Evaluating and reporting at a distance: quality experiences with cost effective web supported evaluations

Abstract

In 2002 and 2003 as the evaluator for two Commonwealth Department of Health and Aging initiatives I conducted evaluations designed to elicit information about what creates and sustains a quality learning environment during off-campus placements for students. The evaluations revealed that both for senior public health undergraduates from 4 Universities undertaking a semi-structured six week placements in private and public health settings around Australia to conduct research projects, and for senior medical students undertaking structured 26 week rural clinical placements in the Spencer Gulf Rural Health School, the quality of the supervision or preceptoring is paramount in creating and sustaining a quality learning environment. This paper considers the quantitative and qualitative evaluation processes engaged to evaluate the impact of these pilot placement initiatives. The evaluation plans included conducting online surveys, online discussion boards, email evaluation, paper-based surveys which were “read” and analysed using optical mark recognition software, as well as the more traditional face-to-face interviews and focus groups. The evaluation results were reported back to various stakeholders including students, preceptors, heath services, Universities and the Commonwealth using emailed reports and video conferencing with associated “Smart Board” graphics. The findings were also disseminated publicly at the 34th Public Health Conference of Australia (September 2002) and ANZAME (July 2003). The focus of the paper is on the way in which the web in particular has enabled an evaluator in Adelaide to cost effectively conduct and report evaluations around Australia using methodologically sound quantitative and qualitative evaluation processes which withstand scrutiny for publication.