A collaborative comparison of objective structured clinical examination (OSCE) standard setting methods at Australian medical schools.

BACKGROUND: A key issue underpinning the usefulness of the OSCE assessment to medical education is standard setting, but the majority of standard-setting methods remain challenging for performance assessment because they produce varying passing marks. Several studies have compared standard-setting methods; however, most of these studies are limited by their experimental scope, or use data on examinee performance at a single OSCE station or from a single medical school. This collaborative study between 10 Australian medical schools investigated the effect of standard-setting methods on OSCE cut scores and failure rates. METHODS: This research used 5256 examinee scores from seven shared OSCE stations to calculate cut scores and failure rates using two different compromise standard-setting methods, namely the Borderline Regression and Cohen's methods. RESULTS: The results of this study indicate that Cohen's method yields similar outcomes to the Borderline Regression method, particularly for large examinee cohort sizes. However, with lower examinee numbers on a station, the Borderline Regression method resulted in higher cut scores and larger difference margins in the failure rates. CONCLUSION: Cohen's method yields similar outcomes as the Borderline Regression method and its application for benchmarking purposes and in resource-limited settings is justifiable, particularly with large examinee numbers.

Frequent users of the Royal Flying Doctor Service primary clinic and aeromedical services in remote New South Wales: a quality study.

OBJECTIVE: To examine activity patterns of the Royal Flying Doctor Service of Australia (RFDS) in far western New South Wales and to determine whether frequent use of RFDS services, particularly emergency evacuations, is a useful indicator of patients who may benefit from care planning and review. DESIGN, SETTING AND PARTICIPANTS: We conducted a retrospective audit of the RFDS South Eastern Section's Broken Hill patient database. Patients with a residential address in the study area who had accessed at least one RFDS medical service between 1 July 2000 and 30 June 2005 were included in the study. MAIN OUTCOME MEASURES: Number of evacuations, clinic consultations and remote consultations; clinic usage by frequent evacuees; number of primary diagnoses recorded for frequent evacuees; number of frequent users who might benefit from multidisciplinary care or specialist shared care. RESULTS: Between July 2000 and June 2005, the number of residents requiring evacuation or remote consultations declined by 26% and 19%, respectively, and the number of residents accessing clinics declined by 6%. (Over the same period, the population of the study area fell by about 24%.) Of the 78 patients who were identified as frequent users of the evacuation service (> or = 3 evacuations/year), 34 had three or more primary diagnoses recorded; 15 were infrequent or non-users of the clinics (< or = 3 attendances/year); 53 may have benefited from multidisciplinary care, and 41 from specialist shared care. CONCLUSIONS: Simple, practical clinical review systems can help health care organisations in rural and remote communities to achieve better outcomes by identifying patients who may benefit from planned care.