Cheating in OSCEs: The Impact of Simulated Security Breaches on OSCE Performance.

Construct: Valid score interpretation is important for constructs in performance assessments such as objective structured clinical examinations (OSCEs). An OSCE is a type of performance assessment in which a series of standardized patients interact with the student or candidate who is scored by either the standardized patient or a physician examiner. BACKGROUND: In high-stakes examinations, test security is an important issue. Students accessing unauthorized test materials can create an unfair advantage and lead to examination scores that do not reflect students' true ability level. The purpose of this study was to assess the impact of various simulated security breaches on OSCE scores. APPROACH: Seventy-six 3rd-year medical students participated in an 8-station OSCE and were randomized to either a control group or to 1 of 2 experimental conditions simulating test security breaches: station topic (i.e., providing a list of station topics prior to the examination) or egregious security breach (i.e., providing detailed content information prior to the examination). Overall total scores were compared for the 3 groups using both a one-way between-subjects analysis of variance and a repeated measure analysis of variance to compare the checklist, rating scales, and oral question subscores across the three conditions. RESULTS: Overall total scores were highest for the egregious security breach condition (81.8%), followed by the station topic condition (73.6%), and they were lowest for the control group (67.4%). This trend was also found with checklist subscores only (79.1%, 64.9%, and 60.3%, respectively for the security breach, station topic, and control conditions). Rating scale subscores were higher for both the station topic and egregious security breach conditions compared to the control group (82.6%, 83.1%, and 77.6%, respectively). Oral question subscores were significantly higher for the egregious security breach condition (88.8%) followed by the station topic condition (64.3%), and they were the lowest for the control group (48.6%). CONCLUSIONS: This simulation of different OSCE security breaches demonstrated that student performance is greatly advantaged by having prior access to test materials. This has important implications for medical educators as they develop policies and procedures regarding the safeguarding and reuse of test content.

Weighting checklist items and station components on a large-scale OSCE: is it worth the effort?

BACKGROUND: Past research suggests that the use of externally-applied scoring weights may not appreciably impact measurement qualities such as reliability or validity. Nonetheless, some credentialing boards and academic institutions apply differential scoring weights based on expert opinion about the relative importance of individual items or test components of Observed Structured Clinical Examinations (OSCEs). AIMS: To investigate the impact of simplified scoring models that make little to no use of differential weighting on the reliability of scores and decisions on a high stakes OSCE required for medical licensure in Canada. METHOD: We applied four different weighting models of various complexities to data from three administrations of the OSCE. We compared score reliability, pass/fail rates, correlations between the scores and classification decision accuracy and consistency across the models and administrations. RESULTS: Less complex weighting models yielded similar reliability and pass rates as the more complex weighting model. Minimal changes in candidates' pass/fail status were observed and there were strong and statistically significant correlations between the scores for all scoring models and administrations. Classification decision accuracy and consistency were very high and similar across the four scoring models. CONCLUSIONS: Adopting a simplified weighting scheme for this OSCE did not diminish its measurement qualities. Instead of developing complex weighting schemes, experts' time and effort could be better spent on other critical test development and assembly tasks with little to no compromise in the quality of scores and decisions on this high-stakes OSCE.

Dried blood spots via remote testing as a possible future application in the doping control process.

The use of DBS as biological matrix in combination with a novel technical remote testing app specifically developed for the doping control process shows that testing is becoming easier and variable through DBS. Supplemented by the remote testing solution, the system could significantly increase in the unpredictability of test dates and sample quantity.

Bone resorption is induced on the second day of bed rest: results of a controlled crossover trial.

The aim of the study was to analyze the kinetics of short-term changes in bone turnover. We studied in a randomized crossover design the effects of 6 days of bed rest on eight healthy male subjects (mean body wt: 70.1 +/- 5.7 kg; mean age: 25.5 +/- 2.9 yr). The metabolic ward period was divided into three parts: 4 ambulatory days, 6 days of either bed rest or non-bed rest periods, and 1 recovery day. The diet was identical in both bed rest and non-bed rest phases. Continuous urine collection started on the first day in the metabolic ward to analyze excretion of bone resorption markers, namely C-telopeptide (CTX) and N-telopeptide (NTX), creatinine, urea, and 3-methylhistidine. On the second ambulatory day and on the fifth day of bed rest or during the non-bed rest phase, blood was drawn to analyze bone formation markers and amino acid concentrations. Urinary calcium excretion was increased as early as the first day of bed rest (P < 0.01). CTX and NTX excretion stayed unchanged during the first 24 h of bed rest compared with the non-bed rest period. However, already on the second day, both resorption markers had increased significantly. NTX excretion increased by 28.7 +/- 14.0% (P < 0.01), whereas CTX excretion rose by 17.8 +/- 8.3% (P < 0.001). Creatinine, urea, and 3-methylhistidine excretion did not change. We conclude that 24 h of bed rest are sufficient to induce a significant rise in osteoclast activity in healthy subjects.