Is the assumption of equal distances between global assessment categories used in borderline regression valid?

BACKGROUND: Standard setting for clinical examinations typically uses the borderline regression method to set the pass mark. An assumption made in using this method is that there are equal intervals between global ratings (GR) (e.g. Fail, Borderline Pass, Clear Pass, Good and Excellent). However, this assumption has never been tested in the medical literature to the best of our knowledge. We examine if the assumption of equal intervals between GR is met, and the potential implications for student outcomes. METHODS: Clinical finals examiners were recruited across two institutions to place the typical 'Borderline Pass', 'Clear Pass' and 'Good' candidate on a continuous slider scale between a typical 'Fail' candidate at point 0 and a typical 'Excellent' candidate at point 1. Results were analysed using one-sample t-testing of each interval to an equal interval size of 0.25. Secondary data analysis was performed on summative assessment scores for 94 clinical stations and 1191 medical student examination outcomes in the final 2 years of study at a single centre. RESULTS: On a scale from 0.00 (Fail) to 1.00 (Excellent), mean examiner GRs for 'Borderline Pass', 'Clear Pass' and 'Good' were 0.33, 0.55 and 0.77 respectively. All of the four intervals between GRs (Fail-Borderline Pass, Borderline Pass-Clear Pass, Clear Pass-Good, Good-Excellent) were statistically significantly different to the expected value of 0.25 (all p-values < 0.0125). An ordinal linear regression using mean examiner GRs was performed for each of the 94 stations, to determine pass marks out of 24. This increased pass marks for all 94 stations compared with the original GR locations (mean increase 0.21), and caused one additional fail by overall exam pass mark (out of 1191 students) and 92 additional station fails (out of 11,346 stations). CONCLUSIONS: Although the current assumption of equal intervals between GRs across the performance spectrum is not met, and an adjusted regression equation causes an increase in station pass marks, the effect on overall exam pass/fail outcomes is modest.

Not All Strokes Are Strokes An Example of Diagnostic Confirmation Bias.

A 72-year-old woman presented with a complex partial seizure and right hemiparesis, after a four-week history of cognitive decline, apraxia and speech disturbance. She previously had chronic lymphocytic leukaemia (CLL) and had finished chemotherapy one year prior to presentation. She was receiving monthly intravenous immunoglobulins for bronchiectasis. Brain imaging showed hypodensity in the left temporo-parietal regions. Cerebrospinal fluid was positive for the JC virus, leading to a diagnosis of progressive multifocal leucoencephalopathy (PML). She remains alive, eight months following initial presentation. The case was valuable for reflective practice in avoiding diagnostic (confirmation) bias because the treating team pursued an incorrect diagnosis of stroke and secondary seizure after radiology findings appeared consistent with this. Additionally, PML has not previously been reported in individuals with CLL receiving immunoglobulin therapy, and may explain the relatively benign course in this individual patient. This offers a potential research question for disease modifying treatments in PML. LEARNING POINTS: This case highlights new insights into an uncommon but important condition: always consider progressive multifocal leucoencephalopathy when immunocompromised patients present with neurological symptoms.A full differential diagnosis should always be considered, even in the context of a more 'plausible' diagnosis.Avoid premature closure and confirmation bias as cognitive errors in diagnostic reasoning.