Validating a script concordance test for assessing neurological localization and emergencies.

BACKGROUND AND PURPOSE: Clinical judgment is the ability to weigh clinical information and make decisions under conditions of uncertainty. Although neurological localization (NL) and neurological emergencies (NE) present such uncertainties, no validated method is reported to assess these decision-making skills. A script concordance test (SCT) was designed and validated to assess clinical judgment in NL and NE. METHODS: Our SCT comprised 14 clinical scenarios (53 questions). Candidates picked the response they considered the best for the questions in each scenario. Undergraduates and internal medicine residents completed the SCT; their responses were scored against the scoring key derived from an expert panel of accredited neurologists. Scores were expressed as a percentage of the maximum score. RESULTS: Mean total scores for undergraduates (n = 52), residents (n = 37) and experts (n = 15) were 61.0 ± 0.9, 68.3 ± 1.1 and 76.6 ± 1.1 (mean ± standard error of the mean, P < 0.001). Mean scores for undergraduates, residents and experts were 59.3 ± 1.1, 66.4 ± 1.4 and 76.1 ± 1.8 (P < 0.001) for NL, and 62.9 ± 1.3, 70.4 ± 1.3 and 77.2 ± 1.6 (P < 0.001) for NE. Senior residents scored higher than junior residents (postgraduate years 2-5 versus postgraduate year 1, 69.7 ± 1.4 vs. 65.3 ± 1.1, P = 0.035). The higher scores with increasing clinical experience supports the construct validity of the SCT. The SCT showed acceptable reliability (G coefficient 0.74 ± 0.05). CONCLUSIONS: Our SCT is validated to reliably assess NL and NE in undergraduate and postgraduate learners; it is generalizable and feasible. It has potential as a valuable adjunct assessment tool for clinical judgment. Future plans to design SCTs to evaluate other topics in clinical neurology, as a multi-center study, are under way.