The Script Concordance Test: a new tool assessing clinical judgement in neurology.

BACKGROUND: Clinical judgment, the ability to make appropriate decisions in uncertain situations, is central to neurological practice, but objective measures of clinical judgment in neurology trainees are lacking. The Script Concordance Test (SCT), based on script theory from cognitive psychology, uses authentic clinical scenarios to compare a trainee's judgment skills with those of experts. The SCT has been validated in several medical disciplines, but has not been investigated in neurology. METHODS: We developed an Internet-based neurology SCT (NSCT) comprising 24 clinical scenarios with three to four questions each. The scenarios were designed to reflect the uncertainty of real-life clinical encounters in adult neurology. The questions explored aspects of the scenario in which several responses might be acceptable; trainees were asked to judge which response they considered to be best. Forty-one PGY1-PGY5 neurology residents and eight medical students from three North American neurology programs (McGill, Calgary, and Mayo Clinic) completed the NSCT. The responses of trainees to each question were compared with the aggregate responses of an expert panel of 16 attending neurologists. RESULTS: The NSCT demonstrated good reliability (Cronbach alpha = 0.79). Neurology residents scored higher than medical students and lower than attending neurologists, supporting the test's construct validity. Furthermore, NSCT scores discriminated between senior (PGY3-5) and junior residents (PGY1-2). CONCLUSIONS: Our NSCT is a practical and reliable instrument, and our findings support its construct validity for assessing judgment in neurology trainees. The NSCT has potentially widespread applications as an evaluation tool, both in neurology training and for licensing examinations.

Learning-by-Concordance (LbC): introducing undergraduate students to the complexity and uncertainty of clinical practice.

BACKGROUND: A current challenge in medical education is the steep exposure to the complexity and uncertainty of clinical practice in early clerkship. The gap between pre-clinical courses and the reality of clinical decision-making can be overwhelming for undergraduate students. The Learning-by-Concordance (LbC) approach aims to bridge this gap by embedding complexity and uncertainty by relying on real-life situations and exposure to expert reasoning processes to support learning. LbC provides three forms of support: 1) expert responses that students compare with their own, 2) expert explanations and 3) recognized scholars' key-messages. METHOD: Three different LbC inspired learning tools were used by 900 undergraduate medical students in three courses: Concordance-of-Reasoning in a 1st-year hematology course; Concordance-of-Perception in a 2nd-year pulmonary physio-pathology course, and; Concordance-of-Professional-Judgment with 3rd-year clerkship students. Thematic analysis was conducted on freely volunteered qualitative comments provided by 404 students. RESULTS: Absence of a right answer was challenging for 1st year concordance-of-reasoning group; the 2nd year visual concordance group found radiology images initially difficult and unnerving and the 3rd year concordance-of-judgment group recognized the importance of divergent expert opinion. CONCLUSIONS: Expert panel answers and explanations constitute an example of "cognitive apprenticeship" that could contribute to the development of appropriate professional reasoning processes.

Online script concordance test for clinical reasoning assessment in otorhinolaryngology: the association between performance and clinical experience.

OBJECTIVE: To report on the creation and administration of an online Script Concordance Test (SCT) for ear, nose, and throat (ENT), the ENT-SCT. DESIGN: Prospective study. SETTING: Two tertiary care university centers. PARTICIPANTS: In total, 132 individuals were asked to test an ENT-SCT of 20 cases and 94 questions based on the major educational objectives of the ENT residency program. MAIN OUTCOME MEASURES: Three levels of experience were tested: medical students, ENT residents, and board-certified otorhinolaryngologists as the expert panel. The test's construct validity-whether scores were related to clinical experience-was statistically analyzed. Reliability was estimated by the Cronbach α internal consistency coefficient. Participants' perception of the test was assessed with the use of a questionnaire. RESULTS: The 65 respondents with usable data were medical students (n = 21), ENT residents (n = 22), and experts (n = 22). Total mean (SD) test scores differed significantly: 76.81 (3.31) for the expert panel, 69.05 (4.35) for residents, and 58.29 (5.86) for students. The Cronbach α coefficient was 0.95. More than two-thirds of the participants found the test to be realistic and relevant for assessing clinical reasoning. The test was also considered fun, interesting, and intuitive. CONCLUSIONS: The Web-based ENT-SCT is feasible, reliable, and useful for assessing clinical reasoning. This online assessment tool may have applications for residency programs and continuing medical education.