Reasoning Report: Engineering Case Conferences to Maximize Clinical Reasoning Education for All Learners.

Case conferences, specifically those in which an unknown case is presented and discussed, are widely utilized in the delivery of medical education. However, the format of case conferences is not always optimized to engage and challenge audience members' clinical reasoning (CR). Based on the current conception of CR and our experience, we provide recommendations on how to better engineer case conferences to maximize CR education for learners at all levels through case selection, conference format, and intentional case construction.

'We're like Spider-Man; with great power comes great responsibility': Coaches' experiences supporting struggling medical students.

BACKGROUND: Medical students can experience a range of academic and non-academic struggles. Coaching is a valuable strategy to support learners, but coaches describe working with struggling learners as taxing. Transformative learning theory (TLT) provides insights into how educators grow from challenging experiences to build resilience. This study explores how coaches evolve as educators through supporting struggling students. METHODS: This qualitative study grounded in an interpretivist paradigm used interviews of longitudinal medical student coaches at two academic institutions. Interviews, using TLT as a sensitizing concept, explored coaches' experience coaching struggling learners. We performed thematic analysis. RESULTS: We interviewed 15 coaches. Coaches described supporting students through multi-faceted struggles which often surprised the coach. Three themes characterized coaches' experiences: personal responsibility, emotional response, and personal learning. Coaches shouldered high personal responsibility for learners' success. For some, this burden felt emotional, raised parental instincts and questions about maintaining boundaries with learners. Coaches evolved their coaching approach, challenged biases, and built skills. Coaches learned to better appreciate the learner point of view and employ resources to support students. DISCUSSION: Through navigating learner struggles, educators can gain self-efficacy, learn to understand learners' perspectives, and evolve their coaching approach to lessen their personal emotional burden through time.

Beyond thinking fast and slow: Implications of a transtheoretical model of clinical reasoning and error on teaching, assessment, and research.

From dual process to a family of theories known collectively as situativity, both micro and macro theories of cognition inform our current understanding of clinical reasoning (CR) and error. CR is a complex process that occurs in a complex environment, and a nuanced, expansive, integrated model of these theories is necessary to fully understand how CR is performed in the present day and in the future. In this perspective, we present these individual theories along with figures and descriptive cases for purposes of comparison before exploring the implications of a transtheoretical model of these theories for teaching, assessment, and research in CR and error.

Concordance of Narrative Comments with Supervision Ratings Provided During Entrustable Professional Activity Assessments.

BACKGROUND: Use of EPA-based entrustment-supervision ratings to determine a learner's readiness to assume patient care responsibilities is expanding. OBJECTIVE: In this study, we investigate the correlation between narrative comments and supervision ratings assigned during ad hoc assessments of medical students' performance of EPA tasks. DESIGN: Data from assessments completed for students enrolled in the clerkship phase over 2 academic years were used to extract a stratified random sample of 100 narrative comments for review by an expert panel. PARTICIPANTS: A review panel, comprised of faculty with specific expertise related to their roles within the EPA program, provided a "gold standard" supervision rating using the comments provided by the original assessor. MAIN MEASURES: Interrater reliability (IRR) between members of review panel and correlation coefficients (CC) between expert ratings and supervision ratings from original assessors. KEY RESULTS: IRR among members of the expert panel ranged from .536 for comments associated with focused history taking to .833 for complete physical exam. CC (Kendall's correlation coefficient W) between panel members' assignment of supervision ratings and the ratings provided by the original assessors for history taking, physical examination, and oral presentation comments were .668, .697, and .735 respectively. The supervision ratings of the expert panel had the highest degree of correlation with ratings provided during assessments done by master assessors, faculty trained to assess students across clinical contexts. Correlation between supervision ratings provided with the narrative comments at the time of observation and supervision ratings assigned by the expert panel differed by clinical discipline, perhaps reflecting the value placed on, and perhaps the comfort level with, assessment of the task in a given specialty. CONCLUSIONS: To realize the full educational and catalytic effect of EPA assessments, assessors must apply established performance expectations and provide high-quality narrative comments aligned with the criteria.

REACT: Rapid Evaluation Assessment of Clinical Reasoning Tool.

INTRODUCTION: Clinical reasoning encompasses the process of data collection, synthesis, and interpretation to generate a working diagnosis and make management decisions. Situated cognition theory suggests that knowledge is relative to contextual factors, and clinical reasoning in urgent situations is framed by pressure of consequential, time-sensitive decision-making for diagnosis and management. These unique aspects of urgent clinical care may limit the effectiveness of traditional tools to assess, teach, and remediate clinical reasoning. METHODS: Using two validated frameworks, a multidisciplinary group of clinicians trained to remediate clinical reasoning and with experience in urgent clinical care encounters designed the novel Rapid Evaluation Assessment of Clinical Reasoning Tool (REACT). REACT is a behaviorally anchored assessment tool scoring five domains used to provide formative feedback to learners evaluating patients during urgent clinical situations. A pilot study was performed to assess fourth-year medical students during simulated urgent clinical scenarios. Learners were scored using REACT by a separate, multidisciplinary group of clinician educators with no additional training in the clinical reasoning process. REACT scores were analyzed for internal consistency across raters and observations. RESULTS: Overall internal consistency for the 41 patient simulations as measured by Cronbach's alpha was 0.86. A weighted kappa statistic was used to assess the overall score inter-rater reliability. Moderate reliability was observed at 0.56. DISCUSSION: To our knowledge, REACT is the first tool designed specifically for formative assessment of a learner's clinical reasoning performance during simulated urgent clinical situations. With evidence of reliability and content validity, this tool guides feedback to learners during high-risk urgent clinical scenarios, with the goal of reducing diagnostic and management errors to limit patient harm.

Characterization of Clinical Skills Remediation: A National Survey of Medical Schools.

OBJECTIVES: Clinical skills instruction is a standard part of medical school curricula, but how institutions address learners who struggle in this area is less clear. Although recommendations for the remediation of clinical skills at an institutional level have been published, how these recommendations are being implemented on a national scale is unknown. In this descriptive study, we characterize current clinical skills remediation practices at US medical schools and US-accredited Caribbean medical schools. METHODS: We conducted a cross-sectional survey of medical educators who work with struggling students. From March 24, 2020 to April 9, 2020, the Directors of Clinical Skills Remediation Working Group conducted an e-mail survey incorporating four aspects of remediation program design and function: identification, assessment, active remediation, and ongoing evaluation. RESULTS: In total, 92 individuals representing 45 institutions provided descriptive information about their respective remediation programs. The majority of respondents have a formal process of identifying (75%) and assessing (86%) students who are identified as struggling with clinical skills, but lack a standardized method of categorizing deficits. Fewer institutions have a standardized approach to active remediation and ongoing evaluation of struggling learners. Fifty-two percent of institutions provide training to faculty involved in the remediation process. CONCLUSIONS: Although most institutions are able to identify struggling students, they lack a standardized approach to intervene. Remediation effectiveness is limited by a lack of student buy-in and institutional time, expertise, and resources. These findings highlight the need for more formalized structure and standardization in remediation program design and implementation.

A High-Value Care Curriculum Using Individual and Group Structured Reflection.

OBJECTIVE: One-third of all healthcare dollars are wasted, primarily in the form of clinician-ordered unnecessary diagnostic tests and treatments. Medical education has likely played a central role in the creation and perpetuation of this problem. We aimed to create a curriculum for medical students to promote their contribution to high-value care conversations in the clinical environment. METHODS: At a large university medical center between March 2017 and February 2018, we implemented a 3-phase curriculum combining multimodal educational initiatives with individual and group reflection for third-year medical students during their 12-week long Internal Medicine clerkship rotation. Students were asked to identify examples of clinical decision making that lacked attention to high-value care, propose solutions to the identified situation, and pinpoint barriers to the implementation of effective solutions using a structured reflection framework and then participate in a debrief debate with fellow students. To assess the curriculum, reflective narratives were coded by frequency and codes were compared with one another and with relevant high-value care literature to identify patterns and themes. RESULTS: In total, 151 medical students participated in phase 1 and 119 in phase 3. For phase 2, 126 reflective narratives (94.7% participation rate) comprised 226 problems, 280 solutions, and 179 barriers. CONCLUSIONS: When provided appropriate resources, medical students are able to identify relevant examples of low-value care, downstream solutions, and barriers to implementation through a structured reflection curriculum comprising written narratives and in-person debate.

Lessons in clinical reasoning - pitfalls, myths, and pearls: a case of tarsal tunnel syndrome caused by an intraneural ganglion cyst.

OBJECTIVES: Intraneural ganglionic cysts are non-neoplastic cysts that can cause signs and symptoms of peripheral neuropathy. However, the scarcity of such cases can lead to cognitive biases. Early surgical exploration of space occupying lesions plays an important role in identification and improving the outcomes for intraneural ganglionic cysts. CASE PRESENTATION: This patient presented with loss of sensation on the right sole with tingling numbness for six months. A diagnosis of tarsal tunnel syndrome was made. Nerve conduction study revealed that the mixed nerve action potential (NAP) was absent in the right medial and lateral plantar nerves. The magnetic resonance imaging (MRI) found a cystic lesion measuring 1.4×1.8×3.8 cm as the presumed cause of the neuropathy. Surgical exploration revealed a ganglionic cyst traversing towards the flexor retinaculum with baby cysts. The latter finding came as a surprise to the treating surgeon and was confirmed to be an intraneural ganglionic cyst based on the histopathology report. CONCLUSIONS: Through integrated commentary by a case discussant and reflection by an orthopedician, this case highlights the significance of the availability heuristic, confirmation bias, and anchoring bias in a case of rare disease. Despite diagnostic delays, a medically knowledgeable patient's involvement in their own care lead to a more positive outcome. A fish-bone diagram is provided to visually demonstrate the major factors that contributed to the diagnostic delay. Finally, this case provides clinical teaching points in addition to a pitfall, myth, and pearl related to availability heuristic and the sunk cost fallacy.

A Targeted Clinical Reasoning Remediation Program for Residents and Fellows in Need.

Background There is no standardized, widely accepted process for individualized clinical reasoning remediation. Objective We describe a novel, targeted assessment and coaching process that allows for individualized intervention for residents and fellows struggling with clinical reasoning. Methods Residents and fellows at the University of Virginia with performance concerns are referred to COACH (Committee on Achieving Competence Through Help) and assessed by a remediation expert. A subset is referred to a clinical reasoning remediation coach who performs an additional assessment and cocreates an individualized remediation plan. Following remediation, residents and fellows are reassessed by their respective programs. We report the frequency of struggle, remediation time invested, and academic outcomes. Results From 2017 to 2022, 114 residents and fellows referred to COACH met inclusion criteria, of which 38 (33%) had a deficiency in clinical reasoning. Targeted assessment revealed the following microskill deficits: hypothesis generation (16 of 38, 42%); data gathering (6 of 38, 16%); problem representation (7 of 38, 18%); hypothesis refinement (3 of 38, 8%); and management (6 of 38, 16%). Remediation required a mean of nearly 23 hours per trainee. Of the 38 trainees, 33 (87%) are in good standing at the time of writing. Conclusions Our unique program offers a feasible, targeted approach to clinical reasoning remediation based on our current understanding of the clinical reasoning process. Early hypothesis generation was the most common microskill deficit identified.

Large Language Model Influence on Diagnostic Reasoning: A Randomized Clinical Trial.

Importance: Large language models (LLMs) have shown promise in their performance on both multiple-choice and open-ended medical reasoning examinations, but it remains unknown whether the use of such tools improves physician diagnostic reasoning. Objective: To assess the effect of an LLM on physicians' diagnostic reasoning compared with conventional resources. Design, Setting, and Participants: A single-blind randomized clinical trial was conducted from November 29 to December 29, 2023. Using remote video conferencing and in-person participation across multiple academic medical institutions, physicians with training in family medicine, internal medicine, or emergency medicine were recruited. Intervention: Participants were randomized to either access the LLM in addition to conventional diagnostic resources or conventional resources only, stratified by career stage. Participants were allocated 60 minutes to review up to 6 clinical vignettes. Main Outcomes and Measures: The primary outcome was performance on a standardized rubric of diagnostic performance based on differential diagnosis accuracy, appropriateness of supporting and opposing factors, and next diagnostic evaluation steps, validated and graded via blinded expert consensus. Secondary outcomes included time spent per case (in seconds) and final diagnosis accuracy. All analyses followed the intention-to-treat principle. A secondary exploratory analysis evaluated the standalone performance of the LLM by comparing the primary outcomes between the LLM alone group and the conventional resource group. Results: Fifty physicians (26 attendings, 24 residents; median years in practice, 3 [IQR, 2-8]) participated virtually as well as at 1 in-person site. The median diagnostic reasoning score per case was 76% (IQR, 66%-87%) for the LLM group and 74% (IQR, 63%-84%) for the conventional resources-only group, with an adjusted difference of 2 percentage points (95% CI, -4 to 8 percentage points; P = .60). The median time spent per case for the LLM group was 519 (IQR, 371-668) seconds, compared with 565 (IQR, 456-788) seconds for the conventional resources group, with a time difference of -82 (95% CI, -195 to 31; P = .20) seconds. The LLM alone scored 16 percentage points (95% CI, 2-30 percentage points; P = .03) higher than the conventional resources group. Conclusions and Relevance: In this trial, the availability of an LLM to physicians as a diagnostic aid did not significantly improve clinical reasoning compared with conventional resources. The LLM alone demonstrated higher performance than both physician groups, indicating the need for technology and workforce development to realize the potential of physician-artificial intelligence collaboration in clinical practice. Trial Registration: ClinicalTrials.gov Identifier: NCT06157944.

Influence of a Large Language Model on Diagnostic Reasoning: A Randomized Clinical Vignette Study.

Importance: Diagnostic errors are common and cause significant morbidity. Large language models (LLMs) have shown promise in their performance on both multiple-choice and open-ended medical reasoning examinations, but it remains unknown whether the use of such tools improves diagnostic reasoning. Objective: To assess the impact of the GPT-4 LLM on physicians' diagnostic reasoning compared to conventional resources. Design: Multi-center, randomized clinical vignette study. Setting: The study was conducted using remote video conferencing with physicians across the country and in-person participation across multiple academic medical institutions. Participants: Resident and attending physicians with training in family medicine, internal medicine, or emergency medicine. Interventions: Participants were randomized to access GPT-4 in addition to conventional diagnostic resources or to just conventional resources. They were allocated 60 minutes to review up to six clinical vignettes adapted from established diagnostic reasoning exams. Main Outcomes and Measures: The primary outcome was diagnostic performance based on differential diagnosis accuracy, appropriateness of supporting and opposing factors, and next diagnostic evaluation steps. Secondary outcomes included time spent per case and final diagnosis. Results: 50 physicians (26 attendings, 24 residents) participated, with an average of 5.2 cases completed per participant. The median diagnostic reasoning score per case was 76.3 percent (IQR 65.8 to 86.8) for the GPT-4 group and 73.7 percent (IQR 63.2 to 84.2) for the conventional resources group, with an adjusted difference of 1.6 percentage points (95% CI -4.4 to 7.6; p=0.60). The median time spent on cases for the GPT-4 group was 519 seconds (IQR 371 to 668 seconds), compared to 565 seconds (IQR 456 to 788 seconds) for the conventional resources group, with a time difference of -82 seconds (95% CI -195 to 31; p=0.20). GPT-4 alone scored 15.5 percentage points (95% CI 1.5 to 29, p=0.03) higher than the conventional resources group. Conclusions and Relevance: In a clinical vignette-based study, the availability of GPT-4 to physicians as a diagnostic aid did not significantly improve clinical reasoning compared to conventional resources, although it may improve components of clinical reasoning such as efficiency. GPT-4 alone demonstrated higher performance than both physician groups, suggesting opportunities for further improvement in physician-AI collaboration in clinical practice.

Large Language Model Influence on Management Reasoning: A Randomized Controlled Trial.

Importance: Large language model (LLM) artificial intelligence (AI) systems have shown promise in diagnostic reasoning, but their utility in management reasoning with no clear right answers is unknown. Objective: To determine whether LLM assistance improves physician performance on open-ended management reasoning tasks compared to conventional resources. Design: Prospective, randomized controlled trial conducted from 30 November 2023 to 21 April 2024. Setting: Multi-institutional study from Stanford University, Beth Israel Deaconess Medical Center, and the University of Virginia involving physicians from across the United States. Participants: 92 practicing attending physicians and residents with training in internal medicine, family medicine, or emergency medicine. Intervention: Five expert-developed clinical case vignettes were presented with multiple open-ended management questions and scoring rubrics created through a Delphi process. Physicians were randomized to use either GPT-4 via ChatGPT Plus in addition to conventional resources (e.g., UpToDate, Google), or conventional resources alone. Main Outcomes and Measures: The primary outcome was difference in total score between groups on expert-developed scoring rubrics. Secondary outcomes included domain-specific scores and time spent per case. Results: Physicians using the LLM scored higher compared to those using conventional resources (mean difference 6.5 %, 95% CI 2.7-10.2, p<0.001). Significant improvements were seen in management decisions (6.1%, 95% CI 2.5-9.7, p=0.001), diagnostic decisions (12.1%, 95% CI 3.1-21.0, p=0.009), and case-specific (6.2%, 95% CI 2.4-9.9, p=0.002) domains. GPT-4 users spent more time per case (mean difference 119.3 seconds, 95% CI 17.4-221.2, p=0.02). There was no significant difference between GPT-4-augmented physicians and GPT-4 alone (-0.9%, 95% CI -9.0 to 7.2, p=0.8). Conclusions and Relevance: LLM assistance improved physician management reasoning compared to conventional resources, with particular gains in contextual and patient-specific decision-making. These findings indicate that LLMs can augment management decision-making in complex cases. Trial registration: ClinicalTrials.gov Identifier: NCT06208423; https://classic.clinicaltrials.gov/ct2/show/NCT06208423.

The future of teaching management reasoning: important questions and potential solutions.

Management reasoning is distinct from but inextricably linked to diagnostic reasoning in the iterative process that is clinical reasoning. Complex and situated, management reasoning skills are distinct from diagnostic reasoning skills and must be developed in order to promote cogent clinical decisions. While there is growing interest in teaching management reasoning, key educational questions remain regarding when it should be taught, how it can best be taught in the clinical setting, and how it can be taught in a way that helps mitigate implicit bias. Here, we describe several useful tools to structure teaching of management reasoning across learner levels and educational settings. The management script provides a scaffold for organizing knowledge around management and can serve as a springboard for discussion of uncertainty, thresholds, high-value care, and shared decision-making. The management pause reserves space for management discussions and exploration of a learner's reasoning. Finally, the equity reflection invites learners to examine management decisions from a health equity perspective, promoting the practice of metacognition around implicit bias. These tools are easily deployable, and - when used regularly - foster a learning environment primed for the successful teaching of management reasoning.

A novel virtual course to teach medical students high-value decision-making.

BACKGROUND: Although a clinician's ability to employ high-value decision-making is influenced by training, many undergraduate medical education programmes lack a formal curriculum in high-value, cost-conscious care. We present a curriculum developed through a cross-institutional collaboration that was used to teach students at two institutions about this topic and can serve as a framework for other institutions to develop similar curricula. APPROACH: The faculty from the University of Virginia and the Johns Hopkins University School of Medicine created a 2-week-long online course to teach medical students the fundamentals of high-value care. The course consisted of learning modules, clinical cases, textbook studies, journal clubs and a competitive 'Shark Tank' final project where students proposed a realistic intervention to promote high-value clinical care. EVALUATION: Over two-thirds of students rated the course's quality as excellent or very good. Most found the online modules (92%), assigned textbook readings (89%) and 'Shark Tank' competition (83%) useful. To evaluate the student's ability to apply the concepts learned during the course in clinical contexts, we developed a scoring rubric based on the New World Kirkpatrick Model to evaluate students' proposals. Groups chosen as finalists (as determined by faculty judges) were more likely to be fourth-year students (56%), achieved higher overall scores (p = 0.03), better incorporated cost impact at several levels (patient, hospital and national) (p = 0.001) and discussed both positive and negative impacts on patient safety (p = 0.04). IMPLICATIONS: This course provides a framework for medical schools to use in their teaching of high-value care. Cross-institutional collaboration and online content overcame local barriers such as contextual factors and lack of faculty expertise, allowed for greater flexibility, and enabled focused curricular time to be spent on a capstone project competition. Prior clinical experience amongst medical students may be an enabling factor in promoting application of learning related to high-value care.

Development and Validation of a Formative Assessment Tool for Nephrology Fellows' Clinical Reasoning.

BACKGROUND: Diagnostic errors are commonly driven by failures in clinical reasoning. Deficits in clinical reasoning are common among graduate medical learners, including nephrology fellows. We created and validated an instrument to assess clinical reasoning in a national cohort of nephrology fellows and established performance thresholds for remedial coaching. METHODS: Experts in nephrology education and clinical reasoning remediation designed an instrument to measure clinical reasoning through a written patient encounter note from a web-based, simulated AKI consult. The instrument measured clinical reasoning in three domains: problem representation, differential diagnosis with justification, and diagnostic plan with justification. Inter-rater reliability was established in a pilot cohort ( n =7 raters) of first-year nephrology fellows using a two-way random effects agreement intraclass correlation coefficient model. The instrument was then administered to a larger cohort of first-year fellows to establish performance standards for coaching using the Hofstee method ( n =6 raters). RESULTS: In the pilot cohort, there were 15 fellows from four training program, and in the study cohort, there were 61 fellows from 20 training programs. The intraclass correlation coefficients for problem representation, differential diagnosis, and diagnostic plan were 0.90, 0.70, and 0.50, respectively. Passing thresholds (% total points) in problem representation, differential diagnosis, and diagnostic plan were 59%, 57%, and 62%, respectively. Fifty-nine percent ( n =36) met the threshold for remedial coaching in at least one domain. CONCLUSIONS: We provide validity evidence for a simulated AKI consult for formative assessment of clinical reasoning in nephrology fellows. Most fellows met criteria for coaching in at least one of three reasoning domains, demonstrating a need for learner assessment and instruction in clinical reasoning.

Lessons in clinical reasoning - pitfalls, myths, and pearls: a case of crushing, substernal chest pain.

OBJECTIVES: Diagnostic error is not uncommon and diagnostic accuracy can be improved with the use of problem representation, pre-test probability, and Bayesian analysis for improved clinical reasoning. CASE PRESENTATION: A 48-year-old female presented as a transfer from another Emergency Department (ED) to our ED with crushing, substernal pain associated with dyspnea, diaphoresis, nausea, and a tingling sensation down both arms with radiation to the back and neck. Troponins were elevated along with an abnormal electrocardiogram. A negative myocardial perfusion scan led to the patient's discharge. The patient presented to the ED 10 days later with an anterior ST-elevation myocardial infarction. CONCLUSIONS: An overemphasis on a single testing modality led to diagnostic error and a severe event. The use of pre-test probabilities guided by history-taking can lead to improved interpretation of test results, ultimately improving diagnostic accuracy and preventing serious medical errors.

Framework and Schema are False Synonyms: Defining Terms to Improve Learning.

Clinical reasoning is an essential expertise of health care professionals that includes the complex cognitive processes that lead to diagnosis and management decisions. In order to optimally teach, learn, and assess clinical reasoning, it is imperative for teachers and learners to have a shared understanding of the language. Currently, educators use the terms schema and framework interchangeably but they are distinct concepts. In this paper, we offer definitions for schema and framework and use the high-stakes field of aviation to demonstrate the interplay of these concepts. We offer examples of framework and schema in the medical education field and discuss how a clear understanding of these concepts allows for greater intentionality when teaching and assessing clinical reasoning.