Influence of a Large Language Model on Diagnostic Reasoning: A Randomized Clinical Vignette Study.

Importance: Diagnostic errors are common and cause significant morbidity. Large language models (LLMs) have shown promise in their performance on both multiple-choice and open-ended medical reasoning examinations, but it remains unknown whether the use of such tools improves diagnostic reasoning. Objective: To assess the impact of the GPT-4 LLM on physicians' diagnostic reasoning compared to conventional resources. Design: Multi-center, randomized clinical vignette study. Setting: The study was conducted using remote video conferencing with physicians across the country and in-person participation across multiple academic medical institutions. Participants: Resident and attending physicians with training in family medicine, internal medicine, or emergency medicine. Interventions: Participants were randomized to access GPT-4 in addition to conventional diagnostic resources or to just conventional resources. They were allocated 60 minutes to review up to six clinical vignettes adapted from established diagnostic reasoning exams. Main Outcomes and Measures: The primary outcome was diagnostic performance based on differential diagnosis accuracy, appropriateness of supporting and opposing factors, and next diagnostic evaluation steps. Secondary outcomes included time spent per case and final diagnosis. Results: 50 physicians (26 attendings, 24 residents) participated, with an average of 5.2 cases completed per participant. The median diagnostic reasoning score per case was 76.3 percent (IQR 65.8 to 86.8) for the GPT-4 group and 73.7 percent (IQR 63.2 to 84.2) for the conventional resources group, with an adjusted difference of 1.6 percentage points (95% CI -4.4 to 7.6; p=0.60). The median time spent on cases for the GPT-4 group was 519 seconds (IQR 371 to 668 seconds), compared to 565 seconds (IQR 456 to 788 seconds) for the conventional resources group, with a time difference of -82 seconds (95% CI -195 to 31; p=0.20). GPT-4 alone scored 15.5 percentage points (95% CI 1.5 to 29, p=0.03) higher than the conventional resources group. Conclusions and Relevance: In a clinical vignette-based study, the availability of GPT-4 to physicians as a diagnostic aid did not significantly improve clinical reasoning compared to conventional resources, although it may improve components of clinical reasoning such as efficiency. GPT-4 alone demonstrated higher performance than both physician groups, suggesting opportunities for further improvement in physician-AI collaboration in clinical practice.

Clinical Reasoning of a Generative Artificial Intelligence Model Compared With Physicians.

This cross-sectional study assesses the ability of a large language model to process medical data and display clinical reasoning compared with the ability of attending physicians and residents.

Finding the right combination for self-directed learning: A focus group study of residents' choice and use of digital resources to support their learning.

BACKGROUND: There has been a shift in postgraduate medical education towards digital educational resources-podcasts, videos, social media and other formats consumed asynchronously and apart from formal curricula. It is unclear what drives residents to select and use these resources. Understanding how and why residents choose digital resources can aid programme directors, faculty and residents in optimising residents' informal learning time. METHOD: This focus group study was conducted with residents at two US internal medicine residency programmes. The authors used the framework approach to content analysis using self-determination theory as guide for deductive coding and iteratively assessing connections among codes and identifying themes. Trustworthiness was addressed through use of analytic memos, reflexive practice and member checking. RESULTS: The authors conducted eight virtual focus groups (n = 23) from 5/27/20 to 6/11/20. Residents described that a feeling of 'should know' drove initial choices towards self-directed learning outside of work. Regular use of a resource was influenced by how the resource fit into a resident's lifestyle, the personal cognitive energy and the perceived 'activation energy' of using a particular resource. Familiarity, increased confidence and in-person social networks gained from digital resources served to reinforce and further guide resource choice. CONCLUSIONS: The selection of digital resources for self-directed learning is driven by multiple factors, suggesting an interdependent relationship between the learning environment and a residents' cognitive capacity. Understanding these interconnections can help residents and clinical educators explicitly choose resources that fit their lifestyle and learning needs.

Artificial Intelligence vs Clinician Performance in Estimating Probabilities of Diagnoses Before and After Testing.

This diagnostic study compares the performance of artificial intelligence (AI) with that of human clinicians in estimating the probability of diagnoses before and after testing.

Accuracy of a Generative Artificial Intelligence Model in a Complex Diagnostic Challenge.

This study assesses the diagnostic accuracy of the Generative Pre-trained Transformer 4 (GPT-4) artificial intelligence (AI) model in a series of challenging cases.

Be in the Digital Room Where it Happens, Part II: Social Media for Neurology Educators.

Social media has changed the way we communicate and interact. Unsurprisingly, it has also changed how we teach and learn. Younger generations of learners have transitioned from traditional educational sources to digital ones. Medical educators need to adapt to trends in medical education and develop fluency in the digital methods used by medical learners today. This is part two of a two-part series on social media and digital education in neurology. This article provides an overview of how social media can be used as a teaching tool in medical education and provides an overview in which it is grounded. We offer practical strategies on how social media can promote lifelong learning, educator development, educator support, and foster educator identity with accompanying neurology-specific examples. We also review considerations for incorporating social media into teaching and learning practices and future directions for integrating these tools in neurology education.

Be in the Digital Room Where it Happens, Part I: Tweeting & Technology for Career Development.

Social media has become a part of everyday life. It has changed the way we obtain and distribute information, connect, and interact with others. As the number of platforms and users grow, medical professionals have learned the value social media can have in education, research, advocacy, and clinical care initiatives. Platforms provide opportunities to network, build collaborations, and develop a reputation. This is part one of a two-part series. This article provides an overview on how social media can benefit professional career development for clinicians and researchers, as well as for advocacy to raise awareness against biases, disparities, and for patient benefit. We review challenges, limitations, and best practices for social media use by medical professionals with neurology-specific examples.

How podcasts teach: A comprehensive analysis of the didactic methods of the top hundred medical podcasts.

PURPOSE: Medical podcasts have grown in popularity, but little is known about their didactic methods. This study sought to systemically describe the pedagogical approach employed by the 100 most popular medical podcasts in the United States. This study also aimed to assess factors related to quality control and conflicts of interest in podcasting. METHODS: The authors averaged the rank positions for Apple podcasts in the Medicine category in the United States from 06/01/18 to 09/30/20 to generate a list of the 100 highest-ranked medical podcasts. They developed and validated a categorization system of didactic methods based on Bloom's taxonomy and collected data on didactic methods, as well as podcast affiliation, target audience, format, advertising, continuing medical education (CME) offerings, and presence of a reference list or review process. RESULTS: Of the 100 most popular medical podcasts, 91 are educational. Of those, 51 are podcasts intended for physician education (PIPEs) while 40 are intended for other audiences, including the general public, nurses, and physical therapists. Compared with podcasts intended for other audiences, PIPEs engage higher levels of Bloom's taxonomy (p < 0.001). Among PIPEs, 18 (35.2%) are affiliated with an individual, 16 (31.4%) with a company, and 12 (23.5%) with a professional journal. 38 PIPEs (74.5%) are targeted toward all levels of medical learners. PIPEs are significantly more likely to list references or have a peer review process in place (n = 37, 72.5% vs. n = 15, 37.5%, p = 0.001) and offer CME credits (n = 20, 39.2% vs. n = 2, 5.0%, p < 0.001) than podcasts intended for other audiences. CONCLUSIONS: Medical podcasts employ a variety of didactic methods, including those ranked highly on Bloom's taxonomy. Unlike traditional medical education, PIPEs are commonly produced by individuals or companies and targeted to all levels of medical learners.

The Last Beat: Contemporary Ethical Controversies Surrounding Determination of Cardiopulmonary Death.

Part one of this series tracked the evolution of the death examination, noting its stability over the last century despite changing diagnostic and therapeutic technologies and social contexts. In part two, we discuss the practical and ethical debates surrounding the exact timing of death. Although the irreversible cessation of cardiopulmonary systems remains the most common criteria used for the determination of death, identification of the moment of irreversibility is imprecise. In most cases, this imprecision is not problematic, but, when the cessation of circulation is used to identify the time of organ procurement for transplantation, it becomes critical. The phenomenon of autoresuscitation highlights these issues because patients who meet all the criteria for circulatory death (sometimes for periods of observation well beyond the norm) apparently return to life. Were these patients resurrected (like Lazarus) or did we simply not wait long enough?

The Last Breath: Historical Controversies Surrounding Determination of Cardiopulmonary Death.

Cardiopulmonary determination of death is a mainstay of the practice of internal medicine and pulmonary physicians. Despite this, there is considerable variability in death examinations. This article tracks the evolution of the tripartite death examination, initially developed in the middle of the 19th century to protect against premature burial. Although the societal context for controversies about death determination has shifted to discussions about end-of-life care in ICUs and organ transplantation, the cardiopulmonary death examination has largely remained unchanged from its original formulation. The recognition of coma dépassé and brain death has further pushed the focus of the death examination onto the neurological system. Despite advancing diagnostics and legislative attempts to standardize the definition of death, cardiopulmonary death determination largely remains an ad hoc process.

Learning Through Listening: A Scoping Review of Podcast Use in Medical Education.

PURPOSE: To improve understanding of podcast use in medical education by examining current research on descriptive attributes and educational outcomes, highlighting implications of the current evidence base for educational practices, and identifying research gaps to guide future investigation. METHOD: The authors conducted a scoping review, searching PubMed and Embase databases in June-July 2020 for English-language studies of audio-only medical education podcast use in undergraduate, graduate, and continuing medical education. The authors excluded studies without original data or with nonphysician data that could not be separated from physician data. From included studies, the authors extracted data regarding descriptive outcomes (e.g., podcast use, content areas, structure) and educational outcomes (classified using Kirkpatrick's 4 levels of evaluation). RESULTS: Of 491 unique articles, 62 met inclusion criteria. Descriptive outcomes were reported in 44 studies. Analysis of these studies revealed podcast use has increased over time, podcasts are a top resource for resident education, and podcasts are being incorporated into formal medical curricula. Educational outcomes were reported in 38 studies. The 29 studies that assessed learner reaction and attitudes to podcasts (Kirkpatrick level 1) showed learners value podcasts for their portability, efficiency, and combined educational and entertainment value. The 10 studies that assessed knowledge retention (Kirkpatrick level 2) showed podcasts to be noninferior to traditional teaching methods. The 11 studies that assessed behavior change (Kirkpatrick level 3) showed improved documentation skills in medical students and self-reported practice change in residents and practicing physicians after listening to podcasts. None of the studies reported system change or patient outcomes (Kirkpatrick level 4). CONCLUSIONS: Future research should focus on the optimal structure of podcasts for learning, higher-level outcomes of podcasts, and the implementation of podcasts into formal curricula. Podcasts may prove to be essential tools for disseminating and implementing the most current, evidence-based practices.