Promoting Longitudinal and Developmental Computer-Based Assessments of Clinical Reasoning: Validity Evidence for a Clinical Reasoning Mapping Exercise.

PURPOSE: Clinical reasoning is vitally important for practitioners across the health professions. However, the assessment of clinical reasoning remains a significant challenge. Combined with other assessment methods, non-workplace-based assessment can increase opportunities to address multiple components of clinical reasoning, evaluate growth, and foster learning, but tools with validity evidence to assess clinical reasoning outside the workplace are scare. This study examined validity evidence for a novel clinical reasoning mapping exercise (CResME). METHOD: Data include CResME performance scores from 120 third-year medical students at the University of Central Florida for 3 topics in May 2022. Each CResME was scored by 2 physician raters based on a scoring rubric that included a combined diagnosis and sequence score. Descriptive statistics were used to examine trends in scores. The authors gathered validity evidence for response process, internal structure, and relations to other variables. RESULTS: The overall mean (SD) score across cases was 66 (29). Internal consistency reliability of cases (Cronbach α) ranged from 0.75 to 0.91. The Phi and G coefficients were 0.45 and 0.56, respectively. Students accounted for 10% of the total variance, indicating the ability to differentiate high and low clinical reasoning skills; the interaction between learner and case accounted for 8.1% of the variance, demonstrating case specificity. There was a moderate correlation between the overall CResME scores and the mean overall score of patient encounter notes from an objective structured clinical examination performed at the end of the third year (0.46; P = .001). Significant associations were also found between the CResME scores and subject exam scores. CONCLUSIONS: The CResME can be used to facilitate the assessment of clinical reasoning, supporting the developmental progress of learners throughout the curriculum. Future research is needed to gather validity evidence for CResMEs with different learners across different settings and formats.

The association between quality of connections and diagnostic accuracy in student-generated concept maps for clinical reasoning education with virtual patients.

Objectives: Concept maps are a learning tool that fosters clinical reasoning skills in healthcare education. They can be developed by students in combination with virtual patients to create a visual representation of the clinical reasoning process while solving a case. However, in order to optimize feedback, there is a need to better understand the role of connections between concepts in student-generated maps. Therefore, in this study we investigated whether the quality of these connections is indicative of diagnostic accuracy. Methods: We analyzed 40 concept maps created by fifth-year medical students in the context of four virtual patients with commonly encountered diagnoses. Half of the maps were created by students who made a correct diagnosis on the first attempt; the other half were created by students who made an error in their first diagnosis. The connections in the maps were rated by two reviewers using a relational scoring system. Analysis of covariance was employed to examine the difference in mean connection scores among groups while controlling for the number of connections. Results: There were no differences between the groups in the number of concepts or connections in the maps; however, maps made by students who made a correct first diagnosis had higher scores for the quality of connections than those created by students who made an incorrect first diagnosis (12.13 vs 9.09; p=0.03). We also observed students' general reluctance to use connections in their concept maps. Conclusion: Our results suggest that the quality, not the quantity, of connections in concept maps is indicative of their diagnostic accuracy.

Editors as Gatekeepers: One Medical Education Journal's Efforts to Resist Racism in Scholarly Publishing.

PURPOSE: Journals have begun to expand the racial diversity of editors as a first step to countering institutional racism. Given the power editors hold as gatekeepers, a diverse team helps ensure that minoritized scholars have equal opportunity to contribute. In 2021, Teaching and Learning in Medicine ( TLM ) created an editorial internship for racially minoritized individuals. This study examines the first 6 months of this program to better understand its creation and initial successes. METHOD: The authors employed critical collaborative autoethnography, a qualitative methodology, focusing on the underlying assumptions around power and hierarchy that are implicit in the design and implementation of the TLM internship. Participants included 13 TLM editorial board members (10 internship selection committee members, 3 mentors, 2 independent researchers), 3 external selection committee members, and 3 interns, with some holding multiple roles. Ten participants served as authors of this report. Data included archival emails, planning documents, and focus groups. The initial analysis explored what happened and how and was followed by a thematic analysis in which participants reflected on their responsibility for implementing an antiracist program. RESULTS: While the program developed interns' editorial skills, which they greatly valued, and diversified the TLM editorial board, it did not achieve the goal of fostering antiracism. Mentors focused on conducting joint peer reviews with interns, assuming that racial experiences can and should be separate from the editorial process, thus working within, rather than trying to change, the existing racist system. CONCLUSIONS: Given these findings, greater structural change is needed to disrupt the existing racist system. These experiences underscore the importance of recognizing the harmful impact a race-neutral lens can have on antiracist efforts. Moving forward, TLM will implement lessons learned ahead of offering the internship again with the goal of creating the transformative change intended with the creation of the program.

Well-being at a Military Medical School and Implications for Military Retention.

INTRODUCTION: Physical and psychological well-being play a critical role in the academic and professional development of medical students and can alter the trajectory of a student's quality of personal and professional life. Military medical students, given their dual role as officer and student, experience unique stressors and issues that may play a role in their future intentions to continue military service, as well as practice medicine. As such, this study explores well-being across the 4 years of medical school at Uniformed Services University (USU) and how well-being relates to a student's likelihood to continue serving in the military and practicing medicine. METHODS: In September 2019, 678 USU medical students were invited to complete a survey consisting of three sections-the Medical Student Well-being Index (MSWBI), a single-item burnout measure, and six questions regarding their likelihood of staying in the military and medical practice. Survey responses were analyzed using descriptive statistics, analysis of variance (ANOVA), and contingency table analysis. Additionally, thematic analysis was conducted on open-ended responses included as part of the likelihood questions. RESULTS: Our MSWBI and burnout scores suggest that the overall state of well-being among medical students at USU is comparable to other studies of the medical student population. ANOVA revealed class differences among the four cohorts, highlighted by improved well-being scores as students transitioned from clerkships to their fourth-year curriculum. Fewer clinical students (MS3s and MS4s), compared to pre-clerkship students, indicated a desire to stay in the military. In contrast, a higher percentage of clinical students seemed to "reconsider" their medical career choice compared to their pre-clerkship student counterparts. "Medicine-oriented" likelihood questions were associated with four unique MSWBI items, whereas "military-oriented" likelihood questions were associated with one unique MSWBI item. CONCLUSION: The present study found that the overall state of well-being in USU medical students is satisfactory, but opportunities for improvement exist. Medical student well-being seemed to have a stronger association with medicine-oriented likelihood items than with military-oriented likelihood items. To obtain and refine best practices for strengthening engagement and commitment, future research should examine if and how military and medical contexts converge and diverge throughout training. This may enhance the medical school and training experience and, ultimately, reinforce, or strengthen, the desire and commitment to practice and serve in military medicine.

Teaching Students How to Think: A Longitudinal Qualitative Study of Preclerkship Clinical Reasoning Instruction.

INTRODUCTION: An objective of undergraduate medical education is to teach students how to think like physicians through a process called clinical reasoning. Currently, clerkship directors often feel that students enter their clinical years with a marginal comprehension of clinical reasoning concepts; instruction in this area could be improved. Although there have been previous educational studies assessing curricular interventions to improve the instruction of clinical reasoning, it is not yet known what happens at an individual level between an instructor and a small group of students in the teaching of clinical reasoning. This research will identify how clinical reasoning is being taught in a longitudinal clinical reasoning course. METHODS: The Introduction to Clinical Reasoning course is a 15-month-long case-based course held in the preclinical curriculum of the USU. Individual sessions involve small-group learning with approximately seven students per group. Throughout the academic year of 2018-2019, 10 of these sessions were videotaped and transcribed. All participants provided informed consent. A thematic analysis was performed using a constant comparative approach. Transcripts were analyzed until thematic sufficiency was reached. RESULTS: Over 300 pages of text were analyzed; new themes ceased to be identified after the eighth session. Topics included obstetrics, general pediatric issues, jaundice, and chest pain; these sessions were taught either by attendings, fellows, or fourth-year medical students with attending supervision. The thematic analysis revealed themes associated with clinical reasoning processes, themes associated with knowledge organization, and a theme associated with clinical reasoning in the military. The clinical reasoning process themes included problem list construction and refinement, differential diagnosis, naming and defending a leading diagnosis, and clinical reasoning heuristics. The knowledge organization themes included illness script development and refinement and semantic competence. The final theme was military relevant care. CONCLUSIONS: In individual teaching sessions, preceptors emphasized problem lists, differential diagnoses, and leading diagnoses in a course designed to strengthen diagnostic reasoning in preclerkship medical students. The use of illness scripts was more often implicitly used rather than explicitly stated, and students used these sessions to use and apply new vocabularies related to a clinical presentation. Instruction in clinical reasoning could be improved by encouraging faculty to provide further context to their thinking, by encouraging the comparing and contrasting of illness scripts, and by using a shared vocabulary for clinical reasoning. Limitations of this study include that it was done in the context of a clinical reasoning course and that it was done at a military medical school, which may limit generalizability. Future studies could determine if faculty development could improve the frequency of references to the clinical reasoning processes that could improve student readiness for clerkship.

The challenge of clinical reasoning in chronic multimorbidity: time and interactions in the Health Issues Network model.

The increasing prevalence of multimorbidity requires new theoretical models and educational approaches to develop physicians' ability to manage multimorbidity patients. The Health Issues Network (HIN) is an educational approach based on a graphical depiction of the evolutions over time of the concurrent health issues of a patient and of their interactions. From a theoretical point of view, the HIN approach is rooted in Prigogine's vision of the "becoming" of the events and in the concept of knowledge organization, intended as the process of storing and structuring of information in a learner's mind. The HIN approach allows to design clinical exercises to foster learners' ability to detect evolutionary paths and interactions among health issues. Recent findings of neuroscience support the expectation that interpreting, completing, and creating diagrams depicting complex clinical cases improves the "sense of time", as a fundamental competence in the management of multimorbidity. The application of the HIN approach is expected to decrease the risk of errors in the management of multimorbidity patients. The approach is still under validation, both for undergraduate students and for the continuous professional development of physicians.

Concept mapping: An aid to teaching and learning: AMEE Guide No. 157.

Concept maps are graphic representations that learners draw to depict their understanding of the meaning of a set of concepts. Concept maps can be a useful learning strategy in medical education. The purpose of this guide is to provide an understanding of the theoretical basis and instructional use of concept mapping in health professions education. The guide describes the key elements of a concept map and underscores the critical components of the implementation process, from the introduction of the activity to the use of different mapping techniques, based on purpose and context. This guide also examines the learning opportunities offered by collaborative concept mapping, including co-construction of knowledge, and provides suggestions for the use of concept mapping as an assessment for learning. Implications for the use of concept mapping as a tool for remediation are mentioned. Finally, the guide describes some of the challenges related to the implementation of this strategy.

Clinical Reasoning Curricula in Preclinical Undergraduate Medical Education: A Scoping Review.

PURPOSE: Clinical reasoning is the process of observing, collecting, analyzing, and interpreting patient information to arrive at a diagnosis and management plan. Although clinical reasoning is foundational in undergraduate medical education (UME), the current literature lacks a clear picture of the clinical reasoning curriculum in preclinical phase of UME. This scoping review explores the mechanisms of clinical reasoning education in preclinical UME. METHOD: A scoping review was performed in accordance with the Arksey and O'Malley framework methodology for scoping reviews and is reported using the Preferred Reporting Items for Systematic Reviews and Meta-Analysis for Scoping Reviews. RESULTS: The initial database search identified 3,062 articles. Of these, 241 articles were selected for a full-text review. Twenty-one articles, each reporting a single clinical reasoning curriculum, were selected for inclusion. Six of the reports included a definition of clinical reasoning, and 7 explicitly reported the theory underlying the curriculum. Reports varied in the identification of clinical reasoning content domains and teaching strategies. Only 4 curricula reported assessment validity evidence. CONCLUSIONS: Based on this scoping review, we recommend 5 key principles for educators to consider when reporting clinical reasoning curricula in preclinical UME: (1) explicitly define clinical reasoning within the report, (2) report clinical reasoning theory(ies) used in the development of the curriculum, (3) clearly identify which clinical reasoning domains are addressed in the curriculum, (4) report validity evidence for assessments when available, and (5) describe how the reported curriculum fits into the larger clinical reasoning education at the institution.

When students' words hurt: 12 tips for helping faculty receive and respond constructively to student evaluations of teaching.

Student evaluations of curricular experiences and instructors are employed by institutions to obtain feedback and guide improvement. However, to be effective, evaluations must prompt faculty action. Unfortunately, evaluative comments that engender strong reactions may undermine the process by hindering innovation and improvement steps. The literature suggests that faculty interpret evaluation feedback as a judgment not just on their teaching ability but on their personal and professional identity. In this context, critical evaluations, even when constructively worded, can result in disappointment, hurt, and shame. The COVID pandemic has challenged institutions and faculty to repeatedly adapt curricula and educational practices, heightening concerns for faculty burnout. In this context, the risk of 'words that hurt' is higher than ever. This article offers guidance for faculty and institutions to support effective responses to critical feedback and ameliorate counterproductive effects of learner evaluations.

A scoping review of distributed cognition in acute care clinical decision-making.

OBJECTIVES: In acute care settings, interactions between providers and tools drive clinical decision-making. Most studies of decision-making focus on individual cognition and fail to capture critical collaborations. Distributed Cognition (DCog) theory provides a framework for examining the dispersal of tasks among agents and artifacts, enhancing the investigation of decision-making and error. CONTENT: This scoping review maps the evidence collected in empiric studies applying DCog to clinical decision-making in acute care settings and identifies gaps in the existing literature. SUMMARY AND OUTLOOK: Thirty-seven articles were included. The majority (n=30) used qualitative methods (observations, interviews, artifact analysis) to examine the work of physicians (n=28), nurses (n=27), residents (n=16), and advanced practice providers (n=12) in intensive care units (n=18), operating rooms (n=7), inpatient units (n=7) and emergency departments (n=5). Information flow (n=30) and task coordination (n=30) were the most frequently investigated elements of DCog. Provider-artifact (n=35) and provider-provider (n=30) interactions were most explored. Electronic (n=18) and paper (n=15) medical records were frequently described artifacts. Seven prominent themes were identified. DCog is an underutilized framework for examining how information is obtained, represented, and transmitted through complex clinical systems. DCog offers mechanisms for exploring how technologies, like EMRs, and workspaces can help or hinder clinical decision-making.

Implementation of three knowledge-oriented instructional strategies to teach clinical reasoning: Self-explanation, a concept mapping exercise, and deliberate reflection: AMEE Guide No. 150.

The teaching of clinical reasoning is essential in medical education. This guide has been written to provide educators with practical advice on the design, development, and implementation of three knowledge-oriented instructional strategies for the teaching of clinical reasoning to medical students: Self-explanation (SE), a Clinical Reasoning Mapping Exercise (CREsME), and Deliberate Reflection (DR). We first synthesize the theoretical tenets that support the use of these strategies, including knowledge organization, and development of illness scripts. We then provide a detailed description of the key components of each strategy, emphasizing the practical applications of each one by sharing specific examples. We also explore the potential for a combined application of these strategies in a longitudinal and developmental approach to teaching clinical reasoning at the undergraduate level. Finally, we discuss enablers and barriers in the implementation and integration of these teaching strategies while taking into consideration curricular needs, context, and resources. We are aware that many strategies exist and are not arguing that SE, CReSME, and DR are the most effective ones or the only ones to be adopted. Nevertheless, we selected these strategies because of overarching theoretical principles, empirical evidence supporting their use, and our own experience with them. We are hoping to provide practical advice on the implementation of these strategies to practicing educators who aim at developing an integrated approach to the teaching of clinical reasoning to medical students at different stages of their development.

An international study on the implementation of programmatic assessment: Understanding challenges and exploring solutions.

INTRODUCTION: Programmatic assessment is an approach to assessment aimed at optimizing the learning and decision function of assessment. It involves a set of key principles and ground rules that are important for its design and implementation. However, despite its intuitive appeal, its implementation remains a challenge. The purpose of this paper is to gain a better understanding of the factors that affect the implementation process of programmatic assessment and how specific implementation challenges are managed across different programs. METHODS: An explanatory multiple case (collective) approach was used for this study. We identified 6 medical programs that had implemented programmatic assessment with variation regarding health profession disciplines, level of education and geographic location. We conducted interviews with a key faculty member from each of the programs and analyzed the data using inductive thematic analysis. RESULTS: We identified two major factors in managing the challenges and complexity of the implementation process: knowledge brokers and a strategic opportunistic approach. Knowledge brokers were the people who drove and designed the implementation process acting by translating evidence into practice allowing for real-time management of the complex processes of implementation. These knowledge brokers used a 'strategic opportunistic' or agile approach to recognize new opportunities, secure leadership support, adapt to the context and take advantage of the unexpected. Engaging in an overall curriculum reform process was a critical factor for a successful implementation of programmatic assessment. DISCUSSION: The study contributes to the understanding of the intricacies of implementation processes of programmatic assessment across different institutions. Managing opportunities, adaptive planning, awareness of context, were all critical aspects of thinking strategically and opportunistically in the implementation of programmatic assessment. Future research is needed to provide a more in-depth understanding of values and beliefs that underpin the assessment culture of an organization, and how such values may affect implementation.

Extending growth curves: a trajectory monitoring approach to identification and interventions in struggling medical student learners.

Given gaps in both identifying and providing targeted interventions to struggling learners, the purpose of this study is to both improve rapid identification and to improve individualized academic advising for learners using this visual representation of performance. Across three graduating classes, individual growth curves were calculated for each student on National Board of Medical Examiners customized assessments during the pre-clerkship period using their deviation from the class average at each assessment point. These deviation scores were cumulatively summed over time and were regressed onto the sequence of exams. We analyzed the difference between the regression slopes of those students placed on Academic Probation (AP) versus not, as well as differences in slopes based on the timing of when a struggling learner was placed on AP to explore learner trajectory after identification. Students on AP had an average growth slope of - 6.06 compared to + 0.89 for those not on AP. Findings also suggested that students who were placed on AP early during pre-clerkship showed significant improvement (positive changes in trajectory) compared to students identified later in the curriculum. Our findings suggest that earlier academic probation and intervention with struggling learners may have a positive effect on academic trajectory. Future research can better explore how academic trajectory monitoring and performance review can be regularly used in advising sessions with students.