Promoting Longitudinal and Developmental Computer-Based Assessments of Clinical Reasoning: Validity Evidence for a Clinical Reasoning Mapping Exercise.

PURPOSE: Clinical reasoning is vitally important for practitioners across the health professions. However, the assessment of clinical reasoning remains a significant challenge. Combined with other assessment methods, non-workplace-based assessment can increase opportunities to address multiple components of clinical reasoning, evaluate growth, and foster learning, but tools with validity evidence to assess clinical reasoning outside the workplace are scare. This study examined validity evidence for a novel clinical reasoning mapping exercise (CResME). METHOD: Data include CResME performance scores from 120 third-year medical students at the University of Central Florida for 3 topics in May 2022. Each CResME was scored by 2 physician raters based on a scoring rubric that included a combined diagnosis and sequence score. Descriptive statistics were used to examine trends in scores. The authors gathered validity evidence for response process, internal structure, and relations to other variables. RESULTS: The overall mean (SD) score across cases was 66 (29). Internal consistency reliability of cases (Cronbach α) ranged from 0.75 to 0.91. The Phi and G coefficients were 0.45 and 0.56, respectively. Students accounted for 10% of the total variance, indicating the ability to differentiate high and low clinical reasoning skills; the interaction between learner and case accounted for 8.1% of the variance, demonstrating case specificity. There was a moderate correlation between the overall CResME scores and the mean overall score of patient encounter notes from an objective structured clinical examination performed at the end of the third year (0.46; P = .001). Significant associations were also found between the CResME scores and subject exam scores. CONCLUSIONS: The CResME can be used to facilitate the assessment of clinical reasoning, supporting the developmental progress of learners throughout the curriculum. Future research is needed to gather validity evidence for CResMEs with different learners across different settings and formats.

Seeing Complexity: Cultural Historical Activity Theory (CHAT) As a Lens for Shared Decision Making.

Shared decision making, a collaborative approach between patient and provider that considers the patient's values and preferences in addition to the scientific evidence, is a complex clinical activity that has not realized its full potential. Gaps in education and training have been cited as barriers to shared decision making, and evidence is inconsistent on effective educational interventions. Because individual agents with their own social and behavioral contexts co-construct a shared decision, the educational approach may need to consider the role of patient agency and sociocultural influences. To address the inherent complexity in shared decision making, the authors identified cultural historical activity theory (CHAT) as a framework for analysis. Although certainly not the only relevant theory, CHAT offers an appropriate lens through which the multivoiced nature of shared decision making can be more clearly appreciated. In this article, the authors demonstrate the application of CHAT as a lens for researchers and educators to examine the complexity of shared decision making. The fictitious case presented in this article describes the use of CHAT with a patient who experiences 2 clinical encounters; during the second, shared decision making takes place. Elements of the case are threaded through the article, demonstrating a sample analysis of the interacting activity systems of the patient and physician and highlighting inherent tensions and contradictions. The authors propose CHAT as a tool for future research around the role of agency in shared decision making and other complex topics and as a framework for design of novel instructional strategies. Although not applicable to all topics and settings, CHAT has significant potential within health professions education.

Decide + Be Ready: A Contraceptive Decision-Making Mobile Application for Servicewomen.

Women in the military have a high rate of unintended pregnancies, which is an issue both personally and with respect to the warfighting mission. One strategy to help servicewomen achieve family planning goals includes increasing education about and access to contraception. Research suggests that preference-sensitive decisions about contraceptives benefit from shared decision-making, and decision aids have been shown to facilitate this patient-centered approach. In this article, we describe the process by which we enhanced an existing evidence-based tool to meet the needs of military servicewomen and created Decide + Be Ready, a contraceptive decision-making mobile application. After extensive research into challenges faced by servicewomen with respect to contraceptive knowledge and access, we developed content for the decision aid and determined that a mobile app format would provide the privacy and convenience needed. Our team developed a prototype that, in collaboration with the Defense Health Agency Connected Health Branch, was tested with servicewomen and providers. User feedback shaped the final version, which can be accessed free from the App Store and Google Play. Early implementation has demonstrated patient and provider satisfaction. Obstacles to full implementation of Decide + Be Ready remain within the Military Health System. We lay out a roadmap for dissemination, implementation, and evaluation and explore the applications of the decision aid for health professions education in the realm of shared decision-making. Finally, we recommend consideration of decision aids for other health care decisions as a way to achieve patient-centered care, improve health outcomes, and potentially reduce costs.

Gathering Validity Evidence on an Internal Medicine Clerkship Multistep Exam to Assess Medical Student Analytic Ability.

Construct: The definition of clinical reasoning may vary among health profession educators. However, for the purpose of this paper, clinical reasoning is defined as the cognitive processes that are involved in the steps of information gathering, problem representation, generating a differential diagnosis, providing a diagnostic justification to arrive at a leading diagnosis, and formulating diagnostic and management plans. Background: Expert performance in clinical reasoning is essential for success as a physician, and has been difficult for clerkship directors to observe and quantify in a way that fosters the instruction and assessment of clinical reasoning. The purpose of this study was to gather validity evidence for the Multistep exam (MSX) format used by our medicine clerkship to assess analytical clinical reasoning abilities; we did this by examining the relationship between scores on the MSX and other external measures of clinical reasoning abilities. This analysis used dual process theory as the main theoretical framework of clinical reasoning, as well as aspects of Kane's validity framework to guide the selection of validity evidence for the investigation. We hypothesized that there would be an association between the MSX (a three-step clinical reasoning tool developed locally), and the USMLE Step 2 CS, as they share similar concepts in assessing the clinical reasoning of students. We examined the relationship between overall scores on the MSX and the Step 2 CS Integrated Clinical Encounter (ICE) score, in which the student articulates their reasoning for simulated patient cases, while controlling for examinee's internal medicine clerkship performance measures such as the NBME subject exam score and the Medicine clerkship OSCE score. Approach: A total 477 of 487 (97.9%) medical students, representing the graduating classes of 2015, 2016, 2017, who took the MSX at the end of each medicine clerkship (2012-2016), and Step 2 CS (2013-2017) were included in this study. Correlation analysis and multiple linear regression analysis were used to examine the impact of the primary explanatory variables of interest (MSX) onto the outcome variable (ICE score) when controlling for baseline variables (Medicine OSCE and NBME Medicine subject exam). Findings: The overall MSX score had a significant, positive correlation with the Step 2 CS ICE score (r = .26, P < .01). The overall MSX score was a significant predictor of Step 2 CS ICE score (ß = .19, P < .001), explaining an additional 4% of the variance of ICE beyond the NBME Medicine subject score and the Medicine OSCE score (Adjusted R2 = 13%). Conclusion: The stepwise format of the MSX provides a tool to observe clinical reasoning performance, which can be used in an assessment system to provide feedback to students on their analytical clinical reasoning. Future studies should focus on gaining additional validity evidence across different learners and multiple medical schools.

Career Accomplishments of Public Health Service Alumni of the Uniformed Services University of the Health Sciences: Classes of 1980-2017.

INTRODUCTION: The purpose of this study was to report on the career accomplishments of Public Health Service (PHS) alumni from the Uniformed Services University (USU) School of Medicine (SOM) in several professional areas such as specialty choice, board certification rates, leadership roles, academic achievements, deployments, and length of service. MATERIALS AND METHODS: This study was conducted using the data obtained from the Long-Term Career Outcomes Study Alumni Survey. The cohort included all alumni who graduated from the SOM classes of 1980-2017. We analyzed the frequency distributions of the survey items of interest of the PHS alumni. RESULTS: Out of the 144 PHS alumni we reached, 39 responded, yielding a 27.1% response rate. 56.0% of PHS alumni entered primary care specialties and 97.0% were board certified in their first residency trained specialty. 30.6% of PHS alumni completed residency training in a military facility. 87.2% of PHS alumni have been in a leadership role since medical school, and 67.6% of PHS alumni currently hold a leadership role. 7.7% of PHS alumni have deployed for ≥30 days to a theater of combat operations. 66.7% and 71.8% of PHS alumni engaged in medical humanitarian and volunteering missions, respectively, and 75% of PHS alumni stayed in active duty longer than their initial commitment. 71.8% of PHS alumni were published in peer-reviewed journals and 17.6% were academically affiliated with USU. CONCLUSIONS: Our findings suggest that PHS alumni are likely to enter primary care specialties, engage in medical volunteering and humanitarian missions, frequently remain in uniform longer than their initial active duty commitment, and are affiliated with numerous universities and medical schools throughout their career. PHS alumni appear to be successfully meeting the mission set forth by the U.S. PHS.

The Impact of Military Pediatrics: Assessing Clinical, Leadership, Academic, and Operational Experience Among Pediatric-trained Graduates From the Uniformed Services University of the Health Sciences (USU).

INTRODUCTION: This article uses alumni survey data from the Uniformed Services University of the Health Sciences (USU) F. Edward Hébert School of Medicine to assess the professional impact of pediatric-trained USU graduates over the course of their careers. We specifically report on the clinical and leadership roles held, career accomplishments, and operational involvement among military pediatricians. MATERIALS AND METHODS: This study analyzed survey data that were collected from alumni of USU. We used descriptive statistics to report the career achievements and operational experiences among USU graduates who completed training in pediatrics. This study was deemed exempt by the USU Institutional Review Board. RESULTS: The survey response rate was 49.5% among 2,400 eligible respondents. Out of 1,189 alumni respondents, 110 (9.3%) trained in pediatrics. Among the pediatric-trained USU graduates, 98.2% spent some time as a full-time clinician, 73.6% served as chief of a clinical service, 42.7% held the role of department chair/chief or the equivalent, and 26.3% filled leadership positions in deployed settings. Forty percent of USU-trained military pediatricians deployed to combat areas at least once, and 35.5% participated in at least one peacetime humanitarian mission. CONCLUSIONS: Our findings contribute to the unique story of military pediatricians who graduated from USU. These uniformed pediatricians participate actively in all realms of military medicine and have demonstrated their versatility through participation in a wide variety of vital functions. Their contributions include the provision of clinical care for both military children and active duty service members, serving in varied leadership positions, engaging in health professions education and other academic pursuits, and participating in operational assignments. Future studies could aim to more fully address the diverse contributions of military pediatricians to the overall mission by including more specific data about career experiences from all uniformed pediatricians.

Exploring the Predictors of Post-Clerkship USMLE Step 1 Scores.

Theory: We used two theoretical frameworks for this study: a) experiential learning, whereby learners construct new knowledge based on prior experience, and learning grows out of a continuous process of reconstructing experience, and b) deliberate practice, whereby the use of testing (test-enhanced learning) promotes learning and produces better long-term retention. Hypothesis: We hypothesized that moving the USMLE Step 1 exam to follow the clerkship year would provide students with a context for basic science learning that may enhance exam performance. We also hypothesized that examination performance variables, specifically National Board of Medical Examiners (NBME) Customized Basic Science Examinations and NBME subject examinations in clinical disciplines would account for a moderate to large amount of the variance in Step 1 scores. Thus we examined predictors of USMLE Step 1 scores when taken after the core clerkship year. Method: In 2011, we revised our medical school curriculum and moved the timing of Step 1 to follow the clerkship year. We performed descriptive statistics, an ANCOVA to compare Step 1 mean scores for three graduating classes of medical students before and after the curriculum changes, and stepwise linear regression to investigate the association between independent variables and the primary outcome measure after curriculum changes. Results: 993 students took the Step 1 exam, which included graduating classes before (2012-2014, N = 491) and after (2015-2017, N = 502) the curriculum change. Step 1 scores increased significantly following curricular revision (mean 218, SD 18.2, vs. 228, SD 16.7, p < 0.01) after controlling for MCAT and undergraduate GPA. Overall, 66.4% of the variance in Step 1 scores after the clerkship year was explained by: the mean score on fourteen pre-clerkship customized NBME exams (p < 0.01, 57.0% R2); performance on the surgery NBME subject exam (p < 0.01, 3.0% R2); the pediatrics NBME subject exam (p < 0.01, 2.0% R2); the Comprehensive Basic Science Self-Assessment (p < .01, 2.0% R2) ; the internal medicine NBME subject exam (p < 0.01, 0.03% R2), pre-clerkship Integrated Clinical Skills score (p < 0.01, 0.05% R2), and the pre-matriculation MCAT (p < 0.01, 0.01% R2). Conclusion: In our institution, nearly two-thirds of the variance in performance on Step 1 taken after the clerkship year was explained mainly by pre-clerkship variables, with a smaller contribution emanating from clerkship measures. Further study is needed to uncover the specific aspects of the clerkship experience that might contribute to success on high stakes licensing exam performance.

Theoretical considerations on programmatic assessment.

Introduction: Programmatic assessment (PA) is an approach to assessment aimed at optimizing learning which continues to gain educational momentum. However, the theoretical underpinnings of PA have not been clearly described. An explanation of the theoretical underpinnings of PA will allow educators to gain a better understanding of this approach and, perhaps, facilitate its use and effective implementation. The purpose of this article is twofold: first, to describe salient theoretical perspectives on PA; second to examine how theory may help educators to develop effective PA programs, helping to overcome challenges around PA.Results: We outline a number of learning theories that underpin key educational principles of PA: constructivist and social constructivist theory supporting meaning making, and longitudinality; cognitivist and cognitive development orientation scaffolding the practice of a continuous feedback process; theory of instructional design underpinning assessment as learning; self-determination theory (SDT), self-regulation learning theory (SRL), and principles of deliberate practice providing theoretical tenets for student agency and accountability.Conclusion: The construction of a plausible and coherent link between key educational principles of PA and learning theories should enable educators to pose new and important inquiries, reflect on their assessment practices and help overcome future challenges in the development and implementation of PA in their programs.

Trust in Group Decisions: a scoping review.

BACKGROUND: Trust is a critical component of competency committees given their high-stakes decisions. Research from outside of medicine on group trust has not focused on trust in group decisions, and "group trust" has not been clearly defined. The purpose was twofold: to examine the definition of trust in the context of group decisions and to explore what factors may influence trust from the perspective of those who rely on competency committees through a proposed group trust model. METHODS: The authors conducted a literature search of four online databases, seeking articles published on trust in group settings. Reviewers extracted, coded, and analyzed key data including definitions of trust and factors pertaining to group trust. RESULTS: The authors selected 42 articles for full text review. Although reviewers found multiple general definitions of trust, they were unable to find a clear definition of group trust and propose the following: a group-directed willingness to accept vulnerability to actions of the members based on the expectation that members will perform a particular action important to the group, encompassing social exchange, collective perceptions, and interpersonal trust. Additionally, the authors propose a model encompassing individual level factors (trustor and trustee), interpersonal interactions, group level factors (structure and processes), and environmental factors. CONCLUSIONS: Higher degrees of trust at the individual and group levels have been associated with attitudinal and performance outcomes, such as quality of group decisions. Developing a deeper understanding of trust in competency committees may help these committees implement more effective and meaningful processes to make collective decisions.

Dynamic Measurement in Health Professions Education: Rationale, Application, and Possibilities.

Dynamic measurement modeling (DMM) is a psychometric paradigm that uses longitudinal data to estimate individual students' growth in measured skills over the course of an educational program (i.e., growth scores). DMM represents a more formal way of assessing learning progress across the health professions education continuum. In this article, the authors provide justification for this approach in health professions education and demonstrate its proof-of-concept use with three time points of United States Medical Licensing Examination Step exams to generate growth scores for 454 current and recent medical learners. The authors demonstrate that learners vary substantially on their growth scores, and those growth scores exhibit psychometric reliability. In addition, growth scores significantly and positively correlated with indicators of medical learner readiness (e.g., undergraduate grade point average and Medical College Admission Test scores). These growth scores were also capable of significantly and positively correlating with future ratings of clinical competencies during internship as assessed through a survey sent to their program directors at the end of the first postgraduate year (e.g., patient care, interpersonal skills). These preliminary findings of reliability and validity for DMM growth scores provide initial evidence for further investigation into the suitability of a dynamic measurement paradigm in health professions education.

The Clinical Reasoning Mapping Exercise (CResME): a new tool for exploring clinical reasoning.

INTRODUCTION: National organizations have identified a need for the creation of novel approaches to teach clinical reasoning throughout medical education. The aim of this project was to develop, implement and evaluate a novel clinical reasoning mapping exercise (CResME). METHODS: Participants included a convenience sample of first and second year medical students at two US medical schools: University of Central Florida (UCF) and Uniformed Services University of Health Sciences (USUHS). The authors describe the creation and implementation of the CResME. The CResME uses clinical information for multiple disease entities as nodes in different domains (history, physical exam, imaging, laboratory results, etc.), requiring learners to connect these nodes of information in an accurate and meaningful way to develop diagnostic and/or management plans in the process. RESULTS: The majority of medical students at both institutions felt that the CResME promoted their understanding of the differential diagnosis and was a valuable tool to compare and contrast elements of a differential diagnosis. Students at both institutions recommended using the CResME for future sessions. DISCUSSION: The CResME is a promising tool to foster students' clinical reasoning early in medical school. Research is needed on the implementation of the CResME as an instructional and assessment strategy for clinical reasoning throughout medical school training.

Honoring Thyself in the Transition to Online Teaching.

Increasingly, health professions education (HPE) faculty are choosing or being required to transition their face-to-face teaching to online teaching. For many faculty, the online learning environment may represent a new context with unfamiliar technology, changing expectations, and unknown challenges. In this context, faculty members may find themselves teaching in ways that are dissonant with the existing assumptions, beliefs, and views that are central to their pedagogical or teaching identity. This "identity dissonance" may lead to dissatisfaction and frustration for faculty members and potentially suboptimal learning experiences for students. In this Perspective, the authors propose that faculty consider using Pratt's five teaching perspectives as a conceptual framework to recognize and mitigate potential identity dissonance as they transition to teaching online. Derived and refined through several years of research, these teaching perspectives are based on interrelated sets of intentions and beliefs that give direction and justification to faculty members' actions. They have been used in higher education to improve faculty satisfaction, self-reflection capabilities, and faculty development. The authors, therefore, believe that these teaching perspectives hold the potential to help HPE faculty enhance their teaching and retain their primary teaching identify, even as they shift to online teaching. Doing so may ensure that the components of teaching they enjoy and draw self-efficacy from are still central to their teaching experience. Pratt's teaching perspectives also provide a conceptual framework for creating future faculty development initiatives and conducting research to better understand and improve the experience of transitioning to online teaching.

Using Relational Reasoning Strategies to Help Improve Clinical Reasoning Practice.

Clinical reasoning-the steps up to and including establishing a diagnosis and/or therapy-is a fundamentally important mental process for physicians. Unfortunately, mounting evidence suggests that errors in clinical reasoning lead to substantial problems for medical professionals and patients alike, including suboptimal care, malpractice claims, and rising health care costs. For this reason, cognitive strategies by which clinical reasoning may be improved-and that many expert clinicians are already using-are highly relevant for all medical professionals, educators, and learners.In this Perspective, the authors introduce one group of cognitive strategies-termed relational reasoning strategies-that have been empirically shown, through limited educational and psychological research, to improve the accuracy of learners' reasoning both within and outside of the medical disciplines. The authors contend that relational reasoning strategies may help clinicians to be metacognitive about their own clinical reasoning; such strategies may also be particularly well suited for explicitly organizing clinical reasoning instruction for learners. Because the particular curricular efforts that may improve the relational reasoning of medical students are not known at this point, the authors describe the nature of previous research on relational reasoning strategies to encourage the future design, implementation, and evaluation of instructional interventions for relational reasoning within the medical education literature. The authors also call for continued research on using relational reasoning strategies and their role in clinical practice and medical education, with the long-term goal of improving diagnostic accuracy.

Interprofessional Education in the Internal Medicine Clerkship Post-LCME Standard Issuance: Results of a National Survey.

BACKGROUND: Several decades of work have detailed the value and goals of interprofessional education (IPE) within the health professions, defining IPE competencies and best practices. In 2013, the Liaison Committee for Medical Education (LCME) elevated IPE to a U.S. medical school accreditation standard. OBJECTIVE: To examine the status of IPE within internal medicine (IM) clerkships including perspectives, curricular content, barriers, and assessment a year after the LCME standard issuance. DESIGN: Anonymous online survey. PARTICIPANTS: IM clerkship directors from each of the Clerkship Directors in Internal Medicine's 121 U.S. and Canadian member medical schools in 2014. METHODS: In 2014, a section on IPE (18 items) was included in the Clerkship Directors in Internal Medicine annual survey of its 121 U.S. and Canadian member medical schools. MAIN MEASURES: Items (18) assessed clerkship director (CD) perspectives, status of IPE curricula in IM clerkships, and barriers to IPE implementation. Data were analyzed using descriptive statistics and qualitative analysis of free-text responses to one of the survey questions. KEY RESULTS: The overall survey response rate was 78% (94/121). The majority (88%) agreed that IPE is important to the practice of IM, and 71% believed IPE should be part of the IM clerkship. Most (76%) CDs agreed there is need for faculty development programs in IPE; 27% had such a program at their institution. Lack of curricular time, scheduling conflicts, and lack of faculty trained in IPE were the most frequently cited barriers. Twenty-nine percent had formal IPE activities within their IM clerkships, and 38% were planning to make changes. Of those with formal IPE activities, over a third (37%) did not involve student assessment. CONCLUSIONS: Since LCME standard issuance, only a minority of IM clerkships have included formal IPE activities, with lectures as the predominant method. Opportunities exist for enhancing educational methods as well as IPE faculty development.