Assessment of management reasoning: Design considerations drawn from analysis of simulated outpatient encounters.

PURPOSE: Management reasoning is a distinct subset of clinical reasoning. We sought to explore features to be considered when designing assessments of management reasoning. METHODS: This is a hybrid empirical research study, narrative review, and expert perspective. In 2021, we reviewed and discussed 10 videos of simulated (staged) physician-patient encounters, actively seeking actions that offered insights into assessment of management reasoning. We analyzed our own observations in conjunction with literature on clinical reasoning assessment, using a constant comparative qualitative approach. RESULTS: Distinguishing features of management reasoning that will influence its assessment include management scripts, shared decision-making, process knowledge, illness-specific knowledge, and tailoring of the encounter and management plan. Performance domains that merit special consideration include communication, integration of patient preferences, adherence to the management script, and prognostication. Additional facets of encounter variation include the clinical problem, clinical and nonclinical patient characteristics (including preferences, values, and resources), team/system characteristics, and encounter features. We cataloged several relevant assessment approaches including written/computer-based, simulation-based, and workplace-based modalities, and a variety of novel response formats. CONCLUSIONS: Assessment of management reasoning could be improved with attention to the performance domains, facets of variation, and variety of approaches herein identified.

The power of written word: Reflection reduces errors of omission.

BACKGROUND: Medical trainees are expected to perform complex tasks while experiencing interruptions, which increases susceptibility to errors of omission. In our study, we examine whether documentation of clinical encounters increases reflective thinking and reduces errors of omission among novice learners in a simulated setting. METHODS: In 2021, 56 senior medical students participated in a simulated paging curriculum involving urgent inpatient cross-cover scenarios (sepsis and atrial fibrillation). Students responded to pages from standardized registered nurses (SRNs) via telephone, gathered history, and discussed clinical decision-making. Following the phone encounter, students documented a brief note (documentation encounter). A 'phone' score (number of checklist items completed in the phone encounter) and a 'combined' score (number of checklist items completed in the phone and documentation encounters) were calculated. Data were analyzed for differences between the phone scores (control) and combined scores using T-tests and McNemar test of symmetry. FINDINGS: Fifty-four students (96%) participated. Combined scores were higher than phone scores for sepsis (72.8 ± 11.3% vs. 67.9 ± 11.9%, p < 0.001) and atrial fibrillation (74.0 ± 10.1% vs. 67.6 ± 10.0%, p < 0.001) cases. Important items, such as ordering blood cultures for sepsis (p = 0.023) and placing the patient on telemetry for atrial fibrillation (p = 0.013), were more likely to be present when a note was documented. DISCUSSION: This study suggests that documentation provides a mechanism for learners to reflect, which could increase important diagnostic and therapeutic interventions. CONCLUSION: Documentation by novice medical learners may improve patient care by allowing for reflection and reducing errors of omission.

Co-creating a suite of patient decision aids for parents of an infant or young child with differences of sex development: A methods roadmap.

Introduction: Parents and guardians of infants and young children with differences of sex development (DSD) often face numerous health and social decisions about their child's condition. While proxy health decisions can be stressful in any circumstance, they are further exacerbated in this clinical context by significant variations in clinical presentation, parental lack of knowledge about DSD, irreversibility of some options (e.g., gonadectomy), a paucity of research available about long-term outcomes, and anticipated decisional regret. This study aimed to engage clinicians, parents, and an adult living with DSD to collaboratively develop a suite of patient decision aids (PDAs) to respond to the decisional needs of parents and guardians of infants and young children diagnosed with DSD. Methods: We used a systematic co-development process guided by the Ottawa Decision Support Framework and the International Patient Decision Aids Standards (IPDAS). The five steps were: literature selection, establish the team, decisional needs assessment, create the PDAs, and alpha testing. Results: We developed four PDAs to support parents/guardians of infants or young children diagnosed with DSD about four priority decisions identified through our decisional needs assessment: genetic testing, gender of rearing, genital surgery and gonadal surgery. All four PDAs include information for parents about DSD, the options, reasons to choose or avoid each option, and opportunities for parents/guardians to rate the importance of features of each option to clarify their values for these features. Qualitative feedback was positive from clinicians, parents and an adult living with DSD. Conclusions: These PDAs are clinical tools designed to support parents/guardians and to promote making an informed and shared DSD-related decision. While these tools are specific to DSD, they contain themes and elements translatable to other pediatric populations.

Healthcare is not about health.

Initiatives designed to reduce the disease burden and improve the health of the US population that focus on increasing access to health care have been disappointing. Progress requires multifaceted change. We must first acknowledge that the healthcare system is focused on reversing or modifying disease, not enhancing health. Our conceptualization of the development of ill health and disease must also change. Scientific advances are clarifying the complex interactions among the development of ill health and disease and an individual's behaviors, their microbiota, and their physical, social, and emotional environments. A person's genetic makeup predisposes them to a wide array of disease conditions but is rarely deterministic in and of itself. Factors extrinsic to the individual, including the social determinants of health, play a major role in disease development, often decades later. The complexity of health and disease requires a "team" accountable for the health of our populations, and these teams must be expanded beyond the medical professions. Governmental officials, architects, business leaders, civic organizations, social and neighborhood groups are among the key stakeholders on the health side of the equation. If and when disease does become manifest, then the care part of the healthcare system assumes the larger role. This has major implications for the education of our clinically focused health science students, but also of professional disciplines previously deemed peripheral to health. Simply redoubling our efforts and focusing on our current healthcare system is insufficient to make progress in the health of the populace. One example of a multipronged approach in Allentown, PA is explored in depth.

The Long Shadow: A Historical Perspective on Racism in Medical Education.

To dismantle racism in U.S. medical education, people must understand how the history of Christian Europe, Enlightenment-era racial science, colonization, slavery, and racism shaped modern American medicine. Beginning with the coalescence of Christian European identity and empire, the authors trace European racial reasoning through the racial science of the Enlightenment into the White supremacist and anti-Black ideology behind Europe's global system of racialized colonization and enslavement. The authors then follow this racist ideology as it becomes an organizing principle of Euro-American medicine and examine how it manifests in medical education in the United States today. Within this historical context, the authors expose the histories of violence underlying contemporary terms such as implicit bias and microaggressions. Through this history, they also gain a deeper appreciation of why racism is so prevalent in medical education and how it affects admissions, assessments, faculty and trainee diversity, retention, racial climate, and the physical environment. The authors then recommend 6 historically informed steps for confronting racism in medical education: (1) incorporate the history of racism into medical education and unmask institutional histories of racism, (2) create centralized reporting mechanisms and implement systematic reviews of bias in educational and clinical activities, (3) adopt mastery-based assessment in medical education, (4) embrace holistic review and expand its possibilities in admissions, (5) increase faculty diversity by using holistic review principles in hiring and promotions, and (6) leverage accreditation to combat bias in medical education. These strategies will help academic medicine begin to acknowledge the harms propagated throughout the history of racism in medicine and start taking meaningful steps to address them. Although the authors have focused on racism in this paper, they recognize there are many forms of bias that impact medical education and intersect with racism, each with its particular history, that deserve their own telling and redress.

A decade of faculty development for health professions educators: lessons learned from the Macy Faculty Scholars Program.

Faculty development (FD) programs are critical for providing the knowledge and skills necessary to drive positive change in health professions education, but they take many forms to attain the program goals. The Macy Faculty Scholars Program (MFSP), created by the Josiah Macy Jr. Foundation (JMJF) in 2010, intends to develop participants as leaders, scholars, teachers, and mentors. After a decade of implementation, an external review committee conducted a program evaluation to determine how well the program met its intended goals and defined options for ongoing improvement.The committee selected Stufflebeam's CIPP (context, input, process, products) framework to guide the program evaluation. Context and input components were derived from the MFSP description and demographic data, respectively. Process and product components were obtained through a mixed-methods approach, utilizing both quantitative and qualitative data obtained from participant survey responses, and curriculum vitae (CV).The evaluation found participants responded favorably to the program and demonstrated an overall increase in academic productivity, most pronounced during the two years of the program. Mentorship, community of practice, and protected time were cited as major strengths. Areas for improvement included: enhancing the diversity of program participants, program leaders and mentors across multiple sociodemographic domains; leveraging technology to strengthen the MFSP community of practice; and improving flexibility of the program.The program evaluation results provide evidence supporting ongoing investment in faculty educators and summarizes key strengths and areas for improvement to inform future FD efforts for both the MFSP and other FD programs.

Management Reasoning: Empirical Determination of Key Features and a Conceptual Model.

PURPOSE: Management reasoning is a critical yet understudied phenomenon in clinical practice and medical education. The authors sought to empirically identify key features of management reasoning and construct a model describing the management reasoning process. METHOD: In November 2020, 4 investigators each reviewed 10 video clips of simulated outpatient physician-patient encounters and used a coding form to document key features and insights related to management reasoning. The team used a constant comparative approach to distill 120 pages of raw observations into an 18-page list of management tasks, processes, and insights. The team then had a series of discussions to iteratively refine these findings into a parsimonious model of management reasoning. RESULTS: The investigators empirically identified 12 distinct features of management reasoning: contrasting and selection among multiple solutions; prioritization of patient, clinician, and system preferences and constraints; communication and shared decision making; ongoing monitoring and adjustment of the management plan; dynamic interplay among people, systems, and competing priorities; illness-specific knowledge; process knowledge; management scripts; clinician roles as patient teacher and salesperson; clinician-patient relationship; prognostication; and organization of the clinical encounter (sequencing and time management). Management scripts seemed to play a prominent and critical role. The model of management reasoning comprised 4 steps: instantiation of a management script, identifying (multiple) options and beginning to teach the patient, shared decision making, and ongoing monitoring and adjustment. This model also conceives 2 overarching features: that management reasoning is personalized to the patient and that it occurs between individuals rather than exclusively within the clinician's mind. CONCLUSIONS: Management scripts constitute a key feature of management reasoning, along with teaching patients about viable options, shared decision making, ongoing monitoring and adjustment, and personalization. Management reasoning seems to be constructed and negotiated between individuals rather than exclusively within the clinician.

Development and validation of conflict management attitude questionnaire for medical students.

BACKGROUND: Medical students should effectively manage conflicts in teamwork and communication with other team members. This study aimed to develop and validate a tool to evaluate attitude of medical students and physicians toward conflict management. METHOD: A multi-step process was employed to develop and validate a Conflict Management Attitude Questionnaire (CMAQ) based on the steps recommended in AMEE Guide No. 87. First, the initial items were obtained from the literature review and focus group. After cognitive interviews with the medical students and revision of the questionnaire, content validity was performed by experts. The construct validity and reliability of the questionnaire were assessed using exploratory Factor Analysis (EFA) and Cronbach's alpha coefficient, respectively. RESULTS: This multi-step process resulted in a 12-item, five-point Likert-type questionnaire with satisfactory construct validity. Exploratory factor analysis revealed three factors, comprising the four items from the "perceived interactions in conflict management" subscale loading on the first factor, and five items from the "perceived value of learning conflict management" subscale loading on the second factor, along with three items from the "perceived application of conflict management" subscale loading on the third factor. All subscales described 56.48% of the variance. Validation results showed that Content Validity Index (CVI) and Content Validity Ratio (CVR) were greater than 0.75. Cronbach's alpha coefficient was 0.791. CONCLUSION: This study showed that CMAQ has valid evidence for assessing the attitude of medical students toward conflict management with favorable psychometric properties and strong evidence of construct validity. However, due to the lack of evidence on any specific questionnaire to evaluate the attitude towards conflict management, future studies should conduct a confirmatory investigation regarding other aspects of medical students' attitudes toward conflict management.

An outcomes research perspective on medical education: Has anything changed in the last 18 years?

PURPOSE: Medical education research focused on patient-centered outcomes holds the promise of improved decision-making by medical educators. In 2001, Prystowsky and Bordage demonstrated that patient-centered outcomes were evaluated in fewer than one percent of studies published in a survey of major medical education journals. Though many have called for increased inclusion of patient-centered outcomes in medical education literature, it remains uncertain to what degree this need has been addressed systematically. METHODS: Using the same data sources as in the original report (Academic Medicine, Medical Education, and Teaching and Learning in Medicine), we sought to replicate Prystowsky and Bordage's study. We extracted data from original empirical research reports from these three journal sources for the years 2014-2016. We selected 652 articles that met the inclusion criteria for further analysis. RESULTS: Study participants were largely trainees (64% of studies) or faculty (25% of studies). Only 2% of studies included patients as active or passive participants. Study outcomes reported were satisfaction (40% of studies), performance (39%), professionalism (20%), and cost (1%). CONCLUSIONS: These results do not differ significantly from the original 2001 study. The medical education literature as represented in these three prominent journals has made little progress in placing a greater focus on patient-centered outcomes.

Management reasoning scripts: Qualitative exploration using simulated physician-patient encounters.

INTRODUCTION: Management reasoning is distinct from diagnostic reasoning and remains incompletely understood. The authors sought to empirically investigate the concept of management scripts. METHODS: In November 2020, 4 investigators each reviewed 10 video clips of simulated outpatient physician-patient encounters, and used a coding form to document observations about management reasoning. The team used constant comparative analysis to integrate empirically-grounded insights with theories related to cognitive scripts and Type 1/Type 2 thinking. RESULTS: Management scripts are precompiled conceptual knowledge structures that represent and connect management options and clinician tasks in a temporal or logical sequence. Management scripts appear to differ substantially from illness scripts. Management scripts varied in quality (in content, sequence, flexibility, and fluency) and generality. The authors empirically identified six key features (components) of management scripts: the problem (diagnosis); management options; preferences, values, and constraints; education needs; interactions; and encounter flow. The authors propose a heuristic framework describing script activation, selection, instantiation with case-specific details, and application to guide development of the management plan. They further propose that management reasoning reflects iterative, back-and-forth involvement of both Type 1 (non-analytic, effortless) and Type 2 (analytic, effortful) thinking. Type 1 thinking likely influences initial script activation, selection, and initial instantiation. Type 2 increasingly influences subsequent script revisions, as activation, selection, and instantiation become more deliberate (effortful) and more hypothetical (involving mental simulation). DISCUSSION: Management scripts constitute a key feature of management reasoning, and could represent a new target for training in clinical reasoning (distinct from illness scripts).

Assessing changes in the quality of quantitative health educations research: a perspective from communities of practice.

BACKGROUND: As a community of practice (CoP), medical education depends on its research literature to communicate new knowledge, examine alternative perspectives, and share methodological innovations. As a key route of communication, the medical education CoP must be concerned about the rigor and validity of its research literature, but prior studies have suggested the need to improve medical education research quality. Of concern in the present study is the question of how responsive the medical education research literature is to changes in the CoP. We examine the nature and extent of changes in the quality of medical education research over a decade, using a widely cited study of research quality in the medical education research literature as a benchmark to compare more recent quality indicators. METHODS: A bibliometric analysis was conducted to examine the methodologic quality of quantitative medical education research studies published in 13 selected journals from September 2013 to December 2014. Quality scores were calculated for 482 medical education studies using a 10-item Medical Education Research Study Quality Instrument (MERSQI) that has demonstrated strong validity evidence. These data were compared with data from the original study for the same journals in the period September 2002 to December 2003. Eleven investigators representing 6 academic medical centers reviewed and scored the research studies that met inclusion and exclusion criteria. Primary outcome measures include MERSQI quality indicators for 6 domains: study design, sampling, type of data, validity, data analysis, and outcomes. RESULTS: There were statistically significant improvements in four sub-domain measures: study design, type of data, validity and outcomes. There were no changes in sampling quality or the appropriateness of data analysis methods. There was a small but significant increase in the use of patient outcomes in these studies. CONCLUSIONS: Overall, we judge this as equivocal evidence for the responsiveness of the research literature to changes in the medical education CoP. This study identified areas of strength as well as opportunities for continued development of medical education research.

Measuring the Master Adaptive Learner: Development and Internal Structure Validity Evidence for a New Instrument.

BACKGROUND: The master adaptive learner (MAL) uses self-regulated learning skills to develop adaptive, efficient, and accurate skills in practice. Given rapid changes in healthcare, it is essential that medical students develop into MALs. There is a need for an instrument that can capture MAL behaviors and characteristics. The objective of this study was to develop an instrument for measuring the MAL process in medical students and evaluate its psychometric properties. METHODS: As part of curriculum evaluation, 818 students completed previously developed instruments with validity evidence including the Self-Regulated Learning Perception Scale, Brief Resilience Scale, Goal Orientation Scale, and Jefferson Scale of Physician Lifelong Learning. The authors performed exploratory factor analysis to examine underlying relationships between items. Items with high factor loadings were retained. Cronbach's alpha was computed. In parallel, the multi-institutional research team rated the same items to provide content validity evidence of the items to MAL model. RESULTS: The original 67 items were reduced to 28 items loading onto four factors: Planning, Learning, Resilience, and Motivation. Each subscale included the following number of items and Cronbach's alpha: Planning (10 items, alpha = 0.88), Learning (6 items, alpha = 0.81), Resilience (6 items, alpha = 0.89), and Motivation (6 items, alpha = 0.81). The findings from the factor analyses aligned with the research team ratings of linkage to the components of MAL. CONCLUSION: These findings serve as a starting point for future work measuring master adaptive learning to identify and support learners. To fully measure the MAL construct, additional items may need to be developed.

Defining successful outcomes and preferences for clinical management in differences/disorders of sex development: Protocol overview and a qualitative phenomenological study of stakeholders' perspectives.

INTRODUCTION: Utilizing a qualitative phenomenological design, the Defining Successful Outcomes and Trade-offs study examined stakeholder perspectives regarding optimal healthcare delivery and outcomes for individuals with a difference/disorder of sex development (DSD). OBJECTIVE: We describe study methods and provide an overview of themes and subthemes. STUDY DESIGN: Interviews were conducted with individuals with a DSD (n = 24), parents of those with a DSD (n = 19), healthcare providers (n = 37), and others (n = 30). Primary questions regarding clinical management of patients with DSD were: "What is a successful outcome?" and "How do you achieve it?" RESULTS: Themes included: understanding of DSD diagnosis and self-efficacy in management is necessary but complex; patient and family psychological well-being; support from others versus being stigmatized; affected person experiences physical health and accepts the implications of their condition; complexities in DSD decision making, roles and expectations; and knowledgeable providers and multidisciplinary teams are essential, notwithstanding persisting barriers. Participants recognized competing values potentially forcing trade-offs in decision making. DISCUSSION: Recognition of diverse and sometimes conflicting perspectives regarding optimal pathways of care and outcomes - both within and among those with DSD and their providers -promises to enhance shared decision making. CONCLUSION: Diverse perspectives and perceptions of trade-offs associated with DSD healthcare emphasize the need to tailor care for patients and families.

LIMEs and LEMONs: Critically Examining the Effect of a Blog Post on Junior Faculty Learners.

INTRODUCTION: The usage of asynchronous resources such as blogs and podcasts is pervasive in academic medicine, despite little understanding of their actual effect on learner knowledge acquisition. This study sought to examine the objective effect of a blog post on knowledge acquisition and application among junior faculty in emergency medicine (EM) via randomized controlled study. METHODS: All accredited EM residency programs in the United States and Canada were contacted to identify assistant and associate program directors and medical education fellows for recruitment into this study. Upon enrollment, participants were randomized as to whether they received access to a supplemental blog post prior to listening to a podcast episode. After listening to the podcast episode, all participants completed an assessment that included a test of knowledge application and knowledge acquisition; demographic information was also obtained. RESULTS: Ultimately, 103 participants completed the study; the study closed for enrollment in July 2019. Data were nonnormally distributed and groups were compared using the Wilcoxon rank-sum test. There were no significant differences between the demographics of the two groups nor was there a significant difference in knowledge between the two groups. CONCLUSION: The addition of a supplementary blog post did not increase junior faculty knowledge of a podcast episode.

Competency-based time-variable training internationally: Ensuring practical next steps in the wake of the COVID-19 pandemic.

Competency-based medical education has been advocated as the future of medical education for nearly a half-century. Inherent to this is the promise that advancement and transitions in training would be defined by readiness to practice rather than by time. Of the logistical problems facing competency-based, time-variable (CBTV) training, enacting time variability may be the largest hurdle to clear. Although it is true that an 'all or nothing' approach to CBTV training would require massive overhauls of both medical education and health care systems, the authors propose that training institutions should gradually evolve within their current environments to incrementally move toward the best version of CBTV training for learners, supervisors, and patients. In support of this evolution, the authors seek to demonstrate the feasibility of advancing toward the goal of realistic CBTV training by detailing examples of successful CBTV training and describing key features of initial steps toward CBTV training implementation.

The Clarion Call of the COVID-19 Pandemic: How Medical Education Can Mitigate Racial and Ethnic Disparities.

Public health crises palpably demonstrate how social determinants of health have led to disparate health outcomes. The staggering mortality rates among African Americans, Native Americans, and Latinx Americans during the COVID-19 pandemic have revealed how recalcitrant structural inequities can exacerbate disparities and render not just individuals but whole communities acutely vulnerable. While medical curricula that educate students about disparities are vital in rousing awareness, it is experience that is most likely to instill passion for change. The authors first consider the roots of health care disparities in relation to the current pandemic. Then, they examine the importance of salient learning experiences that may inspire a commitment to championing social justice. Experiences in diverse communities can imbue medical students with a desire for lifelong learning and advocacy. The authors introduce a 3-pillar framework that consists of trust building, structural competency, and cultural humility. They discuss how these pillars should underpin educational efforts to improve social determinants of health. Effecting systemic change requires passion and resolve; therefore, perseverance in such efforts is predicated on learners caring about the structural inequities in housing, education, economic stability, and neighborhoods-all of which influence the health of individuals and communities.

What faculty write versus what students see? Perspectives on multiple-choice questions using Bloom's taxonomy.

BACKGROUND: Using revised Bloom's taxonomy, some medical educators assume they can write multiple choice questions (MCQs) that specifically assess higher (analyze, apply) versus lower-order (recall) learning. The purpose of this study was to determine whether three key stakeholder groups (students, faculty, and education assessment experts) assign MCQs the same higher- or lower-order level. METHODS: In Phase 1, stakeholders' groups assigned 90 MCQs to Bloom's levels. In Phase 2, faculty wrote 25 MCQs specifically intended as higher- or lower-order. Then, 10 students assigned these questions to Bloom's levels. RESULTS: In Phase 1, there was low interrater reliability within the student group (Krippendorf's alpha = 0.37), the faculty group (alpha = 0.37), and among three groups (alpha = 0.34) when assigning questions as higher- or lower-order. The assessment team alone had high interrater reliability (alpha = 0.90). In Phase 2, 63% of students agreed with the faculty as to whether the MCQs were higher- or lower-order. There was low agreement between paired faculty and student ratings (Cohen's Kappa range .098-.448, mean .256). DISCUSSION: For many questions, faculty and students did not agree whether the questions were lower- or higher-order. While faculty may try to target specific levels of knowledge or clinical reasoning, students may approach the questions differently than intended.

Considerations on Conducting Research on Wellness in the Context of the Learning Environment.

The learning environment (LE) provides a context for many educational phenomena, of which wellness and burnout are particularly important. The LE can be thought of as consisting of a psychosocial dimension of personal, social, and organizational factors and a sociomaterial dimension that consists of spatial and technical factors. The interplay between elements of the LE and wellness of the participants is complex and only partially understood, requiring further research. Using this multidimensional model to describe and to plan to deliberately modify the learning environment can foster more rigorous and meaningful research evidence about the interaction of wellness and the LE. This article highlights four key considerations that scholars of wellness should consider when exploring the impact of the LE or designing interventions to modify the environment. These include 1) a thoughtful definition and theoretical conceptualization of the LE, 2) clarity about the study variables that are essential to the study question(s), 3) thoughtful and appropriate measurement of those variables, and 4) a study design that balances quality with feasibility. We provide a practical illustration of how these considerations can be applied in studies exploring the intersection of wellness and the LE.