Association of Burnout and Other Factors Within Physicians and Advanced Practice Providers in an Academic Surgery Department.

BACKGROUND: Provider burnout is a work-related syndrome that is under-recognized, under-reported, and has negative repercussions on the individual, system, and patients. This study investigated burnout incidence and its association with wellness characteristics such as resilience, psychological safety, and perceptions of the workplace to inform future work in improving well-being. METHODS: Electronic surveys were sent to 153 physicians and advanced practice providers (APPs) in the department of surgery at a single institution. Survey topics included demographics, intention to stay, engagement, and items from validated measures for workplace perceptions including work pace/stress (Mini Z), burnout, psychological safety, and resilience. Descriptive statistics, bivariate associations, and logistic regression were used to evaluate responses. RESULTS: Overall response rate was 47%. The majority of providers reported feeling burned out (56%), and 48% indicated they would probably leave the organization within three years. Additionally, 61% reported being satisfied with their job and 55% felt that they contributed professionally in the ways they value most (meaningful work/engagement). Significant predictors for burnout included negative work environment perceptions (work pace/stress), low resilience, low meaningful work, and professional role (physician vs APP). DISCUSSION: Maintaining a healthy workforce requires investigation into the factors that support workplace well-being. The strongest predictors of burnout were work pace/stress. Protective factors against burnout were psychological safety and resilience. An organizational culture that promotes psychological safety, as well as workplace improvements to enhance providers' sense of meaning in work, and decreasing work pace and stress may contribute to the prevention of burnout and the retention.

Applying the master adaptive learner framework to just-in-time training of procedures.

Background: Just-in-time training (JITT) occurs in the clinical context when learners need immediate guidance for procedures due to a lack of proficiency or the need for knowledge refreshment. The master adaptive learner (MAL) framework presents a comprehensive model of transforming learners into adaptive experts, proficient not only in their current tasks but also in the ongoing development of lifelong skills. With the evolving landscape of procedural competence in emergency medicine (EM), trainees must develop the capacity to acquire and master new techniques consistently. This concept paper will discuss using JITT to support the development of MALs in the emergency department. Methods: In May 2023, an expert panel from the Society for Academic Emergency Medicine (SAEM) Medical Educator's Boot Camp delivered a comprehensive half-day preconference session entitled "Be the Best Teacher" at the society's annual meeting. A subgroup within this panel focused on applying the MAL framework to JITT. This subgroup collaboratively developed a practical guide that underwent iterative review and refinement. Results: The MAL-JITT framework integrates the learner's past experiences with the educator's proficiency, allowing the educational experience to address the unique requirements of each case. We outline a structured five-step process for applying JITT, utilizing the lumbar puncture procedure as an example of integrating the MAL stages of planning, learning, assessing, and adjusting. This innovative approach facilitates prompt procedural competence and cultivates a positive learning environment that fosters acquiring adaptable learning skills with enduring benefits throughout the learner's career trajectory. Conclusions: JITT for procedures holds the potential to cultivate a dynamic learning environment conducive to nurturing the development of MALs in EM.

Defining Types of Leadership Within an Academic Surgery Department to Promote Change for Decreasing Rates of Burnout.

OBJECTIVES: Successful leaders influence the group they represent. Effective surgical care is tied to its leadership climate. However, most surgical providers are not attuned to their individual strengths which if known they could leverage them within their teams. This study identifies leadership types within a department of surgery which may be used to better understand and cultivate their strengths. METHODS: In 2022, 172 providers in an academic surgery department were offered the GallupTM CliftonStrengths assessment, a proprietary instrument that maps 34 strengths across 4 domains of leadership. The assessment provides a respondent with their top 5 strengths and the domain in which they naturally "lead". RESULTS: Of 172 providers, 127 (74%) completed the assessment. While providers have strengths in multiple domains, they "lead with" a specific domain. Mapped from the providers' top 10 strengths, the most common "lead with" domain for surgical providers was Executing: the ability to implement ideas and produce results. Strategic Thinking: those who are analytical and push teams forward and Relationship Building: the ability to create strong and effective teams were followed by the least common domain. Influencing: the ability to communicate ideas and lead others. Formal leaders were significantly more likely to lead with Strategic Thinking. There were no significant differences between APPs and physicians. CONCLUSION: A majority of surgical providers "lead with" the GallupTM Executing domain. Those who lead with executing skills work tirelessly to produce outcomes. Learning to leverage the strengths of our teams to create cohesion and efficiency may improve engagement and retention.

The Association of Master Adaptive Learning With Less Burnout and More Resilience in Obstetrics and Gynecology Residents.

PURPOSE: Medical education should prepare learners for complex and evolving work, and should ideally include the Master Adaptive Learner (MAL) model-meta-learning skills for continuous self-regulated learning. This study aimed to measure obstetrics and gynecology (OB/GYN) residents' MAL attributes, assess associations with burnout and resilience, and explore learning task associations with MAL. METHOD: OB/GYN residents were surveyed electronically at an in-training examination in January 2022. The survey included demographic information, the 2-item Maslach Burnout Inventory, the 2-item Connor-Davidson Resilience Scale, 4 MAL items (e.g., "I take every opportunity to learn new things"), and questions about training and learning experiences. RESULTS: Of 5,761 residents, 3,741 respondents (65%) were included. A total of 1,478 of 3,386 (39%) demonstrated burnout (responded positive for burnout on emotional exhaustion or depersonalization items). The mean (SD) Connor-Davidson Resilience Scale score was 6.4 (1.2) of a total possible score of 8. The mean (SD) MAL score was 16.3 (2.8) of a total possible score of 20. The MAL score was inversely associated with burnout, with lower MAL scores for residents with (mean [SD] MAL score, 16.5 [2.4]) vs without (mean [SD], 16.0 [2.3]) burnout (P < .001). Higher MAL scores were associated with higher resilience (R = 0.29, P < .001). Higher MAL scores were associated with the statement, "I feel that I was well prepared for my first year of residency" (R = 0.19, P < .001) and a plan to complete subspecialty training after residency (mean [SD] of 16.6 [2.4] for "yes" and 16.2 [2.4] for "no," P < .001). CONCLUSIONS: Residents who scored higher on MAL showed more resilience and less burnout. Whether less resilient, burned-out residents did not have the agency to achieve MAL status or whether MAL behaviors filled the resiliency reservoir and protected against burnout is not clear.

Understanding resident wellness: A path analysis of the clinical learning environment at three institutions.

PURPOSE: The clinical learning environment (CLE) affects resident physician well-being. This study assessed how aspects of the learning environment affected the level of resident job stress and burnout. MATERIALS AND METHODS: Three institutions surveyed residents assessing aspects of the CLE and well-being via anonymous survey in fall of 2020 during COVID. Psychological safety (PS) and perceived organizational support (POS) were used to capture the CLE, and the Mini-Z Scale was used to assess resident job stress and burnout. A total of 2,196 residents received a survey link; 889 responded (40% response rate). Path analysis explored both direct and indirect relationships between PS, POS, resident stress, and resident burnout. RESULTS: Both POS and PS had significant negative relationships with experiencing a great deal of job stress; the relationship between PS and stress was noticeably stronger than POS and stress (POS: B= -0.12, p=.025; PS: B= -0.37, p<.001). The relationship between stress and residents' level of burnout was also significant (B = 0.38, p<.001). The overall model explained 25% of the variance in resident burnout. CONCLUSIONS: Organizational support and psychological safety of the learning environment is associated with resident burnout. It is important for educational leaders to recognize and mitigate these factors.

Association between the American Board of Emergency Medicine Oral Certifying Examination and Future State Medical Board Disciplinary Actions.

Objectives: The American Board of Emergency Medicine (ABEM) requires a written examination (the Qualifying Examination) followed by the Oral Certifying Examination (OCE) to obtain ABEM certification. Maintaining ABEM certification is associated with fewer state medical board (SMB) disciplinary actions. We sought to determine the association between poor initial performance on the OCE and subsequent severe SMB disciplinary action. Methods: We included physicians who completed US categorical emergency medicine residencies in 2016 and earlier. We classified OCE performance as good (passed on first attempt) and poor (never passed or required > 1 attempt to pass). We obtained data on physician SMB disciplinary actions from the National Practitioner Data Bank that were limited to actions that denied licensure or altered the status of a medical license (eg, suspension). We determined the association between poor OCE performance and subsequent severe SMB disciplinary action. Results: Of 34,871, 93.5% passed the OCE on the first attempt, 6.1% required multiple attempts, and 0.3% never passed. Of the physicians (93.5%) with good OCE performance, 1.0% received a severe SMB action. Among physicians with poor OCE performance, 2.3% received a severe action; and of those who never passed, 1.7% received a severe action (Table 1). Poor OCE performance was associated with an increased odds of severe SMB disciplinary action (OR 2.21, 95% CI: 1.57-3.12). Conclusion: Physicians with poor OCE performance exhibited higher odds of experiencing a subsequent severe SMB disciplinary action. The OCE may have utility as a predictor of future professionalism or clinical performance.

Learner Assessment and Program Evaluation: Supporting Precision Education.

ABSTRACT: Precision education (PE) systematically leverages data and advanced analytics to inform educational interventions that, in turn, promote meaningful learner outcomes. PE does this by incorporating analytic results back into the education continuum through continuous feedback cycles. These data-informed sequences of planning, learning, assessing, and adjusting foster competence and adaptive expertise. PE cycles occur at individual (micro), program (meso), or system (macro) levels. This article focuses on program- and system-level PE.Data for PE come from a multitude of sources, including learner assessment and program evaluation. The authors describe the link between these data and the vital role evaluation plays in providing evidence of educational effectiveness. By including prior program evaluation research supporting this claim, the authors illustrate the link between training programs and patient outcomes. They also describe existing national reports providing feedback to programs and institutions, as well as 2 emerging, multiorganization program- and system-level PE efforts. The challenges encountered by those implementing PE and the continuing need to advance this work illuminate the necessity for increased cross-disciplinary collaborations and a national cross-organizational data-sharing effort.Finally, the authors propose practical approaches for funding a national initiative in PE as well as potential models for advancing the field of PE. Lessons learned from successes by others illustrate the promise of these recommendations.

Precision Education: The Future of Lifelong Learning in Medicine.

ABSTRACT: The goal of medical education is to produce a physician workforce capable of delivering high-quality equitable care to diverse patient populations and communities. To achieve this aim amidst explosive growth in medical knowledge and increasingly complex medical care, a system of personalized and continuous learning, assessment, and feedback for trainees and practicing physicians is urgently needed. In this perspective, the authors build on prior work to advance a conceptual framework for such a system: precision education (PE).PE is a system that uses data and technology to transform lifelong learning by improving personalization, efficiency, and agency at the individual, program, and organization levels. PE "cycles" start with data inputs proactively gathered from new and existing sources, including assessments, educational activities, electronic medical records, patient care outcomes, and clinical practice patterns. Through technology-enabled analytics , insights are generated to drive precision interventions . At the individual level, such interventions include personalized just-in-time educational programming. Coaching is essential to provide feedback and increase learner participation and personalization. Outcomes are measured using assessment and evaluation of interventions at the individual, program, and organizational levels, with ongoing adjustment for repeated cycles of improvement. PE is rooted in patient, health system, and population data; promotes value-based care and health equity; and generates an adaptive learning culture.The authors suggest fundamental principles for PE, including promoting equity in structures and processes, learner agency, and integration with workflow (harmonization). Finally, the authors explore the immediate need to develop consensus-driven standards: rules of engagement between people, products, and entities that interact in these systems to ensure interoperability, data sharing, replicability, and scale of PE innovations.

How well do workplace-based assessments support summative entrustment decisions? A multi-institutional generalisability study.

BACKGROUND: Assessment of the Core Entrustable Professional Activities for Entering Residency requires direct observation through workplace-based assessments (WBAs). Single-institution studies have demonstrated mixed findings regarding the reliability of WBAs developed to measure student progression towards entrustment. Factors such as faculty development, rater engagement and scale selection have been suggested to improve reliability. The purpose of this investigation was to conduct a multi-institutional generalisability study to determine the influence of specific factors on reliability of WBAs. METHODS: The authors analysed WBA data obtained for clerkship-level students across seven institutions from 2018 to 2020. Institutions implemented a variety of strategies including selection of designated assessors, altered scales and different EPAs. Data were aggregated by these factors. Generalisability theory was then used to examine the internal structure validity evidence of the data. An unbalanced cross-classified random-effects model was used to decompose variance components. A phi coefficient of >0.7 was used as threshold for acceptable reliability. RESULTS: Data from 53 565 WBAs were analysed, and a total of 77 generalisability studies were performed. Most data came from EPAs 1 (n = 17 118, 32%) 2 (n = 10 237, 19.1%), and 6 (n = 6000, 18.5%). Low variance attributed to the learner (<10%) was found for most (59/77, 76%) analyses, resulting in a relatively large number of observations required for reasonable reliability (range = 3 to >560, median = 60). Factors such as DA, scale or EPA were not consistently associated with improved reliability. CONCLUSION: The results from this study describe relatively low reliability in the WBAs obtained across seven sites. Generalisability for these instruments may be less dependent on factors such as faculty development, rater engagement or scale selection. When used for formative feedback, data from these instruments may be useful. However, such instruments do not consistently provide reasonable reliability to justify their use in high-stakes summative entrustment decisions.

Mental Practice: Applying Successful Strategies in Sports to the Practice of Emergency Medicine.

Emergency physicians are expected to learn and maintain a large and varied set of competencies for clinical practice. These include high acuity, low occurrence procedures that may not be encountered frequently in the clinical environment and are difficult to practice with high fidelity and frequency in a simulated environment. Mental practice is a form of a cognitive walk-through that has been shown to be an effective method for improving motor and cognitive skills, with literature in sports science and emerging evidence supporting its use in medicine. In this article, we review the literature on mental practice in sports and medicine as well as the underlying neuroscientific theories that support its use. We review best-known practices and provide a framework to design and use mental imagery scripts to augment learning and maintaining the competencies necessary for physicians at all levels of training and clinical environments in the practice of emergency medicine.

Making Use of Natural Language Processing to Better Understand Medical Students' Self-Assessment of Clinical Skills.

PROBLEM: Reflective practice is necessary for self-regulated learning. Helping medical students develop these skills can be challenging since they are difficult to observe. One common solution is to assign students' reflective self-assessments, which produce large quantities of narrative assessment data. Reflective self-assessments also provide feedback to faculty regarding students' understanding of content, reflective abilities, and areas for course improvement. To maximize student learning and feedback to faculty, reflective self-assessments must be reviewed and analyzed, activities that are often difficult for faculty due to the time-intensive and cumbersome nature of processing large quantities of narrative assessment data. APPROACH: The authors collected narrative assessment data (2,224 students' reflective self-assessments) from 344 medical students' reflective self-assessments. In academic years 2019-2020 and 2021-2022, students at the University of Cincinnati College of Medicine responded to 2 prompts (aspects that surprised students, areas for student improvement) after reviewing their standardized patient encounters. These free-text entries were analyzed using TopEx, an open-source natural language processing (NLP) tool, to identify common topics and themes, which faculty then reviewed. OUTCOMES: TopEx expedited theme identification in students' reflective self-assessments, unveiling 10 themes for prompt 1 such as question organization and history analysis, and 8 for prompt 2, including sensitive histories and exam efficiency. Using TopEx offered a user-friendly, time-saving analysis method without requiring complex NLP implementations. The authors discerned 4 education enhancement implications: aggregating themes for future student reflection, revising self-assessments for common improvement areas, adjusting curriculum to guide students better, and aiding faculty in providing targeted upcoming feedback. NEXT STEPS: The University of Cincinnati College of Medicine aims to refine and expand the utilization of TopEx for deeper narrative assessment analysis, while other institutions may model or extend this approach to uncover broader educational insights and drive curricular advancements.

Using Theory-Informed Group Processes to Make TIMELESS Promotion Decisions.

ABSTRACT: Competency-based medical education (CBME) depends on effective programs of assessment to achieve the desired outcomes and goals of training. Residency programs must be able to defend clinical competency committee (CCC) group decisions about learner readiness for practice, including decisions about time-variable resident promotion and graduation. In this article, the authors describe why CCC group decision-making processes should be supported by theory and review 3 theories they used in designing their group processes: social decision scheme theory, functional theory, and wisdom of crowds. They describe how these theories were applied in a competency-based, time-variable training pilot-Transitioning in Internal Medicine Education Leveraging Entrustment Scores Synthesis (TIMELESS) at the University of Cincinnati internal medicine residency program in 2020-2022-to increase the defensibility of their CCC group decision-making. This work serves as an example of how use of theory can bolster validity arguments supporting group decisions about resident readiness for practice.

Maintaining Community During Disruption: Lessons From the Accelerating Change in Medical Education Consortium.

ABSTRACT: When the COVID-19 pandemic emerged early in 2020, the American Medical Association's (AMA) Accelerating Change in Medical Education Consortium focused on maintaining its community of 37 medical schools and 11 graduate medical education projects along with the core substance of its work. The initial response was to cancel events and reduce the workload of consortium members, but it quickly became clear that the consortium needed additional strategies. The constituents needed resources, support, and community. The authors, along with the rest of the AMA team, learned to maximize the benefits and minimize the drawbacks of operating virtually. These insights supported the continuation, and even expansion, of the consortium's innovations despite extremely stressful circumstances. In this Scholarly Perspective, the authors outline the actions that they and the AMA team took in 2020 and 2021 at the beginning of the pandemic and as that crisis progressed and was further intensified by the significant distress reflected in the protests after the murder of George Floyd. The goal of these actions was to maintain the consortium's core substance, innovation, momentum, and sense of community. The authors describe lessons learned in 2020 and 2021 via a novel model developed to facilitate ongoing collaboration and respond rapidly to the needs of overwhelmed medical educators. This model is composed of 4 phases: assessing needs, mining experts, convening people, and generating products. This model for leveraging a community of practice can help educators optimize collaboration, whether educators are pursuing innovation in the training of physicians or other health care professionals and whether in times of extreme stress or stability, the model provides a pathway for maintaining community. The prepandemic way of working will not return. Virtual participation and collaboration will remain a part of work and daily life for the foreseeable future and beyond.

The Next Era of Assessment: Can Ensuring High-Quality, Equitable Patient Care Be the Defining Characteristic?

ABSTRACT: Previous eras of assessment in medical education have been defined by how assessment is done, from knowledge exams popularized in the 1960s to the emergence of work-based assessment in the 1990s to current efforts to integrate multiple types and sources of performance data through programmatic assessment. Each of these eras was a response to why assessment was performed (e.g., assessing medical knowledge with exams; assessing communication, professionalism, and systems competencies with work-based assessment). Despite the evolution of assessment eras, current evidence highlights the graduation of trainees with foundational gaps in the ability to provide high-quality care to patients presenting with common problems, and training program leaders report they graduate trainees they would not trust to care for themselves or their loved ones. In this article, the authors argue that the next era of assessment should be defined by why assessment is done: to ensure high-quality, equitable care. Assessment should place focus on demanding graduates possess the knowledge, skills, attitudes, and adaptive expertise to meet the needs of all patients and ensuring that graduates are able to do this in an equitable fashion. The authors explore 2 patient-focused assessment approaches that could help realize the promise of this envisioned era: entrustable professional activities (EPAs) and resident sensitive quality measures (RSQMs)/TRainee Attributable and Automatable Care Evaluations in Real-time (TRACERs). These examples illustrate how the envisioned next era of assessment can leverage existing and new data to provide precision education assessment that focuses on providing formative and summative feedback to trainees in a manner that seeks to ensure their learning outcomes prepare them to ensure high-quality, equitable patient outcomes.

AI-Enabled Medical Education: Threads of Change, Promising Futures, and Risky Realities Across Four Potential Future Worlds.

BACKGROUND: The rapid trajectory of artificial intelligence (AI) development and advancement is quickly outpacing society's ability to determine its future role. As AI continues to transform various aspects of our lives, one critical question arises for medical education: what will be the nature of education, teaching, and learning in a future world where the acquisition, retention, and application of knowledge in the traditional sense are fundamentally altered by AI? OBJECTIVE: The purpose of this perspective is to plan for the intersection of health care and medical education in the future. METHODS: We used GPT-4 and scenario-based strategic planning techniques to craft 4 hypothetical future worlds influenced by AI's integration into health care and medical education. This method, used by organizations such as Shell and the Accreditation Council for Graduate Medical Education, assesses readiness for alternative futures and effectively manages uncertainty, risk, and opportunity. The detailed scenarios provide insights into potential environments the medical profession may face and lay the foundation for hypothesis generation and idea-building regarding responsible AI implementation. RESULTS: The following 4 worlds were created using OpenAI's GPT model: AI Harmony, AI conflict, The world of Ecological Balance, and Existential Risk. Risks include disinformation and misinformation, loss of privacy, widening inequity, erosion of human autonomy, and ethical dilemmas. Benefits involve improved efficiency, personalized interventions, enhanced collaboration, early detection, and accelerated research. CONCLUSIONS: To ensure responsible AI use, the authors suggest focusing on 3 key areas: developing a robust ethical framework, fostering interdisciplinary collaboration, and investing in education and training. A strong ethical framework emphasizes patient safety, privacy, and autonomy while promoting equity and inclusivity. Interdisciplinary collaboration encourages cooperation among various experts in developing and implementing AI technologies, ensuring that they address the complex needs and challenges in health care and medical education. Investing in education and training prepares professionals and trainees with necessary skills and knowledge to effectively use and critically evaluate AI technologies. The integration of AI in health care and medical education presents a critical juncture between transformative advancements and significant risks. By working together to address both immediate and long-term risks and consequences, we can ensure that AI integration leads to a more equitable, sustainable, and prosperous future for both health care and medical education. As we engage with AI technologies, our collective actions will ultimately determine the state of the future of health care and medical education to harness AI's power while ensuring the safety and well-being of humanity.

Transition to Residency: National Study of Factors Contributing to Variability in Learner Milestones Ratings in Emergency Medicine and Family Medicine.

PURPOSE: The developmental trajectory of learning during residency may be attributed to multiple factors, including variation in individual trainee performance, program-level factors, graduating medical school effects, and the learning environment. Understanding the relationship between medical school and learner performance during residency is important in prioritizing undergraduate curricular strategies and educational approaches for effective transition to residency and postgraduate training. This study explores factors contributing to longitudinal and developmental variability in resident Milestones ratings, focusing on variability due to graduating medical school, training program, and learners using national cohort data from emergency medicine (EM) and family medicine (FM). METHOD: Data from programs with residents entering training in July 2016 were used (EM: n=1,645 residents, 178 residency programs; FM: n=3,997 residents, 487 residency programs). Descriptive statistics were used to examine data trends. Cross-classified mixed-effects regression were used to decompose variance components in Milestones ratings. RESULTS: During postgraduate year (PGY)-1, graduating medical school accounted for 5% and 6% of the variability in Milestones ratings, decreasing to 2% and 5% by PGY-3 for EM and FM, respectively. Residency program accounted for substantial variability during PGY-1 (EM=70%, FM=53%) but decreased during PGY-3 (EM=62%, FM=44%), with greater variability across training period in patient care (PC), medical knowledge (MK), and systems-based practice (SBP). Learner variance increased significantly between PGY-1 (EM=23%, FM=34%) and PGY-3 (EM=34%, FM=44%), with greater variability in practice-based learning and improvement (PBLI), professionalism (PROF), and interpersonal communication skills (ICS). CONCLUSIONS: The greatest variance in Milestone ratings can be attributed to the residency program and to a lesser degree, learners, and medical school. The dynamic impact of program-level factors on learners shifts during the first year and across the duration of residency training, highlighting the influence of curricular, instructional, and programmatic factors on resident performance throughout residency.