Precision Education: The Future of Lifelong Learning in Medicine.

ABSTRACT: The goal of medical education is to produce a physician workforce capable of delivering high-quality equitable care to diverse patient populations and communities. To achieve this aim amidst explosive growth in medical knowledge and increasingly complex medical care, a system of personalized and continuous learning, assessment, and feedback for trainees and practicing physicians is urgently needed. In this perspective, the authors build on prior work to advance a conceptual framework for such a system: precision education (PE).PE is a system that uses data and technology to transform lifelong learning by improving personalization, efficiency, and agency at the individual, program, and organization levels. PE "cycles" start with data inputs proactively gathered from new and existing sources, including assessments, educational activities, electronic medical records, patient care outcomes, and clinical practice patterns. Through technology-enabled analytics , insights are generated to drive precision interventions . At the individual level, such interventions include personalized just-in-time educational programming. Coaching is essential to provide feedback and increase learner participation and personalization. Outcomes are measured using assessment and evaluation of interventions at the individual, program, and organizational levels, with ongoing adjustment for repeated cycles of improvement. PE is rooted in patient, health system, and population data; promotes value-based care and health equity; and generates an adaptive learning culture.The authors suggest fundamental principles for PE, including promoting equity in structures and processes, learner agency, and integration with workflow (harmonization). Finally, the authors explore the immediate need to develop consensus-driven standards: rules of engagement between people, products, and entities that interact in these systems to ensure interoperability, data sharing, replicability, and scale of PE innovations.

The Next Era of Assessment: Building a Trustworthy Assessment System.

Assessment in medical education has evolved through a sequence of eras each centering on distinct views and values. These eras include measurement (e.g., knowledge exams, objective structured clinical examinations), then judgments (e.g., workplace-based assessments, entrustable professional activities), and most recently systems or programmatic assessment, where over time multiple types and sources of data are collected and combined by competency committees to ensure individual learners are ready to progress to the next stage in their training. Significantly less attention has been paid to the social context of assessment, which has led to an overall erosion of trust in assessment by a variety of stakeholders including learners and frontline assessors. To meaningfully move forward, the authors assert that the reestablishment of trust should be foundational to the next era of assessment. In our actions and interventions, it is imperative that medical education leaders address and build trust in assessment at a systems level. To that end, the authors first review tenets on the social contextualization of assessment and its linkage to trust and discuss consequences should the current state of low trust continue. The authors then posit that trusting and trustworthy relationships can exist at individual as well as organizational and systems levels. Finally, the authors propose a framework to build trust at multiple levels in a future assessment system; one that invites and supports professional and human growth and has the potential to position assessment as a fundamental component of renegotiating the social contract between medical education and the health of the public.

Advancing anti-oppression and social justice in healthcare through competency-based medical education (CBME).

Competency-based medical education (CBME) focuses on preparing physicians to improve the health of patients and populations. In the context of ongoing health disparities worldwide, medical educators must implement CBME in ways that advance social justice and anti-oppression. In this article, authors describe how CBME can be implemented to promote equity pedagogy, an approach to education in which curricular design, teaching, assessment strategies, and learning environments support learners from diverse groups to be successful. The five core components of CBME programs - outcomes competency framework, progressive sequencing of competencies, learning experiences tailored to learners' needs, teaching focused on competencies, and programmatic assessment - enable individualization of learning experiences and teaching and encourage learners to partner with their teachers in driving their learning. These educational approaches appreciate each learner's background, experiences, and strengths. Using an exemplar case study, the authors illustrate how CBME can afford opportunities to enhance anti-oppression and social justice in medical education and promote each learner's success in meeting the expected outcomes of training. The authors provide recommendations for individuals and institutions implementing CBME to enact equity pedagogy.

Maintaining Community During Disruption: Lessons From the Accelerating Change in Medical Education Consortium.

ABSTRACT: When the COVID-19 pandemic emerged early in 2020, the American Medical Association's (AMA) Accelerating Change in Medical Education Consortium focused on maintaining its community of 37 medical schools and 11 graduate medical education projects along with the core substance of its work. The initial response was to cancel events and reduce the workload of consortium members, but it quickly became clear that the consortium needed additional strategies. The constituents needed resources, support, and community. The authors, along with the rest of the AMA team, learned to maximize the benefits and minimize the drawbacks of operating virtually. These insights supported the continuation, and even expansion, of the consortium's innovations despite extremely stressful circumstances. In this Scholarly Perspective, the authors outline the actions that they and the AMA team took in 2020 and 2021 at the beginning of the pandemic and as that crisis progressed and was further intensified by the significant distress reflected in the protests after the murder of George Floyd. The goal of these actions was to maintain the consortium's core substance, innovation, momentum, and sense of community. The authors describe lessons learned in 2020 and 2021 via a novel model developed to facilitate ongoing collaboration and respond rapidly to the needs of overwhelmed medical educators. This model is composed of 4 phases: assessing needs, mining experts, convening people, and generating products. This model for leveraging a community of practice can help educators optimize collaboration, whether educators are pursuing innovation in the training of physicians or other health care professionals and whether in times of extreme stress or stability, the model provides a pathway for maintaining community. The prepandemic way of working will not return. Virtual participation and collaboration will remain a part of work and daily life for the foreseeable future and beyond.

Supporting Medical Education Innovation: Evaluation of a Grants Initiative.

PROBLEM: Medical education must evolve to meet the changing needs of patients and communities. Innovation is a critical component of that evolution. As medical educators pursue innovative curricula, assessments, and evaluation techniques, the impact of innovations may be limited by minimal funding. The American Medical Association (AMA) Innovation Grant Program, launched in 2018, seeks to address the gap in funding and stimulate educational innovation and research in medical education. APPROACH: In 2018 and 2019, the Innovation Grant Program targeted innovation in content areas including health systems science, competency-based medical education, coaching, learning environment, and emerging technology. The authors reviewed the content of applications and final reports for the 27 projects completed during the first 2 years of the program. They also noted measures of success (completion of project, achievement of grant objectives, development of transferrable educational product, dissemination). OUTCOMES: In 2018, the AMA received 52 submissions and funded 13 proposals, dispersing $290,000 ($10,000 and $30,000 grants). In 2019, the AMA received 80 submissions and funded 15 proposals, dispersing $345,000. Among the 27 completed grants, 17 (63%) supported innovations related to health systems science. Fifteen (56%) were used to create educational products that could be shared, such as new assessment tools, curricula, and teaching modules. Five grant recipients (29%) published articles and 15 (56%) presented at national conferences. NEXT STEPS: The grant program advanced educational innovations, particularly in health systems science. The next steps will involve examining the long-term outcomes and impact of the completed projects on medical students, patients, and the health system; the professional development of the grantees; and the adoption and dissemination of the innovations.

Competency-Based Medical Education in a Norm-Referenced World: A Root Cause Analysis of Challenges to the Competency-Based Paradigm in Medical School.

Competency-based medical education (CBME) requires a criterion-referenced approach to assessment. However, despite best efforts to advance CBME, there remains an implicit, and at times, explicit, demand for norm-referencing, particularly at the junction of undergraduate medical education (UME) and graduate medical education (GME). In this manuscript, the authors perform a root cause analysis to determine the underlying reasons for continued norm-referencing in the context of the movement toward CBME. The root cause analysis consisted of 2 processes: (1) identification of potential causes and effects organized into a fishbone diagram and (2) identification of the 5 whys. The fishbone diagram identified 2 primary drivers: the false notion that measures such as grades are truly objective and the importance of different incentives for different key constituents. From these drivers, the importance of norm-referencing for residency selection was identified as a critical component. Exploration of the 5 whys further detailed the reasons for continuation of norm-referenced grading to facilitate selection, including the need for efficient screening in residency selection, dependence upon rank-order lists, perception that there is a best outcome to the match, lack of trust between residency programs and medical schools, and inadequate resources to support progression of trainees. Based on these findings, the authors argue that the implied purpose of assessment in UME is primarily stratification for residency selection. Because stratification requires comparison, a norm-referenced approach is needed. To advance CBME, the authors recommend reconsideration of the approach to assessment in UME to maintain the purpose of selection while also advancing the purpose of rendering a competency decision. Changing the approach will require a collaboration between national organizations, accrediting bodies, GME programs, UME programs, students, and patients/societies. Details are provided regarding the specific approaches required of each key constituent group.

Teaching artificial intelligence as a fundamental toolset of medicine.

Artificial intelligence (AI) is transforming the practice of medicine. Systems assessing chest radiographs, pathology slides, and early warning systems embedded in electronic health records (EHRs) are becoming ubiquitous in medical practice. Despite this, medical students have minimal exposure to the concepts necessary to utilize and evaluate AI systems, leaving them under prepared for future clinical practice. We must work quickly to bolster undergraduate medical education around AI to remedy this. In this commentary, we propose that medical educators treat AI as a critical component of medical practice that is introduced early and integrated with the other core components of medical school curricula. Equipping graduating medical students with this knowledge will ensure they have the skills to solve challenges arising at the confluence of AI and medicine.

Social network analysis of publication collaboration of accelerating change in MedEd consortium.

INTRODUCTION: The American Medical Association formed the Accelerating Change in Medical Education Consortium through grants to effect change in medical education. The dissemination of educational innovations through scholarship was a priority. The objective of this study was to explore the patterns of collaboration of educational innovation through the consortium's publications. METHOD: Publications were identified from grantee schools' semi-annual reports. Each publication was coded for the number of citations, Altmetric score, domain of scholarship, and collaboration with other institutions. Social network analysis explored relationships at the midpoint and end of the grant. RESULTS: Over five years, the 32 Consortium institutions produced 168 publications, ranging from 38 papers from one institution to no manuscripts from another. The two most common domains focused on health system science (92 papers) and competency-based medical education (30 papers). Articles were published in 54 different journals. Forty percent of publications involved more than one institution. Social network analysis demonstrated rich publishing relationships within the Consortium members as well as beyond the Consortium schools. In addition, there was growth of the network connections and density over time. CONCLUSION: The Consortium fostered a scholarship network disseminating a broad range of educational innovations through publications of individual school projects and collaborations.

Defining Successful Practice Within Health Systems Science Among Entering Residents: A Single-Institution Qualitative Study of Graduate Medical Education Faculty Observations.

PURPOSE: The American Medical Association's Accelerating Change in Medical Education consortium defined health systems science (HSS) as the study of how health care is delivered, how health care professionals work together to deliver that care, and how the health system can improve patient care and health care delivery. This framework is increasingly being incorporated into medical school curricula. Graduate medical education (GME) had previously elevated systems-based practice (SBP) as a core competency, but expectations are defined by specialty-specific milestones. The lack of a shared competency framework between undergraduate medical education (UME) and GME makes it challenging to ensure that entering residents are prepared to implement HSS/SBP concepts in the workplace. The authors explored GME faculty observations of residents exemplifying successful practice across HSS domains to inform targets for UME training and assessment. METHOD: Authors performed a single-institution qualitative study using transcribed phone interviews with eligible Vanderbilt residency program directors, associate program directors, and core faculty. Participants described observations of successful residents within each HSS domain. Two researchers independently coded, discussed, and reconciled deidentified transcripts using inductive-deductive approaches to identify themes. RESULTS: Seventeen faculty participated across specialties (17/39, 45%). Faculty responses emphasize precurricular experiences including professional degrees, work experience, extracurriculars, and medical school exposure. Importantly, successful residents exhibit foundational core workforce characteristics including growth mindset, curiosity, and a desire to learn about systems. GME faculty identified HSS domain-specific skills, noting distinctions among learning environments. Outcomes resulting from residents' application of HSS concepts include delivering high-quality, person-centered care and systems improvements. CONCLUSIONS: Descriptions of successful practice within HSS domains highlight preparatory experiences and core workforce characteristics and outline entry-level HSS behaviors. Conceptualized in a logic model framework, these findings describe key inputs, learning activities, outputs, and outcomes for systems-prepared entering residents bridging the UME-GME transition.

Exploiting the power of information in medical education.

The explosion of medical information demands a thorough reconsideration of medical education, including what we teach and assess, how we educate, and whom we educate. Physicians of the future will need to be self-aware, self-directed, resource-effective team players who can synthesize and apply summarized information and communicate clearly. Training in metacognition, data science, informatics, and artificial intelligence is needed. Education programs must shift focus from content delivery to providing students explicit scaffolding for future learning, such as the Master Adaptive Learner model. Additionally, educators should leverage informatics to improve the process of education and foster individualized, precision education. Finally, attributes of the successful physician of the future should inform adjustments in recruitment and admissions processes. This paper explores how member schools of the American Medical Association Accelerating Change in Medical Education Consortium adjusted all aspects of educational programming in acknowledgment of the rapid expansion of information.

The critical role of infrastructure and organizational culture in implementing competency-based education and individualized pathways in undergraduate medical education.

In 2010, several key works in medical education predicted the changes necessary to train modern physicians to meet current and future challenges in health care, including the standardization of learning outcomes paired with individualized learning processes. The reframing of a medical expert as a flexible, adaptive team member and change agent, effective within a larger system and responsive to the community's needs, requires a new approach to education: competency-based medical education (CBME). CBME is an outcomes-based developmental approach to ensuring each trainee's readiness to advance through stages of training and continue to grow in unsupervised practice. Implementation of CBME with fidelity is a complex and challenging endeavor, demanding a fundamental shift in organizational culture and investment in appropriate infrastructure. This paper outlines how member schools of the American Medical Association Accelerating Change in Medical Education Consortium developed and implemented CBME, including common challenges and successes. Critical supporting factors include adoption of the master adaptive learner construct, longitudinal views of learner development, coaching, and a supportive learning environment.

The power of interdependence: Linking health systems, communities, and health professions educational programs to better meet the needs of patients and populations.

Promoting optimal health outcomes for diverse patients and populations requires the acknowledgement and strengthening of interdependent relationships between health professions education programs, health systems, and the communities they serve. Educational programs must recognize their role as integral components of a larger system. Educators must strive to break down silos and synergize efforts to foster a health care workforce positioned for collaborative, equitable, community-oriented practice. Sharing interprofessional and interinstitutional strategies can foster wide propagation of educational innovation while accommodating local contexts. This paper outlines how member schools of the American Medical Association Accelerating Change in Medical Education Consortium leveraged interdependence to accomplish transformative innovations catalyzed by systems thinking and a community of innovation.

Realizing the vision of the Lancet Commission on Education of Health Professionals for the 21st Century: Transforming medical education through the Accelerating Change in Medical Education Consortium.

In the last two decades, prompted by the anticipated arrival of the 21st Century and on the centenary of the publication of the Flexner Report, many in medical education called for change to address the expanding chasm between the requirements of the health care system and the educational systems producing the health care workforce. Calls were uniform. Curricular content was missing. There was a mismatch in where people trained and where they were needed to practice, legacy approaches to pedagogical methods that needed to be challenged, an imbalance in diversity of trainees, and a lack of research on educational outcomes, resulting in a workforce that was described as ill-equipped to provide health care in the current and future environment. The Lancet Commission on Education of Health Professionals for the 21st Century published a widely acclaimed report in 2010 that called for a complete and authoritative re-examination of health professional education. This paper describes the innovations of the American Medical Association Accelerating Change in Medical Education Consortium schools as they relate to the recommendations of the Lancet Commission. We outline the successes, challenges, and lessons learned in working to deeply reform medical education.

Expanding innovation from undergraduate to graduate medical education: A path of continuous professional development.

A hundred years after the Flexner report laid the foundation for modern medical education, a number of authors commemorated the occasion by commenting on how the medical education system had to change once more to serve 21st century patients. Experts called for standardized outcomes and individualized learner pathways, integration of material across traditional areas, attention to an environment of inquiry, and professional identity formation. The medical education community responded and much has been achieved in the last decade, but much work remains to be done. In this paper we outline how the American Medical Association Accelerating Change in Medical Education Consortium, launched in 2013 through significant funding of transformation projects in undergraduate medical education, expanded its work into graduate medical education, and we look to the future of innovation in medical education.

The Accelerating Change in Medical Education Consortium: Key Drivers of Transformative Change.

The American Medical Association's (AMA's) Accelerating Change in Medical Education (ACE) initiative, launched in 2013 to foster advancements in undergraduate medical education, has led to the development and scaling of innovations influencing the full continuum of medical training. Initial grants of $1 million were awarded to 11 U.S. medical schools, with 21 schools joining the consortium in 2016 at a lower funding level. Almost one-fifth of all U.S. MD- and DO-granting medical schools are represented in the 32-member consortium. In the first 5 years, the consortium medical schools have delivered innovative educational experiences to approximately 19,000 medical students, who will provide a potential 33 million patient care visits annually. The core initiative objectives focus on competency-based approaches to medical education and individualized pathways for students, training in health systems science, and enhancing the learning environment. At the close of the initial 5-year grant period, AMA leadership sought to catalogue outputs and understand how the structure of the consortium may have influenced its outcomes. Themes from qualitative analysis of stakeholder interviews as well as other sources of evidence aligned with the 4 elements of the transformational leadership model (inspirational motivation, intellectual stimulation, individualized consideration, and idealized influence) and can be used to inform future innovation interventions. For example, the ACE initiative has been successful in stimulating change at the consortium schools and propagating those innovations broadly, with outputs involving medical students, faculty, medical schools, affiliated health systems, and the broader educational landscape. In summary, the ACE initiative has fostered a far-reaching community of innovation that will continue to drive change across the continuum of medical education.

Assessing medical student performance of Entrustable Professional Activities: A mixed methods comparison of Co-Activity and Supervisory Scales.

Introduction: Observations of medical student participation in entrustable professional activities (EPAs) provide insight into the student's ability to synthesize competencies across domains and effectively function in different clinical scenarios. Both Supervisory and Co-Activity Assessment Scales have been recommended for use with medical students.Methods: Students were assessed on EPAs during Acting Internships in Medicine and Pediatrics. Two rating scales were modified based on expert review and included throughout the 2017-18 academic year. Statistical analysis was conducted to clarify relationships between the scales. Raters were interviewed to explore their interpretations and response processes.Results: The results of the McNemar test suggest that the scales are different (p-value <.01). Co-activity and Supervisory EPA ratings are related, but not interchangeable. This finding concurs with themes that emerged from response process interviews: (1) the scales are not directly parallel (2) rater preference depends on diverse factors and (3) rater comments are crucial for guiding students' future learning.Conclusion: The modified Chen Supervisory Scale and the modified Ottawa Co-Activity Scales are measuring different aspects of the entrustable professional activity landscape. Both scales can provide useful information to the learner and the assessment system, but they should not be treated as interchangeable assessments.

Integrating Foundational Sciences in a Clinical Context in the Post-Clerkship Curriculum.

PURPOSE: To design, implement, and launch courses that integrate foundational science learning and clinical application in a post-clerkship undergraduate medical school curriculum. METHOD: In Academic Year (AY) 15-16, as part of a comprehensive curricular revision, Vanderbilt University School of Medicine (VUSM) formally implemented "Integrated Science Courses" (ISCs) that combined rigorous training in the foundational sciences with meaningful clinical experiences. These courses integrated foundational sciences that could be leveraged in the clinical environment, utilized a variety of instructional modalities, and included quantitative and qualitative (competency-based milestones) student assessments. Each ISC underwent a rigorous quality improvement process that required input on foundational science content, student experience, and student performance assessment. RESULTS: Eleven ISCs were delivered to 173 students in AY15-16, with some students taking more than one ISC. Immediately after completing each course, 93% (n=222) of ISC enrollees completed a course evaluation. Students (91%; n=201) 'agreed' or 'strongly agreed' that foundational science learning informed and enriched the clinical experiences. Furthermore, 94% (n=209) of students thought that the clinical experiences informed and enriched the foundational science learning. Ninety-four percent of the students anticipated using the foundational science knowledge acquired in future clinical training and practice. CONCLUSION: The teaching of foundational sciences in the clinical workplace in the post-clerkship medical curriculum is challenging and resource-intensive, yet feasible. Additional experience with the model will inform the mix of courses as well as the breadth and depth of foundational science instruction that is necessary to foster scientifically-based clinical reasoning skills in each student.

Competency milestones for medical students: Design, implementation, and analysis at one medical school.

Competency-based assessment seeks to align measures of performance directly with desired learning outcomes based upon the needs of patients and the healthcare system. Recognizing that assessment methods profoundly influence student motivation and effort, it is critical to measure all desired aspects of performance throughout an individual's medical training. The Accreditation Council for Graduate Medical Education (ACGME) defined domains of competency for residency; the subsequent Milestones Project seeks to describe each learner's progress toward competence within each domain. Because the various clinical disciplines defined unique competencies and milestones within each domain, it is difficult for undergraduate medical education to adopt existing GME milestones language. This paper outlines the process undertaken by one medical school to design, implement and improve competency milestones for medical students. A team of assessment experts developed milestones for a set of focus competencies; these have now been monitored in medical students over two years. A unique digital dashboard enables individual, aggregate and longitudinal views of student progress by domain. Validation and continuous quality improvement cycles are based upon expert review, user feedback, and analysis of variation between students and between assessors. Experience to date indicates that milestone-based assessment has significant potential to guide the development of medical students.

From Theory to Practice: Utilizing Competency-based Milestones to Assess Professional Growth and Development in the Foundational Science Blocks of a Pre-Clerkship Medical School Curriculum.

Physicians-in-training require skills and attitudes beyond medical knowledge in order to mature into successful clinicians. However, because assessments in pre-clerkship curricula historically have focused almost exclusively on medical knowledge, faculty contributions to early student development often have been limited. To address this challenge and enhance student progress, we re-designed our pre-clerkship curriculum to include settings in which diverse facets of student performance could be observed and fostered. Concurrently, we transitioned to an assessment strategy focused on competency-based milestones. The implementation of this strategy has allowed pre-clerkship science faculty to provide early-stage students with rich holistic feedback designed to stimulate their professional growth.