Precision Education: The Future of Lifelong Learning in Medicine.

ABSTRACT: The goal of medical education is to produce a physician workforce capable of delivering high-quality equitable care to diverse patient populations and communities. To achieve this aim amidst explosive growth in medical knowledge and increasingly complex medical care, a system of personalized and continuous learning, assessment, and feedback for trainees and practicing physicians is urgently needed. In this perspective, the authors build on prior work to advance a conceptual framework for such a system: precision education (PE).PE is a system that uses data and technology to transform lifelong learning by improving personalization, efficiency, and agency at the individual, program, and organization levels. PE "cycles" start with data inputs proactively gathered from new and existing sources, including assessments, educational activities, electronic medical records, patient care outcomes, and clinical practice patterns. Through technology-enabled analytics , insights are generated to drive precision interventions . At the individual level, such interventions include personalized just-in-time educational programming. Coaching is essential to provide feedback and increase learner participation and personalization. Outcomes are measured using assessment and evaluation of interventions at the individual, program, and organizational levels, with ongoing adjustment for repeated cycles of improvement. PE is rooted in patient, health system, and population data; promotes value-based care and health equity; and generates an adaptive learning culture.The authors suggest fundamental principles for PE, including promoting equity in structures and processes, learner agency, and integration with workflow (harmonization). Finally, the authors explore the immediate need to develop consensus-driven standards: rules of engagement between people, products, and entities that interact in these systems to ensure interoperability, data sharing, replicability, and scale of PE innovations.

Scaffolding the Transition to Residency: A Qualitative Study of Coach and Resident Perspectives.

PURPOSE: This study explores coaching during transition from medical school to residency through the perspectives of residents and faculty coaches participating in a coaching program from residency match through the first year of residency. METHOD: From January to September 2020, 15 faculty coaches in internal medicine, obstetrics and gynecology, emergency medicine, orthopedics, and pathology participated in a synchronous, in-person coaching training course. All 94 postgraduate year 1 residents in these 5 training programs participated. Between November 2021 and March 2022, focus groups were held with interns from all residency programs participating in the program. Interviews were conducted with faculty coaches in February 2022. Faculty and residents discussed their experiences with and perceptions of coaching. De-identified transcripts were coded, and researchers organized these codes into broader categories, generated cross-cutting themes from the concepts described in both cohorts, and proposed a model for the potential of coaching to support the transition to residency. Descriptive themes were constructed and analytic themes developed by identifying concepts that crossed the data sets. RESULTS: Seven focus groups were held with 39 residents (42%). Residents discussed the goals of a coaching program, coach attributes, program factors, resident attributes, and the role of the coach. Coaches focused on productivity of coaching, coaching skills and approach, professional development, and scaffolding the coaching experience. Three analytic themes were created: (1) coaching as creating an explicit curriculum for growth through the transition to residency, (2) factors contributing to successful coaching, and (3) ways in which these factors confront graduate medical education norms. CONCLUSIONS: Learner and faculty perspectives on coaching through the transition to residency reveal the potential for coaching to make an explicit and modifiable curriculum for professional growth and development. Creating structures for coaching in graduate medical education may allow for individualized professional development, improved mindset, self-awareness, and self-directed learning.

A Theoretical Foundation to Inform the Implementation of Precision Education and Assessment.

ABSTRACT: Precision education (PE) uses personalized educational interventions to empower trainees and improve learning outcomes. While PE has the potential to represent a paradigm shift in medical education, a theoretical foundation to guide the effective implementation of PE strategies has not yet been described. Here, the authors introduce a theoretical foundation for the implementation of PE, integrating key learning theories with the digital tools that allow them to be operationalized. Specifically, the authors describe how the master adaptive learner (MAL) model, transformative learning theory, and self-determination theory can be harnessed in conjunction with nudge strategies and audit and feedback dashboards to drive learning and meaningful behavior change. The authors also provide practical examples of these theories and tools in action by describing precision interventions already in use at one academic medical center, concretizing PE's potential in the current clinical environment. These examples illustrate how a firm theoretical grounding allows educators to most effectively tailor PE interventions to fit individual learners' needs and goals, facilitating efficient learning and ultimately improving patient and health system outcomes.

Precision Medical Education.

Medical schools and residency programs are increasingly incorporating personalization of content, pathways, and assessments to align with a competency-based model. Yet, such efforts face challenges involving large amounts of data, sometimes struggling to deliver insights in a timely fashion for trainees, coaches, and programs. In this article, the authors argue that the emerging paradigm of precision medical education (PME) may ameliorate some of these challenges. However, PME lacks a widely accepted definition and a shared model of guiding principles and capacities, limiting widespread adoption. The authors propose defining PME as a systematic approach that integrates longitudinal data and analytics to drive precise educational interventions that address each individual learner's needs and goals in a continuous, timely, and cyclical fashion, ultimately improving meaningful educational, clinical, or system outcomes. Borrowing from precision medicine, they offer an adapted shared framework. In the P4 medical education framework, PME should (1) take a proactive approach to acquiring and using trainee data; (2) generate timely personalized insights through precision analytics (including artificial intelligence and decision-support tools); (3) design precision educational interventions (learning, assessment, coaching, pathways) in a participatory fashion, with trainees at the center as co-producers; and (4) ensure interventions are predictive of meaningful educational, professional, or clinical outcomes. Implementing PME will require new foundational capacities: flexible educational pathways and programs responsive to PME-guided dynamic and competency-based progression; comprehensive longitudinal data on trainees linked to educational and clinical outcomes; shared development of requisite technologies and analytics to effect educational decision-making; and a culture that embraces a precision approach, with research to gather validity evidence for this approach and development efforts targeting new skills needed by learners, coaches, and educational leaders. Anticipating pitfalls in the use of this approach will be important, as will ensuring it deepens, rather than replaces, the interaction of trainees and their coaches.

Artificial Intelligence Screening of Medical School Applications: Development and Validation of a Machine-Learning Algorithm.

PURPOSE: To explore whether a machine-learning algorithm could accurately perform the initial screening of medical school applications. METHOD: Using application data and faculty screening outcomes from the 2013 to 2017 application cycles (n = 14,555 applications), the authors created a virtual faculty screener algorithm. A retrospective validation using 2,910 applications from the 2013 to 2017 cycles and a prospective validation using 2,715 applications during the 2018 application cycle were performed. To test the validated algorithm, a randomized trial was performed in the 2019 cycle, with 1,827 eligible applications being reviewed by faculty and 1,873 by algorithm. RESULTS: The retrospective validation yielded area under the receiver operating characteristic (AUROC) values of 0.83, 0.64, and 0.83 and area under the precision-recall curve (AUPRC) values of 0.61, 0.54, and 0.65 for the invite for interview, hold for review, and reject groups, respectively. The prospective validation yielded AUROC values of 0.83, 0.62, and 0.82 and AUPRC values of 0.66, 0.47, and 0.65 for the invite for interview, hold for review, and reject groups, respectively. The randomized trial found no significant differences in overall interview recommendation rates according to faculty or algorithm and among female or underrepresented in medicine applicants. In underrepresented in medicine applicants, there were no significant differences in the rates at which the admissions committee offered an interview (70 of 71 in the faculty reviewer arm and 61 of 65 in the algorithm arm; P = .14). No difference in the rate of the committee agreeing with the recommended interview was found among female applicants (224 of 229 in the faculty reviewer arm and 220 of 227 in the algorithm arm; P = .55). CONCLUSIONS: The virtual faculty screener algorithm successfully replicated faculty screening of medical school applications and may aid in the consistent and reliable review of medical school applicants.

SMARTer Goalsetting: A Pilot Innovation for Coaches During the Transition to Residency.

PROBLEM: Ability to set goals and work with coaches can support individualized, self-directed learning. Understanding the focus and quality of graduating medical student and first-year resident goals and the influence of coaching on goal-setting can inform efforts to support learners through the transition from medical school to residency. APPROACH: This observational study examined goal-setting among graduating medical students and first-year residents from April 2021 to March 2022. The medical students set goals while participating in a Transition to Residency elective. The residents in internal medicine, obstetrics and gynecology, emergency medicine, orthopedics, and pathology set goals through meeting 1:1 with coaches. Raters assessed goals using a 3-point rubric on domains of specific, measurable, attainable, relevant, and timely (i.e., SMART goal framework) and analyzed descriptive statistics, Mann-Whitney U tests, and linear regressions. OUTCOMES: Among 48 medical students, 30 (62.5%) set 108 goals for early residency. Among 134 residents, 62 (46.3%) entered goals. Residents met with coaches 2.8 times on average (range 0-8 meetings, median = 3). Goal quality was higher in residents than medical students (average score for S: 2.71 vs 2.06, P < .001; M: 2.38 vs 1.66, P < .001; A: 2.92 vs 2.64, P < .001; R: 2.94 vs 2.86, P = .002; T: 1.71 vs 1.31, P < .001). The number of coaching meetings was associated with more specific, measurable goals (specific: F [1, 1.02] = 6.56, P = .01, R2 = .10; measurable: F [1, 1.49] = 4.74, P = .03, R2 = .07). NEXT STEPS: Learners set realistic, attainable goals through the transition to residency, but the goals could be more specific, measurable, and timely. The residents set SMARTer goals, with coaching improving goal quality. Understanding how best to scaffold coaching and support goal-setting through this transition may improve trainees' self-directed learning and well-being.

Bridging the Gap from Student to Doctor: Developing Coaches for the Transition to Residency.

BACKGROUND: A lack of educational continuity creates disorienting friction at the onset of residency. Few programs have harnessed the benefits of coaching, which can facilitate self-directed learning, competency development, and professional identity formation, to help ease this transition. OBJECTIVE: To describe the process of training faculty Bridge Coaches for the Transition to Residency Advantage (TRA) program for interns. METHODS: Nineteen graduate faculty educators participated in a coaching training course with formative skills assessment as part of a faculty development program starting in January 2020. Surveys (n = 15; 79%) and a focus group (n = 7; 37%) were conducted to explore the perceived impact of the training course on coaching skills, perceptions of coaching, and further program needs during the pilot year of the TRA program. RESULTS: Faculty had strong skills around establishing trust, authentic listening, and supporting goal-setting. They required more practice around guiding self-discovery and following a coachee-led agenda. Faculty found the training course to be helpful for developing coaching skills. Faculty embraced their new roles as coaches and appreciated having a community of practice with other coaches. Suggestions for improvement included more opportunities to practice and receive feedback on skills and additional structures to further support TRA program encounters with coaches. CONCLUSIONS: The faculty development program was feasible and had good acceptance among participants. Faculty were well-suited to serve as coaches and valued the coaching mindset. Adequate skills reinforcement and program structure were identified as needs to facilitate a coaching program in graduate medical education.

Exploiting the power of information in medical education.

The explosion of medical information demands a thorough reconsideration of medical education, including what we teach and assess, how we educate, and whom we educate. Physicians of the future will need to be self-aware, self-directed, resource-effective team players who can synthesize and apply summarized information and communicate clearly. Training in metacognition, data science, informatics, and artificial intelligence is needed. Education programs must shift focus from content delivery to providing students explicit scaffolding for future learning, such as the Master Adaptive Learner model. Additionally, educators should leverage informatics to improve the process of education and foster individualized, precision education. Finally, attributes of the successful physician of the future should inform adjustments in recruitment and admissions processes. This paper explores how member schools of the American Medical Association Accelerating Change in Medical Education Consortium adjusted all aspects of educational programming in acknowledgment of the rapid expansion of information.

Assessing the Transition of Training in Health Systems Science From Undergraduate to Graduate Medical Education.

BACKGROUND: The American Medical Association Accelerating Change in Medical Education (AMA-ACE) consortium proposes that medical schools include a new 3-pillar model incorporating health systems science (HSS) and basic and clinical sciences. One of the goals of AMA-ACE was to support HSS curricular innovation to improve residency preparation. OBJECTIVE: This study evaluates the effectiveness of HSS curricula by using a large dataset to link medical school graduates to internship Milestones through collaboration with the Accreditation Council for Graduate Medical Education (ACGME). METHODS: ACGME subcompetencies related to the schools' HSS curricula were identified for internal medicine, emergency medicine, family medicine, obstetrics and gynecology (OB/GYN), pediatrics, and surgery. Analysis compared Milestone ratings of ACE school graduates to non-ACE graduates at 6 and 12 months using generalized estimating equation models. RESULTS: At 6 months both groups demonstrated similar HSS-related levels of Milestone performance on the selected ACGME competencies. At 1 year, ACE graduates in OB/GYN scored minimally higher on 2 systems-based practice (SBP) subcompetencies compared to non-ACE school graduates: SBP01 (1.96 vs 1.82, 95% CI 0.03-0.24) and SBP02 (1.87 vs 1.79, 95% CI 0.01-0.16). In internal medicine, ACE graduates scored minimally higher on 3 HSS-related subcompetencies: SBP01 (2.19 vs 2.05, 95% CI 0.04-0.26), PBLI01 (2.13 vs 2.01; 95% CI 0.01-0.24), and PBLI04 (2.05 vs 1.93; 95% CI 0.03-0.21). For the other specialties examined, there were no significant differences between groups. CONCLUSIONS: Graduates from schools with training in HSS had similar Milestone ratings for most subcompetencies and very small differences in Milestone ratings for only 5 subcompetencies across 6 specialties at 1 year, compared to graduates from non-ACE schools. These differences are likely not educationally meaningful.

Signatures of medical student applicants and academic success.

The acceptance of students to a medical school places a considerable emphasis on performance in standardized tests and undergraduate grade point average (uGPA). Traditionally, applicants may be judged as a homogeneous population according to simple quantitative thresholds that implicitly assume a linear relationship between scores and academic success. This 'one-size-fits-all' approach ignores the notion that individuals may show distinct patterns of achievement and follow diverse paths to success. In this study, we examined a dataset composed of 53 variables extracted from the admissions application records of 1,088 students matriculating to NYU School of Medicine between the years 2006-2014. We defined training and test groups and applied K-means clustering to search for distinct groups of applicants. Building an optimized logistic regression model, we then tested the predictive value of this clustering for estimating the success of applicants in medical school, aggregating eight performance measures during the subsequent medical school training as a success factor. We found evidence for four distinct clusters of students-we termed 'signatures'-which differ most substantially according to the absolute level of the applicant's uGPA and its trajectory over the course of undergraduate education. The 'risers' signature showed a relatively higher uGPA and also steeper trajectory; the other signatures showed each remaining combination of these two main factors: 'improvers' relatively lower uGPA, steeper trajectory; 'solids' higher uGPA, flatter trajectory; 'statics' both lower uGPA and flatter trajectory. Examining the success index across signatures, we found that the risers and the statics have significantly higher and lower likelihood of quantifiable success in medical school, respectively. We also found that each signature has a unique set of features that correlate with its success in medical school. The big data approach presented here can more sensitively uncover success potential since it takes into account the inherent heterogeneity within the student population.

How well is each learner learning? Validity investigation of a learning curve-based assessment approach for ECG interpretation.

Learning curves can support a competency-based approach to assessment for learning. When interpreting repeated assessment data displayed as learning curves, a key assessment question is: "How well is each learner learning?" We outline the validity argument and investigation relevant to this question, for a computer-based repeated assessment of competence in electrocardiogram (ECG) interpretation. We developed an on-line ECG learning program based on 292 anonymized ECGs collected from an electronic patient database. After diagnosing each ECG, participants received feedback including the computer interpretation, cardiologist's annotation, and correct diagnosis. In 2015, participants from a single institution, across a range of ECG skill levels, diagnosed at least 60 ECGs. We planned, collected and evaluated validity evidence under each inference of Kane's validity framework. For Scoring, three cardiologists' kappa for agreement on correct diagnosis was 0.92. There was a range of ECG difficulty across and within each diagnostic category. For Generalization, appropriate sampling was reflected in the inclusion of a typical clinical base rate of 39% normal ECGs. Applying generalizability theory presented unique challenges. Under the Extrapolation inference, group learning curves demonstrated expert-novice differences, performance increased with practice and the incremental phase of the learning curve reflected ongoing, effortful learning. A minority of learners had atypical learning curves. We did not collect Implications evidence. Our results support a preliminary validity argument for a learning curve assessment approach for repeated ECG interpretation with deliberate and mixed practice. This approach holds promise for providing educators and researchers, in collaboration with their learners, with deeper insights into how well each learner is learning.

Extended Reality in Medical Education: Driving Adoption through Provider-Centered Design.

Simulation is a widely used technique for medical education. Due to decreased training opportunities with real patients, and increased emphasis on both patient outcomes and remote access, demand has increased for more advanced, realistic simulation methods. Here, we discuss the increasing need for, and benefits of, extended (virtual, augmented, or mixed) reality throughout the continuum of medical education, from anatomy for medical students to procedures for residents. We discuss how to drive the adoption of mixed reality tools into medical school's anatomy, and procedural, curricula.

The Time Is Now: Using Graduates' Practice Data to Drive Medical Education Reform.

Medical educators are not yet taking full advantage of the publicly available clinical practice data published by federal, state, and local governments, which can be attributed to individual physicians and evaluated in the context of where they attended medical school and residency training. Understanding how graduates fare in actual practice, both in terms of the quality of the care they provide and the clinical challenges they face, can aid educators in taking an evidence-based approach to medical education. Although in their infancy, efforts to link clinical outcomes data to educational process data hold the potential to accelerate medical education research and innovation. This approach will enable unprecedented insight into the long-term impact of each stage of medical education on graduates' future practice. More work is needed to determine best practices, but the barrier to using these public data is low, and the potential for early results is immediate. Using practice data to evaluate medical education programs can transform how the future physician workforce is trained and better align continuously learning medical education and health care systems.

The Role for Virtual Patients in the Future of Medical Education.

The medical education community is working-across disciplines and across the continuum-to address the current challenges facing the medical education system and to implement strategies to improve educational outcomes. Educational technology offers the promise of addressing these important challenges in ways not previously possible. The authors propose a role for virtual patients (VPs), which they define as multimedia, screen-based interactive patient scenarios. They believe VPs offer capabilities and benefits particularly well suited to addressing the challenges facing medical education. Well-designed, interactive VP-based learning activities can promote the deep learning that is needed to handle the rapid growth in medical knowledge. Clinically oriented learning from VPs can capture intrinsic motivation and promote mastery learning. VPs can also enhance trainees' application of foundational knowledge to promote the development of clinical reasoning, the foundation of medical practice. Although not the entire solution, VPs can support competency-based education. The data created by the use of VPs can serve as the basis for multi-institutional research that will enable the medical education community both to better understand the effectiveness of educational interventions and to measure progress toward an improved system of medical education.

E-Learning with virtual teammates: A novel approach to interprofessional education.

The Institute of Medicine identified interprofessional education (IPE) as a key innovation for achieving the triple aim of better care, better outcomes, and reduced healthcare costs. Yet, a shortage of qualified faculty and difficulty with aligning learners' schedules often prevent sustainable and scalable IPE. A virtual IPE intervention was developed to circumvent these barriers and compared to a blended-learning IPE intervention. We used a pre-test and post-test design with two comparison interventions to test the effects of these IPE interventions on changes in teamwork knowledge, skills, and attitudes. The interventions were delivered to pre-licensure learners at a large, metropolitan medical and a nursing school. We used one-sample and independent-sample t-tests to analyze data from 220 learners who received the blended-learning intervention in 2011 and 540 learners who received the virtual learning intervention in 2012. The students in the blended-learning intervention did not significantly (p < 0.05) outperform the students in the virtual learning intervention for any of the measured outcomes, except for medical students' attitudes around team value. Virtual IPE learning is an effective, scalable, and sustainable solution for imparting foundational teamwork knowledge in health profession students.