Exploiting the power of information in medical education.

The explosion of medical information demands a thorough reconsideration of medical education, including what we teach and assess, how we educate, and whom we educate. Physicians of the future will need to be self-aware, self-directed, resource-effective team players who can synthesize and apply summarized information and communicate clearly. Training in metacognition, data science, informatics, and artificial intelligence is needed. Education programs must shift focus from content delivery to providing students explicit scaffolding for future learning, such as the Master Adaptive Learner model. Additionally, educators should leverage informatics to improve the process of education and foster individualized, precision education. Finally, attributes of the successful physician of the future should inform adjustments in recruitment and admissions processes. This paper explores how member schools of the American Medical Association Accelerating Change in Medical Education Consortium adjusted all aspects of educational programming in acknowledgment of the rapid expansion of information.

A Theoretical Foundation to Inform the Implementation of Precision Education and Assessment.

ABSTRACT: Precision education (PE) uses personalized educational interventions to empower trainees and improve learning outcomes. While PE has the potential to represent a paradigm shift in medical education, a theoretical foundation to guide the effective implementation of PE strategies has not yet been described. Here, the authors introduce a theoretical foundation for the implementation of PE, integrating key learning theories with the digital tools that allow them to be operationalized. Specifically, the authors describe how the master adaptive learner (MAL) model, transformative learning theory, and self-determination theory can be harnessed in conjunction with nudge strategies and audit and feedback dashboards to drive learning and meaningful behavior change. The authors also provide practical examples of these theories and tools in action by describing precision interventions already in use at one academic medical center, concretizing PE's potential in the current clinical environment. These examples illustrate how a firm theoretical grounding allows educators to most effectively tailor PE interventions to fit individual learners' needs and goals, facilitating efficient learning and ultimately improving patient and health system outcomes.

Precision Education: The Future of Lifelong Learning in Medicine.

ABSTRACT: The goal of medical education is to produce a physician workforce capable of delivering high-quality equitable care to diverse patient populations and communities. To achieve this aim amidst explosive growth in medical knowledge and increasingly complex medical care, a system of personalized and continuous learning, assessment, and feedback for trainees and practicing physicians is urgently needed. In this perspective, the authors build on prior work to advance a conceptual framework for such a system: precision education (PE).PE is a system that uses data and technology to transform lifelong learning by improving personalization, efficiency, and agency at the individual, program, and organization levels. PE "cycles" start with data inputs proactively gathered from new and existing sources, including assessments, educational activities, electronic medical records, patient care outcomes, and clinical practice patterns. Through technology-enabled analytics , insights are generated to drive precision interventions . At the individual level, such interventions include personalized just-in-time educational programming. Coaching is essential to provide feedback and increase learner participation and personalization. Outcomes are measured using assessment and evaluation of interventions at the individual, program, and organizational levels, with ongoing adjustment for repeated cycles of improvement. PE is rooted in patient, health system, and population data; promotes value-based care and health equity; and generates an adaptive learning culture.The authors suggest fundamental principles for PE, including promoting equity in structures and processes, learner agency, and integration with workflow (harmonization). Finally, the authors explore the immediate need to develop consensus-driven standards: rules of engagement between people, products, and entities that interact in these systems to ensure interoperability, data sharing, replicability, and scale of PE innovations.

Precision Medical Education.

Medical schools and residency programs are increasingly incorporating personalization of content, pathways, and assessments to align with a competency-based model. Yet, such efforts face challenges involving large amounts of data, sometimes struggling to deliver insights in a timely fashion for trainees, coaches, and programs. In this article, the authors argue that the emerging paradigm of precision medical education (PME) may ameliorate some of these challenges. However, PME lacks a widely accepted definition and a shared model of guiding principles and capacities, limiting widespread adoption. The authors propose defining PME as a systematic approach that integrates longitudinal data and analytics to drive precise educational interventions that address each individual learner's needs and goals in a continuous, timely, and cyclical fashion, ultimately improving meaningful educational, clinical, or system outcomes. Borrowing from precision medicine, they offer an adapted shared framework. In the P4 medical education framework, PME should (1) take a proactive approach to acquiring and using trainee data; (2) generate timely personalized insights through precision analytics (including artificial intelligence and decision-support tools); (3) design precision educational interventions (learning, assessment, coaching, pathways) in a participatory fashion, with trainees at the center as co-producers; and (4) ensure interventions are predictive of meaningful educational, professional, or clinical outcomes. Implementing PME will require new foundational capacities: flexible educational pathways and programs responsive to PME-guided dynamic and competency-based progression; comprehensive longitudinal data on trainees linked to educational and clinical outcomes; shared development of requisite technologies and analytics to effect educational decision-making; and a culture that embraces a precision approach, with research to gather validity evidence for this approach and development efforts targeting new skills needed by learners, coaches, and educational leaders. Anticipating pitfalls in the use of this approach will be important, as will ensuring it deepens, rather than replaces, the interaction of trainees and their coaches.

Artificial Intelligence Screening of Medical School Applications: Development and Validation of a Machine-Learning Algorithm.

PURPOSE: To explore whether a machine-learning algorithm could accurately perform the initial screening of medical school applications. METHOD: Using application data and faculty screening outcomes from the 2013 to 2017 application cycles (n = 14,555 applications), the authors created a virtual faculty screener algorithm. A retrospective validation using 2,910 applications from the 2013 to 2017 cycles and a prospective validation using 2,715 applications during the 2018 application cycle were performed. To test the validated algorithm, a randomized trial was performed in the 2019 cycle, with 1,827 eligible applications being reviewed by faculty and 1,873 by algorithm. RESULTS: The retrospective validation yielded area under the receiver operating characteristic (AUROC) values of 0.83, 0.64, and 0.83 and area under the precision-recall curve (AUPRC) values of 0.61, 0.54, and 0.65 for the invite for interview, hold for review, and reject groups, respectively. The prospective validation yielded AUROC values of 0.83, 0.62, and 0.82 and AUPRC values of 0.66, 0.47, and 0.65 for the invite for interview, hold for review, and reject groups, respectively. The randomized trial found no significant differences in overall interview recommendation rates according to faculty or algorithm and among female or underrepresented in medicine applicants. In underrepresented in medicine applicants, there were no significant differences in the rates at which the admissions committee offered an interview (70 of 71 in the faculty reviewer arm and 61 of 65 in the algorithm arm; P = .14). No difference in the rate of the committee agreeing with the recommended interview was found among female applicants (224 of 229 in the faculty reviewer arm and 220 of 227 in the algorithm arm; P = .55). CONCLUSIONS: The virtual faculty screener algorithm successfully replicated faculty screening of medical school applications and may aid in the consistent and reliable review of medical school applicants.

New directions in e-learning research in health professions education: Report of two symposia.

BACKGROUND: The use of Computer Assisted Instruction (CAI) is rising across health professions education. Research to date is of limited use in guiding the implementation and selection of CAI innovations. AIMS: In the context of two symposia, systemic reviews were discussed that evaluate literature in Internet-based learning, Virtual Patients, and animations. Each session included a debate with the goal of reaching consensus on best current practices and future research. METHODS: Thematic analysis of the discussions was performed to arrange the questions by theme, eliminate redundancy, and craft them into a cohesive narrative. RESULTS: The question analysis revealed that there are clear advantages to the use of CAI, and that established educational theories should certainly inform the future development and selection of CAI tools. Schools adopting CAI need to carefully consider the benefits, cost, available resources, and capacity for teachers and learners to accept change in their practice of education. Potential areas for future research should focus on the effectiveness of CAI instructional features, integration of e-learning into existing curricula and with other modalities like simulation, and the use of CAI in assessment of higher-level outcomes. CONCLUSIONS: There are numerous opportunities for future research and it will be important to achieve consensus on important themes.

Enhanced virtual microscopy for collaborative education.

BACKGROUND: Curricular reform efforts and a desire to use novel educational strategies that foster student collaboration are challenging the traditional microscope-based teaching of histology. Computer-based histology teaching tools and Virtual Microscopes (VM), computer-based digital slide viewers, have been shown to be effective and efficient educational strategies. We developed an open-source VM system based on the Google Maps engine to transform our histology education and introduce new teaching methods. This VM allows students and faculty to collaboratively create content, annotate slides with markers, and it is enhanced with social networking features to give the community of learners more control over the system. RESULTS: We currently have 1,037 slides in our VM system comprised of 39,386,941 individual JPEG files that take up 349 gigabytes of server storage space. Of those slides 682 are for general teaching and available to our students and the public; the remaining 355 slides are used for practical exams and have restricted access. The system has seen extensive use with 289,352 unique slide views to date. Students viewed an average of 56.3 slides per month during the histology course and accessed the system at all hours of the day. Of the 621 annotations added to 126 slides 26.2% were added by faculty and 73.8% by students. The use of the VM system reduced the amount of time faculty spent administering the course by 210 hours, but did not reduce the number of laboratory sessions or the number of required faculty. Laboratory sessions were reduced from three hours to two hours each due to the efficiencies in the workflow of the VM system. CONCLUSIONS: Our virtual microscope system has been an effective solution to the challenges facing traditional histopathology laboratories and the novel needs of our revised curriculum. The web-based system allowed us to empower learners to have greater control over their content, as well as the ability to work together in collaborative groups. The VM system saved faculty time and there was no significant difference in student performance on an identical practical exam before and after its adoption. We have made the source code of our VM freely available and encourage use of the publically available slides on our website.

Signatures of medical student applicants and academic success.

The acceptance of students to a medical school places a considerable emphasis on performance in standardized tests and undergraduate grade point average (uGPA). Traditionally, applicants may be judged as a homogeneous population according to simple quantitative thresholds that implicitly assume a linear relationship between scores and academic success. This 'one-size-fits-all' approach ignores the notion that individuals may show distinct patterns of achievement and follow diverse paths to success. In this study, we examined a dataset composed of 53 variables extracted from the admissions application records of 1,088 students matriculating to NYU School of Medicine between the years 2006-2014. We defined training and test groups and applied K-means clustering to search for distinct groups of applicants. Building an optimized logistic regression model, we then tested the predictive value of this clustering for estimating the success of applicants in medical school, aggregating eight performance measures during the subsequent medical school training as a success factor. We found evidence for four distinct clusters of students-we termed 'signatures'-which differ most substantially according to the absolute level of the applicant's uGPA and its trajectory over the course of undergraduate education. The 'risers' signature showed a relatively higher uGPA and also steeper trajectory; the other signatures showed each remaining combination of these two main factors: 'improvers' relatively lower uGPA, steeper trajectory; 'solids' higher uGPA, flatter trajectory; 'statics' both lower uGPA and flatter trajectory. Examining the success index across signatures, we found that the risers and the statics have significantly higher and lower likelihood of quantifiable success in medical school, respectively. We also found that each signature has a unique set of features that correlate with its success in medical school. The big data approach presented here can more sensitively uncover success potential since it takes into account the inherent heterogeneity within the student population.

Virtual patients: a critical literature review and proposed next steps.

CONTEXT: The opposing forces of increased training expectations and reduced training resources have greatly impacted health professions education. Virtual patients (VPs), which take the form of interactive computer-based clinical scenarios, may help to reconcile this paradox. METHODS: We summarise research on VPs, highlight the spectrum of potential variation and identify an agenda for future research. We also critically consider the role of VPs in the educational armamentarium. RESULTS: We propose that VPs' most unique and cost-effective function is to facilitate and assess the development of clinical reasoning. Clinical reasoning in experts involves a non-analytical process that matures through deliberate practice with multiple and varied clinical cases. Virtual patients are ideally suited to this task. Virtual patients can also be used in learner assessment, but scoring rubrics should emphasise non-analytical clinical reasoning rather than completeness of information or algorithmic approaches. Potential variations in VP design are practically limitless, yet few studies have rigorously explored design issues. More research is needed to inform instructional design and curricular integration. CONCLUSIONS: Virtual patients should be designed and used to promote clinical reasoning skills. More research is needed to inform how to effectively use VPs.

Extended Reality in Medical Education: Driving Adoption through Provider-Centered Design.

Simulation is a widely used technique for medical education. Due to decreased training opportunities with real patients, and increased emphasis on both patient outcomes and remote access, demand has increased for more advanced, realistic simulation methods. Here, we discuss the increasing need for, and benefits of, extended (virtual, augmented, or mixed) reality throughout the continuum of medical education, from anatomy for medical students to procedures for residents. We discuss how to drive the adoption of mixed reality tools into medical school's anatomy, and procedural, curricula.

The Time Is Now: Using Graduates' Practice Data to Drive Medical Education Reform.

Medical educators are not yet taking full advantage of the publicly available clinical practice data published by federal, state, and local governments, which can be attributed to individual physicians and evaluated in the context of where they attended medical school and residency training. Understanding how graduates fare in actual practice, both in terms of the quality of the care they provide and the clinical challenges they face, can aid educators in taking an evidence-based approach to medical education. Although in their infancy, efforts to link clinical outcomes data to educational process data hold the potential to accelerate medical education research and innovation. This approach will enable unprecedented insight into the long-term impact of each stage of medical education on graduates' future practice. More work is needed to determine best practices, but the barrier to using these public data is low, and the potential for early results is immediate. Using practice data to evaluate medical education programs can transform how the future physician workforce is trained and better align continuously learning medical education and health care systems.

Brief report: Failure of an electronic medical record tool to improve pain assessment documentation.

OBJECTIVE: To comply with pain management standards, Bellevue Hospital in New York City implemented a mandatory computerized pain assessment screen (PAS) in its electronic medical record (EMR) system for every outpatient encounter. We assessed provider acceptance of the instrument and examined whether the intervention led to increased documentation of pain-related diagnoses or inquiries. DESIGN: Cross-sectional survey; a pre- and posthistorically controlled observational study. SUBJECTS AND MEASUREMENTS: The utility of the computerized tool to medicine housestaff and attendings was assessed by an anonymous survey. We conducted an electronic chart review comparing all adult primary care patient encounters over a 2-day period 6 months prior to implementation of the PAS and on 2 days 6 months after its implementation. RESULTS: Forty-seven percent of survey respondents felt that the computerized assessment tool was "somewhat difficult" or "very difficult" to use. The majority of respondents (79%) felt the tool did not change their pain assessment practice. Of 265 preintervention patients and 364 postintervention patients seen in the clinic, 42% and 37% had pain-related diagnoses, respectively (P=.29). Pain inquiry by the physician was noted for 49% of preintervention patients and 44% of the postintervention patients (P=.26). In 55% of postintervention encounters, there was discordance between the pain documentation using the PAS tool and the free text section of the medical note. CONCLUSION: A mandatory computerized pain assessment tool did not lead to an increase in pain-related diagnoses and may have hindered the documentation of pain assessment because of the perceived burden of using the application.

Proposed educational objectives for hospital-based dentists during catastrophic events and disaster response.

The purpose of this project was to define education and training requirements for hospital-based dentists to efficiently and meaningfully participate in a hospital disaster response. Eight dental faculty with hospital-based training and/or military command and CBRNE (chemical, biological, radiological, nuclear, and explosive) expertise were recruited as an expert panel. A consensus set of recommended educational objectives for hospital-based dentists was established using the following process: 1) identify assumptions supported by all expert panelists, 2) determine current advanced dental educational training requirements, and 3) conduct additional training and literature review by various panelists and discussions with other content and systems experts. Using this three-step process, educational objectives that the development group believed necessary for hospital-based dentists to be effective in treatment or management roles in times of a catastrophic event were established. These educational objectives are categorized into five thematic areas: 1) disaster systems, 2) triage/medical assessment, 3) blast and burn injuries, 4) chemical agents, and 5) biological agents. Creation of training programs to help dentists acquire these educational objectives would benefit hospital-based dental training programs and strengthen hospital surge manpower needs. The proposed educational objectives are designed to stimulate discussion and debate among dental, medical, and public health professionals about the roles of dentists in meeting hospital surge manpower needs.

The Medical Reserve Corps. an opportunity for dentists to serve.

The response to the events of Sept. 11, 2001, relied on local resources and personnel. Aware of how important their contribution could be, many people are now inspired to volunteer during times of crisis. The Medical Reserve Corps is a community-based volunteer network of health professionals that trains to respond to large-scale emergencies.

The Role for Virtual Patients in the Future of Medical Education.

The medical education community is working-across disciplines and across the continuum-to address the current challenges facing the medical education system and to implement strategies to improve educational outcomes. Educational technology offers the promise of addressing these important challenges in ways not previously possible. The authors propose a role for virtual patients (VPs), which they define as multimedia, screen-based interactive patient scenarios. They believe VPs offer capabilities and benefits particularly well suited to addressing the challenges facing medical education. Well-designed, interactive VP-based learning activities can promote the deep learning that is needed to handle the rapid growth in medical knowledge. Clinically oriented learning from VPs can capture intrinsic motivation and promote mastery learning. VPs can also enhance trainees' application of foundational knowledge to promote the development of clinical reasoning, the foundation of medical practice. Although not the entire solution, VPs can support competency-based education. The data created by the use of VPs can serve as the basis for multi-institutional research that will enable the medical education community both to better understand the effectiveness of educational interventions and to measure progress toward an improved system of medical education.

Technology induced error and usability: the relationship between usability problems and prescription errors when using a handheld application.

This paper describes an innovative approach to the evaluation of a handheld prescription writing application. Participants (10 physicians) were asked to perform a series of tasks involving entering prescriptions into the application from a medication list. The study procedure involved the collection of data consisting of transcripts of the subjects who were asked to "think aloud" while interacting with the prescription writing program to enter medications. All user interactions with the device were video and audio recorded. Analysis of the protocols was conducted in two phases: (1) usability problems were identified from coding of the transcripts and video data, (2) actual errors in entering prescription data were also identified. The results indicated that there were a variety of usability problems, with most related to interface design issues. In examining the relationship between usability problems and errors, it was found that certain types of usability problems were closely associated with the occurrence of specific types of errors in prescription of medications. Implications for identifying and predicting technology-induced error are discussed in the context of improving the safety of health care information systems.

Computer simulation of pathogen transmission in the medical intensive care unit: a comparison of two probabilistic methods.

The quantitative evaluation of pathogen transmission in the medical intensive care unit (MICU) is difficult given the small number of patients and the complexity and severity of illness. We sought to evaluate the suitability of a probabilistic computer model of our MICU, with which we could rapidly simulate infection control measures and other clinical interventions that would be impossible to perform in the real clinical setting

Enhancing medical and public health capabilities during times of crisis.

Terrorist attacks and other catastrophic events will create demands that severely challenge the capacity of the medical/public health system. To meet the surge, a cadre of professionals should be trained to operate around the nucleus of medical/public health officials. At New York University, an inter-institutional team is considering specific roles for and an approach to training dentists to enable these health care professionals to supplement medical/public surge needs based upon informatics systems that provide critical information.