Powerful medical education improves health care quality and return on investment.

INTRODUCTION: Powerful medical education (PME) involves the use of new technologies informed by the science of expertise that are embedded in laboratories and organizations that value evidence-based education and support innovation. This contrasts with traditional medical education that relies on a dated apprenticeship model that yields uneven results. PME involves an amalgam of features, conditions and assumptions, and contextual variables that comprise an approach to developing clinical competence grounded in education impact metrics including efficiency and cost-effectiveness. METHODS: This article is a narrative review based on SANRA criteria and informed by realist review principles. The review addresses the PME model with an emphasis on mastery learning and deliberate practice principles drawn from the new science of expertise. Pub Med, Scopus, and Web of Science search terms include medical education, the science of expertise, mastery learning, translational outcomes, cost effectiveness, and return on investment. Literature coverage is comprehensive with selective citations. RESULTS: PME is described as an integrated set of twelve features embedded in a group of seven conditions and assumptions and four context variables. PME is illustrated via case examples that demonstrate improved ventilator patient management learning outcomes compared to traditional clinical education and mastery learning of breaking bad news communication skills. Evidence also shows that PME of physicians and other health care providers can have translational, downstream effects on patient care practices, patient outcomes, and return on investment. Several translational health care quality improvements that derive from PME include reduced infections; better communication among physicians, patients, and families; exceptional birth outcomes; more effective patient education; and return on investment. CONCLUSIONS: The article concludes with challenges to hospitals, health systems, and medical education organizations that are responsible for producing physicians who are expected to deliver safe, effective, and cost-conscious health care.

Twelve tips for developing simulation-based mastery learning clinical skills checklists.

Simulation-based mastery learning is a powerful educational paradigm that leads to high levels of performance through a combination of strict standards, deliberate practice, formative feedback, and rigorous assessment. Successful mastery learning curricula often require well-designed checklists that produce reliable data that contribute to valid decisions. The following twelve tips are intended to help educators create defensible and effective clinical skills checklists for use in mastery learning curricula. These tips focus on defining the scope of a checklist using established principles of curriculum development, crafting the checklist based on a literature review and expert input, revising and testing the checklist, and recruiting judges to set a minimum passing standard. While this article has a particular focus on mastery learning, with the exception of the tips related to standard setting, the general principles discussed apply to the development of any clinical skills checklist.

Tomographic Plane Visualization (ToPlaV): a Tool to Enhance Echocardiographic Training.

Visual learning is an important part of echocardiographic training. Our aim is to describe and evaluate a visual teaching tool, tomographic plane visualization (ToPlaV) as an adjunct to skills training in pediatric echocardiography image acquisition. This tool incorporates learning theory by applying psychomotor skills that closely emulate the skills used in echocardiography. ToPlaV was used as part of a transthoracic bootcamp for first year cardiology fellows. A qualitative survey was given to trainees to evaluate their perceptions of its usefulness. There was universal agreement among fellow trainees that ToPlaV is a useful training tool. ToPlaV is a simple, low cost, education tool which can complement a simulator and live models. We propose that ToPlaV should be incorporated into early training in echocardiography skills for pediatric cardiology fellows.

Simulation Based Mastery Learning of Transesophageal Echocardiography.

Transesophageal echocardiography (TEE) education is part of pediatric cardiology fellow training. Simulation-based mastery learning (SBML) is an efficient and valuable education experience. The aim of this project was to equip trainees with the basic knowledge and skill required to perform a pediatric TEE. The secondary aim was to assess the utility of using SBML for pediatric TEE training. The target group is trainees from pediatric cardiology and cardiac anesthesia who participated in a TEE bootcamp. A baseline knowledge pretest was obtained. The knowledge session consisted of preparation via reading material, viewing recorded lectures and completing an iterative multiple-choice examination, which was repeated until a minimum passing score of 90% was achieved. The skills session involved a review of TEE probe manipulation and image acquisition, followed by rapid cycle deliberate practice using simulation to acquire TEE skills at 3 levels, advancing in complexity from level 1 to level 3. Eight individuals (7 pediatric cardiology fellows at varying training levels and one anesthesia attending) participated in the TEE bootcamp. All reached a minimum knowledge post test score of at least 90% before the skills session. All subjects reached mastery in TEE probe manipulation. All reached mastery in image acquisition for the skill level that they attempted (level 1-8/8, level 2-8/8, level 3-4/4, with 4 participants not attempting level 3). A TEE bootcamp using SBML is a powerful medical education strategy. SBML is a rigorous approach that can be used to achieve high and uniform TEE learning outcomes among trainees of different training levels and backgrounds.

Improving cardiology fellow education of right heart catheterization using a simulation based curriculum.

BACKGROUND: Medical procedures are traditionally taught informally at patients' bedside through observation and practice using the adage "see one, do one, teach one." This lack of formalized training can cause trainees to be unprepared to perform procedures independently. Simulation based education (SBE) increases competence, reduces complications, and decreases costs. We developed, implemented, and evaluated the efficacy of a right heart catheterization (RHC) SBE curriculum. METHODS: The RHC curriculum consisted of a pretest, video didactics, deliberate practice, and a posttest. Pre-and posttest skills examinations consisted of a dichotomous 43-item checklist on RHC skills and a 14-item hemodynamic waveform quiz. We enrolled two groups of fellows: 6 first-year, novice cardiology fellows at Northwestern University in their first month of training, and 11 second- and third-year fellows who had completed traditional required, level I training in RHC. We trained the first-year fellows at the beginning of the 2018-2019 year using the SBE curriculum and compared them to the traditionally-trained cardiology fellows who did not complete SBE. RESULTS: The SBE-trained fellows significantly improved RHC skills, hemodynamic knowledge, and confidence from pre- to posttesting. SBE-trained fellows performed similarly to traditionally-trained fellows on simulated RHC skills checklists (88.4% correct vs. 89.2%, p = .84), hemodynamic quizzes (94.0% correct vs. 86.4%, p = .12), and confidence (79.4 vs. 85.9 out of 100, p = .15) despite less clinical experience. CONCLUSIONS: A SBE curriculum for RHC allowed novice cardiology fellows to achieve level I skills and knowledge at the beginning of fellowship and can train cardiology fellows before patient contact.

Simulation-based mastery learning compared to standard education for discussing diagnostic uncertainty with patients in the emergency department: a randomized controlled trial.

BACKGROUND: Diagnostic uncertainty occurs frequently in emergency medical care, with more than one-third of patients leaving the emergency department (ED) without a clear diagnosis. Despite this frequency, ED providers are not adequately trained on how to discuss diagnostic uncertainty with these patients, who often leave the ED confused and concerned. To address this training need, we developed the Uncertainty Communication Education Module (UCEM) to teach physicians how to discuss diagnostic uncertainty. The purpose of the study is to evaluate the effectiveness of the UCEM in improving physician communications. METHODS: The trial is a multicenter, two-arm randomized controlled trial designed to teach communication skills using simulation-based mastery learning (SBML). Resident emergency physicians from two training programs will be randomly assigned to immediate or delayed receipt of the two-part UCEM intervention after completing a baseline standardized patient encounter. The two UCEM components are: 1) a web-based interactive module, and 2) a smart-phone-based game. Both formats teach and reinforce communication skills for patient cases involving diagnostic uncertainty. Following baseline testing, participants in the immediate intervention arm will complete a remote deliberate practice session via a video platform and subsequently return for a second study visit to assess if they have achieved mastery. Participants in the delayed intervention arm will receive access to UCEM and remote deliberate practice after the second study visit. The primary outcome of interest is the proportion of residents in the immediate intervention arm who achieve mastery at the second study visit. DISCUSSION: Patients' understanding of the care they received has implications for care quality, safety, and patient satisfaction, especially when they are discharged without a definitive diagnosis. Developing a patient-centered diagnostic uncertainty communication strategy will improve safety of acute care discharges. Although use of SBML is a resource intensive educational approach, this trial has been deliberately designed to have a low-resource, scalable intervention that would allow for widespread dissemination and uptake. TRIAL REGISTRATION: The trial was registered at clinicaltrials.gov (NCT04021771). Registration date: July 16, 2019.

A Mastery Learning Capstone Course to Teach and Assess Components of Three Entrustable Professional Activities to Graduating Medical Students.

PROBLEM: Thirteen measurable Entrustable Professional Activities (EPAs) have been proposed by the Association of American Medical Colleges as a means to operationalize medical school graduates' patient care qualifications. Mastery learning is an effective method for boosting clinical skills, but its applicability to the EPAs remains to be studied. The authors designed this study to evaluate a mastery learning intervention to teach and assess components of 3 of the 13 EPAs in a 4th-year capstone course. INTERVENTION: The course featured mastery learning principles and addressed three EPA-based skills: (a) obtain informed consent, (b) develop a differential diagnosis and write admission orders, and (c) write discharge prescriptions. All students underwent a baseline skills assessment, received feedback, engaged in deliberate practice with actionable feedback, and completed a similar skills-based posttest assessment. Students continued with practice and testing until the minimum passing standards (MPSs) were reached for each posttest. CONTEXT: All medical students at a single medical school (Northwestern University, Feinberg School of Medicine) who matriculated in 2012 and graduated with the class of 2016 participated in a required transition to residency course immediately prior to graduation. OUTCOME: There were 134 students eligible to participate, and 130 (97.0%) completed all curricular requirements and assessments. All 130 medical students who completed the course met or exceeded the MPS for each of the three EPA-based clinical skills. Reliability coefficients for outcome data were uniformly high. Measures for each of the three clinical skills showed statistically significant improvement. LESSONS LEARNED: The capstone course was an effective approach to teach and assess components of three EPA-based clinical skills to mastery learning standards in a 4th-year capstone course. We learned that this approach for implementation is feasible and results in significant improvement in components of EPA skill performance. Next steps will include developing assessments incorporating the mastery model into components of additional EPAs, identifying the best location within the curriculum to insert this content, and expanding the number of assessments as part of a larger assessment system.

Rescuing the Clinical Breast Examination: Advances in Classifying Technique and Assessing Physician Competency.

OBJECTIVE: Develop new performance evaluation standards for the clinical breast examination (CBE). SUMMARY BACKGROUND DATA: There are several, technical aspects of a proper CBE. Our recent work discovered a significant, linear relationship between palpation force and CBE accuracy. This article investigates the relationship between other technical aspects of the CBE and accuracy. METHODS: This performance assessment study involved data collection from physicians (n = 553) attending 3 different clinical meetings between 2013 and 2014: American Society of Breast Surgeons, American Academy of Family Physicians, and American College of Obstetricians and Gynecologists. Four, previously validated, sensor-enabled breast models were used for clinical skills assessment. Models A and B had solitary, superficial, 2 cm and 1 cm soft masses, respectively. Models C and D had solitary, deep, 2 cm hard and moderately firm masses, respectively. Finger movements (search technique) from 1137 CBE video recordings were independently classified by 2 observers. Final classifications were compared with CBE accuracy. RESULTS: Accuracy rates were model A = 99.6%, model B = 89.7%, model C = 75%, and model D = 60%. Final classification categories for search technique included rubbing movement, vertical movement, piano fingers, and other. Interrater reliability was (k = 0.79). Rubbing movement was 4 times more likely to yield an accurate assessment (odds ratio 3.81, P < 0.001) compared with vertical movement and piano fingers. Piano fingers had the highest failure rate (36.5%). Regression analysis of search pattern, search technique, palpation force, examination time, and 6 demographic variables, revealed that search technique independently and significantly affected CBE accuracy (P < 0.001). CONCLUSIONS: Our results support measurement and classification of CBE techniques and provide the foundation for a new paradigm in teaching and assessing hands-on clinical skills. The newly described piano fingers palpation technique was noted to have unusually high failure rates. Medical educators should be aware of the potential differences in effectiveness for various CBE techniques.

Attending Physician Adherence to a 29-Component Central Venous Catheter Bundle Checklist During Simulated Procedures.

OBJECTIVES: Central venous catheter insertions may lead to preventable adverse events. Attending physicians' central venous catheter insertion skills are not assessed routinely. We aimed to compare attending physicians' simulated central venous catheterinsertion performance to published competency standards. DESIGN: Prospective cohort study of attending physicians' simulated internal jugular and subclavian central venous catheter insertion skills versus a historical comparison group of residents who participated in simulation training. SETTING: Fifty-eight Veterans Affairs Medical Centers from February 2014 to December 2014 during a 2-day simulation-based education curriculum and two academic medical centers in Chicago. SUBJECTS: A total of 108 experienced attending physicians and 143 internal medicine and emergency medicine residents. INTERVENTION: None. MEASUREMENTS AND MAIN RESULTS: Using a previously published central venous catheter insertion skills checklist, we compared Veterans Affairs Medical Centers attending physicians' simulated central venous catheter insertion performance to the same simulated performance by internal medicine and emergency medicine residents from two academic centers. Attending physician performance was compared to residents' baseline and posttest (after simulation training) performance. Minimum passing scores were set previously by an expert panel. Attending physicians performed higher on the internal jugular (median, 75.86% items correct; interquartile range, 68.97-86.21) and subclavian (median, 83.00%; interquartile range, 59.00-86.21) assessments compared to residents' internal jugular (median, 37.04% items correct; interquartile range, 22.22-68.97) and subclavian (median, 33.33%; interquartile range, 0.00-70.37; both p < 0.001) baseline assessments. Overall simulated performance was poor because only 12 of 67 attending physicians (17.9%) met or exceeded the minimum passing score for internal jugular central venous catheter insertion and only 11 of 47 (23.4%) met or exceeded the minimum passing score for subclavian central venous catheter insertion. Resident posttest performance after simulation training was significantly higher than attending physician performance (internal jugular: median, 96%; interquartile range, 93.10-100.00; subclavian: median, 100%; interquartile range, 96.00-100.00; both p < 0.001). CONCLUSIONS: This study demonstrates highly variable simulated central venous catheter insertion performance among a national cohort of experienced attending physicians. Hospitals, healthcare systems, and governing bodies should recognize that even experienced physicians require periodic clinical skill assessment and retraining.

Variables that affect the process and outcome of feedback, relevant for medical training: a meta-review.

CONTEXT: Feedback is considered important in medical education. The literature is not clear about the mechanisms that contribute to its effects, which are often small to moderate and at times contradictory. A variety of variables seem to influence the impact of feedback on learning. The aim of this study was to determine which variables influence the process and outcomes of feedback in settings relevant to medical education. METHODS: A myriad of studies on feedback have been conducted. To determine the most researched variables, we limited our review to meta-analyses and literature reviews published in the period from January 1986 to February 2012. According to our protocol, we first identified features of the feedback process that influence its effects and subsequently variables that influence these features. We used a chronological model of the feedback process to categorise all variables found. RESULTS: A systematic search of ERIC, PsycINFO and MEDLINE yielded 1101 publications, which we reduced to 203, rejecting papers on six exclusion criteria. Of these, 46 met the inclusion criteria. In our four-phase model, we identified 33 variables linked to task performance (e.g. task complexity, task nature) and feedback reception (e.g. self-esteem, goal-setting behaviour) by trainees, and to observation (e.g. focus, intensity) and feedback provision (e.g. form, content) by supervisors that influence the subsequent effects of the feedback process. Variables from all phases influence the feedback process and effects, but variables that influence the quality of the observation and rating of the performance dominate the literature. There is a paucity of studies addressing other, seemingly relevant variables. CONCLUSIONS: The larger picture of variables that influence the process and outcome of feedback, relevant for medical education, shows many open spaces. We suggest that targeted studies be carried out to expand our knowledge of these important aspects of feedback in medical education.

A critical review of simulation-based mastery learning with translational outcomes.

OBJECTIVES: This article has two objectives. Firstly, we critically review simulation-based mastery learning (SBML) research in medical education, evaluate its implementation and immediate results, and document measured downstream translational outcomes in terms of improved patient care practices, better patient outcomes and collateral effects. Secondly, we briefly address implementation science and its importance in the dissemination of innovations in medical education and health care. METHODS: This is a qualitative synthesis of SBML with translational (T) science research reports spanning a period of 7 years (2006-2013). We use the 'critical review' approach proposed by Norman and Eva to synthesise findings from 23 medical education studies that employ the mastery learning model and measure downstream translational outcomes. RESULTS: Research in SBML in medical education has addressed a range of interpersonal and technical skills. Measured outcomes have been achieved in educational laboratories (T1), and as improved patient care practices (T2), patient outcomes (T3) and collateral effects (T4). CONCLUSIONS: Simulation-based mastery learning in medical education can produce downstream results. Such results derive from integrated education and health services research programmes that are thematic, sustained and cumulative. The new discipline of implementation science holds promise to explain why medical education innovations are adopted slowly and how to accelerate innovation dissemination.

Medical students' observations, practices, and attitudes regarding electronic health record documentation.

BACKGROUND: Medical students are increasingly documenting their patient notes in electronic health records (EHRs). Documentation short-cuts, such as copy-paste and templates, have raised concern among clinician-educators because they may perpetuate redundant, inaccurate, or even plagiarized notes. Little is known about medical students' experiences with copy-paste, templates and other "efficiency tools" in EHRs. PURPOSES: We sought to understand medical students' observations, practices, and attitudes regarding electronic documentation efficiency tools. METHODS: We surveyed 3rd-year medical students at one medical school. We asked about efficiency tools including copy-paste, templates, auto-inserted data, and "scribing" (documentation under a supervisor's name). RESULTS: Overall, 123 of 163 students (75%) responded; almost all frequently use an EHR for documentation. Eighty-six percent (102/119) reported at least sometimes observing residents copying data from other providers' notes and 60% (70/116) reported observing attending physicians doing so. Most students (95%, 113/119) reported copying from their own previous notes, and 22% (26/119) reported copying from residents. Only 10% (12/119) indicated that copying from other providers is acceptable, whereas 83% (98/118) believe copying from their own notes is acceptable. Most students use templates and auto-inserted data; 43% (51/120) reported documenting while signed in under an attending's name. Greater use of documentation efficiency tools is associated with plans to enter a procedural specialty and with lack of awareness of the medical school copy-paste policy. CONCLUSIONS: Students frequently use a range of efficiency tools to document in the electronic health record, most commonly copying their own notes. Although the vast majority of students believe it is unacceptable to copy-paste from other providers, most have observed clinical supervisors doing so.

Development and evaluation of cesarean section surgical training using computer-enhanced visual learning.

BACKGROUND: Skilled performance of cesarean deliveries is essential in obstetrics and gynecology residency. A computer-enhanced visual learning module (CEVL Cesarean) was developed to teach cesarean deliveries. METHODS: An online module presented cesarean deliveries as a series of components using text, audio, video and animation. First-year residents used CEVL Cesarean and were evaluated intra-operatively by trained raters, then provided feedback about surgical performance. Clinical outcomes were collected for approximately 50 cesarean deliveries for each resident. RESULTS: From 2010 to 2011, 12 first-year residents participated in the study. About 406 unique observed cesarean deliveries were analyzed. Procedures up to each resident's 70th case were analyzed by grouping cases in 10 s (cases 1-10 and 11-20), or deciles. Resident performance significantly improved by decile [χ(2)(6) = 47.56, p < 0.001]. When examining each resident's performance, surgical skill acquisition plateaued by cases 21-30. Procedural performance, independent of resident, also improved significantly by decile [χ(2)(6) = 186.95, p < 0.001], plateauing by decile 4 (cases 31-40). Throughout the observation period, operative time decreased by 3.84 min (p = 0.006). CONCLUSIONS: Pre-clinical teaching using computer-based modules for cesarean sections is feasible to develop. Novice surgeons required at least 30 procedures before performing the procedure competently. When residents performed competently, operative time and complications decreased.

Using Simulation-Based Mastery Learning to Achieve Excellent Learning Outcomes in Physical Therapist Education.

INTRODUCTION: The 2 aims of this observational study are (a) to describe the implementation and feasibility of a bed mobility skills simulation-based mastery learning (SBML) curricular module for physical therapist students and (b) to measure learning outcomes and student perceptions of this module. REVIEW OF LITERATURE: Simulation-based mastery learning is an outcome-based educational approach that has been successful in other health professions but has not been explored in physical therapy education. SUBJECTS: Eighty-seven students in a single cohort of a Doctor of Physical Therapy program. METHODS: The SBML module in this pretest-posttest study included a pretest, instruction, initial posttest, and additional rounds of instruction and assessment as needed for all learners to achieve the minimum passing standard (MPS) set using the Mastery Angoff and Patient Safety methods. Outcome measures were bed mobility assessment pass rates and scores, additional student and faculty time compared with a traditional approach, and student perceptions of their self-confidence and the module. RESULTS: All students achieved the MPS after 3 rounds of training and assessment beyond the initial posttest. Mean Total Scores improved from 67.6% (12.9%) at pretest to 91.4% (4.8%) at mastery posttest (P < .001, Cohen's d = 1.8, 95% CI [1.4-2.1]); mean Safety Scores improved from 75.2% (16.0%) at pretest to 100.0% (0.0%) at mastery posttest (P < .001, Cohen's d = 1.5, 95% CI [1.2-1.9]). Students who did not achieve the MPS at the initial posttest (n = 30) required a mean of 1.2 hours for additional instruction and assessment. Survey results revealed an increase in student confidence (P < .001) and positive student perceptions of the module. DISCUSSION AND CONCLUSION: Implementation of this SBML module was feasible and resulted in uniformly high levels of bed mobility skill acquisition. Based on rigorous learning outcomes, feasible requirements for implementation, and increased student confidence, SBML offers a promising approach for wider implementation in physical therapy education.

Systematic Development and Validity Evidence for a Checklist to Assess Bed Mobility Skills Among Physical Therapy Students.

INTRODUCTION: Assessments with strong validity evidence are necessary to accurately assess health professions students' performance of clinical skills. The aim of this study was to develop and validate a checklist assessment of physical therapy students' performance of bed mobility skills. METHODS: A checklist was developed using a 4-step process: 1) evidence review and preliminary checklist development, 2) Delphi review to reach consensus on content, 3) pilot testing and checklist editing, 4) final round of Delphi review. Consensus during Delphi review was defined as 100% of participants rating an item "keep as is" and zero comments in Round 1, and >50% of participants rating each item agree/strongly agree in subsequent Delphi rounds. Interrater reliability (IRR) was measured by two raters scoring 32 recorded exam simulations. RESULTS: All 48 items of the checklist reached consensus after three rounds of Delphi review (12 participants in Round 1, 11 participants in Rounds 2-3). IRR was substantial with 88.5% agreement, Cohen's kappa coefficient=0.61, p<0.001, 95% CI [0.56, 0.66]. DISCUSSION: This checklist has potential to be used to assess student readiness to evaluate and train patients in bed mobility tasks for first-time clinical experiences and to serve as a methodological template for future checklist development.

Raising the bar: reassessing standards for procedural competence.

BACKGROUND: How often passing standards for clinical skills examinations should be reexamined is unknown. PURPOSE: The goal is to determine if improved resident performance affects the passing standard imposed by expert judges. METHOD: In 2006, we set a passing standard for a central venous catheter insertion clinical skills examination using an expert panel. From 2007 to 2010, trainee scores improved steadily. In 2010, a new expert panel provided judgments for the examination using 4 consecutive years of resident performance data. Interrater reliabilities and test-retest reliability (stability) were calculated. Passing standards from 2006 and 2010 were compared. RESULTS: Judgments provided were reliable and stable. The new passing standard was 88% for internal jugular and 87% for subclavian central venous catheter insertion compared to 79% for both sites in 2006. CONCLUSIONS: Cumulative performance data influenced experts to set a more stringent minimum passing standard. Standards should be regularly reviewed to ensure they are fair and appropriately rigorous.