Quality of Widely Available Video Instructional Materials for Point-of-Care Ultrasound-Guided Procedure Training in Internal Medicine.

OBJECTIVES: Many instructional materials for point-of-care ultrasound (US)-guided procedures exist; however, their quality is unknown. This study assessed widely available educational videos for point-of-care US-guided procedures relevant to internal medicine: central venous catheterization, thoracentesis, and paracentesis. METHODS: We searched Ovid MEDLINE, YouTube, and Google to identify videos for point-of-care US-guided paracentesis, thoracentesis, and central venous catheterization. Videos were evaluated with a 5-point scale assessing the global educational value and a checklist based on consensus guidelines for competencies in point-of-care US-guided procedures. RESULTS: For point-of-care US-guided central venous catheterization, 12 videos were found, with an average global educational value score ± SD of 4.5 ± 0.7. Indications to abort the procedure were discussed in only 3 videos. Five videos described the indications and contraindications for performing central venous catheterization. For point-of-care US-guided thoracentesis, 8 videos were identified, with an average global educational value score of 4.0 ± 0.9. Only one video discussed indications to abort the procedure, and 3 videos discussed sterile technique. For point-of-care US-guided paracentesis, 7 videos were included, with an average global educational value score of 4.1 ± 0.9. Only 1 video discussed indications to abort the procedure, and 2 described the location of the inferior epigastric artery. CONCLUSIONS: The 27 videos reviewed contained good-quality general instruction. However, we noted a lack of safety-related information in most of the available videos. Further development of resources is required to teach internal medicine trainees skills that focus on the safety of point-of-care US guidance.

Defining Competencies for Ultrasound-Guided Bedside Procedures: Consensus Opinions From Canadian Physicians.

OBJECTIVES: This study sought to define the competencies in ultrasound knowledge and skills that are essential for medical trainees to master to perform ultrasound-guided central venous catheterization, thoracentesis, and paracentesis. METHODS: Experts in the 3 procedures were identified by a snowball technique through 3 Canadian tertiary academic health centers. Experts completed 2 rounds of surveys, including an 88-item central venous catheterization survey, a 96-item thoracentesis survey, and an 89-item paracentesis survey. For each item, experts were asked to determine whether the knowledge/skill described was essential, important, or marginal. Consensus on an item was defined as agreement by at least 80% of the experts. For items on which consensus was not reached during the first round of surveys, a second survey was created in which the experts were asked to rate the item in a binary fashion (essential/important versus marginal/unimportant). RESULTS: Of the 27 experts invited to complete each survey, 25 (93%) completed the central venous catheterization survey; 22 (81%) completed the thoracentesis survey; and 23 (85%) completed the paracentesis survey. The experts represented 8 specialties from 8 cities within Canada. A total of 22, 32, and 28 items were determined to be essential competencies for central venous catheterization, thoracentesis, and paracentesis, respectively, whereas 47, 38, and 42 competencies were determined to be important, and 8, 13, and 10 were determined to be marginal. The ability to perform real-time direct ultrasound guidance was considered essential only for the performance of central venous catheterization insertion. CONCLUSIONS: Our study presents expert consensus-derived ultrasound competencies that should be considered during the design and implementation of procedural skills training for learners.

Simulation in healthcare education: a best evidence practical guide. AMEE Guide No. 82.

Over the past two decades, there has been an exponential and enthusiastic adoption of simulation in healthcare education internationally. Medicine has learned much from professions that have established programs in simulation for training, such as aviation, the military and space exploration. Increased demands on training hours, limited patient encounters, and a focus on patient safety have led to a new paradigm of education in healthcare that increasingly involves technology and innovative ways to provide a standardized curriculum. A robust body of literature is growing, seeking to answer the question of how best to use simulation in healthcare education. Building on the groundwork of the Best Evidence in Medical Education (BEME) Guide on the features of simulators that lead to effective learning, this current Guide provides practical guidance to aid educators in effectively using simulation for training. It is a selective review to describe best practices and illustrative case studies. This Guide is the second part of a two-part AMEE Guide on simulation in healthcare education. The first Guide focuses on building a simulation program, and discusses more operational topics such as types of simulators, simulation center structure and set-up, fidelity management, and scenario engineering, as well as faculty preparation. This Guide will focus on the educational principles that lead to effective learning, and include topics such as feedback and debriefing, deliberate practice, and curriculum integration - all central to simulation efficacy. The important subjects of mastery learning, range of difficulty, capturing clinical variation, and individualized learning are also examined. Finally, we discuss approaches to team training and suggest future directions. Each section follows a framework of background and definition, its importance to effective use of simulation, practical points with examples, and challenges generally encountered. Simulation-based healthcare education has great potential for use throughout the healthcare education continuum, from undergraduate to continuing education. It can also be used to train a variety of healthcare providers in different disciplines from novices to experts. This Guide aims to equip healthcare educators with the tools to use this learning modality to its full capability.

Observational study of agreement between attending and trainee physicians on the surprise question: "Would you be surprised if this patient died in the next 12 months?"

BACKGROUND: Optimal end-of-life care requires identifying patients that are near the end of life. The extent to which attending physicians and trainee physicians agree on the prognoses of their patients is unknown. We investigated agreement between attending and trainee physician on the surprise question: "Would you be surprised if this patient died in the next 12 months?", a question intended to assess mortality risk and unmet palliative care needs. METHODS: This was a multicentre prospective cohort study of general internal medicine patients at 7 tertiary academic hospitals in Ontario, Canada. General internal medicine attending and senior trainee physician dyads were asked the surprise question for each of the patients for whom they were responsible. Surprise question response agreement was quantified by Cohen's kappa using Bayesian multilevel modeling to account for clustering by physician dyad. Mortality was recorded at 12 months. RESULTS: Surprise question responses encompassed 546 patients from 30 attending-trainee physician dyads on academic general internal medicine teams at 7 tertiary academic hospitals in Ontario, Canada. Patients had median age 75 years (IQR 60-85), 260 (48%) were female, and 138 (25%) were dependent for some or all activities of daily living. Trainee and attending physician responses agreed in 406 (75%) patients with adjusted Cohen's kappa of 0.54 (95% credible interval 0.41 to 0.66). Vital status was confirmed for 417 (76%) patients of whom 160 (38% of 417) had died. Using a response of "No" to predict 12-month mortality had positive likelihood ratios of 1.84 (95% CrI 1.55 to 2.22, trainee physicians) and 1.51 (95% CrI 1.30 to 1.72, attending physicians), and negative likelihood ratios of 0.31 (95% CrI 0.17 to 0.48, trainee physicians) and 0.25 (95% CrI 0.10 to 0.46, attending physicians). CONCLUSION: Trainee and attending physician responses to the surprise question agreed in 54% of cases after correcting for chance agreement. Physicians had similar discriminative accuracy; both groups had better accuracy predicting which patients would survive as opposed to which patients would die. Different opinions of a patient's prognosis may contribute to confusion for patients and missed opportunities for engagement with palliative care services.

Cost-Efficient Medical Education: An Innovative Approach to Creating Educational Products.

BACKGROUND: Cost is a barrier to creating educational resources, and new educational initiatives are often limited in distribution. Medical training programs must develop strategies to create and implement cost-effective educational programming. OBJECTIVE: We developed high-quality medical programming in procedural instruction with efficient economics, reaching the most trainees at the lowest cost. METHODS: The Just-In-Time online procedural program was developed at the University of Toronto in Canada, aiming to teach thoracentesis, paracentesis, and lumbar puncture skills to internal medicine trainees. Commercial vendors quoted between CAD $50,000 and $100,000 to create 3 comprehensive e-learning procedural modules-a cost that was prohibitive. Modules were therefore developed internally, utilizing 4 principles aimed at decreasing costs while creating efficiencies: targeting talent, finding value abroad, open source expansion, and extrapolating efficiency. RESULTS: Procedural modules for thoracentesis, paracentesis, and lumbar puncture were created for a total cost of CAD $1,200, less than 3% of the anticipated cost in utilizing traditional commercial vendors. From November 2016 until October 2018, 1800 online instructional sessions have occurred, with over 3600 pageviews of content utilized. While half of the instructional sessions occurred within the city of Toronto, utilization was documented in 10 other cities across Canada. CONCLUSIONS: The Just-in-Time online instructional program successfully created 3 procedural modules at a fraction of the anticipated cost and appeared acceptable to residents based on website utilization.

Professional Responsibilities and Personal Impacts: Residents' Experiences as Participants in Education Research.

PURPOSE: Although the field of medical education research is growing and residents are increasingly recruited to participate as subjects in research studies, little is known about their experiences. The goal of this study was to explore the experiences and perceptions of residents who are study participants in medical education research. METHOD: A phenomenographic approach was chosen to examine the range of residents' experiences as research participants. A maximum variation sampling strategy was used to identify residents with diverse experiences. Semistructured interviews that explored experiences as research participants were conducted with 19 residents in internal medicine, general surgery, and pediatrics at the University of Toronto in 2015-2016. RESULTS: The perceptions and experiences of participants fell into two categories. First, participation was seen as a professional responsibility to advance the profession, including a desire to improve future educational practices and a sense of responsibility to contribute to the academic cause. Second, the experience was noted for its personal impact, including benefits (e.g., receiving monetary incentives or novel educational experiences) and risks (e.g., coercion and breaches of confidentiality). The time required to participate in a study was identified as one of the most important factors affecting willingness to participate and the impact of participation. CONCLUSIONS: Being a participant in medical education research can be perceived in different ways. Understanding the view of resident participants is important to optimize potential benefits and minimize risks and negative consequences for them, thus fostering ready participation and high-quality research.

Does Making the Numerical Values of Verbal Anchors on a Rating Scale Available to Examiners Inflate Scores on a Long Case Examination?

PURPOSE: Rating scales are frequently used for scoring assessments in medical education. The effect of changing the structural elements of a rating scale on students' examination scores has received little attention in the medical education literature. This study assessed the impact of making the numerical values of verbal anchors on a rating scale available to examiners in a long case examination (LCE). METHOD: During the 2011-2012 academic year, the numerical values of verbal anchors on a rating scale for an internal medicine clerkship LCE were made available to faculty examiners. Historically, and specifically in the control year of 2010-2011, examiners only saw the scale's verbal anchors and were blinded to the associated numerical values. To assess the impact of this change, the authors compared students' LCE scores between the two cohort years. To assess for differences between the two cohorts, they compared students' scores on other clerkship assessments, which remained the same between the two cohorts. RESULTS: From 2010-2011 (n = 226) to 2011-2012 (n = 218), the median LCE score increased significantly from 82.11% to 85.02% (P < .01). Students' performance on the other clerkship assessments was similar between cohorts. CONCLUSIONS: Providing examiners with the numerical values of verbal anchors on a rating scale, in addition to the verbal anchors themselves, led to a significant increase in students' scores on an internal medicine clerkship LCE. When constructing or changing rating scales, educators must consider the potential impact of the rating scale structure on students' scores.

An Equivalence Trial Comparing Instructor-Regulated With Directed Self-Regulated Mastery Learning of Advanced Cardiac Life Support Skills.

INTRODUCTION: Instructor-led simulation-based mastery learning of advanced cardiac life support (ACLS) skills is an effective and focused approach to competency-based education. Directed self-regulated learning (DSRL) may be an effective and less resource-intensive way to teach ACLS skills. METHODS: Forty first-year internal medicine residents were randomized to either simulation-based DSRL or simulation-based instructor-regulated learning (IRL) of ACLS skills using a mastery learning model. Residents in each intervention completed pretest, posttest, and retention test of their performance in leading an ACLS response to a simulated scenario. Performance tests were assessed using a standardized checklist. Residents in the DSRL intervention were provided assessment instruments, a debriefing guide, and scenario-specific teaching points, and they were permitted to access relevant online resources. Residents in the IRL intervention had access to the same materials; however, the teaching and debriefing were instructor led. RESULTS: Skills of both the IRL and DSRL interventions showed significant improvement after the intervention, with an average improvement on the posttest of 21.7%. After controlling for pretest score, there was no difference between intervention arms on the posttest [F(1,37) = 0.02, P = 0.94] and retention tests [F(1,17) = 1.43, P = 0.25]. Cost savings were realized in the DSRL intervention after the fourth group (16 residents) had completed each intervention, with an ongoing savings of $80 per resident. CONCLUSIONS: Using a simulation-based mastery learning model, we observed equivalence in learning of ACLS skills for the DSRL and IRL conditions, whereas DSRL was more cost effective.

Workplace-Based Assessment of Internal Medicine Resident Diagnostic Accuracy.

BACKGROUND: Making an accurate diagnosis is a core skill residents must develop. Assessments of this skill and decisions to grant residents clinical independence often are based on global impressions. A workplace-based assessment of diagnostic accuracy could be a useful part of a competency-based assessment program and could inform decisions about granting residents independence. INNOVATION: We developed a method for measuring diagnostic accuracy that was integrated into the workflow of internal medicine residents and attending physicians. METHODS: Four senior medical residents and 6 attending physicians working in the internal medicine clinical teaching unit of a tertiary hospital participated in this study. To determine their diagnostic accuracy, residents documented a leading diagnosis for each patient they evaluated in the emergency department. After reviewing each case with the resident and after examining the patient, the resident's attending physician documented the diagnosis. Discharge diagnosis was determined by retrospective chart review to allow determination of resident and attending physician diagnostic accuracy. Data were collected for 240 consecutive patients referred for a medicine consultation. RESULTS: Resident diagnostic accuracy was 66% (95% CI 60-72), whereas attending physician accuracy was significantly higher at 79% (95% CI 74-84, P < .001). By logistic regression, the accuracy of the attending physician was found to be influenced by the accuracy of the resident. Participants felt this process motivated them to improve their clinical reasoning. CONCLUSIONS: Measuring resident diagnostic accuracy provides information that could be used in a competency-based assessment program to provide feedback and motivation to stimulate performance improvement.