Retention of En Route Cricothyroidotomy Skills in Novice Providers Following a Simulation-Based Mastery Learning Curriculum.

INTRODUCTION: Surgical cricothyroidotomy (SC) is a vital skill that combat first responders must master as airway obstruction is the third most preventable cause of death on the battlefield. Degradation of skills over time is a known problem, and there is inadequate knowledge regarding the rate of SC skill retention. Our prior study showed that simulation-based mastery learning was effective in training 89 novices how to reliably perform an en route SC to mastery performance standards. This study aims to assess the durability of this skill by bringing participants back in 3 separate cohorts at 6, 12, or 24 months following the initial training to perform SC in the same test environment. MATERIALS AND METHODS: This was a randomized prospective trial. Random cohorts of equal subjects who previously underwent SC simulation-based mastery learning training were selected to return at 6, 12, and 24 months to retest in the same en route medical evacuation (MEDEVAC) helicopter scenario. A total of 22, 14, and 10 subjects returned at 6, 12, and 24 months, respectively, due to Coronavirus-19 impacts and travel limitations. Participants in the 24-month cohort received a refresher training prior to retesting. All attempts were recorded and blindly graded using the same 10 item standardized SC checklist used in initial training. Our previous work found that mastery criteria for performing a SC were ≤40 seconds and completion of 9/10 items on the checklist. Outcome measures in this study were time to complete the procedure and percent of subjects who completed at least 9/10 items on the SC checklist. RESULTS: There was an increase in time required to complete the procedure compared to initial training in all three retesting cohorts (initial: median 27.50, interquartile range 25.38-31.07 seconds; 6 months: median 36.33, interquartile range 31.59-55.22 seconds; 12 months: median 49.50, interquartile range 41.75-60.75 seconds; 24 months: median 38.79, interquartile range 30.20-53.08 seconds; P < .0001, P < .0001, P = .0039). There was a decline in median value checklist scores compared to initial training in the 6- and 12-month retesting cohorts (initial: median 10.00/10, interquartile range 9.50-10.00; 6 months: median 8.00/10, interquartile range 6.75-9.00; 12 months: median 8.00/10, interquartile range 6.75-9.25; P < .0001, P < .001). There was no difference in median checklist scores between the initial and 24-month retesting scenario (initial: median 10.00/10, interquartile range 9.50-10.00; 24 months: 10.00/10, interquartile range 9.00-10.00; P= .125). There was a decrease in retention of skills as only 31.82% of subjects at 6 months and 14.29% at 12 months met the defined passing criteria of time to completion of ≤40 seconds and checklist score of ≥9/10. A brief refresher course several months prior to the 24-month cohort retesting greatly increased the retention of SC procedural skills, with 60% of subjects meeting the time and checklist criteria. CONCLUSIONS: This study showed that the skill required to perform a SC after initial mastery training does decay significantly. A brief refresher course can help increase retention of skills. Based on our findings SC skills should be refreshed at a minimum of every 6 months to assure optimal proficiency.

A Review of Verbal and Written Patient Handoffs Applicable to the U.S. Military's Expeditionary Care System.

INTRODUCTION: Long considered a danger point in patient care, handoffs and patient care transitions contribute to medical errors and adverse events. Without standardization of patient handoffs, communication breakdowns arise and critical patient information is lost. Minimal training and informal learning have led to a lack of understanding the process involved in this vital aspect of patient care. In 2017, the U.S. Army commissioned a report to study the process of patient handoffs and identify training gaps. Our report summarizes that process and makes recommendations for implementation. MATERIALS AND METHODS: Scoping literature review of 139 articles published between 1999 and 2017 using PubMed, CINAHL, Cochrane, and Medline databases. Verbal tools for handoffs were evaluated against 12 criteria including patient ID, history, current situation, contingency planning, ability to ask questions, ownership, and read back. Written tools were evaluated against a matrix of 126 casualty/treatment attributes. RESULTS: Among verbal communication protocols, the highest scoring handoff mnemonics were HAND ME AN ISOBAR, IPASS the BATON, and I-SBARQ. Among written handoff tools, the highest scoring documents were the Special Operations Forces (SOF) Mechanism, Injuries, Signs, and Treatment (MIST) Casualty Treatment Card and the Department of Defense (DD) Form 1380 Tactical Combat Casualty Care (TCCC) Card. Four critical process elements for patient handoffs and transfers were identified: (1) interactive communications, (2) limited interruptions, (3) a process for verification, and (4) an opportunity to review any relevant historical data. CONCLUSIONS: The findings in this review highlight the need for standardized tools and techniques for patient handoffs in the U.S. Military's expeditionary care system. Future research is needed to trial verbal and nonverbal handoffs under field conditions to gather observational data to assess effectiveness. The results of our gap analyses may provide researchers insight for determining which handoffs to study. If standardized handoffs are utilized, training programs should incorporate the four critical elements into their curricula.

Innovative Curriculum to Train Medical Students on Lateral Canthotomy and Cantholysis.

PURPOSE: Orbital compartment syndrome (OCS) is an ocular emergency requiring prompt decompression with a lateral canthotomy and cantholysis (LCC) within 2 hours. This study evaluates the feasibility and effectiveness of a standardized LCC curriculum to train medical students to identify and treat OCS. METHODS: This was a prospective, non-randomized, non-comparative cohort study of 39 novice first-year medical students with no prior LCC training who underwent a standardized LCC curriculum incorporating both didactic and hands-on procedural training. Didactic knowledge of orbital anatomy and OCS was evaluated with written pre- and post-knowledge testing. Expert performance criteria were determined by expert consensus based on the performance of three oculoplastic surgeons and were defined as correctly performing all 12 critical checklist steps of an LCC within 3 minutes twice consecutively on a Sonalyst LCC training system eye model. Utilizing the principles of mastery learning, participants learned how to perform an LCC in a classroom environment and were evaluated on a final test of proficiency in a training lane designed to simulate an austere military environment. RESULTS: Participants required a median of 3.0 practice iterations to achieve expert performance in the classroom environment. During the testing phase, all participants correctly identified the eye with OCS, and 77% (n = 30) of learners successfully performed an LCC at the expert level within their first attempt. The median completion time of those who passed on their first testing was 130 seconds. The mean LCC knowledge test scores significantly improved from 48.7% to 71.2% (P < .001). CONCLUSION: This study successfully developed a standardized LCC curriculum utilizing the principles of hands-on mastery learning to train novice learners to perform an LCC efficiently and effectively.

The SOAP Feedback Training Program.

BACKGROUND: Accessible and efficient opportunities for health professional faculty to hone feedback skills are limited. In addition, feedback models to apply to the objective structured clinical examination (OSCE) setting are lacking. APPROACH: Annually, paediatric interns from Children's National Hospital and Walter Reed National Military Medical Center participate in an OSCE, which includes faculty observation and immediate feedback to trainees. In 2018, we incorporated the subjective, objective, assessment, plan (SOAP) Feedback Training Program during 20 min of the pre-OSCE faculty orientation. The SOAP Feedback Training Program introduced the SOAP feedback model (subjective, objective, assessment, plan), facilitated practice in pairs and distributed a cognitive aid referencing the model. We evaluated the quality of faculty feedback exchanges during the 2018 OSCE via retrospective video review using the Direct Observation of Clinical Skills Feedback Scale (DOCS-FBS). We compared the results to the 2015 initial evaluation and used focus groups to understand how and why faculty feedback changed. EVALUATION: Comparison of the initial evaluation to the post-SOAP Feedback Training Program intervention data using a Wilcoxon signed rank test showed statistically significant improvement in six of eight feedback items on the DOCS-FBS. Causal coding of focus group transcripts revealed that the SOAP Feedback Training Program evoked affective responses, reinforced prior practice in feedback delivery, improved feedback organisation and increased feedback delivery preparation. IMPLICATIONS: The SOAP Feedback Training Program is an effective intervention to teach the SOAP feedback model and improve faculty feedback quality in an OSCE setting. It is efficient and low resource, facilitating its potential use in settings beyond the OSCE.

Development of a Simulation Surgical Cricothyroidotomy Curriculum for Novice Providers: A Learning Curve Study.

INTRODUCTION: Airway obstruction is the third most common cause of preventable death on the battlefield, accounting for 1%-2% of total combat fatalities. No previous surgical cricothyroidotomy (SC) studies have analyzed the learning curve required to obtain proficiency despite being studied in numerous other surgical technique training experiments. The aims of this study were to establish expert SC performance criteria, develop a novel standardized SC curriculum, and determine the necessary number of practice iterations required by a novice to reach this pre-determined performance goal. MATERIALS AND METHODS: A standardized checklist and SC performance standards were established based on the performance of 12 board certified Military Health System surgeons with prior experience on performing a SC using a simulated trauma mannequin. Expert-level criteria were defined as a SC time to completion of 40 s or less and checklist score of at least 9/10, including all critical steps. Study subjects included 89 novice providers (54 active-duty first- and second-year medical students and 35 Navy corpsmen). Subjects received instruction on performing a SC using the principles of mastery learning and performed a final test of SC proficiency on a trauma mannequin within a realistic simulated MEDEVAC helicopter. The total number of subject practice attempts, checklist scores, and time to completion were measured and/or blindly scored. Learning curve and exponential plateau equations were used to characterize their improvement in mean time to SC completion and checklist scores. RESULTS: Mean pre-test knowledge scores for the entire group were 11.8 ± 3.1 out of 24 points. Total mean practice learning plateaued at checklist scores of 9.9/10 after 7 iterations and at a mean completion time of 30.4 s after 10 iterations. During the final test performance in the helicopter, 67.4% of subjects achieved expert-level performance on the first attempt. All subjects achieved expert-level performance by the end of two additional attempts. While a significantly larger proportion of medical students (79.9%) successfully completed the helicopter test on the first attempt compared to corpsmen (54.3%), there were no statistically significant differences in mean SC completion times and checklist scores between both groups (P > 0.05). Medical students performed a SC only 1.3 s faster and scored only 0.16 points higher than corpsmen. The effect size for differences were small to negligible (Cohen's d range 0.18-0.33 for SC completion time; Cohen's d range 0.45-0.46 for checklist scores). CONCLUSION: This study successfully defined SC checklist scores and completion times based on the performance of experienced surgeons on a simulator. Using these criteria and the principles of mastery learning, novices with little knowledge and experience in SC were successfully trained to the level of experienced providers. All subjects met performance targets after training and overall performance plateaued after approximately seven iterations. Over two-thirds of subjects achieved the performance target on the first testing attempt in a simulated helicopter environment. Performance was comparable between medical student and corpsmen subgroups. Further research will assess the durability of maintaining SC skills and the timing for introducing refresher courses after initial skill acquisition.

Validation of a Modified Jefferson Scale of Empathy for Observers to Assess Trainees.

OBJECTIVE: "Demonstrate insight and understanding into emotion" is a competency amenable to simulation-based assessment. The Jefferson Scale of Patient Perceptions of Physician Empathy (JSPPPE) has validity evidence for patients to assess provider empathy. A version adapted for a third-party observers does not exist. Our aim was to modify the JSPPPE and use recorded standardized encounters to obtain validity evidence. METHODS: This cross-sectional study used video and data collected from 2 pediatric residencies. In 2018, 4 raters reviewed 24 videos of 12 interns communicating with standardized patients (SP) in 2 encounters and completed a modified JSPPE for observers (JSEO). Reliability between raters was established using Intraclass Correlations (ICC). JSEO mean scores were correlated to Essential Elements of Communication (EEC), JSPPPE, and faculty composite interpersonal communication (IC) scores using Spearman Rank. RESULTS: The mean ICC for all 4 raters was 0.573 (0.376-0.755). When ICC was calculated for pairs of raters, Rater 1 was an outlier. ICCs for mean scores for pairs among the 3 remaining raters was 0.81 to 0.84. Mean JSEO scores from the four raters correlated with the JSPPPE (rho = 0.45, P = .03) and IC (rho = 0.68, P < .001), but not the EEC (rho = 0.345, P = .1). CONCLUSIONS: We found validity evidence for the use of a modified JSPPPE for an observer to assess empathy in a recorded encounter with a SP. This may be useful as medical educators shift toward competency-based tracking. The brevity of this tool and potential assessment using video are also appealing.

I-PASS Mentored Implementation Handoff Curriculum: Frontline Provider Training Materials.

Introduction: The I-PASS Handoff Program is a comprehensive handoff curriculum that has been shown to decrease rates of medical errors and adverse events during patient handoffs. Frontline providers are the key individuals participating in handoffs of patient care. It is important they receive robust handoff training. Methods: The I-PASS Mentored Implementation Handoff Curriculum frontline provider training materials were created as part of the original I-PASS Study and adapted for the Society of Hospital Medicine (SHM) I-PASS Mentored Implementation Program. The adapted materials embrace a flipped classroom approach with an emphasis on adult learning theory principles. The training includes an overview of I-PASS handoff techniques, TeamSTEPPS team communication strategies, verbal handoff simulation scenarios, and a printed handoff document exercise. Results: As part of the SHM I-PASS Mentored Implementation Program, 2,735 frontline providers were trained at 32 study sites (16 adult and 16 pediatric) across North America. At the end of their training, 1,762 frontline providers completed the workshop evaluation form (64% response rate). After receiving the training, over 90% agreed/strongly agreed that they were able to distinguish a good- from a poor-quality handoff, articulate the elements of the I-PASS mnemonic, construct a high-quality patient summary, advocate for an appropriate environment for handoffs, and participate in handoff simulations. Universally, the training provided them with knowledge and skills relevant to their patient care activities. Discussion: The I-PASS frontline training materials were rated highly by those trained and are an integral part of a successful I-PASS Handoff Program implementation.

Rapid Application Temporary Tattoos for Medical Moulage: From Development to Testing and Commercialization.

Military medical education and training often utilize theatrical makeup, or moulage, to simulate injuries or pathologies. Traditional methods of moulage application are incredibly realistic when expertly applied. However, moulage can be expensive in terms of supplies, manpower, and time. We proposed that by creating a library of illustrations for use with temporary tattoos, the overall cost of moulage could decrease with little to no impact on training goals and objectives. The development, detailed testing and plans for commercialization are outlined herein.

A Focus on Feedback: Improving Learner Engagement and Faculty Delivery of Feedback in Hospital Medicine.

Feedback is an integral part of medical education. However, there is great variation of training and effectiveness of feedback delivery, especially in the inpatient setting. The unique learning environment provided in hospital medicine allows teachers the opportunity to provide feedback on learner performance under several longitudinal observations in areas such as direct patient care, procedural tasks, and interdisciplinary team leadership skills. Most important, feedback should occur on more than one occasion to truly empower change in knowledge, attitude, and skills. This article aims to provide the reader with foundational theories on feedback and strategies to use best practices for delivery.

I-PASS Mentored Implementation Handoff Curriculum: Champion Training Materials.

Introduction: The I-PASS Handoff Program is a comprehensive handoff curriculum that has been shown to decrease rates of medical errors and adverse events during patient handoffs. I-PASS champions are a critical part of the implementation and sustainment of this curriculum, and therefore, a rigorous program to support their training is necessary. Methods: The I-PASS Handoff champion training materials were created for the original I-PASS Study and adapted for the Society of Hospital Medicine (SHM) I-PASS Mentored Implementation Program. The adapted materials embrace a flipped classroom approach and adult learning theory. The training includes an overview of I-PASS handoff techniques, an opportunity to practice evaluating handoffs with the I-PASS observation tools using a handoff video vignette, and other key implementation principles. Results: As part of the SHM I-PASS Mentored Implementation Program, 366 champions were trained at 32 sites across North America and participated in a total of 3,491 handoff observations. A total of 346 champions completed the I-PASS Champion Workshop evaluation form at the end of their training (response rate: 94.5%). After receiving the training, over 90% agreed/strongly agreed that it provided them with knowledge or skills critical to their patient care activities and that they were able to distinguish the difference between high- and poor-quality handoffs, competently use the I-PASS handoff assessment tools, and articulate the importance of handoff observations. Conclusion: The I-PASS champion training materials were rated highly by those trained and are an integral part of a successful I-PASS Handoff Program implementation.

Patient safety after implementation of a coproduced family centered communication programme: multicenter before and after intervention study

OBJECTIVE: To determine whether medical errors, family experience, and communication processes improved after implementation of an intervention to standardize the structure of healthcare provider-family communication on family centered rounds. DESIGN: Prospective, multicenter before and after intervention study. SETTING: Pediatric inpatient units in seven North American hospitals, 17 December 2014 to 3 January 2017. PARTICIPANTS: All patients admitted to study units (3106 admissions, 13171 patient days); 2148 parents or caregivers, 435 nurses, 203 medical students, and 586 residents. INTERVENTION: Families, nurses, and physicians coproduced an intervention to standardize healthcare provider-family communication on ward rounds ("family centered rounds"), which included structured, high reliability communication on bedside rounds emphasizing health literacy, family engagement, and bidirectional communication; structured, written real-time summaries of rounds; a formal training programme for healthcare providers; and strategies to support teamwork, implementation, and process improvement. MAIN OUTCOME MEASURES: Medical errors (primary outcome), including harmful errors (preventable adverse events) and non-harmful errors, modeled using Poisson regression and generalized estimating equations clustered by site; family experience; and communication processes (eg, family engagement on rounds). Errors were measured via an established systematic surveillance methodology including family safety reporting. RESULTS: The overall rate of medical errors (per 1000 patient days) was unchanged (41.2 (95% confidence interval 31.2 to 54.5) pre-intervention v 35.8 (26.9 to 47.7) post-intervention, P=0.21), but harmful errors (preventable adverse events) decreased by 37.9% (20.7 (15.3 to 28.1) v 12.9 (8.9 to 18.6), P=0.01) post-intervention. Non-preventable adverse events also decreased (12.6 (8.9 to 17.9) v 5.2 (3.1 to 8.8), P=0.003). Top box (eg, "excellent") ratings for six of 25 components of family reported experience improved; none worsened. Family centered rounds occurred more frequently (72.2% (53.5% to 85.4%) v 82.8% (64.9% to 92.6%), P=0.02). Family engagement 55.6% (32.9% to 76.2%) v 66.7% (43.0% to 84.1%), P=0.04) and nurse engagement (20.4% (7.0% to 46.6%) v 35.5% (17.0% to 59.6%), P=0.03) on rounds improved. Families expressing concerns at the start of rounds (18.2% (5.6% to 45.3%) v 37.7% (17.6% to 63.3%), P=0.03) and reading back plans (4.7% (0.7% to 25.2%) v 26.5% (12.7% to 7.3%), P=0.02) increased. Trainee teaching and the duration of rounds did not change significantly. CONCLUSIONS: Although overall errors were unchanged, harmful medical errors decreased and family experience and communication processes improved after implementation of a structured communication intervention for family centered rounds coproduced by families, nurses, and physicians. Family centered care processes may improve safety and quality of care without negatively impacting teaching or duration of rounds. TRIAL REGISTRATION: ClinicalTrials.gov NCT02320175.

Multiple Patient Casualty Scenarios: A Measurement Tool for Teamwork.

INTRODUCTION: Teamwork is a critical aspect of patient care and is especially salient in response to multiple patient casualties. Effective training and measurement improve team performance. However, the literature currently lacks a scientifically developed measure of team performance within multiple causality scenarios, making training and feedback efforts difficult. The present effort addresses this gap by integrating the input of subject matter experts and the science of multicasualty teams and training to (1) identify overarching teamwork processes and corresponding behaviors necessary for team performance and (2) develop a behavioral observation tool to optimize teamwork in multicasualty training efforts. METHOD: A search of articles including team performance frameworks associated with team training was conducted, leading to the identification of a total of 14 articles. Trained coders extracted teamwork processes and the corresponding team behaviors indicative of effective performance from these articles. Five subject matter experts were interviewed using the critical incident technique to identify additional behaviors. RESULTS: Team situation awareness, team leadership, coordination, and information exchange emerged as the four core team processes required for team performance in scenarios with multiple patient casualties. Relevant behaviors and subbehaviors within these overarching processes were identified to inform a pilot behavioral framework of team performance. CONCLUSIONS: The processes and associated behaviors identified within this effort serve as scientifically grounded behaviors of team performance in the case of multiple patient casualties simulated training scenarios. Future work can use and further refine these results to ensure that measures of team performance are grounded in specific, observable, and scientifically delineated behaviors.

Validity Evidence for a Serious Game to Assess Performance on Critical Pediatric Emergency Medicine Scenarios.

INTRODUCTION: We developed a first-person serious game, PediatricSim, to teach and assess performances on seven critical pediatric scenarios (anaphylaxis, bronchiolitis, diabetic ketoacidosis, respiratory failure, seizure, septic shock, and supraventricular tachycardia). In the game, players are placed in the role of a code leader and direct patient management by selecting from various assessment and treatment options. The objective of this study was to obtain supportive validity evidence for the PediatricSim game scores. METHODS: Game content was developed by 11 subject matter experts and followed the American Heart Association's 2011 Pediatric Advanced Life Support Provider Manual and other authoritative references. Sixty subjects with three different levels of experience were enrolled to play the game. Before game play, subjects completed a 40-item written pretest of knowledge. Game scores were compared between subject groups using scoring rubrics developed for the scenarios. Validity evidence was established and interpreted according to Messick's framework. RESULTS: Content validity was supported by a game development process that involved expert experience, focused literature review, and pilot testing. Subjects rated the game favorably for engagement, realism, and educational value. Interrater agreement on game scoring was excellent (intraclass correlation coefficient = 0.91, 95% confidence interval = 0.89-0.9). Game scores were higher for attendings followed by residents then medical students (Pc < 0.01) with large effect sizes (1.6-4.4) for each comparison. There was a very strong, positive correlation between game and written test scores (r = 0.84, P < 0.01). CONCLUSIONS: These findings contribute validity evidence for PediatricSim game scores to assess knowledge of pediatric emergency medicine resuscitation.

Asymptomatic Kawasaki Disease in a 3-Month-Old Infant.

Kawasaki disease (KD) is the leading cause of acquired heart disease in children in the United States. It is a systemic vasculitis characterized by diffuse inflammation of medium and small blood vessels. If untreated it can lead to myocardial infarction, ischemic heart disease, or sudden death. Early recognition and treatment decrease the incidence of coronary consequences, resulting in improved clinical outcomes. Incomplete KD is much less likely to fulfill major clinical diagnostic criteria. Infants <12 months of age are more likely to have an incomplete presentation, and children <6 months of age are more likely to develop cardiac complications. We present a case of a 3-month-old, previously healthy white boy who was noted to have a new transient cardiac murmur during a routine health assessment. He was completely asymptomatic, and physical examination was otherwise within normal limits. An echocardiogram was performed and showed abnormal dilation of several coronary arteries, consistent with the coronary ectasia associated with KD. Laboratory evaluation was significant for values suggestive of systemic inflammation. Based on these results, a presumed diagnosis of incomplete KD was made and treatment administered. Close surveillance was undertaken, and serial laboratory studies and imaging showed gradual resolution of inflammatory markers and cardiac ectasia. This unique case of incomplete KD without any of the physical signs normally associated with the disease emphasizes the spectrum of presentation and the possibility of missing a diagnosis of incomplete disease, reinforcing the need to remain vigilant.

Do Standardized Patients Have Concerns About Students Not Captured by Traditional Assessment Forms?

Construct: Traditionally, standardized patients (SPs) assess students' clinical skills principally through numerical rating forms-an approach that may not fully capture SPs' concerns. SPs are students' closest approximation to real patients. To maximally benefit students' clinical training and evaluation it is important to find ways to give voice to the totality of SPs' perspectives. BACKGROUND: SPs have been shown to be a reliable and valid means to assess medical students' clinical skills in clinical skills examinations. We noticed, however, that SPs often express "off the record" concerns about students, which they do not include on traditional assessment forms. APPROACH: To explore these "off the record" concerns, we designed a Concerns item and added it to the traditional assessment form for an end-of-3rd-year clinical skills examination shared by three medical schools. We asked SPs to use this Concerns item to identify students about whom they had any "gut-level" concerns and provided them with a narrative opportunity to explain why. SPs were informed that the purpose of the item was to help students with difficulties and was not part of the student's grade. RESULTS: We analyzed the concerns data using quantitative and qualitative methods. Of 551 students at three schools, 223 (∼40%) had concerns recorded. Seventy students received two or more concerns. Qualitative analysis of SPs' comments revealed 3 major categories of concern: communication and interpersonal skills, history taking, and physical exam. Grouped under each were several subcategories. More than half of the written comments from the SPs related to the communication/interpersonal skills category and included subcategories commonly addressed in communications courses: lack of empathy, good listening skills, and lack of connection to the patient. They also included subcategories that in our experience are less commonly addressed: odd or off-putting mannerisms, lack of confidence, unprofessional behavior, domineering behavior, and biased behavior. Another 47% of concerns identified deficiencies in history taking and physical examination. Of the students with concerns noted by two or more SPs, SPs' narrative comments on 84%, 42%, and 48% of the students in the domains of communications, history, and physical exam respectively indicated potential problems not identified by scores on the traditional assessment form. CONCLUSION: The Concerns item is a narrative assessment method that may add value to traditional quantitative scoring by identifying and characterizing problematic student performance not captured by the traditional assessment form. It may thus contribute to giving fuller voice to the totality of SPs' perspective.

Acute Otitis Media Curriculum.

INTRODUCTION: Studies have shown that clinicians do a poor job of diagnosing middle ear infections, and an inaccurate diagnosis may result in inappropriate treatment, such as overuse of antibiotics. Antibiotic overuse is known to lead to increased bacterial resistance and puts patients at risk of deleterious side effects, including allergic reactions, vomiting, diarrhea, and rash. In this curriculum, we teach three important topics for achieving a successful middle ear exam and diagnosis of pathology. METHODS: The topics covered during the session are middle ear examination techniques, diagnosis of middle ear pathology and criteria for diagnosing acute otitis media, and treatment of acute otitis media. Each topic has a 30-minute lesson plan with broad goals, specific and measurable learning objectives, interactive learning activities, and assessment measures. Teaching activities include a mix of knowledge, skill, and attitude activities. Assessment measures aim at the highest level of Miller's pyramid of assessment when possible. Also included is a scholarship map that relates how this learning activity can fit into the assessment of learners at the program level by linking learning objectives with pediatric developmental milestones. RESULTS: The interactive nature of these lessons has been very well received by learners, and pediatric residents have had a self-perceived increase in skills. DISCUSSION: This curriculum provides a standardized approach to teaching the middle ear exam structured by educational rubrics and allows for accurate assessment of learners.

Optimizing Your Mentoring Relationship: A Toolkit for Mentors and Mentees.

INTRODUCTION: Mentorship is a vital component of academic and professional development. Mentees report positive impacts from mentorship programs, yet institutions and societies may struggle to meet their mentees' needs due to factors such as mentor fatigue and lack of mentor training. To address this in our own professional society, the Association of Pediatric Program Directors, we developed a mentor toolkit in order to utilize a variety of mentoring models, provide faculty development for midlevel mentors, and offer guidance to mentees. METHODS: Most of these tools were designed to be administered in an interactive format such as a workshop or seminar with think-pair-share opportunities. The toolkit begins by providing a definition of mentoring and reinforcing the benefits and the characteristics of effective mentoring relationships. Next, we discuss the important role that mentees have in creating and maintaining effective mentoring relationships (i.e., mentee-driven mentoring). We then introduce a mentoring mosaic activity designed to help mentees examine their professional network and think about how they might expand it to fulfill the spectrum of their mentoring needs. Next, we present guidelines for the implementation of four mentoring models that can be used within one's institution: traditional dyadic mentoring, peer group mentoring, meet the professor mentoring, and speed mentoring. We then provide tools that can be used to help facilitate effective mentoring development. RESULTS: This toolkit has successfully served as a self-guided resource at national meetings for many years, garnering positive feedback from mentors and mentees alike. DISCUSSION: The principles and methods are easily generalizable and may be used to guide mentorship programs within institutional and professional societies, as well as to assist mentors and mentees in optimizing their individual mentoring relationships.

The Creation of Standard-Setting Videos to Support Faculty Observations of Learner Performance and Entrustment Decisions.

Entrustable professional activities (EPAs) provide a framework to standardize medical education outcomes and advance competency-based assessment. Direct observation of performance plays a central role in entrustment decisions; however, data obtained from these observations are often insufficient to draw valid high-stakes conclusions. One approach to enhancing the reliability and validity of these assessments is to create videos that establish performance standards to train faculty observers. Little is known about how to create videos that can serve as standards for assessment of EPAs.The authors report their experience developing videos that represent five levels of performance for an EPA for patient handoffs. The authors describe a process that begins with mapping the EPA to the critical competencies needed to make an entrustment decision. Each competency is then defined by five milestones (behavioral descriptors of performance at five advancing levels). Integration of the milestones at each level across competencies enabled the creation of clinical vignettes that were converted into video scripts and ultimately videos. Each video represented a performance standard from novice to expert. The process included multiple assessments by experts to guide iterative improvements, provide evidence of content validity, and ensure that the authors successfully translated behavioral descriptions and vignettes into videos that represented the intended performance level for a learner. The steps outlined are generalizable to other EPAs, serving as a guide for others to develop videos to train faculty. This process provides the level of content validity evidence necessary to support using videos as standards for high-stakes entrustment decisions.

Targeting Simulation-Based Assessment for the Pediatric Milestones: A Survey of Simulation Experts and Program Directors.

OBJECTIVE: To determine which of the 21 general pediatrics milestone subcompetencies are most difficult to assess using traditional methodologies and which are best suited to simulation-based assessment. METHODS: We surveyed 2 samples: pediatric simulation experts and pediatric program directors. Respondents were asked about current use of simulation for assessment and to select 5 of the 21 pediatric subcompetencies most difficult to assess using traditional methods and the 5 best suited to simulation-based assessment. Spearman rank correlation was used to determine a correlation between how the 2 samples ranked the subcompetencies. RESULTS: Forty-eight percent (29 of 60) simulation experts and 20% (115 of 571) program directors completed the survey. Few respondents reported using simulation for summative assessment. There are clear differences across the pediatric subcompetencies in perceived difficulty of assessment and suitability to simulation-based assessment. The 3 most difficult to assess subcompetencies were "recognize ambiguity," "demonstrate emotional insight," and "identify one's own strengths and deficiencies." The subcompetencies most suitable to assessment using simulation were "interprofessional teamwork," "clinical decision making," and "effective communication." Program directors and simulation experts had high agreement for both questions: difficult to assess (rho = 0.76, P < .001) and suitable to simulation-based assessment (rho = 0.94, P < .001). CONCLUSIONS: Several general pediatrics milestone subcompetencies were identified by pediatric simulation experts and pediatric program directors as difficult to assess using current methodologies and as amenable to simulation-based assessment. The pediatric simulation community should target development of simulation-based assessment tools to these areas.

Simulation-based medical education in pediatrics.

The use of simulation-based medical education (SBME) in pediatrics has grown rapidly over the past 2 decades and is expected to continue to grow. Similar to other instructional formats used in medical education, SBME is an instructional methodology that facilitates learning. Successful use of SBME in pediatrics requires attention to basic educational principles, including the incorporation of clear learning objectives. To facilitate learning during simulation the psychological safety of the participants must be ensured, and when done correctly, SBME is a powerful tool to enhance patient safety in pediatrics. Here we provide an overview of SBME in pediatrics and review key topics in the field. We first review the tools of the trade and examine various types of simulators used in pediatric SBME, including human patient simulators, task trainers, standardized patients, and virtual reality simulation. Then we explore several uses of simulation that have been shown to lead to effective learning, including curriculum integration, feedback and debriefing, deliberate practice, mastery learning, and range of difficulty and clinical variation. Examples of how these practices have been successfully used in pediatrics are provided. Finally, we discuss the future of pediatric SBME. As a community, pediatric simulation educators and researchers have been a leading force in the advancement of simulation in medicine. As the use of SBME in pediatrics expands, we hope this perspective will serve as a guide for those interested in improving the state of pediatric SBME.