Twelve tips for developing simulation-based mastery learning clinical skills checklists.

Simulation-based mastery learning is a powerful educational paradigm that leads to high levels of performance through a combination of strict standards, deliberate practice, formative feedback, and rigorous assessment. Successful mastery learning curricula often require well-designed checklists that produce reliable data that contribute to valid decisions. The following twelve tips are intended to help educators create defensible and effective clinical skills checklists for use in mastery learning curricula. These tips focus on defining the scope of a checklist using established principles of curriculum development, crafting the checklist based on a literature review and expert input, revising and testing the checklist, and recruiting judges to set a minimum passing standard. While this article has a particular focus on mastery learning, with the exception of the tips related to standard setting, the general principles discussed apply to the development of any clinical skills checklist.

Where in the world: Mapping medical student learning using the Social and Structural Determinants of Health Curriculum Assessment Tool (SSDH CAT).

INTRODUCTION: Addressing the Social and Structural Determinants of Health (SSDH) is a primary strategy for attaining health equity. Teaching and learning about SSDH has increased across medical schools throughout the world; however, the published literature describing these efforts continues to be limited and many unknowns persist including what should be taught and by whom, what teaching methods and settings should be used, and how medical learners should be assessed. MATERIALS AND METHODS: Based on published studies, input from experts in the field, and elements from the framework developed by the National Academy of Medicine, we created a universal Social and Structural Determinants of Health Curriculum Assessment Tool (SSDH CAT) to assist medical educators to assess existing SSDH curricular content, ascertain critical gaps, and categorize educational methods, delivery, and assessment techniques and tools that could help inform curricular enhancements to advance the goal of training a health care workforce focused on taking action to achieve health equity. To test the usefulness of the tool, we applied the SSDH CAT to map SSDH-related curriculum at a US-based medical school. RESULTS: By applying the SSDH CAT to our undergraduate medical school curriculum, we recognized that our SSDH curriculum relied too heavily on lectures, emphasized knowledge without sufficient skill building, and lacked objective assessment measures. As a result of our curricular review, we added more skill-based activities such as using evidence-based tools for screening patients for social needs, and created and implemented a universal, longitudinal, experiential community health curriculum. DISCUSSION: We created a universal SSDH CAT and applied it to assess and improve our medical school's SSDH curriculum. The SSDH CAT provides a starting point for other medical schools to assess their SSDH content as a strategy to improve teaching and learning about health equity, and to inspire students to act on the SSDH.

Expert Consensus Guidelines for Assessing Students on the Social Determinants of Health.

PHENOMENON: Assessment and evaluation guidelines inform programmatic changes necessary for educational effectiveness. Presently, no widely accepted guidelines exist for educators to assess learners and evaluate programs regarding social determinants of health (SDOH) during physician and physician assistant (PA) education. We sought to garner expert consensus about effective SDOH learner assessment and program evaluation, so as to make recommendations for best practices related to SDOH education. APPROACH: We used a Delphi approach to conduct our study (September 2019 to December 2020). To administer our Delphi survey, we followed a three-step process: 1) literature review, 2) focus groups and semi-structured interviews, 3) question development and refinement. The final survey contained 72 items that addressed SDOH content areas, assessment methods, assessors, assessment integration, and program evaluation. Survey participants included 14 SDOH experts at US medical schools and PA programs. The survey was circulated for three rounds seeking consensus, and when respondents reached consensus on a particular question, that question was removed from subsequent rounds. FINDINGS: The geographically diverse sample of experts reached consensus on many aspects of SDOH assessment and evaluation. The experts selected three important areas to assess learners' knowledge, skills, and attitudes about SDOH. They identified assessment methods that were "essential", "useful, but not essential", and "not necessary." The essential assessment methods are performance rating scales for knowledge and attitudes and skill-based assessments. They favored faculty and patients as assessors, as well as learner self-assessment, over assessments conducted by other health professionals. Questions about separation versus incorporation of SDOH assessment with other educational assessment did not yield consensus opinion. The experts reached consensus on priority outcome measures to evaluate a school's SDOH program which included student attitudes toward SDOH, Competence-Based Assessment Scales, and the percentage of graduates involved in health equity initiatives. INSIGHTS: Based on the Delphi survey results, we make five recommendations that medical and PA educators can apply now when designing learner assessments and evaluating SDOH programming. These recommendations include what should be assessed, using what methods, who should do the assessments, and how they should be incorporated into the curriculum. This expert consensus should guide future development of an assessment and evaluation toolkit to optimize SDOH education and clinical practice.Supplemental data for this article is available online at https://doi.org/10.1080/10401334.2022.2045490 .

Assessment and Evaluation in Social Determinants of Health Education: a National Survey of US Medical Schools and Physician Assistant Programs.

BACKGROUND: Social determinants of health (SDOH) curricular content in medical schools and physician assistant programs are increasing. However, there is little understanding of current practice in SDOH learner assessment and program evaluation, or what the best practices are. OBJECTIVE: Our study aim was to describe the current landscape of assessment and evaluation at US medical schools and physician assistant programs as a first step in developing best practices in SDOH education. DESIGN: We conducted a national survey of SDOH educators from July to December 2020. The 55-item online survey covered learner assessment methods, program evaluation, faculty training, and barriers to effective assessment and evaluation. Results were analyzed using descriptive statistics. PARTICIPANTS: One hundred six SDOH educators representing 26% of medical schools and 23% of PA programs in the USA completed the survey. KEY RESULTS: Most programs reported using a variety of SDOH learner assessment methods. Faculty and self were the most common assessors of learners' SDOH knowledge, attitudes, and skills. Common barriers to effective learner assessment were lack of agreement on "SDOH competency" and lack of faculty training in assessment. Programs reported using evaluation results to refine curricular content, identify the need for new content, and improve assessment strategies. CONCLUSIONS: We identified a heterogeneity of SDOH assessment and evaluation practices among programs, as well as gaps and barriers in their educational practices. Specific guidance from accrediting bodies and professional organizations and agreement on SDOH competency as well as providing faculty with time, resources, and training will improve assessment and evaluation practice and ensure SDOH education is effective for students, patients, and communities.

Varying Estimates of Sepsis among Adults Presenting to US Emergency Departments: Estimates from a National Dataset from 2002-2018.

BACKGROUND: A variety of approaches to defining sepsis using administrative datasets have been previously reported. We aimed to compare estimates, demographics, treatment factors, outcomes and longitudinal trends of patients identified with sepsis in United States emergency departments (EDs) using differing sets of sepsis criteria. METHODS: We performed a cross-sectional study using the National Healthcare Ambulatory Medical Care Survey, a complex survey of nonfederal US ED encounters between 2002 to 2018. We obtained survey-weighted population-adjusted encounters of sepsis using the following criteria: explicit sepsis, severe sepsis, and quick Sequential Organ Failure Assessment (qSOFA) score combined with the presence of infection. RESULTS: Age-adjusted for US adults, 18.6, 16.1 and 8.9 encounters per 10 000 population were identified when using the explicit, severe sepsis and qSOFA definitions, respectively. A higher proportion of the explicit cohort was hospitalized and had blood cultures performed, compared to cohorts ascertained using severe sepsis and qSOFA criteria, though confidence intervals overlapped. Antibiotic use was highest in encounters meeting qSOFA criteria. When inspecting unweighted encounters meeting each set of criteria, there was minimal overlap, with only 3% meeting all three. Encounters meeting the explicit and severe sepsis criteria were increasing over time. CONCLUSION: The explicit, severe sepsis and qSOFA criteria generated similar annual rates of presentation when applied to US ED encounters, with some evidence of the explicit sepsis cohort being higher acuity. There was minimal overlap of cases and instability in estimates when assessed longitudinally. Our findings inform research efforts to accurately identify sepsis among ED encounters using administrative data.

From Concept to Publication: Effectiveness of the International Network for Simulation-Based Pediatric Innovation, Research, and Education Project Development Process at Generating Simulation Scholarship.

BACKGROUND: As simulation matures, it is critical to develop pathways for researchers. A recent analysis, however, demonstrates a low conversion rate between abstract and peer-reviewed journal publication in our field. The International Network for Simulation-based Pediatric Innovation, Research, and Education has used the ALERT Presentation process for the past decade as a means of accelerating research. In this study, we analyze the scholarly products attributable to ALERT Presentations. METHODS: Surveys were distributed to all International Network for Simulation-based Pediatric Innovation, Research, and Education Advanced Look Exploratory Research Template (ALERT) Presentation first authors from January 2011 through January 2020. Presenters were asked to provide information on abstracts, grants, journal publications, and book chapters related to their ALERT Presentation, as well as basic demographic information. A structured literature search was conducted for those ALERT Presentations whose authors did not return a survey. The resulting database was descriptively analyzed, and statistical correlations between demographic variables and scholarship were examined. RESULTS: One hundred sixty-five new ALERT presentations were presented over 10 years. We identified 361 associated scholarly works (170 conference abstracts, 125 peer-reviewed journal publications, 65 grants, and 1 book chapter). Sixty-one percent (101 of 165) of ALERT Presentations produced at least 1 item of scholarship, and 59% (34 of 58) of ALERT Presentations that resulted in at least 1 abstract also led to at least 1 peer-reviewed journal article. Presenter gender was associated with likelihood of journal publication. CONCLUSIONS: The ALERT Presentation process is an effective approach for facilitating the development of projects that result in disseminated scholarship. Wider adoption may benefit other simulation and education research networks.

Impact of Personal Protective Equipment on the Performance of Emergency Pediatric Procedures by Prehospital Providers.

BACKGROUND: Personal protective equipment (PPE) is worn by prehospital providers (PHPs) for protection from hazardous exposures. Evidence regarding the ability of PHPs to perform resuscitation procedures has been described in adult but not pediatric models. This study examined the effects of PPE on the ability of PHPs to perform resuscitation procedures on pediatric patients. METHODS: This prospective study was conducted at a US simulation center. Paramedics wore normal attire at the baseline session and donned full Level B PPE for the second session. During each session, they performed timed sets of psychomotor tasks simulating clinical care of a critically ill pediatric patient. The difference in time to completion between baseline and PPE sessions per task was examined using Wilcoxon signed-rank tests. RESULTS: A total of 50 paramedics completed both sessions. Median times for task completion at the PPE sessions increased significantly from baseline for several procedures: tracheal intubation (+4.5 s; P = 0.01), automated external defibrillator (AED) placement (+9.5 s; P = 0.01), intraosseous line insertion (+7 s; P < 0.0001), tourniquet (+8.5 s; P < 0.0001), intramuscular injection (+21-23 s, P < 0.0001), and pulse oximetry (+4 s; P < 0.0001). There was no significant increase in completion time for bag-mask ventilation or autoinjector use. CONCLUSIONS: PPE did not have a significant impact on PHPs performing critical tasks while caring for a pediatric patient with a highly infectious or chemical exposure. This information may guide PHPs faced with the situation of resuscitating children while wearing Level B PPE.

Outcomes and Factors Associated With Prehospital Treatment of Pediatric Anaphylaxis.

OBJECTIVE: Anaphylaxis is a potentially life-threatening reaction requiring prompt treatment with intramuscular epinephrine (EPI). We sought to describe presenting features of pediatric anaphylaxis and compare patient characteristics and outcomes of children treated with prehospital EPI with those untreated. METHODS: We abstracted data from emergency department (ED) records for children meeting the National Institute of Allergy & Infectious Disease criteria for anaphylaxis (2015-2017) in one tertiary care children's hospital. We analyzed associations between patient characteristics and outcomes and receipt of prehospital EPI using descriptive statistics and multivariate logistic regression. RESULTS: Of 414 children presenting with anaphylaxis, 39.4% received IM EPI and 62.1% received antihistamines before hospital arrival. Children with Medicaid received pre-emergency department EPI less frequently than did children with private insurance (24.5% vs 43.8%, P = 0.001). Factors positively associated with prehospital EPI administration were history of food allergy (odds ratio [OR], 4.4 [95% confidence interval {CI}, 2.4-8.2]) or arrival by emergency medical services (OR, 8.0 [95% CI, 4.2-15.0]). Medicaid insurance was associated with decreased odds of prehospital EPI (OR, 0.33 [95% CI, 0.16-0.66]) and prehospital H1-antihistamine use (OR, 0.30 [95% CI, 0.17-0.56]). Prehospital EPI treatment was also associated with decreased rates of observation (37% vs 63%), inpatient admission (38% vs 62%), and intensive care unit admission (20% vs 80%) compared with no pretreatment (P = 0.03). CONCLUSIONS: Prehospital treatment with EPI remains low, and barriers to optimal treatment are more pronounced in children with public insurance. Prehospital treatment with EPI was associated with decreased morbidity including hospitalization and intensive care unit admission.

Simulation-based Mastery Learning Improves Emergency Medicine Residents' Ability to Perform Temporary Transvenous Cardiac Pacing.

INTRODUCTION: Temporary transvenous cardiac pacing (TVP) is a critical intervention that emergency physicians perform infrequently in clinical practice. Prior simulation studies revealed that emergency medicine (EM) residents and board-certified emergency physicians perform TVP poorly during checklist-based assessments. Our objective in this report was to describe the design and implementation of a simulation-based mastery learning (SBML) curriculum and evaluate its impact on EM residents' ability to perform TVP. METHODS: An expert panel of emergency physicians and cardiologists set a minimum passing standard (MPS) for a previously developed 30-item TVP checklist using the Mastery Angoff approach. Emergency medicine residents were assessed using this checklist and a high-fidelity TVP task trainer. Residents who did not meet the MPS during baseline testing viewed a procedure video and completed a 30-minute individual deliberate practice session before retesting. Residents who did not meet the MPS during initial post-testing completed additional deliberate practice and assessment until meeting or exceeding the MPS. RESULTS: The expert panel set an MPS of correctly performing 28 (93.3%) checklist items. Fifty-seven EM residents participated. Mean checklist scores improved from 13.4 (95% CI 11.8-15.0) during baseline testing to 27.5 (95% CI 26.9-28.1) during initial post-testing (P < 0.01). No residents met the MPS at baseline testing. The 21 (36.8%) residents who did not meet the MPS during initial post-testing all met or exceeded the MPS after completing one additional 30-minute deliberate practice session. CONCLUSION: Emergency medicine residents demonstrated significantly improved TVP performance with reduced variability in checklist scores after completing a simulation-based mastery learning curriculum.

Development of a Rigorously Designed Procedural Checklist for Assessment of Emergency Medicine Resident Performance of Temporary Transvenous Cardiac Pacing.

OBJECTIVES: Temporary transvenous cardiac pacing (TVP) is a potentially lifesaving intervention included in the list of essential core procedures for emergency medicine (EM) training; however, opportunities to perform TVP during residency cannot be guaranteed. EM graduates report feeling subjectively underprepared for this procedure, but objective performance data are lacking. Checklist-based simulated assessment is an increasing focus of competency-based medical education, particularly for invasive procedures like TVP. The objectives of this paper were as follows: first, to enlist a multidisciplinary team of experts to create an assessment tool for TVP using best practices in checklist development; second, to determine the reliability of checklist scoring; and third, to assess EM residents' baseline ability to perform TVP using a dedicated task trainer. METHODS: This study was conducted at a single 4-year EM residency. A panel of emergency physicians and cardiologists designed a TVP checklist using a modified Delphi approach. After consensus was achieved on a final checklist, EM residents were assessed using a dedicated TVP task trainer. Inter-rater reliability was determined using Cohen's kappa coefficient. Resident performance was determined by number of correctly performed checklist items. RESULTS: The expert panel achieved consensus on a 30-item checklist after three rounds of revisions. The Cohen's kappa coefficient for the overall checklist score was 0.87, with individual checklist items ranging from 0.63 to 1.00. In total, 58 residents were assessed with a mean score of 13.5 of 30 checklist items. Scores increased with each year of training. CONCLUSIONS: This study details the rigorous development of a TVP checklist designed by a multidisciplinary team of experts. Checklist scores demonstrated strong inter-rater reliability. The overall poor performance of this cohort suggests the current approach to TVP training does not provide sufficient preparation for EM residents. Competency-based techniques, such as simulation-based mastery learning, should be explored.

Impact of Personal Protective Equipment on the Performance of Emergency Pediatric Tasks.

OBJECTIVES: Personal protective equipment (PPE) is worn by health care providers (HCPs) to protect against hazardous exposures. Studies of HCPs performing critical resuscitation tasks in PPE have yielded mixed results and have not evaluated performance in care of children. We evaluated the impacts of PPE on timeliness or success of emergency procedures performed by pediatric HCPs. METHODS: This prospective study was conducted at 2 tertiary children's hospitals. For session 1, HCPs (medical doctors and registered nurses) wore normal attire; for session 2, they wore full-shroud PPE garb with 2 glove types: Ebola level or chemical. During each session, they performed clinical tasks on a patient simulator: intubation, bag-valve mask ventilation, venous catheter (IV) placement, push-pull fluid bolus, and defibrillation. Differences in completion time per task were compared. RESULTS: There were no significant differences in medical doctor completion time across sessions. For registered nurses, there was a significant difference between baseline and PPE sessions for both defibrillation and IV placement tasks. Registered nurses were faster to defibrillate in Ebola PPE and slower when wearing chemical PPE (median difference, -3.5 vs 2 seconds, respectively; P < 0.01). Registered nurse IV placement took longer in Ebola and chemical PPE (5.5 vs 42 seconds, respectively; P < 0.01). After the PPE session, participants were significantly less likely to indicate that full-body PPE interfered with procedures, was claustrophobic, or slowed them down. CONCLUSIONS: Personal protective equipment did not affect procedure timeliness or success on a simulated child, with the exception of IV placement. Further study is needed to investigate PPE's impact on procedures performed in a clinical care context.

Application of the Improving Pediatric Sepsis Outcomes Definition for Pediatric Sepsis to Nationally Representative Emergency Department Data.

To compare encounter estimates and demographics of pediatric patients (<18 years) meeting modified Improving Pediatric Sepsis Outcomes (IPSO) criteria for sepsis to cohorts obtained using other criteria for pediatric sepsis from administrative datasets. METHODS: We analyzed data from the National Hospital Ambulatory Medical Care Survey for 2003-2018. We report encounter estimates, demographics, and treatments among pediatric sepsis events using 3 criteria: modified IPSO criteria for sepsis, explicit criteria using diagnostic codes, and implicit severe sepsis criteria requiring the presence of infection and organ dysfunction. RESULTS: The modified IPSO, explicit, and severe sepsis criteria estimated the yearly encounter rates as 116,200, 27,900, and 56,000 respectively. The modified IPSO sepsis criteria accounted for 0.4% of emergency department encounters, with a high proportion of patients who received antibiotics (99.2%, 95% CI 97.8%-100.0%), intravenous fluids (100.0%, 95% CI 99.9%-100.0%), and blood cultures (98.7%, 95% CI 96.9%-100.0%). The explicit cohort had lower proportions with blood cultures (60.6%, 95% CI 40.4%-80.7%) and antibiotic use (77.0%, 95% CI 63.1%-90.8%), but a high proportion admitted (84.0% 95% CI 73.4%-95.7%). The severe sepsis definition had low proportions with blood cultures (12.7%, 95% CI 6.3%-19.1%) and admission (21.1%, 95% CI 14.5%-27.8%). CONCLUSIONS: Pediatric sepsis estimates differed based on the criteria used for cohort ascertainment. The modified IPSO sepsis criteria group had higher acuity than the severe sepsis cohort but lower acuity than the cohort identified using the explicit sepsis criteria.

Association of triage hypothermia with in-hospital mortality among patients in the emergency department with suspected sepsis.

PURPOSE: To identify if triage hypothermia (<36.0 °C) among emergency department (ED) encounters with sepsis are independently associated with mortality. METHODS: We analyzed data from a multi-stage probability sample survey of visits to United States EDs between 2007 and 2015, using two inclusion approaches: an explicit definition based on diagnosis codes for sepsis and a severe sepsis definition, combining evidence of infection with organ dysfunction. We used multivariable regression to determine an association between hypothermia and in-hospital mortality. RESULTS: Of 1.2 billion ED encounters (95% confidence interval [CI] 1.0-1.3 billion), 3.1 million (95% CI 2.7-3.5 million) met the explicit sepsis definition; 7.4% (95% CI 75.2-9.7%) had triage hypothermia. The adjusted odds ratio (aOR) for hypothermia for in-hospital mortality was 6.82 (95% CI 3.08-15.22). The severe sepsis definition identified 3.5 million (95% 3.1-4.0 million) encounters; 30.3% (95% CI 25.0-34.6%) had triage hypothermia. The aOR for hypothermia with mortality was 4.08 (95% CI 2.09-7.95). Depending on sepsis definition, 78.1-84.4% had other systemic inflammatory response syndrome vital sign abnormalities. CONCLUSION: Up to one in three patients with sepsis have triage hypothermia, which is independently associated with mortality. 10-20% of patients with hypothermic sepsis do not have other vital sign abnormalities.

Impact of Personal Protective Equipment on Pediatric Cardiopulmonary Resuscitation Performance: A Controlled Trial.

OBJECTIVES: This study aimed to determine whether personal protective equipment (PPE) results in deterioration in chest compression (CC) quality and greater fatigue for administering health care providers (HCPs). METHODS: In this multicenter study, HCPs completed 2 sessions. In session 1 (baseline), HCPs wore normal attire; in session 2, HCPs donned full PPE. During each session, they performed 5 minutes of uninterrupted CCs on a child manikin. Chest compression rate, depth, and release velocity were reported in ten 30-second epochs. Change in CC parameters and self-reported fatigue were measured between the start and 2- and 5-minute epochs. RESULTS: We enrolled 108 HCPs (prehospital and in-hospital providers). The median CC rate did not change significantly between epochs 1 and 10 during baseline sessions. Median CC depth and release velocity decreased for 5 minutes with PPE. There were no significant differences in CC parameters between baseline and PPE sessions in any provider group. Median fatigue scores during baseline sessions were 2 (at start), 4 (at 2 minutes), and 6 (at 5 minutes). There was a significantly higher median fatigue score between 0 and 5 minutes in both study sessions and in all groups. Fatigue scores were significantly higher for providers wearing PPE compared with baseline specifically among prehospital providers. CONCLUSIONS: During a clinically appropriate 2-minute period, neither CC quality nor self-reported fatigue worsened to a significant degree in providers wearing PPE. Our data suggest that Pediatric Basic Life Support recommendations for CC providers to switch every 2 minutes need not be altered with PPE use.

Expert Consensus on Inclusion of the Social Determinants of Health in Undergraduate Medical Education Curricula.

PURPOSE: Accreditation bodies have mandated teaching social determinants of health (SDH) to medical students, but there has been limited guidance for educators on what or how to teach, and how to evaluate students' competence. To fill this gap, this study aimed to develop an SDH curricular consensus guide for teaching SDH to medical students. METHOD: In 2017, the authors used a modified Delphi technique to survey an expert panel of educators, researchers, students, and community advocates about knowledge, skills, and attitudes (KSA) and logistics regarding SDH teaching and assessment. They identified the panel and ranked a comprehensive list of topics based on a scoping review of SDH education studies and discussions with key informants. A total of 57 experts were invited. RESULTS: Twenty-two and 12 panelists participated in Delphi rounds 1 and 2, respectively. The highest-ranked items regarding KSA were "Appreciation that the SDH are some of the root causes of health outcomes and health inequities" and "How to work effectively with community health workers." The panel achieved consensus that SDH should constitute 29% of the total curriculum and be taught continuously throughout the curriculum. Multiple-choice tests were ranked lowest as an assessment method, and patient feedback was ranked highest. Panelists noted that SDH content must be a part of standardized exams to be prioritized by faculty and students. CONCLUSIONS: An expert panel endorsed essential curricular content, teaching methods, and evaluation approaches that can be used to help guide medical educators regarding SDH curriculum development.

Teaching the Social Determinants of Health in Undergraduate Medical Education: a Scoping Review.

BACKGROUND: To provide optimal care, medical students should understand that the social determinants of health (SDH) impact their patients' well-being. Those charged with teaching SDH to future physicians, however, face a paucity of curricular guidance. OBJECTIVE: This review's objective is to map key characteristics from publications about teaching SDH to students in undergraduate medical education (UME). METHODS: In 2016, the authors searched PubMed, Embase, Web of Science, the Cochrane and ERIC databases, bibliographies, and MedEdPORTAL for articles published between January 2010 and November 2016. Four reviewers screened articles for eligibility then extracted and analyzed data descriptively. Scoping review methodology was used to map key concepts and curricular logistics as well as educator and student characteristics. RESULTS: The authors screened 3571 unique articles of which 22 were included in the final review. Many articles focused on community engagement (15). Experiential learning was a common instructional strategy (17) and typically took the form of community or clinic-based learning. Nearly half (10) of the manuscripts described school-wide curricula, of which only three spanned a full year. The majority of assessment was self-reported (20) and often related to affective change. Few studies objectively assessed learner outcomes (2). CONCLUSIONS: The abundance of initial articles screened highlights the growing interest in SDH in medical education. The small number of selected articles with sufficient detail for abstraction demonstrates limited SDH curricular dissemination. A lack of accepted tools or practices that limit development of robust learner or program evaluation was noted. Future research should focus on identifying and evaluating effective instructional and assessment methodologies to address this gap, exploring additional innovative teaching frameworks, and examining the specific contexts and characteristics of marginalized and underserved populations and their coverage in medical education.

Using the METRICS model for defining routes to scholarship in healthcare simulation.

INTRODUCTION: In this paper, we explored the utility and value of the METRICS model for modeling scholarship in healthcare simulation by: (1) describing the distribution of articles in four healthcare simulation journals across the seven areas of METRICS scholarship; and (2) appraising patterns of scholarship expressed in three programs of simulation scholarship and reflecting on how these patterns potentially influence the pursuit of future scholarly activities. METHODS: Two raters reviewed abstracts of papers published between January 2015 and August 2017 in four healthcare simulation journals and coded them using METRICS. Descriptive statistics were calculated for scholarship type and distribution across journals. Twenty-eight articles from three scholars were reviewed, with patterns of scholarship within articles mapped to METRICS. Descriptive synthesis was constructed through discussion between two reviewers. RESULTS: A total of 432 articles from four journals were reviewed. The three most commonly published areas of scholarship were: 32.2% (139/432) evaluation, 18.8% (81/432) innovation, and 15.3% (66/432) conceptual. The METRICS model was able to represent different kinds of scholarship expressed in all of the papers reviewed and across programs of research. Reflecting on patterns of scholarship within their scholarly programs was helpful for research in planning future directions. CONCLUSIONS: The METRICS model for scholarship can describe a wide range of patterns of simulation scholarship within individual articles, programs of research, or across journals.

Use of Simulation to Test Systems and Prepare Staff for a New Hospital Transition.

OBJECTIVE: To describe the development and key outcomes arising from the use of simulation as a method to test systems and prepare staff for a transition to a new hospital. METHODS: We describe a simulation program developed by key parties with the goal of reducing latent safety threats present at the opening of a new hospital and to train staff in new workflows. Issues identified were collected and reported to leadership. Outcomes included the number of learners reached, issues identified (grouped by theme), and results of a postmove survey of hospital-based staff. RESULTS: Approximately 258 hours of simulation were conducted, impacting 514 participants. We conducted 64 hours of system testing and 196 hours of training during the main orientation process. Approximately 641 unique issues were identified (175 equipment, 136 code alarm, 174 barriers to care, and 156 incorrect signage). In a hospital-wide survey, 38% reported simulation as part of their training (39% of nurses and 23% of physicians). 43% of survey respondents reported multidisciplinary simulations; 55% of simulation attendees felt that the simulation was helpful and eased their transition to the new hospital. CONCLUSIONS: Systems testing and education using simulation can play a meaningful role in new facility training. Key lessons included early planning, allocation of resources to the effort, flexibility to adapt to changes, and planned integration with other training activities. A formal a priori plan to address issues identified during the process is necessary.

The Development and Validation of a Concise Instrument for Formative Assessment of Team Leader Performance During Simulated Pediatric Resuscitations.

AIM: The aim of this study was to assess the validity of a formative feedback instrument for leaders of simulated resuscitations. METHODS: This is a prospective validation study with a fully crossed (person × scenario × rater) study design. The Concise Assessment of Leader Management (CALM) instrument was designed by pediatric emergency medicine and graduate medical education experts to be used off the shelf to evaluate and provide formative feedback to resuscitation leaders. Four experts reviewed 16 videos of in situ simulated pediatric resuscitations and scored resuscitation leader performance using the CALM instrument. The videos consisted of 4 pediatric emergency department resuscitation teams each performing in 4 pediatric resuscitation scenarios (cardiac arrest, respiratory arrest, seizure, and sepsis). We report on content and internal structure (reliability) validity of the CALM instrument. RESULTS: Content validity was supported by the instrument development process that involved professional experience, expert consensus, focused literature review, and pilot testing. Internal structure validity (reliability) was supported by the generalizability analysis. The main component that contributed to score variability was the person (33%), meaning that individual leaders performed differently. The rater component had almost zero (0%) contribution to variance, which implies that raters were in agreement and argues for high interrater reliability. CONCLUSIONS: These results provide initial evidence to support the validity of the CALM instrument as a reliable assessment instrument that can facilitate formative feedback to leaders of pediatric simulated resuscitations.