RESUMEN
BACKGROUND: Healthcare curricula need summative assessments relevant to and representative of clinical situations to best select and train learners. Simulation provides multiple benefits with a growing literature base proving its utility for training in a formative context. Advancing to the next step, "the use of simulation for summative assessment" requires rigorous and evidence-based development because any summative assessment is high stakes for participants, trainers, and programs. The first step of this process is to identify the baseline from which we can start. METHODS: First, using a modified nominal group technique, a task force of 34 panelists defined topics to clarify the why, how, what, when, and who for using simulation-based summative assessment (SBSA). Second, each topic was explored by a group of panelists based on state-of-the-art literature reviews technique with a snowball method to identify further references. Our goal was to identify current knowledge and potential recommendations for future directions. Results were cross-checked among groups and reviewed by an independent expert committee. RESULTS: Seven topics were selected by the task force: "What can be assessed in simulation?", "Assessment tools for SBSA", "Consequences of undergoing the SBSA process", "Scenarios for SBSA", "Debriefing, video, and research for SBSA", "Trainers for SBSA", and "Implementation of SBSA in healthcare". Together, these seven explorations provide an overview of what is known and can be done with relative certainty, and what is unknown and probably needs further investigation. Based on this work, we highlighted the trustworthiness of different summative assessment-related conclusions, the remaining important problems and questions, and their consequences for participants and institutions of how SBSA is conducted. CONCLUSION: Our results identified among the seven topics one area with robust evidence in the literature ("What can be assessed in simulation?"), three areas with evidence that require guidance by expert opinion ("Assessment tools for SBSA", "Scenarios for SBSA", "Implementation of SBSA in healthcare"), and three areas with weak or emerging evidence ("Consequences of undergoing the SBSA process", "Debriefing for SBSA", "Trainers for SBSA"). Using SBSA holds much promise, with increasing demand for this application. Due to the important stakes involved, it must be rigorously conducted and supervised. Guidelines for good practice should be formalized to help with conduct and implementation. We believe this baseline can direct future investigation and the development of guidelines.
RESUMEN
OBJECTIVES: We aimed to evaluate the effectiveness of a multifaceted procedure in improving pneumococcal and influenza vaccinations 6 months after an emergency department (ED) visit among patients aged 65 years and older. METHODS: We conducted a cluster-randomized, controlled, parallel-group, open-label implementation trial in 18 EDs in France and Monaco. Participants were recruited from November 2015 to September 2016. EDs were randomly assigned with a 1:1 ratio to provide either a multifaceted procedure that combined structured information about pneumococcal and influenza vaccines and three text message reminders sent to patients every two weeks (intervention arm) or nonstructured information only (control arm). The outcomes were self-reported pneumococcal vaccination and influenza vaccination rates within 6 months of enrollment. RESULTS: A total of 9 EDs were randomized to the intervention arm (n = 780 patients) and 9 to the control arm (n = 695 patients). The median age for all enrolled patients was 74 years (25-75th percentiles, 69 to 82): 50.1% were male, 34.9% had at least one underlying condition, and 30.7% were at risk for invasive pneumococcal infection. In the intention-to-treat analysis, the multifaceted intervention did not alter the pneumococcal vaccination rate (6.4% versus 4.6%, absolute difference: 1.8; 95% CI: [-0.9 to 4.4]; p = 0.19), whereas it improved the influenza vaccination rate (52.1% versus 40.0%, absolute difference: 12.1; 95% CI: [2.4 to 21.8]; p = 0.01). At 12 months, mortality did not differ between the intervention (9.7%) and control (11.2%) arms (p = 0.35). CONCLUSIONS: A multifaceted intervention based on text message reminders provides an opportunity to increase anti-influenza vaccination among elderly patients visiting the ED. Efforts are warranted to provide better information on pneumococcal diseases and the benefits of pneumococcal vaccines, especially in the elderly.
