Repeatability, reproducibility, and diagnostic accuracy of a commercial large language model (ChatGPT) to perform emergency department triage using the Canadian triage and acuity scale.

PURPOSE: The release of the ChatGPT prototype to the public in November 2022 drastically reduced the barrier to using artificial intelligence by allowing easy access to a large language model with only a simple web interface. One situation where ChatGPT could be useful is in triaging patients arriving to the emergency department. This study aimed to address the research problem: "can emergency physicians use ChatGPT to accurately triage patients using the Canadian Triage and Acuity Scale (CTAS)?". METHODS: Six unique prompts were developed independently by five emergency physicians. An automated script was used to query ChatGPT with each of the 6 prompts combined with 61 validated and previously published patient vignettes. Thirty repetitions of each combination were performed for a total of 10,980 simulated triages. RESULTS: In 99.6% of 10,980 queries, a CTAS score was returned. However, there was considerable variations in results. Repeatability (use of the same prompt repeatedly) was responsible for 21.0% of overall variation. Reproducibility (use of different prompts) was responsible for 4.0% of overall variation. Overall accuracy of ChatGPT to triage simulated patients was 47.5% with a 13.7% under-triage rate and a 38.7% over-triage rate. More extensively detailed text given as a prompt was associated with greater reproducibility, but minimal increase in accuracy. CONCLUSIONS: This study suggests that the current ChatGPT large language model is not sufficient for emergency physicians to triage simulated patients using the Canadian Triage and Acuity Scale due to poor repeatability and accuracy. Medical practitioners should be aware that while ChatGPT can be a valuable tool, it may lack consistency and may frequently provide false information.

ABSTRAIT: OBJECTIF: La sortie du prototype ChatGPT au public en novembre 2022 a considérablement réduit l'obstacle à l'utilisation de l'intelligence artificielle en permettant un accès facile à un grand modèle de langage avec une interface web simple. Une situation où ChatGPT pourrait être utile est de trier les patients qui arrivent au service d'urgence. Cette étude visait à résoudre le problème de la recherche : «Les médecins d'urgence peuvent-ils utiliser ChatGPT pour trier avec précision les patients à l'aide de l'Échelle canadienne de triage et d'acuité (ECTC) ?¼. MéTHODES: Six invites uniques ont été élaborées indépendamment par cinq urgentologues. Un script automatisé a été utilisé pour interroger ChatGPT avec chacune des six invites combinées à 61 vignettes de patients validées et précédemment publiées. Trente répétitions de chaque combinaison ont été réalisées pour un total de 10980 triages simulés. RéSULTATS: Dans 99.6 % des 10980 requêtes, un score CTAS a été obtenu. Cependant, il y a eu des variations considérables dans les résultats. La répétabilité (utilisation répétée de la même invite) était responsable de 21.0 % de la variation globale. La reproductibilité (utilisation de différentes invites) était responsable de 4.0 % de la variation globale. La précision globale de ChatGPT pour le triage des patients simulés était de 47.5 %, avec un taux de sous-triage de 13.7 % et un taux de triage supérieur de 38.7 %. Un texte plus détaillé donné à titre d'invite était associé à une plus grande reproductibilité, mais à une augmentation minimale de la précision. CONCLUSIONS: Cette étude suggère que le modèle actuel de ChatGPT en langage large n'est pas suffisant pour permettre aux médecins d'urgence de trier des patients simulés à l'aide de l'échelle canadienne de triage et d'acuité en raison de la faible répétabilité et de la faible précision. Les médecins doivent être conscients que, bien que ChatGPT puisse être un outil précieux, il peut manquer de cohérence et fournir fréquemment de fausses informations.

The Impact of Exposure to Previous Disasters on Hospital Disaster Surge Capacity Preparedness in Finland: Hospital disaster surge capacity preparedness.

OBJECTIVE: As disasters are rare and high-impact events, it is important that the learnings from disasters are maximized. The aim of this study was to explore the effect of exposure to a past disaster or mass casualty incident (MCI) on local hospital surge capacity planning. METHODS: The current hospital preparedness plans of hospitals receiving surgical emergency patients in Finland were collected (n = 28) and analyzed using the World Health Organization (WHO) hospital emergency checklist tool. The surge capacity score was compared between the hospitals that had been exposed to a disaster or MCI with those who had not. RESULTS: The overall median score of all key components on the WHO checklist was 76% (range 24%). The median surge capacity score was 65% (range 39%). There was no statistical difference between the surge capacity score of the hospitals with history of a disaster or MCI compared to those without (65% for both, P = 0.735). CONCLUSION: Exposure to a past disaster or MCI did not appear to be associated with an increased local hospital disaster surge capacity score. The study suggests that disaster planning should include structured post-action processes for enabling meaningful improvement after an experienced disaster or MCI.

Editorial Comments: The 2023 Model Core Content of Disaster Medicine.

The recently published Model Core Content of Disaster Medicine introduces proposed curriculum elements for specialized education and training in Disaster Medicine. This editorial comments on the publishing decision for the manuscript.

Sudden-Onset Disaster Mass-Casualty Incident Response: A Modified Delphi Study on Triage, Prehospital Life Support, and Processes.

The application and provision of prehospital care in disasters and mass-casualty incident response in Europe is currently being explored for opportunities to improve practice. The objective of this translational science study was to align common principles of approach and action and to identify how technology can assist and enhance response. To achieve this objective, the application of a modified Delphi methodology study based on statements derived from key findings of a scoping review was undertaken. This resulted in 18 triage, eight life support and damage control interventions, and 23 process consensus statements. These findings will be utilized in the development of evidence-based prehospital mass-casualty incident response tools and guidelines.

Impact of Point-of-Care Ultrasound on Secondary Triage: A Pilot Study.

OBJECTIVES: In mass casualty scenarios, patients with apparent hemodynamic and respiratory stability might have occult life-threatening injuries. These patients could benefit from more accurate triage methods. This study assessed the impact of point-of-care ultrasound on the accuracy of secondary triage conducted at an advanced medical post to enhance the detection of patients who, despite their apparent clinically stable condition, could benefit from earlier evacuation to definitive care or immediate life-saving treatment. METHODS: A mass casualty simulated event consisting of a bomb blast in a remote area was conducted with 10 simulated casualties classified as YELLOW at the primary triage scene; patients were evaluated by 4 physicians at an advanced medical post. Three patients had, respectively, hemoperitoneum, pneumothorax, and hemothorax. Only 2 physicians had sonographic information. RESULTS: All 4 physicians were able to suspect hemoperitoneum as a possible critical condition to be managed first, but only physicians with additional sonographic information accurately detected pneumothorax and hemothorax, thus deciding to immediately evacuate or treat.

A Qualitative Assessment of Studies Evaluating the Classification Accuracy of Personnel Using START in Disaster Triage: A Scoping Review.

Background: Mass casualty incidents (MCIs) can occur as a consequence of a wide variety of events and often require overwhelming prehospital and emergency support and coordinated emergency response. A variety of disaster triage systems have been developed to assist health care providers in making difficult choices with regards to prioritization of victim treatment. The simple triage and rapid treatment (START) triage system is one of the most widely used triage algorithms; however, the research literature addressing real-world or simulation studies documenting the classification accuracy of personnel using START is lacking. Aims and Objectives: To explore the existing literature related to the current state of knowledge about studies assessing the classification accuracy of the START triage system. Design: Scoping review based on Arksey and O'Malley's methodological framework and narrative synthesis based on methods described by Popay and colleagues were performed. Results: The literature search identified 1,820 citations, of which 32 studies met the inclusion criteria. Thirty were peer-reviewed articles and 28 published in the last 10 years (i.e., 2010 and onward). Primary research studies originated in 13 countries and included 3,706 participants conducting triaging assessments involving 2,950 victims. Included studies consisted of five randomized controlled trials, 17 non-randomized controlled studies, eight descriptive studies, and two mixed-method studies. Simulation techniques, mode of delivery, contextual features, and participants' required skills varied among studies. Overall, there was no consistent reporting of outcomes across studies and results were heterogeneous. Data were extracted from the included studies and categorized into two themes: (1) typology of simulations and (2) START system in MCIs simulations. Each theme contains sub-themes regarding the development of simulation employing START as a system for improving individuals' preparedness. These include types of simulation training, settings, and technologies. Other sub-themes include outcome measures and reference standards. Conclusion: This review demonstrates a variety of factors impacting the development and implementation of simulation to assess characteristics of the START system. To further improve simulation-based assessment of triage systems, we recommend the use of reporting guidelines specifically designed for health care simulation research. In particular, reporting of reference standards and test characteristics need to improve in future studies.

METASTART: A Systematic Review and Meta-Analysis of the Diagnostic Accuracy of the Simple Triage and Rapid Treatment (START) Algorithm for Disaster Triage.

INTRODUCTION: The goal of disaster triage at both the prehospital and in-hospital level is to maximize resources and optimize patient outcomes. Of the disaster-specific triage methods developed to guide health care providers, the Simple Triage and Rapid Treatment (START) algorithm has become the most popular system world-wide. Despite its appeal and global application, the accuracy and effectiveness of the START protocol is not well-known. OBJECTIVES: The purpose of this meta-analysis was two-fold: (1) to estimate overall accuracy, under-triage, and over-triage of the START method when used by providers across a variety of backgrounds; and (2) to obtain specific accuracy for each of the four START categories: red, yellow, green, and black. METHODS: A systematic review and meta-analysis was conducted that searched Medline (OVID), Embase (OVID), Global Health (OVID), CINAHL (EBSCO), Compendex (Engineering Village), SCOPUS, ProQuest Dissertations and Theses Global, Cochrane Library, and PROSPERO. The results were expanded by hand searching of journals, reference lists, and the grey literature. The search was executed in March 2020. The review considered the participants, interventions, context, and outcome (PICO) framework and followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Accuracy outcomes are presented as means with 95% confidence intervals (CI) as calculated using the binomial method. Pooled meta-analyses of accuracy outcomes using fixed and random effects models were calculated and the heterogeneity was assessed using the Q statistic. RESULTS: Thirty-two studies were included in the review, most of which utilized a non-randomized study design (84%). Proportion of victims correctly triaged using START ranged from 0.27 to 0.99 with an overall triage accuracy of 0.73 (95% CI, 0.67 to 0.78). Proportion of over-triage was 0.14 (95% CI, 0.11 to 0.17) while the proportion of under-triage was 0.10 (95% CI, 0.072 to 0.14). There was significant heterogeneity of the studies for all outcomes (P < .0001). CONCLUSION: This meta-analysis suggests that START is not accurate enough to serve as a reliable disaster triage tool. Although the accuracy of START may be similar to other models of disaster triage, development of a more accurate triage method should be urgently pursued.

Tourniquet application by schoolchildren-a randomized crossover study of three commercially available models.

BACKGROUND: Life-threatening hemorrhage is a major cause of preventable mortality in trauma. Studies have demonstrated the effectiveness and safety of commercial tourniquets when used by adult civilians. However, there are no data about tourniquet application by children.This study's goal is to determine which of three commercially available tourniquets is most effective when used by children. METHODS: A randomized crossover study was conducted in four elementary schools in Montreal to compare three commercially available tourniquets. The study population is primary school children aged 10 to 12 years (5th-6th grade). A total of 181 students were invited to participate; 96 obtained parental approval and were recruited. Participants underwent a short 7-minute video training on the use of three commercial tourniquets and were subsequently given a 2-minute practice period. Students were evaluated on their ability to successfully apply the tourniquet and the time to complete application. After applying all three tourniquets, the students selected their favorite model. The primary outcome is the proportion of successful applications per tourniquet model. Secondary outcomes include time to successful application for each tourniquet model and tourniquet model preference. RESULTS: The mechanical advantage tourniquet (MAT) outperformed the combat application tourniquet (CAT) and the stretch wrap and tuck tourniquet (SWATT) in terms of success rate (MAT, 67%; CAT, 44%; SWATT, 24%; p < 0.0001), time to application (MAT, 57 seconds; CAT, 80 seconds; SWATT, 90 seconds; p < 0.0001), and preference (MAT, 64%; CAT, 30%; SWATT, 6%; p < 0.0001). CONCLUSION: In this study, the MAT performs better in terms of success rate, time to application, and preference when used by school-aged children. This study can be helpful when facilities are purchasing tourniquets for use by students.

The Ethical Triage and Management Guidelines of the Entrapped and Mangled Extremity in Resource Scarce Environments: A Systematic Literature Review.

OBJECTIVE: A systematic literature review (SLR) was performed to elucidate the current triage and treatment of an entrapped or mangled extremity in resource scarce environments (RSEs). METHODS: A lead researcher followed the search strategy following inclusion and exclusion criteria. A first reviewer (FR) was randomly assigned sources. One of the 2 lead researchers was the second reviewer (SR). Each determined the level of evidence (LOE) and quality of evidence (QE) from each source. Any differing opinions between the FR and SR were discussed between them, and if differing opinions remained, then a third reviewer (the other lead researcher) discussed the article until a consensus was reached. The final opinion of each article was entered for analysis. RESULTS: Fifty-eight (58) articles were entered into the final study. There was 1 study determined to be LOE 1, 29 LOE 2, and 28 LOE 3, with 15 determined to achieve QE 1, 37 QE 2, and 6 QE 3. CONCLUSION: This SLR showed that there is a lack of studies producing strong evidence to support the triage and treatment of the mangled extremity in RSE. Therefore, a Delphi process is suggested to adapt and modify current civilian and military triage and treatment guidelines to the RSE.

Residents working with Médecins Sans Frontières: training and pilot evaluation.

BACKGROUND: Well-prepared humanitarian workers are now more necessary than ever. Essential to the preparation process are: clearly defined learning objectives, curricula tailored to the nuances of humanitarian settings, simulation-based training, and evaluation. This manuscript describes a training program designed to prepare medical residents for their first field deployment with Médecins Sans Frontières and presents the results of a pilot assessment of its effectiveness. METHODS: The training was jointly developed by the Research Center in Emergency and Disaster Medicine- CRIMEDIM of the Università del Piemonte Orientale, Novara, Italy, and the humanitarian aid organization Médecins Sans Frontières- Italy (MSF-Italy); the following topics were covered: disaster medicine, public health, safety and security, infectious diseases, psychological support, communication, humanitarian law, leadership, and job-specific skills. It used a blended-learning approach consisting of a 3-month distance learning module; 1-week instructor-led coaching; and a field placement with MSF. We assessed its effectiveness using the first three levels of Kirkpatrick's training evaluation model. RESULTS: Eight residents took part in the evaluation. Four were residents in emergency medicine, 3 in anesthesia, and 1 in pediatrics; 3 of them were female and the median age was 31 years. Two residents were deployed in Pakistan, 1 in Afghanistan, 1 in the Democratic Republic of Congo, 1 in Iraq, 2 in Haiti and 1 on board of the MSF Mediterranean search & rescue ship. Mean deployment time was 3 months. The average median score for the overall course was 5 (excellent). There was a significant improvement in post-test multiple choice scores (p = 0.001) and in residents' overall performance scores (P = 0.000001). CONCLUSION: Residents were highly satisfied with the training program and their knowledge and skills improved as a result of participation. TRIAL REGISTRATION: This study was approved by the Institutional Ethics Committee (date 24-02-2016, study code UPO.2015.4.10).

COVID-19 Pandemic: Perspective From Italian Pediatric Emergency Physicians.

OBJECTIVES: To document the lived experience of Italian pediatric emergency physicians during the coronavirus disease 2019 (COVID-19) pandemic. METHODS: We developed a structured interview to collect the lived experience of the staff of the pediatric emergency department (PED) of a tertiary referral university hospital in Northern Italy. The open-ended questions were draft according to the suggestions of Canadian colleagues and administered by 1 interviewer, who was part of the PED staff, at the end of March 2020. All the PED staff was interviewed, on a voluntary basis, using purposive sampling. RESULTS: Most respondents declared to be afraid of becoming infected and of infecting their families. The number of patients seen in the PED has decreased, and the cases tend to be more severe. A shift in the clinical approach to the ill child has occurred, the physical examination is problem-oriented, aiming to avoid un-necessary maneuvers and to minimize the number of practitioners involved. The most challenging aspects reported are: (1) performing a physical examination in personal protective equipment (PPE), (2) being updated with rapidly evolving guidelines, and (3) staying focused on the possible COVID-19 clinical presentation without failing in differential diagnosis. CONCLUSIONS: During the COVID-19 pandemic, it seems that pediatric emergency physicians are radically changing their clinical practice, aiming at prioritizing essential interventions and maneuvers and self-protection.

A Randomized Trial of Instructor-Led Training Versus Video Lesson in Training Health Care Providers in Proper Donning and Doffing of Personal Protective Equipment.

OBJECTIVE: This study compared live instructor-led training with video-based instruction in personal protective equipment (PPE) donning and doffing. It assessed the difference in performance between (1) attending 1 instructor-led training session in donning and doffing PPE at 1 month prior to assessment, and (2) watching training videos for 1 month. METHODS: This randomized controlled trial pilot study divided 21 medical students and junior doctors into 2 groups. Control group participants attended 1 instructor-led training session. Video group participants watched training videos demonstrating the same procedures, which they could freely watch again at home. After 1 month, a doctor performed a blind evaluation of performance using checklists. RESULTS: Nineteen participants were assessed after 1 month. The mean donning score was 84.8/100 for the instructor-led group and 88/100 for the video group; mean effect size was 3.2 (95% CI: -7.5 to 9.5). The mean doffing score was 79.1/100 for the instructor-led group and 73.9/100 for the video group; mean effect size was 5.2 (95% CI: -7.6 to 18). CONCLUSION: Our study found no significant difference in donning and doffing scores between instructor-led and video lessons. Video training could be a fast and resource-efficient method of training in PPE donning and doffing in responding to the COVID-19 pandemic.

Cognitive appraisals and team performance under stress: A simulation study.

OBJECTIVES: The present study explored how challenge and threat responses to stress relate to performance, anxiety, confidence, team identity and team characteristics (time spent in training and postgraduate experience) in a medical simulation-based team competition. METHODS: The study was conducted during a national simulation-based training event for residents, the SIMCUP Italia 2018. The SIMCUP is a simulation competition in which teams of four compete in simulated medical emergency scenarios. Cross-sectional data were collected prior to the 3 days of the competition. Subjects included 95 participants on 24 teams. Before the competition on each day, participants completed brief self-report measures that assessed demands and resources (which underpin challenge and threat responses to stress), cognitive and somatic anxiety, self-confidence and team identification. Participants also reported time (hours) spent practising as a team and years of postgraduate experience. A team of referees judged each scenario for performance and assigned a score. A linear mixed model using demands and resources was built to model performance. RESULTS: The data showed that both demands and resources have positive effects on performance (31 [11-50.3] [P < .01] and 54 [25-83.3] [P < .01] percentage points increase for unitary increases in demands and resources, respectively); however, this is balanced by a negative interaction between the two (demands * resources interaction coefficient = -10 [-16 to -4.2]). A high level of resources is associated with better performance until demands become very high. Cognitive and somatic anxieties were found to be correlated with demands (Pearson's r = .51 [P < .01] and Pearson's r = .48 [P < .01], respectively). Time spent training was associated with greater perceptions of resources (Pearson's r = .36 [P < .01]). CONCLUSIONS: We describe a model of challenge and threat that allows for the estimation of performance according to perceived demands and resources, and the interaction between the two. Higher levels of resources and lower demands were associated with better performance.

Comparing Resource Management Skills in a High- versus Low-Resource Simulation Scenario: A Pilot Study.

BACKGROUND: Low-resource environments, such as those found in humanitarian crises, pose significant challenges to the provision of proper medical treatment. While the lack of training of health providers to such settings has been well-acknowledged in literature, there has yet to be any scientific evidence for this phenomenon. METHODS: This pilot study utilized a randomized crossover experimental design to examine the effects of high- versus low-resource simulated scenarios of a resuscitation of a critically ill obstetric patient on a medical doctors' performance and inter-personal skills. Ten senior residents (fifth-year post-graduate) of the Maggiore Hospital School of Medicine (Novara, NO, Italy) were included in the study. RESULTS: Overall performance score for the high-resource setting was 5.2, as opposed to only 2.3 for the low-resource setting. The mean effect size for the overall score was 2.9 (95% CI, 1.7-4.0; P <.001). The results suggest a significant decrease in both technical (medical) and non-technical skills, such as leadership, problem solving, situation awareness, resource utilization, and communication in the low-resource environment setting. The latter finding is of special important since it was yet to be reported. CONCLUSIONS: This pilot study suggests that untrained physicians in low-resource environments may experience a considerable setback not only to their professional performance, but also to their interpersonal skills, when deployed ill-prepared to humanitarian missions. Consequently, this may endanger the health of local populations.

Intra-operative low-dose ketamine does not reduce the cost of post-operative pain management after surgery: a randomized controlled trial in a low-income country.

BACKGROUND: In developing countries, post-operative pain remains underestimated and undertreated due to economic constraints, lack of awareness and limited resources. In contrast, ketamine is an effective, readily available, easy to use and inexpensive drug frequently used in poor settings. OBJECTIVES: The aim of this study was to explore the overall reduction in the medication treatment cost of acute post-operative pain by adding intra-operative low-dose ketamine to traditional intravenous morphine for surgery in a low-income country. METHODS: A double blind randomized controlled trial with placebo-controlled parallel group was performed in Mulago National Hospital (Uganda). Consenting adults scheduled for elective surgery were randomized into two study arms: Group K received ketamine 0.15mg/kg bolus at induction and a continuous infusion of 0.12 mg/kg/hour till start of skin closure; Group C (control) received normal saline. Both groups received Morphine 0.1 mg/kg IV at debulking. The total medication cost was registered. NRS pain scores and other measurements such vital signs and incidence of major and minor side effects were also recorded. RESULTS: A total of 46 patients were included. Patients' baseline characteristics were comparable in both groups. No statistically significant difference was found between the groups concerning the overall medication cost of post-operative pain management. Pain scores, patients' satisfaction in the first 24 hours after surgery and hospital length of stay were similar in both groups. CONCLUSION: Our results do not support the utilization of intra-operative low dose ketamine as a cost-saving post-operative pain treatment strategy for all types of surgery in low-resource settings.

A Randomized Trial Comparing Telephone Tree, Text Messaging, and Instant Messaging App for Emergency Department Staff Recall for Disaster Response.

IntroductionA crucial component of a hospital's disaster plan is an efficient staff recall communication method. Many hospitals use a "calling tree" protocol to contact staff members and recall them to work. Alternative staff recall methods have been proposed and explored. METHODS: An unannounced, multidisciplinary, randomized emergency department (ED) staff recall drill was conducted at night - when there is the greatest need for back-up personnel and staff is most difficult to reach. The drill was performed on December 14, 2017 at 4:00am and involved ED staff members from three hospitals which are all part of the McGill University Health Centre (MUHC; Montreal, Quebec, Canada). Three tools were compared: manual phone tree, instant messaging application (IMA), and custom-made hospital Short Message Service (SMS) system. The key outcome measures were proportion of responses at 45 minutes and median response time. RESULTS: One-hundred thirty-two participants were recruited. There were 44 participants in each group after randomization. In the manual phone tree group, 18 (41%) responded within 45 minutes. In the IMA group, 11 participants (25%) responded in the first 45 minutes. In the SMS group, seven participants responded in the first 45 minutes (16%). Manual phone tree was significantly better than SMS with an effect size of 25% (95% confidence interval for effect: 4.6% to 45.0%; P=.018). Conversely, there was no significant difference between manual phone tree and IMA with an effect size of 16% (95% confidence interval for effect: -5.7% to 38.0%; P=.17) There was a statistically significant difference in the median response time between the three groups with the phone tree group presenting the lowest median response time (8.5 minutes; range: 2.0 to 8.5 minutes; P=.000006). CONCLUSION: Both the phone tree and IMA groups had a significantly higher response rate than the SMS group. There was no significant difference between the proportion of responses at 45 minutes in the phone tree and the IMA arms. This study suggests that an IMA may be a viable alternative to the traditional phone tree method. Limitations of the study include volunteer bias and the fact that there was only one communication drill, which did not allow staff members randomized to the IMA and SMS groups to fully get familiar with the new staff recall methods. HomierV, HamadR, LarocqueJ, ChasséP, KhalilE, FrancJM. A randomized trial comparing telephone tree, text messaging, and instant messaging app for emergency department staff recall for disaster response. Prehosp Disaster Med. 2018;33(5):471-477.

A novel simulation competition format as an effective instructional tool in post-graduate medical education.

OBJECTIVE: Medical simulation competitions are a growing reality. This study aims at exploring if a novel format of simulation competition (SIMCUP) can be an effective educational format in post-graduate education. DESIGN: We designed a 2-day event that included scientific educational lectures, an orientation to the competition, familiarization with the simulation lab, and competition time. Day 1 was devoted to preliminary rounds and was structured using an Objective Structured Clinical Examination (OSCE)-like system. On day 2, the first four teams advanced to semi-finals and then to finals, which were held using a classical SimWars style. SETTING AND SUBJECTS: A total of 14 four-participant teams participated in the event over two editions (Ed.1 in 2015 and Ed.2 in 2016). INTERVENTIONS: External referees evaluated both technical and non-technical skills for each simulated scenario. Each participant was also administered pre- and post-test questionnaires covering self-perception about the confidence in managing simulated clinical cases, educational effectiveness, satisfaction with the simulation experience, and previous simulation training. MAIN RESULTS: Overall participants found SIMCUP a useful learning experience, rating it 10 [9, 10] and 10 [7.75-10] out of 10 for Ed.1 and Ed.2, respectively. Participants reported, using a 10-point semantic differential scale ranging from "1 - strongly disagree." to "10 - strongly agree," finding both days to be educationally effective: day 1 was rated 9 [7-10] and 9 [8-10] as day 2 was rated 8 [7-10] and 8 [7-10] for Ed. 1 and Ed. 2, respectively.Participants' self-perception regarding the confidence of managing the specific scenarios significantly improved immediately after the event as measured by pre- and post-questionnaires for all stations and during both editions. CONCLUSION: This study suggests that simulation competition can serve as an effective instructional format in residency training.

Prioritization in medical school simulation curriculum development using survey tools and desirability function: a pilot experiment.

BACKGROUND: In Italy, there is no framework of procedural skills that all medical students should be able to perform autonomously at graduation. The study aims at identifying (1) a set of essential procedural skills and (2) which abilities could be potentially taught with simulation. Desirability score was calculated for each procedure to determine the most effective manner to proceed with simulation curriculum development. METHODS: A web poll was conducted at the School of Medicine in Novara, looking at the level of expected and self-perceived competency for common medical procedures. Three groups were enrolled: (1) faculty, (2) junior doctors in their first years of practice, and (3) recently graduated medical students. Level of importance of procedural skills for independent practice expressed by teachers, level of mastery self-perceived by learners (students and junior doctors) and suitability of simulation training for the given technical skills were measured. Desirability function was used to set priorities for future learning. RESULTS: The overall mean expected level of competency for the procedural skills was 7.9/9. Mean level of self reported competency was 4.7/9 for junior doctors and 4.4/9 for recently graduated students. The highest priority skills according to the desirability function were urinary catheter placement, nasogastric tube insertion, and incision and drainage of superficial abscesses. CONCLUSIONS: This study identifies those technical competencies thought by faculty to be important and assessed the junior doctors and recent graduates level of self-perceived confidence in performing these skills. The study also identifies the perceived utility of teaching these skills by simulation. The study prioritizes those skills that have a gap between expected and observed competency and are also thought to be amenable to teaching by simulation. This allows immediate priorities for simulation curriculum development in the most effective manner. This methodology may be useful to researchers in other centers to prioritize simulation training.

Anesthesiology Resident Induction Month: a pilot study showing an effective and safe way to train novice residents through simulation.

BACKGROUND: The transition of new residents from medical school to the post-graduate clinical environment remains challenging. We hypothesized that an introductory simulation course could improve new residents' performance in anesthesiology. METHODS: The Anesthesiology Residents Induction Month (ARIM) program was designed as a non-clinical simulation training program aiming at providing the theoretical and practical skills to safely approach, as junior anesthesiologists, the operating rooms. For each participant, specific knowledge, procedural skills and non-technical performance were assessed with a pre and post-test approach, before and immediately after the participation in the study. RESULTS: Fifteen first-month residents participated in the study. As compared to pre-test, residents significantly improved in all three evaluated areas. Pre-test knowledge assessment mean improved from 56% to 73% in the post-test (P<0.001). In the procedural skills assessment, pre-test mean improved from 43% to 77% (P<0.001) and non-technical skills assessment improved from 3.17 to 4.61 (in a scale out of seven points) in the post-test (P<0.001). CONCLUSIONS: Data suggest that an intensive simulation-based program can be an effective way for first-year residents to rapidly acquire and develop basic skills specific to anesthesiology. There might be benefits to begin residency with a training program aiming at developing and standardizing technical and non-technical skills.