Examining the Effect of Interruptions at Different Breakpoints and Frequencies Within a Task.

OBJECTIVE: The purpose was to explore how event segmentation theory (EST) can be used to determine optimal moments for an interruption relying on hierarchical task analysis (HTA) to identify coarse and fine event boundaries. BACKGROUND: Research on the effects of interruptions shows that they can be either disruptive or beneficial, depending on which aspects of an interruption are manipulated. Two important aspects that contribute to these conflicting results concern when and how often interruptions occur. METHOD: Undergraduates completed a trip planning task divided into three subtasks. The within-subjects factor was interruption timing with three levels: none, coarse breakpoints, and fine breakpoints. The between-subjects factor was interruption frequency with two levels: one and three. The dependent measures included resumption lag, number of errors, mental workload, and frustration. RESULTS: Participants took longer to resume the primary task and reported higher mental workload when interruptions occurred at fine breakpoints. The effect of interruptions at coarse breakpoints was similar to completing the task without interruption. Interruption frequency had no effect on performance; however, participants spent significantly longer attending to interruptions in the initial task, and within a task, the first and second interruptions were attended to significantly longer than the third interruption. CONCLUSION: The disruptiveness of an interruption is tied to the point within the task hierarchy where it occurs. APPLICATION: The performance cost associated with interruptions must be considered within the task structure. Interruptions occurring at coarse breakpoints may not be disruptive or have a negative effect on mental workload.

Medical Students' Experiences and Outcomes Using a Virtual Human Simulation to Improve Communication Skills: Mixed Methods Study.

BACKGROUND: Attending to the wide range of communication behaviors that convey empathy is an important but often underemphasized concept to reduce errors in care, improve patient satisfaction, and improve cancer patient outcomes. A virtual human (VH)-based simulation, MPathic-VR, was developed to train health care providers in empathic communication with patients and in interprofessional settings and evaluated through a randomized controlled trial. OBJECTIVE: This mixed methods study aimed to investigate the differential effects of a VH-based simulation developed to train health care providers in empathic patient-provider and interprofessional communication. METHODS: We employed a mixed methods intervention design, involving a comparison of 2 quantitative measures-MPathic-VR-calculated scores and the objective structured clinical exam (OSCE) scores-with qualitative reflections by medical students about their experiences. This paper is a secondary, focused analysis of intervention arm data from the larger trial. Students at 3 medical schools in the United States (n=206) received simulation to improve empathic communication skills. We conducted analysis of variance, thematic text analysis, and merging mixed methods analysis. RESULTS: OSCE scores were significantly improved for learners in the intervention group (mean 0.806, SD 0.201) compared with the control group (mean 0.752, SD 0.198; F1,414=6.09; P=.01). Qualitative analysis revealed 3 major positive themes for the MPathic-VR group learners: gaining useful communication skills, learning awareness of nonverbal skills in addition to verbal skills, and feeling motivated to learn more about communication. Finally, the results of the mixed methods analysis indicated that most of the variation between high, middle, and lower performers was noted about nonverbal behaviors. Medium and high OSCE scorers most often commented on the importance of nonverbal communication. Themes of motivation to learn about communication were only present in middle and high scorers. CONCLUSIONS: VHs are a promising strategy for improving empathic communication in health care. Higher performers seemed most engaged to learn, particularly nonverbal skills.

Learner Improvement From a Simulation-Enhanced Ultrasonography Curriculum for First-Year Medical Students.

OBJECTIVES: We describe a simulation-enhanced ultrasonography (US) curriculum for first-year medical students as part of a comprehensive curricular integration of US skills. Our goal was to assess student knowledge and performance of US and determine their satisfaction with the integrated curriculum. METHODS: A committee of basic science, clinical, and interinstitutional faculty developed 7 educational US modules integrated into existing anatomy and physiology courses. First-year students in years 2012 through 2014 were administered a demographic survey and a knowledge-based pretest at the outset of the US program and assessed with a posttest, satisfaction survey, and their image acquisition abilities in an objective structured clinical examination with standardized patients on completion of the program. RESULTS: Data from 390 students showed a significant increase in knowledge from the pretest to the posttest [t(389) = 58.027; P < .0001]. Students with higher spatial abilities or some previous US experience performed better on the posttest. The objective structured clinical examination results showed that about 83% of the students were able to capture acceptable or marginally acceptable images. Ninety-five percent of students indicated that the US educational experience enhanced their medical education. CONCLUSIONS: Initial results show that we were able to successfully develop, implement, and evaluate performance of first-year medical students on their fundamental knowledge and performance of basic US using a model that emphasized hands-on simulation-enhanced training. Furthermore, most students found the experience to be a beneficial component of their education and indicated a desire for more US training in the medical curricula.

Running Memory for Clinical Handoffs: A Look at Active and Passive Processing.

OBJECTIVE: The goal of the present study was to examine the effects of domain-relevant expertise on running memory and the ability to process handoffs of information. In addition, the role of active or passive processing was examined. BACKGROUND: Currently, there is little research that addresses how individuals with different levels of expertise process information in running memory when the information is needed to perform a real-world task. METHOD: Three groups of participants differing in their level of clinical expertise (novice, intermediate, and expert) performed an abstract running memory span task and two tasks resembling real-world activities, a clinical handoff task and an air traffic control (ATC) handoff task. For all tasks, list length and the amount of information to be recalled were manipulated. RESULTS: Regarding processing strategy, all participants used passive processing for the running memory span and ATC tasks. The novices also used passive processing for the clinical task. The experts, however, appeared to use more active processing, and the intermediates fell in between. CONCLUSION: Overall, the results indicated that individuals with clinical expertise and a developed mental model rely more on active processing of incoming information for the clinical task while individuals with little or no knowledge rely on passive processing. APPLICATION: The results have implications about how training should be developed to aid less experienced personnel identify what information should be included in a handoff and what should not.

Healthcare Simulation Training Guidelines and Literature Reviews From the Third Society for Simulation in Healthcare Research Summit.

ABSTRACT: The Society for Simulation in Healthcare held its third research summit in January 2023 with the aim of establishing evidence-based guidelines for healthcare simulation training. A panel of researchers, clinicians, and subject-matter experts conducted reviews of the literature addressing 12 key topics and followed a formal process to generate 16 guidelines for simulation-based training in healthcare. Eleven peer-reviewed literature reviews accompany these guidelines. Over the last 12 years, the Society for Simulation in Healthcare research summits have evolved with a consistent aim to advance simulation research, culminating in the formal set of guidelines published in this special issue.

Simulator training to automaticity leads to improved skill transfer compared with traditional proficiency-based training: a randomized controlled trial.

OBJECTIVE: We hypothesized that novices will perform better in the operating room after simulator training to automaticity compared with traditional proficiency based training (current standard training paradigm). BACKGROUND: Simulator-acquired skill translates to the operating room, but the skill transfer is incomplete. Secondary task metrics reflect the ability of trainees to multitask (automaticity) and may improve performance assessment on simulators and skill transfer by indicating when learning is complete. METHODS: Novices (N = 30) were enrolled in an IRB-approved, blinded, randomized, controlled trial. Participants were randomized into an intervention (n = 20) and a control (n = 10) group. The intervention group practiced on the FLS suturing task until they achieved expert levels of time and errors (proficiency), were tested on a live porcine fundoplication model, continued simulator training until they achieved expert levels on a visual spatial secondary task (automaticity) and were retested on the operating room (OR) model. The control group participated only during testing sessions. Performance scores were compared within and between groups during testing sessions. RESULTS: : Intervention group participants achieved proficiency after 54 ± 14 and automaticity after additional 109 ± 57 repetitions. Participants achieved better scores in the OR after automaticity training [345 (range, 0-537)] compared with after proficiency-based training [220 (range, 0-452; P < 0.001]. CONCLUSIONS: Simulator training to automaticity takes more time but is superior to proficiency-based training, as it leads to improved skill acquisition and transfer. Secondary task metrics that reflect trainee automaticity should be implemented during simulator training to improve learning and skill transfer.

Preparing and Presenting Validation Studies: A Guide for the Perplexed.

SUMMARY STATEMENT: Simulated environments are frequently used for learner assessment, and a wide array of assessment instruments have been created to assist with this process. It is important, therefore, that clear, compelling evidence for the validity of these assessments be established. Contemporary theory recognizes instrument validity as a unified construct that links a construct to be assessed with a population, an environment of assessment, and a decision to be made using the scores. In this article, we present a primer on 2 current frameworks (Messick and Kane), define the elements of each, present a rubric that can be used by potential authors to structure their work, and offer examples of published studies showing how each framework has been successfully used to make a validity argument. We offer this with the goal of improving the quality of validity-related publications, thereby advancing the quality of assessment in healthcare simulation.

Time and number of displays impact critical signal detection in fetal heart rate tracings.

Interest in centralized monitoring in labor and delivery units is growing because it affords the opportunity to monitor multiple patients simultaneously. However, a long history of research on sustained attention reveals these types of monitoring tasks can be problematic. The goal of the present experiment was to examine the ability of individuals to detect critical signals in fetal heart rate (FHR) tracings in one or more displays over an extended period of time. Seventy-two participants monitored one, two, or four computer-simulated FHR tracings on a computer display for the appearance of late decelerations over a 48-minute vigil. Measures of subjective stress and workload were also obtained before and after the vigil. The results showed that detection accuracy decreased over time and also declined as the number of displays increased. The subjective reports indicated that participants found the task to be stressful and mentally demanding, effortful, and frustrating. The results suggest that centralized monitoring that allows many patients to be monitored simultaneously may impose a detrimental attentional burden on the observer. Furthermore, this seemingly benign task may impose an additional source of stress and mental workload above what is commonly found in labor and delivery units.

Simulation in training: one-year experience using an efficiency index to assess interventional radiology fellow training status.

PURPOSE: To compare objective fellow and expert efficiency indices for an interventional radiology renal artery stenosis skill set with the use of a high-fidelity simulator. MATERIALS AND METHODS: The Mentice VIST simulator was used for three different renal artery stenosis simulations of varying difficulty, which were used to grade performance. Fellows' indices at three intervals throughout 1 year were compared to expert baseline performance. Seventy-four simulated procedures were performed, 63 of which were captured as audiovisual recordings. Three levels of fellow experience were analyzed: 1, 6, and 12 months of dedicated interventional radiology fellowship. The recordings were compiled on a computer workstation and analyzed. Distinct measurable events in the procedures were identified with task analysis, and data regarding efficiency were extracted. Total scores were calculated as the product of procedure time, fluoroscopy time, tools, and contrast agent volume. The lowest scores, which reflected efficient use of tools, radiation, and time, were considered to indicate proficiency. Subjective analysis of participants' procedural errors was not included in this analysis. RESULTS: Fellows' mean scores diminished from 1 month to 12 months (42,960 at 1 month, 18,726 at 6 months, and 9,636 at 12 months). The experts' mean score was 4,660. In addition, the range of variance in score diminished with increasing experience (from a range of 5,940-120,156 at 1 month to 2,436-85,272 at 6 months and 2,160-32,400 at 12 months). Expert scores ranged from 1,450 to 10,800. CONCLUSIONS: Objective efficiency indices for simulated procedures can demonstrate scores directly comparable to the level of clinical experience.

Monitoring simulated maternal-fetal heart rate signals.

The present study represents the first in a series aimed at examining how individuals monitor maternal-fetal heart rate signals over an extended period of time. Seventy-two undergraduate students monitored simulated maternal-fetal heart rate signals for late decelerations of either high or low amplitude for 48 minutes. The results showed that the ability to detect late decelerations was quite good and did not decline over time. Although the findings regarding detections are encouraging, the participants also made a lot of false alarms. Collectively, these results suggest that the observers still struggled to distinguish critical patterns from the background fetal heart activity.

The Second Society for Simulation in Healthcare Research Summit: Beyond our Boundaries.

STATEMENT: In this article, we report on the Second Society for Simulation in Healthcare Research Summit held in 2017. This meeting succeeded the First Research Summit from 2011 with the goal of advancing the scope of healthcare simulation research. During the one and a half day summit, some of the world's leading experts in simulation, healthcare, and simulation in healthcare convened to discuss ideas about what research goals would be most beneficial to the healthcare simulation community, and what could be done to achieve them. We describe the rationale for the meeting, the organization, the program, and the articles that emerged from the Summit, which are found in this supplemental issue of Simulation in Healthcare.

Human Factors and Simulation in Emergency Medicine.

This consensus group from the 2017 Academic Emergency Medicine Consensus Conference "Catalyzing System Change through Health Care Simulation: Systems, Competency, and Outcomes" held in Orlando, Florida, on May 16, 2017, focused on the use of human factors (HF) and simulation in the field of emergency medicine (EM). The HF discipline is often underutilized within EM but has significant potential in improving the interface between technologies and individuals in the field. The discussion explored the domain of HF, its benefits in medicine, how simulation can be a catalyst for HF work in EM, and how EM can collaborate with HF professionals to effect change. Implementing HF in EM through health care simulation will require a demonstration of clinical and safety outcomes, advocacy to stakeholders and administrators, and establishment of structured collaborations between HF professionals and EM, such as in this breakout group.

Task sequencing effects for open and closed loop laparoscopic skills.

The present study examined laparoscopic skill acquisition on a simulator for different sequences of open and closed loop tasks. Sixteen medical students were divided into four groups distinguished by their initial training task and subsequent transfer task. Group 1 practiced instrument navigation, an open loop task, and then transferred to grasping, a closed loop task. Group 2 practiced grasping, and then transferred to the instrument navigation task. Group 3 practiced instrument navigation and then transferred to a complex cutting task that involved both open and closed loop components. Group 4 practiced grasping and then transferred to the cutting task. The results showed distinct task sequencing effects in favor of initial practice on a closed loop task. Specifically, task completion times declined significantly when participants practiced the closed loop task followed by the open loop task. The benefits of initial practice on a closed loop task, however, were limited primarily to accuracy measures when participants transferred to the complex cutting task. The findings indicate that task order is important and that training on one task can either facilitate or impede skill acquisition on a subsequent task and that these differences reflect fundamental psychomotor characteristics of the tasks.

Differences in mental workload between traditional and single-incision laparoscopic procedures measured with a secondary task.

INTRO: The mental workload associated with laparoscopic suturing can be assessed with a secondary task that requires the same visual-spatial attentional resources. The purpose of this study was to use a secondary task to measure the incremental workload demands of single-incision laparoscopic surgery (SILS) procedures versus traditional laparoscopic procedures. METHOD: 12 surgery residents and surgical assistants who had met FLS criteria on an FLS and SILS simulator performed one trial each of peg transfer, cutting, and intracorporeal suturing tasks simultaneously with the secondary task and provided subjective workload ratings using the NASA-TLX. RESULTS: SILS procedures resulted in lower primary and secondary task scores, p < 0.001 and higher workload ratings, p < 0.0001. Suturing resulted in lower primary (p < 0.003) and secondary task scores (p < 0.017) and higher workload ratings (p < 0.017) compared to the other tasks. CONCLUSIONS: SILS procedures were significantly more mentally demanding than traditional laparoscopic procedures corroborated by primary and secondary tasks scores and subjective ratings.

Effects of a retention interval and refresher session on intracorporeal suturing and knot tying skill and mental workload.

BACKGROUND: The effects of refraining from practice for different intervals on laparoscopic suturing and mental workload was assessed with a secondary task developed by the authors. We expected the inability to practice to produce a decrease in performance on the suturing, knot tying, and secondary task and skills to rebound after a single refresher session. METHODS: In total, 22 surgical assistant and premedical students trained to Fundamentals of Laparoscopic Surgery proficiency in intracorporeal suturing and knot tying were assessed on that task using a secondary task. Participants refrained from practicing any Fundamentals of Laparoscopic Surgery tasks for 1 or 5 months. At the time of their return, they were assessed immediately on suturing and knot tying with the secondary task, practiced suturing and knot tying for 40 minutes, and then were reassessed. RESULTS: The mean suture times from the initial reassessment were greater than the proficiency times but returned to proficiency levels after one practice session, F(2, 40) = 14.5, P < .001, partial η2 = .420. Secondary task scores mirrored the results of suturing time, F(2, 40) = 6.128, P < .005, partial η2 = .235, and were moderated by retention interval. CONCLUSION: When participants who reached proficiency in suturing and knot tying were reassessed after either 1or 5 months without practice, their performance times increased by 35% and secondary task scores decreased by 30%. These deficits, however, were nearly reversed after a single refresher session.

Initial construct validity evidence of a virtual human application for competency assessment in breaking bad news to a cancer patient.

BACKGROUND: Despite interest in using virtual humans (VHs) for assessing health care communication, evidence of validity is limited. We evaluated the validity of a VH application, MPathic-VR, for assessing performance-based competence in breaking bad news (BBN) to a VH patient. METHODS: We used a two-group quasi-experimental design, with residents participating in a 3-hour seminar on BBN. Group A (n=15) completed the VH simulation before and after the seminar, and Group B (n=12) completed the VH simulation only after the BBN seminar to avoid the possibility that testing alone affected performance. Pre- and postseminar differences for Group A were analyzed with a paired t-test, and comparisons between Groups A and B were analyzed with an independent t-test. RESULTS: Compared to the preseminar result, Group A's postseminar scores improved significantly, indicating that the VH program was sensitive to differences in assessing performance-based competence in BBN. Postseminar scores of Group A and Group B were not significantly different, indicating that both groups performed similarly on the VH program. CONCLUSION: Improved pre-post scores demonstrate acquisition of skills in BBN to a VH patient. Pretest sensitization did not appear to influence posttest assessment. These results provide initial construct validity evidence that the VH program is effective for assessing BBN performance-based communication competence.

Using a computer simulation for teaching communication skills: A blinded multisite mixed methods randomized controlled trial.

OBJECTIVES: To assess advanced communication skills among second-year medical students exposed either to a computer simulation (MPathic-VR) featuring virtual humans, or to a multimedia computer-based learning module, and to understand each group's experiences and learning preferences. METHODS: A single-blinded, mixed methods, randomized, multisite trial compared MPathic-VR (N=210) to computer-based learning (N=211). Primary outcomes: communication scores during repeat interactions with MPathic-VR's intercultural and interprofessional communication scenarios and scores on a subsequent advanced communication skills objective structured clinical examination (OSCE). Multivariate analysis of variance was used to compare outcomes. SECONDARY OUTCOMES: student attitude surveys and qualitative assessments of their experiences with MPathic-VR or computer-based learning. RESULTS: MPathic-VR-trained students improved their intercultural and interprofessional communication performance between their first and second interactions with each scenario. They also achieved significantly higher composite scores on the OSCE than computer-based learning-trained students. Attitudes and experiences were more positive among students trained with MPathic-VR, who valued its providing immediate feedback, teaching nonverbal communication skills, and preparing them for emotion-charged patient encounters. CONCLUSIONS: MPathic-VR was effective in training advanced communication skills and in enabling knowledge transfer into a more realistic clinical situation. PRACTICE IMPLICATIONS: MPathic-VR's virtual human simulation offers an effective and engaging means of advanced communication training.