Hierarchical task analysis of endoscopic sleeve gastroplasty.

BACKGROUND: Endoscopic sleeve gastroplasty (ESG) is a minimally invasive endoscopic weight loss procedure used to treat obesity. The long-term goal of this project is to develop a Virtual Bariatric Endoscopy (ViBE) simulator for training and assessment of the ESG procedure. The objectives of this current work are to: (a) perform a task analysis of ESG and (b) create metrics to be validated in the created simulator. METHODS: We performed a hierarchical task analysis (HTA) by identifying the significant tasks of the ESG procedure. We created the HTA to show the breakdown and connection of the tasks of the procedure. Utilizing the HTA and input from ESG experts, performance metrics were derived for objective measurement of the ESG procedure. Three blinded video raters analyzed seven recorded ESG procedures according to the proposed performance metrics. RESULTS: Based on the seven videos, there was a positive correlation between total task times and total performance scores (R = 0.886, P = 0.008). Endoscopists expert were found to be more skilled in reducing the area of the stomach compared to endoscopists novice (34.6% reduction versus 9.4% reduction, P = 0.01). The mean novice performance score was significantly lower than the mean expert performance score (34.7 vs. 23.8, P = 0.047). The inter-rater reliability test showed a perfect agreement among three raters for all tasks except for the suturing task. The suturing task had a significant agreement (Inter-rater Correlation = 0.84, Cronbach's alpha = 0.88). Suturing was determined to be a critical task that is positively correlated with the total score (R = 0.962, P = 0.0005). CONCLUSION: The task analysis and metrics development are critical for the development of the ViBE simulator. This preliminary assessment demonstrates that the performance metrics provide an accurate assessment of the endoscopist's performance. Further validation testing and refinement of the performance metrics are anticipated.

Does your team know how to respond safely to an operating room fire? Outcomes of a virtual reality, AI-enhanced simulation training.

BACKGROUND: Operating room (OR) fires are rare but devastating events requiring immediate and effective response. Virtual Reality (VR) simulation training can provide a safe environment for practice of skills in such highly stressful situation. This study assessed interprofessional participants' ability to respond to VR-simulated OR fire scenarios, attitudes, numbers of attempt of the VR simulation do participants need to successfully respond to OR fires and does prior experience, confidence level, or professional role predict the number of attempts needed to demonstrate safety and pass the simulation. METHODS: 180 surgical team members volunteered to participate in this study at Beth Israel Deaconess Medical Center, Boston, MA. Each participant completed five VR OR simulation trials; the final two trials incorporated AI assistance. Primary outcomes were performance scores, number of attempts needed to pass, and pre- and post-survey results describing participant confidence and experiences. Differences across professional or training role were assessed using chi-square tests and analyses of variance. Differences in pass rates over time were assessed using repeated measures logistic regression. RESULTS: One hundred eighty participants completed simulation testing; 170 (94.4%) completed surveys. Participants included surgeons (17.2%), anesthesiologists (10.0%), allied health professionals (41.7%), and medical trainees (31.1%). Prior to training, 45.4% of participants reported feeling moderately or very confident in their ability to respond to an OR fire. Eight participants (4.4%) responded safely on the first simulation attempt. Forty-three participants (23.9%) passed by the third attempt (VR only); an additional 97 participants (53.9%) passed within the 4-5th attempt (VR with AI assistance). CONCLUSIONS: Providers are unprepared to respond to OR fires. VR-based simulation training provides a practical platform for individuals to improve their knowledge and performance in the management of OR fires with a 79% pass rate in our study. A VR AI approach to teaching this essential skill is innovative, feasible, and effective.

Virtual reality operating room with AI guidance: design and validation of a fire scenario.

BACKGROUND: Operating room (OR) fires are uncommon but disastrous events. Inappropriate handling of OR fires can result in injuries, even death. Aiming to simulate OR fire emergencies and effectively train clinicians to react appropriately, we have developed an artificial intelligence (AI)-based OR fire virtual trainer based on the principle of the "fire triangle" and SAGES FUSE curriculum. The simulator can predict the user's actions in the virtual OR and provide them with timely feedback to assist with training. We conducted a study investigating the validity of the AI-assisted OR fire trainer at the 2019 SAGES Learning Center. METHODS: Fifty-three participants with varying medical experience were voluntarily recruited to participate in our Institutional Review Board approved study. All participants were asked to contain a fire within the virtual OR. Participants were then asked to fill out a 7-point Likert questionnaire consisting of ten questions regarding the face validation of the AI-assisted OR fire simulator. Shapiro-Wilk tests were conducted to test normality of the scores for each trial. A Friedman's ANOVA with post hoc tests was used to evaluate the effect of multiple trials on performance. RESULTS: On a 7-point scale, eight of the ten questions were rated a mean of 6 or greater (72.73%), especially those relating to the usefulness of the simulator for OR fire-containing training. 79.25% of the participants rated the degree of usefulness of AI guidance over 6 out of 7. The performance of individuals improved significantly over the five trials, χ2(4) = 119.89, p < .001, and there was a significant linear trend of performance r = .97, p = 0.006. A pairwise analysis showed that only after the introduction of AI did performance improve significantly. CONCLUSIONS: The AI-guided OR fire trainer offers the potential to assess OR personnel and teach the proper response to an iatrogenic fire scenario in a safe, repeatable, immersive environment.

Validation of a Virtual Simulator With Haptic Feedback for Endotracheal Intubation Training.

INTRODUCTION: Endotracheal intubation (ETI) is a procedure that varies in difficulty because of patient characteristics and clinical conditions. Existing physical simulators do not encompass these variations. The Virtual Airway Skills Trainer for Endotracheal Intubation (VAST-ETI) was developed to provide different patient characteristics and high-fidelity haptic feedback to improve training. METHODS: We demonstrate the effectiveness of VAST-ETI as a training and evaluation tool for ETI. Construct validation was evaluated by scoring the performance of experts ( N = 15) and novices ( N = 15) on the simulator to ensure its ability to distinguish technical proficiency. Convergent and predictive validity were evaluated by performing a learning curve study, in which a group of novices ( N = 7) were trained for 2 weeks using VAST-ETI and then compared with a control group ( N = 9). RESULTS: The VAST-ETI was able to distinguish between expert and novice based on mean simulator scores ( t [88] = -6.61, P < 0.0005). When used during repeated practice, individuals demonstrated a significant increase in their score on VAST-ETI over the learning period ( F [11,220] = 7206, P < 0.001); however when compared with a control group, there was not a significant interaction effect on the simulator score. There was a significant difference between the simulator-trained and control groups ( t [12.85] = -2.258, P = 0.042) when tested in the operating room. CONCLUSIONS: Our results demonstrate the effectiveness of virtual simulation with haptic feedback for assessing performance and training of ETI. The simulator was not able to differentiate performance between more experienced trainees and experts because of limits in simulator difficulty.

Disruptions to shared mental models from poor quality of service in collaborative virtual environments.

Collaborative virtual environments are being used in various applications ranging from online games to complex team training scenarios. The key to the success of such environments is the ability of the participants to form a shared mental model of the collaborative task being performed. Poor quality of service can deteriorate user performance and quality of experience, leading to a disruption of this mental model. While the effects of quality of service have been analyzed for traditional desktop environments, these effects remain unclear in collaborative virtual environments during user-to-user interactions. Here, we analyze the role of latency and packet bursts, two common problems in collaborative applications, on both simulator perception and actual task performance in a collaborative fire-fighting simulator. This exploratory study indicates that large latencies have a significant (p < 0.05) impact on the quality of experience, but not task performance. In contrast, packet bursts have a much larger impact on both the quality of experience and performance. Additionally, the network role, such as whether a user is a client or server, showed a significant (p < 0.05) impact on task performance in conditions impaired by packet bursts.

Development of an Endotracheal Intubation Formative Assessment Tool.

BACKGROUND: Valid methods for providing detailed formative feedback on direct laryngoscopy and endotracheal intubation (ETI) performance do not exist. We are developing an observation-based assessment tool for measuring performance and providing feedback during ETI. METHODS: Based on the literature and interviews of experts, we proposed an initial ETI metric with 22 items. Six anesthesiology experts used it to assess the quality of ETI performance in videotaped intubations. Following metric revisions, 2 expert groups assessed 2 collections of videos (27 total) using the revised metric. Two reference standards for comparison with metric scores were created with a third and fourth group of experts; (1) an average global rating (1-100) of each ETI performance and (2) average rank-ordered performance from best to worst. Rater agreement and correlations between the 2 methods were calculated. Regression analysis determined items that optimally discriminated quality. When calculating a score based on all clinically important terms, multiple weightings were evaluated. RESULTS: Metric items had high average rater agreement (80%) with intraclass correlation coefficients averaging 0.83. Correlations of the reference rank and score were high for both video collections (-0.96, P < .05, and -0.95, P < .05). Regression coefficients for different item weighting methods indicated strong relationships with global ratings (averaging r = 0.89, P < .05) and rankings averaging -0.85, P < .05). Prediction of global ratings using regression achieved high accuracy (R 2 = 0.8218). CONCLUSIONS: High observer agreement and strong correlations between metric and rank data support the validity of using this metric to assess ETI performance. Different weighting models yielded scores that correlated strongly with the ratings and ranks from global assessment. When using the metric to predict competency, a 3-item regression model is most accurate in predicting a global score.

3D force/torque characterization of emergency cricothyroidotomy procedure using an instrumented scalpel.

Emergency Cricothyroidotomy (CCT) is a surgical procedure performed to secure a patient's airway. This high-stakes, but seldom-performed procedure is an ideal candidate for a virtual reality simulator to enhance physician training. For the first time, this study characterizes the force/torque characteristics of the cricothyroidotomy procedure, to guide development of a virtual reality CCT simulator for use in medical training. We analyze the upper force and torque thresholds experienced at the human-scalpel interface. We then group individual surgical cuts based on style of cut and cut medium and perform a regression analysis to create two models that allow us to predict the style of cut performed and the cut medium.

Virtual Airway Skills Trainer (VAST) Simulator.

This paper presents a simulation of Virtual Airway Skill Trainer (VAST) tasks. The simulated tasks are a part of two main airway management techniques; Endotracheal Intubation (ETI) and Cricothyroidotomy (CCT). ETI is a simple nonsurgical airway management technique, while CCT is the extreme surgical alternative to secure the airway of a patient. We developed identification of Mallampati class, finding the optimal angle for positioning pharyngeal/mouth axes tasks for ETI and identification of anatomical landmarks and incision tasks for CCT. Both ETI and CCT simulators were used to get physicians' feedback at Society for Education in Anesthesiology and Association for Surgical Education spring meetings. In this preliminary validation study, total 38 participants for ETI and 48 for CCT performed each simulation task and completed pre and post questionnaires. In this work, we present the details of the simulation for the tasks and also the analysis of the collected data from the validation study.