From the simulation lab to the operating room: simulation performance predicts intraoperative performance in robotic gastrojejunostomy.

BACKGROUND: Simulation and video-based assessment (VBA) offer residents the opportunity to develop operative skills while ensuring patient safety. This study aims to determine whether simulation training can predict residents' operative performance, focusing on the gastrojejunal (GJ) anastomosis during robotic pancreatoduodenectomy. METHODS: Twenty-seven general surgery residents completed simulated robotic GJ drills and subsequently performed GJs in the operating room (OR). Both simulated and intraoperative performances were video recorded and retrospectively assessed by two blinded graders using the Objective Structural Assessment of Technical Skills (OSATS) scale, time to completion, and occurrence of errors. Intraoperative GJ OSATS scores were compared in cases with and without Clinically Relevant Delayed Gastric Emptying (CRDGE). Statistical analysis was performed using Spearman's rho, Chi-square, and Kruskal-Wallis tests. RESULTS: For simulated GJs, the median OSATS score was 29 (IQR 27-33), time to completion was 30 min (IQR 27-35), and 11 cases had at least one error. Intraoperative GJs had a median OSATS of 30 (IQR 27-31), time to completion of 41 min (IQR 36-51), and errors occurred in nine cases. The OSATS score on the simulated GJs demonstrated a significant positive correlation to the OSATS score on the operative GJs (r = 0.74; p < 0.001) and less time to completion (r = - 0.68; p < 0.001). A shorter simulated GJ completion time significantly correlated with a higher intraoperative OSATS score (r = - 0.52; p < 0.01). Residents with at least one error in the simulated GJs had lower OSATS scores and higher times intraoperatively. Those cases with CRDGE had significantly lower intraoperative OSATS scores than those without CRDGE. CONCLUSION: Performance on a simulated robotic GJ environment is a robust predictor of OR GJ performance, demonstrating predictive validity. VBA of residents' operative GJ performance is associated with the presentation of CRDGE. Simulation-based training may be crucial to optimizing surgical outcomes before operating on patients.

Validity of task-specific metrics for assessment in perineal proctectomy.

BACKGROUND: Perineal proctectomy is a complex procedure that requires advanced skills. Currently, there are no simulators for training in this procedure. As part of our objective of developing a virtual reality simulator, our goal was to develop and validate task-specific metrics for the assessment of performance for this procedure. We conducted a three-phase study to establish task-specific metrics, obtain expert consensus on the appropriateness of the developed metrics, and establish the discriminant validity of the developed metrics. METHODS: In phase I, we utilized hierarchical task analysis to formulate the metrics. In phase II, a survey involving expert colorectal surgeons determined the significance of the developed metrics. Phase III was aimed at establishing the discriminant validity for novices (PGY1-3) and experts (PGY4-5 and faculty). They performed a perineal proctectomy on a rectal prolapse model. Video recordings were independently assessed by two raters using global ratings and task-specific metrics for the procedure. Total scores for both metrics were computed and analyzed using the Kruskal-Wallis test. A Mann-Whitney U test with Benjamini-Hochberg correction was used to evaluate between-group differences. Spearman's rank correlation coefficient was computed to assess the correlation between global and task-specific scores. RESULTS: In phase II, a total of 23 colorectal surgeons were recruited and consensus was obtained on all the task-specific metrics. In phase III, participants (n = 22) included novices (n = 15) and experts (n = 7). There was a strong positive correlation between the global and task-specific scores (rs = 0.86; P < 0.001). Significant between-group differences were detected for both global (χ2 = 15.38; P < 0.001; df = 2) and task-specific (χ2 = 11.38; P = 0.003; df = 2) scores. CONCLUSIONS: Using a biotissue rectal prolapse model, this study documented high IRR and significant discriminant validity evidence in support of video-based assessment using task-specific metrics.

The development of a deep learning model for automated segmentation of the robotic pancreaticojejunostomy.

BACKGROUND: Minimally invasive surgery provides an unprecedented opportunity to review video for assessing surgical performance. Surgical video analysis is time-consuming and expensive. Deep learning provides an alternative for analysis. Robotic pancreaticoduodenectomy (RPD) is a complex and morbid operation. Surgeon technical performance of pancreaticojejunostomy (PJ) has been associated with postoperative pancreatic fistula. In this work, we aimed to utilize deep learning to automatically segment PJ RPD videos. METHODS: This was a retrospective review of prospectively collected videos from 2011 to 2022 that were in libraries at tertiary referral centers, including 111 PJ videos. Each frame of a robotic PJ video was categorized based on 6 tasks. A 3D convolutional neural network was trained for frame-level visual feature extraction and classification. All the videos were manually annotated for the start and end of each task. RESULTS: Of the 100 videos assessed, 60 videos were used for the training the model, 10 for hyperparameter optimization, and 30 for the testing of performance. All the frames were extracted (6 frames/second) and annotated. The accuracy and mean per-class F1 scores were 88.01% and 85.34% for tasks. CONCLUSION: The deep learning model performed well for automated segmentation of PJ videos. Future work will focus on skills assessment and outcome prediction.

Validity and reliability evidence support task-specific metrics for laparoscopic fundoplication.

BACKGROUND: Laparoscopic hiatal hernia repair (LHHR) is a complex operation requiring advanced surgical training. Surgical simulation offers a potential solution for learning complex operations without the need for high surgical volume. Our goal is to develop a virtual reality (VR) simulator for LHHR; however, data supporting task-specific metrics for this procedure are lacking. The purpose of this study was to develop and assess validity and reliability evidence of task-specific metrics for the fundoplication phase of LHHR. METHODS: In phase I, structured interviews with expert foregut surgeons were conducted to develop task-specific metrics (TSM). In phase II, participants with varying levels of surgical expertise performed a laparoscopic Nissen fundoplication procedure on a porcine stomach explant. Video recordings were independently assessed by two blinded graders using global and TSM. An intraclass correlation coefficient (ICC) was used to assess interrater reliability (IRR). Performance scores were compared using a Kruskal-Wallis test. Spearman's rank correlation was used to evaluate the association between global and TSM. RESULTS: Phase I of the study consisted of 12 interviews with expert foregut surgeons. Phase II engaged 31 surgery residents, a fellow, and 6 attendings in the simulation. Phase II results showed high IRR for both global (ICC = 0.84, p < 0.001) and TSM (ICC = 0.75, p < 0.001). Significant between-group differences were detected for both global (χ2 = 24.01, p < 0.001) and TSM (χ2 = 18.4, p < 0.001). Post hoc analysis showed significant differences in performance between the three groups for both metrics (p < 0.05). There was a strong positive correlation between the global and TSM (rs = 0.86, p < 0.001). CONCLUSION: We developed task-specific metrics for LHHR and using a fundoplication model, we documented significant reliability and validity evidence. We anticipate that these LHHR task-specific metrics will be useful in our planned VR simulator.

Automated segmentation of phases, steps, and tasks in laparoscopic cholecystectomy using deep learning.

BACKGROUND: Video-based review is paramount for operative performance assessment but can be laborious when performed manually. Hierarchical Task Analysis (HTA) is a well-known method that divides any procedure into phases, steps, and tasks. HTA requires large datasets of videos with consistent definitions at each level. Our aim was to develop an AI model for automated segmentation of phases, steps, and tasks for laparoscopic cholecystectomy videos using a standardized HTA. METHODS: A total of 160 laparoscopic cholecystectomy videos were collected from a publicly available dataset known as cholec80 and from our own institution. All videos were annotated for the beginning and ending of a predefined set of phases, steps, and tasks. Deep learning models were then separately developed and trained for the three levels using a 3D Convolutional Neural Network architecture. RESULTS: Four phases, eight steps, and nineteen tasks were defined through expert consensus. The training set for our deep learning models contained 100 videos with an additional 20 videos for hyperparameter optimization and tuning. The remaining 40 videos were used for testing the performance. The overall accuracy for phases, steps, and tasks were 0.90, 0.81, and 0.65 with the average F1 score of 0.86, 0.76 and 0.48 respectively. Control of bleeding and bile spillage tasks were most variable in definition, operative management, and clinical relevance. CONCLUSION: The use of hierarchical task analysis for surgical video analysis has numerous applications in AI-based automated systems. Our results show that our tiered method of task analysis can successfully be used to train a DL model.

Adapting to a Robotic Era: The Transferability of Open and Laparoscopic Skills to Robotic Surgery.

BACKGROUND: The learning curve of robotic surgical skills is poorly understood. There is a lack of data on the transferability of skills from open and laparoscopic training to robotic surgery. In this retrospective cohort study, we investigated the impact of training acquired during intern year on the development of robotic skills in general surgery residents, prior to formal robotic training. METHODS: Between 2019 and 2021, novice general surgery residents underwent robotic skill assessment using 3 validated inanimate drills before starting intern year. After completing basic open and laparoscopic proficiency-based curricula, they completed an identical robotic skill assessment at the end of intern year. Pre and post intern year robotic performances were recorded and analyzed by 2 blinded graders. Video-based assessment included completion time, errors, and the modified Objective Structured Assessment of Technical Skills (mOSATS) score. RESULTS: The total time needed to complete all 3 robotic drills decreased from a mean of 26 to 17 minutes after intern year (p < 0.001). The number of errors decreased from a mean of 2.16 to 0.56 errors per subject (p < 0.001). The aggregated mOSATS score increased by an average of 41% (p < 0.001), with a greater increase in technical skill domains compared to the knowledge-based domain. The interrater intraclass correlation coefficient was 0.91. CONCLUSIONS: Baseline robotic surgical skills are limited without formal training. Our findings suggest that acquiring basic open and laparoscopic skills, such as knot tying, needle driving, and tissue handling results in improved performance on the robotic platform, prior to formal robotic training. Therefore, requiring trainees to complete fundamental open and laparoscopic training prior to robotic training may be an efficient and effective strategy within a surgical residency curriculum.

Baseline performance in a robotic virtual reality platform predicts rate of skill acquisition in a proficiency-based curriculum: a cohort study of surgical trainees.

BACKGROUND: Residency programs must prepare to train the next generation of surgeons on the robotic platform. The purpose of this study was to determine if baseline skills of residents on a virtual reality (VR) robotic simulator before intern year predicted future performance in a proficiency-based curriculum. METHODS: Across two academic years, 21 general surgery PGY-1s underwent the robotic surgery boot camp at the University of Texas Southwestern. During boot camp, subjects completed five previously validated VR tasks, and their performance metrics (score, time, and economy of motion [EOM]) were extracted retrospectively from their Intuitive learning accounts. The same metrics were assessed during their residency until they reached previously validated proficiency benchmarks. Outcomes were defined as the score at proficiency, attempts to reach proficiency, and time to proficiency. Spearman's rho and Mann-Whitney U tests were used; median (IQR) was reported. Significance level was set at p < 0.05. RESULTS: Twenty-one residents completed at least three out of the five boot camp tasks and achieved proficiency in the former during residency. The median average score at boot camp was 12.3 (IQR: 5.14-18.5). The median average EOM at boot camp was 599.58 cm (IQR: 529.64-676.60). The average score at boot camp significantly correlated with lower time to achieve proficiency (p < 0.05). EOM at boot camp showed a significant correlation with attempts to proficiency and time to proficiency (p < 0.01). Residents with an average baseline EOM below the median showed a significant difference in attempts to proficiency (p < 0.05) and time to proficiency (p < 0.05) compared to those with EOMs above or equal to the median. CONCLUSION: Residents with an innate ability to perform tasks with better EOM may acquire robotic surgery skills faster. Future investigators could explore how these innate differences impact performance throughout residency.

What are essential laparoscopic skills these days? Results of the SAGES Fundamentals of Laparoscopic Surgery (FLS) Committee technical skills survey.

INTRODUCTION: The Fundamentals of Laparoscopic Surgery (FLS) program tests basic knowledge and skills required to perform laparoscopic surgery. Educational experiences in laparoscopic training and development of associated competencies have evolved since FLS inception, making it important to review the definition of fundamental laparoscopic skills. The Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) assigned an FLS Technical Skills Working Group to characterize technical skills used in basic laparoscopic surgery in current practice contexts and their possible application to future FLS tests. METHODS: A group of subject matter experts defined an inventory of 65 laparoscopic skills using a Nominal Group Technique. From these, a survey was developed rating these items for importance, frequency of use, and priority for testing for FLS certification. This survey was distributed to SAGES members, recent recipients of FLS certification, and members of the Association of Program Directors in Surgery (APDS). Results were collected using a secure web-based survey platform. RESULTS: Complete data were available for 1742 surveys. Of these, 1143 comprised results for post-residency participants who performed advanced procedures. Seventeen competencies were identified for FLS testing prioritization by determining the proportion of respondents who identified them of highest priority, at median (50th percentile) of the maximum survey scale rating. These included basic peritoneal access, laparoscope and instrument use, tissue manipulation, and specific problem management skills. Sixteen could be used to show appropriateness of the domain construct by confirmatory factor analysis. Of these 8 could be characterized as manipulative tasks. Of these 5 mapped to current FLS tasks. CONCLUSIONS: This survey-identified competencies, some of which are currently assessed in FLS, with a high level of priority for testing. Further work is needed to determine if this should prompt consideration of changes or additions to the FLS technical skills test component.

Objective metrics for hand-sewn bowel anastomoses can differentiate novice from expert surgeons.

BACKGROUND: Assessing performance automatically in a virtual reality trainer or from recorded videos is advantageous but needs validated objective metrics. The purpose of this study is to obtain expert consensus and validate task-specific metrics developed for assessing performance in double-layered end-to-end anastomosis. MATERIALS AND METHODS: Subjects were recruited into expert (PGY 4-5, colorectal surgery residents, and attendings) and novice (PGY 1-3) groups. Weighted average scores of experts for each metric item, completion time, and the total scores computed using global and task-specific metrics were computed for assessment. RESULTS: A total of 43 expert surgeons rated our task-specific metric items with weighted averages ranging from 3.33 to 4.5 on a 5-point Likert scale. A total of 20 subjects (10 novices and 10 experts) participated in validation study. The novice group completed the task significantly more slowly than the experienced group (37.67 ± 7.09 vs 25.47 ± 7.82 min, p = 0.001). In addition, both the global rating scale (23.47 ± 4.28 vs 28.3 ± 3.85, p = 0.016) and the task-specific metrics showed a significant difference in performance between the two groups (38.77 ± 2.83 vs 42.58 ± 4.56 p = 0.027) following partial least-squares (PLS) regression. Furthermore, PLS regression showed that only two metric items (Stay suture tension and Tool handling) could reliably differentiate the performance between the groups (20.41 ± 2.42 vs 24.28 ± 4.09 vs, p = 0.037). CONCLUSIONS: Our study shows that our task-specific metrics have significant discriminant validity and can be used to evaluate the technical skills for this procedure.

Developing artificial intelligence models for medical student suturing and knot-tying video-based assessment and coaching.

BACKGROUND: Early introduction and distributed learning have been shown to improve student comfort with basic requisite suturing skills. The need for more frequent and directed feedback, however, remains an enduring concern for both remote and in-person training. A previous in-person curriculum for our second-year medical students transitioning to clerkships was adapted to an at-home video-based assessment model due to the social distancing implications of COVID-19. We aimed to develop an Artificial Intelligence (AI) model to perform video-based assessment. METHODS: Second-year medical students were asked to submit a video of a simple interrupted knot on a penrose drain with instrument tying technique after self-training to proficiency. Proficiency was defined as performing the task under two minutes with no critical errors. All the videos were first manually rated with a pass-fail rating and then subsequently underwent task segmentation. We developed and trained two AI models based on convolutional neural networks to identify errors (instrument holding and knot-tying) and provide automated ratings. RESULTS: A total of 229 medical student videos were reviewed (150 pass, 79 fail). Of those who failed, the critical error distribution was 15 knot-tying, 47 instrument-holding, and 17 multiple. A total of 216 videos were used to train the models after excluding the low-quality videos. A k-fold cross-validation (k = 10) was used. The accuracy of the instrument holding model was 89% with an F-1 score of 74%. For the knot-tying model, the accuracy was 91% with an F-1 score of 54%. CONCLUSIONS: Medical students require assessment and directed feedback to better acquire surgical skill, but this is often time-consuming and inadequately done. AI techniques can instead be employed to perform automated surgical video analysis. Future work will optimize the current model to identify discrete errors in order to supplement video-based rating with specific feedback.

The Operating Room Black Box: Understanding Adherence to Surgical Checklists.

OBJECTIVE: We report for the first time the use of the Operating Room Black Box (ORBB) to track checklist compliance, engagement, and quality. BACKGROUND: Implementation of operative checklists is associated with improved outcomes. Compliance is difficult to monitor. Most studies report either no assessment of checklist compliance or deployed in-person short-term assessment. The ORBB a novel artificially intelligence-driven data analytic platform affords the opportunity to assess checklist compliance without disrupting surgical workflow. METHODS: This was a retrospective review of prospectively collected ORBB data. Operative cases included elective surgery at a quaternary referral center. Cases were analyzed as prepolicy change (first 9 months) or as a postpolicy change (last 9 months). Measures of checklist compliance, engagement, and quality were assessed. RESULTS: There were 3879 cases that were performed and monitored for checklist compliance between August 15, 2020, and February 20, 2022. The overall scores for compliance, engagement, and quality were 81%, 84%, and 67% respectively. When broken down by phase, the scores for time-out were compliance 100%, engagement 98%, and quality 61%. Scores for the debrief phase were 81% for compliance, 98% for engagement, and 66% for quality. After a hospital policy change, the debrief scores improved significantly (85%; P <0.001 for compliance, 88%; P <0.001 for engagement and 71%; P <0.001 for quality). CONCLUSIONS: ORBB provides the unprecedented ability to assess not only compliance with surgical safety checklists but also engagement and quality. Utilization of this technology allows the assessment of compliance in near real time and to accurately address safety threats that may arise from noncompliance.

Development and Validation of Task-Specific Metrics for the Assessment of Linear Stapler-Based Small Bowel Anastomosis.

INTRODUCTION: Task-specific metrics facilitate the assessment of surgeon performance. This 3-phased study was designed to (1) develop task-specific metrics for stapled small bowel anastomosis, (2) obtain expert consensus on the appropriateness of the developed metrics, and (3) establish its discriminant validity. METHODS: In Phase I, a hierarchical task analysis was used to develop the metrics. In Phase II, a survey of expert colorectal surgeons established the importance of the developed metrics. In Phase III, to establish discriminant validity, surgical trainees and surgeons, divided into novice and experienced groups, constructed a side-to-side anastomosis on porcine small bowel using a linear cutting stapler. The participants' performances were videotaped and rated by 2 independent observers. Partial least squares regression was used to compute the weights for the task-specific metrics to obtain weighted total score. RESULTS: In Phase II, a total of 45 colorectal surgeons were surveyed: 28 with more than 15 years, 13 with 5 to 15 years, and 4 with less than 5 years of experience. The consensus was obtained on all the task-specific metrics in the more experienced groups. In Phase III, 20 subjects participated equally in both groups. The experienced group performed better than the novice group regardless of the rating scale used: global rating scale (p = 0.009) and the task-specific metrics (p = 0.012). After partial least squares regression, the weighted task-specific metric score continued to show that the experienced group performed better (p < 0.001). CONCLUSION: Task-specific metric items were developed based on expert consensus and showed good discriminant validity compared with a global rating scale between experienced and novice operators. These items can be used for evaluating technical skills in a stapled small bowel anastomosis model.

Training on a virtual reality cricothyroidotomy simulator improves skills and transfers to a simulated procedure.

Objective: The virtual airway skills trainer (VAST) is a virtual reality simulator for training in cricothyroidotomy (CCT). The goal of the study is to test the effectiveness of training and transfer of skills of the VAST-CCT. Methods: Two groups, control (no training) and simulation (2 weeks of proficiency-based training), participated in this study. Subjects in the control condition did not receive any training on the task whereas those in the simulation received a proficiency-based training on the task during a period of 2 weeks. Two weeks post-training, both groups performed CCT on the TraumaMan to demonstrate the transfer of skills. Results: A total of (n=20) subjects participated in the study. The simulation group performed better than the control group at both the post-test (p<0.001) and retention test (p<0.001) on the simulator. The cumulative sum analysis showed that all subjects in the simulation group reached proficiency with acceptable failure rate within the 2 weeks of training. On the transfer test, the simulation group performed better on skin cut (p<0.001), intubation (p<0.001) and total score (p<0.001) than the control group. Conclusions: The VAST-CCT is effective in training and skills transfer for the CCT procedure. Level of evidence: Not applicable. Simulator validation study.

Feasibility, effectiveness and transferability of a novel mastery-based virtual reality robotic training platform for general surgery residents.

BACKGROUND: The annual number of robotic surgical procedures is on the rise. Robotic surgery requires unique skills compared to other surgical approaches. Simulation allows basic robot skill acquisition and enhances patient safety. The purpose of this study was to evaluate the feasibility, effectiveness, and transferability of a mastery-based curriculum using a new virtual reality (VR) robotic simulator for surgery resident training. METHODS: Nineteen PGY2s and 22 PGY4s were enrolled. Residents completed a pretest and posttest consisting of five VR and three previously validated inanimate tasks. Training included practicing 33 VR tasks until a total score ≥ 90% ("mastery") was achieved using automated metrics (time, economy of motion). Inanimate performance was evaluated by two trained, blinded raters using video review metrics (time, errors, and modified OSATS). Outcomes were defined as: curriculum feasibility (completion rate, training time, repetitions), training effectiveness (pre/post training skill improvement), and skill transferability (skill transfer to validated inanimate drills). Wilcoxon signed-rank and Mann-Whitney U tests were used; median (IQR) reported. RESULTS: Thirty-four of 41 residents (83%) achieved mastery on all 33 VR tasks; median training time was 7 h (IQR: 5'26″-8'52″). Pretest vs. post-test performance improved (all p < 0.001) according to all VR and Inanimate metrics for both PGY2 and PGY4 residents. Significant pretest performance differences were observed between PGY2 and PGY4 residents for VR but not inanimate tasks; no PGY2 vs. PGY4 posttest performance differences were observed for both VR and inanimate tasks. CONCLUSION: This mastery-based VR curriculum was associated with a high completion rate and excellent feasibility. Significant performance improvements were noted for both the VR and inanimate tasks, supporting training effectiveness and skill transferability. Additional studies examining validity evidence may help further refine this curriculum.

Artificial Intelligence for Intraoperative Guidance: Using Semantic Segmentation to Identify Surgical Anatomy During Laparoscopic Cholecystectomy.

OBJECTIVE: The aim of this study was to develop and evaluate the performance of artificial intelligence (AI) models that can identify safe and dangerous zones of dissection, and anatomical landmarks during laparoscopic cholecystectomy (LC). SUMMARY BACKGROUND DATA: Many adverse events during surgery occur due to errors in visual perception and judgment leading to misinterpretation of anatomy. Deep learning, a subfield of AI, can potentially be used to provide real-time guidance intraoperatively. METHODS: Deep learning models were developed and trained to identify safe (Go) and dangerous (No-Go) zones of dissection, liver, gallbladder, and hepatocystic triangle during LC. Annotations were performed by 4 high-volume surgeons. AI predictions were evaluated using 10-fold cross-validation against annotations by expert surgeons. Primary outcomes were intersection- over-union (IOU) and F1 score (validated spatial correlation indices), and secondary outcomes were pixel-wise accuracy, sensitivity, specificity, ± standard deviation. RESULTS: AI models were trained on 2627 random frames from 290 LC videos, procured from 37 countries, 136 institutions, and 153 surgeons. Mean IOU, F1 score, accuracy, sensitivity, and specificity for the AI to identify Go zones were 0.53 (±0.24), 0.70 (±0.28), 0.94 (±0.05), 0.69 (±0.20). and 0.94 (±0.03), respectively. For No-Go zones, these metrics were 0.71 (±0.29), 0.83 (±0.31), 0.95 (±0.06), 0.80 (±0.21), and 0.98 (±0.05), respectively. Mean IOU for identification of the liver, gallbladder, and hepatocystic triangle were: 0.86 (±0.12), 0.72 (±0.19), and 0.65 (±0.22), respectively. CONCLUSIONS: AI can be used to identify anatomy within the surgical field. This technology may eventually be used to provide real-time guidance and minimize the risk of adverse events.

A contextual detector of surgical tools in laparoscopic videos using deep learning.

BACKGROUND: The complexity of laparoscopy requires special training and assessment. Analyzing the streaming videos during the surgery can potentially improve surgical education. The tedium and cost of such an analysis can be dramatically reduced using an automated tool detection system, among other things. We propose a new multilabel classifier, called LapTool-Net to detect the presence of surgical tools in each frame of a laparoscopic video. METHODS: The novelty of LapTool-Net is the exploitation of the correlations among the usage of different tools and, the tools and tasks-i.e., the context of the tools' usage. Towards this goal, the pattern in the co-occurrence of the tools is utilized for designing a decision policy for the multilabel classifier based on a Recurrent Convolutional Neural Network (RCNN), which is trained in an end-to-end manner. In the post-processing step, the predictions are corrected by modeling the long-term tasks' order with an RNN. RESULTS: LapTool-Net was trained using publicly available datasets of laparoscopic cholecystectomy, viz., M2CAI16 and Cholec80. For M2CAI16, our exact match accuracies (when all the tools in one frame are predicted correctly) in online and offline modes were 80.95% and 81.84% with per-class F1-score of 88.29% and 90.53%. For Cholec80, the accuracies were 85.77% and 91.92% with F1-scores if 93.10% and 96.11% for online and offline, respectively. CONCLUSIONS: The results show LapTool-Net outperformed state-of-the-art methods significantly, even while using fewer training samples and a shallower architecture. Our context-aware model does not require expert's domain-specific knowledge, and the simple architecture can potentially improve all existing methods.

Developing an immersive virtual reality medication administration scenario using the nominal group technique.

AIM: This paper aims to describe how the Nominal Group Technique was applied to obtain focused content to develop medication administration error scenarios for future use to educate practicing RNs with immersive virtual reality simulation. BACKGROUND: In the United States, medication errors account for up to $46 million in daily loss to hospital operational budgets. Each phase of prescribing, dispensing, administration, monitoring, and reconciliation is crucial in reducing potentially life-threatening outcomes associated with medication errors. Registered Nurses are responsible for safely administering diverse classifications of medications to patients in various healthcare settings. However, human and system factors can contribute to the exposure of hospitalized patients to a medication error. Virtual reality simulation-based education can be a methodology to educate practicing Registered Nurses on safe medication practices. DESIGN: A Nominal Group Technique process was used to generate consensus from participating Registered Nurses on human and system factors that can contribute to medication administration errors. METHODS: The process consisted of (a) preparation, (b) running the group with an introduction of the subject, (c) generation of ideas, (d) listing of ideas, (e) discussion of ideas, (f) ranking of top ideas, (g) voting on top ideas, (h) discussion of the vote outcome, and (i) re-ranking and rating the top items. Human and system factor idea items encompassed medication errors during ordering, prescribing, or administering medications. Both novice and experienced Registered Nurses rank-ordered these factors as those most likely to encounter or which would most likely occur during one working shift. RESULTS: Descriptive statistics of frequencies and percentages were used to analyze the findings when grouped by human and system factor categories. Non-parametric testing with a Kruskal-Wallis test was conducted to compare the human and system factors by categories and years of Registered Nurse experience. Findings revealed that the factors of Time Management: getting behind, hurried, urgent (KW-H 11.2, df 4, p = .025) and Right Medication: medications have similar look and sound-alike names (KW-H 11.1, df 4, p = .025) impacted safe medication administration for both the novice and experienced nurse. CONCLUSION: The NGT process identified human and system factors contributing to errors and impacting safe medication administration practices. Findings will support the creation of medication administration scenarios for use with immersive virtual reality simulation.

Clinical Features of Emergency Department Patients from Early COVID-19 Pandemic that Predict SARS-CoV-2 Infection: Machine-learning Approach.

INTRODUCTION: Within a few months coronavirus disease 2019 (COVID-19) evolved into a pandemic causing millions of cases worldwide, but it remains challenging to diagnose the disease in a timely fashion in the emergency department (ED). In this study we aimed to construct machine-learning (ML) models to predict severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2) infection based on the clinical features of patients visiting an ED during the early COVID-19 pandemic. METHODS: We retrospectively collected the data of all patients who received reverse transcriptase polymerase chain reaction (RT-PCR) testing for SARS-CoV-2 at the ED of Baylor Scott & White All Saints Medical Center, Fort Worth, from February 23-May 12, 2020. The variables collected included patient demographics, ED triage data, clinical symptoms, and past medical history. The primary outcome was the confirmed diagnosis of COVID-19 (or SARS-CoV-2 infection) by a positive RT-PCR test result for SARS-CoV-2, and was used as the label for ML tasks. We used univariate analyses for feature selection, and variables with P<0.1 were selected for model construction. Samples were split into training and testing cohorts on a 60:40 ratio chronologically. We tried various ML algorithms to construct the best predictive model, and we evaluated performances with the area under the receiver operating characteristic curve (AUC) in the testing cohort. RESULTS: A total of 580 ED patients were tested for SARS-CoV-2 during the study periods, and 98 (16.9%) were identified as having the SARS-CoV-2 infection based on the RT-PCR results. Univariate analyses selected 21 features for model construction. We assessed three ML methods for performance: of the three methods, random forest outperformed the others with the best AUC result (0.86), followed by gradient boosting (0.83) and extra trees classifier (0.82). CONCLUSION: This study shows that it is feasible to use ML models as an initial screening tool for identifying patients with SARS-CoV-2 infection. Further validation will be necessary to determine how effectively this prediction model can be used prospectively in clinical practice.

Simulation for Colorectal Surgery.

Introduction: Colorectal surgery is a highly specialized field in surgery that deals with the surgical intervention of disease processes of the colon, rectum, and anus. Gaining proficiency in this field requires training both inside and outside of the operating room. Simulation plays a key role in training surgeons in colorectal surgery. The goal of this study is to review the currently available simulators for training in the field of colorectal surgery. Methods: A review of the literature was conducted to identify simulators that are both physical such as benchtop, live animal, and cadaver, as wells as virtual reality (VR) simulators. Any reported validity evidence for these simulators were also presented. Results: There are several benchtop physical models made of silicone for training in basic colorectal tasks, such as hand-sewn and stapled anastomosis. To improve realism, explanted animal and cadaveric specimens were also used for training. To improve repeatability, objective assessment, both commercial and VR simulators also exist for training in both open and laparoscopic colorectal surgery and emerging areas such as endoscopic submucosal dissection. Conclusion: Simulation-based training in colorectal surgery is here to stay and is going to play a significant role in training, credentialing, and quality improvements.

Training with cognitive load improves performance under similar conditions in a real surgical task.

BACKGROUND: Enhancing cognitive load while performing a bimanual surgical task affects performance. Whether repeated training under this condition could benefit performance in an operating room was tested using a virtual reality simulator with cognitive load applied through two-digit math multiplication questions. METHOD: 11 subjects were randomized to Control, VR and VR + CL groups. After a pre-test, VR and VR + CL groups repeated the peg transfer task 150 times over 15 sessions with cognitive load applied only for the last 100 trials. After training, all groups took a post-test and two weeks later the retention test with and without cognitive load and the transfer task on a pig intestine of 150 cm long under cognitive load. RESULTS AND CONCLUSION: Mixed ANOVA analysis showed significant differences between the control and VR and VR + CL groups (p = 0.013, p = 0.009) but no differences between the VR + CL and the VR groups (p = 1.0). GOALS bimanual dexterity score on transfer test show that VR + CL group outperformed both Control and VR groups (p = 0.016, p = 0.03). Training under cognitive load benefitted performance on an actual surgical task under similar conditions.