Proficiency Levels and Validity Evidence for Scoring Metrics for a Virtual Reality and Inanimate Robotic Surgery Simulation Curriculum.

OBJECTIVE: Our institution recently implemented a virtual reality (VR) skills curriculum for general surgery residents using the SimNow simulator. Based on a content alignment study, we revised the curriculum to include only 20 of 33 VR tasks and we added 3 previously validated inanimate tasks. The purpose of this study was to establish expert-derived proficiency levels for all tasks and to evaluate the validity of the scoring for the VR tasks. DESIGN: Two expert robotic surgeons performed 5 repetitions of each VR and inanimate task. The trimmed mean (lowest scoring attempt and outliers [>2 standard deviations] were eliminated) was defined as the expert level for each task. For the VR tasks, expert levels were compared to resident performance to evaluate validity. SETTING: This study was conducted at the University of Texas Southwestern Medical Center (Dallas, TX), a tertiary care academic teaching hospital. PARTICIPANTS: Two expert robotic surgeons participated in this study. The data from 42 residents (PGY2-4) who completed the original curriculum was used to represent novice performance. RESULTS: Comparison of expert levels and resident performance was statistically significant for 15 VR tasks (supporting validity) and approached significance (p = 0.06, 0.09) for 2 VR tasks; expert levels were designated as proficiency levels for these 17 tasks. Group comparisons were clearly not significant (p = 0.2-0.8) for 3 VR tasks; 2 of these 3 tasks were retained as introductory exercises (with 3 repetitions required) and 1 was excluded. For the 3 inanimate tasks, expert levels minus 2 standard deviations were designated as proficiency levels. CONCLUSIONS: This analysis generated validity evidence for 15 VR tasks and established expert-derived proficiency levels for 17 VR tasks and 3 inanimate tasks. Our proposed curriculum now consists of 19 VR and 3 inanimate tasks using the selected proficiency levels. We anticipate that this design will maximize curriculum efficiency and effectiveness.

Evaluating emergency department tube thoracostomy: A single-center use of trauma video review to assess efficiency and technique.

BACKGROUND: Emergency department tube thoracostomy is a common procedure; however, assessing procedural skills is difficult. We sought to describe procedural variability and technical complications of emergency department tube thoracostomy using trauma video review. We hypothesized that factors such as hemodynamic abnormality lead to increased technical difficulty and malpositioning. METHODS: Using trauma video review, we reviewed all emergency department tube thoracostomy from 2020 to 2022. Patients were stratified into hemodynamically abnormal (systolic blood pressure <90 or heart rate >120) and hemodynamically normal (systolic blood pressure ≥90 or heart rate ≤120). Emergency department tube thoracostomies outside of video-capable rooms, with incomplete visualization, or in patients undergoing cardiopulmonary resuscitation or resuscitative thoracotomy were excluded. The primary outcome was a procedure score modified from the validated tool ranging from 0 to 11 (higher score indicating better performance). Also measured were procedural times to (1) decision to place, (2) pleural entry, and (3) procedure completion. Postprocedure x-ray and chart review were used to determine accuracy. RESULTS: In total, 51 videos met the inclusion criteria. The median age was 34 [interquartile range 24-40] years, body mass index 25.8 [interquartile range 21.8-30.7], predominately male (75%), blunt injury (57%), with Injury Severity Score of 22 [14.5-41]. The median procedure score was 9 [7-10]. Emergency department tube thoracostomies in patients with abnormal hemodynamics had significantly lower procedure scores (8 vs 10, P < .05). Hemodynamically abnormal patients had significantly shorter times from decision to proceed to pleural entry (4.05 vs 8.25 minutes, P < .001), and to completion (6.31 vs 14.23 minutes, P < .001). The most common complication was malpositioning (35.1%), with no significant difference noted when comparing hemodynamically normal and abnormal patients (P = .41). CONCLUSION: Using trauma video review we identified significant procedural variability in emergency department tube thoracostomy, mainly that hemodynamic abnormality led to lower proficiency scores and increased malpositioning. Efforts are needed to define procedural benchmarks and evaluation in the context of patient outcomes. Using this technology and methodology can help establish procedural norms.

Developing a Robotic Surgery Curriculum: Selection of Virtual Reality Drills for Content Alignment.

INTRODUCTION: Despite the importance of simulation-based training for robotic surgery, there is no consensus about its training curricula. Recently, a virtual reality (VR) platform (SimNow, Intuitive, Inc) was introduced with 33 VR drills but without evidence of their validity. As part of our creating a new robotic VR curriculum, we assessed the drills' validity through content mapping and the alignment between learning goals and drill content. METHODS: Three robotically trained surgeons content-mapped all 33 drills for how well the drills incorporated 15 surgery skills and also rated the drills' difficulty, usefulness, relevance, and uniqueness. Drills were added to the new curriculum based on consensus about ratings and historic learner data. The drills were grouped according to similar skill sets and arranged in order of complexity. RESULTS: The 33 drills were judged to have 12/15 surgery skills as primary goals and 13/15 as secondary goals. Twenty of the 33 drills were selected for inclusion in the new curriculum; these had 11/15 skills as primary goals and 11/15 as secondary goals. However, skills regarding energy sources, atraumatic handling, blunt dissection, fine dissection, and running suturing were poorly represented in the drills. Three previously validated inanimate drills were added to the curriculum to address lacking skill domains. CONCLUSIONS: We identified 20 of the 33 SimNow drills as a foundation for a robotic surgery curriculum based on content-oriented evidence. We added 3 other drills to address identified gaps in drill content.

The case for a new post-graduate hernia designation: a review of fellowship council case logs from the past twelve-years.

BACKGROUND: The Fellowship Council (FC) is a robust accreditation body with numerous fellowships; however, no specific criteria exist for hernia fellowships. This study analyzed the case log database to evaluate trends in fellowship exposure to hernia repairs. METHODS: FC hernia case log records (2007-2019) were coded as inguinal or ventral hernias and with or without mesh repair. Retrospective analysis examined total hernia repairs logged, type of repair, program designation, and robotic adoption. Robotic adoption was categorized by quartiles of program performance according to the final year of analysis (2018-2019); yearly performance was then graphed by quartiles. RESULTS: Over this twelve-year period, 93,334 hernia repairs, 5 program designations, 152 unique programs and 1,558 unique fellows were analyzed. The number of fellows grew from 106 (2007-2008) to > 130 (2018-2019). Total hernias repairs per fellow increased from an average of 41.2 in 2007-2008 to 75.7 in 2018-2019 (183.7%). Open and robotic hernia repairs increased by 241.9% and 266.3%, respectively; laparoscopic hernia repairs decreased by 14.8%. Inguinal and ventral hernia repairs comprised 48.1% and 51.9% of total cases, respectively. Advanced GI/MIS and Advanced GI/MIS/Bariatrics programs logged the majority of hernia repairs (86.0-90.2%). 2014 began an exponential rise in robotic adoption, with fellows averaging < 1 robotic repairs before and > 25 repairs in 2019. A significant difference was found between all groups when comparing quartiles of robotic adopters (median robotic repairs per fellow; IQR): first quartile (72.0; 47.9-108.8), second quartile (25.5; 21.0-30.6), third quartile (13.0; 12.0-14.3) and fourth quartile (3.5; 0.5-5.0) (p-value < 0.05). CONCLUSIONS: This twelve-year analysis shows a near doubling in the growth of total hernia repairs, with a decrease in laparoscopic repairs as robotic repairs increased. These data show the importance of hernia repairs in FC fellows' training and warrant further granular analysis to determine specific accreditation criteria for hernia fellowship designations.

Educational Experience Impacts Wellness More than Hours Worked.

OBJECTIVE: The Accreditation Council for Graduate Medical Education has focused its interests on resident wellbeing and the clinical work environment in recent years. Concerns regarding both duty hours as well as service obligations versus education resulted in programs nationwide receiving citations, including ours. This study aimed to evaluate the impact of those 2 factors on surgical residents' general wellbeing, hypothesizing that service obligations would be a stronger predictor. DESIGN: We have previously reported on the use of a "Fuel Gauge" tool developed at our institution for monitoring resident wellbeing. We performed a retrospective comparison of prospectively collected cross-sectional survey data comparing the Fuel Gauge to a bimonthly "Service Versus Education" (SVE) report. This report used similar scaling and allowed residents to provide feedback on the balance of the educational quality of their current rotation in comparison to their perception of service obligation. Pearson's correlation was then used to compare those scores with duty hour logs to determine if a relationship could be identified between the 3 measurements. SETTING: Academic institution of the University of Texas Southwestern in Dallas, Texas. PARTICIPANTS: Active general surgery residents (n = 73). RESULTS: During the study period, 73 residents filled out both a Fuel Gauge assessment and a SVE assessment at least once, with 273 complete data points available for analysis. Our program's Fuel Gauge median was 4, and our program's median SVE score was 4. Fuel Gauge assessment scores demonstrated a moderate positive correlation with SVE (r = 0.65, p < 0.001), while only a weakly negative association with increasing hours worked (r = -0.15, p = 0.015). SVE also demonstrated a weak negative correlation with hours logged (r = -0.225, p = 0.001). CONCLUSIONS: While the Accreditation Council for Graduate Medical Education recognizes that multiple factors contribute to resident wellbeing issues, early efforts were focused on limiting excessive duty hours. Examining our institutional data regarding the previously understudied factor of SVE, we indeed found a stronger correlation with resident perception of low educational value rather than excessive work hours contributing to lower Fuel Gauge scores. These data, if verified, should guide program directors in identifying other institutional factors that may more strongly contribute to their own culture of resident wellness.

Laparoscopic Ambidexterity in Left-Handed Trainees.

INTRODUCTION: There is a paucity of objective data about the advantages or disadvantages of handedness in surgery. Given the need for ambidexterity in laparoscopic surgery, our study aimed to identify the patterns in handedness and performance on basic laparoscopic tasks. METHODS: A retrospective analysis of intern laparoscopic performance on bimanual tasks was assessed for delta time (differences in task time between the dominant and nondominant hand) between left-hand and right-hand dominant interns. RESULTS: Data were analyzed for 16 residents. 25% were left-handed dominant (4/16) and 75% were right-handed dominant (12/16). Of the left-handed surgeons, 75% (3/4) operated primarily with their right hand. There was a significant difference between the time to task completion in Task 2 of left-handed and right-handed residents with median (IQR) time 94 s (90.25-97) and 127 s (104.25-128.5), respectively (P value = 0.02). No significant difference was seen between left-handed and right-handed residents on Task 1 (50 s versus 49 s) and Task 3 (51 s versus 59.5 s). In all three tasks, however, left-handed dominant residents had smaller variability (IQR 4.5-8 s versus 7-24.25 s) and significantly shorter delta times. CONCLUSIONS: Although true ambidexterity is rare, the ability to be facile with both hands is crucial for laparoscopic surgery. Our data show that variability in performance between the dominant hand and nondominant hand was remarkably smaller for left-hand dominant residents. This remains true despite the majority learning to operate with their nondominant hand. These data demonstrate a possible advantage to being left-hand dominant and may lead to further insights into variations of skill acquisition and improved curriculum development.

Feasibility, effectiveness and transferability of a novel mastery-based virtual reality robotic training platform for general surgery residents.

BACKGROUND: The annual number of robotic surgical procedures is on the rise. Robotic surgery requires unique skills compared to other surgical approaches. Simulation allows basic robot skill acquisition and enhances patient safety. The purpose of this study was to evaluate the feasibility, effectiveness, and transferability of a mastery-based curriculum using a new virtual reality (VR) robotic simulator for surgery resident training. METHODS: Nineteen PGY2s and 22 PGY4s were enrolled. Residents completed a pretest and posttest consisting of five VR and three previously validated inanimate tasks. Training included practicing 33 VR tasks until a total score ≥ 90% ("mastery") was achieved using automated metrics (time, economy of motion). Inanimate performance was evaluated by two trained, blinded raters using video review metrics (time, errors, and modified OSATS). Outcomes were defined as: curriculum feasibility (completion rate, training time, repetitions), training effectiveness (pre/post training skill improvement), and skill transferability (skill transfer to validated inanimate drills). Wilcoxon signed-rank and Mann-Whitney U tests were used; median (IQR) reported. RESULTS: Thirty-four of 41 residents (83%) achieved mastery on all 33 VR tasks; median training time was 7 h (IQR: 5'26″-8'52″). Pretest vs. post-test performance improved (all p < 0.001) according to all VR and Inanimate metrics for both PGY2 and PGY4 residents. Significant pretest performance differences were observed between PGY2 and PGY4 residents for VR but not inanimate tasks; no PGY2 vs. PGY4 posttest performance differences were observed for both VR and inanimate tasks. CONCLUSION: This mastery-based VR curriculum was associated with a high completion rate and excellent feasibility. Significant performance improvements were noted for both the VR and inanimate tasks, supporting training effectiveness and skill transferability. Additional studies examining validity evidence may help further refine this curriculum.

Thriving or surviving? A critical examination of funding models for fellowship council fellowships.

BACKGROUND: Since 1997, the Fellowship Council (FC) has evolved into a robust organization responsible for the advanced training of nearly half of the US residency graduates entering general surgery practice. While FC fellowships are competitive (55% match rate) and offer outstanding educational experiences, funding is arguably vulnerable. This study aimed to investigate the current funding models of FC fellowships. METHODS: Under an IRB-approved protocol, an electronic survey was administered to 167 FC programs with subsequent phone interviews to collect data on total cost and funding sources. De-identified data were also obtained via 2020-2021 Foundation for Surgical Fellowships (FSF) grant applications. Means and ranges are reported. RESULTS: Data were obtained from 59 programs (35% response rate) via the FC survey and 116 programs via FSF applications; the average cost to train one fellow per year was $107,957 and $110,816, respectively. Most programs utilized departmental and grants funds. Additionally, 36% (FC data) to 39% (FSF data) of programs indicated billing for their fellow, generating on average $74,824 ($15,000-200,000) and $33,281 ($11,500-66,259), respectively. FC data documented that 14% of programs generated net positive revenue, whereas FSF data documented that all programs were budget-neutral. CONCLUSION: Both data sets yielded similar overall results, supporting the accuracy of our findings. Expenses varied widely, which may, in part, be due to regional cost differences. Most programs relied on multiple funding sources. A minority were able to generate a positive revenue stream. Although fewer than half of programs billed for their fellow, this source accounted for substantial revenue. Institutional support and external grant funding have continued to be important sources for the majority of programs as well. Given the value of these fellowships and inherent vulnerabilities associated with graduate medical education funding, alternative grant funding models and standardization of annual financial reporting are encouraged.

Creating a Proficiency-based Remote Laparoscopic Skills Curriculum for the COVID-19 Era.

OBJECTIVE: Social distancing restrictions due to COVID-19 challenged our ability to educate incoming surgery interns who depend on early simulation training for basic skill acquisition. This study aimed to create a proficiency-based laparoscopic skills curriculum using remote learning. DESIGN: Content experts designed 5 surgical tasks to address hand-eye coordination, depth perception, and precision cutting. A scoring formula was used to measure performance: cutoff time - completion time - (K × errors) = score; the constant K was determined for each task. As a benchmark for proficiency, a fellowship-trained laparoscopic surgeon performed 3 consecutive repetitions of each task; proficiency was defined as the surgeon's mean score minus 2 standard deviations. To train remotely, PGY1 surgery residents (n = 29) were each issued a donated portable laparoscopic training box, task explanations, and score sheets. Remote training included submitting a pre-test video, self-training to proficiency, and submitting a post-test video. Construct validity (expert vs. trainee pre-tests) and skill acquisition (trainee pre-tests vs. post-tests) were compared using a Wilcoxon test (median [IQR] reported). SETTING: The University of Texas Southwestern Medical Center in Dallas, Texas PARTICIPANTS: Surgery interns RESULTS: Expert and trainee pre-test performance was significantly different for all tasks, supporting construct validity. One trainee was proficient at pre-test. After 1 month of self-training, 7 additional residents achieved proficiency on all 5 tasks after 2-18 repetitions; trainee post-test scores were significantly improved versus pre-test on all tasks (p = 0.01). CONCLUSIONS: This proficiency-based curriculum demonstrated construct validity, was feasible as a remote teaching option, and resulted in significant skill acquisition. The remote format, including video-based performance assessment, facilitates effective at-home learning and may allow additional innovations such as video-based coaching for more advanced curricula.

The Impact of COVID-19 on Surgical Education.

BACKGROUND: At the onset of social distancing, our general surgery residency transitioned its educational curriculum to an entirely virtual format with no gaps in conference offerings. The aim of this study is to examine the feasibility of our evolution to a virtual format and report program attitudes toward the changes. METHODS: On March 15, 2020, due to the coronavirus disease (COVID-19) our institution restricted mass gatherings. We immediately transitioned all lectures to a virtual platform. The cancellation of elective surgeries in April 2020 then created the need for augmented resident education opportunities. We responded by creating additional lectures and implementing a daily conference itinerary. To evaluate the success of the changes and inform the development of future curriculum, we surveyed residents and faculty regarding the changes. Classes and faculty answers were compared for perception of value of the online format. RESULTS: Pre-COVID-19, residency-wide educational offerings were concentrated to one half-day per week. Once restrictions were in place, our educational opportunities were expanded to a daily schedule and averaged 16.5 hours/week during April. Overall, 41/63 residents and 25/94 faculty completed the survey. The majority of residents reported an increased ability (56%) or similar ability (34.1%) to attend virtual conferences while 9.9% indicated a decrease. Faculty responses indicated similar effects (64% increased, 32% similar, 4% decreased). PGY-1 residents rated the changes negatively compared to other trainees and faculty. PGY-2 residents reported neutral views and all other trainees and faculty believed the changes positively affected educational value. Comments from PGY1 and 2 residents revealed they could not focus on virtual conferences as it was not "protected time" in a classroom and that they felt responsible for patient care during virtual lectures. A majority of both residents (61%) and faculty (84%) reported they would prefer to continue virtual conferences in the future. CONCLUSIONS: The necessity for adapting our academic offerings during the COVID-19 era has afforded our program the opportunity to recognize the feasibility of virtual platforms and expand our educational offerings. The majority of participants report stable to improved attendance and educational value. Virtual lectures should still be considered protected time in order to maximize the experience for junior residents.

Antifungal Therapy in Fungal Necrotizing Soft Tissue Infections.

BACKGROUND: Necrotizing soft tissue infections (NSTIs) are life-threatening surgical emergencies associated with high morbidity and mortality. Fungal NSTIs are considered rare and have been largely understudied. The purpose of this study was to study the impact of fungal NSTIs and antifungal therapy on mortality after NSTIs. METHODS: A retrospective chart review was performed on patients with NSTIs from 2012 to 2018. Patient baseline characteristics, microbiologic data, antimicrobial therapy, and clinical outcomes were collected. Patients were excluded if they had comfort care before excision. The primary outcome measured was in-hospital mortality. RESULTS: A total of 215 patients met study criteria with a fungal species identified in 29 patients (13.5%). The most prevalent fungal organism was Candida tropicalis (n = 11). Fungal NSTIs were more prevalent in patients taking immunosuppressive medications (17.2% versus 3.2%, P = 0.01). A fungal NSTI was significantly associated with in-hospital mortality (odds ratio, 3.13; 95% confidence interval, 1.16-8.40; P = 0.02). Furthermore, fungal NSTI patients had longer lengths of stay (32 d [interquartile range, 16-53] versus 19 d [interquartile range, 11-31], P < 0.01), more likely to require initiation of renal replacement therapy (24.1% versus 8.6%, P = 0.02), and more likely to require mechanical ventilation (64.5% versus 42.0%, P = 0.02). Initiation of antifungals was associated with a significantly lower rate of in-hospital mortality (6.7% versus 57.1%, P = 0.01). CONCLUSIONS: Fungal NSTIs are more common in patients taking immunosuppressive medications and are significantly associated with in-hospital mortality. Antifungal therapy is associated with decreased in-hospital mortality in those with fungal NSTIs. Consideration should be given to adding antifungals in empiric treatment regimens, especially in those taking immunosuppressive medications.