Surgical Education.

This paper summarizes the proceedings of the joint European Surgical Association ESA/American Surgical Association symposium on Surgical Education that took place in Bordeaux, France, as part of the celebrations for 30 years of ESA scientific meetings. Three presentations on the use of quantitative metrics to understand technical decisions, coaching during training and beyond, and entrustable professional activities were presented by American Surgical Association members and discussed by ESA members in a symposium attended by members of both associations.

Do Individual Surgeon Preferences Affect Procedural Outcomes?

OBJECTIVES: Surgeon preferences such as instrument and suture selection and idiosyncratic approaches to individual procedure steps have been largely viewed as minor differences in the surgical workflow. We hypothesized that idiosyncratic approaches could be quantified and shown to have measurable effects on procedural outcomes. METHODS: At the American College of Surgeons (ACS) Clinical Congress, experienced surgeons volunteered to wear motion tracking sensors and be videotaped while evaluating a loop of porcine intestines to identify and repair 2 preconfigured, standardized enterotomies. Video annotation was used to identify individual surgeon preferences and motion data was used to quantify surgical actions. χ 2 analysis was used to determine whether surgical preferences were associated with procedure outcomes (bowel leak). RESULTS: Surgeons' (N=255) preferences were categorized into 4 technical decisions. Three out of the 4 technical decisions (repaired injuries together, double-layer closure, corner-stitches vs no corner-stitches) played a significant role in outcomes, P <0.05. Running versus interrupted did not affect outcomes. Motion analysis revealed significant differences in average operative times (leak: 6.67 min vs no leak: 8.88 min, P =0.0004) and work effort (leak-path length=36.86 cm vs no leak-path length=49.99 cm, P =0.001). Surgeons who took the riskiest path but did not leak had better bimanual dexterity (leak=0.21/1.0 vs no leak=0.33/1.0, P =0.047) and placed more sutures during the repair (leak=4.69 sutures vs no leak=6.09 sutures, P =0.03). CONCLUSIONS: Our results show that individual preferences affect technical decisions and play a significant role in procedural outcomes. Future analysis in more complex procedures may make major contributions to our understanding of contributors to procedure outcomes.

From Listening to Action: Academic Surgical Departmental Response to Social Injustice Through Curricular Development.

OBJECTIVE: The aim of this study was to describe the development and evaluation of a structured department wide cultural competency curriculum. SUMMARY BACKGROUND DATA: Despite numerous organizational policies and statements, social injustice and bias still exist. Our department committed to assist individuals of the entire department to develop foundational knowledge and skills to combat implicit bias and systemic racism through the creation of a cultural competency curriculum. The purpose of this manuscript is to detail our curriculum and the evaluation of its effectiveness. METHODS: Using a well-established curriculum development framework, a cultural competency curriculum was developed focusing on knowledge, skills and attitudes at the individual level, for all members of the department. The curriculum was implemented through 6-hour-long sessions over a 9-week period. Effectiveness was assessed through a post curriculum survey. RESULTS: Twenty percent of the respondents had experienced bias based on race, ethnicity, or sexual orientation in the past 12 months, whereas 30% had experienced bias based on sex. Seventy-one percent independently explored related topics. The curriculum was overall well received and generally achieved the goals and objectives. CONCLUSION: Using a standard curriculum development framework, an effective department-wide cultural competency curriculum can be developed and implemented.

SAGES consensus recommendations on an annotation framework for surgical video.

BACKGROUND: The growing interest in analysis of surgical video through machine learning has led to increased research efforts; however, common methods of annotating video data are lacking. There is a need to establish recommendations on the annotation of surgical video data to enable assessment of algorithms and multi-institutional collaboration. METHODS: Four working groups were formed from a pool of participants that included clinicians, engineers, and data scientists. The working groups were focused on four themes: (1) temporal models, (2) actions and tasks, (3) tissue characteristics and general anatomy, and (4) software and data structure. A modified Delphi process was utilized to create a consensus survey based on suggested recommendations from each of the working groups. RESULTS: After three Delphi rounds, consensus was reached on recommendations for annotation within each of these domains. A hierarchy for annotation of temporal events in surgery was established. CONCLUSIONS: While additional work remains to achieve accepted standards for video annotation in surgery, the consensus recommendations on a general framework for annotation presented here lay the foundation for standardization. This type of framework is critical to enabling diverse datasets, performance benchmarks, and collaboration.

Situating Artificial Intelligence in Surgery: A Focus on Disease Severity.

OBJECTIVES: Artificial intelligence (AI) has numerous applications in surgical quality assurance. We assessed AI accuracy in evaluating the critical view of safety (CVS) and intraoperative events during laparoscopic cholecystectomy. We hypothesized that AI accuracy and intraoperative events are associated with disease severity. METHODS: One thousand fifty-one laparoscopic cholecystectomy videos were annotated by AI for disease severity (Parkland Scale), CVS achievement (Strasberg Criteria), and intraoperative events. Surgeons performed focused video review on procedures with ≥1 intraoperative events (n = 335). AI versus surgeon annotation of CVS components and intraoperative events were compared. For all cases (n = 1051), intraoperative-event association with CVS achievement and severity was examined using ordinal logistic regression. RESULTS: Using AI annotation, surgeons reviewed 50 videos/hr. CVS was achieved in ≤10% of cases. Hepatocystic triangle and cystic plate visualization was achieved more often in low-severity cases (P < 0.03). AI-surgeon agreement for all CVS components exceeded 75%, with higher agreement in high-severity cases (P < 0.03). Surgeons agreed with 99% of AI-annotated intraoperative events. AI-annotated intraoperative events were associated with both disease severity and number of CVS components not achieved. Intraoperative events occurred more frequently in high-severity versus low-severity cases (0.98 vs 0.40 events/case, P < 0.001). CONCLUSIONS: AI annotation allows for efficient video review and is a promising quality assurance tool. Disease severity may limit its use and surgeon oversight is still required, especially in complex cases. Continued refinement may improve AI applicability and allow for automated assessment.

Benchmarking Accomplishments of Leaders in American Surgery and Justification for Enhancing Diversity and Inclusion.

OBJECTIVE: To comprehensively assess the level of achievement and demographics of national surgical society presidents. BACKGROUND: Data on the accomplishments needed to rise to positions of national surgical leadership is scarce and merit alone does not always yield such opportunities. Recognizing the shortcomings of sex and ethnic diversity within academic surgical leadership, the American College of Surgeon (ACS), American Surgical Association (ASA), Association of Women Surgeons (AWS), and the Society of Black Academic Surgeons (SBAS) partnered to address these challenges by performing a comprehensive assessment of their presidents over the last 16 years. METHODS: ACS, ASA, AWS, and SBAS presidents' CVs, at the time of their presidential term, were assessed for demographics and scholastic achievements. Regression analyses controlling for age were performed to determine relative differences across societies. RESULTS: A total of 62 of the 64 presidents' CVs were received and assessed (97% response rate). There was a large discrepancy in the average age in years of ACS (70) and ASA (66) presidents compared to the AWS (51) and SBAS (53) presidents. For the ACS and ASA cohort, 87% were male and 83% were White, collectively. After controlling for age (52), the AWS and SBAS presidents' scholastic achievements were comparable to the ACS (and ASA) cohort in 9 and 12 of the 15 accessed metrics, respectively. CONCLUSION: The ACS and ASA presidents' CVs displayed unsurpassed scholastic achievement, and although not equivalent, both the AWS and the SBAS presidents had comparable attainment. These findings further substantiate that women and ethnic minority surgeons are deserving of additional national leadership consideration as organized medicine pursues a more diverse and reflective physician workforce.

Modeling Surgical Technical Skill Using Expert Assessment for Automated Computer Rating.

OBJECTIVE: Computer vision was used to predict expert performance ratings from surgeon hand motions for tying and suturing tasks. SUMMARY BACKGROUND DATA: Existing methods, including the objective structured assessment of technical skills (OSATS), have proven reliable, but do not readily discriminate at the task level. Computer vision may be used for evaluating distinct task performance throughout an operation. METHODS: Open surgeries was videoed and surgeon hands were tracked without using sensors or markers. An expert panel of 3 attending surgeons rated tying and suturing video clips on continuous scales from 0 to 10 along 3 task measures adapted from the broader OSATS: motion economy, fluidity of motion, and tissue handling. Empirical models were developed to predict the expert consensus ratings based on the hand kinematic data records. RESULTS: The predicted versus panel ratings for suturing had slopes from 0.73 to 1, and intercepts from 0.36 to 1.54 (Average R2 = 0.81). Predicted versus panel ratings for tying had slopes from 0.39 to 0.88, and intercepts from 0.79 to 4.36 (Average R2 = 0.57). The mean square error among predicted and expert ratings was consistently less than the mean squared difference among individual expert ratings and the eventual consensus ratings. CONCLUSIONS: The computer algorithm consistently predicted the panel ratings of individual tasks, and were more objective and reliable than individual assessment by surgical experts.

Dynamic Visual Feedback During Junctional Tourniquet Training.

BACKGROUND: This project involved the development and evaluation of a new visual bleeding feedback (VBF) system for tourniquet training. We hypothesized that dynamic VBF during junctional tourniquet training would be helpful and well received by trainees. MATERIALS AND METHODS: We designed the VBF to simulate femoral bleeding. Medical students (n = 15) and emergency medical service (EMS) members (n = 4) were randomized in a single-blind, crossover study to the VBF or without feedback groups. Poststudy surveys assessing VBF usefulness and recommendations were conducted along with participants' reported confidence using a 7-point Likert scale. Data from the different groups were compared using Wilcoxon signed-rank and rank-sum tests. RESULTS: Participants rated the helpfulness of the VBF highly (6.53/7.00) and indicated they were very likely to recommend the VBF simulator to others (6.80/7.00). Pre- and post-VBF confidence were not statistically different (P = 0.59). Likewise, tourniquet application times for VBF and without feedback before crossover were not statistically different (P = 0.63). Although participant confidence did not change significantly from beginning to end of the study (P = 0.46), application time was significantly reduced (P = 0.001). CONCLUSIONS: New tourniquet learners liked our VBF prototype and found it useful. Although confidence did not change over the course of the study for any group, application times improved. Future studies using outcomes of this study will allow us to continue VBF development as well as incorporate other quantitative measures of task performance to elucidate VBF's true benefit and help trainees achieve mastery in junctional tourniquet skills.

A structured, extended training program to facilitate adoption of new techniques for practicing surgeons.

INTRODUCTION: Laparoscopic inguinal hernia repair has been shown to have significant benefits when compared to open inguinal hernia repair, yet remains underutilized in the United States. The traditional model of short, hands-on, cognitive courses to enhance the adoption of new techniques fails to lead to significant levels of practice implementation for most surgeons. We hypothesized that a comprehensive program would facilitate the adoption of laparoscopic inguinal hernia repair (TEP) for practicing surgeons. METHODS: A team of experts in simulation, coaching, and hernia care created a comprehensive training program to facilitate the adoption of TEP. Three surgeons who routinely performed open inguinal hernia repair with greater than 50 cases annually were recruited to participate in the program. Coaches were selected based on their procedural expertise and underwent formal training in surgical coaching. Participants were required to evaluate all aspects of the educational program and were surveyed out to one year following completion of the program to assess for sustained adoption of TEP. RESULTS: All three participants successfully completed the first three steps of the seven-step program. Two participants completed the full course, while the third dropped out of the program due to time constraints and low case volume. Participant surgeons rated Orientation (4.7/5), GlovesOn training (5/5), and Preceptored Cases (5/5) as highly important training activities that contributed to advancing their knowledge and technical performance of the TEP procedure. At one year, both participants were performing TEPs for "most of their cases" and were confident in their ability to perform the procedure. The total cost of the program including all travel, personal coaching, and simulation was $8638.60 per participant. DISCUSSION: Our comprehensive educational program led to full and sustained adoption of TEP for those who completed the course. Time constraints, travel costs, and case volume are major considerations for successful completion; however, the program is feasible, acceptable, and affordable.

Rescuing the Clinical Breast Examination: Advances in Classifying Technique and Assessing Physician Competency.

OBJECTIVE: Develop new performance evaluation standards for the clinical breast examination (CBE). SUMMARY BACKGROUND DATA: There are several, technical aspects of a proper CBE. Our recent work discovered a significant, linear relationship between palpation force and CBE accuracy. This article investigates the relationship between other technical aspects of the CBE and accuracy. METHODS: This performance assessment study involved data collection from physicians (n = 553) attending 3 different clinical meetings between 2013 and 2014: American Society of Breast Surgeons, American Academy of Family Physicians, and American College of Obstetricians and Gynecologists. Four, previously validated, sensor-enabled breast models were used for clinical skills assessment. Models A and B had solitary, superficial, 2 cm and 1 cm soft masses, respectively. Models C and D had solitary, deep, 2 cm hard and moderately firm masses, respectively. Finger movements (search technique) from 1137 CBE video recordings were independently classified by 2 observers. Final classifications were compared with CBE accuracy. RESULTS: Accuracy rates were model A = 99.6%, model B = 89.7%, model C = 75%, and model D = 60%. Final classification categories for search technique included rubbing movement, vertical movement, piano fingers, and other. Interrater reliability was (k = 0.79). Rubbing movement was 4 times more likely to yield an accurate assessment (odds ratio 3.81, P < 0.001) compared with vertical movement and piano fingers. Piano fingers had the highest failure rate (36.5%). Regression analysis of search pattern, search technique, palpation force, examination time, and 6 demographic variables, revealed that search technique independently and significantly affected CBE accuracy (P < 0.001). CONCLUSIONS: Our results support measurement and classification of CBE techniques and provide the foundation for a new paradigm in teaching and assessing hands-on clinical skills. The newly described piano fingers palpation technique was noted to have unusually high failure rates. Medical educators should be aware of the potential differences in effectiveness for various CBE techniques.

Residents' response to bleeding during a simulated robotic surgery task.

BACKGROUND: The aim of this study was to assess performance measurement validity of our newly developed robotic surgery task trainer. We hypothesized that residents would exhibit wide variations in their intercohort performance as well as a measurable difference compared to surgeons in fellowship training. MATERIALS AND METHODS: Our laboratory synthesized a model of a pelvic tumor that simulates unexpected bleeding. Surgical residents and fellows of varying specialties completed a demographic survey and were allowed 20 minutes to resect the tumor using the da Vinci robot and achieve hemostasis. At a standardized event in the simulation, venous bleeding began, and participants attempted hemostasis using suture ligation. A motion tracking system, using electromagnetic sensors, recorded participants' hand movements. A postparticipation Likert scale survey evaluated participants' assessment of the model's realism and usefulness. RESULTS: Three of the seven residents (postgraduate year 2-5), and the fellow successfully resected the tumor in the allotted time. Residents showed high variability in performance and blood loss (125-700 mL) both within their cohort and compared to the fellow (150 mL blood). All participants rated the model as having high realism and utility for trainees. CONCLUSIONS: The results support that our bleeding pelvic tumor simulator has the ability to discriminate resident performance in robotic surgery. The combination of motion, decision-making, and blood loss metrics offers a multilevel performance assessment, analyzing both technical and decision-making abilities.

Residents' surgical performance during the laboratory years: an analysis of rule-based errors.

BACKGROUND: Nearly one-third of surgical residents will enter into academic development during their surgical residency by dedicating time to a research fellowship for 1-3 y. Major interest lies in understanding how laboratory residents' surgical skills are affected by minimal clinical exposure during academic development. A widely held concern is that the time away from clinical exposure results in surgical skills decay. This study examines the impact of the academic development years on residents' operative performance. We hypothesize that the use of repeated, annual assessments may result in learning even without individual feedback on participants simulated performance. METHODS: Surgical performance data were collected from laboratory residents (postgraduate years 2-5) during the summers of 2014, 2015, and 2016. Residents had 15 min to complete a shortened, simulated laparoscopic ventral hernia repair procedure. Final hernia repair skins from all participants were scored using a previously validated checklist. An analysis of variance test compared the mean performance scores of repeat participants to those of first time participants. RESULTS: Twenty-seven (37% female) laboratory residents provided 2-year assessment data over the 3-year span of the study. Second time performance revealed improvement from a mean score of 14 (standard error = 1.0) in the first year to 17.2 (SD = 0.9) in the second year, (F[1, 52] = 5.6, P = 0.022). Detailed analysis demonstrated improvement in performance for 3 grading criteria that were considered to be rule-based errors. There was no improvement in operative strategy errors. CONCLUSIONS: Analysis of longitudinal performance of laboratory residents shows higher scores for repeat participants in the category of rule-based errors. These findings suggest that laboratory residents can learn from rule-based mistakes when provided with annual performance-based assessments. This benefit was not seen with operative strategy errors and has important implications for using assessments not only for performance analysis but also as a learning experience.

Errors in bladder catheterization: are residents ready for complex scenarios?

BACKGROUND: The aim of this study was to investigate whether junior surgical residents had successfully mastered bladder catheterization. Our hypothesis was that surgical residents would be overly confident in their abilities and underestimate the potential for case complexity. MATERIALS AND METHODS: PGY 2-4 surgery residents (n = 44) were given 15 min. to complete three of four bladder catheterization simulations. Participants reported their mastery by rating confidence using a 5-point Likert scale. Multiple linear regression analysis was used to test predictors of procedure performance. RESULTS: Participants made a total of 228 errors with an average of 5.1 errors (standard deviation = 2.6) per participant. The most common errors included not maintaining the sterile field (52.0%), failure to get urine return (20.3%), and inflating the catheter balloon before urine return (8.4%). Some residents committed the same error more than once. Presimulation confidence ratings ranged from "1" being not confident to "5" being extremely confident. Average presimulation confidence was 4.42 (range 1-5, standard deviation = 0.85). Sixteen (36%) residents ranked their presimulation confidence in problem-solving abilities as "moderately confident" or below, whereas 28 (64%) were "very confident" or above. The lower the resident's presimulation confidence in problem-solving, the more errors they committed during the simulation (beta = -0.33, t = -2.15, P = 0.04). CONCLUSIONS: The residents did not perform as well as they anticipated when presented with more complicated bladder catheterization scenarios. Simulation can be used to identify and expose potential errors that may occur during complex presentations of basic procedures. This type of training and assessment may facilitate mastery.

Exploring hand coordination as a measure of surgical skill.

BACKGROUND: The study aim was to identify residents' coordination between dominant and nondominant hands while grasping for sutures in a laparoscopic ventral hernia repair procedure simulation. We hypothesize residents will rely on their dominant and nondominant hands unequally while grasping for suture. METHODS: Surgical residents had 15 min to complete the mesh securing and mesh tacking steps of a laparoscopic ventral hernia repair procedure. Procedure videos were coded for manual coordination events during the active suture grasping phase. Manual coordination events were defined as: active motion of dominant, nondominant, or both hands; and bimanual or unimanual manipulation of hands. A chi-square test was used to discriminate between coordination choices. RESULTS: Thirty-six residents (postgraduate year, 1-5) participated in the study. Residents changed manual coordination types during active suture grasping 500 times, ranging between 5 and 24 events (M = 13.9 events, standard deviation [SD] = 4.4). Bimanual coordination was used most (40%) and required the most time on average (M = 20.6 s, SD = 27.2), while unimanual nondominant coordination was used least (2.2%; M = 7.9 s, SD = 6.9). Residents relied on their dominant and nondominant hands unequally (P < 0.001). During 24% of events, residents depended on their nondominant hand (n = 120), which was predominantly used to operate the suture passer device. CONCLUSIONS: Residents appeared to actively coordinate both dominant and nondominant hands almost half of the time to complete suture grasping. Bimanual task durations took longer than other tasks on average suggesting these tasks were characteristically longer or switching hands required a greater degree of coordination. Future work is necessary to understand how task completion time and overall performance are affected by residents' hand utilization and switching between dominant and nondominant hands in surgical tasks.

Do resident's leadership skills relate to ratings of technical skill?

BACKGROUND: This study sought to compare general surgery research residents' survey information regarding self-efficacy ratings to their observed performance during a simulated small bowel repair. Their observed performance ratings were based on their leadership skills in directing their assistant. METHODS: Participants were given 15 min to perform a bowel repair using bovine intestines with standardized injuries. Operative assistants were assigned to help assist with the repair. Before the procedure, participants were asked to rate their expected skills decay, task difficulty, and confidence in addressing the small bowel injury. Interactions were coded to identify the number of instructions given by the participants to the assistant during the repair. Statistical analyses assessed the relationship between the number of directional instructions and participants' perceptions self-efficacy measures. Directional instructions were defined as any dialog by the participant who guided the assistant to perform an action. RESULTS: Thirty-six residents (58.3% female) participated in the study. Participants who rated lower levels of decay in their intraoperative decision-making and small bowel repair skills were noted to use their assistant more by giving more instructions. Similarly, a higher number of instructions correlated with lower perceived difficulty in selecting the correct suture, suture pattern, and completing the entire surgical task. CONCLUSIONS: General surgery research residents' intraoperative leadership skills showed significant correlations to their perceptions of skill decay and task difficulty during a bowel repair. Evaluating resident's directional instructions may provide an additional individualized intraoperative assessment metric. Further evaluation relating to operative performance outcomes is warranted.

Resident performance in complex simulated urinary catheter scenarios.

BACKGROUND: Urinary catheter insertion is a common procedure performed in hospitals. Improper catheterization can lead to unnecessary catheter-associated urinary tract infections and urethral trauma, increasing patient morbidity. To prevent such complications, guidelines were created on how to insert and troubleshoot urinary catheters. As nurses have an increasing responsibility for catheter placement, resident responsibility has shifted to more complex scenarios. This study examines the clinical decision-making skills of surgical residents during simulated urinary catheter scenarios. We hypothesize that during urinary catheterization, residents will make inconsistent decisions relating to catheter choices and clinical presentations. METHODS: Forty-five general surgery residents (postgraduate year 2-4) in Midwest training programs were presented with three of four urinary catheter scenarios of varying difficulty. Residents were allowed 15 min to complete the scenarios with five different urinary catheter choices. A chi-square test was performed to examine the relation between initial and subsequent catheter choices and to evaluate for consistency of decision-making for each scenario. RESULTS: Eighty-two percent of residents performed scenario A; 49% performed scenario B; 64% performed scenario C, and 82% performed scenario D. For initial attempt for scenario A-C, the 16 French Foley catheter was the most common choice (38%, 54%, 50%, P's < 0.001), whereas for scenario D, the 16 French Coude was the most common choice (37%, P < 0.01). Residents were most likely to be successful in achieving urine output in the initial catheterization attempt (P < 0.001). Chi-square analyses showed no relationship between residents' first and subsequent catheter choices for each scenario (P's > 0.05). CONCLUSIONS: Evaluation of clinical decision-making shows that initial catheter choice may have been deliberate based on patient background, as evidenced by the most popular choice in scenario D. Analyses of subsequent choices in each of the catheterization models reveal inconsistency. These findings suggest a possible lack of competence or training in clinical decision-making with regard to urinary catheter choices in residents.

Evaluation of Simulated Clinical Breast Exam Motion Patterns Using Marker-Less Video Tracking.

OBJECTIVE: This study investigates using marker-less video tracking to evaluate hands-on clinical skills during simulated clinical breast examinations (CBEs). BACKGROUND: There are currently no standardized and widely accepted CBE screening techniques. METHODS: Experienced physicians attending a national conference conducted simulated CBEs presenting different pathologies with distinct tumorous lesions. Single hand exam motion was recorded and analyzed using marker-less video tracking. Four kinematic measures were developed to describe temporal (time pressing and time searching) and spatial (area covered and distance explored) patterns. RESULTS: Mean differences between time pressing, area covered, and distance explored varied across the simulated lesions. Exams were objectively categorized as either sporadic, localized, thorough, or efficient for both temporal and spatial categories based on spatiotemporal characteristics. The majority of trials were temporally or spatially thorough (78% and 91%), exhibiting proportionally greater time pressing and time searching (temporally thorough) and greater area probed with greater distance explored (spatially thorough). More efficient exams exhibited proportionally more time pressing with less time searching (temporally efficient) and greater area probed with less distance explored (spatially efficient). Just two (5.9 %) of the trials exhibited both high temporal and spatial efficiency. CONCLUSIONS: Marker-less video tracking was used to discriminate different examination techniques and measure when an exam changes from general searching to specific probing. The majority of participants exhibited more thorough than efficient patterns. APPLICATION: Marker-less video kinematic tracking may be useful for quantifying clinical skills for training and assessment.