Outcome prediction in bariatric surgery through video-based assessment.

INTRODUCTION: The relationship between intraoperative surgical performance scores and patient outcomes has not been demonstrated at a single-case level. The GEARS score is a Likert-based scale that quantifies robotic surgical proficiency in 5 domains. Given that even highly skilled surgeons can have variability in their skill among their cases, we hypothesized that at a patient level, higher surgical skill as determined by the GEARS score will predict individual patient outcomes. METHODS: Patients undergoing robotic sleeve gastrectomy between July 2018 and January 2021 at a single-health care system were captured in a prospective database. Bivariate Pearson's correlation was used to compare continuous variables, one-way ANOVA for categorical variables compared with a continuous variable, and chi-square for two categorical variables. Significant variables in the univariable screen were included in a multivariable linear regression model. Two-tailed p-value < 0.05 was considered significant. RESULTS: Of 162 patients included, 9 patients (5.5%) experienced a serious morbidity within 30 days. The average excess weight loss (EWL) was 72 ± 12% at 6 months and 74 ± 15% at 12 months. GEARS score was not significantly correlated with EWL at 6 months (p = 0.349), 12 months (p = 0.468), or serious morbidity (p = 0.848) on unadjusted analysis. After adjusting, total GEARS score was not correlated with serious morbidity (p = 0.914); however, GEARS score did predict EWL at 6 (p < 0.001) and 12 months (p < 0.001). All GEARS subcomponent scores, bimanual dexterity, depth perception, efficiency, force sensitivity, and robotic control were predictive of EWL at 6 months (p < 0.001) and 12 months (p < 0.001) on multivariable analysis. CONCLUSION: For patients undergoing sleeve gastrectomy, surgical skill as assessed by the GEARS score was correlated with EWL, suggesting that better performance of a sleeve gastrectomy can result in improved postoperative weight loss.

Reducing surgical site infections after colectomy: bundle item compliance, process, and outlier identification.

BACKGROUND: The purpose of this study was to implement a checklist monitoring system and identify critical surgical checklist items associated with post-colectomy surgical site infections (SSI). The relationship between checklist compliance, infection rates, and identification of non-compliant surgeons was explored. MATERIALS AND METHODS: National Health Safety Network (NHSN) data were imported annually to establish baseline incidence of post-colectomy SSI from 2016 to 2019. A colectomy checklist was used to monitor compliance for 1694 random colectomies (1274 elective; 420 emergency). Reports were generated monthly to profile system, hospital, surgeon-specific infection, and checklist compliance rates. RESULTS: Checklist compliance improved in elective and emergent colectomies to > 90% for all items except oral antibiotic and mechanical bowel prep in elective cases. Annualized total SSI and organ space infection rates in elective cases decreased by 33% and 45%, respectively. Elective and emergency SSI's were reduced for Superficial Incisional Primary (SIP), Deep Incisional Primary (DIP), and Intra-Abdominal Abscess (IAB) by 66%, 60.4%, and 78.3%, respectively. Checklist compliance between low (< 3%) and high (> 3%) infection rate surgeons demonstrated significantly lower utilization of oral antibiotic prep (p < 0.03) and mechanical bowel prep (p < 0.02) in high infection rate surgeons. CONCLUSION: Surgeons compliant with colectomy checklists decreased elective and emergency colectomy infection rates. Ceiling compliance rates > 95% for bundle items are suggested to achieve optimal reductions in SSIs and efforts should be focused on surgeons with NHSN infection rates > 3%. Oral antibiotic prep and mechanical bowel prep compliance rates in elective colectomy appeared to differentiate high infection rate surgeons from low infection rate surgeons.

Blinded intraoperative skill evaluations avoid gender-based bias.

INTRODUCTION: Gender bias has been identified consistently in written performance evaluations. Qualitative tools may provide a standardized way to evaluate surgical skill and minimize gender bias. We hypothesized that there is no difference in operative time or GEARS scores in robotic hysterectomy for men vs women surgeons. METHODS: Patients undergoing robotic hysterectomies performed between June 2019 and March 2020 at 8 hospitals within the same hospital system were captured into a prospective database. GEARS scores were assigned by crowd-sourced evaluators by a third party blinded to any surgeon- or patient-identifying information. One-way ANOVA was used to compare the mean operative time and GEARS scores for each group, and significant variables were included in a one-way ANCOVA to control for confounders. Two-tailed p-value < 0.05 was considered significant. RESULTS: Seventeen women and 13 men performed a total of 188 hysterectomies; women performed 34 (18%) and men performed 153 (81%). Women surgeons had a higher mean operative time (133 ± 58 vs 86.3 ± 46 min, p = 0.024); after adjustment, there were no significant differences in operative time (p = 0.607). There was no significant difference between the genders in total GEARS score (20.0 ± 0.77 vs 20.2 ± 0.70, p = 0.415) or GEARS subcomponent scores: bimanual dexterity (3.98 ± 0.03 vs 4.00 ± 0.03, p = 0.705); depth perception (4.04 ± 0.04 vs 4.05 ± 0.02, p = 0.799); efficiency (3.79 ± 0.02 vs 3.82 ± 0.02, p = 0.437); force sensitivity (4.01 ± 0.04 vs 4.05 ± 0.05, p = 0.533); or robotic control (4.16 ± 0.03 vs 4.26 ± 0.01, p = 0.079). CONCLUSION: There was no difference in GEARS score between men vs women surgeons performing robotic hysterectomies. Video-based blinded assessment of skills may minimize gender biases when evaluating surgical skill for competency evaluation and credentialing.

Core Competency Self-Assessment Enhances Critical Review of Complications and Entrustable Activities.

BACKGROUND: The Accreditation Council for Graduate Medical Education has defined six core competencies (CCs) that every successful physician should possess. However, the assessment of CC achievement among trainees is difficult. This project was designed to prospectively evaluate the impact of resident identification of CC as a component of morbidity review on error identification and standard of care (SOC) assessments. The platform was assessed for its reliability as a measure of resident critical analysis of complication causality across postgraduate year (PGY). MATERIALS AND METHODS: A total of 1945 general surgery cases with complications were assessed for error identification and SOC management between January 1, 2016, and December 31, 2018. CC identification was additionally assessed between January 1, 2019, and December 31, 2019, and included 708 general surgery cases. Data were evaluated for error assessments and overall SOC management. PGY4 and 5 residents were compared for number of cases and complications reviewed, severity, error causation, and CC relevance. RESULTS: Study groups were equivalent by Clavien-Dindo scores. Error identification significantly increased in all categories: diagnostic (P < 0.001), technical (P < 0.05), judgment (P < 0.001), system (P < 0.001), and communication (P < 0.001). Overall SOC assessments validated by a supervising surgical quality officer were unchanged. An increased exposure to cases with severe complications, error causation, and CC relevance was noted across PGY. CONCLUSIONS: The addition of CC assessment into morbidity review appears to improve the critical thinking of evaluating residents by increasing the identification of management errors. Used as an element of prospective self-assessment, teaching residents to identify CC principles in cases with complications may assist in learner progression toward clinical competence and critical thinking.

Cognitive Bias Impact on Management of Postoperative Complications, Medical Error, and Standard of Care.

BACKGROUND: Cognitive bias (CB) is increasingly recognized as an important source of medical error and up to 75% of errors in internal medicine are thought to be cognitive in origin (O'Sullivan ED, Schofield SJ. Cognitive bias in clinical medicine. J R Coll Physicans Edinb. 2018;48;225-232). However, primary data regarding the true incidence of bias is lacking. A prospective evaluation of CB in the management of surgical cases with complications has not been reported. This study reports the incidence and distribution of various types of CBs, and evaluates their impact on management errors and standard of care (SOC). METHODS: A prospectively collected series of 736 general surgical cases with complications from three university hospitals was analyzed. Surgical residents evaluated cases for 22 types of CBs (Croskerry P. The importance of cognitive errors in diagnosis and strategies to minimize them. Acad Med. 2003;78:775-780). Supervising quality officers validated all quality assessments. Data were assessed for the incidence of CBs, error assessments (diagnostic, technical, judgment, system, communication, therapeutic, and professionalism), and SOC. RESULTS: CB was attributed in 32.7% (241/736) of all cases with complications. The most common CBs identified, both singly and in groups, were anchoring, confirmation, omission, commission, overconfidence, premature closure, hindsight, diagnosis momentum, outcome, and ascertainment bias. The attribution of CB was correlated to a statistically significant increase in the incidence of management errors by the surgical team and lower SOC assessments. CONCLUSIONS: CBs are identified in the management of cases with complications and are associated with an increase in management errors and a degradation in SOC. Insight into the types of CBs and their association with the type and severity of management errors may prove useful in improving quality care.

Correlation between operative time and crowd-sourced skills assessment for robotic bariatric surgery.

INTRODUCTION: Operative time has been traditionally used as a proxy for surgical skill and is commonly utilized to measure the learning curve, assuming that faster operations indicate a more skilled surgeon. The Global Evaluative Assessment of Robotic Skills (GEARS) rubric is a validated Likert scale for evaluating technical skill. We hypothesize that operative time will not correlate with the GEARS score. METHODS: Patients undergoing elective robotic sleeve gastrectomy at a single bariatric center of excellence hospital from January 2019 to March 2020 were captured in a prospectively maintained database. For step-specific scoring, videos were broken down into three steps: ligation of short gastric vessels, gastric transection, and oversewing the staple line. Overall and step-specific GEARS scores were assigned by crowd-sourced evaluators. Correlation between operative time and GEARS score was assessed with linear regression and calculation of the R2 statistic. RESULTS: Sixty-eight patients were included in the study, with a mean operative time of 112 ± 27.4 min. The mean GEARS score was 20.1 ± 0.81. Mean scores for the GEARS subcomponents were: bimanual dexterity 4.06 ± 0.17; depth perception 3.96 ± 0.24; efficiency 3.82 ± 0.19; force sensitivity 4.06 ± 0.20; robotic control 4.16 ± 0.21. Operative time and overall score showed no correlation (R2 = 0.0146, p = 0.326). Step-specific times and scores showed weak correlation for gastric transection (R2 = 0.0737, p = 0.028) and no correlation for ligation of short gastric vessels (R2 = 0.0262, p = 0.209) or oversewing the staple line (R2 = 0.0142, p = 0.344). CONCLUSIONS: Operative time and crowd-sourced GEARS score were not correlated. Operative time and GEARS scores measure different performance characteristics, and future studies should consider using both a validated skills assessment tool and operative time for a more complete evaluation of skill.

Using Concurrent Complication Reporting to Evaluate Resident Critical Thinking and Enhance Adult Learning.

OBJECTIVE: Critical thinking and accurate case analysis is difficult to quantify even within the context of routine morbidity and mortality reporting. We designed and implemented a HIPAA-compliant adverse outcome reporting system that collects weekly resident assessments of clinical care across multiple domains (case summary, complications, error analysis, Clavien-Dindo Harm, cognitive bias, standard of care, and ACGME core competencies). We hypothesized that incorporation of this system into the residency program's core curriculum would allow for identification of areas of cognitive weakness or strength and provide a longitudinal evaluation of critical thinking development. DESIGN: A validated, password-protected electronic platform linked to our electronic medical record was used to collect cases weekly in which surgical adverse events occurred. General surgery residents critiqued 1932 cases over a 4-year period from 3 major medical centers within our system. These data were reviewed by teaching faculty, corrected for accuracy and graded utilizing the software's critique algorithm. Grades were emailed to the residents at the time of the review, collected prospectively, stratified, and analyzed by post-graduate year (PGY). Evaluation of the resident scores for each domain and the resultant composite scores allowed for comparison of critical thinking skills across post-graduate year (PGY) over time. SETTING: Data was collected from 3 independently ACGME-accredited surgery residency programs over 3 tertiary hospitals within our health system. PARTICIPANTS: General surgery residents in clinical PGY 1-5. RESULTS: Residents scored highest in properly identifying ACGME core competencies and determining Clavien-Dindo scores (p < 0.006) with no improvement in providing accurate and concise clinical summaries. However, residents improved in recording data sufficient to identify error (p < 0.00001). A positive linear trend in median scores for all remaining domains except for cognitive bias was demonstrated (p < 0.001). Senior residents scored significantly higher than junior residents in all domains. Scores > 90% were never achieved. CONCLUSIONS: The use of an electronic standardized critique algorithm in the evaluation and assessment of adverse surgical case outcomes enabled the measure of residents' critical thinking skills. Feedback in the form of teaching faculty-facilitated discussion and emailed grades enhanced adult learning with a steady improvement in performance over PGY. Although residents improved with PGY, the data suggest that further improvement in all categories is possible. Implementing this standardized critique algorithm across PGY allows for evaluation of areas of individual resident weakness vs. strength, progression over time, and comparisons to peers. These data suggest that routine complication reporting may be enhanced as a critical thinking assessment tool and that improvement in critical thinking can be quantified. Incorporation of this platform into M&M conference has the potential to augment executive function and professional identity development.

The Surgical Learning Curve: Does Robotic Technical Skill Explain Differences in Operative Performance?

Background: Prior studies on technical skills use small collections of videos for assessment. However, there is likely heterogeneity of performance among surgeons and likely improvement after training. If technical skill explains these differences, then it should vary among practicing surgeons and improve over time. Materials and Methods: Sleeve gastrectomy cases (n = 162) between July 2018 and January 2021 at one health system were included. Global evaluative assessment of robotic skills (GEARS) scores were assigned by crowdsourced evaluators. Videos were manually annotated. Analysis of variance was used to compare continuous variables between surgeons. Tamhane's post hoc test was used to define differences between surgeons with the eta-squared value for effect size. Linear regression was used for temporal changes. A P value <.05 was considered significant. Results: Variations in operative time discriminated between individuals (e.g., between 2 surgeons, means were 91 and 112 minutes, Tamhane's = 0.001). Overall, GEARS scores did not vary significantly (e.g., between those 2 surgeons, means were 20.32 and 20.6, Tamhane's = 0.151). Operative time and total GEARS score did not change over time (R2 = 0.0001-0.096). Subcomponent scores showed idiosyncratic temporal changes, although force sensitivity increased among all (R2 = 0.172-0.243). For a novice surgeon, phase-adjusted operative time (R2 = 0.24), but not overall GEARS scores (R2 = 0.04), improved over time. Conclusions: GEARS scores showed less variability and did not improve with time for a novice surgeon. Improved technical skill does not explain the learning curve of a novice surgeon or variation among surgeons. More work could define valid surrogate metrics for performance analysis.

Does Surgeon Experience Correlate with Crowd-Sourced Skill Assessment in Robotic Bariatric Surgery?

BACKGROUND: The Global Evaluative Assessment of Robotic Skills (GEARS) rubric provides a measure of skill in robotic surgery. We hypothesize surgery performed by more experienced operators will be associated with higher GEARS scores. METHOD: Patients undergoing sleeve gastrectomy from 2016 to 2020 were analyzed. Three groups were defined by time in practice: less than 5, between 5 and 15, and more than 15 years. Continuous variables were compared with ANOVA and multivariable regression was performed. RESULTS: Fourteen operators performing 154 cases were included. More experienced surgeons had higher GEARS scores and shorter operative times. On multivariable regression, operative time (P = 0.027), efficiency (P = .022), depth perception (P = 0.033), and bimanual dexterity (P = 0.047) were associated with experience. CONCLUSIONS: In our video-based assessment (VBA) model, operative time and several GEARS subcomponent scores were associated with surgical experience. Further studies should determine the association between these metrics and surgical outcomes.

Cognitive bias and severity of harm following surgery: Plan for workflow debiasing strategy.

INTRODUCTION: This study analyzes the relationship between cognitive bias (CB) and harm severity as measured by Clavien-Dindo Scores (CD). METHODS: A prospectively collected series of 655 severity matched general surgical cases with complications were analyzed. Cases were evaluated for CB and assigned harm scores as defined by CD grade. Potentially mitigating "debiasing" strategies were identified for each bias attribution. RESULTS: Among cases with CB, 24% (55/232) were CD(I-II) and 76% (177/232) were CD(III-V). Odds ratio suggests that serious complications occur nearly 60% more frequently when CB is identified. The CBs identified with severe harm were Overconfidence, Commission, Anchoring, Confirmation, and Diagnosis Momentum. Preliminary data on debiasing strategies suggest diagnosis review, linear reasoning and Type II thinking may be relevant in over 85% of complications. CONCLUSION: The incidence of CB is increased in patients sustaining severe harm. Understanding the specific CBs identified and their mitigating debiasing strategies may improve outcomes.

Impact of Timing of Tocilizumab Use in Hospitalized Patients With SARS-CoV-2 Infection.

BACKGROUND: SARS-CoV-2 (severe acute respiratory syndrome coronavirus 2) continues to be a global challenge due to the lack of definitive treatment strategies. We sought to determine the efficacy of early administration of anti-interleukin 6 therapy in reducing hospital mortality and progression to mechanical ventilation. METHODS: This was a retrospective chart review of 11,512 patients infected with SARS-CoV-2 who were admitted to a New York health system from March to May 2020. Tocilizumab was administered to subjects at the nasal cannula level of oxygen support to maintain an oxygen saturation of >88%. The Charlson comorbidity index was used as an objective assessment of the burden of comorbidities to predict 10-year mortality. The primary outcome of interest was hospital mortality. Secondary outcomes were progression to mechanical ventilation; the prevalence of venous thromboembolism and renal failure; and the change in C-reactive protein, D-dimer, and ferritin levels after tocilizumab administration. Propensity score matching by using a 1:2 protocol was used to match the tocilizumab and non-tocilizumab groups to minimize selection bias. The groups were matched on baseline demographic characteristics, including age, sex, and body mass index; Charlson comorbidity index score; laboratory markers, including ferritin, D-dimer, lactate dehydrogenase, and C-reactive protein values; and the maximum oxygen requirement at the time of tocilizumab administration. Mortality outcomes were evaluated based on the level of oxygen requirement and the day of hospitalization at the time of tocilizumab administration. RESULTS: The overall hospital mortality was significantly reduced in the tocilizumab group when tocilizumab was administered at the nasal cannula level (10.4% vs 22.0%; P = .002). In subjects who received tocilizumab at the nasal cannula level, the progression to mechanical ventilation was reduced versus subjects who were initially on higher levels of oxygen support (6.3% vs 18.7%; P < .001). There was no improvement in mortality when tocilizumab was given at the time of requiring non-rebreather, high-flow nasal cannula, noninvasive ventilator, or invasive ventilator. CONCLUSIONS: Early use of anti-interleukin 6 therapy may be associated with improved hospital mortality and reduction in progression to more severe coronavirus disease 2019.