Laparoscopic cholecystectomy critical view of safety (LC-CVS): a multi-national validation study of an objective, procedure-specific assessment using video-based assessment (VBA).

BACKGROUND: A novel 6-item objective, procedure-specific assessment for laparoscopic cholecystectomy incorporating the critical view of safety (LC-CVS OPSA) was developed to support trainee formative and summative assessments. The LC-CVS OPSA included two retraction items (fundus and infundibulum retraction) and four CVS items (hepatocystic triangle visualization, gallbladder-liver separation, cystic artery identification, and cystic duct identification). The scoring rubric for retraction consisted of poor (frequently outside of defined range), adequate (minimally outside of defined range) and excellent (consistently inside defined range) and for CVS items were "poor-unsafe", "adequate-safe", or "excellent-safe". METHODS: A multi-national consortium of 12 expert LC surgeons applied the OPSA-LC CVS to 35 unique LC videos and one duplicate video. Primary outcome measure was inter-rater reliability as measured by Gwet's AC2, a weighted measure that adjusts for scales with high probability of random agreement. Analysis of the inter-rater reliability was conducted on a collapsed dichotomous scoring rubric of "poor-unsafe" vs. "adequate/excellent-safe". RESULTS: Inter-rater reliability was high for all six items ranging from 0.76 (hepatocystic triangle visualization) to 0.86 (cystic duct identification). Intra-rater reliability for the single duplicate video was substantially higher across the six items ranging from 0.91 to 1.00. CONCLUSIONS: The novel 6-item OPSA LC CVS demonstrated high inter-rater reliability when tested with a multi-national consortium of LC expert surgeons. This brief instrument focused on safe surgical practice was designed to support the implementation of entrustable professional activities into busy surgical training programs. Instrument use coupled with video-based assessments creates novel datasets with the potential for artificial intelligence development including computer vision to drive assessment automation.

A multi-national, video-based qualitative study to refine training guidelines for assigning an "unsafe" score in laparoscopic cholecystectomy critical view of safety.

BACKGROUND: The critical view of safety (CVS) was incorporated into a novel 6-item objective procedure-specific assessment for laparoscopic cholecystectomy (LC-CVS OPSA) to enhance focus on safe completion of surgical tasks and advance the American Board of Surgery's entrustable professional activities (EPAs) initiative. To enhance instrument development, a feasibility study was performed to elucidate expert surgeon perspectives regarding "safe" vs. "unsafe" practice. METHODS: A multi-national consortium of 11 expert LC surgeons were asked to apply the LC-CVS OPSA to ten LC videos of varying surgical difficulty using a "safe" vs. "unsafe" scale. Raters were asked to provide written rationale for all "unsafe" ratings and invited to provide additional feedback regarding instrument clarity. A qualitative analysis was performed on written responses to extract major themes. RESULTS: Of the 660 ratings, 238 were scored as "unsafe" with substantial variation in distribution across tasks and raters. Analysis of the comments revealed three major categories of "unsafe" ratings: (a) inability to achieve the critical view of safety (intended outcome), (b) safe task completion but less than optimal surgical technique, and (c) safe task completion but risk for potential future complication. Analysis of reviewer comments also identified the potential for safe surgical practice even when CVS was not achieved, either due to unusual anatomy or severe pathology preventing safe visualization. Based upon findings, modifications to the instructions to raters for the LC-CVS OPSA were incorporated to enhance instrument reliability. CONCLUSIONS: A safety-based LC-CVS OPSA has the potential to significantly improve surgical training by incorporating CVS formally into learner assessment. This study documents the perspectives of expert biliary tract surgeons regarding clear identification and documentation of unsafe surgical practice for LC-CVS and enables the development of training materials to improve instrument reliability. Learnings from the study have been incorporated into rater instructions to enhance instrument reliability.

Using video-based assessment (VBA) to document fellow improvement in safely completing the jejunojejunostomy portion of laparoscopic Roux-en-Y gastric bypass (RYGB) surgery.

BACKGROUND: Surgical assessment instruments are used for formative and summative trainee evaluations. To characterize the features of existing instruments and a novel 12-item objective, procedure-specific assessment tool for Roux-en-Y Gastric Bypass (RYGB-OPSA), we evaluated the progress of a single surgical fellow over 17 consecutive surgeries. METHODS: Seventeen consecutive RYGB videos completed between 8/2021 and 1/2022 by an academic hospital surgical fellow were de-identified and assessed by four board-certified bariatric surgeons using Global Operative Assessment of Laparoscopic Skills (GOALS), General Assessment of Surgical Skill (GASS), and RYGB-OPSA which includes the reflection of transverse colon, identification of ligament of Treitz, biliopancreatic and Roux limbs orientation, jejunal division point selection, stapler use, mesentery division, bleeding control, jejunojejunostomy (JJ) anastomotic site selection, apposition of JJ anastomotic site, JJ creation, common enterotomy closure of JJ, and integrity of anastomosis. The GASS measured economy of motion, tissue handling, appreciating operative anatomy, bimanual dexterity, and achievement of hemostasis. RYGB-OPSA and GASS items were scored "poor-unsafe," "acceptable-safe," or "good-safe." Change in performance was measured by linear trendline slope. RESULTS: Over the course of 17 procedures, significant improvement was demonstrated by three GOALS items, GOALS overall score, GASS bimanual dexterity, and three RYGB-OPSA tasks: JJ creation, jejunal division point selection, and stapler use. Achievement of hemostasis declined but never rated "poor-unsafe." Overall RYGB-OPSA and GOALS trendlines documented significant increase across the 17 procedures. CONCLUSION: This examination of a bariatric surgery fellow's operative training experience as measured by three surgical assessment instruments demonstrated anticipated improvements in general skills and safe completion of procedure-specific tasks. Effective surgical assessment instruments have enough sensitivity to show improvement to enable meaningful trainee feedback (low-stakes assessments) as well as the ability to determine safe surgical practice to enable promotion to greater autonomous practice.

A Practical Approach to Predicting Surgical Site Infection Risk Among Patients Before Leaving the Operating Room.

A surgical site infection (SSI) prediction model that identifies at-risk patients before leaving the operating room can support efforts to improve patient safety. In this study, eight pre-operative and five perioperative patient- and procedure-specific characteristics were tested with two scoring algorithms: 1) count of positive factors (manual), and 2) logistic regression model (automated). Models were developed and validated using data from 3,440 general and oncologic surgical patients. In the automated algorithm, two pre-operative (procedure urgency, odds ratio [OR]: 1.7; and antibiotic administration >2 hours before incision, OR: 1.6) and three intraoperative risk factors (open surgery [OR: 3.7], high-risk procedure [OR: 3.5], and operative time OR: [2.6]) were associated with SSI risk. The manual score achieved an area under the curve (AUC) of 0.831 and the automated algorithm achieved AUC of 0.868. Open surgery had the greatest impact on prediction, followed by procedure risk, operative time, and procedure urgency. At 80% sensitivity, the manual and automated scores achieved a positive predictive value of 16.3% and 22.0%, respectively. Both the manual and automated SSI risk prediction algorithms accurately identified at-risk populations. Use of either model before the patient leaves the operating room can provide the clinical team with evidence-based guidance to consider proactive intervention to prevent SSIs.

Association between Surgical Technical Skills and Clinical Outcomes: A Systematic Literature Review and Meta-Analysis.

Background: A systematic literature review and meta-analysis was conducted to assess the association between intraoperative surgical skill and clinical outcomes. Methods: Peer-reviewed, original research articles published through August 31, 2021 were identified from PubMed and Embase. From the 1,513 potential articles, seven met eligibility requirements, reporting on 151 surgeons and 17,932 procedures. All included retrospective assessment of operative videos. Associations between surgical skill and outcomes were assessed by pooling odds ratios (OR) using random-effects models with the inverse variance method. Eligible studies included pancreaticoduodenectomy, gastric bypass, laparoscopic gastrectomy, prostatectomy, colorectal, and hemicolectomy procedures. Results: Meta-analytic pooling identified significant associations between the highest vs. lowest quartile of surgical skill and reoperation (OR: 0.44; 95% confidence interval [CI]: 0.23, 0.83), hemorrhage (OR: 0.66; 95% CI, 0.65, 0.68), obstruction (OR: 0.33; 95% CI, 0.30, 0.35), and any medical complication (OR: 0.23, 95% CI, 0.19, 0.27). Nonsignificant inverse associations were noted between skill and readmission, emergency department visit, mortality, leak, infection, venous thromboembolism, and cardiac and pulmonary complications. Conclusions: Overall, surgeon technical skill appears to predict clinical outcomes. However, there are surprisingly few articles that evaluate this association. The authors recommend a thoughtful approach for the development of a comprehensive surgical quality infrastructure that could significantly reduce the challenges identified by this study.

Towards interpretable, medically grounded, EMR-based risk prediction models.

Machine-learning based risk prediction models have the potential to improve patient outcomes by assessing risk more accurately than clinicians. Significant additional value lies in these models providing feedback about the factors that amplify an individual patient's risk. Identification of risk factors enables more informed decisions on interventions to mitigate or ameliorate modifiable factors. For these reasons, risk prediction models must be explainable and grounded on medical knowledge. Current machine learning-based risk prediction models are frequently 'black-box' models whose inner workings cannot be understood easily, making it difficult to define risk drivers. Since machine learning models follow patterns in the data rather than looking for medically relevant relationships, possible risk factors identified by these models do not necessarily translate into actionable insights for clinicians. Here, we use the example of risk assessment for postoperative complications to demonstrate how explainable and medically grounded risk prediction models can be developed. Pre- and postoperative risk prediction models are trained based on clinically relevant inputs extracted from electronic medical record data. We show that these models have similar predictive performance as models that incorporate a wider range of inputs and explain the models' decision-making process by visualizing how different model inputs and their values affect the models' predictions.

ClipAssistNet: bringing real-time safety feedback to operating rooms.

PURPOSE: Cholecystectomy is one of the most common laparoscopic procedures. A critical phase of laparoscopic cholecystectomy consists in clipping the cystic duct and artery before cutting them. Surgeons can improve the clipping safety by ensuring full visibility of the clipper, while enclosing the artery or the duct with the clip applier jaws. This can prevent unintentional interaction with neighboring tissues or clip misplacement. In this article, we present a novel real-time feedback to ensure safe visibility of the instrument during this critical phase. This feedback incites surgeons to keep the tip of their clip applier visible while operating. METHODS: We present a new dataset of 300 laparoscopic cholecystectomy videos with frame-wise annotation of clipper tip visibility. We further present ClipAssistNet, a neural network-based image classifier which detects the clipper tip visibility in single frames. ClipAssistNet ensembles predictions from 5 neural networks trained on different subsets of the dataset. RESULTS: Our model learns to classify the clipper tip visibility by detecting its presence in the image. Measured on a separate test set, ClipAssistNet classifies the clipper tip visibility with an AUROC of 0.9107, and 66.15% specificity at 95% sensitivity. Additionally, it can perform real-time inference (16 FPS) on an embedded computing board; this enables its deployment in operating room settings. CONCLUSION: This work presents a new application of computer-assisted surgery for laparoscopic cholecystectomy, namely real-time feedback on adequate visibility of the clip applier. We believe this feedback can increase surgeons' attentiveness when departing from safe visibility during the critical clipping of the cystic duct and artery.