Machine learning models to predict success of endoscopic sleeve gastroplasty using total and excess weight loss percent achievement: a multicentre study.

BACKGROUND: The large amount of heterogeneous data collected in surgical/endoscopic practice calls for data-driven approaches as machine learning (ML) models. The aim of this study was to develop ML models to predict endoscopic sleeve gastroplasty (ESG) efficacy at 12 months defined by total weight loss (TWL) % and excess weight loss (EWL) % achievement. Multicentre data were used to enhance generalizability: evaluate consistency among different center of ESG practice and assess reproducibility of the models and possible clinical application. Models were designed to be dynamic and integrate follow-up clinical data into more accurate predictions, possibly assisting management and decision-making. METHODS: ML models were developed using data of 404 ESG procedures performed at 12 centers across Europe. Collected data included clinical and demographic variables at the time of ESG and at follow-up. Multicentre/external and single center/internal and temporal validation were performed. Training and evaluation of the models were performed on Python's scikit-learn library. Performance of models was quantified as receiver operator curve (ROC-AUC), sensitivity, specificity, and calibration plots. RESULTS: Multicenter external validation: ML models using preoperative data show poor performance. Best performances were reached by linear regression (LR) and support vector machine models for TWL% and EWL%, respectively, (ROC-AUC: TWL% 0.87, EWL% 0.86) with the addition of 6-month follow-up data. Single-center internal validation: Preoperative data only ML models show suboptimal performance. Early, i.e., 3-month follow-up data addition lead to ROC-AUC of 0.79 (random forest classifiers model) and 0.81 (LR models) for TWL% and EWL% achievement prediction, respectively. Single-center temporal validation shows similar results. CONCLUSIONS: Although preoperative data only may not be sufficient for accurate postoperative predictions, the ability of ML models to adapt and evolve with the patients changes could assist in providing an effective and personalized postoperative care. ML models predictive capacity improvement with follow-up data is encouraging and may become a valuable support in patient management and decision-making.

Partial Hepatic Vein Occlusion and Venous Congestion in Liver Exploration Using a Hyperspectral Camera: A Proposal for Monitoring Intraoperative Liver Perfusion.

INTRODUCTION: The changes occurring in the liver in cases of outflow deprivation have rarely been investigated, and no measurements of this phenomenon are available. This investigation explored outflow occlusion in a pig model using a hyperspectral camera. METHODS: Six pigs were enrolled. The right hepatic vein was clamped for 30 min. The oxygen saturation (StO2%), deoxygenated hemoglobin level (de-Hb), near-infrared perfusion (NIR), and total hemoglobin index (THI) were investigated at different time points in four perfused lobes using a hyperspectral camera measuring light absorbance between 500 nm and 995 nm. Differences among lobes at different time points were estimated by mixed-effect linear regression. RESULTS: StO2% decreased over time in the right lateral lobe (RLL, totally occluded) when compared to the left lateral (LLL, outflow preserved) and the right medial (RML, partially occluded) lobes (p < 0.05). De-Hb significantly increased after clamping in RLL when compared to RML and LLL (p < 0.05). RML was further analyzed considering the right portion (totally occluded) and the left portion of the lobe (with an autonomous draining vein). StO2% decreased and de-Hb increased more smoothly when compared to the totally occluded RLL (p < 0.05). CONCLUSIONS: The variations of StO2% and deoxy-Hb could be considered good markers of venous liver congestion.

Towards automatic verification of the critical view of the myopectineal orifice with artificial intelligence.

BACKGROUND: Visualization of key anatomical landmarks is required during surgical Trans Abdominal Pre Peritoneal repair (TAPP) of inguinal hernia. The Critical View of the MyoPectineal Orifice (CVMPO) was proposed to ensure correct dissection. An artificial intelligence (AI) system that automatically validates the presence of key and marks during the procedure is a critical step towards automatic dissection quality assessment and video-based competency evaluation. The aim of this study was to develop an AI system that automatically recognizes the TAPP key CVMPO landmarks in hernia repair videos. METHODS: Surgical videos of 160 TAPP procedures were used in this single-center study. A deep neural network-based object detector was developed to automatically recognize the pubic symphysis, direct hernia orifice, Cooper's ligament, the iliac vein, triangle of Doom, deep inguinal ring, and iliopsoas muscle. The system was trained using 130 videos, annotated and verified by two board-certified surgeons. Performance was evaluated in 30 videos of new patients excluded from the training data. RESULTS: Performance was validated in 2 ways: first, single-image validation where the AI model detected landmarks in a single laparoscopic image (mean average precision (MAP) of 51.2%). The second validation is video evaluation where the model detected landmarks throughout the myopectineal orifice visual inspection phase (mean accuracy and F-score of 77.1 and 75.4% respectively). Annotation objectivity was assessed between 2 surgeons in video evaluation, showing a high agreement of 88.3%. CONCLUSION: This study establishes the first AI-based automated recognition of critical structures in TAPP surgical videos, and a major step towards automatic CVMPO validation with AI. Strong performance was achieved in the video evaluation. The high inter-rater agreement confirms annotation quality and task objectivity.

Hyperspectral Imaging in Major Hepatectomies: Preliminary Results from the Ex-Machyna Trial.

Ischemia-reperfusion injury during major hepatic resections is associated with high rates of post-operative complications and liver failure. Real-time intra-operative detection of liver dysfunction could provide great insight into clinical outcomes. In the present study, we demonstrate the intra-operative application of a novel optical technology, hyperspectral imaging (HSI), to predict short-term post-operative outcomes after major hepatectomy. We considered fifteen consecutive patients undergoing major hepatic resection for malignant liver lesions from January 2020 to June 2021. HSI measures included tissue water index (TWI), organ hemoglobin index (OHI), tissue oxygenation (StO2%), and near infrared (NIR). Pre-operative, intra-operative, and post-operative serum and clinical outcomes were collected. NIR values were higher in unhealthy liver tissue (p = 0.003). StO2% negatively correlated with post-operative serum ALT values (r = -0.602), while ΔStO2% positively correlated with ALP (r = 0.594). TWI significantly correlated with post-operative reintervention and OHI with post-operative sepsis and liver failure. In conclusion, the HSI imaging system is accurate and precise in translating from pre-clinical to human studies in this first clinical trial. HSI indices are related to serum and outcome metrics. Further experimental and clinical studies are necessary to determine clinical value of this technology.

Computer-Assisted Differentiation between Colon-Mesocolon and Retroperitoneum Using Hyperspectral Imaging (HSI) Technology.

Complete mesocolic excision (CME), which involves the adequate resection of the tumor-bearing colonic segment with "en bloc" removal of its mesocolon along embryological fascial planes is associated with superior oncological outcomes. However, CME presents a higher complication rate compared to non-CME resections due to a higher risk of vascular injury. Hyperspectral imaging (HSI) is a contrast-free optical imaging technology, which facilitates the quantitative imaging of physiological tissue parameters and the visualization of anatomical structures. This study evaluates the accuracy of HSI combined with deep learning (DL) to differentiate the colon and its mesenteric tissue from retroperitoneal tissue. In an animal study including 20 pig models, intraoperative hyperspectral images of the sigmoid colon, sigmoid mesentery, and retroperitoneum were recorded. A convolutional neural network (CNN) was trained to distinguish the two tissue classes using HSI data, validated with a leave-one-out cross-validation process. The overall recognition sensitivity of the tissues to be preserved (retroperitoneum) and the tissues to be resected (colon and mesentery) was 79.0 ± 21.0% and 86.0 ± 16.0%, respectively. Automatic classification based on HSI and CNNs is a promising tool to automatically, non-invasively, and objectively differentiate the colon and its mesentery from retroperitoneal tissue.

Automatic optical biopsy for colorectal cancer using hyperspectral imaging and artificial neural networks.

BACKGROUND: Intraoperative identification of cancerous tissue is fundamental during oncological surgical or endoscopic procedures. This relies on visual assessment supported by histopathological evaluation, implying a longer operative time. Hyperspectral imaging (HSI), a contrast-free and contactless imaging technology, provides spatially resolved spectroscopic analysis, with the potential to differentiate tissue at a cellular level. However, HSI produces "big data", which is impossible to directly interpret by clinicians. We hypothesize that advanced machine learning algorithms (convolutional neural networks-CNNs) can accurately detect colorectal cancer in HSI data. METHODS: In 34 patients undergoing colorectal resections for cancer, immediately after extraction, the specimen was opened, the tumor-bearing section was exposed and imaged using HSI. Cancer and normal mucosa were categorized from histopathology. A state-of-the-art CNN was developed to automatically detect regions of colorectal cancer in a hyperspectral image. Accuracy was validated with three levels of cross-validation (twofold, fivefold, and 15-fold). RESULTS: 32 patients had colorectal adenocarcinomas confirmed by histopathology (9 left, 11 right, 4 transverse colon, and 9 rectum). 6 patients had a local initial stage (T1-2) and 26 had a local advanced stage (T3-4). The cancer detection performance of the CNN using 15-fold cross-validation showed high sensitivity and specificity (87% and 90%, respectively) and a ROC-AUC score of 0.95 (considered outstanding). In the T1-2 group, the sensitivity and specificity were 89% and 90%, respectively, and in the T3-4 group, the sensitivity and specificity were 81% and 93%, respectively. CONCLUSIONS: Automatic colorectal cancer detection on fresh specimens using HSI, using a properly trained CNN is feasible and accurate, even with small datasets, regardless of the local tumor extension. In the near future, this approach may become a useful intraoperative tool during oncological endoscopic and surgical procedures, and may result in precise and non-destructive optical biopsies to support objective and consistent tumor-free resection margins.

Knowledge, attitude, and practice of artificial intelligence in emergency and trauma surgery, the ARIES project: an international web-based survey.

AIM: We aimed to evaluate the knowledge, attitude, and practices in the application of AI in the emergency setting among international acute care and emergency surgeons. METHODS: An online questionnaire composed of 30 multiple choice and open-ended questions was sent to the members of the World Society of Emergency Surgery between 29th May and 28th August 2021. The questionnaire was developed by a panel of 11 international experts and approved by the WSES steering committee. RESULTS: 200 participants answered the survey, 32 were females (16%). 172 (86%) surgeons thought that AI will improve acute care surgery. Fifty surgeons (25%) were trained, robotic surgeons and can perform it. Only 19 (9.5%) were currently performing it. 126 (63%) surgeons do not have a robotic system in their institution, and for those who have it, it was mainly used for elective surgery. Only 100 surgeons (50%) were able to define different AI terminology. Participants thought that AI is useful to support training and education (61.5%), perioperative decision making (59.5%), and surgical vision (53%) in emergency surgery. There was no statistically significant difference between males and females in ability, interest in training or expectations of AI (p values 0.91, 0.82, and 0.28, respectively, Mann-Whitney U test). Ability was significantly correlated with interest and expectations (p < 0.0001 Pearson rank correlation, rho 0.42 and 0.47, respectively) but not with experience (p = 0.9, rho - 0.01). CONCLUSIONS: The implementation of artificial intelligence in the emergency and trauma setting is still in an early phase. The support of emergency and trauma surgeons is essential for the progress of AI in their setting which can be augmented by proper research and training programs in this area.

StaSiS-Net: A stacked and siamese disparity estimation network for depth reconstruction in modern 3D laparoscopy.

Developing accurate and real-time algorithms for a non-invasive three-dimensional representation and reconstruction of internal patient structures is one of the main research fields in computer-assisted surgery and endoscopy. Mono and stereo endoscopic images of soft tissues are converted into a three-dimensional representation by the estimation of depth maps. However, automatic, detailed, accurate and robust depth map estimation is a challenging problem that, in the stereo setting, is strictly dependent on a robust estimate of the disparity map. Many traditional algorithms are often inefficient or not accurate. In this work, novel self-supervised stacked and Siamese encoder/decoder neural networks are proposed to compute accurate disparity maps for 3D laparoscopy depth estimation. These networks run in real-time on standard GPU-equipped desktop computers and the outputs may be used for depth map estimation using the a known camera calibration. We compare performance on three different public datasets and on a new challenging simulated dataset and our solutions outperform state-of-the-art mono and stereo depth estimation methods. Extensive robustness and sensitivity analyses on more than 30000 frames has been performed. This work leads to important improvements in mono and stereo real-time depth map estimation of soft tissues and organs with a very low average mean absolute disparity reconstruction error with respect to ground truth.

Surgical data science - from concepts toward clinical translation.

Recent developments in data science in general and machine learning in particular have transformed the way experts envision the future of surgery. Surgical Data Science (SDS) is a new research field that aims to improve the quality of interventional healthcare through the capture, organization, analysis and modeling of data. While an increasing number of data-driven approaches and clinical applications have been studied in the fields of radiological and clinical data science, translational success stories are still lacking in surgery. In this publication, we shed light on the underlying reasons and provide a roadmap for future advances in the field. Based on an international workshop involving leading researchers in the field of SDS, we review current practice, key achievements and initiatives as well as available standards and tools for a number of topics relevant to the field, namely (1) infrastructure for data acquisition, storage and access in the presence of regulatory constraints, (2) data annotation and sharing and (3) data analytics. We further complement this technical perspective with (4) a review of currently available SDS products and the translational progress from academia and (5) a roadmap for faster clinical translation and exploitation of the full potential of SDS, based on an international multi-round Delphi process.

Intraoperative Guidance Using Hyperspectral Imaging: A Review for Surgeons.

Hyperspectral imaging (HSI) is a novel optical imaging modality, which has recently found diverse applications in the medical field. HSI is a hybrid imaging modality, combining a digital photographic camera with a spectrographic unit, and it allows for a contactless and non-destructive biochemical analysis of living tissue. HSI provides quantitative and qualitative information of the tissue composition at molecular level in a contrast-free manner, hence making it possible to objectively discriminate between different tissue types and between healthy and pathological tissue. Over the last two decades, HSI has been increasingly used in the medical field, and only recently it has found an application in the operating room. In the last few years, several research groups have used this imaging modality as an intraoperative guidance tool within different surgical disciplines. Despite its great potential, HSI still remains far from being routinely used in the daily surgical practice, since it is still largely unknown to most of the surgical community. The aim of this study is to provide clinical surgeons with an overview of the capabilities, current limitations, and future directions of HSI for intraoperative guidance.

Automatic Recognition of Colon and Esophagogastric Cancer with Machine Learning and Hyperspectral Imaging.

There are approximately 1.8 million diagnoses of colorectal cancer, 1 million diagnoses of stomach cancer, and 0.6 million diagnoses of esophageal cancer each year globally. An automatic computer-assisted diagnostic (CAD) tool to rapidly detect colorectal and esophagogastric cancer tissue in optical images would be hugely valuable to a surgeon during an intervention. Based on a colon dataset with 12 patients and an esophagogastric dataset of 10 patients, several state-of-the-art machine learning methods have been trained to detect cancer tissue using hyperspectral imaging (HSI), including Support Vector Machines (SVM) with radial basis function kernels, Multi-Layer Perceptrons (MLP) and 3D Convolutional Neural Networks (3DCNN). A leave-one-patient-out cross-validation (LOPOCV) with and without combining these sets was performed. The ROC-AUC score of the 3DCNN was slightly higher than the MLP and SVM with a difference of 0.04 AUC. The best performance was achieved with the 3DCNN for colon cancer and esophagogastric cancer detection with a high ROC-AUC of 0.93. The 3DCNN also achieved the best DICE scores of 0.49 and 0.41 on the colon and esophagogastric datasets, respectively. These scores were significantly improved using a patient-specific decision threshold to 0.58 and 0.51, respectively. This indicates that, in practical use, an HSI-based CAD system using an interactive decision threshold is likely to be valuable. Experiments were also performed to measure the benefits of combining the colorectal and esophagogastric datasets (22 patients), and this yielded significantly better results with the MLP and SVM models.

Prediction of In Vivo Laser-Induced Thermal Damage with Hyperspectral Imaging Using Deep Learning.

Thermal ablation is an acceptable alternative treatment for primary liver cancer, of which laser ablation (LA) is one of the least invasive approaches, especially for tumors in high-risk locations. Precise control of the LA effect is required to safely destroy the tumor. Although temperature imaging techniques provide an indirect measurement of the thermal damage, a degree of uncertainty remains about the treatment effect. Optical techniques are currently emerging as tools to directly assess tissue thermal damage. Among them, hyperspectral imaging (HSI) has shown promising results in image-guided surgery and in the thermal ablation field. The highly informative data provided by HSI, associated with deep learning, enable the implementation of non-invasive prediction models to be used intraoperatively. Here we show a novel paradigm "peak temperature prediction model" (PTPM), convolutional neural network (CNN)-based, trained with HSI and infrared imaging to predict LA-induced damage in the liver. The PTPM demonstrated an optimal agreement with tissue damage classification providing a consistent threshold (50.6 ± 1.5 °C) for the damage margins with high accuracy (~0.90). The high correlation with the histology score (r = 0.9085) and the comparison with the measured peak temperature confirmed that PTPM preserves temperature information accordingly with the histopathological assessment.

Automatic Liver Viability Scoring with Deep Learning and Hyperspectral Imaging.

Hyperspectral imaging (HSI) is a non-invasive imaging modality already applied to evaluate hepatic oxygenation and to discriminate different models of hepatic ischemia. Nevertheless, the ability of HSI to detect and predict the reperfusion damage intraoperatively was not yet assessed. Hypoxia caused by hepatic artery occlusion (HAO) in the liver brings about dreadful vascular complications known as ischemia-reperfusion injury (IRI). Here, we show the evaluation of liver viability in an HAO model with an artificial intelligence-based analysis of HSI. We have combined the potential of HSI to extract quantitative optical tissue properties with a deep learning-based model using convolutional neural networks. The artificial intelligence (AI) score of liver viability showed a significant correlation with capillary lactate from the liver surface (r = -0.78, p = 0.0320) and Suzuki's score (r = -0.96, p = 0.0012). CD31 immunostaining confirmed the microvascular damage accordingly with the AI score. Our results ultimately show the potential of an HSI-AI-based analysis to predict liver viability, thereby prompting for intraoperative tool development to explore its application in a clinical setting.

Deep Learning Analysis of In Vivo Hyperspectral Images for Automated Intraoperative Nerve Detection.

Nerves are critical structures that may be difficult to recognize during surgery. Inadvertent nerve injuries can have catastrophic consequences for the patient and lead to life-long pain and a reduced quality of life. Hyperspectral imaging (HSI) is a non-invasive technique combining photography with spectroscopy, allowing non-invasive intraoperative biological tissue property quantification. We show, for the first time, that HSI combined with deep learning allows nerves and other tissue types to be automatically recognized in in vivo hyperspectral images. An animal model was used, and eight anesthetized pigs underwent neck midline incisions, exposing several structures (nerve, artery, vein, muscle, fat, skin). State-of-the-art machine learning models were trained to recognize these tissue types in HSI data. The best model was a convolutional neural network (CNN), achieving an overall average sensitivity of 0.91 and a specificity of 1.0, validated with leave-one-patient-out cross-validation. For the nerve, the CNN achieved an average sensitivity of 0.76 and a specificity of 0.99. In conclusion, HSI combined with a CNN model is suitable for in vivo nerve recognition.

A novel knockout mouse for the small EDRK-rich factor 2 (Serf2) showing developmental and other deficits.

The small EDRK-rich factor 2 (SERF2) is a highly conserved protein that modifies amyloid fibre assembly in vitro and promotes protein misfolding. However, the role of SERF2 in regulating age-related proteotoxicity remains largely unexplored due to a lack of in vivo models. Here, we report the generation of Serf2 knockout mice using an ES cell targeting approach, with Serf2 knockout alleles being bred onto different defined genetic backgrounds. We highlight phenotyping data from heterozygous Serf2+/- mice, including unexpected male-specific phenotypes in startle response and pre-pulse inhibition. We report embryonic lethality in Serf2-/- null animals when bred onto a C57BL/6 N background. However, homozygous null animals were viable on a mixed genetic background and, remarkably, developed without obvious abnormalities. The Serf2 knockout mice provide a powerful tool to further investigate the role of SERF2 protein in previously unexplored pathophysiological pathways in the context of a whole organism.

Bridging the Gap Between RCTs and RWE Through Endpoint Selection.

This commentary is authored by several industry real-world evidence (RWE) experts, with support from IQVIA, as part of the 'RWE Leadership Forum': a group of Industry Leaders who have come together as non-competitive partners to understand and respond to RWD/E challenges and opportunities with a single expert voice. Here, the forum discusses the value in bridging the industry disconnect between RTCs and RWE, with a view to promoting the use of RWE in the RCT environment. RCT endpoints are explored along several axes including their clinical relevance and their measure of direct patient benefit, and then compared with their real-world counterparts to identify suitable paths, or gaps, for assimilating RWE endpoints into the RCT environment.

Hyperspectral evaluation of hepatic oxygenation in a model of total vs. arterial liver ischaemia.

Liver ischaemia reperfusion injury (IRI) is a dreaded pathophysiological complication which may lead to an impaired liver function. The level of oxygen hypoperfusion affects the level of cellular damage during the reperfusion phase. Consequently, intraoperative localisation and quantification of oxygen impairment would help in the early detection of liver ischaemia. To date, there is no real-time, non-invasive, and intraoperative tool which can compute an organ oxygenation map, quantify and discriminate different types of vascular occlusions intraoperatively. Hyperspectral imaging (HSI) is a non-invasive optical methodology which can quantify tissue oxygenation and which has recently been applied to the medical field. A hyperspectral camera detects the relative reflectance of a tissue in the range of 500 to 1000 nm, allowing the quantification of organic compounds such as oxygenated and deoxygenated haemoglobin at different depths. Here, we show the first comparative study of liver oxygenation by means of HSI quantification in a model of total vascular inflow occlusion (VIO) vs. hepatic artery occlusion (HAO), correlating optical properties with capillary lactate and histopathological evaluation. We found that liver HSI could discriminate between VIO and HAO. These results were confirmed via cross-validation of HSI which detected and quantified intestinal congestion in VIO. A significant correlation between the near-infrared spectra and capillary lactate was found (r = - 0.8645, p = 0.0003 VIO, r = - 0.7113, p = 0.0120 HAO). Finally, a statistically significant negative correlation was found between the histology score and the near-infrared parameter index (NIR) (r = - 0.88, p = 0.004). We infer that HSI, by predicting capillary lactates and the histopathological score, would be a suitable non-invasive tool for intraoperative liver perfusion assessment.

Automatic task recognition in a flexible endoscopy benchtop trainer with semi-supervised learning.

PURPOSE: Inexpensive benchtop training systems offer significant advantages to meet the increasing demand of training surgeons and gastroenterologists in flexible endoscopy. Established scoring systems exist, based on task duration and mistake evaluation. However, they require trained human raters, which limits broad and low-cost adoption. There is an unmet and important need to automate rating with machine learning. METHOD: We present a general and robust approach for recognizing training tasks from endoscopic training video, which consequently automates task duration computation. Our main technical novelty is to show the performance of state-of-the-art CNN-based approaches can be improved significantly with a novel semi-supervised learning approach, using both labelled and unlabelled videos. In the latter case, we assume only the task execution order is known a priori. RESULTS: Two video datasets are presented: the first has 19 videos recorded in examination conditions, where the participants complete their tasks in predetermined order. The second has 17 h of videos recorded in self-assessment conditions, where participants complete one or more tasks in any order. For the first dataset, we obtain a mean task duration estimation error of 3.65 s, with a mean task duration of 159 s ([Formula: see text] relative error). For the second dataset, we obtain a mean task duration estimation error of 3.67 s. We reduce an average of 5.63% in error to 3.67% thanks to our semi-supervised learning approach. CONCLUSION: This work is the first significant step forward to automate rating of flexible endoscopy students using a low-cost benchtop trainer. Thanks to our semi-supervised learning approach, we can scale easily to much larger unlabelled training datasets. The approach can also be used for other phase recognition tasks.