RESUMO
BACKGROUND: Hyperspectral imaging (HSI), combined with machine learning, can help to identify characteristic tissue signatures enabling automatic tissue recognition during surgery. This study aims to develop the first HSI-based automatic abdominal tissue recognition with human data in a prospective bi-center setting. METHODS: Data were collected from patients undergoing elective open abdominal surgery at two international tertiary referral hospitals from September 2020 to June 2021. HS images were captured at various time points throughout the surgical procedure. Resulting RGB images were annotated with 13 distinct organ labels. Convolutional Neural Networks (CNNs) were employed for the analysis, with both external and internal validation settings utilized. RESULTS: A total of 169 patients were included, 73 (43.2%) from Strasbourg and 96 (56.8%) from Verona. The internal validation within centers combined patients from both centers into a single cohort, randomly allocated to the training (127 patients, 75.1%, 585 images) and test sets (42 patients, 24.9%, 181 images). This validation setting showed the best performance. The highest true positive rate was achieved for the skin (100%) and the liver (97%). Misclassifications included tissues with a similar embryological origin (omentum and mesentery: 32%) or with overlaying boundaries (liver and hepatic ligament: 22%). The median DICE score for ten tissue classes exceeded 80%. CONCLUSION: To improve automatic surgical scene segmentation and to drive clinical translation, multicenter accurate HSI datasets are essential, but further work is needed to quantify the clinical value of HSI. HSI might be included in a new omics science, namely surgical optomics, which uses light to extract quantifiable tissue features during surgery.
Assuntos
Aprendizado Profundo , Imageamento Hiperespectral , Humanos , Estudos Prospectivos , Imageamento Hiperespectral/métodos , Masculino , Feminino , Pessoa de Meia-Idade , Idoso , Abdome/cirurgia , Abdome/diagnóstico por imagem , Cirurgia Assistida por Computador/métodosRESUMO
BACKGROUND: The large amount of heterogeneous data collected in surgical/endoscopic practice calls for data-driven approaches as machine learning (ML) models. The aim of this study was to develop ML models to predict endoscopic sleeve gastroplasty (ESG) efficacy at 12 months defined by total weight loss (TWL) % and excess weight loss (EWL) % achievement. Multicentre data were used to enhance generalizability: evaluate consistency among different center of ESG practice and assess reproducibility of the models and possible clinical application. Models were designed to be dynamic and integrate follow-up clinical data into more accurate predictions, possibly assisting management and decision-making. METHODS: ML models were developed using data of 404 ESG procedures performed at 12 centers across Europe. Collected data included clinical and demographic variables at the time of ESG and at follow-up. Multicentre/external and single center/internal and temporal validation were performed. Training and evaluation of the models were performed on Python's scikit-learn library. Performance of models was quantified as receiver operator curve (ROC-AUC), sensitivity, specificity, and calibration plots. RESULTS: Multicenter external validation: ML models using preoperative data show poor performance. Best performances were reached by linear regression (LR) and support vector machine models for TWL% and EWL%, respectively, (ROC-AUC: TWL% 0.87, EWL% 0.86) with the addition of 6-month follow-up data. Single-center internal validation: Preoperative data only ML models show suboptimal performance. Early, i.e., 3-month follow-up data addition lead to ROC-AUC of 0.79 (random forest classifiers model) and 0.81 (LR models) for TWL% and EWL% achievement prediction, respectively. Single-center temporal validation shows similar results. CONCLUSIONS: Although preoperative data only may not be sufficient for accurate postoperative predictions, the ability of ML models to adapt and evolve with the patients changes could assist in providing an effective and personalized postoperative care. ML models predictive capacity improvement with follow-up data is encouraging and may become a valuable support in patient management and decision-making.
Assuntos
Gastroplastia , Obesidade Mórbida , Humanos , Gastroplastia/métodos , Obesidade/cirurgia , Reprodutibilidade dos Testes , Resultado do Tratamento , Redução de Peso , Aprendizado de Máquina , Obesidade Mórbida/cirurgiaRESUMO
INTRODUCTION: The changes occurring in the liver in cases of outflow deprivation have rarely been investigated, and no measurements of this phenomenon are available. This investigation explored outflow occlusion in a pig model using a hyperspectral camera. METHODS: Six pigs were enrolled. The right hepatic vein was clamped for 30 min. The oxygen saturation (StO2%), deoxygenated hemoglobin level (de-Hb), near-infrared perfusion (NIR), and total hemoglobin index (THI) were investigated at different time points in four perfused lobes using a hyperspectral camera measuring light absorbance between 500 nm and 995 nm. Differences among lobes at different time points were estimated by mixed-effect linear regression. RESULTS: StO2% decreased over time in the right lateral lobe (RLL, totally occluded) when compared to the left lateral (LLL, outflow preserved) and the right medial (RML, partially occluded) lobes (p < 0.05). De-Hb significantly increased after clamping in RLL when compared to RML and LLL (p < 0.05). RML was further analyzed considering the right portion (totally occluded) and the left portion of the lobe (with an autonomous draining vein). StO2% decreased and de-Hb increased more smoothly when compared to the totally occluded RLL (p < 0.05). CONCLUSIONS: The variations of StO2% and deoxy-Hb could be considered good markers of venous liver congestion.
RESUMO
BACKGROUND: Visualization of key anatomical landmarks is required during surgical Trans Abdominal Pre Peritoneal repair (TAPP) of inguinal hernia. The Critical View of the MyoPectineal Orifice (CVMPO) was proposed to ensure correct dissection. An artificial intelligence (AI) system that automatically validates the presence of key and marks during the procedure is a critical step towards automatic dissection quality assessment and video-based competency evaluation. The aim of this study was to develop an AI system that automatically recognizes the TAPP key CVMPO landmarks in hernia repair videos. METHODS: Surgical videos of 160 TAPP procedures were used in this single-center study. A deep neural network-based object detector was developed to automatically recognize the pubic symphysis, direct hernia orifice, Cooper's ligament, the iliac vein, triangle of Doom, deep inguinal ring, and iliopsoas muscle. The system was trained using 130 videos, annotated and verified by two board-certified surgeons. Performance was evaluated in 30 videos of new patients excluded from the training data. RESULTS: Performance was validated in 2 ways: first, single-image validation where the AI model detected landmarks in a single laparoscopic image (mean average precision (MAP) of 51.2%). The second validation is video evaluation where the model detected landmarks throughout the myopectineal orifice visual inspection phase (mean accuracy and F-score of 77.1 and 75.4% respectively). Annotation objectivity was assessed between 2 surgeons in video evaluation, showing a high agreement of 88.3%. CONCLUSION: This study establishes the first AI-based automated recognition of critical structures in TAPP surgical videos, and a major step towards automatic CVMPO validation with AI. Strong performance was achieved in the video evaluation. The high inter-rater agreement confirms annotation quality and task objectivity.
Assuntos
Hérnia Inguinal , Laparoscopia , Cirurgiões , Humanos , Inteligência Artificial , Laparoscopia/métodos , Peritônio , Hérnia Inguinal/cirurgiaRESUMO
Ischemia-reperfusion injury during major hepatic resections is associated with high rates of post-operative complications and liver failure. Real-time intra-operative detection of liver dysfunction could provide great insight into clinical outcomes. In the present study, we demonstrate the intra-operative application of a novel optical technology, hyperspectral imaging (HSI), to predict short-term post-operative outcomes after major hepatectomy. We considered fifteen consecutive patients undergoing major hepatic resection for malignant liver lesions from January 2020 to June 2021. HSI measures included tissue water index (TWI), organ hemoglobin index (OHI), tissue oxygenation (StO2%), and near infrared (NIR). Pre-operative, intra-operative, and post-operative serum and clinical outcomes were collected. NIR values were higher in unhealthy liver tissue (p = 0.003). StO2% negatively correlated with post-operative serum ALT values (r = -0.602), while ΔStO2% positively correlated with ALP (r = 0.594). TWI significantly correlated with post-operative reintervention and OHI with post-operative sepsis and liver failure. In conclusion, the HSI imaging system is accurate and precise in translating from pre-clinical to human studies in this first clinical trial. HSI indices are related to serum and outcome metrics. Further experimental and clinical studies are necessary to determine clinical value of this technology.
RESUMO
Complete mesocolic excision (CME), which involves the adequate resection of the tumor-bearing colonic segment with "en bloc" removal of its mesocolon along embryological fascial planes is associated with superior oncological outcomes. However, CME presents a higher complication rate compared to non-CME resections due to a higher risk of vascular injury. Hyperspectral imaging (HSI) is a contrast-free optical imaging technology, which facilitates the quantitative imaging of physiological tissue parameters and the visualization of anatomical structures. This study evaluates the accuracy of HSI combined with deep learning (DL) to differentiate the colon and its mesenteric tissue from retroperitoneal tissue. In an animal study including 20 pig models, intraoperative hyperspectral images of the sigmoid colon, sigmoid mesentery, and retroperitoneum were recorded. A convolutional neural network (CNN) was trained to distinguish the two tissue classes using HSI data, validated with a leave-one-out cross-validation process. The overall recognition sensitivity of the tissues to be preserved (retroperitoneum) and the tissues to be resected (colon and mesentery) was 79.0 ± 21.0% and 86.0 ± 16.0%, respectively. Automatic classification based on HSI and CNNs is a promising tool to automatically, non-invasively, and objectively differentiate the colon and its mesentery from retroperitoneal tissue.
RESUMO
BACKGROUND: Intraoperative identification of cancerous tissue is fundamental during oncological surgical or endoscopic procedures. This relies on visual assessment supported by histopathological evaluation, implying a longer operative time. Hyperspectral imaging (HSI), a contrast-free and contactless imaging technology, provides spatially resolved spectroscopic analysis, with the potential to differentiate tissue at a cellular level. However, HSI produces "big data", which is impossible to directly interpret by clinicians. We hypothesize that advanced machine learning algorithms (convolutional neural networks-CNNs) can accurately detect colorectal cancer in HSI data. METHODS: In 34 patients undergoing colorectal resections for cancer, immediately after extraction, the specimen was opened, the tumor-bearing section was exposed and imaged using HSI. Cancer and normal mucosa were categorized from histopathology. A state-of-the-art CNN was developed to automatically detect regions of colorectal cancer in a hyperspectral image. Accuracy was validated with three levels of cross-validation (twofold, fivefold, and 15-fold). RESULTS: 32 patients had colorectal adenocarcinomas confirmed by histopathology (9 left, 11 right, 4 transverse colon, and 9 rectum). 6 patients had a local initial stage (T1-2) and 26 had a local advanced stage (T3-4). The cancer detection performance of the CNN using 15-fold cross-validation showed high sensitivity and specificity (87% and 90%, respectively) and a ROC-AUC score of 0.95 (considered outstanding). In the T1-2 group, the sensitivity and specificity were 89% and 90%, respectively, and in the T3-4 group, the sensitivity and specificity were 81% and 93%, respectively. CONCLUSIONS: Automatic colorectal cancer detection on fresh specimens using HSI, using a properly trained CNN is feasible and accurate, even with small datasets, regardless of the local tumor extension. In the near future, this approach may become a useful intraoperative tool during oncological endoscopic and surgical procedures, and may result in precise and non-destructive optical biopsies to support objective and consistent tumor-free resection margins.
Assuntos
Neoplasias Colorretais , Imageamento Hiperespectral , Humanos , Redes Neurais de Computação , Algoritmos , Margens de Excisão , Neoplasias Colorretais/diagnóstico por imagem , Neoplasias Colorretais/cirurgia , BiópsiaRESUMO
AIM: We aimed to evaluate the knowledge, attitude, and practices in the application of AI in the emergency setting among international acute care and emergency surgeons. METHODS: An online questionnaire composed of 30 multiple choice and open-ended questions was sent to the members of the World Society of Emergency Surgery between 29th May and 28th August 2021. The questionnaire was developed by a panel of 11 international experts and approved by the WSES steering committee. RESULTS: 200 participants answered the survey, 32 were females (16%). 172 (86%) surgeons thought that AI will improve acute care surgery. Fifty surgeons (25%) were trained, robotic surgeons and can perform it. Only 19 (9.5%) were currently performing it. 126 (63%) surgeons do not have a robotic system in their institution, and for those who have it, it was mainly used for elective surgery. Only 100 surgeons (50%) were able to define different AI terminology. Participants thought that AI is useful to support training and education (61.5%), perioperative decision making (59.5%), and surgical vision (53%) in emergency surgery. There was no statistically significant difference between males and females in ability, interest in training or expectations of AI (p values 0.91, 0.82, and 0.28, respectively, Mann-Whitney U test). Ability was significantly correlated with interest and expectations (p < 0.0001 Pearson rank correlation, rho 0.42 and 0.47, respectively) but not with experience (p = 0.9, rho - 0.01). CONCLUSIONS: The implementation of artificial intelligence in the emergency and trauma setting is still in an early phase. The support of emergency and trauma surgeons is essential for the progress of AI in their setting which can be augmented by proper research and training programs in this area.
Assuntos
Inteligência Artificial , Cirurgiões , Feminino , Conhecimentos, Atitudes e Prática em Saúde , Humanos , Internet , Masculino , Inquéritos e QuestionáriosRESUMO
Developing accurate and real-time algorithms for a non-invasive three-dimensional representation and reconstruction of internal patient structures is one of the main research fields in computer-assisted surgery and endoscopy. Mono and stereo endoscopic images of soft tissues are converted into a three-dimensional representation by the estimation of depth maps. However, automatic, detailed, accurate and robust depth map estimation is a challenging problem that, in the stereo setting, is strictly dependent on a robust estimate of the disparity map. Many traditional algorithms are often inefficient or not accurate. In this work, novel self-supervised stacked and Siamese encoder/decoder neural networks are proposed to compute accurate disparity maps for 3D laparoscopy depth estimation. These networks run in real-time on standard GPU-equipped desktop computers and the outputs may be used for depth map estimation using the a known camera calibration. We compare performance on three different public datasets and on a new challenging simulated dataset and our solutions outperform state-of-the-art mono and stereo depth estimation methods. Extensive robustness and sensitivity analyses on more than 30000 frames has been performed. This work leads to important improvements in mono and stereo real-time depth map estimation of soft tissues and organs with a very low average mean absolute disparity reconstruction error with respect to ground truth.
Assuntos
Laparoscopia , Cirurgia Assistida por Computador , Algoritmos , Humanos , Imageamento Tridimensional/métodos , Redes Neurais de Computação , Cirurgia Assistida por Computador/métodosRESUMO
Recent developments in data science in general and machine learning in particular have transformed the way experts envision the future of surgery. Surgical Data Science (SDS) is a new research field that aims to improve the quality of interventional healthcare through the capture, organization, analysis and modeling of data. While an increasing number of data-driven approaches and clinical applications have been studied in the fields of radiological and clinical data science, translational success stories are still lacking in surgery. In this publication, we shed light on the underlying reasons and provide a roadmap for future advances in the field. Based on an international workshop involving leading researchers in the field of SDS, we review current practice, key achievements and initiatives as well as available standards and tools for a number of topics relevant to the field, namely (1) infrastructure for data acquisition, storage and access in the presence of regulatory constraints, (2) data annotation and sharing and (3) data analytics. We further complement this technical perspective with (4) a review of currently available SDS products and the translational progress from academia and (5) a roadmap for faster clinical translation and exploitation of the full potential of SDS, based on an international multi-round Delphi process.
Assuntos
Ciência de Dados , Aprendizado de Máquina , HumanosRESUMO
Hyperspectral imaging (HSI) is a novel optical imaging modality, which has recently found diverse applications in the medical field. HSI is a hybrid imaging modality, combining a digital photographic camera with a spectrographic unit, and it allows for a contactless and non-destructive biochemical analysis of living tissue. HSI provides quantitative and qualitative information of the tissue composition at molecular level in a contrast-free manner, hence making it possible to objectively discriminate between different tissue types and between healthy and pathological tissue. Over the last two decades, HSI has been increasingly used in the medical field, and only recently it has found an application in the operating room. In the last few years, several research groups have used this imaging modality as an intraoperative guidance tool within different surgical disciplines. Despite its great potential, HSI still remains far from being routinely used in the daily surgical practice, since it is still largely unknown to most of the surgical community. The aim of this study is to provide clinical surgeons with an overview of the capabilities, current limitations, and future directions of HSI for intraoperative guidance.
RESUMO
Thermal ablation is an acceptable alternative treatment for primary liver cancer, of which laser ablation (LA) is one of the least invasive approaches, especially for tumors in high-risk locations. Precise control of the LA effect is required to safely destroy the tumor. Although temperature imaging techniques provide an indirect measurement of the thermal damage, a degree of uncertainty remains about the treatment effect. Optical techniques are currently emerging as tools to directly assess tissue thermal damage. Among them, hyperspectral imaging (HSI) has shown promising results in image-guided surgery and in the thermal ablation field. The highly informative data provided by HSI, associated with deep learning, enable the implementation of non-invasive prediction models to be used intraoperatively. Here we show a novel paradigm "peak temperature prediction model" (PTPM), convolutional neural network (CNN)-based, trained with HSI and infrared imaging to predict LA-induced damage in the liver. The PTPM demonstrated an optimal agreement with tissue damage classification providing a consistent threshold (50.6 ± 1.5 °C) for the damage margins with high accuracy (~0.90). The high correlation with the histology score (r = 0.9085) and the comparison with the measured peak temperature confirmed that PTPM preserves temperature information accordingly with the histopathological assessment.
Assuntos
Aprendizado Profundo , Terapia a Laser , Imageamento Hiperespectral , Lasers , Redes Neurais de ComputaçãoRESUMO
There are approximately 1.8 million diagnoses of colorectal cancer, 1 million diagnoses of stomach cancer, and 0.6 million diagnoses of esophageal cancer each year globally. An automatic computer-assisted diagnostic (CAD) tool to rapidly detect colorectal and esophagogastric cancer tissue in optical images would be hugely valuable to a surgeon during an intervention. Based on a colon dataset with 12 patients and an esophagogastric dataset of 10 patients, several state-of-the-art machine learning methods have been trained to detect cancer tissue using hyperspectral imaging (HSI), including Support Vector Machines (SVM) with radial basis function kernels, Multi-Layer Perceptrons (MLP) and 3D Convolutional Neural Networks (3DCNN). A leave-one-patient-out cross-validation (LOPOCV) with and without combining these sets was performed. The ROC-AUC score of the 3DCNN was slightly higher than the MLP and SVM with a difference of 0.04 AUC. The best performance was achieved with the 3DCNN for colon cancer and esophagogastric cancer detection with a high ROC-AUC of 0.93. The 3DCNN also achieved the best DICE scores of 0.49 and 0.41 on the colon and esophagogastric datasets, respectively. These scores were significantly improved using a patient-specific decision threshold to 0.58 and 0.51, respectively. This indicates that, in practical use, an HSI-based CAD system using an interactive decision threshold is likely to be valuable. Experiments were also performed to measure the benefits of combining the colorectal and esophagogastric datasets (22 patients), and this yielded significantly better results with the MLP and SVM models.
RESUMO
Hyperspectral imaging (HSI) is a non-invasive imaging modality already applied to evaluate hepatic oxygenation and to discriminate different models of hepatic ischemia. Nevertheless, the ability of HSI to detect and predict the reperfusion damage intraoperatively was not yet assessed. Hypoxia caused by hepatic artery occlusion (HAO) in the liver brings about dreadful vascular complications known as ischemia-reperfusion injury (IRI). Here, we show the evaluation of liver viability in an HAO model with an artificial intelligence-based analysis of HSI. We have combined the potential of HSI to extract quantitative optical tissue properties with a deep learning-based model using convolutional neural networks. The artificial intelligence (AI) score of liver viability showed a significant correlation with capillary lactate from the liver surface (r = -0.78, p = 0.0320) and Suzuki's score (r = -0.96, p = 0.0012). CD31 immunostaining confirmed the microvascular damage accordingly with the AI score. Our results ultimately show the potential of an HSI-AI-based analysis to predict liver viability, thereby prompting for intraoperative tool development to explore its application in a clinical setting.
RESUMO
Nerves are critical structures that may be difficult to recognize during surgery. Inadvertent nerve injuries can have catastrophic consequences for the patient and lead to life-long pain and a reduced quality of life. Hyperspectral imaging (HSI) is a non-invasive technique combining photography with spectroscopy, allowing non-invasive intraoperative biological tissue property quantification. We show, for the first time, that HSI combined with deep learning allows nerves and other tissue types to be automatically recognized in in vivo hyperspectral images. An animal model was used, and eight anesthetized pigs underwent neck midline incisions, exposing several structures (nerve, artery, vein, muscle, fat, skin). State-of-the-art machine learning models were trained to recognize these tissue types in HSI data. The best model was a convolutional neural network (CNN), achieving an overall average sensitivity of 0.91 and a specificity of 1.0, validated with leave-one-patient-out cross-validation. For the nerve, the CNN achieved an average sensitivity of 0.76 and a specificity of 0.99. In conclusion, HSI combined with a CNN model is suitable for in vivo nerve recognition.
RESUMO
The small EDRK-rich factor 2 (SERF2) is a highly conserved protein that modifies amyloid fibre assembly in vitro and promotes protein misfolding. However, the role of SERF2 in regulating age-related proteotoxicity remains largely unexplored due to a lack of in vivo models. Here, we report the generation of Serf2 knockout mice using an ES cell targeting approach, with Serf2 knockout alleles being bred onto different defined genetic backgrounds. We highlight phenotyping data from heterozygous Serf2+/- mice, including unexpected male-specific phenotypes in startle response and pre-pulse inhibition. We report embryonic lethality in Serf2-/- null animals when bred onto a C57BL/6 N background. However, homozygous null animals were viable on a mixed genetic background and, remarkably, developed without obvious abnormalities. The Serf2 knockout mice provide a powerful tool to further investigate the role of SERF2 protein in previously unexplored pathophysiological pathways in the context of a whole organism.
Assuntos
Deficiências do Desenvolvimento/diagnóstico , Deficiências do Desenvolvimento/genética , Estudos de Associação Genética , Predisposição Genética para Doença , Peptídeos e Proteínas de Sinalização Intracelular/genética , Fenótipo , Fatores Etários , Alelos , Processamento Alternativo , Animais , Linhagem Celular , Modelos Animais de Doenças , Células-Tronco Embrionárias/metabolismo , Feminino , Regulação da Expressão Gênica , Estudos de Associação Genética/métodos , Patrimônio Genético , Loci Gênicos , Genótipo , Masculino , Camundongos , Camundongos Knockout , Especificidade de Órgãos , Microtomografia por Raio-XRESUMO
This commentary is authored by several industry real-world evidence (RWE) experts, with support from IQVIA, as part of the 'RWE Leadership Forum': a group of Industry Leaders who have come together as non-competitive partners to understand and respond to RWD/E challenges and opportunities with a single expert voice. Here, the forum discusses the value in bridging the industry disconnect between RTCs and RWE, with a view to promoting the use of RWE in the RCT environment. RCT endpoints are explored along several axes including their clinical relevance and their measure of direct patient benefit, and then compared with their real-world counterparts to identify suitable paths, or gaps, for assimilating RWE endpoints into the RCT environment.