Surgical optomics: hyperspectral imaging and deep learning towards precision intraoperative automatic tissue recognition-results from the EX-MACHYNA trial.

BACKGROUND: Hyperspectral imaging (HSI), combined with machine learning, can help to identify characteristic tissue signatures enabling automatic tissue recognition during surgery. This study aims to develop the first HSI-based automatic abdominal tissue recognition with human data in a prospective bi-center setting. METHODS: Data were collected from patients undergoing elective open abdominal surgery at two international tertiary referral hospitals from September 2020 to June 2021. HS images were captured at various time points throughout the surgical procedure. Resulting RGB images were annotated with 13 distinct organ labels. Convolutional Neural Networks (CNNs) were employed for the analysis, with both external and internal validation settings utilized. RESULTS: A total of 169 patients were included, 73 (43.2%) from Strasbourg and 96 (56.8%) from Verona. The internal validation within centers combined patients from both centers into a single cohort, randomly allocated to the training (127 patients, 75.1%, 585 images) and test sets (42 patients, 24.9%, 181 images). This validation setting showed the best performance. The highest true positive rate was achieved for the skin (100%) and the liver (97%). Misclassifications included tissues with a similar embryological origin (omentum and mesentery: 32%) or with overlaying boundaries (liver and hepatic ligament: 22%). The median DICE score for ten tissue classes exceeded 80%. CONCLUSION: To improve automatic surgical scene segmentation and to drive clinical translation, multicenter accurate HSI datasets are essential, but further work is needed to quantify the clinical value of HSI. HSI might be included in a new omics science, namely surgical optomics, which uses light to extract quantifiable tissue features during surgery.

Partial Hepatic Vein Occlusion and Venous Congestion in Liver Exploration Using a Hyperspectral Camera: A Proposal for Monitoring Intraoperative Liver Perfusion.

INTRODUCTION: The changes occurring in the liver in cases of outflow deprivation have rarely been investigated, and no measurements of this phenomenon are available. This investigation explored outflow occlusion in a pig model using a hyperspectral camera. METHODS: Six pigs were enrolled. The right hepatic vein was clamped for 30 min. The oxygen saturation (StO2%), deoxygenated hemoglobin level (de-Hb), near-infrared perfusion (NIR), and total hemoglobin index (THI) were investigated at different time points in four perfused lobes using a hyperspectral camera measuring light absorbance between 500 nm and 995 nm. Differences among lobes at different time points were estimated by mixed-effect linear regression. RESULTS: StO2% decreased over time in the right lateral lobe (RLL, totally occluded) when compared to the left lateral (LLL, outflow preserved) and the right medial (RML, partially occluded) lobes (p < 0.05). De-Hb significantly increased after clamping in RLL when compared to RML and LLL (p < 0.05). RML was further analyzed considering the right portion (totally occluded) and the left portion of the lobe (with an autonomous draining vein). StO2% decreased and de-Hb increased more smoothly when compared to the totally occluded RLL (p < 0.05). CONCLUSIONS: The variations of StO2% and deoxy-Hb could be considered good markers of venous liver congestion.

In Vitro Antibody Quantification with Hyperspectral Imaging in a Large Field of View for Clinical Applications.

Hyperspectral imaging (HSI) is a non-invasive, contrast-free optical-based tool that has recently been applied in medical and basic research fields. The opportunity to use HSI to identify exogenous tumor markers in a large field of view (LFOV) could increase precision in oncological diagnosis and surgical treatment. In this study, the anti-high mobility group B1 (HMGB1) labeled with Alexa fluorophore (647 nm) was used as the target molecule. This is the proof-of-concept of HSI's ability to quantify antibodies via an in vitro setting. A first test was performed to understand whether the relative absorbance provided by the HSI camera was dependent on volume at a 1:1 concentration. A serial dilution of 1:1, 10, 100, 1000, and 10,000 with phosphatase-buffered saline (PBS) was then used to test the sensitivity of the camera at the minimum and maximum volumes. For the analysis, images at 640 nm were extracted from the hypercubes according to peak signals matching the specificities of the antibody manufacturer. The results showed a positive correlation between relative absorbance and volume (r = 0.9709, p = 0.0013). The correlation between concentration and relative absorbance at min (1 µL) and max (20 µL) volume showed r = 0.9925, p < 0.0001, and r = 0.9992, p < 0.0001, respectively. These results demonstrate the HSI potential in quantifying HMGB1, hence deserving further studies in ex vivo and in vivo settings.

Surgical Models of Liver Regeneration in Pigs: A Practical Review of the Literature for Researchers.

The remarkable capacity of regeneration of the liver is well known, although the involved mechanisms are far from being understood. Furthermore, limits concerning the residual functional mass of the liver remain critical in both fields of hepatic resection and transplantation. The aim of the present study was to review the surgical experiments regarding liver regeneration in pigs to promote experimental methodological standardization. The Pubmed, Medline, Scopus, and Cochrane Library databases were searched. Studies evaluating liver regeneration through surgical experiments performed on pigs were included. A total of 139 titles were screened, and 41 articles were included in the study, with 689 pigs in total. A total of 29 studies (71% of all) had a survival design, with an average study duration of 13 days. Overall, 36 studies (88%) considered partial hepatectomy, of which four were an associating liver partition and portal vein ligation for staged hepatectomy (ALPPS). Remnant liver volume ranged from 10% to 60%. Only 2 studies considered a hepatotoxic pre-treatment, while 25 studies evaluated additional liver procedures, such as stem cell application, ischemia/reperfusion injury, portal vein modulation, liver scaffold application, bio-artificial, and pharmacological liver treatment. Only nine authors analysed how cytokines and growth factors changed in response to liver resection. The most used imaging system to evaluate liver volume was CT-scan volumetry, even if performed only by nine authors. The pig represents one of the best animal models for the study of liver regeneration. However, it remains a mostly unexplored field due to the lack of experiments reproducing the chronic pathological aspects of the liver and the heterogeneity of existing studies.

Hyperspectral Imaging in Major Hepatectomies: Preliminary Results from the Ex-Machyna Trial.

Ischemia-reperfusion injury during major hepatic resections is associated with high rates of post-operative complications and liver failure. Real-time intra-operative detection of liver dysfunction could provide great insight into clinical outcomes. In the present study, we demonstrate the intra-operative application of a novel optical technology, hyperspectral imaging (HSI), to predict short-term post-operative outcomes after major hepatectomy. We considered fifteen consecutive patients undergoing major hepatic resection for malignant liver lesions from January 2020 to June 2021. HSI measures included tissue water index (TWI), organ hemoglobin index (OHI), tissue oxygenation (StO2%), and near infrared (NIR). Pre-operative, intra-operative, and post-operative serum and clinical outcomes were collected. NIR values were higher in unhealthy liver tissue (p = 0.003). StO2% negatively correlated with post-operative serum ALT values (r = -0.602), while ΔStO2% positively correlated with ALP (r = 0.594). TWI significantly correlated with post-operative reintervention and OHI with post-operative sepsis and liver failure. In conclusion, the HSI imaging system is accurate and precise in translating from pre-clinical to human studies in this first clinical trial. HSI indices are related to serum and outcome metrics. Further experimental and clinical studies are necessary to determine clinical value of this technology.

Computer-Assisted Differentiation between Colon-Mesocolon and Retroperitoneum Using Hyperspectral Imaging (HSI) Technology.

Complete mesocolic excision (CME), which involves the adequate resection of the tumor-bearing colonic segment with "en bloc" removal of its mesocolon along embryological fascial planes is associated with superior oncological outcomes. However, CME presents a higher complication rate compared to non-CME resections due to a higher risk of vascular injury. Hyperspectral imaging (HSI) is a contrast-free optical imaging technology, which facilitates the quantitative imaging of physiological tissue parameters and the visualization of anatomical structures. This study evaluates the accuracy of HSI combined with deep learning (DL) to differentiate the colon and its mesenteric tissue from retroperitoneal tissue. In an animal study including 20 pig models, intraoperative hyperspectral images of the sigmoid colon, sigmoid mesentery, and retroperitoneum were recorded. A convolutional neural network (CNN) was trained to distinguish the two tissue classes using HSI data, validated with a leave-one-out cross-validation process. The overall recognition sensitivity of the tissues to be preserved (retroperitoneum) and the tissues to be resected (colon and mesentery) was 79.0 ± 21.0% and 86.0 ± 16.0%, respectively. Automatic classification based on HSI and CNNs is a promising tool to automatically, non-invasively, and objectively differentiate the colon and its mesentery from retroperitoneal tissue.

Endoplasmic Reticulum and Mitochondria Contacts Correlate with the Presence and Severity of NASH in Humans.

The interaction between the mitochondria and the endoplasmic reticulum (ER) is essential for hepatocyte function. An increase in ER-mitochondria contacts (ERMCs) is associated with various metabolic diseases. Non-alcoholic fatty liver disease (NAFLD) is associated with obesity and type 2 diabetes, and its progressive form non-alcoholic steatohepatitis (NASH) can lead to cirrhosis and hepatocellular carcinoma. However, the role of ERMCs in the progression of NAFL to NASH is still unclear. We assessed whether ERMCs could correlate with NAFLD severity. We used a proximity ligation assay to measure the abundance of ERMCs in liver biopsies from patients with biopsy-proven NAFLD (n = 48) and correlated the results with histological and metabolic syndrome (MetS) features. NAFLD patients were included according to inclusion and exclusion criteria, and then assigned to NAFL (n = 9) and NASH (n = 39) groups. ERMCs density could discriminate NASH from NAFL (sensitivity 61.5%, specificity 100%). ERMCs abundance correlated with hepatocellular ballooning. Moreover, the density of ERMCs increased with an increase in the number of MetS features. In conclusion, ERMCs increased from NAFL to NASH, in parallel with the number of MetS features, supporting a role for this interaction in the pathophysiology of NASH.

Treatment strategies for neuroendocrine liver metastases: a systematic review.

BACKGROUND: Gastroenteropancreatic neuroendocrine tumors are often diagnosed when metastatic. The liver is the main site of metastases. Unfortunately, optimal management of neuroendocrine liver metastases remains a topic of debate. The aim of this study was to make a systematic review of the current literature about the results of the different treatments of neuroendocrine liver metastases. METHODS: A systematic review was conducted for English language publications from 1995 to 2021. Outcomes were analyzed according to survival, disease-free survival, and in the case of systemic therapies, progression-free survival. RESULTS: 5509 patients were analyzed in the review. 67% of patients underwent surgery achieving 5 years overall survival despite only 30% percent without a recurrence. 60% of patients that had received a transplant reached 5 years survival with a low disease-free survival rate (20%). Five-year survival rate was 36.2% for patients undergoing loco-regional therapies. CONCLUSION: Surgical resection is the best treatment when metastases are resectable, with the highest rate of survival, although liver transplantation shows good results for patients not eligible for surgery. Loco-regional therapies may be useful when surgical resection is contraindicated, or selectively used as a bridge to surgery or transplantation. Systemic therapies are indicated in patients for whom curative treatment cannot be obtained.

Perioperative Hyperspectral Imaging to Assess Mastectomy Skin Flap and DIEP Flap Perfusion in Immediate Autologous Breast Reconstruction: A Pilot Study.

Mastectomy skin flap necrosis (MSFN) and partial DIEP (deep inferior epigastric artery perforator) flap loss represent two frequently reported complications in immediate autologous breast reconstruction. These complications could be prevented when areas of insufficient tissue perfusion are detected intraoperatively. Hyperspectral imaging (HSI) is a relatively novel, non-invasive imaging technique, which could be used to objectively assess tissue perfusion through analysis of tissue oxygenation patterns (StO2%), near-infrared (NIR%), tissue hemoglobin (THI%), and tissue water (TWI%) perfusion indices. This prospective clinical pilot study aimed to evaluate the efficacy of HSI for tissue perfusion assessment and to identify a cut-off value for flap necrosis. Ten patients with a mean age of 55.4 years underwent immediate unilateral autologous breast reconstruction. Prior, during and up to 72 h after surgery, a total of 19 hyperspectral images per patient were acquired. MSFN was observed in 3 out of 10 patients. No DIEP flap necrosis was observed. In all MSFN cases, an increased THI% and decreased StO2%, NIR%, and TWI% were observed when compared to the vital group. StO2% was found to be the most sensitive parameter to detect MSFN with a statistically significant lower mean StO2% (51% in the vital group versus 32% in the necrosis group, p < 0.0001) and a cut-off value of 36.29% for flap necrosis. HSI has the potential to accurately assess mastectomy skin flap perfusion and discriminate between vital and necrotic skin flap during the early postoperative period prior to clinical observation. Although the results should be confirmed in future studies, including DIEP flap necrosis specifically, these findings suggest that HSI can aid clinicians in postoperative mastectomy skin flap and DIEP flap monitoring.

Near infrared fluorescence imaging of the urethra: a systematic review of the literature.

BACKGROUND: Urethral injury is a dreaded complication during laparoscopic, perineal and transanal surgery and is mainly a result of a failed visualization of the urethra. The aim of this systematic review is to provide an overview of the available literature on the near-infrared fluorescence (NIRF) imaging technique using contrast agents for the intra-operative visualization of the urethra. MATERIAL AND METHODS: A systematic review of the literature was conducted including studies on NIRF imaging using contrast agents to visualize the urethra. All studies describing a NIRF imaging technique and demonstrating visual findings of the urethra were included. RESULTS: Five studies were identified. Four studies examined indocyanine green, one of which also studied the IRDye® 800BK agent and one examined the CP-IRT dye. All studies showed that the NIRF imaging technique was feasible for an early identification of the urethra. No complications related to NIRF imaging were reported. CONCLUSION: We conclude that the use of a NIRF imaging technique is feasible and that it can contribute to prevent iatrogenic injury to the urethra. However, based on the limited available data, no solid conclusion can yet be drawn and further translation to the clinical practice is necessary.

Single Snapshot Imaging of Optical Properties (SSOP) for Perfusion Assessment during Gastric Conduit Creation for Esophagectomy: An Experimental Study on Pigs.

Anastomotic leakage (AL) is a serious complication occurring after esophagectomy. The current knowledge suggests that inadequate intraoperative perfusion in the anastomotic site contributes to an increase in the AL rate. Presently, clinical estimation undertaken by surgeons is not accurate and new technology is necessary to improve the intraoperative assessment of tissue oxygenation. In the present study, we demonstrate the application of a novel optical technology, namely Single Snapshot imaging of Optical Properties (SSOP), used to quantify StO2% in an open surgery experimental gastric conduit (GC) model. After the creation of a gastric conduit, local StO2% was measured with a preclinical SSOP system for 60 min in the antrum (ROI-A), corpus (ROI-C), and fundus (ROI-F). The removed region (ROI-R) acted as ischemic control. ROI-R had statistically significant lower StO2% when compared to all other ROIs at T15, T30, T45, and T60 (p < 0.0001). Local capillary lactates (LCLs) and StO2% correlation was statistically significant (R = -0.8439, 95% CI -0.9367 to -0.6407, p < 0.0001). Finally, SSOP could discriminate resected from perfused regions and ROI-A from ROI-F (the future anastomotic site). In conclusion, SSOP could well be a suitable technology to assess intraoperative perfusion of GC, providing consistent StO2% quantification and ROIs discrimination.

Intraoperative Guidance Using Hyperspectral Imaging: A Review for Surgeons.

Hyperspectral imaging (HSI) is a novel optical imaging modality, which has recently found diverse applications in the medical field. HSI is a hybrid imaging modality, combining a digital photographic camera with a spectrographic unit, and it allows for a contactless and non-destructive biochemical analysis of living tissue. HSI provides quantitative and qualitative information of the tissue composition at molecular level in a contrast-free manner, hence making it possible to objectively discriminate between different tissue types and between healthy and pathological tissue. Over the last two decades, HSI has been increasingly used in the medical field, and only recently it has found an application in the operating room. In the last few years, several research groups have used this imaging modality as an intraoperative guidance tool within different surgical disciplines. Despite its great potential, HSI still remains far from being routinely used in the daily surgical practice, since it is still largely unknown to most of the surgical community. The aim of this study is to provide clinical surgeons with an overview of the capabilities, current limitations, and future directions of HSI for intraoperative guidance.

Prediction of In Vivo Laser-Induced Thermal Damage with Hyperspectral Imaging Using Deep Learning.

Thermal ablation is an acceptable alternative treatment for primary liver cancer, of which laser ablation (LA) is one of the least invasive approaches, especially for tumors in high-risk locations. Precise control of the LA effect is required to safely destroy the tumor. Although temperature imaging techniques provide an indirect measurement of the thermal damage, a degree of uncertainty remains about the treatment effect. Optical techniques are currently emerging as tools to directly assess tissue thermal damage. Among them, hyperspectral imaging (HSI) has shown promising results in image-guided surgery and in the thermal ablation field. The highly informative data provided by HSI, associated with deep learning, enable the implementation of non-invasive prediction models to be used intraoperatively. Here we show a novel paradigm "peak temperature prediction model" (PTPM), convolutional neural network (CNN)-based, trained with HSI and infrared imaging to predict LA-induced damage in the liver. The PTPM demonstrated an optimal agreement with tissue damage classification providing a consistent threshold (50.6 ± 1.5 °C) for the damage margins with high accuracy (~0.90). The high correlation with the histology score (r = 0.9085) and the comparison with the measured peak temperature confirmed that PTPM preserves temperature information accordingly with the histopathological assessment.

Deep Learning Analysis of In Vivo Hyperspectral Images for Automated Intraoperative Nerve Detection.

Nerves are critical structures that may be difficult to recognize during surgery. Inadvertent nerve injuries can have catastrophic consequences for the patient and lead to life-long pain and a reduced quality of life. Hyperspectral imaging (HSI) is a non-invasive technique combining photography with spectroscopy, allowing non-invasive intraoperative biological tissue property quantification. We show, for the first time, that HSI combined with deep learning allows nerves and other tissue types to be automatically recognized in in vivo hyperspectral images. An animal model was used, and eight anesthetized pigs underwent neck midline incisions, exposing several structures (nerve, artery, vein, muscle, fat, skin). State-of-the-art machine learning models were trained to recognize these tissue types in HSI data. The best model was a convolutional neural network (CNN), achieving an overall average sensitivity of 0.91 and a specificity of 1.0, validated with leave-one-patient-out cross-validation. For the nerve, the CNN achieved an average sensitivity of 0.76 and a specificity of 0.99. In conclusion, HSI combined with a CNN model is suitable for in vivo nerve recognition.

Impact of Aging on Liver Cells and Liver Disease: Focus on the Biliary and Vascular Compartments.

The aging process is represented by the time-dependent decay in physiologic functions of living beings. Major interest has been focused in recent years on the determinants of this progressive condition due to its correlative relationship with the onset of diseases. Several hallmark features have been observed in aging, such as genetic alterations, mitochondrial impairment, and telomere shortening. At the cellular level, a senescent phenotype has been identified in response to aging that is characterized by a flat appearance, proliferative arrest, and production of specific molecules. The net effect of these cells in the course of diseases is an argument of debate. In fact, while the onset of a senescent phenotype may prevent tumor spreading, these cells appear to support pathological processes in some conditions. Several studies are now focused on clarifying the specific molecular pathways of aging/senescence in different cells, tissues, or organs. Biliary and vascular components, within the liver, have emerged as important determinants of some form of liver disease. In this review we summarize the most recent achievements on aging/senescence, focusing on the biliary and vascular liver system. Conclusion: Several findings, in both preclinical animal models and on human liver specimens, converge in supporting the presence of specific aging hallmarks in the diseases involving these hepatic compartments.

Intraoperative ureter identification with a novel fluorescent catheter.

Iatrogenic ureteral injuries (IUI) occur in 0.5-1.3% of cases during abdominal surgery. If not recognized intraoperatively, IUI increase morbidity/mortality. A universally accepted method to prevent IUI is lacking. Near-infrared fluorescent imaging (NIRF), penetrating deeper than normal light within the tissue, might be useful, therefore ureter visualization combining NIRF with special dyes (i.e. IRDye 800BK) is promising. Aim of this work is to evaluate the detection of ureters using stents coated with a novel biocompatible fluorescent material (NICE: near-infrared coating of equipment), during laparoscopy. female pigs underwent placement of NICE-coated stents (NS). NIRF was performed, and fluorescence intensity (FI) was computed. Successively, 0.15 mg/kg of IRDye 800BK was administered intravenously, and FI was computed at different timepoints. Ureter visualization using NS only was further assessed in a human cadaver. Both methods allowed in vivo ureter visualization, with equal FI. However, NS were constantly visible whereas IRDye 800BK allowed visualization exclusively during the ureteral peristaltic phases. In the human cadaver, NS provided excellent ureter visualization in its natural anatomical position. NS provided continuous ureteral visualization with similar FI as the IRDye 800BK, which exclusively allowed intermittent visualization, dependent on ureteral peristalsis. NS might prove useful to visualize ureters intraoperatively, potentially preventing IUI.

Fluorescence-based bowel anastomosis perfusion evaluation: results from the IHU-IRCAD-EAES EURO-FIGS registry.

BACKGROUND: Anastomotic leakage (AL) is one of the dreaded complications following surgery in the digestive tract. Near-infrared fluorescence (NIRF) imaging is a means to intraoperatively visualize anastomotic perfusion, facilitating fluorescence image-guided surgery (FIGS) with the purpose to reduce the incidence of AL. The aim of this study was to analyze the current practices and results of NIRF imaging of the anastomosis in digestive tract surgery through the EURO-FIGS registry. METHODS: Analysis of data prospectively collected by the registry members provided patient and procedural data along with the ICG dose, timing, and consequences of NIRF imaging. Among the included upper-GI, colorectal, and bariatric surgeries, subgroup analysis was performed to identify risk factors associated with complications. RESULTS: A total of 1240 patients were included in the study. The included patients, 74.8% of whom were operated on for cancer, originated from 8 European countries and 30 hospitals. A total of 54 surgeons performed the procedures. In 83.8% of cases, a pre-anastomotic ICG dose was administered, and in 60.1% of cases, a post-anastomotic ICG dose was administered. A significant difference (p < 0.001) was found in the ICG dose given in the four pathology groups registered (range: 0.013-0.89 mg/kg) and a significant (p < 0.001) negative correlation was found between the ICG dose and BMI. In 27.3% of the procedures, the choice of the anastomotic level was guided by means of NIRF imaging which means that in these cases NIRF imaging changed the level of anastomosis which was first decided based on visual findings in conventional white light imaging. In 98.7% of the procedures, the use of ICG partly or strongly provided a sense of confidence about the anastomosis. A total of 133 complications occurred, without any statistical significance in the incidence of complications in the anastomoses, whether they were ICG-guided or not. CONCLUSION: The EURO-FIGS registry provides an insight into the current clinical practice across Europe with respect to NIRF imaging of anastomotic perfusion during digestive tract surgery.

Hyperspectral Imagery for Assessing Laser-Induced Thermal State Change in Liver.

This work presents the potential of hyperspectral imaging (HSI) to monitor the thermal outcome of laser ablation therapy used for minimally invasive tumor removal. Our main goal is the establishment of indicators of the thermal damage of living tissues, which can be used to assess the effect of the procedure. These indicators rely on the spectral variation of temperature-dependent tissue chromophores, i.e., oxyhemoglobin, deoxyhemoglobin, methemoglobin, and water. Laser treatment was performed at specific temperature thresholds (from 60 to 110 °C) on in-vivo animal liver and was assessed with a hyperspectral camera (500-995 nm) during and after the treatment. The indicators were extracted from the hyperspectral images after the following processing steps: the breathing motion compensation and the spectral and spatial filtering, the selection of spectral bands corresponding to specific tissue chromophores, and the analysis of the areas under the curves for each spectral band. Results show that properly combining spectral information related to deoxyhemoglobin, methemoglobin, lipids, and water allows for the segmenting of different zones of the laser-induced thermal damage. This preliminary investigation provides indicators for describing the thermal state of the liver, which can be employed in the future as clinical endpoints of the procedure outcome.