Computational Polarization Imaging In Vivo through Surgical Smoke Using Refined Polarization Difference.

In surgery, the surgical smoke generated during tissue dissection and hemostasis can degrade the image quality, affecting tissue visibility and interfering with the further image processing. Developing reliable and interpretable computational imaging methods for restoring smoke-affected surgical images is crucial, as typical image restoration methods relying on color-texture information are insufficient. Here a computational polarization imaging method through surgical smoke is demonstrated, including a refined polarization difference estimation based on the discrete electric field direction, and a corresponding prior-based estimation method, for better parameter estimation and image restoration performance. Results and analyses for ex vivo, the first in vivo animal experiments, and human oral cavity tests show that the proposed method achieves visibility restoration and color recovery of higher quality, and exhibits good generalization across diverse imaging scenarios with interpretability. The method is expected to enhance the precision, safety, and efficiency of advanced image-guided and robotic surgery.

Fluorescence guided surgery imaging systems for breast cancer identification: a systematic review.

Significance: Breast-conserving surgery (BCS) is limited by high rates of positive margins and re-operative interventions. Fluorescence-guided surgery seeks to detect the entire lesion in real time, thus guiding the surgeons to remove all the tumor at the index procedure. Aim: Our aim was to identify the optimal combination of a camera system and fluorophore for fluorescence-guided BCS. Approach: A systematic review of medical databases using the terms "fluorescence," "breast cancer," "surgery," and "fluorescence imaging" was performed. Cameras were compared using the ratio between the fluorescent signal from the tumor compared to background fluorescence, as well as diagnostic accuracy measures, such as sensitivity, specificity, and positive predictive value. Results: Twenty-one studies identified 14 camera systems using nine different fluorophores. Twelve cameras worked in the infrared spectrum. Ten studies reported on the difference in strength of the fluorescence signal between cancer and normal tissue, with results ranging from 1.72 to 4.7. In addition, nine studies reported on whether any tumor remained in the resection cavity (5.4% to 32.5%). To date, only three studies used the fluorescent signal for guidance during real BCS. Diagnostic accuracy ranged from 63% to 98% sensitivity, 32% to 97% specificity, and 75% to 100% positive predictive value. Conclusion: In this systematic review, all the studies reported a clinically significant difference in signal between the tumor and normal tissue using various camera/fluorophore combinations. However, given the heterogeneity in protocols, including camera setup, fluorophore studied, data acquisition, and reporting structure, it was impossible to determine the optimal camera and fluorophore combination for use in BCS. It would be beneficial to develop a standardized reporting structure using similar metrics to provide necessary data for a comparison between camera systems.

Monoclonal Antibodies for Targeted Fluorescence-Guided Surgery: A Review of Applicability across Multiple Solid Tumors.

This study aims to review the status of the clinical use of monoclonal antibodies (mAbs) that have completed or are in ongoing clinical trials for targeted fluorescence-guided surgery (T-FGS) for the intraoperative identification of the tumor margins of extra-hematological solid tumors. For each of them, the targeted antigen, the mAb generic/commercial name and format, and clinical indications are presented, together with utility, doses, and the timing of administration. Based on the current scientific evidence in humans, the top three mAbs that could be prepared in a GMP-compliant bank ready to be delivered for surgical purposes are proposed to speed up the translation to the operating room and produce a few readily available "off-the-shelf" injectable fluorescent probes for safer and more effective solid tumor resection.

Real-time classification of tumour and non-tumour tissue in colorectal cancer using diffuse reflectance spectroscopy and neural networks to aid margin assessment.

BACKGROUND: Colorectal cancer is the third most commonly diagnosed malignancy and the second leading cause of mortality worldwide. A positive resection margin following surgery for colorectal cancer is linked with higher rates of local recurrence and poorer survival. The authors investigated diffuse reflectance spectroscopy (DRS) to distinguish tumour and non-tumour tissue in ex-vivo colorectal specimens, to aid margin assessment and provide augmented visual maps to the surgeon in real-time. METHODS: Patients undergoing elective colorectal cancer resection surgery at a London-based hospital were prospectively recruited. A hand-held DRS probe was used on the surface of freshly resected ex-vivo colorectal tissue. Spectral data were acquired for tumour and non-tumour tissue. Binary classification was achieved using conventional machine learning classifiers and a convolutional neural network (CNN), which were evaluated in terms of sensitivity, specificity, accuracy and the area under the curve. RESULTS: A total of 7692 mean spectra were obtained for tumour and non-tumour colorectal tissue. The CNN-based classifier was the best performing machine learning algorithm, when compared to contrastive approaches, for differentiating tumour and non-tumour colorectal tissue, with an overall diagnostic accuracy of 90.8% and area under the curve of 96.8%. Live on-screen classification of tissue type was achieved using a graduated colourmap. CONCLUSION: A high diagnostic accuracy for a DRS probe and tracking system to differentiate ex-vivo tumour and non-tumour colorectal tissue in real-time with on-screen visual feedback was highlighted by this study. Further in-vivo studies are needed to ensure integration into a surgical workflow.

A YOLOv5-based network for the detection of a diffuse reflectance spectroscopy probe to aid surgical guidance in gastrointestinal cancer surgery.

PURPOSE: A positive circumferential resection margin (CRM) for oesophageal and gastric carcinoma is associated with local recurrence and poorer long-term survival. Diffuse reflectance spectroscopy (DRS) is a non-invasive technology able to distinguish tissue type based on spectral data. The aim of this study was to develop a deep learning-based method for DRS probe detection and tracking to aid classification of tumour and non-tumour gastrointestinal (GI) tissue in real time. METHODS: Data collected from both ex vivo human tissue specimen and sold tissue phantoms were used for the training and retrospective validation of the developed neural network framework. Specifically, a neural network based on the You Only Look Once (YOLO) v5 network was developed to accurately detect and track the tip of the DRS probe on video data acquired during an ex vivo clinical study. RESULTS: Different metrics were used to analyse the performance of the proposed probe detection and tracking framework, such as precision, recall, mAP 0.5, and Euclidean distance. Overall, the developed framework achieved a 93% precision at 23 FPS for probe detection, while the average Euclidean distance error was 4.90 pixels. CONCLUSION: The use of a deep learning approach for markerless DRS probe detection and tracking system could pave the way for real-time classification of GI tissue to aid margin assessment in cancer resection surgery and has potential to be applied in routine surgical practice.

A Comparison of Spectroscopy and Imaging Techniques Utilizing Spectrally Resolved Diffusely Reflected Light for Intraoperative Margin Assessment in Breast-Conserving Surgery: A Systematic Review and Meta-Analysis.

Up to 19% of patients require re-excision surgery due to positive margins in breast-conserving surgery (BCS). Intraoperative margin assessment tools (IMAs) that incorporate tissue optical measurements could help reduce re-excision rates. This review focuses on methods that use and assess spectrally resolved diffusely reflected light for breast cancer detection in the intraoperative setting. Following PROSPERO registration (CRD42022356216), an electronic search was performed. The modalities searched for were diffuse reflectance spectroscopy (DRS), multispectral imaging (MSI), hyperspectral imaging (HSI), and spatial frequency domain imaging (SFDI). The inclusion criteria encompassed studies of human in vivo or ex vivo breast tissues, which presented data on accuracy. The exclusion criteria were contrast use, frozen samples, and other imaging adjuncts. 19 studies were selected following PRISMA guidelines. Studies were divided into point-based (spectroscopy) or whole field-of-view (imaging) techniques. A fixed-or random-effects model analysis generated pooled sensitivity/specificity for the different modalities, following heterogeneity calculations using the Q statistic. Overall, imaging-based techniques had better pooled sensitivity/specificity (0.90 (CI 0.76-1.03)/0.92 (CI 0.78-1.06)) compared with probe-based techniques (0.84 (CI 0.78-0.89)/0.85 (CI 0.79-0.91)). The use of spectrally resolved diffusely reflected light is a rapid, non-contact technique that confers accuracy in discriminating between normal and malignant breast tissue, and it constitutes a potential IMA tool.

Surgical polarimetric endoscopy for the detection of laryngeal cancer.

The standard-of-care for the detection of laryngeal pathologies involves distinguishing suspicious lesions from surrounding healthy tissue via contrasts in colour and texture captured by white-light endoscopy. However, the technique is insufficiently sensitive and thus leads to unsatisfactory rates of false negatives. Here we show that laryngeal lesions can be better detected in real time by taking advantage of differences in the light-polarization properties of cancer and healthy tissues. By measuring differences in polarized-light retardance and depolarization, the technique, which we named 'surgical polarimetric endoscopy' (SPE), generates about one-order-of-magnitude greater contrast than white-light endoscopy, and hence allows for the better discrimination of cancerous lesions, as we show with patients diagnosed with squamous cell carcinoma. Polarimetric imaging of excised and stained slices of laryngeal tissue indicated that changes in the retardance of polarized light can be largely attributed to architectural features of the tissue. We also assessed SPE to aid routine transoral laser surgery for the removal of a cancerous lesion, indicating that SPE can complement white-light endoscopy for the detection of laryngeal cancer.

Introduction to the Biophotonics Congress 2022 feature issue.

A feature issue is being presented by a team of guest editors containing papers based on studies presented at the Optica Biophotonics Congress: Biomedical Optics held on April 24-27, 2022 in Fort Lauderdale, Florida, USA.

Real-time Tracking and Classification of Tumor and Nontumor Tissue in Upper Gastrointestinal Cancers Using Diffuse Reflectance Spectroscopy for Resection Margin Assessment.

Importance: Cancers of the upper gastrointestinal tract remain a major contributor to the global cancer burden. The accurate mapping of tumor margins is of particular importance for curative cancer resection and improvement in overall survival. Current mapping techniques preclude a full resection margin assessment in real time. Objective: To evaluate whether diffuse reflectance spectroscopy (DRS) on gastric and esophageal cancer specimens can differentiate tissue types and provide real-time feedback to the operator. Design, Setting, and Participants: This was a prospective ex vivo validation study. Patients undergoing esophageal or gastric cancer resection were prospectively recruited into the study between July 2020 and July 2021 at Hammersmith Hospital in London, United Kingdom. Tissue specimens were included for patients undergoing elective surgery for either esophageal carcinoma (adenocarcinoma or squamous cell carcinoma) or gastric adenocarcinoma. Exposures: A handheld DRS probe and tracking system was used on freshly resected ex vivo tissue to obtain spectral data. Binary classification, following histopathological validation, was performed using 4 supervised machine learning classifiers. Main Outcomes and Measures: Data were divided into training and testing sets using a stratified 5-fold cross-validation method. Machine learning classifiers were evaluated in terms of sensitivity, specificity, overall accuracy, and the area under the curve. Results: Of 34 included patients, 22 (65%) were male, and the median (range) age was 68 (35-89) years. A total of 14 097 mean spectra for normal and cancerous tissue were collected. For normal vs cancer tissue, the machine learning classifier achieved a mean (SD) overall diagnostic accuracy of 93.86% (0.66) for stomach tissue and 96.22% (0.50) for esophageal tissue and achieved a mean (SD) sensitivity and specificity of 91.31% (1.5) and 95.13% (0.8), respectively, for stomach tissue and of 94.60% (0.9) and 97.28% (0.6) for esophagus tissue. Real-time tissue tracking and classification was achieved and presented live on screen. Conclusions and Relevance: This study provides ex vivo validation of the DRS technology for real-time differentiation of gastric and esophageal cancer from healthy tissue using machine learning with high accuracy. As such, it is a step toward the development of a real-time in vivo tumor mapping tool for esophageal and gastric cancers that can aid decision-making of resection margins intraoperatively.

Simultaneous Depth Estimation and Surgical Tool Segmentation in Laparoscopic Images.

Surgical instrument segmentation and depth estimation are crucial steps to improve autonomy in robotic surgery. Most recent works treat these problems separately, making the deployment challenging. In this paper, we propose a unified framework for depth estimation and surgical tool segmentation in laparoscopic images. The network has an encoder-decoder architecture and comprises two branches for simultaneously performing depth estimation and segmentation. To train the network end to end, we propose a new multi-task loss function that effectively learns to estimate depth in an unsupervised manner, while requiring only semi-ground truth for surgical tool segmentation. We conducted extensive experiments on different datasets to validate these findings. The results showed that the end-to-end network successfully improved the state-of-the-art for both tasks while reducing the complexity during their deployment.

Self-supervised Monocular Depth Estimation with 3D Displacement Module for Laparoscopic Images.

We present a novel self-supervised training framework with 3D displacement (3DD) module for accurately estimating per-pixel depth maps from single laparoscopic images. Recently, several self-supervised learning based monocular depth estimation models have achieved good results on the KITTI dataset, under the hypothesis that the camera is dynamic and the objects are stationary, however this hypothesis is often reversed in the surgical setting (laparoscope is stationary, the surgical instruments and tissues are dynamic). Therefore, a 3DD module is proposed to establish the relation between frames instead of ego-motion estimation. In the 3DD module, a convolutional neural network (CNN) analyses source and target frames to predict the 3D displacement of a 3D point cloud from a target frame to a source frame in the coordinates of the camera. Since it is difficult to constrain the depth displacement from two 2D images, a novel depth consistency module is proposed to maintain depth consistency between displacement-updated depth and model-estimated depth to constrain 3D displacement effectively. Our proposed method achieves remarkable performance for monocular depth estimation on the Hamlyn surgical dataset and acquired ground truth depth maps, outperforming monodepth, monodepth2 and packnet models.

Polarization-based smoke removal method for surgical images.

Smoke generated during surgery affects tissue visibility and degrades image quality, affecting surgical decisions and limiting further image processing and analysis. Polarization is a fundamental property of light and polarization-resolved imaging has been studied and applied to general visibility restoration scenarios such as for smog or mist removal or in underwater environments. However, there is no related research or application for surgical smoke removal. Due to differences between surgical smoke and general haze scenarios, we propose an alternative imaging degradation model by redefining the form of the transmission parameters. The analysis of the propagation of polarized light interacting with the mixed medium of smoke and tissue is proposed to realize polarization-based smoke removal (visibility restoration). Theoretical analysis and observation of experimental data shows that the cross-polarized channel data generated by multiple scattering is less affected by smoke compared to the co-polarized channel. The polarization difference calculation for different color channels can estimate the model transmission parameters and reconstruct the image with restored visibility. Qualitative and quantitative comparison with alternative methods show that the polarization-based image smoke-removal method can effectively reduce the degradation of biomedical images caused by surgical smoke and partially restore the original degree of polarization of the samples.

Indocyanine green fluorescence image processing techniques for breast cancer macroscopic demarcation.

Re-operation due to disease being inadvertently close to the resection margin is a major challenge in breast conserving surgery (BCS). Indocyanine green (ICG) fluorescence imaging could be used to visualize the tumor boundaries and help surgeons resect disease more efficiently. In this work, ICG fluorescence and color images were acquired with a custom-built camera system from 40 patients treated with BCS. Images were acquired from the tumor in-situ, surgical cavity post-excision, freshly excised tumor and histopathology tumour grossing. Fluorescence image intensity and texture were used as individual or combined predictors in both logistic regression (LR) and support vector machine models to predict the tumor extent. ICG fluorescence spectra in formalin-fixed histopathology grossing tumor were acquired and analyzed. Our results showed that ICG remains in the tissue after formalin fixation. Therefore, tissue imaging could be validated in freshly excised and in formalin-fixed grossing tumor. The trained LR model with combined fluorescence intensity (pixel values) and texture (slope of power spectral density curve) identified the tumor's extent in the grossing images with pixel-level resolution and sensitivity, specificity of 0.75 ± 0.3, 0.89 ± 0.2.This model was applied on tumor in-situ and surgical cavity (post-excision) images to predict tumor presence.

Real-time tracking of a diffuse reflectance spectroscopy probe used to aid histological validation of margin assessment in upper gastrointestinal cancer resection surgery.

SIGNIFICANCE: Diffuse reflectance spectroscopy (DRS) allows discrimination of tissue type. Its application is limited by the inability to mark the scanned tissue and the lack of real-time measurements. AIM: This study aimed to develop a real-time tracking system to enable localization of a DRS probe to aid the classification of tumor and non-tumor tissue. APPROACH: A green-colored marker attached to the DRS probe was detected using hue-saturation-value (HSV) segmentation. A live, augmented view of tracked optical biopsy sites was recorded in real time. Supervised classifiers were evaluated in terms of sensitivity, specificity, and overall accuracy. A developed software was used for data collection, processing, and statistical analysis. RESULTS: The measured root mean square error (RMSE) of DRS probe tip tracking was 1.18 ± 0.58 mm and 1.05 ± 0.28 mm for the x and y dimensions, respectively. The diagnostic accuracy of the system to classify tumor and non-tumor tissue in real time was 94% for stomach and 96% for the esophagus. CONCLUSIONS: We have successfully developed a real-time tracking and classification system for a DRS probe. When used on stomach and esophageal tissue for tumor detection, the accuracy derived demonstrates the strength and clinical value of the technique to aid margin assessment in cancer resection surgery.

A systematic review of robotic surgery: From supervised paradigms to fully autonomous robotic approaches.

BACKGROUND: From traditional open surgery to laparoscopic surgery and robot-assisted surgery, advances in robotics, machine learning, and imaging are pushing the surgical approach to-wards better clinical outcomes. Pre-clinical and clinical evidence suggests that automation may standardise techniques, increase efficiency, and reduce clinical complications. METHODS: A PRISMA-guided search was conducted across PubMed and OVID. RESULTS: Of the 89 screened articles, 51 met the inclusion criteria, with 10 included in the final review. Automatic data segmentation, trajectory planning, intra-operative registration, trajectory drilling, and soft tissue robotic surgery were discussed. CONCLUSION: Although automated surgical systems remain conceptual, several research groups have developed supervised autonomous robotic surgical systems with increasing consideration for ethico-legal issues for automation. Automation paves the way for precision surgery and improved safety and opens new possibilities for deploying more robust artificial intelligence models, better imaging modalities and robotics to improve clinical outcomes.

The Impact of Temporal Variation in Indocyanine Green Administration on Tumor Identification During Fluorescence Guided Breast Surgery.

BACKGROUND: On average, 21% of women in the USA treated with Breast Conserving Surgery (BCS) undergo a second operation because of close positive margins. Tumor identification with fluorescence imaging could improve positive margin rates through demarcating location, size, and invasiveness of tumors. We investigated the technique's diagnostic accuracy in detecting tumors during BCS using intravenous indocyanine green (ICG) and a custom-built fluorescence camera system. METHODS: In this single-center prospective clinical study, 40 recruited BCS patients were sub-categorized into two cohorts. In the first 'enhanced permeability and retention' (EPR) cohort, 0.25 mg/kg ICG was injected ~ 25 min prior to tumor excision, and in the second 'angiography' cohort, ~ 5 min prior to tumor excision. Subsequently, an in-house imaging system was used to image the tumor in situ prior to resection, ex vivo following resection, the resection bed, and during grossing in the histopathology laboratory to compare the technique's diagnostic accuracy between the cohorts. RESULTS: The two cohorts were matched in patient and tumor characteristics. The majority of patients had invasive ductal carcinoma with concomitant ductal carcinoma in situ. Tumor-to-background ratio (TBR) in the angiography cohort was superior to the EPR cohort (TBR = 3.18 ± 1.74 vs 2.10 ± 0.92 respectively, p = 0.023). Tumor detection reached sensitivity and specificity scores of 0.82 and 0.93 for the angiography cohort and 0.66 and 0.90 for the EPR cohort, respectively (p = 0.1051 and p = 0.9099). DISCUSSION: ICG administration timing during the angiography phase compared with the EPR phase improved TBR and diagnostic accuracy. Future work will focus on image pattern analysis and adaptation of the camera system to targeting fluorophores specific to breast cancer.