Surgeon assessment of significant rectal polyps using white light endoscopy alone and in comparison to fluorescence-augmented AI lesion classification.

PURPOSE: Perioperative decision making for large (> 2 cm) rectal polyps with ambiguous features is complex. The most common intraprocedural assessment is clinician judgement alone while radiological and endoscopic biopsy can provide periprocedural detail. Fluorescence-augmented machine learning (FA-ML) methods may optimise local treatment strategy. METHODS: Surgeons of varying grades, all performing colonoscopies independently, were asked to visually judge endoscopic videos of large benign and early-stage malignant (potentially suitable for local excision) rectal lesions on an interactive video platform (Mindstamp) with results compared with and between final pathology, radiology and a novel FA-ML classifier. Statistical analyses of data used Fleiss Multi-rater Kappa scoring, Spearman Coefficient and Frequency tables. RESULTS: Thirty-two surgeons judged 14 ambiguous polyp videos (7 benign, 7 malignant). In all cancers, initial endoscopic biopsy had yielded false-negative results. Five of each lesion type had had a pre-excision MRI with a 60% false-positive malignancy prediction in benign lesions and a 60% over-staging and 40% equivocal rate in cancers. Average clinical visual cancer judgement accuracy was 49% (with only 'fair' inter-rater agreement), many reporting uncertainty and higher reported decision confidence did not correspond to higher accuracy. This compared to 86% ML accuracy. Size was misjudged visually by a mean of 20% with polyp size underestimated in 4/6 and overestimated in 2/6. Subjective narratives regarding decision-making requested for 7/14 lesions revealed wide rationale variation between participants. CONCLUSION: Current available clinical means of ambiguous rectal lesion assessment is suboptimal with wide inter-observer variation. Fluorescence based AI augmentation may advance this field via objective, explainable ML methods.

Clinical and computational development of a patient-calibrated ICGFA bowel transection recommender.

INTRODUCTION: Intraoperative indocyanine green fluorescence angiography (ICGFA) aims to reduce colorectal anastomotic complications. However, signal interpretation is inconsistent and confounded by patient physiology and system behaviours. Here, we demonstrate a proof of concept of a novel clinical and computational method for patient calibrated quantitative ICGFA (QICGFA) bowel transection recommendation. METHODS: Patients undergoing elective colorectal resection had colonic ICGFA both immediately after operative commencement prior to any dissection and again, as usual, just before anastomotic construction. Video recordings of both ICGFA acquisitions were blindly quantified post hoc across selected colonic regions of interest (ROIs) using tracking-quantification software and computationally compared with satisfactory perfusion assumed in second time-point ROIs, demonstrating 85% agreement with baseline ICGFA. ROI quantification outputs detailing projected perfusion sufficiency-insufficiency zones were compared to the actual surgeon-selected transection/anastomotic construction site for left/right-sided resections, respectively. Anastomotic outcomes were recorded, and tissue lactate was also measured in the devascularised colonic segment in a subgroup of patients. The novel perfusion zone projections were developed as full-screen recommendations via overlay heatmaps. RESULTS: No patient suffered intra- or early postoperative anastomotic complications. Following computational development (n = 14) the software recommended zone (ROI) contained the expert surgical site of transection in almost all cases (Jaccard similarity index 0.91) of the nine patient validation series. Previously published ICGFA time-series milestone descriptors correlated moderately well, but lactate measurements did not. High resolution augmented reality heatmaps presenting recommendations from all pixels of the bowel ICGFA were generated for all cases. CONCLUSIONS: By benchmarking to the patient's own baseline perfusion, this novel QICGFA method could allow the deployment of algorithmic personalised NIR bowel transection point recommendation in a way fitting existing clinical workflow.

Geotemporal Fluorophore Biodistribution Mapping of Colorectal Cancer: Micro and Macroscopic Insights.

Fluorescence-guided oncology promises to improve both the detection and treatment of malignancy. We sought to investigate the temporal distribution of indocyanine green (ICG), an exogenous fluorophore in human colorectal cancer. This analysis aims to enhance our understanding of ICG's effectiveness in current tumour detection and inform potential future diagnostic and therapeutic enhancements. METHODS: Fifty consenting patients undergoing treatment for suspected/confirmed colorectal neoplasia provided near infrared (NIR) video and imagery of transanally recorded and ex vivo resected rectal lesions following intravenous ICG administration (0.25 mg/kg), with a subgroup providing tissue samples for microscopic (including near infrared) analysis. Computer vision techniques detailed macroscopic 'early' (<15 min post ICG administration) and 'late' (>2 h) tissue fluorescence appearances from surgical imagery with digital NIR scanning (Licor, Lincoln, NE, USA) and from microscopic analysis (Nikon, Tokyo, Japan) undertaken by a consultant pathologist detailing tissue-level fluorescence distribution over the same time. RESULTS: Significant intra-tumoural fluorescence heterogeneity was seen 'early' in malignant versus benign lesions. In all 'early' samples, fluorescence was predominantly within the tissue stroma, with uptake within plasma cells, blood vessels and lymphatics, but not within malignant or healthy glands. At 'late' stage observation, fluorescence was visualised non-uniformly within the intracellular cytoplasm of malignant tissue but not retained in benign glands. Fluorescence also accumulated within any present peritumoural inflammatory tissue. CONCLUSION: This study demonstrates the time course diffusion patterns of ICG through both benign and malignant tumours in vivo in human patients at both macroscopic and microscopic levels, demonstrating important cellular drivers and features of geolocalisation and how they differ longitudinally after exposure to ICG.

Evaluating clinical near-infrared surgical camera systems with a view to optimizing operator and computational signal analysis.

Significance: As clinical evidence on the colorectal application of indocyanine green (ICG) perfusion angiography accrues, there is also interest in computerizing decision support. However, user interpretation and software development may be impacted by system factors affecting the displayed near-infrared (NIR) signal. Aim: We aim to assess the impact of camera positioning on the displayed NIR signal across different open and laparoscopic camera systems. Approach: The effects of distance, movement, and target location (center versus periphery) on the displayed fluorescence signal of different systems were measured under electromagnetic stereotactic guidance from an ICG-albumin model and in vivo during surgery. Results: Systems displayed distinct fluorescence performances with variance apparent with scope optical lens configuration (0 deg versus 30 deg), movement, target positioning, and distance. Laparoscopic system readings fitted inverse square function distance-intensity curves with one device and demonstrated a direction dependent sigmoid curve. Laparoscopic cameras presented central targets as brighter than peripheral ones, and laparoscopes with angled optical lens configurations had a diminished field of view. One handheld open system also showed a distance-intensity relationship, whereas the other maintained a consistent signal despite distance, but both presented peripheral targets brighter than central ones. Conclusions: Optimal clinical use and signal computational development requires detailed appreciation of system behaviors.

Real-time administration of indocyanine green in combination with computer vision and artificial intelligence for the identification and delineation of colorectal liver metastases.

Introduction: Fluorescence guided surgery for the identification of colorectal liver metastases (CRLM) can be better with low specificity and antecedent dosing impracticalities limiting indocyanine green (ICG) usefulness currently. We investigated the application of artificial intelligence methods (AIM) to demonstrate and characterise CLRMs based on dynamic signalling immediately following intraoperative ICG administration. Methods: Twenty-five patients with liver surface lesions (24 CRLM and 1 benign cyst) undergoing open/laparoscopic/robotic procedures were studied. ICG (0.05 mg/kg) was administered with near-infrared recording of fluorescence perfusion. User-selected region-of-interest (ROI) perfusion profiles were generated, milestones relating to ICG inflow/outflow extracted and used to train a machine learning (ML) classifier. 2D heatmaps were constructed in a subset using AIM to depict whole screen imaging based on dynamic tissue-ICG interaction. Fluorescence appearances were also assessed microscopically (using H&E and fresh-frozen preparations) to provide tissue-level explainability of such methods. Results: The ML algorithm correctly classified 97.2 % of CRLM ROIs (n = 132) and all benign lesion ROIs (n = 6) within 90-s of ICG administration following initial mathematical curve analysis identifying ICG inflow/outflow differentials between healthy liver and CRLMs. Time-fluorescence plots extracted for each pixel in 10 lesions enabled creation of 2D characterising heatmaps using flow parameters and through unsupervised ML. Microscopy confirmed statistically less CLRM fluorescence vs adjacent liver (mean ± std deviation signal/area 2.46 ± 9.56 vs 507.43 ± 160.82 respectively p < 0.001) with H&E diminishing ICG signal (n = 4). Conclusion: ML accurately identifies CRLMs from surrounding liver tissue enabling representative 2D mapping of such lesions from their fluorescence perfusion patterns using AIM. This may assist in reducing positive margin rates at metastatectomy and in identifying unexpected/occult malignancies.

Clinical application of machine learning and computer vision to indocyanine green quantification for dynamic intraoperative tissue characterisation: how to do it.

INTRODUCTION: Indocyanine green (ICG) quantification and assessment by machine learning (ML) could discriminate tissue types through perfusion characterisation, including delineation of malignancy. Here, we detail the important challenges overcome before effective clinical validation of such capability in a prospective patient series of quantitative fluorescence angiograms regarding primary and secondary colorectal neoplasia. METHODS: ICG perfusion videos from 50 patients (37 with benign (13) and malignant (24) rectal tumours and 13 with colorectal liver metastases) of between 2- and 15-min duration following intravenously administered ICG were formally studied (clinicaltrials.gov: NCT04220242). Video quality with respect to interpretative ML reliability was studied observing practical, technical and technological aspects of fluorescence signal acquisition. Investigated parameters included ICG dosing and administration, distance-intensity fluorescent signal variation, tissue and camera movement (including real-time camera tracking) as well as sampling issues with user-selected digital tissue biopsy. Attenuating strategies for the identified problems were developed, applied and evaluated. ML methods to classify extracted data, including datasets with interrupted time-series lengths with inference simulated data were also evaluated. RESULTS: Definable, remediable challenges arose across both rectal and liver cohorts. Varying ICG dose by tissue type was identified as an important feature of real-time fluorescence quantification. Multi-region sampling within a lesion mitigated representation issues whilst distance-intensity relationships, as well as movement-instability issues, were demonstrated and ameliorated with post-processing techniques including normalisation and smoothing of extracted time-fluorescence curves. ML methods (automated feature extraction and classification) enabled ML algorithms glean excellent pathological categorisation results (AUC-ROC > 0.9, 37 rectal lesions) with imputation proving a robust method of compensation for interrupted time-series data with duration discrepancies. CONCLUSION: Purposeful clinical and data-processing protocols enable powerful pathological characterisation with existing clinical systems. Video analysis as shown can inform iterative and definitive clinical validation studies on how to close the translation gap between research applications and real-world, real-time clinical utility.

Practical Perfusion Quantification in Multispectral Endoscopic Video: Using the Minutes after ICG Administration to Assess Tissue Pathology.

The wide availability of near infrared light sources in interventional medical imaging stacks enables non-invasive quantification of perfusion by using fluorescent dyes, typically Indocyanine Green (ICG). Due to their often leaky and chaotic vasculatures, intravenously administered ICG perfuses through cancerous tissues differently. We investigate here how a few characteristic values derived from the time series of fluorescence can be used in simple machine learning algorithms to distinguish benign lesions from cancers. These features capture the initial uptake of ICG in the colon, its peak fluorescence, and its early wash-out. By using simple, explainable algorithms we demonstrate, in clinical cases, that sensitivity (specificity) rates of over 95% (95%) for cancer classification can be achieved.