Interobserver Variability in the Assessment of Fluorescence Angiography in the Colon.

BACKGROUND: Fluorescence angiography in colorectal surgery is a technique that may lead to lower anastomotic leak rates. However, the interpretation of the fluorescent signal is not standardised and there is a paucity of data regarding interobserver agreement. The aim of this study is to assess interobserver variability in selection of the transection point during fluorescence angiography before anastomosis. METHODS: An online survey with still images of fluorescence angiography was distributed through colorectal surgery channels containing images from 13 patients where several areas for transection were displayed to be chosen by raters. Agreement was assessed overall and between pre-planned rater cohorts (experts vs non-experts; trainees vs consultants; colorectal specialists vs non colorectal specialists), using Fleiss' kappa statistic. RESULTS: 101 raters had complete image ratings. No significant difference was found between raters when choosing a point of optimal bowel transection based on fluorescence angiography still images. There was no difference between pre-planned cohorts analysed (experts vs non-experts; trainees vs consultants; colorectal specialists vs non colorectal specialists). Agreement between these cohorts was poor (<.26). CONCLUSION: Whilst there is no learning curve for the technical adoption of FA, understanding the fluorescent signal characteristics is key to successful use. We found significant variation exists in interpretation of static fluorescence angiography data. Further efforts should be employed to standardise fluorescence angiography assessment.

Multisensor perfusion assessment cohort study: Preliminary evidence toward a standardized assessment of indocyanine green fluorescence in colorectal surgery.

BACKGROUND: Traditional methods of assessing colonic perfusion are based on the surgeon's visual inspection of tissue. Fluorescence angiography provides qualitative information, but there remains disagreement on how the observed signal should be interpreted. It is unclear whether fluorescence correlates with physiological properties of the tissue, such as tissue oxygen saturation. The aim of this study was to correlate fluorescence intensity and colonic tissue oxygen saturation. METHODS: Prospective cohort study performed in a single academic tertiary referral center. Patients undergoing colorectal surgery who required an anastomosis underwent dual-modality perfusion assessment of a segment of bowel before transection and creation of the anastomosis, using near-infrared and multispectral imaging. Perfusion was assessed using maximal fluorescence intensity measurement during fluorescence angiography, and its correlation with tissue oxygen saturation was calculated. RESULTS: In total, 18 patients were included. Maximal fluorescence intensity occurred at a mean of 101 seconds after indocyanine green injection. The correlation coefficient was 0.73 (95% confidence interval of 0.65-0.79) with P < .0001, showing a statistically significant strong positive correlation between normalized fluorescence intensity and tissue oxygen saturation. The use of time averaging improved the correlation coefficient to 0.78. CONCLUSION: Fluorescence intensity is a potential surrogate for tissue oxygenation. This is expected to lead to improved decision making when transecting the bowel and, consequently, a reduction in anastomotic leak rates. A larger, phase II study is needed to confirm this result and form the basis of computational algorithms to infer biological or physiological information from the fluorescence imaging data.

Stain-free identification of tissue pathology using a generative adversarial network to infer nanomechanical signatures.

Intraoperative frozen section analysis can be used to improve the accuracy of tumour margin estimation during cancer resection surgery through rapid processing and pathological assessment of excised tissue. Its applicability is limited in some cases due to the additional risks associated with prolonged surgery, largely from the time-consuming staining procedure. Our work uses a measurable property of bulk tissue to bypass the staining process: as tumour cells proliferate, they influence the surrounding extra-cellular matrix, and the resulting change in elastic modulus provides a signature of the underlying pathology. In this work we accurately localise atomic force microscopy measurements of human liver tissue samples and train a generative adversarial network to infer elastic modulus from low-resolution images of unstained tissue sections. Pathology is predicted through unsupervised clustering of parameters characterizing the distributions of inferred values, achieving 89% accuracy for all samples based on the nominal assessment (n = 28), and 95% for samples that have been validated by two independent pathologists through post hoc staining (n = 20). Our results demonstrate that this technique could increase the feasibility of intraoperative frozen section analysis for use during resection surgery and improve patient outcomes.

Intraoperative colon perfusion assessment using multispectral imaging.

In colorectal surgery an anastomosis performed using poorly-perfused, ischaemic bowel segments may result in a leak and consequent morbidity. Traditional measures of perfusion assessment rely on clinical judgement and are mainly subjective, based on tissue appearance, leading to variability between clinicians. This paper describes a multispectral imaging (MSI) laparoscope that can derive quantitative measures of tissue oxygen saturation (SO2 ). The system uses a xenon surgical light source and fast filter wheel camera to capture eight narrow waveband images across the visible range in approximately 0.3 s. Spectral validation measurements were performed by imaging standardised colour tiles and comparing reflectance with ground truth spectrometer data. Tissue spectra were decomposed into individual contributions from haemoglobin, adipose tissue and scattering, using a previously-developed regression approach. Initial clinical results from seven patients undergoing colorectal surgery are presented and used to characterise measurement stability and reproducibility in vivo. Strategies to improve signal-to-noise ratio and correct for motion are described. Images of healthy bowel tissue (in vivo) indicate that baseline SO2 is approximately 75 ± 6%. The SO2 profile along a bowel segment following ligation of the inferior mesenteric artery (IMA) shows a decrease from the proximal to distal end. In the clinical cases shown, imaging results concurred with clinical judgements of the location of well-perfused tissue. Adipose tissue, visibly yellow in the RGB images, is shown to surround the mesentery and cover some of the serosa. SO2 in this tissue is consistently high, with mean value of 90%. These results show that MSI is a potential intraoperative guidance tool for assessment of perfusion. Mapping of SO2 in the colon could be used by surgeons to guide choice of transection points and ensure that well-perfused tissue is used to form an anastomosis. The observation of high mesenteric SO2 agrees with work in the literature and warrants further exploration. Larger studies incorporating with a wider cohort of clinicians will help to provide retrospective evidence of how this imaging technique may be able to reduce inter-operator variability.

Surgical spectral imaging.

Recent technological developments have resulted in the availability of miniaturised spectral imaging sensors capable of operating in the multi- (MSI) and hyperspectral imaging (HSI) regimes. Simultaneous advances in image-processing techniques and artificial intelligence (AI), especially in machine learning and deep learning, have made these data-rich modalities highly attractive as a means of extracting biological information non-destructively. Surgery in particular is poised to benefit from this, as spectrally-resolved tissue optical properties can offer enhanced contrast as well as diagnostic and guidance information during interventions. This is particularly relevant for procedures where inherent contrast is low under standard white light visualisation. This review summarises recent work in surgical spectral imaging (SSI) techniques, taken from Pubmed, Google Scholar and arXiv searches spanning the period 2013-2019. New hardware, optimised for use in both open and minimally-invasive surgery (MIS), is described, and recent commercial activity is summarised. Computational approaches to extract spectral information from conventional colour images are reviewed, as tip-mounted cameras become more commonplace in MIS. Model-based and machine learning methods of data analysis are discussed in addition to simulation, phantom and clinical validation experiments. A wide variety of surgical pilot studies are reported but it is apparent that further work is needed to quantify the clinical value of MSI/HSI. The current trend toward data-driven analysis emphasises the importance of widely-available, standardised spectral imaging datasets, which will aid understanding of variability across organs and patients, and drive clinical translation.

Estimation of tissue oxygen saturation from RGB images and sparse hyperspectral signals based on conditional generative adversarial network.

PURPOSE: Intra-operative measurement of tissue oxygen saturation ([Formula: see text]) is important in detection of ischaemia, monitoring perfusion and identifying disease. Hyperspectral imaging (HSI) measures the optical reflectance spectrum of the tissue and uses this information to quantify its composition, including [Formula: see text]. However, real-time monitoring is difficult due to capture rate and data processing time. METHODS: An endoscopic system based on a multi-fibre probe was previously developed to sparsely capture HSI data (sHSI). These were combined with RGB images, via a deep neural network, to generate high-resolution hypercubes and calculate [Formula: see text]. To improve accuracy and processing speed, we propose a dual-input conditional generative adversarial network, Dual2StO2, to directly estimate [Formula: see text] by fusing features from both RGB and sHSI. RESULTS: Validation experiments were carried out on in vivo porcine bowel data, where the ground truth [Formula: see text] was generated from the HSI camera. Performance was also compared to our previous super-spectral-resolution network, SSRNet in terms of mean [Formula: see text] prediction accuracy and structural similarity metrics. Dual2StO2 was also tested using simulated probe data with varying fibre number. CONCLUSIONS: [Formula: see text] estimation by Dual2StO2 is visually closer to ground truth in general structure and achieves higher prediction accuracy and faster processing speed than SSRNet. Simulations showed that results improved when a greater number of fibres are used in the probe. Future work will include refinement of the network architecture, hardware optimization based on simulation results, and evaluation of the technique in clinical applications beyond [Formula: see text] estimation.

Spectral Imaging Of Thermal Damage Induced During Microwave Ablation In The Liver.

Induction of thermal damage to tissue through delivery of microwave energy is frequently applied in surgery to destroy diseased tissue such as cancer cells. Minimization of unwanted harm to healthy tissue is still achieved subjectively, and the surgeon has few tools at their disposal to monitor the spread of the induced damage. This work describes the use of optical methods to monitor the time course of changes to the tissue during delivery of microwave energy in the porcine liver. Multispectral imaging and diffuse reflectance spectroscopy are used to monitor temporal changes in optical properties in parallel with thermal imaging. The results demonstrate the ability to monitor the spatial extent of thermal damage on a whole organ, including possible secondary effects due to vascular damage. Future applications of this type of imaging may see the multispectral data used as a feedback mechanism to avoid collateral damage to critical healthy structures and to potentially verify sufficient application of energy to the diseased tissue.

Use of Laser Speckle Contrast Analysis during pelvic surgery in a uterine transplantation model.

AIM: Uterine transplantation (UTx) is proposed for treatment of uterine factor infertility. Our aim was to assess whether Endoscopic Laser Speckle Contrast Analysis (eLASCA) could evaluate pelvic blood flow at anastomotic sites required for sheep and rabbit UTx. RESULTS/METHODOLOGY: eLASCA detected blood flow in rabbit UTx #7 and #9. In sheep UTx #2, #3 and #5, the results allowed us to conclude that blood flow was present in the uterine graft following transplantation; and post-UTx, the animal had heart and respiratory rates, and oxygen saturation compatible with a normal hemodynamic status. CONCLUSION: These preliminary results establish the potential of Laser Speckle Contrast Analysis as noncontact and real-time tool for observation of spatially-resolved blood flow from which other parameters can be derived.

Dual-modality endoscopic probe for tissue surface shape reconstruction and hyperspectral imaging enabled by deep neural networks.

Surgical guidance and decision making could be improved with accurate and real-time measurement of intra-operative data including shape and spectral information of the tissue surface. In this work, a dual-modality endoscopic system has been proposed to enable tissue surface shape reconstruction and hyperspectral imaging (HSI). This system centers around a probe comprised of an incoherent fiber bundle, whose fiber arrangement is different at the two ends, and miniature imaging optics. For 3D reconstruction with structured light (SL), a light pattern formed of randomly distributed spots with different colors is projected onto the tissue surface, creating artificial texture. Pattern decoding with a Convolutional Neural Network (CNN) model and a customized feature descriptor enables real-time 3D surface reconstruction at approximately 12 frames per second (FPS). In HSI mode, spatially sparse hyperspectral signals from the tissue surface can be captured with a slit hyperspectral imager in a single snapshot. A CNN based super-resolution model, namely "super-spectral-resolution" network (SSRNet), has also been developed to estimate pixel-level dense hypercubes from the endoscope cameras standard RGB images and the sparse hyperspectral signals, at approximately 2 FPS. The probe, with a 2.1 mm diameter, enables the system to be used with endoscope working channels. Furthermore, since data acquisition in both modes can be accomplished in one snapshot, operation of this system in clinical applications is minimally affected by tissue surface movement and deformation. The whole apparatus has been validated on phantoms and tissue (ex vivo and in vivo), while initial measurements on patients during laryngeal surgery show its potential in real-world clinical applications.

Use of biomedical photonics in gynecological surgery: a uterine transplantation model.

AIM: Uterine transplantation (UTx) has been proposed as a treatment for permanent absolute uterine factor infertility. The study aims were to compare pulse oximetry and multispectral imaging (MSI), for intraoperative tracking of uterine oxygen saturation in animal UTx models (rabbit and sheep). RESULTS/METHODOLOGY: Imaging results confirmed the re-establishment of adequate perfusion in the transplanted organ after surgery. Comparison of oxygen saturation values between the pre-UTx donor and post-UTx recipient, and pre-UTx and post-UTx recipient reveals a statistically significant decrease in saturation levels post-UTx. CONCLUSION: The use of MSI is the first case in gynecology and has demonstrated promise of possible future human use. MSI technique has advantages over pulse oximetry - it provides spatial information in a real-time, noncontact manner.

Tissue classification for laparoscopic image understanding based on multispectral texture analysis.

Intraoperative tissue classification is one of the prerequisites for providing context-aware visualization in computer-assisted minimally invasive surgeries. As many anatomical structures are difficult to differentiate in conventional RGB medical images, we propose a classification method based on multispectral image patches. In a comprehensive ex vivo study through statistical analysis, we show that (1) multispectral imaging data are superior to RGB data for organ tissue classification when used in conjunction with widely applied feature descriptors and (2) combining the tissue texture with the reflectance spectrum improves the classification performance. The classifier reaches an accuracy of 98.4% on our dataset. Multispectral tissue analysis could thus evolve as a key enabling technique in computer-assisted laparoscopy.

Bayesian Estimation of Intrinsic Tissue Oxygenation and Perfusion From RGB Images.

Multispectral imaging (MSI) can potentially assist the intra-operative assessment of tissue structure, function and viability, by providing information about oxygenation. In this paper, we present a novel technique for recovering intrinsic MSI measurements from endoscopic RGB images without custom hardware adaptations. The advantage of this approach is that it requires no modification to existing surgical and diagnostic endoscopic imaging systems. Our method uses a radiometric color calibration of the endoscopic camera's sensor in conjunction with a Bayesian framework to recover a per-pixel measurement of the total blood volume (THb) and oxygen saturation (SO2) in the observed tissue. The sensor's pixel measurements are modeled as weighted sums over a mixture of Poisson distributions and we optimize the variables SO2 and THb to maximize the likelihood of the observations. To validate our technique, we use synthetic images generated from Monte Carlo physics simulation of light transport through soft tissue containing sub-surface blood vessels. We also validate our method on in vivo data by comparing it to a MSI dataset acquired with a hardware system that sequentially images multiple spectral bands without overlap. Our results are promising and show that we are able to provide surgeons with additional relevant information by processing endoscopic images with our modeling and inference framework.

Multispectral imaging of organ viability during uterine transplantation surgery in rabbits and sheep.

Uterine transplantation surgery (UTx) has been proposed as a treatment for permanent absolute uterine factor infertility (AUFI) in the case of the congenital absence or surgical removal of the uterus. Successful surgical attachment of the organ and its associated vasculature is essential for the organ's reperfusion and long-term viability. Spectral imaging techniques have demonstrated the potential for the measurement of hemodynamics in medical applications. These involve the measurement of reflectance spectra by acquiring images of the tissue in different wavebands. Measures of tissue constituents at each pixel can then be extracted from these spectra through modeling of the light­tissue interaction. A multispectral imaging (MSI) laparoscope was used in sheep and rabbit UTx models to study short- and long-term changes in oxygen saturation following surgery. The whole organ was imaged in the donor and recipient animals in parallel with point measurements from a pulse oximeter. Imaging results confirmed the re-establishment of adequate perfusion in the transplanted organ after surgery. Cornual oxygenation trends measured with MSI are consistent with pulse oximeter readings, showing decreased StO2 immediately after anastomosis of the blood vessels. Long-term results show recovery of StO2 to preoperative levels.

Robust near real-time estimation of physiological parameters from megapixel multispectral images with inverse Monte Carlo and random forest regression.

PURPOSE: Multispectral imaging can provide reflectance measurements at multiple spectral bands for each image pixel. These measurements can be used for estimation of important physiological parameters, such as oxygenation, which can provide indicators for the success of surgical treatment or the presence of abnormal tissue. The goal of this work was to develop a method to estimate physiological parameters in an accurate and rapid manner suited for modern high-resolution laparoscopic images. METHODS: While previous methods for oxygenation estimation are based on either simple linear methods or complex model-based approaches exclusively suited for off-line processing, we propose a new approach that combines the high accuracy of model-based approaches with the speed and robustness of modern machine learning methods. Our concept is based on training random forest regressors using reflectance spectra generated with Monte Carlo simulations. RESULTS: According to extensive in silico and in vivo experiments, the method features higher accuracy and robustness than state-of-the-art online methods and is orders of magnitude faster than other nonlinear regression based methods. CONCLUSION: Our current implementation allows for near real-time oxygenation estimation from megapixel multispectral images and is thus well suited for online tissue analysis.

Intraoperative measurement of bowel oxygen saturation using a multispectral imaging laparoscope.

Intraoperative monitoring of tissue oxygen saturation (StO2 ) has potentially important applications in procedures such as organ transplantation or colorectal surgery, where successful reperfusion affects the viability and integrity of repaired tissues. In this paper a liquid crystal tuneable filter-based multispectral imaging (MSI) laparoscope is described. Motion-induced image misalignments are reduced, using feature-based registration, before regression of the tissue reflectance spectra to calculate relative quantities of oxy- and deoxyhaemoglobin. The laparoscope was validated in vivo, during porcine abdominal surgery, by making parallel MSI and blood gas measurements of the small bowel vasculature. Ischaemic conditions were induced by local occlusion of the mesenteric arcade and monitored using the system. The MSI laparoscope was capable of measuring StO2 over a wide range (30-100%) with a temporal error of ± 7.5%. The imager showed sensitivity to spatial changes in StO2 during dynamic local occlusions, as well as tracking the recovery of tissues post-occlusion.

An endoscopic structured light system using multispectral detection.

PURPOSE: In clinical examinations, the tissue surface topology is an important feature for detecting the tissue pathology and implementing augmented reality. We have previously presented a miniaturised structured light (SL) system for recovery of tissue surface shape in minimally invasive surgery (MIS), based on a flexible multispectral structured illumination probe (1.9 mm diameter) (Clancy et al. in Biomed Opt Express 2(11):3119-3128, 2011. doi: 10.1364/BOE.2.003119 ). This paper reports further hardware and analytical developments to improve the light pattern decoding result and increase the reconstruction accuracy. METHODS: The feasibility of using an 8-band multispectral camera with higher pattern-colour discrimination ability than normal RGB camera in this system was studied. Additionally, the "normalised cut" algorithm was investigated to improve pattern segmentation. RESULTS: The whole SL system was evaluated by phantom and in vivo experiments. Higher pattern identification performance than that of an RGB camera was recorded by using the multispectral camera (average precision >97%, average sensitivity >62%). An average of [Formula: see text] reconstruction error was achieved using the proposed pattern decoding method on a heart phantom at a working distance of approximately 10 cm. CONCLUSIONS: The experiment showed the superiority of the multispectral camera over the RGB camera in the spot identification step. The proposed pattern decoding algorithm underwent evaluations using different experiments, showing that it provided promising reconstruction results. The potential of using this system in MIS environments has been demonstrated.

Robust surface tracking combining features, intensity and illumination compensation.

PURPOSE: Recovering tissue deformation during robotic-assisted minimally invasive surgery procedures is important for providing intra-operative guidance, enabling in vivo imaging modalities and enhanced robotic control. The tissue motion can also be used to apply motion stabilization and to prescribe dynamic constraints for avoiding critical anatomical structures. METHODS: Image-based methods based independently on salient features or on image intensity have limitations when dealing with homogeneous soft tissues or complex reflectance. In this paper, we use a triangular geometric mesh model in order to combine the advantages of both feature and intensity information and track the tissue surface reliably and robustly. RESULTS: Synthetic and in vivo experiments are performed to provide quantitative analysis of the tracking accuracy of our method, and we also show exemplar results for registering multispectral images where there is only a weak image signal. CONCLUSION: Compared to traditional methods, our hybrid tracking method is more robust and has improved convergence in the presence of larger displacements, tissue dynamics and illumination changes.

Polarised stereo endoscope and narrowband detection for minimal access surgery.

Polarisation imaging has the potential to provide enhanced contrast based on variations in the optical properties, such as scattering or birefringence, of the tissue of interest. Examining the signal at different wavebands in the visible spectrum also allows interrogation of different depths and structures. A stereo endoscope has been adapted to allow snapshot acquisition of orthogonal linear polarisation images to generate difference of linear polarisation images. These images are acquired in three narrow bands using a triple-bandpass filter and pair of colour cameras. The first in vivo results, acquired during a surgical procedure on a porcine subject, are presented that show wavelength dependent variations in vessel visibility and an increase in contrast under polarised detection.

Narrow band 3 × 3 Mueller polarimetric endoscopy.

Mueller matrix polarimetric imaging has shown potential in tissue diagnosis but is challenging to implement endoscopically. In this work, a narrow band 3 × 3 Mueller matrix polarimetric endoscope was designed by rotating the endoscope to generate 0°, 45° and 90° linearly polarized illumination and positioning a rotating filter wheel in front of the camera containing three polarisers to permit polarization state analysis for backscattered light. The system was validated with a rotating linear polarizer and a diffuse reflection target. Initial measurements of 3 × 3 Mueller matrices on a rat are demonstrated, followed by matrix decomposition into the depolarization and retardance matrices for further analysis. Our work shows the feasibility of implementing polarimetric imaging in a rigid endoscope conveniently and economically in order to reveal diagnostic information.