Targeted multispectral filter array design for the optimization of endoscopic cancer detection in the gastrointestinal tract.

Significance: Color differences between healthy and diseased tissue in the gastrointestinal (GI) tract are detected visually by clinicians during white light endoscopy; however, the earliest signs of cancer are often just a slightly different shade of pink compared to healthy tissue making it hard to detect. Improving contrast in endoscopy is important for early detection of disease in the GI tract during routine screening and surveillance. Aim: We aim to target alternative colors for imaging to improve contrast using custom multispectral filter arrays (MSFAs) that could be deployed in an endoscopic "chip-on-tip" configuration. Approach: Using an open-source toolbox, Opti-MSFA, we examined the optimal design of MSFAs for early cancer detection in the GI tract. The toolbox was first extended to use additional classification models (k-nearest neighbor, support vector machine, and spectral angle mapper). Using input spectral data from published clinical trials examining the esophagus and colon, we optimized the design of MSFAs with three to nine different bands. Results: We examined the variation of the spectral and spatial classification accuracies as a function of the number of bands. The MSFA configurations tested showed good classification accuracies when compared to the full hyperspectral data available from the clinical spectra used in these studies. Conclusion: The ability to retain good classification accuracies with a reduced number of spectral bands could enable the future deployment of multispectral imaging in an endoscopic chip-on-tip configuration using simplified MSFA hardware. Further studies using an expanded clinical dataset are needed to confirm these findings.

Enhancing intraoperative tumor delineation with multispectral short-wave infrared fluorescence imaging and machine learning.

Significance: Fluorescence-guided surgery (FGS) provides specific real-time visualization of tumors, but intensity-based measurement of fluorescence is prone to errors. Multispectral imaging (MSI) in the short-wave infrared (SWIR) has the potential to improve tumor delineation by enabling machine-learning classification of pixels based on their spectral characteristics. Aim: Determine whether MSI can be applied to FGS and combined with machine learning to provide a robust method for tumor visualization. Approach: A multispectral SWIR fluorescence imaging device capable of collecting data from six spectral filters was constructed and deployed on neuroblastoma (NB) subcutaneous xenografts ( n = 6 ) after the injection of a NB-specific NIR-I fluorescent probe (Dinutuximab-IRDye800). We constructed image cubes representing fluorescence collected from ∼ 850 to 1450 nm and compared the performance of seven learning-based methods for pixel-by-pixel classification, including linear discriminant analysis, k -nearest neighbor classification, and a neural network. Results: The spectra of tumor and non-tumor tissue were subtly different and conserved between individuals. In classification, a combine principal component analysis and k -nearest-neighbor approach with area under curve normalization performed best, achieving 97.5% per-pixel classification accuracy (97.1%, 93.5%, and 99.2% for tumor, non-tumor tissue and background, respectively). Conclusions: The development of dozens of new imaging agents provides a timely opportunity for multispectral SWIR imaging to revolutionize next-generation FGS.

Shortwave Infrared Imaging Enables High-Contrast Fluorescence-Guided Surgery in Neuroblastoma.

Fluorescence-guided surgery is set to play a pivotal role in the intraoperative management of pediatric tumors. Shortwave infrared imaging (SWIR) has advantages over conventional near-infrared I (NIR-I) imaging with reduced tissue scattering and autofluorescence. Here, two NIR-I dyes (IRDye800CW and IR12), with long tails emitting in the SWIR range, were conjugated with a clinical-grade anti-GD2 monoclonal antibody (dinutuximab-beta) to compare NIR-I and SWIR imaging for neuroblastoma surgery. A first-of-its-kind multispectral NIR-I/SWIR fluorescence imaging device was constructed to allow an objective comparison between the two imaging windows. Conjugates were first characterized in vitro. Tissue-mimicking phantoms, imaging specimens of known geometric and material composition, were used to assess the sensitivity and depth penetration of the NIR-I/SWIR device, showing a minimum detectable volume of ∼0.9 mm3 and depth penetration up to 3 mm. In vivo, fluorescence imaging using the NIR-I/SWIR device showed a high tumor-to-background ratio (TBR) for both dyes, with anti-GD2-IR800 being significantly brighter than anti-GD2-IR12. Crucially, the system enabled higher TBR at SWIR wavelengths than at NIR-I wavelengths, verifying SWIR imaging enables high-contrast delineation of tumor margins. This work demonstrates that by combining the high specificity of anti-GD2 antibodies with the availability and translatability of existing NIR-I dyes, along with the advantages of SWIR in terms of depth and tumor signal-to-background ratio, GD2-targeted NIR-I/SWIR-guided surgery could improve the treatment of patients with neuroblastoma, warranting investigation in future clinical trials. SIGNIFICANCE: Multispectral near-infrared I/shortwave infrared fluorescence imaging is a versatile system enabling high tumor-to-background signal for safer and more complete resection of pediatric tumors during surgery.

First-in-human pilot study of snapshot multispectral endoscopy for early detection of Barrett's-related neoplasia.

SIGNIFICANCE: The early detection of dysplasia in patients with Barrett's esophagus could improve outcomes by enabling curative intervention; however, dysplasia is often inconspicuous using conventional white-light endoscopy. AIM: We sought to determine whether multispectral imaging (MSI) could be applied in endoscopy to improve detection of dysplasia in the upper gastrointestinal (GI) tract. APPROACH: We used a commercial fiberscope to relay imaging data from within the upper GI tract to a snapshot MSI camera capable of collecting data from nine spectral bands. The system was deployed in a pilot clinical study of 20 patients (ClinicalTrials.gov NCT03388047) to capture 727 in vivo image cubes matched with gold-standard diagnosis from histopathology. We compared the performance of seven learning-based methods for data classification, including linear discriminant analysis, k-nearest neighbor classification, and a neural network. RESULTS: Validation of our approach using a Macbeth color chart achieved an image-based classification accuracy of 96.5%. Although our patient cohort showed significant intra- and interpatient variance, we were able to resolve disease-specific contributions to the recorded MSI data. In classification, a combined principal component analysis and k-nearest-neighbor approach performed best, achieving accuracies of 95.8%, 90.7%, and 76.1%, respectively, for squamous, non-dysplastic Barrett's esophagus and neoplasia based on majority decisions per-image. CONCLUSIONS: MSI shows promise for disease classification in Barrett's esophagus and merits further investigation as a tool in high-definition "chip-on-tip" endoscopes.

Spectral Endoscopy Enhances Contrast for Neoplasia in Surveillance of Barrett's Esophagus.

Early detection of esophageal neoplasia enables curative endoscopic therapy, but the current diagnostic standard of care has low sensitivity because early neoplasia is often inconspicuous with conventional white-light endoscopy. Here, we hypothesized that spectral endoscopy could enhance contrast for neoplasia in surveillance of patients with Barrett's esophagus. A custom spectral endoscope was deployed in a pilot clinical study of 20 patients to capture 715 in vivo tissue spectra matched with gold standard diagnosis from histopathology. Spectral endoscopy was sensitive to changes in neovascularization during the progression of disease; both non-dysplastic and neoplastic Barrett's esophagus showed higher blood volume relative to healthy squamous tissue (P = 0.001 and 0.02, respectively), and vessel radius appeared larger in neoplasia relative to non-dysplastic Barrett's esophagus (P = 0.06). We further developed a deep learning algorithm capable of classifying spectra of neoplasia versus non-dysplastic Barrett's esophagus with high accuracy (84.8% accuracy, 83.7% sensitivity, 85.5% specificity, 78.3% positive predictive value, and 89.4% negative predictive value). Exploiting the newly acquired library of labeled spectra to model custom color filter sets identified a potential 12-fold enhancement in contrast between neoplasia and non-dysplastic Barrett's esophagus using application-specific color filters compared with standard-of-care white-light imaging (perceptible color difference = 32.4 and 2.7, respectively). This work demonstrates the potential of endoscopic spectral imaging to extract vascular properties in Barrett's esophagus, to classify disease stages using deep learning, and to enable high-contrast endoscopy. SIGNIFICANCE: The results of this pilot first-in-human clinical trial demonstrate the potential of spectral endoscopy to reveal disease-associated vascular changes and to provide high-contrast delineation of neoplasia in the esophagus. GRAPHICAL ABSTRACT: http://cancerres.aacrjournals.org/content/canres/81/12/3415/F1.large.jpg.

First experience in clinical application of hyperspectral endoscopy for evaluation of colonic polyps.

Early detection and resection of adenomatous polyps prevents their progression to colorectal cancer (CRC), significantly improving patient outcomes. Polyps are typically identified and removed during white-light colonoscopy. Unfortunately, the rate of interval cancers that arise between CRC screening events remains high, linked to poor visualization of polyps during screening and incomplete polyp removal. Here, we sought to evaluate the potential of a hyperspectral endoscope (HySE) to enhance polyp discrimination for detection and resection. We designed, built and tested a new compact HySE in a proof-of-concept clinical study. We successfully collected spectra from three tissue types in seven patients undergoing routine colonoscopy screening. The acquired spectral data from normal tissue and polyps, both pre- and post- resection, were subjected to quantitative analysis using spectral angle mapping and machine learning, which discriminated the data by tissue type, meriting further investigation of HySE as a clinical tool.

A roadmap for the clinical implementation of optical-imaging biomarkers.

Clinical workflows for the non-invasive detection and characterization of disease states could benefit from optical-imaging biomarkers. In this Perspective, we discuss opportunities and challenges towards the clinical implementation of optical-imaging biomarkers for the early detection of cancer by analysing two case studies: the assessment of skin lesions in primary care, and the surveillance of patients with Barrett's oesophagus in specialist care. We stress the importance of technical and biological validations and clinical-utility assessments, and the need to address implementation bottlenecks. In addition, we define a translational roadmap for the widespread clinical implementation of optical-imaging technologies.

A clinically translatable hyperspectral endoscopy (HySE) system for imaging the gastrointestinal tract.

Hyperspectral imaging (HSI) enables visualisation of morphological and biochemical information, which could improve disease diagnostic accuracy. Unfortunately, the wide range of image distortions that arise during flexible endoscopy in the clinic have made integration of HSI challenging. To address this challenge, we demonstrate a hyperspectral endoscope (HySE) that simultaneously records intrinsically co-registered hyperspectral and standard-of-care white light images, which allows image distortions to be compensated computationally and an accurate hyperspectral data cube to be reconstructed as the endoscope moves in the lumen. Evaluation of HySE performance shows excellent spatial, spectral and temporal resolution and high colour fidelity. Application of HySE enables: quantification of blood oxygenation levels in tissue mimicking phantoms; differentiation of spectral profiles from normal and pathological ex vivo human tissues; and recording of hyperspectral data under freehand motion within an intact ex vivo pig oesophagus model. HySE therefore shows potential for enabling HSI in clinical endoscopy.

Quantitative evaluation of comb-structure correction methods for multispectral fibrescopic imaging.

Removing the comb artifact introduced by imaging fibre bundles, or 'fibrescopes', for example in medical endoscopy, is essential to provide high quality images to the observer. Multispectral imaging (MSI) is an emerging method that combines morphological (spatial) and chemical (spectral) information in a single data 'cube'. When a fibrescope is coupled to a spectrally resolved detector array (SRDA) to perform MSI, comb removal is complicated by the demosaicking step required to reconstruct the multispectral data cube. To understand the potential for using SRDAs as multispectral imaging sensors in medical endoscopy, we assessed five comb correction methods with respect to five performance metrics relevant to biomedical imaging applications: processing time, resolution, smoothness, signal and the accuracy of spectral reconstruction. By assigning weights to each metric, which are determined by the particular imaging application, our results can be used to select the correction method to achieve best overall performance. In most cases, interpolation gave the best compromise between the different performance metrics when imaging using an SRDA.

Bimodal reflectance and fluorescence multispectral endoscopy based on spectrally resolving detector arrays.

Emerging clinical interest in combining standard white light endoscopy with targeted near-infrared (NIR) fluorescent contrast agents for improved early cancer detection has created demand for multimodal imaging endoscopes. We used two spectrally resolving detector arrays (SRDAs) to realize a bimodal endoscope capable of simultaneous reflectance-based imaging in the visible spectral region and multiplexed fluorescence-based imaging in the NIR. The visible SRDA was composed of 16 spectral bands, with peak wavelengths in the range of 463 to 648 nm and full-width at half-maximum (FWHM) between 9 and 26 nm. The NIR SRDA was composed of 25 spectral bands, with peak wavelengths in the range 659 to 891 nm and FWHM 7 to 15 nm. The spectral endoscope design was based on a "babyscope" model using a commercially available imaging fiber bundle. We developed a spectral transmission model to select optical components and provide reference endmembers for linear spectral unmixing of the recorded image data. The technical characterization of the spectral endoscope is presented, including evaluation of the angular field-of-view, barrel distortion, spatial resolution and spectral fidelity, which showed encouraging performance. An agarose phantom containing oxygenated and deoxygenated blood with three fluorescent dyes was then imaged. After spectral unmixing, the different chemical components of the phantom could be successfully identified via majority decision with high signal-to-background ratio (>3). Imaging performance was further assessed in an ex vivo porcine esophagus model. Our preliminary imaging results demonstrate the capability to simultaneously resolve multiple biological components using a compact spectral endoscopy system.

Emerging optical methods for endoscopic surveillance of Barrett's oesophagus.

Barrett's oesophagus is an acquired metaplastic condition that predisposes patients to the development of oesophageal adenocarcinoma, prompting the use of surveillance regimes to detect early malignancy for endoscopic therapy with curative intent. The currently accepted surveillance regime uses white light endoscopy together with random biopsies, but has poor sensitivity and discards information from numerous light-tissue interactions that could be exploited to probe structural, functional, and molecular changes in the tissue. Advanced optical methods are now emerging that are highly sensitive to these changes and hold potential to improve surveillance of Barrett's oesophagus if they can be applied endoscopically. The next decade will see some of these exciting new methods applied to surveillance of Barrett's oesophagus in new device architectures for the first time, potentially leading to a long-awaited improvement in the standard of care.

Detection of early neoplasia in Barrett's esophagus using lectin-based near-infrared imaging: an ex vivo study on human tissue.

BACKGROUND AND STUDY AIMS: Endoscopic surveillance for Barrett's esophagus (BE) is limited by long procedure times and sampling error. Near-infrared (NIR) fluorescence imaging minimizes tissue autofluorescence and optical scattering. We assessed the feasibility of a topically applied NIR dye-labeled lectin for the detection of early neoplasia in BE in an ex vivo setting. METHODS: Consecutive patients undergoing endoscopic mucosal resection (EMR) for BE-related early neoplasia were recruited. Freshly collected EMR specimens were sprayed at the bedside with fluorescent lectin and then imaged. Punch biopsies were collected from each EMR under NIR light guidance. We compared the fluorescence intensity from dysplastic and nondysplastic areas within EMRs and from punch biopsies with different histological grades. RESULTS: 29 EMR specimens were included from 17 patients. A significantly lower fluorescence was found for dysplastic regions across whole EMR specimens (P < 0.001). We found a 41 % reduction in the fluorescence of dysplastic compared to nondysplastic punch biopsies (P < 0.001), with a sensitivity and specificity for dysplasia detection of 80 % and 82.9 %, respectively. CONCLUSION: Lectin-based NIR imaging can differentiate dysplastic from nondysplastic Barrett's mucosa ex vivo.

Design and validation of a near-infrared fluorescence endoscope for detection of early esophageal malignancy.

Barrett's esophagus is a known precursor lesion to esophageal adenocarcinoma. In these patients, early detection of premalignant disease, known as dysplasia, allows curative minimally invasive endoscopic therapy, but is confounded by a lack of contrast in white light endoscopy. Imaging fluorescently labeled lectins applied topically to the tissue has the potential to more accurately delineate dysplasia, but tissue autofluorescence limits both sensitivity and contrast when operating in the visible region. To overcome this challenge, we synthesized near-infrared (NIR) fluorescent wheat germ agglutinin (WGA-IR800CW) and constructed a clinically translatable bimodal NIR and white light endoscope. Images of NIR and white light with a field of view of 63 deg and an image resolution of 182 µm are coregistered and the honeycomb artifact arising from the fiber bundle is removed. A minimum detectable concentration of 110 nM was determined using a dilution series of WGA-IR800CW. We demonstrated ex vivo that this system can distinguish between gastric and squamous tissue types in mouse stomachs (p=0.0005) and accurately detect WGA-IR800CW fluorescence in human esophageal resections (compared with a gold standard imaging system, rs>0.90). Based on these findings, future work will optimize the bimodal endoscopic system for clinical trials in Barrett's surveillance.