Optical characterization of rigid endoscopes and polarization calibration methods.

Polarization imaging can reveal orthogonal information with respect to color about the structural composition of biological tissue, and with the advance of superior polarimeters its use for biomedical applications has proliferated in the last decade. Polarimetry can be used in pre-clinical and clinical settings for the early detection of cancerous tissue. Polarization-based endoscopy with the complementary near-infrared fluorescence imaging modality improves the early diagnosis of flat cancerous lesions in colorectal tumor models. With the development of new polarization sensors the need to use standard laboratory optics to create custom imaging systems increases. These additional optics can behave as polarization filters effectively degrading and modifying the original tissue's polarization signatures leading to erroneous judgments. Here, we present a framework to characterize the spectral and polarization properties of rigid endoscopes for polarization-based endoscopic imaging. We describe and evaluate two calibration schemes based on Mueller calculus to reconstruct the original polarization information. Optical limitations of the endoscopes and minimum polarimeter requirements are discussed that may be of interest to other researchers working with custom polarization-based imaging systems.

Decoupling channel count from field of view and spatial resolution in single-sensor imaging systems for fluorescence image-guided surgery.

Significance: Near-infrared fluorescence image-guided surgery is often thought of as a spectral imaging problem where the channel count is the critical parameter, but it should also be thought of as a multiscale imaging problem where the field of view and spatial resolution are similarly important. Aim: Conventional imaging systems based on division-of-focal-plane architectures suffer from a strict relationship between the channel count on one hand and the field of view and spatial resolution on the other, but bioinspired imaging systems that combine stacked photodiode image sensors and long-pass/short-pass filter arrays offer a weaker tradeoff. Approach: In this paper, we explore how the relevant changes to the image sensor and associated image processing routines affect image fidelity during image-guided surgeries for tumor removal in an animal model of breast cancer and nodal mapping in women with breast cancer. Results: We demonstrate that a transition from a conventional imaging system to a bioinspired one, along with optimization of the image processing routines, yields improvements in multiple measures of spectral and textural rendition relevant to surgical decision-making. Conclusions: These results call for a critical examination of the devices and algorithms that underpin image-guided surgery to ensure that surgeons receive high-quality guidance and patients receive high-quality outcomes as these technologies enter clinical practice.

Comparison of the polarization contrast of gelatinous zooplankton and a transparent single-use plastic bag-Implications for marine animals.

Plastic pollution in the ocean is an increasingly detrimental issue for marine organisms. As a form of polarized light pollution, transparent plastic debris may be more visible and pose additional threats to organisms that can detect and interpret polarized light. Plastic can mimic the visual features of common marine prey items, such as transparent gelatinous zooplankton, which may lead to more significant plastic ingestion. We measured, in situ, the polarization and radiance contrast between a transparent plastic bag and gelatinous zooplankton with an underwater video polarimeter. The plastic bag had significantly higher polarization contrast than the gelatinous zooplankton, yet both shared similar radiance contrasts. This higher polarization contrast may contribute to the observed high ingestion rates of transparent plastic by marine organisms. Further study into the connection between polarization-sensitive organisms and plastic ingestion is recommended.

Hexachromatic bioinspired camera for image-guided cancer surgery.

Cancer affects one in three people worldwide. Surgery remains the primary curative option for localized cancers, but good prognoses require complete removal of primary tumors and timely recognition of metastases. To expand surgical capabilities and enhance patient outcomes, we developed a six-channel color/near-infrared image sensor inspired by the mantis shrimp visual system that enabled near-infrared fluorescence image guidance during surgery. The mantis shrimp's unique eye, which maximizes the number of photons contributing to and the amount of information contained in each glimpse of its surroundings, is recapitulated in our single-chip imaging system that integrates arrays of vertically stacked silicon photodetectors and pixelated spectral filters. To provide information about tumor location unavailable from a single instrument, we tuned three color channels to permit an intuitive perspective of the surgical procedure and three near-infrared channels to permit multifunctional imaging of optical probes highlighting cancerous tissue. In nude athymic mice bearing human prostate tumors, our image sensor enabled simultaneous detection of two tumor-targeted fluorophores, distinguishing diseased from healthy tissue in an estimated 92% of cases. It also permitted extraction of near-infrared structured illumination enabling the mapping of the three-dimensional topography of tumors and surgical sites to within 1.2-mm error. In the operating room, during surgical resection in 18 patients with breast cancer, our image sensor further enabled sentinel lymph node mapping using clinically approved near-infrared fluorophores. The flexibility and performance afforded by this simple and compact architecture highlights the benefits of biologically inspired sensors in image-guided surgery.

Multichannel spectrometers in animals.

Multispectral, hyperspectral, polarimetric, and other types of multichannel imaging spectrometers are coming into common use for a variety of applications, including remote sensing, material identification, forensics, and medical diagnosis. These instruments are often bulky and intolerant of field abuse, so designing compact, reliable, portable, and robust devices is a priority. In contrast to most engineering designs, animals have been building compact and robust multichannel imaging systems for millennia-their eyes. Biological sensors arise by evolution, of course, and are not designed 'for' a particular use; they exist because the creatures that were blessed with useful mutations were better able to survive and reproduce than their competitors. While this is an inefficient process for perfecting a sensor, it brings unexpected innovations and novel concepts into visual system design-concepts that may be useful in the inspiration of new engineered solutions to problematic challenges, like the ones mentioned above. Here, we review a diversity of multichannel visual systems from both vertebrate and invertebrate animals, considering the receptor molecules and cells, spectral sensitivity and its tuning, and some aspects of the higher-level processing systems used to shape spectral (and polarizational) channels in vision. The eyes of mantis shrimps are presented as potential models for biomimetic multichannel imaging systems. We end with a description of a bioinspired, newly developed multichannel spectral/polarimetric imaging system based on mantis shrimp vision that is highly adaptable to field application.

Bio-inspired imager improves sensitivity in near-infrared fluorescence image-guided surgery.

Image-guided surgery can enhance cancer treatment by decreasing, and ideally eliminating, positive tumor margins and iatrogenic damage to healthy tissue. Current state-of-the-art near-infrared fluorescence imaging systems are bulky and costly, lack sensitivity under surgical illumination, and lack co-registration accuracy between multimodal images. As a result, an overwhelming majority of physicians still rely on their unaided eyes and palpation as the primary sensing modalities for distinguishing cancerous from healthy tissue. Here we introduce an innovative design, comprising an artificial multispectral sensor inspired by the Morpho butterfly's compound eye, which can significantly improve image-guided surgery. By monolithically integrating spectral tapetal filters with photodetectors, we have realized a single-chip multispectral imager with 1000 × higher sensitivity and 7 × better spatial co-registration accuracy compared to clinical imaging systems in current use. Preclinical and clinical data demonstrate that this technology seamlessly integrates into the surgical workflow while providing surgeons with real-time information on the location of cancerous tissue and sentinel lymph nodes. Due to its low manufacturing cost, our bio-inspired sensor will provide resource-limited hospitals with much-needed technology to enable more accurate value-based health care.

Near-infrared angiography for critical limb ischemia in a diabetic murine model.

Peripheral arterial disease (PAD) is a highly prevalent disease process that afflicts more than 20% of individuals with diabetes. Progression of PAD in the setting of diabetes can lead to critical limb ischemia (CLI), which is associated with increased risk of wounds, gangrene, and limb loss. Prompt noninvasive evaluation of limbs affected by PAD progression and CLI is currently limited. Here, we evaluate the utility of a custom-designed multispectral imaging system for fluorescence-based near-infrared angiography and compare it to the existing gold standard of laser-scanning Doppler perfusion assessments. Due to its higher resolution and fluorescence sensitivity, near-infrared angiography demonstrates a greater capacity to characterize altered dynamic arterial perfusion in a clinically relevant diabetic murine model for CLI. Furthermore, we demonstrate that our imaging system can accurately track arterial perfusion recovery over time following induced ischemia, and reveal unique phenotypic differences in the setting of diabetes.

Trimodal color-fluorescence-polarization endoscopy aided by a tumor selective molecular probe accurately detects flat lesions in colitis-associated cancer.

Colitis-associated cancer (CAC) arises from premalignant flat lesions of the colon, which are difficult to detect with current endoscopic screening approaches. We have developed a complementary fluorescence and polarization reporting strategy that combines the unique biochemical and physical properties of dysplasia and cancer for real-time detection of these lesions. Using azoxymethane-dextran sodium sulfate (AOM-DSS) treated mice, which recapitulates human CAC and dysplasia, we show that an octapeptide labeled with a near-infrared (NIR) fluorescent dye selectively identified all precancerous and cancerous lesions. A new thermoresponsive sol-gel formulation allowed topical application of the molecular probe during endoscopy. This method yielded high contrast-to-noise ratios (CNR) between adenomatous tumors (20.6 ± 1.65) and flat lesions (12.1 ± 1.03) and surrounding uninvolved colon tissue versus CNR of inflamed tissues (1.62±0.42) Incorporation of nanowire-filtered polarization imaging into NIR fluorescence endoscopy shows a high depolarization contrast in both adenomatous tumors and flat lesions in CAC, reflecting compromised structural integrity of these tissues. Together, the real-time polarization imaging provides real-time validation of suspicious colon tissue highlighted by molecular fluorescence endoscopy.