Bioinspired, vertically stacked, and perovskite nanocrystal-enhanced CMOS imaging sensors for resolving UV spectral signatures.

Imaging and identifying target signatures and biomedical markers in the ultraviolet (UV) spectrum is broadly important to medical imaging, military target tracking, remote sensing, and industrial automation. However, current silicon-based imaging sensors are fundamentally limited because of the rapid absorption and attenuation of UV light, hindering their ability to resolve UV spectral signatures. Here, we present a bioinspired imaging sensor capable of wavelength-resolved imaging in the UV range. Inspired by the UV-sensitive visual system of the Papilio xuthus butterfly, the sensor monolithically combines vertically stacked photodiodes and perovskite nanocrystals. This imaging design combines two complementary UV detection mechanisms: The nanocrystal layer converts a portion of UV signals into visible fluorescence, detected by the photodiode array, while the remaining UV light is detected by the top photodiode. Our label-free UV fluorescence imaging data from aromatic amino acids and cancer/normal cells enables real-time differentiation of these biomedical materials with 99% confidence.

Fluorescence-guided surgical system using holographic display: from phantom studies to canine patients.

Significance: Holographic display technology is a promising area of research that can lead to significant advancements in cancer surgery. We present the benefits of combining bioinspired multispectral imaging technology with holographic goggles for fluorescence-guided cancer surgery. Through a series of experiments with 43D-printed phantoms, small animal models of cancer, and surgeries on canine patients with head and neck cancer, we showcase the advantages of this holistic approach. Aim: The aim of our study is to demonstrate the feasibility and potential benefits of utilizing holographic display for fluorescence-guided surgery through a series of experiments involving 3D-printed phantoms and canine patients with head and neck cancer. Approach: We explore the integration of a bioinspired camera with a mixed reality headset to project fluorescent images as holograms onto a see-through display, and we demonstrate the potential benefits of this technology through benchtop and in vivo animal studies. Results: Our complete imaging and holographic display system showcased improved delineation of fluorescent targets in phantoms compared with the 2D monitor display approach and easy integration into the veterinarian surgical workflow. Conclusions: Based on our findings, it is evident that our comprehensive approach, which combines a bioinspired multispectral imaging sensor with holographic goggles, holds promise in enhancing the presentation of fluorescent information to surgeons during intraoperative scenarios while minimizing disruptions.

Bioinspired color-near infrared endoscopic imaging system for molecular guided cancer surgery.

Significance: Fluorescently guided minimally invasive surgery is improving patient outcomes and disease-free survival, but biomarker variability hinders complete tumor resection with single molecular probes. To overcome this, we developed a bioinspired endoscopic system that images multiple tumor-targeted probes, quantifies volumetric ratios in cancer models, and detects tumors in ex vivo samples. Aim: We present a new rigid endoscopic imaging system (EIS) that can capture color images while simultaneously resolving two near-infrared (NIR) probes. Approach: Our optimized EIS integrates a hexa-chromatic image sensor, a rigid endoscope optimized for NIR-color imaging, and a custom illumination fiber bundle. Results: Our optimized EIS achieves a 60% improvement in NIR spatial resolution when compared to a leading FDA-approved endoscope. Ratio-metric imaging of two tumor-targeted probes is demonstrated in vials and animal models of breast cancer. Clinical data gathered from fluorescently tagged lung cancer samples on the operating room's back table demonstrate a high tumor-to-background ratio and consistency with the vial experiments. Conclusions: We investigate key engineering breakthroughs for the single-chip endoscopic system, which can capture and distinguish numerous tumor-targeting fluorophores. As the molecular imaging field shifts toward a multi-tumor targeted probe methodology, our imaging instrument can aid in assessing these concepts during surgical procedures.

Protease-activated indocyanine green nanoprobes for intraoperative NIR fluorescence imaging of primary tumors.

Tumor-targeted fluorescent probes in the near-infrared spectrum can provide invaluable information about the location and extent of primary and metastatic tumors during intraoperative procedures to ensure no residual tumors are left in the patient's body. Even though the first fluorescence-guided surgery was performed more than 50 years ago, it is still not accepted as a standard of care in part due to the lack of efficient and non-toxic targeted probes approved by regulatory agencies around the world. Herein, we report protease-activated cationic gelatin nanoparticles encapsulating indocyanine green (ICG) for the detection of primary breast tumors in murine models with high tumor-to-background ratios. Upon intravenous administration, these nanoprobes remain optically silent due to the energy resonance transfer among the bound ICG molecules. As the nanoprobes extravasate and are exposed to the acidic tumor microenvironment, their positive surface charges increase, facilitating cellular uptake. The internalized nanoprobes are activated upon proteolytic degradation of gelatin to allow high contrast between the tumor and normal tissue. Since both gelatin and ICG are FDA-approved for intravenous administration, this activatable nanoprobe can lead to quick clinical adoption and improve the treatment of patients undergoing image-guided cancer surgery.

Decoupling channel count from field of view and spatial resolution in single-sensor imaging systems for fluorescence image-guided surgery.

Significance: Near-infrared fluorescence image-guided surgery is often thought of as a spectral imaging problem where the channel count is the critical parameter, but it should also be thought of as a multiscale imaging problem where the field of view and spatial resolution are similarly important. Aim: Conventional imaging systems based on division-of-focal-plane architectures suffer from a strict relationship between the channel count on one hand and the field of view and spatial resolution on the other, but bioinspired imaging systems that combine stacked photodiode image sensors and long-pass/short-pass filter arrays offer a weaker tradeoff. Approach: In this paper, we explore how the relevant changes to the image sensor and associated image processing routines affect image fidelity during image-guided surgeries for tumor removal in an animal model of breast cancer and nodal mapping in women with breast cancer. Results: We demonstrate that a transition from a conventional imaging system to a bioinspired one, along with optimization of the image processing routines, yields improvements in multiple measures of spectral and textural rendition relevant to surgical decision-making. Conclusions: These results call for a critical examination of the devices and algorithms that underpin image-guided surgery to ensure that surgeons receive high-quality guidance and patients receive high-quality outcomes as these technologies enter clinical practice.

Hexachromatic bioinspired camera for image-guided cancer surgery.

Cancer affects one in three people worldwide. Surgery remains the primary curative option for localized cancers, but good prognoses require complete removal of primary tumors and timely recognition of metastases. To expand surgical capabilities and enhance patient outcomes, we developed a six-channel color/near-infrared image sensor inspired by the mantis shrimp visual system that enabled near-infrared fluorescence image guidance during surgery. The mantis shrimp's unique eye, which maximizes the number of photons contributing to and the amount of information contained in each glimpse of its surroundings, is recapitulated in our single-chip imaging system that integrates arrays of vertically stacked silicon photodetectors and pixelated spectral filters. To provide information about tumor location unavailable from a single instrument, we tuned three color channels to permit an intuitive perspective of the surgical procedure and three near-infrared channels to permit multifunctional imaging of optical probes highlighting cancerous tissue. In nude athymic mice bearing human prostate tumors, our image sensor enabled simultaneous detection of two tumor-targeted fluorophores, distinguishing diseased from healthy tissue in an estimated 92% of cases. It also permitted extraction of near-infrared structured illumination enabling the mapping of the three-dimensional topography of tumors and surgical sites to within 1.2-mm error. In the operating room, during surgical resection in 18 patients with breast cancer, our image sensor further enabled sentinel lymph node mapping using clinically approved near-infrared fluorophores. The flexibility and performance afforded by this simple and compact architecture highlights the benefits of biologically inspired sensors in image-guided surgery.