Signal to Noise Ratio as a Cross-Platform Metric for Intraoperative Fluorescence Imaging.

Real-time molecular imaging to guide curative cancer surgeries is critical to ensure removal of all tumor cells; however, visualization of microscopic tumor foci remains challenging. Wide variation in both imager instrumentation and molecular labeling agents demands a common metric conveying the ability of a system to identify tumor cells. Microscopic disease, comprised of a small number of tumor cells, has a signal on par with the background, making the use of signal (or tumor) to background ratio inapplicable in this critical regime. Therefore, a metric that incorporates the ability to subtract out background, evaluating the signal itself relative to the sources of uncertainty, or noise is required. Here we introduce the signal to noise ratio (SNR) to characterize the ultimate sensitivity of an imaging system and optimize factors such as pixel size. Variation in the background (noise) is due to electronic sources, optical sources, and spatial sources (heterogeneity in tumor marker expression, fluorophore binding, and diffusion). Here, we investigate the impact of these noise sources and ways to limit its effect on SNR. We use empirical tumor and noise measurements to procedurally generate tumor images and run a Monte Carlo simulation of microscopic disease imaging to optimize parameters such as pixel size.

Angle-insensitive amorphous silicon optical filter for fluorescence contact imaging.

We introduce a novel amorphous silicon absorption filter that has high rejection for all angles of incident light for wavelengths below approximately 700 nm. This filter is used for microscopic cancer tissue detection in a small intraoperative contact fluorescence imaging system that requires excitation light at oblique angles. Our 15 µm thick filter presents over five orders of magnitude rejection at 633 nm, making it compatible with several clinically tested fluorophores, including IR700DX. We have demonstrated imaging of fluorescently labeled human epidermal growth factor receptor 2+ breast cancer tissue using the filter, and we can reliably detect microscopic clusters of breast cancer cells with only a 75 ms integration time.

Multicolor fluorescence microscopy for surgical guidance using a chip-scale imager with a low-NA fiber optic plate and a multi-bandpass interference filter.

In curative-intent cancer surgery, intraoperative fluorescence imaging of both diseased and healthy tissue can help to ensure successful removal of all gross and microscopic disease with minimal damage to neighboring critical structures, such as nerves. Current fluorescence-guided surgery (FGS) systems, however, rely on bulky and rigid optics that incur performance-limiting trade-offs between sensitivity and maneuverability. Moreover, many FGS systems are incapable of multiplexed imaging. As a result, clinical FGS is currently limited to millimeter-scale detection of a single fluorescent target. Here we present a scalable, lens-less fluorescence imaging chip, VISION, capable of sensitive and multiplexed detection within a compact form factor. Central to VISION is a novel optical frontend design combining a low-numerical-aperture fiber optic plate (LNA-FOP) and a multi-bandpass interference filter, which is affixed to a custom CMOS image sensor. The LNA-FOP acts as a planar collimator to improve resolution and compensate for the angle-sensitivity of the interference filter, enabling high-resolution and multiplexed fluorescence imaging without lenses. We show VISION is capable of detecting tumor foci of less than 100 cells at near video framerates and, as proof of principle, can simultaneously visualize both tumor and nerves in ex vivo prostate tissue.

Multicolor fluorescence microscopy for surgical guidance using a chip-scale imager with a low-NA fiber optic plate and a multi-bandpass interference filter.

In curative-intent cancer surgery, intraoperative fluorescence imaging of both diseased and healthy tissue can help to ensure the successful removal of all gross and microscopic diseases with minimal damage to neighboring critical structures, such as nerves. Current fluorescence-guided surgery (FGS) systems, however, rely on bulky and rigid optics that incur performance-limiting trade-offs between sensitivity and maneuverability. Moreover, many FGS systems are incapable of multiplexed imaging. As a result, clinical FGS is currently limited to millimeter-scale detection of a single fluorescent target. Here, we present a scalable, lens-less fluorescence imaging chip, VISION, capable of sensitive and multiplexed detection within a compact form factor. Central to VISION is a novel optical frontend design combining a low-numerical-aperture fiber optic plate (LNA-FOP) and a multi-bandpass interference filter, which is affixed to a custom CMOS image sensor. The LNA-FOP acts as a planar collimator to improve resolution and compensate for the angle-sensitivity of the interference filter, enabling high-resolution and multiplexed fluorescence imaging without lenses. We show VISION is capable of detecting tumor foci of less than 100 cells at near video framerates and, as proof of principle, can simultaneously visualize both tumors and nerves in ex vivo prostate tissue.

Towards A Wireless Image Sensor for Real-Time Fluorescence Microscopy in Cancer Therapy.

We present a mm-sized, ultrasonically powered lensless CMOS image sensor as a progress towards wireless fluorescence microscopy. Access to biological information within the tissue has the potential to provide insights guiding diagnosis and treatment across numerous medical conditions including cancer therapy. This information, in conjunction with current clinical imaging techniques that have limitations in obtaining images continuously and lack wireless compatibility, can improve continual detection of multicell clusters deep within tissue. The proposed platform incorporates a 2.4×4.7 mm2 integrated circuit (IC) fabricated in TSMC 0.18 µm, a micro laser diode (µLD), a single piezoceramic and off-chip storage capacitors. The IC consists of a 36×40 array of capacitive trans-impedance amplifier-based pixels, wireless power management and communication via ultrasound and a laser driver all controlled by a Finite State Machine. The piezoceramic harvests energy from the acoustic waves at a depth of 2 cm to power up the IC and transfer 11.5 kbits/frame via backscattering. During Charge-Up, the off-chip capacitor stores charge to later supply a high-power 78 mW µLD during Imaging. Proof of concept of the imaging front end is shown by imaging distributions of CD8 T-cells, an indicator of the immune response to cancer, ex vivo, in the lymph nodes of a functional immune system (BL6 mice) against colorectal cancer consistent with the results of a fluorescence microscope. The overall system performance is verified by detecting 140 µm features on a USAF resolution target with 32 ms exposure time and 389 ms ultrasound backscattering.

Towards A Wireless Image Sensor for Real-Time Fluorescence Microscopy in Cancer Therapy.

We present a mm-sized, ultrasonically powered lensless CMOS image sensor as a progress towards wireless fluorescence microscopy. Access to biological information within the tissue has the potential to provide insights guiding diagnosis and treatment across numerous medical conditions including cancer therapy. This information, in conjunction with current clinical imaging techniques that have limitations in obtaining images continuously and lack wireless compatibility, can improve continual detection of multicell clusters deep within tissue. The proposed platform incorporates a 2.4×4.7 mm2 integrated circuit (IC) fabricated in TSMC 0.18 µm, a micro laser diode (µLD), a single piezoceramic and off-chip storage capacitors. The IC consists of a 36×40 array of capacitive trans-impedance amplifier-based pixels, wireless power management and communication via ultrasound and a laser driver all controlled by a Finite State Machine. The piezoceramic harvests energy from the acoustic waves at a depth of 2 cm to power up the IC and transfer 11.5 kbits/frame via backscattering. During Charge-Up, the off-chip capacitor stores charge to later supply a high-power 78 mW µLD during Imaging. Proof of concept of the imaging front end is shown by imaging distributions of CD8 T-cells, an indicator of the immune response to cancer, ex vivo, in the lymph nodes of a functional immune system (BL6 mice) against colorectal cancer consistent with the results of a fluorescence microscope. The overall system performance is verified by detecting 140 µm features on a USAF resolution target with 32 ms exposure time and 389 ms ultrasound backscattering.

3D Reconstruction of cellular images from microfabricated imagers using fully-adaptive deep neural networks.

Millimeter-scale multi-cellular level imagers enable various applications, ranging from intraoperative surgical navigation to implantable sensors. However, the tradeoffs for miniaturization compromise resolution, making extracting 3D cell locations challenging-critical for tumor margin assessment and therapy monitoring. This work presents three machine-learning-based modules that extract spatial information from single image acquisitions using custom-made millimeter-scale imagers. The neural networks were trained on synthetically-generated (using Perlin noise) cell images. The first network is a convolutional neural network estimating the depth of a single layer of cells, the second is a deblurring module correcting for the point spread function (PSF). The final module extracts spatial information from a single image acquisition of a 3D specimen and reconstructs cross-sections, by providing a layered "map" of cell locations. The maximum depth error of the first module is 100 µm, with 87% test accuracy. The second module's PSF correction achieves a least-square-error of only 4%. The third module generates a binary "cell" or "no cell" per-pixel labeling with an accuracy ranging from 89% to 85%. This work demonstrates the synergy between ultra-small silicon-based imagers that enable in vivo imaging but face a trade-off in spatial resolution, and the processing power of neural networks to achieve enhancements beyond conventional linear optimization techniques.

Towards an Implantable Fluorescence Image Sensor for Real-Time Monitoring of Immune Response in Cancer Therapy.

Real-time monitoring of cellular-level changes inside the body provides key information regarding disease progression and therapy assessment for critical care including cancer therapy. Current state-of-the-art oncological imaging methods impose unnecessary latencies to detect small cell foci. Invasive methods such as biopsies, on the other hand, cause disruption if deployed on a repeated basis. Therefore, they are not practical for real-time assessments of the tumor tissue. This work presents a proof-of-concept design for an implantable fluorescence lensless image sensor to address the pervasive challenge of real-time tracking of the immune response in immunotherapy. The 2.4x4.7 mm2 integrated circuit (IC) prototype consists of a 36 by 40 pixel array, a laser driver and a power management unit harvesting power and transferring 11.5 kbits/frame through a wireless ultrasound link while implanted 2 cm deep inside the body. Compared to prior art, this is the first full-fledged wireless system implementing chip-scale fluorescence microscopy to the best of our knowledge.Clinical relevance- This prototype can be used to personalize immunotherapy for the 50% of cancer patients who do not initially respond to the therapy.

Chip-Scale Angle-Selective Imager for In Vivo Microscopic Cancer Detection.

We present an image sensor incorporating angle-selective gratings for resolution enhancement in contact imaging applications. Optical structures designed in the CMOS metal layers above each photodiode form the angle-selective gratings that limit the sensor angle of view to ±18 °, rejecting background light and deblurring the image. The imager is based on a high-gain capacitive transimpedance amplifier pixel using a custom 11fF MOM capacitor, achieving [Formula: see text] sensitivity. The pixel includes a leakage current minimization circuit to remove signal-dependent reset switch leakage and the corresponding dark current is [Formula: see text]. The resulting 4.7 mm by 2.25 mm sensor (80 by 36 pixels) is designed specifically for intraoperative cancer imaging in order to solve the pervasive challenge of identifying microscopic residual cancer foci in vivo, where they can be removed. We demonstrate imaging and detection of foci containing less than 200 cancer cells labeled with fluorescent biomarkers in 50 ms with signal-to-noise ratios greater than 15 dB and the detection of microscopic residual tumor in mice models. The absence of large optical elements enables extreme miniaturization, allowing manipulation within a small, morphologically complex, tumor cavity.

A 25 micron-thin microscope for imaging upconverting nanoparticles with NIR-I and NIR-II illumination.

Rationale: Intraoperative visualization in small surgical cavities and hard-to-access areas are essential requirements for modern, minimally invasive surgeries and demand significant miniaturization. However, current optical imagers require multiple hard-to-miniaturize components including lenses, filters and optical fibers. These components restrict both the form-factor and maneuverability of these imagers, and imagers largely remain stand-alone devices with centimeter-scale dimensions. Methods: We have engineered INSITE (Immunotargeted Nanoparticle Single-Chip Imaging Technology), which integrates the unique optical properties of lanthanide-based alloyed upconverting nanoparticles (aUCNPs) with the time-resolved imaging of a 25-micron thin CMOS-based (complementary metal oxide semiconductor) imager. We have synthesized core/shell aUCNPs of different compositions and imaged their visible emission with INSITE under either NIR-I and NIR-II photoexcitation. We characterized aUCNP imaging with INSITE across both varying aUCNP composition and 980 nm and 1550 nm excitation wavelengths. To demonstrate clinical experimental validity, we also conducted an intratumoral injection into LNCaP prostate tumors in a male nude mouse that was subsequently excised and imaged with INSITE. Results: Under the low illumination fluences compatible with live animal imaging, we measure aUCNP radiative lifetimes of 600 µs - 1.3 ms, which provides strong signal for time-resolved INSITE imaging. Core/shell NaEr0.6Yb0.4F4 aUCNPs show the highest INSITE signal when illuminated at either 980 nm or 1550 nm, with signal from NIR-I excitation about an order of magnitude brighter than from NIR-II excitation. The 55 µm spatial resolution achievable with this approach is demonstrated through imaging of aUCNPs in PDMS (polydimethylsiloxane) micro-wells, showing resolution of micrometer-scale targets with single-pixel precision. INSITE imaging of intratumoral NaEr0.8Yb0.2F4 aUCNPs shows a signal-to-background ratio of 9, limited only by photodiode dark current and electronic noise. Conclusion: This work demonstrates INSITE imaging of aUCNPs in tumors, achieving an imaging platform that is thinned to just a 25 µm-thin, planar form-factor, with both NIR-I and NIR-II excitation. Based on a highly paralleled array structure INSITE is scalable, enabling direct coupling with a wide array of surgical and robotic tools for seamless integration with tissue actuation, resection or ablation.

Real-time cancer detection with an integrated lensless fluorescence contact imager.

Microscopic tumor cell foci left in a patient after surgery significantly increase the chance of cancer recurrence. However, fluorescence microscopes used for intraoperative navigation lack the necessary sensitivity for imaging microscopic disease and are too bulky to maneuver within the resection cavity. We have developed a scalable chip-scale fluorescence contact imager for detecting microscopic cancer in vivo and in real-time. The imager has been characterized under simulated in vivo conditions using ex vivo samples, providing strong evidence that our device can be used in vivo. Angle-selective gratings enhance the resolution of the imager without impacting its physical size. We demonstrate detection of cancer cell clusters containing as few as 25 HCC1569 breast cancer cells and 400 LNCaP prostate cancer cells with integration times of only 50 ms and 70 ms, respectively. A cell cluster recognition algorithm is used to achieve both a sensitivity and specificity of 92 % for HCC1569 cell samples, indicating the reliability of the imager. The signal-to-noise ratio (SNR) degradation with increased separation is only 1.5 dB at 250 µm. Blood scattering and absorption reduce the SNR by less than 2 dB for typical concentrations. Moreover, HER2+ breast cancer tissue taken from a patient is distinguished from normal breast tissue with an integration time of only 75 ms.

Imaging of IR700DX Labeled Mouse Breast Tumor Using a Custom Angle-Selective Fluorescence Contact Imaging System.

Cancer treatment faces the challenge of identifying small clusters of residual tumor cells in the resection cavity after the gross section of tumor is surgically removed. Despite the introduction of targeted fluorescent probes to guide cancer surgeries, large, bulky, optical components restrict the ability of fluorescence imaging devices to detect small clusters of tumor cells in the complex surgical cavity. We have developed a small size-scale contact fluorescence image sensor that incorporates angle-selective gratings and a thin 15 m amorphous silicon optical wavelength filter for detecting residual cancer tissue in vivo. Using a custom fluorescent probe combining a fluorescent dye, IR700DX, with a targeted antibody, Trastuzumab, we label and visualize breast tissue in in vivo mouse models of breast cancer. When imaging tumorbearing mice injected with the probe, HER2+ breast cancer tissue intensity is 3.80.8 times brighter than other tissue. Excised cancer tumors and residual cancer attached to healthy tissue are imaged using the custom image sensor. Residual cancer tissue can be detected in real-time and is imaged with a high SNR of 45 dB using an integration time of only 40 ms.