QuATON: Quantization Aware Training of Optical Neurons.

Optical processors, built with "optical neurons", can efficiently perform high-dimensional linear operations at the speed of light. Thus they are a promising avenue to accelerate large-scale linear computations. With the current advances in micro-fabrication, such optical processors can now be 3D fabricated, but with a limited precision. This limitation translates to quantization of learnable parameters in optical neurons, and should be handled during the design of the optical processor in order to avoid a model mismatch. Specifically, optical neurons should be trained or designed within the physical-constraints at a predefined quantized precision level. To address this critical issues we propose a physics-informed quantization-aware training framework. Our approach accounts for physical constraints during the training process, leading to robust designs. We demonstrate that our approach can design state of the art optical processors using diffractive networks for multiple physics based tasks despite quantized learnable parameters. We thus lay the foundation upon which improved optical processors may be 3D fabricated in the future.

Optical diffraction tomography for assessing single cell models in angular light scattering.

Angularly resolved light scattering (ALS) has become a useful tool for assessing the size and refractive index of biological scatterers at cellular and organelle length scales. Sizing organelle populations with ALS relies on Mie scattering theory models, which require significant assumptions about the object, including spherical scatterers and a homogeneous medium. These assumptions may incur greater error at the single cell level, where there are fewer scatterers to be averaged over. We investigate the validity of these assumptions using 3D refractive index (RI) tomograms measured via optical diffraction tomography (ODT). We compute the angular scattering on digitally manipulated tomograms with increasingly strong model assumptions, including RI-matched immersion media, homogeneous cytosol, and spherical organelles. We also compare the tomogram-computed angular scattering to experimental measurements of angular scattering from the same cells to ensure that the ODT-based approach accurately models angular scattering. We show that enforced RI-matching with the immersion medium and a homogeneous cytosol significantly affects the angular scattering intensity shape, suggesting that these assumptions can reduce the accuracy of size distribution estimates.

Mid-infrared chemical imaging of intracellular tau fibrils using fluorescence-guided computational photothermal microscopy.

Amyloid proteins are associated with a broad spectrum of neurodegenerative diseases. However, it remains a grand challenge to extract molecular structure information from intracellular amyloid proteins in their native cellular environment. To address this challenge, we developed a computational chemical microscope integrating 3D mid-infrared photothermal imaging with fluorescence imaging, termed Fluorescence-guided Bond-Selective Intensity Diffraction Tomography (FBS-IDT). Based on a low-cost and simple optical design, FBS-IDT enables chemical-specific volumetric imaging and 3D site-specific mid-IR fingerprint spectroscopic analysis of tau fibrils, an important type of amyloid protein aggregates, in their intracellular environment. Label-free volumetric chemical imaging of human cells with/without seeded tau fibrils is demonstrated to show the potential correlation between lipid accumulation and tau aggregate formation. Depth-resolved mid-infrared fingerprint spectroscopy is performed to reveal the protein secondary structure of the intracellular tau fibrils. 3D visualization of the ß-sheet for tau fibril structure is achieved.

Pupil engineering for extended depth-of-field imaging in a fluorescence miniscope.

Significance: Fluorescence head-mounted microscopes, i.e., miniscopes, have emerged as powerful tools to analyze in-vivo neural populations but exhibit a limited depth-of-field (DoF) due to the use of high numerical aperture (NA) gradient refractive index (GRIN) objective lenses. Aim: We present extended depth-of-field (EDoF) miniscope, which integrates an optimized thin and lightweight binary diffractive optical element (DOE) onto the GRIN lens of a miniscope to extend the DoF by 2.8× between twin foci in fixed scattering samples. Approach: We use a genetic algorithm that considers the GRIN lens' aberration and intensity loss from scattering in a Fourier optics-forward model to optimize a DOE and manufacture the DOE through single-step photolithography. We integrate the DOE into EDoF-Miniscope with a lateral accuracy of 70 µm to produce high-contrast signals without compromising the speed, spatial resolution, size, or weight. Results: We characterize the performance of EDoF-Miniscope across 5- and 10-µm fluorescent beads embedded in scattering phantoms and demonstrate that EDoF-Miniscope facilitates deeper interrogations of neuronal populations in a 100-µm-thick mouse brain sample and vessels in a whole mouse brain sample. Conclusions: Built from off-the-shelf components and augmented by a customizable DOE, we expect that this low-cost EDoF-Miniscope may find utility in a wide range of neural recording applications.

Mid-infrared Chemical Imaging of Intracellular Tau Fibrils using Fluorescence-guided Computational Photothermal Microscopy.

Amyloid proteins are associated with a broad spectrum of neurodegenerative diseases. However, it remains a grand challenge to extract molecular structure information from intracellular amyloid proteins in their native cellular environment. To address this challenge, we developed a computational chemical microscope integrating 3D mid-infrared photothermal imaging with fluorescence imaging, termed Fluorescence-guided Bond-Selective Intensity Diffraction Tomography (FBS-IDT). Based on a low-cost and simple optical design, FBS-IDT enables chemical-specific volumetric imaging and 3D site-specific mid-IR fingerprint spectroscopic analysis of tau fibrils, an important type of amyloid protein aggregates, in their intracellular environment. Label-free volumetric chemical imaging of human cells with/without seeded tau fibrils is demonstrated to show the potential correlation between lipid accumulation and tau aggregate formation. Depth-resolved mid-infrared fingerprint spectroscopy is performed to reveal the protein secondary structure of the intracellular tau fibrils. 3D visualization of the \b{eta}-sheet for tau fibril structure is achieved.

Multiple-scattering simulator-trained neural network for intensity diffraction tomography.

Recovering 3D phase features of complex biological samples traditionally sacrifices computational efficiency and processing time for physical model accuracy and reconstruction quality. Here, we overcome this challenge using an approximant-guided deep learning framework in a high-speed intensity diffraction tomography system. Applying a physics model simulator-based learning strategy trained entirely on natural image datasets, we show our network can robustly reconstruct complex 3D biological samples. To achieve highly efficient training and prediction, we implement a lightweight 2D network structure that utilizes a multi-channel input for encoding the axial information. We demonstrate this framework on experimental measurements of weakly scattering epithelial buccal cells and strongly scattering C. elegans worms. We benchmark the network's performance against a state-of-the-art multiple-scattering model-based iterative reconstruction algorithm. We highlight the network's robustness by reconstructing dynamic samples from a living worm video. We further emphasize the network's generalization capabilities by recovering algae samples imaged from different experimental setups. To assess the prediction quality, we develop a quantitative evaluation metric to show that our predictions are consistent with both multiple-scattering physics and experimental measurements.

Bond-selective intensity diffraction tomography.

Recovering molecular information remains a grand challenge in the widely used holographic and computational imaging technologies. To address this challenge, we developed a computational mid-infrared photothermal microscope, termed Bond-selective Intensity Diffraction Tomography (BS-IDT). Based on a low-cost brightfield microscope with an add-on pulsed light source, BS-IDT recovers both infrared spectra and bond-selective 3D refractive index maps from intensity-only measurements. High-fidelity infrared fingerprint spectra extraction is validated. Volumetric chemical imaging of biological cells is demonstrated at a speed of ~20 s per volume, with a lateral and axial resolution of ~350 nm and ~1.1 µm, respectively. BS-IDT's application potential is investigated by chemically quantifying lipids stored in cancer cells and volumetric chemical imaging on Caenorhabditis elegans with a large field of view (~100 µm x 100 µm).

LED array reflectance microscopy for scattering-based multi-contrast imaging.

LED array microscopy is an emerging platform for computational imaging with significant utility for biological imaging. Existing LED array systems often exploit transmission imaging geometries of standard brightfield microscopes that leave the rich backscattered field undetected. This backscattered signal contains high-resolution sample information with superb sensitivity to subtle structural features that make it ideal for biological sensing and detection. Here, we develop an LED array reflectance microscope capturing the sample's backscattered signal. In particular, we demonstrate multimodal brightfield, darkfield, and differential phase contrast imaging on fixed and living biological specimens including Caenorhabditis elegans (C. elegans), zebrafish embryos, and live cell cultures. Video-rate multimodal imaging at 20 Hz records real time features of freely moving C. elegans and the fast beating heart of zebrafish embryos. Our new reflectance mode is a valuable addition to the LED array microscopic toolbox.

Inverse scattering for reflection intensity phase microscopy.

Reflection phase imaging provides label-free, high-resolution characterization of biological samples, typically using interferometric-based techniques. Here, we investigate reflection phase microscopy from intensity-only measurements under diverse illumination. We evaluate the forward and inverse scattering model based on the first Born approximation for imaging scattering objects above a glass slide. Under this design, the measured field combines linear forward-scattering and height-dependent nonlinear back-scattering from the object that complicates object phase recovery. Using only the forward-scattering, we derive a linear inverse scattering model and evaluate this model's validity range in simulation and experiment using a standard reflection microscope modified with a programmable light source. Our method provides enhanced contrast of thin, weakly scattering samples that complement transmission techniques. This model provides a promising development for creating simplified intensity-based reflection quantitative phase imaging systems easily adoptable for biological research.

High-Throughput, High-Resolution Interferometric Light Microscopy of Biological Nanoparticles.

Label-free, visible light microscopy is an indispensable tool for studying biological nanoparticles (BNPs). However, conventional imaging techniques have two major challenges: (i) weak contrast due to low-refractive-index difference with the surrounding medium and exceptionally small size and (ii) limited spatial resolution. Advances in interferometric microscopy have overcome the weak contrast limitation and enabled direct detection of BNPs, yet lateral resolution remains as a challenge in studying BNP morphology. Here, we introduce a wide-field interferometric microscopy technique augmented by computational imaging to demonstrate a 2-fold lateral resolution improvement over a large field-of-view (>100 × 100 µm2), enabling simultaneous imaging of more than 104 BNPs at a resolution of ∼150 nm without any labels or sample preparation. We present a rigorous vectorial-optics-based forward model establishing the relationship between the intensity images captured under partially coherent asymmetric illumination and the complex permittivity distribution of nanoparticles. We demonstrate high-throughput morphological visualization of a diverse population of Ebola virus-like particles and a structurally distinct Ebola vaccine candidate. Our approach offers a low-cost and robust label-free imaging platform for high-throughput and high-resolution characterization of a broad size range of BNPs.

High-throughput, volumetric quantitative phase imaging with multiplexed intensity diffraction tomography.

Intensity diffraction tomography (IDT) provides quantitative, volumetric refractive index reconstructions of unlabeled biological samples from intensity-only measurements. IDT is scanless and easily implemented in standard optical microscopes using an LED array but suffers from large data requirements and slow acquisition speeds. Here, we develop multiplexed IDT (mIDT), a coded illumination framework providing high volume-rate IDT for evaluating dynamic biological samples. mIDT combines illuminations from an LED grid using physical model-based design choices to improve acquisition rates and reduce dataset size with minimal loss to resolution and reconstruction quality. We analyze the optimal design scheme with our mIDT framework in simulation using the reconstruction error compared to conventional IDT and theoretical acquisition speed. With the optimally determined mIDT scheme, we achieve hardware-limited 4Hz acquisition rates enabling 3D refractive index distribution recovery on live Caenorhabditis elegans worms and embryos as well as epithelial buccal cells. Our mIDT architecture provides a 60 × speed improvement over conventional IDT and is robust across different illumination hardware designs, making it an easily adoptable imaging tool for volumetrically quantifying biological samples in their natural state.

Vertical-cavity surface-emitting laser sources for gigahertz-bandwidth, multiwavelength frequency-domain photon migration.

Frequency-domain photon migration (FDPM) uses modulated laser light to measure the bulk optical properties of turbid media and is increasingly applied for noninvasive functional medical imaging in the near-infrared. Although semiconductor edge-emitting laser diodes have been traditionally used as miniature light sources for this application, we show that vertical-cavity surface-emitting lasers (VCSELs) exhibit output power and modulation performance characteristics suitable for FDPM measurements of tissue optical properties at modulation frequencies exceeding 1 GHz. We also show that an array of multiple VCSEL devices can be coherently modulated at frequencies suitable for FDPM and can improve optical power. In addition, their small size and simple packaging make them an attractive choice as components in wearable sensors and clinical FDPM-based optical spectroscopy systems. We demonstrate the benefits of VCSEL technology by fabricating and testing a unique, compact VCSEL-based optical probe with an integrated avalanche photodiode. We demonstrate sensitivity of the VCSEL-based probe to subcutaneous tissue hemodynamics that was induced during an arterial cuff occlusion of the upper arm in a human subject.

Optical imaging correlates with magnetic resonance imaging breast density and reveals composition changes during neoadjuvant chemotherapy.

INTRODUCTION: In addition to being a risk factor for breast cancer, breast density has been hypothesized to be a surrogate biomarker for predicting response to endocrine-based chemotherapies. The purpose of this study was to evaluate whether a noninvasive bedside scanner based on diffuse optical spectroscopic imaging (DOSI) provides quantitative metrics to measure and track changes in breast tissue composition and density. To access a broad range of densities in a limited patient population, we performed optical measurements on the contralateral normal breast of patients before and during neoadjuvant chemotherapy (NAC). In this work, DOSI parameters, including tissue hemoglobin, water, and lipid concentrations, were obtained and correlated with magnetic resonance imaging (MRI)-measured fibroglandular tissue density. We evaluated how DOSI could be used to assess breast density while gaining new insight into the impact of chemotherapy on breast tissue. METHODS: This was a retrospective study of 28 volunteers undergoing NAC treatment for breast cancer. Both 3.0-T MRI and broadband DOSI (650 to 1,000 nm) were obtained from the contralateral normal breast before and during NAC. Longitudinal DOSI measurements were used to calculate breast tissue concentrations of oxygenated and deoxygenated hemoglobin, water, and lipid. These values were compared with MRI-measured fibroglandular density before and during therapy. RESULTS: Water (r = 0.843; P < 0.001), deoxyhemoglobin (r = 0.785; P = 0.003), and lipid (r = -0.707; P = 0.010) concentration measured with DOSI correlated strongly with MRI-measured density before therapy. Mean DOSI parameters differed significantly between pre- and postmenopausal subjects at baseline (water, P < 0.001; deoxyhemoglobin, P = 0.024; lipid, P = 0.006). During NAC treatment measured at about 90 days, significant reductions were observed in oxyhemoglobin for pre- (-20.0%; 95% confidence interval (CI), -32.7 to -7.4) and postmenopausal subjects (-20.1%; 95% CI, -31.4 to -8.8), and water concentration for premenopausal subjects (-11.9%; 95% CI, -17.1 to -6.7) compared with baseline. Lipid increased slightly in premenopausal subjects (3.8%; 95% CI, 1.1 to 6.5), and water increased slightly in postmenopausal subjects (4.4%; 95% CI, 0.1 to 8.6). Percentage change in water at the end of therapy compared with baseline correlated strongly with percentage change in MRI-measured density (r = 0.864; P = 0.012). CONCLUSIONS: DOSI functional measurements correlate with MRI fibroglandular density, both before therapy and during NAC. Although from a limited patient dataset, these results suggest that DOSI may provide new functional indices of density based on hemoglobin and water that could be used at the bedside to assess response to therapy and evaluate disease risk.