Shot-noise limited optical hybrid based on fused fiber couplers.

We describe a fiber-based coherent receiver topology which utilizes intrinsic phase shifts from fiber couplers to enable instantaneous quadrature projection with shot-noise limited signal-to-noise ratio (SNR). Fused 3 × 3 fiber couplers generate three phase-shifted signals simultaneously that can be combined with quadrature projection methods to detect magnitude and phase unambiguously. We present a novel, to the best of our knowledge, differential detection topology which utilizes a combination of 3 × 3 and 2 × 2 couplers to enable quadrature projection with fully differential detection. We present a mathematical analysis of this 3 × 3 differential detection topology, extended methods for signal calibration, and SNR analysis. We characterize the SNR advantage of this approach and demonstrate a sample application illustrating simultaneous magnitude and phase imaging of a chrome-on-glass test chart.

Parallelized computational 3D video microscopy of freely moving organisms at multiple gigapixels per second.

Wide field of view microscopy that can resolve 3D information at high speed and spatial resolution is highly desirable for studying the behaviour of freely moving model organisms. However, it is challenging to design an optical instrument that optimises all these properties simultaneously. Existing techniques typically require the acquisition of sequential image snapshots to observe large areas or measure 3D information, thus compromising on speed and throughput. Here, we present 3D-RAPID, a computational microscope based on a synchronized array of 54 cameras that can capture high-speed 3D topographic videos over an area of 135 cm2, achieving up to 230 frames per second at spatiotemporal throughputs exceeding 5 gigapixels per second. 3D-RAPID employs a 3D reconstruction algorithm that, for each synchronized snapshot, fuses all 54 images into a composite that includes a co-registered 3D height map. The self-supervised 3D reconstruction algorithm trains a neural network to map raw photometric images to 3D topography using stereo overlap redundancy and ray-propagation physics as the only supervision mechanism. The resulting reconstruction process is thus robust to generalization errors and scales to arbitrarily long videos from arbitrarily sized camera arrays. We demonstrate the broad applicability of 3D-RAPID with collections of several freely behaving organisms, including ants, fruit flies, and zebrafish larvae.

Conserved chamber-specific polyploidy maintains heart function in Drosophila.

Developmentally programmed polyploidy (whole-genome duplication) of cardiomyocytes is common across evolution. Functions of such polyploidy are essentially unknown. Here, in both Drosophila larvae and human organ donors, we reveal distinct polyploidy levels in cardiac organ chambers. In Drosophila, differential growth and cell cycle signal sensitivity leads the heart chamber to reach a higher ploidy/cell size relative to the aorta chamber. Cardiac ploidy-reduced animals exhibit reduced heart chamber size, stroke volume and cardiac output, and acceleration of circulating hemocytes. These Drosophila phenotypes mimic human cardiomyopathies. Our results identify productive and likely conserved roles for polyploidy in cardiac chambers and suggest that precise ploidy levels sculpt many developing tissues. These findings of productive cardiomyocyte polyploidy impact efforts to block developmental polyploidy to improve heart injury recovery.

Multi-modal imaging using a cascaded microscope design.

We present a multi-modal fiber array snapshot technique (M-FAST) based on an array of 96 compact cameras placed behind a primary objective lens and a fiber bundle array. Our technique is capable of large-area, high-resolution, multi-channel video acquisition. The proposed design provides two key improvements to prior cascaded imaging system approaches: a novel optical arrangement that accommodates the use of planar camera arrays, and a new ability to acquire multi-modal image data acquisition. M-FAST is a multi-modal, scalable imaging system that can acquire snapshot dual-channel fluorescence images as well as differential phase contrast measurements over a large 6.59 mm × 9.74 mm field-of-view at 2.2-µm center full-pitch resolution.

Conserved Chamber-Specific Polyploidy Maintains Heart Function in Drosophila.

Developmentally programmed polyploidy (whole-genome-duplication) of cardiomyocytes is common across evolution. Functions of such polyploidy are essentially unknown. Here, we reveal roles for precise polyploidy levels in cardiac tissue. We highlight a conserved asymmetry in polyploidy level between cardiac chambers in Drosophila larvae and humans. In Drosophila , differential Insulin Receptor (InR) sensitivity leads the heart chamber to reach a higher ploidy/cell size relative to the aorta chamber. Cardiac ploidy-reduced animals exhibit reduced heart chamber size, stroke volume, cardiac output, and acceleration of circulating hemocytes. These Drosophila phenotypes mimic systemic human heart failure. Using human donor hearts, we reveal asymmetry in nuclear volume (ploidy) and insulin signaling between the left ventricle and atrium. Our results identify productive and likely conserved roles for polyploidy in cardiac chambers and suggest precise ploidy levels sculpt many developing tissues. These findings of productive cardiomyocyte polyploidy impact efforts to block developmental polyploidy to improve heart injury recovery.

Parallelized computational 3D video microscopy of freely moving organisms at multiple gigapixels per second.

To study the behavior of freely moving model organisms such as zebrafish (Danio rerio) and fruit flies (Drosophila) across multiple spatial scales, it would be ideal to use a light microscope that can resolve 3D information over a wide field of view (FOV) at high speed and high spatial resolution. However, it is challenging to design an optical instrument to achieve all of these properties simultaneously. Existing techniques for large-FOV microscopic imaging and for 3D image measurement typically require many sequential image snapshots, thus compromising speed and throughput. Here, we present 3D-RAPID, a computational microscope based on a synchronized array of 54 cameras that can capture high-speed 3D topographic videos over a 135-cm^2 area, achieving up to 230 frames per second at throughputs exceeding 5 gigapixels (GPs) per second. 3D-RAPID features a 3D reconstruction algorithm that, for each synchronized temporal snapshot, simultaneously fuses all 54 images seamlessly into a globally-consistent composite that includes a coregistered 3D height map. The self-supervised 3D reconstruction algorithm itself trains a spatiotemporally-compressed convolutional neural network (CNN) that maps raw photometric images to 3D topography, using stereo overlap redundancy and ray-propagation physics as the only supervision mechanism. As a result, our end-to-end 3D reconstruction algorithm is robust to generalization errors and scales to arbitrarily long videos from arbitrarily sized camera arrays. The scalable hardware and software design of 3D-RAPID addresses a longstanding problem in the field of behavioral imaging, enabling parallelized 3D observation of large collections of freely moving organisms at high spatiotemporal throughputs, which we demonstrate in ants (Pogonomyrmex barbatus), fruit flies, and zebrafish larvae.

Transient Motion Classification Through Turbid Volumes via Parallelized Single-Photon Detection and Deep Contrastive Embedding.

Fast noninvasive probing of spatially varying decorrelating events, such as cerebral blood flow beneath the human skull, is an essential task in various scientific and clinical settings. One of the primary optical techniques used is diffuse correlation spectroscopy (DCS), whose classical implementation uses a single or few single-photon detectors, resulting in poor spatial localization accuracy and relatively low temporal resolution. Here, we propose a technique termed C lassifying R apid decorrelation E vents via P arallelized single photon d E tection (CREPE), a new form of DCS that can probe and classify different decorrelating movements hidden underneath turbid volume with high sensitivity using parallelized speckle detection from a 32 × 32 pixel SPAD array. We evaluate our setup by classifying different spatiotemporal-decorrelating patterns hidden beneath a 5 mm tissue-like phantom made with rapidly decorrelating dynamic scattering media. Twelve multi-mode fibers are used to collect scattered light from different positions on the surface of the tissue phantom. To validate our setup, we generate perturbed decorrelation patterns by both a digital micromirror device (DMD) modulated at multi-kilo-hertz rates, as well as a vessel phantom containing flowing fluid. Along with a deep contrastive learning algorithm that outperforms classic unsupervised learning methods, we demonstrate our approach can accurately detect and classify different transient decorrelation events (happening in 0.1-0.4 s) underneath turbid scattering media, without any data labeling. This has the potential to be applied to non-invasively monitor deep tissue motion patterns, for example identifying normal or abnormal cerebral blood flow events, at multi-Hertz rates within a compact and static detection probe.

Imaging Dynamics Beneath Turbid Media via Parallelized Single-Photon Detection.

Noninvasive optical imaging through dynamic scattering media has numerous important biomedical applications but still remains a challenging task. While standard diffuse imaging methods measure optical absorption or fluorescent emission, it is also well-established that the temporal correlation of scattered coherent light diffuses through tissue much like optical intensity. Few works to date, however, have aimed to experimentally measure and process such temporal correlation data to demonstrate deep-tissue video reconstruction of decorrelation dynamics. In this work, a single-photon avalanche diode array camera is utilized to simultaneously monitor the temporal dynamics of speckle fluctuations at the single-photon level from 12 different phantom tissue surface locations delivered via a customized fiber bundle array. Then a deep neural network is applied to convert the acquired single-photon measurements into video of scattering dynamics beneath rapidly decorrelating tissue phantoms. The ability to reconstruct images of transient (0.1-0.4 s) dynamic events occurring up to 8 mm beneath a decorrelating tissue phantom with millimeter-scale resolution is demonstrated, and it is highlighted how the model can flexibly extend to monitor flow speed within buried phantom vessels.

Quantitative Jones matrix imaging using vectorial Fourier ptychography.

This paper presents a microscopic imaging technique that uses variable-angle illumination to recover the complex polarimetric properties of a specimen at high resolution and over a large field-of-view. The approach extends Fourier ptychography, which is a synthetic aperture-based imaging approach to improve resolution with phaseless measurements, to additionally account for the vectorial nature of light. After images are acquired using a standard microscope outfitted with an LED illumination array and two polarizers, our vectorial Fourier ptychography (vFP) algorithm solves for the complex 2x2 Jones matrix of the anisotropic specimen of interest at each resolved spatial location. We introduce a new sequential Gauss-Newton-based solver that additionally jointly estimates and removes polarization-dependent imaging system aberrations. We demonstrate effective vFP performance by generating large-area (29 mm2), high-resolution (1.24 µm full-pitch) reconstructions of sample absorption, phase, orientation, diattenuation, and retardance for a variety of calibration samples and biological specimens.

Video-rate high-precision time-frequency multiplexed 3D coherent ranging.

Frequency-modulated continuous wave (FMCW) light detection and ranging (LiDAR) is an emerging 3D ranging technology that offers high sensitivity and ranging precision. Due to the limited bandwidth of digitizers and the speed limitations of beam steering using mechanical scanners, meter-scale FMCW LiDAR systems typically suffer from a low 3D frame rate, which greatly restricts their applications in real-time imaging of dynamic scenes. In this work, we report a high-speed FMCW based 3D imaging system, combining a grating for beam steering with a compressed time-frequency analysis approach for depth retrieval. We thoroughly investigate the localization accuracy and precision of our system both theoretically and experimentally. Finally, we demonstrate 3D imaging results of multiple static and moving objects, including a flexing human hand. The demonstrated technique achieves submillimeter localization accuracy over a tens-of-centimeter imaging range with an overall depth voxel acquisition rate of 7.6 MHz, enabling densely sampled 3D imaging at video rate.

Increasing a microscope's effective field of view via overlapped imaging and machine learning.

This work demonstrates a multi-lens microscopic imaging system that overlaps multiple independent fields of view on a single sensor for high-efficiency automated specimen analysis. Automatic detection, classification and counting of various morphological features of interest is now a crucial component of both biomedical research and disease diagnosis. While convolutional neural networks (CNNs) have dramatically improved the accuracy of counting cells and sub-cellular features from acquired digital image data, the overall throughput is still typically hindered by the limited space-bandwidth product (SBP) of conventional microscopes. Here, we show both in simulation and experiment that overlapped imaging and co-designed analysis software can achieve accurate detection of diagnostically-relevant features for several applications, including counting of white blood cells and the malaria parasite, leading to multi-fold increase in detection and processing throughput with minimal reduction in accuracy.

Computational 3D microscopy with optical coherence refraction tomography.

Optical coherence tomography (OCT) has seen widespread success as an in vivo clinical diagnostic 3D imaging modality, impacting areas including ophthalmology, cardiology, and gastroenterology. Despite its many advantages, such as high sensitivity, speed, and depth penetration, OCT suffers from several shortcomings that ultimately limit its utility as a 3D microscopy tool, such as its pervasive coherent speckle noise and poor lateral resolution required to maintain millimeter-scale imaging depths. Here, we present 3D optical coherence refraction tomography (OCRT), a computational extension of OCT which synthesizes an incoherent contrast mechanism by combining multiple OCT volumes, acquired across two rotation axes, to form a resolution-enhanced, speckle-reduced, refraction-corrected 3D reconstruction. Our label-free computational 3D microscope features a novel optical design incorporating a parabolic mirror to enable the capture of 5D plenoptic datasets, consisting of millimetric 3D fields of view over up to ±75° without moving the sample. We demonstrate that 3D OCRT reveals 3D features unobserved by conventional OCT in fruit fly, zebrafish, and mouse samples.

In vivo quantitative analysis of anterior chamber white blood cell mixture composition using spectroscopic optical coherence tomography.

Anterior uveitis is the most common form of intraocular inflammation, and one of its main signs is the presence of white blood cells (WBCs) in the anterior chamber (AC). Clinically, the true composition of cells can currently only be obtained using AC paracentesis, an invasive procedure to obtain AC fluid requiring needle insertion into the AC. We previously developed a spectroscopic optical coherence tomography (SOCT) analysis method to differentiate between populations of RBCs and subtypes of WBCs, including granulocytes, lymphocytes and monocytes, both in vitro and in ACs of excised porcine eyes. We have shown that different types of WBCs have distinct characteristic size distributions, extracted from the backscattered reflectance spectrum of individual cells using Mie theory. Here, we further develop our method to estimate the composition of blood cell mixtures, both in vitro and in vivo. To do so, we estimate the size distribution of unknown cell mixtures by fitting the distribution observed using SOCT with a weighted combination of reference size distributions of each WBC type calculated using kernel density estimation. We validate the accuracy of our estimation in an in vitro study, by comparing our results for a given WBC sample mixture with the cellular concentrations measured by a hemocytometer and SOCT images before mixing. We also conducted a small in vivo quantitative cell mixture validation pilot study which demonstrates congruence between our method and AC paracentesis in two patients with uveitis. The SOCT based method appears promising to provide quantitative diagnostic information of cellular responses in the ACs of patients with uveitis.

Diffraction tomography with a deep image prior.

We present a tomographic imaging technique, termed Deep Prior Diffraction Tomography (DP-DT), to reconstruct the 3D refractive index (RI) of thick biological samples at high resolution from a sequence of low-resolution images collected under angularly varying illumination. DP-DT processes the multi-angle data using a phase retrieval algorithm that is extended by a deep image prior (DIP), which reparameterizes the 3D sample reconstruction with an untrained, deep generative 3D convolutional neural network (CNN). We show that DP-DT effectively addresses the missing cone problem, which otherwise degrades the resolution and quality of standard 3D reconstruction algorithms. As DP-DT does not require pre-captured data or pre-training, it is not biased towards any particular dataset. Hence, it is a general technique that can be applied to a wide variety of 3D samples, including scenarios in which large datasets for supervised training would be infeasible or expensive. We applied DP-DT to obtain 3D RI maps of bead phantoms and complex biological specimens, both in simulation and experiment, and show that DP-DT produces higher-quality results than standard regularization techniques. We further demonstrate the generality of DP-DT, using two different scattering models, the first Born and multi-slice models. Our results point to the potential benefits of DP-DT for other 3D imaging modalities, including X-ray computed tomography, magnetic resonance imaging, and electron microscopy.

Fourier ptychography: current applications and future promises.

Traditional imaging systems exhibit a well-known trade-off between the resolution and the field of view of their captured images. Typical cameras and microscopes can either "zoom in" and image at high-resolution, or they can "zoom out" to see a larger area at lower resolution, but can rarely achieve both effects simultaneously. In this review, we present details about a relatively new procedure termed Fourier ptychography (FP), which addresses the above trade-off to produce gigapixel-scale images without requiring any moving parts. To accomplish this, FP captures multiple low-resolution, large field-of-view images and computationally combines them in the Fourier domain into a high-resolution, large field-of-view result. Here, we present details about the various implementations of FP and highlight its demonstrated advantages to date, such as aberration recovery, phase imaging, and 3D tomographic reconstruction, to name a few. After providing some basics about FP, we list important details for successful experimental implementation, discuss its relationship with other computational imaging techniques, and point to the latest advances in the field while highlighting persisting challenges.

Spectroscopic optical coherence refraction tomography.

In optical coherence tomography (OCT), the axial resolution is often superior to the lateral resolution, which is sacrificed for long imaging depths. To address this anisotropy, we previously developed optical coherence refraction tomography (OCRT), which uses images from multiple angles to computationally reconstruct an image with isotropic resolution, given by the OCT axial resolution. On the other hand, spectroscopic OCT (SOCT), an extension of OCT, trades axial resolution for spectral resolution and hence often has superior lateral resolution. Here, we present spectroscopic OCRT (SOCRT), which uses SOCT images from multiple angles to reconstruct a spectroscopic image with isotropic spatial resolution limited by the OCT lateral resolution. We experimentally show that SOCRT can estimate bead size based on Mie theory at simultaneously high spectral and isotropic spatial resolution. We also applied SOCRT to a biological sample, achieving axial resolution enhancement limited by the lateral resolution.

Learned sensing: jointly optimized microscope hardware for accurate image classification.

Since its invention, the microscope has been optimized for interpretation by a human observer. With the recent development of deep learning algorithms for automated image analysis, there is now a clear need to re-design the microscope's hardware for specific interpretation tasks. To increase the speed and accuracy of automated image classification, this work presents a method to co-optimize how a sample is illuminated in a microscope, along with a pipeline to automatically classify the resulting image, using a deep neural network. By adding a "physical layer" to a deep classification network, we are able to jointly optimize for specific illumination patterns that highlight the most important sample features for the particular learning task at hand, which may not be obvious under standard illumination. We demonstrate how our learned sensing approach for illumination design can automatically identify malaria-infected cells with up to 5-10% greater accuracy than standard and alternative microscope lighting designs. We show that this joint hardware-software design procedure generalizes to offer accurate diagnoses for two different blood smear types, and experimentally show how our new procedure can translate across different experimental setups while maintaining high accuracy.

Ocular anterior chamber blood cell population differentiation using spectroscopic optical coherence tomography.

There is potential clinical significance in identifying cellular responses in the anterior chamber (AC) of the eye, which can indicate hyphema (an accumulation of red blood cells [RBCs]) or aberrant intraocular inflammation (an accumulation of white blood cells [WBCs]). In this work, we developed a spectroscopic OCT analysis method to differentiate between populations of RBCs and subtypes of WBCs, including granulocytes, lymphocytes and monocytes, both in vitro and in ACs of porcine eyes. We developed an algorithm to track single cells within OCT data sets, and extracted the backscatter reflectance spectrum of each single cell from the detected interferograms using the short-time Fourier transform (STFT). A look-up table of Mie back-scattering spectra was generated and used to correlate the backscatter spectral features of single cells to their characteristic sizes. The extracted size distributions based on the best Mie spectra fit were significantly different between each cell type. We also studied theoretical backscattering models of single RBCs to further validate our experimental results. The described work is a promising step towards clinically differentiating and quantifying AC blood cell types.

Unraveling the molecular nature of melanin changes in metastatic cancer.

More people die from melanoma after a stage I diagnosis than after a stage IV diagnosis, because the tools available to clinicians do not readily identify which early-stage cancers will be aggressive. Near-infrared pump-probe microscopy detects fundamental differences in melanin structure between benign human moles and melanoma and also correlates with metastatic potential. However, the biological mechanisms of these changes have been difficult to quantify, as many different mechanisms can contribute to the pump-probe signal. We use model systems (sepia, squid, and synthetic eumelanin), cellular uptake studies, and a range of pump and probe wavelengths to demonstrate that the clinically observed effects come from alterations of the aggregated mode from "thick oligomer stacks" to "thin oligomer stacks" (due to changes in monomer composition) and (predominantly) deaggregation of the assembled melanin structure. This provides the opportunity to use pump-probe microscopy for the detection and study of melanin-associated diseases.

Optical coherence refraction tomography.

Optical coherence tomography (OCT) is a cross-sectional, micron-scale imaging modality with widespread clinical application. Typical OCT systems sacrifice lateral resolution to achieve long depths of focus for bulk tissue imaging, and hence tend to have better axial than lateral resolution. Such anisotropic resolution can obscure fine ultrastructural features. Furthermore, conventional OCT suffers from refraction-induced image distortions. Here, we introduce optical coherence refraction tomography (OCRT), which extends the superior axial resolution to the lateral dimension, synthesising undistorted cross-sectional image reconstructions from multiple conventional images acquired with angular diversity. In correcting refraction-induced distortions to register the OCT images, OCRT also achieves spatially resolved refractive index imaging. We demonstrate >3-fold improvement in lateral resolution as well as speckle reduction in imaging tissue ultrastructure, consistent with histology. With further optimisation in optical designs to incorporate angular diversity into clinical instruments, OCRT could be widely applied as an enhancement over conventional OCT.