Multiline orthogonal scanning temporal focusing (mosTF) microscopy for scattering reduction in in vivo brain imaging.

Temporal focusing two-photon microscopy has been utilized for high-resolution imaging of neuronal and synaptic structures across volumes spanning hundreds of microns in vivo. However, a limitation of temporal focusing is the rapid degradation of the signal-to-background ratio and resolution with increasing imaging depth. This degradation is due to scattered emission photons being widely distributed, resulting in a strong background. To overcome this challenge, we have developed multiline orthogonal scanning temporal focusing (mosTF) microscopy. mosTF captures a sequence of images at each scan location of the excitation line. A reconstruction algorithm then reassigns scattered photons back to their correct scan positions. We demonstrate the effectiveness of mosTF by acquiring neuronal images of mice in vivo. Our results show remarkable improvements in in vivo brain imaging with mosTF, while maintaining its speed advantage.

QuATON: Quantization Aware Training of Optical Neurons.

Optical processors, built with "optical neurons", can efficiently perform high-dimensional linear operations at the speed of light. Thus they are a promising avenue to accelerate large-scale linear computations. With the current advances in micro-fabrication, such optical processors can now be 3D fabricated, but with a limited precision. This limitation translates to quantization of learnable parameters in optical neurons, and should be handled during the design of the optical processor in order to avoid a model mismatch. Specifically, optical neurons should be trained or designed within the physical-constraints at a predefined quantized precision level. To address this critical issues we propose a physics-informed quantization-aware training framework. Our approach accounts for physical constraints during the training process, leading to robust designs. We demonstrate that our approach can design state of the art optical processors using diffractive networks for multiple physics based tasks despite quantized learnable parameters. We thus lay the foundation upon which improved optical processors may be 3D fabricated in the future.

Towards ultrafast quantitative phase imaging via differentiable microscopy [Invited].

With applications ranging from metabolomics to histopathology, quantitative phase microscopy (QPM) is a powerful label-free imaging modality. Despite significant advances in fast multiplexed imaging sensors and deep-learning-based inverse solvers, the throughput of QPM is currently limited by the pixel-rate of the image sensors. Complementarily, to improve throughput further, here we propose to acquire images in a compressed form so that more information can be transferred beyond the existing hardware bottleneck of the image sensor. To this end, we present a numerical simulation of a learnable optical compression-decompression framework that learns content-specific features. The proposed differentiable quantitative phase microscopy (∂-QPM) first uses learnable optical processors as image compressors. The intensity representations produced by these optical processors are then captured by the imaging sensor. Finally, a reconstruction network running on a computer decompresses the QPM images post aquisition. In numerical experiments, the proposed system achieves compression of × 64 while maintaining the SSIM of ∼0.90 and PSNR of ∼30 dB on cells. The results demonstrated by our experiments open up a new pathway to QPM systems that may provide unprecedented throughput improvements.

Multiline Orthogonal Scanning Temporal Focusing (mosTF) Microscopy for Scattering Reduction in High-speed in vivo Brain Imaging.

Temporal focusing two-photon microscopy enables high resolution imaging of fine structures in vivo over a large volume. A limitation of temporal focusing is that signal-to-background ratio and resolution degrade rapidly with increasing imaging depth. This degradation originates from the scattered emission photons are widely distributed resulting in a strong background. We have developed Multiline Orthogonal Scanning Temporal Focusing (mosTF) microscopy that overcomes this problem. mosTF captures a sequence of images at each scan location of the excitation line, followed by a reconstruction algorithm reassigns scattered photons back to the correct scan position. We demonstrate mosTF by acquiring mice neuronal images in vivo. Our results show remarkably improvements with mosTF for in vivo brain imaging while maintaining its speed advantage.

DEEP-squared: deep learning powered De-scattering with Excitation Patterning.

Limited throughput is a key challenge in in vivo deep tissue imaging using nonlinear optical microscopy. Point scanning multiphoton microscopy, the current gold standard, is slow especially compared to the widefield imaging modalities used for optically cleared or thin specimens. We recently introduced "De-scattering with Excitation Patterning" or "DEEP" as a widefield alternative to point-scanning geometries. Using patterned multiphoton excitation, DEEP encodes spatial information inside tissue before scattering. However, to de-scatter at typical depths, hundreds of such patterned excitations were needed. In this work, we present DEEP2, a deep learning-based model that can de-scatter images from just tens of patterned excitations instead of hundreds. Consequently, we improve DEEP's throughput by almost an order of magnitude. We demonstrate our method in multiple numerical and experimental imaging studies, including in vivo cortical vasculature imaging up to 4 scattering lengths deep in live mice.

De-scattering Deep Neural Network Enables Fast Imaging of Spines through Scattering Media by Temporal Focusing Microscopy.

Today the gold standard for in vivo imaging through scattering tissue is point-scanning two-photon microscopy (PSTPM). Especially in neuroscience, PSTPM is widely used for deep-tissue imaging in the brain. However, due to sequential scanning, PSTPM is slow. Temporal focusing microscopy (TFM), on the other hand, focuses femtosecond pulsed laser light temporally while keeping wide-field illumination, and is consequently much faster. However, due to the use of a camera detector, TFM suffers from the scattering of emission photons. As a result, TFM produces images of poor quality, obscuring fluorescent signals from small structures such as dendritic spines. In this work, we present a de-scattering deep neural network (DeScatterNet) to improve the quality of TFM images. Using a 3D convolutional neural network (CNN) we build a map from TFM to PSTPM modalities, to enable fast TFM imaging while maintaining high image quality through scattering media. We demonstrate this approach for in vivo imaging of dendritic spines on pyramidal neurons in the mouse visual cortex. We quantitatively show that our trained network rapidly outputs images that recover biologically relevant features previously buried in the scattered fluorescence in the TFM images. In vivo imaging that combines TFM and the proposed neural network is one to two orders of magnitude faster than PSTPM but retains the high quality necessary to analyze small fluorescent structures. The proposed approach could also be beneficial for improving the performance of many speed-demanding deep-tissue imaging applications, such as in vivo voltage imaging.

SERS Nanowire Chip and Machine Learning-Enabled Classification of Wild-Type and Antibiotic-Resistant Bacteria at Species and Strain Levels.

Antimicrobial resistance (AMR) is a major health threat worldwide and the culture-based bacterial detection methods are slow. Surface-enhanced Raman spectroscopy (SERS) can be used to identify target analytes in real time with sensitivity down to the single-molecule level, providing a promising solution for the culture-free bacterial detection. We report the fabrication of SERS substrates having tightly packed silver (Ag) nanoparticles loaded onto long silicon nanowires (Si NWs) grown by the metal-assisted chemical etching (MACE) method for the detection of bacteria. The optimized SERS chips exhibited sensitivity down to 10-12 M concentration of R6G molecules and detected reproducible Raman spectra of bacteria down to a concentration of 100 colony forming units (CFU)/mL, which is a thousand times lower than the clinical threshold of bacterial infections like UTI (105 CFU/mL). A Siamese neural network model was used to classify SERS spectra from bacteria specimens. The trained model identified 12 different bacterial species, including those which are causative agents for tuberculosis and urinary tract infection (UTI). Next, the SERS chips and another Siamese neural network model were used to differentiate AMR strains from susceptible strains of Escherichia coli (E. coli). The enhancement offered by SERS chip-enabled acquisitions of Raman spectra of bacteria directly in the synthetic urine by spiking the sample with only 103 CFU/mL E. coli. Thus, the present study lays the ground for the identification and quantification of bacteria on SERS chips, thereby offering a potential future use for rapid, reproducible, label-free, and low limit detection of clinical pathogens.

Highly sensitive quantitative phase microscopy and deep learning aided with whole genome sequencing for rapid detection of infection and antimicrobial resistance.

Current state-of-the-art infection and antimicrobial resistance (AMR) diagnostics are based on culture-based methods with a detection time of 48-96 h. Therefore, it is essential to develop novel methods that can do real-time diagnoses. Here, we demonstrate that the complimentary use of label-free optical assay with whole-genome sequencing (WGS) can enable rapid diagnosis of infection and AMR. Our assay is based on microscopy methods exploiting label-free, highly sensitive quantitative phase microscopy (QPM) followed by deep convolutional neural networks-based classification. The workflow was benchmarked on 21 clinical isolates from four WHO priority pathogens that were antibiotic susceptibility tested, and their AMR profile was determined by WGS. The proposed optical assay was in good agreement with the WGS characterization. Accurate classification based on the gram staining (100% recall for gram-negative and 83.4% for gram-positive), species (98.6%), and resistant/susceptible type (96.4%), as well as at the individual strain level (100% sensitivity in predicting 19 out of the 21 strains, with an overall accuracy of 95.45%). The results from this initial proof-of-concept study demonstrate the potential of the QPM assay as a rapid and first-stage tool for species, strain-level classification, and the presence or absence of AMR, which WGS can follow up for confirmation. Overall, a combined workflow with QPM and WGS complemented with deep learning data analyses could, in the future, be transformative for detecting and identifying pathogens and characterization of the AMR profile and antibiotic susceptibility.

De-scattering with Excitation Patterning enables rapid wide-field imaging through scattering media.

Nonlinear optical microscopy has enabled in vivo deep tissue imaging on the millimeter scale. A key unmet challenge is its limited throughput especially compared to rapid wide-field modalities that are used ubiquitously in thin specimens. Wide-field imaging methods in tissue specimens have found successes in optically cleared tissues and at shallower depths, but the scattering of emission photons in thick turbid samples severely degrades image quality at the camera. To address this challenge, we introduce a novel technique called De-scattering with Excitation Patterning or "DEEP," which uses patterned nonlinear excitation followed by computational imaging-assisted wide-field detection. Multiphoton temporal focusing allows high-resolution excitation patterns to be projected deep inside specimen at multiple scattering lengths due to the use of long wavelength light. Computational reconstruction allows high-resolution structural features to be reconstructed from tens to hundreds of DEEP images instead of millions of point-scanning measurements.

Excision of mutagenic replication-blocking lesions suppresses cancer but promotes cytotoxicity and lethality in nitrosamine-exposed mice.

N-Nitrosodimethylamine (NDMA) is a DNA-methylating agent that has been discovered to contaminate water, food, and drugs. The alkyladenine DNA glycosylase (AAG) removes methylated bases to initiate the base excision repair (BER) pathway. To understand how gene-environment interactions impact disease susceptibility, we study Aag-knockout (Aag-/-) and Aag-overexpressing mice that harbor increased levels of either replication-blocking lesions (3-methyladenine [3MeA]) or strand breaks (BER intermediates), respectively. Remarkably, the disease outcome switches from cancer to lethality simply by changing AAG levels. To understand the underlying basis for this observation, we integrate a suite of molecular, cellular, and physiological analyses. We find that unrepaired 3MeA is somewhat toxic, but highly mutagenic (promoting cancer), whereas excess strand breaks are poorly mutagenic and highly toxic (suppressing cancer and promoting lethality). We demonstrate that the levels of a single DNA repair protein tip the balance between blocks and breaks and thus dictate the disease consequences of DNA damage.

Analysis of mutations in tumor and normal adjacent tissue via fluorescence detection.

Inflammation is a major risk factor for many types of cancer, including colorectal. There are two fundamentally different mechanisms by which inflammation can contribute to carcinogenesis. First, reactive oxygen and nitrogen species (RONS) can damage DNA to cause mutations that initiate cancer. Second, inflammatory cytokines and chemokines promote proliferation, migration, and invasion. Although it is known that inflammation-associated RONS can be mutagenic, the extent to which they induce mutations in intestinal stem cells has been little explored. Furthermore, it is now widely accepted that cancer is caused by successive rounds of clonal expansion with associated de novo mutations that further promote tumor development. As such, we aimed to understand the extent to which inflammation promotes clonal expansion in normal and tumor tissue. Using an engineered mouse model that is prone to cancer and within which mutant cells fluoresce, here we have explored the impact of inflammation on de novo mutagenesis and clonal expansion in normal and tumor tissue. While inflammation is strongly associated with susceptibility to cancer and a concomitant increase in the overall proportion of mutant cells in the tissue, we did not observe an increase in mutations in normal adjacent tissue. These results are consistent with opportunities for de novo mutations and clonal expansion during tumor growth, and they suggest protective mechanisms that suppress the risk of inflammation-induced accumulation of mutant cells in normal tissue.

Advances in Chromatin and Chromosome Research: Perspectives from Multiple Fields.

Nucleosomes package genomic DNA into chromatin. By regulating DNA access for transcription, replication, DNA repair, and epigenetic modification, chromatin forms the nexus of most nuclear processes. In addition, dynamic organization of chromatin underlies both regulation of gene expression and evolution of chromosomes into individualized sister objects, which can segregate cleanly to different daughter cells at anaphase. This collaborative review shines a spotlight on technologies that will be crucial to interrogate key questions in chromatin and chromosome biology including state-of-the-art microscopy techniques, tools to physically manipulate chromatin, single-cell methods to measure chromatin accessibility, computational imaging with neural networks and analytical tools to interpret chromatin structure and dynamics. In addition, this review provides perspectives on how these tools can be applied to specific research fields such as genome stability and developmental biology and to test concepts such as phase separation of chromatin.

Scattering reduction by structured light illumination in line-scanning temporal focusing microscopy.

Line-scanning temporal focusing microscopy (LineTFM) is capable of imaging biological samples more than 10 times faster than two-photon laser point-scanning microscopy (TPLSM), while achieving nearly the same lateral and axial spatial resolution. However, the image contrast taken by LineTFM is lower than that by TPLSM because LineTFM is severely influenced by biological tissue scattering. To reject the scattered photons, we implemented LineTFM using both structured illumination and uniform illumination combined with the HiLo post-processing algorithm, called HiLL microscopy (HiLo-Line-scanning temporal focusing microscopy). HiLL microscopy significantly reduces tissue scattering and improves image contrast. We demonstrate HiLL microscopy with in vivo brain imaging. This approach could potentially find applications in monitoring fast dynamic events and in mapping high resolution structures over a large volume.

Automated fluorescence intensity and gradient analysis enables detection of rare fluorescent mutant cells deep within the tissue of RaDR mice.

Homologous recombination (HR) events are key drivers of cancer-promoting mutations, and the ability to visualize these events in situ provides important information regarding mutant cell type, location, and clonal expansion. We have previously created the Rosa26 Direct Repeat (RaDR) mouse model wherein HR at an integrated substrate gives rise to a fluorescent cell. To fully leverage this in situ approach, we need better ways to quantify rare fluorescent cells deep within tissues. Here, we present a robust, automated event quantification algorithm that uses image intensity and gradient features to detect fluorescent cells in deep tissue specimens. To analyze the performance of our algorithm, we simulate fluorescence behavior in tissue using Monte Carlo methods. Importantly, this approach reduces the potential for bias in manual counting and enables quantification of samples with highly dense HR events. Using this approach, we measured the relative frequency of HR within a chromosome and between chromosomes and found that HR within a chromosome is more frequent, which is consistent with the close proximity of sister chromatids. Our approach is both objective and highly rapid, providing a powerful tool, not only to researchers interested in HR, but also to many other researchers who are similarly using fluorescence as a marker for understanding mammalian biology in tissues.

Near-common-path interferometer for imaging Fourier-transform spectroscopy in wide-field microscopy.

Imaging Fourier-transform spectroscopy (IFTS) is a powerful method for biological hyperspectral analysis based on various imaging modalities, such as fluorescence or Raman. Since the measurements are taken in the Fourier space of the spectrum, it can also take advantage of compressed sensing strategies. IFTS has been readily implemented in high-throughput, high-content microscope systems based on wide-field imaging modalities. However, there are limitations in existing wide-field IFTS designs. Non-common-path approaches are less phase-stable. Alternatively, designs based on the common-path Sagnac interferometer are stable, but incompatible with high-throughput imaging. They require exhaustive sequential scanning over large interferometric path delays, making compressive strategic data acquisition impossible. In this paper, we present a novel phase-stable, near-common-path interferometer enabling high-throughput hyperspectral imaging based on strategic data acquisition. Our results suggest that this approach can improve throughput over those of many other wide-field spectral techniques by more than an order of magnitude without compromising phase stability.

Three-dimensional image cytometer based on widefield structured light microscopy and high-speed remote depth scanning.

A high throughput 3D image cytometer have been developed that improves imaging speed by an order of magnitude over current technologies. This imaging speed improvement was realized by combining several key components. First, a depth-resolved image can be rapidly generated using a structured light reconstruction algorithm that requires only two wide field images, one with uniform illumination and the other with structured illumination. Second, depth scanning is implemented using the high speed remote depth scanning. Finally, the large field of view, high NA objective lens and the high pixelation, high frame rate sCMOS camera enable high resolution, high sensitivity imaging of a large cell population. This system can image at 800 cell/sec in 3D at submicron resolution corresponding to imaging 1 million cells in 20 min. The statistical accuracy of this instrument is verified by quantitatively measuring rare cell populations with ratio ranging from 1:1 to 1:10(5) . © 2014 International Society for Advancement of Cytometry.

Depth resolved hyperspectral imaging spectrometer based on structured light illumination and Fourier transform interferometry.

A depth resolved hyperspectral imaging spectrometer can provide depth resolved imaging both in the spatial and the spectral domain. Images acquired through a standard imaging Fourier transform spectrometer do not have the depth-resolution. By post processing the spectral cubes (x, y, λ) obtained through a Sagnac interferometer under uniform illumination and structured illumination, spectrally resolved images with depth resolution can be recovered using structured light illumination algorithms such as the HiLo method. The proposed scheme is validated with in vitro specimens including fluorescent solution and fluorescent beads with known spectra. The system is further demonstrated in quantifying spectra from 3D resolved features in biological specimens. The system has demonstrated depth resolution of 1.8 µm and spectral resolution of 7 nm respectively.

Rosa26-GFP direct repeat (RaDR-GFP) mice reveal tissue- and age-dependence of homologous recombination in mammals in vivo.

Homologous recombination (HR) is critical for the repair of double strand breaks and broken replication forks. Although HR is mostly error free, inherent or environmental conditions that either suppress or induce HR cause genomic instability. Despite its importance in carcinogenesis, due to limitations in our ability to detect HR in vivo, little is known about HR in mammalian tissues. Here, we describe a mouse model in which a direct repeat HR substrate is targeted to the ubiquitously expressed Rosa26 locus. In the Rosa26 Direct Repeat-GFP (RaDR-GFP) mice, HR between two truncated EGFP expression cassettes can yield a fluorescent signal. In-house image analysis software provides a rapid method for quantifying recombination events within intact tissues, and the frequency of recombinant cells can be evaluated by flow cytometry. A comparison among 11 tissues shows that the frequency of recombinant cells varies by more than two orders of magnitude among tissues, wherein HR in the brain is the lowest. Additionally, de novo recombination events accumulate with age in the colon, showing that this mouse model can be used to study the impact of chronic exposures on genomic stability. Exposure to N-methyl-N-nitrosourea, an alkylating agent similar to the cancer chemotherapeutic temozolomide, shows that the colon, liver and pancreas are susceptible to DNA damage-induced HR. Finally, histological analysis of the underlying cell types reveals that pancreatic acinar cells and liver hepatocytes undergo HR and also that HR can be specifically detected in colonic somatic stem cells. Taken together, the RaDR-GFP mouse model provides new understanding of how tissue and age impact susceptibility to HR, and enables future studies of genetic, environmental and physiological factors that modulate HR in mammals.