DBlink: dynamic localization microscopy in super spatiotemporal resolution via deep learning.

Single-molecule localization microscopy (SMLM) has revolutionized biological imaging, improving the spatial resolution of traditional microscopes by an order of magnitude. However, SMLM techniques require long acquisition times, typically a few minutes, to yield a single super-resolved image, because they depend on accumulation of many localizations over thousands of recorded frames. Hence, the capability of SMLM to observe dynamics at high temporal resolution has always been limited. In this work, we present DBlink, a deep-learning-based method for super spatiotemporal resolution reconstruction from SMLM data. The input to DBlink is a recorded video of SMLM data and the output is a super spatiotemporal resolution video reconstruction. We use a convolutional neural network combined with a bidirectional long short-term memory network architecture, designed for capturing long-term dependencies between different input frames. We demonstrate DBlink performance on simulated filaments and mitochondria-like structures, on experimental SMLM data under controlled motion conditions and on live-cell dynamic SMLM. DBlink's spatiotemporal interpolation constitutes an important advance in super-resolution imaging of dynamic processes in live cells.

Diffractive optical system design by cascaded propagation.

Modern design of complex optical systems relies heavily on computational tools. These frequently use geometrical optics as well as Fourier optics. Fourier optics is typically used for designing thin diffractive elements, placed in the system's aperture, generating a shift-invariant Point Spread Function (PSF). A major bottleneck in applying Fourier Optics in many cases of interest, e.g. when dealing with multiple, or out-of-aperture elements, comes from numerical complexity. In this work, we propose and implement an efficient and differentiable propagation model based on the Collins integral, which enables the optimization of diffractive optical systems with unprecedented design freedom using backpropagation. We demonstrate the applicability of our method, numerically and experimentally, by engineering shift-variant PSFs via thin plate elements placed in arbitrary planes inside complex imaging systems, performing cascaded optimization of multiple planes, and designing optimal machine-vision systems by deep learning.

Deep-ROCS: from speckle patterns to superior-resolved images by deep learning in rotating coherent scattering microscopy.

Rotating coherent scattering (ROCS) microscopy is a label-free imaging technique that overcomes the optical diffraction limit by adding up the scattered laser light from a sample obliquely illuminated from different angles. Although ROCS imaging achieves 150 nm spatial and 10 ms temporal resolution, simply summing different speckle patterns may cause loss of sample information. In this paper we present Deep-ROCS, a neural network-based technique that generates a superior-resolved image by efficient numerical combination of a set of differently illuminated images. We show that Deep-ROCS can reconstruct super-resolved images more accurately than conventional ROCS microscopy, retrieving high-frequency information from a small number (6) of speckle images. We demonstrate the performance of Deep-ROCS experimentally on 200 nm beads and by computer simulations, where we show its potential for even more complex structures such as a filament network.

Neural network-assisted single-molecule localization microscopy with a weak-affinity protein tag.

Single-molecule localization microscopy achieves nanometer spatial resolution by localizing single fluorophores separated in space and time. A major challenge of single-molecule localization microscopy is the long acquisition time, leading to low throughput, as well as to a poor temporal resolution that limits its use to visualize the dynamics of cellular structures in live cells. Another challenge is photobleaching, which reduces information density over time and limits throughput and the available observation time in live-cell applications. To address both challenges, we combine two concepts: first, we integrate the neural network DeepSTORM to predict super-resolution images from high-density imaging data, which increases acquisition speed. Second, we employ a direct protein label, HaloTag7, in combination with exchangeable ligands (xHTLs), for fluorescence labeling. This labeling method bypasses photobleaching by providing a constant signal over time and is compatible with live-cell imaging. The combination of both a neural network and a weak-affinity protein label reduced the acquisition time up to ∼25-fold. Furthermore, we demonstrate live-cell imaging with increased temporal resolution, and capture the dynamics of the endoplasmic reticulum over extended time without signal loss.

High-Throughput Imaging of CRISPR- and Recombinant Adeno-Associated Virus-Induced DNA Damage Response in Human Hematopoietic Stem and Progenitor Cells.

CRISPR-Cas technology has revolutionized gene editing, but concerns remain due to its propensity for off-target interactions. This, combined with genotoxicity related to both CRISPR-Cas9-induced double-strand breaks and transgene delivery, poses a significant liability for clinical genome-editing applications. Current best practice is to optimize genome-editing parameters in preclinical studies. However, quantitative tools that measure off-target interactions and genotoxicity are costly and time-consuming, limiting the practicality of screening large numbers of potential genome-editing reagents and conditions. Here, we show that flow-based imaging facilitates DNA damage characterization of hundreds of human hematopoietic stem and progenitor cells per minute after treatment with CRISPR-Cas9 and recombinant adeno-associated virus serotype 6. With our web-based platform that leverages deep learning for image analysis, we find that greater DNA damage response is observed for guide RNAs with higher genome-editing activity, differentiating even single on-target guide RNAs with different levels of off-target interactions. This work simplifies the characterization and screening process of genome-editing parameters toward enabling safer and more effective gene-therapy applications.

Automated Analysis of Fluorescence Kinetics in Single-Molecule Localization Microscopy Data Reveals Protein Stoichiometry.

Understanding the function of protein complexes requires information on their molecular organization, specifically, their oligomerization level. Optical super-resolution microscopy can localize single protein complexes in cells with high precision, however, the quantification of their oligomerization level, remains a challenge. Here, we present a Quantitative Algorithm for Fluorescent Kinetics Analysis (QAFKA), that serves as a fully automated workflow for quantitative analysis of single-molecule localization microscopy (SMLM) data by extracting fluorophore "blinking" events. QAFKA includes an automated localization algorithm, the extraction of emission features per localization cluster, and a deep neural network-based estimator that reports the ratios of cluster types within the population. We demonstrate molecular quantification of protein monomers and dimers on simulated and experimental SMLM data. We further demonstrate that QAFKA accurately reports quantitative information on the monomer/dimer equilibrium of membrane receptors in single immobilized cells, opening the door to single-cell single-protein analysis.