Universal inverse modeling of point spread functions for SMLM localization and microscope characterization.

The point spread function (PSF) of a microscope describes the image of a point emitter. Knowing the accurate PSF model is essential for various imaging tasks, including single-molecule localization, aberration correction and deconvolution. Here we present universal inverse modeling of point spread functions (uiPSF), a toolbox to infer accurate PSF models from microscopy data, using either image stacks of fluorescent beads or directly images of blinking fluorophores, the raw data in single-molecule localization microscopy (SMLM). Our modular framework is applicable to a variety of microscope modalities and the PSF model incorporates system- or sample-specific characteristics, for example, the bead size, field- and depth- dependent aberrations, and transformations among channels. We demonstrate its application in single or multiple channels or large field-of-view SMLM systems, 4Pi-SMLM, and lattice light-sheet microscopes using either bead data or single-molecule blinking data.

Depth-enhanced high-throughput microscopy by compact PSF engineering.

High-throughput microscopy is vital for screening applications, where three-dimensional (3D) cellular models play a key role. However, due to defocus susceptibility, current 3D high-throughput microscopes require axial scanning, which lowers throughput and increases photobleaching and photodamage. Point spread function (PSF) engineering is an optical method that enables various 3D imaging capabilities, yet it has not been implemented in high-throughput microscopy due to the cumbersome optical extension it typically requires. Here we demonstrate compact PSF engineering in the objective lens, which allows us to enhance the imaging depth of field and, combined with deep learning, recover 3D information using single snapshots. Beyond the applications shown here, this work showcases the usefulness of high-throughput microscopy in obtaining training data for deep learning-based algorithms, applicable to a variety of microscopy modalities.

OM2Seq: learning retrieval embeddings for optical genome mapping.

Motivation: Genomics-based diagnostic methods that are quick, precise, and economical are essential for the advancement of precision medicine, with applications spanning the diagnosis of infectious diseases, cancer, and rare diseases. One technology that holds potential in this field is optical genome mapping (OGM), which is capable of detecting structural variations, epigenomic profiling, and microbial species identification. It is based on imaging of linearized DNA molecules that are stained with fluorescent labels, that are then aligned to a reference genome. However, the computational methods currently available for OGM fall short in terms of accuracy and computational speed. Results: This work introduces OM2Seq, a new approach for the rapid and accurate mapping of DNA fragment images to a reference genome. Based on a Transformer-encoder architecture, OM2Seq is trained on acquired OGM data to efficiently encode DNA fragment images and reference genome segments to a common embedding space, which can be indexed and efficiently queried using a vector database. We show that OM2Seq significantly outperforms the baseline methods in both computational speed (by 2 orders of magnitude) and accuracy. Availability and implementation: https://github.com/yevgenin/om2seq.

Large-FOV 3D localization microscopy by spatially variant point spread function generation.

Accurate characterization of the microscopic point spread function (PSF) is crucial for achieving high-performance localization microscopy (LM). Traditionally, LM assumes a spatially invariant PSF to simplify the modeling of the imaging system. However, for large fields of view (FOV) imaging, it becomes important to account for the spatially variant nature of the PSF. Here, we propose an accurate and fast principal components analysis-based field-dependent 3D PSF generator (PPG3D) and localizer for LM. Through simulations and experimental three-dimensional (3D) single-molecule localization microscopy (SMLM), we demonstrate the effectiveness of PPG3D, enabling super-resolution imaging of mitochondria and microtubules with high fidelity over a large FOV. A comparison of PPG3D with a shift-variant PSF generator for 3D LM reveals a threefold improvement in accuracy. Moreover, PPG3D is approximately 100 times faster than existing PSF generators, when used in image plane-based interpolation mode. Given its user-friendliness, we believe that PPG3D holds great potential for widespread application in SMLM and other imaging modalities.

Universal inverse modelling of point spread functions for SMLM localization and microscope characterization.

The point spread function (PSF) of a microscope describes the image of a point emitter. Knowing the accurate PSF model is essential for various imaging tasks, including single molecule localization, aberration correction and deconvolution. Here we present uiPSF (universal inverse modelling of Point Spread Functions), a toolbox to infer accurate PSF models from microscopy data, using either image stacks of fluorescent beads or directly images of blinking fluorophores, the raw data in single molecule localization microscopy (SMLM). The resulting PSF model enables accurate 3D super-resolution imaging using SMLM. Additionally, uiPSF can be used to characterize and optimize a microscope system by quantifying the aberrations, including field-dependent aberrations, and resolutions. Our modular framework is applicable to a variety of microscope modalities and the PSF model incorporates system or sample specific characteristics, e.g., the bead size, depth dependent aberrations and transformations among channels. We demonstrate its application in single or multiple channels or large field-of-view SMLM systems, 4Pi-SMLM, and lattice light-sheet microscopes using either bead data or single molecule blinking data.

Neural network-assisted single-molecule localization microscopy with a weak-affinity protein tag.

Single-molecule localization microscopy achieves nanometer spatial resolution by localizing single fluorophores separated in space and time. A major challenge of single-molecule localization microscopy is the long acquisition time, leading to low throughput, as well as to a poor temporal resolution that limits its use to visualize the dynamics of cellular structures in live cells. Another challenge is photobleaching, which reduces information density over time and limits throughput and the available observation time in live-cell applications. To address both challenges, we combine two concepts: first, we integrate the neural network DeepSTORM to predict super-resolution images from high-density imaging data, which increases acquisition speed. Second, we employ a direct protein label, HaloTag7, in combination with exchangeable ligands (xHTLs), for fluorescence labeling. This labeling method bypasses photobleaching by providing a constant signal over time and is compatible with live-cell imaging. The combination of both a neural network and a weak-affinity protein label reduced the acquisition time up to ∼25-fold. Furthermore, we demonstrate live-cell imaging with increased temporal resolution, and capture the dynamics of the endoplasmic reticulum over extended time without signal loss.

Near index matching enables solid diffractive optical element fabrication via additive manufacturing.

Diffractive optical elements (DOEs) have a wide range of applications in optics and photonics, thanks to their capability to perform complex wavefront shaping in a compact form. However, widespread applicability of DOEs is still limited, because existing fabrication methods are cumbersome and expensive. Here, we present a simple and cost-effective fabrication approach for solid, high-performance DOEs. The method is based on conjugating two nearly refractive index-matched solidifiable transparent materials. The index matching allows for extreme scaling up of the elements in the axial dimension, which enables simple fabrication of a template using commercially available 3D printing at tens-of-micrometer resolution. We demonstrated the approach by fabricating and using DOEs serving as microlens arrays, vortex plates, including for highly sensitive applications such as vector beam generation and super-resolution microscopy using MINSTED, and phase-masks for three-dimensional single-molecule localization microscopy. Beyond the advantage of making DOEs widely accessible by drastically simplifying their production, the method also overcomes difficulties faced by existing methods in fabricating highly complex elements, such as high-order vortex plates, and spectrum-encoding phase masks for microscopy.

Design of optimal labeling patterns for optical genome mapping via information theory.

MOTIVATION: Optical genome mapping (OGM) is a technique that extracts partial genomic information from optically imaged and linearized DNA fragments containing fluorescently labeled short sequence patterns. This information can be used for various genomic analyses and applications, such as the detection of structural variations and copy-number variations, epigenomic profiling, and microbial species identification. Currently, the choice of labeled patterns is based on the available biochemical methods and is not necessarily optimized for the application. RESULTS: In this work, we develop a model of OGM based on information theory, which enables the design of optimal labeling patterns for specific applications and target organism genomes. We validated the model through experimental OGM on human DNA and simulations on bacterial DNA. Our model predicts up to 10-fold improved accuracy by optimal choice of labeling patterns, which may guide future development of OGM biochemical labeling methods and significantly improve its accuracy and yield for applications such as epigenomic profiling and cultivation-free pathogen identification in clinical samples. AVAILABILITY AND IMPLEMENTATION: https://github.com/yevgenin/PatternCode.

DBlink: dynamic localization microscopy in super spatiotemporal resolution via deep learning.

Single-molecule localization microscopy (SMLM) has revolutionized biological imaging, improving the spatial resolution of traditional microscopes by an order of magnitude. However, SMLM techniques require long acquisition times, typically a few minutes, to yield a single super-resolved image, because they depend on accumulation of many localizations over thousands of recorded frames. Hence, the capability of SMLM to observe dynamics at high temporal resolution has always been limited. In this work, we present DBlink, a deep-learning-based method for super spatiotemporal resolution reconstruction from SMLM data. The input to DBlink is a recorded video of SMLM data and the output is a super spatiotemporal resolution video reconstruction. We use a convolutional neural network combined with a bidirectional long short-term memory network architecture, designed for capturing long-term dependencies between different input frames. We demonstrate DBlink performance on simulated filaments and mitochondria-like structures, on experimental SMLM data under controlled motion conditions and on live-cell dynamic SMLM. DBlink's spatiotemporal interpolation constitutes an important advance in super-resolution imaging of dynamic processes in live cells.

Fluorescence Microscopy: a statistics-optics perspective.

Fundamental properties of light unavoidably impose features on images collected using fluorescence microscopes. Modeling these features is ever more important in quantitatively interpreting microscopy images collected at scales on par or smaller than light's wavelength. Here we review the optics responsible for generating fluorescent images, fluorophore properties, microscopy modalities leveraging properties of both light and fluorophores, in addition to the necessarily probabilistic modeling tools imposed by the stochastic nature of light and measurement.

Roadmap on Deep Learning for Microscopy.

Through digital imaging, microscopy has evolved from primarily being a means for visual observation of life at the micro- and nano-scale, to a quantitative tool with ever-increasing resolution and throughput. Artificial intelligence, deep neural networks, and machine learning are all niche terms describing computational methods that have gained a pivotal role in microscopy-based research over the past decade. This Roadmap is written collectively by prominent researchers and encompasses selected aspects of how machine learning is applied to microscopy image data, with the aim of gaining scientific knowledge by improved image quality, automated detection, segmentation, classification and tracking of objects, and efficient merging of information from multiple imaging modalities. We aim to give the reader an overview of the key developments and an understanding of possibilities and limitations of machine learning for microscopy. It will be of interest to a wide cross-disciplinary audience in the physical sciences and life sciences.

DeepOM: single-molecule optical genome mapping via deep learning.

MOTIVATION: Efficient tapping into genomic information from a single microscopic image of an intact DNA molecule is an outstanding challenge and its solution will open new frontiers in molecular diagnostics. Here, a new computational method for optical genome mapping utilizing deep learning is presented, termed DeepOM. Utilization of a convolutional neural network, trained on simulated images of labeled DNA molecules, improves the success rate in the alignment of DNA images to genomic references. RESULTS: The method is evaluated on acquired images of human DNA molecules stretched in nano-channels. The accuracy of the method is benchmarked against state-of-the-art commercial software Bionano Solve. The results show a significant advantage in alignment success rate for molecules shorter than 50 kb. DeepOM improves the yield, sensitivity, and throughput of optical genome mapping experiments in applications of human genomics and microbiology. AVAILABILITY AND IMPLEMENTATION: The source code for the presented method is publicly available at https://github.com/yevgenin/DeepOM.

Diffractive optical system design by cascaded propagation.

Modern design of complex optical systems relies heavily on computational tools. These frequently use geometrical optics as well as Fourier optics. Fourier optics is typically used for designing thin diffractive elements, placed in the system's aperture, generating a shift-invariant Point Spread Function (PSF). A major bottleneck in applying Fourier Optics in many cases of interest, e.g. when dealing with multiple, or out-of-aperture elements, comes from numerical complexity. In this work, we propose and implement an efficient and differentiable propagation model based on the Collins integral, which enables the optimization of diffractive optical systems with unprecedented design freedom using backpropagation. We demonstrate the applicability of our method, numerically and experimentally, by engineering shift-variant PSFs via thin plate elements placed in arbitrary planes inside complex imaging systems, performing cascaded optimization of multiple planes, and designing optimal machine-vision systems by deep learning.

Monocular kilometer-scale passive ranging by point-spread function engineering.

Standard imaging systems are designed for 2D representation of objects, while information about the third dimension remains implicit, as imaging-based distance estimation is a difficult challenge. Existing long-range distance estimation technologies mostly rely on active emission of signal, which as a subsystem, constitutes a significant portion of the complexity, size and cost of the active-ranging apparatus. Despite the appeal of alleviating the requirement for signal-emission, passive distance estimation methods are essentially nonexistent for ranges greater than a few hundreds of meters. Here, we present monocular long-range, telescope-based passive ranging, realized by integration of point-spread-function engineering into a telescope, extending the scale of point-spread-function engineering-based ranging to distances where it has never been tested before. We provide experimental demonstrations of the optical system in a variety of challenging imaging scenarios, including adversarial weather conditions, dynamic targets and scenes of diversified textures, at distances extending beyond 1.7 km. We conclude with brief quantification of the effect of atmospheric turbulence on estimation precision, which becomes a significant error source in long-range optical imaging.

THY1-mediated mechanisms converge to drive YAP activation in skin homeostasis and repair.

Anchored cells of the basal epidermis constantly undergo proliferation in an overcrowded environment. An important regulator of epidermal proliferation is YAP, which can be controlled by both cell-matrix and cell-cell interactions. Here, we report that THY1, a GPI-anchored protein, inhibits epidermal YAP activity through converging molecular mechanisms. THY1 deficiency leads to increased adhesion by activating the integrin-ß1-SRC module. Notably, regardless of high cellular densities, the absence of THY1 leads to the dissociation of an adherens junction complex that enables the release and translocation of YAP. Due to increased YAP-dependent proliferation, Thy1-/- mice display enhanced wound repair and hair follicle regeneration. Taken together, our work reveals THY1 as a crucial regulator of cell-matrix and cell-cell interactions that controls YAP activity in skin homeostasis and regeneration.

Quantifying cell-cycle-dependent chromatin dynamics during interphase by live 3D tracking.

The study of cell cycle progression and regulation is important to our understanding of fundamental biophysics, aging, and disease mechanisms. Local chromatin movements are generally considered to be constrained and relatively consistent during all interphase stages, although recent advances in our understanding of genome organization challenge this claim. Here, we use high spatiotemporal resolution, 4D (x, y, z and time) localization microscopy by point-spread-function (PSF) engineering and deep learning-based image analysis, for live imaging of mouse embryonic fibroblast (MEF 3T3) and MEF 3T3 double Lamin A Knockout (LmnaKO) cell lines, to characterize telomere diffusion during the interphase. We detected varying constraint levels imposed on chromatin, which are prominently decreased during G0/G1. Our 4D measurements of telomere diffusion offer an effective method to investigate chromatin dynamics and reveal cell-cycle-dependent motion constraints, which may be caused by various cellular processes.

Synthetic cells with self-activating optogenetic proteins communicate with natural cells.

Development of regulated cellular processes and signaling methods in synthetic cells is essential for their integration with living materials. Light is an attractive tool to achieve this, but the limited penetration depth into tissue of visible light restricts its usability for in-vivo applications. Here, we describe the design and implementation of bioluminescent intercellular and intracellular signaling mechanisms in synthetic cells, dismissing the need for an external light source. First, we engineer light generating SCs with an optimized lipid membrane and internal composition, to maximize luciferase expression levels and enable high-intensity emission. Next, we show these cells' capacity to trigger bioprocesses in natural cells by initiating asexual sporulation of dark-grown mycelial cells of the fungus Trichoderma atroviride. Finally, we demonstrate regulated transcription and membrane recruitment in synthetic cells using bioluminescent intracellular signaling with self-activating fusion proteins. These functionalities pave the way for deploying synthetic cells as embeddable microscale light sources that are capable of controlling engineered processes inside tissues.

High-Throughput Imaging of CRISPR- and Recombinant Adeno-Associated Virus-Induced DNA Damage Response in Human Hematopoietic Stem and Progenitor Cells.

CRISPR-Cas technology has revolutionized gene editing, but concerns remain due to its propensity for off-target interactions. This, combined with genotoxicity related to both CRISPR-Cas9-induced double-strand breaks and transgene delivery, poses a significant liability for clinical genome-editing applications. Current best practice is to optimize genome-editing parameters in preclinical studies. However, quantitative tools that measure off-target interactions and genotoxicity are costly and time-consuming, limiting the practicality of screening large numbers of potential genome-editing reagents and conditions. Here, we show that flow-based imaging facilitates DNA damage characterization of hundreds of human hematopoietic stem and progenitor cells per minute after treatment with CRISPR-Cas9 and recombinant adeno-associated virus serotype 6. With our web-based platform that leverages deep learning for image analysis, we find that greater DNA damage response is observed for guide RNAs with higher genome-editing activity, differentiating even single on-target guide RNAs with different levels of off-target interactions. This work simplifies the characterization and screening process of genome-editing parameters toward enabling safer and more effective gene-therapy applications.

Deep-ROCS: from speckle patterns to superior-resolved images by deep learning in rotating coherent scattering microscopy.

Rotating coherent scattering (ROCS) microscopy is a label-free imaging technique that overcomes the optical diffraction limit by adding up the scattered laser light from a sample obliquely illuminated from different angles. Although ROCS imaging achieves 150 nm spatial and 10 ms temporal resolution, simply summing different speckle patterns may cause loss of sample information. In this paper we present Deep-ROCS, a neural network-based technique that generates a superior-resolved image by efficient numerical combination of a set of differently illuminated images. We show that Deep-ROCS can reconstruct super-resolved images more accurately than conventional ROCS microscopy, retrieving high-frequency information from a small number (6) of speckle images. We demonstrate the performance of Deep-ROCS experimentally on 200 nm beads and by computer simulations, where we show its potential for even more complex structures such as a filament network.