Content-aware frame interpolation (CAFI): deep learning-based temporal super-resolution for fast bioimaging.

The development of high-resolution microscopes has made it possible to investigate cellular processes in 3D and over time. However, observing fast cellular dynamics remains challenging because of photobleaching and phototoxicity. Here we report the implementation of two content-aware frame interpolation (CAFI) deep learning networks, Zooming SlowMo and Depth-Aware Video Frame Interpolation, that are highly suited for accurately predicting images in between image pairs, therefore improving the temporal resolution of image series post-acquisition. We show that CAFI is capable of understanding the motion context of biological structures and can perform better than standard interpolation methods. We benchmark CAFI's performance on 12 different datasets, obtained from four different microscopy modalities, and demonstrate its capabilities for single-particle tracking and nuclear segmentation. CAFI potentially allows for reduced light exposure and phototoxicity on the sample for improved long-term live-cell imaging. The models and the training and testing data are available via the ZeroCostDL4Mic platform.

High-fidelity 3D live-cell nanoscopy through data-driven enhanced super-resolution radial fluctuation.

Live-cell super-resolution microscopy enables the imaging of biological structure dynamics below the diffraction limit. Here we present enhanced super-resolution radial fluctuations (eSRRF), substantially improving image fidelity and resolution compared to the original SRRF method. eSRRF incorporates automated parameter optimization based on the data itself, giving insight into the trade-off between resolution and fidelity. We demonstrate eSRRF across a range of imaging modalities and biological systems. Notably, we extend eSRRF to three dimensions by combining it with multifocus microscopy. This realizes live-cell volumetric super-resolution imaging with an acquisition speed of ~1 volume per second. eSRRF provides an accessible super-resolution approach, maximizing information extraction across varied experimental conditions while minimizing artifacts. Its optimal parameter prediction strategy is generalizable, moving toward unbiased and optimized analyses in super-resolution microscopy.

Fast4DReg - fast registration of 4D microscopy datasets.

Unwanted sample drift is a common issue that plagues microscopy experiments, preventing accurate temporal visualization and quantification of biological processes. Although multiple methods and tools exist to correct images post acquisition, performing drift correction of three-dimensional (3D) videos using open-source solutions remains challenging and time consuming. Here, we present a new tool developed for ImageJ or Fiji called Fast4DReg that can quickly correct axial and lateral drift in 3D video-microscopy datasets. Fast4DReg works by creating intensity projections along multiple axes and estimating the drift between frames using two-dimensional cross-correlations. Using synthetic and acquired datasets, we demonstrate that Fast4DReg can perform better than other state-of-the-art open-source drift-correction tools and significantly outperforms them in speed. We also demonstrate that Fast4DReg can be used to register misaligned channels in 3D using either calibration slides or misaligned images directly. Altogether, Fast4DReg provides a quick and easy-to-use method to correct 3D imaging data before further visualization and analysis.

TrackMate 7: integrating state-of-the-art segmentation algorithms into tracking pipelines.

TrackMate is an automated tracking software used to analyze bioimages and is distributed as a Fiji plugin. Here, we introduce a new version of TrackMate. TrackMate 7 is built to address the broad spectrum of modern challenges researchers face by integrating state-of-the-art segmentation algorithms into tracking pipelines. We illustrate qualitatively and quantitatively that these new capabilities function effectively across a wide range of bio-imaging experiments.

A method for the fast and photon-efficient analysis of time-domain fluorescence lifetime image data over large dynamic ranges.

Fluorescence lifetime imaging (FLIM) allows the quantification of sub-cellular processes in situ, in living cells. A number of approaches have been developed to extract the lifetime from time-domain FLIM data, but they are often limited in terms of speed, photon efficiency, precision or the dynamic range of lifetimes they can measure. Here, we focus on one of the best performing methods in the field, the centre-of-mass method (CMM), that conveys advantages in terms of speed and photon efficiency over others. In this paper, however, we identify a loss of photon efficiency of CMM for short lifetimes when background noise is present. We subsequently present a new development and generalization of CMM that provides for the rapid and accurate extraction of fluorescence lifetime over a large lifetime dynamic range. We provide software tools to simulate, validate and analyse FLIM data sets and compare the performance of our approach against the standard CMM and the commonly employed least-square minimization (LSM) methods. Our method features a better photon efficiency than standard CMM and LSM and is robust in the presence of background noise. The algorithm is applicable to any time-domain FLIM data set.

Fluctuation-Based Super-Resolution Traction Force Microscopy.

Cellular mechanics play a crucial role in tissue homeostasis and are often misregulated in disease. Traction force microscopy is one of the key methods that has enabled researchers to study fundamental aspects of mechanobiology; however, traction force microscopy is limited by poor resolution. Here, we propose a simplified protocol and imaging strategy that enhances the output of traction force microscopy by increasing i) achievable bead density and ii) the accuracy of bead tracking. Our approach relies on super-resolution microscopy, enabled by fluorescence fluctuation analysis. Our pipeline can be used on spinning-disk confocal or widefield microscopes and is compatible with available analysis software. In addition, we demonstrate that our workflow can be used to gain biologically relevant information and is suitable for fast long-term live measurement of traction forces even in light-sensitive cells. Finally, using fluctuation-based traction force microscopy, we observe that filopodia align to the force field generated by focal adhesions.

Artificial intelligence for microscopy: what you should know.

Artificial Intelligence based on Deep Learning (DL) is opening new horizons in biomedical research and promises to revolutionize the microscopy field. It is now transitioning from the hands of experts in computer sciences to biomedical researchers. Here, we introduce recent developments in DL applied to microscopy, in a manner accessible to non-experts. We give an overview of its concepts, capabilities and limitations, presenting applications in image segmentation, classification and restoration. We discuss how DL shows an outstanding potential to push the limits of microscopy, enhancing resolution, signal and information content in acquired data. Its pitfalls are discussed, along with the future directions expected in this field.

NanoJ: a high-performance open-source super-resolution microscopy toolbox.

Super-resolution microscopy (SRM) has become essential for the study of nanoscale biological processes. This type of imaging often requires the use of specialised image analysis tools to process a large volume of recorded data and extract quantitative information. In recent years, our team has built an open-source image analysis framework for SRM designed to combine high performance and ease of use. We named it NanoJ-a reference to the popular ImageJ software it was developed for. In this paper, we highlight the current capabilities of NanoJ for several essential processing steps: spatio-temporal alignment of raw data (NanoJ-Core), super-resolution image reconstruction (NanoJ-SRRF), image quality assessment (NanoJ-SQUIRREL), structural modelling (NanoJ-VirusMapper) and control of the sample environment (NanoJ-Fluidics). We expect to expand NanoJ in the future through the development of new tools designed to improve quantitative data analysis and measure the reliability of fluorescent microscopy studies.

Nanoscopic insights into seeding mechanisms and toxicity of α-synuclein species in neurons.

New strategies for visualizing self-assembly processes at the nanoscale give deep insights into the molecular origins of disease. An example is the self-assembly of misfolded proteins into amyloid fibrils, which is related to a range of neurodegenerative disorders, such as Parkinson's and Alzheimer's diseases. Here, we probe the links between the mechanism of α-synuclein (AS) aggregation and its associated toxicity by using optical nanoscopy directly in a neuronal cell culture model of Parkinson's disease. Using superresolution microscopy, we show that protein fibrils are taken up by neuronal cells and act as prion-like seeds for elongation reactions that both consume endogenous AS and suppress its de novo aggregation. When AS is internalized in its monomeric form, however, it nucleates and triggers the aggregation of endogenous AS, leading to apoptosis, although there are no detectable cross-reactions between externally added and endogenous protein species. Monomer-induced apoptosis can be reduced by pretreatment with seed fibrils, suggesting that partial consumption of the externally added or excess soluble AS can be significantly neuroprotective.

HSV-1 Glycoproteins Are Delivered to Virus Assembly Sites Through Dynamin-Dependent Endocytosis.

Herpes simplex virus-1 (HSV-1) is a large enveloped DNA virus that belongs to the family of Herpesviridae. It has been recently shown that the cytoplasmic membranes that wrap the newly assembled capsids are endocytic compartments derived from the plasma membrane. Here, we show that dynamin-dependent endocytosis plays a major role in this process. Dominant-negative dynamin and clathrin adaptor AP180 significantly decrease virus production. Moreover, inhibitors targeting dynamin and clathrin lead to a decreased transport of glycoproteins to cytoplasmic capsids, confirming that glycoproteins are delivered to assembly sites via endocytosis. We also show that certain combinations of glycoproteins colocalize with each other and with the components of clathrin-dependent and -independent endocytosis pathways. Importantly, we demonstrate that the uptake of neutralizing antibodies that bind to glycoproteins when they become exposed on the cell surface during virus particle assembly leads to the production of non-infectious HSV-1. Our results demonstrate that transport of viral glycoproteins to the plasma membrane prior to endocytosis is the major route by which these proteins are localized to the cytoplasmic virus assembly compartments. This highlights the importance of endocytosis as a major protein-sorting event during HSV-1 envelopment.

Retarded PDI diffusion and a reductive shift in poise of the calcium depleted endoplasmic reticulum.

BACKGROUND: Endoplasmic reticulum (ER) lumenal protein thiol redox balance resists dramatic variation in unfolded protein load imposed by diverse physiological challenges including compromise in the key upstream oxidases. Lumenal calcium depletion, incurred during normal cell signaling, stands out as a notable exception to this resilience, promoting a rapid and reversible shift towards a more reducing poise. Calcium depletion induced ER redox alterations are relevant to physiological conditions associated with calcium signaling, such as the response of pancreatic cells to secretagogues and neuronal activity. The core components of the ER redox machinery are well characterized; however, the molecular basis for the calcium-depletion induced shift in redox balance is presently obscure. RESULTS: In vitro, the core machinery for generating disulfides, consisting of ERO1 and the oxidizing protein disulfide isomerase, PDI1A, was indifferent to variation in calcium concentration within the physiological range. However, ER calcium depletion in vivo led to a selective 2.5-fold decline in PDI1A mobility, whereas the mobility of the reducing PDI family member, ERdj5 was unaffected. In vivo, fluorescence resonance energy transfer measurements revealed that declining PDI1A mobility correlated with formation of a complex with the abundant ER chaperone calreticulin, whose mobility was also inhibited by calcium depletion and the calcium depletion-mediated reductive shift was attenuated in cells lacking calreticulin. Measurements with purified proteins confirmed that the PDI1A-calreticulin complex dissociated as Ca(2+) concentrations approached those normally found in the ER lumen ([Ca(2+)]K(0.5max) = 190 µM). CONCLUSIONS: Our findings suggest that selective sequestration of PDI1A in a calcium depletion-mediated complex with the abundant chaperone calreticulin attenuates the effective concentration of this major lumenal thiol oxidant, providing a plausible and simple mechanism for the observed shift in ER lumenal redox poise upon physiological calcium depletion.

A method to quantify FRET stoichiometry with phasor plot analysis and acceptor lifetime ingrowth.

FRET is widely used for the study of protein-protein interactions in biological samples. However, it is difficult to quantify both the FRET efficiency (E) and the affinity (Kd) of the molecular interaction from intermolecular FRET signals in samples of unknown stoichiometry. Here, we present a method for the simultaneous quantification of the complete set of interaction parameters, including fractions of bound donors and acceptors, local protein concentrations, and dissociation constants, in each image pixel. The method makes use of fluorescence lifetime information from both donor and acceptor molecules and takes advantage of the linear properties of the phasor plot approach. We demonstrate the capability of our method in vitro in a microfluidic device and also in cells, via the determination of the binding affinity between tagged versions of glutathione and glutathione S-transferase, and via the determination of competitor concentration. The potential of the method is explored with simulations.

In Situ Visualization of Block Copolymer Self-Assembly in Organic Media by Super-Resolution Fluorescence Microscopy.

Analytical methods that enable visualization of nanomaterials derived from solution self-assembly processes in organic solvents are highly desirable. Herein, we demonstrate the use of stimulated emission depletion microscopy (STED) and single molecule localization microscopy (SMLM) to map living crystallization-driven block copolymer (BCP) self-assembly in organic media at the sub-diffraction scale. Four different dyes were successfully used for single-colour super-resolution imaging of the BCP nanostructures allowing micelle length distributions to be determined in situ. Dual-colour SMLM imaging was used to measure and compare the rate of addition of red fluorescent BCP to the termini of green fluorescent seed micelles to generate block comicelles. Although well-established for aqueous systems, the results highlight the potential of super-resolution microscopy techniques for the interrogation of self-assembly processes in organic media.

ALFA-PRF: a novel approach to detect murine perforin release from CTLs into the immune synapse.

When killing through the granule exocytosis pathway, cytotoxic lymphocytes release key effector molecules into the immune synapse, perforin and granzymes, to initiate target cell killing. The pore-forming perforin is essential for the function of cytotoxic lymphocytes, as its pores disrupt the target cell membrane and allow diffusion of pro-apoptotic serine proteases, granzyme, into the target cell, where they initiate various cell death cascades. Unlike human perforin, the detection of its murine counterpart in a live cell system has been problematic due its relatively low expression level and the lack of sensitive antibodies. The lack of a suitable methodology to visualise murine perforin secretion into the synapse hinders the study of the cytotoxic lymphocyte secretory machinery in murine models of human disease. Here, we describe a novel recombinant technology, whereby a short ALFA-tag sequence has been fused with the amino-terminus of a mature murine perforin, and this allowed its detection by the highly specific FluoTag®-X2 anti-ALFA nanobodies using both Total Internal Reflection Fluorescence (TIRF) microscopy of an artificial synapse, and confocal microscopy of the physiological immune synapse with a target cell. This methodology can have broad application in the field of cytotoxic lymphocyte biology and for the many models of human disease.

TissUExM enables quantitative ultrastructural analysis in whole vertebrate embryos by expansion microscopy.

Super-resolution microscopy reveals the molecular organization of biological structures down to the nanoscale. While it allows the study of protein complexes in single cells, small organisms, or thin tissue sections, there is currently no versatile approach for ultrastructural analysis compatible with whole vertebrate embryos. Here, we present tissue ultrastructure expansion microscopy (TissUExM), a method to expand millimeter-scale and mechanically heterogeneous whole embryonic tissues, including Drosophila wing discs, whole zebrafish, and mouse embryos. TissUExM is designed for the observation of endogenous proteins. It permits quantitative characterization of protein complexes in various organelles at super-resolution in a range of ∼3 mm-sized tissues using conventional microscopes. We demonstrate its strength by investigating tissue-specific ciliary architecture heterogeneity and ultrastructural defects observed upon ciliary protein overexpression. Overall, TissUExM is ideal for performing ultrastructural studies and molecular mapping in situ in whole embryos.

DeepBacs for multi-task bacterial image analysis using open-source deep learning approaches.

This work demonstrates and guides how to use a range of state-of-the-art artificial neural-networks to analyse bacterial microscopy images using the recently developed ZeroCostDL4Mic platform. We generated a database of image datasets used to train networks for various image analysis tasks and present strategies for data acquisition and curation, as well as model training. We showcase different deep learning (DL) approaches for segmenting bright field and fluorescence images of different bacterial species, use object detection to classify different growth stages in time-lapse imaging data, and carry out DL-assisted phenotypic profiling of antibiotic-treated cells. To also demonstrate the ability of DL to enhance low-phototoxicity live-cell microscopy, we showcase how image denoising can allow researchers to attain high-fidelity data in faster and longer imaging. Finally, artificial labelling of cell membranes and predictions of super-resolution images allow for accurate mapping of cell shape and intracellular targets. Our purposefully-built database of training and testing data aids in novice users' training, enabling them to quickly explore how to analyse their data through DL. We hope this lays a fertile ground for the efficient application of DL in microbiology and fosters the creation of tools for bacterial cell biology and antibiotic research.

FLIM FRET technology for drug discovery: automated multiwell-plate high-content analysis, multiplexed readouts and application in situ.

A fluorescence lifetime imaging (FLIM) technology platform intended to read out changes in Förster resonance energy transfer (FRET) efficiency is presented for the study of protein interactions across the drug-discovery pipeline. FLIM provides a robust, inherently ratiometric imaging modality for drug discovery that could allow the same sensor constructs to be translated from automated cell-based assays through small transparent organisms such as zebrafish to mammals. To this end, an automated FLIM multiwell-plate reader is described for high content analysis of fixed and live cells, tomographic FLIM in zebrafish and FLIM FRET of live cells via confocal endomicroscopy. For cell-based assays, an exemplar application reading out protein aggregation using FLIM FRET is presented, and the potential for multiple simultaneous FLIM (FRET) readouts in microscopy is illustrated.

Förster resonance energy transfer imaging in vivo with approximated radiative transfer equation.

We describe a new light transport model, which was applied to three-dimensional lifetime imaging of Förster resonance energy transfer in mice in vivo. The model is an approximation to the radiative transfer equation and combines light diffusion and ray optics. This approximation is well adopted to wide-field time-gated intensity-based data acquisition. Reconstructed image data are presented and compared with results obtained by using the telegraph equation approximation. The new approach provides improved recovery of absorption and scattering parameters while returning similar values for the fluorescence parameters.

Imaging in focus: An introduction to denoising bioimages in the era of deep learning.

Fluorescence microscopy enables the direct observation of previously hidden dynamic processes of life, allowing profound insights into mechanisms of health and disease. However, imaging of live samples is fundamentally limited by the toxicity of the illuminating light and images are often acquired using low light conditions. As a consequence, images can become very noisy which severely complicates their interpretation. In recent years, deep learning (DL) has emerged as a very successful approach to remove this noise while retaining the useful signal. Unlike classical algorithms which use well-defined mathematical functions to remove noise, DL methods learn to denoise from example data, providing a powerful content-aware approach. In this review, we first describe the different types of noise that typically corrupt fluorescence microscopy images and introduce the denoising task. We then present the main DL-based denoising methods and their relative advantages and disadvantages. We aim to provide insights into how DL-based denoising methods operate and help users choose the most appropriate tools for their applications.