Content-aware frame interpolation (CAFI): deep learning-based temporal super-resolution for fast bioimaging.

The development of high-resolution microscopes has made it possible to investigate cellular processes in 3D and over time. However, observing fast cellular dynamics remains challenging because of photobleaching and phototoxicity. Here we report the implementation of two content-aware frame interpolation (CAFI) deep learning networks, Zooming SlowMo and Depth-Aware Video Frame Interpolation, that are highly suited for accurately predicting images in between image pairs, therefore improving the temporal resolution of image series post-acquisition. We show that CAFI is capable of understanding the motion context of biological structures and can perform better than standard interpolation methods. We benchmark CAFI's performance on 12 different datasets, obtained from four different microscopy modalities, and demonstrate its capabilities for single-particle tracking and nuclear segmentation. CAFI potentially allows for reduced light exposure and phototoxicity on the sample for improved long-term live-cell imaging. The models and the training and testing data are available via the ZeroCostDL4Mic platform.

High-fidelity 3D live-cell nanoscopy through data-driven enhanced super-resolution radial fluctuation.

Live-cell super-resolution microscopy enables the imaging of biological structure dynamics below the diffraction limit. Here we present enhanced super-resolution radial fluctuations (eSRRF), substantially improving image fidelity and resolution compared to the original SRRF method. eSRRF incorporates automated parameter optimization based on the data itself, giving insight into the trade-off between resolution and fidelity. We demonstrate eSRRF across a range of imaging modalities and biological systems. Notably, we extend eSRRF to three dimensions by combining it with multifocus microscopy. This realizes live-cell volumetric super-resolution imaging with an acquisition speed of ~1 volume per second. eSRRF provides an accessible super-resolution approach, maximizing information extraction across varied experimental conditions while minimizing artifacts. Its optimal parameter prediction strategy is generalizable, moving toward unbiased and optimized analyses in super-resolution microscopy.

Fast4DReg - fast registration of 4D microscopy datasets.

Unwanted sample drift is a common issue that plagues microscopy experiments, preventing accurate temporal visualization and quantification of biological processes. Although multiple methods and tools exist to correct images post acquisition, performing drift correction of three-dimensional (3D) videos using open-source solutions remains challenging and time consuming. Here, we present a new tool developed for ImageJ or Fiji called Fast4DReg that can quickly correct axial and lateral drift in 3D video-microscopy datasets. Fast4DReg works by creating intensity projections along multiple axes and estimating the drift between frames using two-dimensional cross-correlations. Using synthetic and acquired datasets, we demonstrate that Fast4DReg can perform better than other state-of-the-art open-source drift-correction tools and significantly outperforms them in speed. We also demonstrate that Fast4DReg can be used to register misaligned channels in 3D using either calibration slides or misaligned images directly. Altogether, Fast4DReg provides a quick and easy-to-use method to correct 3D imaging data before further visualization and analysis.

TrackMate 7: integrating state-of-the-art segmentation algorithms into tracking pipelines.

TrackMate is an automated tracking software used to analyze bioimages and is distributed as a Fiji plugin. Here, we introduce a new version of TrackMate. TrackMate 7 is built to address the broad spectrum of modern challenges researchers face by integrating state-of-the-art segmentation algorithms into tracking pipelines. We illustrate qualitatively and quantitatively that these new capabilities function effectively across a wide range of bio-imaging experiments.

A method for the fast and photon-efficient analysis of time-domain fluorescence lifetime image data over large dynamic ranges.

Fluorescence lifetime imaging (FLIM) allows the quantification of sub-cellular processes in situ, in living cells. A number of approaches have been developed to extract the lifetime from time-domain FLIM data, but they are often limited in terms of speed, photon efficiency, precision or the dynamic range of lifetimes they can measure. Here, we focus on one of the best performing methods in the field, the centre-of-mass method (CMM), that conveys advantages in terms of speed and photon efficiency over others. In this paper, however, we identify a loss of photon efficiency of CMM for short lifetimes when background noise is present. We subsequently present a new development and generalization of CMM that provides for the rapid and accurate extraction of fluorescence lifetime over a large lifetime dynamic range. We provide software tools to simulate, validate and analyse FLIM data sets and compare the performance of our approach against the standard CMM and the commonly employed least-square minimization (LSM) methods. Our method features a better photon efficiency than standard CMM and LSM and is robust in the presence of background noise. The algorithm is applicable to any time-domain FLIM data set.

Fluctuation-Based Super-Resolution Traction Force Microscopy.

Cellular mechanics play a crucial role in tissue homeostasis and are often misregulated in disease. Traction force microscopy is one of the key methods that has enabled researchers to study fundamental aspects of mechanobiology; however, traction force microscopy is limited by poor resolution. Here, we propose a simplified protocol and imaging strategy that enhances the output of traction force microscopy by increasing i) achievable bead density and ii) the accuracy of bead tracking. Our approach relies on super-resolution microscopy, enabled by fluorescence fluctuation analysis. Our pipeline can be used on spinning-disk confocal or widefield microscopes and is compatible with available analysis software. In addition, we demonstrate that our workflow can be used to gain biologically relevant information and is suitable for fast long-term live measurement of traction forces even in light-sensitive cells. Finally, using fluctuation-based traction force microscopy, we observe that filopodia align to the force field generated by focal adhesions.

Artificial intelligence for microscopy: what you should know.

Artificial Intelligence based on Deep Learning (DL) is opening new horizons in biomedical research and promises to revolutionize the microscopy field. It is now transitioning from the hands of experts in computer sciences to biomedical researchers. Here, we introduce recent developments in DL applied to microscopy, in a manner accessible to non-experts. We give an overview of its concepts, capabilities and limitations, presenting applications in image segmentation, classification and restoration. We discuss how DL shows an outstanding potential to push the limits of microscopy, enhancing resolution, signal and information content in acquired data. Its pitfalls are discussed, along with the future directions expected in this field.

NanoJ: a high-performance open-source super-resolution microscopy toolbox.

Super-resolution microscopy (SRM) has become essential for the study of nanoscale biological processes. This type of imaging often requires the use of specialised image analysis tools to process a large volume of recorded data and extract quantitative information. In recent years, our team has built an open-source image analysis framework for SRM designed to combine high performance and ease of use. We named it NanoJ-a reference to the popular ImageJ software it was developed for. In this paper, we highlight the current capabilities of NanoJ for several essential processing steps: spatio-temporal alignment of raw data (NanoJ-Core), super-resolution image reconstruction (NanoJ-SRRF), image quality assessment (NanoJ-SQUIRREL), structural modelling (NanoJ-VirusMapper) and control of the sample environment (NanoJ-Fluidics). We expect to expand NanoJ in the future through the development of new tools designed to improve quantitative data analysis and measure the reliability of fluorescent microscopy studies.

Nanoscopic insights into seeding mechanisms and toxicity of α-synuclein species in neurons.

New strategies for visualizing self-assembly processes at the nanoscale give deep insights into the molecular origins of disease. An example is the self-assembly of misfolded proteins into amyloid fibrils, which is related to a range of neurodegenerative disorders, such as Parkinson's and Alzheimer's diseases. Here, we probe the links between the mechanism of α-synuclein (AS) aggregation and its associated toxicity by using optical nanoscopy directly in a neuronal cell culture model of Parkinson's disease. Using superresolution microscopy, we show that protein fibrils are taken up by neuronal cells and act as prion-like seeds for elongation reactions that both consume endogenous AS and suppress its de novo aggregation. When AS is internalized in its monomeric form, however, it nucleates and triggers the aggregation of endogenous AS, leading to apoptosis, although there are no detectable cross-reactions between externally added and endogenous protein species. Monomer-induced apoptosis can be reduced by pretreatment with seed fibrils, suggesting that partial consumption of the externally added or excess soluble AS can be significantly neuroprotective.

HSV-1 Glycoproteins Are Delivered to Virus Assembly Sites Through Dynamin-Dependent Endocytosis.

Herpes simplex virus-1 (HSV-1) is a large enveloped DNA virus that belongs to the family of Herpesviridae. It has been recently shown that the cytoplasmic membranes that wrap the newly assembled capsids are endocytic compartments derived from the plasma membrane. Here, we show that dynamin-dependent endocytosis plays a major role in this process. Dominant-negative dynamin and clathrin adaptor AP180 significantly decrease virus production. Moreover, inhibitors targeting dynamin and clathrin lead to a decreased transport of glycoproteins to cytoplasmic capsids, confirming that glycoproteins are delivered to assembly sites via endocytosis. We also show that certain combinations of glycoproteins colocalize with each other and with the components of clathrin-dependent and -independent endocytosis pathways. Importantly, we demonstrate that the uptake of neutralizing antibodies that bind to glycoproteins when they become exposed on the cell surface during virus particle assembly leads to the production of non-infectious HSV-1. Our results demonstrate that transport of viral glycoproteins to the plasma membrane prior to endocytosis is the major route by which these proteins are localized to the cytoplasmic virus assembly compartments. This highlights the importance of endocytosis as a major protein-sorting event during HSV-1 envelopment.

A method to quantify FRET stoichiometry with phasor plot analysis and acceptor lifetime ingrowth.

FRET is widely used for the study of protein-protein interactions in biological samples. However, it is difficult to quantify both the FRET efficiency (E) and the affinity (Kd) of the molecular interaction from intermolecular FRET signals in samples of unknown stoichiometry. Here, we present a method for the simultaneous quantification of the complete set of interaction parameters, including fractions of bound donors and acceptors, local protein concentrations, and dissociation constants, in each image pixel. The method makes use of fluorescence lifetime information from both donor and acceptor molecules and takes advantage of the linear properties of the phasor plot approach. We demonstrate the capability of our method in vitro in a microfluidic device and also in cells, via the determination of the binding affinity between tagged versions of glutathione and glutathione S-transferase, and via the determination of competitor concentration. The potential of the method is explored with simulations.

In Situ Visualization of Block Copolymer Self-Assembly in Organic Media by Super-Resolution Fluorescence Microscopy.

Analytical methods that enable visualization of nanomaterials derived from solution self-assembly processes in organic solvents are highly desirable. Herein, we demonstrate the use of stimulated emission depletion microscopy (STED) and single molecule localization microscopy (SMLM) to map living crystallization-driven block copolymer (BCP) self-assembly in organic media at the sub-diffraction scale. Four different dyes were successfully used for single-colour super-resolution imaging of the BCP nanostructures allowing micelle length distributions to be determined in situ. Dual-colour SMLM imaging was used to measure and compare the rate of addition of red fluorescent BCP to the termini of green fluorescent seed micelles to generate block comicelles. Although well-established for aqueous systems, the results highlight the potential of super-resolution microscopy techniques for the interrogation of self-assembly processes in organic media.

ALFA-PRF: a novel approach to detect murine perforin release from CTLs into the immune synapse.

When killing through the granule exocytosis pathway, cytotoxic lymphocytes release key effector molecules into the immune synapse, perforin and granzymes, to initiate target cell killing. The pore-forming perforin is essential for the function of cytotoxic lymphocytes, as its pores disrupt the target cell membrane and allow diffusion of pro-apoptotic serine proteases, granzyme, into the target cell, where they initiate various cell death cascades. Unlike human perforin, the detection of its murine counterpart in a live cell system has been problematic due its relatively low expression level and the lack of sensitive antibodies. The lack of a suitable methodology to visualise murine perforin secretion into the synapse hinders the study of the cytotoxic lymphocyte secretory machinery in murine models of human disease. Here, we describe a novel recombinant technology, whereby a short ALFA-tag sequence has been fused with the amino-terminus of a mature murine perforin, and this allowed its detection by the highly specific FluoTag®-X2 anti-ALFA nanobodies using both Total Internal Reflection Fluorescence (TIRF) microscopy of an artificial synapse, and confocal microscopy of the physiological immune synapse with a target cell. This methodology can have broad application in the field of cytotoxic lymphocyte biology and for the many models of human disease.

TissUExM enables quantitative ultrastructural analysis in whole vertebrate embryos by expansion microscopy.

Super-resolution microscopy reveals the molecular organization of biological structures down to the nanoscale. While it allows the study of protein complexes in single cells, small organisms, or thin tissue sections, there is currently no versatile approach for ultrastructural analysis compatible with whole vertebrate embryos. Here, we present tissue ultrastructure expansion microscopy (TissUExM), a method to expand millimeter-scale and mechanically heterogeneous whole embryonic tissues, including Drosophila wing discs, whole zebrafish, and mouse embryos. TissUExM is designed for the observation of endogenous proteins. It permits quantitative characterization of protein complexes in various organelles at super-resolution in a range of ∼3 mm-sized tissues using conventional microscopes. We demonstrate its strength by investigating tissue-specific ciliary architecture heterogeneity and ultrastructural defects observed upon ciliary protein overexpression. Overall, TissUExM is ideal for performing ultrastructural studies and molecular mapping in situ in whole embryos.

DeepBacs for multi-task bacterial image analysis using open-source deep learning approaches.

This work demonstrates and guides how to use a range of state-of-the-art artificial neural-networks to analyse bacterial microscopy images using the recently developed ZeroCostDL4Mic platform. We generated a database of image datasets used to train networks for various image analysis tasks and present strategies for data acquisition and curation, as well as model training. We showcase different deep learning (DL) approaches for segmenting bright field and fluorescence images of different bacterial species, use object detection to classify different growth stages in time-lapse imaging data, and carry out DL-assisted phenotypic profiling of antibiotic-treated cells. To also demonstrate the ability of DL to enhance low-phototoxicity live-cell microscopy, we showcase how image denoising can allow researchers to attain high-fidelity data in faster and longer imaging. Finally, artificial labelling of cell membranes and predictions of super-resolution images allow for accurate mapping of cell shape and intracellular targets. Our purposefully-built database of training and testing data aids in novice users' training, enabling them to quickly explore how to analyse their data through DL. We hope this lays a fertile ground for the efficient application of DL in microbiology and fosters the creation of tools for bacterial cell biology and antibiotic research.

Imaging in focus: An introduction to denoising bioimages in the era of deep learning.

Fluorescence microscopy enables the direct observation of previously hidden dynamic processes of life, allowing profound insights into mechanisms of health and disease. However, imaging of live samples is fundamentally limited by the toxicity of the illuminating light and images are often acquired using low light conditions. As a consequence, images can become very noisy which severely complicates their interpretation. In recent years, deep learning (DL) has emerged as a very successful approach to remove this noise while retaining the useful signal. Unlike classical algorithms which use well-defined mathematical functions to remove noise, DL methods learn to denoise from example data, providing a powerful content-aware approach. In this review, we first describe the different types of noise that typically corrupt fluorescence microscopy images and introduce the denoising task. We then present the main DL-based denoising methods and their relative advantages and disadvantages. We aim to provide insights into how DL-based denoising methods operate and help users choose the most appropriate tools for their applications.

Application of Super-Resolution and Advanced Quantitative Microscopy to the Spatio-Temporal Analysis of Influenza Virus Replication.

With an estimated three to five million human cases annually and the potential to infect domestic and wild animal populations, influenza viruses are one of the greatest health and economic burdens to our society, and pose an ongoing threat of large-scale pandemics. Despite our knowledge of many important aspects of influenza virus biology, there is still much to learn about how influenza viruses replicate in infected cells, for instance, how they use entry receptors or exploit host cell trafficking pathways. These gaps in our knowledge are due, in part, to the difficulty of directly observing viruses in living cells. In recent years, advances in light microscopy, including super-resolution microscopy and single-molecule imaging, have enabled many viral replication steps to be visualised dynamically in living cells. In particular, the ability to track single virions and their components, in real time, now allows specific pathways to be interrogated, providing new insights to various aspects of the virus-host cell interaction. In this review, we discuss how state-of-the-art imaging technologies, notably quantitative live-cell and super-resolution microscopy, are providing new nanoscale and molecular insights into influenza virus replication and revealing new opportunities for developing antiviral strategies.

Single-Molecule Super-Resolution Imaging of T-Cell Plasma Membrane CD4 Redistribution upon HIV-1 Binding.

The first step of cellular entry for the human immunodeficiency virus type-1 (HIV-1) occurs through the binding of its envelope protein (Env) with the plasma membrane receptor CD4 and co-receptor CCR5 or CXCR4 on susceptible cells, primarily CD4+ T cells and macrophages. Although there is considerable knowledge of the molecular interactions between Env and host cell receptors that lead to successful fusion, the precise way in which HIV-1 receptors redistribute to sites of virus binding at the nanoscale remains unknown. Here, we quantitatively examine changes in the nanoscale organisation of CD4 on the surface of CD4+ T cells following HIV-1 binding. Using single-molecule super-resolution imaging, we show that CD4 molecules are distributed mostly as either individual molecules or small clusters of up to 4 molecules. Following virus binding, we observe a local 3-to-10-fold increase in cluster diameter and molecule number for virus-associated CD4 clusters. Moreover, a similar but smaller magnitude reorganisation of CD4 was also observed with recombinant gp120. For one of the first times, our results quantify the nanoscale CD4 reorganisation triggered by HIV-1 on host CD4+ T cells. Our quantitative approach provides a robust methodology for characterising the nanoscale organisation of plasma membrane receptors in general with the potential to link spatial organisation to function.

Democratising deep learning for microscopy with ZeroCostDL4Mic.

Deep Learning (DL) methods are powerful analytical tools for microscopy and can outperform conventional image processing pipelines. Despite the enthusiasm and innovations fuelled by DL technology, the need to access powerful and compatible resources to train DL networks leads to an accessibility barrier that novice users often find difficult to overcome. Here, we present ZeroCostDL4Mic, an entry-level platform simplifying DL access by leveraging the free, cloud-based computational resources of Google Colab. ZeroCostDL4Mic allows researchers with no coding expertise to train and apply key DL networks to perform tasks including segmentation (using U-Net and StarDist), object detection (using YOLOv2), denoising (using CARE and Noise2Void), super-resolution microscopy (using Deep-STORM), and image-to-image translation (using Label-free prediction - fnet, pix2pix and CycleGAN). Importantly, we provide suitable quantitative tools for each network to evaluate model performance, allowing model optimisation. We demonstrate the application of the platform to study multiple biological processes.