Live-cell imaging powered by computation.

The proliferation of microscopy methods for live-cell imaging offers many new possibilities for users but can also be challenging to navigate. The prevailing challenge in live-cell fluorescence microscopy is capturing intra-cellular dynamics while preserving cell viability. Computational methods can help to address this challenge and are now shifting the boundaries of what is possible to capture in living systems. In this Review, we discuss these computational methods focusing on artificial intelligence-based approaches that can be layered on top of commonly used existing microscopies as well as hybrid methods that integrate computation and microscope hardware. We specifically discuss how computational approaches can improve the signal-to-noise ratio, spatial resolution, temporal resolution and multi-colour capacity of live-cell imaging.

Sequence-Independent Self-Assembly of Germ Granule mRNAs into Homotypic Clusters.

mRNAs enriched in membraneless condensates provide functional compartmentalization within cells. The mechanisms that recruit transcripts to condensates are under intense study; however, how mRNAs organize once they reach a granule remains poorly understood. Here, we report on a self-sorting mechanism by which multiple mRNAs derived from the same gene assemble into discrete homotypic clusters. We demonstrate that in vivo mRNA localization to granules and self-assembly within granules are governed by different mRNA features: localization is encoded by specific RNA regions, whereas self-assembly involves the entire mRNA, does not involve sequence-specific, ordered intermolecular RNA:RNA interactions, and is thus RNA sequence independent. We propose that the ability of mRNAs to self-sort into homotypic assemblies is an inherent property of an messenger ribonucleoprotein (mRNP) that is augmented under conditions that increase RNA concentration, such as upon enrichment in RNA-protein granules, a process that appears conserved in diverse cellular contexts and organisms.

Multiview confocal super-resolution microscopy.

Confocal microscopy1 remains a major workhorse in biomedical optical microscopy owing to its reliability and flexibility in imaging various samples, but suffers from substantial point spread function anisotropy, diffraction-limited resolution, depth-dependent degradation in scattering samples and volumetric bleaching2. Here we address these problems, enhancing confocal microscopy performance from the sub-micrometre to millimetre spatial scale and the millisecond to hour temporal scale, improving both lateral and axial resolution more than twofold while simultaneously reducing phototoxicity. We achieve these gains using an integrated, four-pronged approach: (1) developing compact line scanners that enable sensitive, rapid, diffraction-limited imaging over large areas; (2) combining line-scanning with multiview imaging, developing reconstruction algorithms that improve resolution isotropy and recover signal otherwise lost to scattering; (3) adapting techniques from structured illumination microscopy, achieving super-resolution imaging in densely labelled, thick samples; (4) synergizing deep learning with these advances, further improving imaging speed, resolution and duration. We demonstrate these capabilities on more than 20 distinct fixed and live samples, including protein distributions in single cells; nuclei and developing neurons in Caenorhabditis elegans embryos, larvae and adults; myoblasts in imaginal disks of Drosophila wings; and mouse renal, oesophageal, cardiac and brain tissues.

Structural and developmental principles of neuropil assembly in C. elegans.

Neuropil is a fundamental form of tissue organization within the brain1, in which densely packed neurons synaptically interconnect into precise circuit architecture2,3. However, the structural and developmental principles that govern this nanoscale precision remain largely unknown4,5. Here we use an iterative data coarse-graining algorithm termed 'diffusion condensation'6 to identify nested circuit structures within the Caenorhabditis elegans neuropil, which is known as the nerve ring. We show that the nerve ring neuropil is largely organized into four strata that are composed of related behavioural circuits. The stratified architecture of the neuropil is a geometrical representation of the functional segregation of sensory information and motor outputs, with specific sensory organs and muscle quadrants mapping onto particular neuropil strata. We identify groups of neurons with unique morphologies that integrate information across strata and that create neural structures that cage the strata within the nerve ring. We use high resolution light-sheet microscopy7,8 coupled with lineage-tracing and cell-tracking algorithms9,10 to resolve the developmental sequence and reveal principles of cell position, migration and outgrowth that guide stratified neuropil organization. Our results uncover conserved structural design principles that underlie the architecture and function of the nerve ring neuropil, and reveal a temporal progression of outgrowth-based on pioneer neurons-that guides the hierarchical development of the layered neuropil. Our findings provide a systematic blueprint for using structural and developmental approaches to understand neuropil organization within the brain.

Cell division machinery drives cell-specific gene activation during differentiation in Bacillus subtilis.

When faced with starvation, the bacterium Bacillus subtilis transforms itself into a dormant cell type called a "spore". Sporulation initiates with an asymmetric division event, which requires the relocation of the core divisome components FtsA and FtsZ, after which the sigma factor σF is exclusively activated in the smaller daughter cell. Compartment-specific activation of σF requires the SpoIIE phosphatase, which displays a biased localization on one side of the asymmetric division septum and associates with the structural protein DivIVA, but the mechanism by which this preferential localization is achieved is unclear. Here, we isolated a variant of DivIVA that indiscriminately activates σF in both daughter cells due to promiscuous localization of SpoIIE, which was corrected by overproduction of FtsA and FtsZ. We propose that the core components of the redeployed cell division machinery drive the asymmetric localization of DivIVA and SpoIIE to trigger the initiation of the sporulation program.

Practical considerations for quantitative light sheet fluorescence microscopy.

Fluorescence microscopy has evolved from a purely observational tool to a platform for quantitative, hypothesis-driven research. As such, the demand for faster and less phototoxic imaging modalities has spurred a rapid growth in light sheet fluorescence microscopy (LSFM). By restricting the excitation to a thin plane, LSFM reduces the overall light dose to a specimen while simultaneously improving image contrast. However, the defining characteristics of light sheet microscopes subsequently warrant unique considerations in their use for quantitative experiments. In this Perspective, we outline many of the pitfalls in LSFM that can compromise analysis and confound interpretation. Moreover, we offer guidance in addressing these caveats when possible. In doing so, we hope to provide a useful resource for life scientists seeking to adopt LSFM to quantitatively address complex biological hypotheses.

Incorporating the image formation process into deep learning improves network performance.

We present Richardson-Lucy network (RLN), a fast and lightweight deep learning method for three-dimensional fluorescence microscopy deconvolution. RLN combines the traditional Richardson-Lucy iteration with a fully convolutional network structure, establishing a connection to the image formation process and thereby improving network performance. Containing only roughly 16,000 parameters, RLN enables four- to 50-fold faster processing than purely data-driven networks with many more parameters. By visual and quantitative analysis, we show that RLN provides better deconvolution, better generalizability and fewer artifacts than other networks, especially along the axial dimension. RLN outperforms classic Richardson-Lucy deconvolution on volumes contaminated with severe out of focus fluorescence or noise and provides four- to sixfold faster reconstructions of large, cleared-tissue datasets than classic multi-view pipelines. We demonstrate RLN's performance on cells, tissues and embryos imaged with widefield-, light-sheet-, confocal- and super-resolution microscopy.

Myc Regulates Chromatin Decompaction and Nuclear Architecture during B Cell Activation.

50 years ago, Vincent Allfrey and colleagues discovered that lymphocyte activation triggers massive acetylation of chromatin. However, the molecular mechanisms driving epigenetic accessibility are still unknown. We here show that stimulated lymphocytes decondense chromatin by three differentially regulated steps. First, chromatin is repositioned away from the nuclear periphery in response to global acetylation. Second, histone nanodomain clusters decompact into mononucleosome fibers through a mechanism that requires Myc and continual energy input. Single-molecule imaging shows that this step lowers transcription factor residence time and non-specific collisions during sampling for DNA targets. Third, chromatin interactions shift from long range to predominantly short range, and CTCF-mediated loops and contact domains double in numbers. This architectural change facilitates cognate promoter-enhancer contacts and also requires Myc and continual ATP production. Our results thus define the nature and transcriptional impact of chromatin decondensation and reveal an unexpected role for Myc in the establishment of nuclear topology in mammalian cells.

Three-dimensional residual channel attention networks denoise and sharpen fluorescence microscopy image volumes.

We demonstrate residual channel attention networks (RCAN) for the restoration and enhancement of volumetric time-lapse (four-dimensional) fluorescence microscopy data. First we modify RCAN to handle image volumes, showing that our network enables denoising competitive with three other state-of-the-art neural networks. We use RCAN to restore noisy four-dimensional super-resolution data, enabling image capture of over tens of thousands of images (thousands of volumes) without apparent photobleaching. Second, using simulations we show that RCAN enables resolution enhancement equivalent to, or better than, other networks. Third, we exploit RCAN for denoising and resolution improvement in confocal microscopy, enabling ~2.5-fold lateral resolution enhancement using stimulated emission depletion microscopy ground truth. Fourth, we develop methods to improve spatial resolution in structured illumination microscopy using expansion microscopy data as ground truth, achieving improvements of ~1.9-fold laterally and ~3.6-fold axially. Finally, we characterize the limits of denoising and resolution enhancement, suggesting practical benchmarks for evaluation and further enhancement of network performance.

The HIV-1 Viral Protease Is Activated during Assembly and Budding Prior to Particle Release.

HIV-1 encodes a viral protease that is essential for the maturation of infectious viral particles. While protease inhibitors are effective antiretroviral agents, recent studies have shown that prematurely activating, rather than inhibiting, protease function leads to the pyroptotic death of infected cells, with exciting implications for efforts to eradicate viral reservoirs. Despite 40 years of research into the kinetics of protease activation, it remains unclear exactly when protease becomes activated. Recent reports have estimated that protease activation occurs minutes to hours after viral release, suggesting that premature protease activation is challenging to induce efficiently. Here, monitoring viral protease activity with sensitive techniques, including nanoscale flow cytometry and instant structured illumination microscopy, we demonstrate that the viral protease is activated within cells prior to the release of free virions. Using genetic mutants that lock protease into a precursor conformation, we further show that both the precursor and mature protease have rapid activation kinetics and that the activity of the precursor protease is sufficient for viral fusion with target cells. Our finding that HIV-1 protease is activated within producer cells prior to release of free virions helps resolve a long-standing question of when protease is activated and suggests that only a modest acceleration of protease activation kinetics is required to induce potent and specific elimination of HIV-infected cells. IMPORTANCE HIV-1 protease inhibitors have been a mainstay of antiretroviral therapy for more than 2 decades. Although antiretroviral therapy is effective at controlling HIV-1 replication, persistent reservoirs of latently infected cells quickly reestablish replication if therapy is halted. A promising new strategy to eradicate the latent reservoir involves prematurely activating the viral protease, which leads to the pyroptotic killing of infected cells. Here, we use highly sensitive techniques to examine the kinetics of protease activation during and shortly after particle formation. We found that protease is fully activated before virus is released from the cell membrane, which is hours earlier than recent estimates. Our findings help resolve a long-standing debate as to when the viral protease is initially activated during viral assembly and confirm that prematurely activating HIV-1 protease is a viable strategy to eradicate infected cells following latency reversal.

Human deafness-associated variants alter the dynamics of key molecules in hair cell stereocilia F-actin cores.

Stereocilia protrude up to 100 µm from the apical surface of vertebrate inner ear hair cells and are packed with cross-linked filamentous actin (F-actin). They function as mechanical switches to convert sound vibration into electrochemical neuronal signals transmitted to the brain. Several genes encode molecular components of stereocilia including actin monomers, actin regulatory and bundling proteins, motor proteins and the proteins of the mechanotransduction complex. A stereocilium F-actin core is a dynamic system, which is continuously being remodeled while maintaining an outwardly stable architecture under the regulation of F-actin barbed-end cappers, severing proteins and crosslinkers. The F-actin cores of stereocilia also provide a pathway for motor proteins to transport cargos including components of tip-link densities, scaffolding proteins and actin regulatory proteins. Deficiencies and mutations of stereocilia components that disturb this "dynamic equilibrium" in stereocilia can induce morphological changes and disrupt mechanotransduction causing sensorineural hearing loss, best studied in mouse and zebrafish models. Currently, at least 23 genes, associated with human syndromic and nonsyndromic hearing loss, encode proteins involved in the development and maintenance of stereocilia F-actin cores. However, it is challenging to predict how variants associated with sensorineural hearing loss segregating in families affect protein function. Here, we review the functions of several molecular components of stereocilia F-actin cores and provide new data from our experimental approach to directly evaluate the pathogenicity and functional impact of reported and novel variants of DIAPH1 in autosomal-dominant DFNA1 hearing loss using single-molecule fluorescence microscopy.

Author Correction: Faster, sharper, and deeper: structured illumination microscopy for biological imaging.

In the version of this Perspective originally published, Fig. 4g included an incorrect inset adapted from a different figure than the main image in the panel. This error has been corrected in the PDF and HTML versions of the paper.

Multiscale fluorescence imaging of living samples.

Fluorescence microscopy is a highly effective tool for interrogating biological structure and function, particularly when imaging across multiple spatiotemporal scales. Here we survey recent innovations and applications in the relatively understudied area of multiscale fluorescence imaging of living samples. We discuss fundamental challenges in live multiscale imaging and describe successful examples that highlight the power of this approach. We attempt to synthesize general strategies from these test cases, aiming to help accelerate progress in this exciting area.

Signaling through polymerization and degradation: Analysis and simulations of T cell activation mediated by Bcl10.

The adaptive immune system serves as a potent and highly specific defense mechanism against pathogen infection. One component of this system, the effector T cell, facilitates pathogen clearance upon detection of specific antigens by the T cell receptor (TCR). A critical process in effector T cell activation is transmission of signals from the TCR to a key transcriptional regulator, NF-κB. The transmission of this signal involves a highly dynamic process in which helical filaments of Bcl10, a key protein constituent of the TCR signaling cascade, undergo competing processes of polymeric assembly and macroautophagy-dependent degradation. Through computational analysis of three-dimensional, super-resolution optical micrographs, we quantitatively characterize TCR-stimulated Bcl10 filament assembly and length dynamics, and demonstrate that filaments become shorter over time. Additionally, we develop an image-based, bootstrap-like resampling method that demonstrates the preferred association between autophagosomes and both Bcl10-filament ends and punctate-Bcl10 structures, implying that autophagosome-driven macroautophagy is directly responsible for Bcl10 filament shortening. We probe Bcl10 polymerization-depolymerization dynamics with a stochastic Monte-Carlo simulation of nucleation-limited filament assembly and degradation, and we show that high probabilities of filament nucleation in response to TCR engagement could provide the observed robust, homogeneous, and tunable response dynamic. Furthermore, we demonstrate that the speed of filament disassembly preferentially at filament ends provides effective regulatory control. Taken together, these data suggest that Bcl10 filament growth and degradation act as an excitable system that provides a digital response mechanism and the reliable timing critical for T cell activation and regulatory processes.

A 2-dimensional ratchet model describes assembly initiation of a specialized bacterial cell surface.

Bacterial spores are dormant cells that are encased in a thick protein shell, the "coat," which participates in protecting the organism's DNA from environmental insults. The coat is composed of dozens of proteins that assemble in an orchestrated fashion during sporulation. In Bacillus subtilis, 2 proteins initiate coat assembly: SpoVM, which preferentially binds to micron-scale convex membranes and marks the surface of the developing spore as the site for coat assembly; and SpoIVA, a structural protein recruited by SpoVM that uses ATP hydrolysis to drive its irreversible polymerization around the developing spore. Here, we describe the initiation of coat assembly by SpoVM and SpoIVA. Using single-molecule fluorescence microscopy in vivo in sporulating cells and in vitro on synthetic spores, we report that SpoVM's localization is primarily driven by a lower off-rate on membranes of preferred curvature in the absence of other coat proteins. Recruitment and polymerization of SpoIVA results in the entrapment of SpoVM on the forespore surface. Using experimentally derived reaction parameters, we show that a 2-dimensional ratchet model can describe the interdependent localization dynamics of SpoVM and SpoIVA, wherein SpoVM displays a longer residence time on the forespore surface, which favors recruitment of SpoIVA to that location. Localized SpoIVA polymerization in turn prevents further sampling of other membranes by prelocalized SpoVM molecules. Our model therefore describes the dynamics of structural proteins as they localize and assemble at the correct place and time within a cell to form a supramolecular complex.

Nanoscale Pattern Extraction from Relative Positions of Sparse 3D Localizations.

Inferring the organization of fluorescently labeled nanosized structures from single molecule localization microscopy (SMLM) data, typically obscured by stochastic noise and background, remains challenging. To overcome this, we developed a method to extract high-resolution ordered features from SMLM data that requires only a low fraction of targets to be localized with high precision. First, experimentally measured localizations are analyzed to produce relative position distributions (RPDs). Next, model RPDs are constructed using hypotheses of how the molecule is organized. Finally, a statistical comparison is used to select the most likely model. This approach allows pattern recognition at sub-1% detection efficiencies for target molecules, in large and heterogeneous samples and in 2D and 3D data sets. As a proof-of-concept, we infer ultrastructure of Nup107 within the nuclear pore, DNA origami structures, and α-actinin-2 within the cardiomyocyte Z-disc and assess the quality of images of centrioles to improve the averaged single-particle reconstruction.

Faster, sharper, and deeper: structured illumination microscopy for biological imaging.

Structured illumination microscopy (SIM) allows rapid, super-resolution (SR) imaging in live specimens. We review recent technical advances in SR-SIM, with emphasis on imaging speed, resolution, and depth. Since its introduction decades ago, the technique has grown to offer myriad implementations, each with its own strengths and weaknesses. We discuss these, aiming to provide a practical guide for biologists and to highlight which approach is best suited to a given application.

Single-shot super-resolution total internal reflection fluorescence microscopy.

We combined instant structured illumination microscopy (iSIM) with total internal reflection fluorescence microscopy (TIRFM) in an approach referred to as instant TIRF-SIM, thereby improving the lateral spatial resolution of TIRFM to 115 ± 13 nm without compromising speed, and enabling imaging frame rates up to 100 Hz over hundreds of time points. We applied instant TIRF-SIM to multiple live samples and achieved rapid, high-contrast super-resolution imaging close to the coverslip surface.

Assessing phototoxicity in live fluorescence imaging.

Are the answers to biological questions obtained via live fluorescence microscopy substantially affected by phototoxicity? Although a single set of standards for assessing phototoxicity cannot exist owing to the breadth of samples and experimental questions associated with biological imaging, we need quantitative, practical assessments and reporting standards to ensure that imaging has a minimal impact on observed biological processes and sample health. Here we discuss the problem of phototoxicity in biology and suggest guidelines to improve its reporting and assessment.