Multiview confocal super-resolution microscopy.

Confocal microscopy1 remains a major workhorse in biomedical optical microscopy owing to its reliability and flexibility in imaging various samples, but suffers from substantial point spread function anisotropy, diffraction-limited resolution, depth-dependent degradation in scattering samples and volumetric bleaching2. Here we address these problems, enhancing confocal microscopy performance from the sub-micrometre to millimetre spatial scale and the millisecond to hour temporal scale, improving both lateral and axial resolution more than twofold while simultaneously reducing phototoxicity. We achieve these gains using an integrated, four-pronged approach: (1) developing compact line scanners that enable sensitive, rapid, diffraction-limited imaging over large areas; (2) combining line-scanning with multiview imaging, developing reconstruction algorithms that improve resolution isotropy and recover signal otherwise lost to scattering; (3) adapting techniques from structured illumination microscopy, achieving super-resolution imaging in densely labelled, thick samples; (4) synergizing deep learning with these advances, further improving imaging speed, resolution and duration. We demonstrate these capabilities on more than 20 distinct fixed and live samples, including protein distributions in single cells; nuclei and developing neurons in Caenorhabditis elegans embryos, larvae and adults; myoblasts in imaginal disks of Drosophila wings; and mouse renal, oesophageal, cardiac and brain tissues.

Structural and developmental principles of neuropil assembly in C. elegans.

Neuropil is a fundamental form of tissue organization within the brain1, in which densely packed neurons synaptically interconnect into precise circuit architecture2,3. However, the structural and developmental principles that govern this nanoscale precision remain largely unknown4,5. Here we use an iterative data coarse-graining algorithm termed 'diffusion condensation'6 to identify nested circuit structures within the Caenorhabditis elegans neuropil, which is known as the nerve ring. We show that the nerve ring neuropil is largely organized into four strata that are composed of related behavioural circuits. The stratified architecture of the neuropil is a geometrical representation of the functional segregation of sensory information and motor outputs, with specific sensory organs and muscle quadrants mapping onto particular neuropil strata. We identify groups of neurons with unique morphologies that integrate information across strata and that create neural structures that cage the strata within the nerve ring. We use high resolution light-sheet microscopy7,8 coupled with lineage-tracing and cell-tracking algorithms9,10 to resolve the developmental sequence and reveal principles of cell position, migration and outgrowth that guide stratified neuropil organization. Our results uncover conserved structural design principles that underlie the architecture and function of the nerve ring neuropil, and reveal a temporal progression of outgrowth-based on pioneer neurons-that guides the hierarchical development of the layered neuropil. Our findings provide a systematic blueprint for using structural and developmental approaches to understand neuropil organization within the brain.

Practical considerations for quantitative light sheet fluorescence microscopy.

Fluorescence microscopy has evolved from a purely observational tool to a platform for quantitative, hypothesis-driven research. As such, the demand for faster and less phototoxic imaging modalities has spurred a rapid growth in light sheet fluorescence microscopy (LSFM). By restricting the excitation to a thin plane, LSFM reduces the overall light dose to a specimen while simultaneously improving image contrast. However, the defining characteristics of light sheet microscopes subsequently warrant unique considerations in their use for quantitative experiments. In this Perspective, we outline many of the pitfalls in LSFM that can compromise analysis and confound interpretation. Moreover, we offer guidance in addressing these caveats when possible. In doing so, we hope to provide a useful resource for life scientists seeking to adopt LSFM to quantitatively address complex biological hypotheses.

Incorporating the image formation process into deep learning improves network performance.

We present Richardson-Lucy network (RLN), a fast and lightweight deep learning method for three-dimensional fluorescence microscopy deconvolution. RLN combines the traditional Richardson-Lucy iteration with a fully convolutional network structure, establishing a connection to the image formation process and thereby improving network performance. Containing only roughly 16,000 parameters, RLN enables four- to 50-fold faster processing than purely data-driven networks with many more parameters. By visual and quantitative analysis, we show that RLN provides better deconvolution, better generalizability and fewer artifacts than other networks, especially along the axial dimension. RLN outperforms classic Richardson-Lucy deconvolution on volumes contaminated with severe out of focus fluorescence or noise and provides four- to sixfold faster reconstructions of large, cleared-tissue datasets than classic multi-view pipelines. We demonstrate RLN's performance on cells, tissues and embryos imaged with widefield-, light-sheet-, confocal- and super-resolution microscopy.

Three-dimensional residual channel attention networks denoise and sharpen fluorescence microscopy image volumes.

We demonstrate residual channel attention networks (RCAN) for the restoration and enhancement of volumetric time-lapse (four-dimensional) fluorescence microscopy data. First we modify RCAN to handle image volumes, showing that our network enables denoising competitive with three other state-of-the-art neural networks. We use RCAN to restore noisy four-dimensional super-resolution data, enabling image capture of over tens of thousands of images (thousands of volumes) without apparent photobleaching. Second, using simulations we show that RCAN enables resolution enhancement equivalent to, or better than, other networks. Third, we exploit RCAN for denoising and resolution improvement in confocal microscopy, enabling ~2.5-fold lateral resolution enhancement using stimulated emission depletion microscopy ground truth. Fourth, we develop methods to improve spatial resolution in structured illumination microscopy using expansion microscopy data as ground truth, achieving improvements of ~1.9-fold laterally and ~3.6-fold axially. Finally, we characterize the limits of denoising and resolution enhancement, suggesting practical benchmarks for evaluation and further enhancement of network performance.

Author Correction: Faster, sharper, and deeper: structured illumination microscopy for biological imaging.

In the version of this Perspective originally published, Fig. 4g included an incorrect inset adapted from a different figure than the main image in the panel. This error has been corrected in the PDF and HTML versions of the paper.

Multiscale fluorescence imaging of living samples.

Fluorescence microscopy is a highly effective tool for interrogating biological structure and function, particularly when imaging across multiple spatiotemporal scales. Here we survey recent innovations and applications in the relatively understudied area of multiscale fluorescence imaging of living samples. We discuss fundamental challenges in live multiscale imaging and describe successful examples that highlight the power of this approach. We attempt to synthesize general strategies from these test cases, aiming to help accelerate progress in this exciting area.

Faster, sharper, and deeper: structured illumination microscopy for biological imaging.

Structured illumination microscopy (SIM) allows rapid, super-resolution (SR) imaging in live specimens. We review recent technical advances in SR-SIM, with emphasis on imaging speed, resolution, and depth. Since its introduction decades ago, the technique has grown to offer myriad implementations, each with its own strengths and weaknesses. We discuss these, aiming to provide a practical guide for biologists and to highlight which approach is best suited to a given application.

Single-shot super-resolution total internal reflection fluorescence microscopy.

We combined instant structured illumination microscopy (iSIM) with total internal reflection fluorescence microscopy (TIRFM) in an approach referred to as instant TIRF-SIM, thereby improving the lateral spatial resolution of TIRFM to 115 ± 13 nm without compromising speed, and enabling imaging frame rates up to 100 Hz over hundreds of time points. We applied instant TIRF-SIM to multiple live samples and achieved rapid, high-contrast super-resolution imaging close to the coverslip surface.

Adaptive optics improves multiphoton super-resolution imaging.

We improve multiphoton structured illumination microscopy using a nonlinear guide star to determine optical aberrations and a deformable mirror to correct them. We demonstrate our method on bead phantoms, cells in collagen gels, nematode larvae and embryos, Drosophila brain, and zebrafish embryos. Peak intensity is increased (up to 40-fold) and resolution recovered (up to 176 ± 10 nm laterally, 729 ± 39 nm axially) at depths ∼250 µm from the coverslip surface.

Correction: Influenza A Virus Assembly Intermediates Fuse in the Cytoplasm.

[This corrects the article DOI: 10.1371/journal.ppat.1003971.].

Influenza a virus assembly intermediates fuse in the cytoplasm.

Reassortment of influenza viral RNA (vRNA) segments in co-infected cells can lead to the emergence of viruses with pandemic potential. Replication of influenza vRNA occurs in the nucleus of infected cells, while progeny virions bud from the plasma membrane. However, the intracellular mechanics of vRNA assembly into progeny virions is not well understood. Here we used recent advances in microscopy to explore vRNA assembly and transport during a productive infection. We visualized four distinct vRNA segments within a single cell using fluorescent in situ hybridization (FISH) and observed that foci containing more than one vRNA segment were found at the external nuclear periphery, suggesting that vRNA segments are not exported to the cytoplasm individually. Although many cytoplasmic foci contain multiple vRNA segments, not all vRNA species are present in every focus, indicating that assembly of all eight vRNA segments does not occur prior to export from the nucleus. To extend the observations made in fixed cells, we used a virus that encodes GFP fused to the viral polymerase acidic (PA) protein (WSN PA-GFP) to explore the dynamics of vRNA assembly in live cells during a productive infection. Since WSN PA-GFP colocalizes with viral nucleoprotein and influenza vRNA segments, we used it as a surrogate for visualizing vRNA transport in 3D and at high speed by inverted selective-plane illumination microscopy. We observed cytoplasmic PA-GFP foci colocalizing and traveling together en route to the plasma membrane. Our data strongly support a model in which vRNA segments are exported from the nucleus as complexes that assemble en route to the plasma membrane through dynamic colocalization events in the cytoplasm.

WormGUIDES: an interactive single cell developmental atlas and tool for collaborative multidimensional data exploration.

BACKGROUND: Imaging and image analysis advances are yielding increasingly complete and complicated records of cellular events in tissues and whole embryos. The ability to follow hundreds to thousands of cells at the individual level demands a spatio-temporal data infrastructure: tools to assemble and collate knowledge about development spatially in a manner analogous to geographic information systems (GIS). Just as GIS indexes items or events based on their spatio-temporal or 4D location on the Earth these tools would organize knowledge based on location within the tissues or embryos. Developmental processes are highly context-specific, but the complexity of the 4D environment in which they unfold is a barrier to assembling an understanding of any particular process from diverse sources of information. In the same way that GIS aids the understanding and use of geo-located large data sets, software can, with a proper frame of reference, allow large biological data sets to be understood spatially. Intuitive tools are needed to navigate the spatial structure of complex tissue, collate large data sets and existing knowledge with this spatial structure and help users derive hypotheses about developmental mechanisms. RESULTS: Toward this goal we have developed WormGUIDES, a mobile application that presents a 4D developmental atlas for Caenorhabditis elegans. The WormGUIDES mobile app enables users to navigate a 3D model depicting the nuclear positions of all cells in the developing embryo. The identity of each cell can be queried with a tap, and community databases searched for available information about that cell. Information about ancestry, fate and gene expression can be used to label cells and craft customized visualizations that highlight cells as potential players in an event of interest. Scenes are easily saved, shared and published to other WormGUIDES users. The mobile app is available for Android and iOS platforms. CONCLUSION: WormGUIDES provides an important tool for examining developmental processes and developing mechanistic hypotheses about their control. Critically, it provides the typical end user with an intuitive interface for developing and sharing custom visualizations of developmental processes. Equally important, because users can select cells based on their position and search for information about them, the app also serves as a spatially organized index into the large body of knowledge available to the C. elegans community online. Moreover, the app can be used to create and publish the result of exploration: interactive content that brings other researchers and students directly to the spatio-temporal point of insight. Ultimately the app will incorporate a detailed time lapse record of cell shape, beginning with neurons. This will add the key ability to navigate and understand the developmental events that result in the coordinated and precise emergence of anatomy, particularly the wiring of the nervous system.

Incoherent structured illumination improves optical sectioning and contrast in multiphoton super-resolution microscopy.

Three-dimensional super-resolution imaging in thick, semi-transparent biological specimens is hindered by light scattering, which increases background and degrades both contrast and optical sectioning. We describe a simple method that mitigates these issues, improving image quality in our recently developed two-photon instant structured illumination microscope without requiring any hardware modifications to the instrument. By exciting the specimen with three laterally-structured, phase-shifted illumination patterns and post-processing the resulting images, we digitally remove both scattered and out-of-focus emissions that would otherwise contaminate our raw data. We demonstrate the improved performance of our approach in biological samples, including pollen grains, primary mouse aortic endothelial cells cultured in a three-dimensional collagen matrix and live tumor-like cell spheroids.

Inverted selective plane illumination microscopy (iSPIM) enables coupled cell identity lineaging and neurodevelopmental imaging in Caenorhabditis elegans.

The Caenorhabditis elegans embryo is a powerful model for studying neural development, but conventional imaging methods are either too slow or phototoxic to take full advantage of this system. To solve these problems, we developed an inverted selective plane illumination microscopy (iSPIM) module for noninvasive high-speed volumetric imaging of living samples. iSPIM is designed as a straightforward add-on to an inverted microscope, permitting conventional mounting of specimens and facilitating SPIM use by development and neurobiology laboratories. iSPIM offers a volumetric imaging rate 30× faster than currently used technologies, such as spinning-disk confocal microscopy, at comparable signal-to-noise ratio. This increased imaging speed allows us to continuously monitor the development of C, elegans embryos, scanning volumes every 2 s for the 14-h period of embryogenesis with no detectable phototoxicity. Collecting ∼25,000 volumes over the entirety of embryogenesis enabled in toto visualization of positions and identities of cell nuclei. By merging two-color iSPIM with automated lineaging techniques we realized two goals: (i) identification of neurons expressing the transcription factor CEH-10/Chx10 and (ii) visualization of their neurodevelopmental dynamics. We found that canal-associated neurons use somal translocation and amoeboid movement as they migrate to their final position in the embryo. We also visualized axon guidance and growth cone dynamics as neurons circumnavigate the nerve ring and reach their targets in the embryo. The high-speed volumetric imaging rate of iSPIM effectively eliminates motion blur from embryo movement inside the egg case, allowing characterization of dynamic neurodevelopmental events that were previously inaccessible.

A radiomics-boosted deep-learning for risk assessment of synchronous peritoneal metastasis in colorectal cancer.

OBJECTIVES: Synchronous colorectal cancer peritoneal metastasis (CRPM) has a poor prognosis. This study aimed to create a radiomics-boosted deep learning model by PET/CT image for risk assessment of synchronous CRPM. METHODS: A total of 220 colorectal cancer (CRC) cases were enrolled in this study. We mapped the feature maps (Radiomic feature maps (RFMs)) of radiomic features across CT and PET image patches by a 2D sliding kernel. Based on ResNet50, a radiomics-boosted deep learning model was trained using PET/CT image patches and RFMs. Besides that, we explored whether the peritumoral region contributes to the assessment of CRPM. In this study, the performance of each model was evaluated by the area under the curves (AUC). RESULTS: The AUCs of the radiomics-boosted deep learning model in the training, internal, external, and all validation datasets were 0.926 (95% confidence interval (CI): 0.874-0.978), 0.897 (95% CI: 0.801-0.994), 0.885 (95% CI: 0.795-0.975), and 0.889 (95% CI: 0.823-0.954), respectively. This model exhibited consistency in the calibration curve, the Delong test and IDI identified it as the most predictive model. CONCLUSIONS: The radiomics-boosted deep learning model showed superior estimated performance in preoperative prediction of synchronous CRPM from pre-treatment PET/CT, offering potential assistance in the development of more personalized treatment methods and follow-up plans. CRITICAL RELEVANCE STATEMENT: The onset of synchronous colorectal CRPM is insidious, and using a radiomics-boosted deep learning model to assess the risk of CRPM before treatment can help make personalized clinical treatment decisions or choose more sensitive follow-up plans. KEY POINTS: Prognosis for patients with CRPM is bleak, and early detection poses challenges. The synergy between radiomics and deep learning proves advantageous in evaluating CRPM. The radiomics-boosted deep-learning model proves valuable in tailoring treatment approaches for CRC patients.

Phase diversity-based wavefront sensing for fluorescence microscopy.

Fluorescence microscopy is an invaluable tool in biology, yet its performance is compromised when the wavefront of light is distorted due to optical imperfections or the refractile nature of the sample. Such optical aberrations can dramatically lower the information content of images by degrading image contrast, resolution, and signal. Adaptive optics (AO) methods can sense and subsequently cancel the aberrated wavefront, but are too complex, inefficient, slow, or expensive for routine adoption by most labs. Here we introduce a rapid, sensitive, and robust wavefront sensing scheme based on phase diversity, a method successfully deployed in astronomy but underused in microscopy. Our method enables accurate wavefront sensing to less than λ/35 root mean square (RMS) error with few measurements, and AO with no additional hardware besides a corrective element. After validating the method with simulations, we demonstrate calibration of a deformable mirror > 100-fold faster than comparable methods (corresponding to wavefront sensing on the ~100 ms scale), and sensing and subsequent correction of severe aberrations (RMS wavefront distortion exceeding λ/2), restoring diffraction-limited imaging on extended biological samples.

Three-dimensional spatio-angular fluorescence microscopy with a polarized dual-view inverted selective-plane illumination microscope (pol-diSPIM).

Polarized fluorescence microscopy is a valuable tool for measuring molecular orientations, but techniques for recovering three-dimensional orientations and positions of fluorescent ensembles are limited. We report a polarized dual-view light-sheet system for determining the three-dimensional orientations and diffraction-limited positions of ensembles of fluorescent dipoles that label biological structures, and we share a set of visualization, histogram, and profiling tools for interpreting these positions and orientations. We model our samples, their excitation, and their detection using coarse-grained representations we call orientation distribution functions (ODFs). We apply ODFs to create physics-informed models of image formation with spatio-angular point-spread and transfer functions. We use theory and experiment to conclude that light-sheet tilting is a necessary part of our design for recovering all three-dimensional orientations. We use our system to extend known two-dimensional results to three dimensions in FM1-43-labelled giant unilamellar vesicles, fast-scarlet-labelled cellulose in xylem cells, and phalloidin-labelled actin in U2OS cells. Additionally, we observe phalloidin-labelled actin in mouse fibroblasts grown on grids of labelled nanowires and identify correlations between local actin alignment and global cell-scale orientation, indicating cellular coordination across length scales.

3D-printed biomimetic scaffolds with precisely controlled and tunable structures guide cell migration and promote regeneration of osteochondral defect.

Untreated osteochondral defects will develop into osteoarthritis, affecting patients' quality of life. Since articular cartilage and subchondral bone exhibit distinct biological characteristics, repairing osteochondral defects remains a major challenge. Previous studies have tried to fabricate multilayer scaffolds with traditional methods or 3D printing technology. However, the efficacy is unsatisfactory because of poor control over internal structures or a lack of integrity between adjacent layers, severely compromising repair outcomes. Therefore, there is a need for a biomimetic scaffold that can simultaneously boost osteochondral defect regeneration in both structure and function. Herein, an integrated bilayer scaffold with precisely controlled structures is successfully 3D-printed in one step via digital light processing (DLP) technology. The upper layer has both 'lotus- and radial-' distribution pores, and the bottom layer has 'lotus-' pores to guide and facilitate the migration of chondrocytes and bone marrow mesenchymal stem cells, respectively, to the defect area. Tuning pore sizes could modulate the mechanical properties of scaffolds easily. Results show that 3D-printed porous structures allow significantly more cells to infiltrate into the area of 'lotus- and radial-' distribution pores during cell migration assay, subcutaneous implantation, andin situtransplantation, which are essential for osteochondral repair. Transplantation of this 3D-printed bilayer scaffold exhibits a promising osteochondral repair effect in rabbits. Incorporation of Kartogenin into the upper layer of scaffolds further induces better cartilage formation. Combining small molecules/drugs and precisely size-controlled and layer-specific porous structure via DLP technology, this 3D-printed bilayer scaffold is expected to be a potential strategy for osteochondral regeneration.

Augmented Super-Resolution Radial Fluctuations (aSRRF) Pushing the Limits of Structured Illumination Microscopy.

Structured illumination microscopy (SIM) is a versatile super-resolution technique known for its compatibility with a wide range of probes and fast implementation. While 3D SIM is capable of achieving a spatial resolution of ∼120 nm laterally and ∼300 nm axially, attempting to further enhance the resolution through methods such as nonlinear SIM or 4-beam SIM introduces complexities in optical configurations, increased phototoxicity, and reduced temporal resolution. Here, we have developed a novel method that combines SIM with augmented super-resolution radial fluctuations (aSRRF) utilizing a single image through image augmentation. By applying aSRRF reconstruction to SIM images, we can enhance the SIM resolution to ∼50 nm isotopically, without requiring any modifications to the optical system or sample acquisition process. Additionaly, we have incorporated the aSRRF approach into an ImageJ plugin and demonstrated its versatility across various fluorescence microscopy images, showcasing a remarkable two-fold resolution increase.