Automated segmentation of cell organelles in volume electron microscopy using deep learning.

Recent advances in computing power triggered the use of artificial intelligence in image analysis in life sciences. To train these algorithms, a large enough set of certified labeled data is required. The trained neural network is then capable of producing accurate instance segmentation results that will then need to be re-assembled into the original dataset: the entire process requires substantial expertise and time to achieve quantifiable results. To speed-up the process, from cell organelle detection to quantification across electron microscopy modalities, we propose a deep-learning based approach for fast automatic outline segmentation (FAMOUS), that involves organelle detection combined with image morphology, and 3D meshing to automatically segment, visualize and quantify cell organelles within volume electron microscopy datasets. From start to finish, FAMOUS provides full segmentation results within a week on previously unseen datasets. FAMOUS was showcased on a HeLa cell dataset acquired using a focused ion beam scanning electron microscope, and on yeast cells acquired by transmission electron tomography. RESEARCH HIGHLIGHTS: Introducing a rapid, multimodal machine-learning workflow for the automatic segmentation of 3D cell organelles. Successfully applied to a variety of volume electron microscopy datasets and cell lines. Outperforming manual segmentation methods in time and accuracy. Enabling high-throughput quantitative cell biology.

Femtosecond laser preparation of resin embedded samples for correlative microscopy workflows in life sciences.

Correlative multimodal imaging is a useful approach to investigate complex structural relations in life sciences across multiple scales. For these experiments, sample preparation workflows that are compatible with multiple imaging techniques must be established. In one such implementation, a fluorescently labeled region of interest in a biological soft tissue sample can be imaged with light microscopy before staining the specimen with heavy metals, enabling follow-up higher resolution structural imaging at the targeted location, bringing context where it is required. Alternatively, or in addition to fluorescence imaging, other microscopy methods, such as synchrotron x-ray computed tomography with propagation-based phase contrast or serial blockface scanning electron microscopy, might also be applied. When combining imaging techniques across scales, it is common that a volumetric region of interest (ROI) needs to be carved from the total sample volume before high resolution imaging with a subsequent technique can be performed. In these situations, the overall success of the correlative workflow depends on the precise targeting of the ROI and the trimming of the sample down to a suitable dimension and geometry for downstream imaging. Here, we showcase the utility of a femtosecond laser (fs laser) device to prepare microscopic samples (1) of an optimized geometry for synchrotron x-ray tomography as well as (2) for volume electron microscopy applications and compatible with correlative multimodal imaging workflows that link both imaging modalities.

Ectocytosis renders T cell receptor signaling self-limiting at the immune synapse.

Cytotoxic T lymphocytes (CTLs) kill virus-infected and cancer cells through T cell receptor (TCR) recognition. How CTLs terminate signaling and disengage to allow serial killing has remained a mystery. TCR activation triggers membrane specialization within the immune synapse, including the production of diacylglycerol (DAG), a lipid that can induce negative membrane curvature. We found that activated TCRs were shed into DAG-enriched ectosomes at the immune synapse rather than internalized through endocytosis, suggesting that DAG may contribute to the outward budding required for ectocytosis. Budding ectosomes were endocytosed directly by target cells, thereby terminating TCR signaling and simultaneously disengaging the CTL from the target cell to allow serial killing. Thus, ectocytosis renders TCR signaling self-limiting.

Crosshair, semi-automated targeting for electron microscopy with a motorised ultramicrotome.

Volume electron microscopy (EM) is a time-consuming process - often requiring weeks or months of continuous acquisition for large samples. In order to compare the ultrastructure of a number of individuals or conditions, acquisition times must therefore be reduced. For resin-embedded samples, one solution is to selectively target smaller regions of interest by trimming with an ultramicrotome. This is a difficult and labour-intensive process, requiring manual positioning of the diamond knife and sample, and much time and training to master. Here, we have developed a semi-automated workflow for targeting with a modified ultramicrotome. We adapted two recent commercial systems to add motors for each rotational axis (and also each translational axis for one system), allowing precise and automated movement. We also developed a user-friendly software to convert X-ray images of resin-embedded samples into angles and cutting depths for the ultramicrotome. This is provided as an open-source Fiji plugin called Crosshair. This workflow is demonstrated by targeting regions of interest in a series of Platynereis dumerilii samples.

Pathological structural conversion of α-synuclein at the mitochondria induces neuronal toxicity.

Aggregation of alpha-synuclein (α-Syn) drives Parkinson's disease (PD), although the initial stages of self-assembly and structural conversion have not been directly observed inside neurons. In this study, we tracked the intracellular conformational states of α-Syn using a single-molecule Förster resonance energy transfer (smFRET) biosensor, and we show here that α-Syn converts from a monomeric state into two distinct oligomeric states in neurons in a concentration-dependent and sequence-specific manner. Three-dimensional FRET-correlative light and electron microscopy (FRET-CLEM) revealed that intracellular seeding events occur preferentially on membrane surfaces, especially at mitochondrial membranes. The mitochondrial lipid cardiolipin triggers rapid oligomerization of A53T α-Syn, and cardiolipin is sequestered within aggregating lipid-protein complexes. Mitochondrial aggregates impair complex I activity and increase mitochondrial reactive oxygen species (ROS) generation, which accelerates the oligomerization of A53T α-Syn and causes permeabilization of mitochondrial membranes and cell death. These processes were also observed in induced pluripotent stem cell (iPSC)-derived neurons harboring A53T mutations from patients with PD. Our study highlights a mechanism of de novo α-Syn oligomerization at mitochondrial membranes and subsequent neuronal toxicity.

DNGR-1-tracing marks an ependymal cell subset with damage-responsive neural stem cell potential.

Cells with latent stem ability can contribute to mammalian tissue regeneration after damage. Whether the central nervous system (CNS) harbors such cells remains controversial. Here, we report that DNGR-1 lineage tracing in mice identifies an ependymal cell subset, wherein resides latent regenerative potential. We demonstrate that DNGR-1-lineage-traced ependymal cells arise early in embryogenesis (E11.5) and subsequently spread across the lining of cerebrospinal fluid (CSF)-filled compartments to form a contiguous sheet from the brain to the end of the spinal cord. In the steady state, these DNGR-1-traced cells are quiescent, committed to their ependymal cell fate, and do not contribute to neuronal or glial lineages. However, trans-differentiation can be induced in adult mice by CNS injury or in vitro by culture with suitable factors. Our findings highlight previously unappreciated ependymal cell heterogeneity and identify across the entire CNS an ependymal cell subset wherein resides damage-responsive neural stem cell potential.

Functional and multiscale 3D structural investigation of brain tissue through correlative in vivo physiology, synchrotron microtomography and volume electron microscopy.

Understanding the function of biological tissues requires a coordinated study of physiology and structure, exploring volumes that contain complete functional units at a detail that resolves the relevant features. Here, we introduce an approach to address this challenge: Mouse brain tissue sections containing a region where function was recorded using in vivo 2-photon calcium imaging were stained, dehydrated, resin-embedded and imaged with synchrotron X-ray computed tomography with propagation-based phase contrast (SXRT). SXRT provided context at subcellular detail, and could be followed by targeted acquisition of multiple volumes using serial block-face electron microscopy (SBEM). In the olfactory bulb, combining SXRT and SBEM enabled disambiguation of in vivo-assigned regions of interest. In the hippocampus, we found that superficial pyramidal neurons in CA1a displayed a larger density of spine apparati than deeper ones. Altogether, this approach can enable a functional and structural investigation of subcellular features in the context of cells and tissues.

Volume electron microscopy.

Life exists in three dimensions, but until the turn of the century most electron microscopy methods provided only 2D image data. Recently, electron microscopy techniques capable of delving deep into the structure of cells and tissues have emerged, collectively called volume electron microscopy (vEM). Developments in vEM have been dubbed a quiet revolution as the field evolved from established transmission and scanning electron microscopy techniques, so early publications largely focused on the bioscience applications rather than the underlying technological breakthroughs. However, with an explosion in the uptake of vEM across the biosciences and fast-paced advances in volume, resolution, throughput and ease of use, it is timely to introduce the field to new audiences. In this Primer, we introduce the different vEM imaging modalities, the specialized sample processing and image analysis pipelines that accompany each modality and the types of information revealed in the data. We showcase key applications in the biosciences where vEM has helped make breakthrough discoveries and consider limitations and future directions. We aim to show new users how vEM can support discovery science in their own research fields and inspire broader uptake of the technology, finally allowing its full adoption into mainstream biological imaging.

Deep learning for automatic segmentation of the nuclear envelope in electron microscopy data, trained with volunteer segmentations.

Advancements in volume electron microscopy mean it is now possible to generate thousands of serial images at nanometre resolution overnight, yet the gold standard approach for data analysis remains manual segmentation by an expert microscopist, resulting in a critical research bottleneck. Although some machine learning approaches exist in this domain, we remain far from realizing the aspiration of a highly accurate, yet generic, automated analysis approach, with a major obstacle being lack of sufficient high-quality ground-truth data. To address this, we developed a novel citizen science project, Etch a Cell, to enable volunteers to manually segment the nuclear envelope (NE) of HeLa cells imaged with serial blockface scanning electron microscopy. We present our approach for aggregating multiple volunteer annotations to generate a high-quality consensus segmentation and demonstrate that data produced exclusively by volunteers can be used to train a highly accurate machine learning algorithm for automatic segmentation of the NE, which we share here, in addition to our archived benchmark data.

Semantic segmentation of HeLa cells: An objective comparison between one traditional algorithm and four deep-learning architectures.

The quantitative study of cell morphology is of great importance as the structure and condition of cells and their structures can be related to conditions of health or disease. The first step towards that, is the accurate segmentation of cell structures. In this work, we compare five approaches, one traditional and four deep-learning, for the semantic segmentation of the nuclear envelope of cervical cancer cells commonly known as HeLa cells. Images of a HeLa cancer cell were semantically segmented with one traditional image-processing algorithm and four three deep learning architectures: VGG16, ResNet18, Inception-ResNet-v2, and U-Net. Three hundred slices, each 2000 × 2000 pixels, of a HeLa Cell were acquired with Serial Block Face Scanning Electron Microscopy. The first three deep learning architectures were pre-trained with ImageNet and then fine-tuned with transfer learning. The U-Net architecture was trained from scratch with 36, 000 training images and labels of size 128 × 128. The image-processing algorithm followed a pipeline of several traditional steps like edge detection, dilation and morphological operators. The algorithms were compared by measuring pixel-based segmentation accuracy and Jaccard index against a labelled ground truth. The results indicated a superior performance of the traditional algorithm (Accuracy = 99%, Jaccard = 93%) over the deep learning architectures: VGG16 (93%, 90%), ResNet18 (94%, 88%), Inception-ResNet-v2 (94%, 89%), and U-Net (92%, 56%).

M. tuberculosis infection of human iPSC-derived macrophages reveals complex membrane dynamics during xenophagy evasion.

Xenophagy is an important cellular defence mechanism against cytosol-invading pathogens, such as Mycobacterium tuberculosis (Mtb). Activation of xenophagy in macrophages targets Mtb to autophagosomes; however, how Mtb is targeted to autophagosomes in human macrophages at a high spatial and temporal resolution is unknown. Here, we use human induced pluripotent stem cell-derived macrophages (iPSDMs) to study the human macrophage response to Mtb infection and the role of the ESX-1 type VII secretion system. Using RNA-seq, we identify ESX-1-dependent transcriptional responses in iPSDMs after infection with Mtb. This analysis revealed differential inflammatory responses and dysregulated pathways such as eukaryotic initiation factor 2 (eIF2) signalling and protein ubiquitylation. Moreover, live-cell imaging revealed that Mtb infection in human macrophages induces dynamic ESX-1-dependent, LC3B-positive tubulovesicular autophagosomes (LC3-TVS). Through a correlative live-cell and focused ion beam scanning electron microscopy (FIB SEM) approach, we show that upon phagosomal rupture, Mtb induces the formation of LC3-TVS, from which the bacterium is able to escape to reside in the cytosol. Thus, iPSDMs represent a valuable model for studying spatiotemporal dynamics of human macrophage-Mtb interactions, and Mtb is able to evade capture by autophagic compartments.

Enteric glia as a source of neural progenitors in adult zebrafish.

The presence and identity of neural progenitors in the enteric nervous system (ENS) of vertebrates is a matter of intense debate. Here, we demonstrate that the non-neuronal ENS cell compartment of teleosts shares molecular and morphological characteristics with mammalian enteric glia but cannot be identified by the expression of canonical glial markers. However, unlike their mammalian counterparts, which are generally quiescent and do not undergo neuronal differentiation during homeostasis, we show that a relatively high proportion of zebrafish enteric glia proliferate under physiological conditions giving rise to progeny that differentiate into enteric neurons. We also provide evidence that, similar to brain neural stem cells, the activation and neuronal differentiation of enteric glia are regulated by Notch signalling. Our experiments reveal remarkable similarities between enteric glia and brain neural stem cells in teleosts and open new possibilities for use of mammalian enteric glia as a potential source of neurons to restore the activity of intestinal neural circuits compromised by injury or disease.

The zebrafish as a novel model for the in vivo study of Toxoplasma gondii replication and interaction with macrophages.

Toxoplasma gondii is an obligate intracellular parasite capable of invading any nucleated cell. Three main clonal lineages (type I, II, III) exist and murine models have driven the understanding of general and strain-specific immune mechanisms underlying Toxoplasma infection. However, murine models are limited for studying parasite-leukocyte interactions in vivo, and discrepancies exist between cellular immune responses observed in mouse versus human cells. Here, we developed a zebrafish infection model to study the innate immune response to Toxoplasma in vivo By infecting the zebrafish hindbrain ventricle, and using high-resolution microscopy techniques coupled with computer vision-driven automated image analysis, we reveal that Toxoplasma invades brain cells and replicates inside a parasitophorous vacuole to which type I and III parasites recruit host cell mitochondria. We also show that type II and III strains maintain a higher infectious burden than type I strains. To understand how parasites are cleared in vivo, we further analyzed Toxoplasma-macrophage interactions using time-lapse microscopy and three-dimensional correlative light and electron microscopy (3D CLEM). Time-lapse microscopy revealed that macrophages are recruited to the infection site and play a key role in Toxoplasma control. High-resolution 3D CLEM revealed parasitophorous vacuole breakage in brain cells and macrophages in vivo, suggesting that cell-intrinsic mechanisms may be used to destroy the intracellular niche of tachyzoites. Together, our results demonstrate in vivo control of Toxoplasma by macrophages, and highlight the possibility that zebrafish may be further exploited as a novel model system for discoveries within the field of parasite immunity.This article has an associated First Person interview with the first author of the paper.

Differential requirements for cyclase-associated protein (CAP) in actin-dependent processes of Toxoplasma gondii.

Toxoplasma gondii contains a limited subset of actin binding proteins. Here we show that the putative actin regulator cyclase-associated protein (CAP) is present in two different isoforms and its deletion leads to significant defects in some but not all actin dependent processes. We observe defects in cell-cell communication, daughter cell orientation and the juxtanuclear accumulation of actin, but only modest defects in synchronicity of division and no defect in the replication of the apicoplast. 3D electron microscopy reveals that loss of CAP results in a defect in formation of a normal central residual body, but parasites remain connected within the vacuole. This dissociates synchronicity of division and parasite rosetting and reveals that establishment and maintenance of the residual body may be more complex than previously thought. These results highlight the different spatial requirements for F-actin regulation in Toxoplasma which appear to be achieved by partially overlapping functions of actin regulators.

Origins of Enterovirus Replication Organelles Established by Whole-Cell Electron Microscopy.

Enterovirus genome replication occurs at virus-induced structures derived from cellular membranes and lipids. However, the origin of these replication organelles (ROs) remains uncertain. Ultrastructural evidence of the membrane donor is lacking, suggesting that the sites of its transition into ROs are rare or fleeting. To overcome this challenge, we combined live-cell imaging and serial block-face scanning electron microscopy of whole cells to capture emerging enterovirus ROs. The first foci of fluorescently labeled viral protein correlated with ROs connected to the endoplasmic reticulum (ER) and preceded the appearance of ROs stemming from the trans-Golgi network. Whole-cell data sets further revealed striking contact regions between ROs and lipid droplets that may represent a route for lipid shuttling to facilitate RO proliferation and genome replication. Our data provide direct evidence that enteroviruses use ER and then Golgi membranes to initiate RO formation, demonstrating the remarkable flexibility with which enteroviruses usurp cellular organelles.IMPORTANCE Enteroviruses are causative agents of a range of human diseases. The replication of these viruses within cells relies on specialized membranous structures termed replication organelles (ROs) that form during infection but whose origin remains elusive. To capture the emergence of enterovirus ROs, we use correlative light and serial block-face scanning electron microscopy, a powerful method to pinpoint rare events in their whole-cell ultrastructural context. RO biogenesis was found to occur first at ER and then at Golgi membranes. Extensive contacts were found between early ROs and lipid droplets (LDs), which likely serve to provide LD-derived lipids required for replication. Together, these data establish the dual origin of enterovirus ROs and the chronology of their biogenesis at different supporting cellular membranes.

Segmentation and Modelling of the Nuclear Envelope of HeLa Cells Imaged with Serial Block Face Scanning Electron Microscopy.

This paper describes an unsupervised algorithm, which segments the nuclear envelope of HeLa cells imaged by Serial Block Face Scanning Electron Microscopy. The algorithm exploits the variations of pixel intensity in different cellular regions by calculating edges, which are then used to generate superpixels. The superpixels are morphologically processed and those that correspond to the nuclear region are selected through the analysis of size, position, and correspondence with regions detected in neighbouring slices. The nuclear envelope is segmented from the nuclear region. The three-dimensional segmented nuclear envelope is then modelled against a spheroid to create a two-dimensional (2D) surface. The 2D surface summarises the complex 3D shape of the nuclear envelope and allows the extraction of metrics that may be relevant to characterise the nature of cells. The algorithm was developed and validated on a single cell and tested in six separate cells, each with 300 slices of 2000 × 2000 pixels. Ground truth was available for two of these cells, i.e., 600 hand-segmented slices. The accuracy of the algorithm was evaluated with two similarity metrics: Jaccard Similarity Index and Mean Hausdorff distance. Jaccard values of the first/second segmentation were 93%/90% for the whole cell, and 98%/94% between slices 75 and 225, as the central slices of the nucleus are more regular than those on the extremes. Mean Hausdorff distances were 9/17 pixels for the whole cells and 4/13 pixels for central slices. One slice was processed in approximately 8 s and a whole cell in 40 min. The algorithm outperformed active contours in both accuracy and time.

The Rényi divergence enables accurate and precise cluster analysis for localization microscopy.

Motivation: Clustering analysis is a key technique for quantitatively characterizing structures in localization microscopy images. To build up accurate information about biological structures, it is critical that the quantification is both accurate (close to the ground truth) and precise (has small scatter and is reproducible). Results: Here, we describe how the Rényi divergence can be used for cluster radius measurements in localization microscopy data. We demonstrate that the Rényi divergence can operate with high levels of background and provides results which are more accurate than Ripley's functions, Voronoi tesselation or DBSCAN. Availability and implementation: The data supporting this research and the software described are accessible at the following site: https://dx.doi.org/10.18742/RDM01-316. Correspondence and requests for materials should be addressed to the corresponding author. Supplementary information: Supplementary data are available at Bioinformatics online.

Correlative super-resolution fluorescence and electron microscopy using conventional fluorescent proteins in vacuo.

Super-resolution light microscopy, correlative light and electron microscopy, and volume electron microscopy are revolutionising the way in which biological samples are examined and understood. Here, we combine these approaches to deliver super-accurate correlation of fluorescent proteins to cellular structures. We show that YFP and GFP have enhanced blinking properties when embedded in acrylic resin and imaged under partial vacuum, enabling in vacuo single molecule localisation microscopy. In conventional section-based correlative microscopy experiments, the specimen must be moved between imaging systems and/or further manipulated for optimal viewing. These steps can introduce undesirable alterations in the specimen, and complicate correlation between imaging modalities. We avoided these issues by using a scanning electron microscope with integrated optical microscope to acquire both localisation and electron microscopy images, which could then be precisely correlated. Collecting data from ultrathin sections also improved the axial resolution and signal-to-noise ratio of the raw localisation microscopy data. Expanding data collection across an array of sections will allow 3-dimensional correlation over unprecedented volumes. The performance of this technique is demonstrated on vaccinia virus (with YFP) and diacylglycerol in cellular membranes (with GFP).