Deep learning for automatic segmentation of the nuclear envelope in electron microscopy data, trained with volunteer segmentations.

Advancements in volume electron microscopy mean it is now possible to generate thousands of serial images at nanometre resolution overnight, yet the gold standard approach for data analysis remains manual segmentation by an expert microscopist, resulting in a critical research bottleneck. Although some machine learning approaches exist in this domain, we remain far from realizing the aspiration of a highly accurate, yet generic, automated analysis approach, with a major obstacle being lack of sufficient high-quality ground-truth data. To address this, we developed a novel citizen science project, Etch a Cell, to enable volunteers to manually segment the nuclear envelope (NE) of HeLa cells imaged with serial blockface scanning electron microscopy. We present our approach for aggregating multiple volunteer annotations to generate a high-quality consensus segmentation and demonstrate that data produced exclusively by volunteers can be used to train a highly accurate machine learning algorithm for automatic segmentation of the NE, which we share here, in addition to our archived benchmark data.

Online citizen science with the Zooniverse for analysis of biological volumetric data.

Public participation in research, also known as citizen science, is being increasingly adopted for the analysis of biological volumetric data. Researchers working in this domain are applying online citizen science as a scalable distributed data analysis approach, with recent research demonstrating that non-experts can productively contribute to tasks such as the segmentation of organelles in volume electron microscopy data. This, alongside the growing challenge to rapidly process the large amounts of biological volumetric data now routinely produced, means there is increasing interest within the research community to apply online citizen science for the analysis of data in this context. Here, we synthesise core methodological principles and practices for applying citizen science for analysis of biological volumetric data. We collate and share the knowledge and experience of multiple research teams who have applied online citizen science for the analysis of volumetric biological data using the Zooniverse platform ( www.zooniverse.org ). We hope this provides inspiration and practical guidance regarding how contributor effort via online citizen science may be usefully applied in this domain.

Bridging imaging users to imaging analysis - A community survey.

The 'Bridging Imaging Users to Imaging Analysis' survey was conducted in 2022 by the Center for Open Bioimage Analysis (COBA), BioImaging North America (BINA) and the Royal Microscopical Society Data Analysis in Imaging Section (RMS DAIM) to understand the needs of the imaging community. Through multichoice and open-ended questions, the survey inquired about demographics, image analysis experiences, future needs and suggestions on the role of tool developers and users. Participants of the survey were from diverse roles and domains of the life and physical sciences. To our knowledge, this is the first attempt to survey cross-community to bridge knowledge gaps between physical and life sciences imaging. Survey results indicate that respondents' overarching needs are documentation, detailed tutorials on the usage of image analysis tools, user-friendly intuitive software, and better solutions for segmentation, ideally in a format tailored to their specific use cases. The tool creators suggested the users familiarise themselves with the fundamentals of image analysis, provide constant feedback and report the issues faced during image analysis while the users would like more documentation and an emphasis on tool friendliness. Regardless of the computational experience, there is a strong preference for 'written tutorials' to acquire knowledge on image analysis. We also observed that the interest in having 'office hours' to get an expert opinion on their image analysis methods has increased over the years. The results also showed less-than-expected usage of online discussion forums in the imaging community for solving image analysis problems. Surprisingly, we also observed a decreased interest among the survey respondents in deep/machine learning despite the increasing adoption of artificial intelligence in biology. In addition, the community suggests the need for a common repository for the available image analysis tools and their applications. The opinions and suggestions of the community, released here in full, will help the image analysis tool creation and education communities to design and deliver the resources accordingly.

3D correlative light and electron microscopy of cultured cells using serial blockface scanning electron microscopy.

The processes of life take place in multiple dimensions, but imaging these processes in even three dimensions is challenging. Here, we describe a workflow for 3D correlative light and electron microscopy (CLEM) of cell monolayers using fluorescence microscopy to identify and follow biological events, combined with serial blockface scanning electron microscopy to analyse the underlying ultrastructure. The workflow encompasses all steps from cell culture to sample processing, imaging strategy, and 3D image processing and analysis. We demonstrate successful application of the workflow to three studies, each aiming to better understand complex and dynamic biological processes, including bacterial and viral infections of cultured cells and formation of entotic cell-in-cell structures commonly observed in tumours. Our workflow revealed new insight into the replicative niche of Mycobacterium tuberculosis in primary human lymphatic endothelial cells, HIV-1 in human monocyte-derived macrophages, and the composition of the entotic vacuole. The broad application of this 3D CLEM technique will make it a useful addition to the correlative imaging toolbox for biomedical research.

PolNet: A Tool to Quantify Network-Level Cell Polarity and Blood Flow in Vascular Remodeling.

In this article, we present PolNet, an open-source software tool for the study of blood flow and cell-level biological activity during vessel morphogenesis. We provide an image acquisition, segmentation, and analysis protocol to quantify endothelial cell polarity in entire in vivo vascular networks. In combination, we use computational fluid dynamics to characterize the hemodynamics of the vascular networks under study. The tool enables, to our knowledge for the first time, a network-level analysis of polarity and flow for individual endothelial cells. To date, PolNet has proven invaluable for the study of endothelial cell polarization and migration during vascular patterning, as demonstrated by two recent publications. Additionally, the tool can be easily extended to correlate blood flow with other experimental observations at the cellular/molecular level. We release the source code of our tool under the Lesser General Public License.

Dynamic endothelial cell rearrangements drive developmental vessel regression.

Patterning of functional blood vessel networks is achieved by pruning of superfluous connections. The cellular and molecular principles of vessel regression are poorly understood. Here we show that regression is mediated by dynamic and polarized migration of endothelial cells, representing anastomosis in reverse. Establishing and analyzing the first axial polarity map of all endothelial cells in a remodeling vascular network, we propose that balanced movement of cells maintains the primitive plexus under low shear conditions in a metastable dynamic state. We predict that flow-induced polarized migration of endothelial cells breaks symmetry and leads to stabilization of high flow/shear segments and regression of adjacent low flow/shear segments.

Correction: dynamic endothelial cell rearrangements drive developmental vessel regression.

[This corrects the article DOI: 10.1371/journal.pbio.1002125.].

Neutral processes and species sorting in benthic microalgal community assembly: effects of tidal resuspension.

Benthic microalgae (BMA) provide vital food resources for heterotrophs and stabilize sediments with their extracellular secretions. A central goal in ecology is to understand how processes such as species interactions and dispersal, contribute to observed patterns of species abundance and distribution. Our objectives were to assess the effects of sediment resuspension on microalgal community structure. We tested whether taxa-abundance distributions could be predicted using neutral community models (NCMs) and also specific hypotheses about passive migration: (i) As migration decreases in sediment patches, BMA α-diversity will decrease, and (ii) As migration decreases, BMA community dissimilarity (ß-diversity) will increase. Co-occurrence indices (checkerboard score and variance ratio) were also computed to test for deterministic factors, such as competition and niche differentiation, in shaping communities. Two intertidal sites (mudflat and sand bar) differing in resuspension regime were sampled throughout the tidal cycle. Fluorometry and denaturing gradient gel electrophoresis were utilized to investigate diatom community structure. Observed taxa-abundances fit those predicted from NCMs reasonably well (R2 of 0.68-0.93), although comparisons of observed local communities to artificial randomly assembled communities rejected the null hypothesis that diatom communities were assembled solely by stochastic processes. No co-occurrence tests indicated a significant role for competitive exclusion or niche partitioning in microalgal community assembly. In general, predictions about relationships between migration and species diversity were supported for local community dynamics. BMA at low tide (lowest migration) exhibited reduced α-diversity as compared to periods of immersion at both mudflat and sand bar sites. ß-diversity was higher during low tide emersion on the mudflat, but did not differ temporally at the sand bar site. In between-site metacommunity comparisons, low- and high-resuspension sites exhibited distinct community compositions while the low-energy mudflats contained higher microalgal biomass and greater α-diversity. To our knowledge this is the first study to test the relevance of neutral processes in structuring marine microalgal communities. Our results demonstrate a prominent role for stochastic factors in structuring local BMA community assembly, although unidentified nonrandom processes also appear to play some role. High passive migration, in particular, appears to help maintain species diversity and structure communities in both sand and muddy habitats.

Bridging Imaging Users to Imaging Analysis - A community survey.

The "Bridging Imaging Users to Imaging Analysis" survey was conducted in 2022 by the Center for Open Bioimage Analysis (COBA), Bioimaging North America (BINA), and the Royal Microscopical Society Data Analysis in Imaging Section (RMS DAIM) to understand the needs of the imaging community. Through multi-choice and open-ended questions, the survey inquired about demographics, image analysis experiences, future needs, and suggestions on the role of tool developers and users. Participants of the survey were from diverse roles and domains of the life and physical sciences. To our knowledge, this is the first attempt to survey cross-community to bridge knowledge gaps between physical and life sciences imaging. Survey results indicate that respondents' overarching needs are documentation, detailed tutorials on the usage of image analysis tools, user-friendly intuitive software, and better solutions for segmentation, ideally in a format tailored to their specific use cases. The tool creators suggested the users familiarize themselves with the fundamentals of image analysis, provide constant feedback, and report the issues faced during image analysis while the users would like more documentation and an emphasis on tool friendliness. Regardless of the computational experience, there is a strong preference for 'written tutorials' to acquire knowledge on image analysis. We also observed that the interest in having 'office hours' to get an expert opinion on their image analysis methods has increased over the years. In addition, the community suggests the need for a common repository for the available image analysis tools and their applications. The opinions and suggestions of the community, released here in full, will help the image analysis tool creation and education communities to design and deliver the resources accordingly.

Current Progress and Challenges in Large-Scale 3D Mitochondria Instance Segmentation.

In this paper, we present the results of the MitoEM challenge on mitochondria 3D instance segmentation from electron microscopy images, organized in conjunction with the IEEE-ISBI 2021 conference. Our benchmark dataset consists of two large-scale 3D volumes, one from human and one from rat cortex tissue, which are 1,986 times larger than previously used datasets. At the time of paper submission, 257 participants had registered for the challenge, 14 teams had submitted their results, and six teams participated in the challenge workshop. Here, we present eight top-performing approaches from the challenge participants, along with our own baseline strategies. Posterior to the challenge, annotation errors in the ground truth were corrected without altering the final ranking. Additionally, we present a retrospective evaluation of the scoring system which revealed that: 1) challenge metric was permissive with the false positive predictions; and 2) size-based grouping of instances did not correctly categorize mitochondria of interest. Thus, we propose a new scoring system that better reflects the correctness of the segmentation results. Although several of the top methods are compared favorably to our own baselines, substantial errors remain unsolved for mitochondria with challenging morphologies. Thus, the challenge remains open for submission and automatic evaluation, with all volumes available for download.

Sargasso Sea bacterioplankton community structure and drivers of variance as revealed by DNA metabarcoding analysis.

Marine microbes provide the backbone for pelagic ecosystems by cycling and fixing nutrients and establishing the base of food webs. Microbial communities are often assumed to be highly connected and genetically mixed, with localized environmental filters driving minor changes in structure. Our study applied high-throughput Illumina 16S ribosomal RNA gene amplicon sequencing on whole-community bacterial samples to characterize geographic, environmental, and stochastic drivers of community diversity. DNA was extracted from seawater collected from the surface (N = 18) and at depth just below the deep chlorophyll-a maximum (DCM mean depth = 115.4 m; N = 22) in the Sargasso Sea and adjacent oceanographic regions. Discrete bacterioplankton assemblages were observed at varying depths in the North Sargasso Sea, with a signal for distance-decay of bacterioplankton community similarity found only in surface waters. Bacterial communities from different oceanic regions could be distinguished statistically but exhibited a low magnitude of divergence. Redundancy analysis identified temperature as the key environmental variable correlated with community structuring. The effect of dispersal limitation was weak, while variation partitioning and neutral community modeling demonstrated stochastic processes influencing the communities. This study advances understanding of microbial biogeography in the pelagic ocean and highlights the use of high-throughput sequencing methods in studying microbial community structure.

Crosshair, semi-automated targeting for electron microscopy with a motorised ultramicrotome.

Volume electron microscopy (EM) is a time-consuming process - often requiring weeks or months of continuous acquisition for large samples. In order to compare the ultrastructure of a number of individuals or conditions, acquisition times must therefore be reduced. For resin-embedded samples, one solution is to selectively target smaller regions of interest by trimming with an ultramicrotome. This is a difficult and labour-intensive process, requiring manual positioning of the diamond knife and sample, and much time and training to master. Here, we have developed a semi-automated workflow for targeting with a modified ultramicrotome. We adapted two recent commercial systems to add motors for each rotational axis (and also each translational axis for one system), allowing precise and automated movement. We also developed a user-friendly software to convert X-ray images of resin-embedded samples into angles and cutting depths for the ultramicrotome. This is provided as an open-source Fiji plugin called Crosshair. This workflow is demonstrated by targeting regions of interest in a series of Platynereis dumerilii samples.

Democratising deep learning for microscopy with ZeroCostDL4Mic.

Deep Learning (DL) methods are powerful analytical tools for microscopy and can outperform conventional image processing pipelines. Despite the enthusiasm and innovations fuelled by DL technology, the need to access powerful and compatible resources to train DL networks leads to an accessibility barrier that novice users often find difficult to overcome. Here, we present ZeroCostDL4Mic, an entry-level platform simplifying DL access by leveraging the free, cloud-based computational resources of Google Colab. ZeroCostDL4Mic allows researchers with no coding expertise to train and apply key DL networks to perform tasks including segmentation (using U-Net and StarDist), object detection (using YOLOv2), denoising (using CARE and Noise2Void), super-resolution microscopy (using Deep-STORM), and image-to-image translation (using Label-free prediction - fnet, pix2pix and CycleGAN). Importantly, we provide suitable quantitative tools for each network to evaluate model performance, allowing model optimisation. We demonstrate the application of the platform to study multiple biological processes.

Semantic segmentation of HeLa cells: An objective comparison between one traditional algorithm and four deep-learning architectures.

The quantitative study of cell morphology is of great importance as the structure and condition of cells and their structures can be related to conditions of health or disease. The first step towards that, is the accurate segmentation of cell structures. In this work, we compare five approaches, one traditional and four deep-learning, for the semantic segmentation of the nuclear envelope of cervical cancer cells commonly known as HeLa cells. Images of a HeLa cancer cell were semantically segmented with one traditional image-processing algorithm and four three deep learning architectures: VGG16, ResNet18, Inception-ResNet-v2, and U-Net. Three hundred slices, each 2000 × 2000 pixels, of a HeLa Cell were acquired with Serial Block Face Scanning Electron Microscopy. The first three deep learning architectures were pre-trained with ImageNet and then fine-tuned with transfer learning. The U-Net architecture was trained from scratch with 36, 000 training images and labels of size 128 × 128. The image-processing algorithm followed a pipeline of several traditional steps like edge detection, dilation and morphological operators. The algorithms were compared by measuring pixel-based segmentation accuracy and Jaccard index against a labelled ground truth. The results indicated a superior performance of the traditional algorithm (Accuracy = 99%, Jaccard = 93%) over the deep learning architectures: VGG16 (93%, 90%), ResNet18 (94%, 88%), Inception-ResNet-v2 (94%, 89%), and U-Net (92%, 56%).

Segmentation and Modelling of the Nuclear Envelope of HeLa Cells Imaged with Serial Block Face Scanning Electron Microscopy.

This paper describes an unsupervised algorithm, which segments the nuclear envelope of HeLa cells imaged by Serial Block Face Scanning Electron Microscopy. The algorithm exploits the variations of pixel intensity in different cellular regions by calculating edges, which are then used to generate superpixels. The superpixels are morphologically processed and those that correspond to the nuclear region are selected through the analysis of size, position, and correspondence with regions detected in neighbouring slices. The nuclear envelope is segmented from the nuclear region. The three-dimensional segmented nuclear envelope is then modelled against a spheroid to create a two-dimensional (2D) surface. The 2D surface summarises the complex 3D shape of the nuclear envelope and allows the extraction of metrics that may be relevant to characterise the nature of cells. The algorithm was developed and validated on a single cell and tested in six separate cells, each with 300 slices of 2000 × 2000 pixels. Ground truth was available for two of these cells, i.e., 600 hand-segmented slices. The accuracy of the algorithm was evaluated with two similarity metrics: Jaccard Similarity Index and Mean Hausdorff distance. Jaccard values of the first/second segmentation were 93%/90% for the whole cell, and 98%/94% between slices 75 and 225, as the central slices of the nucleus are more regular than those on the extremes. Mean Hausdorff distances were 9/17 pixels for the whole cells and 4/13 pixels for central slices. One slice was processed in approximately 8 s and a whole cell in 40 min. The algorithm outperformed active contours in both accuracy and time.

Mycobacterium tuberculosis replicates within necrotic human macrophages.

Mycobacterium tuberculosis modulation of macrophage cell death is a well-documented phenomenon, but its role during bacterial replication is less characterized. In this study, we investigate the impact of plasma membrane (PM) integrity on bacterial replication in different functional populations of human primary macrophages. We discovered that IFN-γ enhanced bacterial replication in macrophage colony-stimulating factor-differentiated macrophages more than in granulocyte-macrophage colony-stimulating factor-differentiated macrophages. We show that permissiveness in the different populations of macrophages to bacterial growth is the result of a differential ability to preserve PM integrity. By combining live-cell imaging, correlative light electron microscopy, and single-cell analysis, we found that after infection, a population of macrophages became necrotic, providing a niche for M. tuberculosis replication before escaping into the extracellular milieu. Thus, in addition to bacterial dissemination, necrotic cells provide first a niche for bacterial replication. Our results are relevant to understanding the environment of M. tuberculosis replication in the host.

Building bridges between cellular and molecular structural biology.

The integration of cellular and molecular structural data is key to understanding the function of macromolecular assemblies and complexes in their in vivo context. Here we report on the outcomes of a workshop that discussed how to integrate structural data from a range of public archives. The workshop identified two main priorities: the development of tools and file formats to support segmentation (that is, the decomposition of a three-dimensional volume into regions that can be associated with defined objects), and the development of tools to support the annotation of biological structures.

ultraLM and miniLM: Locator tools for smart tracking of fluorescent cells in correlative light and electron microscopy.

In-resin fluorescence (IRF) protocols preserve fluorescent proteins in resin-embedded cells and tissues for correlative light and electron microscopy, aiding interpretation of macromolecular function within the complex cellular landscape. Dual-contrast IRF samples can be imaged in separate fluorescence and electron microscopes, or in dual-modality integrated microscopes for high resolution correlation of fluorophore to organelle. IRF samples also offer a unique opportunity to automate correlative imaging workflows. Here we present two new locator tools for finding and following fluorescent cells in IRF blocks, enabling future automation of correlative imaging. The ultraLM is a fluorescence microscope that integrates with an ultramicrotome, which enables 'smart collection' of ultrathin sections containing fluorescent cells or tissues for subsequent transmission electron microscopy or array tomography. The miniLM is a fluorescence microscope that integrates with serial block face scanning electron microscopes, which enables 'smart tracking' of fluorescent structures during automated serial electron image acquisition from large cell and tissue volumes.

Synchronization of endothelial Dll4-Notch dynamics switch blood vessels from branching to expansion.

Formation of a regularly branched blood vessel network is crucial in development and physiology. Here we show that the expression of the Notch ligand Dll4 fluctuates in individual endothelial cells within sprouting vessels in the mouse retina in vivo and in correlation with dynamic cell movement in mouse embryonic stem cell-derived sprouting assays. We also find that sprout elongation and branching associates with a highly differential phase pattern of Dll4 between endothelial cells. Stimulation with pathologically high levels of Vegf, or overexpression of Dll4, leads to Notch dependent synchronization of Dll4 fluctuations within clusters, both in vitro and in vivo. Our results demonstrate that the Vegf-Dll4/Notch feedback system normally operates to generate heterogeneity between endothelial cells driving branching, whilst synchronization drives vessel expansion. We propose that this sensitive phase transition in the behaviour of the Vegf-Dll4/Notch feedback loop underlies the morphogen function of Vegfa in vascular patterning.