Current Progress and Challenges in Large-Scale 3D Mitochondria Instance Segmentation.

In this paper, we present the results of the MitoEM challenge on mitochondria 3D instance segmentation from electron microscopy images, organized in conjunction with the IEEE-ISBI 2021 conference. Our benchmark dataset consists of two large-scale 3D volumes, one from human and one from rat cortex tissue, which are 1,986 times larger than previously used datasets. At the time of paper submission, 257 participants had registered for the challenge, 14 teams had submitted their results, and six teams participated in the challenge workshop. Here, we present eight top-performing approaches from the challenge participants, along with our own baseline strategies. Posterior to the challenge, annotation errors in the ground truth were corrected without altering the final ranking. Additionally, we present a retrospective evaluation of the scoring system which revealed that: 1) challenge metric was permissive with the false positive predictions; and 2) size-based grouping of instances did not correctly categorize mitochondria of interest. Thus, we propose a new scoring system that better reflects the correctness of the segmentation results. Although several of the top methods are compared favorably to our own baselines, substantial errors remain unsolved for mitochondria with challenging morphologies. Thus, the challenge remains open for submission and automatic evaluation, with all volumes available for download.

Bridging imaging users to imaging analysis - A community survey.

The 'Bridging Imaging Users to Imaging Analysis' survey was conducted in 2022 by the Center for Open Bioimage Analysis (COBA), BioImaging North America (BINA) and the Royal Microscopical Society Data Analysis in Imaging Section (RMS DAIM) to understand the needs of the imaging community. Through multichoice and open-ended questions, the survey inquired about demographics, image analysis experiences, future needs and suggestions on the role of tool developers and users. Participants of the survey were from diverse roles and domains of the life and physical sciences. To our knowledge, this is the first attempt to survey cross-community to bridge knowledge gaps between physical and life sciences imaging. Survey results indicate that respondents' overarching needs are documentation, detailed tutorials on the usage of image analysis tools, user-friendly intuitive software, and better solutions for segmentation, ideally in a format tailored to their specific use cases. The tool creators suggested the users familiarise themselves with the fundamentals of image analysis, provide constant feedback and report the issues faced during image analysis while the users would like more documentation and an emphasis on tool friendliness. Regardless of the computational experience, there is a strong preference for 'written tutorials' to acquire knowledge on image analysis. We also observed that the interest in having 'office hours' to get an expert opinion on their image analysis methods has increased over the years. The results also showed less-than-expected usage of online discussion forums in the imaging community for solving image analysis problems. Surprisingly, we also observed a decreased interest among the survey respondents in deep/machine learning despite the increasing adoption of artificial intelligence in biology. In addition, the community suggests the need for a common repository for the available image analysis tools and their applications. The opinions and suggestions of the community, released here in full, will help the image analysis tool creation and education communities to design and deliver the resources accordingly.

Bridging Imaging Users to Imaging Analysis - A community survey.

The "Bridging Imaging Users to Imaging Analysis" survey was conducted in 2022 by the Center for Open Bioimage Analysis (COBA), Bioimaging North America (BINA), and the Royal Microscopical Society Data Analysis in Imaging Section (RMS DAIM) to understand the needs of the imaging community. Through multi-choice and open-ended questions, the survey inquired about demographics, image analysis experiences, future needs, and suggestions on the role of tool developers and users. Participants of the survey were from diverse roles and domains of the life and physical sciences. To our knowledge, this is the first attempt to survey cross-community to bridge knowledge gaps between physical and life sciences imaging. Survey results indicate that respondents' overarching needs are documentation, detailed tutorials on the usage of image analysis tools, user-friendly intuitive software, and better solutions for segmentation, ideally in a format tailored to their specific use cases. The tool creators suggested the users familiarize themselves with the fundamentals of image analysis, provide constant feedback, and report the issues faced during image analysis while the users would like more documentation and an emphasis on tool friendliness. Regardless of the computational experience, there is a strong preference for 'written tutorials' to acquire knowledge on image analysis. We also observed that the interest in having 'office hours' to get an expert opinion on their image analysis methods has increased over the years. In addition, the community suggests the need for a common repository for the available image analysis tools and their applications. The opinions and suggestions of the community, released here in full, will help the image analysis tool creation and education communities to design and deliver the resources accordingly.

Online citizen science with the Zooniverse for analysis of biological volumetric data.

Public participation in research, also known as citizen science, is being increasingly adopted for the analysis of biological volumetric data. Researchers working in this domain are applying online citizen science as a scalable distributed data analysis approach, with recent research demonstrating that non-experts can productively contribute to tasks such as the segmentation of organelles in volume electron microscopy data. This, alongside the growing challenge to rapidly process the large amounts of biological volumetric data now routinely produced, means there is increasing interest within the research community to apply online citizen science for the analysis of data in this context. Here, we synthesise core methodological principles and practices for applying citizen science for analysis of biological volumetric data. We collate and share the knowledge and experience of multiple research teams who have applied online citizen science for the analysis of volumetric biological data using the Zooniverse platform ( www.zooniverse.org ). We hope this provides inspiration and practical guidance regarding how contributor effort via online citizen science may be usefully applied in this domain.

Crosshair, semi-automated targeting for electron microscopy with a motorised ultramicrotome.

Volume electron microscopy (EM) is a time-consuming process - often requiring weeks or months of continuous acquisition for large samples. In order to compare the ultrastructure of a number of individuals or conditions, acquisition times must therefore be reduced. For resin-embedded samples, one solution is to selectively target smaller regions of interest by trimming with an ultramicrotome. This is a difficult and labour-intensive process, requiring manual positioning of the diamond knife and sample, and much time and training to master. Here, we have developed a semi-automated workflow for targeting with a modified ultramicrotome. We adapted two recent commercial systems to add motors for each rotational axis (and also each translational axis for one system), allowing precise and automated movement. We also developed a user-friendly software to convert X-ray images of resin-embedded samples into angles and cutting depths for the ultramicrotome. This is provided as an open-source Fiji plugin called Crosshair. This workflow is demonstrated by targeting regions of interest in a series of Platynereis dumerilii samples.

Sargasso Sea bacterioplankton community structure and drivers of variance as revealed by DNA metabarcoding analysis.

Marine microbes provide the backbone for pelagic ecosystems by cycling and fixing nutrients and establishing the base of food webs. Microbial communities are often assumed to be highly connected and genetically mixed, with localized environmental filters driving minor changes in structure. Our study applied high-throughput Illumina 16S ribosomal RNA gene amplicon sequencing on whole-community bacterial samples to characterize geographic, environmental, and stochastic drivers of community diversity. DNA was extracted from seawater collected from the surface (N = 18) and at depth just below the deep chlorophyll-a maximum (DCM mean depth = 115.4 m; N = 22) in the Sargasso Sea and adjacent oceanographic regions. Discrete bacterioplankton assemblages were observed at varying depths in the North Sargasso Sea, with a signal for distance-decay of bacterioplankton community similarity found only in surface waters. Bacterial communities from different oceanic regions could be distinguished statistically but exhibited a low magnitude of divergence. Redundancy analysis identified temperature as the key environmental variable correlated with community structuring. The effect of dispersal limitation was weak, while variation partitioning and neutral community modeling demonstrated stochastic processes influencing the communities. This study advances understanding of microbial biogeography in the pelagic ocean and highlights the use of high-throughput sequencing methods in studying microbial community structure.

Democratising deep learning for microscopy with ZeroCostDL4Mic.

Deep Learning (DL) methods are powerful analytical tools for microscopy and can outperform conventional image processing pipelines. Despite the enthusiasm and innovations fuelled by DL technology, the need to access powerful and compatible resources to train DL networks leads to an accessibility barrier that novice users often find difficult to overcome. Here, we present ZeroCostDL4Mic, an entry-level platform simplifying DL access by leveraging the free, cloud-based computational resources of Google Colab. ZeroCostDL4Mic allows researchers with no coding expertise to train and apply key DL networks to perform tasks including segmentation (using U-Net and StarDist), object detection (using YOLOv2), denoising (using CARE and Noise2Void), super-resolution microscopy (using Deep-STORM), and image-to-image translation (using Label-free prediction - fnet, pix2pix and CycleGAN). Importantly, we provide suitable quantitative tools for each network to evaluate model performance, allowing model optimisation. We demonstrate the application of the platform to study multiple biological processes.

Deep learning for automatic segmentation of the nuclear envelope in electron microscopy data, trained with volunteer segmentations.

Advancements in volume electron microscopy mean it is now possible to generate thousands of serial images at nanometre resolution overnight, yet the gold standard approach for data analysis remains manual segmentation by an expert microscopist, resulting in a critical research bottleneck. Although some machine learning approaches exist in this domain, we remain far from realizing the aspiration of a highly accurate, yet generic, automated analysis approach, with a major obstacle being lack of sufficient high-quality ground-truth data. To address this, we developed a novel citizen science project, Etch a Cell, to enable volunteers to manually segment the nuclear envelope (NE) of HeLa cells imaged with serial blockface scanning electron microscopy. We present our approach for aggregating multiple volunteer annotations to generate a high-quality consensus segmentation and demonstrate that data produced exclusively by volunteers can be used to train a highly accurate machine learning algorithm for automatic segmentation of the NE, which we share here, in addition to our archived benchmark data.

Volumetric Semantic Instance Segmentation of the Plasma Membrane of HeLa Cells.

In this work, an unsupervised volumetric semantic instance segmentation of the plasma membrane of HeLa cells as observed with serial block face scanning electron microscopy is described. The resin background of the images was segmented at different slices of a 3D stack of 518 slices with 8192 × 8192 pixels each. The background was used to create a distance map, which helped identify and rank the cells by their size at each slice. The centroids of the cells detected at different slices were linked to identify them as a single cell that spanned a number of slices. A subset of these cells, i.e., the largest ones and those not close to the edges were selected for further processing. The selected cells were then automatically cropped to smaller regions of interest of 2000 × 2000 × 300 voxels that were treated as cell instances. Then, for each of these volumes, the nucleus was segmented, and the cell was separated from any neighbouring cells through a series of traditional image processing steps that followed the plasma membrane. The segmentation process was repeated for all the regions of interest previously selected. For one cell for which the ground truth was available, the algorithm provided excellent results in Accuracy (AC) and the Jaccard similarity Index (JI): nucleus: JI =0.9665, AC =0.9975, cell including nucleus JI =0.8711, AC =0.9655, cell excluding nucleus JI =0.8094, AC =0.9629. A limitation of the algorithm for the plasma membrane segmentation was the presence of background. In samples with tightly packed cells, this may not be available. When tested for these conditions, the segmentation of the nuclear envelope was still possible. All the code and data were released openly through GitHub, Zenodo and EMPIAR.

Semantic segmentation of HeLa cells: An objective comparison between one traditional algorithm and four deep-learning architectures.

The quantitative study of cell morphology is of great importance as the structure and condition of cells and their structures can be related to conditions of health or disease. The first step towards that, is the accurate segmentation of cell structures. In this work, we compare five approaches, one traditional and four deep-learning, for the semantic segmentation of the nuclear envelope of cervical cancer cells commonly known as HeLa cells. Images of a HeLa cancer cell were semantically segmented with one traditional image-processing algorithm and four three deep learning architectures: VGG16, ResNet18, Inception-ResNet-v2, and U-Net. Three hundred slices, each 2000 × 2000 pixels, of a HeLa Cell were acquired with Serial Block Face Scanning Electron Microscopy. The first three deep learning architectures were pre-trained with ImageNet and then fine-tuned with transfer learning. The U-Net architecture was trained from scratch with 36, 000 training images and labels of size 128 × 128. The image-processing algorithm followed a pipeline of several traditional steps like edge detection, dilation and morphological operators. The algorithms were compared by measuring pixel-based segmentation accuracy and Jaccard index against a labelled ground truth. The results indicated a superior performance of the traditional algorithm (Accuracy = 99%, Jaccard = 93%) over the deep learning architectures: VGG16 (93%, 90%), ResNet18 (94%, 88%), Inception-ResNet-v2 (94%, 89%), and U-Net (92%, 56%).

Segmentation and Modelling of the Nuclear Envelope of HeLa Cells Imaged with Serial Block Face Scanning Electron Microscopy.

This paper describes an unsupervised algorithm, which segments the nuclear envelope of HeLa cells imaged by Serial Block Face Scanning Electron Microscopy. The algorithm exploits the variations of pixel intensity in different cellular regions by calculating edges, which are then used to generate superpixels. The superpixels are morphologically processed and those that correspond to the nuclear region are selected through the analysis of size, position, and correspondence with regions detected in neighbouring slices. The nuclear envelope is segmented from the nuclear region. The three-dimensional segmented nuclear envelope is then modelled against a spheroid to create a two-dimensional (2D) surface. The 2D surface summarises the complex 3D shape of the nuclear envelope and allows the extraction of metrics that may be relevant to characterise the nature of cells. The algorithm was developed and validated on a single cell and tested in six separate cells, each with 300 slices of 2000 × 2000 pixels. Ground truth was available for two of these cells, i.e., 600 hand-segmented slices. The accuracy of the algorithm was evaluated with two similarity metrics: Jaccard Similarity Index and Mean Hausdorff distance. Jaccard values of the first/second segmentation were 93%/90% for the whole cell, and 98%/94% between slices 75 and 225, as the central slices of the nucleus are more regular than those on the extremes. Mean Hausdorff distances were 9/17 pixels for the whole cells and 4/13 pixels for central slices. One slice was processed in approximately 8 s and a whole cell in 40 min. The algorithm outperformed active contours in both accuracy and time.

PolNet: A Tool to Quantify Network-Level Cell Polarity and Blood Flow in Vascular Remodeling.

In this article, we present PolNet, an open-source software tool for the study of blood flow and cell-level biological activity during vessel morphogenesis. We provide an image acquisition, segmentation, and analysis protocol to quantify endothelial cell polarity in entire in vivo vascular networks. In combination, we use computational fluid dynamics to characterize the hemodynamics of the vascular networks under study. The tool enables, to our knowledge for the first time, a network-level analysis of polarity and flow for individual endothelial cells. To date, PolNet has proven invaluable for the study of endothelial cell polarization and migration during vascular patterning, as demonstrated by two recent publications. Additionally, the tool can be easily extended to correlate blood flow with other experimental observations at the cellular/molecular level. We release the source code of our tool under the Lesser General Public License.

Building bridges between cellular and molecular structural biology.

The integration of cellular and molecular structural data is key to understanding the function of macromolecular assemblies and complexes in their in vivo context. Here we report on the outcomes of a workshop that discussed how to integrate structural data from a range of public archives. The workshop identified two main priorities: the development of tools and file formats to support segmentation (that is, the decomposition of a three-dimensional volume into regions that can be associated with defined objects), and the development of tools to support the annotation of biological structures.

Mycobacterium tuberculosis replicates within necrotic human macrophages.

Mycobacterium tuberculosis modulation of macrophage cell death is a well-documented phenomenon, but its role during bacterial replication is less characterized. In this study, we investigate the impact of plasma membrane (PM) integrity on bacterial replication in different functional populations of human primary macrophages. We discovered that IFN-γ enhanced bacterial replication in macrophage colony-stimulating factor-differentiated macrophages more than in granulocyte-macrophage colony-stimulating factor-differentiated macrophages. We show that permissiveness in the different populations of macrophages to bacterial growth is the result of a differential ability to preserve PM integrity. By combining live-cell imaging, correlative light electron microscopy, and single-cell analysis, we found that after infection, a population of macrophages became necrotic, providing a niche for M. tuberculosis replication before escaping into the extracellular milieu. Thus, in addition to bacterial dissemination, necrotic cells provide first a niche for bacterial replication. Our results are relevant to understanding the environment of M. tuberculosis replication in the host.

3D correlative light and electron microscopy of cultured cells using serial blockface scanning electron microscopy.

The processes of life take place in multiple dimensions, but imaging these processes in even three dimensions is challenging. Here, we describe a workflow for 3D correlative light and electron microscopy (CLEM) of cell monolayers using fluorescence microscopy to identify and follow biological events, combined with serial blockface scanning electron microscopy to analyse the underlying ultrastructure. The workflow encompasses all steps from cell culture to sample processing, imaging strategy, and 3D image processing and analysis. We demonstrate successful application of the workflow to three studies, each aiming to better understand complex and dynamic biological processes, including bacterial and viral infections of cultured cells and formation of entotic cell-in-cell structures commonly observed in tumours. Our workflow revealed new insight into the replicative niche of Mycobacterium tuberculosis in primary human lymphatic endothelial cells, HIV-1 in human monocyte-derived macrophages, and the composition of the entotic vacuole. The broad application of this 3D CLEM technique will make it a useful addition to the correlative imaging toolbox for biomedical research.

Neutral processes and species sorting in benthic microalgal community assembly: effects of tidal resuspension.

Benthic microalgae (BMA) provide vital food resources for heterotrophs and stabilize sediments with their extracellular secretions. A central goal in ecology is to understand how processes such as species interactions and dispersal, contribute to observed patterns of species abundance and distribution. Our objectives were to assess the effects of sediment resuspension on microalgal community structure. We tested whether taxa-abundance distributions could be predicted using neutral community models (NCMs) and also specific hypotheses about passive migration: (i) As migration decreases in sediment patches, BMA α-diversity will decrease, and (ii) As migration decreases, BMA community dissimilarity (ß-diversity) will increase. Co-occurrence indices (checkerboard score and variance ratio) were also computed to test for deterministic factors, such as competition and niche differentiation, in shaping communities. Two intertidal sites (mudflat and sand bar) differing in resuspension regime were sampled throughout the tidal cycle. Fluorometry and denaturing gradient gel electrophoresis were utilized to investigate diatom community structure. Observed taxa-abundances fit those predicted from NCMs reasonably well (R2 of 0.68-0.93), although comparisons of observed local communities to artificial randomly assembled communities rejected the null hypothesis that diatom communities were assembled solely by stochastic processes. No co-occurrence tests indicated a significant role for competitive exclusion or niche partitioning in microalgal community assembly. In general, predictions about relationships between migration and species diversity were supported for local community dynamics. BMA at low tide (lowest migration) exhibited reduced α-diversity as compared to periods of immersion at both mudflat and sand bar sites. ß-diversity was higher during low tide emersion on the mudflat, but did not differ temporally at the sand bar site. In between-site metacommunity comparisons, low- and high-resuspension sites exhibited distinct community compositions while the low-energy mudflats contained higher microalgal biomass and greater α-diversity. To our knowledge this is the first study to test the relevance of neutral processes in structuring marine microalgal communities. Our results demonstrate a prominent role for stochastic factors in structuring local BMA community assembly, although unidentified nonrandom processes also appear to play some role. High passive migration, in particular, appears to help maintain species diversity and structure communities in both sand and muddy habitats.

Synchronization of endothelial Dll4-Notch dynamics switch blood vessels from branching to expansion.

Formation of a regularly branched blood vessel network is crucial in development and physiology. Here we show that the expression of the Notch ligand Dll4 fluctuates in individual endothelial cells within sprouting vessels in the mouse retina in vivo and in correlation with dynamic cell movement in mouse embryonic stem cell-derived sprouting assays. We also find that sprout elongation and branching associates with a highly differential phase pattern of Dll4 between endothelial cells. Stimulation with pathologically high levels of Vegf, or overexpression of Dll4, leads to Notch dependent synchronization of Dll4 fluctuations within clusters, both in vitro and in vivo. Our results demonstrate that the Vegf-Dll4/Notch feedback system normally operates to generate heterogeneity between endothelial cells driving branching, whilst synchronization drives vessel expansion. We propose that this sensitive phase transition in the behaviour of the Vegf-Dll4/Notch feedback loop underlies the morphogen function of Vegfa in vascular patterning.

Non-canonical Wnt signalling modulates the endothelial shear stress flow sensor in vascular remodelling.

Endothelial cells respond to molecular and physical forces in development and vascular homeostasis. Deregulation of endothelial responses to flow-induced shear is believed to contribute to many aspects of cardiovascular diseases including atherosclerosis. However, how molecular signals and shear-mediated physical forces integrate to regulate vascular patterning is poorly understood. Here we show that endothelial non-canonical Wnt signalling regulates endothelial sensitivity to shear forces. Loss of Wnt5a/Wnt11 renders endothelial cells more sensitive to shear, resulting in axial polarization and migration against flow at lower shear levels. Integration of flow modelling and polarity analysis in entire vascular networks demonstrates that polarization against flow is achieved differentially in artery, vein, capillaries and the primitive sprouting front. Collectively our data suggest that non-canonical Wnt signalling stabilizes forming vascular networks by reducing endothelial shear sensitivity, thus keeping vessels open under low flow conditions that prevail in the primitive plexus.

Computational fluid dynamics assisted characterization of parafoveal hemodynamics in normal and diabetic eyes using adaptive optics scanning laser ophthalmoscopy.

Diabetic retinopathy (DR) is the leading cause of visual loss in working-age adults worldwide. Previous studies have found hemodynamic changes in the diabetic eyes, which precede clinically evident pathological alterations of the retinal microvasculature. There is a pressing need for new methods to allow greater understanding of these early hemodynamic changes that occur in DR. In this study, we propose a noninvasive method for the assessment of hemodynamics around the fovea (a region of the eye of paramount importance for vision). The proposed methodology combines adaptive optics scanning laser ophthalmoscopy and computational fluid dynamics modeling. We compare results obtained with this technique with in vivo measurements of blood flow based on blood cell aggregation tracking. Our results suggest that parafoveal hemodynamics, such as capillary velocity, wall shear stress, and capillary perfusion pressure can be noninvasively and reliably characterized with this method in both healthy and diabetic retinopathy patients.