A deep learning-based toolkit for 3D nuclei segmentation and quantitative analysis in cellular and tissue context.

We present a new set of computational tools that enable accurate and widely applicable 3D segmentation of nuclei in various 3D digital organs. We have developed an approach for ground truth generation and iterative training of 3D nuclear segmentation models, which we applied to popular CellPose, PlantSeg and StarDist algorithms. We provide two high-quality models trained on plant nuclei that enable 3D segmentation of nuclei in datasets obtained from fixed or live samples, acquired from different plant and animal tissues, and stained with various nuclear stains or fluorescent protein-based nuclear reporters. We also share a diverse high-quality training dataset of about 10,000 nuclei. Furthermore, we advanced the MorphoGraphX analysis and visualization software by, among other things, providing a method for linking 3D segmented nuclei to their surrounding cells in 3D digital organs. We found that the nuclear-to-cell volume ratio varies between different ovule tissues and during the development of a tissue. Finally, we extended the PlantSeg 3D segmentation pipeline with a proofreading tool that uses 3D segmented nuclei as seeds to correct cell segmentation errors in difficult-to-segment tissues.

Scalable Optical Nose Realized with a Chemiresistively Modulated Light-Emitter Array.

Biological olfaction relies on a large number of receptors that function as sensors to detect gaseous molecules. It is challenging to realize artificial olfactory systems that contain similarly large numbers of sensory materials. It is shown that combinatorial materials processing with vapor deposition can be used to fabricate large arrays of distinct chemiresistive sensing materials. By combining these with light-emitting diodes, an array of chemiresistively-modulated light-emitting diodes, or ChemLEDs, that permit a simultaneous optical read-out in response to an analyte is obtained. The optical nose uses a common voltage source and ground for all sensing elements and thus eliminates the need for complex wiring of individual sensors. This optical nose contains one hundred ChemLEDs and generates unique light patterns in response to gases and their mixtures. Optical pattern recognition methods enable the quantitative prediction of the corresponding concentrations and compositions, thereby paving the way for massively parallel artificial olfactory systems. ChemLEDs open the possibility to explore demanding gas sensing applications, including in environmental, food quality monitoring, and potentially diagnostic settings.

Visualizing single-cell data with the neighbor embedding spectrum.

The two-dimensional embedding methods t-SNE and UMAP are ubiquitously used for visualizing single-cell data. Recent theoretical research in machine learning has shown that, despite their very different formulation and implementation, t-SNE and UMAP are closely connected, and a single parameter suffices to interpolate between them. This leads to a whole spectrum of visualization methods that focus on different aspects of the data. Along the spectrum, this focus changes from representing local structures to representing continuous ones. In single-cell context, this leads to a trade-off between highlighting rare cell types or continuous variation, such as developmental trajectories. Visualizing the entire spectrum as an animation can provide a more nuanced understanding of the high-dimensional dataset than individual visualizations with either t-SNE or UMAP.

Visualizing hierarchies in scRNA-seq data using a density tree-biased autoencoder.

MOTIVATION: Single-cell RNA sequencing (scRNA-seq) allows studying the development of cells in unprecedented detail. Given that many cellular differentiation processes are hierarchical, their scRNA-seq data are expected to be approximately tree-shaped in gene expression space. Inference and representation of this tree structure in two dimensions is highly desirable for biological interpretation and exploratory analysis. RESULTS: Our two contributions are an approach for identifying a meaningful tree structure from high-dimensional scRNA-seq data, and a visualization method respecting the tree structure. We extract the tree structure by means of a density-based maximum spanning tree on a vector quantization of the data and show that it captures biological information well. We then introduce density-tree biased autoencoder (DTAE), a tree-biased autoencoder that emphasizes the tree structure of the data in low dimensional space. We compare to other dimension reduction methods and demonstrate the success of our method both qualitatively and quantitatively on real and toy data. AVAILABILITY AND IMPLEMENTATION: Our implementation relying on PyTorch and Higra is available at github.com/hci-unihd/DTAE. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Temporal control of the integrated stress response by a stochastic molecular switch.

Stress granules (SGs) are formed in the cytosol as an acute response to environmental cues and activation of the integrated stress response (ISR), a central signaling pathway controlling protein synthesis. Using chronic virus infection as stress model, we previously uncovered a unique temporal control of the ISR resulting in recurrent phases of SG assembly and disassembly. Here, we elucidate the molecular network generating this fluctuating stress response by integrating quantitative experiments with mathematical modeling and find that the ISR operates as a stochastic switch. Key elements controlling this switch are the cooperative activation of the stress-sensing kinase PKR, the ultrasensitive response of SG formation to the phosphorylation of the translation initiation factor eIF2α, and negative feedback via GADD34, a stress-induced subunit of protein phosphatase 1. We identify GADD34 messenger RNA levels as the molecular memory of the ISR that plays a central role in cell adaptation to acute and chronic stress.

Seipin forms a flexible cage at lipid droplet formation sites.

Lipid droplets (LDs) form in the endoplasmic reticulum by phase separation of neutral lipids. This process is facilitated by the seipin protein complex, which consists of a ring of seipin monomers, with a yet unclear function. Here, we report a structure of S. cerevisiae seipin based on cryogenic-electron microscopy and structural modeling data. Seipin forms a decameric, cage-like structure with the lumenal domains forming a stable ring at the cage floor and transmembrane segments forming the cage sides and top. The transmembrane segments interact with adjacent monomers in two distinct, alternating conformations. These conformations result from changes in switch regions, located between the lumenal domains and the transmembrane segments, that are required for seipin function. Our data indicate a model for LD formation in which a closed seipin cage enables triacylglycerol phase separation and subsequently switches to an open conformation to allow LD growth and budding.

Integration of Cell Growth and Asymmetric Division during Lateral Root Initiation in Arabidopsis thaliana.

Lateral root formation determines to a large extent the ability of plants to forage their environment and thus their growth. In Arabidopsis thaliana and other angiosperms, lateral root initiation requires radial cell expansion and several rounds of anticlinal cell divisions that give rise to a central core of small cells, which express different markers than the larger surrounding cells. These small central cells then switch their plane of divisions to periclinal and give rise to seemingly morphologically similar daughter cells that have different identities and establish the different cell types of the new root. Although the execution of these anticlinal and periclinal divisions is tightly regulated and essential for the correct development of the lateral root, we know little about their geometrical features. Here, we generate a four-dimensional reconstruction of the first stages of lateral root formation and analyze the geometric features of the anticlinal and periclinal divisions. We identify that the periclinal divisions of the small central cells are morphologically dissimilar and asymmetric. We show that mother cell volume is different when looking at anticlinal vs. periclinal divisions and the repeated anticlinal divisions do not lead to reduction in cell volume, although cells are shorter. Finally, we show that cells undergoing a periclinal division are characterized by a strong cell expansion. Our results indicate that cells integrate growth and division to precisely partition their volume upon division during the first two stages of lateral root formation.

A digital 3D reference atlas reveals cellular growth patterns shaping the Arabidopsis ovule.

A fundamental question in biology is how morphogenesis integrates the multitude of processes that act at different scales, ranging from the molecular control of gene expression to cellular coordination in a tissue. Using machine-learning-based digital image analysis, we generated a three-dimensional atlas of ovule development in Arabidopsis thaliana, enabling the quantitative spatio-temporal analysis of cellular and gene expression patterns with cell and tissue resolution. We discovered novel morphological manifestations of ovule polarity, a new mode of cell layer formation, and previously unrecognized subepidermal cell populations that initiate ovule curvature. The data suggest an irregular cellular build-up of WUSCHEL expression in the primordium and new functions for INNER NO OUTER in restricting nucellar cell proliferation and the organization of the interior chalaza. Our work demonstrates the analytical power of a three-dimensional digital representation when studying the morphogenesis of an organ of complex architecture that eventually consists of 1900 cells.

The Mutex Watershed and its Objective: Efficient, Parameter-Free Graph Partitioning.

Image partitioning, or segmentation without semantics, is the task of decomposing an image into distinct segments, or equivalently to detect closed contours. Most prior work either requires seeds, one per segment; or a threshold; or formulates the task as multicut / correlation clustering, an NP-hard problem. Here, we propose an efficient algorithm for graph partitioning, the "Mutex Watershed". Unlike seeded watershed, the algorithm can accommodate not only attractive but also repulsive cues, allowing it to find a previously unspecified number of segments without the need for explicit seeds or a tunable threshold. We also prove that this simple algorithm solves to global optimality an objective function that is intimately related to the multicut / correlation clustering integer linear programming formulation. The algorithm is deterministic, very simple to implement, and has empirically linearithmic complexity. When presented with short-range attractive and long-range repulsive cues from a deep neural network, the Mutex Watershed gives the best results currently known for the competitive ISBI 2012 EM segmentation benchmark.

Microscopy-based assay for semi-quantitative detection of SARS-CoV-2 specific antibodies in human sera: A semi-quantitative, high throughput, microscopy-based assay expands existing approaches to measure SARS-CoV-2 specific antibody levels in human sera.

Emergence of the novel pathogenic coronavirus SARS-CoV-2 and its rapid pandemic spread presents challenges that demand immediate attention. Here, we describe the development of a semi-quantitative high-content microscopy-based assay for detection of three major classes (IgG, IgA, and IgM) of SARS-CoV-2 specific antibodies in human samples. The possibility to detect antibodies against the entire viral proteome together with a robust semi-automated image analysis workflow resulted in specific, sensitive and unbiased assay that complements the portfolio of SARS-CoV-2 serological assays. Sensitive, specific and quantitative serological assays are urgently needed for a better understanding of humoral immune response against the virus as a basis for developing public health strategies to control viral spread. The procedure described here has been used for clinical studies and provides a general framework for the application of quantitative high-throughput microscopy to rapidly develop serological assays for emerging virus infections.

Accurate and versatile 3D segmentation of plant tissues at cellular resolution.

Quantitative analysis of plant and animal morphogenesis requires accurate segmentation of individual cells in volumetric images of growing organs. In the last years, deep learning has provided robust automated algorithms that approach human performance, with applications to bio-image analysis now starting to emerge. Here, we present PlantSeg, a pipeline for volumetric segmentation of plant tissues into cells. PlantSeg employs a convolutional neural network to predict cell boundaries and graph partitioning to segment cells based on the neural network predictions. PlantSeg was trained on fixed and live plant organs imaged with confocal and light sheet microscopes. PlantSeg delivers accurate results and generalizes well across different tissues, scales, acquisition settings even on non plant samples. We present results of PlantSeg applications in diverse developmental contexts. PlantSeg is free and open-source, with both a command line and a user-friendly graphical interface.

ilastik: interactive machine learning for (bio)image analysis.

We present ilastik, an easy-to-use interactive tool that brings machine-learning-based (bio)image analysis to end users without substantial computational expertise. It contains pre-defined workflows for image segmentation, object classification, counting and tracking. Users adapt the workflows to the problem at hand by interactively providing sparse training annotations for a nonlinear classifier. ilastik can process data in up to five dimensions (3D, time and number of channels). Its computational back end runs operations on-demand wherever possible, allowing for interactive prediction on data larger than RAM. Once the classifiers are trained, ilastik workflows can be applied to new data from the command line without further user interaction. We describe all ilastik workflows in detail, including three case studies and a discussion on the expected performance.

Experimental and computational analyses reveal that environmental restrictions shape HIV-1 spread in 3D cultures.

Pathogens face varying microenvironments in vivo, but suitable experimental systems and analysis tools to dissect how three-dimensional (3D) tissue environments impact pathogen spread are lacking. Here we develop an Integrative method to Study Pathogen spread by Experiment and Computation within Tissue-like 3D cultures (INSPECT-3D), combining quantification of pathogen replication with imaging to study single-cell and cell population dynamics. We apply INSPECT-3D to analyze HIV-1 spread between primary human CD4 T-lymphocytes using collagen as tissue-like 3D-scaffold. Measurements of virus replication, infectivity, diffusion, cellular motility and interactions are combined by mathematical analyses into an integrated spatial infection model to estimate parameters governing HIV-1 spread. This reveals that environmental restrictions limit infection by cell-free virions but promote cell-associated HIV-1 transmission. Experimental validation identifies cell motility and density as essential determinants of efficacy and mode of HIV-1 spread in 3D. INSPECT-3D represents an adaptable method for quantitative time-resolved analyses of 3D pathogen spread.

DiversePathsJ: diverse shortest paths for bioimage analysis.

Motivation: We introduce a formulation for the general task of finding diverse shortest paths between two end-points. Our approach is not linked to a specific biological problem and can be applied to a large variety of images thanks to its generic implementation as a user-friendly ImageJ/Fiji plugin. It relies on the introduction of additional layers in a Viterbi path graph, which requires slight modifications to the standard Viterbi algorithm rules. This layered graph construction allows for the specification of various constraints imposing diversity between solutions. Results: The software allows obtaining a collection of diverse shortest paths under some user-defined constraints through a convenient and user-friendly interface. It can be used alone or be integrated into larger image analysis pipelines. Availability and implementation: http://bigwww.epfl.ch/algorithms/diversepathsj. Contact: michael.unser@epfl.ch or fred.hamprecht@iwr.uni-heidelberg.de. Supplementary information: Supplementary data are available at Bioinformatics online.

An objective comparison of cell-tracking algorithms.

We present a combined report on the results of three editions of the Cell Tracking Challenge, an ongoing initiative aimed at promoting the development and objective evaluation of cell segmentation and tracking algorithms. With 21 participating algorithms and a data repository consisting of 13 data sets from various microscopy modalities, the challenge displays today's state-of-the-art methodology in the field. We analyzed the challenge results using performance measures for segmentation and tracking that rank all participating methods. We also analyzed the performance of all of the algorithms in terms of biological measures and practical usability. Although some methods scored high in all technical aspects, none obtained fully correct solutions. We found that methods that either take prior information into account using learning strategies or analyze cells in a global spatiotemporal video context performed better than other methods under the segmentation and tracking scenarios included in the challenge.

Segmenting and Tracking Multiple Dividing Targets Using ilastik.

Tracking crowded cells or other targets in biology is often a challenging task due to poor signal-to-noise ratio, mutual occlusion, large displacements, little discernibility, and the ability of cells to divide. We here present an open source implementation of conservation tracking (Schiegg et al., IEEE international conference on computer vision (ICCV). IEEE, New York, pp 2928-2935, 2013) in the ilastik software framework. This robust tracking-by-assignment algorithm explicitly makes allowance for false positive detections, undersegmentation, and cell division. We give an overview over the underlying algorithm and parameters, and explain the use for a light sheet microscopy sequence of a Drosophila embryo. Equipped with this knowledge, users will be able to track targets of interest in their own data.

Virtual Raters for Reproducible and Objective Assessments in Radiology.

Volumetric measurements in radiologic images are important for monitoring tumor growth and treatment response. To make these more reproducible and objective we introduce the concept of virtual raters (VRs). A virtual rater is obtained by combining knowledge of machine-learning algorithms trained with past annotations of multiple human raters with the instantaneous rating of one human expert. Thus, he is virtually guided by several experts. To evaluate the approach we perform experiments with multi-channel magnetic resonance imaging (MRI) data sets. Next to gross tumor volume (GTV) we also investigate subcategories like edema, contrast-enhancing and non-enhancing tumor. The first data set consists of N = 71 longitudinal follow-up scans of 15 patients suffering from glioblastoma (GB). The second data set comprises N = 30 scans of low- and high-grade gliomas. For comparison we computed Pearson Correlation, Intra-class Correlation Coefficient (ICC) and Dice score. Virtual raters always lead to an improvement w.r.t. inter- and intra-rater agreement. Comparing the 2D Response Assessment in Neuro-Oncology (RANO) measurements to the volumetric measurements of the virtual raters results in one-third of the cases in a deviating rating. Hence, we believe that our approach will have an impact on the evaluation of clinical studies as well as on routine imaging diagnostics.

Three-dimensional immersive virtual reality for studying cellular compartments in 3D models from EM preparations of neural tissues.

Advances in the application of electron microscopy (EM) to serial imaging are opening doors to new ways of analyzing cellular structure. New and improved algorithms and workflows for manual and semiautomated segmentation allow us to observe the spatial arrangement of the smallest cellular features with unprecedented detail in full three-dimensions. From larger samples, higher complexity models can be generated; however, they pose new challenges to data management and analysis. Here we review some currently available solutions and present our approach in detail. We use the fully immersive virtual reality (VR) environment CAVE (cave automatic virtual environment), a room in which we are able to project a cellular reconstruction and visualize in 3D, to step into a world created with Blender, a free, fully customizable 3D modeling software with NeuroMorph plug-ins for visualization and analysis of EM preparations of brain tissue. Our workflow allows for full and fast reconstructions of volumes of brain neuropil using ilastik, a software tool for semiautomated segmentation of EM stacks. With this visualization environment, we can walk into the model containing neuronal and astrocytic processes to study the spatial distribution of glycogen granules, a major energy source that is selectively stored in astrocytes. The use of CAVE was key to the observation of a nonrandom distribution of glycogen, and led us to develop tools to quantitatively analyze glycogen clustering and proximity to other subcellular features.