Modular segmentation, spatial analysis and visualization of volume electron microscopy datasets.

Volume electron microscopy is the method of choice for the in situ interrogation of cellular ultrastructure at the nanometer scale, and with the increase in large raw image datasets generated, improving computational strategies for image segmentation and spatial analysis is necessary. Here we describe a practical and annotation-efficient pipeline for organelle-specific segmentation, spatial analysis and visualization of large volume electron microscopy datasets using freely available, user-friendly software tools that can be run on a single standard workstation. The procedures are aimed at researchers in the life sciences with modest computational expertise, who use volume electron microscopy and need to generate three-dimensional (3D) segmentation labels for different types of cell organelles while minimizing manual annotation efforts, to analyze the spatial interactions between organelle instances and to visualize the 3D segmentation results. We provide detailed guidelines for choosing well-suited segmentation tools for specific cell organelles, and to bridge compatibility issues between freely available open-source tools, we distribute the critical steps as easily installable Album solutions for deep learning segmentation, spatial analysis and 3D rendering. Our detailed description can serve as a reference for similar projects requiring particular strategies for single- or multiple-organelle analysis, which can be achieved with computational resources commonly available to single-user setups.

CoNIC Challenge: Pushing the frontiers of nuclear detection, segmentation, classification and counting.

Nuclear detection, segmentation and morphometric profiling are essential in helping us further understand the relationship between histology and patient outcome. To drive innovation in this area, we setup a community-wide challenge using the largest available dataset of its kind to assess nuclear segmentation and cellular composition. Our challenge, named CoNIC, stimulated the development of reproducible algorithms for cellular recognition with real-time result inspection on public leaderboards. We conducted an extensive post-challenge analysis based on the top-performing models using 1,658 whole-slide images of colon tissue. With around 700 million detected nuclei per model, associated features were used for dysplasia grading and survival analysis, where we demonstrated that the challenge's improvement over the previous state-of-the-art led to significant boosts in downstream performance. Our findings also suggest that eosinophils and neutrophils play an important role in the tumour microevironment. We release challenge models and WSI-level results to foster the development of further methods for biomarker discovery.

Neuronal migration prevents spatial competition in retinal morphogenesis.

The concomitant occurrence of tissue growth and organization is a hallmark of organismal development1-3. This often means that proliferating and differentiating cells are found at the same time in a continuously changing tissue environment. How cells adapt to architectural changes to prevent spatial interference remains unclear. Here, to understand how cell movements that are key for growth and organization are orchestrated, we study the emergence of photoreceptor neurons that occur during the peak of retinal growth, using zebrafish, human tissue and human organoids. Quantitative imaging reveals that successful retinal morphogenesis depends on the active bidirectional translocation of photoreceptors, leading to a transient transfer of the entire cell population away from the apical proliferative zone. This pattern of migration is driven by cytoskeletal machineries that differ depending on the direction: microtubules are exclusively required for basal translocation, whereas actomyosin is involved in apical movement. Blocking the basal translocation of photoreceptors induces apical congestion, which hampers the apical divisions of progenitor cells and leads to secondary defects in lamination. Thus, photoreceptor migration is crucial to prevent competition for space, and to allow concurrent tissue growth and lamination. This shows that neuronal migration, in addition to its canonical role in cell positioning4, can be involved in coordinating morphogenesis.

CenFind: a deep-learning pipeline for efficient centriole detection in microscopy datasets.

BACKGROUND: High-throughput and selective detection of organelles in immunofluorescence images is an important but demanding task in cell biology. The centriole organelle is critical for fundamental cellular processes, and its accurate detection is key for analysing centriole function in health and disease. Centriole detection in human tissue culture cells has been achieved typically by manual determination of organelle number per cell. However, manual cell scoring of centrioles has a low throughput and is not reproducible. Published semi-automated methods tally the centrosome surrounding centrioles and not centrioles themselves. Furthermore, such methods rely on hard-coded parameters or require a multichannel input for cross-correlation. Therefore, there is a need for developing an efficient and versatile pipeline for the automatic detection of centrioles in single channel immunofluorescence datasets. RESULTS: We developed a deep-learning pipeline termed CenFind that automatically scores cells for centriole numbers in immunofluorescence images of human cells. CenFind relies on the multi-scale convolution neural network SpotNet, which allows the accurate detection of sparse and minute foci in high resolution images. We built a dataset using different experimental settings and used it to train the model and evaluate existing detection methods. The resulting average F1-score achieved by CenFind is > 90% across the test set, demonstrating the robustness of the pipeline. Moreover, using the StarDist-based nucleus detector, we link the centrioles and procentrioles detected with CenFind to the cell containing them, overall enabling automatic scoring of centriole numbers per cell. CONCLUSIONS: Efficient, accurate, channel-intrinsic and reproducible detection of centrioles is an important unmet need in the field. Existing methods are either not discriminative enough or focus on a fixed multi-channel input. To fill this methodological gap, we developed CenFind, a command line interface pipeline that automates cell scoring of centrioles, thereby enabling channel-intrinsic, accurate and reproducible detection across experimental modalities. Moreover, the modular nature of CenFind enables its integration in other pipelines. Overall, we anticipate CenFind to prove critical for accelerating discoveries in the field.

MarrowQuant 2.0: A Digital Pathology Workflow Assisting Bone Marrow Evaluation in Experimental and Clinical Hematology.

Bone marrow (BM) cellularity assessment is a crucial step in the evaluation of BM trephine biopsies for hematologic and nonhematologic disorders. Clinical assessment is based on a semiquantitative visual estimation of the hematopoietic and adipocytic components by hematopathologists, which does not provide quantitative information on other stromal compartments. In this study, we developed and validated MarrowQuant 2.0, an efficient, user-friendly digital hematopathology workflow integrated within QuPath software, which serves as BM quantifier for 5 mutually exclusive compartments (bone, hematopoietic, adipocytic, and interstitial/microvasculature areas and other) and derives the cellularity of human BM trephine biopsies. Instance segmentation of individual adipocytes is realized through the adaptation of the machine-learning-based algorithm StarDist. We calculated BM compartments and adipocyte size distributions of hematoxylin and eosin images obtained from 250 bone specimens, from control subjects and patients with acute myeloid leukemia or myelodysplastic syndrome, at diagnosis and follow-up, and measured the agreement of cellularity estimates by MarrowQuant 2.0 against visual scores from 4 hematopathologists. The algorithm was capable of robust BM compartment segmentation with an average mask accuracy of 86%, maximal for bone (99%), hematopoietic (92%), and adipocyte (98%) areas. MarrowQuant 2.0 cellularity score and hematopathologist estimations were highly correlated (R2 = 0.92-0.98, intraclass correlation coefficient [ICC] = 0.98; interobserver ICC = 0.96). BM compartment segmentation quantitatively confirmed the reciprocity of the hematopoietic and adipocytic compartments. MarrowQuant 2.0 performance was additionally tested for cellularity assessment of specimens prospectively collected from clinical routine diagnosis. After special consideration for the choice of the cellularity equation in specimens with expanded stroma, performance was similar in this setting (R2 = 0.86, n = 42). Thus, we conclude that these validation experiments establish MarrowQuant 2.0 as a reliable tool for BM cellularity assessment. We expect this workflow will serve as a clinical research tool to explore novel biomarkers related to BM stromal components and may contribute to further validation of future digitalized diagnostic hematopathology workstreams.

Deep neural network automated segmentation of cellular structures in volume electron microscopy.

Volume electron microscopy is an important imaging modality in contemporary cell biology. Identification of intracellular structures is a laborious process limiting the effective use of this potentially powerful tool. We resolved this bottleneck with automated segmentation of intracellular substructures in electron microscopy (ASEM), a new pipeline to train a convolutional neural network to detect structures of a wide range in size and complexity. We obtained dedicated models for each structure based on a small number of sparsely annotated ground truth images from only one or two cells. Model generalization was improved with a rapid, computationally effective strategy to refine a trained model by including a few additional annotations. We identified mitochondria, Golgi apparatus, endoplasmic reticulum, nuclear pore complexes, caveolae, clathrin-coated pits, and vesicles imaged by focused ion beam scanning electron microscopy. We uncovered a wide range of membrane-nuclear pore diameters within a single cell and derived morphological metrics from clathrin-coated pits and vesicles, consistent with the classical constant-growth assembly model.

Event-driven acquisition for content-enriched microscopy.

A common goal of fluorescence microscopy is to collect data on specific biological events. Yet, the event-specific content that can be collected from a sample is limited, especially for rare or stochastic processes. This is due in part to photobleaching and phototoxicity, which constrain imaging speed and duration. We developed an event-driven acquisition framework, in which neural-network-based recognition of specific biological events triggers real-time control in an instant structured illumination microscope. Our setup adapts acquisitions on-the-fly by switching between a slow imaging rate while detecting the onset of events, and a fast imaging rate during their progression. Thus, we capture mitochondrial and bacterial divisions at imaging rates that match their dynamic timescales, while extending overall imaging durations. Because event-driven acquisition allows the microscope to respond specifically to complex biological events, it acquires data enriched in relevant content.

LAPTM5 mediates immature B cell apoptosis and B cell tolerance by regulating the WWP2-PTEN-AKT pathway.

Elimination of autoreactive developing B cells is an important mechanism to prevent autoantibody production. However, how B cell receptor (BCR) signaling triggers apoptosis of immature B cells remains poorly understood. We show that BCR stimulation up-regulates the expression of the lysosomal-associated transmembrane protein 5 (LAPTM5), which in turn triggers apoptosis of immature B cells through two pathways. LAPTM5 causes BCR internalization, resulting in decreased phosphorylation of SYK and ERK. In addition, LAPTM5 targets the E3 ubiquitin ligase WWP2 for lysosomal degradation, resulting in the accumulation of its substrate PTEN. Elevated PTEN levels suppress AKT phosphorylation, leading to increased FOXO1 expression and up-regulation of the cell cycle inhibitor p27Kip1 and the proapoptotic molecule BIM. In vivo, LAPTM5 is involved in the elimination of autoreactive B cells and its deficiency exacerbates autoantibody production. Our results reveal a previously unidentified mechanism that contributes to immature B cell apoptosis and B cell tolerance.

EASI-FISH for thick tissue defines lateral hypothalamus spatio-molecular organization.

Determining the spatial organization and morphological characteristics of molecularly defined cell types is a major bottleneck for characterizing the architecture underpinning brain function. We developed Expansion-Assisted Iterative Fluorescence In Situ Hybridization (EASI-FISH) to survey gene expression in brain tissue, as well as a turnkey computational pipeline to rapidly process large EASI-FISH image datasets. EASI-FISH was optimized for thick brain sections (300 µm) to facilitate reconstruction of spatio-molecular domains that generalize across brains. Using the EASI-FISH pipeline, we investigated the spatial distribution of dozens of molecularly defined cell types in the lateral hypothalamic area (LHA), a brain region with poorly defined anatomical organization. Mapping cell types in the LHA revealed nine spatially and molecularly defined subregions. EASI-FISH also facilitates iterative reanalysis of scRNA-seq datasets to determine marker-genes that further dissociated spatial and morphological heterogeneity. The EASI-FISH pipeline democratizes mapping molecularly defined cell types, enabling discoveries about brain organization.

Deep Learning Enables Individual Xenograft Cell Classification in Histological Images by Analysis of Contextual Features.

Patient-Derived Xenografts (PDXs) are the preclinical models which best recapitulate inter- and intra-patient complexity of human breast malignancies, and are also emerging as useful tools to study the normal breast epithelium. However, data analysis generated with such models is often confounded by the presence of host cells and can give rise to data misinterpretation. For instance, it is important to discriminate between xenografted and host cells in histological sections prior to performing immunostainings. We developed Single Cell Classifier (SCC), a data-driven deep learning-based computational tool that provides an innovative approach for automated cell species discrimination based on a multi-step process entailing nuclei segmentation and single cell classification. We show that human and murine cell contextual features, more than cell-intrinsic ones, can be exploited to discriminate between cell species in both normal and malignant tissues, yielding up to 96% classification accuracy. SCC will facilitate the interpretation of H&E- and DAPI-stained histological sections of xenografted human-in-mouse tissues and it is open to new in-house built models for further applications. SCC is released as an open-source plugin in ImageJ/Fiji available at the following link: https://github.com/Biomedical-Imaging-Group/SingleCellClassifier .

Deep learning-enhanced light-field imaging with continuous validation.

Visualizing dynamic processes over large, three-dimensional fields of view at high speed is essential for many applications in the life sciences. Light-field microscopy (LFM) has emerged as a tool for fast volumetric image acquisition, but its effective throughput and widespread use in biology has been hampered by a computationally demanding and artifact-prone image reconstruction process. Here, we present a framework for artificial intelligence-enhanced microscopy, integrating a hybrid light-field light-sheet microscope and deep learning-based volume reconstruction. In our approach, concomitantly acquired, high-resolution two-dimensional light-sheet images continuously serve as training data and validation for the convolutional neural network reconstructing the raw LFM data during extended volumetric time-lapse imaging experiments. Our network delivers high-quality three-dimensional reconstructions at video-rate throughput, which can be further refined based on the high-resolution light-sheet images. We demonstrate the capabilities of our approach by imaging medaka heart dynamics and zebrafish neural activity with volumetric imaging rates up to 100 Hz.

3D FIB-SEM reconstruction of microtubule-organelle interaction in whole primary mouse ß cells.

Microtubules play a major role in intracellular trafficking of vesicles in endocrine cells. Detailed knowledge of microtubule organization and their relation to other cell constituents is crucial for understanding cell function. However, their role in insulin transport and secretion is under debate. Here, we use FIB-SEM to image islet ß cells in their entirety with unprecedented resolution. We reconstruct mitochondria, Golgi apparati, centrioles, insulin secretory granules, and microtubules of seven ß cells, and generate a comprehensive spatial map of microtubule-organelle interactions. We find that microtubules form nonradial networks that are predominantly not connected to either centrioles or endomembranes. Microtubule number and length, but not microtubule polymer density, vary with glucose stimulation. Furthermore, insulin secretory granules are enriched near the plasma membrane, where they associate with microtubules. In summary, we provide the first 3D reconstructions of complete microtubule networks in primary mammalian cells together with evidence regarding their importance for insulin secretory granule positioning and thus their supportive role in insulin secretion.

A convolutional neural network segments yeast microscopy images with high accuracy.

The identification of cell borders ('segmentation') in microscopy images constitutes a bottleneck for large-scale experiments. For the model organism Saccharomyces cerevisiae, current segmentation methods face challenges when cells bud, crowd, or exhibit irregular features. We present a convolutional neural network (CNN) named YeaZ, the underlying training set of high-quality segmented yeast images (>10 000 cells) including mutants, stressed cells, and time courses, as well as a graphical user interface and a web application ( www.quantsysbio.com/data-and-software ) to efficiently employ, test, and expand the system. A key feature is a cell-cell boundary test which avoids the need for fluorescent markers. Our CNN is highly accurate, including for buds, and outperforms existing methods on benchmark images, indicating it transfers well to other conditions. To demonstrate how efficient large-scale image processing uncovers new biology, we analyze the geometries of ≈2200 wild-type and cyclin mutant cells and find that morphogenesis control occurs unexpectedly early and gradually.

Practical sensorless aberration estimation for 3D microscopy with deep learning.

Estimation of optical aberrations from volumetric intensity images is a key step in sensorless adaptive optics for 3D microscopy. Recent approaches based on deep learning promise accurate results at fast processing speeds. However, collecting ground truth microscopy data for training the network is typically very difficult or even impossible thereby limiting this approach in practice. Here, we demonstrate that neural networks trained only on simulated data yield accurate predictions for real experimental images. We validate our approach on simulated and experimental datasets acquired with two different microscopy modalities and also compare the results to non-learned methods. Additionally, we study the predictability of individual aberrations with respect to their data requirements and find that the symmetry of the wavefront plays a crucial role. Finally, we make our implementation freely available as open source software in Python.

Rod nuclear architecture determines contrast transmission of the retina and behavioral sensitivity in mice.

Rod photoreceptors of nocturnal mammals display a striking inversion of nuclear architecture, which has been proposed as an evolutionary adaptation to dark environments. However, the nature of visual benefits and the underlying mechanisms remains unclear. It is widely assumed that improvements in nocturnal vision would depend on maximization of photon capture at the expense of image detail. Here, we show that retinal optical quality improves 2-fold during terminal development, and that this enhancement is caused by nuclear inversion. We further demonstrate that improved retinal contrast transmission, rather than photon-budget or resolution, enhances scotopic contrast sensitivity by 18-27%, and improves motion detection capabilities up to 10-fold in dim environments. Our findings therefore add functional significance to a prominent exception of nuclear organization and establish retinal contrast transmission as a decisive determinant of mammalian visual perception.

Automated detection of the HER2 gene amplification status in Fluorescence in situ hybridization images for the diagnostics of cancer tissues.

The human epidermal growth factor receptor 2 (HER2) gene amplification status is a crucial marker for evaluating clinical therapies of breast or gastric cancer. We propose a deep learning-based pipeline for the detection, localization and classification of interphase nuclei depending on their HER2 gene amplification state in Fluorescence in situ hybridization (FISH) images. Our pipeline combines two RetinaNet-based object localization networks which are trained (1) to detect and classify interphase nuclei into distinct classes normal, low-grade and high-grade and (2) to detect and classify FISH signals into distinct classes HER2 or centromere of chromosome 17 (CEN17). By independently classifying each nucleus twice, the two-step pipeline provides both robustness and interpretability for the automated detection of the HER2 amplification status. The accuracy of our deep learning-based pipeline is on par with that of three pathologists and a set of 57 validation images containing several hundreds of nuclei are accurately classified. The automatic pipeline is a first step towards assisting pathologists in evaluating the HER2 status of tumors using FISH images, for analyzing FISH images in retrospective studies, and for optimizing the documentation of each tumor sample by automatically annotating and reporting of the HER2 gene amplification specificities.

Differential lateral and basal tension drive folding of Drosophila wing discs through two distinct mechanisms.

Epithelial folding transforms simple sheets of cells into complex three-dimensional tissues and organs during animal development. Epithelial folding has mainly been attributed to mechanical forces generated by an apically localized actomyosin network, however, contributions of forces generated at basal and lateral cell surfaces remain largely unknown. Here we show that a local decrease of basal tension and an increased lateral tension, but not apical constriction, drive the formation of two neighboring folds in developing Drosophila wing imaginal discs. Spatially defined reduction of extracellular matrix density results in local decrease of basal tension in the first fold; fluctuations in F-actin lead to increased lateral tension in the second fold. Simulations using a 3D vertex model show that the two distinct mechanisms can drive epithelial folding. Our combination of lateral and basal tension measurements with a mechanical tissue model reveals how simple modulations of surface and edge tension drive complex three-dimensional morphological changes.

Content-aware image restoration: pushing the limits of fluorescence microscopy.

Fluorescence microscopy is a key driver of discoveries in the life sciences, with observable phenomena being limited by the optics of the microscope, the chemistry of the fluorophores, and the maximum photon exposure tolerated by the sample. These limits necessitate trade-offs between imaging speed, spatial resolution, light exposure, and imaging depth. In this work we show how content-aware image restoration based on deep learning extends the range of biological phenomena observable by microscopy. We demonstrate on eight concrete examples how microscopy images can be restored even if 60-fold fewer photons are used during acquisition, how near isotropic resolution can be achieved with up to tenfold under-sampling along the axial direction, and how tubular and granular structures smaller than the diffraction limit can be resolved at 20-times-higher frame rates compared to state-of-the-art methods. All developed image restoration methods are freely available as open source software in Python, FIJI, and KNIME.

The influence of body composition on exercise-associated skin temperature changes after resistance training.

Resistance exercise leads to an increase in skin temperature (Tskin) in the area of the exercised muscle. Infrared thermography seems to be applicable to identify these primary used functional muscles with measuring Tskin changes. The aim of the current study was to investigate the influence of body composition on Tskin patterns after resistance exercise. 38 male subjects (19-32 years, BMI 20.4-55.2 kg/m2) participated. Body fat percentage and biceps skinfold thickness were calculated. The subjects were divided into two groups: lean group (LG) with body fat percentage < 25%, obese group (OG) with body fat percentage ≥ 25%. All participants completed three sets with ten repetitions of unilateral biceps curl at 50% of the one repetition maximum. To represent exercise-induced changes of Tskin to rest (Trest), the algebraic difference of each time point to Trest was calculated. The resulting delta values (∆) are as follows: immediately after the first, second, and third set (∆Tset1,∆Tset2,∆Tset3), and at 1,2,3,4,5,6,7,8,9,10,15,20,25,30 min after the third set (∆T1-∆T30). The maximum positive difference to Trest was defined as ∆Tmax, and the time to reach ∆Tmax was defined as Time to ∆Tmax. LG and OG differed significantly at Trest (32.8 ±â€¯0.9 vs. 31.1 ±â€¯1.4 °C), ∆Tmax (1.9 ±â€¯0.4 vs. 0.9 ±â€¯0.8 °C), Time to ∆Tmax (4.5 ±â€¯2.0 vs. 17.6 ±â€¯10.2 min) and at ∆Tset2 to ∆T15 (p < 0.005). Correlations between body composition (BMI, body fat percentage, biceps skinfold thickness) and Trest, ∆Tset2, ∆Tset3, ∆Tmax (-0.47