Current Progress and Challenges in Large-Scale 3D Mitochondria Instance Segmentation.

In this paper, we present the results of the MitoEM challenge on mitochondria 3D instance segmentation from electron microscopy images, organized in conjunction with the IEEE-ISBI 2021 conference. Our benchmark dataset consists of two large-scale 3D volumes, one from human and one from rat cortex tissue, which are 1,986 times larger than previously used datasets. At the time of paper submission, 257 participants had registered for the challenge, 14 teams had submitted their results, and six teams participated in the challenge workshop. Here, we present eight top-performing approaches from the challenge participants, along with our own baseline strategies. Posterior to the challenge, annotation errors in the ground truth were corrected without altering the final ranking. Additionally, we present a retrospective evaluation of the scoring system which revealed that: 1) challenge metric was permissive with the false positive predictions; and 2) size-based grouping of instances did not correctly categorize mitochondria of interest. Thus, we propose a new scoring system that better reflects the correctness of the segmentation results. Although several of the top methods are compared favorably to our own baselines, substantial errors remain unsolved for mitochondria with challenging morphologies. Thus, the challenge remains open for submission and automatic evaluation, with all volumes available for download.

CartoCell, a high-content pipeline for 3D image analysis, unveils cell morphology patterns in epithelia.

Decades of research have not yet fully explained the mechanisms of epithelial self-organization and 3D packing. Single-cell analysis of large 3D epithelial libraries is crucial for understanding the assembly and function of whole tissues. Combining 3D epithelial imaging with advanced deep-learning segmentation methods is essential for enabling this high-content analysis. We introduce CartoCell, a deep-learning-based pipeline that uses small datasets to generate accurate labels for hundreds of whole 3D epithelial cysts. Our method detects the realistic morphology of epithelial cells and their contacts in the 3D structure of the tissue. CartoCell enables the quantification of geometric and packing features at the cellular level. Our single-cell cartography approach then maps the distribution of these features on 2D plots and 3D surface maps, revealing cell morphology patterns in epithelial cysts. Additionally, we show that CartoCell can be adapted to other types of epithelial tissues.

3D Domain Adaptive Instance Segmentation via Cyclic Segmentation GANs.

3D instance segmentation for unlabeled imaging modalities is a challenging but essential task as collecting expert annotation can be expensive and time-consuming. Existing works segment a new modality by either deploying pre-trained models optimized on diverse training data or sequentially conducting image translation and segmentation with two relatively independent networks. In this work, we propose a novel Cyclic Segmentation Generative Adversarial Network (CySGAN) that conducts image translation and instance segmentation simultaneously using a unified network with weight sharing. Since the image translation layer can be removed at inference time, our proposed model does not introduce additional computational cost upon a standard segmentation model. For optimizing CySGAN, besides the CycleGAN losses for image translation and supervised losses for the annotated source domain, we also utilize self-supervised and segmentation-based adversarial objectives to enhance the model performance by leveraging unlabeled target domain images. We benchmark our approach on the task of 3D neuronal nuclei segmentation with annotated electron microscopy (EM) images and unlabeled expansion microscopy (ExM) data. The proposed CySGAN outperforms pre-trained generalist models, feature-level domain adaptation models, and the baselines that conduct image translation and segmentation sequentially. Our implementation and the newly collected, densely annotated ExM zebrafish brain nuclei dataset, named NucExM, are publicly available at https://connectomics-bazaar.github.io/proj/CySGAN/index.html.

A quantitative biophysical principle to explain the 3D cellular connectivity in curved epithelia.

Epithelial cell organization and the mechanical stability of tissues are closely related. In this context, it has been recently shown that packing optimization in bended or folded epithelia is achieved by an energy minimization mechanism that leads to a complex cellular shape: the "scutoid". Here, we focus on the relationship between this shape and the connectivity between cells. We use a combination of computational, experimental, and biophysical approaches to examine how energy drivers affect the three-dimensional (3D) packing of tubular epithelia. We propose an energy-based stochastic model that explains the 3D cellular connectivity. Then, we challenge it by experimentally reducing the cell adhesion. As a result, we observed an increment in the appearance of scutoids that correlated with a decrease in the energy barrier necessary to connect with new cells. We conclude that tubular epithelia satisfy a quantitative biophysical principle that links tissue geometry and energetics with the average cellular connectivity.

Deep learning based domain adaptation for mitochondria segmentation on EM volumes.

BACKGROUND AND OBJECTIVE: Accurate segmentation of electron microscopy (EM) volumes of the brain is essential to characterize neuronal structures at a cell or organelle level. While supervised deep learning methods have led to major breakthroughs in that direction during the past years, they usually require large amounts of annotated data to be trained, and perform poorly on other data acquired under similar experimental and imaging conditions. This is a problem known as domain adaptation, since models that learned from a sample distribution (or source domain) struggle to maintain their performance on samples extracted from a different distribution or target domain. In this work, we address the complex case of deep learning based domain adaptation for mitochondria segmentation across EM datasets from different tissues and species. METHODS: We present three unsupervised domain adaptation strategies to improve mitochondria segmentation in the target domain based on (1) state-of-the-art style transfer between images of both domains; (2) self-supervised learning to pre-train a model using unlabeled source and target images, and then fine-tune it only with the source labels; and (3) multi-task neural network architectures trained end-to-end with both labeled and unlabeled images. Additionally, to ensure good generalization in our models, we propose a new training stopping criterion based on morphological priors obtained exclusively in the source domain. The code and its documentation are publicly available at https://github.com/danifranco/EM_domain_adaptation. RESULTS: We carried out all possible cross-dataset experiments using three publicly available EM datasets. We evaluated our proposed strategies and those of others based on the mitochondria semantic labels predicted on the target datasets. CONCLUSIONS: The methods introduced here outperform the baseline methods and compare favorably to the state of the art. In the absence of validation labels, monitoring our proposed morphology-based metric is an intuitive and effective way to stop the training process and select in average optimal models.

Brain virtual histology with X-ray phase-contrast tomography Part II:3D morphologies of amyloid-ß plaques in Alzheimer's disease models.

While numerous transgenic mouse strains have been produced to model the formation of amyloid-ß (Aß) plaques in the brain, efficient methods for whole-brain 3D analysis of Aß deposits have to be validated and standardized. Moreover, routine immunohistochemistry performed on brain slices precludes any shape analysis of Aß plaques, or require complex procedures for serial acquisition and reconstruction. The present study shows how in-line (propagation-based) X-ray phase-contrast tomography (XPCT) combined with ethanol-induced brain sample dehydration enables hippocampus-wide detection and morphometric analysis of Aß plaques. Performed in three distinct Alzheimer mouse strains, the proposed workflow identified differences in signal intensity and 3D shape parameters: 3xTg displayed a different type of Aß plaques, with a larger volume and area, greater elongation, flatness and mean breadth, and more intense average signal than J20 and APP/PS1. As a label-free non-destructive technique, XPCT can be combined with standard immunohistochemistry. XPCT virtual histology could thus become instrumental in quantifying the 3D spreading and the morphological impact of seeding when studying prion-like properties of Aß aggregates in animal models of Alzheimer's disease. This is Part II of a series of two articles reporting the value of in-line XPCT for virtual histology of the brain; Part I shows how in-line XPCT enables 3D myelin mapping in the whole rodent brain and in human autopsy brain tissue.

A Comparative Analysis of Human Behavior Prediction Approaches in Intelligent Environments.

Behavior modeling has multiple applications in the intelligent environment domain. It has been used in different tasks, such as the stratification of different pathologies, prediction of the user actions and activities, or modeling the energy usage. Specifically, behavior prediction can be used to forecast the future evolution of the users and to identify those behaviors that deviate from the expected conduct. In this paper, we propose the use of embeddings to represent the user actions, and study and compare several behavior prediction approaches. We test multiple model (LSTM, CNNs, GCNs, and transformers) architectures to ascertain the best approach to using embeddings for behavior modeling and also evaluate multiple embedding retrofitting approaches. To do so, we use the Kasteren dataset for intelligent environments, which is one of the most widely used datasets in the areas of activity recognition and behavior modeling.

Stable Deep Neural Network Architectures for Mitochondria Segmentation on Electron Microscopy Volumes.

Electron microscopy (EM) allows the identification of intracellular organelles such as mitochondria, providing insights for clinical and scientific studies. In recent years, a number of novel deep learning architectures have been published reporting superior performance, or even human-level accuracy, compared to previous approaches on public mitochondria segmentation datasets. Unfortunately, many of these publications make neither the code nor the full training details public, leading to reproducibility issues and dubious model comparisons. Thus, following a recent code of best practices in the field, we present an extensive study of the state-of-the-art architectures and compare them to different variations of U-Net-like models for this task. To unveil the impact of architectural novelties, a common set of pre- and post-processing operations has been implemented and tested with each approach. Moreover, an exhaustive sweep of hyperparameters has been performed, running each configuration multiple times to measure their stability. Using this methodology, we found very stable architectures and training configurations that consistently obtain state-of-the-art results in the well-known EPFL Hippocampus mitochondria segmentation dataset and outperform all previous works on two other available datasets: Lucchi++ and Kasthuri++. The code and its documentation are publicly available at https://github.com/danifranco/EM_Image_Segmentation .

Deep Learning on Chest X-ray Images to Detect and Evaluate Pneumonia Cases at the Era of COVID-19.

Coronavirus disease 2019 (COVID-19) is an infectious disease with first symptoms similar to the flu. COVID-19 appeared first in China and very quickly spreads to the rest of the world, causing then the 2019-20 coronavirus pandemic. In many cases, this disease causes pneumonia. Since pulmonary infections can be observed through radiography images, this paper investigates deep learning methods for automatically analyzing query chest X-ray images with the hope to bring precision tools to health professionals towards screening the COVID-19 and diagnosing confirmed patients. In this context, training datasets, deep learning architectures and analysis strategies have been experimented from publicly open sets of chest X-ray images. Tailored deep learning models are proposed to detect pneumonia infection cases, notably viral cases. It is assumed that viral pneumonia cases detected during an epidemic COVID-19 context have a high probability to presume COVID-19 infections. Moreover, easy-to-apply health indicators are proposed for estimating infection status and predicting patient status from the detected pneumonia cases. Experimental results show possibilities of training deep learning models over publicly open sets of chest X-ray images towards screening viral pneumonia. Chest X-ray test images of COVID-19 infected patients are successfully diagnosed through detection models retained for their performances. The efficiency of proposed health indicators is highlighted through simulated scenarios of patients presenting infections and health problems by combining real and synthetic health data.

MitoEM Dataset: Large-scale 3D Mitochondria Instance Segmentation from EM Images.

Electron microscopy (EM) allows the identification of intracellular organelles such as mitochondria, providing insights for clinical and scientific studies. However, public mitochondria segmentation datasets only contain hundreds of instances with simple shapes. It is unclear if existing methods achieving human-level accuracy on these small datasets are robust in practice. To this end, we introduce the MitoEM dataset, a 3D mitochondria instance segmentation dataset with two (30µm)3 volumes from human and rat cortices respectively, 3, 600× larger than previous benchmarks. With around 40K instances, we find a great diversity of mitochondria in terms of shape and density. For evaluation, we tailor the implementation of the average precision (AP) metric for 3D data with a 45× speedup. On MitoEM, we find existing instance segmentation methods often fail to correctly segment mitochondria with complex shapes or close contacts with other instances. Thus, our MitoEM dataset poses new challenges to the field. We release our code and data: https://donglaiw.github.io/page/mitoEM/index.html.

Freeze-frame imaging of synaptic activity using SynTagMA.

Information within the brain travels from neuron to neuron across billions of synapses. At any given moment, only a small subset of neurons and synapses are active, but finding the active synapses in brain tissue has been a technical challenge. Here we introduce SynTagMA to tag active synapses in a user-defined time window. Upon 395-405 nm illumination, this genetically encoded marker of activity converts from green to red fluorescence if, and only if, it is bound to calcium. Targeted to presynaptic terminals, preSynTagMA allows discrimination between active and silent axons. Targeted to excitatory postsynapses, postSynTagMA creates a snapshot of synapses active just before photoconversion. To analyze large datasets, we show how to identify and track the fluorescence of thousands of individual synapses in an automated fashion. Together, these tools provide an efficient method for repeatedly mapping active neurons and synapses in cell culture, slice preparations, and in vivo during behavior.

ANHIR: Automatic Non-Rigid Histological Image Registration Challenge.

Automatic Non-rigid Histological Image Registration (ANHIR) challenge was organized to compare the performance of image registration algorithms on several kinds of microscopy histology images in a fair and independent manner. We have assembled 8 datasets, containing 355 images with 18 different stains, resulting in 481 image pairs to be registered. Registration accuracy was evaluated using manually placed landmarks. In total, 256 teams registered for the challenge, 10 submitted the results, and 6 participated in the workshop. Here, we present the results of 7 well-performing methods from the challenge together with 6 well-known existing methods. The best methods used coarse but robust initial alignment, followed by non-rigid registration, used multiresolution, and were carefully tuned for the data at hand. They outperformed off-the-shelf methods, mostly by being more robust. The best methods could successfully register over 98% of all landmarks and their mean landmark registration accuracy (TRE) was 0.44% of the image diagonal. The challenge remains open to submissions and all images are available for download.

The human remains from Axlor (Dima, Biscay, northern Iberian Peninsula).

OBJECTIVES: We provide the description and comparative analysis of all the human fossil remains found at Axlor during the excavations carried out by J. M. de Barandiarán from 1967 to 1974: a cranial vault fragment and seven teeth, five of which likely belonged to the same individual, although two are currently lost. Our goal is to describe in detail all these human remains and discuss both their taxonomic attribution and their stratigraphic context. MATERIALS AND METHODS: We describe external and internal anatomy, and use classic and geometric morphometrics. The teeth from Axlor are compared to Neandertals, Upper Paleolithic, and recent modern humans. RESULTS: Two teeth (a left dm2 , a left di1 ) and the parietal fragment show morphological features consistent with a Neandertal classification, and were found in an undisturbed Mousterian context. The remaining three teeth (plus the two lost ones), initially classified as Neandertals, show morphological features and a general size that are more compatible with their classification as modern humans. DISCUSSION: A left parietal fragment (Level VIII) from a single probably adult Neandertal individual was recovered during the old excavations performed by Barandiarán. Additionally, two different Neandertal children lost deciduous teeth during the formations of levels V (left di1 ) and IV (right dm2 ). In addition, a modern human individual is represented by five remains (two currently lost) from a complex stratigraphic setting. Some of the morphological features of these remains suggest that they may represent one of the scarce examples of Upper Paleolithic modern human remains in the northern Iberian Peninsula, which should be confirmed by direct dating.

Correction: Division of labor and brain evolution in insect societies: Neurobiology of extreme specialization in the turtle ant Cephalotes varians.

[This corrects the article DOI: 10.1371/journal.pone.0213618.].

Publisher Correction: Multicolor multiscale brain imaging with chromatic multiphoton serial microscopy.

Affiliation 4 incorrectly read 'University of the Basque Country (Ikerbasque), University of the Basque Country and Donostia International Physics Center, San Sebastian 20018, Spain.'Also, the affiliations of Ignacio Arganda-Carreras with 'IKERBASQUE, Basque Foundation for Science, Bilbao, 48013, Spain' and 'Donostia International Physics Center (DIPC), San Sebastian, 20018, Spain' were inadvertently omitted.Additionally, the third sentence of the first paragraph of the Results section entitled 'Multicontrast organ-scale imaging with ChroMS microscopy' incorrectly read 'For example, one can choose lambda1 = 850 and lambda2 = 110 nm for optimal two-photon excitation of blue and red chromophores.'. The correct version reads 'lambda2 = 1100 nm' instead of 'lambda2 = 110 nm'. These errors have now been corrected in the PDF and HTML versions of the Article.

Multicolor multiscale brain imaging with chromatic multiphoton serial microscopy.

Large-scale microscopy approaches are transforming brain imaging, but currently lack efficient multicolor contrast modalities. We introduce chromatic multiphoton serial (ChroMS) microscopy, a method integrating one-shot multicolor multiphoton excitation through wavelength mixing and serial block-face image acquisition. This approach provides organ-scale micrometric imaging of spectrally distinct fluorescent proteins and label-free nonlinear signals with constant micrometer-scale resolution and sub-micron channel registration over the entire imaged volume. We demonstrate tridimensional (3D) multicolor imaging over several cubic millimeters as well as brain-wide serial 2D multichannel imaging. We illustrate the strengths of this method through color-based 3D analysis of astrocyte morphology and contacts in the mouse cerebral cortex, tracing of individual pyramidal neurons within densely Brainbow-labeled tissue, and multiplexed whole-brain mapping of axonal projections labeled with spectrally distinct tracers. ChroMS will be an asset for multiscale and system-level studies in neuroscience and beyond.

Division of labor and brain evolution in insect societies: Neurobiology of extreme specialization in the turtle ant Cephalotes varians.

Strongly polyphenic social insects provide excellent models to examine the neurobiological basis of division of labor. Turtle ants, Cephalotes varians, have distinct minor worker, soldier, and reproductive (gyne/queen) morphologies associated with their behavioral profiles: small-bodied task-generalist minors lack the phragmotic shield-shaped heads of soldiers, which are specialized to block and guard the nest entrance. Gynes found new colonies and during early stages of colony growth overlap behaviorally with soldiers. Here we describe patterns of brain structure and synaptic organization associated with division of labor in C. varians minor workers, soldiers, and gynes. We quantified brain volumes, determined scaling relationships among brain regions, and quantified the density and size of microglomeruli, synaptic complexes in the mushroom body calyxes important to higher-order processing abilities that may underpin behavioral performance. We found that brain volume was significantly larger in gynes; minor workers and soldiers had similar brain sizes. Consistent with their larger behavioral repertoire, minors had disproportionately larger mushroom bodies than soldiers and gynes. Soldiers and gynes had larger optic lobes, which may be important for flight and navigation in gynes, but serve different functions in soldiers. Microglomeruli were larger and less dense in minor workers; soldiers and gynes did not differ. Correspondence in brain structure despite differences in soldiers and gyne behavior may reflect developmental integration, suggesting that neurobiological metrics not only advance our understanding of brain evolution in social insects, but may also help resolve questions of the origin of novel castes.

WDR20 regulates shuttling of the USP12 deubiquitinase complex between the plasma membrane, cytoplasm and nucleus.

The human deubiquitinases USP12 and USP46 are very closely related paralogs with critical functions as tumor suppressors. The catalytic activity of these enzymes is regulated by two cofactors: UAF1 and WDR20. USP12 and USP46 show nearly 90% amino acid sequence identity and share some cellular activities, but have also evolved non-overlapping functions. We hypothesized that, correlating with their functional divergence, the subcellular localization of USP12 and USP46 might be differentially regulated by their cofactors. We used confocal and live microscopy analyses of epitope-tagged proteins to determine the effect of UAF1 and WDR20 on the localization of USP12 and USP46. We found that WDR20 differently modulated the localization of the DUBs, promoting recruitment of USP12, but not USP46, to the plasma membrane. Using site-directed mutagenesis, we generated a large set of USP12 and WDR20 mutants to characterize in detail the mechanisms and sequence determinants that modulate the subcellular localization of the USP12/UAF1/WDR20 complex. Our data suggest that the USP12/UAF1/WDR20 complex dynamically shuttles between the plasma membrane, cytoplasm and nucleus. This shuttling involved active nuclear export mediated by the CRM1 pathway, and required a short N-terminal motif (1MEIL4) in USP12, as well as a novel nuclear export sequence (450MDGAIASGVSKFATLSLHD468) in WDR20. In conclusion, USP12 and USP46 have evolved divergently in terms of cofactor binding-regulated subcellular localization. WDR20 plays a crucial role in as a "targeting subunit" that modulates CRM1-dependent shuttling of the USP12/UAF1/WDR20 complex between the plasma membrane, cytoplasm and nucleus.