A Comparative Analysis of Human Behavior Prediction Approaches in Intelligent Environments.

Behavior modeling has multiple applications in the intelligent environment domain. It has been used in different tasks, such as the stratification of different pathologies, prediction of the user actions and activities, or modeling the energy usage. Specifically, behavior prediction can be used to forecast the future evolution of the users and to identify those behaviors that deviate from the expected conduct. In this paper, we propose the use of embeddings to represent the user actions, and study and compare several behavior prediction approaches. We test multiple model (LSTM, CNNs, GCNs, and transformers) architectures to ascertain the best approach to using embeddings for behavior modeling and also evaluate multiple embedding retrofitting approaches. To do so, we use the Kasteren dataset for intelligent environments, which is one of the most widely used datasets in the areas of activity recognition and behavior modeling.

Deep Learning on Chest X-ray Images to Detect and Evaluate Pneumonia Cases at the Era of COVID-19.

Coronavirus disease 2019 (COVID-19) is an infectious disease with first symptoms similar to the flu. COVID-19 appeared first in China and very quickly spreads to the rest of the world, causing then the 2019-20 coronavirus pandemic. In many cases, this disease causes pneumonia. Since pulmonary infections can be observed through radiography images, this paper investigates deep learning methods for automatically analyzing query chest X-ray images with the hope to bring precision tools to health professionals towards screening the COVID-19 and diagnosing confirmed patients. In this context, training datasets, deep learning architectures and analysis strategies have been experimented from publicly open sets of chest X-ray images. Tailored deep learning models are proposed to detect pneumonia infection cases, notably viral cases. It is assumed that viral pneumonia cases detected during an epidemic COVID-19 context have a high probability to presume COVID-19 infections. Moreover, easy-to-apply health indicators are proposed for estimating infection status and predicting patient status from the detected pneumonia cases. Experimental results show possibilities of training deep learning models over publicly open sets of chest X-ray images towards screening viral pneumonia. Chest X-ray test images of COVID-19 infected patients are successfully diagnosed through detection models retained for their performances. The efficiency of proposed health indicators is highlighted through simulated scenarios of patients presenting infections and health problems by combining real and synthetic health data.

The human remains from Axlor (Dima, Biscay, northern Iberian Peninsula).

OBJECTIVES: We provide the description and comparative analysis of all the human fossil remains found at Axlor during the excavations carried out by J. M. de Barandiarán from 1967 to 1974: a cranial vault fragment and seven teeth, five of which likely belonged to the same individual, although two are currently lost. Our goal is to describe in detail all these human remains and discuss both their taxonomic attribution and their stratigraphic context. MATERIALS AND METHODS: We describe external and internal anatomy, and use classic and geometric morphometrics. The teeth from Axlor are compared to Neandertals, Upper Paleolithic, and recent modern humans. RESULTS: Two teeth (a left dm2 , a left di1 ) and the parietal fragment show morphological features consistent with a Neandertal classification, and were found in an undisturbed Mousterian context. The remaining three teeth (plus the two lost ones), initially classified as Neandertals, show morphological features and a general size that are more compatible with their classification as modern humans. DISCUSSION: A left parietal fragment (Level VIII) from a single probably adult Neandertal individual was recovered during the old excavations performed by Barandiarán. Additionally, two different Neandertal children lost deciduous teeth during the formations of levels V (left di1 ) and IV (right dm2 ). In addition, a modern human individual is represented by five remains (two currently lost) from a complex stratigraphic setting. Some of the morphological features of these remains suggest that they may represent one of the scarce examples of Upper Paleolithic modern human remains in the northern Iberian Peninsula, which should be confirmed by direct dating.

Trainable Weka Segmentation: a machine learning tool for microscopy pixel classification.

SUMMARY: State-of-the-art light and electron microscopes are capable of acquiring large image datasets, but quantitatively evaluating the data often involves manually annotating structures of interest. This process is time-consuming and often a major bottleneck in the evaluation pipeline. To overcome this problem, we have introduced the Trainable Weka Segmentation (TWS), a machine learning tool that leverages a limited number of manual annotations in order to train a classifier and segment the remaining data automatically. In addition, TWS can provide unsupervised segmentation learning schemes (clustering) and can be customized to employ user-designed image features or classifiers. AVAILABILITY AND IMPLEMENTATION: TWS is distributed as open-source software as part of the Fiji image processing distribution of ImageJ at http://imagej.net/Trainable_Weka_Segmentation . CONTACT: ignacio.arganda@ehu.eus. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

MorphoLibJ: integrated library and plugins for mathematical morphology with ImageJ.

MOTIVATION: Mathematical morphology (MM) provides many powerful operators for processing 2D and 3D images. However, most MM plugins currently implemented for the popular ImageJ/Fiji platform are limited to the processing of 2D images. RESULTS: The MorphoLibJ library proposes a large collection of generic tools based on MM to process binary and grey-level 2D and 3D images, integrated into user-friendly plugins. We illustrate how MorphoLibJ can facilitate the exploitation of 3D images of plant tissues. AVAILABILITY AND IMPLEMENTATION: MorphoLibJ is freely available at http://imagej.net/MorphoLibJ CONTACT: david.legland@nantes.inra.frSupplementary information: Supplementary data are available at Bioinformatics online.

NucleusJ: an ImageJ plugin for quantifying 3D images of interphase nuclei.

UNLABELLED: NucleusJ is a simple and user-friendly ImageJ plugin dedicated to the characterization of nuclear morphology and chromatin organization in 3D. Starting from image stacks, the nuclear boundary is delimited by combining the Otsu segmentation method with optimization of nuclear sphericity. Chromatin domains are segmented by partitioning the nucleus using a 3D watershed algorithm and by thresholding a contrast measure over the resulting regions. As output, NucleusJ quantifies 15 parameters including shape and size of nuclei as well as intra-nuclear objects and their position within the nucleus. A step-by-step documentation is available for self-training, together with data sets of nuclei with different nuclear organization. AVAILABILITY AND IMPLEMENTATION: Dataset of nuclei is available at https://www.gred-clermont.fr/media/WorkDirectory.zip. NucleusJ is available at http://imagejdocu.tudor.lu/doku.php?id=plugin:stacks:nuclear_analysis_plugin:start. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Serial two-photon tomography for automated ex vivo mouse brain imaging.

Here we describe an automated method, named serial two-photon (STP) tomography, that achieves high-throughput fluorescence imaging of mouse brains by integrating two-photon microscopy and tissue sectioning. STP tomography generates high-resolution datasets that are free of distortions and can be readily warped in three dimensions, for example, for comparing multiple anatomical tracings. This method opens the door to routine systematic studies of neuroanatomy in mouse models of human brain disorders.

Fiji: an open-source platform for biological-image analysis.

Fiji is a distribution of the popular open-source software ImageJ focused on biological-image analysis. Fiji uses modern software engineering practices to combine powerful software libraries with a broad range of scripting languages to enable rapid prototyping of image-processing algorithms. Fiji facilitates the transformation of new algorithms into ImageJ plugins that can be shared with end users through an integrated update system. We propose Fiji as a platform for productive collaboration between computer science and biology research communities.

Phenotyping nematode feeding sites: three-dimensional reconstruction and volumetric measurements of giant cells induced by root-knot nematodes in Arabidopsis.

The control of plant parasitic nematodes is an increasing problem. A key process during the infection is the induction of specialized nourishing cells, called giant cells (GCs), in roots. Understanding the function of genes required for GC development is crucial to identify targets for new control strategies. We propose a standardized method for GC phenotyping in different plant genotypes, like those with modified genes essential for GC development. The method combines images obtained by bright-field microscopy from the complete serial sectioning of galls with TrakEM2, specialized three-dimensional (3D) reconstruction software for biological structures. The volumes and shapes from 162 3D models of individual GCs induced by Meloidogyne javanica in Arabidopsis were analyzed for the first time along their life cycle. A high correlation between the combined volume of all GCs within a gall and the total area occupied by all the GCs in the section/s where they show maximum expansion, and a proof of concept from two Arabidopsis transgenic lines (J0121 â‰« DTA and J0121 â‰« GFP) demonstrate the reliability of the method. We phenotyped GCs and developed a reliable simplified method based on a two-dimensional (2D) parameter for comparison of GCs from different Arabidopsis genotypes, which is also applicable to galls from different plant species and in different growing conditions, as thickness/transparency is not a restriction.

A generic classification-based method for segmentation of nuclei in 3D images of early embryos.

BACKGROUND: Studying how individual cells spatially and temporally organize within the embryo is a fundamental issue in modern developmental biology to better understand the first stages of embryogenesis. In order to perform high-throughput analyses in three-dimensional microscopic images, it is essential to be able to automatically segment, classify and track cell nuclei. Many 3D/4D segmentation and tracking algorithms have been reported in the literature. Most of them are specific to particular models or acquisition systems and often require the fine tuning of parameters. RESULTS: We present a new automatic algorithm to segment and simultaneously classify cell nuclei in 3D/4D images. Segmentation relies on training samples that are interactively provided by the user and on an iterative thresholding process. This algorithm can correctly segment nuclei even when they are touching, and remains effective under temporal and spatial intensity variations. The segmentation is coupled to a classification of nuclei according to cell cycle phases, allowing biologists to quantify the effect of genetic perturbations and drug treatments. Robust 3D geometrical shape descriptors are used as training features for classification. Segmentation and classification results of three complete datasets are presented. In our working dataset of the Caenorhabditis elegans embryo, only 21 nuclei out of 3,585 were not detected, the overall F-score for segmentation reached 0.99, and more than 95% of the nuclei were classified in the correct cell cycle phase. No merging of nuclei was found. CONCLUSION: We developed a novel generic algorithm for segmentation and classification in 3D images. The method, referred to as Adaptive Generic Iterative Thresholding Algorithm (AGITA), is freely available as an ImageJ plug-in.

3D Domain Adaptive Instance Segmentation via Cyclic Segmentation GANs.

3D instance segmentation for unlabeled imaging modalities is a challenging but essential task as collecting expert annotation can be expensive and time-consuming. Existing works segment a new modality by either deploying pre-trained models optimized on diverse training data or sequentially conducting image translation and segmentation with two relatively independent networks. In this work, we propose a novel Cyclic Segmentation Generative Adversarial Network (CySGAN) that conducts image translation and instance segmentation simultaneously using a unified network with weight sharing. Since the image translation layer can be removed at inference time, our proposed model does not introduce additional computational cost upon a standard segmentation model. For optimizing CySGAN, besides the CycleGAN losses for image translation and supervised losses for the annotated source domain, we also utilize self-supervised and segmentation-based adversarial objectives to enhance the model performance by leveraging unlabeled target domain images. We benchmark our approach on the task of 3D neuronal nuclei segmentation with annotated electron microscopy (EM) images and unlabeled expansion microscopy (ExM) data. The proposed CySGAN outperforms pre-trained generalist models, feature-level domain adaptation models, and the baselines that conduct image translation and segmentation sequentially. Our implementation and the newly collected, densely annotated ExM zebrafish brain nuclei dataset, named NucExM, are publicly available at https://connectomics-bazaar.github.io/proj/CySGAN/index.html.

CartoCell, a high-content pipeline for 3D image analysis, unveils cell morphology patterns in epithelia.

Decades of research have not yet fully explained the mechanisms of epithelial self-organization and 3D packing. Single-cell analysis of large 3D epithelial libraries is crucial for understanding the assembly and function of whole tissues. Combining 3D epithelial imaging with advanced deep-learning segmentation methods is essential for enabling this high-content analysis. We introduce CartoCell, a deep-learning-based pipeline that uses small datasets to generate accurate labels for hundreds of whole 3D epithelial cysts. Our method detects the realistic morphology of epithelial cells and their contacts in the 3D structure of the tissue. CartoCell enables the quantification of geometric and packing features at the cellular level. Our single-cell cartography approach then maps the distribution of these features on 2D plots and 3D surface maps, revealing cell morphology patterns in epithelial cysts. Additionally, we show that CartoCell can be adapted to other types of epithelial tissues.

Current Progress and Challenges in Large-Scale 3D Mitochondria Instance Segmentation.

In this paper, we present the results of the MitoEM challenge on mitochondria 3D instance segmentation from electron microscopy images, organized in conjunction with the IEEE-ISBI 2021 conference. Our benchmark dataset consists of two large-scale 3D volumes, one from human and one from rat cortex tissue, which are 1,986 times larger than previously used datasets. At the time of paper submission, 257 participants had registered for the challenge, 14 teams had submitted their results, and six teams participated in the challenge workshop. Here, we present eight top-performing approaches from the challenge participants, along with our own baseline strategies. Posterior to the challenge, annotation errors in the ground truth were corrected without altering the final ranking. Additionally, we present a retrospective evaluation of the scoring system which revealed that: 1) challenge metric was permissive with the false positive predictions; and 2) size-based grouping of instances did not correctly categorize mitochondria of interest. Thus, we propose a new scoring system that better reflects the correctness of the segmentation results. Although several of the top methods are compared favorably to our own baselines, substantial errors remain unsolved for mitochondria with challenging morphologies. Thus, the challenge remains open for submission and automatic evaluation, with all volumes available for download.

Stable Deep Neural Network Architectures for Mitochondria Segmentation on Electron Microscopy Volumes.

Electron microscopy (EM) allows the identification of intracellular organelles such as mitochondria, providing insights for clinical and scientific studies. In recent years, a number of novel deep learning architectures have been published reporting superior performance, or even human-level accuracy, compared to previous approaches on public mitochondria segmentation datasets. Unfortunately, many of these publications make neither the code nor the full training details public, leading to reproducibility issues and dubious model comparisons. Thus, following a recent code of best practices in the field, we present an extensive study of the state-of-the-art architectures and compare them to different variations of U-Net-like models for this task. To unveil the impact of architectural novelties, a common set of pre- and post-processing operations has been implemented and tested with each approach. Moreover, an exhaustive sweep of hyperparameters has been performed, running each configuration multiple times to measure their stability. Using this methodology, we found very stable architectures and training configurations that consistently obtain state-of-the-art results in the well-known EPFL Hippocampus mitochondria segmentation dataset and outperform all previous works on two other available datasets: Lucchi++ and Kasthuri++. The code and its documentation are publicly available at https://github.com/danifranco/EM_Image_Segmentation .

Deep learning based domain adaptation for mitochondria segmentation on EM volumes.

BACKGROUND AND OBJECTIVE: Accurate segmentation of electron microscopy (EM) volumes of the brain is essential to characterize neuronal structures at a cell or organelle level. While supervised deep learning methods have led to major breakthroughs in that direction during the past years, they usually require large amounts of annotated data to be trained, and perform poorly on other data acquired under similar experimental and imaging conditions. This is a problem known as domain adaptation, since models that learned from a sample distribution (or source domain) struggle to maintain their performance on samples extracted from a different distribution or target domain. In this work, we address the complex case of deep learning based domain adaptation for mitochondria segmentation across EM datasets from different tissues and species. METHODS: We present three unsupervised domain adaptation strategies to improve mitochondria segmentation in the target domain based on (1) state-of-the-art style transfer between images of both domains; (2) self-supervised learning to pre-train a model using unlabeled source and target images, and then fine-tune it only with the source labels; and (3) multi-task neural network architectures trained end-to-end with both labeled and unlabeled images. Additionally, to ensure good generalization in our models, we propose a new training stopping criterion based on morphological priors obtained exclusively in the source domain. The code and its documentation are publicly available at https://github.com/danifranco/EM_domain_adaptation. RESULTS: We carried out all possible cross-dataset experiments using three publicly available EM datasets. We evaluated our proposed strategies and those of others based on the mitochondria semantic labels predicted on the target datasets. CONCLUSIONS: The methods introduced here outperform the baseline methods and compare favorably to the state of the art. In the absence of validation labels, monitoring our proposed morphology-based metric is an intuitive and effective way to stop the training process and select in average optimal models.

Brain virtual histology with X-ray phase-contrast tomography Part II:3D morphologies of amyloid-ß plaques in Alzheimer's disease models.

While numerous transgenic mouse strains have been produced to model the formation of amyloid-ß (Aß) plaques in the brain, efficient methods for whole-brain 3D analysis of Aß deposits have to be validated and standardized. Moreover, routine immunohistochemistry performed on brain slices precludes any shape analysis of Aß plaques, or require complex procedures for serial acquisition and reconstruction. The present study shows how in-line (propagation-based) X-ray phase-contrast tomography (XPCT) combined with ethanol-induced brain sample dehydration enables hippocampus-wide detection and morphometric analysis of Aß plaques. Performed in three distinct Alzheimer mouse strains, the proposed workflow identified differences in signal intensity and 3D shape parameters: 3xTg displayed a different type of Aß plaques, with a larger volume and area, greater elongation, flatness and mean breadth, and more intense average signal than J20 and APP/PS1. As a label-free non-destructive technique, XPCT can be combined with standard immunohistochemistry. XPCT virtual histology could thus become instrumental in quantifying the 3D spreading and the morphological impact of seeding when studying prion-like properties of Aß aggregates in animal models of Alzheimer's disease. This is Part II of a series of two articles reporting the value of in-line XPCT for virtual histology of the brain; Part I shows how in-line XPCT enables 3D myelin mapping in the whole rodent brain and in human autopsy brain tissue.

A quantitative biophysical principle to explain the 3D cellular connectivity in curved epithelia.

Epithelial cell organization and the mechanical stability of tissues are closely related. In this context, it has been recently shown that packing optimization in bended or folded epithelia is achieved by an energy minimization mechanism that leads to a complex cellular shape: the "scutoid". Here, we focus on the relationship between this shape and the connectivity between cells. We use a combination of computational, experimental, and biophysical approaches to examine how energy drivers affect the three-dimensional (3D) packing of tubular epithelia. We propose an energy-based stochastic model that explains the 3D cellular connectivity. Then, we challenge it by experimentally reducing the cell adhesion. As a result, we observed an increment in the appearance of scutoids that correlated with a decrease in the energy barrier necessary to connect with new cells. We conclude that tubular epithelia satisfy a quantitative biophysical principle that links tissue geometry and energetics with the average cellular connectivity.

MitoEM Dataset: Large-scale 3D Mitochondria Instance Segmentation from EM Images.

Electron microscopy (EM) allows the identification of intracellular organelles such as mitochondria, providing insights for clinical and scientific studies. However, public mitochondria segmentation datasets only contain hundreds of instances with simple shapes. It is unclear if existing methods achieving human-level accuracy on these small datasets are robust in practice. To this end, we introduce the MitoEM dataset, a 3D mitochondria instance segmentation dataset with two (30µm)3 volumes from human and rat cortices respectively, 3, 600× larger than previous benchmarks. With around 40K instances, we find a great diversity of mitochondria in terms of shape and density. For evaluation, we tailor the implementation of the average precision (AP) metric for 3D data with a 45× speedup. On MitoEM, we find existing instance segmentation methods often fail to correctly segment mitochondria with complex shapes or close contacts with other instances. Thus, our MitoEM dataset poses new challenges to the field. We release our code and data: https://donglaiw.github.io/page/mitoEM/index.html.