Deep neural network automated segmentation of cellular structures in volume electron microscopy.

Volume electron microscopy is an important imaging modality in contemporary cell biology. Identification of intracellular structures is a laborious process limiting the effective use of this potentially powerful tool. We resolved this bottleneck with automated segmentation of intracellular substructures in electron microscopy (ASEM), a new pipeline to train a convolutional neural network to detect structures of a wide range in size and complexity. We obtained dedicated models for each structure based on a small number of sparsely annotated ground truth images from only one or two cells. Model generalization was improved with a rapid, computationally effective strategy to refine a trained model by including a few additional annotations. We identified mitochondria, Golgi apparatus, endoplasmic reticulum, nuclear pore complexes, caveolae, clathrin-coated pits, and vesicles imaged by focused ion beam scanning electron microscopy. We uncovered a wide range of membrane-nuclear pore diameters within a single cell and derived morphological metrics from clathrin-coated pits and vesicles, consistent with the classical constant-growth assembly model.

Elastic Registration-driven Deep Learning for Longitudinal Assessment of Systemic Sclerosis Interstitial Lung Disease at CT.

Background Longitudinal follow-up of interstitial lung diseases (ILDs) at CT mainly relies on the evaluation of the extent of ILD, without accounting for lung shrinkage. Purpose To develop a deep learning-based method to depict worsening of ILD based on lung shrinkage detection from elastic registration of chest CT scans in patients with systemic sclerosis (SSc). Materials and Methods Patients with SSc evaluated between January 2009 and October 2017 who had undergone at least two unenhanced supine CT scans of the chest and pulmonary function tests (PFTs) performed within 3 months were retrospectively included. Morphologic changes on CT scans were visually assessed by two observers and categorized as showing improvement, stability, or worsening of ILD. Elastic registration between baseline and follow-up CT images was performed to obtain deformation maps of the whole lung. Jacobian determinants calculated from the deformation maps were given as input to a deep learning-based classifier to depict morphologic and functional worsening. For this purpose, the set was randomly split into training, validation, and test sets. Correlations between mean Jacobian values and changes in PFT measurements were evaluated with the Spearman correlation. Results A total of 212 patients (median age, 53 years; interquartile range, 45-62 years; 177 women) were included as follows: 138 for the training set (65%), 34 for the validation set (16%), and 40 for the test set (21%). Jacobian maps demonstrated lung parenchyma shrinkage of the posterior lung bases in patients found to have worsened ILD at visual assessment. The classifier detected morphologic and functional worsening with an accuracy of 80% (32 of 40 patients; 95% confidence interval [CI]: 64%, 91%) and 83% (33 of 40 patients; 95% CI: 67%, 93%), respectively. Jacobian values correlated with changes in forced vital capacity (R = -0.38; 95% CI: -0.25, -0.49; P < .001) and diffusing capacity for carbon monoxide (R = -0.42; 95% CI: -0.27, -0.54; P < .001). Conclusion Elastic registration of CT scans combined with a deep learning classifier aided in the diagnosis of morphologic and functional worsening of interstitial lung disease in patients with systemic sclerosis. © RSNA, 2020 Online supplemental material is available for this article. See also the editorial by Verschakelen in this issue.

Deep Multi-Instance Learning Using Multi-Modal Data for Diagnosis of Lymphocytosis.

We investigate the use of recent advances in deep learning and propose an end-to-end trainable multi-instance convolutional neural network within a mixture-of-experts formulation that combines information from two types of data-images and clinical attributes-for the diagnosis of lymphocytosis. The convolutional network learns to extract meaningful features from images of blood cells using an embedding level approach and aggregates them. Moreover, the mixture-of-experts model combines information from these images as well as clinical attributes to form an end-to-end trainable pipeline for diagnosis of lymphocytosis. Our results demonstrate that even the convolutional network by itself is able to discover meaningful associations between the images and the diagnosis, indicating the presence of important unexploited information in the images. The mixture-of-experts formulation is shown to be more robust while maintaining performance via. a repeatability study to assess the effect of variability in data acquisition on the predictions. The proposed methods are compared with different methods from literature based both on conventional handcrafted features and machine learning, and on recent deep learning models based on attention mechanisms. Our method reports a balanced accuracy of [Formula: see text] and outperfroms the handcrafted feature-based and attention-based approaches as well that of biologists which scored [Formula: see text], [Formula: see text] and [Formula: see text] respectively. These results give insights on the potentials of the applicability of the proposed method in clinical practice. Our code and datasets can be found at https://github.com/msahasrabudhe/lymphoMIL.

Deep Learning-Based Concurrent Brain Registration and Tumor Segmentation.

Image registration and segmentation are the two most studied problems in medical image analysis. Deep learning algorithms have recently gained a lot of attention due to their success and state-of-the-art results in variety of problems and communities. In this paper, we propose a novel, efficient, and multi-task algorithm that addresses the problems of image registration and brain tumor segmentation jointly. Our method exploits the dependencies between these tasks through a natural coupling of their interdependencies during inference. In particular, the similarity constraints are relaxed within the tumor regions using an efficient and relatively simple formulation. We evaluated the performance of our formulation both quantitatively and qualitatively for registration and segmentation problems on two publicly available datasets (BraTS 2018 and OASIS 3), reporting competitive results with other recent state-of-the-art methods. Moreover, our proposed framework reports significant amelioration (p < 0.005) for the registration performance inside the tumor locations, providing a generic method that does not need any predefined conditions (e.g., absence of abnormalities) about the volumes to be registered. Our implementation is publicly available online at https://github.com/TheoEst/joint_registration_tumor_segmentation.