Artificial Intelligence for Automatic Analysis of Shunt Treatment in Presurgery and Postsurgery Computed Tomography Brain Scans of Patients With Idiopathic Normal Pressure Hydrocephalus.

BACKGROUND AND OBJECTIVES: Ventriculo-peritoneal shunt procedures can improve idiopathic normal pressure hydrocephalus (iNPH) symptoms. However, there are no automated methods that quantify the presurgery and postsurgery changes in the ventricular volume for computed tomography scans. Hence, the main goal of this research was to quantify longitudinal changes in the ventricular volume and its correlation with clinical improvement in iNPH symptoms. Furthermore, our objective was to develop an end-to-end graphical interface where surgeons can directly drag-drop a brain scan for quantified analysis. METHODS: A total of 15 patients with 47 longitudinal computed tomography scans were taken before and after shunt surgery. Postoperative scans were collected between 1 and 45 months. We use a UNet-based model to develop a fully automated metric. Center slices of the scan that are most representative (80%) of the ventricular volume of the brain are used. Clinical symptoms of gait, balance, cognition, and bladder continence are studied with respect to the proposed metric. RESULTS: Fifteen patients with iNPH demonstrate a decrease in ventricular volume (as shown by our metric) postsurgery and a concurrent clinical improvement in their iNPH symptomatology. The decrease in postoperative central ventricular volume varied between 6 cc and 33 cc (mean: 20, SD: 9) among patients who experienced improvements in gait, bladder continence, and cognition. Two patients who showed improvement in only one or two of these symptoms had <4 cc of cerebrospinal fluid drained. Our artificial intelligence-based metric and the graphical user interface facilitate this quantified analysis. CONCLUSION: Proposed metric quantifies changes in ventricular volume before and after shunt surgery for patients with iNPH, serving as an automated and effective radiographic marker for a functioning shunt in a patient with iNPH.

A robust approach to 3D neuron shape representation for quantification and classification.

We consider the problem of finding an accurate representation of neuron shapes, extracting sub-cellular features, and classifying neurons based on neuron shapes. In neuroscience research, the skeleton representation is often used as a compact and abstract representation of neuron shapes. However, existing methods are limited to getting and analyzing "curve" skeletons which can only be applied for tubular shapes. This paper presents a 3D neuron morphology analysis method for more general and complex neuron shapes. First, we introduce the concept of skeleton mesh to represent general neuron shapes and propose a novel method for computing mesh representations from 3D surface point clouds. A skeleton graph is then obtained from skeleton mesh and is used to extract sub-cellular features. Finally, an unsupervised learning method is used to embed the skeleton graph for neuron classification. Extensive experiment results are provided and demonstrate the robustness of our method to analyze neuron morphology.

ReeBundle: A Method for Topological Modeling of White Matter Pathways Using Diffusion MRI.

Tractography can generate millions of complex curvilinear fibers (streamlines) in 3D that exhibit the geometry of white matter pathways in the brain. Common approaches to analyzing white matter connectivity are based on adjacency matrices that quantify connection strength but do not account for any topological information. A critical element in neurological and developmental disorders is the topological deterioration and irregularities in streamlines. In this paper, we propose a novel Reeb graph-based method "ReeBundle" that efficiently encodes the topology and geometry of white matter fibers. Given the trajectories of neuronal fiber pathways (neuroanatomical bundle), we re-bundle the streamlines by modeling their spatial evolution to capture geometrically significant events (akin to a fingerprint). ReeBundle parameters control the granularity of the model and handle the presence of improbable streamlines commonly produced by tractography. Further, we propose a new Reeb graph-based distance metric that quantifies topological differences for automated quality control and bundle comparison. We show the practical usage of our method using two datasets: (1) For International Society for Magnetic Resonance in Medicine (ISMRM) dataset, ReeBundle handles the morphology of the white matter tract configurations due to branching and local ambiguities in complicated bundle tracts like anterior and posterior commissures; (2) For the longitudinal repeated measures in the Cognitive Resilience and Sleep History (CRASH) dataset, repeated scans of a given subject acquired weeks apart lead to provably similar Reeb graphs that differ significantly from other subjects, thus highlighting ReeBundle's potential for clinical fingerprinting of brain regions.

3D Neuron Morphology Analysis.

We consider the problem of finding an accurate representation of neuron shapes, extracting sub-cellular features, and classifying neurons based on neuron shapes. In neuroscience research, the skeleton representation is often used as a compact and abstract representation of neuron shapes. However, existing methods are limited to getting and analyzing"curve"skeletons which can only be applied for tubular shapes. This paper presents a 3D neuron morphology analysis method for more general and complex neuron shapes. First, we introduce the concept of skeleton mesh to represent general neuron shapes and propose a novel method for computing mesh representations from 3D surface point clouds. A skeleton graph is then obtained from skeleton mesh and is used to extract sub-cellular features. Finally, an unsupervised learning method is used to embed the skeleton graph for neuron classification. Extensive experiment results are provided and demonstrate the robustness of our method to analyze neuron morphology.

Segmentation, tracking, and sub-cellular feature extraction in 3D time-lapse images.

This paper presents a method for time-lapse 3D cell analysis. Specifically, we consider the problem of accurately localizing and quantitatively analyzing sub-cellular features, and for tracking individual cells from time-lapse 3D confocal cell image stacks. The heterogeneity of cells and the volume of multi-dimensional images presents a major challenge for fully automated analysis of morphogenesis and development of cells. This paper is motivated by the pavement cell growth process, and building a quantitative morphogenesis model. We propose a deep feature based segmentation method to accurately detect and label each cell region. An adjacency graph based method is used to extract sub-cellular features of the segmented cells. Finally, the robust graph based tracking algorithm using multiple cell features is proposed for associating cells at different time instances. We also demonstrate the generality of our tracking method on C. elegans fluorescent nuclei imagery. Extensive experiment results are provided and demonstrate the robustness of the proposed method. The code is available on GitHub and the method is available as a service through the BisQue portal.

Erratum: Automatic classification and neurotransmitter prediction of synapses in electron microscopy - CORRIGENDUM.

[This corrects the article DOI: 10.1017/S2633903X2200006X.].

Automated Segmentation and Connectivity Analysis for Normal Pressure Hydrocephalus.

Objective and Impact Statement. We propose an automated method of predicting Normal Pressure Hydrocephalus (NPH) from CT scans. A deep convolutional network segments regions of interest from the scans. These regions are then combined with MRI information to predict NPH. To our knowledge, this is the first method which automatically predicts NPH from CT scans and incorporates diffusion tractography information for prediction. Introduction. Due to their low cost and high versatility, CT scans are often used in NPH diagnosis. No well-defined and effective protocol currently exists for analysis of CT scans for NPH. Evans' index, an approximation of the ventricle to brain volume using one 2D image slice, has been proposed but is not robust. The proposed approach is an effective way to quantify regions of interest and offers a computational method for predicting NPH. Methods. We propose a novel method to predict NPH by combining regions of interest segmented from CT scans with connectome data to compute features which capture the impact of enlarged ventricles by excluding fiber tracts passing through these regions. The segmentation and network features are used to train a model for NPH prediction. Results. Our method outperforms the current state-of-the-art by 9 precision points and 29 recall points. Our segmentation model outperforms the current state-of-the-art in segmenting the ventricle, gray-white matter, and subarachnoid space in CT scans. Conclusion. Our experimental results demonstrate that fast and accurate volumetric segmentation of CT brain scans can help improve the NPH diagnosis process, and network properties can increase NPH prediction accuracy.

Automatic classification and neurotransmitter prediction of synapses in electron microscopy.

This paper presents a deep-learning-based workflow to detect synapses and predict their neurotransmitter type in the primitive chordate Ciona intestinalis (Ciona) electron microscopic (EM) images. Identifying synapses from EM images to build a full map of connections between neurons is a labor-intensive process and requires significant domain expertise. Automation of synapse classification would hasten the generation and analysis of connectomes. Furthermore, inferences concerning neuron type and function from synapse features are in many cases difficult to make. Finding the connection between synapse structure and function is an important step in fully understanding a connectome. Class Activation Maps derived from the convolutional neural network provide insights on important features of synapses based on cell type and function. The main contribution of this work is in the differentiation of synapses by neurotransmitter type through the structural information in their EM images. This enables the prediction of neurotransmitter types for neurons in Ciona, which were previously unknown. The prediction model with code is available on GitHub.

A Computational Geometry Approach for Modeling Neuronal Fiber Pathways.

We propose a novel and efficient algorithm to model high-level topological structures of neuronal fibers. Tractography constructs complex neuronal fibers in three dimensions that exhibit the geometry of white matter pathways in the brain. However, most tractography analysis methods are time consuming and intractable. We develop a computational geometry-based tractography representation that aims to simplify the connectivity of white matter fibers. Given the trajectories of neuronal fiber pathways, we model the evolution of trajectories that encodes geometrically significant events and calculate their point correspondence in the 3D brain space. Trajectory inter-distance is used as a parameter to control the granularity of the model that allows local or global representation of the tractogram. Using diffusion MRI data from Alzheimer's patient study, we extract tractography features from our model for distinguishing the Alzheimer's subject from the normal control. Software implementation of our algorithm is available on GitHub (https://github.com/UCSB-VRL/ReebGraph.

Corrigendum: Improving Patch-Based Convolutional Neural Networks for MRI Brain Tumor Segmentation by Leveraging Location Information.

[This corrects the article DOI: 10.3389/fnins.2019.01449.].

Predicting Clinical Outcome of Stroke Patients with Tractographic Feature.

The volume of stroke lesion is the gold standard for predicting the clinical outcome of stroke patients. However, the presence of stroke lesion may cause neural disruptions to other brain regions, and these potentially damaged regions may affect the clinical outcome of stroke patients. In this paper, we introduce the tractographic feature to capture these potentially damaged regions and predict the modified Rankin Scale (mRS), which is a widely used outcome measure in stroke clinical trials. The tractographic feature is built from the stroke lesion and average connectome information from a group of normal subjects. The tractographic feature takes into account different functional regions that may be affected by the stroke, thus complementing the commonly used stroke volume features. The proposed tractographic feature is tested on a public stroke benchmark Ischemic Stroke Lesion Segmentation 2017 and achieves higher accuracy than the stroke volume and the state-of-the-art feature on predicting the mRS grades of stroke patients. Also, the tractographic feature yields a lower average absolute error than the commonly used stroke volume feature.

Superpixel Embedding Network.

Superpixel segmentation is a fundamental computer vision technique that finds application in a multitude of high level computer vision tasks. Most state-of-the-art superpixel segmentation methods are unsupervised in nature and thus cannot fully utilize frequently occurring texture patterns or incorporate multiscale context. In this paper, we show that superpixel segmentation can be improved by leveraging the superior modeling power of deep convolutional autoencoders in a fully unsupervised manner. We pose the superpixel segmentation problem as one of manifold learning where pixels that belong to similar texture patterns are assigned near identical embedding vectors. The proposed deep network is able to learn image-wide and dataset-wide feature patterns and the relationships between them. This knowledge is used to segment and group pixels in a way that is consistent with a more global definition of pattern coherence. Experiments demonstrate that the superpixels obtained from the embeddings learned by the proposed method outperform the state-of-theart superpixel segmentation methods for boundary precision and recall values. Additionally, we find that semantic edges obtained from the superpixel embeddings to be significantly better than the contemporary unsupervised approaches.

Gamma-irradiation-induced micro-structural variations in flame-retardant polyurethane foam using synchrotron X-ray micro-tomography.

Flame-retardant polyurethane foams are potential packing materials for the transport casks of highly active nuclear materials for shock absorption and insulation purposes. Exposure of high doses of gamma radiation causes cross-linking and chain sectioning of macromolecules in this polymer foam, which leads to reorganization of their cellular microstructure and thereby variations in physico-mechanical properties. In this study, in-house-developed flame-retardant rigid polyurethane foam samples were exposed to gamma irradiation doses in the 0-20 kGy range and synchrotron radiation X-ray micro-computed tomography (SR-µCT) imaging was employed for the analysis of radiation-induced morphological variations in their cellular microstructure. Qualitative and quantitative analysis of SR-µCT images has revealed significant variations in the average cell size, shape, wall thickness, orientations and spatial anisotropy of the cellular microstructure in polyurethane foam.

Hybrid LSTM and Encoder-Decoder Architecture for Detection of Image Forgeries.

With advanced image journaling tools, one can easily alter the semantic meaning of an image by exploiting certain manipulation techniques such as copy clone, object splicing, and removal, which mislead the viewers. In contrast, the identification of these manipulations becomes a very challenging task as manipulated regions are not visually apparent. This paper proposes a high-confidence manipulation localization architecture that utilizes resampling features, long short-term memory (LSTM) cells, and an encoder-decoder network to segment out manipulated regions from non-manipulated ones. Resampling features are used to capture artifacts, such as JPEG quality loss, upsampling, downsampling, rotation, and shearing. The proposed network exploits larger receptive fields (spatial maps) and frequency-domain correlation to analyze the discriminative characteristics between the manipulated and non-manipulated regions by incorporating the encoder and LSTM network. Finally, the decoder network learns the mapping from low-resolution feature maps to pixel-wise predictions for image tamper localization. With the predicted mask provided by the final layer (softmax) of the proposed architecture, end-to-end training is performed to learn the network parameters through back-propagation using the ground-truth masks. Furthermore, a large image splicing dataset is introduced to guide the training process. The proposed method is capable of localizing image manipulations at the pixel level with high precision, which is demonstrated through rigorous experimentation on three diverse datasets.

Improving Patch-Based Convolutional Neural Networks for MRI Brain Tumor Segmentation by Leveraging Location Information.

The manual brain tumor annotation process is time consuming and resource consuming, therefore, an automated and accurate brain tumor segmentation tool is greatly in demand. In this paper, we introduce a novel method to integrate location information with the state-of-the-art patch-based neural networks for brain tumor segmentation. This is motivated by the observation that lesions are not uniformly distributed across different brain parcellation regions and that a locality-sensitive segmentation is likely to obtain better segmentation accuracy. Toward this, we use an existing brain parcellation atlas in the Montreal Neurological Institute (MNI) space and map this atlas to the individual subject data. This mapped atlas in the subject data space is integrated with structural Magnetic Resonance (MR) imaging data, and patch-based neural networks, including 3D U-Net and DeepMedic, are trained to classify the different brain lesions. Multiple state-of-the-art neural networks are trained and integrated with XGBoost fusion in the proposed two-level ensemble method. The first level reduces the uncertainty of the same type of models with different seed initializations, and the second level leverages the advantages of different types of neural network models. The proposed location information fusion method improves the segmentation performance of state-of-the-art networks including 3D U-Net and DeepMedic. Our proposed ensemble also achieves better segmentation performance compared to the state-of-the-art networks in BraTS 2017 and rivals state-of-the-art networks in BraTS 2018. Detailed results are provided on the public multimodal brain tumor segmentation (BraTS) benchmarks.

Healthcare Event and Activity Logging.

The health of patients in the intensive care unit (ICU) can change frequently and inexplicably. Crucial events and activities responsible for these changes often go unnoticed. This paper introduces healthcare event and action logging (HEAL) which automatically and unobtrusively monitors and reports on events and activities that occur in a medical ICU room. HEAL uses a multimodal distributed camera network to monitor and identify ICU activities and estimate sanitation-event qualifiers. At the core is a novel approach to infer person roles based on semantic interactions, a critical requirement in many healthcare settings where individuals' identities must not be identified. The proposed approach for activity representation identifies contextual aspects basis and estimates aspect weights for proper action representation and reconstruction. The flexibility of the proposed algorithms enables the identification of people roles by associating them with inferred interactions and detected activities. A fully working prototype system is developed, tested in a mock ICU room and then deployed in two ICU rooms at a community hospital, thus offering unique capabilities for data gathering and analytics. The proposed method achieves a role identification accuracy of 84% and a backtracking role identification of 79% for obscured roles using interaction and appearance features on real ICU data. Detailed experimental results are provided in the context of four event-sanitation qualifiers: clean, transmission, contamination, and unclean.

CellECT: cell evolution capturing tool.

BACKGROUND: Robust methods for the segmentation and analysis of cells in 3D time sequences (3D+t) are critical for quantitative cell biology. While many automated methods for segmentation perform very well, few generalize reliably to diverse datasets. Such automated methods could significantly benefit from at least minimal user guidance. Identification and correction of segmentation errors in time-series data is of prime importance for proper validation of the subsequent analysis. The primary contribution of this work is a novel method for interactive segmentation and analysis of microscopy data, which learns from and guides user interactions to improve overall segmentation. RESULTS: We introduce an interactive cell analysis application, called CellECT, for 3D+t microscopy datasets. The core segmentation tool is watershed-based and allows the user to add, remove or modify existing segments by means of manipulating guidance markers. A confidence metric learns from the user interaction and highlights regions of uncertainty in the segmentation for the user's attention. User corrected segmentations are then propagated to neighboring time points. The analysis tool computes local and global statistics for various cell measurements over the time sequence. Detailed results on two large datasets containing membrane and nuclei data are presented: a 3D+t confocal microscopy dataset of the ascidian Phallusia mammillata consisting of 18 time points, and a 3D+t single plane illumination microscopy (SPIM) dataset consisting of 192 time points. Additionally, CellECT was used to segment a large population of jigsaw-puzzle shaped epidermal cells from Arabidopsis thaliana leaves. The cell coordinates obtained using CellECT are compared to those of manually segmented cells. CONCLUSIONS: CellECT provides tools for convenient segmentation and analysis of 3D+t membrane datasets by incorporating human interaction into automated algorithms. Users can modify segmentation results through the help of guidance markers, and an adaptive confidence metric highlights problematic regions. Segmentations can be propagated to multiple time points, and once a segmentation is available for a time sequence cells can be analyzed to observe trends. The segmentation and analysis tools presented here generalize well to membrane or cell wall volumetric time series datasets.

Characterizing spatial distributions of astrocytes in the mammalian retina.

MOTIVATION: In addition to being involved in retinal vascular growth, astrocytes play an important role in diseases and injuries, such as glaucomatous neuro-degeneration and retinal detachment. Studying astrocytes, their morphological cell characteristics and their spatial relationships to the surrounding vasculature in the retina may elucidate their role in these conditions. RESULTS: Our results show that in normal healthy retinas, the distribution of observed astrocyte cells does not follow a uniform distribution. The cells are significantly more densely packed around the blood vessels than a uniform distribution would predict. We also show that compared with the distribution of all cells, large cells are more dense in the vicinity of veins and toward the optic nerve head whereas smaller cells are often more dense in the vicinity of arteries. We hypothesize that since veinal astrocytes are known to transport toxic metabolic waste away from neurons they may be more critical than arterial astrocytes and therefore require larger cell bodies to process waste more efficiently. AVAILABILITY AND IMPLEMENTATION: A 1/8th size down-sampled version of the seven retinal image mosaics described in this article can be found on BISQUE (Kvilekval et al., 2010) at http://bisque.ece.ucsb.edu/client_service/view?resource=http://bisque.ece.ucsb.edu/data_service/dataset/6566968.

Robust Simultaneous Registration and Segmentation with sparse error reconstruction.

We introduce a fast and efficient variational framework for Simultaneous Registration and Segmentation (SRS) applicable to a wide variety of image sequences. We demonstrate that a dense correspondence map (between consecutive frames) can be reconstructed correctly even in the presence of partial occlusion, shading, and reflections. The errors are efficiently handled by exploiting their sparse nature. In addition, the segmentation functional is reformulated using a dual Rudin-Osher-Fatemi (ROF) model for fast implementation. Moreover, nonparametric shape prior terms that are suited for this dual-ROF model are proposed. The efficacy of the proposed method is validated with extensive experiments on both indoor, outdoor natural and biological image sequences, demonstrating the higher accuracy and efficiency compared to various state-of-the-art methods.

A linear program formulation for the segmentation of Ciona membrane volumes.

We address the problem of cell segmentation in confocal microscopy membrane volumes of the ascidian Ciona used in the study of morphogenesis. The primary challenges are non-uniform and patchy membrane staining and faint spurious boundaries from other organelles (e.g. nuclei). Traditional segmentation methods incorrectly attach to faint boundaries producing spurious edges. To address this problem, we propose a linear optimization framework for the joint correction of multiple over-segmentations obtained from different methods. The main idea motivating this approach is that multiple over-segmentations, resulting from a pool of methods with various parameters, are likely to agree on the correct segment boundaries, while spurious boundaries are methodor parameter-dependent. The challenge is to make an optimized decision on selecting the correct boundaries while discarding the spurious ones. The proposed unsupervised method achieves better performance than state of the art methods for cell segmentation from membrane images.