Ultrasound-Based Phenotyping of Lateral Ventricles to Predict Hydrocephalus Outcome in Premature Neonates.

OBJECTIVE: Prediction of post-hemorrhagic hydrocephalus (PHH) outcome-i.e., whether it requires intervention or not-in premature neonates using cranial ultrasound (CUS) images is challenging. In this paper, we present a novel fully-automatic method to perform phenotyping of the brain lateral ventricles and predict PHH outcome from CUS. METHODS: Our method consists of two parts: ventricle quantification followed by prediction of PHH outcome. First, cranial bounding box and brain interhemispheric fissure are detected to determine the anatomical position of ventricles and correct the cranium rotation. Then, lateral ventricles are extracted using a new deep learning-based method by incorporating the convolutional neural network into a probabilistic atlas-based weighted loss function and an image-specific adaption. PHH outcome is predicted using a support vector machine classifier trained using ventricular morphological phenotypes and clinical information. RESULTS: Experiments demonstrated that our method achieves accurate ventricle segmentation results with an average Dice similarity coefficient of 0.86, as well as very good PHH outcome prediction with accuracy of 0.91. CONCLUSION: Automatic CUS-based ventricular phenotyping in premature newborns could objectively and accurately predict the progression to severe PHH. SIGNIFICANCE: Early prediction of severe PHH development in premature newborns could potentially advance criteria for diagnosis and offer an opportunity for early interventions to improve outcome.

Explainable Anatomical Shape Analysis Through Deep Hierarchical Generative Models.

Quantification of anatomical shape changes currently relies on scalar global indexes which are largely insensitive to regional or asymmetric modifications. Accurate assessment of pathology-driven anatomical remodeling is a crucial step for the diagnosis and treatment of many conditions. Deep learning approaches have recently achieved wide success in the analysis of medical images, but they lack interpretability in the feature extraction and decision processes. In this work, we propose a new interpretable deep learning model for shape analysis. In particular, we exploit deep generative networks to model a population of anatomical segmentations through a hierarchy of conditional latent variables. At the highest level of this hierarchy, a two-dimensional latent space is simultaneously optimised to discriminate distinct clinical conditions, enabling the direct visualisation of the classification space. Moreover, the anatomical variability encoded by this discriminative latent space can be visualised in the segmentation space thanks to the generative properties of the model, making the classification task transparent. This approach yielded high accuracy in the categorisation of healthy and remodelled left ventricles when tested on unseen segmentations from our own multi-centre dataset as well as in an external validation set, and on hippocampi from healthy controls and patients with Alzheimer's disease when tested on ADNI data. More importantly, it enabled the visualisation in three-dimensions of both global and regional anatomical features which better discriminate between the conditions under exam. The proposed approach scales effectively to large populations, facilitating high-throughput analysis of normal anatomy and pathology in large-scale studies of volumetric imaging.

A Generic Approach to Lung Field Segmentation From Chest Radiographs Using Deep Space and Shape Learning.

Computer-aided diagnosis (CAD) techniques for lung field segmentation from chest radiographs (CXR) have been proposed for adult cohorts, but rarely for pediatric subjects. Statistical shape models (SSMs), the workhorse of most state-of-the-art CXR-based lung field segmentation methods, do not efficiently accommodate shape variation of the lung field during the pediatric developmental stages. The main contributions of our work are: 1) a generic lung field segmentation framework from CXR accommodating large shape variation for adult and pediatric cohorts; 2) a deep representation learning detection mechanism, ensemble space learning, for robust object localization; and 3) marginal shape deep learning for the shape deformation parameter estimation. Unlike the iterative approach of conventional SSMs, the proposed shape learning mechanism transforms the parameter space into marginal subspaces that are solvable efficiently using the recursive representation learning mechanism. Furthermore, our method is the first to include the challenging retro-cardiac region in the CXR-based lung segmentation for accurate lung capacity estimation. The framework is evaluated on 668 CXRs of patients between 3 month to 89 year of age. We obtain a mean Dice similarity coefficient of 0.96 ±0.03 (including the retro-cardiac region). For a given accuracy, the proposed approach is also found to be faster than conventional SSM-based iterative segmentation methods. The computational simplicity of the proposed generic framework could be similarly applied to the fast segmentation of other deformable objects.

Computational anatomy for multi-organ analysis in medical imaging: A review.

The medical image analysis field has traditionally been focused on the development of organ-, and disease-specific methods. Recently, the interest in the development of more comprehensive computational anatomical models has grown, leading to the creation of multi-organ models. Multi-organ approaches, unlike traditional organ-specific strategies, incorporate inter-organ relations into the model, thus leading to a more accurate representation of the complex human anatomy. Inter-organ relations are not only spatial, but also functional and physiological. Over the years, the strategies proposed to efficiently model multi-organ structures have evolved from the simple global modeling, to more sophisticated approaches such as sequential, hierarchical, or machine learning-based models. In this paper, we present a review of the state of the art on multi-organ analysis and associated computation anatomy methodology. The manuscript follows a methodology-based classification of the different techniques available for the analysis of multi-organs and multi-anatomical structures, from techniques using point distribution models to the most recent deep learning-based approaches. With more than 300 papers included in this review, we reflect on the trends and challenges of the field of computational anatomy, the particularities of each anatomical region, and the potential of multi-organ analysis to increase the impact of medical imaging applications on the future of healthcare.

Ventricular shape evaluation on early ultrasound predicts post-hemorrhagic hydrocephalus.

BACKGROUND: To compare the ability of ventricular morphology on cranial ultrasound (CUS) versus standard clinical variables to predict the need for temporizing cerebrospinal fluid drainage in newborns with intraventricular hemorrhage (IVH). METHOD: This is a retrospective study of newborns (gestational age <29 weeks) diagnosed with IVH. Clinical variables known to increase the risk for post-hemorrhagic hydrocephalus were collected. The first CUS with IVH was identified and a slice in the coronal plane was selected. The frontal horns of the lateral ventricles were manually segmented. Automated quantitative morphological features were extracted from both lateral ventricles. Predictive models of the need of temporizing intervention were compared. RESULTS: Sixty-two newborns met inclusion criteria. Fifteen out of the 62 had a temporizing intervention. The morphological features had a better accuracy predicting temporizing interventions when compared to clinical variables: 0.94 versus 0.85, respectively; p < 0.01 for both. By considering both morphological and clinical variables, our method predicts the need of temporizing intervention with positive and negative predictive values of 0.83 and 1, respectively, and accuracy of 0.97. CONCLUSION: Early cranial ultrasound-based quantitative ventricular evaluation in premature newborns can predict the eventual use of a temporizing intervention to treat post-hemorrhagic hydrocephalus. This may be helpful for early monitoring and treatment.

Fetal Skull Reconstruction via Deep Convolutional Autoencoders.

Ultrasound (US) imaging is arguably the most commonly used modality for fetal screening. Recently, 3DUS has been progressively adopted in modern obstetric practice, showing promising diagnosis capabilities, and alleviating many of the inherent limitations of traditional 2DUS, such as subjectivity and operator dependence. However, the involuntary movements of the fetus, and the difficulty for the operator to inspect the entire volume in real-time can hinder the acquisition of the entire region of interest. In this paper, we present two deep convolutional architectures for the reconstruction of the fetal skull in partially occluded 3DUS volumes: a TL deep convolutional network (TL-Net), and a conditional variational autoencoder (CVAE). The performance of the two networks was evaluated for occlusion rates up to 50%, both showing accurate results even when only 60% of the skull is included in the US volume (Dice coeff. $0.84\pm 0.04$ for CVAE and $0.83\pm 0.03$ for TL-Net). The reconstruction networks proposed here have the potential to optimize image acquisition protocols in obstetric sonography, reducing the acquisition time and providing comprehensive anatomical information even from partially occluded images.

Automatic Segmentation of Neonatal Ventricles from Cranial Ultrasound for Prediction of Intraventricular Hemorrhage Outcome.

Intraventricular hemorrhage (IVH) followed by post hemorrhagic hydrocephalus (PHH) in premature neonates is one of the recognized reasons of brain injury in newborns. Cranial ultrasound (CUS) is a noninvasive imaging tool that has been used widely to diagnose and monitor neonates with IVH. In our previous work, we showed the potential of quantitative morphological analysis of lateral ventricles from early CUS to predict the PHH outcome in neonates with IVH. In this paper, we first present a new automatic method for ventricle segmentation in 2D CUS images. We detect the brain bounding box and brain mid-line to estimate the anatomical positions of ventricles and correct the brain rotation. The ventricles are segmented using a combination of fuzzy c-means, phase congruency, and active contour algorithms. Finally, we compare this fully automated approach with our previous work for the prediction of the outcome of PHH on a set of 2D CUS images taken from 60 premature neonates with different IVH grades. Experimental results showed that our method could segment ventricles with an average Dice similarity coefficient of 0.8 ± 0.12. In addition, our fully automated method could predict the outcome of PHH based on the extracted ventricle regions with similar accuracy to our previous semi-automated approach (83% vs. 84%, respectively, p-value = 0.8). This method has the potential to standardize the evaluation of CUS images and can be a helpful clinical tool for early monitoring and treatment of IVH and PHH.

Fast Multiple Landmark Localisation Using a Patch-based Iterative Network.

We propose a new Patch-based Iterative Network (PIN) for fast and accurate landmark localisation in 3D medical volumes. PIN utilises a Convolutional Neural Network (CNN) to learn the spatial relationship between an image patch and anatomical landmark positions. During inference, patches are repeatedly passed to the CNN until the estimated landmark position converges to the true landmark location. PIN is computationally efficient since the inference stage only selectively samples a small number of patches in an iterative fashion rather than a dense sampling at every location in the volume. Our approach adopts a multitask learning framework that combines regression and classification to improve localisation accuracy. We extend PIN to localise multiple landmarks by using principal component analysis, which models the global anatomical relationships between landmarks. We have evaluated PIN using 72 3D ultrasound images from fetal screening examinations. PIN achieves quantitatively an average landmark localisation error of 5.59mm and a runtime of 0.44s to predict 10 landmarks per volume. Qualitatively, anatomical 2D standard scan planes derived from the predicted landmark locations are visually similar to the clinical ground truth.

Design of patient-specific concentric tube robots using path planning from 3-D ultrasound.

Percutaneous techniques and robot-assisted surgical systems have enabled minimally invasive procedures that offer reduced scarring, recovery time, and complications compared to traditional open surgeries. Despite these improvements, access to diseased sites using the standard, straight needle-based percutaneous techniques is still limited for certain procedures due to intervening tissues. These limitations can be further exacerbated in specific patient groups, particularly pediatric patients, whose anatomy does not fit the traditional tools and systems. We therefore propose a patient-specific paradigm to design and fabricate dexterous, robotic tools based on the patient's preoperative images. In this paper, we present the main steps of our proposed paradigm - image-based path planning, robot design, and fabrication - along with an example case that focuses on a class of dexterous, snake-like tools called concentric tube robots. We demonstrate planning a safe path using a patient's preoperative ultrasound images. We then determine the concentric tube robot parameters needed to achieve this path, and finally, we use 3-D printing to fabricate the patient-specific robot.

Cranial ultrasound-based prediction of post hemorrhagic hydrocephalus outcome in premature neonates with intraventricular hemorrhage.

Premature neonates with intraventricular hemorrhage (IVH) followed by post hemorrhagic hydrocephalus (PHH) are at high risk for brain injury. Cranial ultrasound (CUS) is used for monitoring of premature neonates during the first weeks after birth to identify IVH and follow the progression to PHH. However, the lack of a standardized method for CUS evaluation has led to significant variability in decision making regarding treatment. We propose a quantitative imaging tool for the evaluation of PHH on CUS for premature neonates based on morphological features of the cerebral ventricles. We retrospectively studied 64 extremely premature neonates born less than 29 weeks gestational age, less than 1,500 grams weight at birth, admitted to our center within two weeks of life, and diagnosed with different grades of IVH. We extracted morphological features of the lateral ventricles from CUS imaging using image analysis techniques to compare neonates who needed a temporizing intervention to treat PHH to the ones who did not. From the original set of features, an optimal ranking was obtained based on linear support vector machine. A subset of features was subsequently selected that maximizes the overall accuracy level. Regarding whether or not there was a need for temporizing intervention, we predicted the outcome of PHH with an improved accuracy level of 84%, compared to the 76% rate obtained by linear manual measurement. The proposed imaging tool allowed us to establish a quantitative method for PHH evaluation on CUS in extremely premature neonates with IVH. Further studies will help standardize the evaluation of CUS in those neonates to institute treatments earlier and improve outcomes.

Marginal Shape Deep Learning: Applications to Pediatric Lung Field Segmentation.

Representation learning through deep learning (DL) architecture has shown tremendous potential for identification, localization, and texture classification in various medical imaging modalities. However, DL applications to segmentation of objects especially to deformable objects are rather limited and mostly restricted to pixel classification. In this work, we propose marginal shape deep learning (MaShDL), a framework that extends the application of DL to deformable shape segmentation by using deep classifiers to estimate the shape parameters. MaShDL combines the strength of statistical shape models with the automated feature learning architecture of DL. Unlike the iterative shape parameters estimation approach of classical shape models that often leads to a local minima, the proposed framework is robust to local minima optimization and illumination changes. Furthermore, since the direct application of DL framework to a multi-parameter estimation problem results in a very high complexity, our framework provides an excellent run-time performance solution by independently learning shape parameter classifiers in marginal eigenspaces in the decreasing order of variation. We evaluated MaShDL for segmenting the lung field from 314 normal and abnormal pediatric chest radiographs and obtained a mean Dice similarity coefficient of 0.927 using only the four highest modes of variation (compared to 0.888 with classical ASM1 (p-value=0.01) using same configuration). To the best of our knowledge this is the first demonstration of using DL framework for parametrized shape learning for the delineation of deformable objects.

Renal Segmentation From 3D Ultrasound via Fuzzy Appearance Models and Patient-Specific Alpha Shapes.

Ultrasound (US) imaging is the primary imaging modality for pediatric hydronephrosis, which manifests as the dilation of the renal collecting system (CS). In this paper, we present a new framework for the segmentation of renal structures, kidney and CS, from 3DUS scans. First, the kidney is segmented using an active shape model-based approach, tailored to deal with the challenges raised by US images. A weighted statistical shape model allows to compensate the image variation with the propagation direction of the US wavefront. The model is completed with a new fuzzy appearance model and a multi-scale omnidirectional Gabor-based appearance descriptor. Next, the CS is segmented using an active contour formulation, which combines contour- and intensity-based terms. The new positive alpha detector presented here allows to control the propagation process by means of a patient-specific stopping function created from the bands of adipose tissue within the kidney. The performance of the new segmentation approach was evaluated on a dataset of 39 cases, showing an average Dice's coefficient of 0.86±0.05 for the kidney, and 0.74 ± 0.10 for the CS segmentation, respectively. These promising results demonstrate the potential utility of this framework for the US-based assessment of the severity of pediatric hydronephrosis.

Deep Learning Guided Partitioned Shape Model for Anterior Visual Pathway Segmentation.

Analysis of cranial nerve systems, such as the anterior visual pathway (AVP), from MRI sequences is challenging due to their thin long architecture, structural variations along the path, and low contrast with adjacent anatomic structures. Segmentation of a pathologic AVP (e.g., with low-grade gliomas) poses additional challenges. In this work, we propose a fully automated partitioned shape model segmentation mechanism for AVP steered by multiple MRI sequences and deep learning features. Employing deep learning feature representation, this framework presents a joint partitioned statistical shape model able to deal with healthy and pathological AVP. The deep learning assistance is particularly useful in the poor contrast regions, such as optic tracts and pathological areas. Our main contributions are: 1) a fast and robust shape localization method using conditional space deep learning, 2) a volumetric multiscale curvelet transform-based intensity normalization method for robust statistical model, and 3) optimally partitioned statistical shape and appearance models based on regional shape variations for greater local flexibility. Our method was evaluated on MRI sequences obtained from 165 pediatric subjects. A mean Dice similarity coefficient of 0.779 was obtained for the segmentation of the entire AVP (optic nerve only =0.791 ) using the leave-one-out validation. Results demonstrated that the proposed localized shape and sparse appearance-based learning approach significantly outperforms current state-of-the-art segmentation approaches and is as robust as the manual segmentation.

Quantitative Ultrasound for Measuring Obstructive Severity in Children with Hydronephrosis.

PURPOSE: We define sonographic biomarkers for hydronephrotic renal units that can predict the necessity of diuretic nuclear renography. MATERIALS AND METHODS: We selected a cohort of 50 consecutive patients with hydronephrosis of varying severity in whom 2-dimensional sonography and diuretic mercaptoacetyltriglycine renography had been performed. A total of 131 morphological parameters were computed using quantitative image analysis algorithms. Machine learning techniques were then applied to identify ultrasound based safety thresholds that agreed with the t½ for washout. A best fit model was then derived for each threshold level of t½ that would be clinically relevant at 20, 30 and 40 minutes. Receiver operating characteristic curve analysis was performed. Sensitivity, specificity and area under the receiver operating characteristic curve were determined. Improvement obtained by the quantitative imaging method compared to the Society for Fetal Urology grading system and the hydronephrosis index was statistically verified. RESULTS: For the 3 thresholds considered and at 100% sensitivity the specificities of the quantitative imaging method were 94%, 70% and 74%, respectively. Corresponding area under the receiver operating characteristic curve values were 0.98, 0.94 and 0.94, respectively. Improvement obtained by the quantitative imaging method over the Society for Fetal Urology grade and hydronephrosis index was statistically significant (p <0.05 in all cases). CONCLUSIONS: Quantitative imaging analysis of renal sonograms in children with hydronephrosis can identify thresholds of clinically significant washout times with 100% sensitivity to decrease the number of diuretic renograms in up to 62% of children.

Automatic multi-resolution shape modeling of multi-organ structures.

Point Distribution Models (PDM) are among the most popular shape description techniques and their usefulness has been demonstrated in a wide variety of medical imaging applications. However, to adequately characterize the underlying modeled population it is essential to have a representative number of training samples, which is not always possible. This problem is especially relevant as the complexity of the modeled structure increases, being the modeling of ensembles of multiple 3D organs one of the most challenging cases. In this paper, we introduce a new GEneralized Multi-resolution PDM (GEM-PDM) in the context of multi-organ analysis able to efficiently characterize the different inter-object relations, as well as the particular locality of each object separately. Importantly, unlike previous approaches, the configuration of the algorithm is automated thanks to a new agglomerative landmark clustering method proposed here, which equally allows us to identify smaller anatomically significant regions within organs. The significant advantage of the GEM-PDM method over two previous approaches (PDM and hierarchical PDM) in terms of shape modeling accuracy and robustness to noise, has been successfully verified for two different databases of sets of multiple organs: six subcortical brain structures, and seven abdominal organs. Finally, we propose the integration of the new shape modeling framework into an active shape-model-based segmentation algorithm. The resulting algorithm, named GEMA, provides a better overall performance than the two classical approaches tested, ASM, and hierarchical ASM, when applied to the segmentation of 3D brain MRI.

Quantification of kidneys from 3D ultrasound in pediatric hydronephrosis.

This paper introduces a complete framework for the quantification of renal structures (parenchyma, and collecting system) in 3D ultrasound (US) images. First, the segmentation of the kidney is performed using Gabor-based appearance models (GAM), a variant of the popular active shape models, properly tailored to the imaging physics of US image data. The framework also includes a new graph-cut based method for the segmentation of the collecting system, including brightness and contrast normalization, and positional prior information. The significant advantage (p = 0.03) of the new method over previous approaches in terms of segmentation accuracy has been successfully verified on clinical 3DUS data from pediatric cases with hydronephrosis. The promising results obtained in the estimation of the volumetric hydronephrosis index demonstrate the potential of our new framework to quantify anatomy in US and asses the severity of hydronephrosis.

Generalized multiresolution hierarchical shape models via automatic landmark clusterization.

Point Distribution Models (PDM) are some of the most popular shape description techniques in medical imaging. However, to create an accurate shape model it is essential to have a representative sample of the underlying population, which is often challenging. This problem is particularly relevant as the dimensionality of the modeled structures increases, and becomes critical when dealing with complex 3D shapes. In this paper, we introduce a new generalized multiresolution hierarchical PDM (GMRH-PDM) able to efficiently address the high-dimension-low-sample-size challenge when modeling complex structures. Unlike previous approaches, our new and general framework extends hierarchical modeling to any type of structure (multi- and single-object shapes) allowing to describe efficiently the shape variability at different levels of resolution. Importantly, the configuration of the algorithm is automatized thanks to the new agglomerative landmark clustering method presented here. Our new and automatic GMRH-PDM framework performed significantly better than classical approaches, and as well as the state-of-the-art with the best manual configuration. Evaluations have been studied for two different cases, the right kidney, and a multi-object case composed of eight subcortical structures.

Multiresolution hierarchical shape models in 3D subcortical brain structures.

Point distribution models (PDM) are one of the most extended methods to characterize the underlying population of set of samples, whose usefulness has been demonstrated in a wide variety of applications, including medical imaging. However, one important issue remains unsolved: the large number of training samples required. This problem becomes critical as the complexity of the problem increases, and the modeling of 3D multiobjects/organs represents one of the most challenging cases. Based on the 3D wavelet transform, this paper introduces a multiresolution hierarchical variant of PDM (MRH-PDM) able to efficiently characterize the different inter-object relationships, as well as the particular locality of each element separately. The significant advantage of this new method over two previous approaches in terms of accuracy has been successfully verified for the particular case of 3D subcortical brain structures.

Hierarchical statistical shape models of multiobject anatomical structures: application to brain MRI.

The accurate segmentation of subcortical brain structures in magnetic resonance (MR) images is of crucial importance in the interdisciplinary field of medical imaging. Although statistical approaches such as active shape models (ASMs) have proven to be particularly useful in the modeling of multiobject shapes, they are inefficient when facing challenging problems. Based on the wavelet transform, the fully generic multiresolution framework presented in this paper allows us to decompose the interobject relationships into different levels of detail. The aim of this hierarchical decomposition is twofold: to efficiently characterize the relationships between objects and their particular localities. Experiments performed on an eight-object structure defined in axial cross sectional MR brain images show that the new hierarchical segmentation significantly improves the accuracy of the segmentation, and while it exhibits a remarkable robustness with respect to the size of the training set.