Development and validation of an AI-enabled digital breast cancer assay to predict early-stage breast cancer recurrence within 6 years.

BACKGROUND: Breast cancer (BC) grading plays a critical role in patient management despite the considerable inter- and intra-observer variability, highlighting the need for decision support tools to improve reproducibility and prognostic accuracy for use in clinical practice. The objective was to evaluate the ability of a digital artificial intelligence (AI) assay (PDxBr) to enrich BC grading and improve risk categorization for predicting recurrence. METHODS: In our population-based longitudinal clinical development and validation study, we enrolled 2075 patients from Mount Sinai Hospital with infiltrating ductal carcinoma of the breast. With 3:1 balanced training and validation cohorts, patients were retrospectively followed for a median of 6 years. The main outcome was to validate an automated BC phenotyping system combined with clinical features to produce a binomial risk score predicting BC recurrence at diagnosis. RESULTS: The PDxBr training model (n = 1559 patients) had a C-index of 0.78 (95% CI, 0.76-0.81) versus clinical 0.71 (95% CI, 0.67-0.74) and image feature models 0.72 (95% CI, 0.70-0.74). A risk score of 58 (scale 0-100) stratified patients as low or high risk, hazard ratio (HR) 5.5 (95% CI 4.19-7.2, p < 0.001), with a sensitivity 0.71, specificity 0.77, NPV 0.95, and PPV 0.32 for predicting BC recurrence within 6 years. In the validation cohort (n = 516), the C-index was 0.75 (95% CI, 0.72-0.79) versus clinical 0.71 (95% CI 0.66-0.75) versus image feature models 0.67 (95% CI, 0.63-071). The validation cohort had an HR of 4.4 (95% CI 2.7-7.1, p < 0.001), sensitivity of 0.60, specificity 0.77, NPV 0.94, and PPV 0.24 for predicting BC recurrence within 6 years. PDxBr also improved Oncotype Recurrence Score (RS) performance: RS 31 cutoff, C-index of 0.36 (95% CI 0.26-0.45), sensitivity 37%, specificity 48%, HR 0.48, p = 0.04 versus Oncotype RS plus AI-grade C-index 0.72 (95% CI 0.67-0.79), sensitivity 78%, specificity 49%, HR 4.6, p < 0.001 versus Oncotype RS plus PDxBr, C-index 0.76 (95% CI 0.70-0.82), sensitivity 67%, specificity 80%, HR 6.1, p < 0.001. CONCLUSIONS: PDxBr is a digital BC test combining automated AI-BC prognostic grade with clinical-pathologic features to predict the risk of early-stage BC recurrence. With future validation studies, we anticipate the PDxBr model will enrich current gene expression assays and enhance treatment decision-making.

Artificial intelligence in neuropathology: deep learning-based assessment of tauopathy.

Accumulation of abnormal tau in neurofibrillary tangles (NFT) occurs in Alzheimer disease (AD) and a spectrum of tauopathies. These tauopathies have diverse and overlapping morphological phenotypes that obscure classification and quantitative assessments. Recently, powerful machine learning-based approaches have emerged, allowing the recognition and quantification of pathological changes from digital images. Here, we applied deep learning to the neuropathological assessment of NFT in postmortem human brain tissue to develop a classifier capable of recognizing and quantifying tau burden. The histopathological material was derived from 22 autopsy brains from patients with tauopathies. We used a custom web-based informatics platform integrated with an in-house information management system to manage whole slide images (WSI) and human expert annotations as ground truth. We utilized fully annotated regions to train a deep learning fully convolutional neural network (FCN) implemented in PyTorch against the human expert annotations. We found that the deep learning framework is capable of identifying and quantifying NFT with a range of staining intensities and diverse morphologies. With our FCN model, we achieved high precision and recall in naive WSI semantic segmentation, correctly identifying tangle objects using a SegNet model trained for 200 epochs. Our FCN is efficient and well suited for the practical application of WSIs with average processing times of 45 min per WSI per GPU, enabling reliable and reproducible large-scale detection of tangles. We measured performance on test data of 50 pre-annotated regions on eight naive WSI across various tauopathies, resulting in the recall, precision, and an F1 score of 0.92, 0.72, and 0.81, respectively. Machine learning is a useful tool for complex pathological assessment of AD and other tauopathies. Using deep learning classifiers, we have the potential to integrate cell- and region-specific annotations with clinical, genetic, and molecular data, providing unbiased data for clinicopathological correlations that will enhance our knowledge of the neurodegeneration.

Morphometry of anatomical shape complexes with dense deformations and sparse parameters.

We propose a generic method for the statistical analysis of collections of anatomical shape complexes, namely sets of surfaces that were previously segmented and labeled in a group of subjects. The method estimates an anatomical model, the template complex, that is representative of the population under study. Its shape reflects anatomical invariants within the dataset. In addition, the method automatically places control points near the most variable parts of the template complex. Vectors attached to these points are parameters of deformations of the ambient 3D space. These deformations warp the template to each subject's complex in a way that preserves the organization of the anatomical structures. Multivariate statistical analysis is applied to these deformation parameters to test for group differences. Results of the statistical analysis are then expressed in terms of deformation patterns of the template complex, and can be visualized and interpreted. The user needs only to specify the topology of the template complex and the number of control points. The method then automatically estimates the shape of the template complex, the optimal position of control points and deformation parameters. The proposed approach is completely generic with respect to any type of application and well adapted to efficient use in clinical studies, in that it does not require point correspondence across surfaces and is robust to mesh imperfections such as holes, spikes, inconsistent orientation or irregular meshing. The approach is illustrated with a neuroimaging study of Down syndrome (DS). The results demonstrate that the complex of deep brain structures shows a statistically significant shape difference between control and DS subjects. The deformation-based modelingis able to classify subjects with very high specificity and sensitivity, thus showing important generalization capability even given a low sample size. We show that the results remain significant even if the number of control points, and hence the dimension of variables in the statistical model, are drastically reduced. The analysis may even suggest that parsimonious models have an increased statistical performance. The method has been implemented in the software Deformetrica, which is publicly available at www.deformetrica.org.

Analytical Validation of the PreciseDx Digital Prognostic Breast Cancer Test in Early-Stage Breast Cancer.

BACKGROUND: PreciseDx Breast (PDxBr) is a digital test that predicts early-stage breast cancer recurrence within 6-years of diagnosis. MATERIALS AND METHODS: Using hematoxylin and eosin-stained whole slide images of invasive breast cancer (IBC) and artificial intelligence-enabled morphology feature array, microanatomic features are generated. Morphometric attributes in combination with patient's age, tumor size, stage, and lymph node status predict disease free survival using a proprietary algorithm. Here, analytical validation of the automated annotation process and extracted histologic digital features of the PDxBr test, including impact of methodologic variability on the composite risk score is presented. Studies of precision, repeatability, reproducibility and interference were performed on morphology feature array-derived features. The final risk score was assessed over 20-days with 2-operators, 2-runs/day, and 2-replicates across 8-patients, allowing for calculation of within-run repeatability, between-run and within-laboratory reproducibility. RESULTS: Analytical validation of features derived from whole slide images demonstrated a high degree of precision for tumor segmentation (0.98, 0.98), lymphocyte detection (0.91, 0.93), and mitotic figures (0.85, 0.84). Correlation of variation of the assay risk score for both reproducibility and repeatability were less than 2%, and interference from variation in hematoxylin and eosin staining or tumor thickness was not observed demonstrating assay robustness across standard histopathology preparations. CONCLUSION: In summary, the analytical validation of the digital IBC risk assessment test demonstrated a strong performance across all features in the model and complimented the clinical validation of the assay previously shown to accurately predict recurrence within 6-years in early-stage invasive breast cancer patients.

Regional characterization of longitudinal DT-MRI to study white matter maturation of the early developing brain.

The human brain undergoes rapid and dynamic development early in life. Assessment of brain growth patterns relevant to neurological disorders and disease requires a normative population model of growth and variability in order to evaluate deviation from typical development. In this paper, we focus on maturation of brain white matter as shown in diffusion tensor MRI (DT-MRI), measured by fractional anisotropy (FA), mean diffusivity (MD), as well as axial and radial diffusivities (AD, RD). We present a novel methodology to model temporal changes of white matter diffusion from longitudinal DT-MRI data taken at discrete time points. Our proposed framework combines nonlinear modeling of trajectories of individual subjects, population analysis, and testing for regional differences in growth pattern. We first perform deformable mapping of longitudinal DT-MRI of healthy infants imaged at birth, 1 year, and 2 years of age, into a common unbiased atlas. An existing template of labeled white matter regions is registered to this atlas to define anatomical regions of interest. Diffusivity properties of these regions, presented over time, serve as input to the longitudinal characterization of changes. We use non-linear mixed effect (NLME) modeling where temporal change is described by the Gompertz function. The Gompertz growth function uses intuitive parameters related to delay, rate of change, and expected asymptotic value; all descriptive measures which can answer clinical questions related to quantitative analysis of growth patterns. Results suggest that our proposed framework provides descriptive and quantitative information on growth trajectories that can be interpreted by clinicians using natural language terms that describe growth. Statistical analysis of regional differences between anatomical regions which are known to mature differently demonstrates the potential of the proposed method for quantitative assessment of brain growth and differences thereof. This will eventually lead to a prediction of white matter diffusion properties and associated cognitive development at later stages given imaging data at early stages.

Artificial intelligence-derived neurofibrillary tangle burden is associated with antemortem cognitive impairment.

Tauopathies are a category of neurodegenerative diseases characterized by the presence of abnormal tau protein-containing neurofibrillary tangles (NFTs). NFTs are universally observed in aging, occurring with or without the concomitant accumulation of amyloid-beta peptide (Aß) in plaques that typifies Alzheimer disease (AD), the most common tauopathy. Primary age-related tauopathy (PART) is an Aß-independent process that affects the medial temporal lobe in both cognitively normal and impaired subjects. Determinants of symptomology in subjects with PART are poorly understood and require clinicopathologic correlation; however, classical approaches to staging tau pathology have limited quantitative reproducibility. As such, there is a critical need for unbiased methods to quantitatively analyze tau pathology on the histological level. Artificial intelligence (AI)-based convolutional neural networks (CNNs) generate highly accurate and precise computer vision assessments of digitized pathology slides, yielding novel histology metrics at scale. Here, we performed a retrospective autopsy study of a large cohort (n = 706) of human post-mortem brain tissues from normal and cognitively impaired elderly individuals with mild or no Aß plaques (average age of death of 83.1 yr, range 55-110). We utilized a CNN trained to segment NFTs on hippocampus sections immunohistochemically stained with antisera recognizing abnormal hyperphosphorylated tau (p-tau), which yielded metrics of regional NFT counts, NFT positive pixel density, as well as a novel graph-theory based metric measuring the spatial distribution of NFTs. We found that several AI-derived NFT metrics significantly predicted the presence of cognitive impairment in both the hippocampus proper and entorhinal cortex (p < 0.0001). When controlling for age, AI-derived NFT counts still significantly predicted the presence of cognitive impairment (p = 0.04 in the entorhinal cortex; p = 0.04 overall). In contrast, Braak stage did not predict cognitive impairment in either age-adjusted or unadjusted models. These findings support the hypothesis that NFT burden correlates with cognitive impairment in PART. Furthermore, our analysis strongly suggests that AI-derived metrics of tau pathology provide a powerful tool that can deepen our understanding of the role of neurofibrillary degeneration in cognitive impairment.

Antemortem detection of Parkinson's disease pathology in peripheral biopsies using artificial intelligence.

The diagnosis of Parkinson's disease (PD) is challenging at all stages due to variable symptomatology, comorbidities, and mimicking conditions. Postmortem assessment remains the gold standard for a definitive diagnosis. While it is well recognized that PD manifests pathologically in the central nervous system with aggregation of α-synuclein as Lewy bodies and neurites, similar Lewy-type synucleinopathy (LTS) is additionally found in the peripheral nervous system that may be useful as an antemortem biomarker. We have previously found that detection of LTS in submandibular gland (SMG) biopsies is sensitive and specific for advanced PD; however, the sensitivity is suboptimal especially for early-stage disease. Further, visual microscopic assessment of biopsies by a neuropathologist to identify LTS is impractical for large-scale adoption. Here, we trained and validated a convolutional neural network (CNN) for detection of LTS on 283 digital whole slide images (WSI) from 95 unique SMG biopsies. A total of 8,450 LTS and 35,066 background objects were annotated following an inter-rater reliability study with Fleiss Kappa = 0.72. We used transfer learning to train a CNN model to classify image patches (151 × 151 pixels at 20× magnification) with and without the presence of LTS objects. The trained CNN model showed the following performance on image patches: sensitivity: 0.99, specificity: 0.99, precision: 0.81, accuracy: 0.99, and F-1 score: 0.89. We further tested the trained network on 1230 naïve WSI from the same cohort of research subjects comprising 42 PD patients and 14 controls. Logistic regression models trained on features engineered from the CNN predictions on the WSI resulted in sensitivity: 0.71, specificity: 0.65, precision: 0.86, accuracy: 0.69, and F-1 score: 0.76 in predicting clinical PD status, and 0.64 accuracy in predicting PD stage, outperforming expert neuropathologist LTS density scoring in terms of sensitivity but not specificity. These findings demonstrate the practical utility of a CNN detector in screening for LTS, which can translate into a computational tool to facilitate the antemortem tissue-based diagnosis of PD in clinical settings.

BACH: Grand challenge on breast cancer histology images.

Breast cancer is the most common invasive cancer in women, affecting more than 10% of women worldwide. Microscopic analysis of a biopsy remains one of the most important methods to diagnose the type of breast cancer. This requires specialized analysis by pathologists, in a task that i) is highly time- and cost-consuming and ii) often leads to nonconsensual results. The relevance and potential of automatic classification algorithms using hematoxylin-eosin stained histopathological images has already been demonstrated, but the reported results are still sub-optimal for clinical use. With the goal of advancing the state-of-the-art in automatic classification, the Grand Challenge on BreAst Cancer Histology images (BACH) was organized in conjunction with the 15th International Conference on Image Analysis and Recognition (ICIAR 2018). BACH aimed at the classification and localization of clinically relevant histopathological classes in microscopy and whole-slide images from a large annotated dataset, specifically compiled and made publicly available for the challenge. Following a positive response from the scientific community, a total of 64 submissions, out of 677 registrations, effectively entered the competition. The submitted algorithms improved the state-of-the-art in automatic classification of breast cancer with microscopy images to an accuracy of 87%. Convolutional neuronal networks were the most successful methodology in the BACH challenge. Detailed analysis of the collective results allowed the identification of remaining challenges in the field and recommendations for future developments. The BACH dataset remains publicly available as to promote further improvements to the field of automatic classification in digital pathology.

Regional gray matter growth, sexual dimorphism, and cerebral asymmetry in the neonatal brain.

Although there has been recent interest in the study of childhood and adolescent brain development, very little is known about normal brain development in the first few months of life. In older children, there are regional differences in cortical gray matter development, whereas cortical gray and white matter growth after birth has not been studied to a great extent. The adult human brain is also characterized by cerebral asymmetries and sexual dimorphisms, although very little is known about how these asymmetries and dimorphisms develop. We used magnetic resonance imaging and an automatic segmentation methodology to study brain structure in 74 neonates in the first few weeks after birth. We found robust cortical gray matter growth compared with white matter growth, with occipital regions growing much faster than prefrontal regions. Sexual dimorphism is present at birth, with males having larger total brain cortical gray and white matter volumes than females. In contrast to adults and older children, the left hemisphere is larger than the right hemisphere, and the normal pattern of fronto-occipital asymmetry described in older children and adults is not present. Regional differences in cortical gray matter growth are likely related to differential maturation of sensory and motor systems compared with prefrontal executive function after birth. These findings also indicate that whereas some adult patterns of sexual dimorphism and cerebral asymmetries are present at birth, others develop after birth.

Geodesic shape regression with multiple geometries and sparse parameters.

Many problems in medicine are inherently dynamic processes which include the aspect of change over time, such as childhood development, aging, and disease progression. From medical images, numerous geometric structures can be extracted with various representations, such as landmarks, point clouds, curves, and surfaces. Different sources of geometry may characterize different aspects of the anatomy, such as fiber tracts from DTI and subcortical shapes from structural MRI, and therefore require a modeling scheme which can include various shape representations in any combination. In this paper, we present a geodesic regression model in the large deformation (LDDMM) framework applicable to multi-object complexes in a variety of shape representations. Our model decouples the deformation parameters from the specific shape representations, allowing the complexity of the model to reflect the nature of the shape changes, rather than the sampling of the data. As a consequence, the sparse representation of diffeomorphic flow allows for the straightforward embedding of a variety of geometry in different combinations, which all contribute towards the estimation of a single deformation of the ambient space. Additionally, the sparse representation along with the geodesic constraint results in a compact statistical model of shape change by a small number of parameters defined by the user. Experimental validation on multi-object complexes demonstrate robust model estimation across a variety of parameter settings. We further demonstrate the utility of our method to support the analysis of derived shape features, such as volume, and explore shape model extrapolation. Our method is freely available in the software package deformetrica which can be downloaded at www.deformetrica.org.

Comparison of compressed sensing diffusion spectrum imaging and diffusion tensor imaging in patients with intracranial masses.

PURPOSE: To compare compressed diffusion spectrum imaging (CS-DSI) with diffusion tensor imaging (DTI) in patients with intracranial masses. We hypothesized that CS-DSI would provide superior visualization of the motor and language tracts. MATERIALS AND METHODS: We retrospectively analyzed 25 consecutive patients with intracranial masses who underwent DTI and CS-DSI for preoperative planning. Directionally-encoded anisotropy maps, and streamline hand corticospinal motor tracts and arcuate fasciculus language tracts were graded according to a 3-point scale. Tract counts, anisotropy, and lengths were also calculated. Comparisons were made using exact marginal homogeneity, McNemar's and Wilcoxon signed-rank tests. RESULTS: Readers preferred the CS-DSI over DTI anisotropy maps in 92% of the cases, and the CS-DSI over DTI tracts in 84%. The motor tracts were graded as excellent in 80% of cases for CS-DSI versus 52% for DTI; 58% of the motor tracts graded as acceptable in DTI were graded as excellent in CS-DSI (p=0.02). The language tracts were graded as excellent in 68% for CS-DSI versus none for DTI; 78% of the language tracts graded as acceptable by DTI were graded as excellent by CS-DSI (p<0.001). CS-DSI demonstrated smaller normalized mean differences than DTI for motor tract counts, anisotropy and language tract counts (p≤0.01). CONCLUSION: CS-DSI was preferred over DTI for the evaluation of motor and language white matter tracts in patients with intracranial masses. Results suggest that CS-DSI may be more useful than DTI for preoperative planning purposes.

Multi-modal image set registration and atlas formation.

In this paper, we present a Bayesian framework for both generating inter-subject large deformation transformations between two multi-modal image sets of the brain and for forming multi-class brain atlases. In this framework, the estimated transformations are generated using maximal information about the underlying neuroanatomy present in each of the different modalities. This modality independent registration framework is achieved by jointly estimating the posterior probabilities associated with the multi-modal image sets and the high-dimensional registration transformations mapping these posteriors. To maximally use the information present in all the modalities for registration, Kullback-Leibler divergence between the estimated posteriors is minimized. Registration results for image sets composed of multi-modal MR images of healthy adult human brains are presented. Atlas formation results are presented for a population of five infant human brains.

Modeling 4D Pathological Changes by Leveraging Normative Models.

With the increasing use of efficient multimodal 3D imaging, clinicians are able to access longitudinal imaging to stage pathological diseases, to monitor the efficacy of therapeutic interventions, or to assess and quantify rehabilitation efforts. Analysis of such four-dimensional (4D) image data presenting pathologies, including disappearing and newly appearing lesions, represents a significant challenge due to the presence of complex spatio-temporal changes. Image analysis methods for such 4D image data have to include not only a concept for joint segmentation of 3D datasets to account for inherent correlations of subject-specific repeated scans but also a mechanism to account for large deformations and the destruction and formation of lesions (e.g., edema, bleeding) due to underlying physiological processes associated with damage, intervention, and recovery. In this paper, we propose a novel framework that provides a joint segmentation-registration framework to tackle the inherent problem of image registration in the presence of objects not present in all images of the time series. Our methodology models 4D changes in pathological anatomy across time and and also provides an explicit mapping of a healthy normative template to a subject's image data with pathologies. Since atlas-moderated segmentation methods cannot explain appearance and locality pathological structures that are not represented in the template atlas, the new framework provides different options for initialization via a supervised learning approach, iterative semisupervised active learning, and also transfer learning, which results in a fully automatic 4D segmentation method. We demonstrate the effectiveness of our novel approach with synthetic experiments and a 4D multimodal MRI dataset of severe traumatic brain injury (TBI), including validation via comparison to expert segmentations. However, the proposed methodology is generic in regard to different clinical applications requiring quantitative analysis of 4D imaging representing spatio-temporal changes of pathologies.

Automatic segmentation of MR images of the developing newborn brain.

This paper describes an automatic tissue segmentation method for newborn brains from magnetic resonance images (MRI). The analysis and study of newborn brain MRI is of great interest due to its potential for studying early growth patterns and morphological changes in neurodevelopmental disorders. Automatic segmentation of newborn MRI is a challenging task mainly due to the low intensity contrast and the growth process of the white matter tissue. Newborn white matter tissue undergoes a rapid myelination process, where the nerves are covered in myelin sheathes. It is necessary to identify the white matter tissue as myelinated or non-myelinated regions. The degree of myelination is a fractional voxel property that represents regional changes of white matter as a function of age. Our method makes use of a registered probabilistic brain atlas. The method first uses robust graph clustering and parameter estimation to find the initial intensity distributions. The distribution estimates are then used together with the spatial priors to perform bias correction. Finally, the method refines the segmentation using training sample pruning and non-parametric kernel density estimation. Our results demonstrate that the method is able to segment the brain tissue and identify myelinated and non-myelinated white matter regions.

Automatic Tissue Segmentation of Neonate Brain MR Images with Subject-specific Atlases.

Automatic tissue segmentation of the neonate brain using Magnetic Resonance Images (MRI) is extremely important to study brain development and perform early diagnostics but is challenging due to high variability and inhomogeneity in contrast throughout the image due to incomplete myelination of the white matter tracts. For these reasons, current methods often totally fail or give unsatisfying results. Furthermore, most of the subcortical midbrain structures are misclassified due to a lack of contrast in these regions. We have developed a novel method that creates a probabilistic subject-specific atlas based on a population atlas currently containing a number of manually segmented cases. The generated subject-specific atlas is sharp and adapted to the subject that is being processed. We then segment brain tissue classes using the newly created atlas with a single-atlas expectation maximization based method. Our proposed method leads to a much lower failure rate in our experiments. The overall segmentation results are considerably improved when compared to using a non-subject-specific, population average atlas. Additionally, we have incorporated diffusion information obtained from Diffusion Tensor Images (DTI) to improve the detection of white matter that is not visible at this early age in structural MRI (sMRI) due to a lack of myelination. Although this necessitates the acquisition of an additional sequence, the diffusion information improves the white matter segmentation throughout the brain, especially for the mid-brain structures such as the corpus callosum and the internal capsule.

Automatic brain tumor segmentation by subject specific modification of atlas priors.

RATIONALE AND OBJECTIVES: Manual segmentation of brain tumors from magnetic resonance images is a challenging and time-consuming task. An automated system has been developed for brain tumor segmentation that will provide objective, reproducible segmentations that are close to the manual results. Additionally, the method segments white matter, grey matter, cerebrospinal fluid, and edema. The segmentation of pathology and healthy structures is crucial for surgical planning and intervention. MATERIALS AND METHODS: The method performs the segmentation of a registered set of magnetic resonance images using an expectation-maximization scheme. The segmentation is guided by a spatial probabilistic atlas that contains expert prior knowledge about brain structures. This atlas is modified with the subject-specific brain tumor prior that is computed based on contrast enhancement. RESULTS: Five cases with different types of tumors are selected for evaluation. The results obtained from the automatic segmentation program are compared with results from manual and semi-automated methods. The automated method yields results that have surface distances at roughly 1-4 mm compared with the manual results. CONCLUSION: The automated method can be applied to different types of tumors. Although its performance is below that of the semi-automated method, it has the advantage of requiring no user supervision.

A brain tumor segmentation framework based on outlier detection.

This paper describes a framework for automatic brain tumor segmentation from MR images. The detection of edema is done simultaneously with tumor segmentation, as the knowledge of the extent of edema is important for diagnosis, planning, and treatment. Whereas many other tumor segmentation methods rely on the intensity enhancement produced by the gadolinium contrast agent in the T1-weighted image, the method proposed here does not require contrast enhanced image channels. The only required input for the segmentation procedure is the T2 MR image channel, but it can make use of any additional non-enhanced image channels for improved tissue segmentation. The segmentation framework is composed of three stages. First, we detect abnormal regions using a registered brain atlas as a model for healthy brains. We then make use of the robust estimates of the location and dispersion of the normal brain tissue intensity clusters to determine the intensity properties of the different tissue types. In the second stage, we determine from the T2 image intensities whether edema appears together with tumor in the abnormal regions. Finally, we apply geometric and spatial constraints to the detected tumor and edema regions. The segmentation procedure has been applied to three real datasets, representing different tumor shapes, locations, sizes, image intensities, and enhancement.

GEODESIC REGRESSION OF IMAGE AND SHAPE DATA FOR IMPROVED MODELING OF 4D TRAJECTORIES.

A variety of regression schemes have been proposed on images or shapes, although available methods do not handle them jointly. In this paper, we present a framework for joint image and shape regression which incorporates images as well as anatomical shape information in a consistent manner. Evolution is described by a generative model that is the analog of linear regression, which is fully characterized by baseline images and shapes (intercept) and initial momenta vectors (slope). Further, our framework adopts a control point parameterization of deformations, where the dimensionality of the deformation is determined by the complexity of anatomical changes in time rather than the sampling of the image and/or the geometric data. We derive a gradient descent algorithm which simultaneously estimates baseline images and shapes, location of control points, and momenta. Experiments on real medical data demonstrate that our framework effectively combines image and shape information, resulting in improved modeling of 4D (3D space + time) trajectories.

A JOINT FRAMEWORK FOR 4D SEGMENTATION AND ESTIMATION OF SMOOTH TEMPORAL APPEARANCE CHANGES.

Medical imaging studies increasingly use longitudinal images of individual subjects in order to follow-up changes due to development, degeneration, disease progression or efficacy of therapeutic intervention. Repeated image data of individuals are highly correlated, and the strong causality of information over time lead to the development of procedures for joint segmentation of the series of scans, called 4D segmentation. A main aim was improved consistency of quantitative analysis, most often solved via patient-specific atlases. Challenging open problems are contrast changes and occurance of subclasses within tissue as observed in multimodal MRI of infant development, neurodegeneration and disease. This paper proposes a new 4D segmentation framework that enforces continuous dynamic changes of tissue contrast patterns over time as observed in such data. Moreover, our model includes the capability to segment different contrast patterns within a specific tissue class, for example as seen in myelinated and unmyelinated white matter regions in early brain development. Proof of concept is shown with validation on synthetic image data and with 4D segmentation of longitudinal, multimodal pediatric MRI taken at 6, 12 and 24 months of age, but the methodology is generic w.r.t. different application domains using serial imaging.

Characterizing growth patterns in longitudinal MRI using image contrast.

Understanding the growth patterns of the early brain is crucial to the study of neuro-development. In the early stages of brain growth, a rapid sequence of biophysical and chemical processes take place. A crucial component of these processes, known as myelination, consists of the formation of a myelin sheath around a nerve fiber, enabling the effective transmission of neural impulses. As the brain undergoes myelination, there is a subsequent change in the contrast between gray matter and white matter as observed in MR scans. In this work, gray-white matter contrast is proposed as an effective measure of appearance which is relatively invariant to location, scanner type, and scanning conditions. To validate this, contrast is computed over various cortical regions for an adult human phantom. MR (Magnetic Resonance) images of the phantom were repeatedly generated using different scanners, and at different locations. Contrast displays less variability over changing conditions of scan compared to intensity-based measures, demonstrating that it is less dependent than intensity on external factors. Additionally, contrast is used to analyze longitudinal MR scans of the early brain, belonging to healthy controls and Down's Syndrome (DS) patients. Kernel regression is used to model subject-specific trajectories of contrast changing with time. Trajectories of contrast changing with time, as well as time-based biomarkers extracted from contrast modeling, show large differences between groups. The preliminary applications of contrast based analysis indicate its future potential to reveal new information not covered by conventional volumetric or deformation-based analysis, particularly for distinguishing between normal and abnormal growth patterns.