A Deep Convolutional Neural Network for Annotation of Magnetic Resonance Imaging Sequence Type.

The explosion of medical imaging data along with the advent of big data analytics has launched an exciting era for clinical research. One factor affecting the ability to aggregate large medical image collections for research is the lack of infrastructure for automated data annotation. Among all imaging modalities, annotation of magnetic resonance (MR) images is particularly challenging due to the non-standard labeling of MR image types. In this work, we aimed to train a deep neural network to annotate MR image sequence type for scans of brain tumor patients. We focused on the four most common MR sequence types within neuroimaging: T1-weighted (T1W), T1-weighted post-gadolinium contrast (T1Gd), T2-weighted (T2W), and T2-weighted fluid-attenuated inversion recovery (FLAIR). Our repository contains images acquired using a variety of pulse sequences, sequence parameters, field strengths, and scanner manufacturers. Image selection was agnostic to patient demographics, diagnosis, and the presence of tumor in the imaging field of view. We used a total of 14,400 two-dimensional images, each visualizing a different part of the brain. Data was split into train, validation, and test sets (9600, 2400, and 2400 images, respectively) and sets consisted of equal-sized groups of image types. Overall, the model reached an accuracy of 99% on the test set. Our results showed excellent performance of deep learning techniques in predicting sequence types for brain tumor MR images. We conclude deep learning models can serve as tools to support clinical research and facilitate efficient database management.

Biologically informed deep neural networks provide quantitative assessment of intratumoral heterogeneity in post treatment glioblastoma.

Intratumoral heterogeneity poses a significant challenge to the diagnosis and treatment of recurrent glioblastoma. This study addresses the need for non-invasive approaches to map heterogeneous landscape of histopathological alterations throughout the entire lesion for each patient. We developed BioNet, a biologically-informed neural network, to predict regional distributions of two primary tissue-specific gene modules: proliferating tumor (Pro) and reactive/inflammatory cells (Inf). BioNet significantly outperforms existing methods (p < 2e-26). In cross-validation, BioNet achieved AUCs of 0.80 (Pro) and 0.81 (Inf), with accuracies of 80% and 75%, respectively. In blind tests, BioNet achieved AUCs of 0.80 (Pro) and 0.76 (Inf), with accuracies of 81% and 74%. Competing methods had AUCs lower or around 0.6 and accuracies lower or around 70%. BioNet's voxel-level prediction maps reveal intratumoral heterogeneity, potentially improving biopsy targeting and treatment evaluation. This non-invasive approach facilitates regular monitoring and timely therapeutic adjustments, highlighting the role of ML in precision medicine.

Quantifying intra-tumoral genetic heterogeneity of glioblastoma toward precision medicine using MRI and a data-inclusive machine learning algorithm.

BACKGROUND AND OBJECTIVE: Glioblastoma (GBM) is one of the most aggressive and lethal human cancers. Intra-tumoral genetic heterogeneity poses a significant challenge for treatment. Biopsy is invasive, which motivates the development of non-invasive, MRI-based machine learning (ML) models to quantify intra-tumoral genetic heterogeneity for each patient. This capability holds great promise for enabling better therapeutic selection to improve patient outcome. METHODS: We proposed a novel Weakly Supervised Ordinal Support Vector Machine (WSO-SVM) to predict regional genetic alteration status within each GBM tumor using MRI. WSO-SVM was applied to a unique dataset of 318 image-localized biopsies with spatially matched multiparametric MRI from 74 GBM patients. The model was trained to predict the regional genetic alteration of three GBM driver genes (EGFR, PDGFRA and PTEN) based on features extracted from the corresponding region of five MRI contrast images. For comparison, a variety of existing ML algorithms were also applied. Classification accuracy of each gene were compared between the different algorithms. The SHapley Additive exPlanations (SHAP) method was further applied to compute contribution scores of different contrast images. Finally, the trained WSO-SVM was used to generate prediction maps within the tumoral area of each patient to help visualize the intra-tumoral genetic heterogeneity. RESULTS: WSO-SVM achieved 0.80 accuracy, 0.79 sensitivity, and 0.81 specificity for classifying EGFR; 0.71 accuracy, 0.70 sensitivity, and 0.72 specificity for classifying PDGFRA; 0.80 accuracy, 0.78 sensitivity, and 0.83 specificity for classifying PTEN; these results significantly outperformed the existing ML algorithms. Using SHAP, we found that the relative contributions of the five contrast images differ between genes, which are consistent with findings in the literature. The prediction maps revealed extensive intra-tumoral region-to-region heterogeneity within each individual tumor in terms of the alteration status of the three genes. CONCLUSIONS: This study demonstrated the feasibility of using MRI and WSO-SVM to enable non-invasive prediction of intra-tumoral regional genetic alteration for each GBM patient, which can inform future adaptive therapies for individualized oncology.

Towards Longitudinal Glioma Segmentation: Evaluating combined pre- and post-treatment MRI training data for automated tumor segmentation using nnU-Net.

Identification of key phenotypic regions such as necrosis, contrast enhancement, and edema on magnetic resonance imaging (MRI) is important for understanding disease evolution and treatment response in patients with glioma. Manual delineation is time intensive and not feasible for a clinical workflow. Automating phenotypic region segmentation overcomes many issues with manual segmentation, however, current glioma segmentation datasets focus on pre-treatment, diagnostic scans, where treatment effects and surgical cavities are not present. Thus, existing automatic segmentation models are not applicable to post-treatment imaging that is used for longitudinal evaluation of care. Here, we present a comparison of three-dimensional convolutional neural networks (nnU-Net architecture) trained on large temporally defined pre-treatment, post-treatment, and mixed cohorts. We used a total of 1563 imaging timepoints from 854 patients curated from 13 different institutions as well as diverse public data sets to understand the capabilities and limitations of automatic segmentation on glioma images with different phenotypic and treatment appearance. We assessed the performance of models using Dice coefficients on test cases from each group comparing predictions with manual segmentations generated by trained technicians. We demonstrate that training a combined model can be as effective as models trained on just one temporal group. The results highlight the importance of a diverse training set, that includes images from the course of disease and with effects from treatment, in the creation of a model that can accurately segment glioma MRIs at multiple treatment time points.

Image-based models of T-cell distribution identify a clinically meaningful response to a dendritic cell vaccine in patients with glioblastoma.

BACKGROUND: Glioblastoma is an extraordinarily heterogeneous tumor, yet the current treatment paradigm is a "one size fits all" approach. Hundreds of glioblastoma clinical trials have been deemed failures because they did not extend median survival, but these cohorts are comprised of patients with diverse tumors. Current methods of assessing treatment efficacy fail to fully account for this heterogeneity. METHODS: Using an image-based modeling approach, we predicted T-cell abundance from serial MRIs of patients enrolled in the dendritic cell (DC) vaccine clinical trial. T-cell predictions were quantified in both the contrast-enhancing and non-enhancing regions of the imageable tumor, and changes over time were assessed. RESULTS: A subset of patients in a DC vaccine clinical trial, who had previously gone undetected, were identified as treatment responsive and benefited from prolonged survival. A mere two months after initial vaccine administration, responsive patients had a decrease in model-predicted T-cells within the contrast-enhancing region, with a simultaneous increase in the T2/FLAIR region. CONCLUSIONS: In a field that has yet to see breakthrough therapies, these results highlight the value of machine learning in enhancing clinical trial assessment, improving our ability to prospectively prognosticate patient outcomes, and advancing the pursuit towards individualized medicine.

Integrated molecular and multiparametric MRI mapping of high-grade glioma identifies regional biologic signatures.

Sampling restrictions have hindered the comprehensive study of invasive non-enhancing (NE) high-grade glioma (HGG) cell populations driving tumor progression. Here, we present an integrated multi-omic analysis of spatially matched molecular and multi-parametric magnetic resonance imaging (MRI) profiling across 313 multi-regional tumor biopsies, including 111 from the NE, across 68 HGG patients. Whole exome and RNA sequencing uncover unique genomic alterations to unresectable invasive NE tumor, including subclonal events, which inform genomic models predictive of geographic evolution. Infiltrative NE tumor is alternatively enriched with tumor cells exhibiting neuronal or glycolytic/plurimetabolic cellular states, two principal transcriptomic pathway-based glioma subtypes, which respectively demonstrate abundant private mutations or enrichment in immune cell signatures. These NE phenotypes are non-invasively identified through normalized K2 imaging signatures, which discern cell size heterogeneity on dynamic susceptibility contrast (DSC)-MRI. NE tumor populations predicted to display increased cellular proliferation by mean diffusivity (MD) MRI metrics are uniquely associated with EGFR amplification and CDKN2A homozygous deletion. The biophysical mapping of infiltrative HGG potentially enables the clinical recognition of tumor subpopulations with aggressive molecular signatures driving tumor progression, thereby informing precision medicine targeting.

Image-localized biopsy mapping of brain tumor heterogeneity: A single-center study protocol.

Brain cancers pose a novel set of difficulties due to the limited accessibility of human brain tumor tissue. For this reason, clinical decision-making relies heavily on MR imaging interpretation, yet the mapping between MRI features and underlying biology remains ambiguous. Standard (clinical) tissue sampling fails to capture the full heterogeneity of the disease. Biopsies are required to obtain a pathological diagnosis and are predominantly taken from the tumor core, which often has different traits to the surrounding invasive tumor that typically leads to recurrent disease. One approach to solving this issue is to characterize the spatial heterogeneity of molecular, genetic, and cellular features of glioma through the intraoperative collection of multiple image-localized biopsy samples paired with multi-parametric MRIs. We have adopted this approach and are currently actively enrolling patients for our 'Image-Based Mapping of Brain Tumors' study. Patients are eligible for this research study (IRB #16-002424) if they are 18 years or older and undergoing surgical intervention for a brain lesion. Once identified, candidate patients receive dynamic susceptibility contrast (DSC) perfusion MRI and diffusion tensor imaging (DTI), in addition to standard sequences (T1, T1Gd, T2, T2-FLAIR) at their presurgical scan. During surgery, sample anatomical locations are tracked using neuronavigation. The collected specimens from this research study are used to capture the intra-tumoral heterogeneity across brain tumors including quantification of genetic aberrations through whole-exome and RNA sequencing as well as other tissue analysis techniques. To date, these data (made available through a public portal) have been used to generate, test, and validate predictive regional maps of the spatial distribution of tumor cell density and/or treatment-related key genetic marker status to identify biopsy and/or treatment targets based on insight from the entire tumor makeup. This type of methodology, when delivered within clinically feasible time frames, has the potential to further inform medical decision-making by improving surgical intervention, radiation, and targeted drug therapy for patients with glioma.

Weakly Supervised Skull Stripping of Magnetic Resonance Imaging of Brain Tumor Patients.

Automatic brain tumor segmentation is particularly challenging on magnetic resonance imaging (MRI) with marked pathologies, such as brain tumors, which usually cause large displacement, abnormal appearance, and deformation of brain tissue. Despite an abundance of previous literature on learning-based methodologies for MRI segmentation, few works have focused on tackling MRI skull stripping of brain tumor patient data. This gap in literature can be associated with the lack of publicly available data (due to concerns about patient identification) and the labor-intensive nature of generating ground truth labels for model training. In this retrospective study, we assessed the performance of Dense-Vnet in skull stripping brain tumor patient MRI trained on our large multi-institutional brain tumor patient dataset. Our data included pretreatment MRI of 668 patients from our in-house institutional review board-approved multi-institutional brain tumor repository. Because of the absence of ground truth, we used imperfect automatically generated training labels using SPM12 software. We trained the network using common MRI sequences in oncology: T1-weighted with gadolinium contrast, T2-weighted fluid-attenuated inversion recovery, or both. We measured model performance against 30 independent brain tumor test cases with available manual brain masks. All images were harmonized for voxel spacing and volumetric dimensions before model training. Model training was performed using the modularly structured deep learning platform NiftyNet that is tailored toward simplifying medical image analysis. Our proposed approach showed the success of a weakly supervised deep learning approach in MRI brain extraction even in the presence of pathology. Our best model achieved an average Dice score, sensitivity, and specificity of, respectively, 94.5, 96.4, and 98.5% on the multi-institutional independent brain tumor test set. To further contextualize our results within existing literature on healthy brain segmentation, we tested the model against healthy subjects from the benchmark LBPA40 dataset. For this dataset, the model achieved an average Dice score, sensitivity, and specificity of 96.2, 96.6, and 99.2%, which are, although comparable to other publications, slightly lower than the performance of models trained on healthy patients. We associate this drop in performance with the use of brain tumor data for model training and its influence on brain appearance.

Uncertainty quantification in the radiogenomics modeling of EGFR amplification in glioblastoma.

Radiogenomics uses machine-learning (ML) to directly connect the morphologic and physiological appearance of tumors on clinical imaging with underlying genomic features. Despite extensive growth in the area of radiogenomics across many cancers, and its potential role in advancing clinical decision making, no published studies have directly addressed uncertainty in these model predictions. We developed a radiogenomics ML model to quantify uncertainty using transductive Gaussian Processes (GP) and a unique dataset of 95 image-localized biopsies with spatially matched MRI from 25 untreated Glioblastoma (GBM) patients. The model generated predictions for regional EGFR amplification status (a common and important target in GBM) to resolve the intratumoral genetic heterogeneity across each individual tumor-a key factor for future personalized therapeutic paradigms. The model used probability distributions for each sample prediction to quantify uncertainty, and used transductive learning to reduce the overall uncertainty. We compared predictive accuracy and uncertainty of the transductive learning GP model against a standard GP model using leave-one-patient-out cross validation. Additionally, we used a separate dataset containing 24 image-localized biopsies from 7 high-grade glioma patients to validate the model. Predictive uncertainty informed the likelihood of achieving an accurate sample prediction. When stratifying predictions based on uncertainty, we observed substantially higher performance in the group cohort (75% accuracy, n = 95) and amongst sample predictions with the lowest uncertainty (83% accuracy, n = 72) compared to predictions with higher uncertainty (48% accuracy, n = 23), due largely to data interpolation (rather than extrapolation). On the separate validation set, our model achieved 78% accuracy amongst the sample predictions with lowest uncertainty. We present a novel approach to quantify radiogenomics uncertainty to enhance model performance and clinical interpretability. This should help integrate more reliable radiogenomics models for improved medical decision-making.

Days gained response discriminates treatment response in patients with recurrent glioblastoma receiving bevacizumab-based therapies.

BACKGROUND: Accurate assessments of patient response to therapy are a critical component of personalized medicine. In glioblastoma (GBM), the most aggressive form of brain cancer, tumor growth dynamics are heterogenous across patients, complicating assessment of treatment response. This study aimed to analyze days gained (DG), a burgeoning model-based dynamic metric, for response assessment in patients with recurrent GBM who received bevacizumab-based therapies. METHODS: DG response scores were calculated using volumetric tumor segmentations for patients receiving bevacizumab with and without concurrent cytotoxic therapy (N = 62). Kaplan-Meier and Cox proportional hazards analyses were implemented to examine DG prognostic relationship to overall (OS) and progression-free survival (PFS) from the onset of treatment for recurrent GBM. RESULTS: In patients receiving concurrent bevacizumab and cytotoxic therapy, Kaplan-Meier analysis showed significant differences in OS and PFS at DG cutoffs consistent with previously identified values from newly diagnosed GBM using T1-weighted gadolinium-enhanced magnetic resonance imaging (T1Gd). DG scores for bevacizumab monotherapy patients only approached significance for PFS. Cox regression showed that increases of 25 DG on T1Gd imaging were significantly associated with a 12.5% reduction in OS hazard for concurrent therapy patients and a 4.4% reduction in PFS hazard for bevacizumab monotherapy patients. CONCLUSION: DG has significant meaning in recurrent therapy as a metric of treatment response, even in the context of anti-angiogenic therapies. This provides further evidence supporting the use of DG as an adjunct response metric that quantitatively connects treatment response and clinical outcomes.

Image-based metric of invasiveness predicts response to adjuvant temozolomide for primary glioblastoma.

BACKGROUND: Temozolomide (TMZ) has been the standard-of-care chemotherapy for glioblastoma (GBM) patients for more than a decade. Despite this long time in use, significant questions remain regarding how best to optimize TMZ therapy for individual patients. Understanding the relationship between TMZ response and factors such as number of adjuvant TMZ cycles, patient age, patient sex, and image-based tumor features, might help predict which GBM patients would benefit most from TMZ, particularly for those whose tumors lack O6-methylguanine-DNA methyltransferase (MGMT) promoter methylation. METHODS AND FINDINGS: Using a cohort of 90 newly-diagnosed GBM patients treated according to the standard of care, we examined the relationships between several patient and tumor characteristics and volumetric and survival outcomes during adjuvant chemotherapy. Volumetric changes in MR imaging abnormalities during adjuvant therapy were used to assess TMZ response. T1Gd volumetric response is associated with younger patient age, increased number of TMZ cycles, longer time to nadir volume, and decreased tumor invasiveness. Moreover, increased adjuvant TMZ cycles corresponded with improved volumetric response only among more nodular tumors, and this volumetric response was associated with improved survival outcomes. Finally, in a subcohort of patients with known MGMT methylation status, methylated tumors were more diffusely invasive than unmethylated tumors, suggesting the improved response in nodular tumors is not driven by a preponderance of MGMT methylated tumors. CONCLUSIONS: Our finding that less diffusely invasive tumors are associated with greater volumetric response to TMZ suggests patients with these tumors may benefit from additional adjuvant TMZ cycles, even for those without MGMT methylation.

Deep neural network to locate and segment brain tumors outperformed the expert technicians who created the training data.

Purpose: Deep learning (DL) algorithms have shown promising results for brain tumor segmentation in MRI. However, validation is required prior to routine clinical use. We report the first randomized and blinded comparison of DL and trained technician segmentations. Approach: We compiled a multi-institutional database of 741 pretreatment MRI exams. Each contained a postcontrast T1-weighted exam, a T2-weighted fluid-attenuated inversion recovery exam, and at least one technician-derived tumor segmentation. The database included 729 unique patients (470 males and 259 females). Of these exams, 641 were used for training the DL system, and 100 were reserved for testing. We developed a platform to enable qualitative, blinded, controlled assessment of lesion segmentations made by technicians and the DL method. On this platform, 20 neuroradiologists performed 400 side-by-side comparisons of segmentations on 100 test cases. They scored each segmentation between 0 (poor) and 10 (perfect). Agreement between segmentations from technicians and the DL method was also evaluated quantitatively using the Dice coefficient, which produces values between 0 (no overlap) and 1 (perfect overlap). Results: The neuroradiologists gave technician and DL segmentations mean scores of 6.97 and 7.31, respectively ( p < 0.00007 ). The DL method achieved a mean Dice coefficient of 0.87 on the test cases. Conclusions: This was the first objective comparison of automated and human segmentation using a blinded controlled assessment study. Our DL system learned to outperform its "human teachers" and produced output that was better, on average, than its training data.

Sex differences in GBM revealed by analysis of patient imaging, transcriptome, and survival data.

Sex differences in the incidence and outcome of human disease are broadly recognized but, in most cases, not sufficiently understood to enable sex-specific approaches to treatment. Glioblastoma (GBM), the most common malignant brain tumor, provides a case in point. Despite well-established differences in incidence and emerging indications of differences in outcome, there are few insights that distinguish male and female GBM at the molecular level or allow specific targeting of these biological differences. Here, using a quantitative imaging-based measure of response, we found that standard therapy is more effective in female compared with male patients with GBM. We then applied a computational algorithm to linked GBM transcriptome and outcome data and identified sex-specific molecular subtypes of GBM in which cell cycle and integrin signaling are the critical determinants of survival for male and female patients, respectively. The clinical relevance of cell cycle and integrin signaling pathway signatures was further established through correlations between gene expression and in vitro chemotherapy sensitivity in a panel of male and female patient-derived GBM cell lines. Together, these results suggest that greater precision in GBM molecular subtyping can be achieved through sex-specific analyses and that improved outcomes for all patients might be accomplished by tailoring treatment to sex differences in molecular mechanisms.

Integration of machine learning and mechanistic models accurately predicts variation in cell density of glioblastoma using multiparametric MRI.

Glioblastoma (GBM) is a heterogeneous and lethal brain cancer. These tumors are followed using magnetic resonance imaging (MRI), which is unable to precisely identify tumor cell invasion, impairing effective surgery and radiation planning. We present a novel hybrid model, based on multiparametric intensities, which combines machine learning (ML) with a mechanistic model of tumor growth to provide spatially resolved tumor cell density predictions. The ML component is an imaging data-driven graph-based semi-supervised learning model and we use the Proliferation-Invasion (PI) mechanistic tumor growth model. We thus refer to the hybrid model as the ML-PI model. The hybrid model was trained using 82 image-localized biopsies from 18 primary GBM patients with pre-operative MRI using a leave-one-patient-out cross validation framework. A Relief algorithm was developed to quantify relative contributions from the data sources. The ML-PI model statistically significantly outperformed (p < 0.001) both individual models, ML and PI, achieving a mean absolute predicted error (MAPE) of 0.106 ± 0.125 versus 0.199 ± 0.186 (ML) and 0.227 ± 0.215 (PI), respectively. Associated Pearson correlation coefficients for ML-PI, ML, and PI were 0.838, 0.518, and 0.437, respectively. The Relief algorithm showed the PI model had the greatest contribution to the result, emphasizing the importance of the hybrid model in achieving the high accuracy.

Motivating the additional use of external validity: examining transportability in a model of glioblastoma multiforme.

Despite the growing ubiquity of data in the medical domain, it remains difficult to apply results from experimental and observational studies to additional populations suffering from the same disease. Many methods are employed for testing internal validity; yet limited effort is made in testing generalizability, or external validity. The development of disease models often suffers from this lack of validity testing and trained models frequently have worse performance on different populations, rendering them ineffective. In this work, we discuss the use of transportability theory, a causal graphical model examination, as a mechanism for determining what elements of a data resource can be shared or moved between a source and target population. A simplified Bayesian model of glioblastoma multiforme serves as the example for discussion and preliminary analysis. Examination over data collection hospitals from the TCGA dataset demonstrated improvement of prediction in a transported model over a baseline model.

Comparing predictive models of glioblastoma multiforme built using multi-institutional and local data sources.

The growing amount of electronic data collected from patient care and clinical trials is motivating the creation of national repositories where multiple institutions share data about their patient cohorts. Such efforts aim to provide sufficient sample sizes for data mining and predictive modeling, ultimately improving treatment recommendations and patient outcome prediction. While these repositories offer the potential to improve our understanding of a disease, potential issues need to be addressed to ensure that multi-site data and resultant predictive models are useful to non-contributing institutions. In this paper we examine the challenges of utilizing National Cancer Institute datasets for modeling glioblastoma multiforme. We created several types of prognostic models and compared their results against models generated using data solely from our institution. While overall model performance between the data sources was similar, different variables were selected during model generation, suggesting that mapping data resources between models is not a straightforward issue.

Wireless data collection of self-administered surveys using tablet computers.

The accurate and expeditious collection of survey data by coordinators in the field is critical in the support of research studies. Early methods that used paper documentation have slowly evolved into electronic capture systems. Indeed, tools such as REDCap and others illustrate this transition. However, many current systems are tailored web-browsers running on desktop/laptop computers, requiring keyboard and mouse input. We present a system that utilizes a touch screen interface running on a tablet PC with consideration for portability, limited screen space, wireless connectivity, and potentially inexperienced and low literacy users. The system was developed using C#, ASP.net, and SQL Server by multiple programmers over the course of a year. The system was developed in coordination with UCLA Family Medicine and is currently deployed for the collection of data in a group of Los Angeles area clinics of community health centers for a study on drug addiction and intervention.