Improving the Automatic Classification of Brain MRI Acquisition Contrast with Machine Learning.

Automated quantification of data acquired as part of an MRI exam requires identification of the specific acquisition of relevance to a particular analysis. This motivates the development of methods capable of reliably classifying MRI acquisitions according to their nominal contrast type, e.g., T1 weighted, T1 post-contrast, T2 weighted, T2-weighted FLAIR, proton-density weighted. Prior studies have investigated using imaging-based methods and DICOM metadata-based methods with success on cohorts of patients acquired as part of a clinical trial. This study compares the performance of these methods on heterogeneous clinical datasets acquired with many different scanners from many institutions. RF and CNN models were trained on metadata and pixel data, respectively. A combined RF model incorporated CNN logits from the pixel-based model together with metadata. Four cohorts were used for model development and evaluation: MS research (n = 11,106 series), MS clinical (n = 3244 series), glioma research (n = 612 series, test/validation only), and ADNI PTSD (n = 477 series, training only). Together, these cohorts represent a broad range of acquisition contexts (scanners, sequences, institutions) and subject pathologies. Pixel-based CNN and combined models achieved accuracies between 97 and 98% on the clinical MS cohort. Validation/test accuracies with the glioma cohort were 99.7% (metadata only) and 98.4 (CNN). Accurate and generalizable classification of MRI acquisition contrast types was demonstrated. Such methods are important for enabling automated data selection in high-throughput and big-data image analysis applications.

The Conditional Super Learner.

Using cross validation to select the best model from a library is standard practice in machine learning. Similarly, meta learning is a widely used technique where models previously developed are combined (mainly linearly) with the expectation of improving performance with respect to individual models. In this article we consider the Conditional Super Learner (CSL), an algorithm that selects the best model candidate from a library of models conditional on the covariates. The CSL expands the idea of using cross validation to select the best model and merges it with meta learning. We propose an optimization algorithm that finds a local minimum to the problem posed and proves that it converges at a rate faster than Op(n-1/4). We offer empirical evidence that: (1) CSL is an excellent candidate to substitute stacking and (2) CLS is suitable for the analysis of Hierarchical problems. Additionally, implications for global interpretability are emphasized.

Improving the noninvasive classification of glioma genetic subtype with deep learning and diffusion-weighted imaging.

BACKGROUND: Diagnostic classification of diffuse gliomas now requires an assessment of molecular features, often including IDH-mutation and 1p19q-codeletion status. Because genetic testing requires an invasive process, an alternative noninvasive approach is attractive, particularly if resection is not recommended. The goal of this study was to evaluate the effects of training strategy and incorporation of biologically relevant images on predicting genetic subtypes with deep learning. METHODS: Our dataset consisted of 384 patients with newly diagnosed gliomas who underwent preoperative MRI with standard anatomical and diffusion-weighted imaging, and 147 patients from an external cohort with anatomical imaging. Using tissue samples acquired during surgery, each glioma was classified into IDH-wildtype (IDHwt), IDH-mutant/1p19q-noncodeleted (IDHmut-intact), and IDH-mutant/1p19q-codeleted (IDHmut-codel) subgroups. After optimizing training parameters, top performing convolutional neural network (CNN) classifiers were trained, validated, and tested using combinations of anatomical and diffusion MRI with either a 3-class or tiered structure. Generalization to an external cohort was assessed using anatomical imaging models. RESULTS: The best model used a 3-class CNN containing diffusion-weighted imaging as an input, achieving 85.7% (95% CI: [77.1, 100]) overall test accuracy and correctly classifying 95.2%, 88.9%, 60.0% of the IDHwt, IDHmut-intact, and IDHmut-codel tumors. In general, 3-class models outperformed tiered approaches by 13.5%-17.5%, and models that included diffusion-weighted imaging were 5%-8.8% more accurate than those that used only anatomical imaging. CONCLUSION: Training a classifier to predict both IDH-mutation and 1p19q-codeletion status outperformed a tiered structure that first predicted IDH-mutation, then 1p19q-codeletion. Including apparent diffusion coefficient (ADC), a surrogate marker of cellularity, more accurately captured differences between subgroups.

The Physiological Deep Learner: First application of multitask deep learning to predict hypotension in critically ill patients.

Critical care clinicians are trained to analyze simultaneously multiple physiological parameters to predict critical conditions such as hemodynamic instability. We developed the Multi-task Learning Physiological Deep Learner (MTL-PDL), a deep learning algorithm that predicts simultaneously the mean arterial pressure (MAP) and the heart rate (HR). In an external validation dataset, our model exhibited very good calibration: R2 of 0.747 (95% confidence interval, 0.692 to 0.794) and 0.850 (0.815 to 0.879) for respectively, MAP and HR prediction 60-minutes ahead of time. For acute hypotensive episodes defined as a MAP below 65 mmHg for 5 min, our MTL-PDL reached a predictive value of 90% for patients at very high risk (predicted MAP ≤ 60 mmHg) and 2‰ for patients at low risk (predicted MAP >70 mmHg). Based on its excellent prediction performance, the Physiological Deep Learner has the potential to help the clinician proactively adjust the treatment in order to avoid hypotensive episodes and end-organ hypoperfusion.

Targeted transfer learning to improve performance in small medical physics datasets.

PURPOSE: To perform an in-depth evaluation of current state of the art techniques in training neural networks to identify appropriate approaches in small datasets. METHOD: In total, 112,120 frontal-view X-ray images from the NIH ChestXray14 dataset were used in our analysis. Two tasks were studied: unbalanced multi-label classification of 14 diseases, and binary classification of pneumonia vs non-pneumonia. All datasets were randomly split into training, validation, and testing (70%, 10%, and 20%). Two popular convolution neural networks (CNNs), DensNet121 and ResNet50, were trained using PyTorch. We performed several experiments to test: (a) whether transfer learning using pretrained networks on ImageNet are of value to medical imaging/physics tasks (e.g., predicting toxicity from radiographic images after training on images from the internet), (b) whether using pretrained networks trained on problems that are similar to the target task helps transfer learning (e.g., using X-ray pretrained networks for X-ray target tasks), (c) whether freeze deep layers or change all weights provides an optimal transfer learning strategy, (d) the best strategy for the learning rate policy, and (e) what quantity of data is needed in order to appropriately deploy these various strategies (N = 50 to N = 77 880). RESULTS: In the multi-label problem, DensNet121 needed at least 1600 patients to be comparable to, and 10 000 to outperform, radiomics-based logistic regression. In classifying pneumonia vs non-pneumonia, both CNN and radiomics-based methods performed poorly when N < 2000. For small datasets ( < 2000), however, a significant boost in performance (>15% increase on AUC) comes from a good selection of the transfer learning dataset, dropout, cycling learning rate, and freezing and unfreezing of deep layers as training progresses. In contrast, if sufficient data are available (>35 000), little or no tweaking is needed to obtain impressive performance. While transfer learning using X-ray images from other anatomical sites improves performance, we also observed a similar boost by using pretrained networks from ImageNet. Having source images from the same anatomical site, however, outperforms every other methodology, by up to 15%. In this case, DL models can be trained with as little as N = 50. CONCLUSIONS: While training DL models in small datasets (N < 2000) is challenging, no tweaking is necessary for bigger datasets (N > 35 000). Using transfer learning with images from the same anatomical site can yield remarkable performance in new tasks with as few as N = 50. Surprisingly, we did not find any advantage for using images from other anatomical sites over networks that have been trained using ImageNet. This indicates that features learned may not be as general as currently believed, and performance decays rapidly even by just changing the anatomical site of the images.

Expert-augmented machine learning.

Machine learning is proving invaluable across disciplines. However, its success is often limited by the quality and quantity of available data, while its adoption is limited by the level of trust afforded by given models. Human vs. machine performance is commonly compared empirically to decide whether a certain task should be performed by a computer or an expert. In reality, the optimal learning strategy may involve combining the complementary strengths of humans and machines. Here, we present expert-augmented machine learning (EAML), an automated method that guides the extraction of expert knowledge and its integration into machine-learned models. We used a large dataset of intensive-care patient data to derive 126 decision rules that predict hospital mortality. Using an online platform, we asked 15 clinicians to assess the relative risk of the subpopulation defined by each rule compared to the total sample. We compared the clinician-assessed risk to the empirical risk and found that, while clinicians agreed with the data in most cases, there were notable exceptions where they overestimated or underestimated the true risk. Studying the rules with greatest disagreement, we identified problems with the training data, including one miscoded variable and one hidden confounder. Filtering the rules based on the extent of disagreement between clinician-assessed risk and empirical risk, we improved performance on out-of-sample data and were able to train with less data. EAML provides a platform for automated creation of problem-specific priors, which help build robust and dependable machine-learning models in critical applications.

Integrated models incorporating radiologic and radiomic features predict meningioma grade, local failure, and overall survival.

BACKGROUND: We investigated prognostic models based on clinical, radiologic, and radiomic feature to preoperatively identify meningiomas at risk for poor outcomes. METHODS: Retrospective review was performed for 303 patients who underwent resection of 314 meningiomas (57% World Health Organization grade I, 35% grade II, and 8% grade III) at two independent institutions, which comprised primary and external datasets. For each patient in the primary dataset, 16 radiologic and 172 radiomic features were extracted from preoperative magnetic resonance images, and prognostic features for grade, local failure (LF) or overall survival (OS) were identified using the Kaplan-Meier method, log-rank tests and recursive partitioning analysis. Regressions and random forests were used to generate and test prognostic models, which were validated using the external dataset. RESULTS: Multivariate analysis revealed that apparent diffusion coefficient hypointensity (HR 5.56, 95% CI 2.01-16.7, P = .002) was associated with high grade meningioma, and low sphericity was associated both with increased LF (HR 2.0, 95% CI 1.1-3.5, P = .02) and worse OS (HR 2.94, 95% CI 1.47-5.56, P = .002). Both radiologic and radiomic predictors of adverse meningioma outcomes were significantly associated with molecular markers of aggressive meningioma biology, such as somatic mutation burden, DNA methylation status, and FOXM1 expression. Integrated prognostic models combining clinical, radiologic, and radiomic features demonstrated improved accuracy for meningioma grade, LF, and OS (area under the curve 0.78, 0.75, and 0.78, respectively) compared to models based on clinical features alone. CONCLUSIONS: Preoperative radiologic and radiomic features such as apparent diffusion coefficient and sphericity can predict tumor grade, LF, and OS in patients with meningioma.

Salvage HDR Brachytherapy: Multiple Hypothesis Testing Versus Machine Learning Analysis.

PURPOSE: Salvage high-dose-rate brachytherapy (sHDRB) is a treatment option for recurrences after prior radiation therapy. However, only approximately 50% of patients benefit, with the majority of second recurrences after salvage brachytherapy occurring distantly. Therefore, identification of characteristics that can help select patients who may benefit most from sHDRB is critical. Machine learning may be used to identify characteristics that predict outcome following sHDRB. We aimed to use machine learning to identify patient characteristics associated with biochemical failure (BF) following prostate sHDRB. METHODS AND MATERIALS: We analyzed data for 52 patients treated with sHDRB for locally recurrent prostate cancer after previous definitive radiation therapy between 1998 and 2009. Following pathologic confirmation of locally recurrent disease without evidence of metastatic disease, 36 Gy in 6 fractions was administered to the prostate and seminal vesicles. BF following sHDRB was defined using the Phoenix definition. Sixteen different clinical risk features were collected, and machine learning analysis was executed to identify subpopulations at higher risk of BF. Decision tree-based algorithms including classification and regression trees, MediBoost, and random forests were constructed. RESULTS: Patients were followed up for a minimum of 5 years after sHDRB. Those with a fraction of positive cores ≥0.35 and a disease-free interval <4.12 years after their initial radiation treatment experienced a higher failure rate after sHDRB of 0.75 versus 0.38 for the rest of the population. CONCLUSIONS: Using machine learning, we have identified that patients with a fraction of positive cores ≥0.35 and a disease-free interval <4.1 years might be associated with a high risk of BF following sHDRB.

Deep nets vs expert designed features in medical physics: An IMRT QA case study.

PURPOSE: The purpose of this study was to compare the performance of Deep Neural Networks against a technique designed by domain experts in the prediction of gamma passing rates for Intensity Modulated Radiation Therapy Quality Assurance (IMRT QA). METHOD: A total of 498 IMRT plans across all treatment sites were planned in Eclipse version 11 and delivered using a dynamic sliding window technique on Clinac iX or TrueBeam Linacs. Measurements were performed using a commercial 2D diode array, and passing rates for 3%/3 mm local dose/distance-to-agreement (DTA) were recorded. Separately, fluence maps calculated for each plan were used as inputs to a convolution neural network (CNN). The CNNs were trained to predict IMRT QA gamma passing rates using TensorFlow and Keras. A set of model architectures, inspired by the convolutional blocks of the VGG-16 ImageNet model, were constructed and implemented. Synthetic data, created by rotating and translating the fluence maps during training, was created to boost the performance of the CNNs. Dropout, batch normalization, and data augmentation were utilized to help train the model. The performance of the CNNs was compared to a generalized Poisson regression model, previously developed for this application, which used 78 expert designed features. RESULTS: Deep Neural Networks without domain knowledge achieved comparable performance to a baseline system designed by domain experts in the prediction of 3%/3 mm Local gamma passing rates. An ensemble of neural nets resulted in a mean absolute error (MAE) of 0.70 ± 0.05 and the domain expert model resulted in a 0.74 ± 0.06. CONCLUSIONS: Convolutional neural networks (CNNs) with transfer learning can predict IMRT QA passing rates by automatically designing features from the fluence maps without human expert supervision. Predictions from CNNs are comparable to a system carefully designed by physicist experts.

Comment on 'Deep convolutional neural network with transfer learning for rectum toxicity prediction in cervical cancer radiotherapy: a feasibility study'.

The application of machine learning (ML) presents tremendous opportunities for the field of oncology, thus we read 'Deep convolutional neural network with transfer learning for rectum toxicity prediction in cervical cancer radiotherapy: a feasibility study' with great interest. In this article, the authors used state of the art techniques: a pre-trained convolutional neural network (VGG-16 CNN), transfer learning, data augmentation, drop out and early stopping, all of which are directly responsible for the success and the excitement that these algorithms have created in other fields. We believe that the use of these techniques can offer tremendous opportunities in the field of Medical Physics and as such we would like to praise the authors for their pioneering application to the field of Radiation Oncology. That being said, given that the field of Medical Physics has unique characteristics that differentiate us from those fields where these techniques have been applied successfully, we would like to raise some points for future discussion and follow up studies that could help the community understand the limitations and nuances of deep learning techniques.