Nearest Neighbor-Based Strategy to Optimize Multi-View Triplet Network for Classification of Small-Sample Medical Imaging Data.

Multi-view classification with limited sample size and data augmentation is a very common machine learning (ML) problem in medicine. With limited data, a triplet network approach for two-stage representation learning has been proposed. However, effective training and verifying the features from the representation network for their suitability in subsequent classifiers are still unsolved problems. Although typical distance-based metrics for the training capture the overall class separability of the features, the performance according to these metrics does not always lead to an optimal classification. Consequently, an exhaustive tuning with all feature-classifier combinations is required to search for the best end result. To overcome this challenge, we developed a novel nearest-neighbor (NN) validation strategy based on the triplet metric. This strategy is supported by a theoretical foundation to provide the best selection of the features with a lower bound of the highest end performance. The proposed strategy is a transparent approach to identify whether to improve the features or the classifier. This avoids the need for repeated tuning. Our evaluations on real-world medical imaging tasks (i.e., radiation therapy delivery error prediction and sarcoma survival prediction) show that our strategy is superior to other common deep representation learning baselines [i.e., autoencoder (AE) and softmax]. The strategy addresses the issue of feature's interpretability which enables more holistic feature creation such that the medical experts can focus on specifying relevant data as opposed to tedious feature engineering.

Gradient Boosted Trees for Spatial Data and Its Application to Medical Imaging Data.

Boosting Trees are one of the most successful statistical learning approaches that involve sequentially growing an ensemble of simple regression trees ("weak learners"). This paper proposes a gradient Boosted Trees algorithm for Spatial Data (Boost-S) with covariate information. Boost-S integrates the spatial correlation into the classical framework of eXtreme Gradient Boosting. Each tree is constructed by solving a regularized optimization problem, where the objective function takes into account the underlying spatial correlation and involves two penalty terms on tree complexity. A computationally-efficient greedy heuristic algorithm is proposed to obtain an ensemble of trees. The proposed Boost-S is applied to the spatially-correlated FDG-PET (fluorodeoxyglucose-positron emission tomography) imaging data collected from clinical trials of cancer chemoradiotherapy. Our numerical investigations successfully demonstrate the advantages of the proposed Boost-S over existing approaches for this particular application.

Multitask Learning Radiomics on Longitudinal Imaging to Predict Survival Outcomes following Risk-Adaptive Chemoradiation for Non-Small Cell Lung Cancer.

Medical imaging provides quantitative and spatial information to evaluate treatment response in the management of patients with non-small cell lung cancer (NSCLC). High throughput extraction of radiomic features on these images can potentially phenotype tumors non-invasively and support risk stratification based on survival outcome prediction. The prognostic value of radiomics from different imaging modalities and time points prior to and during chemoradiation therapy of NSCLC, relative to conventional imaging biomarker or delta radiomics models, remains uncharacterized. We investigated the utility of multitask learning of multi-time point radiomic features, as opposed to single-task learning, for improving survival outcome prediction relative to conventional clinical imaging feature model benchmarks. Survival outcomes were prospectively collected for 45 patients with unresectable NSCLC enrolled on the FLARE-RT phase II trial of risk-adaptive chemoradiation and optional consolidation PD-L1 checkpoint blockade (NCT02773238). FDG-PET, CT, and perfusion SPECT imaging pretreatment and week 3 mid-treatment was performed and 110 IBSI-compliant pyradiomics shape-/intensity-/texture-based features from the metabolic tumor volume were extracted. Outcome modeling consisted of a fused Laplacian sparse group LASSO with component-wise gradient boosting survival regression in a multitask learning framework. Testing performance under stratified 10-fold cross-validation was evaluated for multitask learning radiomics of different imaging modalities and time points. Multitask learning models were benchmarked against conventional clinical imaging and delta radiomics models and evaluated with the concordance index (c-index) and index of prediction accuracy (IPA). FDG-PET radiomics had higher prognostic value for overall survival in test folds (c-index 0.71 [0.67, 0.75]) than CT radiomics (c-index 0.64 [0.60, 0.71]) or perfusion SPECT radiomics (c-index 0.60 [0.57, 0.63]). Multitask learning of pre-/mid-treatment FDG-PET radiomics (c-index 0.71 [0.67, 0.75]) outperformed benchmark clinical imaging (c-index 0.65 [0.59, 0.71]) and FDG-PET delta radiomics (c-index 0.52 [0.48, 0.58]) models. Similarly, the IPA for multitask learning FDG-PET radiomics (30%) was higher than clinical imaging (26%) and delta radiomics (15%) models. Radiomics models performed consistently under different voxel resampling conditions. Multitask learning radiomics for outcome modeling provides a clinical decision support platform that leverages longitudinal imaging information. This framework can reveal the relative importance of different imaging modalities and time points when designing risk-adaptive cancer treatment strategies.

Sensitivity analysis of FDG PET tumor voxel cluster radiomics and dosimetry for predicting mid-chemoradiation regional response of locally advanced lung cancer.

We investigated the sensitivity of regional tumor response prediction to variability in voxel clustering techniques, imaging features, and machine learning algorithms in 25 patients with locally advanced non-small cell lung cancer (LA-NSCLC) enrolled on the FLARE-RT clinical trial. Metabolic tumor volumes (MTV) from pre-chemoradiation (PETpre) and mid-chemoradiation fluorodeoxyglucose-positron emission tomography (FDG PET) images (PETmid) were subdivided into K-means or hierarchical voxel clusters by standardized uptake values (SUV) and 3D-positions. MTV cluster separability was evaluated by CH index, and morphologic changes were captured by Dice similarity and centroid Euclidean distance. PETpre conventional features included SUVmean, MTV/MTV cluster size, and mean radiation dose. PETpre radiomics consisted of 41 intensity histogram and 3D texture features (PET Oncology Radiomics Test Suite) extracted from MTV or MTV clusters. Machine learning models (multiple linear regression, support vector regression, logistic regression, support vector machines) of conventional features or radiomic features were constructed to predict PETmid response. Leave-one-out-cross-validated root-mean-squared-error (RMSE) for continuous response regression (ΔSUVmean) and area-under-receiver-operating-characteristic-curve (AUC) for binary response classification were calculated. K-means MTV 2-clusters (MTVhi, MTVlo) achieved maximum CH index separability (Friedman p < 0.001). Between PETpre and PETmid, MTV cluster pairs overlapped (Dice 0.70-0.87) and migrated 0.6-1.1 cm. PETmid ΔSUVmean response prediction was superior in MTV and MTVlo (RMSE = 0.17-0.21) compared to MTVhi (RMSE = 0.42-0.52, Friedman p < 0.001). PETmid ΔSUVmean response class prediction performance trended higher in MTVlo (AUC = 0.83-0.88) compared to MTVhi (AUC = 0.44-0.58, Friedman p = 0.052). Models were more sensitive to MTV/MTV cluster regions (Friedman p = 0.026) than feature sets/algorithms (Wilcoxon signed-rank p = 0.36). Top-ranked radiomic features included GLZSM-LZHGE (large-zone-high-SUV), GTSDM-CP (cluster-prominence), GTSDM-CS (cluster-shade) and NGTDM-CNT (contrast). Top-ranked features were consistent between MTVhi and MTVlo cluster pairs but varied between MTVhi-MTVlo clusters, reflecting distinct regional radiomic phenotypes. Variability in tumor voxel cluster response prediction can inform robust radiomic target definition for risk-adaptive chemoradiation in patients with LA-NSCLC. FLARE-RT trial: NCT02773238.

Deep learning for patient-specific quality assurance: Identifying errors in radiotherapy delivery by radiomic analysis of gamma images with convolutional neural networks.

PURPOSE: Patient-specific quality assurance (QA) for intensity-modulated radiation therapy (IMRT) is a ubiquitous clinical procedure, but conventional methods have often been criticized as being insensitive to errors or less effective than other common physics checks. Recently, there has been interest in the application of radiomics, quantitative extraction of image features, to radiotherapy QA. In this work, we investigate a deep learning approach to classify the presence or absence of introduced radiotherapy treatment delivery errors from patient-specific QA. METHODS: Planar dose maps from 186 IMRT beams from 23 IMRT plans were evaluated. Each plan was transferred to a cylindrical phantom CT geometry. Three sets of planar doses were exported from each plan corresponding to (a) the error-free case, (b) a random multileaf collimator (MLC) error case, and (c) a systematic MLC error case. Each plan was delivered to the electronic portal imaging device (EPID), and planned and measured doses were used to calculate gamma images in an EPID dosimetry software package (for a total of 558 gamma images). Two radiomic approaches were used. In the first, a convolutional neural network with triplet learning was used to extract image features from the gamma images. In the second, a handcrafted approach using texture features was used. The resulting metrics from both approaches were input into four machine learning classifiers (support vector machines, multilayer perceptrons, decision trees, and k-nearest-neighbors) in order to determine whether images contained the introduced errors. Two experiments were considered: the two-class experiment classified images as error-free or containing any MLC error, and the three-class experiment classified images as error-free, containing a random MLC error, or containing a systematic MLC error. Additionally, threshold-based passing criteria were calculated for comparison. RESULTS: In total, 303 gamma images were used for model training and 255 images were used for model testing. The highest classification accuracy was achieved with the deep learning approach, with a maximum accuracy of 77.3% in the two-class experiment and 64.3% in the three-class experiment. The performance of the handcrafted approach with texture features was lower, with a maximum accuracy of 66.3% in the two-class experiment and 53.7% in the three-class experiment. Variability between the results of the four machine learning classifiers was lower for the deep learning approach vs the texture feature approach. Both radiomic approaches were superior to threshold-based passing criteria. CONCLUSIONS: Deep learning with convolutional neural networks can be used to classify the presence or absence of introduced radiotherapy treatment delivery errors from patient-specific gamma images. The performance of the deep learning network was superior to a handcrafted approach with texture features, and both radiomic approaches were better than threshold-based passing criteria. The results suggest that radiomic QA is a promising direction for clinical radiotherapy.

Voxel Forecast for Precision Oncology: Predicting Spatially Variant and Multiscale Cancer Therapy Response on Longitudinal Quantitative Molecular Imaging.

PURPOSE: Prediction of spatially variant response to cancer therapies can inform risk-adaptive management within precision oncology. We developed the "Voxel Forecast" multiscale regression framework for predicting spatially variant tumor response to chemoradiotherapy on fluorodeoxyglucose (FDG) positron emission tomography/computed tomography (PET/CT) imaging. EXPERIMENTAL DESIGN: Twenty-five patients with locally advanced non-small cell lung cancer, enrolled on the FLARE-RT phase II trial (NCT02773238), underwent FDG PET/CT imaging prior to (PETpre) and during week 3 (PETmid) of concurrent chemoradiotherapy. Voxel Forecast was designed to predict tumor voxel standardized uptake value (SUV) on PETmid from baseline patient-level and voxel-level covariates using a custom generalized least squares (GLS) algorithm. Matérn covariance matrices were fit to patient- specific empirical variograms of distance-dependent intervoxel correlation. Regression coefficients from variogram-based weights and corresponding standard errors were estimated using the jackknife technique. The framework was validated using statistical simulations of known spatially variant tumor response. Mean absolute prediction errors (MAEs) of Voxel Forecast models were calculated under leave-one-patient-out cross-validation. RESULTS: Patient-level forecasts resulted in tumor voxel SUV MAE on PETmid of 1.5 g/mL while combined patient- and voxel-level forecasts achieved lower MAE of 1.0 g/mL (P < 0.0001). PETpre voxel SUV was the most important predictor of PETmid voxel SUV. Patients with a greater percentage of under-responding tumor voxels were classified as PETmid nonresponders (P = 0.030) with worse overall survival prognosis (P < 0.001). CONCLUSIONS: Voxel Forecast multiscale regression provides a statistical framework to predict voxel-wise response patterns during therapy. Voxel Forecast can be extended to predict spatially variant response on multimodal quantitative imaging and may eventually guide optimized spatial-temporal dose distributions for precision cancer therapy.