Predicting stereotactic radiosurgery outcomes with multi-observer qualitative appearance labelling versus MRI radiomics.

Qualitative observer-based and quantitative radiomics-based analyses of T1w contrast-enhanced magnetic resonance imaging (T1w-CE MRI) have both been shown to predict the outcomes of brain metastasis (BM) stereotactic radiosurgery (SRS). Comparison of these methods and interpretation of radiomics-based machine learning (ML) models remains limited. To address this need, we collected a dataset of n = 123 BMs from 99 patients including 12 clinical features, 107 pre-treatment T1w-CE MRI radiomic features, and BM post-SRS progression scores. A previously published outcome model using SRS dose prescription and five-way BM qualitative appearance scoring was evaluated. We found high qualitative scoring interobserver variability across five observers that negatively impacted the model's risk stratification. Radiomics-based ML models trained to replicate the qualitative scoring did so with high accuracy (bootstrap-corrected AUC = 0.84-0.94), but risk stratification using these replicated qualitative scores remained poor. Radiomics-based ML models trained to directly predict post-SRS progression offered enhanced risk stratification (Kaplan-Meier rank-sum p = 0.0003) compared to using qualitative appearance. The qualitative appearance scoring enabled interpretation of the progression radiomics-based ML model, with necrotic BMs and a subset of heterogeneous BMs predicted as being at high-risk of post-SRS progression, in agreement with current radiobiological understanding. Our study's results show that while radiomics-based SRS outcome models out-perform qualitative appearance analysis, qualitative appearance still provides critical insight into ML model operation.

Dual-center validation of using magnetic resonance imaging radiomics to predict stereotactic radiosurgery outcomes.

Background: MRI radiomic features and machine learning have been used to predict brain metastasis (BM) stereotactic radiosurgery (SRS) outcomes. Previous studies used only single-center datasets, representing a significant barrier to clinical translation and further research. This study, therefore, presents the first dual-center validation of these techniques. Methods: SRS datasets were acquired from 2 centers (n = 123 BMs and n = 117 BMs). Each dataset contained 8 clinical features, 107 pretreatment T1w contrast-enhanced MRI radiomic features, and post-SRS BM progression endpoints determined from follow-up MRI. Random decision forest models were used with clinical and/or radiomic features to predict progression. 250 bootstrap repetitions were used for single-center experiments. Results: Training a model with one center's dataset and testing it with the other center's dataset required using a set of features important for outcome prediction at both centers, and achieved area under the receiver operating characteristic curve (AUC) values up to 0.70. A model training methodology developed using the first center's dataset was locked and externally validated with the second center's dataset, achieving a bootstrap-corrected AUC of 0.80. Lastly, models trained on pooled data from both centers offered balanced accuracy across centers with an overall bootstrap-corrected AUC of 0.78. Conclusions: Using the presented validated methodology, radiomic models trained at a single center can be used externally, though they must utilize features important across all centers. These models' accuracies are inferior to those of models trained using each individual center's data. Pooling data across centers shows accurate and balanced performance, though further validation is required.

Performance sensitivity analysis of brain metastasis stereotactic radiosurgery outcome prediction using MRI radiomics.

Recent studies have used T1w contrast-enhanced (T1w-CE) magnetic resonance imaging (MRI) radiomic features and machine learning to predict post-stereotactic radiosurgery (SRS) brain metastasis (BM) progression, but have not examined the effects of combining clinical and radiomic features, BM primary cancer, BM volume effects, and using multiple scanner models. To investigate these effects, a dataset of n = 123 BMs from 99 SRS patients with 12 clinical features, 107 pre-treatment T1w-CE radiomic features, and BM progression determined by follow-up MRI was used with a random decision forest model and 250 bootstrapped repetitions. Repeat experiments assessed the relative accuracy across primary cancer sites, BM volume groups, and scanner model pairings. Correction for accuracy imbalances across volume groups was investigated by removing volume-correlated features. We found that using clinical and radiomic features together produced the most accurate model with a bootstrap-corrected area under the receiver operating characteristic curve of 0.77. Accuracy also varied by primary cancer site, BM volume, and scanner model pairings. The effect of BM volume was eliminated by removing features at a volume-correlation coefficient threshold of 0.25. These results show that feature type, primary cancer, volume, and scanner model are all critical factors in the accuracy of radiomics-based prognostic models for BM SRS that must be characterised and controlled for before clinical translation.