RadSigBench: a framework for benchmarking functional genomics signatures of cancer cell radiosensitivity.

ABSTRACT

Multiple transcriptomic predictors of tumour cell radiosensitivity (RS) have been proposed, but they have not been benchmarked against one another or to control models. To address this, we present RadSigBench, a comprehensive benchmarking framework for RS signatures. The approach compares candidate models to those developed from randomly resampled control signatures and from cellular processes integral to the radiation response. Robust evaluation of signature accuracy, both overall and for individual tissues, is performed. The NCI60 and Cancer Cell Line Encyclopaedia datasets are integrated into our workflow. Prediction of two measures of RS is assessed survival fraction after 2 Gy and mean inactivation dose. We apply the RadSigBench framework to seven prominent published signatures of radiation sensitivity and test for equivalence to control signatures. The mean out-of-sample R2 for the published models on test data was very poor at 0.01 (range -0.05 to 0.09) for Cancer Cell Line Encyclopedia and 0.00 (range -0.19 to 0.19) in the NCI60 data. The accuracy of both published and cellular process signatures investigated was equivalent to the resampled controls, suggesting that these signatures contain limited radiation-specific information. Enhanced modelling strategies are needed for effective prediction of intrinsic RS to inform clinical treatment regimes. We make recommendations for methodological improvements, for example the inclusion of perturbation data, multiomics, advanced machine learning and mechanistic modelling. Our validation framework provides for robust performance assessment of ongoing developments in intrinsic RS prediction.