Development and external validation of multivariate prediction models for erectile dysfunction in men with localized prostate cancer.

While the 10-year survival rate for localized prostate cancer patients is very good (>98%), side effects of treatment may limit quality of life significantly. Erectile dysfunction (ED) is a common burden associated with increasing age as well as prostate cancer treatment. Although many studies have investigated the factors affecting erectile dysfunction (ED) after prostate cancer treatment, only limited studies have investigated whether ED can be predicted before the start of treatment. The advent of machine learning (ML) based prediction tools in oncology offers a promising approach to improve the accuracy of prediction and quality of care. Predicting ED may help aid shared decision-making by making the advantages and disadvantages of certain treatments clear, so that a tailored treatment for an individual patient can be chosen. This study aimed to predict ED at 1-year and 2-year post-diagnosis based on patient demographics, clinical data and patient-reported outcomes (PROMs) measured at diagnosis. We used a subset of the ProZIB dataset collected by the Netherlands Comprehensive Cancer Organization (Integraal Kankercentrum Nederland; IKNL) that contained information on 964 localized prostate cancer cases from 69 Dutch hospitals for model training and external validation. Two models were generated using a logistic regression algorithm coupled with Recursive Feature Elimination (RFE). The first predicted ED 1 year post-diagnosis and required 10 pre-treatment variables; the second predicted ED 2 years post-diagnosis with 9 pre-treatment variables. The validation AUCs were 0.84 and 0.81 for 1 year and 2 years post-diagnosis respectively. To immediately allow patients and clinicians to use these models in the clinical decision-making process, nomograms were generated. In conclusion, we successfully developed and validated two models that predicted ED in patients with localized prostate cancer. These models will allow physicians and patients alike to make informed evidence-based decisions about the most suitable treatment with quality of life in mind.

Automatic quality assurance of radiotherapy treatment plans using Bayesian networks: A multi-institutional study.

Purpose: Artificial intelligence applications in radiation oncology have been the focus of study in the last decade. The introduction of automated and intelligent solutions for routine clinical tasks, such as treatment planning and quality assurance, has the potential to increase safety and efficiency of radiotherapy. In this work, we present a multi-institutional study across three different institutions internationally on a Bayesian network (BN)-based initial plan review assistive tool that alerts radiotherapy professionals for potential erroneous or suboptimal treatment plans. Methods: Clinical data were collected from the oncology information systems in three institutes in Europe (Maastro clinic - 8753 patients treated between 2012 and 2020) and the United States of America (University of Vermont Medical Center [UVMMC] - 2733 patients, University of Washington [UW] - 6180 patients, treated between 2018 and 2021). We trained the BN model to detect potential errors in radiotherapy treatment plans using different combinations of institutional data and performed single-site and cross-site validation with simulated plans with embedded errors. The simulated errors consisted of three different categories: i) patient setup, ii) treatment planning and iii) prescription. We also compared the strategy of using only diagnostic parameters or all variables as evidence for the BN. We evaluated the model performance utilizing the area under the receiver-operating characteristic curve (AUC). Results: The best network performance was observed when the BN model is trained and validated using the dataset in the same center. In particular, the testing and validation using UVMMC data has achieved an AUC of 0.92 with all parameters used as evidence. In cross-validation studies, we observed that the BN model performed better when it was trained and validated in institutes with similar technology and treatment protocols (for instance, when testing on UVMMC data, the model trained on UW data achieved an AUC of 0.84, compared with an AUC of 0.64 for the model trained on Maastro data). Also, combining training data from larger clinics (UW and Maastro clinic) and using it on smaller clinics (UVMMC) leads to satisfactory performance with an AUC of 0.85. Lastly, we found that in general the BN model performed better when all variables are considered as evidence. Conclusion: We have developed and validated a Bayesian network model to assist initial treatment plan review using multi-institutional data with different technology and clinical practices. The model has shown good performance even when trained on data from clinics with divergent profiles, suggesting that the model is able to adapt to different data distributions.

External validation of a prediction model for timely implementation of innovations in radiotherapy.

BACKGROUND AND PURPOSE: The aim of this study was to externally validate a model that predicts timely innovation implementation, which can support radiotherapy professionals to be more successful in innovation implementation. MATERIALS AND METHODS: A multivariate prediction model was built based on the TRIPOD (Transparent Reporting of a multivariate prediction model for Individual Prognosis Or Diagnosis) criteria for a type 4 study (1). The previously built internally validated model had an AUC of 0.82, and was now validated using a completely new multicentre dataset. Innovation projects that took place between 2017-2019 were included in this study. Semi-structured interviews were performed to retrieve the prognostic variables of the previously built model. Projects were categorized according to the size of the project; the success of the project and thepresence of pre-defined success factors were analysed. RESULTS: Of the 80 included innovation projects (32.5% technological, 35% organisational and 32.5% treatment innovations), 55% were successfully implemented within the planned timeframe. Comparing the outcome predictions with the observed outcomes of all innovations resulted in an AUC of the external validation of the prediction model of 0.72 (0.60-0.84, 95% CI). Factors related to successful implementation included in the model are sufficient and competent employees, desirability and feasibility, clear goals and processes and the complexity of a project. CONCLUSION: For the first time, a prediction model focusing on the timely implementation of innovations has been successfully built and externally validated. This model can now be widely used to enable more successful innovation in radiotherapy.

Technical note: A knowledge graph approach to registering tumour specific data of patient-candidates for proton therapy in the Netherlands.

The registration of multi-source radiation oncology data is a time-consuming and labour-intensive procedure. The standardisation of data collection offers the possibility for the acquisition of quality data for research and clinical purposes. With this study, we present an overview of the different tumour group data lists in the Dutch national proton therapy registry. Furthermore, as a representative example of the workings of these different tumour-specific knowledge graphs, we present the FAIR (Findable, Accessible, Interoperable, Reusable) data principles-compliant knowledge graph approach describing the head and neck tumour variables using radiotherapy domain ontologies and semantic web technologies. Our goal is to provide the radiotherapy community with a flexible and interoperable data model for data exchange between centres. We highlight data variables that are needed for models used in the model-based approach (MBA), which ensures a fair selection of patients that will benefit most from proton therapy.

Independent validation of a dysphagia dose response model for the selection of head and neck cancer patients to proton therapy.

Background and purpose: The model based approach involves the use of normal tissue complication models for selection of head and neck cancer patients to proton therapy. Our goal was to validate the clinical utility of the related dysphagia model using an independent patient cohort. Materials and Methods: A dataset of 277 head and neck cancer (pharynx and larynx) patients treated with (chemo)radiotherapy between 2019 and 2021 was acquired. For the evaluation of the model discrimination we used statistical metrics such as the sensitivity, specificity and the area under the receiver operating characteristic curve. After the validation we evaluated if the dysphagia model can be improved using the closed testing procedure, the Brier and the Hosmer-Lemeshow score. Results: The performance of the original normal tissue complication probability model for dysphagia grade II-IV at 6 months was good (AUC = 0.80). According to the graphical calibration assessment, the original model showed underestimated dysphagia risk predictions. The closed testing procedure indicated that the model had to be updated and selected a revised model with new predictor coefficients as an optimal model. The revised model had also satisfactory discrimination (AUC = 0.83) with improved calibration. Conclusion: The validation of the normal tissue complication probability model for grade II-IV dysphagia was successful in our independent validation cohort. However, the closed testing procedure indicated that the model should be updated with new coefficients.

A knowledge graph representation of baseline characteristics for the Dutch proton therapy research registry.

Cancer registries collect multisource data and provide valuable information that can lead to unique research opportunities. In the Netherlands, a registry and model-based approach (MBA) are used for the selection of patients that are eligible for proton therapy. We collected baseline characteristics including demographic, clinical, tumour and treatment information. These data were transformed into a machine readable format using the FAIR (Findable, Accessible, Interoperable, Reusable) data principles and resulted in a knowledge graph with baseline characteristics of proton therapy patients. With this approach, we enable the possibility of linking external data sources and optimal flexibility to easily adapt the data structure of the existing knowledge graph to the needs of the clinic.

Making radiotherapy more efficient with FAIR data.

Given the rapid growth of artificial intelligence (AI) applications in radiotherapy and the related transformations toward the data-driven healthcare domain, this article summarizes the need and usage of the FAIR (Findable, Accessible, Interoperable, Reusable) data principles in radiotherapy. This work introduces the FAIR data concept, presents practical and relevant use cases and the future role of the different parties involved. The goal of this article is to provide guidance and potential applications of FAIR to various radiotherapy stakeholders, focusing on the central role of medical physicists.

Prediction of lymph node metastases using pre-treatment PET radiomics of the primary tumour in esophageal adenocarcinoma: an external validation study.

OBJECTIVES: To improve clinical lymph node staging (cN-stage) in oesophageal adenocarcinoma by developing and externally validating three prediction models; one with clinical variables only, one with positron emission tomography (PET) radiomics only, and a combined clinical and radiomics model. METHODS: Consecutive patients with fluorodeoxyglucose (FDG) avid tumours treated with neoadjuvant therapy between 2010 and 2016 in two international centres (n = 130 and n = 60, respectively) were included. Four clinical variables (age, gender, clinical T-stage and tumour regression grade) and PET radiomics from the primary tumour were used for model development. Diagnostic accuracy, area under curve (AUC), discrimination and calibration were calculated for each model. The prognostic significance was also assessed. RESULTS: The incidence of lymph node metastases was 58% in both cohorts. The areas under the curve of the clinical, radiomics and combined models were 0.79, 0.69 and 0.82 in the developmental cohort, and 0.65, 0.63 and 0.69 in the external validation cohort, with good calibration demonstrated. The area under the curve of current cN-stage in development and validation cohorts was 0.60 and 0.66, respectively. For overall survival, the combined clinical and radiomics model achieved the best discrimination performance in the external validation cohort (X2 = 6.08, df = 1, p = 0.01). CONCLUSION: Accurate diagnosis of lymph node metastases is crucial for prognosis and guiding treatment decisions. Despite finding improved predictive performance in the development cohort, the models using PET radiomics derived from the primary tumour were not fully replicated in an external validation cohort. ADVANCES IN KNOWLEDGE: This international study attempted to externally validate a new prediction model for lymph node metastases using PET radiomics. A model combining clinical variables and PET radiomics improved discrimination of lymph node metastases, but these results were not externally replicated.

External validation of nodal failure prediction models including radiomics in head and neck cancer.

PURPOSE: To externally validate the previously published pre-treatment prediction models for lymph nodes failure after definitive radiotherapy in head and neck squamous cell carcinoma (HNSCC) patients. MATERIALS AND METHODS: This external validation cohort consisted of 143 node positive HNSCC patients treated between July 2007 and June 2016 by curative radiotherapy with or without either cisplatin or cetuximab. Imaging and pathology reports during follow-up were analyzed to indicate persisting or recurring nodes. The previously established clinical, radiomic and combined models were validated on this cohort by assessing the concordance index (c-index) and model calibration. RESULTS: Overall 113 patients with 374 pLNs were suitable for final analysis. There were 20 (5.3%) nodal failures from 15 patients after a median follow-up of 36.1 months. Baseline characteristics and radiomic features were comparable to the training cohort. Both the radiomic model (Least-axis-length of lymph node (LALLN) and correlation of gray level co-occurrence matrix (Corre-GLCM)) and the combined model (T stage, gender, WHO performance score, LALLN and Corre-GLCM) showed good agreement between predicted and observed nodal control probabilities. The radiomic (c-index: 0.71; 95% confidence interval (CI): 0.59-0.84) and combined (c-index: 0.71; 95% CI: 0.59-0.82) models performed better than the clinical model (c-index: 0.57; 95% CI: 0.47-0.68) on this cohort, with a significant difference between the combined and clinical models (z-score test: p = 0.005). CONCLUSION: The combined model including clinical and radiomic features was externally validated and proved useful to predict nodal failures and could be helpful to guide treatment choices before and after curative radiation treatment for node positive HNSCC patients.

FAIR-compliant clinical, radiomics and DICOM metadata of RIDER, interobserver, Lung1 and head-Neck1 TCIA collections.

PURPOSE: One of the most frequently cited radiomics investigations showed that features automatically extracted from routine clinical images could be used in prognostic modeling. These images have been made publicly accessible via The Cancer Imaging Archive (TCIA). There have been numerous requests for additional explanatory metadata on the following datasets - RIDER, Interobserver, Lung1, and Head-Neck1. To support repeatability, reproducibility, generalizability, and transparency in radiomics research, we publish the subjects' clinical data, extracted radiomics features, and digital imaging and communications in medicine (DICOM) headers of these four datasets with descriptive metadata, in order to be more compliant with findable, accessible, interoperable, and reusable (FAIR) data management principles. ACQUISITION AND VALIDATION METHODS: Overall survival time intervals were updated using a national citizens registry after internal ethics board approval. Spatial offsets of the primary gross tumor volume (GTV) regions of interest (ROIs) associated with the Lung1 CT series were improved on the TCIA. GTV radiomics features were extracted using the open-source Ontology-Guided Radiomics Analysis Workflow (O-RAW). We reshaped the output of O-RAW to map features and extraction settings to the latest version of Radiomics Ontology, so as to be consistent with the Image Biomarker Standardization Initiative (IBSI). Digital imaging and communications in medicine metadata was extracted using a research version of Semantic DICOM (SOHARD, GmbH, Fuerth; Germany). Subjects' clinical data were described with metadata using the Radiation Oncology Ontology. All of the above were published in Resource Descriptor Format (RDF), that is, triples. Example SPARQL queries are shared with the reader to use on the online triples archive, which are intended to illustrate how to exploit this data submission. DATA FORMAT: The accumulated RDF data are publicly accessible through a SPARQL endpoint where the triples are archived. The endpoint is remotely queried through a graph database web application at http://sparql.cancerdata.org. SPARQL queries are intrinsically federated, such that we can efficiently cross-reference clinical, DICOM, and radiomics data within a single query, while being agnostic to the original data format and coding system. The federated queries work in the same way even if the RDF data were partitioned across multiple servers and dispersed physical locations. POTENTIAL APPLICATIONS: The public availability of these data resources is intended to support radiomics features replication, repeatability, and reproducibility studies by the academic community. The example SPARQL queries may be freely used and modified by readers depending on their research question. Data interoperability and reusability are supported by referencing existing public ontologies. The RDF data are readily findable and accessible through the aforementioned link. Scripts used to create the RDF are made available at a code repository linked to this submission: https://gitlab.com/UM-CDS/FAIR-compliant_clinical_radiomics_and_DICOM_metadata.

Distributed radiomics as a signature validation study using the Personal Health Train infrastructure.

Prediction modelling with radiomics is a rapidly developing research topic that requires access to vast amounts of imaging data. Methods that work on decentralized data are urgently needed, because of concerns about patient privacy. Previously published computed tomography medical image sets with gross tumour volume (GTV) outlines for non-small cell lung cancer have been updated with extended follow-up. In a previous study, these were referred to as Lung1 (n = 421) and Lung2 (n = 221). The Lung1 dataset is made publicly accessible via The Cancer Imaging Archive (TCIA; https://www.cancerimagingarchive.net ). We performed a decentralized multi-centre study to develop a radiomic signature (hereafter "ZS2019") in one institution and validated the performance in an independent institution, without the need for data exchange and compared this to an analysis where all data was centralized. The performance of ZS2019 for 2-year overall survival validated in distributed radiomics was not statistically different from the centralized validation (AUC 0.61 vs 0.61; p = 0.52). Although slightly different in terms of data and methods, no statistically significant difference in performance was observed between the new signature and previous work (c-index 0.58 vs 0.65; p = 0.37). Our objective was not the development of a new signature with the best performance, but to suggest an approach for distributed radiomics. Therefore, we used a similar method as an earlier study. We foresee that the Lung1 dataset can be further re-used for testing radiomic models and investigating feature reproducibility.

Learning from scanners: Bias reduction and feature correction in radiomics.

PURPOSE: Radiomics are quantitative features extracted from medical images. Many radiomic features depend not only on tumor properties, but also on non-tumor related factors such as scanner signal-to-noise ratio (SNR), reconstruction kernel and other image acquisition settings. This causes undesirable value variations in the features and reduces the performance of prediction models. In this paper, we investigate whether we can use phantom measurements to characterize and correct for the scanner SNR dependence. METHODS: We used a phantom with 17 regions of interest (ROI) to investigate the influence of different SNR values. CT scans were acquired with 9 different exposure settings. We developed an additive correction model to reduce scanner SNR influence. RESULTS: Sixty-two of 92 radiomic features showed high variance due to the scanner SNR. Of these 62 features, 47 showed at least a factor 2 significant standard deviation reduction by using the additive correction model. We assessed the clinical relevance of radiomics instability by using a 221 NSCLC patient cohort measured with the same scanner. CONCLUSIONS: Phantom measurements show that roughly two third of the radiomic features depend on the exposure setting of the scanner. The dependence can be modeled and corrected significantly reducing the variation in feature values with at least a factor of 2. More complex models will likely increase the correctability. Scanner SNR correction will result in more reliable radiomics predictions in NSCLC.

Stability of radiomic features of apparent diffusion coefficient (ADC) maps for locally advanced rectal cancer in response to image pre-processing.

Quantitative imaging features (radiomics) extracted from apparent diffusion coefficient (ADC) maps of rectal cancer patients can provide additional information to support treatment decision. Most available radiomic computational packages allow extraction of hundreds to thousands of features. However, two major factors can influence the reproducibility of radiomic features: interobserver variability, and imaging filtering applied prior to features extraction. In this exploratory study we seek to determine to what extent various commonly-used features are reproducible with regards to the mentioned factors using ADC maps from two different clinics (56 patients). Features derived from intensity distribution histograms are less sensitive to manual tumour delineation differences, noise in ADC images, pixel size resampling and intensity discretization. Shape features appear to be strongly affected by delineation quality. On the whole, textural features appear to be poorly or moderately reproducible with respect to the image pre-processing perturbations we reproduced.

Multicenter CT phantoms public dataset for radiomics reproducibility tests.

PURPOSE: The aim of this paper is to describe a public, open-access, computed tomography (CT) phantom image set acquired at three centers and collected especially for radiomics reproducibility research. The dataset is useful to test radiomic features reproducibility with respect to various parameters, such as acquisition settings, scanners, and reconstruction algorithms. ACQUISITION AND VALIDATION METHODS: Three phantoms were scanned in three independent institutions. Images of the following phantoms were acquired: Catphan 700 and COPDGene Phantom II (Phantom Laboratory, Greenwich, NY, USA), and the Triple modality 3D Abdominal Phantom (CIRS, Norfolk, VA, USA). Data were collected at three Dutch medical centers: MAASTRO Clinic (Maastricht, NL), Radboud University Medical Center (Nijmegen, NL), and University Medical Center Groningen (Groningen, NL) with scanners from two different manufacturers Siemens Healthcare and Philips Healthcare. The following acquisition parameter were varied in the phantom scans: slice thickness, reconstruction kernels, and tube current. DATA FORMAT AND USAGE NOTES: We made the dataset publically available on the Dutch instance of "Extensible Neuroimaging Archive Toolkit-XNAT" (https://xnat.bmia.nl). The dataset is freely available and reusable with attribution (Creative Commons 3.0 license). POTENTIAL APPLICATIONS: Our goal was to provide a findable, open-access, annotated, and reusable CT phantom dataset for radiomics reproducibility studies. Reproducibility testing and harmonization are fundamental requirements for wide generalizability of radiomics-based clinical prediction models. It is highly desirable to include only reproducible features into models, to be more assured of external validity across hitherto unseen contexts. In this view, phantom data from different centers represent a valuable source of information to exclude CT radiomic features that may already be unstable with respect to simplified structures and tightly controlled scan settings. The intended extension of our shared dataset is to include other modalities and phantoms with more realistic lesion simulations.

External validation of a prognostic model incorporating quantitative PET image features in oesophageal cancer.

AIM: Enhanced prognostic models are required to improve risk stratification of patients with oesophageal cancer so treatment decisions can be optimised. The primary aim was to externally validate a published prognostic model incorporating PET image features. Transferability of the model was compared using only clinical variables. METHODS: This was a Transparent Reporting of a multivariate prediction model for Individual Prognosis Or Diagnosis (TRIPOD) type 3 study. The model was validated against patients treated with neoadjuvant chemoradiotherapy according to the Neoadjuvant chemoradiotherapy plus surgery versus surgery alone for oesophageal or junctional cancer (CROSS) trial regimen using pre- and post-harmonised image features. The Kaplan-Meier method with log-rank significance tests assessed risk strata discrimination. A Cox proportional hazards model assessed model calibration. Primary outcome was overall survival (OS). RESULTS: Between 2010 and 2015, 449 patients were included in the development (n = 302), internal validation (n = 101) and external validation (n = 46) cohorts. No statistically significant difference in OS between patient quartiles was demonstrated in prognostic models incorporating PET image features (X2 = 1.42, df = 3, p = 0.70) or exclusively clinical variables (age, disease stage and treatment; X2 = 1.19, df = 3, p = 0.75). The calibration slope ß of both models was not significantly different from unity (p = 0.29 and 0.29, respectively). Risk groups defined using only clinical variables suggested differences in OS, although these were not statistically significant (X2 = 0.71, df = 2, p = 0.70). CONCLUSION: The prognostic model did not enable significant discrimination between the validation risk groups, but a second model with exclusively clinical variables suggested some transferable prognostic ability. PET harmonisation did not significantly change the results of model validation.