Developing an ensemble machine learning study: Insights from a multi-center proof-of-concept study.

BACKGROUND: To address the numerous unmeet clinical needs, in recent years several Machine Learning models applied to medical images and clinical data have been introduced and developed. Even when they achieve encouraging results, they lack evolutionary progression, thus perpetuating their status as autonomous entities. We postulated that different algorithms which have been proposed in the literature to address the same diagnostic task, can be aggregated to enhance classification performance. We suggested a proof of concept to define an ensemble approach useful for integrating different algorithms proposed to solve the same clinical task. METHODS: The proposed approach was developed starting from a public database consisting of radiomic features extracted from CT images relating to 535 patients suffering from lung cancer. Seven algorithms were trained independently by participants in the AI4MP working group on Artificial Intelligence of the Italian Association of Physics in Medicine to discriminate metastatic from non-metastatic patients. The classification scores generated by these algorithms are used to train SVM classifier. The Explainable Artificial Intelligence approach is applied to the final model. The ensemble model was validated following an 80-20 hold-out and leave-one-out scheme on the training set. RESULTS: Compared to individual algorithms, a more accurate result was achieved. On the independent test the ensemble model achieved an accuracy of 0.78, a F1-score of 0.57 and a log-loss of 0.49. Shapley values representing the contribution of each algorithm to the final classification result of the ensemble model were calculated. This information represents an added value for the end user useful for evaluating the appropriateness of the classification result on a particular case. It also allows us to evaluate on a global level which methodological approaches of the individual algorithms are likely to have the most impact. CONCLUSION: Our proposal represents an innovative approach useful for integrating different algorithms that populate the literature and which lays the foundations for future evaluations in broader application scenarios.

Small Renal Masses: Developing a Robust Radiomic Signature.

(1) Background and (2) Methods: In this retrospective, observational, monocentric study, we selected a cohort of eighty-five patients (age range 38-87 years old, 51 men), enrolled between January 2014 and December 2020, with a newly diagnosed renal mass smaller than 4 cm (SRM) that later underwent nephrectomy surgery (partial or total) or tumorectomy with an associated histopatological study of the lesion. The radiomic features (RFs) of eighty-five SRMs were extracted from abdominal CTs bought in the portal venous phase using three different CT scanners. Lesions were manually segmented by an abdominal radiologist. Image analysis was performed with the Pyradiomic library of 3D-Slicer. A total of 108 RFs were included for each volume. A machine learning model based on radiomic features was developed to distinguish between benign and malignant small renal masses. The pipeline included redundant RFs elimination, RFs standardization, dataset balancing, exclusion of non-reproducible RFs, feature selection (FS), model training, model tuning and validation of unseen data. (3) Results: The study population was composed of fifty-one RCCs and thirty-four benign lesions (twenty-five oncocytomas, seven lipid-poor angiomyolipomas and two renal leiomyomas). The final radiomic signature included 10 RFs. The average performance of the model on unseen data was 0.79 ± 0.12 for ROC-AUC, 0.73 ± 0.12 for accuracy, 0.78 ± 0.19 for sensitivity and 0.63 ± 0.15 for specificity. (4) Conclusions: Using a robust pipeline, we found that the developed RFs signature is capable of distinguishing RCCs from benign renal tumors.

Lung dual energy CT: Impact of different technological solutions on quantitative analysis.

PURPOSE: To evaluated the accuracy of spectral parameters quantification of four different CT scanners in dual energy examinations of the lung using a dedicated phantom. METHOD: Measurements were made with different technologies of the same vendor: one dual source CT scanner (DSCT), one TwinBeam (i.e. split filter) and two sequential acquisition single source scanners (SSCT). Angular separation of Calcium and Iodine signals were calculated from scatter plots of low-kVp versus high-kVp HUs. Electron density (ρe), effective atomic number (Zeff) and Iodine concentration (Iconc) were measured using Syngo.via software. Accuracy (A) of ρe, Zeff and Iconc was evaluated as the absolute percentage difference (D%) between reference values and measured ones, while precision (P) was evaluated as the variability σ obtained by repeating the measurement with different acquisition/reconstruction settings. RESULTS: Angular separation was significantly larger for DSCT (α = 9.7°) and for sequential SSCT (α = 9.9°) systems. TwinBeam was less performing in material separation (α = 5.0°). The lowest average A was observed for TwinBeam (Aρe = [4.7 ± 1.0], AZ = [9.1 ± 3.1], AIconc = [19.4 ± 4.4]), while the best average A was obtained for Flash (Aρe = [1.8 ± 0.4], AZ = [3.5 ± 0.7], AIconc = [7.3 ± 1.8]). TwinBeam presented inferior average P (Pρe = [0.6 ± 0.1], PZ = [1.1 ± 0.2], PIconc = [10.9 ± 4.9]), while other technologies demonstrate a comparable average. CONCLUSIONS: Different technologies performed material separation and spectral parameter quantification with different degrees of accuracy and precision. DSCT performed better while TwinBeam demonstrated not excellent performance. Iodine concentration measurements exhibited high variability due to low Iodine absolute content in lung nodules, thus limiting its clinical usefulness in pulmonary applications.

Notch-Jagged1 signaling and response to bevacizumab therapy in advanced colorectal cancer: A glance to radiomics or back to physiopathology?

Introduction: The Notch intracellular domain (NICD) and its ligands Jagged-1(Jag1), Delta-like ligand (DLL-3) and DLL4 play an important role in neoangiogenesis. Previous studies suggest a correlation between the tissue levels of NICD and response to therapy with bevacizumab in colorectal cancer (CRC). Another marker that may predict outcome in CRC is radiomics of liver metastases. The aim of this study was to investigate the expression of NICD and its ligands and the role of radiomics in the selection of treatment-naive metastatic CRC patients receiving bevacizumab. Methods: Immunohistochemistry (IHC) for NICD, Jag1 and E-cadherin was performed on the tissue microarrays (TMAs) of 111 patients with metastatic CRC treated with bevacizumab and chemotherapy. Both the intensity and the percentage of stained cells were evaluated. The absolute number of CD4+ and CD8+ lymphocytes was counted in three different high-power fields and the mean values obtained were used to determine the CD4/CD8 ratio. The positivity of tumor cells to DLL3 and DLL4 was studied. The microvascular density (MVD) was assessed in fifteen cases by counting the microvessels at 20x magnification and expressed as MVD score. Abdominal CT scans were retrieved and imported into a dedicated workstation for radiomic analysis. Manually drawn regions of interest (ROI) allowed the extraction of radiomic features (RFs) from the tumor. Results: A positive association was found between NICD and Jag1 expression (p < 0.001). Median PFS was significantly shorter in patients whose tumors expressed high NICD and Jag1 (6.43 months vs 11.53 months for negative cases; p = 0.001). Those with an MVD score ≥5 (CD31-high, NICD/Jag1 positive) experienced significantly poorer survival. The radiomic model developed to predict short and long-term survival and PFS yielded a ROC-AUC of 0.709; when integrated with clinical and histopathological data, the integrated model improved the predictive score (ROC-AUC of 0.823). Discussion: These results show that high NICD and Jag1 expression are associated with progressive disease and early disease progression to anti VEGF-based therapy; the preliminary radiomic analyses show that the integration of quantitative information with clinical and histological data display the highest performance in predicting the outcome of CRC patients.

MR-based simplified extraprostatic extension evaluation: comparison of performances of different predictive models.

OBJECTIVES: To test reproducibility and predictive value of a simplified score for assessment of extraprostatic tumor extension (sEPE grade). METHODS: Sixty-five patients (mean age ± SD, 67 years ± 6.3) treated with radical prostatectomy for prostate cancer who underwent 1.5-Tesla multiparametric magnetic resonance imaging (mpMRI) 6 months before surgery were enrolled. sEPE grade was derived from mpMRI metrics: curvilinear contact length > 15 mm (CCL) and capsular bulging/irregularity. The diameter of the index lesion (dIL) was also measured. Evaluations were independently performed by seven radiologists, and inter-reader agreement was tested by weighted Cohen K coefficient. A nested (two levels) Monte Carlo cross-validation was used. The best cut-off value for dIL was selected by means of the Youden J index to classify values into a binary variable termed dIL*. Logistic regression models based on sEPE grade, dIL, and clinical scores were developed to predict pathologic EPE. Results on validation set were assessed by the main metrics of the receiver operating characteristics curve (ROC) and by decision curve analysis (DCA). Based on our findings, we defined and tested an alternative sEPE grade formulation. RESULTS: Pathologic EPE was found in 31/65 (48%) patients. Average κw was 0.65 (95% CI 0.51-0.79), 0.66 (95% CI 0.48-0.84), 0.67 (95% CI 0.50-0.84), and 0.43 (95% CI 0.22-0.63) for sEPE grading, CLL ≥ 15 mm, dIL*, and capsular bulging/irregularity, respectively. The highest diagnostic yield in predicting EPE was obtained by combining both sEPE grade and dIL*(ROC-AUC 0.81). CONCLUSIONS: sEPE grade is reproducible and when combined with the dIL* accurately predicts extraprostatic tumor extension. KEY POINTS: • Simple and reproducible mpMRI semi-quantitative scoring system for extraprostatic tumor extension. • sEPE grade accurately predicts extraprostatic tumor extension regardless of reader expertise. • Accurate pre-operative staging and risk stratification for optimized patient management.

Integrated prognostication of intrahepatic cholangiocarcinoma by contrast-enhanced computed tomography: the adjunct yield of radiomics.

PURPOSE: To test radiomics for prognostication of intrahepatic mass-forming cholangiocarcinoma (IMCC) and to develop a comprehensive risk model. METHODS: Histologically proven IMCC (representing the full range of stages) were retrospectively analyzed by volume segmentation on baseline hepatic venous phase computed tomography (CT), by two readers with different experience (R1 and R2). Morphological CT features included: tumor size, hepatic satellite lesions, lymph node and distant metastases. Radiomic features (RF) were compared across CT protocols and readers. Univariate analysis against overall survival (OS) warranted ranking and selection of RF into radiomic signature (RSign), which was dichotomized into high and low-risk strata (RSign*). Models without and with RSign* (Model 1 and 2, respectively) were compared. RESULTS: Among 78 patients (median follow-up 262 days, IQR 73-957), 62/78 (79%) died during the study period, 46/78 (59%) died within 1 year. Up to 10% RF showed variability across CT protocols; 37/108 (34%) RF showed variability due to manual segmentation. RSign stratified OS (univariate: HR 1.37 for R1, HR 1.28 for R2), RSign* was different between readers (R1 0.39; R2 0.57). Model 1 showed AUC 0.71, which increased in Model 2: AUC 0.81 (p < 0.001) and AIC 89 for R1, AUC 0.81 (p = 0.001) and AIC 90.2 for R2. CONCLUSION: The use of RF into a unified RSign score stratified OS in patients with IMCC. Dichotomized RSign* classified survival strata, its inclusion in risk models showed adjunct yield. The cut-off value of RSign* was different between readers, suggesting that the use of reference values is hampered by interobserver variability.

Automatic feathering algorithm for VMAT craniospinal irradiation: A comprehensive comparison with other VMAT planning strategies.

In craniospinal irradiation, field matching is very sensitive to intrafraction positional uncertainties in cranio-caudal direction, which could lead to severe overdoses/underdoses inside the planning target volume. During the last decade, significant efforts were made to develop volumetric-modulated arc therapy strategies, which were less sensitive to setup uncertainties. In this study, a treatment planning system-integrated method, named automatic feathering (AF) algorithm, was compared against other volumetric-modulated arc therapy strategies. Three patients were retrospectively included. Five different planning techniques were compared, including overlap (O), staggered overlap (SO), gradient optimization (GO), overlap with AF algorithm turned on (O-AF), and staggered overlap with AF algorithm turned on (SO-AF). Three overlapping lengths were considered (5 cm, 7.5 cm, and 10 cm). The middle isocenter was shifted of ±1 mm, ±3 mm, and ±5 mm to simulate setup uncertainties. Plan robustness against simulated uncertainties was evaluated by calculating near maximum and near minimum dose differences between shifted and nonshifted plans (ΔD2%, ΔD98%). Dose differences among combinations of techniques and junction lengths were tested using Wilcoxon signed-rank test. Higher ΔD2% and ΔD98% were obtained using the overlap technique (ΔD2% = 15.4%, ΔD98% = 15.0%). O-AF and SO-AF provided comparable plan robustness to GO technique. Their performance improved significantly for grater overlapping length. For 10-cm overlap and 5-mm shift, GO, O-AF, and SO-AF yielded to the better plan robustness (5.7% < ΔD2% < 6.0%, 6.1% < ΔD98% < 7.6%). SO provided an intermediate plan robustness (9.8% < ΔD2% < 10.8%, 8.9% < ΔD98% < 10.3%). The addition of AF to the overlap technique significantly improves plan robustness especially if larger overlapping lengths are used. Using the AF algorithm, plans become as robust as plans optimized with more sophisticated and time-consuming approaches (like GO).

Computed tomography use in a large Italian region: trend analysis 2004-2014 of emergency and outpatient CT examinations in children and adults.

OBJECTIVES: To analyse CT use in recent years in a high-density Italian area (±10 million inhabitants, including 1 million children), focusing on developing age. METHODS: Retrospective analysis of records from HealthCare IT System, covering >400 hospitals and clinics. Description of CT use between 2004-2014 in emergency and outpatient care and assessment of radiation exposure trend. RESULTS: Over 9 million scans were performed. Emergency procedures showed a global increase of 230 %, mainly head examinations. In the global outpatient setting, the annual number of CT scans/person increased ±19 %. A moderate increase in CT examinations was observed in the developing age population, while a remarkable increase in dental, chest and abdominal procedures occurred for the 10- to 30-year age range. The increase in mean annual dose/capita in the global patient pool was approximately 42 %, increasing from 0.72-1.03 mSv. The population rate receiving an annual CT radiation dose/capita higher than 1 mSv tripled in the 11-year interval, increasing from 16-48 %. CONCLUSIONS: The remarkable increase in radiation exposure raises a special concern for teenagers and young adults, whose risk tends to be underestimated. The fivefold increase in dental CTs in the younger age groups requires further investigations. KEY POINTS: • Literature highlights a remarkable increase in CT use over the last decades. • The paediatric age had higher exposure to X-ray risk. • A detailed retrospective analysis of more than 9 million scans was performed. • Dental, chest, abdominal procedures increased remarkably in 10- to 30-year age range. • This study raises concern about exposure for teenagers and young adults.

Validation of a free software for unsupervised assessment of abdominal fat in MRI.

PURPOSE: To demonstrate the accuracy of an unsupervised (fully automated) software for fat segmentation in magnetic resonance imaging. The proposed software is a freeware solution developed in ImageJ that enables the quantification of metabolically different adipose tissues in large cohort studies. METHODS: The lumbar part of the abdomen (19cm in craniocaudal direction, centered in L3) of eleven healthy volunteers (age range: 21-46years, BMI range: 21.7-31.6kg/m2) was examined in a breath hold on expiration with a GE T1 Dixon sequence. Single-slice and volumetric data were considered for each subject. The results of the visceral and subcutaneous adipose tissue assessments obtained by the unsupervised software were compared to supervised segmentations of reference. The associated statistical analysis included Pearson correlations, Bland-Altman plots and volumetric differences (VD%). RESULTS: Values calculated by the unsupervised software significantly correlated with corresponding supervised segmentations of reference for both subcutaneous adipose tissue - SAT (R=0.9996, p<0.001) and visceral adipose tissue - VAT (R=0.995, p<0.001). Bland-Altman plots showed the absence of systematic errors and a limited spread of the differences. In the single-slice analysis, VD% were (1.6±2.9)% for SAT and (4.9±6.9)% for VAT. In the volumetric analysis, VD% were (1.3±0.9)% for SAT and (2.9±2.7)% for VAT. CONCLUSIONS: The developed software is capable of segmenting the metabolically different adipose tissues with a high degree of accuracy. This free add-on software for ImageJ can easily have a widespread and enable large-scale population studies regarding the adipose tissue and its related diseases.

High-resolution-cone beam tomography analysis of bone microarchitecture in patients with acromegaly and radiological vertebral fractures.

Vertebral fractures are an emerging complication of acromegaly but their prediction is still difficult occurring even in patients with normal bone mineral density. In this study we evaluated the ability of high-resolution cone-beam computed tomography to provide information on skeletal abnormalities associated with vertebral fractures in acromegaly. 40 patients (24 females, 16 males; median age 57 years, range 25-72) and 21 healthy volunteers (10 females, 11 males; median age 60 years, range: 25-68) were evaluated for trabecular (bone volume/trabecular volume ratio, mean trabecular separation, and mean trabecular thickness) and cortical (thickness and porosity) parameters at distal radius using a high-resolution cone-beam computed tomography system. All acromegaly patients were evaluated for morphometric vertebral fractures and for mineral bone density by dual-energy X-ray absorptiometry at lumbar spine, total hip, femoral neck, and distal radius. Acromegaly patients with vertebral fractures (15 cases) had significantly (p < 0.05) lower bone volume/trabecular volume ratio, greater mean trabecular separation, and higher cortical porosity vs. nonfractured patients, without statistically significant differences in mean trabecular thickness and cortical thickness. Fractured and nonfractured acromegaly patients did not have significant differences in bone density at either skeletal site. Patients with acromegaly showed lower bone volume/trabecular volume ratio (p = 0.003) and mean trabecular thickness (p < 0.001) and greater mean trabecular separation (p = 0.02) as compared to control subjects, without significant differences in cortical thickness and porosity. This study shows for the first time that abnormalities of bone microstructure are associated with radiological vertebral fractures in acromegaly. High-resolution cone-beam computed tomography at the distal radius may be useful to evaluate and predict the effects of acromegaly on bone microstructure.