Best imaging signs identified by radiomics could outperform the model: application to differentiating lung carcinoid tumors from atypical hamartomas.

OBJECTIVES: Lung carcinoids and atypical hamartomas may be difficult to differentiate but require different treatment. The aim was to differentiate these tumors using contrast-enhanced CT semantic and radiomics criteria. METHODS: Between November 2009 and June 2020, consecutives patient operated for hamartomas or carcinoids with contrast-enhanced chest-CT were retrospectively reviewed. Semantic criteria were recorded and radiomics features were extracted from 3D segmentations using Pyradiomics. Reproducible and non-redundant radiomics features were used to training a random forest algorithm with cross-validation. A validation-set from another institution was used to evaluate of the radiomics signature, the 3D 'median' attenuation feature (3D-median) alone and the mean value from 2D-ROIs. RESULTS: Seventy-three patients (median 58 years [43‒70]) were analyzed (16 hamartomas; 57 carcinoids). The radiomics signature predicted hamartomas vs carcinoids on the external dataset (22 hamartomas; 32 carcinoids) with an AUC = 0.76. The 3D-median was the most important in the model. Density thresholds < 10 HU to predict hamartoma and > 60 HU to predict carcinoids were chosen for their high specificity > 0.90. On the external dataset, sensitivity and specificity of the 3D-median and 2D-ROIs were, respectively, 0.23, 1.00 and 0.13, 1.00 < 10 HU; 0.63, 0.95 and 0.69, 0.91 > 60 HU. The 3D-median was more reproducible than 2D-ROIs (ICC = 0.97 95% CI [0.95‒0.99]; bias: 3 ± 7 HU limits of agreement (LoA) [- 10‒16] vs. ICC = 0.90 95% CI [0.85‒0.94]; bias: - 0.7 ± 21 HU LoA [- 4‒40], respectively). CONCLUSIONS: A radiomics signature can distinguish hamartomas from carcinoids with an AUC = 0.76. Median density < 10 HU and > 60 HU on 3D or 2D-ROIs may be useful in clinical practice to diagnose these tumors with confidence, but 3D is more reproducible. CRITICAL RELEVANCE STATEMENT: Radiomic features help to identify the most discriminating imaging signs using random forest. 'Median' attenuation value (Hounsfield units), extracted from 3D-segmentations on contrast-enhanced chest-CTs, could distinguish carcinoids from atypical hamartomas (AUC = 0.85), was reproducible (ICC = 0.97), and generalized to an external dataset. KEY POINTS: • 3D-'Median' was the best feature to differentiate carcinoids from atypical hamartomas (AUC = 0.85). • 3D-'Median' feature is reproducible (ICC = 0.97) and was generalized to an external dataset. • Radiomics signature from 3D-segmentations differentiated carcinoids from atypical hamartomas with an AUC = 0.76. • 2D-ROI value reached similar performance to 3D-'median' but was less reproducible (ICC = 0.90).

Comparative performances of machine learning algorithms in radiomics and impacting factors.

There are no current recommendations on which machine learning (ML) algorithms should be used in radiomics. The objective was to compare performances of ML algorithms in radiomics when applied to different clinical questions to determine whether some strategies could give the best and most stable performances regardless of datasets. This study compares the performances of nine feature selection algorithms combined with fourteen binary classification algorithms on ten datasets. These datasets included radiomics features and clinical diagnosis for binary clinical classifications including COVID-19 pneumonia or sarcopenia on CT, head and neck, orbital or uterine lesions on MRI. For each dataset, a train-test split was created. Each of the 126 (9 × 14) combinations of feature selection algorithms and classification algorithms was trained and tuned using a ten-fold cross validation, then AUC was computed. This procedure was repeated three times per dataset. Best overall performances were obtained with JMI and JMIM as feature selection algorithms and random forest and linear regression models as classification algorithms. The choice of the classification algorithm was the factor explaining most of the performance variation (10% of total variance). The choice of the feature selection algorithm explained only 2% of variation, while the train-test split explained 9%.