Observer variability for Lung-RADS categorisation of lung cancer screening CTs: impact on patient management.

OBJECTIVES: Lung-RADS represents a categorical system published by the American College of Radiology to standardise management in lung cancer screening. The purpose of the study was to quantify how well readers agree in assigning Lung-RADS categories to screening CTs; secondary goals were to assess causes of disagreement and evaluate its impact on patient management. METHODS: For the observer study, 80 baseline and 80 follow-up scans were randomly selected from the NLST trial covering all Lung-RADS categories in an equal distribution. Agreement of seven observers was analysed using Cohen's kappa statistics. Discrepancies were correlated with patient management, test performance and diagnosis of malignancy within the scan year. RESULTS: Pairwise interobserver agreement was substantial (mean kappa 0.67, 95% CI 0.58-0.77). Lung-RADS category disagreement was seen in approximately one-third (29%, 971) of 3360 reading pairs, resulting in different patient management in 8% (278/3360). Out of the 91 reading pairs that referred to scans with a tumour diagnosis within 1 year, discrepancies in only two would have resulted in a substantial management change. CONCLUSIONS: Assignment of lung cancer screening CT scans to Lung-RADS categories achieves substantial interobserver agreement. Impact of disagreement on categorisation of malignant nodules was low. KEY POINTS: • Lung-RADS categorisation of low-dose lung screening CTs achieved substantial interobserver agreement. • Major cause for disagreement was assigning a different nodule as risk-dominant. • Disagreement led to a different follow-up time in 8% of reading pairs.

Malignancy risk estimation of pulmonary nodules in screening CTs: Comparison between a computer model and human observers.

PURPOSE: To compare human observers to a mathematically derived computer model for differentiation between malignant and benign pulmonary nodules detected on baseline screening computed tomography (CT) scans. METHODS: A case-cohort study design was chosen. The study group consisted of 300 chest CT scans from the Danish Lung Cancer Screening Trial (DLCST). It included all scans with proven malignancies (n = 62) and two subsets of randomly selected baseline scans with benign nodules of all sizes (n = 120) and matched in size to the cancers, respectively (n = 118). Eleven observers and the computer model (PanCan) assigned a malignancy probability score to each nodule. Performances were expressed by area under the ROC curve (AUC). Performance differences were tested using the Dorfman, Berbaum and Metz method. Seven observers assessed morphological nodule characteristics using a predefined list. Differences in morphological features between malignant and size-matched benign nodules were analyzed using chi-square analysis with Bonferroni correction. A significant difference was defined at p < 0.004. RESULTS: Performances of the model and observers were equivalent (AUC 0.932 versus 0.910, p = 0.184) for risk-assessment of malignant and benign nodules of all sizes. However, human readers performed superior to the computer model for differentiating malignant nodules from size-matched benign nodules (AUC 0.819 versus 0.706, p < 0.001). Large variations between observers were seen for ROC areas and ranges of risk scores. Morphological findings indicative of malignancy referred to border characteristics (spiculation, p < 0.001) and perinodular architectural deformation (distortion of surrounding lung parenchyma architecture, p < 0.001; pleural retraction, p = 0.002). CONCLUSIONS: Computer model and human observers perform equivalent for differentiating malignant from randomly selected benign nodules, confirming the high potential of computer models for nodule risk estimation in population based screening studies. However, computer models highly rely on size as discriminator. Incorporation of other morphological criteria used by human observers to superiorly discriminate size-matched malignant from benign nodules, will further improve computer performance.

Malignancy risk estimation of screen-detected nodules at baseline CT: comparison of the PanCan model, Lung-RADS and NCCN guidelines.

OBJECTIVES: To compare the PanCan model, Lung-RADS and the 1.2016 National Comprehensive Cancer Network (NCCN) guidelines for discriminating malignant from benign pulmonary nodules on baseline screening CT scans and the impact diameter measurement methods have on performances. METHODS: From the Danish Lung Cancer Screening Trial database, 64 CTs with malignant nodules and 549 baseline CTs with benign nodules were included. Performance of the systems was evaluated applying the system's original diameter definitions: Dlongest-C (PanCan), DmeanAxial (NCCN), both obtained from axial sections, and Dmean3D (Lung-RADS). Subsequently all diameter definitions were applied uniformly to all systems. Areas under the ROC curves (AUC) were used to evaluate risk discrimination. RESULTS: PanCan performed superiorly to Lung-RADS and NCCN (AUC 0.874 vs. 0.813, p = 0.003; 0.874 vs. 0.836, p = 0.010), using the original diameter specifications. When uniformly applying Dlongest-C, Dmean3D and DmeanAxial, PanCan remained superior to Lung-RADS (p < 0.001 - p = 0.001) and NCCN (p < 0.001 - p = 0.016). Diameter definition significantly influenced NCCN's performance with Dlongest-C being the worst (Dlongest-C vs. Dmean3D, p = 0.005; Dlongest-C vs. DmeanAxial, p = 0.016). CONCLUSIONS: Without follow-up information, the PanCan model performs significantly superiorly to Lung-RADS and the 1.2016 NCCN guidelines for discriminating benign from malignant nodules. The NCCN guidelines are most sensitive to nodule size definition. KEY POINTS: • PanCan model outperforms Lung-RADS and 1.2016 NCCN guidelines in identifying malignant pulmonary nodules. • Nodule size definition had no significant impact on Lung-RADS and PanCan model. • 1.2016 NCCN guidelines were significantly superior when using mean diameter to longest diameter. • Longest diameter achieved lowest performance for all models. • Mean diameter performed equivalently when derived from axial sections and from volumetry.

Predictive Accuracy of the PanCan Lung Cancer Risk Prediction Model -External Validation based on CT from the Danish Lung Cancer Screening Trial.

OBJECTIVES: Lung cancer risk models should be externally validated to test generalizability and clinical usefulness. The Danish Lung Cancer Screening Trial (DLCST) is a population-based prospective cohort study, used to assess the discriminative performances of the PanCan models. METHODS: From the DLCST database, 1,152 nodules from 718 participants were included. Parsimonious and full PanCan risk prediction models were applied to DLCST data, and also coefficients of the model were recalculated using DLCST data. Receiver operating characteristics (ROC) curves and area under the curve (AUC) were used to evaluate risk discrimination. RESULTS: AUCs of 0.826-0.870 were found for DLCST data based on PanCan risk prediction models. In the DLCST, age and family history were significant predictors (p = 0.001 and p = 0.013). Female sex was not confirmed to be associated with higher risk of lung cancer; in fact opposing effects of sex were observed in the two cohorts. Thus, female sex appeared to lower the risk (p = 0.047 and p = 0.040) in the DLCST. CONCLUSIONS: High risk discrimination was validated in the DLCST cohort, mainly determined by nodule size. Age and family history of lung cancer were significant predictors and could be included in the parsimonious model. Sex appears to be a less useful predictor. KEY POINTS: • High accuracy in logistic modelling for lung cancer risk stratification of nodules. • Lung cancer risk prediction is primarily based on size of pulmonary nodules. • Nodule spiculation, age and family history of lung cancer are significant predictors. • Sex does not appear to be a useful risk predictor.