Artificial Intelligence Tool for Assessment of Indeterminate Pulmonary Nodules Detected with CT.

Background Limited data are available regarding whether computer-aided diagnosis (CAD) improves assessment of malignancy risk in indeterminate pulmonary nodules (IPNs). Purpose To evaluate the effect of an artificial intelligence-based CAD tool on clinician IPN diagnostic performance and agreement for both malignancy risk categories and management recommendations. Materials and Methods This was a retrospective multireader multicase study performed in June and July 2020 on chest CT studies of IPNs. Readers used only CT imaging data and provided an estimate of malignancy risk and a management recommendation for each case without and with CAD. The effect of CAD on average reader diagnostic performance was assessed using the Obuchowski-Rockette and Dorfman-Berbaum-Metz method to calculate estimates of area under the receiver operating characteristic curve (AUC), sensitivity, and specificity. Multirater Fleiss κ statistics were used to measure interobserver agreement for malignancy risk and management recommendations. Results A total of 300 chest CT scans of IPNs with maximal diameters of 5-30 mm (50.0% malignant) were reviewed by 12 readers (six radiologists, six pulmonologists) (patient median age, 65 years; IQR, 59-71 years; 164 [55%] men). Readers' average AUC improved from 0.82 to 0.89 with CAD (P < .001). At malignancy risk thresholds of 5% and 65%, use of CAD improved average sensitivity from 94.1% to 97.9% (P = .01) and from 52.6% to 63.1% (P < .001), respectively. Average reader specificity improved from 37.4% to 42.3% (P = .03) and from 87.3% to 89.9% (P = .05), respectively. Reader interobserver agreement improved with CAD for both the less than 5% (Fleiss κ, 0.50 vs 0.71; P < .001) and more than 65% (Fleiss κ, 0.54 vs 0.71; P < .001) malignancy risk categories. Overall reader interobserver agreement for management recommendation categories (no action, CT surveillance, diagnostic procedure) also improved with CAD (Fleiss κ, 0.44 vs 0.52; P = .001). Conclusion Use of computer-aided diagnosis improved estimation of indeterminate pulmonary nodule malignancy risk on chest CT scans and improved interobserver agreement for both risk stratification and management recommendations. © RSNA, 2022 Online supplemental material is available for this article. See also the editorial by Yanagawa in this issue.

Lung cancer prediction by Deep Learning to identify benign lung nodules.

INTRODUCTION: Deep Learning has been proposed as promising tool to classify malignant nodules. Our aim was to retrospectively validate our Lung Cancer Prediction Convolutional Neural Network (LCP-CNN), which was trained on US screening data, on an independent dataset of indeterminate nodules in an European multicentre trial, to rule out benign nodules maintaining a high lung cancer sensitivity. METHODS: The LCP-CNN has been trained to generate a malignancy score for each nodule using CT data from the U.S. National Lung Screening Trial (NLST), and validated on CT scans containing 2106 nodules (205 lung cancers) detected in patients from from the Early Lung Cancer Diagnosis Using Artificial Intelligence and Big Data (LUCINDA) study, recruited from three tertiary referral centers in the UK, Germany and Netherlands. We pre-defined a benign nodule rule-out test, to identify benign nodules whilst maintaining a high sensitivity, by calculating thresholds on the malignancy score that achieve at least 99 % sensitivity on the NLST data. Overall performance per validation site was evaluated using Area-Under-the-ROC-Curve analysis (AUC). RESULTS: The overall AUC across the European centers was 94.5 % (95 %CI 92.6-96.1). With a high sensitivity of 99.0 %, malignancy could be ruled out in 22.1 % of the nodules, enabling 18.5 % of the patients to avoid follow-up scans. The two false-negative results both represented small typical carcinoids. CONCLUSION: The LCP-CNN, trained on participants with lung nodules from the US NLST dataset, showed excellent performance on identification of benign lung nodules in a multi-center external dataset, ruling out malignancy with high accuracy in about one fifth of the patients with 5-15 mm nodules.

The utility of a convolutional neural network (CNN) model score for cancer risk in indeterminate small solid pulmonary nodules, compared to clinical practice according to British Thoracic Society guidelines.

PURPOSE: To determine how implementation of an artificial intelligence nodule algorithm, the Lung Cancer Prediction Convolutional Neural Network (LCP-CNN), at the point of incidental nodule detection would have influenced further investigation and management using a series of threshold scores at both the benign and malignant end of the spectrum. METHOD: An observational retrospective study was performed in the assessment of nodules between 5-15 mm (158 benign, 32 malignant) detected on CT scans, which were performed as part of routine practice. The LCP-CNN was applied to the baseline CT scan producing a percentage score, and subsequent imaging and management determined for each threshold group. We hypothesized that the 5% low risk threshold group requires only one follow-up, the 0.56% very low risk threshold group requires no follow-up and the 80% high risk threshold group warrants expedited intervention. RESULTS: The 158 benign nodules had an LCP-CNN score between 0.1 and 70.8%, median 5.5% (IQR 1.4-18.0), whilst the 32 cancer nodules had an LCP-CNN score between 10.1 and 98.7%, median 59.0% (IQR 37.1-83.9). 24/61 CT scans in the 0.56-5% group (n = 37) and 21/21 CT scans <0.56% group (n = 13) could be obviated resulting in an overall reduction of 18.6% (45/242) CT scans in the benign cohort. In the 80% group (n = 10), expedited intervention of malignant nodules could result in a 3.6-month reduction in time delay in 5 cancer patients. CONCLUSION: We show the potential of artificial intelligence to reduce the need for follow-up scans and intervention in low-scoring benign nodules, whilst potentially accelerating the investigation and treatment of high-scoring cancer nodules.

Evaluation of a novel deep learning-based classifier for perifissural nodules.

OBJECTIVES: To evaluate the performance of a novel convolutional neural network (CNN) for the classification of typical perifissural nodules (PFN). METHODS: Chest CT data from two centers in the UK and The Netherlands (1668 unique nodules, 1260 individuals) were collected. Pulmonary nodules were classified into subtypes, including "typical PFNs" on-site, and were reviewed by a central clinician. The dataset was divided into a training/cross-validation set of 1557 nodules (1103 individuals) and a test set of 196 nodules (158 individuals). For the test set, three radiologically trained readers classified the nodules into three nodule categories: typical PFN, atypical PFN, and non-PFN. The consensus of the three readers was used as reference to evaluate the performance of the PFN-CNN. Typical PFNs were considered as positive results, and atypical PFNs and non-PFNs were grouped as negative results. PFN-CNN performance was evaluated using the ROC curve, confusion matrix, and Cohen's kappa. RESULTS: Internal validation yielded a mean AUC of 91.9% (95% CI 90.6-92.9) with 78.7% sensitivity and 90.4% specificity. For the test set, the reader consensus rated 45/196 (23%) of nodules as typical PFN. The classifier-reader agreement (k = 0.62-0.75) was similar to the inter-reader agreement (k = 0.64-0.79). Area under the ROC curve was 95.8% (95% CI 93.3-98.4), with a sensitivity of 95.6% (95% CI 84.9-99.5), and specificity of 88.1% (95% CI 81.8-92.8). CONCLUSION: The PFN-CNN showed excellent performance in classifying typical PFNs. Its agreement with radiologically trained readers is within the range of inter-reader agreement. Thus, the CNN-based system has potential in clinical and screening settings to rule out perifissural nodules and increase reader efficiency. KEY POINTS: • Agreement between the PFN-CNN and radiologically trained readers is within the range of inter-reader agreement. • The CNN model for the classification of typical PFNs achieved an AUC of 95.8% (95% CI 93.3-98.4) with 95.6% (95% CI 84.9-99.5) sensitivity and 88.1% (95% CI 81.8-92.8) specificity compared to the consensus of three readers.

Assessing the Accuracy of a Deep Learning Method to Risk Stratify Indeterminate Pulmonary Nodules.

Rationale: The management of indeterminate pulmonary nodules (IPNs) remains challenging, resulting in invasive procedures and delays in diagnosis and treatment. Strategies to decrease the rate of unnecessary invasive procedures and optimize surveillance regimens are needed.Objectives: To develop and validate a deep learning method to improve the management of IPNs.Methods: A Lung Cancer Prediction Convolutional Neural Network model was trained using computed tomography images of IPNs from the National Lung Screening Trial, internally validated, and externally tested on cohorts from two academic institutions.Measurements and Main Results: The areas under the receiver operating characteristic curve in the external validation cohorts were 83.5% (95% confidence interval [CI], 75.4-90.7%) and 91.9% (95% CI, 88.7-94.7%), compared with 78.1% (95% CI, 68.7-86.4%) and 81.9 (95% CI, 76.1-87.1%), respectively, for a commonly used clinical risk model for incidental nodules. Using 5% and 65% malignancy thresholds defining low- and high-risk categories, the overall net reclassifications in the validation cohorts for cancers and benign nodules compared with the Mayo model were 0.34 (Vanderbilt) and 0.30 (Oxford) as a rule-in test, and 0.33 (Vanderbilt) and 0.58 (Oxford) as a rule-out test. Compared with traditional risk prediction models, the Lung Cancer Prediction Convolutional Neural Network was associated with improved accuracy in predicting the likelihood of disease at each threshold of management and in our external validation cohorts.Conclusions: This study demonstrates that this deep learning algorithm can correctly reclassify IPNs into low- or high-risk categories in more than a third of cancers and benign nodules when compared with conventional risk models, potentially reducing the number of unnecessary invasive procedures and delays in diagnosis.

External validation of a convolutional neural network artificial intelligence tool to predict malignancy in pulmonary nodules.

BACKGROUND: Estimation of the risk of malignancy in pulmonary nodules detected by CT is central in clinical management. The use of artificial intelligence (AI) offers an opportunity to improve risk prediction. Here we compare the performance of an AI algorithm, the lung cancer prediction convolutional neural network (LCP-CNN), with that of the Brock University model, recommended in UK guidelines. METHODS: A dataset of incidentally detected pulmonary nodules measuring 5-15 mm was collected retrospectively from three UK hospitals for use in a validation study. Ground truth diagnosis for each nodule was based on histology (required for any cancer), resolution, stability or (for pulmonary lymph nodes only) expert opinion. There were 1397 nodules in 1187 patients, of which 234 nodules in 229 (19.3%) patients were cancer. Model discrimination and performance statistics at predefined score thresholds were compared between the Brock model and the LCP-CNN. RESULTS: The area under the curve for LCP-CNN was 89.6% (95% CI 87.6 to 91.5), compared with 86.8% (95% CI 84.3 to 89.1) for the Brock model (p≤0.005). Using the LCP-CNN, we found that 24.5% of nodules scored below the lowest cancer nodule score, compared with 10.9% using the Brock score. Using the predefined thresholds, we found that the LCP-CNN gave one false negative (0.4% of cancers), whereas the Brock model gave six (2.5%), while specificity statistics were similar between the two models. CONCLUSION: The LCP-CNN score has better discrimination and allows a larger proportion of benign nodules to be identified without missing cancers than the Brock model. This has the potential to substantially reduce the proportion of surveillance CT scans required and thus save significant resources.

Development and validation of clinical prediction models to risk stratify patients presenting with small pulmonary nodules: a research protocol.

INTRODUCTION: Lung cancer is a common cancer, with over 1.3 million cases worldwide each year. Early diagnosis using computed tomography (CT) screening has been shown to reduce mortality but also detect non-malignant nodules that require follow-up scanning or alternative methods of investigation. Practical and accurate tools that can predict the probability that a lung nodule is benign or malignant will help reduce costs and the risk of morbidity and mortality associated with lung cancer. METHODS: Retrospectively collected data from 1500 patients with pulmonary nodule(s) of up to 15 mm detected on routinely performed CT chest scans aged 18 years old or older from three academic centres in the UK will be used to to develop risk stratification models. Radiological, clinical and patient characteristics will be combined in multivariable logistic regression models to predict nodule malignancy. Data from over 1000 participants recruited in a prospective phase of the study will be used to evaluate model performance. Discrimination, calibration and clinical utility measures will be presented.

Comparison of view-based and reconstruction-based models of human navigational strategy.

There is good evidence that simple animals, such as bees, use view-based strategies to return to a familiar location, whereas humans might use a 3-D reconstruction to achieve the same goal. Assuming some noise in the storage and retrieval process, these two types of strategy give rise to different patterns of predicted errors in homing. We describe an experiment that can help distinguish between these models. Participants wore a head-mounted display to carry out a homing task in immersive virtual reality. They viewed three long, thin, vertical poles and had to remember where they were in relation to the poles before being transported (virtually) to a new location in the scene from where they had to walk back to the original location. The experiment was conducted in both a rich-cue scene (a furnished room) and a sparse scene (no background and no floor or ceiling). As one would expect, in a rich-cue environment, the overall error was smaller, and in this case, the ability to separate the models was reduced. However, for the sparse-cue environment, the view-based model outperforms the reconstruction-based model. Specifically, the likelihood of the experimental data is similar to the likelihood of samples drawn from the view-based model (but assessed under both models), and this is not true for samples drawn from the reconstruction-based model.

Importance of deformable image registration and biological dose summation in planning of radiotherapy retreatments.

The purpose of this study is to evaluate the effect of nonrigid and fractionation-corrected dose summation on total doses in radiotherapy and to demonstrate the benefits of such dose summation on clinical decision-making for planning of retreatments. Dose summation of organs at risk (OARs) was investigated for 3 clinical cases with need of retreatment to the same site: head and neck, brain, and mediastinum. Three different summation methods over old and new radiotherapy treatment plans are presented and compared: (1) rigid raw sum with rigid registration of the planning images and direct dose summing; (2) deformable raw sum with deformable image registration and direct dose summing; and (3) deformable biological sum with deformable registration and takes into account the dose per fraction in biological manner in certain critical organs. In 2 cases, a user-defined dose downscaling is applied to take into account the time between the treatments and the healing from the radiation-induced effects. Of the 3 summation methods presented, the deformable biological sum was considered to offer the most biologically plausible account of the treatment. There were remarkable differences between near-maximum doses (D0.1cc) and dose-volume histogram (DVH) curves for OARs between different summation methods. The differences between deformable raw sum and rigid raw sum D0.1cc doses are in the range from -8 Gy to 2 Gy. Similarly, the deviation was from -14 Gy to 5 Gy for the deformable biological sum compared with the rigid raw sum. These differences come from incorrect summation of doses in the rigid raw sum case, and from the dose per fraction effect in biological summation. We conclude that computing the 3-dimensional deformable biological summation could be a valuable tool for treating patients with complex retreatments. It has the potential to assist the oncologist in refining plans for maximally curative doses while respecting appropriate tissue tolerances.

Modelling human visual navigation using multi-view scene reconstruction.

It is often assumed that humans generate a 3D reconstruction of the environment, either in egocentric or world-based coordinates, but the steps involved are unknown. Here, we propose two reconstruction-based models, evaluated using data from two tasks in immersive virtual reality. We model the observer's prediction of landmark location based on standard photogrammetric methods and then combine location predictions to compute likelihood maps of navigation behaviour. In one model, each scene point is treated independently in the reconstruction; in the other, the pertinent variable is the spatial relationship between pairs of points. Participants viewed a simple environment from one location, were transported (virtually) to another part of the scene and were asked to navigate back. Error distributions varied substantially with changes in scene layout; we compared these directly with the likelihood maps to quantify the success of the models. We also measured error distributions when participants manipulated the location of a landmark to match the preceding interval, providing a direct test of the landmark-location stage of the navigation models. Models such as this, which start with scenes and end with a probabilistic prediction of behaviour, are likely to be increasingly useful for understanding 3D vision.