Minimum perceivable size difference: how well can radiologists visually detect a change in lung nodule size from CT images?

OBJECTIVE: The purpose of this study was to determine how well radiologists could visually detect a change in lung nodule size on the basis of visual image perception alone. SUBJECTS AND METHODS: Under IRB approval, 109 standard chest CT image series were anonymized and exported from PACS. Nine hundred forty virtual lung nodule pairs (six baseline diameters, six relative volume differences, two nodule types-solid and ground glass-and 14 repeats) were digitally inserted into the chest CT image series (same location, different sizes between the pair). These digitally altered CT image pairs were shown to nine radiologists who were tasked to visually determine which image contained the larger nodule using a two-alternative forced-choice perception experimental design. These data were statistically analyzed using a generalized linear mixed effects model to determine how accurately the radiologists were able to correctly identify the larger nodule. RESULTS: Nominal baseline nodule diameter, relative volume difference, and nodule type were found to be statistically significant factors (p < 0.001) in influencing the radiologists' accuracy. For solid (ground-glass) nodules, the baseline diameter needed to be at least 6.3 mm (13.2 mm) to be able to visually detect a 25% change in volume with 95 ± 1.4% accuracy. Accuracy was lowest for the nodules with the smallest baseline diameters and smallest relative volume differences. Additionally, accuracy was lower for ground-glass nodules compared to solid nodules. CONCLUSIONS: Factors that impacted visual size assessment were baseline nodule diameter, relative volume difference, and solid versus non-solid nodule type, with larger and more solid lesions offering a more precise assessment of change. KEY POINTS: • For solid nodules, radiologists could visually detect a 25% change in volume with 95% accuracy for nodules having greater than 6.3-mm baseline diameter. • For ground-glass nodules, radiologists could visually detect a 25% change in volume with 95% accuracy for nodules having greater than 13.2-mm baseline diameter. • Accuracy in detecting a change in nodule size began to stabilize around 90-100% for nodules with larger baseline diameters (> 8 mm for solid nodules, > 12 mm for ground-glass nodules) and larger relative volume differences (>15% for solid nodules, > 25% for ground-glass nodules).

Influence of Lung Reconstruction Algorithms on Interstitial Lung Pattern Recognition on CT.

Despite current recommendations, there is no recent scientific study comparing the influence of CT reconstruction kernels on lung pattern recognition in interstitial lung disease (ILD).To evaluate the sensitivity of lung (i70) and soft (i30) CT kernel algorithms for the diagnosis of ILD patterns.We retrospectively extracted between 15-25 pattern annotations per case (1 annotation = 15 slices of 1 mm) from 23 subjects resulting in 408 annotation stacks per lung kernel and soft kernel reconstructions. Two subspecialized chest radiologists defined the ground truth in consensus. 4 residents, 2 fellows, and 2 general consultants in radiology with 3 to 13 years of experience in chest imaging performed a blinded readout. In order to account for data clustering, a generalized linear mixed model (GLMM) with random intercept for reader and nested for patient and image and a kernel/experience interaction term was used to analyze the results.The results of the GLMM indicated, that the odds of correct pattern recognition is 12 % lower with lung kernel compared to soft kernel; however, this was not statistically significant (OR 0.88; 95%-CI, 0.73-1.06; p = 0.187). Furthermore, the consultants' odds of correct pattern recognition was 78 % higher than the residents' odds, although this finding did not reach statistical significance either (OR 1.78; 95%-CI, 0.62-5.06; p = 0.283). There was no significant interaction between the two fixed terms kernel and experience. Intra-rater agreement between lung and soft kernel was substantial (κ = 0.63 ±â€Š0.19). The mean inter-rater agreement for lung/soft kernel was κ = 0.37 ±â€Š0.17/κ = 0.38 ±â€Š0.17.There is no significant difference between lung and soft kernel reconstructed CT images for the correct pattern recognition in ILD. There are non-significant trends indicating that the use of soft kernels and a higher level of experience lead to a higher probability of correct pattern identification. · There is no significant difference between lung and soft kernel reconstructed CT images for the correct pattern recognition in interstitial lung disease.. · There are even non-significant tendencies that the use of soft kernels lead to a higher probability of correct pattern identification.. · These results challenge the current recommendations and the routinely performed separate lung kernel reconstructions for lung parenchyma analysis.. CITATION FORMAT: · Klaus JB, Christodoulidis S, Peters AA et al. Influence of Lung Reconstruction Algorithms on Interstitial Lung Pattern Recognition on CT. Fortschr Röntgenstr 2023; 195: 47 - 54.

Performance of an AI based CAD system in solid lung nodule detection on chest phantom radiographs compared to radiology residents and fellow radiologists.

BACKGROUND: Despite the decreasing relevance of chest radiography in lung cancer screening, chest radiography is still frequently applied to assess for lung nodules. The aim of the current study was to determine the accuracy of a commercial AI based CAD system for the detection of artificial lung nodules on chest radiograph phantoms and compare the performance to radiologists in training. METHODS: Sixty-one anthropomorphic lung phantoms were equipped with 140 randomly deployed artificial lung nodules (5, 8, 10, 12 mm). A random generator chose nodule size and distribution before a two-plane chest X-ray (CXR) of each phantom was performed. Seven blinded radiologists in training (2 fellows, 5 residents) with 2 to 5 years of experience in chest imaging read the CXRs on a PACS-workstation independently. Results of the software were recorded separately. McNemar test was used to compare each radiologist's results to the AI-computer-aided-diagnostic (CAD) software in a per-nodule and a per-phantom approach and Fleiss-Kappa was applied for inter-rater and intra-observer agreements. RESULTS: Five out of seven readers showed a significantly higher accuracy than the AI algorithm. The pooled accuracies of the radiologists in a nodule-based and a phantom-based approach were 0.59 and 0.82 respectively, whereas the AI-CAD showed accuracies of 0.47 and 0.67, respectively. Radiologists' average sensitivity for 10 and 12 mm nodules was 0.80 and dropped to 0.66 for 8 mm (P=0.04) and 0.14 for 5 mm nodules (P<0.001). The radiologists and the algorithm both demonstrated a significant higher sensitivity for peripheral compared to central nodules (0.66 vs. 0.48; P=0.004 and 0.64 vs. 0.094; P=0.025, respectively). Inter-rater agreements were moderate among the radiologists and between radiologists and AI-CAD software (K'=0.58±0.13 and 0.51±0.1). Intra-observer agreement was calculated for two readers and was almost perfect for the phantom-based (K'=0.85±0.05; K'=0.80±0.02); and substantial to almost perfect for the nodule-based approach (K'=0.83±0.02; K'=0.78±0.02). CONCLUSIONS: The AI based CAD system as a primary reader acts inferior to radiologists regarding lung nodule detection in chest phantoms. Chest radiography has reasonable accuracy in lung nodule detection if read by a radiologist alone and may be further optimized by an AI based CAD system as a second reader.

A Deep-Learning Diagnostic Support System for the Detection of COVID-19 Using Chest Radiographs: A Multireader Validation Study.

MATERIALS AND METHODS: Five publicly available databases comprising normal CXR, confirmed COVID-19 pneumonia cases, and other pneumonias were used. After the harmonization of the data, the training set included 7966 normal cases, 5451 with other pneumonia, and 258 CXRs with COVID-19 pneumonia, whereas in the testing data set, each category was represented by 100 cases. Eleven blinded radiologists with various levels of expertise independently read the testing data set. The data were analyzed separately with the newly proposed artificial intelligence-based system and by consultant radiologists and residents, with respect to positive predictive value (PPV), sensitivity, and F-score (harmonic mean for PPV and sensitivity). The χ2 test was used to compare the sensitivity, specificity, accuracy, PPV, and F-scores of the readers and the system. RESULTS: The proposed system achieved higher overall diagnostic accuracy (94.3%) than the radiologists (61.4% ± 5.3%). The radiologists reached average sensitivities for normal CXR, other type of pneumonia, and COVID-19 pneumonia of 85.0% ± 12.8%, 60.1% ± 12.2%, and 53.2% ± 11.2%, respectively, which were significantly lower than the results achieved by the algorithm (98.0%, 88.0%, and 97.0%; P < 0.00032). The mean PPVs for all 11 radiologists for the 3 categories were 82.4%, 59.0%, and 59.0% for the healthy, other pneumonia, and COVID-19 pneumonia, respectively, resulting in an F-score of 65.5% ± 12.4%, which was significantly lower than the F-score of the algorithm (94.3% ± 2.0%, P < 0.00001). When other pneumonia and COVID-19 pneumonia cases were pooled, the proposed system reached an accuracy of 95.7% for any pathology and the radiologists, 88.8%. The overall accuracy of consultants did not vary significantly compared with residents (65.0% ± 5.8% vs 67.4% ± 4.2%); however, consultants detected significantly more COVID-19 pneumonia cases (P = 0.008) and less healthy cases (P < 0.00001). CONCLUSIONS: The system showed robust accuracy for COVID-19 pneumonia detection on CXR and surpassed radiologists at various training levels.