Improving the Robustness of Diagnostic Accuracy Results By Asking Study Readers to Further Distinguish Subjects Who Appear to be Without the Condition Of Interest.

RATIONALE AND OBJECTIVES: In diagnostic accuracy studies, cases in which a reader does not see the condition of interest are often given the same score for ROC analysis (e.g. confidence score of 0%). However, many of these cases can be further distinguished and doing so may result in more robust ROC results. MATERIALS AND METHODS: We examined two recent, real-world studies which used different procedures to encourage readers to further distinguish subjects who appear to be without the condition of interest. For each study, we calculated the results under two conditions. In the "zeros distinguished" (ZD) condition, we incorporated the confidence scores collected to further distinguish the normal-looking subjects. In the "zeros not distinguished" (ZND) condition, we disregarded these scores and simply gave the unit of analysis a score of zero whenever the reader did not identify the condition of interest in that unit. We compared the two conditions on (1) coverage of the ROC space and (2) discrepancy between parametric and nonparametric results. RESULTS: Compared to the ZND condition, coverage of the ROC space was improved in the ZD condition for all ROC curves in both studies. In the first study, there was a significant reduction in the discrepancy between parametric and nonparametric results (median discrepancy in ZND vs ZD condition: 0.033 vs 0.011, p = 0.012). A similar reduction was not seen in the second study, though the discrepancies were very low in both conditions (0.003 vs 0.006, p = 0.313). CONCLUSION: Prompting readers to further distinguish cases in which they do not see the condition of interest may result in more robust ROC results, though further exploration of this topic is warranted.

Structure Correction for Robust Volume Segmentation in Presence of Tumors.

CNN based lung segmentation models in absence of diverse training dataset fail to segment lung volumes in presence of severe pathologies such as large masses, scars, and tumors. To rectify this problem, we propose a multi-stage algorithm for lung volume segmentation from CT scans. The algorithm uses a 3D CNN in the first stage to obtain a coarse segmentation of the left and right lungs. In the second stage, shape correction is performed on the segmentation mask using a 3D structure correction CNN. A novel data augmentation strategy is adopted to train a 3D CNN which helps in incorporating global shape prior. Finally, the shape corrected segmentation mask is up-sampled and refined using a parallel flood-fill operation. The proposed multi-stage algorithm is robust in the presence of large nodules/tumors and does not require labeled segmentation masks for entire pathological lung volume for training. Through extensive experiments conducted on publicly available datasets such as NSCLC, LUNA, and LOLA11 we demonstrate that the proposed approach improves the recall of large juxtapleural tumor voxels by at least 15% over state-of-the-art models without sacrificing segmentation accuracy in case of normal lungs. The proposed method also meets the requirement of CAD software by performing segmentation within 5 seconds which is significantly faster than present methods.

Prospective Pilot Evaluation of Radiologists and Computer-aided Pulmonary Nodule Detection on Ultra-low-Dose CT With Tin Filtration.

PURPOSE: The aim of this study was to evaluate the ability of computer-aided detection (CAD) and human readers to detect pulmonary nodules ≥5 mm using 100 kV ultra-low-dose computed tomography (ULDCT) utilizing a tin filter. MATERIALS AND METHODS: After informed consent, 55 patients prospectively underwent standard-dose chest CT (SDCT) using 120 kV followed by ULDCT using 100 kV/tin. Reference nodules ≥5 mm were identified by a thoracic radiologist using SDCT. Four thoracic radiologists marked detected nodules on SDCT and ULDCT examinations using a dedicated computer workstation. After a 6-month memory extinction, readers were shown the same ULDCT cases with all CAD markings as well as their original detections, and characterized CAD detections as true positive or false positive. RESULTS: Volume CT Dose index (CTDIvol) for SDCT and ULDCT were 5.3±2 and 0.4±0.2 mGy (P<0.0001), respectively. Forty-five reference nodules were detected in 30 patients. Reader sensitivity varied widely but similarly for SDCT (ranging from 45% to 87%) and ULDCT (45% to 83%). CAD sensitivity was 76% (34/45) for SDCT and 71% (32/45) for ULDCT. After CAD, reader sensitivity substantially improved by 19% and 18% for 2 readers, and remained nearly unchanged for the other 2 readers (0% and 2%), despite reader perception that many more nodules were identified with CAD. There was a mean of 2 false-positive CAD detections/case. CONCLUSIONS: ULDCT with 100 kV/tin reduced patient dose by over 90% without compromising pulmonary nodule detection sensitivity. CAD can substantially improve nodule detection sensitivity at ULDCT for some readers, maintaining interobserver performance.

CT colonography: effect of computer-aided detection of colonic polyps as a second and concurrent reader for general radiologists with moderate experience in CT colonography.

OBJECTIVES: To assess the effectiveness of computer-aided detection (CAD) as a second reader or concurrent reader in helping radiologists who are moderately experienced in computed tomographic colonography (CTC) to detect colorectal polyps. METHODS: Seventy CTC datasets (34 patients: 66 polyps ≥6 mm; 36 patients: no abnormalities) were retrospectively reviewed by seven radiologists with moderate CTC experience. After primary unassisted evaluation, a CAD second read and, after a time interval of ≥4 weeks, a CAD concurrent read were performed. Areas under the receiver operating characteristic (ROC) curve (AUC), along with per-segment, per-polyp and per-patient sensitivities, and also reading times, were calculated for each reader with and without CAD. RESULTS: Of seven readers, 86% and 71% achieved a higher accuracy (segment-level AUC) when using CAD as second and concurrent reader respectively. Average segment-level AUCs with second and concurrent CAD (0.853 and 0.864) were significantly greater (p < 0.0001) than average AUC in the unaided evaluation (0.781). Per-segment, per-polyp, and per-patient sensitivities for polyps ≥6 mm were significantly higher in both CAD reading paradigms compared with unaided evaluation. Second-read CAD reduced readers' average segment and patient specificity by 0.007 and 0.036 (p = 0.005 and 0.011), respectively. CONCLUSIONS: CAD significantly improves the sensitivities of radiologists moderately experienced in CTC for polyp detection, both as second reader and concurrent reader. KEY POINTS: • CAD helps radiologists with moderate CTC experience to detect polyps ≥6 mm. • Second and concurrent read CAD increase the radiologist's sensitivity for detecting polyps ≥6 mm. • Second read CAD slightly decreases specificity compared with an unassisted read. • Concurrent read CAD is significantly more time-efficient than second read CAD.

Benefit of computer-aided detection analysis for the detection of subsolid and solid lung nodules on thin- and thick-section CT.

OBJECTIVE: The objective of our study was to evaluate the impact of computer-aided detection (CAD) on the identification of subsolid and solid lung nodules on thin- and thick-section CT. MATERIALS AND METHODS: For 46 chest CT examinations with ground-glass opacity (GGO) nodules, CAD marks computed using thin data were evaluated in two phases. First, four chest radiologists reviewed thin sections (reader(thin)) for nodules and subsequently CAD marks (reader(thin) + CAD(thin)). After 4 months, the same cases were reviewed on thick sections (reader(thick)) and subsequently with CAD marks (reader(thick) + CAD(thick)). Sensitivities were evaluated. Additionally, reader(thick) sensitivity with assessment of CAD marks on thin sections was estimated (reader(thick) + CAD(thin)). RESULTS: For 155 nodules (mean, 5.5 mm; range, 4.0-27.5 mm)-74 solid nodules, 22 part-solid (part-solid nodules), and 59 GGO nodules-CAD stand-alone sensitivity was 80%, 95%, and 71%, respectively, with three false-positives on average (0-12) per CT study. Reader(thin) + CAD(thin) sensitivities were higher than reader(thin) for solid nodules (82% vs 57%, p < 0.001), part-solid nodules (97% vs 81%, p = 0.0027), and GGO nodules (82% vs 69%, p < 0.001) for all readers (p < 0.001). Respective sensitivities for reader(thick), reader(thick) + CAD(thick), reader(thick) + CAD(thin) were 40%, 58% (p < 0.001), and 77% (p < 0.001) for solid nodules; 72%, 73% (p = 0.322), and 94% (p < 0.001) for part-solid nodules; and 53%, 58% (p = 0.008), and 79% (p < 0.001) for GGO nodules. For reader(thin), false-positives increased from 0.64 per case to 0.90 with CAD(thin) (p < 0.001) but not for reader(thick); false-positive rates were 1.17, 1.19, and 1.26 per case for reader(thick), reader(thick) + CAD(thick), and reader(thick) + CAD(thin), respectively. CONCLUSION: Detection of GGO nodules and solid nodules is significantly improved with CAD. When interpretation is performed on thick sections, the benefit is greater when CAD marks are reviewed on thin rather than thick sections.

Time-efficient CT colonography interpretation using an advanced image-gallery-based, computer-aided "first-reader" workflow for the detection of colorectal adenomas.

OBJECTIVES: To assess the performance of an advanced "first-reader" workflow for computer-aided detection (CAD) of colorectal adenomas ≥ 6 mm at computed tomographic colonography (CTC) in a low-prevalence cohort. METHODS: A total of 616 colonoscopy-validated CTC patient-datasets were retrospectively reviewed by a radiologist using a "first-reader" CAD workflow. CAD detections were presented as galleries of six automatically generated two-dimensional (2D) and three-dimensional (3D) images together with interactive 3D target views and 2D multiplanar views of the complete dataset. Each patient-dataset was interpreted by initially using CAD image-galleries followed by a fast 2D review to address unprompted colonic areas. Per-patient, per-polyp, and per-adenoma sensitivities were calculated for lesions ≥ 6 mm. Statistical testing employed Fisher's exact and McNemar tests. RESULTS: In 91/616 patients, 131 polyps (92 adenomas, 39 non-adenomas) ≥ 6 mm and two cancers were identified by reference standard. Using the CAD gallery-based first-reader workflow, the radiologist detected all adenomas ≥ 10 mm (34/34) and cancers. Per-patient and polyp sensitivities for lesions ≥ 6 mm were 84.3 % (75/89), and 83.2 % (109/131), respectively, with 89.1 % (57/64) and 85.9 % (79/92) for adenomas. Overall specificity was 95.6 % (504/527). Mean interpretation time was 3.1 min per patient. CONCLUSIONS: A CAD algorithm, applied in an image-gallery-based first-reader workflow, can substantially decrease reading times while enabling accurate detection of colorectal adenomas in a low-prevalence population. KEY POINTS: Computer-aided detection (CAD) is increasingly used to help interpret CT colonography (CTC). An image-gallery first-reader CAD-workflow is feasible for detection of colorectal adenomas ≥ 6 mm. Image-gallery first-reader CAD yields per-patient sensitivity of 89.1 % and specificity of 95.6 %. The mean reading time for CTC was 3.1 min, making screening feasible. No large adenoma was missed by the radiologist who reviewed with CAD galleries.

Performance of computer-aided detection of pulmonary nodules in low-dose CT: comparison with double reading by nodule volume.

OBJECTIVE: To evaluate performance of computer-aided detection (CAD) beyond double reading for pulmonary nodules on low-dose computed tomography (CT) by nodule volume. METHODS: A total of 400 low-dose chest CT examinations were randomly selected from the NELSON lung cancer screening trial. CTs were evaluated by two independent readers and processed by CAD. A total of 1,667 findings marked by readers and/or CAD were evaluated by a consensus panel of expert chest radiologists. Performance was evaluated by calculating sensitivity of pulmonary nodule detection and number of false positives, by nodule characteristics and volume. RESULTS: According to the screening protocol, 90.9 % of the findings could be excluded from further evaluation, 49.2 % being small nodules (less than 50 mm(3)). Excluding small nodules reduced false-positive detections by CAD from 3.7 to 1.9 per examination. Of 151 findings that needed further evaluation, 33 (21.9 %) were detected by CAD only, one of them being diagnosed as lung cancer the following year. The sensitivity of nodule detection was 78.1 % for double reading and 96.7 % for CAD. A total of 69.7 % of nodules undetected by readers were attached nodules of which 78.3 % were vessel-attached. CONCLUSIONS: CAD is valuable in lung cancer screening to improve sensitivity of pulmonary nodule detection beyond double reading, at a low false-positive rate when excluding small nodules. KEY POINTS: • Computer-aided detection (CAD) has known advantages for computed tomography (CT). • Combined CAD/nodule size cut-off parameters assist CT lung cancer screening. • This combination improves the sensitivity of pulmonary nodule detection by CT. • It increases the positive predictive value for cancer detection.

Impact of a computer-aided detection (CAD) system integrated into a picture archiving and communication system (PACS) on reader sensitivity and efficiency for the detection of lung nodules in thoracic CT exams.

The objective of this study is to assess the impact on nodule detection and efficiency using a computer-aided detection (CAD) device seamlessly integrated into a commercially available picture archiving and communication system (PACS). Forty-eight consecutive low-dose thoracic computed tomography studies were retrospectively included from an ongoing multi-institutional screening study. CAD results were sent to PACS as a separate image series for each study. Five fellowship-trained thoracic radiologists interpreted each case first on contiguous 5 mm sections, then evaluated the CAD output series (with CAD marks on corresponding axial sections). The standard of reference was based on three-reader agreement with expert adjudication. The time to interpret CAD marking was automatically recorded. A total of 134 true-positive nodules, measuring 3 mm and larger were included in our study; with 85 ≥ 4 and 50 ≥ 5 mm in size. Readers detection improved significantly in each size category when using CAD, respectively, from 44 to 57 % for ≥3 mm, 48 to 61 % for ≥4 mm, and 44 to 60 % for ≥5 mm. CAD stand-alone sensitivity was 65, 68, and 66 % for nodules ≥3, ≥4, and ≥5 mm, respectively, with CAD significantly increasing the false positives for two readers only. The average time to interpret and annotate a CAD mark was 15.1 s, after localizing it in the original image series. The integration of CAD into PACS increases reader sensitivity with minimal impact on interpretation time and supports such implementation into daily clinical practice.

Improved efficiency of CT interpretation using an automated lung nodule matching program.

OBJECTIVE: The purpose of this study was to assess the impact of an automated program on improvement in lung nodule matching efficiency. MATERIALS AND METHODS: Four thoracic radiologists independently reviewed two serial chest CT examinations from each of 57 patients. Each radiologist performed timed manual lung nodule matching. After 6 weeks, all radiologists independently repeated the timed matching portion using an automated nodule matching program. The time required for manual and automated matching was compared. The impact of nodule size and number on matching efficiency was determined. RESULTS: An average of 325 (range, 244-413) noncalcified solid pulmonary nodules was identified. Nodule matching was significantly faster with the automated program irrespective of the interpreting radiologist (p < 0.0001 for each). The maximal time saved with automated matching was 11.4 minutes (mean, 2.3 ± 2.0 minutes). Matching was faster in 56 of 57 cases (98.2%) for three readers and in 46 of 57 cases (80.7%) for one reader. There were no differences among readers with respect to the mean time saved per matched nodule (p > 0.5). The automated program achieved 90%, 90%, 79%, and 92% accuracy for the four readers. The improvement in efficiency for a given patient using the automated technique was proportional to the number of matched nodules (p < 0.0001) and inversely proportional to nodule size (p < 0.05). CONCLUSION: Use of the automated lung nodule matching program significantly improves diagnostic efficiency. The time saved is proportionate to the number of nodules identified and inversely proportional to nodule size. Adoption of such a program should expedite CT examination interpretation and improve report turnaround time.

Computer-aided detection of colorectal polyps in CT colonography with and without fecal tagging: a stand-alone evaluation.

PURPOSE: To evaluate the stand-alone performance of a computer-aided detection (CAD) algorithm for colorectal polyps in a large heterogeneous CT colonography (CTC) database that included both tagged and untagged datasets. METHODS: Written, informed consent was waived for this institutional review board-approved, HIPAA-compliant retrospective study. CTC datasets from 2063 patients were assigned to training (n = 374) and testing (n = 1689). The test set consisted of 836 untagged and 853 tagged examinations not used for CAD training. Examinations were performed at 15 sites in the United States, Asia, and Europe, using 4- to 64-multidetector-row computed tomography and various acquisition parameters. CAD sensitivities were calculated on a per-patient and per-polyp basis for polyps measuring ≥6 mm. The reference standard was colonoscopy in 1588 (94%) and consensus interpretation by expert radiologists in 101 (6%) patients. Statistical testing employed χ, logistic regression, and Mann-Whitney U tests. RESULTS: In 383 of 1689 individuals, 564 polyps measuring ≥6 mm were identified by the reference standard (347 polyps: 6-9 mm and 217 polyps: ≥10 mm). Overall, CAD per-patient sensitivity was 89.6% (343/383), with 89.0% (187/210) for untagged and 90.2% (156/173) for tagged datasets (P = 0.72). Overall, per-polyp sensitivity was 86.9% (490/564), with 84.4% (270/320) for untagged and 90.2% (220/244) for tagged examinations (P = 068). The mean false-positive rate per patient was 5.14 (median, 4) in untagged and 4.67 (median, 4) in tagged patient datasets (P = 0.353). CONCLUSION: Stand-alone CAD can be applied to both tagged and untagged CTC studies without significant performance differences. Detection rates are comparable to human readers at a relatively low false-positive rate, making CAD a useful tool in clinical practice.

Computer-aided detection of pulmonary embolism at CT pulmonary angiography: can it improve performance of inexperienced readers?

PURPOSE: To evaluate the effect of a computer-aided detection (CAD) algorithm on the performance of novice readers for detection of pulmonary embolism (PE) at CT pulmonary angiography (CTPA). MATERIALS AND METHODS: We included CTPA examinations of 79 patients (50 female, 52 ± 18 years). Studies were evaluated by two independent inexperienced readers who marked all vessels containing PE. After 3 months all studies were reevaluated by the same two readers, this time aided by CAD prototype. A consensus read by three expert radiologists served as the reference standard. Statistical analysis used χ(2) and McNemar testing. RESULTS: Expert consensus revealed 119 PEs in 32 studies. For PE detection, the sensitivity of CAD alone was 78%. Inexperienced readers' initial interpretations had an average per-PE sensitivity of 50%, which improved to 71% (p < 0.001) with CAD as a second reader. False positives increased from 0.18 to 0.25 per study (p = 0.03). Per-study, the readers initially detected 27/32 positive studies (84%); with CAD this number increased to 29.5 studies (92%; p = 0.125). CONCLUSION: Our results suggest that CAD significantly improves the sensitivity of PE detection for inexperienced readers with a small but appreciable increase in the rate of false positives.

Sensitivity of CT colonography for nonpolypoid colorectal lesions interpreted by human readers and with computer-aided detection.

OBJECTIVE: The purpose of our study was to determine the sensitivity of CT colonography (CTC) interpreted by human readers and with computer-aided detection (CAD) for genuinely nonpolypoid colorectal lesions, defined as 2 mm or less in lesion height at colonoscopy. MATERIALS AND METHODS: A computerized database search for a 33-month period found 21 patients who had undergone both colonoscopy and CTC and who had a total of 23 genuinely nonpolypoid colorectal lesions: eight adenomas (9-30 mm in width), 10 stage Tis or T1 adenocarcinomas (10-25 mm), and five nonadenomatous lesions (8-20 mm). CTC was performed using a cathartic preparation and fecal tagging and was interpreted by experienced readers in a blinded manner using a primary 3D method and with CAD. RESULTS: The sensitivities of human readers for nonpolypoid adenomatous lesions (i.e., both adenomas and adenocarcinomas), adenocarcinomas, and nonadenomatous lesions were 66.7% (12/18), 90% (9/10), and 0% (0/5), respectively. Sensitivities were 55.6% (10/18), 90% (9/10), and 0% (0/5) for CAD. A 10-mm stage T1 adenocarcinoma was missed by a human reader on blinded review but was detected with CAD. Both human readers and CAD yielded significantly higher sensitivity for adenomatous lesions than for nonadenomatous lesions (p = 0.014 and 0.046, respectively) and for adenocarcinomas than for noncancerous lesions (p = 0.003 and 0.0001, respectively). CONCLUSION: CTC showed a high sensitivity for nonpolypoid stage Tis and T1 adenocarcinomas 10 mm or greater in width despite the limited overall sensitivity for nonpolypoid adenomatous lesions, when performed using cathartic preparation and fecal tagging.

Computer-aided detection of colorectal polyps: can it improve sensitivity of less-experienced readers? Preliminary findings.

PURPOSE: To determine whether computer-aided detection (CAD) applied to computed tomographic (CT) colonography can help improve sensitivity of polyp detection by less-experienced radiologist readers, with colonoscopy or consensus used as the reference standard. MATERIALS AND METHODS: The release of the CT colonographic studies was approved by the individual institutional review boards of each institution. Institutions from the United States were HIPAA compliant. Written informed consent was waived at all institutions. The CT colonographic studies in 30 patients from six institutions were collected; 24 images depicted at least one confirmed polyp 6 mm or larger (39 total polyps) and six depicted no polyps. By using an investigational software package, seven less-experienced readers from two institutions evaluated the CT colonographic images and marked or scored polyps by using a five-point scale before and after CAD. The time needed to interpret the CT colonographic findings without CAD and then to re-evaluate them with CAD was recorded. For each reader, the McNemar test, adjusted for clustered data, was used to compare sensitivities for readers without and with CAD; a Wilcoxon signed-rank test was used to analyze the number of false-positive results per patient. RESULTS: The average sensitivity of the seven readers for polyp detection was significantly improved with CAD-from 0.810 to 0.908 (P=.0152). The number of false-positive results per patient without and with CAD increased from 0.70 to 0.96 (95% confidence interval for the increase: -0.39, 0.91). The mean total time for the readings was 17 minutes 54 seconds; for interpretation of CT colonographic findings alone, the mean time was 14 minutes 16 seconds; and for review of CAD findings, the mean time was 3 minutes 38 seconds. CONCLUSION: Results of this feasibility study suggest that CAD for CT colonography significantly improves per-polyp detection for less-experienced readers.

Effect of computer-aided detection as a second reader in multidetector-row CT colonography.

Our purpose was to assess the effect of computer-aided detection (CAD) on lesion detection as a second reader in computed tomographic colonography, and to compare the influence of CAD on the performance of readers with different levels of expertise. Fifty-two CT colonography patient data-sets (37 patients: 55 endoscopically confirmed polyps > or =0.5 cm, seven cancers; 15 patients: no abnormalities) were retrospectively reviewed by four radiologists (two expert, two nonexpert). After primary data evaluation, a second reading augmented with findings of CAD (polyp-enhanced view, Siemens) was performed. Sensitivities and reading time were calculated for each reader without CAD and supported by CAD findings. The sensitivity of expert readers was 91% each, and of nonexpert readers, 76% and 75%, respectively, for polyp detection. CAD increased the sensitivity of expert readers to 96% (P = 0.25) and 93% (P = 1), and that of nonexpert readers to 91% (P = 0.008) and 95% (P = 0.001), respectively. All four readers diagnosed 100% of cancers, but CAD alone only 43%. CAD increased reading time by 2.1 min (mean). CAD as a second reader significantly improves sensitivity for polyp detection in a high disease prevalence population for nonexpert readers. CAD causes a modest increase in reading time. CAD is of limited value in the detection of cancer.

Symmetric curvature patterns for colonic polyp detection.

A novel approach for generating a set of features derived from properties of patterns of curvature is introduced as a part of a computer aided colonic polyp detection system. The resulting sensitivity was 84% with 4.8 false positives per volume on an independent test set of 72 patients (56 polyps). When used in conjunction with other features, it allowed the detection system to reach an overall sensitivity of 94% with a false positive rate of 4.3 per volume.

Hierarchical part-based detection of 3D flexible tubes: application to CT colonoscopy.

In this paper, we present a learning-based method for the detection and segmentation of 3D free-form tubular structures, such as the rectal tubes in CT colonoscopy. This method can be used to reduce the false alarms introduced by rectal tubes in current polyp detection algorithms. The method is hierarchical, detecting parts of the tube in increasing order of complexity, from tube cross sections and tube segments to the whole flexible tube. To increase the speed of the algorithm, candidate parts are generated using a voting strategy. The detected tube segments are combined into a flexible tube using a dynamic programming algorithm. Testing the algorithm on 210 unseen datasets resulted in a tube detection rate of 94.7% and 0.12 false alarms per volume. The method can be easily retrained to detect and segment other tubular 3D structures.