Assessment of the Incremental Benefit of Computer-Aided Detection (CAD) for Interpretation of CT Colonography by Experienced and Inexperienced Readers.

OBJECTIVES: To quantify the incremental benefit of computer-assisted-detection (CAD) for polyps, for inexperienced readers versus experienced readers of CT colonography. METHODS: 10 inexperienced and 16 experienced radiologists interpreted 102 colonography studies unassisted and with CAD utilised in a concurrent paradigm. They indicated any polyps detected on a study sheet. Readers' interpretations were compared against a ground-truth reference standard: 46 studies were normal and 56 had at least one polyp (132 polyps in total). The primary study outcome was the difference in CAD net benefit (a combination of change in sensitivity and change in specificity with CAD, weighted towards sensitivity) for detection of patients with polyps. RESULTS: Inexperienced readers' per-patient sensitivity rose from 39.1% to 53.2% with CAD and specificity fell from 94.1% to 88.0%, both statistically significant. Experienced readers' sensitivity rose from 57.5% to 62.1% and specificity fell from 91.0% to 88.3%, both non-significant. Net benefit with CAD assistance was significant for inexperienced readers but not for experienced readers: 11.2% (95%CI 3.1% to 18.9%) versus 3.2% (95%CI -1.9% to 8.3%) respectively. CONCLUSIONS: Concurrent CAD resulted in a significant net benefit when used by inexperienced readers to identify patients with polyps by CT colonography. The net benefit was nearly four times the magnitude of that observed for experienced readers. Experienced readers did not benefit significantly from concurrent CAD.

CT colonography: external clinical validation of an algorithm for computer-assisted prone and supine registration.

PURPOSE: To perform external validation of a computer-assisted registration algorithm for prone and supine computed tomographic (CT) colonography and to compare the results with those of an existing centerline method. MATERIALS AND METHODS: All contributing centers had institutional review board approval; participants provided informed consent. A validation sample of CT colonographic examinations of 51 patients with 68 polyps (6-55 mm) was selected from a publicly available, HIPAA compliant, anonymized archive. No patients were excluded because of poor preparation or inadequate distension. Corresponding prone and supine polyp coordinates were recorded, and endoluminal surfaces were registered automatically by using a computer algorithm. Two observers independently scored three-dimensional endoluminal polyp registration success. Results were compared with those obtained by using the normalized distance along the colonic centerline (NDACC) method. Pairwise Wilcoxon signed rank tests were used to compare gross registration error and McNemar tests were used to compare polyp conspicuity. RESULTS: Registration was possible in all 51 patients, and 136 paired polyp coordinates were generated (68 polyps) to test the algorithm. Overall mean three-dimensional polyp registration error (mean ± standard deviation, 19.9 mm ± 20.4) was significantly less than that for the NDACC method (mean, 27.4 mm ± 15.1; P = .001). Accuracy was unaffected by colonic segment (P = .76) or luminal collapse (P = .066). During endoluminal review by two observers (272 matching tasks, 68 polyps, prone to supine and supine to prone coordinates), 223 (82%) polyp matches were visible (120° field of view) compared with just 129 (47%) when the NDACC method was used (P < .001). By using multiplanar visualization, 48 (70%) polyps were visible after scrolling ± 15 mm in any multiplanar axis compared with 16 (24%) for NDACC (P < .001). CONCLUSION: Computer-assisted registration is more accurate than the NDACC method for mapping the endoluminal surface and matching the location of polyps in corresponding prone and supine CT colonographic acquisitions.

Incremental benefit of computer-aided detection when used as a second and concurrent reader of CT colonographic data: multiobserver study.

PURPOSE: To quantify the changes in reader performance levels, if any, during interpretation of computed tomographic (CT) colonographic data when a computer-aided detection (CAD) system is used as a second or concurrent reader. MATERIALS AND METHODS: After institutional review board approval was obtained, 16 experienced radiologists searched for polyps in 112 patients, 56 of whom had 132 polyps. Each case was interpreted on three separate occasions by using an unassisted (without CAD), second-read CAD, or concurrent CAD reading paradigm. The reading paradigm and case order were randomized, with a minimal interval of 1 month between consecutive interpretations. The readers' findings were compared with the reference-truth interpretation. The mean per-patient sensitivity and mean per-patient specificity with CAD were compared with those achieved with unassisted reading. An increase in per-patient sensitivity was considered to be clinically more important than an equivalent decrease in specificity. RESULTS: The mean per-patient sensitivity for identification of patients with polyps of any size increased significantly with use of second-read CAD (mean increase, 7.0%; 95% confidence interval [CI]: 4.0%, 9.8%) and concurrent CAD (mean increase, 4.5%; 95% CI: 0.8%, 8.2%). The mean per-patient specificity did not decrease significantly with use of second-read CAD (mean decrease, -2.5%; 95% CI: -5.2%, 0.1%) or concurrent CAD (mean decrease, -2.2%; 95% CI: -4.6%, 0.2%). With analysis restricted to patients with polyps 6 mm or larger, the benefit in sensitivity with second-read CAD remained (mean increase, 7.1%; 95% CI: 3.0%, 11.1%), whereas the increase with concurrent CAD was not significant (mean increase, 4.2%; 95% CI: -0.5%, 8.9%). Use of second-read CAD significantly increased the per-polyp sensitivity for polyps 6 mm or larger (mean increase, 9.0%; 98.3% CI: 4.9%, 12.8%) and polyps 5 mm or smaller (mean increase, 5.9%; 98.3% CI: 3.2%, 9.1%), but use of concurrent CAD increased the per-polyp sensitivity for only those polyps 5 mm or smaller (mean increase, 4.8%; 98.3% CI: 2.2%, 7.9%). CONCLUSION: Use of second-read CAD significantly improves readers' per-patient and per-polyp detection. Concurrent CAD is less effective. SUPPLEMENTAL MATERIAL: http://radiology.rsna.org/lookup/suppl/doi:10.1148/radiol.10100354/-/DC1.

Computer assisted detection software for CT colonography: effect of sphericity filter on performance characteristics for patients with and without fecal tagging.

The aim of this study is to investigate the effect of changing sphericity filter values on performance of a computer assisted detection (CAD) system for CT colonography for data with and without fecal tagging. Colonography data from 138 patients with 317 validated polyps were divided into those with (86) and without (52) fecal tagging. Polyp coordinates were established by three observers and datasets analysed subsequently by a proprietary CAD system used at four discrete sphericity filter settings. Prompts were compared with the known coordinates in order to determine sensitivity and specificity. Sensitivity was highest at low sphericity; of 164 polyps 6 mm or more, 144 (87.8%) were detected at sphericity 0.3, and 132 (80.1%) at sphericity 0.9. Of 42 polyps measuring 10 mm or more, 40 (95.2%) were detected at sphericity 0.3, and 36 (85.7%) at sphericity 0.9. There was no significant difference in sensitivity for tagged and un-tagged data but specificity was reduced in tagged data at low sphericity and significantly reduced in untagged data at high sphericity. CAD had a sensitivity of 95.2% for polyps measuring 1 cm or more and 87.8% for polyps 6 mm or more when used at a sphericity setting of 0.3. Higher sphericity settings increased specificity while reducing sensitivity. The bowel preparation used significantly impacts on specificity.

Computed tomographic colonography: assessment of radiologist performance with and without computer-aided detection.

BACKGROUND & AIMS: In isolation, computer-aided detection (CAD) for computed tomographic (CT) colonography is as effective as optical colonoscopy for detection of significant adenomas. However, the unavoidable interaction between CAD and the reader has not been addressed. METHODS: Ten readers trained in CT but without special expertise in colonography interpreted CT colonography images of 107 patients (60 with 142 polyps), first without CAD and then with CAD after temporal separation of 2 months. Per-patient and per-polyp detection were determined by comparing responses with known patient status. RESULTS: With CAD, 41 (68%; 95% confidence interval [CI], 55%-80%) of the 60 patients with polyps were identified more frequently by readers. Per-patient sensitivity increased significantly in 70% of readers, while specificity dropped significantly in only one. Polyp detection increased significantly with CAD; on average, 12 more polyps were detected by each reader (9.1%, 95% CI, 5.2%-12.8%). Small- (< or =5 mm) and medium-sized (6-9 mm) polyps were significantly more likely to be detected when prompted correctly by CAD. However, overall performance was relatively poor; even with CAD, on average readers detected only 10 polyps (51.0%) > or =10 mm and 24 (38.2%) > or =6 mm. Interpretation time was shortened significantly with CAD: by 1.9 minutes (95% CI, 1.4-2.4 minutes) for patients with polyps and by 2.9 minutes (95% CI, 2.5-3.3 minutes) for patients without. Overall, 9 readers (90%) benefited significantly from CAD, either by increased sensitivity and/or by reduced interpretation time. CONCLUSIONS: CAD for CT colonography significantly increases per-patient and per-polyp detection and significantly reduces interpretation times but cannot substitute for adequate training.

Polyp detection with CT colonography: primary 3D endoluminal analysis versus primary 2D transverse analysis with computer-assisted reader software.

PURPOSE: To retrospectively compare primary three-dimensional (3D) endoluminal analysis with primary two-dimensional (2D) transverse analysis supplemented by computer-assisted reader (CAR) software for computed tomographic (CT) polyp detection and reader reporting times. MATERIALS AND METHODS: Ethical permission and patient consent were obtained from all donor institutions for use of CT colonography data sets. Twenty CT colonography data sets from 14 men (median age, 61 years; age range, 52-78 years) with 48 endoscopically proved polyps were selected. Polyp coordinates were documented in consensus by three unblinded radiologists to create a reference standard. Two radiologists read the data sets, which were randomized between primary 3D endoluminal views with 2D problem solving and 2D views supplemented by CAR software. Reading times and diagnostic confidence were documented. The CAR software highlighted possible polyps by superimposing circles on the 2D transverse images. Data sets were reread after 1 month by using the opposing analysis method. Detection rates were compared by using the McNemar test. Reporting times and diagnostic confidence were compared by using the paired t test and Mann-Whitney U test, respectively. RESULTS: Mean sensitivity values for polyps measuring 1-5, 6-9, and 10 mm or larger were 14%, 53%, and 83%, respectively, for 2D CAR analysis and 16%, 53%, and 67%, respectively, for primary 3D analysis. Overall sensitivity values were 41% for 2D CAR analysis and 39% for primary 3D analysis (P=.77). Reader 1 detected more polyps than reader 2, particularly when using the 3D fly-through method (P=.002). Mean reading times were significantly longer with the 3D method (P=.001). Mean false-positive findings were 1.5 for 2D analysis and 5.5 for 3D analysis. Reader confidence was not significantly different between analysis methods (P=.42). CONCLUSION: Two-dimensional CAR analysis is quicker and at least matches the sensitivity of primary 3D endoluminal analysis, with fewer false-positive findings.

Computer-assisted reader software versus expert reviewers for polyp detection on CT colonography.

OBJECTIVE: The purpose of our study was to assess the sensitivity of computer-assisted reader (CAR) software for polyp detection compared with the performance of expert reviewers. MATERIALS AND METHODS: A library of colonoscopically validated CT colonography cases were collated and separated into training and test sets according to the time of accrual. Training data sets were annotated in consensus by three expert radiologists who were aware of the colonoscopy report. A subset of 45 training cases containing 100 polyps underwent batch analysis using ColonCAR version 1.2 software to determine the optimum polyp enhancement filter settings for polyp detection. Twenty-five consecutive positive test data sets were subsequently interpreted individually by each expert, who was unaware of the endoscopy report, and before generation of the annotated reference via an unblinded consensus interpretation. ColonCAR version 1.2 software was applied to the test cases, at optimized polyp enhancement filter settings, to determine diagnostic performance. False-positive findings were classified according to importance. RESULTS: The 25 test cases contained 32 nondiminutive polyps ranging from 6 to 35 mm in diameter. The ColonCAR version 1.2 software identified 26 (81%) of 32 polyps compared with an average sensitivity of 70% for the expert reviewers. Eleven (92%) of 12 polyps > or = 10 mm were detected by ColonCAR version 1.2. All polyps missed by experts 1 (n = 4) and 2 (n = 3) and 12 (86%) of 14 polyps missed by expert 3 were detected by ColonCAR version 1.2. The median number of false-positive highlights per case was 13, of which 91% were easily dismissed. CONCLUSION: ColonCAR version 1.2 is sensitive for polyp detection, with a clinically acceptable false-positive rate. ColonCAR version 1.2 has a synergistic effect to the reviewer alone, and its standalone performance may exceed even that of experts.

Computed tomography colonography: automated diameter and volume measurement of colonic polyps compared with a manual technique--in vitro study.

OBJECTIVE: To investigate inter- and intraobserver agreement of automated measurement of polyp diameter in vitro. METHODS: Two phantoms ("QRM" and "Whiting") containing simulated polyps of known diameter and volume were scanned using 16-detector row computed tomography. Two observers estimated polyp diameter using 3 methods: software calipers ("manual"), freehand boundary identification ("semiautomatic"), and automated software segmentation ("fully automatic"). RESULTS: Intraobserver 95% limits of agreement for diameter were narrowest for the fully automatic method (QRM span: 0.39 mm, 0.48 mm; Whiting span: 0.24 mm, 0 mm). Manual estimates were approximately 10 times wider (QRM span: 3.57 mm, 3.21 mm; Whiting span: 3.2 mm, 2.02 mm). Volume estimates were narrowest for the fully automatic method (span: 24.2 mm, 24.1 mm vs. 97.9 mm, 102.9 mm for semiautomatic measurement). Interobserver agreement for diameter was narrowest for the fully automatic method (QRM span: 0.12 mm, Whiting span: 0.16 mm), with the manual method approximately 18 times wider (QRM span: 2.87 mm, Whiting span: 2.18 mm). CONCLUSION: Fully automated measurement of polyp diameter and volume is technically feasible and results in superior inter- and intraobserver agreement.