Diagnostic Accuracy and Failure Mode Analysis of a Deep Learning Algorithm for the Detection of Cervical Spine Fractures.

BACKGROUND AND PURPOSE: Artificial intelligence decision support systems are a rapidly growing class of tools to help manage ever-increasing imaging volumes. The aim of this study was to evaluate the performance of an artificial intelligence decision support system, Aidoc, for the detection of cervical spinal fractures on noncontrast cervical spine CT scans and to conduct a failure mode analysis to identify areas of poor performance. MATERIALS AND METHODS: This retrospective study included 1904 emergent noncontrast cervical spine CT scans of adult patients (60 [SD, 22] years, 50.3% men). The presence of cervical spinal fracture was determined by Aidoc and an attending neuroradiologist; discrepancies were independently adjudicated. Algorithm performance was assessed by calculation of the diagnostic accuracy, and a failure mode analysis was performed. RESULTS: Aidoc and the neuroradiologist's interpretation were concordant in 91.5% of cases. Aidoc correctly identified 67 of 122 fractures (54.9%) with 106 false-positive flagged studies. Diagnostic performance was calculated as the following: sensitivity, 54.9% (95% CI, 45.7%-63.9%); specificity, 94.1% (95% CI, 92.9%-95.1%); positive predictive value, 38.7% (95% CI, 33.1%-44.7%); and negative predictive value, 96.8% (95% CI, 96.2%-97.4%). Worsened performance was observed in the detection of chronic fractures; differences in diagnostic performance were not altered by study indication or patient characteristics. CONCLUSIONS: We observed poor diagnostic accuracy of an artificial intelligence decision support system for the detection of cervical spine fractures. Many similar algorithms have also received little or no external validation, and this study raises concerns about their generalizability, utility, and rapid pace of deployment. Further rigorous evaluations are needed to understand the weaknesses of these tools before widespread implementation.

Risk for colon adenomas in patients with rectosigmoid hyperplastic polyps.

OBJECTIVE: To determine whether hyperplastic polyps found in the rectosigmoid area of the colon are associated with proximal adenomas, and to judge whether patients with distal hyperplastic polyps found during sigmoidoscopy might benefit from full colonoscopy. DESIGN: Data on patients having colonoscopy collected prospectively according to a set protocol. The size and location of all polyps were noted, and all polyps were biopsied. SETTING: Two university hospitals. PATIENTS: One thousand eight hundred and thirty-six consecutive patients referred for colonoscopy between 31 December 1987 and 31 August 1989. RESULTS: Of the 970 patients who met eligibility requirements, 274 (28.3%) had adenomas and 108 (11.1%) had hyperplastic polyps. The proportion of patients with distal hyperplastic polyps and proximal adenomas (31.9%) was similar to the proportion of those without distal hyperplastic polyps (23.0%) (crude odds ratio, 1.57; 95% CI, 0.77 to 3.06). After adjusting for age and sex, the results were unchanged (adjusted odds ratio, 1.53; CI, 0.82 to 2.88). Patients with distal adenomas, on the other hand, were three times more likely to have proximal adenomas than those without distal adenomas (adjusted odds ratio, 3.42; CI, 1.99 to 5.88). CONCLUSIONS: Distal hyperplastic polyps are not strong predictors of risk for proximal adenomas. Based on the magnitude of the risk difference, we do not believe that finding a hyperplastic polyp during sigmoidoscopy justifies doing a full colonoscopy to search for proximal adenomas. Because rectosigmoid adenomas are associated with proximal adenomas, however, small polyps seen during sigmoidoscopy should be biopsied to determine their type. Colonoscopy should be reserved for patients who are proved to have adenomas.

Health status in inflammatory bowel disease. Biological and behavioral considerations.

The existing clinical measures of disease activity for inflammatory bowel disease are insufficient to explain a patient's illness experience or health outcomes. Although many disease activity measures have been devised, they are not widely accepted by clinicians because existing ones are no better than a carefully obtained clinical assessment. Furthermore, health status is determined not only by disease activity, but also by the psychological state, cultural influences, degree of social support, and effects of complications, previous surgery, and medication. To develop more accurate appraisals of the impact and severity of IBD, we must prospectively evaluate the biological and psychosocial measures that predict clinically relevant outcomes. We should then be able to develop statistically weighted scales related to specific outcome variables. Such knowledge will help us to develop more sensitive measures of illness, particularly in patients with mild disease for whom present indices are insensitive. This type of assessment should also aid in the understanding of health care utilization, medical vs. surgical options, resource allocation, and the efficacy of therapeutic trials.