Potholes and molehills: bias in the diagnostic performance of diffusion-tensor imaging in concussion.

PURPOSE: To investigate the extent of bias in a clinical study involving "pothole analysis" of diffusion-tensor imaging (DTI) data used to quantify white matter lesion load in diseases with a heterogeneous spatial distribution of pathologic findings, such as mild traumatic brain injury (TBI), and create a mathematical model of the bias. MATERIALS AND METHODS: Use of the same reference population to define normal findings and make comparisons with a patient group introduces bias, which potentially inflates reported diagnostic performance. In this institutional review board-approved prospective observational cohort study, DTI data were obtained in 20 patients admitted to the emergency department with mild TBI and in 16 control subjects. Potholes and molehills were defined as clusters of voxels with fractional anisotropy values more than 2 standard deviations below and above the mean of the corresponding voxels in the reference population, respectively. The number and volume of potholes and molehills in the two groups were compared by using a Mann-Whitney U test. RESULTS: Standard analysis showed significantly more potholes in mild TBI than in the control group (102.5 ± 34.3 vs 50.6 ± 28.9, P < .001). Repeat analysis by using leave-one-out cross-validation decreased the apparent difference in potholes between groups (mild TBI group, 102.5 ± 34.3; control group, 93.4 ± 27.2; P = .369). It was demonstrated that even with 100 subjects, this bias can decrease the voxelwise false-positive rate by more than 30% in the control group. CONCLUSION: The pothole approach to neuroimaging data may introduce bias, which can be minimized by independent training and test groups or cross-validation methods. This bias is sufficient to call into question the previously reported diagnostic performance of DTI for mild TBI.