Algorithm Variability in the Estimation of Lung Nodule Volume From Phantom CT Scans: Results of the QIBA 3A Public Challenge.

RATIONALE AND OBJECTIVES: Quantifying changes in lung tumor volume is important for diagnosis, therapy planning, and evaluation of response to therapy. The aim of this study was to assess the performance of multiple algorithms on a reference data set. The study was organized by the Quantitative Imaging Biomarker Alliance (QIBA). MATERIALS AND METHODS: The study was organized as a public challenge. Computed tomography scans of synthetic lung tumors in an anthropomorphic phantom were acquired by the Food and Drug Administration. Tumors varied in size, shape, and radiodensity. Participants applied their own semi-automated volume estimation algorithms that either did not allow or allowed post-segmentation correction (type 1 or 2, respectively). Statistical analysis of accuracy (percent bias) and precision (repeatability and reproducibility) was conducted across algorithms, as well as across nodule characteristics, slice thickness, and algorithm type. RESULTS: Eighty-four percent of volume measurements of QIBA-compliant tumors were within 15% of the true volume, ranging from 66% to 93% across algorithms, compared to 61% of volume measurements for all tumors (ranging from 37% to 84%). Algorithm type did not affect bias substantially; however, it was an important factor in measurement precision. Algorithm precision was notably better as tumor size increased, worse for irregularly shaped tumors, and on the average better for type 1 algorithms. Over all nodules meeting the QIBA Profile, precision, as measured by the repeatability coefficient, was 9.0% compared to 18.4% overall. CONCLUSION: The results achieved in this study, using a heterogeneous set of measurement algorithms, support QIBA quantitative performance claims in terms of volume measurement repeatability for nodules meeting the QIBA Profile criteria.

Measurement of tumor volumes improves RECIST-based response assessments in advanced lung cancer.

OBJECTIVE: This study was designed to characterize the reproducibility of measurement for tumor volumes and their longest tumor diameters (LDs) and estimate the potential impact of using changes in tumor volumes instead of LDs as the basis for response assessments. METHODS: We studied patients with advanced lung cancer who have been observed longitudinally with x-ray computed tomography in a multinational trial. A total of 71 time points from 10 patients with 13 morphologically complex target lesions were analyzed. A total of 6461 volume measurements and their corresponding LDs were made by seven independent teams using their own work flows and image analysis tools. Interteam agreement and overall interrater concurrence were characterized. RESULTS: Interteam agreement between volume measurements was better than between LD measurements (i = 0.945 vs 0.734, P = .005). The variability in determining the nadir was lower for volumes than for LDs (P = .005). Use of standard thresholds for the RECIST-based method and use of experimentally determined cutoffs for categorizing responses showed that volume measurements had a significantly greater sensitivity for detecting partial responses and disease progression. Earlier detection of progression would have led to earlier changes in patient management in most cases. CONCLUSIONS: Our findings indicate that measurement of changes in tumor volumes is adequately reproducible. Using tumor volumes as the basis for response assessments could have a positive impact on both patient management and clinical trials. More authoritative work to qualify or discard changes in volume as the basis for response assessments should proceed.