Radiology workflow for RECIST assessment in clinical trials: Can we reconcile time-efficiency and quality?

PURPOSE: In oncology clinical trials, nonconformity issues are frequently reported. Radiological workload is increasing, thus reducing radiologists' availability and affecting diagnostic quality. We compared performances of a standard radiological workflow (SW) and a novel "hybrid workflow" (HW). METHOD: We prospectively studied imaging data of 40 patients included in RECIST 1.1 clinical trials. Ninety-six time-points were reviewed by 7 radiologists and one trained technologist. Nonconformities using the SW were retrieved from hospital archives. For the HW, radiologists performed all baseline evaluations; the technologist made subsequent measurements. Finally, the radiologists checked the technologist's findings before confirming the evaluations. The HW enabled implementation of an electronic reporting system. An independent body compared SW and HW reading times and nonconformity occurrences. RESULTS: Using SW, 19 types of nonconformity were found: blank report (13%); unsigned report (11%); undocumented change of tumor burden (10%); undocumented new lesions (9%); missing/wrong patients' appointment dates (7%); undocumented tumor location (5%); error in tumor burden change (5%). SW and HW nonconformities affected 55% (179/323) and 5% (2/40) of reports, respectively (p < 0.001). HW nonconformities were: one inaccurate login name was used on the platform, and one erroneous time-point number. On average, SW required 11'30″ [10'06″; 13'20″] per time-point. HW required 1'35″ [40″; 5'08″] for radiologists, and 12'18″ [11'12″; 14'18″] for the technologist. CONCLUSIONS: HW significantly reduced the number of trial nonconformities and saved 87% of radiologists' time while enabling them to apply their expertise to final decisions. HW could offer an effective opportunity for cost reduction associated with improved imaging trial quality.

Inter-Method Performance Study of Tumor Volumetry Assessment on Computed Tomography Test-Retest Data.

RATIONALE AND OBJECTIVES: Tumor volume change has potential as a biomarker for diagnosis, therapy planning, and treatment response. Precision was evaluated and compared among semiautomated lung tumor volume measurement algorithms from clinical thoracic computed tomography data sets. The results inform approaches and testing requirements for establishing conformance with the Quantitative Imaging Biomarker Alliance (QIBA) Computed Tomography Volumetry Profile. MATERIALS AND METHODS: Industry and academic groups participated in a challenge study. Intra-algorithm repeatability and inter-algorithm reproducibility were estimated. Relative magnitudes of various sources of variability were estimated using a linear mixed effects model. Segmentation boundaries were compared to provide a basis on which to optimize algorithm performance for developers. RESULTS: Intra-algorithm repeatability ranged from 13% (best performing) to 100% (least performing), with most algorithms demonstrating improved repeatability as the tumor size increased. Inter-algorithm reproducibility was determined in three partitions and was found to be 58% for the four best performing groups, 70% for the set of groups meeting repeatability requirements, and 84% when all groups but the least performer were included. The best performing partition performed markedly better on tumors with equivalent diameters greater than 40 mm. Larger tumors benefitted by human editing but smaller tumors did not. One-fifth to one-half of the total variability came from sources independent of the algorithms. Segmentation boundaries differed substantially, not ony in overall volume but also in detail. CONCLUSIONS: Nine of the 12 participating algorithms pass precision requirements similar to what is indicated in the QIBA Profile, with the caveat that the present study was not designed to explicitly evaluate algorithm profile conformance. Change in tumor volume can be measured with confidence to within ±14% using any of these nine algorithms on tumor sizes greater than 10 mm. No partition of the algorithms was able to meet the QIBA requirements for interchangeability down to 10 mm, although the partition comprising best performing algorithms did meet this requirement for a tumor size of greater than approximately 40 mm.