Uncertainties in RECIST as a measure of volume for lung nodules and liver tumors.

PURPOSE: The authors wish to determine the extent to which the Response Evaluation Criteria in Solid Tumors (RECIST) and the criteria of the World Health Organization (WHO) can predict tumor volumes and changes in volume using clinical data. METHODS: The data presented are a reanalysis of data acquired in other studies, including the public database from the Lung Image Database Consortium (LIDC) and from a study of liver tumors. RESULTS: The principal result is that a given RECIST diameter predicts volume to a factor of 16 or 10 for the two data sets, respectively, by examining 95% prediction bounds and that changes in volume are predicted only little better: to within a factor of 7 for the liver data. The WHO criteria reduce the prediction bounds by a factor of 1.3 in all cases. Also, the RECIST threshold of 10 mm to measure a nodule corresponds to a transition zone width of a factor of more than 2 in volume for the nodules in the LIDC database. CONCLUSIONS: While the RECIST diameter is certainly correlated with the volume, and similarly for changes in these quantities, the use of the diameter introduces additional variation assuming volume is the quantity of interest. Exactly how much this reduces the statistical power of clinical drug trials is a key open question for future research.

The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): a completed reference database of lung nodules on CT scans.

PURPOSE: The development of computer-aided diagnostic (CAD) methods for lung nodule detection, classification, and quantitative assessment can be facilitated through a well-characterized repository of computed tomography (CT) scans. The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI) completed such a database, establishing a publicly available reference for the medical imaging research community. Initiated by the National Cancer Institute (NCI), further advanced by the Foundation for the National Institutes of Health (FNIH), and accompanied by the Food and Drug Administration (FDA) through active participation, this public-private partnership demonstrates the success of a consortium founded on a consensus-based process. METHODS: Seven academic centers and eight medical imaging companies collaborated to identify, address, and resolve challenging organizational, technical, and clinical issues to provide a solid foundation for a robust database. The LIDC/IDRI Database contains 1018 cases, each of which includes images from a clinical thoracic CT scan and an associated XML file that records the results of a two-phase image annotation process performed by four experienced thoracic radiologists. In the initial blinded-read phase, each radiologist independently reviewed each CT scan and marked lesions belonging to one of three categories ("nodule > or =3 mm," "nodule <3 mm," and "non-nodule > or =3 mm"). In the subsequent unblinded-read phase, each radiologist independently reviewed their own marks along with the anonymized marks of the three other radiologists to render a final opinion. The goal of this process was to identify as completely as possible all lung nodules in each CT scan without requiring forced consensus. RESULTS: The Database contains 7371 lesions marked "nodule" by at least one radiologist. 2669 of these lesions were marked "nodule > or =3 mm" by at least one radiologist, of which 928 (34.7%) received such marks from all four radiologists. These 2669 lesions include nodule outlines and subjective nodule characteristic ratings. CONCLUSIONS: The LIDC/IDRI Database is expected to provide an essential medical imaging research resource to spur CAD development, validation, and dissemination in clinical practice.

Determining the Variability of Lesion Size Measurements from CT Patient Data Sets Acquired under "No Change" Conditions.

PURPOSE: To determine the variability of lesion size measurements in computed tomography data sets of patients imaged under a "no change" ("coffee break") condition and to determine the impact of two reading paradigms on measurement variability. METHOD AND MATERIALS: Using data sets from 32 non-small cell lung cancer patients scanned twice within 15 minutes ("no change"), measurements were performed by five radiologists in two phases: (1) independent reading of each computed tomography dataset (timepoint): (2) a locked, sequential reading of datasets. Readers performed measurements using several sizing methods, including one-dimensional (1D) longest in-slice dimension and 3D semi-automated segmented volume. Change in size was estimated by comparing measurements performed on both timepoints for the same lesion, for each reader and each measurement method. For each reading paradigm, results were pooled across lesions, across readers, and across both readers and lesions, for each measurement method. RESULTS: The mean percent difference (±SD) when pooled across both readers and lesions for 1D and 3D measurements extracted from contours was 2.8 ± 22.2% and 23.4 ± 105.0%, respectively, for the independent reads. For the locked, sequential reads, the mean percent differences (±SD) reduced to 2.52 ± 14.2% and 7.4 ± 44.2% for the 1D and 3D measurements, respectively. CONCLUSION: Even under a "no change" condition between scans, there is variation in lesion size measurements due to repeat scans and variations in reader, lesion, and measurement method. This variation is reduced when using a locked, sequential reading paradigm compared to an independent reading paradigm.

Three-Axis Coil Probe Dimensions and Uncertainties During Measurement of Magnetic Fields from Appliances.

Comparisons are made between the average magnetic flux density for a three-axis circular coil probe and the flux density at the center of the probe. The results, which are determined assuming a dipole magnetic field, provide information on the uncertainty associated with measurements of magnetic fields from some electrical appliances and other electrical equipment. The present investigation extends an earlier treatment of the problem, which did not consider all orientations of the probe. A more comprehensive examination of the problem leaves unchanged the conclusions reached previously.

Comparison of 1D, 2D, and 3D nodule sizing methods by radiologists for spherical and complex nodules on thoracic CT phantom images.

RATIONALE AND OBJECTIVES: To estimate and statistically compare the bias and variance of radiologists measuring the size of spherical and complex synthetic nodules. MATERIALS AND METHODS: This study did not require the institutional review board approval. Six radiologists estimated the size of 10 synthetic nodules embedded within an anthropomorphic thorax phantom from computed tomography scans at 0.8- and 5-mm slice thicknesses. The readers measured the nodule size using unidimensional (1D) longest in-slice dimension, bidimensional (2D) area from longest in-slice and longest perpendicular dimension, and three-dimensional (3D) semiautomated volume. Intercomparisons of bias (difference between average and true size) and variance among methods were performed after converting the 2D and 3D estimates to a compatible 1D scale. RESULTS: The relative biases of radiologists with the 3D tool were -1.8%, -0.4%, -0.7%, -0.4%, and -1.6% for 10-mm spherical, 20-mm spherical, 20-mm elliptical, 10-mm lobulated, and 10-mm spiculated nodules compared to 1.4%, -0.1%, -26.5%, -7.8%, and -39.8% for 1D. The three-dimensional measurements were significantly less biased than 1D for elliptical, lobulated, and spiculated nodules. The relative standard deviations for 3D were 7.5%, 3.9%, 3.6%, 9.7%, and 8.3% compared to 5.7%, 2.6%, 20.3%, 5.3%, and 16.4% for 1D. Unidimensional sizing was significantly less variable than 3D for the lobulated nodule and significantly more variable for the ellipsoid and spiculated nodules. Three-dimensional bias and variability were smaller for thin 0.8-mm slice data compared to thick 5.0-mm data. CONCLUSIONS: The study shows that radiologist-controlled 3D volumetric lesion sizing can not only achieve smaller bias but also achieve similar or smaller variability compared to 1D sizing, especially for complex lesion shapes.

Volumetric CT in lung cancer: an example for the qualification of imaging as a biomarker.

RATIONALE AND OBJECTIVES: New ways to understand biology as well as increasing interest in personalized treatments requires new capabilities for the assessment of therapy response. The lack of consensus methods and qualification evidence needed for large-scale multicenter trials, and in turn the standardization that allows them, are widely acknowledged to be the limiting factor in the deployment of qualified imaging biomarkers. MATERIALS AND METHODS: The Quantitative Imaging Biomarker Alliance is organized to establish a methodology whereby multiple stakeholders collaborate. It has charged the Volumetric Computed Tomography (CT) Technical Subcommittee with investigating the technical feasibility and clinical value of quantifying changes over time in either volume or other parameters as biomarkers. The group selected solid tumors of the chest in subjects with lung cancer as its first case in point. Success is defined as sufficiently rigorous improvements in CT-based outcome measures to allow individual patients in clinical settings to switch treatments sooner if they are no longer responding to their current regimens, and reduce the costs of evaluating investigational new drugs to treat lung cancer. RESULTS: The team has completed a systems engineering analysis, has begun a roadmap of experimental groundwork, documented profile claims and protocols, and documented a process for imaging biomarker qualification as a general paradigm for qualifying other imaging biomarkers as well. CONCLUSION: This report addresses a procedural template for the qualification of quantitative imaging biomarkers. This mechanism is cost-effective for stakeholders while simultaneously advancing the public health by promoting the use of measures that prove effective.

Quantitative imaging to assess tumor response to therapy: common themes of measurement, truth data, and error sources.

RATIONALE: Early detection of tumor response to therapy is a key goal. Finding measurement algorithms capable of early detection of tumor response could individualize therapy treatment as well as reduce the cost of bringing new drugs to market. On an individual basis, the urgency arises from the desire to prevent continued treatment of the patient with a high-cost and/or high-risk regimen with no demonstrated individual benefit and rapidly switch the patient to an alternative efficacious therapy for that patient. In the context of bringing new drugs to market, such algorithms could demonstrate efficacy in much smaller populations, which would allow phase 3 trials to achieve statistically significant decisions with fewer subjects in shorter trials. MATERIALS AND METHODS: This consensus-based article describes multiple, image modality-independent means to assess the relative performance of algorithms for measuring tumor change in response to therapy. In this setting, we describe specifically the example of measurement of tumor volume change from anatomic imaging as well as provide an overview of other promising generic analytic methods that can be used to assess change in heterogeneous tumors. To support assessment of the relative performance of algorithms for measuring small tumor change, data sources of truth are required. RESULTS: Very short interval clinical imaging examinations and phantom scans provide known truth for comparative evaluation of algorithms. CONCLUSIONS: For a given category of measurement methods, the algorithm that has the smallest measurement noise and least bias on average will perform best in early detection of true tumor change.

Computed tomography assessment of response to therapy: tumor volume change measurement, truth data, and error.

RATIONALE AND OBJECTIVES: This article describes issues and methods that are specific to the measurement of change in tumor volume as measured from computed tomographic (CT) images and how these would relate to the establishment of CT tumor volumetrics as a biomarker of patient response to therapy. The primary focus is on the measurement of lung tumors, but the approach should be generalizable to other anatomic regions. MATERIALS AND METHODS: The first issues addressed are the various sources of bias and variance in the measurement of tumor volumes, which are discussed in the context of measurement variation and its impact on the early detection of response to therapy. RESULTS AND RESOURCES: Research that seeks to identify the magnitude of some of these sources of error is ongoing, and several of these efforts are described herein. In addition, several resources for these investigations are being made available through the National Institutes of Health-funded Reference Image Database to Evaluate Response to therapy in cancer project, and these are described as well. Other measures derived from CT image data that might be predictive of patient response are described briefly, as well as the additional issues that each of these metrics may encounter in real-life applications. CONCLUSIONS: The article concludes with a brief discussion of moving from the assessment of measurement variation to the steps necessary to establish the efficacy of a metric as a biomarker for response.