The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): a completed reference database of lung nodules on CT scans.

PURPOSE: The development of computer-aided diagnostic (CAD) methods for lung nodule detection, classification, and quantitative assessment can be facilitated through a well-characterized repository of computed tomography (CT) scans. The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI) completed such a database, establishing a publicly available reference for the medical imaging research community. Initiated by the National Cancer Institute (NCI), further advanced by the Foundation for the National Institutes of Health (FNIH), and accompanied by the Food and Drug Administration (FDA) through active participation, this public-private partnership demonstrates the success of a consortium founded on a consensus-based process. METHODS: Seven academic centers and eight medical imaging companies collaborated to identify, address, and resolve challenging organizational, technical, and clinical issues to provide a solid foundation for a robust database. The LIDC/IDRI Database contains 1018 cases, each of which includes images from a clinical thoracic CT scan and an associated XML file that records the results of a two-phase image annotation process performed by four experienced thoracic radiologists. In the initial blinded-read phase, each radiologist independently reviewed each CT scan and marked lesions belonging to one of three categories ("nodule > or =3 mm," "nodule <3 mm," and "non-nodule > or =3 mm"). In the subsequent unblinded-read phase, each radiologist independently reviewed their own marks along with the anonymized marks of the three other radiologists to render a final opinion. The goal of this process was to identify as completely as possible all lung nodules in each CT scan without requiring forced consensus. RESULTS: The Database contains 7371 lesions marked "nodule" by at least one radiologist. 2669 of these lesions were marked "nodule > or =3 mm" by at least one radiologist, of which 928 (34.7%) received such marks from all four radiologists. These 2669 lesions include nodule outlines and subjective nodule characteristic ratings. CONCLUSIONS: The LIDC/IDRI Database is expected to provide an essential medical imaging research resource to spur CAD development, validation, and dissemination in clinical practice.

The Lung Image Database Consortium (LIDC): an evaluation of radiologist variability in the identification of lung nodules on CT scans.

RATIONALE AND OBJECTIVES: The purpose of this study was to analyze the variability of experienced thoracic radiologists in the identification of lung nodules on computed tomography (CT) scans and thereby to investigate variability in the establishment of the "truth" against which nodule-based studies are measured. MATERIALS AND METHODS: Thirty CT scans were reviewed twice by four thoracic radiologists through a two-phase image annotation process. During the initial "blinded read" phase, radiologists independently marked lesions they identified as "nodule >or=3 mm (diameter)," "nodule <3 mm," or "non-nodule >or=3 mm." During the subsequent "unblinded read" phase, the blinded read results of all four radiologists were revealed to each radiologist, who then independently reviewed their marks along with the anonymous marks of their colleagues; a radiologist's own marks then could be deleted, added, or left unchanged. This approach was developed to identify, as completely as possible, all nodules in a scan without requiring forced consensus. RESULTS: After the initial blinded read phase, 71 lesions received "nodule >or=3 mm" marks from at least one radiologist; however, all four radiologists assigned such marks to only 24 (33.8%) of these lesions. After the unblinded reads, a total of 59 lesions were marked as "nodule >or=3 mm" by at least one radiologist. Twenty-seven (45.8%) of these lesions received such marks from all four radiologists, three (5.1%) were identified as such by three radiologists, 12 (20.3%) were identified by two radiologists, and 17 (28.8%) were identified by only a single radiologist. CONCLUSION: The two-phase image annotation process yields improved agreement among radiologists in the interpretation of nodules >or=3 mm. Nevertheless, substantial variability remains across radiologists in the task of lung nodule identification.

Database design and implementation for quantitative image analysis research.

Quantitative image analysis (QIA) goes beyond subjective visual assessment to provide computer measurements of the image content, typically following image segmentation to identify anatomical regions of interest (ROIs). Commercially available picture archiving and communication systems focus on storage of image data. They are not well suited to efficient storage and mining of new types of quantitative data. In this paper, we present a system that integrates image segmentation, quantitation, and characterization with database and data mining facilities. The paper includes generic process and data models for QIA in medicine and describes their practical use. The data model is based upon the Digital Imaging and Communications in Medicine (DICOM) data hierarchy, which is augmented with tables to store segmentation results (ROIs) and quantitative data from multiple experiments. Data mining for statistical analysis of the quantitative data is described along with example queries. The database is implemented in PostgreSQL on a UNIX server. Database requirements and capabilities are illustrated through two quantitative imaging experiments related to lung cancer screening and assessment of emphysema lung disease. The system can manage the large amounts of quantitative data necessary for research, development, and deployment of computer-aided diagnosis tools.

Comparison of treatment response classifications between unidimensional, bidimensional, and volumetric measurements of metastatic lung lesions on chest computed tomography.

RATIONALE AND OBJECTIVES: To study the agreement in treatment response classifications between unidimensional (1D), bidimensional (2D), and volumetric (3D) methods of measuring metastatic lung nodules on chest computed tomography (CT). MATERIALS AND METHODS: Chest CT scans of 15 patients undergoing treatment for metastatic colorectal, renal cell, or breast carcinoma to the lungs were analyzed. CT images were acquired with 3 mm collimation and contiguous reconstruction. Two or three lung lesions were selected for each patient. Lesions were analyzed at baseline and two follow-up intervals of 1-4 months. 1D and 2D measurements were made with electronic calipers, while nodule volume was measured using a semiautomated segmentation system. Following the World Health Organization and RECIST (Response Evaluation Criteria in Solid Tumors) criteria, patients were categorized into four treatment response classifications. Volumetric criteria were used to classify response based on 3D measurements. RESULTS: Thirty-two lesions from 15 patients were analyzed. Because each patient had a baseline and two follow-up scans, this yielded 30 response classifications for each measurement technique. The 1D, 2D, and 3D measurements were concordant in 21 of 30 classifications. The 1D and 3D measurements were concordant in 29 of 30 classifications, while the 2D and 3D measurements were concordant in 23 of 30 classifications. Level of agreement among the three methods was measured using a kappa statistic (K). For 1D compared with 3D, K = 0.739 +/- 0.345 (visits 1, 2) and 0.273 +/- 0.323 (visits 2, 3). For 2D compared with 3D, K = 0.655 +/- 0.325 (visits 1, 2) and 0.200 +/- 0.208 (visits 2, 3). Agreement among the methods for round and ovoid nodules was also fair to poor. CONCLUSION: The three methods of tumor measurement show fair to poor agreement in treatment response classification. These findings have negative implications for the accuracy in which patients are classified under the World Health Organization or RECIST criteria and managed under cancer treatment protocols.