Computerized segmentation and measurement of malignant pleural mesothelioma.

PURPOSE: The current linear method to track tumor progression and evaluate treatment efficacy is insufficient for malignant pleural mesothelioma (MPM). A volumetric method for tumor measurement could improve the evaluation of novel treatments, but a fully manual implementation of volume measurement is too tedious and time-consuming. This manuscript presents a computerized method for the three-dimensional segmentation and volumetric analysis of MPM. METHODS: The computerized MPM segmentation method segments the lung parenchyma and hemithoracic cavities to define the pleural space. Nonlinear diffusion and a k-means classifier are then implemented to identify MPM in the pleural space. A database of 31 computed tomography scans from 31 patients with pathologically confirmed MPM was retrospectively collected. Three observers independently outlined five randomly selected sections in each scan. The Jaccard similarity coefficient (J) between each of the observers and between the observer-defined and computer-defined segmentations was calculated. The computer-defined and the observer-defined segmentation areas (averaged over all observers) were both calculated for each axial section and compared using Bland-Altman plots. RESULTS: The median J value among observers averaged over all sections was 0.517. The median J between the computer-defined and manual segmentations was 0.484. The difference between these values was not statistically significant. The area delineated by the computerized method demonstrated variability and bias comparable to the tumor area calculated from manual delineations. CONCLUSIONS: A computerized method for segmentation and measurement of MPM was developed. This method requires minimal initialization by the user and demonstrated good agreement with manually drawn outlines and area measurements. This method will allow volumetric tracking of tumor progression and may improve the evaluation of novel MPM treatments.

The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): a completed reference database of lung nodules on CT scans.

PURPOSE: The development of computer-aided diagnostic (CAD) methods for lung nodule detection, classification, and quantitative assessment can be facilitated through a well-characterized repository of computed tomography (CT) scans. The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI) completed such a database, establishing a publicly available reference for the medical imaging research community. Initiated by the National Cancer Institute (NCI), further advanced by the Foundation for the National Institutes of Health (FNIH), and accompanied by the Food and Drug Administration (FDA) through active participation, this public-private partnership demonstrates the success of a consortium founded on a consensus-based process. METHODS: Seven academic centers and eight medical imaging companies collaborated to identify, address, and resolve challenging organizational, technical, and clinical issues to provide a solid foundation for a robust database. The LIDC/IDRI Database contains 1018 cases, each of which includes images from a clinical thoracic CT scan and an associated XML file that records the results of a two-phase image annotation process performed by four experienced thoracic radiologists. In the initial blinded-read phase, each radiologist independently reviewed each CT scan and marked lesions belonging to one of three categories ("nodule > or =3 mm," "nodule <3 mm," and "non-nodule > or =3 mm"). In the subsequent unblinded-read phase, each radiologist independently reviewed their own marks along with the anonymized marks of the three other radiologists to render a final opinion. The goal of this process was to identify as completely as possible all lung nodules in each CT scan without requiring forced consensus. RESULTS: The Database contains 7371 lesions marked "nodule" by at least one radiologist. 2669 of these lesions were marked "nodule > or =3 mm" by at least one radiologist, of which 928 (34.7%) received such marks from all four radiologists. These 2669 lesions include nodule outlines and subjective nodule characteristic ratings. CONCLUSIONS: The LIDC/IDRI Database is expected to provide an essential medical imaging research resource to spur CAD development, validation, and dissemination in clinical practice.

Discrete-space versus continuous-space lesion boundary and area definitions.

Measurement of the size of anatomic regions of interest in medical images is used to diagnose disease, track growth, and evaluate response to therapy. The discrete nature of medical images allows for both continuous and discrete definitions of region boundary. These definitions may, in turn, support several methods of area calculation that give substantially different quantitative values. This study investigated several boundary definitions (e.g., continuous polygon, internal discrete, and external discrete) and area calculation methods (pixel counting and Green's theorem). These methods were applied to three separate databases: A synthetic image database, the Lung Image Database Consortium database of lung nodules and a database of adrenal gland outlines. Average percent differences in area on the order of 20% were found among the different methods applied to the clinical databases. These results support the idea that inconsistent application of region boundary definition and area calculation may substantially impact measurement accuracy.

The Lung Image Database Consortium (LIDC): ensuring the integrity of expert-defined "truth".

RATIONALE AND OBJECTIVES: Computer-aided diagnostic (CAD) systems fundamentally require the opinions of expert human observers to establish "truth" for algorithm development, training, and testing. The integrity of this "truth," however, must be established before investigators commit to this "gold standard" as the basis for their research. The purpose of this study was to develop a quality assurance (QA) model as an integral component of the "truth" collection process concerning the location and spatial extent of lung nodules observed on computed tomography (CT) scans to be included in the Lung Image Database Consortium (LIDC) public database. MATERIALS AND METHODS: One hundred CT scans were interpreted by four radiologists through a two-phase process. For the first of these reads (the "blinded read phase"), radiologists independently identified and annotated lesions, assigning each to one of three categories: "nodule >or=3 mm," "nodule <3 mm," or "non-nodule >or=3 mm." For the second read (the "unblinded read phase"), the same radiologists independently evaluated the same CT scans, but with all of the annotations from the previously performed blinded reads presented; each radiologist could add to, edit, or delete their own marks; change the lesion category of their own marks; or leave their marks unchanged. The post-unblinded read set of marks was grouped into discrete nodules and subjected to the QA process, which consisted of identification of potential errors introduced during the complete image annotation process and correction of those errors. Seven categories of potential error were defined; any nodule with a mark that satisfied the criterion for one of these categories was referred to the radiologist who assigned that mark for either correction or confirmation that the mark was intentional. RESULTS: A total of 105 QA issues were identified across 45 (45.0%) of the 100 CT scans. Radiologist review resulted in modifications to 101 (96.2%) of these potential errors. Twenty-one lesions erroneously marked as lung nodules after the unblinded reads had this designation removed through the QA process. CONCLUSIONS: The establishment of "truth" must incorporate a QA process to guarantee the integrity of the datasets that will provide the basis for the development, training, and testing of CAD systems.

The Lung Image Database Consortium (LIDC): an evaluation of radiologist variability in the identification of lung nodules on CT scans.

RATIONALE AND OBJECTIVES: The purpose of this study was to analyze the variability of experienced thoracic radiologists in the identification of lung nodules on computed tomography (CT) scans and thereby to investigate variability in the establishment of the "truth" against which nodule-based studies are measured. MATERIALS AND METHODS: Thirty CT scans were reviewed twice by four thoracic radiologists through a two-phase image annotation process. During the initial "blinded read" phase, radiologists independently marked lesions they identified as "nodule >or=3 mm (diameter)," "nodule <3 mm," or "non-nodule >or=3 mm." During the subsequent "unblinded read" phase, the blinded read results of all four radiologists were revealed to each radiologist, who then independently reviewed their marks along with the anonymous marks of their colleagues; a radiologist's own marks then could be deleted, added, or left unchanged. This approach was developed to identify, as completely as possible, all nodules in a scan without requiring forced consensus. RESULTS: After the initial blinded read phase, 71 lesions received "nodule >or=3 mm" marks from at least one radiologist; however, all four radiologists assigned such marks to only 24 (33.8%) of these lesions. After the unblinded reads, a total of 59 lesions were marked as "nodule >or=3 mm" by at least one radiologist. Twenty-seven (45.8%) of these lesions received such marks from all four radiologists, three (5.1%) were identified as such by three radiologists, 12 (20.3%) were identified by two radiologists, and 17 (28.8%) were identified by only a single radiologist. CONCLUSION: The two-phase image annotation process yields improved agreement among radiologists in the interpretation of nodules >or=3 mm. Nevertheless, substantial variability remains across radiologists in the task of lung nodule identification.

Assessment of radiologist performance in the detection of lung nodules: dependence on the definition of "truth".

RATIONALE AND OBJECTIVES: Studies that evaluate the lung nodule detection performance of radiologists or computerized methods depend on an initial inventory of the nodules within the thoracic images (the "truth"). The purpose of this study was to analyze (1) variability in the "truth" defined by different combinations of experienced thoracic radiologists and (2) variability in the performance of other experienced thoracic radiologists based on these definitions of "truth" in the context of lung nodule detection in computed tomographic (CT) scans. MATERIALS AND METHODS: Twenty-five thoracic CT scans were reviewed by four thoracic radiologists, who independently marked lesions they considered to be nodules >or=3 mm in maximum diameter. Panel "truth" sets of nodules were then derived from the nodules marked by different combinations of two and three of these four radiologists. The nodule detection performance of the other radiologists was evaluated based on these panel "truth" sets. RESULTS: The number of "true" nodules in the different panel "truth" sets ranged from 15 to 89 (mean 49.8 +/- 25.6). The mean radiologist nodule detection sensitivities across radiologists and panel "truth" sets for different panel "truth" conditions ranged from 51.0 to 83.2%; mean false-positive rates ranged from 0.33 to 1.39 per case. CONCLUSIONS: Substantial variability exists across radiologists in the task of lung nodule identification in CT scans. The definition of "truth" on which lung nodule detection studies are based must be carefully considered, because even experienced thoracic radiologists may not perform well when measured against the "truth" established by other experienced thoracic radiologists.