COVID-19 on Chest Radiographs: A Multireader Evaluation of an Artificial Intelligence System.

Background Chest radiography may play an important role in triage for coronavirus disease 2019 (COVID-19), particularly in low-resource settings. Purpose To evaluate the performance of an artificial intelligence (AI) system for detection of COVID-19 pneumonia on chest radiographs. Materials and Methods An AI system (CAD4COVID-XRay) was trained on 24 678 chest radiographs, including 1540 used only for validation while training. The test set consisted of a set of continuously acquired chest radiographs (n = 454) obtained in patients suspected of having COVID-19 pneumonia between March 4 and April 6, 2020, at one center (223 patients with positive reverse transcription polymerase chain reaction [RT-PCR] results, 231 with negative RT-PCR results). Radiographs were independently analyzed by six readers and by the AI system. Diagnostic performance was analyzed with the receiver operating characteristic curve. Results For the test set, the mean age of patients was 67 years ± 14.4 (standard deviation) (56% male). With RT-PCR test results as the reference standard, the AI system correctly classified chest radiographs as COVID-19 pneumonia with an area under the receiver operating characteristic curve of 0.81. The system significantly outperformed each reader (P < .001 using the McNemar test) at their highest possible sensitivities. At their lowest sensitivities, only one reader significantly outperformed the AI system (P = .04). Conclusion The performance of an artificial intelligence system in the detection of coronavirus disease 2019 on chest radiographs was comparable with that of six independent readers. © RSNA, 2020.

Computer aided detection of tuberculosis on chest radiographs: An evaluation of the CAD4TB v6 system.

There is a growing interest in the automated analysis of chest X-Ray (CXR) as a sensitive and inexpensive means of screening susceptible populations for pulmonary tuberculosis. In this work we evaluate the latest version of CAD4TB, a commercial software platform designed for this purpose. Version 6 of CAD4TB was released in 2018 and is here tested on a fully independent dataset of 5565 CXR images with GeneXpert (Xpert) sputum test results available (854 Xpert positive subjects). A subset of 500 subjects (50% Xpert positive) was reviewed and annotated by 5 expert observers independently to obtain a radiological reference standard. The latest version of CAD4TB is found to outperform all previous versions in terms of area under receiver operating curve (ROC) with respect to both Xpert and radiological reference standards. Improvements with respect to Xpert are most apparent at high sensitivity levels with a specificity of 76% obtained at a fixed 90% sensitivity. When compared with the radiological reference standard, CAD4TB v6 also outperformed previous versions by a considerable margin and achieved 98% specificity at the 90% sensitivity setting. No substantial difference was found between the performance of CAD4TB v6 and any of the various expert observers against the Xpert reference standard. A cost and efficiency analysis on this dataset demonstrates that in a standard clinical situation, operating at 90% sensitivity, users of CAD4TB v6 can process 132 subjects per day at an average cost per screen of $5.95 per subject, while users of version 3 process only 85 subjects per day at a cost of $8.38 per subject. At all tested operating points version 6 is shown to be more efficient and cost effective than any other version.

Computer-aided diagnosis for World Health Organization-defined chest radiograph primary-endpoint pneumonia in children.

BACKGROUND: The chest radiograph is the most common imaging modality to assess childhood pneumonia. It has been used in epidemiological and vaccine efficacy/effectiveness studies on childhood pneumonia. OBJECTIVE: To develop computer-aided diagnosis (CAD4Kids) for chest radiography in children and to evaluate its accuracy in identifying World Health Organization (WHO)-defined chest radiograph primary-endpoint pneumonia compared to a consensus interpretation. MATERIALS AND METHODS: Chest radiographs were independently evaluated by three radiologists based on WHO criteria. Automatic lung field segmentation was followed by manual inspection and correction, training, feature extraction and classification. Radiographs were filtered with Gaussian derivatives on multiple scales, extracting texture features to classify each pixel in the lung region. To obtain an image score, the 95th percentile score of the pixels was used. Training and testing were done in 10-fold cross validation. RESULTS: The radiologist majority consensus reading of 858 interpretable chest radiographs included 333 (39%) categorised as primary-endpoint pneumonia, 208 (24%) as other infiltrate only and 317 (37%) as no primary-endpoint pneumonia or other infiltrate. Compared to the reference radiologist consensus reading, CAD4Kids had an area under the receiver operator characteristic (ROC) curve of 0.850 (95% confidence interval [CI] 0.823-0.876), with a sensitivity of 76% and specificity of 80% for identifying primary-endpoint pneumonia on chest radiograph. Furthermore, the ROC curve was 0.810 (95% CI 0.772-0.846) for CAD4Kids identifying primary-endpoint pneumonia compared to other infiltrate only. CONCLUSION: Further development of the CAD4Kids software and validation in multicentre studies are important for future research on computer-aided diagnosis and artificial intelligence in paediatric radiology.

Fast and effective quantification of symmetry in medical images for pathology detection: Application to chest radiography.

PURPOSE: Symmetry is an important feature of human anatomy and the absence of symmetry in medical images can indicate the presence of pathology. Quantification of image symmetry can then be used to improve the automatic analysis of medical images. METHODS: A method is presented that computes both local and global symmetry in 2D medical images. A symmetry axis is determined to define for each position p in the image a mirrored position p' on the contralateral side of the axis. In the neighborhood of p', an optimally corresponding position ps is determined by minimizing a cost function d that combines intensity differences in a patch around p and the mirrored patch around ps and the spatial distance between p' and ps. The optimal value of d is used as a measure of local symmetry s. The average of all values of s, indicated as S, quantifies global symmetry. Starting from an initial approximation of the symmetry axis, the optimal orientation and position of the axis is determined by greedy minimization of S. RESULTS: The method was evaluated in three experiments concerning abnormality detection in frontal chest radiographs. In the first experiment, global symmetry S was used to discriminate between 174 normal images and 174 images containing diffuse textural abnormalities from the publicly available CRASS database of tuberculosis suspects. Performance, measured as area under the receiver operating characteristic curve Az was 0.838. The second experiment investigated whether adding the local symmetry s as an additional feature to a set of 106 texture features resulted in improvements in classifying local patches in the same image database. We found that Az increased from 0.878 to 0.891 (P = 0.001). In the third experiment, it was shown that the contrast of pulmonary nodules, obtained from the publicly available JSRT database, increased significantly in the local symmetry map compared to the original image. CONCLUSIONS: We conclude that the proposed algorithm for symmetry computation provides informative features which can be used to improve abnormality detection in medical images both at a local and a global level.

An automated tuberculosis screening strategy combining X-ray-based computer-aided detection and clinical information.

Lack of human resources and radiological interpretation expertise impair tuberculosis (TB) screening programmes in TB-endemic countries. Computer-aided detection (CAD) constitutes a viable alternative for chest radiograph (CXR) reading. However, no automated techniques that exploit the additional clinical information typically available during screening exist. To address this issue and optimally exploit this information, a machine learning-based combination framework is introduced. We have evaluated this framework on a database containing 392 patient records from suspected TB subjects prospectively recruited in Cape Town, South Africa. Each record comprised a CAD score, automatically computed from a CXR, and 12 clinical features. Comparisons with strategies relying on either CAD scores or clinical information alone were performed. Our results indicate that the combination framework outperforms the individual strategies in terms of the area under the receiving operating characteristic curve (0.84 versus 0.78 and 0.72), specificity at 95% sensitivity (49% versus 24% and 31%) and negative predictive value (98% versus 95% and 96%). Thus, it is believed that combining CAD and clinical information to estimate the risk of active disease is a promising tool for TB screening.

On Combining Multiple-Instance Learning and Active Learning for Computer-Aided Detection of Tuberculosis.

The major advantage of multiple-instance learning (MIL) applied to a computer-aided detection (CAD) system is that it allows optimizing the latter with case-level labels instead of accurate lesion outlines as traditionally required for a supervised approach. As shown in previous work, a MIL-based CAD system can perform comparably to its supervised counterpart considering complex tasks such as chest radiograph scoring in tuberculosis (TB) detection. However, despite this remarkable achievement, the uncertainty inherent to MIL can lead to a less satisfactory outcome if analysis at lower levels (e.g., regions or pixels) is needed. This issue may seriously compromise the applicability of MIL to tasks related to quantification or grading, or detection of highly localized lesions. In this paper, we propose to reduce uncertainty by embedding a MIL classifier within an active learning (AL) framework. To minimize the labeling effort, we develop a novel instance selection mechanism that exploits the MIL problem definition through one-class classification. We adapt this mechanism to provide meaningful regions instead of individual instances for expert labeling, which is a more appropriate strategy given the application domain. In addition, and contrary to usual AL methods, a single iteration is performed. To show the effectiveness of our approach, we compare the output of a MIL-based CAD system trained with and without the proposed AL framework. The task is to detect textural abnormalities related to TB. Both quantitative and qualitative evaluations at the pixel level are carried out. Our method significantly improves the MIL-based classification.

A novel multiple-instance learning-based approach to computer-aided detection of tuberculosis on chest X-rays.

To reach performance levels comparable to human experts, computer-aided detection (CAD) systems are typically optimized following a supervised learning approach that relies on large training databases comprising manually annotated lesions. However, manually outlining those lesions constitutes a difficult and time-consuming process that renders detailedly annotated data difficult to obtain. In this paper, we investigate an alternative approach, namely multiple-instance learning (MIL), that does not require detailed information for optimization. We have applied MIL to a CAD system for tuberculosis detection. Only the case condition (normal or abnormal) was required during training. Based upon the well-known miSVM technique, we propose an improved algorithm that overcomes miSVM's drawbacks related to positive instance underestimation and costly iteration. To show the advantages of our MIL-based approach as compared with a traditional supervised one, experiments with three X-ray databases were conducted. The area under the receiver operating characteristic curve was utilized as a performance measure. With the first database, for which training lesion annotations were available, our MIL-based method was comparable to the supervised system ( 0.86 versus 0.88 ). When evaluating the remaining databases, given their large difference with the previous image set, the most appealing strategy was to retrain the CAD systems. However, since only the case condition was available, only the MIL-based system could be retrained. This scenario, which is common in real-world applications, demonstrates the better adaptation capabilities of the proposed approach. After retraining, our MIL-based system significantly outperformed the supervised one ( 0.86 versus 0.79 and 0.91 versus 0.85 , and p=0.0002 , respectively).

Diagnostic accuracy of computer-aided detection of pulmonary tuberculosis in chest radiographs: a validation study from sub-Saharan Africa.

BACKGROUND: Chest radiography to diagnose and screen for pulmonary tuberculosis has limitations, especially due to inter-reader variability. Automating the interpretation has the potential to overcome this drawback and to deliver objective and reproducible results. The CAD4TB software is a computer-aided detection system that has shown promising preliminary findings. Evaluation studies in different settings are needed to assess diagnostic accuracy and practicability of use. METHODS: CAD4TB was evaluated on chest radiographs of patients with symptoms suggestive of pulmonary tuberculosis enrolled in two cohort studies in Tanzania. All patients were characterized by sputum smear microscopy and culture including subsequent antigen or molecular confirmation of Mycobacterium tuberculosis (M.tb) to determine the reference standard. Chest radiographs were read by the software and two human readers, one expert reader and one clinical officer. The sensitivity and specificity of CAD4TB was depicted using receiver operating characteristic (ROC) curves, the area under the curve calculated and the performance of the software compared to the results of human readers. RESULTS: Of 861 study participants, 194 (23%) were culture-positive for M.tb. The area under the ROC curve of CAD4TB for the detection of culture-positive pulmonary tuberculosis was 0.84 (95% CI 0.80-0.88). CAD4TB was significantly more accurate for the discrimination of smear-positive cases against non TB patients than for smear-negative cases (p-value<0.01). It differentiated better between TB cases and non TB patients among HIV-negative compared to HIV-positive individuals (p<0.01). CAD4TB significantly outperformed the clinical officer, but did not reach the accuracy of the expert reader (p = 0.02), for a tuberculosis specific reading threshold. CONCLUSION: CAD4TB accurately distinguished between the chest radiographs of culture-positive TB cases and controls. Further studies on cost-effectiveness, operational and ethical aspects should determine its place in diagnostic and screening algorithms.