Crowdsourcing pneumothorax annotations using machine learning annotations on the NIH chest X-ray dataset.

Pneumothorax is a potentially life-threatening condition that requires prompt recognition and often urgent intervention. In the ICU setting, large numbers of chest radiographs are performed and must be interpreted on a daily basis which may delay diagnosis of this entity. Development of artificial intelligence (AI) techniques to detect pneumothorax could help expedite detection as well as localize and potentially quantify pneumothorax. Open image analysis competitions are useful in advancing state-of-the art AI algorithms but generally require large expert annotated datasets. We have annotated and adjudicated a large dataset of chest radiographs to be made public with the goal of sparking innovation in this space. Because of the cumbersome and time-consuming nature of image labeling, we explored the value of using AI models to generate annotations for review. Utilization of this machine learning annotation (MLA) technique appeared to expedite our annotation process with relatively high sensitivity at the expense of specificity. Further research is required to confirm and better characterize the value of MLAs. Our adjudicated dataset is now available for public consumption in the form of a challenge.

Deep Learning-Based Digitally Reconstructed Tomography of the Chest in the Evaluation of Solitary Pulmonary Nodules: A Feasibility Study.

RATIONALE AND OBJECTIVES: Computed tomography (CT) is preferred for evaluating solitary pulmonary nodules (SPNs) but access or availability may be lacking, in addition, overlapping anatomy can hinder detection of SPNs on chest radiographs. We developed and evaluated the clinical feasibility of a deep learning algorithm to generate digitally reconstructed tomography (DRT) images of the chest from digitally reconstructed frontal and lateral radiographs (DRRs) and use them to detect SPNs. METHODS: This single-institution retrospective study included 637 patients with noncontrast helical CT of the chest (mean age 68 years, median age 69 years, standard deviation 11.7 years; 355 women) between 11/2012 and 12/2020, with SPNs measuring 10-30 mm. A deep learning model was trained on 562 patients, validated on 60 patients, and tested on the remaining 15 patients. Diagnostic performance (SPN detection) from planar radiography (DRRs and CT scanograms, PR) alone or with DRT was evaluated by two radiologists in an independent blinded fashion. The quality of the DRT SPN image in terms of nodule size and location, morphology, and opacity was also evaluated, and compared to the ground-truth CT images RESULTS: Diagnostic performance was higher from DRT plus PR than from PR alone (area under the receiver operating characteristic curve 0.95-0.98 versus 0.80-0.85; p < 0.05). DRT plus PR enabled diagnosis of SPNs in 11 more patients than PR alone. Interobserver agreement was 0.82 for DRT plus PR and 0.89 for PR alone; and interobserver agreement for size and location, morphology, and opacity of the DRT SPN was 0.94, 0.68, and 0.38, respectively. CONCLUSION: For SPN detection, DRT plus PR showed better diagnostic performance than PR alone. Deep learning can be used to generate DRT images and improve detection of SPNs.

Opportunistic detection of type 2 diabetes using deep learning from frontal chest radiographs.

Deep learning (DL) models can harness electronic health records (EHRs) to predict diseases and extract radiologic findings for diagnosis. With ambulatory chest radiographs (CXRs) frequently ordered, we investigated detecting type 2 diabetes (T2D) by combining radiographic and EHR data using a DL model. Our model, developed from 271,065 CXRs and 160,244 patients, was tested on a prospective dataset of 9,943 CXRs. Here we show the model effectively detected T2D with a ROC AUC of 0.84 and a 16% prevalence. The algorithm flagged 1,381 cases (14%) as suspicious for T2D. External validation at a distinct institution yielded a ROC AUC of 0.77, with 5% of patients subsequently diagnosed with T2D. Explainable AI techniques revealed correlations between specific adiposity measures and high predictivity, suggesting CXRs' potential for enhanced T2D screening.

Detecting Racial/Ethnic Health Disparities Using Deep Learning From Frontal Chest Radiography.

PURPOSE: The aim of this study was to assess racial/ethnic and socioeconomic disparities in the difference between atherosclerotic vascular disease prevalence measured by a multitask convolutional neural network (CNN) deep learning model using frontal chest radiographs (CXRs) and the prevalence reflected by administrative hierarchical condition category codes in two cohorts of patients with coronavirus disease 2019 (COVID-19). METHODS: A CNN model, previously published, was trained to predict atherosclerotic disease from ambulatory frontal CXRs. The model was then validated on two cohorts of patients with COVID-19: 814 ambulatory patients from a suburban location (presenting from March 14, 2020, to October 24, 2020, the internal ambulatory cohort) and 485 hospitalized patients from an inner-city location (hospitalized from March 14, 2020, to August 12, 2020, the external hospitalized cohort). The CNN model predictions were validated against electronic health record administrative codes in both cohorts and assessed using the area under the receiver operating characteristic curve (AUC). The CXRs from the ambulatory cohort were also reviewed by two board-certified radiologists and compared with the CNN-predicted values for the same cohort to produce a receiver operating characteristic curve and the AUC. The atherosclerosis diagnosis discrepancy, Δvasc, referring to the difference between the predicted value and presence or absence of the vascular disease HCC categorical code, was calculated. Linear regression was performed to determine the association of Δvasc with the covariates of age, sex, race/ethnicity, language preference, and social deprivation index. Logistic regression was used to look for an association between the presence of any hierarchical condition category codes with Δvasc and other covariates. RESULTS: The CNN prediction for vascular disease from frontal CXRs in the ambulatory cohort had an AUC of 0.85 (95% confidence interval, 0.82-0.89) and in the hospitalized cohort had an AUC of 0.69 (95% confidence interval, 0.64-0.75) against the electronic health record data. In the ambulatory cohort, the consensus radiologists' reading had an AUC of 0.89 (95% confidence interval, 0.86-0.92) relative to the CNN. Multivariate linear regression of Δvasc in the ambulatory cohort demonstrated significant negative associations with non-English-language preference (ß = -0.083, P < .05) and Black or Hispanic race/ethnicity (ß = -0.048, P < .05) and positive associations with age (ß = 0.005, P < .001) and sex (ß = 0.044, P < .05). For the hospitalized cohort, age was also significant (ß = 0.003, P < .01), as was social deprivation index (ß = 0.002, P < .05). The Δvasc variable (odds ratio [OR], 0.34), Black or Hispanic race/ethnicity (OR, 1.58), non-English-language preference (OR, 1.74), and site (OR, 0.22) were independent predictors of having one or more hierarchical condition category codes (P < .01 for all) in the combined patient cohort. CONCLUSIONS: A CNN model was predictive of aortic atherosclerosis in two cohorts (one ambulatory and one hospitalized) with COVID-19. The discrepancy between the CNN model and the administrative code, Δvasc, was associated with language preference in the ambulatory cohort; in the hospitalized cohort, this discrepancy was associated with social deprivation index. The absence of administrative code(s) was associated with Δvasc in the combined cohorts, suggesting that Δvasc is an independent predictor of health disparities. This may suggest that biomarkers extracted from routine imaging studies and compared with electronic health record data could play a role in enhancing value-based health care for traditionally underserved or disadvantaged patients for whom barriers to care exist.

Validation of a deep learning, value-based care model to predict mortality and comorbidities from chest radiographs in COVID-19.

We validate a deep learning model predicting comorbidities from frontal chest radiographs (CXRs) in patients with coronavirus disease 2019 (COVID-19) and compare the model's performance with hierarchical condition category (HCC) and mortality outcomes in COVID-19. The model was trained and tested on 14,121 ambulatory frontal CXRs from 2010 to 2019 at a single institution, modeling select comorbidities using the value-based Medicare Advantage HCC Risk Adjustment Model. Sex, age, HCC codes, and risk adjustment factor (RAF) score were used. The model was validated on frontal CXRs from 413 ambulatory patients with COVID-19 (internal cohort) and on initial frontal CXRs from 487 COVID-19 hospitalized patients (external cohort). The discriminatory ability of the model was assessed using receiver operating characteristic (ROC) curves compared to the HCC data from electronic health records, and predicted age and RAF score were compared using correlation coefficient and absolute mean error. The model predictions were used as covariables in logistic regression models to evaluate the prediction of mortality in the external cohort. Predicted comorbidities from frontal CXRs, including diabetes with chronic complications, obesity, congestive heart failure, arrhythmias, vascular disease, and chronic obstructive pulmonary disease, had a total area under ROC curve (AUC) of 0.85 (95% CI: 0.85-0.86). The ROC AUC of predicted mortality for the model was 0.84 (95% CI,0.79-0.88) for the combined cohorts. This model using only frontal CXRs predicted select comorbidities and RAF score in both internal ambulatory and external hospitalized COVID-19 cohorts and was discriminatory of mortality, supporting its potential use in clinical decision making.

Machine Learning Principles for Radiology Investigators.

Artificial intelligence and deep learning are areas of high interest for radiology investigators at present. However, the field of machine learning encompasses multiple statistics-based techniques useful for investigators, which may be complementary to deep learning approaches. After a refresher in basic statistical concepts, relevant considerations for machine learning practitioners are reviewed: regression, classification, decision boundaries, and bias-variance tradeoff. Regularization, ground truth, and populations are discussed along with compute and data management principles. Advanced statistical machine learning techniques including bootstrapping, bagging, boosting, decision trees, random forest, XGboost, and support vector machines are reviewed along with relevant examples from the radiology literature.