Mitigating bias in deep learning for diagnosis of coronary artery disease from myocardial perfusion SPECT images.

PURPOSE: Artificial intelligence (AI) has high diagnostic accuracy for coronary artery disease (CAD) from myocardial perfusion imaging (MPI). However, when trained using high-risk populations (such as patients with correlating invasive testing), the disease probability can be overestimated due to selection bias. We evaluated different strategies for training AI models to improve the calibration (accurate estimate of disease probability), using external testing. METHODS: Deep learning was trained using 828 patients from 3 sites, with MPI and invasive angiography within 6 months. Perfusion was assessed using upright (U-TPD) and supine total perfusion deficit (S-TPD). AI training without data augmentation (model 1) was compared to training with augmentation (increased sampling) of patients without obstructive CAD (model 2), and patients without CAD and TPD < 2% (model 3). All models were tested in an external population of patients with invasive angiography within 6 months (n = 332) or low likelihood of CAD (n = 179). RESULTS: Model 3 achieved the best calibration (Brier score 0.104 vs 0.121, p < 0.01). Improvement in calibration was particularly evident in women (Brier score 0.084 vs 0.124, p < 0.01). In external testing (n = 511), the area under the receiver operating characteristic curve (AUC) was higher for model 3 (0.930), compared to U-TPD (AUC 0.897) and S-TPD (AUC 0.900, p < 0.01 for both). CONCLUSION: Training AI models with augmentation of low-risk patients can improve calibration of AI models developed to identify patients with CAD, allowing more accurate assignment of disease probability. This is particularly important in lower-risk populations and in women, where overestimation of disease probability could significantly influence down-stream patient management.

Outcome of Turf Toe Injuries in NFL Players.

The increase in artificial turf in the 1970s was thought to have contributed to increased incidence of turf toe injury in National Football League (NFL) players. To our knowledge, there are no publications that have analyzed the impact of this injury on performance. This is a retrospective case series. Online resources were used to identify NFL players who sustained a turf toe injury between the 2011 and 2014 seasons. The performance of each offensive skill player was analyzed separately by calculating their power rating (PR) over 6 seasons. Injured offensive skill players were then compared to a control group consisting of all RBs and WRs without a turf toe injury who competed in the 2012 season. Seventy-one turf toe injuries were identified. Twenty-nine occurred on grass, 29 on turf and the playing surface of 13 injuries could not be identified. The average PR prior to injury was 105.7/season (7.3/game), 87.3 (6.9/game) for the season of injury and 115.5 (8.1/ game) for postinjury seasons. The PR was not significantly different after a turf toe injury compared to before injury or to uninjured control player. There was no significant difference in NFL players' performances after turf toe injury based on power ratings.

Improved myocardial blood flow estimation with residual activity correction and motion correction in 18F-flurpiridaz PET myocardial perfusion imaging.

PURPOSE: We sought to evaluate the diagnostic performance for coronary artery disease (CAD) of myocardial blood flow (MBF) quantification with 18F-flurpiridaz PET using motion correction (MC) and residual activity correction (RAC). METHODS: In total, 231 patients undergoing same-day pharmacologic rest and stress 18F-flurpiridaz PET from Phase III Flurpiridaz trial (NCT01347710) were studied. Frame-by-frame MC was performed and RAC was accomplished by subtracting the rest residual counts from the dynamic stress polar maps. MBF and myocardial flow reserve (MFR) were derived with a two-compartment early kinetic model for the entire left ventricle (global), each coronary territory, and 17-segment. Global and minimal values of three territorial (minimal vessel) and segmental estimation (minimal segment) of stress MBF and MFR were evaluated in the prediction of CAD. MBF and MFR were evaluated with and without MC and RAC (1: no MC/no RAC, 2: no MC/RAC, 3: MC/RAC). RESULTS: The area-under the receiver operating characteristics curve (AUC [95% confidence interval]) of stress MBF with MC/RAC was higher for minimal segment (0.89 [0.85-0.94]) than for minimal vessel (0.86 [0.81-0.92], p = 0.03) or global estimation (0.81 [0.75-0.87], p < 0.0001). The AUC of MFR with MC/RAC was higher for minimal segment (0.87 [0.81-0.93]) than for minimal vessel (0.83 [0.76-0.90], p = 0.014) or global estimation (0.77 [0.69-0.84], p < 0.0001). The AUCs of minimal segment stress MBF and MFR with MC/RAC were higher compared to those with no MC/RAC (p < 0.001 for both) or no MC/no RAC (p < 0.0001 for both). CONCLUSIONS: Minimal segment MBF or MFR estimation with MC and RAC improves the diagnostic performance for obstructive CAD compared to global assessment.

Deep Learning for Explainable Estimation of Mortality Risk From Myocardial Positron Emission Tomography Images.

BACKGROUND: We aim to develop an explainable deep learning (DL) network for the prediction of all-cause mortality directly from positron emission tomography myocardial perfusion imaging flow and perfusion polar map data and evaluate it using prospective testing. METHODS: A total of 4735 consecutive patients referred for stress and rest 82Rb positron emission tomography between 2010 and 2018 were followed up for all-cause mortality for 4.15 (2.24-6.3) years. DL network utilized polar maps of stress and rest perfusion, myocardial blood flow, myocardial flow reserve, and spill-over fraction combined with cardiac volumes, singular indices, and sex. Patients scanned from 2010 to 2016 were used for training and validation. The network was tested in a set of 1135 patients scanned from 2017 to 2018 to simulate prospective clinical implementation. RESULTS: In prospective testing, the area under the receiver operating characteristic curve for all-cause mortality prediction by DL (0.82 [95% CI, 0.77-0.86]) was higher than ischemia (0.60 [95% CI, 0.54-0.66]; P <0.001), myocardial flow reserve (0.70 [95% CI, 0.64-0.76], P <0.001) or a comprehensive logistic regression model (0.75 [95% CI, 0.69-0.80], P <0.05). The highest quartile of patients by DL had an annual all-cause mortality rate of 11.87% and had a 16.8 ([95% CI, 6.12%-46.3%]; P <0.001)-fold increase in the risk of death compared with the lowest quartile patients. DL showed a 21.6% overall reclassification improvement as compared with established measures of ischemia. CONCLUSIONS: The DL model trained directly on polar maps allows improved patient risk stratification in comparison with established methods for positron emission tomography flow or perfusion assessments.

Handling missing values in machine learning to predict patient-specific risk of adverse cardiac events: Insights from REFINE SPECT registry.

BACKGROUND: Machine learning (ML) models can improve prediction of major adverse cardiovascular events (MACE), but in clinical practice some values may be missing. We evaluated the influence of missing values in ML models for patient-specific prediction of MACE risk. METHODS: We included 20,179 patients from the multicenter REFINE SPECT registry with MACE follow-up data. We evaluated seven methods for handling missing values: 1) removal of variables with missing values (ML-Remove), 2) imputation with median and unique category for continuous and categorical variables, respectively (ML-Traditional), 3) unique category for missing variables (ML-Unique), 4) cluster-based imputation (ML-Cluster), 5) regression-based imputation (ML-Regression), 6) missRanger imputation (ML-MR), and 7) multiple imputation (ML-MICE). We trained ML models with full data and simulated missing values in testing patients. Prediction performance was evaluated using area under the receiver-operating characteristic curve (AUC) and compared with a model without missing values (ML-All), expert visual diagnosis and total perfusion deficit (TPD). RESULTS: During mean follow-up of 4.7 ± 1.5 years, 3,541 patients experienced at least one MACE (3.7% annualized risk). ML-All (reference model-no missing values) had AUC 0.799 for MACE risk prediction. All seven models with missing values had lower AUC (ML-Remove: 0.778, ML-MICE: 0.774, ML-Cluster: 0.771, ML-Traditional: 0.771, ML-Regression: 0.770, ML-MR: 0.766, and ML-Unique: 0.766; p < 0.01 for ML-Remove vs remaining methods). Stress TPD (AUC 0.698) and visual diagnosis (0.681) had the lowest AUCs. CONCLUSION: Missing values reduce the accuracy of ML models when predicting MACE risk. Removing variables with missing values and retraining the model may yield superior patient-level prediction performance.

Explainable Deep Learning Improves Physician Interpretation of Myocardial Perfusion Imaging.

Artificial intelligence may improve accuracy of myocardial perfusion imaging (MPI) but will likely be implemented as an aid to physician interpretation rather than an autonomous tool. Deep learning (DL) has high standalone diagnostic accuracy for obstructive coronary artery disease (CAD), but its influence on physician interpretation is unknown. We assessed whether access to explainable DL predictions improves physician interpretation of MPI. Methods: We selected a representative cohort of patients who underwent MPI with reference invasive coronary angiography. Obstructive CAD, defined as stenosis ≥50% in the left main artery or ≥70% in other coronary segments, was present in half of the patients. We used an explainable DL model (CAD-DL), which was previously developed in a separate population from different sites. Three physicians interpreted studies first with clinical history, stress, and quantitative perfusion, then with all the data plus the DL results. Diagnostic accuracy was assessed using area under the receiver-operating-characteristic curve (AUC). Results: In total, 240 patients with a median age of 65 y (interquartile range 58-73) were included. The diagnostic accuracy of physician interpretation with CAD-DL (AUC 0.779) was significantly higher than that of physician interpretation without CAD-DL (AUC 0.747, P = 0.003) and stress total perfusion deficit (AUC 0.718, P < 0.001). With matched specificity, CAD-DL had higher sensitivity when operating autonomously compared with readers without DL results (P < 0.001), but not compared with readers interpreting with DL results (P = 0.122). All readers had numerically higher accuracy with CAD-DL, with AUC improvement 0.02-0.05, and interpretation with DL resulted in overall net reclassification improvement of 17.2% (95% CI 9.2%-24.4%, P < 0.001). Conclusion: Explainable DL predictions lead to meaningful improvements in physician interpretation; however, the improvement varied across the readers, reflecting the acceptance of this new technology. This technique could be implemented as an aid to physician diagnosis, improving the diagnostic accuracy of MPI.

Determining a minimum set of variables for machine learning cardiovascular event prediction: results from REFINE SPECT registry.

AIMS: Optimal risk stratification with machine learning (ML) from myocardial perfusion imaging (MPI) includes both clinical and imaging data. While most imaging variables can be derived automatically, clinical variables require manual collection, which is time-consuming and prone to error. We determined the fewest manually input and imaging variables required to maintain the prognostic accuracy for major adverse cardiac events (MACE) in patients undergoing a single-photon emission computed tomography (SPECT) MPI. METHODS AND RESULTS: This study included 20 414 patients from the multicentre REFINE SPECT registry and 2984 from the University of Calgary for training and external testing of the ML models, respectively. ML models were trained using all variables (ML-All) and all image-derived variables (including age and sex, ML-Image). Next, ML models were sequentially trained by incrementally adding manually input and imaging variables to baseline ML models based on their importance ranking. The fewest variables were determined as the ML models (ML-Reduced, ML-Minimum, and ML-Image-Reduced) that achieved comparable prognostic performance to ML-All and ML-Image. Prognostic accuracy of the ML models was compared with visual diagnosis, stress total perfusion deficit (TPD), and traditional multivariable models using area under the receiver-operating characteristic curve (AUC). ML-Minimum (AUC 0.798) obtained comparable prognostic accuracy to ML-All (AUC 0.799, P = 0.19) by including 12 of 40 manually input variables and 11 of 58 imaging variables. ML-Reduced achieved comparable accuracy (AUC 0.796) with a reduced set of manually input variables and all imaging variables. In external validation, the ML models also obtained comparable or higher prognostic accuracy than traditional multivariable models. CONCLUSION: Reduced ML models, including a minimum set of manually collected or imaging variables, achieved slightly lower accuracy compared to a full ML model but outperformed standard interpretation methods and risk models. ML models with fewer collected variables may be more practical for clinical implementation.

Clinical Deployment of Explainable Artificial Intelligence of SPECT for Diagnosis of Coronary Artery Disease.

BACKGROUND: Explainable artificial intelligence (AI) can be integrated within standard clinical software to facilitate the acceptance of the diagnostic findings during clinical interpretation. OBJECTIVES: This study sought to develop and evaluate a novel, general purpose, explainable deep learning model (coronary artery disease-deep learning [CAD-DL]) for the detection of obstructive CAD following single-photon emission computed tomography (SPECT) myocardial perfusion imaging (MPI). METHODS: A total of 3,578 patients with suspected CAD undergoing SPECT MPI and invasive coronary angiography within a 6-month interval from 9 centers were studied. CAD-DL computes the probability of obstructive CAD from stress myocardial perfusion, wall motion, and wall thickening maps, as well as left ventricular volumes, age, and sex. Myocardial regions contributing to the CAD-DL prediction are highlighted to explain the findings to the physician. A clinical prototype was integrated using a standard clinical workstation. Diagnostic performance by CAD-DL was compared to automated quantitative total perfusion deficit (TPD) and reader diagnosis. RESULTS: In total, 2,247 patients (63%) had obstructive CAD. In 10-fold repeated testing, the area under the receiver-operating characteristic curve (AUC) (95% CI) was higher according to CAD-DL (AUC: 0.83 [95% CI: 0.82-0.85]) than stress TPD (AUC: 0.78 [95% CI: 0.77-0.80]) or reader diagnosis (AUC: 0.71 [95% CI: 0.69-0.72]; P < 0.0001 for both). In external testing, the AUC in 555 patients was higher according to CAD-DL (AUC: 0.80 [95% CI: 0.76-0.84]) than stress TPD (AUC: 0.73 [95% CI: 0.69-0.77]) or reader diagnosis (AUC: 0.65 [95% CI: 0.61-0.69]; P < 0.001 for all). The present model can be integrated within standard clinical software and generates results rapidly (<12 seconds on a standard clinical workstation) and therefore could readily be incorporated into a typical clinical workflow. CONCLUSIONS: The deep-learning model significantly surpasses the diagnostic accuracy of standard quantitative analysis and clinical visual reading for MPI. Explainable artificial intelligence can be integrated within standard clinical software to facilitate acceptance of artificial intelligence diagnosis of CAD following MPI.

Prognostic Value of Phase Analysis for Predicting Adverse Cardiac Events Beyond Conventional Single-Photon Emission Computed Tomography Variables: Results From the REFINE SPECT Registry.

BACKGROUND: Phase analysis of single-photon emission computed tomography myocardial perfusion imaging provides dyssynchrony information which correlates well with assessments by echocardiography, but the independent prognostic significance is not well defined. This study assessed the independent prognostic value of single-photon emission computed tomography-myocardial perfusion imaging phase analysis in the largest multinational registry to date across all modalities. METHODS: From the REFINE SPECT (Registry of Fast Myocardial Perfusion Imaging With Next Generation SPECT), a total of 19 210 patients were included (mean age 63.8±12.0 years and 56% males). Poststress total perfusion deficit, left ventricular ejection fraction, and phase variables (phase entropy, bandwidth, and SD) were obtained automatically. Cox proportional hazards analyses were performed to assess associations with major adverse cardiac events (MACE). RESULTS: During a follow-up of 4.5±1.7 years, 2673 (13.9%) patients experienced MACE. Annualized MACE rates increased with phase variables and were ≈4-fold higher between the second and highest decile group for entropy (1.7% versus 6.7%). Optimal phase variable cutoff values stratified MACE risk in patients with normal and abnormal total perfusion deficit and left ventricular ejection fraction. Only entropy was independently associated with MACE. The addition of phase entropy significantly improved the discriminatory power for MACE prediction when added to the model with total perfusion deficit and left ventricular ejection fraction (P<0.0001). CONCLUSIONS: In a largest to date imaging study, widely representative, international cohort, phase variables were independently associated with MACE and improved risk stratification for MACE beyond the prediction by perfusion and left ventricular ejection fraction assessment alone. Phase analysis can be obtained fully automatically, without additional radiation exposure or cost to improve MACE risk prediction and, therefore, should be routinely reported for single-photon emission computed tomography-myocardial perfusion imaging studies.