Efficient adversarial debiasing with concept activation vector - Medical image case-studies.

BACKGROUND: A major hurdle for the real time deployment of the AI models is ensuring trustworthiness of these models for the unseen population. More often than not, these complex models are black boxes in which promising results are generated. However, when scrutinized, these models begin to reveal implicit biases during the decision making, particularly for the minority subgroups. METHOD: We develop an efficient adversarial de-biasing approach with partial learning by incorporating the existing concept activation vectors (CAV) methodology, to reduce racial disparities while preserving the performance of the targeted task. CAV is originally a model interpretability technique which we adopted to identify convolution layers responsible for learning race and only fine-tune up to that layer instead of fine-tuning the complete network, limiting the drop in performance RESULTS:: The methodology has been evaluated on two independent medical image case-studies - chest X-ray and mammograms, and we also performed external validation on a different racial population. On the external datasets for the chest X-ray use-case, debiased models (averaged AUC 0.87 ) outperformed the baseline convolution models (averaged AUC 0.57 ) as well as the models trained with the popular fine-tuning strategy (averaged AUC 0.81). Moreover, the mammogram models is debiased using a single dataset (white, black and Asian) and improved the performance on an external datasets (averaged AUC 0.8 to 0.86 ) with completely different population (primarily Hispanic patients). CONCLUSION: In this study, we demonstrated that the adversarial models trained only with internal data performed equally or often outperformed the standard fine-tuning strategy with data from an external setting. The adversarial training approach described can be applied regardless of predictor's model architecture, as long as the convolution model is trained using a gradient-based method. We release the training code with academic open-source license - https://github.com/ramon349/JBI2023_TCAV_debiasing.

Systematic Review of Generative Adversarial Networks (GANs) for Medical Image Classification and Segmentation.

In recent years, generative adversarial networks (GANs) have gained tremendous popularity for various imaging related tasks such as artificial image generation to support AI training. GANs are especially useful for medical imaging-related tasks where training datasets are usually limited in size and heavily imbalanced against the diseased class. We present a systematic review, following the PRISMA guidelines, of recent GAN architectures used for medical image analysis to help the readers in making an informed decision before employing GANs in developing medical image classification and segmentation models. We have extracted 54 papers that highlight the capabilities and application of GANs in medical imaging from January 2015 to August 2020 and inclusion criteria for meta-analysis. Our results show four main architectures of GAN that are used for segmentation or classification in medical imaging. We provide a comprehensive overview of recent trends in the application of GANs in clinical diagnosis through medical image segmentation and classification and ultimately share experiences for task-based GAN implementations.

Just in Time Radiology Decision Support Using Real-time Data Feeds.

Ready access to relevant real-time information in medical imaging offers several potential benefits. Knowing both when important information will be available and that important information is available can facilitate optimization of workflow and management of time. Unexpected findings, as well as deficiencies in reporting and documentation, can be immediately managed. Herein, we present our experience developing and implementing a real-time web-centric dashboard system for radiologists, clinicians, and support staff. The dashboards are driven by multi-sourced HL7 message streams that are monitored, analyzed, aggregated, and transformed into multiple real-time displays to improve operations within our department. We call this framework Pipeline. Ruby on Rails, JavaScript, HTML, and SQL serve as the foundations of the Pipeline application. HL7 messages are processed in real-time by a Mirth interface engine which posts exam data into SQL. Users utilize web browsers to visit the Ruby on Rails-based dashboards on any device connected to our hospital network. The dashboards will automatically refresh every 30 seconds using JavaScript. The Pipeline application has been well received by clinicians and radiologists.

Machine Learning for the Interventional Radiologist.

OBJECTIVE. The purpose of this article is to describe key potential areas of application of machine learning in interventional radiology. CONCLUSION. Machine learning, although in the early stages of development within the field of interventional radiology, has great potential to influence key areas such as image analysis, clinical predictive modeling, and trainee education. A proactive approach from current interventional radiologists and trainees is needed to shape future directions for machine learning and artificial intelligence.

A Platform for Innovation and Standards Evaluation: a Case Study from the OpenMRS Open-Source Radiology Information System.

Open-source development can provide a platform for innovation by seeking feedback from community members as well as providing tools and infrastructure to test new standards. Vendors of proprietary systems may delay adoption of new standards until there are sufficient incentives such as legal mandates or financial incentives to encourage/mandate adoption. Moreover, open-source systems in healthcare have been widely adopted in low- and middle-income countries and can be used to bridge gaps that exist in global health radiology. Since 2011, the authors, along with a community of open-source contributors, have worked on developing an open-source radiology information system (RIS) across two communities-OpenMRS and LibreHealth. The main purpose of the RIS is to implement core radiology workflows, on which others can build and test new radiology standards. This work has resulted in three major releases of the system, with current architectural changes driven by changing technology, development of new standards in health and imaging informatics, and changing user needs. At their core, both these communities are focused on building general-purpose EHR systems, but based on user contributions from the fringes, we have been able to create an innovative system that has been used by hospitals and clinics in four different countries. We provide an overview of the history of the LibreHealth RIS, the architecture of the system, overview of standards integration, describe challenges of developing an open-source product, and future directions. Our goal is to attract more participation and involvement to further develop the LibreHealth RIS into an Enterprise Imaging System that can be used in other clinical imaging including pathology and dermatology.

The limits of fair medical imaging AI in real-world generalization.

As artificial intelligence (AI) rapidly approaches human-level performance in medical imaging, it is crucial that it does not exacerbate or propagate healthcare disparities. Previous research established AI's capacity to infer demographic data from chest X-rays, leading to a key concern: do models using demographic shortcuts have unfair predictions across subpopulations? In this study, we conducted a thorough investigation into the extent to which medical AI uses demographic encodings, focusing on potential fairness discrepancies within both in-distribution training sets and external test sets. Our analysis covers three key medical imaging disciplines-radiology, dermatology and ophthalmology-and incorporates data from six global chest X-ray datasets. We confirm that medical imaging AI leverages demographic shortcuts in disease classification. Although correcting shortcuts algorithmically effectively addresses fairness gaps to create 'locally optimal' models within the original data distribution, this optimality is not true in new test settings. Surprisingly, we found that models with less encoding of demographic attributes are often most 'globally optimal', exhibiting better fairness during model evaluation in new test environments. Our work establishes best practices for medical imaging models that maintain their performance and fairness in deployments beyond their initial training contexts, underscoring critical considerations for AI clinical deployments across populations and sites.

Identifying and improving the "ground truth" of race in disparities research through improved EMR data reporting. A systematic review.

BACKGROUND: Studies about racial disparities in healthcare are increasing in quantity; however, they are subject to vast differences in definition, classification, and utilization of race/ethnicity data. Improved standardization of this information can strengthen conclusions drawn from studies using such data. The objective of this study is to examine how data related to race/ethnicity are recorded in research through examining articles on race/ethnicity health disparities and examine problems and solutions in data reporting that may impact overall data quality. METHODS: In this systematic review, Business Source Complete, Embase.com, IEEE Xplore, PubMed, Scopus and Web of Science Core Collection were searched for relevant articles published from 2000 to 2020. Search terms related to the concepts of electronic medical records, race/ethnicity, and data entry related to race/ethnicity were used. Exclusion criteria included articles not in the English language and those describing pediatric populations. Data were extracted from published articles. This review was organized and reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 statement for systematic reviews. FINDINGS: In this systematic review, 109 full text articles were reviewed. Weaknesses and possible solutions have been discussed in current literature, with the predominant problem and solution as follows: the electronic medical record (EMR) is vulnerable to inaccuracies and incompleteness in the methods that research staff collect this data; however, improved standardization of the collection and use of race data in patient care may help alleviate these inaccuracies. INTERPRETATION: Conclusions drawn from large datasets concerning peoples of certain race/ethnic groups should be made cautiously, and a careful review of the methodology of each publication should be considered prior to implementation in patient care.

Utility of Skin Tone on Pulse Oximetry in Critically Ill Patients: A Prospective Cohort Study.

OBJECTIVE: Pulse oximetry, a ubiquitous vital sign in modern medicine, has inequitable accuracy that disproportionately affects minority Black and Hispanic patients, with associated increases in mortality, organ dysfunction, and oxygen therapy. Previous retrospective studies used self-reported race or ethnicity as a surrogate for skin tone which is believed to be the root cause of the disparity. Our objective was to determine the utility of skin tone in explaining pulse oximetry discrepancies. DESIGN: Prospective cohort study. SETTING: Patients were eligible if they had pulse oximetry recorded up to 5 minutes before arterial blood gas (ABG) measurements. Skin tone was measured using administered visual scales, reflectance colorimetry, and reflectance spectrophotometry. PARTICIPANTS: Admitted hospital patients at Duke University Hospital. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: Sao2-Spo2 bias, variation of bias, and accuracy root mean square, comparing pulse oximetry, and ABG measurements. Linear mixed-effects models were fitted to estimate Sao2-Spo2 bias while accounting for clinical confounders.One hundred twenty-eight patients (57 Black, 56 White) with 521 ABG-pulse oximetry pairs were recruited. Skin tone data were prospectively collected using six measurement methods, generating eight measurements. The collected skin tone measurements were shown to yield differences among each other and overlap with self-reported racial groups, suggesting that skin tone could potentially provide information beyond self-reported race. Among the eight skin tone measurements in this study, and compared with self-reported race, the Monk Scale had the best relationship with differences in pulse oximetry bias (point estimate: -2.40%; 95% CI, -4.32% to -0.48%; p = 0.01) when comparing patients with lighter and dark skin tones. CONCLUSIONS: We found clinical performance differences in pulse oximetry, especially in darker skin tones. Additional studies are needed to determine the relative contributions of skin tone measures and other potential factors on pulse oximetry discrepancies.

Racial Differences in Accuracy of Predictive Models for High-Flow Nasal Cannula Failure in COVID-19.

OBJECTIVES: To develop and validate machine learning (ML) models to predict high-flow nasal cannula (HFNC) failure in COVID-19, compare their performance to the respiratory rate-oxygenation (ROX) index, and evaluate model accuracy by self-reported race. DESIGN: Retrospective cohort study. SETTING: Four Emory University Hospitals in Atlanta, GA. PATIENTS: Adult patients hospitalized with COVID-19 between March 2020 and April 2022 who received HFNC therapy within 24 hours of ICU admission were included. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: Four types of supervised ML models were developed for predicting HFNC failure (defined as intubation or death within 7 d of HFNC initiation), using routine clinical variables from the first 24 hours of ICU admission. Models were trained on the first 60% (n = 594) of admissions and validated on the latter 40% (n = 390) of admissions to simulate prospective implementation. Among 984 patients included, 317 patients (32.2%) developed HFNC failure. eXtreme Gradient Boosting (XGB) model had the highest area under the receiver-operator characteristic curve (AUROC) for predicting HFNC failure (0.707), and was the only model with significantly better performance than the ROX index (AUROC 0.616). XGB model had significantly worse performance in Black patients compared with White patients (AUROC 0.663 vs. 0.808, p = 0.02). Racial differences in the XGB model were reduced and no longer statistically significant when restricted to patients with nonmissing arterial blood gas data, and when XGB model was developed to predict mortality (rather than the composite outcome of failure, which could be influenced by biased clinical decisions for intubation). CONCLUSIONS: Our XGB model had better discrimination for predicting HFNC failure in COVID-19 than the ROX index, but had racial differences in accuracy of predictions. Further studies are needed to understand and mitigate potential sources of biases in clinical ML models and to improve their equitability.

Utility of skin tone on pulse oximetry in critically ill patients: a prospective cohort study.

Importance: Pulse oximetry, a ubiquitous vital sign in modern medicine, has inequitable accuracy that disproportionately affects Black and Hispanic patients, with associated increases in mortality, organ dysfunction, and oxygen therapy. Although the root cause of these clinical performance discrepancies is believed to be skin tone, previous retrospective studies used self-reported race or ethnicity as a surrogate for skin tone. Objective: To determine the utility of objectively measured skin tone in explaining pulse oximetry discrepancies. Design Setting and Participants: Admitted hospital patients at Duke University Hospital were eligible for this prospective cohort study if they had pulse oximetry recorded up to 5 minutes prior to arterial blood gas (ABG) measurements. Skin tone was measured across sixteen body locations using administered visual scales (Fitzpatrick Skin Type, Monk Skin Tone, and Von Luschan), reflectance colorimetry (Delfin SkinColorCatch [L*, individual typology angle {ITA}, Melanin Index {MI}]), and reflectance spectrophotometry (Konica Minolta CM-700D [L*], Variable Spectro 1 [L*]). Main Outcomes and Measures: Mean directional bias, variability of bias, and accuracy root mean square (ARMS), comparing pulse oximetry and ABG measurements. Linear mixed-effects models were fitted to estimate mean directional bias while accounting for clinical confounders. Results: 128 patients (57 Black, 56 White) with 521 ABG-pulse oximetry pairs were recruited, none with hidden hypoxemia. Skin tone data was prospectively collected using 6 measurement methods, generating 8 measurements. The collected skin tone measurements were shown to yield differences among each other and overlap with self-reported racial groups, suggesting that skin tone could potentially provide information beyond self-reported race. Among the eight skin tone measurements in this study, and compared to self-reported race, the Monk Scale had the best relationship with differences in pulse oximetry bias (point estimate: -2.40%; 95% CI: -4.32%, -0.48%; p=0.01) when comparing patients with lighter and dark skin tones. Conclusions and relevance: We found clinical performance differences in pulse oximetry, especially in darker skin tones. Additional studies are needed to determine the relative contributions of skin tone measures and other potential factors on pulse oximetry discrepancies.