Causal machine learning for predicting treatment outcomes.

Causal machine learning (ML) offers flexible, data-driven methods for predicting treatment outcomes including efficacy and toxicity, thereby supporting the assessment and safety of drugs. A key benefit of causal ML is that it allows for estimating individualized treatment effects, so that clinical decision-making can be personalized to individual patient profiles. Causal ML can be used in combination with both clinical trial data and real-world data, such as clinical registries and electronic health records, but caution is needed to avoid biased or incorrect predictions. In this Perspective, we discuss the benefits of causal ML (relative to traditional statistical or ML approaches) and outline the key components and steps. Finally, we provide recommendations for the reliable use of causal ML and effective translation into the clinic.

Machine learning for healthcare that matters: Reorienting from technical novelty to equitable impact.

Despite significant technical advances in machine learning (ML) over the past several years, the tangible impact of this technology in healthcare has been limited. This is due not only to the particular complexities of healthcare, but also due to structural issues in the machine learning for healthcare (MLHC) community which broadly reward technical novelty over tangible, equitable impact. We structure our work as a healthcare-focused echo of the 2012 paper "Machine Learning that Matters", which highlighted such structural issues in the ML community at large, and offered a series of clearly defined "Impact Challenges" to which the field should orient itself. Drawing on the expertise of a diverse and international group of authors, we engage in a narrative review and examine issues in the research background environment, training processes, evaluation metrics, and deployment protocols which act to limit the real-world applicability of MLHC. Broadly, we seek to distinguish between machine learning ON healthcare data and machine learning FOR healthcare-the former of which sees healthcare as merely a source of interesting technical challenges, and the latter of which regards ML as a tool in service of meeting tangible clinical needs. We offer specific recommendations for a series of stakeholders in the field, from ML researchers and clinicians, to the institutions in which they work, and the governments which regulate their data access.

U-PASS: An uncertainty-guided deep learning pipeline for automated sleep staging.

With the increasing prevalence of machine learning in critical fields like healthcare, ensuring the safety and reliability of these systems is crucial. Estimating uncertainty plays a vital role in enhancing reliability by identifying areas of high and low confidence and reducing the risk of errors. This study introduces U-PASS, a specialized human-centered machine learning pipeline tailored for clinical applications, which effectively communicates uncertainty to clinical experts and collaborates with them to improve predictions. U-PASS incorporates uncertainty estimation at every stage of the process, including data acquisition, training, and model deployment. Training is divided into a supervised pre-training step and a semi-supervised recording-wise finetuning step. We apply U-PASS to the challenging task of sleep staging and demonstrate that it systematically improves performance at every stage. By optimizing the training dataset, actively seeking feedback from domain experts for informative samples, and deferring the most uncertain samples to experts, U-PASS achieves an impressive expert-level accuracy of 85% on a challenging clinical dataset of elderly sleep apnea patients. This represents a significant improvement over the starting point at 75% accuracy. The largest improvement gain is due to the deferral of uncertain epochs to a sleep expert. U-PASS presents a promising AI approach to incorporating uncertainty estimation in machine learning pipelines, improving their reliability and unlocking their potential in clinical settings.

Factors influencing clinician and patient interaction with machine learning-based risk prediction models: a systematic review.

Machine learning (ML)-based risk prediction models hold the potential to support the health-care setting in several ways; however, use of such models is scarce. We aimed to review health-care professional (HCP) and patient perceptions of ML risk prediction models in published literature, to inform future risk prediction model development. Following database and citation searches, we identified 41 articles suitable for inclusion. Article quality varied with qualitative studies performing strongest. Overall, perceptions of ML risk prediction models were positive. HCPs and patients considered that models have the potential to add benefit in the health-care setting. However, reservations remain; for example, concerns regarding data quality for model development and fears of unintended consequences following ML model use. We identified that public views regarding these models might be more negative than HCPs and that concerns (eg, extra demands on workload) were not always borne out in practice. Conclusions are tempered by the low number of patient and public studies, the absence of participant ethnic diversity, and variation in article quality. We identified gaps in knowledge (particularly views from under-represented groups) and optimum methods for model explanation and alerts, which require future research.

Linear Deconfounded Score Method: Scoring DAGs With Dense Unobserved Confounding.

This article deals with the discovery of causal relations from a combination of observational data and qualitative assumptions about the nature of causality in the presence of unmeasured confounding. We focus on applications where unobserved variables are known to have a widespread effect on many of the observed ones, which makes the problem particularly difficult for constraint-based methods, because most pairs of variables are conditionally dependent given any other subset, rendering the causal effect unidentifiable. In this article, we show that under the principle of independent mechanisms, unobserved confounding in this setting leaves a statistical footprint in the observed data distribution that allows for disentangling spurious and causal effects. Using this insight, we demonstrate that a sparse linear Gaussian directed acyclic graph (DAG) among observed variables may be recovered approximately and propose a simple adjusted score-based causal discovery algorithm that may be implemented with general-purpose solvers and scales to high-dimensional problems. We find, in addition, that despite the conditions we pose to guarantee causal recovery, performance in practice is robust to large deviations in model assumptions, and extensions to nonlinear structural models are possible.

Using Machine Learning to Individualize Treatment Effect Estimation: Challenges and Opportunities.

The use of data from randomized clinical trials to justify treatment decisions for real-world patients is the current state of the art. It relies on the assumption that average treatment effects from the trial can be extrapolated to patients with personal and/or disease characteristics different from those treated in the trial. Yet, because of heterogeneity of treatment effects between patients and between the trial population and real-world patients, this assumption may not be correct for many patients. Using machine learning to estimate the expected conditional average treatment effect (CATE) in individual patients from observational data offers the potential for more accurate estimation of the expected treatment effects in each patient based on their observed characteristics. In this review, we discuss some of the challenges and opportunities for machine learning to estimate CATE, including ensuring identification assumptions are met, managing covariate shift, and learning without access to the true label of interest. We also discuss the potential applications as well as future work and collaborations needed to further improve identification and utilization of CATE estimates to increase patient benefit.

Novel Preoperative Risk Stratification Using Digital Phenotyping Applying a Scalable Machine-Learning Approach.

BACKGROUND: Classification of perioperative risk is important for patient care, resource allocation, and guiding shared decision-making. Using discriminative features from the electronic health record (EHR), machine-learning algorithms can create digital phenotypes among heterogenous populations, representing distinct patient subpopulations grouped by shared characteristics, from which we can personalize care, anticipate clinical care trajectories, and explore therapies. We hypothesized that digital phenotypes in preoperative settings are associated with postoperative adverse events including in-hospital and 30-day mortality, 30-day surgical redo, intensive care unit (ICU) admission, and hospital length of stay (LOS). METHODS: We identified all laminectomies, colectomies, and thoracic surgeries performed over a 9-year period from a large hospital system. Seventy-seven readily extractable preoperative features were first selected from clinical consensus, including demographics, medical history, and lab results. Three surgery-specific datasets were built and split into derivation and validation cohorts using chronological occurrence. Consensus k -means clustering was performed independently on each derivation cohort, from which phenotypes' characteristics were explored. Cluster assignments were used to train a random forest model to assign patient phenotypes in validation cohorts. We reconducted descriptive analyses on validation cohorts to confirm the similarity of patient characteristics with derivation cohorts, and quantified the association of each phenotype with postoperative adverse events by using the area under receiver operating characteristic curve (AUROC). We compared our approach to American Society of Anesthesiologists (ASA) alone and investigated a combination of our phenotypes with the ASA score. RESULTS: A total of 7251 patients met inclusion criteria, of which 2770 were held out in a validation dataset based on chronological occurrence. Using segmentation metrics and clinical consensus, 3 distinct phenotypes were created for each surgery. The main features used for segmentation included urgency of the procedure, preoperative LOS, age, and comorbidities. The most relevant characteristics varied for each of the 3 surgeries. Low-risk phenotype alpha was the most common (2039 of 2770, 74%), while high-risk phenotype gamma was the rarest (302 of 2770, 11%). Adverse outcomes progressively increased from phenotypes alpha to gamma, including 30-day mortality (0.3%, 2.1%, and 6.0%, respectively), in-hospital mortality (0.2%, 2.3%, and 7.3%), and prolonged hospital LOS (3.4%, 22.1%, and 25.8%). When combined with the ASA score, digital phenotypes achieved higher AUROC than the ASA score alone (hospital mortality: 0.91 vs 0.84; prolonged hospitalization: 0.80 vs 0.71). CONCLUSIONS: For 3 frequently performed surgeries, we identified 3 digital phenotypes. The typical profiles of each phenotype were described and could be used to anticipate adverse postoperative events.

Assessing eligibility for lung cancer screening using parsimonious ensemble machine learning models: A development and validation study.

BACKGROUND: Risk-based screening for lung cancer is currently being considered in several countries; however, the optimal approach to determine eligibility remains unclear. Ensemble machine learning could support the development of highly parsimonious prediction models that maintain the performance of more complex models while maximising simplicity and generalisability, supporting the widespread adoption of personalised screening. In this work, we aimed to develop and validate ensemble machine learning models to determine eligibility for risk-based lung cancer screening. METHODS AND FINDINGS: For model development, we used data from 216,714 ever-smokers recruited between 2006 and 2010 to the UK Biobank prospective cohort and 26,616 high-risk ever-smokers recruited between 2002 and 2004 to the control arm of the US National Lung Screening (NLST) randomised controlled trial. The NLST trial randomised high-risk smokers from 33 US centres with at least a 30 pack-year smoking history and fewer than 15 quit-years to annual CT or chest radiography screening for lung cancer. We externally validated our models among 49,593 participants in the chest radiography arm and all 80,659 ever-smoking participants in the US Prostate, Lung, Colorectal and Ovarian (PLCO) Screening Trial. The PLCO trial, recruiting from 1993 to 2001, analysed the impact of chest radiography or no chest radiography for lung cancer screening. We primarily validated in the PLCO chest radiography arm such that we could benchmark against comparator models developed within the PLCO control arm. Models were developed to predict the risk of 2 outcomes within 5 years from baseline: diagnosis of lung cancer and death from lung cancer. We assessed model discrimination (area under the receiver operating curve, AUC), calibration (calibration curves and expected/observed ratio), overall performance (Brier scores), and net benefit with decision curve analysis. Models predicting lung cancer death (UCL-D) and incidence (UCL-I) using 3 variables-age, smoking duration, and pack-years-achieved or exceeded parity in discrimination, overall performance, and net benefit with comparators currently in use, despite requiring only one-quarter of the predictors. In external validation in the PLCO trial, UCL-D had an AUC of 0.803 (95% CI: 0.783, 0.824) and was well calibrated with an expected/observed (E/O) ratio of 1.05 (95% CI: 0.95, 1.19). UCL-I had an AUC of 0.787 (95% CI: 0.771, 0.802), an E/O ratio of 1.0 (95% CI: 0.92, 1.07). The sensitivity of UCL-D was 85.5% and UCL-I was 83.9%, at 5-year risk thresholds of 0.68% and 1.17%, respectively, 7.9% and 6.2% higher than the USPSTF-2021 criteria at the same specificity. The main limitation of this study is that the models have not been validated outside of UK and US cohorts. CONCLUSIONS: We present parsimonious ensemble machine learning models to predict the risk of lung cancer in ever-smokers, demonstrating a novel approach that could simplify the implementation of risk-based lung cancer screening in multiple settings.

Bridging the Worlds of Pharmacometrics and Machine Learning.

Precision medicine requires individualized modeling of disease and drug dynamics, with machine learning-based computational techniques gaining increasing popularity. The complexity of either field, however, makes current pharmacological problems opaque to machine learning practitioners, and state-of-the-art machine learning methods inaccessible to pharmacometricians. To help bridge the two worlds, we provide an introduction to current problems and techniques in pharmacometrics that ranges from pharmacokinetic and pharmacodynamic modeling to pharmacometric simulations, model-informed precision dosing, and systems pharmacology, and review some of the machine learning approaches to address them. We hope this would facilitate collaboration between experts, with complementary strengths of principled pharmacometric modeling and flexibility of machine learning leading to synergistic effects in pharmacological applications.

Relationship Between Azithromycin and Cardiovascular Outcomes in Unvaccinated Patients With COVID-19 and Preexisting Cardiovascular Disease.

Background Empiric antimicrobial therapy with azithromycin is highly used in patients admitted to the hospital with COVID-19, despite prior research suggesting that azithromycin may be associated with increased risk of cardiovascular events. Methods and Results This study was conducted using data from the ISACS-COVID-19 (International Survey of Acute Coronavirus Syndromes-COVID-19) registry. Patients with a confirmed diagnosis of SARS-CoV-2 infection were eligible for inclusion. The study included 793 patients exposed to azithromycin within 24 hours from hospital admission and 2141 patients who received only standard care. The primary exposure was cardiovascular disease (CVD). Main outcome measures were 30-day mortality and acute heart failure (AHF). Among 2934 patients, 1066 (36.4%) had preexisting CVD. A total of 617 (21.0%) died, and 253 (8.6%) had AHF. Azithromycin therapy was consistently associated with an increased risk of AHF in patients with preexisting CVD (risk ratio [RR], 1.48 [95% CI, 1.06-2.06]). Receiving azithromycin versus standard care was not significantly associated with death (RR, 0.94 [95% CI, 0.69-1.28]). By contrast, we found significantly reduced odds of death (RR, 0.57 [95% CI, 0.42-0.79]) and no significant increase in AHF (RR, 1.23 [95% CI, 0.75-2.04]) in patients without prior CVD. The relative risks of death from the 2 subgroups were significantly different from each other (Pinteraction=0.01). Statistically significant association was observed between AHF and death (odds ratio, 2.28 [95% CI, 1.34-3.90]). Conclusions These findings suggest that azithromycin use in patients with COVID-19 and prior history of CVD is significantly associated with an increased risk of AHF and all-cause 30-day mortality. Registration URL: https://www.clinicaltrials.gov; Unique identifier: NCT05188612.

AutoPrognosis 2.0: Democratizing diagnostic and prognostic modeling in healthcare with automated machine learning.

Diagnostic and prognostic models are increasingly important in medicine and inform many clinical decisions. Recently, machine learning approaches have shown improvement over conventional modeling techniques by better capturing complex interactions between patient covariates in a data-driven manner. However, the use of machine learning introduces technical and practical challenges that have thus far restricted widespread adoption of such techniques in clinical settings. To address these challenges and empower healthcare professionals, we present an open-source machine learning framework, AutoPrognosis 2.0, to facilitate the development of diagnostic and prognostic models. AutoPrognosis leverages state-of-the-art advances in automated machine learning to develop optimized machine learning pipelines, incorporates model explainability tools, and enables deployment of clinical demonstrators, without requiring significant technical expertise. To demonstrate AutoPrognosis 2.0, we provide an illustrative application where we construct a prognostic risk score for diabetes using the UK Biobank, a prospective study of 502,467 individuals. The models produced by our automated framework achieve greater discrimination for diabetes than expert clinical risk scores. We have implemented our risk score as a web-based decision support tool, which can be publicly accessed by patients and clinicians. By open-sourcing our framework as a tool for the community, we aim to provide clinicians and other medical practitioners with an accessible resource to develop new risk scores, personalized diagnostics, and prognostics using machine learning techniques. Software: https://github.com/vanderschaarlab/AutoPrognosis.

Synthetic Model Combination: A new machine-learning method for pharmacometric model ensembling.

When aiming to make predictions over targets in the pharmacological setting, a data-focused approach aims to learn models based on a collection of labeled examples. Unfortunately, data sharing is not always possible, and this can result in many different models trained on disparate populations, leading to the natural question of how best to use and combine them when making a new prediction. Previous work has focused on global model selection or ensembling, with the result of a single final model across the feature space. Machine-learning models perform notoriously poorly on data outside their training domain, however, due to a problem known as covariate shift, and so we argue that when ensembling models the weightings for individual instances must reflect their respective domains-in other words, models that are more likely to have seen information on that instance should have more attention paid to them. We introduce a method for such an instance-wise ensembling of models called Synthetic Model Combination (SMC), including a novel representation learning step for handling sparse high-dimensional domains. We demonstrate the use of SMC on an example with dosing predictions for vancomycin, although emphasize the applicability of the method to any scenario involving the use of multiple models.

Development and External Validation of a Risk Calculator for Prediction of Major Complications and Readmission After Anterior Cervical Discectomy and Fusion.

STUDY DESIGN: A retrospective, case-control study. OBJECTIVE: We aim to build a risk calculator predicting major perioperative complications after anterior cervical fusion. In addition, we aim to externally validate this calculator with an institutional cohort of patients who underwent anterior cervical discectomy and fusion (ACDF). SUMMARY OF BACKGROUND DATA: The average age and proportion of patients with at least one comorbidity undergoing ACDF have increased in recent years. Given the increased morbidity and cost associated with perioperative complications and unplanned readmission, accurate risk stratification of patients undergoing ACDF is of great clinical utility. METHODS: This is a retrospective cohort study of adults who underwent anterior cervical fusion at any nonfederal California hospital between 2015 and 2017. The primary outcome was major perioperative complication or 30-day readmission. We built standard and ensemble machine learning models for risk prediction, assessing discrimination, and calibration. The best-performing model was validated on an external cohort comprised of consecutive adult patients who underwent ACDF at our institution between 2013 and 2020. RESULTS: A total of 23,184 patients were included in this study; there were 1886 cases of major complication or readmissions. The ensemble model was well calibrated and demonstrated an area under the receiver operating characteristic curve of 0.728. The variables most important for the ensemble model include male sex, medical comorbidities, history of complications, and teaching hospital status. The ensemble model was evaluated on the validation cohort (n=260) with an area under the receiver operating characteristic curve of 0.802. The ensemble algorithm was used to build a web-based risk calculator. CONCLUSION: We report derivation and external validation of an ensemble algorithm for prediction of major perioperative complications and 30-day readmission after anterior cervical fusion. This model has excellent discrimination and is well calibrated when tested on a contemporaneous external cohort of ACDF cases.

Sex differences and disparities in cardiovascular outcomes of COVID-19.

AIMS: Previous analyses on sex differences in case fatality rates at population-level data had limited adjustment for key patient clinical characteristics thought to be associated with coronavirus disease 2019 (COVID-19) outcomes. We aimed to estimate the risk of specific organ dysfunctions and mortality in women and men. METHODS AND RESULTS: This retrospective cross-sectional study included 17 hospitals within 5 European countries participating in the International Survey of Acute Coronavirus Syndromes COVID-19 (NCT05188612). Participants were individuals hospitalized with positive severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) from March 2020 to February 2022. Risk-adjusted ratios (RRs) of in-hospital mortality, acute respiratory failure (ARF), acute heart failure (AHF), and acute kidney injury (AKI) were calculated for women vs. men. Estimates were evaluated by inverse probability weighting and logistic regression models. The overall care cohort included 4499 patients with COVID-19-associated hospitalizations. Of these, 1524 (33.9%) were admitted to intensive care unit (ICU), and 1117 (24.8%) died during hospitalization. Compared with men, women were less likely to be admitted to ICU [RR: 0.80; 95% confidence interval (CI): 0.71-0.91]. In general wards (GWs) and ICU cohorts, the adjusted women-to-men RRs for in-hospital mortality were of 1.13 (95% CI: 0.90-1.42) and 0.86 (95% CI: 0.70-1.05; pinteraction = 0.04). Development of AHF, AKI, and ARF was associated with increased mortality risk (odds ratios: 2.27, 95% CI: 1.73-2.98; 3.85, 95% CI: 3.21-4.63; and 3.95, 95% CI: 3.04-5.14, respectively). The adjusted RRs for AKI and ARF were comparable among women and men regardless of intensity of care. In contrast, female sex was associated with higher odds for AHF in GW, but not in ICU (RRs: 1.25; 95% CI: 0.94-1.67 vs. 0.83; 95% CI: 0.59-1.16, pinteraction = 0.04). CONCLUSIONS: Women in GW were at increased risk of AHF and in-hospital mortality for COVID-19 compared with men. For patients receiving ICU care, fatal complications including AHF and mortality appeared to be independent of sex. Equitable access to COVID-19 ICU care is needed to minimize the unfavourable outcome of women presenting with COVID-19-related complications.