Deployment and validation of the CLL treatment infection model adjoined to an EHR system.

Research algorithms are seldom externally validated or integrated into clinical practice, leaving unknown challenges in deployment. In such efforts, one needs to address challenges related to data harmonization, the performance of an algorithm in unforeseen missingness, automation and monitoring of predictions, and legal frameworks. We here describe the deployment of a high-dimensional data-driven decision support model into an EHR and derive practical guidelines informed by this deployment that includes the necessary processes, stakeholders and design requirements for a successful deployment. For this, we describe our deployment of the chronic lymphocytic leukemia (CLL) treatment infection model (CLL-TIM) as a stand-alone platform adjoined to an EPIC-based Danish Electronic Health Record (EHR), with the presentation of personalized predictions in a clinical context. CLL-TIM is an 84-variable data-driven prognostic model utilizing 7-year medical patient records and predicts the 2-year risk composite outcome of infection and/or treatment post-CLL diagnosis. As an independent validation cohort for this deployment, we used a retrospective population-based cohort of patients diagnosed with CLL from 2018 onwards (n = 1480). Unexpectedly high levels of missingness for key CLL-TIM variables were exhibited upon deployment. High dimensionality, with the handling of missingness, and predictive confidence were critical design elements that enabled trustworthy predictions and thus serves as a priority for prognostic models seeking deployment in new EHRs. Our setup for deployment, including automation and monitoring into EHR that meets Medical Device Regulations, may be used as step-by-step guidelines for others aiming at designing and deploying research algorithms into clinical practice.

Identifying CLL patients at high risk of atrial fibrillation on treatment using machine learning.

An increased risk of developing atrial fibrillation (AF) has been observed in patients with chronic lymphocytic leukemia (CLL) who were treated with ibrutinib and other BTK inhibitors. Previous studies have explored the prevalence of AF in CLL and the risk of developing AF at time of diagnosis. However, the interaction between treatment type with other risk factors on risk of developing atrial fibrillation at the time of treatment initiation has not been investigated. This becomes particularly crucial in CLL, as there is often a substantial time gap between diagnosis and treatment, unlike many other cancers. We propose a treatment-aware approach using predictive modeling to identify the risk factors associated with AF at time of treatment initiation. Moreover, the model provides treatment-dependent risk factors by including the interaction between the treatment types and other risk factors. The results demonstrated that the treatment-aware modeling including interactions outperformed currentrisk scores.

Erratum: Author Correction: Early stimulated immune responses predict clinical disease severity in hospitalized COVID-19 patients.

[This corrects the article DOI: 10.1038/s43856-022-00178-5.].

Early stimulated immune responses predict clinical disease severity in hospitalized COVID-19 patients.

Background: The immune pathogenesis underlying the diverse clinical course of COVID-19 is poorly understood. Currently, there is an unmet need in daily clinical practice for early biomarkers and improved risk stratification tools to help identify and monitor COVID-19 patients at risk of severe disease. Methods: We performed longitudinal assessment of stimulated immune responses in 30 patients hospitalized with COVID-19. We used the TruCulture whole-blood ligand-stimulation assay applying standardized stimuli to activate distinct immune pathways, allowing quantification of cytokine responses. We further characterized immune cell subsets by flow cytometry and used this deep immunophenotyping data to map the course of clinical disease within and between patients. Results: Here we demonstrate impairments in innate immune response pathways at time of COVID-19 hospitalization that are associated with the development of severe disease. We show that these impairments are transient in those discharged from hospital, as illustrated by functional and cellular immune reconstitution. Specifically, we identify lower levels of LPS-stimulated IL-1ß, and R848-stimulated IL-12 and IL-17A, at hospital admission to be significantly associated with increasing COVID-19 disease severity during hospitalization. Furthermore, we propose a stimulated immune response signature for predicting risk of developing severe or critical COVID-19 disease at time of hospitalization, to validate in larger cohorts. Conclusions: We identify early impairments in innate immune responses that are associated with subsequent COVID-19 disease severity. Our findings provide basis for early identification of patients at risk of severe disease which may have significant implications for the early management of patients hospitalized with COVID-19.

Personalized survival probabilities for SARS-CoV-2 positive patients by explainable machine learning.

Interpretable risk assessment of SARS-CoV-2 positive patients can aid clinicians to implement precision medicine. Here we trained a machine learning model to predict mortality within 12 weeks of a first positive SARS-CoV-2 test. By leveraging data on 33,938 confirmed SARS-CoV-2 cases in eastern Denmark, we considered 2723 variables extracted from electronic health records (EHR) including demographics, diagnoses, medications, laboratory test results and vital parameters. A discrete-time framework for survival modelling enabled us to predict personalized survival curves and explain individual risk factors. Performance on the test set was measured with a weighted concordance index of 0.95 and an area under the curve for precision-recall of 0.71. Age, sex, number of medications, previous hospitalizations and lymphocyte counts were identified as top mortality risk factors. Our explainable survival model developed on EHR data also revealed temporal dynamics of the 22 selected risk factors. Upon further validation, this model may allow direct reporting of personalized survival probabilities in routine care.

Prediction of clinical outcome in CLL based on recurrent gene mutations, CLL-IPI variables, and (para)clinical data.

A highly variable clinical course, immune dysfunction, and a complex genetic blueprint pose challenges for treatment decisions and the management of risk of infection in patients with chronic lymphocytic leukemia (CLL). In recent years, the use of machine learning (ML) technologies has made it possible to attempt to untangle such heterogeneous disease entities. In this study, using 3 classes of variables (international prognostic index for CLL [CLL-IPI] variables, baseline [para]clinical data, and data on recurrent gene mutations), we built ML predictive models to identify the individual risk of 4 clinical outcomes: death, treatment, infection, and the combined outcome of treatment or infection. Using the predictive models, we assessed to what extent the different classes of variables are predictive of the 4 different outcomes, within both a short-term 2-year outlook and a long-term 5-year outlook after CLL diagnosis. By adding the baseline (para)clinical data to CLL-IPI variables, predictive performance was improved, whereas no further improvement was observed when including the data on recurrent genetic mutations. We discovered 2 main clusters of variables predictive of treatment and infection. Further emphasizing the high mortality resulting from infection in CLL, we found a close similarity between variables predictive of infection in the short-term outlook and those predictive of death in the long-term outlook. We conclude that at the time of CLL diagnosis, routine (para)clinical data are more predictive of patient outcome than recurrent mutations. Future studies on modeling genetics and clinical outcome should always consider the inclusion of several (para)clinical data to improve performance.

Identifying patients with chronic lymphocytic leukemia without need of treatment: End of endless watch and wait?

INTRODUCTION: Early-stage chronic lymphocytic leukemia (CLL) challenges specialized management and follow-up. METHODS: We developed and validated a prognostic index to identify newly diagnosed patients without need of treatment (CLL-WONT) by a training/validation approach using data on 4708 patients. Composite scores derived from weighted hazards by multivariable analysis defined CLL-WONT risk groups. RESULTS: Age (>65 years: 1 point), Binet stage (B: 2 points), lactate dehydrogenase (LDH) (>205 U/L: 1 point), absolute lymphocyte count (15-30 × 109 /L: 1 point; >30 × 109 /L; 2 points), ß2-microglobulin (>4 mg/L: 1 point), IGHV mutation status (unmutated: 1 point), and 11q or 17p deletion (1 point) were independently associated with shorter time to first treatment (TTFT). Low-risk patients demonstrated 5-year TTFT of 2% by internal validation, but 7-19% by external validation. Including all patients with complete scores, the 5-year TTFT was 10% for the 756 (39%) CLL-WONT low-risk patients, and the 704 (37%) patients who were both CLL-WONT and CLL-IPI low risk demonstrated even lower 5-year TTFT (8%). CONCLUSION: We have adopted the CLL-WONT at an institution covering 1 800 000 individuals to allow patients with both low-risk CLL-WONT and CLL-IPI to be managed by primary healthcare providers, thereby prioritizing specialized hematology services for patients in dire need.

Artificial intelligence models in chronic lymphocytic leukemia - recommendations toward state-of-the-art.

Artificial intelligence (AI), machine learning and predictive modeling are becoming enabling technologies in many day-to-day applications. Translation of these advances to the patient's bedside for AI assisted interventions is not yet the norm. With specific emphasis on CLL, here, we review the progress of prognostic models in hematology and highlight sources of stagnation that may be limiting significant improvements in prognostication in the near future. We discuss issues related to performance, trust, modeling simplicity, and prognostic marker robustness and find that the major limiting factor in progressing toward state-of-the-art prognostication within the hematological community, is not the lack of able AI algorithms but rather, the lack of their adoption. Current models in CLL still deal with the 'average' patient while the use of patient-centric approaches remains absent. Using lessons from research areas where machine learning has become an enabling technology, we derive recommendations and propose methods for achieving state-of-the-art predictions in modeling health data, that can be readily adopted by the CLL modeling community.

Machine learning can identify newly diagnosed patients with CLL at high risk of infection.

Infections have become the major cause of morbidity and mortality among patients with chronic lymphocytic leukemia (CLL) due to immune dysfunction and cytotoxic CLL treatment. Yet, predictive models for infection are missing. In this work, we develop the CLL Treatment-Infection Model (CLL-TIM) that identifies patients at risk of infection or CLL treatment within 2 years of diagnosis as validated on both internal and external cohorts. CLL-TIM is an ensemble algorithm composed of 28 machine learning algorithms based on data from 4,149 patients with CLL. The model is capable of dealing with heterogeneous data, including the high rates of missing data to be expected in the real-world setting, with a precision of 72% and a recall of 75%. To address concerns regarding the use of complex machine learning algorithms in the clinic, for each patient with CLL, CLL-TIM provides explainable predictions through uncertainty estimates and personalized risk factors.

Characterizing changes in the rate of protein-protein dissociation upon interface mutation using hotspot energy and organization.

Predicting the effects of mutations on the kinetic rate constants of protein-protein interactions is central to both the modeling of complex diseases and the design of effective peptide drug inhibitors. However, while most studies have concentrated on the determination of association rate constants, dissociation rates have received less attention. In this work we take a novel approach by relating the changes in dissociation rates upon mutation to the energetics and architecture of hotspots and hotregions, by performing alanine scans pre- and post-mutation. From these scans, we design a set of descriptors that capture the change in hotspot energy and distribution. The method is benchmarked on 713 kinetically characterized mutations from the SKEMPI database. Our investigations show that, with the use of hotspot descriptors, energies from single-point alanine mutations may be used for the estimation of off-rate mutations to any residue type and also multi-point mutations. A number of machine learning models are built from a combination of molecular and hotspot descriptors, with the best models achieving a Pearson's Correlation Coefficient of 0.79 with experimental off-rates and a Matthew's Correlation Coefficient of 0.6 in the detection of rare stabilizing mutations. Using specialized feature selection models we identify descriptors that are highly specific and, conversely, broadly important to predicting the effects of different classes of mutations, interface regions and complexes. Our results also indicate that the distribution of the critical stability regions across protein-protein interfaces is a function of complex size more strongly than interface area. In addition, mutations at the rim are critical for the stability of small complexes, but consistently harder to characterize. The relationship between hotregion size and the dissociation rate is also investigated and, using hotspot descriptors which model cooperative effects within hotregions, we show how the contribution of hotregions of different sizes, changes under different cooperative effects.

A Markov-chain model description of binding funnels to enhance the ranking of docked solutions.

Within the crowded, seemingly chaotic environment of the cell, proteins are still able to find their binding partners. This is achieved via an ensemble of trajectories, which funnel them towards their functional binding sites, the binding funnel. Here, we characterize funnel-like energy structures on the global energy landscape using time-homogeneous finite state Markov chain models. These models are based on the idea that transitions can occur between structurally similar docking solutions, with transition probabilities determined by their difference in binding energy. Funnel-like energy structures are those containing solutions with very high equilibrium populations. Although these are found surrounding both near-native and false positive binding sites, we show that the removal of nonfunnel-like energy structures, by filtering away solutions with low maximum equilibrium population, can significantly improve the ranking of docked poses.

Community-wide evaluation of methods for predicting the effect of mutations on protein-protein interactions.

Community-wide blind prediction experiments such as CAPRI and CASP provide an objective measure of the current state of predictive methodology. Here we describe a community-wide assessment of methods to predict the effects of mutations on protein-protein interactions. Twenty-two groups predicted the effects of comprehensive saturation mutagenesis for two designed influenza hemagglutinin binders and the results were compared with experimental yeast display enrichment data obtained using deep sequencing. The most successful methods explicitly considered the effects of mutation on monomer stability in addition to binding affinity, carried out explicit side-chain sampling and backbone relaxation, evaluated packing, electrostatic, and solvation effects, and correctly identified around a third of the beneficial mutations. Much room for improvement remains for even the best techniques, and large-scale fitness landscapes should continue to provide an excellent test bed for continued evaluation of both existing and new prediction methodologies.

Understanding cancer mechanisms through network dynamics.

Cancer is a complex, multifaceted disease. Cellular systems are perturbed both during the onset and development of cancer, and the behavioural change of tumour cells usually involves a broad range of dynamic variations. To an extent, the difficulty of monitoring the systemic change has been alleviated by recent developments in the high-throughput technologies. At both the genomic as well as proteomic levels, the technological advances in microarray and mass spectrometry, in conjunction with computational simulations and the construction of human interactome maps have facilitated the progress of identifying disease-associated genes. On a systems level, computational approaches developed for network analysis are becoming especially useful for providing insights into the mechanism behind tumour development and metastasis. This review emphasizes network approaches that have been developed to study cancer and provides an overview of our current knowledge of protein-protein interaction networks, and how their systemic perturbation can be analysed by two popular network simulation methods: Boolean network and ordinary differential equations.

Protein-protein binding affinity prediction on a diverse set of structures.

MOTIVATION: Accurate binding free energy functions for protein-protein interactions are imperative for a wide range of purposes. Their construction is predicated upon ascertaining the factors that influence binding and their relative importance. A recent benchmark of binding affinities has allowed, for the first time, the evaluation and construction of binding free energy models using a diverse set of complexes, and a systematic assessment of our ability to model the energetics of conformational changes. RESULTS: We construct a large set of molecular descriptors using commonly available tools, introducing the use of energetic factors associated with conformational changes and disorder to order transitions, as well as features calculated on structural ensembles. The descriptors are used to train and test a binding free energy model using a consensus of four machine learning algorithms, whose performance constitutes a significant improvement over the other state of the art empirical free energy functions tested. The internal workings of the learners show how the descriptors are used, illuminating the determinants of protein-protein binding. AVAILABILITY: The molecular descriptor set and descriptor values for all complexes are available in the Supplementary Material. A web server for the learners and coordinates for the bound and unbound structures can be accessed from the website: http://bmm.cancerresearchuk.org/~Affinity. CONTACT: paul.bates@cancer.org.uk. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.