Aiming for Relevance.

Vital signs are crucial in intensive care units (ICUs). They are used to track the patient's state and to identify clinically significant changes. Predicting vital sign trajectories is valuable for early detection of adverse events. However, conventional machine learning metrics like RMSE often fail to capture the true clinical relevance of such predictions. We introduce novel vital sign prediction performance metrics that align with clinical contexts, focusing on deviations from clinical norms, overall trends, and trend deviations. These metrics are derived from empirical utility curves obtained in a previous study through interviews with ICU clinicians. We validate the metrics' usefulness using simulated and real clinical datasets (MIMIC and eICU). Furthermore, we employ these metrics as loss functions for neural networks, resulting in models that excel in predicting clinically significant events. This research paves the way for clinically relevant machine learning model evaluation and optimization, promising to improve ICU patient care.

Machine learning for healthcare that matters: Reorienting from technical novelty to equitable impact.

Despite significant technical advances in machine learning (ML) over the past several years, the tangible impact of this technology in healthcare has been limited. This is due not only to the particular complexities of healthcare, but also due to structural issues in the machine learning for healthcare (MLHC) community which broadly reward technical novelty over tangible, equitable impact. We structure our work as a healthcare-focused echo of the 2012 paper "Machine Learning that Matters", which highlighted such structural issues in the ML community at large, and offered a series of clearly defined "Impact Challenges" to which the field should orient itself. Drawing on the expertise of a diverse and international group of authors, we engage in a narrative review and examine issues in the research background environment, training processes, evaluation metrics, and deployment protocols which act to limit the real-world applicability of MLHC. Broadly, we seek to distinguish between machine learning ON healthcare data and machine learning FOR healthcare-the former of which sees healthcare as merely a source of interesting technical challenges, and the latter of which regards ML as a tool in service of meeting tangible clinical needs. We offer specific recommendations for a series of stakeholders in the field, from ML researchers and clinicians, to the institutions in which they work, and the governments which regulate their data access.

Evaluating the heterogeneous effect of extended culture to blastocyst transfer on the implantation outcome via causal inference in fresh ICSI cycles.

PURPOSE: In IVF treatments, extended culture to single blastocyst transfer is the recommended protocol over cleavage-stage transfer. However, evidence-based criteria for assessing the heterogeneous implications on implantation outcomes are lacking. The purpose of this work is to estimate the causal effect of blastocyst transfer on implantation outcome. METHODS: We fit a causal forest model using a multicenter observational dataset that includes an exogenous source of variability in treatment assignment and has a strong claim for satisfying the assumptions needed for valid causal inference from observational data. RESULTS: We quantified the probability difference in embryo implantation if transferred as a blastocyst versus cleavage stage. Blastocyst transfer increased the average implantation rate; however, we revealed a subpopulation of embryos whose implantation potential is predicted to increase via cleavage-stage transfer. CONCLUSION: Relative to the current policy, the proposed embryo transfer policy retrospectively improves implantation rate from 0.2 to 0.27. Our work demonstrates the efficacy of implementing causal inference in reproductive medicine and motivates its utilization in medical disciplines that are dominated by retrospective datasets.

iCVS-Inferring Cardio-Vascular hidden States from physiological signals available at the bedside.

Intensive care medicine is complex and resource-demanding. A critical and common challenge lies in inferring the underlying physiological state of a patient from partially observed data. Specifically for the cardiovascular system, clinicians use observables such as heart rate, arterial and venous blood pressures, as well as findings from the physical examination and ancillary tests to formulate a mental model and estimate hidden variables such as cardiac output, vascular resistance, filling pressures and volumes, and autonomic tone. Then, they use this mental model to derive the causes for instability and choose appropriate interventions. Not only this is a very hard problem due to the nature of the signals, but it also requires expertise and a clinician's ongoing presence at the bedside. Clinical decision support tools based on mechanistic dynamical models offer an appealing solution due to their inherent explainability, corollaries to the clinical mental process, and predictive power. With a translational motivation in mind, we developed iCVS: a simple, with high explanatory power, dynamical mechanistic model to infer hidden cardiovascular states. Full model estimation requires no prior assumptions on physiological parameters except age and weight, and the only inputs are arterial and venous pressure waveforms. iCVS also considers autonomic and non-autonomic modulations. To gain more information without increasing model complexity, both slow and fast timescales of the blood pressure traces are exploited, while the main inference and dynamic evolution are at the longer, clinically relevant, timescale of minutes. iCVS is designed to allow bedside deployment at pediatric and adult intensive care units and for retrospective investigation of cardiovascular mechanisms underlying instability. In this paper, we describe iCVS and inference system in detail, and using a dataset of critically-ill children, we provide initial indications to its ability to identify bleeding, distributive states, and cardiac dysfunction, in isolation and in combination.

What drives performance in machine learning models for predicting heart failure outcome?

Aims: The development of acute heart failure (AHF) is a critical decision point in the natural history of the disease and carries a dismal prognosis. The lack of appropriate risk-stratification tools at hospital discharge of AHF patients significantly limits clinical ability to precisely tailor patient-specific therapeutic regimen at this pivotal juncture. Machine learning-based strategies may improve risk stratification by incorporating analysis of high-dimensional patient data with multiple covariates and novel prediction methodologies. In the current study, we aimed at evaluating the drivers for success in prediction models and establishing an institute-tailored artificial Intelligence-based prediction model for real-time decision support. Methods and results: We used a cohort of all 10 868 patients AHF patients admitted to a tertiary hospital during a 12 years period. A total of 372 covariates were collected from admission to the end of the hospitalization. We assessed model performance across two axes: (i) type of prediction method and (ii) type and number of covariates. The primary outcome was 1-year survival from hospital discharge. For the model-type axis, we experimented with seven different methods: logistic regression (LR) with either L1 or L2 regularization, random forest (RF), Cox proportional hazards model (Cox), extreme gradient boosting (XGBoost), a deep neural-net (NeuralNet) and an ensemble classifier of all the above methods. We were able to achieve an area under receiver operator curve (AUROC) prediction accuracy of more than 80% with most prediction models including L1/L2-LR (80.4%/80.3%), Cox (80.2%), XGBoost (80.5%), NeuralNet (80.4%). RF was inferior to other methods (78.8%), and the ensemble model was slightly superior (81.2%). The number of covariates was a significant modifier (P < 0.001) of prediction success, the use of multiplex-covariates preformed significantly better (AUROC 80.4% for L1-LR) compared with a set of known clinical covariates (AUROC 77.8%). Demographics followed by lab-tests and administrative data resulted in the largest gain in model performance. Conclusions: The choice of the predictive modelling method is secondary to the multiplicity and type of covariates for predicting AHF prognosis. The application of a structured data pre-processing combined with the use of multiple-covariates results in an accurate, institute-tailored risk prediction in AHF.

Tell me something interesting: Clinical utility of machine learning prediction models in the ICU.

In recent years, extensive resources are dedicated to the development of machine learning (ML) based clinical prediction models for intensive care unit (ICU) patients. These models are transforming patient care into a collaborative human-AI task, yet prediction of patient-related events is mostly treated as a standalone goal, without considering clinicians' roles, tasks or workflow in depth. We conducted a mixed methods study aimed at understanding clinicians' needs and expectations from such systems, informing the design of machine learning based prediction models. Our findings identify several areas of focus where clinicians' needs deviate from current practice, including desired prediction targets, timescales stemming from actionability requirements, and concerns regarding the evaluation and trust in these algorithms. Based on our findings, we suggest several design implications for ML-based prediction tools in the ICU.

Socioeconomic disparities and COVID-19 vaccination acceptance: a nationwide ecologic study.

OBJECTIVE: To analyse the correlation between COVID-19 vaccination percentage and socioeconomic status (SES). METHODS: A nationwide ecologic study based on open-sourced, anonymized, aggregated data provided by the Israel Ministry of Health. The correlations between municipal SES, vaccination percentage and active COVID-19 cases during the vaccination campaign were analysed by using weighted Pearson correlations. To assess the adequacy of first dose vaccination rollout relative to the municipality COVID-19 disease burden, a metric termed the vaccination need ratio was devised by dividing the total number of active cases (per 10 000 people) by the vaccination percentage of the population over 60 in each municipality, and its correlation with the SES was examined. RESULTS: 23 days after initiation of the vaccination campaign, 760 916 (56.8%) individuals over the age of 60 were vaccinated in Israel with the first dose of the BNT162b2 COVID-19 vaccine. A negative correlation was found between the COVID-19 active case burden and the vaccination percentage of the study population in each municipality (r = -0.47, 95% CI -0.59 to -0.30). The vaccination percentage significantly correlated with the municipal SES (r = 0.83, 95% CI 0.79 to 0.87). This finding persisted but was attenuated over a 5-week period. A negative correlation between the vaccination need ratio and municipal SES (r = -0.80, 95% CI -0.88 to -0.66) was found. DISCUSSION: Lower COVID-19 vaccination percentage was associated with lower SES and high active disease burden. Vaccination efforts should focus on areas with lower SES and high disease burden to assure equality of vaccine allocation and potentially provide a more diligent disease mitigation.

Artificial Intelligence and Early Detection of Pancreatic Cancer: 2020 Summative Review.

ABSTRACT: Despite considerable research efforts, pancreatic cancer is associated with a dire prognosis and a 5-year survival rate of only 10%. Early symptoms of the disease are mostly nonspecific. The premise of improved survival through early detection is that more individuals will benefit from potentially curative treatment. Artificial intelligence (AI) methodology has emerged as a successful tool for risk stratification and identification in general health care. In response to the maturity of AI, Kenner Family Research Fund conducted the 2020 AI and Early Detection of Pancreatic Cancer Virtual Summit (www.pdac-virtualsummit.org) in conjunction with the American Pancreatic Association, with a focus on the potential of AI to advance early detection efforts in this disease. This comprehensive presummit article was prepared based on information provided by each of the interdisciplinary participants on one of the 5 following topics: Progress, Problems, and Prospects for Early Detection; AI and Machine Learning; AI and Pancreatic Cancer-Current Efforts; Collaborative Opportunities; and Moving Forward-Reflections from Government, Industry, and Advocacy. The outcome from the robust Summit conversations, to be presented in a future white paper, indicate that significant progress must be the result of strategic collaboration among investigators and institutions from multidisciplinary backgrounds, supported by committed funders.

COVID-19 dynamics after a national immunization program in Israel.

Studies on the real-life effect of the BNT162b2 vaccine for Coronavirus Disease 2019 (COVID-19) prevention are urgently needed. In this study, we conducted a retrospective analysis of data from the Israeli Ministry of Health collected between 28 August 2020 and 24 February 2021. We studied the temporal dynamics of the number of new COVID-19 cases and hospitalizations after the vaccination campaign, which was initiated on 20 December 2020. To distinguish the possible effects of the vaccination on cases and hospitalizations from other factors, including a third lockdown implemented on 8 January 2021, we performed several comparisons: (1) individuals aged 60 years and older prioritized to receive the vaccine first versus younger age groups; (2) the January lockdown versus the September lockdown; and (3) early-vaccinated versus late-vaccinated cities. A larger and earlier decrease in COVID-19 cases and hospitalization was observed in individuals older than 60 years, followed by younger age groups, by the order of vaccination prioritization. This pattern was not observed in the previous lockdown and was more pronounced in early-vaccinated cities. Our analysis demonstrates the real-life effect of a national vaccination campaign on the pandemic dynamics.

Hospital load and increased COVID-19 related mortality in Israel.

The spread of Coronavirus disease 19 (COVID-19) has led to many healthcare systems being overwhelmed by the rapid emergence of new cases. Here, we study the ramifications of hospital load due to COVID-19 morbidity on in-hospital mortality of patients with COVID-19 by analyzing records of all 22,636 COVID-19 patients hospitalized in Israel from mid-July 2020 to mid-January 2021. We show that even under moderately heavy patient load (>500 countrywide hospitalized severely-ill patients; the Israeli Ministry of Health defined 800 severely-ill patients as the maximum capacity allowing adequate treatment), in-hospital mortality rate of patients with COVID-19 significantly increased compared to periods of lower patient load (250-500 severely-ill patients): 14-day mortality rates were 22.1% (Standard Error 3.1%) higher (mid-September to mid-October) and 27.2% (Standard Error 3.3%) higher (mid-December to mid-January). We further show this higher mortality rate cannot be attributed to changes in the patient population during periods of heavier load.

Development and validation of a machine learning model predicting illness trajectory and hospital utilization of COVID-19 patients: A nationwide study.

OBJECTIVE: The spread of coronavirus disease 2019 (COVID-19) has led to severe strain on hospital capacity in many countries. We aim to develop a model helping planners assess expected COVID-19 hospital resource utilization based on individual patient characteristics. MATERIALS AND METHODS: We develop a model of patient clinical course based on an advanced multistate survival model. The model predicts the patient's disease course in terms of clinical states-critical, severe, or moderate. The model also predicts hospital utilization on the level of entire hospitals or healthcare systems. We cross-validated the model using a nationwide registry following the day-by-day clinical status of all hospitalized COVID-19 patients in Israel from March 1 to May 2, 2020 (n = 2703). RESULTS: Per-day mean absolute errors for predicted total and critical care hospital bed utilization were 4.72 ± 1.07 and 1.68 ± 0.40, respectively, over cohorts of 330 hospitalized patients; areas under the curve for prediction of critical illness and in-hospital mortality were 0.88 ± 0.04 and 0.96 ± 0.04, respectively. We further present the impact of patient influx scenarios on day-by-day healthcare system utilization. We provide an accompanying R software package. DISCUSSION: The proposed model accurately predicts total and critical care hospital utilization. The model enables evaluating impacts of patient influx scenarios on utilization, accounting for the state of currently hospitalized patients and characteristics of incoming patients. We show that accurate hospital load predictions were possible using only a patient's age, sex, and day-by-day clinical state (critical, severe, or moderate). CONCLUSIONS: The multistate model we develop is a powerful tool for predicting individual-level patient outcomes and hospital-level utilization.

Developing a COVID-19 mortality risk prediction model when individual-level data are not available.

At the COVID-19 pandemic onset, when individual-level data of COVID-19 patients were not yet available, there was already a need for risk predictors to support prevention and treatment decisions. Here, we report a hybrid strategy to create such a predictor, combining the development of a baseline severe respiratory infection risk predictor and a post-processing method to calibrate the predictions to reported COVID-19 case-fatality rates. With the accumulation of a COVID-19 patient cohort, this predictor is validated to have good discrimination (area under the receiver-operating characteristics curve of 0.943) and calibration (markedly improved compared to that of the baseline predictor). At a 5% risk threshold, 15% of patients are marked as high-risk, achieving a sensitivity of 88%. We thus demonstrate that even at the onset of a pandemic, shrouded in epidemiologic fog of war, it is possible to provide a useful risk predictor, now widely used in a large healthcare organization.

FuncISH: learning a functional representation of neural ISH images.

MOTIVATION: High-spatial resolution imaging datasets of mammalian brains have recently become available in unprecedented amounts. Images now reveal highly complex patterns of gene expression varying on multiple scales. The challenge in analyzing these images is both in extracting the patterns that are most relevant functionally and in providing a meaningful representation that allows neuroscientists to interpret the extracted patterns. RESULTS: Here, we present FuncISH--a method to learn functional representations of neural in situ hybridization (ISH) images. We represent images using a histogram of local descriptors in several scales, and we use this representation to learn detectors of functional (GO) categories for every image. As a result, each image is represented as a point in a low-dimensional space whose axes correspond to meaningful functional annotations. The resulting representations define similarities between ISH images that can be easily explained by functional categories. We applied our method to the genomic set of mouse neural ISH images available at the Allen Brain Atlas, finding that most neural biological processes can be inferred from spatial expression patterns with high accuracy. Using functional representations, we predict several gene interaction properties, such as protein-protein interactions and cell-type specificity, more accurately than competing methods based on global correlations. We used FuncISH to identify similar expression patterns of GABAergic neuronal markers that were not previously identified and to infer new gene function based on image-image similarities. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Descending systems translate transient cortical commands into a sustained muscle activation signal.

Controlling motor actions requires online adjustments of time-varying parameters. Although numerous studies have attempted to identify the parameters coded in different motor sites, the relationships between the temporal profile of neuronal responses and the dynamics of motor behavior remain poorly understood in particular because motor parameters such as force and movement direction often change over time. We studied time-dependent coding of cortical and spinal neurons in primates performing an isometric wrist task with an active hold period, which made it possible to segregate motor behavior into its phasic and sustained components. Here, we show that cortical neurons transiently code motor-related parameters when actively acquiring a goal, whereas spinal interneurons provide persistent information regarding maintained torque level and posture. Moreover, motor cortical neurons differed substantially from spinal neurons with regard to the evolvement of parameter-specific coding over the course of a trial. These results suggest that the motor cortex and spinal cord use different control policies: Cortical neurons produce transient motor commands governing ensuing actions, whereas spinal neurons exhibit sustained coding of ongoing motor states. Hence, motor structures downstream to M1 need to integrate cortical commands to produce state-dependent spinal firing.

Computation in spinal circuitry: lessons from behaving primates.

Performing voluntary motor actions requires the translation of motor commands into a specific set of muscle activation. While it is assumed that this process is carried out via cooperative interactions between supraspinal and spinal neurons, the unique contribution of each of these areas to the process is still unknown. Many studies have focused on the neuronal representation of the motor command, mostly in the motor cortex. Nonetheless, to execute these commands there must be a mechanism that can translate this representation into a sustained drive to the spinal motoneurons (MNs). Here we review different candidate mechanisms for activating MNs and their possible role in voluntary movements. We discuss recent studies which directly estimate the contribution of segmental INs to the transmission of cortical command to MNs, both in terms of functional connectivity and as a computational link. Finally, we suggest a conceptual framework in which the cortical motor command is processed simultaneously via MNs and INs. In this model, the motor cortex provides a transient signal which is important for initiating new patterns of recruited muscles, whereas the INs translate this command into a sustained, amplified and muscle-based signal which is necessary to maintain ongoing muscle activity.