Patients' Perspectives and Feasibility of Home Monitoring in Acute Care: The AcuteCare@Home Flash Mob Study.

Objective: To determine patients' perspectives on home monitoring at emergency department (ED) presentation and shortly after admission and compare these with their physicians' perspectives. Methods: Forty Dutch hospitals participated in this prospective flash mob study. Adult patients with acute medical conditions, treated by internal medicine specialties, presenting at the ED or admitted at the admission ward within the previous 24 h were included. The primary outcome was the proportion of patients who were able and willing to undergo home monitoring. Secondary outcomes included identifying barriers to home monitoring, patient's prerequisites, and assessing the agreement between the perspectives of patients and treating physicians. Results: On February 2, 2023, in total 665 patients [median age 69 (interquartile range: 55-78) years; 95.5% community dwelling; 29.3% Modified Early Warning Score ≥3; 29.5% clinical frailty score ≥5] were included. In total, 19.6% of ED patients were admitted and 26% of ward patients preferred home monitoring as continuation of care. Guaranteed readmission (87.8%), ability to contact the hospital 24/7 (77.3%), and a family caregiver at home (55.7%) were the most often reported prerequisites. Barriers for home monitoring were feeling too severely ill (78.8%) and inability to receive the required treatment at home (64.4%). The agreement between patients and physicians was fair (Cohens kappa coefficient 0.26). Conclusions: A substantial proportion of acutely ill patients stated that they were willing and able to be monitored at home. Guaranteed readmission, availability of a treatment team (24/7), and a home support system are needed for successful implementation of home monitoring in acute care.

Development and validation of a risk prediction model for hospital admission in COVID-19 patients presenting to primary care.

BACKGROUND: There is a paucity of prognostic models for COVID-19 that are usable for in-office patient assessment in general practice (GP). OBJECTIVES: To develop and validate a risk prediction model for hospital admission with readily available predictors. METHODS: A retrospective cohort study linking GP records from 8 COVID-19 centres and 55 general practices in the Netherlands to hospital admission records. The development cohort spanned March to June 2020, the validation cohort March to June 2021. The primary outcome was hospital admission within 14 days. We used geographic leave-region-out cross-validation in the development cohort and temporal validation in the validation cohort. RESULTS: In the development cohort, 4,806 adult patients with COVID-19 consulted their GP (median age 56, 56% female); in the validation cohort 830 patients did (median age 56, 52% female). In the development and validation cohort respectively, 292 (6.1%) and 126 (15.2%) were admitted to the hospital within 14 days, respectively. A logistic regression model based on sex, smoking, symptoms, vital signs and comorbidities predicted hospital admission with a c-index of 0.84 (95% CI 0.83 to 0.86) at geographic cross-validation and 0.79 (95% CI 0.74 to 0.83) at temporal validation, and was reasonably well calibrated (intercept -0.08, 95% CI -0.98 to 0.52, slope 0.89, 95% CI 0.71 to 1.07 at geographic cross-validation and intercept 0.02, 95% CI -0.21 to 0.24, slope 0.82, 95% CI 0.64 to 1.00 at temporal validation). CONCLUSION: We derived a risk model using readily available variables at GP assessment to predict hospital admission for COVID-19. It performed accurately across regions and waves. Further validation on cohorts with acquired immunity and newer SARS-CoV-2 variants is recommended.

A general practice prediction model based on signs and symptoms of COVID-19 patients reliably predicted hospitalisation.The model performed well in second-wave data with other dominant variants and changed testing and vaccination policies.In an emerging pandemic, GP data can be leveraged to develop prognostic models for decision support and to predict hospitalisation rates.

Explainable Machine Learning Models for Rapid Risk Stratification in the Emergency Department: A Multicenter Study.

BACKGROUND: Risk stratification of patients presenting to the emergency department (ED) is important for appropriate triage. Diagnostic laboratory tests are an essential part of the workup and risk stratification of these patients. Using machine learning, the prognostic power and clinical value of these tests can be amplified greatly. In this study, we applied machine learning to develop an accurate and explainable clinical decision support tool model that predicts the likelihood of 31-day mortality in ED patients (the RISKINDEX). This tool was developed and evaluated in four Dutch hospitals. METHODS: Machine learning models included patient characteristics and available laboratory data collected within the first 2 h after ED presentation, and were trained using 5 years of data from consecutive ED patients from the Maastricht University Medical Center (Maastricht), Meander Medical Center (Amersfoort), and Zuyderland Medical Center (Sittard and Heerlen). A sixth year of data was used to evaluate the models using area under the receiver-operating-characteristic curve (AUROC) and calibration curves. The Shapley additive explanations (SHAP) algorithm was used to obtain explainable machine learning models. RESULTS: The present study included 266 327 patients with 7.1 million laboratory results available. Models show high diagnostic performance with AUROCs of 0.94, 0.98, 0.88, and 0.90 for Maastricht, Amersfoort, Sittard and Heerlen, respectively. The SHAP algorithm was utilized to visualize patient characteristics and laboratory data patterns that underlie individual RISKINDEX predictions. CONCLUSIONS: Our clinical decision support tool has excellent diagnostic performance in predicting 31-day mortality in ED patients. Follow-up studies will assess whether implementation of these algorithms can improve clinically relevant end points.

Acute internal medicine physicians' clinical intuition based on acute care telephone referral: A prospective study.

INTRODUCTION: In the Netherlands, most emergency department (ED) patients are referred by a general practitioner (GP) or a hospital specialist. Early risk stratification during telephone referral could allow the physician to assess the severity of the patients' illness in the prehospital setting. We aim to assess the discriminatory value of the acute internal medicine (AIM) physicians' clinical intuition based on telephone referral of ED patients to predict short-term adverse outcomes, and to investigate on which information their predictions are based. METHODS: In this prospective study, we included adult ED patients who were referred for internal medicine by a GP or a hospital specialist. Primary outcomes were hospital admission and triage category according to the Manchester Triage System (MTS). Secondary outcome was 31-day mortality. The discriminatory performance of the clinical intuition was assessed using an area under the receiver operating characteristics curve (AUC). To identify which information is important to predict adverse outcomes, we performed univariate regression analysis. Agreement between predicted and observed MTS triage category was assessed using intraclass and Spearman's correlation. RESULTS: We included 333 patients, of whom 172 (51.7%) were referred by a GP, 146 (43.8%) by a hospital specialist, and 12 (3.6%) by another health professional. The AIM physician's clinical intuition showed good discriminatory performance regarding hospital admission (AUC 0.72, 95% CI: 0.66-0.78) and 31-day mortality (AUC 0.73, 95% CI: 0.64-0.81). Univariate regression analysis showed that age ≥65 years and a sense of alarm were significant predictors. The predicted and observed triage category were similar in 45.2%, but in 92.5% the prediction did not deviate by more than one category. Intraclass and Spearman's correlation showed fair agreement between predicted and observed triage category (ICC 0.48, Spearman's 0.29). CONCLUSION: Clinical intuition based on relevant information during a telephone referral can be used to accurately predict short-term outcomes, allowing for early risk stratification in the prehospital setting and managing ED patient flow more effectively.

Machine learning for risk stratification in the emergency department (MARS-ED) study protocol for a randomized controlled pilot trial on the implementation of a prediction model based on machine learning technology predicting 31-day mortality in the emergency department.

BACKGROUND: Many prediction models have been developed to help identify emergency department (ED) patients at high risk of poor outcome. However, these models often underperform in clinical practice and their actual clinical impact has hardly ever been evaluated. We aim to perform a clinical trial to investigate the clinical impact of a prediction model based on machine learning (ML) technology. METHODS: The study is a prospective, randomized, open-label, non-inferiority pilot clinical trial. We will investigate the clinical impact of a prediction model based on ML technology, the RISKINDEX, which has been developed to predict the risk of 31-day mortality based on the results of laboratory tests and demographic characteristics. In previous studies, the RISKINDEX was shown to outperform internal medicine specialists and to have high discriminatory performance. Adults patients (18 years or older) will be recruited in the ED. All participants will be randomly assigned to the control group or the intervention group in a 1:1 ratio. Participants in the control group will receive care as usual in which the study team asks the attending physicians questions about their clinical intuition. Participants in the intervention group will also receive care as usual, but in addition to asking the clinical impression questions, the study team presents the RISKINDEX to the attending physician in order to assess the extent to which clinical treatment is influenced by the results. DISCUSSION: This pilot clinical trial investigates the clinical impact and implementation of an ML based prediction model in the ED. By assessing the clinical impact and prognostic accuracy of the RISKINDEX, this study aims to contribute valuable insights to optimize patient care and inform future research in the field of ML based clinical prediction models. TRIAL REGISTRATION: ClinicalTrials.gov NCT05497830. Machine Learning for Risk Stratification in the Emergency Department (MARS-ED). Registered on August 11, 2022. URL: https://clinicaltrials.gov/study/NCT05497830 .