Acute internal medicine physicians' clinical intuition based on acute care telephone referral: A prospective study.

INTRODUCTION: In the Netherlands, most emergency department (ED) patients are referred by a general practitioner (GP) or a hospital specialist. Early risk stratification during telephone referral could allow the physician to assess the severity of the patients' illness in the prehospital setting. We aim to assess the discriminatory value of the acute internal medicine (AIM) physicians' clinical intuition based on telephone referral of ED patients to predict short-term adverse outcomes, and to investigate on which information their predictions are based. METHODS: In this prospective study, we included adult ED patients who were referred for internal medicine by a GP or a hospital specialist. Primary outcomes were hospital admission and triage category according to the Manchester Triage System (MTS). Secondary outcome was 31-day mortality. The discriminatory performance of the clinical intuition was assessed using an area under the receiver operating characteristics curve (AUC). To identify which information is important to predict adverse outcomes, we performed univariate regression analysis. Agreement between predicted and observed MTS triage category was assessed using intraclass and Spearman's correlation. RESULTS: We included 333 patients, of whom 172 (51.7%) were referred by a GP, 146 (43.8%) by a hospital specialist, and 12 (3.6%) by another health professional. The AIM physician's clinical intuition showed good discriminatory performance regarding hospital admission (AUC 0.72, 95% CI: 0.66-0.78) and 31-day mortality (AUC 0.73, 95% CI: 0.64-0.81). Univariate regression analysis showed that age ≥65 years and a sense of alarm were significant predictors. The predicted and observed triage category were similar in 45.2%, but in 92.5% the prediction did not deviate by more than one category. Intraclass and Spearman's correlation showed fair agreement between predicted and observed triage category (ICC 0.48, Spearman's 0.29). CONCLUSION: Clinical intuition based on relevant information during a telephone referral can be used to accurately predict short-term outcomes, allowing for early risk stratification in the prehospital setting and managing ED patient flow more effectively.

Machine learning for risk stratification in the emergency department (MARS-ED) study protocol for a randomized controlled pilot trial on the implementation of a prediction model based on machine learning technology predicting 31-day mortality in the emergency department.

BACKGROUND: Many prediction models have been developed to help identify emergency department (ED) patients at high risk of poor outcome. However, these models often underperform in clinical practice and their actual clinical impact has hardly ever been evaluated. We aim to perform a clinical trial to investigate the clinical impact of a prediction model based on machine learning (ML) technology. METHODS: The study is a prospective, randomized, open-label, non-inferiority pilot clinical trial. We will investigate the clinical impact of a prediction model based on ML technology, the RISKINDEX, which has been developed to predict the risk of 31-day mortality based on the results of laboratory tests and demographic characteristics. In previous studies, the RISKINDEX was shown to outperform internal medicine specialists and to have high discriminatory performance. Adults patients (18 years or older) will be recruited in the ED. All participants will be randomly assigned to the control group or the intervention group in a 1:1 ratio. Participants in the control group will receive care as usual in which the study team asks the attending physicians questions about their clinical intuition. Participants in the intervention group will also receive care as usual, but in addition to asking the clinical impression questions, the study team presents the RISKINDEX to the attending physician in order to assess the extent to which clinical treatment is influenced by the results. DISCUSSION: This pilot clinical trial investigates the clinical impact and implementation of an ML based prediction model in the ED. By assessing the clinical impact and prognostic accuracy of the RISKINDEX, this study aims to contribute valuable insights to optimize patient care and inform future research in the field of ML based clinical prediction models. TRIAL REGISTRATION: ClinicalTrials.gov NCT05497830. Machine Learning for Risk Stratification in the Emergency Department (MARS-ED). Registered on August 11, 2022. URL: https://clinicaltrials.gov/study/NCT05497830 .

Explainable Machine Learning Models for Rapid Risk Stratification in the Emergency Department: A Multicenter Study.

BACKGROUND: Risk stratification of patients presenting to the emergency department (ED) is important for appropriate triage. Diagnostic laboratory tests are an essential part of the workup and risk stratification of these patients. Using machine learning, the prognostic power and clinical value of these tests can be amplified greatly. In this study, we applied machine learning to develop an accurate and explainable clinical decision support tool model that predicts the likelihood of 31-day mortality in ED patients (the RISKINDEX). This tool was developed and evaluated in four Dutch hospitals. METHODS: Machine learning models included patient characteristics and available laboratory data collected within the first 2 h after ED presentation, and were trained using 5 years of data from consecutive ED patients from the Maastricht University Medical Center (Maastricht), Meander Medical Center (Amersfoort), and Zuyderland Medical Center (Sittard and Heerlen). A sixth year of data was used to evaluate the models using area under the receiver-operating-characteristic curve (AUROC) and calibration curves. The Shapley additive explanations (SHAP) algorithm was used to obtain explainable machine learning models. RESULTS: The present study included 266 327 patients with 7.1 million laboratory results available. Models show high diagnostic performance with AUROCs of 0.94, 0.98, 0.88, and 0.90 for Maastricht, Amersfoort, Sittard and Heerlen, respectively. The SHAP algorithm was utilized to visualize patient characteristics and laboratory data patterns that underlie individual RISKINDEX predictions. CONCLUSIONS: Our clinical decision support tool has excellent diagnostic performance in predicting 31-day mortality in ED patients. Follow-up studies will assess whether implementation of these algorithms can improve clinically relevant end points.

Head-to-head comparison of 19 prediction models for short-term outcome in medical patients in the emergency department: a retrospective study.

INTRODUCTION: Prediction models for identifying emergency department (ED) patients at high risk of poor outcome are often not externally validated. We aimed to perform a head-to-head comparison of the discriminatory performance of several prediction models in a large cohort of ED patients. METHODS: In this retrospective study, we selected prediction models that aim to predict poor outcome and we included adult medical ED patients. Primary outcome was 31-day mortality, secondary outcomes were 1-day mortality, 7-day mortality, and a composite endpoint of 31-day mortality and admission to intensive care unit (ICU).The discriminatory performance of the prediction models was assessed using an area under the receiver operating characteristic curve (AUC). Finally, the prediction models with the highest performance to predict 31-day mortality were selected to further examine calibration and appropriate clinical cut-off points. RESULTS: We included 19 prediction models and applied these to 2185 ED patients. Thirty-one-day mortality was 10.6% (231 patients), 1-day mortality was 1.4%, 7-day mortality was 4.4%, and 331 patients (15.1%) met the composite endpoint. The RISE UP and COPE score showed similar and very good discriminatory performance for 31-day mortality (AUC 0.86), 1-day mortality (AUC 0.87), 7-day mortality (AUC 0.86) and for the composite endpoint (AUC 0.81). Both scores were well calibrated. Almost no patients with RISE UP and COPE scores below 5% had an adverse outcome, while those with scores above 20% were at high risk of adverse outcome. Some of the other prediction models (i.e. APACHE II, NEWS, WPSS, MEWS, EWS and SOFA) showed significantly higher discriminatory performance for 1-day and 7-day mortality than for 31-day mortality. CONCLUSIONS: Head-to-head validation of 19 prediction models in medical ED patients showed that the RISE UP and COPE score outperformed other models regarding 31-day mortality.

Performance of prediction models for short-term outcome in COVID-19 patients in the emergency department: a retrospective study.

INTRODUCTION: Coronavirus disease 2019 (COVID-19) has a high burden on the healthcare system. Prediction models may assist in triaging patients. We aimed to assess the value of several prediction models in COVID-19 patients in the emergency department (ED). METHODS: In this retrospective study, ED patients with COVID-19 were included. Prediction models were selected based on their feasibility. Primary outcome was 30-day mortality, secondary outcomes were 14-day mortality and a composite outcome of 30-day mortality and admission to medium care unit (MCU) or intensive care unit (ICU). The discriminatory performance of the prediction models was assessed using an area under the receiver operating characteristic curve (AUC). RESULTS: We included 403 patients. Thirty-day mortality was 23.6%, 14-day mortality was 19.1%, 66 patients (16.4%) were admitted to ICU, 48 patients (11.9%) to MCU, and 152 patients (37.7%) met the composite endpoint. Eleven prediction models were included. The RISE UP score and 4 C mortality scores showed very good discriminatory performance for 30-day mortality (AUC 0.83 and 0.84, 95% CI 0.79-0.88 for both), significantly higher than that of the other models. CONCLUSION: The RISE UP score and 4 C mortality score can be used to recognise patients at high risk for poor outcome and may assist in guiding decision-making and allocating resources.

An 86-year-old man with acute abdominal pain.

An 86-year-old man presented with severe pain in the upper abdomen along with fever. On physical examination, we found an arterial blood pressure of 84/43 mm Hg, a heart rate of 80 bpm and a temperature of 38.3°C. The abdomen was painful and peristalsis was absent. Empiric antibiotic therapy for sepsis was started with amoxicillin/clavulanate and gentamicin. CT scan of the abdomen revealed an emphysematous cholecystitis. Percutaneous ultrasound-guided cholecystostomy was applied. Bile cultures revealed Clostridium perfringens. Emphysematous cholecystitis is a life-threatening form of acute cholecystitis that occurs as a consequence of ischaemic injury to the gallbladder, followed by translocation of gas-forming bacteria (ie, C. perfringens, Escherichia coli, Klebsiella and Streptococci). The mortality associated with emphysematous cholecystitis is higher than in non-emphysematous cholecystitis (15% vs 4%). Therefore, early diagnosis with radiological imaging is of vital importance.

Symptomatic cholecystolithiasis after cholecystectomy.

A 43-year-old woman was admitted to the gastroenterology department with colicky pain in the upper abdomen. Four years earlier, she had undergone a laparoscopic cholecystectomy because of cholecystitis. She recognised her current complaints from that previous episode. An endoscopic retrograde cholangiopancreatography showed a cavity with a diameter of 2 cm which contained multiple concrements near the liver hilus. An elective surgical exploration was performed. Near the clip of the previous cholecystectomy a bulging of the biliary tract with its own duct was visualised and resected. Histological examination of this "neo" gallbladder showed that the bulging was consistent with the formation of a reservoir secondary to bile leakage, probably caused by a small peroperative lesion of the common bile duct during the previous cholecystectomy. In conclusion, our patient presented with colicky pain caused by concrements inside a 'neo' gallbladder.