External validation of triage tools for adults with suspected COVID-19 in a middle-income setting: an observational cohort study.

BACKGROUND: Tools proposed to triage ED acuity in suspected COVID-19 were derived and validated in higher income settings during early waves of the pandemic. We estimated the accuracy of seven risk-stratification tools recommended to predict severe illness in the Western Cape, South Africa. METHODS: An observational cohort study using routinely collected data from EDs across the Western Cape, from 27 August 2020 to 11 March 2022, was conducted to assess the performance of the PRIEST (Pandemic Respiratory Infection Emergency System Triage) tool, NEWS2 (National Early Warning Score, version 2), TEWS (Triage Early Warning Score), the WHO algorithm, CRB-65, Quick COVID-19 Severity Index and PMEWS (Pandemic Medical Early Warning Score) in suspected COVID-19. The primary outcome was intubation or non-invasive ventilation, death or intensive care unit admission at 30 days. RESULTS: Of the 446 084 patients, 15 397 (3.45%, 95% CI 34% to 35.1%) experienced the primary outcome. Clinical decision-making for inpatient admission achieved a sensitivity of 0.77 (95% CI 0.76 to 0.78), specificity of 0.88 (95% CI 0.87 to 0.88) and the negative predictive value (NPV) of 0.99 (95% CI 0.99 to 0.99). NEWS2, PMEWS and PRIEST scores achieved good estimated discrimination (C-statistic 0.79 to 0.82) and identified patients at risk of adverse outcomes at recommended cut-offs with moderate sensitivity (>0.8) and specificity ranging from 0.41 to 0.64. Use of the tools at recommended thresholds would have more than doubled admissions, with only a 0.01% reduction in false negative triage. CONCLUSION: No risk score outperformed existing clinical decision-making in determining the need for inpatient admission based on prediction of the primary outcome in this setting. Use of the PRIEST score at a threshold of one point higher than the previously recommended best approximated existing clinical accuracy.

Prognostic accuracy of triage tools for adults with suspected COVID-19 in a prehospital setting: an observational cohort study.

BACKGROUND: Tools proposed to triage patient acuity in COVID-19 infection have only been validated in hospital populations. We estimated the accuracy of five risk-stratification tools recommended to predict severe illness and compared accuracy to existing clinical decision making in a prehospital setting. METHODS: An observational cohort study using linked ambulance service data for patients attended by Emergency Medical Service (EMS) crews in the Yorkshire and Humber region of England between 26 March 2020 and 25 June 2020 was conducted to assess performance of the Pandemic Respiratory Infection Emergency System Triage (PRIEST) tool, National Early Warning Score (NEWS2), WHO algorithm, CRB-65 and Pandemic Medical Early Warning Score (PMEWS) in patients with suspected COVID-19 infection. The primary outcome was death or need for organ support. RESULTS: Of the 7549 patients in our cohort, 17.6% (95% CI 16.8% to 18.5%) experienced the primary outcome. The NEWS2 (National Early Warning Score, version 2), PMEWS, PRIEST tool and WHO algorithm identified patients at risk of adverse outcomes with a high sensitivity (>0.95) and specificity ranging from 0.3 (NEWS2) to 0.41 (PRIEST tool). The high sensitivity of NEWS2 and PMEWS was achieved by using lower thresholds than previously recommended. On index assessment, 65% of patients were transported to hospital and EMS decision to transfer patients achieved a sensitivity of 0.84 (95% CI 0.83 to 0.85) and specificity of 0.39 (95% CI 0.39 to 0.40). CONCLUSION: Use of NEWS2, PMEWS, PRIEST tool and WHO algorithm could improve sensitivity of EMS triage of patients with suspected COVID-19 infection. Use of the PRIEST tool would improve sensitivity of triage without increasing the number of patients conveyed to hospital.

Prognostic accuracy of eight triage scores in suspected COVID-19 in an Emergency Department low-income setting: An observational cohort study.

Introduction: Previous studies deriving and validating triage scores for patients with suspected COVID-19 in Emergency Department settings have been conducted in high- or middle-income settings. We assessed eight triage scores' accuracy for death or organ support in patients with suspected COVID-19 in Sudan. Methods: We conducted an observational cohort study using Covid-19 registry data from eight emergency unit isolation centres in Khartoum State, Sudan. We assessed performance of eight triage scores including: PRIEST, LMIC-PRIEST, NEWS2, TEWS, the WHO algorithm, CRB-65, Quick COVID-19 Severity Index and PMEWS in suspected COVID-19. A composite primary outcome included death, ventilation or ICU admission. Results: In total 874 (33.84 %, 95 % CI:32.04 % to 35.69 %) of 2,583 patients died, required intubation/non-invasive ventilation or HDU/ICU admission . All risk-stratification scores assessed had worse estimated discrimination in this setting, compared to studies conducted in higher-income settings: C-statistic range for primary outcome: 0.56-0.64. At previously recommended thresholds NEWS2, PRIEST and LMIC-PRIEST had high estimated sensitivities (≥0.95) for the primary outcome. However, the high baseline risk meant that low-risk patients identified at these thresholds still had a between 8 % and 17 % risk of death, ventilation or ICU admission. Conclusion: None of the triage scores assessed demonstrated sufficient accuracy to be used clinically. This is likely due to differences in the health care system and population (23 % of patients died) compared to higher-income settings in which the scores were developed. Risk-stratification scores developed in this setting are needed to provide the necessary accuracy to aid triage of patients with suspected COVID-19.

LMIC-PRIEST: Derivation and validation of a clinical severity score for acutely ill adults with suspected COVID-19 in a middle-income setting.

BACKGROUND: Uneven vaccination and less resilient health care systems mean hospitals in LMICs are at risk of being overwhelmed during periods of increased COVID-19 infection. Risk-scores proposed for rapid triage of need for admission from the emergency department (ED) have been developed in higher-income settings during initial waves of the pandemic. METHODS: Routinely collected data for public hospitals in the Western Cape, South Africa from the 27th August 2020 to 11th March 2022 were used to derive a cohort of 446,084 ED patients with suspected COVID-19. The primary outcome was death or ICU admission at 30 days. The cohort was divided into derivation and Omicron variant validation sets. We developed the LMIC-PRIEST score based on the coefficients from multivariable analysis in the derivation cohort and existing triage practices. We externally validated accuracy in the Omicron period and a UK cohort. RESULTS: We analysed 305,564 derivation, 140,520 Omicron and 12,610 UK validation cases. Over 100 events per predictor parameter were modelled. Multivariable analyses identified eight predictor variables retained across models. We used these findings and clinical judgement to develop a score based on South African Triage Early Warning Scores and also included age, sex, oxygen saturation, inspired oxygen, diabetes and heart disease. The LMIC-PRIEST score achieved C-statistics: 0.82 (95% CI: 0.82 to 0.83) development cohort; 0.79 (95% CI: 0.78 to 0.80) Omicron cohort; and 0.79 (95% CI: 0.79 to 0.80) UK cohort. Differences in prevalence of outcomes led to imperfect calibration in external validation. However, use of the score at thresholds of three or less would allow identification of very low-risk patients (NPV ≥0.99) who could be rapidly discharged using information collected at initial assessment. CONCLUSION: The LMIC-PRIEST score shows good discrimination and high sensitivity at lower thresholds and can be used to rapidly identify low-risk patients in LMIC ED settings.

Training and testing of a gradient boosted machine learning model to predict adverse outcome in patients presenting to emergency departments with suspected covid-19 infection in a middle-income setting.

COVID-19 infection rates remain high in South Africa. Clinical prediction models may be helpful for rapid triage, and supporting clinical decision making, for patients with suspected COVID-19 infection. The Western Cape, South Africa, has integrated electronic health care data facilitating large-scale linked routine datasets. The aim of this study was to develop a machine learning model to predict adverse outcome in patients presenting with suspected COVID-19 suitable for use in a middle-income setting. A retrospective cohort study was conducted using linked, routine data, from patients presenting with suspected COVID-19 infection to public-sector emergency departments (EDs) in the Western Cape, South Africa between 27th August 2020 and 31st October 2021. The primary outcome was death or critical care admission at 30 days. An XGBoost machine learning model was trained and internally tested using split-sample validation. External validation was performed in 3 test cohorts: Western Cape patients presenting during the Omicron COVID-19 wave, a UK cohort during the ancestral COVID-19 wave, and a Sudanese cohort during ancestral and Eta waves. A total of 282,051 cases were included in a complete case training dataset. The prevalence of 30-day adverse outcome was 4.0%. The most important features for predicting adverse outcome were the requirement for supplemental oxygen, peripheral oxygen saturations, level of consciousness and age. Internal validation using split-sample test data revealed excellent discrimination (C-statistic 0.91, 95% CI 0.90 to 0.91) and calibration (CITL of 1.05). The model achieved C-statistics of 0.84 (95% CI 0.84 to 0.85), 0.72 (95% CI 0.71 to 0.73), and 0.62, (95% CI 0.59 to 0.65) in the Omicron, UK, and Sudanese test cohorts. Results were materially unchanged in sensitivity analyses examining missing data. An XGBoost machine learning model achieved good discrimination and calibration in prediction of adverse outcome in patients presenting with suspected COVID19 to Western Cape EDs. Performance was reduced in temporal and geographical external validation.

LMIC-PRIEST: Derivation and validation of a clinical severity score for acutely ill adults with suspected COVID-19 in a middle-income setting.

Background: Uneven vaccination and less resilient health care systems mean hospitals in LMICs are at risk of being overwhelmed during periods of increased COVID-19 infection. Risk-scores proposed for rapid triage of need for admission from the emergency department (ED) have been developed in higher-income settings during initial waves of the pandemic. Methods: Routinely collected data for public hospitals in the Western Cape, South Africa from the 27 th August 2020 to 11 th March 2022 were used to derive a cohort of 446,084 ED patients with suspected COVID-19. The primary outcome was death or ICU admission at 30 days. The cohort was divided into derivation and Omicron variant validation sets. We developed the LMIC-PRIEST score based on the coefficients from multivariable analysis in the derivation cohort and existing triage practices. We externally validated accuracy in the Omicron period and a UK cohort. Results: We analysed 305,564, derivation 140,520 Omicron and 12,610 UK validation cases. Over 100 events per predictor parameter were modelled. Multivariable analyses identified eight predictor variables retained across models. We used these findings and clinical judgement to develop a score based on South African Triage Early Warning Scores and also included age, sex, oxygen saturation, inspired oxygen, diabetes and heart disease. The LMIC-PRIEST score achieved C-statistics: 0.82 (95% CI: 0.82 to 0.83) development cohort; 0.79 (95% CI: 0.78 to 0.80) Omicron cohort; and 0.79 (95% CI: 0.79 to 0.80) UK cohort. Differences in prevalence of outcomes led to imperfect calibration in external validation. However, use of the score at thresholds of three or less would allow identification of very low-risk patients (NPV ≥0.99) who could be rapidly discharged using information collected at initial assessment. Conclusion: The LMIC-PRIEST score shows good discrimination and high sensitivity at lower thresholds and can be used to rapidly identify low-risk patients in LMIC ED settings. What is already known on this subject: Uneven vaccination in low- and middle-income countries (LMICs) coupled with less resilient health care provision mean that emergency health care systems in LMICs may still be at risk of being overwhelmed during periods of increased COVID-19 infection.Risk-stratification scores may help rapidly triage need for hospitalisation. However, those proposed for use in the ED for patients with suspected COVID-19 have been developed and validated in high-income settings. What this study adds: The LMIC-PRIEST score has been robustly developed using a large routine dataset from the Western Cape, South Africa and is directly applicable to existing triage practices in LMICs.External validation across both income settings and COVID-19 variants showed good discrimination and high sensitivity (at lower thresholds) to a composite outcome indicating need for inpatient admission from the ED. How this study might affect research practice or policy: Use of the LMIC-PRIEST score at thresholds of three or less would allow identification of very low-risk patients (negative predictive value ≥0.99) across all settings assessedDuring periods of increased demand, this could allow the rapid identification and discharge of patients from the ED using information collected at initial assessment.

Accuracy of emergency medical service telephone triage of need for an ambulance response in suspected COVID-19: an observational cohort study.

OBJECTIVE: To assess accuracy of emergency medical service (EMS) telephone triage in identifying patients who need an EMS response and identify factors which affect triage accuracy. DESIGN: Observational cohort study. SETTING: Emergency telephone triage provided by Yorkshire Ambulance Service (YAS) National Health Service (NHS) Trust. PARTICIPANTS: 12 653 adults who contacted EMS telephone triage services provided by YAS between 2 April 2020 and 29 June 2020 assessed by COVID-19 telephone triage pathways were included. OUTCOME: Accuracy of call handler decision to dispatch an ambulance was assessed in terms of death or need for organ support at 30 days from first contact with the telephone triage service. RESULTS: Callers contacting EMS dispatch services had an 11.1% (1405/12 653) risk of death or needing organ support. In total, 2000/12 653 (16%) of callers did not receive an emergency response and they had a 70/2000 (3.5%) risk of death or organ support. Ambulances were dispatched to 4230 callers (33.4%) who were not conveyed to hospital and did not deteriorate. Multivariable modelling found variables of older age (1 year increase, OR: 1.05, 95% CI: 1.04 to 1.05) and presence of pre-existing respiratory disease (OR: 1.35, 95% CI: 1.13 to 1.60) to be predictors of false positive triage. CONCLUSION: Telephone triage can reduce ambulance responses but, with low specificity. A small but significant proportion of patients who do not receive an initial emergency response deteriorated. Research to improve accuracy of EMS telephone triage is needed and, due to limitations of routinely collected data, this is likely to require prospective data collection.

Accuracy of telephone triage for predicting adverse outcomes in suspected COVID-19: an observational cohort study.

OBJECTIVE: To assess accuracy of telephone triage in identifying need for emergency care among those with suspected COVID-19 infection and identify factors which affect triage accuracy. DESIGN: Observational cohort study. SETTING: Community telephone triage provided in the UK by Yorkshire Ambulance Service NHS Trust (YAS). PARTICIPANTS: 40 261 adults who contacted National Health Service (NHS) 111 telephone triage services provided by YAS between 18 March 2020 and 29 June 2020 with symptoms indicating COVID-19 infection were linked to Office for National Statistics death registrations and healthcare data collected by NHS Digital. OUTCOME: Accuracy of triage disposition was assessed in terms of death or need for organ support up to 30 days from first contact. RESULTS: Callers had a 3% (1200/40 261) risk of serious adverse outcomes (death or organ support). Telephone triage recommended self-care or non-urgent assessment for 60% (24 335/40 261), with a 1.3% (310/24 335) risk of adverse outcomes. Telephone triage had 74.2% sensitivity (95% CI: 71.6 to 76.6%) and 61.5% specificity (95% CI: 61% to 62%) for the primary outcome. Multivariable analysis suggested respiratory comorbidities may be overappreciated, and diabetes underappreciated as predictors of deterioration. Repeat contact with triage service appears to be an important under-recognised predictor of deterioration with 2 contacts (OR 1.77, 95% CI: 1.14 to 2.75) and 3 or more contacts (OR 4.02, 95% CI: 1.68 to 9.65) associated with false negative triage. CONCLUSION: Patients advised to self-care or receive non-urgent clinical assessment had a small but non-negligible risk of serious clinical deterioration. Repeat contact with telephone services needs recognition as an important predictor of subsequent adverse outcomes.