Protocol to assess performance of crisis standards of care guidelines for clinical triage.

During the COVID-19 pandemic, US states developed Crisis Standards of Care (CSC) algorithms to triage allocation of scarce resources to maximize population-wide benefit. While CSC algorithms were developed by ethical debate, this protocol guides their quantitative assessment. For CSC algorithms, this protocol addresses (1) adapting algorithms for empirical study, (2) quantifying predictive accuracy, and (3) simulating clinical decision-making. This protocol provides a framework for healthcare systems and governments to test the performance of CSC algorithms to ensure they meet their stated ethical goals. For complete details on the use and execution of this protocol, please refer to Jezmir et al. (2021).

Empirical Assessment of U.S. Coronavirus Disease 2019 Crisis Standards of Care Guidelines.

To establish the feasibility of empirically testing crisis standards of care guidelines. DESIGN: Retrospective single-center study. SETTING: ICUs at a large academic medical center in the United States. SUBJECTS: Adult, critically ill patients admitted to ICU, with 27 patients admitted for acute respiratory failure due to coronavirus disease 2019 and 37 patients admitted for diagnoses other than coronavirus disease 2019. INTERVENTIONS: Review of electronic health record. MEASUREMENTS AND MAIN RESULTS: Many U.S. states released crisis standards of care guidelines with algorithms to allocate scarce healthcare resources during the coronavirus disease 2019 pandemic. We compared state guidelines that represent different approaches to incorporating disease severity and comorbidities: New York, Maryland, Pennsylvania, and Colorado. Following each algorithm, we calculated priority scores at the time of ICU admission for a cohort of patients with primary diagnoses of coronavirus disease 2019 and diseases other than coronavirus disease 2019 (n = 64). We assessed discrimination of 28-day mortality by area under the receiver operating characteristic curve. We simulated real-time decision-making by applying the triage algorithms to groups of two, five, or 10 patients. For prediction of 28-day mortality by priority scores, area under the receiver operating characteristic curve was 0.56, 0.49, 0.53, 0.66, and 0.69 for New York, Maryland, Pennsylvania, Colorado, and raw Sequential Organ Failure Assessment score algorithms, respectively. For groups of five patients, the percentage of decisions made without deferring to a lottery were 1%, 57%, 80%, 88%, and 95% for New York, Maryland, Pennsylvania, Colorado, and raw Sequential Organ Failure Assessment score algorithms, respectively. The percentage of decisions made without lottery was higher in the subcohort without coronavirus disease 2019, compared with the subcohort with coronavirus disease 2019. CONCLUSIONS: Inclusion of comorbidities does not consistently improve an algorithm's performance in predicting 28-day mortality. Crisis standards of care algorithms result in a substantial percentage of tied priority scores. Crisis standards of care algorithms operate differently in cohorts with and without coronavirus disease 2019. This proof-of-principle study demonstrates the feasibility and importance of empirical testing of crisis standards of care guidelines to understand whether they meet their goals.

Performance of crisis standards of care guidelines in a cohort of critically ill COVID-19 patients in the United States.

Many US states published crisis standards of care (CSC) guidelines for allocating scarce critical care resources during the COVID-19 pandemic. However, the performance of these guidelines in maximizing their population benefit has not been well tested. In 2,272 adults with COVID-19 requiring mechanical ventilation drawn from the Study of the Treatment and Outcomes in Critically Ill Patients with COVID-19 (STOP-COVID) multicenter cohort, we test the following three approaches to CSC algorithms: Sequential Organ Failure Assessment (SOFA) scores grouped into ranges, SOFA score ranges plus comorbidities, and a hypothetical approach using raw SOFA scores not grouped into ranges. We find that area under receiver operating characteristic (AUROC) curves for all three algorithms demonstrate only modest discrimination for 28-day mortality. Adding comorbidity scoring modestly improves algorithm performance over SOFA scores alone. The algorithm incorporating comorbidities has modestly worse predictive performance for Black compared to white patients. CSC algorithms should be empirically examined to refine approaches to the allocation of scarce resources during pandemics and to avoid potential exacerbation of racial inequities.

Empirical Assessment of COVID-19 Crisis Standards of Care Guidelines.

BACKGROUND: Several states have released Crisis Standards of Care (CSC) guidelines for the allocation of scarce critical care resources. Most guidelines rely on Sequential Organ Failure Assessment (SOFA) scores to maximize lives saved, but states have adopted different stances on whether to maximize long-term outcomes (life-years saved) by accounting for patient comorbidities. METHODS: We compared 4 representative state guidelines with varying approaches to comorbidities and analyzed how CSC prioritization correlates with clinical outcomes. We included 27 laboratory-confirmed COVID-19 patients admitted to ICUs at Brigham and Women's Hospital from March 12 to April 3, 2020. We compared prioritization algorithms from New York, which assigns priority based on SOFA alone; Maryland, which uses SOFA plus severe comorbidities; Pennsylvania, which uses SOFA plus major and severe comorbidities; and Colorado, which uses SOFA plus a modified Charlson comorbidity index. RESULTS: In pairwise comparisons across all possible pairs, we found that state guidelines frequently resulted in tie-breakers based on age or lottery: New York 100% of the time (100% resolved by lottery), Pennsylvania 86% of the time (18% by lottery), Maryland 93% of the time (35% by lottery), and Colorado: 32% of the time (10% by lottery). The prioritization algorithm with the strongest correlation with 14-day outcomes was Colorado (rs = -0.483. p = 0.011) followed by Maryland (rs = -0.394, p =0.042), Pennsylvania (rs = -0.382, p = 0.049), and New York (rs = 0). An alternative model using raw SOFA scores alone was moderately correlated with outcomes (rs = -0.448, p = 0.019). CONCLUSIONS: State guidelines for scarce resource allocation frequently resulted in identical priority scores, requiring tie-breakers based on age or lottery. These findings suggest that state CSC guidelines should be further assessed empirically to understand whether they meet their goals.

Inflammatory Biomarker Trends Predict Respiratory Decline in COVID-19 Patients.

In this single-center, retrospective cohort analysis of hospitalized coronavirus disease 2019 (COVID-19) patients, we investigate whether inflammatory biomarker levels predict respiratory decline in patients who initially present with stable disease. Examination of C-reactive protein (CRP) trends reveals that a rapid rise in CRP levels precedes respiratory deterioration and intubation, although CRP levels plateau in patients who remain stable. Increasing CRP during the first 48 h of hospitalization is a better predictor (with higher sensitivity) of respiratory decline than initial CRP levels or ROX indices (a physiological score of respiratory function). CRP, the proinflammatory cytokine interleukin-6 (IL-6), and physiological measures of hypoxemic respiratory failure are correlated, which suggests a mechanistic link. Our work shows that rising CRP predicts subsequent respiratory deterioration in COVID-19 and may suggest mechanistic insight and a potential role for targeted immunomodulation in a subset of patients early during hospitalization.