Evaluation of S/F94 as a proxy for COVID-19 severity

Optimising statistical power in early-stage trials and observational studies accelerates discovery and improves the reliability of results. Ideally, intermediate outcomes should be continuously distributed and lie on the causal pathway between an intervention and a definitive outcome such as mortality. In order to optimise power for an intermediate outcome in the RECOVERY trial, we devised and evaluated a modification to a simple, pragmatic measure of oxygenation function - the SaO2/FIO2 (S/F) ratio. We demonstrate that, because of the ceiling effect in oxyhaemoglobin saturation, S/F ceases to reflect pulmonary oxygenation function at high values of SaO2. Using synthetic and real data, we found that the correlation of S/F with a gold standard (PaO2/FIO2, P/F ratio) improved substantially when measurements with SaO2 [≥] 0.94 are excluded (Spearman r, synthetic data: S/F : 0.31; S/F94: 0.85). We refer to this measure as S/F94. In order to test the underlying assumptions and validity of S/F94 as a predictor of a definitive outcome (mortality), we collected an observational dataset including over 39,000 hospitalised patients with COVID-19 in the ISARIC4C study. We first demonstrated that S/F94 is predictive of mortality in COVID-19. We then compared the sample sizes required for trials using different outcome measures (S/F94, the WHO ordinal scale, sustained improvement at day 28 and mortality at day 28) ensuring comparable effect sizes. The smallest sample size was needed when S/F94 on day 5 was used as an outcome measure. To facilitate future study design, we provide an online user interface to quantify real-world power for a range of outcomes and inclusion criteria, using a synthetic dataset retaining the population-level clinical associations in real data accrued in ISARIC4C https://isaric4c.net/endpoints. We demonstrated that S/F94 is superior to S/F as a measure of pulmonary oxygenation function and is an effective intermediate outcome measure in COVID-19. It is a simple and non-invasive measurement, representative of disease severity and provides greater statistical power to detect treatment differences than other intermediate endpoints.

Blood transcriptional biomarkers of acute viral infection for detection of pre-symptomatic SARS-CoV 2 infection

We hypothesised that host-response biomarkers of viral infections may contribute to early identification of SARS-CoV-2 infected individuals, critical to breaking chains of transmission. We identified 20 candidate blood transcriptomic signatures of viral infection by systematic review and evaluated their ability to detect SARS-CoV-2 infection, compared to the gold-standard of virus PCR tests, among a prospective cohort of 400 hospital staff subjected to weekly testing when fit to attend work. The transcriptional signatures had limited overlap, but were mostly co-correlated as components of type 1 interferon responses. We reconstructed each signature score in blood RNA sequencing data from 41 individuals over sequential weeks spanning a first positive SARS-CoV-2 PCR, and after 6-month convalescence. A single blood transcript for IFI27 provided the highest accuracy for discriminating individuals at the time of their first positive viral PCR result from uninfected controls, with area under the receiver operating characteristic curve (AUROC) of 0.95 (95% confidence interval 0.91-0.99), sensitivity 0.84 (0.7-0.93) and specificity 0.95 (0.85-0.98) at a predefined test threshold. The test performed equally well in individuals with and without symptoms, correlated with viral load, and identified incident infections one week before the first positive viral PCR with sensitivity 0.4 (0.17-0.69) and specificity 0.95 (0.85-0.98). Our findings strongly support further urgent evaluation and development of blood IFI27 transcripts as a biomarker for early phase SARS-CoV-2 infection, for screening individuals such as contacts of index cases, in order to facilitate early case isolation and early antiviral treatments as they emerge.

Characterising heterogeneity and sero-reversion in antibody responses to mild SARS-CoV-2 infection: a cohort study using time series analysis and mechanistic modelling

BackgroundSARS-CoV-2 serology is used to identify prior infection at individual and at population level. Extended longitudinal studies with multi-timepoint sampling to evaluate dynamic changes in antibody levels are required to identify the time horizon in which these applications of serology are valid, and to explore the longevity of protective humoral immunity. MethodsHealth-care workers were recruited to a prospective cohort study from the first SARS-CoV-2 epidemic peak in London, undergoing weekly symptom screen, viral PCR and blood sampling over 16-21 weeks. Serological analysis (n=12,990) was performed using semi-quantitative Euroimmun IgG to viral spike S1 domain and Roche total antibody to viral nucleocapsid protein (NP) assays. Comparisons were made to previously reported pseudovirus neutralising antibody measurements. FindingsA total of 157/729 (21.5%) participants developed positive SARS-CoV-2 serology by one or other assay, of whom 31.0% were asymptomatic and there were no deaths. Peak Euroimmun anti-S1 and Roche anti-NP measurements correlated (r=0.57, p<0.0001) but only anti-S1 measurements correlated with near-contemporary pseudovirus neutralising antibody titres (measured at 16-18 weeks, r=0.57, p<0.0001). By 21 weeks follow-up, 31/143 (21.7%) anti-S1 and 6/150 (4.0%) anti-NP measurements reverted to negative. Mathematical modelling suggested faster clearance of anti-S1 compared to anti-NP (median half-life of 2.5 weeks versus 4.0 weeks), earlier transition to lower levels of antibody production (median of 8 versus 13 weeks), and greater reductions in relative antibody production rate after the transition (median of 35% versus 50%). InterpretationMild SARS-CoV-2 infection is associated with heterogenous serological responses in Euroimmun anti-S1 and Roche anti-NP assays. Anti-S1 responses showed faster rates of clearance, more rapid transition from high to low level production rate and greater reduction in production rate after this transition. The application of individual assays for diagnostic and epidemiological serology requires validation in time series analysis. FundingCharitable donations via Barts Charity Research in contextO_ST_ABSEvidence before this studyC_ST_ABSWe searched PubMed, medRxiv, and bioRxiv for ["antibody" OR "serology"] AND ["SARS-CoV-2" OR "COVID-19"]. The available literature highlights widespread use of serology to detect recent SARS-CoV-2 infection in individual patients and in population epidemiological surveys. Antibody to virus spike protein S1 domain is widely reported to correlate with neutralising antibody titres. The existing assays have good sensitivity to detect seroconversion within 14 days of incident infection, but the available longitudinal studies have reported variable rates of decline in antibody levels and reversion to undetectable levels in some people over 3 months. High frequency multi-time point serology data for different antibody targets or assays in longitudinal cohorts from the time of incident infection to greater than 3 months follow up are lacking. Added value of this studyWe combine detailed longitudinal serology using the Euroimmun anti-S1 and Roche anti-nucleocapsid protein (NP) assays in 731 health care workers from the time of the first SARS-CoV-2 epidemic peak in London, UK. In 157 seroconverters (using either assay) we show substantial heterogeneity in semiquantitative antibody measurements over time between individuals and between assays. Mathematical modelling of individual participant antibody production and clearance rates in individuals with at least 8 data points over 21 weeks showed anti-S1 antibodies to have a faster clearance rate, earlier transition from the initial antibody production rate to lower rates, and greater reduction in antibody production rate after this transition, compared to anti-NP antibodies as measured by these assays. As a result, Euroimmun anti-S1 measurements peaked earlier and then reduced more rapidly than Roche anti-NP measurements. In this study, these differences led to 21% anti-S1 sero-reversion, compared to 4% anti-NP sero-reversion over 4-5 months. Implications of all of the available evidenceThe rapid decline in anti-S1 antibodies measured by the Euroimmun assay following infection limits its application for diagnostic and epidemiological screening. If generalisable, these data are consistent with the hypothesis that anti-S1 mediated humoral immunity may not be sustained in some people beyond the initial post-infective period. Further work is required to understand the mechanisms behind the heterogeneity in antibody kinetics between individuals to SARS-CoV-2. Our data point to differential mechanisms regulating humoral immunity against these two viral targets.

Development and validation of the 4C Deterioration model for adults hospitalised with COVID-19

Prognostic models to predict the risk of clinical deterioration in acute COVID-19 are required to inform clinical management decisions. Among 75,016 consecutive adults across England, Scotland and Wales prospectively recruited to the ISARIC Coronavirus Clinical Characterisation Consortium (ISARIC4C) study, we developed and validated a multivariable logistic regression model for in-hospital clinical deterioration (defined as any requirement of ventilatory support or critical care, or death) using 11 routinely measured variables. We used internal-external cross-validation to show consistent measures of discrimination, calibration and clinical utility across eight geographical regions. We further validated the final model in held-out data from 8,252 individuals in London, with similarly consistent performance (C-statistic 0.77 (95% CI 0.75 to 0.78); calibration-in-the-large 0.01 (-0.04 to 0.06); calibration slope 0.96 (0.90 to 1.02)). Importantly, this model demonstrated higher net benefit than using other candidate scores to inform decision-making. Our 4C Deterioration model thus demonstrates unprecedented clinical utility and generalisability to predict clinical deterioration among adults hospitalised with COVID-19.

Risk stratification of patients admitted to hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: development and validation of the 4C Mortality Score

ObjectivesTo develop and validate a pragmatic risk score to predict mortality for patients admitted to hospital with covid-19. DesignProspective observational cohort study: ISARIC WHO CCP-UK study (ISARIC Coronavirus Clinical Characterisation Consortium [4C]). Model training was performed on a cohort of patients recruited between 6 February and 20 May 2020, with validation conducted on a second cohort of patients recruited between 21 May and 29 June 2020. Setting260 hospitals across England, Scotland, and Wales. ParticipantsAdult patients ([≥]18 years) admitted to hospital with covid-19 admitted at least four weeks before final data extraction. Main outcome measuresIn-hospital mortality. ResultsThere were 34 692 patients included in the derivation dataset (mortality rate 31.7%) and 22 454 in the validation dataset (mortality 31.5%). The final 4C Mortality Score included eight variables readily available at initial hospital assessment: age, sex, number of comorbidities, respiratory rate, peripheral oxygen saturation, level of consciousness, urea, and C-reactive protein (score range 0-21 points). The 4C risk stratification score demonstrated high discrimination for mortality (derivation cohort: AUROC 0.79; 95% CI 0.78 - 0.79; validation cohort 0.78, 0.77-0.79) with excellent calibration (slope = 1.0). Patients with a score [≥]15 (n = 2310, 17.4%) had a 67% mortality (i.e., positive predictive value 67%) compared with 1.0% mortality for those with a score [≤]3 (n = 918, 7%; negative predictive value 99%). Discriminatory performance was higher than 15 pre-existing risk stratification scores (AUROC range 0.60-0.76), with scores developed in other covid-19 cohorts often performing poorly (range 0.63-0.73). ConclusionsWe have developed and validated an easy-to-use risk stratification score based on commonly available parameters at hospital presentation. This outperformed existing scores, demonstrated utility to directly inform clinical decision making, and can be used to stratify inpatients with covid-19 into different management groups. The 4C Mortality Score may help clinicians identify patients with covid-19 at high risk of dying during current and subsequent waves of the pandemic. Study registrationISRCTN66726260

Systematic evaluation and external validation of 22 prognostic models among hospitalised adults with COVID-19: An observational cohort study

BackgroundThe number of proposed prognostic models for COVID-19, which aim to predict disease outcomes, is growing rapidly. It is not known whether any are suitable for widespread clinical implementation. We addressed this question by independent and systematic evaluation of their performance among hospitalised COVID-19 cases. MethodsWe conducted an observational cohort study to assess candidate prognostic models, identified through a living systematic review. We included consecutive adults admitted to a secondary care hospital with PCR-confirmed or clinically diagnosed community-acquired COVID-19 (1st February to 30th April 2020). We reconstructed candidate models as per their original descriptions and evaluated performance for their original intended outcomes (clinical deterioration or mortality) and time horizons. We assessed discrimination using the area under the receiver operating characteristic curve (AUROC), and calibration using calibration plots, slopes and calibration-in-the-large. We calculated net benefit compared to the default strategies of treating all and no patients, and against the most discriminating predictor in univariable analyses, based on a limited subset of a priori candidates. ResultsWe tested 22 candidate prognostic models among a cohort of 411 participants, of whom 180 (43.8%) and 115 (28.0%) met the endpoints of clinical deterioration and mortality, respectively. The highest AUROCs were achieved by the NEWS2 score for prediction of deterioration over 24 hours (0.78; 95% CI 0.73-0.83), and a novel model for prediction of deterioration <14 days from admission (0.78; 0.74-0.82). Calibration appeared generally poor for models that used probability outcomes. In univariable analyses, admission oxygen saturation on room air was the strongest predictor of in-hospital deterioration (AUROC 0.76; 0.71-0.81), while age was the strongest predictor of in-hospital mortality (AUROC 0.76; 0.71-0.81). No prognostic model demonstrated consistently higher net benefit than using the most discriminating univariable predictors to stratify treatment, across a range of threshold probabilities. ConclusionsOxygen saturation on room air and patient age are strong predictors of deterioration and mortality among hospitalised adults with COVID-19, respectively. None of the prognostic models evaluated offer incremental value for patient stratification to these univariable predictors.

Supplementing the National Early Warning Score (NEWS2) for anticipating early deterioration among patients with COVID-19 infection

BackgroundThe National Early Warning Score (NEWS2) is currently recommended in the United Kingdom for risk stratification of COVID outcomes, but little is known about its ability to detect severe cases. We aimed to evaluate NEWS2 for severe COVID outcome and identify and validate a set of routinely-collected blood and physiological parameters taken at hospital admission to improve the score. MethodsTraining cohorts comprised 1276 patients admitted to Kings College Hospital NHS Foundation Trust with COVID-19 disease from 1st March to 30th April 2020. External validation cohorts included 5037 patients from four UK NHS Trusts (Guys and St Thomas Hospitals, University Hospitals Southampton, University Hospitals Bristol and Weston NHS Foundation Trust, University College London Hospitals), and two hospitals in Wuhan, China (Wuhan Sixth Hospital and Taikang Tongji Hospital). The outcome was severe COVID disease (transfer to intensive care unit or death) at 14 days after hospital admission. Age, physiological measures, blood biomarkers, sex, ethnicity and comorbidities (hypertension, diabetes, cardiovascular, respiratory and kidney diseases) measured at hospital admission were considered in the models. ResultsA baseline model of NEWS2 + age had poor-to-moderate discrimination for severe COVID infection at 14 days (AUC in training sample = 0.700; 95% CI: 0.680, 0.722; Brier score = 0.192; 95% CI: 0.186, 0.197). A supplemented model adding eight routinely-collected blood and physiological parameters (supplemental oxygen flow rate, urea, age, oxygen saturation, CRP, estimated GFR, neutrophil count, neutrophil/lymphocyte ratio) improved discrimination (AUC = 0.735; 95% CI: 0.715, 0.757) and these improvements were replicated across five UK and non-UK sites. However, there was evidence of miscalibration with the model tending to underestimate risks in most sites. ConclusionsNEWS2 score had poor-to-moderate discrimination for medium-term COVID outcome which raises questions about its use as a screening tool at hospital admission. Risk stratification was improved by including readily available blood and physiological parameters measured at hospital admission, but there was evidence of miscalibration in external sites. This highlights the need for a better understanding of the use of early warning scores for COVID. KO_SCPLOWEYC_SCPLOWO_SCPCAP C_SCPCAPO_SCPLOWMESSAGESC_SCPLOWO_LIThe National Early Warning Score (NEWS2), currently recommended for stratification of severe COVID-19 disease in the UK, showed poor-to-moderate discrimination for medium-term outcomes (14-day transfer to ICU or death) among COVID-19 patients. C_LIO_LIRisk stratification was improved by the addition of routinely-measured blood and physiological parameters routinely at hospital admission (supplemental oxygen, urea, oxygen saturation, CRP, estimated GFR, neutrophil count, neutrophil/lymphocyte ratio) which provided moderate improvements in a risk stratification model for 14-day ICU/death. C_LIO_LIThis improvement over NEWS2 alone was maintained across multiple hospital trusts but the model tended to be miscalibrated with risks of severe outcomes underestimated in most sites. C_LIO_LIWe benefited from existing pipelines for informatics at KCH such as CogStack that allowed rapid extraction and processing of electronic health records. This methodological approach provided rapid insights and allowed us to overcome the complications associated with slow data centralisation approaches. C_LI