Observational methods for COVID-19 vaccine effectiveness research: a trial emulation and empirical evaluation

Despite much research on the topic, little work has been done comparing the use of methods to control for confounding in the estimation of COVID-19 vaccine effectiveness in routinely collected medical record data. We conducted a trial emulation study to replicate the ChAdOx1 (Oxford/AstraZeneca) and BNT162b2 (BioNTech/Pfizer) COVID-19 phase 3 efficacy studies. We conducted a cohort study including individuals aged 75+ from UK CPRD AURUM (N = 916,128) in early 2021. Three different methods were assessed: Overlap weighting, inverse probability treatment weighting, and propensity score matching. All three methods successfully replicated the findings from both phase 3 trials, and overlap weighting performed best in terms of confounding, systematic error, and precision. Despite lack of trial data beyond 3 weeks, we found that even 1 dose of BNT162b2 was effective against SARS-CoV-2 infection for up to 12 weeks before a second dose was administered. These results support the UK Joint Committee on Vaccination and Immunisation modelling and related UK vaccination strategies implemented in early 2021. Key messagesO_LIReal world evidence generated using weighting (overlapping weights and inverse probability of treatment weights) and propensity score matching: all methods successfully replicate the findings of Phase 3 trials for COVID-19 vaccine effectiveness. C_LIO_LIOverlap weighting provides the least biased estimates in our study and should be considered amongst the most suitable methods for future COVID-19 vaccine effectiveness research. C_LIO_LIDespite a lack of trial data, our findings suggest that first-dose BNT162b2 provides effective protection against SARS-COV-2 infection for up to 12 weeks, in line with UKs Joint Committee on Vaccination and Immunisation modelling and subsequent vaccination strategies. C_LI

Predictive performance of multi-model ensemble forecasts of COVID-19 across European nations

BackgroundShort-term forecasts of infectious disease burden can contribute to situational awareness and aid capacity planning. Based on best practice in other fields and recent insights in infectious disease epidemiology, one can maximise the predictive performance of such forecasts if multiple models are combined into an ensemble. Here we report on the performance of ensembles in predicting COVID-19 cases and deaths across Europe between 08 March 2021 and 07 March 2022. MethodsWe used open-source tools to develop a public European COVID-19 Forecast Hub. We invited groups globally to contribute weekly forecasts for COVID-19 cases and deaths reported from a standardised source over the next one to four weeks. Teams submitted forecasts from March 2021 using standardised quantiles of the predictive distribution. Each week we created an ensemble forecast, where each predictive quantile was calculated as the equally-weighted average (initially the mean and then from 26th July the median) of all individual models predictive quantiles. We measured the performance of each model using the relative Weighted Interval Score (WIS), comparing models forecast accuracy relative to all other models. We retrospectively explored alternative methods for ensemble forecasts, including weighted averages based on models past predictive performance. ResultsOver 52 weeks we collected and combined up to 28 forecast models for 32 countries. We found a weekly ensemble had a consistently strong performance across countries over time. Across all horizons and locations, the ensemble performed better on relative WIS than 84% of participating models forecasts of incident cases (with a total N=862), and 92% of participating models forecasts of deaths (N=746). Across a one to four week time horizon, ensemble performance declined with longer forecast periods when forecasting cases, but remained stable over four weeks for incident death forecasts. In every forecast across 32 countries, the ensemble outperformed most contributing models when forecasting either cases or deaths, frequently outperforming all of its individual component models. Among several choices of ensemble methods we found that the most influential and best choice was to use a median average of models instead of using the mean, regardless of methods of weighting component forecast models. ConclusionsOur results support the use of combining forecasts from individual models into an ensemble in order to improve predictive performance across epidemiological targets and populations during infectious disease epidemics. Our findings further suggest that median ensemble methods yield better predictive performance more than ones based on means. Our findings also highlight that forecast consumers should place more weight on incident death forecasts than incident case forecasts at forecast horizons greater than two weeks. Code and data availabilityAll data and code are publicly available on Github: covid19-forecast-hub-europe/euro-hub-ensemble.

The impact of COVID passport mandates on the number of cases of and hospitalizations with COVID-19 in the UK: a difference-in-differences analysis

BackgroundMandatory COVID-19 certification was introduced at different times in the four countries of the UK. We aimed to study the effect of this intervention on the incidence of cases and hospital admissions. MethodsThe main outcome was the weekly averaged incidence of COVID-19 confirmed cases and hospital admissions. We performed Negative Binomial Segmented Regression (NBSR) and Autoregressive Integrated Moving Average (ARIMA) analyses for the four countries (England, Northern Ireland, Scotland and Wales), and fitted Difference-in-Differences (DiD) models to compare the latter three to England, where COVID-19 certification was imposed the latest. FindingsNBSR methods suggested COVID-19 certification led to a decrease in the incidence of cases in Northern Ireland, but not in hospitalizations. In Wales, they also caused a decrease in the incidence of cases but not in hospital admissions. In Scotland, we observed a decrease in both cases and admissions. ARIMA models confirmed these results. The DiD model showed that the intervention decreased the incidence of COVID compared to England in all countries except Wales, in October. Then, the incidence rate of cases already had a decreasing tendency, as well as in England, hence a particular impact of Covid Passport was less obvious. In Wales, the model coefficients were 2.2 (95% CI -6.24,10.70) for cases and -0.144 (95% CI -0.248, -0.039) for admissions in October and -7.75 (95% CI -13.1, -2.46) for cases and -0.169 (95% CI-0.308, -0.031) for admissions in November. In Northern Ireland, -10.1 (95% CI -18.4, -1.79) for cases and -0.269 (95% CI -0.385, -0.153) for admissions. In Scotland they were 7.91 (95% CI 4.46,11.4) for cases and -0.097 (95% CI - 0.219,0.024) for admissions. InterpretationThe introduction of mandatory certificates decreased cases in all countries except in England. Differences on concomitant measures, on vaccination uptake or Omicron variant prevalence could explain this discrepancy.

Epidemiological dynamics of the incidence of COVID-19 in children and the relationship with the opening of schools in Catalonia (Spain)

Here we analyse the epidemiological trend of the incidence of COVID-19 in children in Catalonia (Spain) during the first 20 weeks of the 2020-2021 school year. This study demonstrates that while schools were open the incidence rate among children remained significantly lower than in general population, despite a greater diagnostic effort in children. These results suggest that schools have not played a significant role in the SARS-CoV-2 dissemination in Catalonia.

Empiric model for short-time prediction of COVID-19spreading

Covid-19 appearance and fast spreading took by surprise the international community. Collaboration between researchers, public health workers and politicians has been established to deal with the epidemic. One important contribution from researchers in epidemiology is the analysis of trends so that both current state and short-term future trends can be carefully evaluated. Gompertz model has shown to correctly describe the dynamics of cumulative confirmed cases, since it is characterized by a decrease in growth rate that is able to show the effect of control measures. Thus, it provides a way to systematically quantify the Covid-19 spreading velocity. Moreover, it allows to carry out short-term predictions and long-term estimations that may facilitate policy decisions and the revision of in-place confinement measures and the development of new protocols. This model has been employed to fit the cumulative cases of Covid-19 from several Chinese provinces and from other countries with a successful containment of the disease. Results show that there are systematic differences in spreading velocity between countries. In countries that are in the initial stages of the Covid-19 outbreak, model predictions provide a reliable picture of its short-term evolution and may permit to unveil some characteristics of the long-term evolution. These predictions can also be generalized to short-term hospital and Intensive Care Units (ICU) requirements, which together with the equivalent predictions on mortality provide key information for health officials. Author summaryCovid-19 has brought international scientific community into the eye of a storm. Collaboration between researchers, public health workers and politicians is essential to deal with this challenge. One of the pieces of the puzzle is the analysis of May 7, epidemiological trends so that both current and immediate future situation can be carefully evaluated. For this reason we have daily employed a generic growing function to describe the cumulative cases of Covid-19 in several countries and regions around the world and particularly for European countries during the Covid-19 outbreak in Europe. Our model is completely empiric and it is not using any assumption to make the predictions, only the daily update of new cases. In this manuscript, we detail the methods employed and the degree of confidence we have obtained during this process. This can be used for other researchers collaborating and advising health institutions around the world for the Covid-19 outbreak or any other epidemic that follows the same pattern.

Robust estimation of diagnostic rate and real incidence of COVID-19 for European policymakers

Policymakers need a clear and fast assessment of the real spread of the epidemic of COVID-19 in each of their respective countries. Standard measures of the situation provided by the governments include reported positive cases and total deaths. While total deaths immediately indicate that countries like Italy and Spain have the worst situation as of mid April 2020, on its own, reported cases do not provide a correct picture of the situation. The reason is that different countries diagnose diversely and present very distinctive reported case fatality rate (CFR). The same levels of reported incidence and mortality might hide a very different underlying picture. Here we present a straightforward and robust estimation of the diagnostic rate in each European country. From that estimation we obtain an uniform unbiased incidence of the epidemic. The method to obtain the diagnostic rate is transparent and empiric. The key assumption of the method is that the real CFR in Europe of COVID-19 is not strongly country-dependent. We show that this number is not expected to be biased due to demography nor the way total deaths are reported. The estimation protocol has a dynamic nature, and it has been giving converging numbers for diagnostic rates in all European countries as of mid April 2020. From this diagnostic rate, policy makers can obtain an Effective Potential Growth (EPG) updated everyday providing an unbiased assessment of the countries with more potential to have an uncontrolled situation. The method developed will be used to track possible improvements on the diagnostic rate in European countries as the epidemic evolves.