Infectious disease surveillance needs for the United States: lessons from Covid-19.

The COVID-19 pandemic has highlighted the need to upgrade systems for infectious disease surveillance and forecasting and modeling of the spread of infection, both of which inform evidence-based public health guidance and policies. Here, we discuss requirements for an effective surveillance system to support decision making during a pandemic, drawing on the lessons of COVID-19 in the U.S., while looking to jurisdictions in the U.S. and beyond to learn lessons about the value of specific data types. In this report, we define the range of decisions for which surveillance data are required, the data elements needed to inform these decisions and to calibrate inputs and outputs of transmission-dynamic models, and the types of data needed to inform decisions by state, territorial, local, and tribal health authorities. We define actions needed to ensure that such data will be available and consider the contribution of such efforts to improving health equity.

Title evaluation of FluSight influenza forecasting in the 2021-22 and 2022-23 seasons with a new target laboratory-confirmed influenza hospitalizations.

Accurate forecasts can enable more effective public health responses during seasonal influenza epidemics. For the 2021-22 and 2022-23 influenza seasons, 26 forecasting teams provided national and jurisdiction-specific probabilistic predictions of weekly confirmed influenza hospital admissions for one-to-four weeks ahead. Forecast skill is evaluated using the Weighted Interval Score (WIS), relative WIS, and coverage. Six out of 23 models outperform the baseline model across forecast weeks and locations in 2021-22 and 12 out of 18 models in 2022-23. Averaging across all forecast targets, the FluSight ensemble is the 2nd most accurate model measured by WIS in 2021-22 and the 5th most accurate in the 2022-23 season. Forecast skill and 95% coverage for the FluSight ensemble and most component models degrade over longer forecast horizons. In this work we demonstrate that while the FluSight ensemble was a robust predictor, even ensembles face challenges during periods of rapid change.

Scenario design for infectious disease projections: Integrating concepts from decision analysis and experimental design.

Across many fields, scenario modeling has become an important tool for exploring long-term projections and how they might depend on potential interventions and critical uncertainties, with relevance to both decision makers and scientists. In the past decade, and especially during the COVID-19 pandemic, the field of epidemiology has seen substantial growth in the use of scenario projections. Multiple scenarios are often projected at the same time, allowing important comparisons that can guide the choice of intervention, the prioritization of research topics, or public communication. The design of the scenarios is central to their ability to inform important questions. In this paper, we draw on the fields of decision analysis and statistical design of experiments to propose a framework for scenario design in epidemiology, with relevance also to other fields. We identify six different fundamental purposes for scenario designs (decision making, sensitivity analysis, situational awareness, horizon scanning, forecasting, and value of information) and discuss how those purposes guide the structure of scenarios. We discuss other aspects of the content and process of scenario design, broadly for all settings and specifically for multi-model ensemble projections. As an illustrative case study, we examine the first 17 rounds of scenarios from the U.S. COVID-19 Scenario Modeling Hub, then reflect on future advancements that could improve the design of scenarios in epidemiological settings.

New Vibrio cholerae sequences from Eastern and Southern Africa alter our understanding of regional cholera transmission.

Despite ongoing containment and vaccination efforts, cholera remains prevalent in many countries in sub-Saharan Africa. Part of the difficulty in containing cholera comes from our lack of understanding of how it circulates throughout the region. To better characterize regional transmission, we generated and analyzed 118 Vibrio cholerae genomes collected between 2007-2019 from five different countries in Southern and Eastern Africa. We showed that V. cholerae sequencing can be successful from a variety of sample types and filled in spatial and temporal gaps in our understanding of circulating lineages, including providing some of the first sequences from the 2018-2019 outbreaks in Uganda, Kenya, Tanzania, Zambia, and Malawi. Our results present a complex picture of cholera transmission in the region, with multiple lineages found to be co-circulating within several countries. We also find evidence that previously identified sporadic cases may be from larger, undersampled outbreaks, highlighting the need for careful examination of sampling biases and underscoring the need for continued and expanded cholera surveillance across the African continent.

Reconstructed influenza A/H3N2 infection histories reveal variation in incidence and antibody dynamics over the life course.

Humans experience many influenza infections over their lives, resulting in complex and varied immunological histories. Although experimental and quantitative analyses have improved our understanding of the immunological processes defining an individual's antibody repertoire, how these within-host processes are linked to population-level influenza epidemiology remains unclear. Here, we used a multi-level mathematical model to jointly infer antibody dynamics and individual-level lifetime influenza A/H3N2 infection histories for 1,130 individuals in Guangzhou, China, using 67,683 haemagglutination inhibition (HI) assay measurements against 20 A/H3N2 strains from repeat serum samples collected between 2009 and 2015. These estimated infection histories allowed us to reconstruct historical seasonal influenza patterns and to investigate how influenza incidence varies over time, space and age in this population. We estimated median annual influenza infection rates to be approximately 18% from 1968 to 2015, but with substantial variation between years. 88% of individuals were estimated to have been infected at least once during the study period (2009-2015), and 20% were estimated to have three or more infections in that time. We inferred decreasing infection rates with increasing age, and found that annual attack rates were highly correlated across all locations, regardless of their distance, suggesting that age has a stronger impact than fine-scale spatial effects in determining an individual's antibody profile. Finally, we reconstructed each individual's expected antibody profile over their lifetime and inferred an age-stratified relationship between probability of infection and HI titre. Our analyses show how multi-strain serological panels provide rich information on long term, epidemiological trends, within-host processes and immunity when analyzed using appropriate inference methods, and adds to our understanding of the life course epidemiology of influenza A/H3N2.

flepiMoP: The evolution of a flexible infectious disease modeling pipeline during the COVID-19 pandemic.

The COVID-19 pandemic led to an unprecedented demand for projections of disease burden and healthcare utilization under scenarios ranging from unmitigated spread to strict social distancing policies. In response, members of the Johns Hopkins Infectious Disease Dynamics Group developed flepiMoP (formerly called the COVID Scenario Modeling Pipeline), a comprehensive open-source software pipeline designed for creating and simulating compartmental models of infectious disease transmission and inferring parameters through these models. The framework has been used extensively to produce short-term forecasts and longer-term scenario projections of COVID-19 at the state and county level in the US, for COVID-19 in other countries at various geographic scales, and more recently for seasonal influenza. In this paper, we highlight how the flepiMoP has evolved throughout the COVID-19 pandemic to address changing epidemiological dynamics, new interventions, and shifts in policy-relevant model outputs. As the framework has reached a mature state, we provide a detailed overview of flepiMoP's key features and remaining limitations, thereby distributing flepiMoP and its documentation as a flexible and powerful tool for researchers and public health professionals to rapidly build and deploy large-scale complex infectious disease models for any pathogen and demographic setup.

HIV epidemiologic trends among occupational groups in Rakai, Uganda: A population-based longitudinal study, 1999-2016.

Certain occupations have been associated with heightened risk of HIV acquisition and spread in sub-Saharan Africa, including female bar and restaurant work and male transportation work. However, data on changes in population prevalence of HIV infection and HIV incidence within occupations following mass scale-up of African HIV treatment and prevention programs is very limited. We evaluated prospective data collected between 1999 and 2016 from the Rakai Community Cohort Study, a longitudinal population-based study of 15- to 49-year-old persons in Uganda. Adjusted prevalence risk ratios for overall, treated, and untreated, prevalent HIV infection, and incidence rate ratios for HIV incidence with 95% confidence intervals were estimated using Poisson regression to assess changes in HIV outcomes by occupation. Analyses were stratified by gender. There were 33,866 participants, including 19,113 (56%) women. Overall, HIV seroprevalence declined in most occupational subgroups among men, but increased or remained mostly stable among women. In contrast, prevalence of untreated HIV substantially declined between 1999 and 2016 in most occupations, irrespective of gender, including by 70% among men (12.3 to 4.2%; adjPRR = 0.30; 95%CI:0.23-0.41) and by 78% among women (14.7 to 4.0%; adjPRR = 0.22; 95%CI:0.18-0.27) working in agriculture, the most common self-reported primary occupation. Exceptions included men working in transportation. HIV incidence similarly declined in most occupations, but there were no reductions in incidence among female bar and restaurant workers, women working in local crafts, or men working in transportation. In summary, untreated HIV infection and HIV incidence have declined within most occupational groups in Uganda. However, women working in bars/restaurants and local crafts and men working in transportation continue to have a relatively high burden of untreated HIV and HIV incidence, and as such, should be considered priority populations for HIV programming.

Ensemble2: Scenarios ensembling for communication and performance analysis.

Throughout the COVID-19 pandemic, scenario modeling played a crucial role in shaping the decision-making process of public health policies. Unlike forecasts, scenario projections rely on specific assumptions about the future that consider different plausible states-of-the-world that may or may not be realized and that depend on policy interventions, unpredictable changes in the epidemic outlook, etc. As a consequence, long-term scenario projections require different evaluation criteria than the ones used for traditional short-term epidemic forecasts. Here, we propose a novel ensemble procedure for assessing pandemic scenario projections using the results of the Scenario Modeling Hub (SMH) for COVID-19 in the United States (US). By defining a "scenario ensemble" for each model and the ensemble of models, termed "Ensemble2", we provide a synthesis of potential epidemic outcomes, which we use to assess projections' performance, bypassing the identification of the most plausible scenario. We find that overall the Ensemble2 models are well-calibrated and provide better performance than the scenario ensemble of individual models. The ensemble procedure accounts for the full range of plausible outcomes and highlights the importance of scenario design and effective communication. The scenario ensembling approach can be extended to any scenario design strategy, with potential refinements including weighting scenarios and allowing the ensembling process to evolve over time.

The US COVID-19 and Influenza Scenario Modeling Hubs: Delivering long-term projections to guide policy.

Between December 2020 and April 2023, the COVID-19 Scenario Modeling Hub (SMH) generated operational multi-month projections of COVID-19 burden in the US to guide pandemic planning and decision-making in the context of high uncertainty. This effort was born out of an attempt to coordinate, synthesize and effectively use the unprecedented amount of predictive modeling that emerged throughout the COVID-19 pandemic. Here we describe the history of this massive collective research effort, the process of convening and maintaining an open modeling hub active over multiple years, and attempt to provide a blueprint for future efforts. We detail the process of generating 17 rounds of scenarios and projections at different stages of the COVID-19 pandemic, and disseminating results to the public health community and lay public. We also highlight how SMH was expanded to generate influenza projections during the 2022-23 season. We identify key impacts of SMH results on public health and draw lessons to improve future collaborative modeling efforts, research on scenario projections, and the interface between models and policy.

Effectiveness of one dose of killed oral cholera vaccine in an endemic community in the Democratic Republic of the Congo: a matched case-control study.

BACKGROUND: A global shortage of cholera vaccines has increased the use of single-dose regimens, rather than the standard two-dose regimen. There is sparse evidence on single-dose protection, particularly in children. In 2020, a mass vaccination campaign was conducted in Uvira, an endemic urban setting in eastern Democratic Republic of the Congo, resulting in largely single-dose coverage. We examined the effectiveness of a single-dose of the oral cholera vaccine Euvichol-Plus in this high-burden setting. METHODS: In this matched case-control study, we recruited individuals with medically attended confirmed cholera in the two cholera treatment facilities in the city of Uvira. The control group consisted of age-matched, sex-matched, and neighbourhood-matched community individuals. We recruited across two distinct periods: Oct 14, 2021, to March 10, 2022 (12-17 months after vaccination), and Nov 21, 2022, to Oct 18, 2023 (24-36 months after vaccination). Study staff administered structured questionnaires to all participants to capture demographics, household conditions, potential confounding variables, and vaccination status. The odds of vaccination for the case and control groups were contrasted in conditional logistic regression models to estimate unadjusted and adjusted vaccine effectiveness. FINDINGS: We enrolled 658 individuals with confirmed cholera and 2274 matched individuals for the control group. 99 (15·1%) individuals in the case group were younger than 5 years at the time of vaccination. The adjusted single-dose vaccine effectiveness was 52·7% (95% CI 31·4 to 67·4) 12-17 months after vaccination and 44·7% (24·8 to 59·4) 24-36 months after vaccination. Although protection in the first 12-17 months after vaccination was similar for children aged 1-4 years and older individuals, the estimate of protection in children aged 1-4 years appeared to wane during the third year after vaccination (adjusted vaccine effectiveness 32·9%, 95% CI -30·7 to 65·5), with CIs spanning the null. INTERPRETATION: A single dose of Euvichol-Plus provided substantial protection against medically attended cholera for at least 36 months after vaccination in this cholera-endemic setting. Although the evidence provides support for similar levels of protection in young children and others in the short term, protection among children younger than 5 years might wane significantly during the third year after vaccination. FUNDING: Wellcome Trust and Gavi, the Vaccine Alliance.

Transportability Without Positivity: A Synthesis of Statistical and Simulation Modeling.

Studies designed to estimate the effect of an action in a randomized or observational setting often do not represent a random sample of the desired target population. Instead, estimates from that study can be transported to the target population. However, transportability methods generally rely on a positivity assumption, such that all relevant covariate patterns in the target population are also observed in the study sample. Strict eligibility criteria, particularly in the context of randomized trials, may lead to violations of this assumption. Two common approaches to address positivity violations are restricting the target population and restricting the relevant covariate set. As neither of these restrictions is ideal, we instead propose a synthesis of statistical and simulation models to address positivity violations. We propose corresponding g-computation and inverse probability weighting estimators. The restriction and synthesis approaches to addressing positivity violations are contrasted with a simulation experiment and an illustrative example in the context of sexually transmitted infection testing uptake. In both cases, the proposed synthesis approach accurately addressed the original research question when paired with a thoughtfully selected simulation model. Neither of the restriction approaches was able to accurately address the motivating question. As public health decisions must often be made with imperfect target population information, model synthesis is a viable approach given a combination of empirical data and external information based on the best available knowledge.

Inferring the proportion of undetected cholera infections from serological and clinical surveillance in an immunologically naive population.

Most infections with pandemic Vibrio cholerae are thought to result in subclinical disease and are not captured by surveillance. Previous estimates of the ratio of infections to clinical cases have varied widely (2 to 100). Understanding cholera epidemiology and immunity relies on the ability to translate between numbers of clinical cases and the underlying number of infections in the population. We estimated the infection incidence during the first months of an outbreak in a cholera-naive population using a Bayesian vibriocidal antibody titer decay model combining measurements from a representative serosurvey and clinical surveillance data. 3,880 suspected cases were reported in Grande Saline, Haiti, between 20 October 2010 and 6 April 2011 (clinical attack rate 18.4%). We found that more than 52.6% (95% Credible Interval (CrI) 49.4-55.7) of the population ≥2 years showed serologic evidence of infection, with a lower infection rate among children aged 2-4 years (35.5%; 95%CrI 24.2-51.6) compared with people ≥5 years (53.1%; 95%CrI 49.4-56.4). This estimated infection rate, nearly three times the clinical attack rate, with underdetection mainly seen in those ≥5 years, has likely impacted subsequent outbreak dynamics. Our findings show how seroincidence estimates improve understanding of links between cholera burden, transmission dynamics and immunity.

Scenario Design for Infectious Disease Projections: Integrating Concepts from Decision Analysis and Experimental Design.

Across many fields, scenario modeling has become an important tool for exploring long-term projections and how they might depend on potential interventions and critical uncertainties, with relevance to both decision makers and scientists. In the past decade, and especially during the COVID-19 pandemic, the field of epidemiology has seen substantial growth in the use of scenario projections. Multiple scenarios are often projected at the same time, allowing important comparisons that can guide the choice of intervention, the prioritization of research topics, or public communication. The design of the scenarios is central to their ability to inform important questions. In this paper, we draw on the fields of decision analysis and statistical design of experiments to propose a framework for scenario design in epidemiology, with relevance also to other fields. We identify six different fundamental purposes for scenario designs (decision making, sensitivity analysis, value of information, situational awareness, horizon scanning, and forecasting) and discuss how those purposes guide the structure of scenarios. We discuss other aspects of the content and process of scenario design, broadly for all settings and specifically for multi-model ensemble projections. As an illustrative case study, we examine the first 17 rounds of scenarios from the U.S. COVID-19 Scenario Modeling Hub, then reflect on future advancements that could improve the design of scenarios in epidemiological settings.

Estimating the proportion of clinically suspected cholera cases that are true Vibrio cholerae infections: A systematic review and meta-analysis.

BACKGROUND: Cholera surveillance relies on clinical diagnosis of acute watery diarrhea. Suspected cholera case definitions have high sensitivity but low specificity, challenging our ability to characterize cholera burden and epidemiology. Our objective was to estimate the proportion of clinically suspected cholera that are true Vibrio cholerae infections and identify factors that explain variation in positivity. METHODS AND FINDINGS: We conducted a systematic review of studies that tested ≥10 suspected cholera cases for V. cholerae O1/O139 using culture, PCR, and/or a rapid diagnostic test. We searched PubMed, Embase, Scopus, and Google Scholar for studies that sampled at least one suspected case between January 1, 2000 and April 19, 2023, to reflect contemporary patterns in V. cholerae positivity. We estimated diagnostic test sensitivity and specificity using a latent class meta-analysis. We estimated V. cholerae positivity using a random-effects meta-analysis, adjusting for test performance. We included 119 studies from 30 countries. V. cholerae positivity was lower in studies with representative sampling and in studies that set minimum ages in suspected case definitions. After adjusting for test performance, on average, 52% (95% credible interval (CrI): 24%, 80%) of suspected cases represented true V. cholerae infections. After adjusting for test performance and study methodology, the odds of a suspected case having a true infection were 5.71 (odds ratio 95% CrI: 1.53, 15.43) times higher when surveillance was initiated in response to an outbreak than in non-outbreak settings. Variation across studies was high, and a limitation of our approach was that we were unable to explain all the heterogeneity with study-level attributes, including diagnostic test used, setting, and case definitions. CONCLUSIONS: In this study, we found that burden estimates based on suspected cases alone may overestimate the incidence of medically attended cholera by 2-fold. However, accounting for cases missed by traditional clinical surveillance is key to unbiased cholera burden estimates. Given the substantial variability in positivity between settings, extrapolations from suspected to confirmed cases, which is necessary to estimate cholera incidence rates without exhaustive testing, should be based on local data.

Consistency between Household and County Measures of Onsite Schooling during the COVID-19 Pandemic.

The academic, socioemotional, and health impacts of school policies throughout the COVID-19 pandemic have been a source of many questions that require accurate information about the extent of onsite schooling occurring. This article investigates school operational status datasets during the pandemic, comparing (1) self-report data collected nationally on the household level through a Facebook-based survey, (2) county-level school policy data, and (3) a school-level closure status dataset based on phone GPS tracking. The percentage of any onsite instruction within states and counties are compared across datasets from December 2020 to May 2021. Sources were relatively consistent at the state level and for large counties, but key differences were revealed between units of measurement, showing differences between policy and household decisions surrounding children's schooling experiences. The consistency levels across sources support the usage of each of the school policy sources to answer questions about the educational experiences, factors, and impacts related to K-12 education across the nation during the pandemic, but it remains vital to think critically as to which unit of measurement is most relevant to targeted research questions.

Implications of rapid population growth on survey design and HIV estimates in the Rakai Community Cohort Study (RCCS), Uganda.

OBJECTIVE: Since rapid population growth challenges longitudinal population-based HIV cohorts in Africa to maintain coverage of their target populations, this study evaluated whether the exclusion of some residents due to growing population size biases key HIV metrics like prevalence and population-level viremia. DESIGN, SETTING AND PARTICIPANTS: Data were obtained from the Rakai Community Cohort Study (RCCS) in south central Uganda, an open population-based cohort which began excluding some residents of newly constructed household structures within its surveillance boundaries in 2008. The study includes adults aged 15-49 years who were censused from 2019 to 2020. MEASURES: We fit ensemble machine learning models to RCCS census and survey data to predict HIV seroprevalence and viremia (prevalence of those with viral load >1000 copies/mL) in the excluded population and evaluated whether their inclusion would change overall estimates. RESULTS: Of the 24 729 census-eligible residents, 2920 (12%) residents were excluded from the RCCS because they were living in new households. The predicted seroprevalence for these excluded residents was 10.8% (95% CI: 9.6% to 11.8%)-somewhat lower than 11.7% (95% CI: 11.2% to 12.3%) in the observed sample. Predicted seroprevalence for younger excluded residents aged 15-24 years was 4.9% (95% CI: 3.6% to 6.1%)-significantly higher than that in the observed sample for the same age group (2.6% (95% CI: 2.2% to 3.1%)), while predicted seroprevalence for older excluded residents aged 25-49 years was 15.0% (95% CI: 13.3% to 16.4%)-significantly lower than their counterparts in the observed sample (17.2% (95% CI: 16.4% to 18.1%)). Over all ages, the predicted prevalence of viremia in excluded residents (3.7% (95% CI: 3.0% to 4.5%)) was significantly higher than that in the observed sample (1.7% (95% CI: 1.5% to 1.9%)), resulting in a higher overall population-level viremia estimate of 2.1% (95% CI: 1.8% to 2.4%). CONCLUSIONS: Exclusion of residents in new households may modestly bias HIV viremia estimates and some age-specific seroprevalence estimates in the RCCS. Overall, HIV seroprevalence estimates were not significantly affected.

Water, Sanitation, and Cholera in Sub-Saharan Africa.

Improvements in water and sanitation should reduce cholera risk though the associations between cholera and specific water and sanitation access measures remain unclear. We estimated the association between eight water and sanitation measures and annual cholera incidence access across sub-Saharan Africa (2010-2016) for data aggregated at the country and district levels. We fit random forest regression and classification models to understand how well these measures combined might be able to predict cholera incidence rates and identify high cholera incidence areas. Across spatial scales, piped or "other improved" water access was inversely associated with cholera incidence. Access to piped water, septic or sewer sanitation, and septic, sewer, or "other improved" sanitation were associated with decreased district-level cholera incidence. The classification model had moderate performance in identifying high cholera incidence areas (cross-validated-AUC 0.81, 95% CI 0.78-0.83) with high negative predictive values (93-100%) indicating the utility of water and sanitation measures for screening out areas that are unlikely to be at high cholera risk. While comprehensive cholera risk assessments must incorporate other data sources (e.g., historical incidence), our results suggest that water and sanitation measures could alone be useful in narrowing the geographic focus for detailed risk assessments.

Accounting for assay performance when estimating the temporal dynamics in SARS-CoV-2 seroprevalence in the U.S.

Reconstructing the incidence of SARS-CoV-2 infection is central to understanding the state of the pandemic. Seroprevalence studies are often used to assess cumulative infections as they can identify asymptomatic infection. Since July 2020, commercial laboratories have conducted nationwide serosurveys for the U.S. CDC. They employed three assays, with different sensitivities and specificities, potentially introducing biases in seroprevalence estimates. Using models, we show that accounting for assays explains some of the observed state-to-state variation in seroprevalence, and when integrating case and death surveillance data, we show that when using the Abbott assay, estimates of proportions infected can differ substantially from seroprevalence estimates. We also found that states with higher proportions infected (before or after vaccination) had lower vaccination coverages, a pattern corroborated using a separate dataset. Finally, to understand vaccination rates relative to the increase in cases, we estimated the proportions of the population that received a vaccine prior to infection.

Sample size calculations for pathogen variant surveillance in the presence of biological and systematic biases.

Tracking the emergence and spread of pathogen variants is an important component of monitoring infectious disease outbreaks. To that end, accurately estimating the number and prevalence of pathogen variants in a population requires carefully designed surveillance programs. However, current approaches to calculating the number of pathogen samples needed for effective surveillance often do not account for the various processes that can bias which infections are detected and which samples are ultimately characterized as a specific variant. In this article, we introduce a framework that accounts for the logistical and epidemiological processes that may bias variant characterization, and we demonstrate how to use this framework (implemented in a publicly available tool) to calculate the number of sequences needed for surveillance. Our framework is designed to be easy to use while also flexible enough to be adapted to various pathogens and surveillance scenarios.

Context-dependent representation of within- and between-model uncertainty: aggregating probabilistic predictions in infectious disease epidemiology.

Probabilistic predictions support public health planning and decision making, especially in infectious disease emergencies. Aggregating outputs from multiple models yields more robust predictions of outcomes and associated uncertainty. While the selection of an aggregation method can be guided by retrospective performance evaluations, this is not always possible. For example, if predictions are conditional on assumptions about how the future will unfold (e.g. possible interventions), these assumptions may never materialize, precluding any direct comparison between predictions and observations. Here, we summarize literature on aggregating probabilistic predictions, illustrate various methods for infectious disease predictions via simulation, and present a strategy for choosing an aggregation method when empirical validation cannot be used. We focus on the linear opinion pool (LOP) and Vincent average, common methods that make different assumptions about between-prediction uncertainty. We contend that assumptions of the aggregation method should align with a hypothesis about how uncertainty is expressed within and between predictions from different sources. The LOP assumes that between-prediction uncertainty is meaningful and should be retained, while the Vincent average assumes that between-prediction uncertainty is akin to sampling error and should not be preserved. We provide an R package for implementation. Given the rising importance of multi-model infectious disease hubs, our work provides useful guidance on aggregation and a deeper understanding of the benefits and risks of different approaches.