An open challenge to advance probabilistic forecasting for dengue epidemics.

A wide range of research has promised new tools for forecasting infectious disease dynamics, but little of that research is currently being applied in practice, because tools do not address key public health needs, do not produce probabilistic forecasts, have not been evaluated on external data, or do not provide sufficient forecast skill to be useful. We developed an open collaborative forecasting challenge to assess probabilistic forecasts for seasonal epidemics of dengue, a major global public health problem. Sixteen teams used a variety of methods and data to generate forecasts for 3 epidemiological targets (peak incidence, the week of the peak, and total incidence) over 8 dengue seasons in Iquitos, Peru and San Juan, Puerto Rico. Forecast skill was highly variable across teams and targets. While numerous forecasts showed high skill for midseason situational awareness, early season skill was low, and skill was generally lowest for high incidence seasons, those for which forecasts would be most valuable. A comparison of modeling approaches revealed that average forecast skill was lower for models including biologically meaningful data and mechanisms and that both multimodel and multiteam ensemble forecasts consistently outperformed individual model forecasts. Leveraging these insights, data, and the forecasting framework will be critical to improve forecast skill and the application of forecasts in real time for epidemic preparedness and response. Moreover, key components of this project-integration with public health needs, a common forecasting framework, shared and standardized data, and open participation-can help advance infectious disease forecasting beyond dengue.

Clinical Cholera Surveillance Sensitivity in Bangladesh and Implications for Large-Scale Disease Control.

BACKGROUND: A surveillance system that is sensitive to detecting high burden areas is critical for achieving widespread disease control. In 2014, Bangladesh established a nationwide, facility-based cholera surveillance system for Vibrio cholerae infection. We sought to measure the sensitivity of this surveillance system to detect cases to assess whether cholera elimination targets outlined by the Bangladesh national control plan can be adequately measured. METHODS: We overlaid maps of nationally representative annual V cholerae seroincidence onto maps of the catchment areas of facilities where confirmatory laboratory testing for cholera was conducted, and we identified its spatial complement as surveillance greyspots, areas where cases likely occur but go undetected. We assessed surveillance system sensitivity and changes to sensitivity given alternate surveillance site selection strategies. RESULTS: We estimated that 69% of Bangladeshis (111.7 million individuals) live in surveillance greyspots and that 23% (25.5 million) of these individuals live in areas with the highest V cholerae infection rates. CONCLUSIONS: The cholera surveillance system in Bangladesh has the ability to monitor progress towards cholera elimination goals among 31% of the country's population, which may be insufficient for accurately measuring progress. Increasing surveillance coverage, particularly in the highest risk areas, should be considered.

Seroprevalence of anti-SARS-CoV-2 IgG antibodies in Geneva, Switzerland (SEROCoV-POP): a population-based study.

BACKGROUND: Assessing the burden of COVID-19 on the basis of medically attended case numbers is suboptimal given its reliance on testing strategy, changing case definitions, and disease presentation. Population-based serosurveys measuring anti-severe acute respiratory syndrome coronavirus 2 (anti-SARS-CoV-2) antibodies provide one method for estimating infection rates and monitoring the progression of the epidemic. Here, we estimate weekly seroprevalence of anti-SARS-CoV-2 antibodies in the population of Geneva, Switzerland, during the epidemic. METHODS: The SEROCoV-POP study is a population-based study of former participants of the Bus Santé study and their household members. We planned a series of 12 consecutive weekly serosurveys among randomly selected participants from a previous population-representative survey, and their household members aged 5 years and older. We tested each participant for anti-SARS-CoV-2-IgG antibodies using a commercially available ELISA. We estimated seroprevalence using a Bayesian logistic regression model taking into account test performance and adjusting for the age and sex of Geneva's population. Here we present results from the first 5 weeks of the study. FINDINGS: Between April 6 and May 9, 2020, we enrolled 2766 participants from 1339 households, with a demographic distribution similar to that of the canton of Geneva. In the first week, we estimated a seroprevalence of 4·8% (95% CI 2·4-8·0, n=341). The estimate increased to 8·5% (5·9-11·4, n=469) in the second week, to 10·9% (7·9-14·4, n=577) in the third week, 6·6% (4·3-9·4, n=604) in the fourth week, and 10·8% (8·2-13·9, n=775) in the fifth week. Individuals aged 5-9 years (relative risk [RR] 0·32 [95% CI 0·11-0·63]) and those older than 65 years (RR 0·50 [0·28-0·78]) had a significantly lower risk of being seropositive than those aged 20-49 years. After accounting for the time to seroconversion, we estimated that for every reported confirmed case, there were 11·6 infections in the community. INTERPRETATION: These results suggest that most of the population of Geneva remained uninfected during this wave of the pandemic, despite the high prevalence of COVID-19 in the region (5000 reported clinical cases over <2·5 months in the population of half a million people). Assuming that the presence of IgG antibodies is associated with immunity, these results highlight that the epidemic is far from coming to an end by means of fewer susceptible people in the population. Further, a significantly lower seroprevalence was observed for children aged 5-9 years and adults older than 65 years, compared with those aged 10-64 years. These results will inform countries considering the easing of restrictions aimed at curbing transmission. FUNDING: Swiss Federal Office of Public Health, Swiss School of Public Health (Corona Immunitas research program), Fondation de Bienfaisance du Groupe Pictet, Fondation Ancrage, Fondation Privée des Hôpitaux Universitaires de Genève, and Center for Emerging Viral Diseases.

Modeling of Future COVID-19 Cases, Hospitalizations, and Deaths, by Vaccination Rates and Nonpharmaceutical Intervention Scenarios - United States, April-September 2021.

After a period of rapidly declining U.S. COVID-19 incidence during January-March 2021, increases occurred in several jurisdictions (1,2) despite the rapid rollout of a large-scale vaccination program. This increase coincided with the spread of more transmissible variants of SARS-CoV-2, the virus that causes COVID-19, including B.1.1.7 (1,3) and relaxation of COVID-19 prevention strategies such as those for businesses, large-scale gatherings, and educational activities. To provide long-term projections of potential trends in COVID-19 cases, hospitalizations, and deaths, COVID-19 Scenario Modeling Hub teams used a multiple-model approach comprising six models to assess the potential course of COVID-19 in the United States across four scenarios with different vaccination coverage rates and effectiveness estimates and strength and implementation of nonpharmaceutical interventions (NPIs) (public health policies, such as physical distancing and masking) over a 6-month period (April-September 2021) using data available through March 27, 2021 (4). Among the four scenarios, an accelerated decline in NPI adherence (which encapsulates NPI mandates and population behavior) was shown to undermine vaccination-related gains over the subsequent 2-3 months and, in combination with increased transmissibility of new variants, could lead to surges in cases, hospitalizations, and deaths. A sharp decline in cases was projected by July 2021, with a faster decline in the high-vaccination scenarios. High vaccination rates and compliance with public health prevention measures are essential to control the COVID-19 pandemic and to prevent surges in hospitalizations and deaths in the coming months.

Variation in False-Negative Rate of Reverse Transcriptase Polymerase Chain Reaction-Based SARS-CoV-2 Tests by Time Since Exposure.

BACKGROUND: Tests for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) based on reverse transcriptase polymerase chain reaction (RT-PCR) are being used to rule out infection among high-risk persons, such as exposed inpatients and health care workers. It is critical to understand how the predictive value of the test varies with time from exposure and symptom onset to avoid being falsely reassured by negative test results. OBJECTIVE: To estimate the false-negative rate by day since infection. DESIGN: Literature review and pooled analysis. SETTING: 7 previously published studies providing data on RT-PCR performance by time since symptom onset or SARS-CoV-2 exposure using samples from the upper respiratory tract (n = 1330). PATIENTS: A mix of inpatients and outpatients with SARS-CoV-2 infection. MEASUREMENTS: A Bayesian hierarchical model was fitted to estimate the false-negative rate by day since exposure and symptom onset. RESULTS: Over the 4 days of infection before the typical time of symptom onset (day 5), the probability of a false-negative result in an infected person decreases from 100% (95% CI, 100% to 100%) on day 1 to 67% (CI, 27% to 94%) on day 4. On the day of symptom onset, the median false-negative rate was 38% (CI, 18% to 65%). This decreased to 20% (CI, 12% to 30%) on day 8 (3 days after symptom onset) then began to increase again, from 21% (CI, 13% to 31%) on day 9 to 66% (CI, 54% to 77%) on day 21. LIMITATION: Imprecise estimates due to heterogeneity in the design of studies on which results were based. CONCLUSION: Care must be taken in interpreting RT-PCR tests for SARS-CoV-2 infection-particularly early in the course of infection-when using these results as a basis for removing precautions intended to prevent onward transmission. If clinical suspicion is high, infection should not be ruled out on the basis of RT-PCR alone, and the clinical and epidemiologic situation should be carefully considered. PRIMARY FUNDING SOURCE: National Institute of Allergy and Infectious Diseases, Johns Hopkins Health System, and U.S. Centers for Disease Control and Prevention.

The Incubation Period of Coronavirus Disease 2019 (COVID-19) From Publicly Reported Confirmed Cases: Estimation and Application.

BACKGROUND: A novel human coronavirus, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), was identified in China in December 2019. There is limited support for many of its key epidemiologic features, including the incubation period for clinical disease (coronavirus disease 2019 [COVID-19]), which has important implications for surveillance and control activities. OBJECTIVE: To estimate the length of the incubation period of COVID-19 and describe its public health implications. DESIGN: Pooled analysis of confirmed COVID-19 cases reported between 4 January 2020 and 24 February 2020. SETTING: News reports and press releases from 50 provinces, regions, and countries outside Wuhan, Hubei province, China. PARTICIPANTS: Persons with confirmed SARS-CoV-2 infection outside Hubei province, China. MEASUREMENTS: Patient demographic characteristics and dates and times of possible exposure, symptom onset, fever onset, and hospitalization. RESULTS: There were 181 confirmed cases with identifiable exposure and symptom onset windows to estimate the incubation period of COVID-19. The median incubation period was estimated to be 5.1 days (95% CI, 4.5 to 5.8 days), and 97.5% of those who develop symptoms will do so within 11.5 days (CI, 8.2 to 15.6 days) of infection. These estimates imply that, under conservative assumptions, 101 out of every 10 000 cases (99th percentile, 482) will develop symptoms after 14 days of active monitoring or quarantine. LIMITATION: Publicly reported cases may overrepresent severe cases, the incubation period for which may differ from that of mild cases. CONCLUSION: This work provides additional evidence for a median incubation period for COVID-19 of approximately 5 days, similar to SARS. Our results support current proposals for the length of quarantine or active monitoring of persons potentially exposed to SARS-CoV-2, although longer monitoring periods might be justified in extreme cases. PRIMARY FUNDING SOURCE: U.S. Centers for Disease Control and Prevention, National Institute of Allergy and Infectious Diseases, National Institute of General Medical Sciences, and Alexander von Humboldt Foundation.

Prospective forecasts of annual dengue hemorrhagic fever incidence in Thailand, 2010-2014.

Dengue hemorrhagic fever (DHF), a severe manifestation of dengue viral infection that can cause severe bleeding, organ impairment, and even death, affects between 15,000 and 105,000 people each year in Thailand. While all Thai provinces experience at least one DHF case most years, the distribution of cases shifts regionally from year to year. Accurately forecasting where DHF outbreaks occur before the dengue season could help public health officials prioritize public health activities. We develop statistical models that use biologically plausible covariates, observed by April each year, to forecast the cumulative DHF incidence for the remainder of the year. We perform cross-validation during the training phase (2000-2009) to select the covariates for these models. A parsimonious model based on preseason incidence outperforms the 10-y median for 65% of province-level annual forecasts, reduces the mean absolute error by 19%, and successfully forecasts outbreaks (area under the receiver operating characteristic curve = 0.84) over the testing period (2010-2014). We find that functions of past incidence contribute most strongly to model performance, whereas the importance of environmental covariates varies regionally. This work illustrates that accurate forecasts of dengue risk are possible in a policy-relevant timeframe.

The potential impact of COVID-19 in refugee camps in Bangladesh and beyond: A modeling study.

BACKGROUND: COVID-19 could have even more dire consequences in refugees camps than in general populations. Bangladesh has confirmed COVID-19 cases and hosts almost 1 million Rohingya refugees from Myanmar, with 600,000 concentrated in the Kutupalong-Balukhali Expansion Site (mean age, 21 years; standard deviation [SD], 18 years; 52% female). Projections of the potential COVID-19 burden, epidemic speed, and healthcare needs in such settings are critical for preparedness planning. METHODS AND FINDINGS: To explore the potential impact of the introduction of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in the Kutupalong-Balukhali Expansion Site, we used a stochastic Susceptible Exposed Infectious Recovered (SEIR) transmission model with parameters derived from emerging literature and age as the primary determinant of infection severity. We considered three scenarios with different assumptions about the transmission potential of SARS-CoV-2. From the simulated infections, we estimated hospitalizations, deaths, and healthcare needs expected, age-adjusted for the Kutupalong-Balukhali Expansion Site age distribution. Our findings suggest that a large-scale outbreak is likely after a single introduction of the virus into the camp, with 61%-92% of simulations leading to at least 1,000 people infected across scenarios. On average, in the first 30 days of the outbreak, we expect 18 (95% prediction interval [PI], 2-65), 54 (95% PI, 3-223), and 370 (95% PI, 4-1,850) people infected in the low, moderate, and high transmission scenarios, respectively. These reach 421,500 (95% PI, 376,300-463,500), 546,800 (95% PI, 499,300-567,000), and 589,800 (95% PI, 578,800-595,600) people infected in 12 months, respectively. Hospitalization needs exceeded the existing hospitalization capacity of 340 beds after 55-136 days, between the low and high transmission scenarios. We estimate 2,040 (95% PI, 1,660-2,500), 2,650 (95% PI, 2,030-3,380), and 2,880 (95% PI, 2,090-3,830) deaths in the low, moderate, and high transmission scenarios, respectively. Due to limited data at the time of analyses, we assumed that age was the primary determinant of infection severity and hospitalization. We expect that comorbidities, limited hospitalization, and intensive care capacity may increase this risk; thus, we may be underestimating the potential burden. CONCLUSIONS: Our findings suggest that a COVID-19 epidemic in a refugee settlement may have profound consequences, requiring large increases in healthcare capacity and infrastructure that may exceed what is currently feasible in these settings. Detailed and realistic planning for the worst case in Kutupalong-Balukhali and all refugee camps worldwide must begin now. Plans should consider novel and radical strategies to reduce infectious contacts and fill health worker gaps while recognizing that refugees may not have access to national health systems.

Evaluating the ALERT algorithm for local outbreak onset detection in seasonal infectious disease surveillance data.

Estimation of epidemic onset timing is an important component of controlling the spread of seasonal infectious diseases within community healthcare sites. The Above Local Elevated Respiratory Illness Threshold (ALERT) algorithm uses a threshold-based approach to suggest incidence levels that historically have indicated the transition from endemic to epidemic activity. In this paper, we present the first detailed overview of the computational approach underlying the algorithm. In the motivating example section, we evaluate the performance of ALERT in determining the onset of increased respiratory virus incidence using laboratory testing data from the Children's Hospital of Colorado. At a threshold of 10 cases per week, ALERT-selected intervention periods performed better than the observed hospital site periods (2004/2005-2012/2013) and a CUSUM method. Additional simulation studies show how data properties may effect ALERT performance on novel data. We found that the conditions under which ALERT showed ideal performance generally included high seasonality and low off-season incidence.

Infectious disease prediction with kernel conditional density estimation.

Creating statistical models that generate accurate predictions of infectious disease incidence is a challenging problem whose solution could benefit public health decision makers. We develop a new approach to this problem using kernel conditional density estimation (KCDE) and copulas. We obtain predictive distributions for incidence in individual weeks using KCDE and tie those distributions together into joint distributions using copulas. This strategy enables us to create predictions for the timing of and incidence in the peak week of the season. Our implementation of KCDE incorporates 2 novel kernel components: a periodic component that captures seasonality in disease incidence and a component that allows for a full parameterization of the bandwidth matrix with discrete variables. We demonstrate via simulation that a fully parameterized bandwidth matrix can be beneficial for estimating conditional densities. We apply the method to predicting dengue fever and influenza and compare to a seasonal autoregressive integrated moving average model and HHH4, a previously published extension to the generalized linear model framework developed for infectious disease incidence. The KCDE outperforms the baseline methods for predictions of dengue incidence in individual weeks. The KCDE also offers more consistent performance than the baseline models for predictions of incidence in the peak week and is comparable to the baseline models on the other prediction targets. Using the periodic kernel function led to better predictions of incidence. Our approach and extensions of it could yield improved predictions for public health decision makers, particularly in diseases with heterogeneous seasonal dynamics such as dengue fever.

Triggering interventions for influenza: the ALERT algorithm.

BACKGROUND: Early, accurate predictions of the onset of influenza season enable targeted implementation of control efforts. Our objective was to develop a tool to assist public health practitioners, researchers, and clinicians in defining the community-level onset of seasonal influenza epidemics. METHODS: Using recent surveillance data on virologically confirmed infections of influenza, we developed the Above Local Elevated Respiratory Illness Threshold (ALERT) algorithm, a method to identify the period of highest seasonal influenza activity. We used data from 2 large hospitals that serve Baltimore, Maryland and Denver, Colorado, and the surrounding geographic areas. The data used by ALERT are routinely collected surveillance data: weekly case counts of laboratory-confirmed influenza A virus. The main outcome is the percentage of prospective seasonal influenza cases identified by the ALERT algorithm. RESULTS: When ALERT thresholds designed to capture 90% of all cases were applied prospectively to the 2011-2012 and 2012-2013 influenza seasons in both hospitals, 71%-91% of all reported cases fell within the ALERT period. CONCLUSIONS: The ALERT algorithm provides a simple, robust, and accurate metric for determining the onset of elevated influenza activity at the community level. This new algorithm provides valuable information that can impact infection prevention recommendations, public health practice, and healthcare delivery.

Correction: Challenges in Real-Time Prediction of Infectious Disease: A Case Study of Dengue in Thailand.

[This corrects the article DOI: 10.1371/journal.pntd.0004761.].

A scenario modeling pipeline for COVID-19 emergency planning.

Coronavirus disease 2019 (COVID-19) has caused strain on health systems worldwide due to its high mortality rate and the large portion of cases requiring critical care and mechanical ventilation. During these uncertain times, public health decision makers, from city health departments to federal agencies, sought the use of epidemiological models for decision support in allocating resources, developing non-pharmaceutical interventions, and characterizing the dynamics of COVID-19 in their jurisdictions. In response, we developed a flexible scenario modeling pipeline that could quickly tailor models for decision makers seeking to compare projections of epidemic trajectories and healthcare impacts from multiple intervention scenarios in different locations. Here, we present the components and configurable features of the COVID Scenario Pipeline, with a vignette detailing its current use. We also present model limitations and active areas of development to meet ever-changing decision maker needs.

Insights into household transmission of SARS-CoV-2 from a population-based serological survey.

Understanding the risk of infection from household- and community-exposures and the transmissibility of asymptomatic infections is critical to SARS-CoV-2 control. Limited previous evidence is based primarily on virologic testing, which disproportionately misses mild and asymptomatic infections. Serologic measures are more likely to capture all previously infected individuals. We apply household transmission models to data from a cross-sectional, household-based population serosurvey of 4,534 people ≥5 years from 2,267 households enrolled April-June 2020 in Geneva, Switzerland. We found that the risk of infection from exposure to a single infected household member aged ≥5 years (17.3%,13.7-21.7) was more than three-times that of extra-household exposures over the first pandemic wave (5.1%,4.5-5.8). Young children had a lower risk of infection from household members. Working-age adults had the highest extra-household infection risk. Seropositive asymptomatic household members had 69.4% lower odds (95%CrI,31.8-88.8%) of infecting another household member compared to those reporting symptoms, accounting for 14.5% (95%CrI, 7.2-22.7%) of all household infections.

Vibrio cholerae O1 transmission in Bangladesh: insights from a nationally representative serosurvey.

BACKGROUND: Pandemic Vibrio cholerae from cholera-endemic countries around the Bay of Bengal regularly seed epidemics globally. Without reducing cholera in these countries, including Bangladesh, global cholera control might never be achieved. Little is known about the geographical distribution and magnitude of V cholerae O1 transmission nationally. We aimed to describe infection risk across Bangladesh, making use of advances in cholera seroepidemiology, therefore overcoming many of the limitations of current clinic-based surveillance. METHODS: We tested serum samples from a nationally representative serosurvey in Bangladesh with eight V cholerae-specific assays. Using these data with a machine-learning model previously validated within a cohort of confirmed cholera cases and their household contacts, we estimated the proportion of the population with evidence of infection by V cholerae O1 in the previous year (annual seroincidence) and used Bayesian geostatistical models to create high-resolution national maps of infection risk. FINDINGS: Between Oct 16, 2015, and Jan 24, 2016, we obtained and tested serum samples from 2930 participants (707 households) in 70 communities across Bangladesh. We estimated national annual seroincidence of V cholerae O1 infection of 17·3% (95% CI 10·5-24·1). Our high-resolution maps showed large heterogeneity of infection risk, with community-level annual infection risk within the sampled population ranging from 4·3% to 62·9%. Across Bangladesh, we estimated that 28·1 (95% CI 17·1-39·2) million infections occurred in the year before the survey. Despite having an annual seroincidence of V cholerae O1 infection lower than much of Bangladesh, Dhaka (the capital of Bangladesh and largest city in the country) had 2·0 (95% CI 0·6-3·9) million infections during the same year, primarily because of its large population. INTERPRETATION: Serosurveillance provides an avenue for identifying areas with high V cholerae O1 transmission and investigating key risk factors for infection across geographical scales. Serosurveillance could serve as an important method for countries to plan and monitor progress towards 2030 cholera elimination goals. FUNDING: The Bill & Melinda Gates Foundation, National Institutes of Health, and US Centers for Disease Control and Prevention.

Dynamics and significance of the antibody response to SARS-CoV-2 infection.

BACKGROUND: Characterizing the humoral immune response to SARS-CoV-2 and developing accurate serologic assays are needed for diagnostic purposes and estimating population-level seroprevalence. METHODS: We measured the kinetics of early antibody responses to the receptor-binding domain (RBD) of the spike (S) protein of SARS-CoV-2 in a cohort of 259 symptomatic North American patients infected with SARS-CoV-2 (up to 75 days after symptom onset) compared to antibody levels in 1548 individuals whose blood samples were obtained prior to the pandemic. RESULTS: Between 14-28 days from onset of symptoms, IgG, IgA, or IgM antibody responses to RBD were all accurate in identifying recently infected individuals, with 100% specificity and a sensitivity of 97%, 91%, and 81% respectively. Although the estimated median time to becoming seropositive was similar across isotypes, IgA and IgM antibodies against RBD were short-lived with most individuals estimated to become seronegative again by 51 and 47 days after symptom onset, respectively. IgG antibodies against RBD lasted longer and persisted through 75 days post-symptoms. IgG antibodies to SARS-CoV-2 RBD were highly correlated with neutralizing antibodies targeting the S protein. No cross-reactivity of the SARS-CoV-2 RBD-targeted antibodies was observed with several known circulating coronaviruses, HKU1, OC 229 E, OC43, and NL63. CONCLUSIONS: Among symptomatic SARS-CoV-2 cases, RBD-targeted antibodies can be indicative of previous and recent infection. IgG antibodies are correlated with neutralizing antibodies and are possibly a correlate of protective immunity.

Persistence and decay of human antibody responses to the receptor binding domain of SARS-CoV-2 spike protein in COVID-19 patients.

We measured plasma and/or serum antibody responses to the receptor-binding domain (RBD) of the spike (S) protein of SARS-CoV-2 in 343 North American patients infected with SARS-CoV-2 (of which 93% required hospitalization) up to 122 days after symptom onset and compared them to responses in 1548 individuals whose blood samples were obtained prior to the pandemic. After setting seropositivity thresholds for perfect specificity (100%), we estimated sensitivities of 95% for IgG, 90% for IgA, and 81% for IgM for detecting infected individuals between 15 and 28 days after symptom onset. While the median time to seroconversion was nearly 12 days across all three isotypes tested, IgA and IgM antibodies against RBD were short-lived with median times to seroreversion of 71 and 49 days after symptom onset. In contrast, anti-RBD IgG responses decayed slowly through 90 days with only 3 seropositive individuals seroreverting within this time period. IgG antibodies to SARS-CoV-2 RBD were strongly correlated with anti-S neutralizing antibody titers, which demonstrated little to no decrease over 75 days since symptom onset. We observed no cross-reactivity of the SARS-CoV-2 RBD-targeted antibodies with other widely circulating coronaviruses (HKU1, 229 E, OC43, NL63). These data suggest that RBD-targeted antibodies are excellent markers of previous and recent infection, that differential isotype measurements can help distinguish between recent and older infections, and that IgG responses persist over the first few months after infection and are highly correlated with neutralizing antibodies.

Case study in evaluating time series prediction models using the relative mean absolute error.

Statistical prediction models inform decision-making processes in many real-world settings. Prior to using predictions in practice, one must rigorously test and validate candidate models to ensure that the proposed predictions have sufficient accuracy to be used in practice. In this paper, we present a framework for evaluating time series predictions that emphasizes computational simplicity and an intuitive interpretation using the relative mean absolute error metric. For a single time series, this metric enables comparisons of candidate model predictions against naïve reference models, a method that can provide useful and standardized performance benchmarks. Additionally, in applications with multiple time series, this framework facilitates comparisons of one or more models' predictive performance across different sets of data. We illustrate the use of this metric with a case study comparing predictions of dengue hemorrhagic fever incidence in two provinces of Thailand. This example demonstrates the utility and interpretability of the relative mean absolute error metric in practice, and underscores the practical advantages of using relative performance metrics when evaluating predictions.

Challenges in Real-Time Prediction of Infectious Disease: A Case Study of Dengue in Thailand.

Epidemics of communicable diseases place a huge burden on public health infrastructures across the world. Producing accurate and actionable forecasts of infectious disease incidence at short and long time scales will improve public health response to outbreaks. However, scientists and public health officials face many obstacles in trying to create such real-time forecasts of infectious disease incidence. Dengue is a mosquito-borne virus that annually infects over 400 million people worldwide. We developed a real-time forecasting model for dengue hemorrhagic fever in the 77 provinces of Thailand. We created a practical computational infrastructure that generated multi-step predictions of dengue incidence in Thai provinces every two weeks throughout 2014. These predictions show mixed performance across provinces, out-performing seasonal baseline models in over half of provinces at a 1.5 month horizon. Additionally, to assess the degree to which delays in case reporting make long-range prediction a challenging task, we compared the performance of our real-time predictions with predictions made with fully reported data. This paper provides valuable lessons for the implementation of real-time predictions in the context of public health decision making.