Scenario design for infectious disease projections: Integrating concepts from decision analysis and experimental design.

Across many fields, scenario modeling has become an important tool for exploring long-term projections and how they might depend on potential interventions and critical uncertainties, with relevance to both decision makers and scientists. In the past decade, and especially during the COVID-19 pandemic, the field of epidemiology has seen substantial growth in the use of scenario projections. Multiple scenarios are often projected at the same time, allowing important comparisons that can guide the choice of intervention, the prioritization of research topics, or public communication. The design of the scenarios is central to their ability to inform important questions. In this paper, we draw on the fields of decision analysis and statistical design of experiments to propose a framework for scenario design in epidemiology, with relevance also to other fields. We identify six different fundamental purposes for scenario designs (decision making, sensitivity analysis, situational awareness, horizon scanning, forecasting, and value of information) and discuss how those purposes guide the structure of scenarios. We discuss other aspects of the content and process of scenario design, broadly for all settings and specifically for multi-model ensemble projections. As an illustrative case study, we examine the first 17 rounds of scenarios from the U.S. COVID-19 Scenario Modeling Hub, then reflect on future advancements that could improve the design of scenarios in epidemiological settings.

Scenario Design for Infectious Disease Projections: Integrating Concepts from Decision Analysis and Experimental Design.

Across many fields, scenario modeling has become an important tool for exploring long-term projections and how they might depend on potential interventions and critical uncertainties, with relevance to both decision makers and scientists. In the past decade, and especially during the COVID-19 pandemic, the field of epidemiology has seen substantial growth in the use of scenario projections. Multiple scenarios are often projected at the same time, allowing important comparisons that can guide the choice of intervention, the prioritization of research topics, or public communication. The design of the scenarios is central to their ability to inform important questions. In this paper, we draw on the fields of decision analysis and statistical design of experiments to propose a framework for scenario design in epidemiology, with relevance also to other fields. We identify six different fundamental purposes for scenario designs (decision making, sensitivity analysis, value of information, situational awareness, horizon scanning, and forecasting) and discuss how those purposes guide the structure of scenarios. We discuss other aspects of the content and process of scenario design, broadly for all settings and specifically for multi-model ensemble projections. As an illustrative case study, we examine the first 17 rounds of scenarios from the U.S. COVID-19 Scenario Modeling Hub, then reflect on future advancements that could improve the design of scenarios in epidemiological settings.

Optimal spatial evaluation of a pro rata vaccine distribution rule for COVID-19.

The COVID-19 Vaccines Global Access (COVAX) is a World Health Organization (WHO) initiative that aims for an equitable access of COVID-19 vaccines. Despite potential heterogeneous infection levels across a country, countries receiving allotments of vaccines may follow WHO's allocation guidelines and distribute vaccines based on a jurisdictions' relative population size. Utilizing economic-epidemiological modeling, we benchmark the performance of this pro rata allocation rule by comparing it to an optimal one that minimizes the economic damages and expenditures over time, including a penalty representing the social costs of deviating from the pro rata strategy. The pro rata rule performs better when the duration of naturally- and vaccine-acquired immunity is short, when there is population mixing, when the supply of vaccine is high, and when there is minimal heterogeneity in demographics. Despite behavioral and epidemiological uncertainty diminishing the performance of the optimal allocation, it generally outperforms the pro rata vaccine distribution rule.

Synergistic interventions to control COVID-19: Mass testing and isolation mitigates reliance on distancing.

Stay-at-home orders and shutdowns of non-essential businesses are powerful, but socially costly, tools to control the pandemic spread of SARS-CoV-2. Mass testing strategies, which rely on widely administered frequent and rapid diagnostics to identify and isolate infected individuals, could be a potentially less disruptive management strategy, particularly where vaccine access is limited. In this paper, we assess the extent to which mass testing and isolation strategies can reduce reliance on socially costly non-pharmaceutical interventions, such as distancing and shutdowns. We develop a multi-compartmental model of SARS-CoV-2 transmission incorporating both preventative non-pharmaceutical interventions (NPIs) and testing and isolation to evaluate their combined effect on public health outcomes. Our model is designed to be a policy-guiding tool that captures important realities of the testing system, including constraints on test administration and non-random testing allocation. We show how strategic changes in the characteristics of the testing system, including test administration, test delays, and test sensitivity, can reduce reliance on preventative NPIs without compromising public health outcomes in the future. The lowest NPI levels are possible only when many tests are administered and test delays are short, given limited immunity in the population. Reducing reliance on NPIs is highly dependent on the ability of a testing program to identify and isolate unreported, asymptomatic infections. Changes in NPIs, including the intensity of lockdowns and stay at home orders, should be coordinated with increases in testing to ensure epidemic control; otherwise small additional lifting of these NPIs can lead to dramatic increases in infections, hospitalizations and deaths. Importantly, our results can be used to guide ramp-up of testing capacity in outbreak settings, allow for the flexible design of combined interventions based on social context, and inform future cost-benefit analyses to identify efficient pandemic management strategies.

Strategic testing approaches for targeted disease monitoring can be used to inform pandemic decision-making.

More than 1.6 million Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) tests were administered daily in the United States at the peak of the epidemic, with a significant focus on individual treatment. Here, we show that objective-driven, strategic sampling designs and analyses can maximize information gain at the population level, which is necessary to increase situational awareness and predict, prepare for, and respond to a pandemic, while also continuing to inform individual treatment. By focusing on specific objectives such as individual treatment or disease prediction and control (e.g., via the collection of population-level statistics to inform lockdown measures or vaccine rollout) and drawing from the literature on capture-recapture methods to deal with nonrandom sampling and testing errors, we illustrate how public health objectives can be achieved even with limited test availability when testing programs are designed a priori to meet those objectives.

Political economy of renewable resource federalism.

The authority to manage natural capital often follows political boundaries rather than ecological. This mismatch can lead to unsustainable outcomes, as spillovers from one management area to the next may create adverse incentives for local decision making, even within a single country. At the same time, one-size-fits-all approaches of federal (centralized) authority can fail to respond to state (decentralized) heterogeneity and can result in inefficient economic or detrimental ecological outcomes. Here we utilize a spatially explicit coupled natural-human system model of a fishery to illuminate trade-offs posed by the choice between federal vs. state control of renewable resources. We solve for the dynamics of fishing effort and fish stocks that result from different approaches to federal management that vary in terms of flexibility. Adapting numerical methods from engineering, we also solve for the open-loop Nash equilibrium characterizing state management outcomes, where each state anticipates and responds to the choices of the others. We consider traditional federalism questions (state vs. federal management) as well as more contemporary questions about the economic and ecological impacts of shifting regulatory authority from one level to another. The key mechanisms behind the trade-offs include whether differences in local conditions are driven by biological or economic mechanisms; degree of flexibility embedded in the federal management; the spatial and temporal distribution of economic returns across states; and the status-quo management type. While simple rules-of-thumb are elusive, our analysis reveals the complex political economy dimensions of renewable resource federalism.

Anticipating future learning affects current control decisions: A comparison between passive and active adaptive management in an epidemiological setting.

Infectious disease epidemics present a difficult task for policymakers, requiring the implementation of control strategies under significant time constraints and uncertainty. Mathematical models can be used to predict the outcome of control interventions, providing useful information to policymakers in the event of such an epidemic. However, these models suffer in the early stages of an outbreak from a lack of accurate, relevant information regarding the dynamics and spread of the disease and the efficacy of control. As such, recommendations provided by these models are often incorporated in an ad hoc fashion, as and when more reliable information becomes available. In this work, we show that such trial-and-error-type approaches to management, which do not formally take into account the resolution of uncertainty and how control actions affect this, can lead to sub-optimal management outcomes. We compare three approaches to managing a theoretical epidemic: a non-adaptive management (AM) approach that does not use real-time outbreak information to adapt control, a passive AM approach that incorporates real-time information if and when it becomes available, and an active AM approach that explicitly incorporates the future resolution of uncertainty through gathering real-time information into its initial recommendations. The structured framework of active AM encourages the specification of quantifiable objectives, models of system behaviour and possible control and monitoring actions, followed by an iterative learning and control phase that is able to employ complex control optimisations and resolve system uncertainty. The result is a management framework that is able to provide dynamic, long-term projections to help policymakers meet the objectives of management. We investigate in detail the effect of different methods of incorporating up-to-date outbreak information. We find that, even in a highly simplified system, the method of incorporating new data can lead to different results that may influence initial policy decisions, with an active AM approach to management providing better information that can lead to more desirable outcomes from an epidemic.

Concurrent assessment of epidemiological and operational uncertainties for optimal outbreak control: Ebola as a case study.

Determining how best to manage an infectious disease outbreak may be hindered by both epidemiological uncertainty (i.e. about epidemiological processes) and operational uncertainty (i.e. about the effectiveness of candidate interventions). However, these two uncertainties are rarely addressed concurrently in epidemic studies. We present an approach to simultaneously address both sources of uncertainty, to elucidate which source most impedes decision-making. In the case of the 2014 West African Ebola outbreak, epidemiological uncertainty is represented by a large ensemble of published models. Operational uncertainty about three classes of interventions is assessed for a wide range of potential intervention effectiveness. We ranked each intervention by caseload reduction in each model, initially assuming an unlimited budget as a counterfactual. We then assessed the influence of three candidate cost functions relating intervention effectiveness and cost for different budget levels. The improvement in management outcomes to be gained by resolving uncertainty is generally high in this study; appropriate information gain could reduce expected caseload by more than 50%. The ranking of interventions is jointly determined by the underlying epidemiological process, the effectiveness of the interventions and the size of the budget. An epidemiologically effective intervention might not be optimal if its costs outweigh its epidemiological benefit. Under higher-budget conditions, resolution of epidemiological uncertainty is most valuable. When budgets are tight, however, operational and epidemiological uncertainty are equally important. Overall, our study demonstrates that significant reductions in caseload could result from a careful examination of both epidemiological and operational uncertainties within the same modelling structure. This approach can be applied to decision-making for the management of other diseases for which multiple models and multiple interventions are available.

Real-time decision-making during emergency disease outbreaks.

In the event of a new infectious disease outbreak, mathematical and simulation models are commonly used to inform policy by evaluating which control strategies will minimize the impact of the epidemic. In the early stages of such outbreaks, substantial parameter uncertainty may limit the ability of models to provide accurate predictions, and policymakers do not have the luxury of waiting for data to alleviate this state of uncertainty. For policymakers, however, it is the selection of the optimal control intervention in the face of uncertainty, rather than accuracy of model predictions, that is the measure of success that counts. We simulate the process of real-time decision-making by fitting an epidemic model to observed, spatially-explicit, infection data at weekly intervals throughout two historical outbreaks of foot-and-mouth disease, UK in 2001 and Miyazaki, Japan in 2010, and compare forward simulations of the impact of switching to an alternative control intervention at the time point in question. These are compared to policy recommendations generated in hindsight using data from the entire outbreak, thereby comparing the best we could have done at the time with the best we could have done in retrospect. Our results show that the control policy that would have been chosen using all the data is also identified from an early stage in an outbreak using only the available data, despite high variability in projections of epidemic size. Critically, we find that it is an improved understanding of the locations of infected farms, rather than improved estimates of transmission parameters, that drives improved prediction of the relative performance of control interventions. However, the ability to estimate undetected infectious premises is a function of uncertainty in the transmission parameters. Here, we demonstrate the need for both real-time model fitting and generating projections to evaluate alternative control interventions throughout an outbreak. Our results highlight the use of using models at outbreak onset to inform policy and the importance of state-dependent interventions that adapt in response to additional information throughout an outbreak.

Quantifying the Value of Perfect Information in Emergency Vaccination Campaigns.

Foot-and-mouth disease outbreaks in non-endemic countries can lead to large economic costs and livestock losses but the use of vaccination has been contentious, partly due to uncertainty about emergency FMD vaccination. Value of information methods can be applied to disease outbreak problems such as FMD in order to investigate the performance improvement from resolving uncertainties. Here we calculate the expected value of resolving uncertainty about vaccine efficacy, time delay to immunity after vaccination and daily vaccination capacity for a hypothetical FMD outbreak in the UK. If it were possible to resolve all uncertainty prior to the introduction of control, we could expect savings of £55 million in outbreak cost, 221,900 livestock culled and 4.3 days of outbreak duration. All vaccination strategies were found to be preferable to a culling only strategy. However, the optimal vaccination radius was found to be highly dependent upon vaccination capacity for all management objectives. We calculate that by resolving the uncertainty surrounding vaccination capacity we would expect to return over 85% of the above savings, regardless of management objective. It may be possible to resolve uncertainty about daily vaccination capacity before an outbreak, and this would enable decision makers to select the optimal control action via careful contingency planning.

Adaptive management and the value of information: learning via intervention in epidemiology.

Optimal intervention for disease outbreaks is often impeded by severe scientific uncertainty. Adaptive management (AM), long-used in natural resource management, is a structured decision-making approach to solving dynamic problems that accounts for the value of resolving uncertainty via real-time evaluation of alternative models. We propose an AM approach to design and evaluate intervention strategies in epidemiology, using real-time surveillance to resolve model uncertainty as management proceeds, with foot-and-mouth disease (FMD) culling and measles vaccination as case studies. We use simulations of alternative intervention strategies under competing models to quantify the effect of model uncertainty on decision making, in terms of the value of information, and quantify the benefit of adaptive versus static intervention strategies. Culling decisions during the 2001 UK FMD outbreak were contentious due to uncertainty about the spatial scale of transmission. The expected benefit of resolving this uncertainty prior to a new outbreak on a UK-like landscape would be £45-£60 million relative to the strategy that minimizes livestock losses averaged over alternate transmission models. AM during the outbreak would be expected to recover up to £20.1 million of this expected benefit. AM would also recommend a more conservative initial approach (culling of infected premises and dangerous contact farms) than would a fixed strategy (which would additionally require culling of contiguous premises). For optimal targeting of measles vaccination, based on an outbreak in Malawi in 2010, AM allows better distribution of resources across the affected region; its utility depends on uncertainty about both the at-risk population and logistical capacity. When daily vaccination rates are highly constrained, the optimal initial strategy is to conduct a small, quick campaign; a reduction in expected burden of approximately 10,000 cases could result if campaign targets can be updated on the basis of the true susceptible population. Formal incorporation of a policy to update future management actions in response to information gained in the course of an outbreak can change the optimal initial response and result in significant cost savings. AM provides a framework for using multiple models to facilitate public-health decision making and an objective basis for updating management actions in response to improved scientific understanding.