Adaptive metrics for an evolving pandemic: A dynamic approach to area-level COVID-19 risk designations.

Throughout the COVID-19 pandemic, policymakers have proposed risk metrics, such as the CDC Community Levels, to guide local and state decision-making. However, risk metrics have not reliably predicted key outcomes and have often lacked transparency in terms of prioritization of false-positive versus false-negative signals. They have also struggled to maintain relevance over time due to slow and infrequent updates addressing new variants and shifts in vaccine- and infection-induced immunity. We make two contributions to address these weaknesses. We first present a framework to evaluate predictive accuracy based on policy targets related to severe disease and mortality, allowing for explicit preferences toward false-negative versus false-positive signals. This approach allows policymakers to optimize metrics for specific preferences and interventions. Second, we propose a method to update risk thresholds in real time. We show that this adaptive approach to designating areas as "high risk" improves performance over static metrics in predicting 3-wk-ahead mortality and intensive care usage at both state and county levels. We also demonstrate that with our approach, using only new hospital admissions to predict 3-wk-ahead mortality and intensive care usage has performed consistently as well as metrics that also include cases and inpatient bed usage. Our results highlight that a key challenge for COVID-19 risk prediction is the changing relationship between indicators and outcomes of policy interest. Adaptive metrics therefore have a unique advantage in a rapidly evolving pandemic context.

The US COVID-19 Trends and Impact Survey: Continuous real-time measurement of COVID-19 symptoms, risks, protective behaviors, testing, and vaccination.

The US COVID-19 Trends and Impact Survey (CTIS) is a large, cross-sectional, internet-based survey that has operated continuously since April 6, 2020. By inviting a random sample of Facebook active users each day, CTIS collects information about COVID-19 symptoms, risks, mitigating behaviors, mental health, testing, vaccination, and other key priorities. The large scale of the survey-over 20 million responses in its first year of operation-allows tracking of trends over short timescales and allows comparisons at fine demographic and geographic detail. The survey has been repeatedly revised to respond to emerging public health priorities. In this paper, we describe the survey methods and content and give examples of CTIS results that illuminate key patterns and trends and help answer high-priority policy questions relevant to the COVID-19 epidemic and response. These results demonstrate how large online surveys can provide continuous, real-time indicators of important outcomes that are not subject to public health reporting delays and backlogs. The CTIS offers high value as a supplement to official reporting data by supplying essential information about behaviors, attitudes toward policy and preventive measures, economic impacts, and other topics not reported in public health surveillance systems.

First Do No Harm? Modeling Risks and Benefits of Challenge Trials for Hepatitis C Vaccine Development.

BACKGROUND: In 2019, about 58 million individuals were chronically infected with hepatitis C virus. Some experts have proposed challenge trials for hepatitis C virus vaccine development. METHODS: We modeled incremental infections averted through a challenge approach, under varying assumptions regarding trial duration, number of candidates, and vaccine uptake. We computed the benefit-risk ratio of incremental benefits to risks for challenge versus traditional approaches. We also benchmarked against monetary costs of achieving incremental benefits through treatment. RESULTS: Our base case assumes 3 vaccine candidates, each with an 11% chance of success, corresponding to a 30% probability of successfully developing a vaccine. Given this probability, and assuming a 5-year difference in duration between challenge and traditional trials, a challenge approach would avert an expected 185 000 incremental infections with 20% steady-state uptake compared to a traditional approach and 832 000 with 90% uptake (quality-adjusted life-year benefit-risk ratio, 72 000 & 323 000). It would cost at least $92 million and $416 million, respectively, to obtain equivalent benefits through treatment. BRRs vary considerably across scenarios, depending on input assumptions. CONCLUSIONS: Benefits of a challenge approach increase with more vaccine candidates, faster challenge trials, and greater uptake.

Evaluating the Performance of Centers for Disease Control and Prevention COVID-19 Community Levels as Leading Indicators of COVID-19 Mortality.

BACKGROUND: Centers for Disease Control and Prevention (CDC) defines low, medium, and high "COVID-19 community levels" to guide interventions, but associated mortality rates have not been reported. OBJECTIVE: To evaluate the diagnostic performance of CDC COVID-19 community level metrics as predictors of elevated community mortality risk. DESIGN: Time series analysis over the period of 30 May 2021 through 4 June 2022. SETTING: U.S. states and counties. PARTICIPANTS: U.S. population. MEASUREMENTS: CDC "COVID-19 community level" metrics based on hospital admissions, bed occupancy, and reported cases; reported COVID-19 deaths; and sensitivity, specificity, and predictive values for CDC and alternative metrics. RESULTS: Mean and median weekly mortality rates per 100 000 population after onset of high COVID-19 community level 3 weeks prior were, respectively, 2.6 and 2.4 (interquartile range [IQR], 1.7 to 3.1) across 90 high episodes in states and 4.3 and 2.1 (IQR, 0 to 5.4) across 7987 high episodes in counties. In 85 of 90 (94%) episodes in states and 4801 of 7987 (60%) episodes in counties, lagged weekly mortality after onset exceeded 0.9 per 100 000 population, and in 57 of 90 (63%) episodes in states and 4018 of 7987 (50%) episodes in counties, lagged weekly mortality after onset exceeded 2.1 per 100 000, which is equivalent to approximately 1000 daily deaths in the national population. Alternative metrics based on lower hospital admissions or case thresholds were associated with lower mortality and had higher sensitivity and negative predictive value for elevated mortality, but the CDC metrics had higher specificity and positive predictive value. Ratios between cases, hospitalizations, and deaths have varied substantially over time. LIMITATIONS: Aggregate mortality does not account for nonfatal outcomes or disparities. Continuing evolution of viral variants, immunity, clinical interventions, and public health mitigation strategies complicate prediction for future waves. CONCLUSION: Designing metrics for public health decision making involves tradeoffs between identifying early signals for action and avoiding undue restrictions when risks are modest. Explicit frameworks for evaluating surveillance metrics can improve transparency and decision support. PRIMARY FUNDING SOURCE: Council of State and Territorial Epidemiologists.

Affordability and Value in Decision Rules for Cost-Effectiveness: A Survey of Health Economists.

OBJECTIVES: New health technologies are often expensive, but may nevertheless meet standard thresholds for cost effectiveness, a situation exemplified by recent hepatitis C cures. Currently, cost-effectiveness analysis (CEA) does not supply practical means of weighing trade-offs between cost-effectiveness and affordability, particularly when costs and benefits are temporally separated and in health systems with multiple payers, such as the United States. We formally characterized disagreements in CEA theory and identified how these trade-offs are presently addressed in practice. METHODS: We surveyed 170 health economics researchers. RESULTS: When presented with a hypothetical cost-effective drug therapy in the United States that would require 20% of a state's Medicaid budget over 5 years, 34% of survey respondents recommended that policy makers fund the drug for all patients and 26% for a subset. By contrast, 26% recommended against funding the drug. We found additional disagreement regarding whether the willingness-to-pay threshold should be based on the budget (42%) or societal preferences (41%) and identified 4 approaches to weighing cost-effectiveness and affordability. A total of 61% of respondents did not believe that the threshold used in their last article (most often 1×-3× per capita gross domestic product) represented either the budget or societal willingness-to-pay threshold. CONCLUSIONS: We use these findings to recommend metrics that can inform translation of CEA theory into practice. By contextualizing cost and value, researchers can provide more actionable policy recommendations.

Passing the Test: A Model-Based Analysis of Safe School-Reopening Strategies.

BACKGROUND: The COVID-19 pandemic has induced historic educational disruptions. In April 2021, about 40% of U.S. public school students were not offered full-time in-person education. OBJECTIVE: To assess the risk for SARS-CoV-2 transmission in schools. DESIGN: An agent-based network model was developed to simulate transmission in elementary and high school communities, including home, school, and interhousehold interactions. SETTING: School structure was parametrized to reflect average U.S. classrooms, with elementary schools of 638 students and high schools of 1451 students. Daily local incidence was varied from 1 to 100 cases per 100 000 persons. PARTICIPANTS: Students, faculty, staff, and adult household members. INTERVENTION: Isolation of symptomatic individuals, quarantine of an infected individual's contacts, reduced class sizes, alternative schedules, staff vaccination, and weekly asymptomatic screening. MEASUREMENTS: Transmission was projected among students, staff, and families after a single infection in school and over an 8-week quarter, contingent on local incidence. RESULTS: School transmission varies according to student age and local incidence and is substantially reduced with mitigation measures. Nevertheless, when transmission occurs, it may be difficult to detect without regular testing because of the subclinical nature of most children's infections. Teacher vaccination can reduce transmission to staff, and asymptomatic screening improves understanding of local circumstances and reduces transmission. LIMITATION: Uncertainty exists about the susceptibility and infectiousness of children, and precision is low regarding the effectiveness of specific countermeasures, particularly with new variants. CONCLUSION: With controlled community transmission and moderate mitigation, elementary schools can open safety, but high schools require more intensive mitigation. Asymptomatic screening can facilitate reopening at higher local incidence while minimizing transmission risk. PRIMARY FUNDING SOURCE: Centers for Disease Control and Prevention through the Council of State and Territorial Epidemiologists, National Institute of Allergy and Infectious Diseases, National Institute on Drug Abuse, and Facebook.

One Health approach to cost-effective rabies control in India.

Over 20,000 rabies deaths occur annually in India, representing one-third of global human rabies. The Indian state of Tamil Nadu has pioneered a "One Health" committee to address the challenge of rabies in dogs and humans. Currently, rabies control in Tamil Nadu involves postexposure vaccination of humans after dog bites, whereas potential supplemental approaches include canine vaccination and sterilization. We developed a data-driven rabies transmission model fit to human rabies autopsy data and human rabies surveillance data from Tamil Nadu. Integrating local estimates for canine demography and costs, we predicted the impact of canine vaccination and sterilization on human health outcomes and evaluated cost-effectiveness according to the WHO criteria for India, which correspond to thresholds of $1,582 and $4,746 per disability-adjusted life-years (DALYs) for very cost-effective and cost-effective strategies, respectively. We found that highly feasible strategies focused on stray dogs, vaccinating as few as 7% of dogs annually, could very cost-effectively reduce human rabies deaths by 70% within 5 y, and a modest expansion to vaccinating 13% of stray dogs could cost-effectively reduce human rabies by almost 90%. Through integration over parameter uncertainty, we find that, for a cost-effectiveness threshold above $1,400 per DALY, canine interventions are at least 95% likely to be optimal. If owners are willing to bring dogs to central point campaigns at double the rate that campaign teams can capture strays, expanded annual targets become cost-effective. This case study of cost-effective canine interventions in Tamil Nadu may have applicability to other settings in India and beyond.

COVID-19 and Excess All-Cause Mortality in the US and 20 Comparison Countries, June 2021-March 2022.

This study compares the COVID-19 per capita overall and excess mortality rates in the US vs rates for 20 Organization for Economic Co-operation and Development countries and the timing of any increases in excess mortality between June 2021 and December 2021 (Delta) and December 2021 to March 2022 (Omicron).

When cost-effective interventions are unaffordable: Integrating cost-effectiveness and budget impact in priority setting for global health programs.

Potential cost-effective barriers in cost-effectiveness studies mean that budgetary impact analyses should also be included in post-2015 Sustainable Development Goal projects says Joshua Salomon and colleagues.

Optimal frequency of rabies vaccination campaigns in Sub-Saharan Africa.

Rabies causes more than 24 000 human deaths annually in Sub-Saharan Africa. The World Health Organization recommends annual canine vaccination campaigns with at least 70% coverage to control the disease. While previous studies have considered optimal coverage of animal rabies vaccination, variation in the frequency of vaccination campaigns has not been explored. To evaluate the cost-effectiveness of rabies canine vaccination campaigns at varying coverage and frequency, we parametrized a rabies virus transmission model to two districts of northwest Tanzania, Ngorongoro (pastoral) and Serengeti (agro-pastoral). We found that optimal vaccination strategies were every 2 years, at 80% coverage in Ngorongoro and annually at 70% coverage in Serengeti. We further found that the optimality of these strategies was sensitive to the rate of rabies reintroduction from outside the district. Specifically, if a geographically coordinated campaign could reduce reintroduction, vaccination campaigns every 2 years could effectively manage rabies in both districts. Thus, coordinated campaigns may provide monetary savings in addition to public health benefits. Our results indicate that frequency and coverage of canine vaccination campaigns should be evaluated simultaneously and tailored to local canine ecology as well as to the risk of disease reintroduction from surrounding regions.

Screening Strategies to Reduce COVID-19 Mortality in Nursing Homes.

Importance: Nursing home residents continue to bear a disproportionate share of COVID-19 morbidity and mortality, accounting for 9% of all US COVID-19 deaths in 2023, despite comprising only 0.4% of the population. Objective: To evaluate the cost-effectiveness of screening strategies in reducing COVID-19 mortality in nursing homes. Design and Setting: An agent-based model was developed to simulate SARS-CoV-2 transmission in the nursing home setting. Parameters were determined using SARS-CoV-2 virus data and COVID-19 data from the Centers for Medicare & Medicaid Services and US Centers for Disease Control and Prevention that were published between 2020 and 2023, as well as data on nursing homes published between 2010 and 2023. The model used in this study simulated interactions and SARS-CoV-2 transmission between residents, staff, and visitors in a nursing home setting. The population used in the simulation model was based on the size of the average US nursing home and recommended staffing levels, with 90 residents, 90 visitors (1 per resident), and 83 nursing staff members. Exposure: Screening frequency (none, weekly, and twice weekly) was varied over 30 days against varying levels of COVID-19 community incidence, booster uptake, and antiviral use. Main Outcomes and Measures: The main outcomes were SARS-CoV-2 infections, detected cases per 1000 tests, and incremental cost of screening per life-year gained. Results: Nursing home interactions were modeled between 90 residents, 90 visitors, and 83 nursing staff over 30 days, completing 4000 to 8000 simulations per parameter combination. The incremental cost-effectiveness ratios of weekly and twice-weekly screening were less than $150 000 per resident life-year with moderate (50 cases per 100 000) and high (100 cases per 100 000) COVID-19 community incidence across low-booster uptake and high-booster uptake levels. When COVID-19 antiviral use reached 100%, screening incremental cost-effectiveness ratios increased to more than $150 000 per life-year when booster uptake was low and community incidence was high. Conclusions and Relevance: The results of this cost-effectiveness analysis suggest that screening may be effective for reducing COVID-19 mortality in nursing homes when COVID-19 community incidence is high and/or booster uptake is low. Nursing home administrators can use these findings to guide planning in the context of widely varying levels of SARS-CoV-2 transmission and intervention measures across the US.

Geographic poverty and racial/ethnic disparities in cervical cancer precursor rates in Connecticut, 2008-2009.

OBJECTIVES: We examined associations of geographic measures of poverty, race, ethnicity, and city status with rates of cervical intraepithelial neoplasia grade 2 or higher and adenocarcinoma in situ (CIN2+/AIS), known precursors to cervical cancer. METHODS: We identified 3937 cases of CIN2+/AIS among women aged 20 to 39 years in statewide surveillance data from Connecticut for 2008 to 2009. We geocoded cases to census tracts and used census data to calculate overall and age-specific rates. Poisson regression determined whether rates differed by geographic measures. RESULTS: The average annual rate of CIN2+/AIS was 417.6 per 100,000 women. Overall, higher rates of CIN2+/AIS were associated with higher levels of poverty and higher proportions of Black residents. Poverty was the strongest and most consistently associated measure. However, among women aged 20 to 24 years, we observed inverse associations between poverty and CIN2+/AIS rates. CONCLUSIONS: Disparities in cervical cancer precursors exist for poverty and race, but these effects are age dependent. This information is necessary to monitor human papillomavirus vaccine impact and target vaccination strategies.

Adaptive metrics for an evolving pandemic A dynamic approach to area-level COVID-19 risk designations.

Throughout the COVID-19 pandemic, policymakers have proposed risk metrics, such as the CDC Community Levels, to guide local and state decision-making. However, risk metrics have not reliably predicted key outcomes and often lack transparency in terms of prioritization of false positive versus false negative signals. They have also struggled to maintain relevance over time due to slow and infrequent updates addressing new variants and shifts in vaccine- and infection-induced immunity. We make two contributions to address these weaknesses of risk metrics. We first present a framework to evaluate predictive accuracy based on policy targets related to severe disease and mortality, allowing for explicit preferences toward false negative versus false positive signals. This approach allows policymakers to optimize metrics for specific preferences and interventions. Second, we propose a novel method to update risk thresholds in real-time. We show that this adaptive approach to designating areas as "high risk" improves performance over static metrics in predicting 3-week-ahead mortality and intensive care usage at both state and county levels. We also demonstrate that with our approach, using only new hospital admissions to predict 3-week-ahead mortality and intensive care usage has performed consistently as well as metrics that also include cases and inpatient bed usage. Our results highlight that a key challenge for COVID-19 risk prediction is the changing relationship between indicators and outcomes of policy interest. Adaptive metrics therefore have a unique advantage in a rapidly evolving pandemic context. Significance Statement: In the rapidly-evolving COVID-19 pandemic, public health risk metrics often become less relevant over time. Risk metrics are designed to predict future severe disease and mortality based on currently-available surveillance data, such as cases and hospitalizations. However, the relationship between cases, hospitalizations, and mortality has varied considerably over the course of the pandemic, in the context of new variants and shifts in vaccine- and infection-induced immunity. We propose an adaptive approach that regularly updates metrics based on the relationship between surveillance inputs and future outcomes of policy interest. Our method captures changing pandemic dynamics, requires only hospitalization input data, and outperforms static risk metrics in predicting high-risk states and counties.

Prevalence and Risk Factors for School-Associated Transmission of SARS-CoV-2.

Importance: School-associated SARS-CoV-2 transmission is described as uncommon, although the true transmission rate is unknown. Objective: To identify the SARS-CoV-2 secondary attack rate (SAR) in schools and factors associated with transmission. Design, Setting, and Participants: This cohort study examined the risk of school-based transmission of SARS-CoV-2 among kindergarten through grade 12 students and staff in 10 Massachusetts school districts during 2 periods: fall 2020/spring 2021 (F20/S21) and fall 2021 (F21). School staff collected data on SARS-CoV-2 index cases and school-based contacts, and SAR was defined as the proportion of contacts acquiring SARS-CoV-2 infection. Exposure: SARS-CoV-2. Main Outcomes and Measures: Potential factors associated with transmission, including grade level, masking, exposure location, vaccination history, and Social Vulnerability Index (SVI), were analyzed using univariable and multivariable logistic regression models. Results: For F20/S21, 8 school districts (70 schools, >33 000 students) were included and reported 435 index cases (151 staff, 216 students, and 68 missing role) with 1771 school-based contacts (278 staff, 1492 students, and 1 missing role). For F21, 5 districts (34 schools, >18 000 students) participated and reported 309 index cases (37 staff, 207 students, and 65 missing role) with 1673 school-based contacts (107 staff and 1566 students). The F20/S21 SAR was 2.2% (lower bound, 1.6%; upper bound, 26.7%), and the F21 SAR was 2.8% (lower bound, 2.6%; upper bound, 7.4%). In multivariable analysis, during F20/S21, masking was associated with a lower odds of transmission compared with not masking (odds radio [OR], 0.12; 95% CI, 0.04-0.40; P < .001). In F21, classroom exposure vs out-of-classroom exposure was associated with increased odds of transmission (OR, 2.47; 95% CI, 1.07-5.66; P = .02); a fully vaccinated vs unvaccinated contact was associated with a lower odds of transmission (OR, 0.04; 95% CI, 0.00-0.62; P < .001). In both periods, a higher SVI was associated with a greater odds of transmission. Conclusions and Relevance: In this study of Massachusetts schools, the SAR for SARS-CoV-2 among school-based contacts was low during 2 periods, and factors associated with transmission risk varied over time. These findings suggest that ongoing surveillance efforts may be essential to ensure that both targeted resources and mitigation practices remain optimal and relevant for disease prevention.

Estimated Transmission Outcomes and Costs of SARS-CoV-2 Diagnostic Testing, Screening, and Surveillance Strategies Among a Simulated Population of Primary School Students.

Importance: In addition to illness, the COVID-19 pandemic has led to historic educational disruptions. In March 2021, the federal government allocated $10 billion for COVID-19 testing in US schools. Objective: Costs and benefits of COVID-19 testing strategies were evaluated in the context of full-time, in-person kindergarten through eighth grade (K-8) education at different community incidence levels. Design, Setting, and Participants: An updated version of a previously published agent-based network model was used to simulate transmission in elementary and middle school communities in the United States. Assuming dominance of the delta SARS-CoV-2 variant, the model simulated an elementary school (638 students in grades K-5, 60 staff) and middle school (460 students grades 6-8, 51 staff). Exposures: Multiple strategies for testing students and faculty/staff, including expanded diagnostic testing (test to stay) designed to avoid symptom-based isolation and contact quarantine, screening (routinely testing asymptomatic individuals to identify infections and contain transmission), and surveillance (testing a random sample of students to identify undetected transmission and trigger additional investigation or interventions). Main Outcomes and Measures: Projections included 30-day cumulative incidence of SARS-CoV-2 infection, proportion of cases detected, proportion of planned and unplanned days out of school, cost of testing programs, and childcare costs associated with different strategies. For screening policies, the cost per SARS-CoV-2 infection averted in students and staff was estimated, and for surveillance, the probability of correctly or falsely triggering an outbreak response was estimated at different incidence and attack rates. Results: Compared with quarantine policies, test-to-stay policies are associated with similar model-projected transmission, with a mean of less than 0.25 student days per month of quarantine or isolation. Weekly universal screening is associated with approximately 50% less in-school transmission at one-seventh to one-half the societal cost of hybrid or remote schooling. The cost per infection averted in students and staff by weekly screening is lowest for schools with less vaccination, fewer other mitigation measures, and higher levels of community transmission. In settings where local student incidence is unknown or rapidly changing, surveillance testing may detect moderate to large in-school outbreaks with fewer resources compared with schoolwide screening. Conclusions and Relevance: In this modeling study of a simulated population of primary school students and simulated transmission of COVID-19, test-to-stay policies and/or screening tests facilitated consistent in-person school attendance with low transmission risk across a range of community incidence. Surveillance was a useful reduced-cost option for detecting outbreaks and identifying school environments that would benefit from increased mitigation.

Model-Estimated Association Between Simulated US Elementary School-Related SARS-CoV-2 Transmission, Mitigation Interventions, and Vaccine Coverage Across Local Incidence Levels.

Importance: With recent surges in COVID-19 incidence and vaccine authorization for children aged 5 to 11 years, elementary schools face decisions about requirements for masking and other mitigation measures. These decisions require explicit determination of community objectives (eg, acceptable risk level for in-school SARS-CoV-2 transmission) and quantitative estimates of the consequences of changing mitigation measures. Objective: To estimate the association between adding or removing in-school mitigation measures (eg, masks) and COVID-19 outcomes within an elementary school community at varying student vaccination and local incidence rates. Design, Setting, and Participants: This decision analytic model used an agent-based model to simulate SARS-CoV-2 transmission within a school community, with a simulated population of students, teachers and staff, and their household members (ie, immediate school community). Transmission was evaluated for a range of observed local COVID-19 incidence (0-50 cases per 100 000 residents per day, assuming 33% of all infections detected). The population used in the model reflected the mean size of a US elementary school, including 638 students and 60 educators and staff members in 6 grades with 5 classes per grade. Exposures: Variant infectiousness (representing wild-type virus, Alpha variant, and Delta variant), mitigation effectiveness (0%-100% reduction in the in-school secondary attack rate, representing increasingly intensive combinations of mitigations including masking and ventilation), and student vaccination levels were varied. Main Outcomes and Measures: The main outcomes were (1) probability of at least 1 in-school transmission per month and (2) mean increase in total infections per month among the immediate school community associated with a reduction in mitigation; multiple decision thresholds were estimated for objectives associated with each outcome. Sensitivity analyses on adult vaccination uptake, vaccination effectiveness, and testing approaches (for selected scenarios) were conducted. Results: With student vaccination coverage of 70% or less and moderate assumptions about mitigation effectiveness (eg, masking), mitigation could only be reduced when local case incidence was 14 or fewer cases per 100 000 residents per day to keep the mean additional cases associated with reducing mitigation to 5 or fewer cases per month. To keep the probability of any in-school transmission to less than 50% per month, the local case incidence would have to be 4 or fewer cases per 100 000 residents per day. Conclusions and Relevance: In this study, in-school mitigation measures (eg, masks) and student vaccinations were associated with substantial reductions in transmissions and infections, but the level of reduction varied across local incidence. These findings underscore the potential role for responsive plans that deploy mitigation strategies based on local COVID-19 incidence, vaccine uptake, and explicit consideration of community objectives.