Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the United States.

Short-term probabilistic forecasts of the trajectory of the COVID-19 pandemic in the United States have served as a visible and important communication channel between the scientific modeling community and both the general public and decision-makers. Forecasting models provide specific, quantitative, and evaluable predictions that inform short-term decisions such as healthcare staffing needs, school closures, and allocation of medical supplies. Starting in April 2020, the US COVID-19 Forecast Hub (https://covid19forecasthub.org/) collected, disseminated, and synthesized tens of millions of specific predictions from more than 90 different academic, industry, and independent research groups. A multimodel ensemble forecast that combined predictions from dozens of groups every week provided the most consistently accurate probabilistic forecasts of incident deaths due to COVID-19 at the state and national level from April 2020 through October 2021. The performance of 27 individual models that submitted complete forecasts of COVID-19 deaths consistently throughout this year showed high variability in forecast skill across time, geospatial units, and forecast horizons. Two-thirds of the models evaluated showed better accuracy than a naïve baseline model. Forecast accuracy degraded as models made predictions further into the future, with probabilistic error at a 20-wk horizon three to five times larger than when predicting at a 1-wk horizon. This project underscores the role that collaboration and active coordination between governmental public-health agencies, academic modeling teams, and industry partners can play in developing modern modeling capabilities to support local, state, and federal response to outbreaks.

Large benefits to youth-focused HIV treatment-as-prevention efforts in generalized heterosexual populations: An agent-based simulation model.

Predominantly heterosexual HIV-1 epidemics like those in sub-Saharan Africa continue to have high HIV incidence in young people. We used a stochastic, agent-based model for age-disparate networks to test the hypothesis that focusing uptake and retention of ART among youth could enhance the efficiency of treatment as prevention (TasP) campaigns. We used the model to identify strategies that reduce incidence to negligible levels (i.e., < 0.1 cases/100 person-years) 20-25 years after initiation of a targeted TasP campaign. The model was parameterized using behavioral, demographic, and clinical data from published papers and national reports. To keep a focus on the underlying age effects we model a generalized heterosexual population with average risks (i.e., no MSM, no PWIDs, no sex workers) and no entry of HIV+ people from other regions. The model assumes that most people (default 95%, range in variant simulations 60-95%) are "linkable"; i.e., could get linked to effective care given sufficient resources. To simplify the accounting, we assume a rapid jump in the number of people receiving treatment at the start of the TasP campaign, followed by a 2% annual increase that continues until all linkable HIV+ people have been treated. Under historical scenarios of CD4-based targeted ART allocation and current policies of untargeted (random) ART allocation, our model predicts that viral replication would need to be suppressed in 60-85% of infected people at the start of the TasP campaign to drive incidence to negligible levels. Under age-based strategies, by contrast, this percentage dropped by 18-54%, depending on the strength of the epidemic and the age target. For our baseline model, targeting those under age 30 halved the number of people who need to be treated. Age-based targeting also minimized total and time-discounted AIDS deaths over 25 years. Age-based targeting yielded benefits without being highly exclusive; in a model in which 60% of infected people were treated, ~87% and ~58% of those initiating therapy during a campaign targeting those <25 and <30 years, respectively, fell outside the target group. Sensitivity analyses revealed that youth-focused TasP is beneficial due to age-related risk factors (e.g. shorter relationship durations), and an age-specific herd immunity (ASHI) effect that protects uninfected adolescents entering the sexually active population. As testing rates increase in response to UNAIDS 90-90-90 goals, efforts to link all young people to care and treatment could contribute enormously to ending the HIV epidemic.

Visualization and analytics tools for infectious disease epidemiology: a systematic review.

BACKGROUND: A myriad of new tools and algorithms have been developed to help public health professionals analyze and visualize the complex data used in infectious disease control. To better understand approaches to meet these users' information needs, we conducted a systematic literature review focused on the landscape of infectious disease visualization tools for public health professionals, with a special emphasis on geographic information systems (GIS), molecular epidemiology, and social network analysis. The objectives of this review are to: (1) identify public health user needs and preferences for infectious disease information visualization tools; (2) identify existing infectious disease information visualization tools and characterize their architecture and features; (3) identify commonalities among approaches applied to different data types; and (4) describe tool usability evaluation efforts and barriers to the adoption of such tools. METHODS: We identified articles published in English from January 1, 1980 to June 30, 2013 from five bibliographic databases. Articles with a primary focus on infectious disease visualization tools, needs of public health users, or usability of information visualizations were included in the review. RESULTS: A total of 88 articles met our inclusion criteria. Users were found to have diverse needs, preferences and uses for infectious disease visualization tools, and the existing tools are correspondingly diverse. The architecture of the tools was inconsistently described, and few tools in the review discussed the incorporation of usability studies or plans for dissemination. Many studies identified concerns regarding data sharing, confidentiality and quality. Existing tools offer a range of features and functions that allow users to explore, analyze, and visualize their data, but the tools are often for siloed applications. Commonly cited barriers to widespread adoption included lack of organizational support, access issues, and misconceptions about tool use. DISCUSSION AND CONCLUSION: As the volume and complexity of infectious disease data increases, public health professionals must synthesize highly disparate data to facilitate communication with the public and inform decisions regarding measures to protect the public's health. Our review identified several themes: consideration of users' needs, preferences, and computer literacy; integration of tools into routine workflow; complications associated with understanding and use of visualizations; and the role of user trust and organizational support in the adoption of these tools. Interoperability also emerged as a prominent theme, highlighting challenges associated with the increasingly collaborative and interdisciplinary nature of infectious disease control and prevention. Future work should address methods for representing uncertainty and missing data to avoid misleading users as well as strategies to minimize cognitive overload.

Rapid Development of a Registry to Accelerate COVID-19 Vaccine Clinical Trials.

Background: The unprecedented scientific response to the SARS-Cov-2 pandemic in 2020 required the rapid development and activation of extensive clinical trial networks to study vaccines and therapeutics. The COVID-19 Prevention Network (CoVPN) coordinated hundreds of sites conducting phase 2 and 3 clinical trials of vaccines and antibody therapeutics. To facilitate these clinical trials, the CoVPN Volunteer Screening Registry (VSR) was created to collect volunteer information at scale, identify volunteers at risk of COVID-19 who met enrollment criteria, distribute candidates across clinical trial sites, and enable monitoring of volunteering and enrollment progress. Methods: We developed a secure database to support three primary web-based interfaces: a national volunteer questionnaire intake form, a clinical trial site portal, and an Administrative Portal. The Site Portal supported filters based on volunteer attributes, visual analytics, enrollment status tracking, geographic search, and clinical risk prediction. The Administrative Portal supported oversight and development with pre-specified reports aggregated by geography, trial, and trial site; charts of volunteer rates over time; volunteer risk score calculation; and dynamic, user-defined reports. Findings: Over 650,000 volunteers joined the VSR, and 1094 users were trained to utilize the system. The VSR played a key role in recruitment for the Moderna, Oxford-AstraZeneca, Janssen, and Novavax vaccine clinical trials, provided support to the Pfizer and Sanofi vaccine and prophylactic antibody clinical trials, and enhanced the diversity of trial participants. Clinical trial sites selected 166,729 volunteer records for follow-up screening, and of these 47·7% represented groups prioritized for increased enrollment. Despite the unprecedented urgency of its development, the system maintained 99·99% uptime. Interpretation: The success of the VSR demonstrates that information tools can be rapidly yet safely developed through a public-private partnership and integrated into a distributed and accelerated clinical trial setting. We further summarize the requirements, design, and development of the system, and discuss lessons learned for future pandemic preparedness.

Technical desiderata for the integration of genomic data into Electronic Health Records.

The era of "Personalized Medicine," guided by individual molecular variation in DNA, RNA, expressed proteins and other forms of high volume molecular data brings new requirements and challenges to the design and implementation of Electronic Health Records (EHRs). In this article we describe the characteristics of biomolecular data that differentiate it from other classes of data commonly found in EHRs, enumerate a set of technical desiderata for its management in healthcare settings, and offer a candidate technical approach to its compact and efficient representation in operational systems.

Why does age at HIV infection correlate with set point viral load? An evolutionary hypothesis.

BACKGROUND: Set-point viral load (SPVL) correlates with the age at which people acquire HIV. Although immunosenescence may seem like a parsimonious explanation for this, it does not easily explain the observation that the relationship between age and SPVL attenuates when accounting for source partner SPVL. Here we propose an alternative explanation that encompasses this latter finding: that decreasing risk of acquisition with older age generates a selection bottleneck that selects for more virulent strains with age. METHODS: We adapted a previously published model of HIV transmission and evolution (EvoNetHIV), parameterized here for men who have sex with men (MSM). We conducted a series of simulation experiments that vary seven behavioral or clinical parameters that affect exposure risk as people age. We conducted regressions to determine the mean increase in SPVL per 10-year increase in seroconversion age, with and without source SPVL in the model. RESULTS: All runs generated significant relationships between seroconversion age and SPVL when not including source SPVL. All saw attenuated relationships, most to near 0, with source SPVL included. Four of our behavioral measures (relational duration, age-related homophily, coital frequency, and mean age at relationship formation) had clear effects on this relationship, all in the hypothesized direction. Combining multiple forms of behavioral heterogeneity yielded an increase of 0.056 log10 copies/mL SPVL per 10-year increase in seroconversion age, nearly as large as that seen in two empirical studies of age-SPVL correlations in MSM. CONCLUSION: The higher virulence of HIV among those infected later in life may be partly explained by a combination of selective bottlenecks and behavioral heterogeneity by age. Variation in the strength of this effect across populations may be in part due to different behavioral, epidemiological and clinical conditions, and not require assumptions about differences in patterns of immunosenescence among populations.

Test-and-treat coverage and HIV virulence evolution among men who have sex with men.

HIV set point viral load (SPVL), the viral load established shortly after initial infection, is a proxy for HIV virulence: higher SPVLs lead to higher risk of transmission and faster disease progression. Three models of test-and-treat scenarios, mainly in heterosexual populations, found that increasing treatment coverage selected for more virulent viruses. We modeled virulence evolution in a population of men who have sex with men (MSM) with increasing test-and-treat coverage. We extended a stochastic, dynamic network model (EvoNetHIV). We varied relationship patterns (MSM vs. heterosexual), HIV transmission models (increasing vs. plateauing probability of transmission at very high viral loads), and treatment roll-out (with explicit testing or fixed intervals between infection and treatment). In scenarios most similar to previous models (longer relational durations and the plateauing transmission function), we replicated trends previously found: increasing treatment coverage led to increased virulence (0.12 log10 increase in mean population SPVL between 20% and 100% treatment coverage). In scenarios reflecting MSM behavioral data using the increasing transmission function, increasing treatment coverage selected for viruses with lower virulence (0.16 log10 decrease in mean population SPVL between 20% and 100% treatment coverage). These findings emphasize the impact of sexual network conditions and transmission function details on predicted epidemiological and evolutionary outcomes. Varying these features creates very different evolutionary environments, which in turn lead to opposite effects in mean population SPVL evolution. Our results suggest that, under some realistic conditions, effective test-and-treat strategies may not face the previously reported tradeoff in which increasing coverage leads to evolution of greater virulence. This suggests instead that a virtuous cycle of increasing treatment coverage and diminishing virulence is possible.

Relational concurrency, stages of infection, and the evolution of HIV set point viral load.

HIV viral load (VL) predicts both transmission potential and rate of disease progression. For reasons that are still not fully understood, the set point viral load (SPVL) established after acute infection varies across individuals and populations. Previous studies have suggested that population mean SPVL (MSPVL) has evolved near an optimum that reflects a trade-off between transmissibility and host survival. Sexual network structures affect rates of potential exposure during different within-host phases of infection marked by different transmission probabilities, and thus affect the number and timing of transmission events. These structures include relational concurrency, which has been argued to explain key differences in HIV burden across populations. We hypothesize that concurrency will alter the fitness landscape for SPVL in ways that differ from other network features whose impacts accrue at other times during infection. To quantitatively test this hypothesis, we developed a dynamic, stochastic, data-driven network model of HIV transmission, and evolution to assess the impact of key sexual network phenomena on MSPVL evolution. Experiments were repeated in sensitivity runs that made different assumptions about transmissibility during acute infection, SPVL heritability, and the functional form of the relationship between VL and transmissibility. For our main transmission model, scenarios yielded MSPVLs ranging from 4.4 to 4.75 log10 copies/ml, covering much of the observed empirical range. MSPVL evolved to be higher in populations with high concurrency and shorter relational durations, with values varying over a clinically significant range. In linear regression analyses on these and other predictors, main effects were significant (P < 0.05), as were interaction terms, indicating that effects are interdependent. We also noted a strong correlation between two key emergent properties measured at the end of the simulations-MSPVL and HIV prevalence-most clearly for phenomena that affect transmission networks early in infection. Controlling for prevalence, high concurrency yielded higher MSPVL than other network phenomena. Interestingly, we observed lower prevalence in runs in which SPVL heritability was zero, indicating the potential for viral evolution to exacerbate disease burden over time. Future efforts to understand empirical variation in MSPVL should consider local HIV burden and basic sexual behavioral and network structure.

Expanding assessments of translational research programs: supplementing metrics with value judgments.

The University of Washington (UW) Institute for Translational Health Sciences (ITHS), funded by a Clinical and Translational Sciences Award program, has supplemented its initial Kellogg Logic Model-based program evaluation with the eight judgment-based evaluative elements of the World Health Organization's (WHO) Health Services Assessment Model. This article describes the relationship between the two models, the rationale for the decision to supplement the evaluation with WHO evaluative elements, the value-added results of the WHO evaluative elements, and plans for further developing the WHO assessments.

Timeliness and data element completeness of immunization data in Washington State in 2010: a comparison of data exchange methods.

Health information systems receive data through various methods. These data exchange methods have the potential to influence data quality. We assessed a de-identified 2010 dataset including 757,476 demographic records and 2,634,101 vaccination records from Washington State's Immunization Information System (IIS) to describe timeliness and completeness of IIS data across several data exchange methods: manual entry, HL7, and flat file upload. Overall, manually-entered data and HL7 records were more timely than records imported as flat files. Completeness, though very high overall, was slightly higher for records arriving via flat file. Washington State IIS users, including clinicians and public health, rely on its data to inform patient care and determine population coverage of immunizations. Our results suggest that although data element completeness in systems like Washington's IIS will likely not be immediately or significantly impacted by provider's migration to HL7 connections with IISs, timeliness could be substantially improved when using HL7 connections.

Methods to identify standard data elements in clinical and public health forms.

The fragmentation of clinical and public health systems results in divergent information collection practices, presenting challenges to standardization and EHR certification efforts. Data forms employed in public health jurisdictions nationwide reflect these differences in patient treatment, monitoring and evaluation, and follow-up, presenting challenges for data integration. To study these variations, we surveyed tuberculosis contact investigation forms from all fifty states, three municipalities and two countries. We apply statistics and cluster analysis to analyze the divergent content of contact investigation forms with the goal of characterizing normative practices and identifying a common core of data fields. We found widespread variation in data elements between states in the study, with the "Name" field being the only ubiquitous data element. Our method reveals distinct groupings of data fields employed in certain regions, allowing the simultaneous identification of core standard data fields as well as variations in practice.

Microevolution of the direct repeat locus of Mycobacterium tuberculosis in a strain prevalent in San Francisco.

We describe a microevolutionary event of a prevalent strain of Mycobacterium tuberculosis that caused two outbreaks in San Francisco. During the second outbreak, a direct variable repeat was lost. We discuss the mechanisms of this change and the implications of analyzing multiple genetic loci in this context.