Challenges of COVID-19 Case Forecasting in the US, 2020-2021.

During the COVID-19 pandemic, forecasting COVID-19 trends to support planning and response was a priority for scientists and decision makers alike. In the United States, COVID-19 forecasting was coordinated by a large group of universities, companies, and government entities led by the Centers for Disease Control and Prevention and the US COVID-19 Forecast Hub (https://covid19forecasthub.org). We evaluated approximately 9.7 million forecasts of weekly state-level COVID-19 cases for predictions 1-4 weeks into the future submitted by 24 teams from August 2020 to December 2021. We assessed coverage of central prediction intervals and weighted interval scores (WIS), adjusting for missing forecasts relative to a baseline forecast, and used a Gaussian generalized estimating equation (GEE) model to evaluate differences in skill across epidemic phases that were defined by the effective reproduction number. Overall, we found high variation in skill across individual models, with ensemble-based forecasts outperforming other approaches. Forecast skill relative to the baseline was generally higher for larger jurisdictions (e.g., states compared to counties). Over time, forecasts generally performed worst in periods of rapid changes in reported cases (either in increasing or decreasing epidemic phases) with 95% prediction interval coverage dropping below 50% during the growth phases of the winter 2020, Delta, and Omicron waves. Ideally, case forecasts could serve as a leading indicator of changes in transmission dynamics. However, while most COVID-19 case forecasts outperformed a naïve baseline model, even the most accurate case forecasts were unreliable in key phases. Further research could improve forecasts of leading indicators, like COVID-19 cases, by leveraging additional real-time data, addressing performance across phases, improving the characterization of forecast confidence, and ensuring that forecasts were coherent across spatial scales. In the meantime, it is critical for forecast users to appreciate current limitations and use a broad set of indicators to inform pandemic-related decision making.

Evaluation of FluSight influenza forecasting in the 2021-22 and 2022-23 seasons with a new target laboratory-confirmed influenza hospitalizations.

Accurate forecasts can enable more effective public health responses during seasonal influenza epidemics. Forecasting teams were asked to provide national and jurisdiction-specific probabilistic predictions of weekly confirmed influenza hospital admissions for one through four weeks ahead for the 2021-22 and 2022-23 influenza seasons. Across both seasons, 26 teams submitted forecasts, with the submitting teams varying between seasons. Forecast skill was evaluated using the Weighted Interval Score (WIS), relative WIS, and coverage. Six out of 23 models outperformed the baseline model across forecast weeks and locations in 2021-22 and 12 out of 18 models in 2022-23. Averaging across all forecast targets, the FluSight ensemble was the 2nd most accurate model measured by WIS in 2021-22 and the 5th most accurate in the 2022-23 season. Forecast skill and 95% coverage for the FluSight ensemble and most component models degraded over longer forecast horizons and during periods of rapid change. Current influenza forecasting efforts help inform situational awareness, but research is needed to address limitations, including decreased performance during periods of changing epidemic dynamics.

The United States COVID-19 Forecast Hub dataset.

Academic researchers, government agencies, industry groups, and individuals have produced forecasts at an unprecedented scale during the COVID-19 pandemic. To leverage these forecasts, the United States Centers for Disease Control and Prevention (CDC) partnered with an academic research lab at the University of Massachusetts Amherst to create the US COVID-19 Forecast Hub. Launched in April 2020, the Forecast Hub is a dataset with point and probabilistic forecasts of incident cases, incident hospitalizations, incident deaths, and cumulative deaths due to COVID-19 at county, state, and national, levels in the United States. Included forecasts represent a variety of modeling approaches, data sources, and assumptions regarding the spread of COVID-19. The goal of this dataset is to establish a standardized and comparable set of short-term forecasts from modeling teams. These data can be used to develop ensemble models, communicate forecasts to the public, create visualizations, compare models, and inform policies regarding COVID-19 mitigation. These open-source data are available via download from GitHub, through an online API, and through R packages.

Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the United States.

Short-term probabilistic forecasts of the trajectory of the COVID-19 pandemic in the United States have served as a visible and important communication channel between the scientific modeling community and both the general public and decision-makers. Forecasting models provide specific, quantitative, and evaluable predictions that inform short-term decisions such as healthcare staffing needs, school closures, and allocation of medical supplies. Starting in April 2020, the US COVID-19 Forecast Hub (https://covid19forecasthub.org/) collected, disseminated, and synthesized tens of millions of specific predictions from more than 90 different academic, industry, and independent research groups. A multimodel ensemble forecast that combined predictions from dozens of groups every week provided the most consistently accurate probabilistic forecasts of incident deaths due to COVID-19 at the state and national level from April 2020 through October 2021. The performance of 27 individual models that submitted complete forecasts of COVID-19 deaths consistently throughout this year showed high variability in forecast skill across time, geospatial units, and forecast horizons. Two-thirds of the models evaluated showed better accuracy than a naïve baseline model. Forecast accuracy degraded as models made predictions further into the future, with probabilistic error at a 20-wk horizon three to five times larger than when predicting at a 1-wk horizon. This project underscores the role that collaboration and active coordination between governmental public-health agencies, academic modeling teams, and industry partners can play in developing modern modeling capabilities to support local, state, and federal response to outbreaks.

Developing a decision-making framework for insect pest management: a case study using Aphis glycines (Hemiptera: Aphididae).

BACKGROUND: The profitability of farming varies based on factors such as a crop's market value, input costs and occurrence of resistant pests, all capable of altering the value of pest management tactics in an integrated pest management program. We provide a framework for calculating expected yield and expected net revenue of pest management scenarios, using the soybean aphid (Aphis glycines) as a case study. Foliar insecticide and host-plant resistance are effective management tactics for preventing yield loss from soybean aphid outbreaks; however, pyrethroid-resistant aphid populations pose a management challenge for farmers. We evaluated eight scenarios relevant to soybean aphid management in Iowa with varying probabilities of aphid outbreaks and insecticide-resistant aphids occurring. RESULTS: Our equation suggests that insecticide use is profitable when the probability of an aphid outbreak is ≥29%, and soybean production will become more costly with increasing probability of pyrethroid-resistant aphids. If farmers continue to use pyrethroids, they will not experience financial consequences from pyrethroid-resistant aphids until the chance of insecticide resistance is 48%. Aphid-resistant varieties provided consistent yield and offered the highest net revenue under all conditions. CONCLUSION: This framework can be used for other crop-pest systems to evaluate the profitability of management tactics and investigate how resistance impacts revenue for farmers. Including the cost of resistance in crop budgets can help farmers and agronomic consultants comprehend these impacts and enhance decision-making to increase revenue and curb resistance development.

Knot selection in sparse Gaussian processes with a variational objective function.

Sparse, knot-based Gaussian processes have enjoyed considerable success as scalable approximations of full Gaussian processes. Certain sparse models can be derived through specific variational approximations to the true posterior, and knots can be selected to minimize the Kullback-Leibler divergence between the approximate and true posterior. While this has been a successful approach, simultaneous optimization of knots can be slow due to the number of parameters being optimized. Furthermore, there have been few proposed methods for selecting the number of knots, and no experimental results exist in the literature. We propose using a one-at-a-time knot selection algorithm based on Bayesian optimization to select the number and locations of knots. We showcase the competitive performance of this method relative to optimization of knots simultaneously on three benchmark datasets, but at a fraction of the computational cost.

Fully Bayesian analysis of allele-specific RNA-seq data.

Diploid organisms have two copies of each gene, called alleles, that can be separately transcribed. The RNA abundance associated to any particular allele is known as allele-specific expression (ASE). When two alleles have polymorphisms in transcribed regions, ASE can be studied using RNA-seq read count data. ASE has characteristics different from the regular RNA-seq expression: ASE cannot be assessed for every gene, measures of ASE can be biased towards one of the alleles (reference allele), and ASE provides two measures of expression for a single gene for each biological samples with leads to additional complications for single-gene models. We present statistical methods for modeling ASE and detecting genes with differential allelic expression. We propose a hierarchical, overdispersed, count regression model to deal with ASE counts. The model accommodates gene-specific overdispersion, has an internal measure of the reference allele bias, and uses random effects to model the gene-specific regression parameters. Fully Bayesian inference is obtained using the fbseq package that implements a parallel strategy to make the computational times reasonable. Simulation and real data analysis suggest the proposed model is a practical and powerful tool for the study of differential ASE.

The impact of dietary supplementation of arginine during gestation in a commercial swine herd: II. Offspring performance.

Arginine (Arg) is an important amino acid of pig fetal development; however, whether Arg improves postnatal performance is ill-defined. Therefore, the influence of Arg supplementation at different gestational stages on offspring performance was evaluated in a commercial swine herd. Sows (n = 548) were allocated into 4, diet by stage of gestation treatments: Control (n = 143; 0% suppl. Arg), or dietary treatments supplemented with 1% L-Arg (free-base; Ajinomoto Animal Nutrition North America, Inc., Chicago, IL): from 15 to 45 d of gestation (n = 138; Early-Arg); 15 d of gestation to farrowing (n = 139; Full-Arg); and from day 85 of gestation to farrowing (n = 128; Late-Arg). All offspring were individually identified and weighed at birth; at weaning, a subset was selected for evaluation of carcass performance at market. All data were analyzed using birth weight (BiWt) and age as covariates. Wean weights (WW) and prewean (PW) ADG tended to increase (P = 0.06) in progeny from sows supplemented with Arg, as compared to progeny from Control sows. Preplanned contrast comparisons revealed an increased (P = 0.03) BiWt for pigs from sows receiving 1% L-Arg prior to day 45 of gestation (Early-Arg and Full-Arg; 1.38 kg/pig), as compared to pigs from sows not supplemented prior to day 45 of gestation (Control and Late-Arg; 1.34 kg/pig). No difference in BiWt was observed (1.36 kg/pig; P = 0.68) for Arg supplementation after day 85 of gestation (Full-Arg and Late-Arg), as compared to those not receiving Arg supplementation after day 85 (Control and Early-Arg); although WW and PW ADG were greater (P = 0.02), respectively. A 3.6% decrease (P = 0.05) in peak lean accretion ADG occurred when dams received 1% L-Arg prior to day 45 of gestation (Early-Arg and Full-Arg), however, no other significant differences were detected in finishing growth parameters or carcass characteristics (P ≥ 0.1). Pig mortality rates tended (P = 0.07) to decrease in progeny of dams supplemented Arg after day 85 (3.6%) compared to dams not provided additional Arg during late gestation (4.9%). Collectively, these data suggest that Arg provided during late gestation may improve WW and PW ADG, however, finishing performance was not affected. While Arg supplementation provided some moderate production benefits, further investigation is warranted to comprehensively understand the gestational timing and biological role of Arg supplementation during fetal and postnatal development in commercial production systems.

The impact of dietary supplementation of arginine during gestation in a commercial swine herd: I. Gilt reproductive performance.

Supplemental arginine (Arg) during gestation purportedly benefits fetal development. However, the benefits of a gestational Arg dietary strategy in commercial production are unclear. Therefore, the objectives of this study examined Arg supplementation during different gestational stages and the effects on gilt reproductive performance. Pubertal gilts (n = 548) were allocated into 4 treatment groups: Control (n = 143; 0% supplemental Arg) or 1 of 3 supplemental Arg (1% as fed) treatments: from 15 to 45 d of gestation (n = 138; Early-Arg); from 15 d of gestation until farrowing (n = 139; Full-Arg); or from 85 d of gestation until farrowing (n = 128; Late-Arg). At farrowing, the number of total born (TB), born alive (BA), stillborn piglets (SB), mummified fetuses (MM), and individual piglet birth weights (BiWt) were recorded. The wean-to-estrus interval (WEI) and subsequent sow reproductive performance (to third parity) were also monitored. No significant effect of supplemental Arg during any part of P0 gestation was observed for TB, BA, SB, or MM (P ≥ 0.29). Offspring BiWt and variation among individual piglet birth weights did not differ (P = 0.42 and 0.89, respectively) among treatment groups. Following weaning, the WEI was similar among treatments (average of 8.0 ± 0.8 d; P = 0.88). Litter performance over 3 parities revealed a decrease (P = 0.02) in BA for Early-Arg fed gilts compared with all other treatments, whereas TB and WEI were similar among treatments over 3 parities (P > 0.05). There was an increased proportion of sows with average size litters (12 to 16 TB) from the Full-Arg treatment sows (76.8% ± 3.7%) when compared with Control (58.7% ± 4.2%; P = 0.01); however, the proportion of sows with high (>16 TB) and low (<12 TB) litters was not different among treatments (P = 0.20). These results suggest that gestational Arg supplementation had a minimal impact on reproductive performance in first parity sows. These data underscore the complexity of AA supplementation and the need for continued research into understanding how and when utilizing a gestational dietary Arg strategy can optimize fetal development and sow performance.

Fully Bayesian analysis of RNA-seq counts for the detection of gene expression heterosis.

Heterosis, or hybrid vigor, is the enhancement of the phenotype of hybrid progeny relative to their inbred parents. Heterosis is extensively used in agriculture, and the underlying mechanisms are unclear. To investigate the molecular basis of phenotypic heterosis, researchers search tens of thousands of genes for heterosis with respect to expression in the transcriptome. Difficulty arises in the assessment of heterosis due to composite null hypotheses and non-uniform distributions for p-values under these null hypotheses. Thus, we develop a general hierarchical model for count data and a fully Bayesian analysis in which an efficient parallelized Markov chain Monte Carlo algorithm ameliorates the computational burden. We use our method to detect gene expression heterosis in a two-hybrid plant-breeding scenario, both in a real RNA-seq maize dataset and in simulation studies. In the simulation studies, we show our method has well-calibrated posterior probabilities and credible intervals when the model assumed in analysis matches the model used to simulate the data. Although model misspecification can adversely affect calibration, the methodology is still able to accurately rank genes. Finally, we show that hyperparameter posteriors are extremely narrow and an empirical Bayes (eBayes) approach based on posterior means from the fully Bayesian analysis provides virtually equivalent posterior probabilities, credible intervals, and gene rankings relative to the fully Bayesian solution. This evidence of equivalence provides support for the use of eBayes procedures in RNA-seq data analysis if accurate hyperparameter estimates can be obtained.

Multivariate temporal modeling of crime with dynamic linear models.

Interest in modeling contemporary crime trends, a task that has historically been considered valuable to the public, researchers, and policymakers, is resurging. Advancements in criminology have made it clear that understanding crime trends necessarily involves understanding trends in how likely individuals are to report crimes to the police, as well as how likely the police are to accurately record those crimes. In this paper, we use dynamic linear models to simultaneously model the time series for several crime types in order to gain insight into trends in crime and crime reporting. We analyze crime data from Chicago spanning 2007 through 2016 and show how correlations in the way crime trends evolve may contain information about drivers of crime and crime reporting. We provide evidence of substantial differences in the relationships between the trends of crimes of different types depending on whether crimes are violent or nonviolent and whether or not crimes are tracked in the FBI's Uniform Crime Report.

Collaborative efforts to forecast seasonal influenza in the United States, 2015-2016.

Since 2013, the Centers for Disease Control and Prevention (CDC) has hosted an annual influenza season forecasting challenge. The 2015-2016 challenge consisted of weekly probabilistic forecasts of multiple targets, including fourteen models submitted by eleven teams. Forecast skill was evaluated using a modified logarithmic score. We averaged submitted forecasts into a mean ensemble model and compared them against predictions based on historical trends. Forecast skill was highest for seasonal peak intensity and short-term forecasts, while forecast skill for timing of season onset and peak week was generally low. Higher forecast skill was associated with team participation in previous influenza forecasting challenges and utilization of ensemble forecasting techniques. The mean ensemble consistently performed well and outperformed historical trend predictions. CDC and contributing teams will continue to advance influenza forecasting and work to improve the accuracy and reliability of forecasts to facilitate increased incorporation into public health response efforts.

Prairie strips improve biodiversity and the delivery of multiple ecosystem services from corn-soybean croplands.

Loss of biodiversity and degradation of ecosystem services from agricultural lands remain important challenges in the United States despite decades of spending on natural resource management. To date, conservation investment has emphasized engineering practices or vegetative strategies centered on monocultural plantings of nonnative plants, largely excluding native species from cropland. In a catchment-scale experiment, we quantified the multiple effects of integrating strips of native prairie species amid corn and soybean crops, with prairie strips arranged to arrest run-off on slopes. Replacing 10% of cropland with prairie strips increased biodiversity and ecosystem services with minimal impacts on crop production. Compared with catchments containing only crops, integrating prairie strips into cropland led to greater catchment-level insect taxa richness (2.6-fold), pollinator abundance (3.5-fold), native bird species richness (2.1-fold), and abundance of bird species of greatest conservation need (2.1-fold). Use of prairie strips also reduced total water runoff from catchments by 37%, resulting in retention of 20 times more soil and 4.3 times more phosphorus. Corn and soybean yields for catchments with prairie strips decreased only by the amount of the area taken out of crop production. Social survey results indicated demand among both farming and nonfarming populations for the environmental outcomes produced by prairie strips. If federal and state policies were aligned to promote prairie strips, the practice would be applicable to 3.9 million ha of cropland in Iowa alone.

Empirical Bayes analysis of RNA-seq data for detection of gene expression heterosis.

An important type of heterosis, known as hybrid vigor, refers to the enhancements in the phenotype of hybrid progeny relative to their inbred parents. Although hybrid vigor is extensively utilized in agriculture, its molecular basis is still largely unknown. In an effort to understand phenotypic heterosis at the molecular level, researchers are measuring transcript abundance levels of thousands of genes in parental inbred lines and their hybrid offspring using RNA sequencing (RNA-seq) technology. The resulting data allow researchers to search for evidence of gene expression heterosis as one potential molecular mechanism underlying heterosis of agriculturally important traits. The null hypotheses of greatest interest in testing for gene expression heterosis are composite null hypotheses that are difficult to test with standard statistical approaches for RNA-seq analysis. To address these shortcomings, we develop a hierarchical negative binomial model and draw inferences using a computationally tractable empirical Bayes approach to inference. We demonstrate improvements over alternative methods via a simulation study based on a maize experiment and then analyze that maize experiment with our newly proposed methodology. This article has supplementary material online.

Student interpretations of phylogenetic trees in an introductory biology course.

Phylogenetic trees are widely used visual representations in the biological sciences and the most important visual representations in evolutionary biology. Therefore, phylogenetic trees have also become an important component of biology education. We sought to characterize reasoning used by introductory biology students in interpreting taxa relatedness on phylogenetic trees, to measure the prevalence of correct taxa-relatedness interpretations, and to determine how student reasoning and correctness change in response to instruction and over time. Counting synapomorphies and nodes between taxa were the most common forms of incorrect reasoning, which presents a pedagogical dilemma concerning labeled synapomorphies on phylogenetic trees. Students also independently generated an alternative form of correct reasoning using monophyletic groups, the use of which decreased in popularity over time. Approximately half of all students were able to correctly interpret taxa relatedness on phylogenetic trees, and many memorized correct reasoning without understanding its application. Broad initial instruction that allowed students to generate inferences on their own contributed very little to phylogenetic tree understanding, while targeted instruction on evolutionary relationships improved understanding to some extent. Phylogenetic trees, which can directly affect student understanding of evolution, appear to offer introductory biology instructors a formidable pedagogical challenge.

Comparison of the performance of particle filter algorithms applied to tracking of a disease epidemic.

We present general methodology for sequential inference in nonlinear stochastic state-space models to simultaneously estimate dynamic states and fixed parameters. We show that basic particle filters may fail due to degeneracy in fixed parameter estimation and suggest the use of a kernel density approximation to the filtered distribution of the fixed parameters to allow the fixed parameters to regenerate. In addition, we show that "seemingly" uninformative uniform priors on fixed parameters can affect posterior inferences and suggest the use of priors bounded only by the support of the parameter. We show the negative impact of using multinomial resampling and suggest the use of either stratified or residual resampling within the particle filter. As a motivating example, we use a model for tracking and prediction of a disease outbreak via a syndromic surveillance system. Finally, we use this improved particle filtering methodology to relax prior assumptions on model parameters yet still provide reasonable estimates for model parameters and disease states.

Accelerated maximum likelihood parameter estimation for stochastic biochemical systems.

BACKGROUND: A prerequisite for the mechanistic simulation of a biochemical system is detailed knowledge of its kinetic parameters. Despite recent experimental advances, the estimation of unknown parameter values from observed data is still a bottleneck for obtaining accurate simulation results. Many methods exist for parameter estimation in deterministic biochemical systems; methods for discrete stochastic systems are less well developed. Given the probabilistic nature of stochastic biochemical models, a natural approach is to choose parameter values that maximize the probability of the observed data with respect to the unknown parameters, a.k.a. the maximum likelihood parameter estimates (MLEs). MLE computation for all but the simplest models requires the simulation of many system trajectories that are consistent with experimental data. For models with unknown parameters, this presents a computational challenge, as the generation of consistent trajectories can be an extremely rare occurrence. RESULTS: We have developed Monte Carlo Expectation-Maximization with Modified Cross-Entropy Method (MCEM(2)): an accelerated method for calculating MLEs that combines advances in rare event simulation with a computationally efficient version of the Monte Carlo expectation-maximization (MCEM) algorithm. Our method requires no prior knowledge regarding parameter values, and it automatically provides a multivariate parameter uncertainty estimate. We applied the method to five stochastic systems of increasing complexity, progressing from an analytically tractable pure-birth model to a computationally demanding model of yeast-polarization. Our results demonstrate that MCEM(2) substantially accelerates MLE computation on all tested models when compared to a stand-alone version of MCEM. Additionally, we show how our method identifies parameter values for certain classes of models more accurately than two recently proposed computationally efficient methods. CONCLUSIONS: This work provides a novel, accelerated version of a likelihood-based parameter estimation method that can be readily applied to stochastic biochemical systems. In addition, our results suggest opportunities for added efficiency improvements that will further enhance our ability to mechanistically simulate biological processes.

Adaptive Mixture Modelling Metropolis Methods for Bayesian Analysis of Non-linear State-Space Models.

We describe a strategy for Markov chain Monte Carlo analysis of non-linear, non-Gaussian state-space models involving batch analysis for inference on dynamic, latent state variables and fixed model parameters. The key innovation is a Metropolis-Hastings method for the time series of state variables based on sequential approximation of filtering and smoothing densities using normal mixtures. These mixtures are propagated through the non-linearities using an accurate, local mixture approximation method, and we use a regenerating procedure to deal with potential degeneracy of mixture components. This provides accurate, direct approximations to sequential filtering and retrospective smoothing distributions, and hence a useful construction of global Metropolis proposal distributions for simulation of posteriors for the set of states. This analysis is embedded within a Gibbs sampler to include uncertain fixed parameters. We give an example motivated by an application in systems biology. Supplemental materials provide an example based on a stochastic volatility model as well as MATLAB code.

Image segmentation and dynamic lineage analysis in single-cell fluorescence microscopy.

An increasingly common component of studies in synthetic and systems biology is analysis of dynamics of gene expression at the single-cell level, a context that is heavily dependent on the use of time-lapse movies. Extracting quantitative data on the single-cell temporal dynamics from such movies remains a major challenge. Here, we describe novel methods for automating key steps in the analysis of single-cell, fluorescent images-segmentation and lineage reconstruction-to recognize and track individual cells over time. The automated analysis iteratively combines a set of extended morphological methods for segmentation, and uses a neighborhood-based scoring method for frame-to-frame lineage linking. Our studies with bacteria, budding yeast and human cells, demonstrate the portability and usability of these methods, whether using phase, bright field or fluorescent images. These examples also demonstrate the utility of our integrated approach in facilitating analyses of engineered and natural cellular networks in diverse settings. The automated methods are implemented in freely available, open-source software.

Cytotype regulation by telomeric P elements in Drosophila melanogaster: interactions with P elements from M' strains.

P strains of Drosophila are distinguished from M strains by having P elements in their genomes and also by having the P cytotype, a maternally inherited condition that strongly represses P-element-induced hybrid dysgenesis. The P cytotype is associated with P elements inserted near the left telomere of the X chromosome. Repression by the telomeric P elements TP5 and TP6 is significantly enhanced when these elements are crossed into M' strains, which, like P strains, carry P elements, but have little or no ability to repress dysgenesis. The telomeric and M' P elements must coexist in females for this enhanced repression ability to develop. However, once established, it is transmitted maternally to the immediate offspring independently of the telomeric P elements themselves. Females that carry a telomeric P element but that do not carry M' P elements may also transmit an ability to repress dysgenesis to their offspring independently of the telomeric P element. Cytotype regulation therefore involves a maternally transmissible product of telomeric P elements that can interact synergistically with products from paternally inherited M' P elements. This synergism between TP and M' P elements also appears to persist for at least one generation after the TP has been removed from the genotype.