Mortality Attributable to Long-Term Exposure to Ambient Fine Particulate Matter: Insights from the Epidemiologic Evidence for Understudied Locations.

Epidemiologic cohort studies have consistently demonstrated that long-term exposure to ambient fine particles (PM2.5) is associated with mortality. Nevertheless, extrapolating results to understudied locations may involve considerable uncertainty. To explore this issue, this review discusses the evidence for (i) the associated risk of mortality, (ii) the shape of the concentration-response function, (iii) a causal interpretation, and (iv) how the source mix/composition of PM2.5 and population characteristics may alter the effect. The accumulated evidence suggests the following: (i) In the United States, the change in all-cause mortality risk per µg/m3 is about 0.8%. (ii) The concentration-response function appears nonlinear. (iii) Causation is overwhelmingly supported. (iv) Fossil fuel combustion-related sources are likely more toxic than others, and age, race, and income may modify the effect. To illustrate the use of our findings in support of a risk assessment in an understudied setting, we consider Kuwait. However, given the complexity of this relationship and the heterogeneity in reported effects, it is unreasonable to think that, in such circumstances, point estimates can be meaningful. Consequently, quantitative probabilistic estimates, which cannot be derived objectively, become essential. Formally elicited expert judgment can provide such estimates, and this review provides the evidence to support an elicitation.

Ice sheet contributions to future sea-level rise from structured expert judgment.

Despite considerable advances in process understanding, numerical modeling, and the observational record of ice sheet contributions to global mean sea-level rise (SLR) since the Fifth Assessment Report (AR5) of the Intergovernmental Panel on Climate Change, severe limitations remain in the predictive capability of ice sheet models. As a consequence, the potential contributions of ice sheets remain the largest source of uncertainty in projecting future SLR. Here, we report the findings of a structured expert judgement study, using unique techniques for modeling correlations between inter- and intra-ice sheet processes and their tail dependences. We find that since the AR5, expert uncertainty has grown, in particular because of uncertain ice dynamic effects. For a +2 °C temperature scenario consistent with the Paris Agreement, we obtain a median estimate of a 26 cm SLR contribution by 2100, with a 95th percentile value of 81 cm. For a +5 °C temperature scenario more consistent with unchecked emissions growth, the corresponding values are 51 and 178 cm, respectively. Inclusion of thermal expansion and glacier contributions results in a global total SLR estimate that exceeds 2 m at the 95th percentile. Our findings support the use of scenarios of 21st century global total SLR exceeding 2 m for planning purposes. Beyond 2100, uncertainty and projected SLR increase rapidly. The 95th percentile ice sheet contribution by 2200, for the +5 °C scenario, is 7.5 m as a result of instabilities coming into play in both West and East Antarctica. Introducing process correlations and tail dependences increases estimates by roughly 15%.

Vine Regression with Bayes Nets: A Critical Comparison with Traditional Approaches Based on a Case Study on the Effects of Breastfeeding on IQ.

Regular vines (R-vines) copulas build high dimensional joint densities from arbitrary one-dimensional margins and (conditional) bivariate copula densities. Vine densities enable the computation of all conditional distributions, though the calculations can be numerically intensive. Saturated continuous nonparametric Bayes nets (CNPBN) are regular vines. Computing regression functions from the vine copula density is termed vine regression. The epicycles of regression-including/excluding covariates, interactions, higher order terms, multicollinearity, model fit, transformations, heteroscedasticity, bias-are dispelled. One simply computes the regressions from the vine copula density. Only the question of finding an adequate vine copula remains. Vine regression is applied to a data set from the National Longitudinal Study of Youth relating breastfeeding to IQ. The expected effects of breastfeeding on IQ depend on IQ, on the baseline level of breastfeeding, on the duration of additional breastfeeding and on the values of other covariates. A child given two weeks breastfeeding can expect to increase his/her IQ by 1.5-2 IQ points by adding 10 weeks of breastfeeding, depending on values of other covariates. A child given two years breastfeeding can expect to gain from 0.48-0.65 IQ points from 10 additional weeks. Adding 10 weeks breastfeeding to each of the 3,179 children in this data set has a net present value $50,700,000 according to the Bayes net, compared to $29,000,000 according to the linear regression.

Attribution of Illnesses Transmitted by Food and Water to Comprehensive Transmission Pathways Using Structured Expert Judgment, United States.

Illnesses transmitted by food and water cause a major disease burden in the United States despite advancements in food safety, water treatment, and sanitation. We report estimates from a structured expert judgment study using 48 experts who applied Cooke's classical model of the proportion of disease attributable to 5 major transmission pathways (foodborne, waterborne, person-to-person, animal contact, and environmental) and 6 subpathways (food handler-related, under foodborne; recreational, drinking, and nonrecreational/nondrinking, under waterborne; and presumed person-to-person-associated and presumed animal contact-associated, under environmental). Estimates for 33 pathogens were elicited, including bacteria such as Salmonella enterica, Campylobacter spp., Legionella spp., and Pseudomonas spp.; protozoa such as Acanthamoeba spp., Cyclospora cayetanensis, and Naegleria fowleri; and viruses such as norovirus, rotavirus, and hepatitis A virus. The results highlight the importance of multiple pathways in the transmission of the included pathogens and can be used to guide prioritization of public health interventions.

Rating impacts in a multi-stressor world: a quantitative assessment of 50 stressors affecting the Great Lakes.

Ecosystems often experience multiple environmental stressors simultaneously that can differ widely in their pathways and strengths of impact. Differences in the relative impact of environmental stressors can guide restoration and management prioritization, but few studies have empirically assessed a comprehensive suite of stressors acting on a given ecosystem. To fill this gap in the Laurentian Great Lakes, where considerable restoration investments are currently underway, we used expert elicitation via a detailed online survey to develop ratings of the relative impacts of 50 potential stressors. Highlighting the multiplicity of stressors in this system, experts assessed all 50 stressors as having some impact on ecosystem condition, but ratings differed greatly among stressors. Individual stressors related to invasive and nuisance species (e.g., dreissenid mussels and ballast invasion risk) and climate change were assessed as having the greatest potential impacts. These results mark a shift away from the longstanding emphasis on nonpoint phosphorus and persistent bioaccumulative toxic substances in the Great Lakes. Differences in impact ratings among lakes and ecosystem zones were weak, and experts exhibited surprisingly high levels of agreement on the relative impacts of most stressors. Our results provide a basin-wide, quantitative summary of expert opinion on the present-day influence of all major Great Lakes stressors. The resulting ratings can facilitate prioritizing stressors to achieve management objectives in a given location, as well as providing a baseline for future stressor impact assessments in the Great Lakes and elsewhere.

Use of structured expert judgment to forecast invasions by bighead and silver carp in Lake Erie.

Identifying which nonindigenous species will become invasive and forecasting the damage they will cause is difficult and presents a significant problem for natural resource management. Often, the data or resources necessary for ecological risk assessment are incomplete or absent, leaving environmental decision makers ill equipped to effectively manage valuable natural resources. Structured expert judgment (SEJ) is a mathematical and performance-based method of eliciting, weighting, and aggregating expert judgments. In contrast to other methods of eliciting and aggregating expert judgments (where, for example, equal weights may be assigned to experts), SEJ weights each expert on the basis of his or her statistical accuracy and informativeness through performance measurement on a set of calibration variables. We used SEJ to forecast impacts of nonindigenous Asian carp (Hypophthalmichthys spp.) in Lake Erie, where it is believed not to be established. Experts quantified Asian carp biomass, production, and consumption and their impact on 4 fish species if Asian carp were to become established. According to experts, in Lake Erie Asian carp have the potential to achieve biomass levels that are similar to the sum of biomasses for several fishes that are harvested commercially or recreationally. However, the impact of Asian carp on the biomass of these fishes was estimated by experts to be small, relative to long term average biomasses, with little uncertainty. Impacts of Asian carp in tributaries and on recreational activities, water quality, or other species were not addressed. SEJ can be used to quantify key uncertainties of invasion biology and also provide a decision-support tool when the necessary information for natural resource management and policy is not available.

The aggregation of expert judgment: do good things come to those who weight?

This responds to an "evaluation" of the classical model for structured expert judgment by Bolger and Rowe in this issue. This response references extensive expert judgment performance data in the public domain which played no role in their evaluation.

Using structured expert judgment to assess invasive species prevention: Asian carp and the Mississippi-Great Lakes hydrologic connection.

Recently, authors have theorized that invasive species prevention is more cost-effective than control in protecting ecosystem services. However, quantification of the effectiveness of prevention is rare because experiments at field scales are expensive or infeasible. We therefore used structured expert judgment to quantify the efficacy of 17 proposed strategies to prevent Asian carp invasion of the Laurentian Great Lakes via the hydrologic connection between the Mississippi and Great Lakes watersheds. Performance-weighted expert estimates indicated that hydrologic separation would prevent 99% (95,100; median, 5th and 95th percentiles) of Asian carp access, while electric and acoustic-bubble-strobe barriers would prevent 92% (85,95) and 92% (75,95), respectively. For all other strategies, estimated effectiveness was lower, with greater uncertainty. When potential invasions by other taxa are considered, the effectiveness of hydrologic separation increases relative to strategies that are effective primarily for fishes. These results could help guide invasive species management in many waterways globally.

Ice Sheet and Climate Processes Driving the Uncertainty in Projections of Future Sea Level Rise: Findings From a Structured Expert Judgement Approach.

The ice sheets covering Antarctica and Greenland present the greatest uncertainty in, and largest potential contribution to, future sea level rise. The uncertainty arises from a paucity of suitable observations covering the full range of ice sheet behaviors, incomplete understanding of the influences of diverse processes, and limitations in defining key boundary conditions for the numerical models. To investigate the impact of these uncertainties on ice sheet projections we undertook a structured expert judgement study. Here, we interrogate the findings of that study to identify the dominant drivers of uncertainty in projections and their relative importance as a function of ice sheet and time. We find that for the 21st century, Greenland surface melting, in particular the role of surface albedo effects, and West Antarctic ice dynamics, specifically the role of ice shelf buttressing, dominate the uncertainty. The importance of these effects holds under both a high-end 5°C global warming scenario and another that limits global warming to 2°C. During the 22nd century the dominant drivers of uncertainty shift. Under the 5°C scenario, East Antarctic ice dynamics dominate the uncertainty in projections, driven by the possible role of ice flow instabilities. These dynamic effects only become dominant, however, for a temperature scenario above the Paris Agreement 2°C target and beyond 2100. Our findings identify key processes and factors that need to be addressed in future modeling and observational studies in order to reduce uncertainties in ice sheet projections.

A retrospective assessment of COVID-19 model performance in the USA.

Coronavirus disease 2019 (COVID-19) forecasts from over 100 models are readily available. However, little published information exists regarding the performance of their uncertainty estimates (i.e. probabilistic performance). To evaluate their probabilistic performance, we employ the classical model (CM), an established method typically used to validate expert opinion. In this analysis, we assess both the predictive and probabilistic performance of COVID-19 forecasting models during 2021. We also compare the performance of aggregated forecasts (i.e. ensembles) based on equal and CM performance-based weights to an established ensemble from the Centers for Disease Control and Prevention (CDC). Our analysis of forecasts of COVID-19 mortality from 22 individual models and three ensembles across 49 states indicates that-(i) good predictive performance does not imply good probabilistic performance, and vice versa; (ii) models often provide tight but inaccurate uncertainty estimates; (iii) most models perform worse than a naive baseline model; (iv) both the CDC and CM performance-weighted ensembles perform well; but (v) while the CDC ensemble was more informative, the CM ensemble was more statistically accurate across states. This study presents a worthwhile method for appropriately assessing the performance of probabilistic forecasts and can potentially improve both public health decision-making and COVID-19 modelling.

Quantifying uncertainty about future antimicrobial resistance: Comparing structured expert judgment and statistical forecasting methods.

The increase of multidrug resistance and resistance to last-line antibiotics is a major global public health threat. Although surveillance programs provide useful current and historical information on the scale of the problem, the future emergence and spread of antibiotic resistance is uncertain, and quantifying this uncertainty is crucial for guiding decisions about investment in antibiotics and resistance control strategies. Mathematical and statistical models capable of projecting future rates are challenged by the paucity of data and the complexity of the emergence and spread of resistance, but experts have relevant knowledge. We use the Classical Model of structured expert judgment to elicit projections with uncertainty bounds of resistance rates through 2026 for nine pathogen-antibiotic pairs in four European countries and empirically validate the assessments against data on a set of calibration questions. The performance-weighted combination of experts in France, Spain, and the United Kingdom projected that resistance for five pairs on the World Health Organization's priority pathogens list (E. coli and K. pneumoniae resistant to third-generation cephalosporins and carbapenems and MRSA) would remain below 50% in 2026. In Italy, although upper bounds of 90% credible ranges exceed 50% resistance for some pairs, the medians suggest Italy will sustain or improve its current rates. We compare these expert projections to statistical forecasts based on historical data from the European Antimicrobial Resistance Surveillance Network (EARS-Net). Results from the statistical models differ from each other and from the judgmental forecasts in many cases. The judgmental forecasts include information from the experts about the impact of current and future shifts in infection control, antibiotic usage, and other factors that cannot be easily captured in statistical forecasts, demonstrating the potential of structured expert judgment as a tool for better understanding the uncertainty about future antibiotic resistance.

Uncertainty and sensitivity analyses of a dynamic economic evaluation model for vaccination programs.

UNLABELLED: With public health policy increasingly relying on mathematical models to provide insights about the impacts of potential policy options, the demand for uncertainty and sensitivity analyses that explore the implications of different assumptions in a model continues to expand. Although analysts continue to develop methods to meet the demand, most modelers rely on a single method in the context of their assessments and presentations of results, and few analysts provide results that facilitate comparisons between uncertainty and sensitivity analysis methods. METHODS: vary in their degree of analytical difficulty and in the nature of the information that they provide, and analysts should communicate results with a note that not all methods yield the same insights. The authors explore several sensitivity analysis methods to test whether the choice of method affects the insights and importance rankings of inputs from the analysis. They use a dynamic cost-effectiveness model of a hypothetical infectious disease as the basis to perform 1-way and multi-way sensitivity analyses, design of experiments, and Morris' method. They also compute partial derivatives as well as a number of probabilistic sensitivity measures, including correlations, regression coefficients, and the correlation ratio, to demonstrate the existing methods and to compare them. The authors find that the magnitudes and rankings of sensitivity measures depend on the selected method(s) and make recommendations regarding the choice of method depending on the complexity of the model, number of uncertain inputs, and desired types of insights from the sensitivity analysis.

Regulating under uncertainty: newsboy for exposure limits.

Setting action levels or limits for health protection is complicated by uncertainty in the dose-response relation across a range of hazards and exposures. To address this issue, we consider the classic newsboy problem. The principles used to manage uncertainty for that case are applied to two stylized exposure examples, one for high dose and high dose rate radiation and the other for ammonia. Both incorporate expert judgment on uncertainty quantification in the dose-response relationship. The mathematical technique of probabilistic inversion also plays a key role. We propose a coupled approach, whereby scientists quantify the dose-response uncertainty using techniques such as structured expert judgment with performance weights and probabilistic inversion, and stakeholders quantify associated loss rates.

Attribution of foodborne pathogens using structured expert elicitation.

OBJECTIVES: To estimate the fraction of human cases of enterically transmitted illness by five major pathways (food, environment, direct animal contact, human-human transmission, and travel) and by 11 groups within the food pathway. METHODS: Food safety experts were asked to provide their estimates of the most likely range for each of the parameters. Joint probability distributions were created by probabilistic inversion (PI). RESULTS: Sixteen experts participated in the study. PI resulted in good fits for most pathogens. Qualitatively, expert estimates were similar to earlier published studies but the estimated fraction of foodborne transmission was lower for most pathogens. Biologically less plausible pathways were given some weight by the experts. Uncertainties were smallest for pathogens with dominant transmission routes. CONCLUSIONS: Structured expert studies are a feasible method for source attribution, but methods need further development. APPLICATIONS: These estimates can be combined with data on incidence, disease burden and costs to provide specific estimates of the public health impact of foodborne illness, and to identify the food groups that have the highest impact.

Probabilistic reasoning about measurements of equilibrium climate sensitivity: combining disparate lines of evidence.

Where policy and science intersect, there are always issues of ambiguous and conflicting lines of evidence. Combining disparate information sources is mathematically complex; common heuristics based on simple statistical models easily lead us astray. Here, we use Bayesian Nets (BNs) to illustrate the complexity in reasoning under uncertainty. Data from joint research at Resources for the Future and NASA Langley are used to populate a BN for predicting equilibrium climate sensitivity (ECS). The information sources consist of measuring the rate of decadal temperature rise (DTR) and measuring the rate of percentage change in cloud radiative forcing (CRF), with both the existing configuration of satellites and with a proposed enhanced measuring system. The goal of all measurements is to reduce uncertainty in equilibrium climate sensitivity. Subtle aspects of probabilistic reasoning with concordant and discordant measurements are illustrated. Relative to the current prior distribution on ECS, we show that after 30 years of observing with the current systems, the 2σ uncertainty band for ECS would be shrunk on average to 73% of its current value. With the enhanced systems over the same time, it would be shrunk to 32% of its current value. The actual shrinkage depends on the values actually observed. These results are based on models recommended by the Social Cost of Carbon methodology and assume a Business as Usual emissions path.

Out-of-sample validation for structured expert judgment of Asian carp establishment in Lake Erie.

Structured expert judgment (SEJ) is used to quantify the uncertainty of nonindigenous fish (bighead carp [Hypophthalmichthys nobilis] and silver carp [H. molitrix]) establishment in Lake Erie. The classical model for structured expert judgment model is applied. Forming a weighted combination (called a decision maker) of experts' distributions, with weights derived from performance on a set of calibration variables from the experts' field, exhibits greater statistical accuracy and greater informativeness than simple averaging with equal weights. New methods of cross validation are applied and suggest that performance characteristics relative to equal weighting could be predicted with a small number (1-2) of calibration variables. The performance-based decision maker is somewhat degraded on out-of-sample prediction, but remained superior to the equal weight decision maker in terms of statistical accuracy and informativeness.

Model uncertainty in economic impacts of climate change: Bernoulli versus Lotka Volterra dynamics.

The dynamic economic behavior in most integrated assessment models linking economic growth to climate change involves a differential equation solved by Jacob Bernoulli in 1695. Using the dynamic integrated climate economy (DICE) model and freezing exogenous variables at initial values, this dynamic is shown to produce implausible projections on a 60-year time frame. If world capital started at US$1, after 60 years the world economy would be indistinguishable from one starting with 10 times the current capitalization. Such behavior points to uncertainty at the level of the fundamental dynamics, and suggests that discussions of discounting, utility, damage functions, and ethics should be conducted within a more general modeling vocabulary. Lotka Volterra dynamics is proposed as an alternative with greater prime facie plausibility. With near universality, economists assume that economic growth will go on forever. Lotka Volterra dynamics alert us to the possibility of collapse.

Prioritizing emerging zoonoses in the Netherlands.

BACKGROUND: To support the development of early warning and surveillance systems of emerging zoonoses, we present a general method to prioritize pathogens using a quantitative, stochastic multi-criteria model, parameterized for the Netherlands. METHODOLOGY/PRINCIPAL FINDINGS: A risk score was based on seven criteria, reflecting assessments of the epidemiology and impact of these pathogens on society. Criteria were weighed, based on the preferences of a panel of judges with a background in infectious disease control. CONCLUSIONS/SIGNIFICANCE: Pathogens with the highest risk for the Netherlands included pathogens in the livestock reservoir with a high actual human disease burden (e.g. Campylobacter spp., Toxoplasma gondii, Coxiella burnetii) or a low current but higher historic burden (e.g. Mycobacterium bovis), rare zoonotic pathogens in domestic animals with severe disease manifestations in humans (e.g. BSE prion, Capnocytophaga canimorsus) as well as arthropod-borne and wildlife associated pathogens which may pose a severe risk in future (e.g. Japanese encephalitis virus and West-Nile virus). These agents are key targets for development of early warning and surveillance.

A probabilistic characterization of the relationship between fine particulate matter and mortality: elicitation of European experts.

In support of an assessment of the mortality impacts of the Kuwait Oil Fires we interviewed six European experts in epidemiology and toxicology using formal procedures for elicitation of expert judgment. While the primary focus of the elicitations was to characterize the public health impacts of the fires, the experts provided quantitative estimates of the mortality impacts of hypothetical changes in the levels of ambient fine particulate matter (PM2.5) in both the United States and Europe. Uncertainty was assessed by asking each expert to provide the 5th, 25th, 50th, 75th, and 90th percentiles of their subjective cumulative probability density function for each quantity of interest. The results suggest that many regulatory risk assessments underestimate the impacts of PM2.5 mortality; confirm that only a small fraction of the mortality impact occurs within the first few months after exposure; and indicate that it may be important to better address the differential toxicities of particles from various source classes. By providing quantitative estimates of the uncertainty in current estimates of PM2.5 mortality risks, the study facilitates structured analysis of the value of further research on PM2.5 and its impacts.

A structured expert judgment study for a model of Campylobacter transmission during broiler-chicken processing.

A structured expert judgment study was organized to obtain input data for a microbial risk-assessment model describing the transmission of campylobacter during broiler-chicken processing in the Netherlands. More specially, the expert study was aimed at quantifying the uncertainty on input parameters of this model and focused on the contamination of broiler-chicken carcasses with campylobacter during processing. Following the protocol for structured expert judgment studies, expert assessments were elicited individually through subjective probability distribution functions. The classical model was used to aggregate the individual experts' distributions in order to obtain a single combined distribution per variable. Three different weighting schemes were applied, including equal weighting and performance-based weighting with and without optimalization of the combined distributions. The individual experts' weights were based on their performance on the seed variables. Results of the various weighting schemes are presented in terms of performance, robustness, and combined distributions of the seed variables and some of the query variables. All three weighting schemes had adequate performance, with the optimized combined distributions significantly outperforming both the equal weight and the nonoptimized combined distributions. Hence, this weighting scheme, having adequate robustness, was chosen for further processing of the results.