Adequacy of sample size for estimating a value from field observational data.

In 2011, the U.S. EPA Office of Research and Development released a field-based method for deriving aquatic life benchmarks for conductivity. Since its release, it has been verified, validated, and corroborated by the authors, reviewers, and independent researchers. However, the method and published results have been recently challenged as being artifacts of small sample sizes, prompting this re-evaluation. This paper supplements prior causal analyses by weighing evidence that specifically addresses the hypothesis that the benchmark is a statistical artifact. Four types of evidence are presented: (1) Permutation analyses show that the data sets are able to reliably estimate the extirpation of 5% of genera. (2) Analyses show that 25 occurrences of a genus are sufficient to estimate extirpation. (3) Coherent ecological explanations show that the claimed influence of sample size is actually a result of community ecology. (4) A review of relevant independent studies supports the benchmark. The permutation test is a useful test of the adequacy of field data sets. Furthermore, this weight-of-evidence approach and the individual types of evidence can be a model for analysis of other field-based benchmark values.

Modeling Spatial and Temporal Variation in Natural Background Specific Conductivity.

Understanding how background levels of dissolved minerals vary in streams temporally and spatially is needed to assess salinization of fresh water, establish reasonable thresholds and restoration goals, and determine vulnerability to extreme climate events like drought. We developed a random forest model that predicts natural background specific conductivity (SC), a measure of total dissolved ions, for all stream segments in the contiguous United States at monthly time steps between the years 2001 to 2015. Models were trained using 11 796 observations made at 1785 minimally impaired stream segments and validated with observations from an additional 92 segments. Static predictors of SC included geology, soils, and vegetation parameters. Temporal predictors were related to climate and enabled the model to make predictions for different dates. The model explained 95% of the variation in SC among validation observations (mean absolute error = 29 µS/cm, Nash-Sutcliffe efficiency = 0.85). The model performed well across the period of interest but exhibited bias in Coastal Plain and Xeric regions (26 and 30%, respectively). National model predictions showed large spatial variation with the greatest SC predicted to occur in the desert southwest and plains. Model predictions also reflected changes at individual streams during drought.

Using our brains to develop better policy.

Current governmental practices often use a method called weight of evidence (WoE) to integrate and weigh different sources of information in the process of reaching a decision. Recent advances in cognitive neuroscience have identified WoE-like processes in the brain, and we believe that these advances have the potential to improve current decision-making practices. In this article, we describe five specific areas where knowledge emerging from cognitive neuroscience may be applied to the challenges confronting decisionmakers who manage risks: (1) quantifying evidence, (2) comparing the value of different sources of evidence, (3) reaching a decision, (4) illuminating the role of subjectivity, and (5) adapting to new information. We believe that the brain is an appropriate model for structuring decision-making processes because the brain's network is designed for complex, flexible decision making, and because policy decisions that must ultimately depend on human judgment will be best served by methods that complement human abilities. Future discoveries in cognitive neuroscience will likely bring further applications to decision practice.

Systematic Review and Weight of Evidence Are Integral to Ecological and Human Health Assessments: They Need an Integrated Framework.

Scientific assessments synthesize the various results of scientific research for policy and decision making. Synthesizing evidence in environmental assessments can involve either or both of 2 systems: systematic review (SR) and weight of evidence (WoE). Systematic review was developed to systematically assemble results of clinical trials to be combined by meta-analysis. Weight-of-evidence approaches have evolved from jurisprudence to make inferences from diverse bodies of evidence in various fields. Our objectives are to describe the similarities and differences between SR and WoE and to suggest how their best practices can be combined into a general framework that is applicable to human health and ecological assessments. Integrating SR and WoE is based on the recognition that 2 processes are required, assembling evidence and making an inference. Systematic review is characterized by methodical literature searching, screening, and data extraction, originally for meta-analysis but now for various inferential methods. Weight of evidence is characterized by systematically relating heterogeneous evidence to considerations appropriate to the inference and making the inference by weighing the evidence. Systematic review enables the unbiased assembly of evidence from literature, but methods for assembling other information must be considered as well. If only 1 type of quantitative study estimates the assessment endpoint, meta-analysis is appropriate for inference. Otherwise, the heterogeneous evidence must be weighed. A framework is presented that integrates best practices into a methodical assembly and weighing of evidence. A glossary of terms for the combined practice and a history of the origins of SR and WoE are provided in Supplemental Data. Integr Environ Assess Manag 2020;16:718-728. Published 2020. This article is a US Government work and is in the public domain in the USA.

Field-based method for evaluating the annual maximum specific conductivity tolerated by freshwater invertebrates.

Most water quality criteria are based on laboratory toxicity tests and usually include chronic and acute magnitudes. Field-based criteria are typically based on long-term or continuous exposures, so they are chronic. Biological responses of quantified, short-term aqueous exposures are seldom documented in the field. However, acute values may be derived by estimating an upper limit using temporal variance and chronic values. This method estimates an upper limit from the variance of pollutant measurements from stream locations that attain the chronic criterion. The formula for deriving a 90th centile of a standard normal distribution is used to identify the upper limit, a criterion maximum exposure concentration (CMEC). The calculated CMEC is interpreted as a maximum exposure that 95% of organisms may tolerate if the chronic exposure is not exceeded. The methods of deriving chronic and acute criteria are illustrated with specific conductivity in a mountainous area in the eastern United States. The biological relevance of the CMEC was assessed using the maximum annual exposure during the life cycle of the most salt-intolerant genera. The method using the chronic criterion and the variance of water chemistry data is practical, whereas frequently collecting and analyzing paired biological and chemical samples at numerous sites is impractical and may give misleading results due to lags in biological response. This method can be used anywhere with sufficient data to estimate the temporal variability and may be applicable for field-based criteria other than the specific conductivity criteria illustrated here.

A field-based model of the relationship between extirpation of salt-intolerant benthic invertebrates and background conductivity.

Field-collected measures of dissolved salts and occurrences of aquatic invertebrates have been used to develop protective levels. However, sufficiently large field data sets of exposures and biota are often not available. Therefore, a model was developed to predict the exposure extirpating 5% of benthic invertebrate genera using only measures of specific conductivity (SC) as the independent variable. The model is based on 3 assumptions: (1) a genus will rarely occur where the background exceeds its upper physiological limit; (2) the lowest possible tolerance limit of a genus in a region is defined by the natural background; and (3) as a result, there will be a regular association between natural background SC and the SC at which salt-intolerant genera are present. Three steps were used to develop the model. First, background SC was characterized as the 25th centile of sampled sites for each of 24 areas in the United States with streams dominated by bicarbonate and sulfate ions. Second, the extirpation concentration (XC95), an estimate of the upper tolerance limit with respect to SC, was calculated for genera in 24 data sets. Next, the lower 5th centile of each set of XC95 values (XCD05) was identified for the most salt-intolerant members in each data set. Finally, the relationship between the 24 background SC and the 24 XCD05 values was empirically modeled to develop a background-to-criterion model. The least squares regression of XCD05 values on log background SC (log Y = 0.658logX + 1.071) yields a strong linear relationship (r = 0.93). The regression model makes it possible to use SC background to predict the SC likely to extirpate the most salt-intolerant genera in an area. The results also suggest that species distribute along natural background gradients of SC and that this relationship can be used to develop criteria for ionic concentration.

Using extirpation to evaluate ionic tolerance of freshwater fish.

Field data of fish occurrences and specific conductivity were used to estimate the tolerance of freshwater fish to elevated ion concentrations and to compare the differences between species- and genus-level analyses for individual effects. We derived extirpation concentrations at the 95th percentile (XC95) of a weighted cumulative frequency distribution for fish species inhabiting streams of the central and southern Appalachians by customizing methods used previously with macroinvertebrate genera. Weighting factors were calculated based on the number of sites in basins where each species occurred, reducing overweighting observations of species restricted to fewer basins. Comparing the species- and genus-level fish XC95 values, XC95s for fish genera were near the XC95s for the most salt-tolerant species in the genus. Therefore, a genus-level effect threshold is not reliably predictive of species-level extirpation, unless the genus is monospecific in the assessed assemblage. Of the 101 fish species XC95 values, 5% were <509 and 10% were <565 µS/cm. The lowest XC95 for a species was 322 µS/cm, which is >300 µS/cm, the exposure estimated to extirpate 5% of macroinvertebrate genera in the central Appalachians. Above 509 µS/cm, 41 of the 101 species are expected to decline in occurrence. Environ Toxicol Chem 2018;37:871-883. Published 2017 Wiley Periodicals Inc. on behalf of SETAC. This article is a US government work and, as such, is in the public domain in the United States of America.

Assessing background levels of specific conductivity using weight of evidence.

There are many ways to estimate background levels, and many types of evidence may contribute to determining whether a water, air, or soil is at background. As a result, it is important to define background in each case and to weigh the available evidence to determine the best estimate of background. A weight-of-evidence approach is demonstrated that assesses whether the background SC is sufficiently similar in streams of Ecoregion 70 in West Virginia and Ohio. During planning, five relevant considerations were identified to assess background SC: physical properties, measured SC, spatial distribution of low SC sites, biological properties, and data relevance and reliability. For each consideration, diverse types of evidence were generated, evaluated, and synthesized using weight of evidence. In the example, evidence was weighed for the hypothesis that background SC is similar in two areas in Ecoregion 70, the Western Allegheny Plateau in the eastern United States. Where, as in this case, background is not well characterized by measurements, because data sets are small or sampling designs or anthropogenic inputs may influence estimates of background, it is suggested that information about regional properties, related to and affected by SC, may be used to determine whether SC in the less characterized area is sufficiently similar to a well characterized area.

A field-based characterization of conductivity in areas of minimal alteration: A case example in the Cascades of northwestern United States.

The concentration of salts in streams is increasing world-wide making freshwater a declining resource. Developing thresholds for freshwater with low specific conductivity (SC), a measure of dissolved ions in water, may protect high quality resources that are refugia for aquatic life and that dilute downstream waters. In this case example, methods are illustrated for estimating protective levels for streams with low SC. The Cascades in the Pacific Northwest of the United States of America was selected for the case study because a geophysical model indicated that the SC of freshwater streams was likely to be very low. Also, there was an insufficient range in the SC data to accurately derive a criterion using the 2011, US Environmental Protection Agency field-based extirpation concentration distribution method. Instead, background and a regression model was used to estimate chronic and acute SC levels that could extirpate 5% of benthic invertebrate genera. Background SC was estimated at the 25th centile (33µS/cm) of the measured data and used as the independent variable in a least squares empirical background-to-criteria (B-C) model. Because no comparison could be made with effect levels estimated from a paired SC and biological data set from the Cascades, the lower 50% prediction limit (PL) was identified as an example chronic water quality criterion (97µS/cm). The maximum exposure threshold was estimated at the 90th centile SC of streams meeting the chronic SC level. The example acute SC level was 190µS/cm. Because paired aquatic life and SC data are often sparse, the B-C method is useful for developing SC criteria for other systems with limited data.

Step-by-step calculation and spreadsheet tools for predicting stressor levels that extirpate genera and species.

In 2011, the United States Environmental Protection Agency (USEPA) released a field-based method for estimating the extirpation of freshwater aquatic benthic invertebrates by ionic mixtures dominated by HCO3- , SO42- , and Ca2+ measured as specific conductivity (SC). The estimate of extirpation was SC at the 95th centile (XC95) of a weighted cumulative frequency distribution (CFD) of a genus or species over a range of SC. A CFD of XC95 values was used to predict the SC at which 5% of genera were likely to be extirpated. Because there are many uses for XC95 values and many data sets that could be analyzed using this method, we laid out a step-by-step method for calculating XC95 values and the stressor level that predicts a 5% extirpation of genera (HC05). Although the calculations can be done with a handheld calculator, we developed 2 downloadable Microsoft Excel® spreadsheet calculation tools that are easy to use to calculate XC95 values, to plot a taxon's XC95 cumulative frequency distribution with increasing SC, and to plot probabilities of observing a taxon at a particular SC. They also plot cumulative frequency distributions of XC95 values and calculate HC05 values. In addition to the tools, we share an example and the output of XC95 values for 176 distinct aquatic benthic invertebrates in Appalachia, in West Virginia, USA. Integr Environ Assess Manag 2018;14:174-180. © 2017 SETAC.

A flow-chart for developing water quality criteria from two field-based methods.

Field-based methods increase relevance and realism when setting water quality criteria. They also pose challenges. To enable a consistent process, a flow chart was developed for choosing between two field-based methods and then selecting among candidate results. The two field-based methods estimated specific conductivity (SC) levels likely to extirpate 5% of benthic invertebrate genera: an extirpation concentration distribution (XCD) method and a background-to-criterion (B-C) model developed by the U.S. Environmental Protection Agency. The B-C model is a least squares regression of the 5th centile of XCD (XCD05) values against estimates of background SC. Selection of an XCD05 from the flowchart is determined by characteristics of the paired chemical and biological data sets and method for estimating the XCD05 values. Confidence in these example SC XCD05 values is based on the size of the data sets and ecoregional SC disturbance. The level of ecoregional SC disturbance was judged by comparing the background SC (the 25th centile of the data set used to calculate a XCD05) and an estimate of natural base-flow SC modeled from geophysical attributes in the region. The B-C approach appears to be a viable option for estimating a SC benchmark with inexpensive estimates of SC background while the XCD method is used when the data are abundant. To illustrate the use of the flow chart, example SC XCD05 values were calculated for 63 of 86 Level III ecoregions in the conterminous United States of America.

A weight of evidence framework for environmental assessments: Inferring quantities.

The US Environmental Protection Agency (USEPA) has developed a generally applicable framework for a weight-of-evidence (WoE) process for deriving quantitative values from multiple estimates. These guidelines are intended for environmental assessments that require the generation of quantitative parameters such as degradation rates or that develop quantitative products such as criterion values or magnitudes of effects. The basic steps are to weigh evidence for the environmental quality to be quantified, generate the value by merging estimates or by identifying the best estimate, and weight the results to determine confidence in the numerical value. When multiple data sets or outputs of multiple models are available, it may be appropriate to weigh the evidence. Use of the framework to weigh multiple estimates may increase the accuracy of quantitative results compared to a single estimate from a default method. Its use can provide greater transparency compared to ad hoc weighing of evidence. Integr Environ Assess Manag 2017;13:1045-1051. Published 2017. This article is a US Government work and is in the public domain in the USA.

A weight of evidence framework for environmental assessments: Inferring qualities.

The weighing of heterogeneous evidence such as conventional laboratory toxicity tests, field tests, biomarkers, and community surveys is essential to environmental assessments. Evidence synthesis and weighing is needed to determine causes of observed effects, hazards posed by chemicals or other agents, the completeness of remediation, and other environmental qualities. As part of its guidelines for weight of evidence (WoE) in ecological assessments, the US Environmental Protection Agency has developed a generally applicable framework. Its basic steps are these: assemble evidence, weight the evidence, and weigh the body of evidence. Use of the framework can increase the consistency and rigor of WoE practices and provide greater transparency than ad hoc and narrative-based approaches. Integr Environ Assess Manag 2017;13:1038-1044. Published 2017. This article is a US Government work and is in the public domain in the USA.

Why care about aquatic insects: uses, benefits, and services.

Aquatic insects are common subjects of ecological research and environmental monitoring and assessment. However, their important role in protecting and restoring aquatic ecosystems is often challenged because their benefits and services to humans are not obvious to decision makers or the public. Insects are food for fish, amphibians, and wildlife. They are important contributors to energy and nutrient processing, including capturing nutrients and returning them to terrestrial ecosystems and purifying water. They provide recreation to fishermen and nature lovers and are cultural symbols. Monetary benefits to fishermen can be quantified, but most other benefits have been described qualitatively. Integr Environ Assess Manag 2015;11:188-194.

A methodology for inferring the causes of observed impairments in aquatic ecosystems.

Biological surveys have become a common technique for determining whether aquatic communities have been injured. However, their results are not useful for identifying management options until the causes of apparent injuries have been identified. Techniques for determining causation have been largely informal and ad hoc. This paper presents a logical system for causal inference. It begins by analyzing the available information to generate causal evidence; available information may include spatial or temporal associations of potential cause and effect, field or laboratory experimental results, and diagnostic evidence from the affected organisms. It then uses a series of three alternative methods to infer the cause: Elimination of causes, diagnostic protocols, and analysis of the strength of evidence. If the cause cannot be identified with sufficient confidence, the reality of the effects is examined, and if the effects are determined to be real, more information is obtained to reiterate the process.

Determining the causes of impairments in the Little Scioto River, Ohio, USA: part 2. Characterization of causes.

Two stream reaches in the Little Scioto River (OH, USA) were characterized for the causes of impairments measured at two locations. By inductive inference, six candidate causes were winnowed down to three and five candidate causes for each of the two stream reaches. Using a formal strength-of-evidence process, a single cause was determined. At the most upstream location, habitat alterations, including fine-textured substrates and low DO, were characterized as the probable causes for an increased percentage of anomalies of fish, a decreased percentage of mayflies, and an increased percentage of tolerant macroinvertebrates. An increase in the relative weight of fish was attributed to an artificially narrow, deepened channel. Approximately 2 km downstream, polycyclic aromatic hydrocarbon (PAH)-contaminated sediments were identified as the cause for both fish and macroinvertebrate impairments. Causal characterization using first elimination and then a strength-of-evidence approach narrowed and defined the causes of ecological impairment even in this situation, where many complex and interacting candidate causes existed. Applying a formal method highlighted types of data and associations that can strengthen and present a more convincing determination of the causes of impairment.

Predicting levels of stress from biological assessment data: empirical models from the Eastern Corn Belt Plains, Ohio, USA.

Interest is increasing in using biological community data to provide information on the specific types of anthropogenic influences impacting streams. We built empirical models that predict the level of six different types of stress with fish and benthic macroinvertebrate data as explanatory variables. Significant models were found for six stressor factors: stream corridor structure; siltation; total suspended solids (TSS), biochemical oxygen demand (BOD), and iron (Fe); chemical oxygen demand (COD) and BOD; zinc (Zn) and lead (Pb); and nitrate and nitrite (NOx) and phosphorus (P). Model R2 values were lowest for the siltation factor and highest for TSS, BOD, and Fe. Model R2 values increased when spatial relationships were incorporated into the model. The models generally performed well when applied to a random subset of the data. Performance was more mixed when models were applied to data collected from a previous time period, perhaps because of a change in the spatial structure of these systems. These models may provide a useful indication of the levels of different stresses impacting stream reaches in the Eastern Corn Belt Plains ecoregion of Ohio, USA. More generally, the models provide additional evidence that biological communities can serve as useful indicators of the types of anthropogenic stress impacting aquatic systems.

Determining probable causes of ecological impairment in the Little Scioto River, Ohio, USA: part 1. Listing candidate causes and analyzing evidence.

The Little Scioto River in north-central Ohio, USA, is considered to be biologically impaired based on the results of fish and invertebrate surveys. The causes for these impairments were evaluated by means of a formal method. Two of the impairments identified on the stream reach were characterized in detail to support the causal assessment. A list of six candidate causes was developed that included habitat alteration, polycyclic aromatic hydrocarbon contamination, metals contamination, low dissolved oxygen, ammonia toxicity, and nutrient enrichment. Evidence for the causal evaluation was developed with data from the site that associated each candidate cause with the biological responses. Evidence was also developed that drew on data from other locations and laboratory studies, including comparisons of site exposures with screening values and criteria. The formal method increased the transparency of the assessment; candidate causes were clearly listed and the pathways by which they may have produced effects were shown. Analysis of the evidence maximized the utility of available data, which were collected as part of monitoring and research programs rather than to specifically support a causal assessment. This case study illustrates how the stressor identification method can be used to draw conclusions from available data about the most likely causes of impairment and to show what additional studies would be useful.

A method for assessing the potential for confounding applied to ionic strength in central Appalachian streams.

Causal relationships derived from field data are potentially confounded by variables that are correlated with both the cause and its effect. The present study presents a method for assessing the potential for confounding and applies it to the relationship between ionic strength and impairment of benthic invertebrate assemblages in central Appalachian streams. The method weighs all available evidence for and against confounding by each potential confounder. It identifies 10 types of evidence for confounding, presents a qualitative scoring system, and provides rules for applying the scores. Twelve potential confounders were evaluated: habitat, organic enrichment, nutrients, deposited sediments, pH, selenium, temperature, lack of headwaters, catchment area, settling ponds, dissolved oxygen, and metals. One potential confounder, low pH, was found to be biologically significant and eliminated by removing sites with pH < 6. Other potential confounders were eliminated based on the weight of evidence. This method was found to be useful and defensible. It could be applied to other environmental assessments that use field data to develop causal relationships, including contaminated site remediation or management of natural resources.