Accounting for Precision Uncertainty of Toxicity Testing: Methods to Define Borderline Ranges and Implications for Hazard Assessment of Chemicals.

For hazard classifications of chemicals, continuous data from animal- or nonanimal testing methods are often dichotomized into binary positive/negative outcomes by defining classification thresholds (CT). Experimental data are, however, subject to biological and technical variability. Each test method's precision is limited resulting in uncertainty of the positive/negative outcome if the experimental result is close to the CT. Borderline ranges (BR) around the CT were suggested, which represent ranges in which the study result is ambiguous, that is, positive or negative results are equally likely. The BR reflects a method's precision uncertainty. This article explores and compares different approaches to quantify the BR. Besides using the pooled standard deviation, we determine the BR by means of the median absolute deviation (MAD), with a sequential combination of both methods, and by using nonparametric bootstrapping. Furthermore, we quantify the BR for different hazardous effects, including nonanimal tests for skin corrosion, eye irritation, skin irritation, and skin sensitization as well as for an animal test on skin sensitization (the local lymph node assay, LLNA). Additionally, for one method (direct peptide reactivity assay) the BR was determined experimentally and compared to calculated BRs. Our results demonstrate that (i) the precision of the methods is determining the size of their BRs, (ii) there is no "perfect" method to derive a BR, alas, (iii) a consensus on BR is needed to account for the limited precision of testing methods.

Choosing the best for preventing the worst: A structured analysis of the selection of risk management options in REACH restriction dossiers.

Under the European chemicals legislation REACH (Registration, Evaluation, Authorisation and restriction of CHemicals), the use of chemicals posing an unacceptable risk for humans and the environment can be restricted. This requires that regulatory authorities of EU member states, or the European Chemicals Agency on request of the Commission, submit a restriction proposal in which they suggest one or multiple risk management options (RMOs). The options are recommended to be evaluated in a socio-economic analysis (SEA) using defined criteria. This paper explores the drivers of the selection of the preferred RMO in 32 restriction dossiers. Applying principal component analysis reveals that the selection of the preferred RMO, and the evaluation of possible trade-offs between alternative RMOs, is determined by criteria characterizing a measure's effectiveness and practicality, in particular its risk reduction capacity (R) and proportionality. A logistic regression using quantitative estimates provided in SEA suggests that the probability for an RMO to be selected is the higher the higher its R and the lower the costs of the restriction. Based on our analysis we conclude that the selection process of RMOs in REACH restriction dossiers could be strengthened by defining a limited but unambiguous set of criteria, conducting a score-based evaluation as a default, and by defining transparent decision rules.

Socio-economic analysis in REACH restriction dossiers for chemicals management: A critical review.

This paper offers a critical review of socio-economic analysis (SEA) in Registration, Evaluation, Authorisation and Restriction of Chemicals (REACH) restriction dossiers. We examine the conceptual setup of SEA and identify the methods used for impact assessment. Moreover, we analyse the outcomes of quantitative impact assessment across restriction dossiers and substance groups. We find that impact assessment has largely focused on economic and health impacts. Environmental, social, wider economic and distributional impacts have either been evaluated qualitatively or not at all. While this can be explained by the specific scope of the proposed restriction or by lacking data, we also observe a lack of approaches for environmental and health impact assessment. This underlines the need to develop integrated methods that transform information about chemical effects and risks into impacts and, ultimately, into benefits and damages. Furthermore, to strengthen the function of SEA as decision-support tool in REACH restriction procedures, a comparative SEA of at least two alternative restriction options should be the default.

The impact of precision uncertainty on predictive accuracy metrics of non-animal testing methods.

The ability of non-animal methods to correctly predict the outcome of in vivo testing in repeated applications is referred to as precision. Due to dichotomising continuous read-outs into discrete "positive/negative" hazard data, non-animal methods can reveal discordant classifications if results are sufficiently close to a defined classification threshold. This paper explores the impact of precision uncertainty on the predictive accuracy of non-animal methods. Using selected non-animal methods for assessing skin sensitisation hazard as case study examples, we explore the impact of precision uncertainty separately and in combination with uncertainty due to varying composition and size of experimental samples. Our results underline that discrete numbers of non-animal methods' sensitivity, specificity and concordance are of limited information for evaluations of non-animal testing methods' predictivity. Instead, information on the variability, and the upper and lower limits of accuracy metrics, should be provided to ensure a transparent assessment of testing methods' predictivity, and to allow for a meaningful comparison of the predictivity of non-animal methods with that of animal tests.

The Essential Elements of a Risk Governance Framework for Current and Future Nanotechnologies.

Societies worldwide are investing considerable resources into the safe development and use of nanomaterials. Although each of these protective efforts is crucial for governing the risks of nanomaterials, they are insufficient in isolation. What is missing is a more integrative governance approach that goes beyond legislation. Development of this approach must be evidence based and involve key stakeholders to ensure acceptance by end users. The challenge is to develop a framework that coordinates the variety of actors involved in nanotechnology and civil society to facilitate consideration of the complex issues that occur in this rapidly evolving research and development area. Here, we propose three sets of essential elements required to generate an effective risk governance framework for nanomaterials. (1) Advanced tools to facilitate risk-based decision making, including an assessment of the needs of users regarding risk assessment, mitigation, and transfer. (2) An integrated model of predicted human behavior and decision making concerning nanomaterial risks. (3) Legal and other (nano-specific and general) regulatory requirements to ensure compliance and to stimulate proactive approaches to safety. The implementation of such an approach should facilitate and motivate good practice for the various stakeholders to allow the safe and sustainable future development of nanotechnology.

The borderline range of toxicological methods: Quantification and implications for evaluating precision.

Test methods to assess the skin sensitization potential of a substance usually use threshold criteria to dichotomize continuous experimental read-outs into yes/no conclusions. The threshold criteria are prescribed in the respective OECD test guidelines and the conclusion is used for regulatory hazard assessment, i.e., classification and labelling of the substance. We can identify a borderline range (BR) around the classification threshold within which test results are inconclusive due to a test method's biological and technical variability. We quantified BRs in the prediction models of the non-animal test methods DPRA, LuSens and h-CLAT, and of the animal test LLNA, respectively. Depending on the size of the BR, we found that between 6% and 28% of the substances in the sets tested with these methods were considered borderline. When the results of individual non-animal test methods were combined into integrated testing strategies (ITS), borderline test results of individual tests also affected the overall assessment of the skin sensitization potential of the testing strategy. This was analyzed for the 2-out-of-3 ITS: Four out of 40 substances (10%) were considered borderline. Based on our findings we propose expanding the standard binary classification of substances into "positive"/"negative" or "hazardous"/"non-hazardous" by adding a "borderline" or "inconclusive" alert for cases where test results fall within the borderline range.

Nuclear Waste Management under Approaching Disaster: A Comparison of Decommissioning Strategies for the German Repository Asse II.

This article compares different strategies for handling low- and medium-level nuclear waste buried in a retired potassium mine in Germany (Asse II) that faces significant risk of uncontrollable brine intrusion and, hence, long-term groundwater contamination. We survey the policy process that has resulted in the identification of three possible so-called decommissioning options: complete backfilling, relocation of the waste to deeper levels in the mine, and retrieval. The selection of a decommissioning strategy must compare expected investment costs with expected social damage costs (economic, environmental, and health damage costs) caused by flooding and subsequent groundwater contamination. We apply a cost minimization approach that accounts for the uncertainty regarding the stability of the rock formation and the risk of an uncontrollable brine intrusion. Since economic and health impacts stretch out into the far future, we examine the impact of different discounting methods and rates. Due to parameter uncertainty, we conduct a sensitivity analysis concerning key assumptions. We find that retrieval, the currently preferred option by policymakers, has the lowest expected social damage costs for low discount rates. However, this advantage is overcompensated by higher expected investment costs. Considering all costs, backfilling is the best option for all discounting scenarios considered.

Economic analysis of adaptive strategies for flood risk management under climate change.

Climate change requires reconsideration of flood risk management strategies. Cost-benefit analysis (CBA), an economic decision-support tool, has been widely applied to assess these strategies. This paper aims to describe and discuss probabilistic extensions of CBA to identify welfare-maximising flood risk management strategies under climate change. First, uncertainty about the changes in return periods of hydro-meteorological extremes is introduced by probability-weighted climate scenarios. Second, the analysis is extended by learning about climate change impacts. Learning occurs upon the probabilistic arrival of information. We distinguish between learning from scientific progress, from statistical evidence and from flood disasters. These probabilistic extensions can be used to analyse and compare the economic efficiency and flexibility of flood risk management strategies under climate change. We offer a critical discussion of the scope of such extensions and options for increasing flexibility. We find that uncertainty reduction from scientific progress may reduce initial investments, while other types of learning may increase initial investments. This requires analysing effects of different types of learning. We also find that probabilistic information about climate change impacts and learning is imprecise. We conclude that risk-based CBA with learning improves the flexibility of flood risk management strategies under climate change. However, CBA provides subjective estimates of expected outcomes and reflects different decision-maker preferences than those captured in robustness analyses. We therefore advocate robustness analysis in addition to, or combined with, cost-benefit analysis to support local investment decisions on flood risk reduction and global strategies on allocation of adaptation funds for flood risk management.

Time matters: A stock-pollution approach to authorisation decision-making for PBT/vPvB chemicals under REACH.

A core aim of the European chemicals legislation REACH is to ensure that the risks caused by substances of very high concern (SVHC) are adequately controlled. Authorisation - i.e. the formal approval of certain uses of SVHC for a limited time - is a key regulatory instrument in order to achieve this goal. For SVHC which are, in addition to their toxicity, (very) persistent and/or (very) bioaccumulative (PBT/vPvB chemicals), decision-making on the authorisation is conditional on a socio-economic analysis (SEA). In a SEA companies must demonstrate that the gains from keeping a chemical in use outweigh expected damage costs for society. The current setup of the REACH authorisation process, including existing guidance on performing a SEA, ignores that PBT/vPvB chemicals are stock pollutants. This paper explores the implications of incorporating stock pollution effects of these chemicals into a SEA on authorisation decision-making. We develop a cost-benefit approach which includes stock dynamics of PBT/vPvB chemicals. This allows identifying the decision rules for granting or refusing an authorisation. Furthermore, we generalize the model to an entire set of damage functions. We show that ignoring stock pollution effects in a SEA may lead to erroneous decisions on the use of PBT/vPvB chemicals because long-term impacts are not adequately captured. Using a historic case of DDT soil contamination as an illustrative example we discuss information requirements and challenges for authorisation decisions on the use of PBT/vPvB chemicals under REACH.

Evaluation of non-animal methods for assessing skin sensitisation hazard: A Bayesian Value-of-Information analysis.

This paper offers a Bayesian Value-of-Information (VOI) analysis for guiding the development of non-animal testing strategies, balancing information gains from testing with the expected social gains and costs from the adoption of regulatory decisions. Testing is assumed to have value, if, and only if, the information revealed from testing triggers a welfare-improving decision on the use (or non-use) of a substance. As an illustration, our VOI model is applied to a set of five individual non-animal prediction methods used for skin sensitisation hazard assessment, seven battery combinations of these methods, and 236 sequential 2-test and 3-test strategies. Their expected values are quantified and compared to the expected value of the local lymph node assay (LLNA) as the animal method. We find that battery and sequential combinations of non-animal prediction methods reveal a significantly higher expected value than the LLNA. This holds for the entire range of prior beliefs. Furthermore, our results illustrate that the testing strategy with the highest expected value does not necessarily have to follow the order of key events in the sensitisation adverse outcome pathway (AOP).

Socio-economic analysis for the authorisation of chemicals under REACH: a case of very high concern?

Under the European chemicals' legislation, REACH, substances that are identified to be of "very high concern" will de facto be removed from the market unless the European Commission grants authorisations permitting specific uses. Companies who apply for an authorisation without demonstrating "adequate control" of the risks have to show by means of a socio-economic analysis (SEA) that positive impacts of use outweigh negative impacts for human health and ecosystems. This paper identifies core challenges where further in-depth guidance is urgently required in order to ensure that a SEA can deliver meaningful results and that it can effectively support decision-making on authorisation. In particular, we emphasise the need (i) to better guide the selection of tools for impact assessment, (ii) to explicitly account for stock pollution effects in impact assessments for persistent and very persistent chemicals, (iii) to define suitable impact indicators for PBT/vPvB chemicals given the lack of reliable information about safe concentration levels, (iv) to guide how impacts can be transformed into values for decision-making, and (v) to provide a well-balanced discussion of discounting of long-term impacts of chemicals.

A tutorial for analysing the cost-effectiveness of alternative methods for assessing chemical toxicity: the case of acute oral toxicity prediction.

Compared with traditional animal methods for toxicity testing, in vitro and in silico methods are widely considered to permit a more cost-effective assessment of chemicals. However, how to assess the cost-effectiveness of alternative methods has remained unclear. This paper offers a user-oriented tutorial for applying cost-effectiveness analysis (CEA) to alternative (non-animal) methods. The purpose is to illustrate how CEA facilitates the identification of the alternative method, or the combination of methods, that offers the highest information gain per unit of cost. We illustrate how information gains and costs of single methods and method combinations can be assessed. By using acute oral toxicity as an example, we apply CEA to a set of four in silico methods (ToxSuite, TOPKAT, TEST, ADMET Predictor), one in vitro method (the 3T3 Neutral Red Uptake cytotoxicity assay), and various combinations of these methods. Our results underline that in silico tools are more cost-effective than the in vitro test. Battery combinations of alternative methods, however, do not necessarily outperform single methods, because additional information gains from the battery are easily outweighed by additional costs.

A comparative survey of chemistry-driven in silico methods to identify hazardous substances under REACH.

This paper presents an inventory of in silico screening tools to identify substance properties of concern under the European chemicals' legislation REACH. The objective is to support the selection and implementation of appropriate tools as building blocks within integrated testing strategies (ITS). The relevant concerns addressed are persistence, bioaccumulation potential, acute and long-term aquatic toxicity, PBT/vPvB properties ((very) persistent, (very) bioaccumulative, toxic), CMR (carcinogenicity, mutagenicity, reproductive toxicity), endocrine disruption and skin sensitisation. The inventory offers a comparative evaluation of methods with respect to the underlying algorithms (how does the method work?) and the applicability domains (when does the method work?) as well as their limitations (when does the method not work?). The inventory explicitly addresses the reliability of predictions of different in silico models for diverse chemicals by applicability domain considerations. The confidence in predictions can be greatly improved by consensus modelling that allows for taking conflicting results into account. The inventory is complemented by a brief discussion of socio-economic tools for assessing the potential efficiency gains of using in silico methods compared to traditional in vivo testing of chemical hazards.

Training needs for toxicity testing in the 21st century: a survey-informed analysis.

Current training needs on the use of alternative methods in predictive toxicology, including new approaches based on mode-of-action (MoA) and adverse outcome pathway (AOP) concepts, are expected to evolve rapidly. In order to gain insight into stakeholder preferences for training, the European Commission's Joint Research Centre (JRC) conducted a single-question survey with twelve experts in regulatory agencies, industry, national research organisations, NGOs and consultancies. Stakeholder responses were evaluated by means of theory-based qualitative data analysis. Overall, a set of training topics were identified that relate both to general background information and to guidance for applying alternative testing methods. In particular, for the use of in silico methods, stakeholders emphasised the need for training on data integration and evaluation, in order to increase confidence in applying these methods for regulatory purposes. Although the survey does not claim to offer an exhaustive overview of the training requirements, its findings support the conclusion that the development of well-targeted and tailor-made training opportunities that inform about the usefulness of alternative methods, in particular those that offer practical experience in the application of in silico methods, deserves more attention. This should be complemented by transparent information and guidance on the interpretation of the results generated by these methods and software tools.

Towards optimization of chemical testing under REACH: a Bayesian network approach to Integrated Testing Strategies.

Integrated Testing Strategies (ITSs) are considered tools for guiding resource efficient decision-making on chemical hazard and risk management. Originating in the mid-nineties from research initiatives on minimizing animal use in toxicity testing, ITS development still lacks a methodologically consistent framework for incorporating all relevant information, for updating and reducing uncertainty across testing stages, and for handling conditionally dependent evidence. This paper presents a conceptual and methodological proposal for improving ITS development. We discuss methodological shortcomings of current ITS approaches, and we identify conceptual requirements for ITS development and optimization. First, ITS development should be based on probabilistic methods in order to quantify and update various uncertainties across testing stages. Second, reasoning should reflect a set of logic rules for consistently combining probabilities of related events. Third, inference should be hypothesis-driven and should reflect causal relationships in order to coherently guide decision-making across testing stages. To meet these requirements, we propose an information-theoretic approach to ITS development, the "ITS inference framework", which can be made operational by using Bayesian networks. As an illustration, we examine a simple two-test battery for assessing rodent carcinogenicity. Finally, we demonstrate how running the Bayesian network reveals a quantitative measure of Weight-of-Evidence.

Improving efficiency of uncertainty analysis in complex integrated assessment models: the case of the RAINS emission module.

Ever since the Regional Acidification Information and Simulation Model (RAINS) has been constructed, the treatment of uncertainty has remained an issue of major interest. In a recent review of the model performed for the Clean Air for Europe (CAFE) programme of the European Commission, a more systematic and structured uncertainty analysis has been recommended. This paper aims at contributing to the scientific debate how this can be achieved. Because of its complex structure on the one hand and limited research resources (time, computational capacities) on the other hand a full-blown uncertainty analysis in RAINS is hardly feasible. Therefore, all types of uncertainty require more efficient ways for uncertainty analysis. With respect to parameter uncertainty, we propose to focus research efforts for uncertainty analysis on key parameters. Among different approaches to select key parameters that have been discussed in the literature screening methods seem to be particularly appropriate for complex, deterministic Integrated Assessment models such as RAINS. Surprisingly, in Integrated Assessment modelling for air pollution problems of screening design have not been taken up so far. As a case study we consider the emission module of RAINS. We show that its structure allows for a straightforward and effective screening procedure.