Acceptance criteria for new approach methods in toxicology and human health-relevant life science research - part I.

Every test procedure, scientific and non-scientific, has inherent uncertainties, even when performed according to a standard operating procedure (SOP). In addition, it is prone to errors, defects, and mistakes introduced by operators, laboratory equipment, or materials used. Adherence to an SOP and comprehensive validation of the test method cannot guarantee that each test run produces data within the acceptable range of variability and with the precision and accuracy determined during the method validation. We illustrate here (part I) why controlling the validity of each test run is an important element of experimental design. The definition and application of acceptance criteria (AC) for the validity of test runs is important for the setup and use of test methods, particularly for the use of new approach methods (NAM) in toxicity testing. AC can be used for decision rules on how to handle data, e.g., to accept the data for further use (AC fulfilled) or to reject the data (AC not fulfilled). The adherence to AC has important requirements and consequences that may seem surprising at first sight: (i) AC depend on a test method's objectives, e.g., on the types/concentrations of chemicals tested, the regulatory context, the desired throughput; (ii) AC are applied and documented at each test run, while validation of a method (including the definition of AC) is only performed once; (iii) if AC are altered, then the set of data produced by a method can change. AC, if missing, are the blind spot of quality assurance: Test results may not be reliable and comparable. The establishment and uses of AC will be further detailed in part II of this series.

High Accuracy Classification of Developmental Toxicants by In Vitro Tests of Human Neuroepithelial and Cardiomyoblast Differentiation.

Human-relevant tests to predict developmental toxicity are urgently needed. A currently intensively studied approach makes use of differentiating human stem cells to measure chemically-induced deviations of the normal developmental program, as in a recent study based on cardiac differentiation (UKK2). Here, we (i) tested the performance of an assay modeling neuroepithelial differentiation (UKN1), and (ii) explored the benefit of combining assays (UKN1 and UKK2) that model different germ layers. Substance-induced cytotoxicity and genome-wide expression profiles of 23 teratogens and 16 non-teratogens at human-relevant concentrations were generated and used for statistical classification, resulting in accuracies of the UKN1 assay of 87-90%. A comparison to the UKK2 assay (accuracies of 90-92%) showed, in general, a high congruence in compound classification that may be explained by the fact that there was a high overlap of signaling pathways. Finally, the combination of both assays improved the prediction compared to each test alone, and reached accuracies of 92-95%. Although some compounds were misclassified by the individual tests, we conclude that UKN1 and UKK2 can be used for a reliable detection of teratogens in vitro, and that a combined analysis of tests that differentiate hiPSCs into different germ layers and cell types can even further improve the prediction of developmental toxicants.

Classification of Developmental Toxicants in a Human iPSC Transcriptomics-Based Test.

Despite the progress made in developmental toxicology, there is a great need for in vitro tests that identify developmental toxicants in relation to human oral doses and blood concentrations. In the present study, we established the hiPSC-based UKK2 in vitro test and analyzed genome-wide expression profiles of 23 known teratogens and 16 non-teratogens. Compounds were analyzed at the maximal plasma concentration (Cmax) and at 20-fold Cmax for a 24 h incubation period in three independent experiments. Based on the 1000 probe sets with the highest variance and including information on cytotoxicity, penalized logistic regression with leave-one-out cross-validation was used to classify the compounds as test-positive or test-negative, reaching an area under the curve (AUC), accuracy, sensitivity, and specificity of 0.96, 0.92, 0.96, and 0.88, respectively. Omitting the cytotoxicity information reduced the test performance to an AUC of 0.94, an accuracy of 0.79, and a sensitivity of 0.74. A second method, which used the number of significantly deregulated probe sets to classify the compounds, resulted in a specificity of 1; however, the AUC (0.90), accuracy (0.90), and sensitivity (0.83) were inferior compared to those of the logistic regression-based procedure. Finally, no increased performance was achieved when the high test concentrations (20-fold Cmax) were used, in comparison to testing within the realistic clinical range (1-fold Cmax). In conclusion, although further optimization is required, for example, by including additional readouts and cell systems that model different developmental processes, the UKK2-test in its present form can support the early discovery-phase detection of human developmental toxicants.

Kinetic modeling of stem cell transcriptome dynamics to identify regulatory modules of normal and disturbed neuroectodermal differentiation.

Thousands of transcriptome data sets are available, but approaches for their use in dynamic cell response modelling are few, especially for processes affected simultaneously by two orthogonal influencing variables. We approached this problem for neuroepithelial development of human pluripotent stem cells (differentiation variable), in the presence or absence of valproic acid (signaling variable). Using few basic assumptions (sequential differentiation states of cells; discrete on/off states for individual genes in these states), and time-resolved transcriptome data, a comprehensive model of spontaneous and perturbed gene expression dynamics was developed. The model made reliable predictions (average correlation of 0.85 between predicted and subsequently tested expression values). Even regulations predicted to be non-monotonic were successfully validated by PCR in new sets of experiments. Transient patterns of gene regulation were identified from model predictions. They pointed towards activation of Wnt signaling as a candidate pathway leading to a redirection of differentiation away from neuroepithelial cells towards neural crest. Intervention experiments, using a Wnt/beta-catenin antagonist, led to a phenotypic rescue of this disturbed differentiation. Thus, our broadly applicable model allows the analysis of transcriptome changes in complex time/perturbation matrices.

Development of a neural rosette formation assay (RoFA) to identify neurodevelopmental toxicants and to characterize their transcriptome disturbances.

The first in vitro tests for developmental toxicity made use of rodent cells. Newer teratology tests, e.g. developed during the ESNATS project, use human cells and measure mechanistic endpoints (such as transcriptome changes). However, the toxicological implications of mechanistic parameters are hard to judge, without functional/morphological endpoints. To address this issue, we developed a new version of the human stem cell-based test STOP-tox(UKN). For this purpose, the capacity of the cells to self-organize to neural rosettes was assessed as functional endpoint: pluripotent stem cells were allowed to differentiate into neuroepithelial cells for 6 days in the presence or absence of toxicants. Then, both transcriptome changes were measured (standard STOP-tox(UKN)) and cells were allowed to form rosettes. After optimization of staining methods, an imaging algorithm for rosette quantification was implemented and used for an automated rosette formation assay (RoFA). Neural tube toxicants (like valproic acid), which are known to disturb human development at stages when rosette-forming cells are present, were used as positive controls. Established toxicants led to distinctly different tissue organization and differentiation stages. RoFA outcome and transcript changes largely correlated concerning (1) the concentration-dependence, (2) the time dependence, and (3) the set of positive hits identified amongst 24 potential toxicants. Using such comparative data, a prediction model for the RoFA was developed. The comparative analysis was also used to identify gene dysregulations that are particularly predictive for disturbed rosette formation. This 'RoFA predictor gene set' may be used for a simplified and less costly setup of the STOP-tox(UKN) assay.

Stem Cell Transcriptome Responses and Corresponding Biomarkers That Indicate the Transition from Adaptive Responses to Cytotoxicity.

Analysis of transcriptome changes has become an established method to characterize the reaction of cells to toxicants. Such experiments are mostly performed at compound concentrations close to the cytotoxicity threshold. At present, little information is available on concentration-dependent features of transcriptome changes, in particular, at the transition from noncytotoxic concentrations to conditions that are associated with cell death. Thus, it is unclear in how far cell death confounds the results of transcriptome studies. To explore this gap of knowledge, we treated pluripotent stem cells differentiating to human neuroepithelial cells (UKN1 assay) for short periods (48 h) with increasing concentrations of valproic acid (VPA) and methyl mercury (MeHg), two compounds with vastly different modes of action. We developed various visualization tools to describe cellular responses, and the overall response was classified as "tolerance" (minor transcriptome changes), "functional adaptation" (moderate/strong transcriptome responses, but no cytotoxicity), and "degeneration". The latter two conditions were compared, using various statistical approaches. We identified (i) genes regulated at cytotoxic, but not at noncytotoxic, concentrations and (ii) KEGG pathways, gene ontology term groups, and superordinate biological processes that were only regulated at cytotoxic concentrations. The consensus markers and processes found after 48 h treatment were then overlaid with those found after prolonged (6 days) treatment. The study highlights the importance of careful concentration selection and of controlling viability for transcriptome studies. Moreover, it allowed identification of 39 candidate "biomarkers of cytotoxicity". These could serve to provide alerts that data sets of interest may have been affected by cell death in the model system studied.

Grouping of histone deacetylase inhibitors and other toxicants disturbing neural crest migration by transcriptional profiling.

Functional assays, such as the "migration inhibition of neural crest cells" (MINC) developmental toxicity test, can identify toxicants without requiring knowledge on their mode of action (MoA). Here, we were interested, whether (i) inhibition of migration by structurally diverse toxicants resulted in a unified signature of transcriptional changes; (ii) whether statistically-identified transcript patterns would inform on compound grouping even though individual genes were little regulated, and (iii) whether analysis of a small group of biologically-relevant transcripts would allow the grouping of compounds according to their MoA. We analyzed transcripts of 35 'migration genes' after treatment with 16 migration-inhibiting toxicants. Clustering, principal component analysis and correlation analyses of the data showed that mechanistically related compounds (e.g. histone deacetylase inhibitors (HDACi), PCBs) triggered similar transcriptional changes, but groups of structurally diverse toxicants largely differed in their transcriptional effects. Linear discriminant analysis (LDA) confirmed the specific clustering of HDACi across multiple separate experiments. Similarity of the signatures of the HDACi trichostatin A and suberoylanilide hydroxamic acid to the one of valproic acid (VPA), suggested that the latter compound acts as HDACi when impairing neural crest migration. In conclusion, the data suggest that (i) a given functional effect (e.g. inhibition of migration) can be associated with highly diverse signatures of transcript changes; (ii) statistically significant grouping of mechanistically-related compounds can be achieved on the basis of few genes with small regulations. Thus, incorporation of mechanistic markers in functional in vitro tests may support read-across procedures, also for structurally un-related compounds.