Neural systems for choice and valuation with counterfactual learning signals.

The purpose of this experiment was to test a computational model of reinforcement learning with and without fictive prediction error (FPE) signals to investigate how counterfactual consequences contribute to acquired representations of action-specific expected value, and to determine the functional neuroanatomy and neuromodulator systems that are involved. 80 male participants underwent dietary depletion of either tryptophan or tyrosine/phenylalanine to manipulate serotonin (5HT) and dopamine (DA), respectively. They completed 80 rounds (240 trials) of a strategic sequential investment task that required accepting interim losses in order to access a lucrative state and maximize long-term gains, while being scanned. We extended the standard Q-learning model by incorporating both counterfactual gains and losses into separate error signals. The FPE model explained the participants' data significantly better than a model that did not include counterfactual learning signals. Expected value from the FPE model was significantly correlated with BOLD signal change in the ventromedial prefrontal cortex (vmPFC) and posterior orbitofrontal cortex (OFC), whereas expected value from the standard model did not predict changes in neural activity. The depletion procedure revealed significantly different neural responses to expected value in the vmPFC, caudate, and dopaminergic midbrain in the vicinity of the substantia nigra (SN). Differences in neural activity were not evident in the standard Q-learning computational model. These findings demonstrate that FPE signals are an important component of valuation for decision making, and that the neural representation of expected value incorporates cortical and subcortical structures via interactions among serotonergic and dopaminergic modulator systems.

Reward-based decision-making and aging.

Healthy aging is associated with a number of neuroanatomical and neurobiological alterations that result in various cognitive changes. Both, the dopaminergic as well as the serotonergic system are subject to change during aging. Receptor loss and severe structural changes in PFC and striatum have been reported. Aging is associated with a progressive decline in several cognitive functions, such as episodic memory, working memory, and processing speed. Furthermore, it is associated with deficits in tasks requiring adaptation to external feedback of right or wrong, or task-switching. Here, we develop the hypothesis that this loss of behavioral flexibility is caused by structural and functional alterations of the reward system leading to impairments in reward processing, learning stimulus reinforcement associations, and reward-based decision-making. We review (a) data on neural correlates and substrates of reward processing in young healthy animals and humans, (b) evidence for age related functional and structural alterations of the reward system, and (c) behavioral and neuroimaging data of age effects on reward-based decision-making processes. Implications for neuroeconomics and neurodegenerative diseases are discussed.

Effect of aging on stimulus-reward association learning.

The flexible learning of stimulus-reward associations when required by situational context is essential for everyday behavior. Older adults experience a progressive decline in several cognitive functions and show deficiencies in neuropsychological tasks requiring flexible adaptation to external feedback, which could be related to impairments in reward association learning. To study the effect of aging on stimulus-reward association learning 20 young and 20 older adults performed a probabilistic object reversal task (pORT) along with a battery of tests assessing executive functions and general intellectual abilities. The pORT requires learning and reversing associations between actions and their outcomes. Older participants collected fewer points, needed more trials to reach the learning criterion, and completed less blocks successfully compared to young adults. This difference remained statistically significant after correcting for the age effect of other tests assessing executive functions. This suggests that there is an age-related difference in reward association learning as measured using the pORT, which is not closely related to other executive functions with respect to the age effect. In human aging, structural alterations of reward detecting structures and functional changes of the dopaminergic as well as the serotonergic system might contribute to the deficit in reward association learning observed in this study.

Hazard assessment of chemicals in soil : Proposed ecotoxicological test strategy.

Based on a stepwise (tiered) approach, degradation, adsorption and leaching tests as well as various effect tests (using plants, microorganisms and animals) are recommended for the testing of environmental chemicals. If, after the tests of tiers 1 and 2, the results of a monospecies-effect-test (including a safety factor) are within the range of the predicted exposure, the ecotoxicological hazard should be determined using a terrestrial model ecosystem. Some of the tests for the proposed strategy were selected from practical experience in testing environmental chemicals in the laboratory, and some on the basis of a comprehensive literature review.

Environmental risk assessment of existing chemicals.

Most of the existing chemicals of high priority have been released into the environment for many years. Risk assessments for existing chemicals are now conducted within the framework of the German Existing Chemicals Program and by the EC Regulation on Existing Substances. The environmental assessment of a chemical involves: a) exposure assessment leading to the derivation of a predicted environmental concentration (PEC) of a chemical from releases due to its production, processing, use, and disposal. The calculation of a PEC takes into account the dispersion of a chemical into different environmental compartments, elimination and dilution processes, as well as degradation. Monitoring data are also considered. b) effects assessment. Data obtained from acute or long-term toxicity tests are used for extrapolation on environmental conditions. In order to calculate the concentration with expectedly no adverse effect on organisms (Predicted No Effect Concentration, PNEC) the effect values are divided by an assessment factor. This assessment factor depends on the quantity and quality of toxicity data available. In the last step of the initial risk assessment, the measured or estimated PEC is compared with the PNEC. This "risk characterization" is conducted for each compartment separately (water, sediment, soil, and atmosphere). In case PEC > PNEC an attempt should be made to revise data of exposure and/or effects to conduct a refined risk characterization. In case PEC is again larger than PNEC risk reduction measures have to be considered.