Cortical ß Power Reflects a Neural Implementation of Decision Boundary Collapse in Speeded Decisions.

A prominent account of decision-making assumes that information is accumulated until a fixed response threshold is crossed. However, many decisions require weighting of information appropriately against time. Collapsing response thresholds are a mathematically optimal solution to this decision problem. However, our understanding of the neurocomputational mechanisms underlying dynamic response thresholds remains significantly incomplete. To investigate this issue, we used a multistage drift-diffusion model (DDM) and also analyzed EEG ß power lateralization (BPL). The latter served as a neural proxy for decision signals. We analyzed a large dataset (n = 863; 434 females and 429 males) from a speeded flanker task and data from an independent confirmation sample (n = 119; 70 females and 49 males). We showed that a DDM with collapsing decision thresholds, a process wherein the decision boundary reduces over time, captured participants' time-dependent decision policy more accurately than a model with fixed thresholds. Previous research suggests that BPL over motor cortices reflects features of a decision signal and that its peak, coinciding with the motor response, may serve as a neural proxy for the decision threshold. We show that BPL around the response decreased with increasing RTs. Together, our findings offer compelling evidence for the existence of collapsing decision thresholds in decision-making processes.

Transdiagnostic inflexible learning dynamics explain deficits in depression and schizophrenia.

Deficits in reward learning are core symptoms across many mental disorders. Recent work suggests that such learning impairments arise by a diminished ability to use reward history to guide behaviour, but the neuro-computational mechanisms through which these impairments emerge remain unclear. Moreover, limited work has taken a transdiagnostic approach to investigate whether the psychological and neural mechanisms that give rise to learning deficits are shared across forms of psychopathology. To provide insight into this issue, we explored probabilistic reward learning in patients diagnosed with major depressive disorder (n = 33) or schizophrenia (n = 24) and 33 matched healthy controls by combining computational modelling and single-trial EEG regression. In our task, participants had to integrate the reward history of a stimulus to decide whether it is worthwhile to gamble on it. Adaptive learning in this task is achieved through dynamic learning rates that are maximal on the first encounters with a given stimulus and decay with increasing stimulus repetitions. Hence, over the course of learning, choice preferences would ideally stabilize and be less susceptible to misleading information. We show evidence of reduced learning dynamics, whereby both patient groups demonstrated hypersensitive learning (i.e. less decaying learning rates), rendering their choices more susceptible to misleading feedback. Moreover, there was a schizophrenia-specific approach bias and a depression-specific heightened sensitivity to disconfirmational feedback (factual losses and counterfactual wins). The inflexible learning in both patient groups was accompanied by altered neural processing, including no tracking of expected values in either patient group. Taken together, our results thus provide evidence that reduced trial-by-trial learning dynamics reflect a convergent deficit across depression and schizophrenia. Moreover, we identified disorder distinct learning deficits.

Disentangling performance-monitoring signals encoded in feedback-related EEG dynamics.

The feedback-related negativity (FRN) is a well-established electrophysiological correlate of feedback-processing. However, there is still an ongoing debate whether the FRN is driven by negative or positive reward prediction errors (RPE), valence of feedback, or mere surprise. Our study disentangles independent contributions of valence, surprise, and RPE on the feedback-related neuronal signal including the FRN and P3 components using the statistical power of a sample of N = 992 healthy individuals. The participants performed a modified time-estimation task, while EEG from 64 scalp electrodes was recorded. Our results show that valence coding is present during the FRN with larger amplitudes for negative feedback. The FRN is further modulated by surprise in a valence-dependent way being more positive-going for surprising positive outcomes. The P3 was strongly driven by both global and local surprise, with larger amplitudes for unexpected feedback and local deviants. Behavioral adaptations after feedback and FRN just show small associations. Results support the theory of the FRN as a representation of a signed RPE. Additionally, our data indicates that surprising positive feedback enhances the EEG response in the time window of the P3. These results corroborate previous findings linking the P3 to the evaluation of PEs in decision making and learning tasks.

Unbiased post-error slowing in interference tasks: A confound and a simple solution.

We typically slow down after committing an error, an effect termed post-error slowing (PES). Traditionally, PES has been calculated by subtracting post-correct from post-error RTs. Dutilh et al. (Journal of Mathematical Psychology, 56(3), 208-216, 2012), however, showed PES values calculated in this way are potentially biased. Therefore, they proposed to compute robust PES scores by subtracting pre-error RTs from post-error RTs. Based on data from a large-scale study using the flanker task, we show that both traditional and robust PES estimates can be biased. The source of the bias are differential imbalances in the percentage of congruent vs. incongruent post-correct, pre-error, and post-error trials. Specifically, we found that post-correct, pre-error, and post-error trials were more likely to be congruent than incongruent, with the size of the imbalance depending on the trial type as well as the length of the response-stimulus interval (RSI). In our study, for trials preceded by a 700-ms RSI, the percentages of congruent trials were 62% for post-correct trials, 66% for pre-error trials, and 56% for post-error trials. Relative to unbiased estimates, these imbalances inflated traditional PES estimates by 37% (9 ms) and robust PES estimates by 42% (16 ms) when individual-participant means were calculated. When individual-participant medians were calculated, the biases were even more pronounced (40% and 50% inflation, respectively). To obtain unbiased PES scores for interference tasks, we propose to compute unweighted individual-participant means by initially calculating mean RTs for congruent and incongruent trials separately, before averaging congruent and incongruent mean RTs to calculate means for post-correct, pre-error and post-error trials.

Publisher Correction: Learning relative values in the striatum induces violations of normative decision making.

An amendment to this paper has been published and can be accessed via a link at the top of the paper.

Cortical beta power reflects decision dynamics and uncovers multiple facets of post-error adaptation.

Adapting to errors quickly is essential for survival. Reaction slowing after errors is commonly observed but whether this slowing is adaptive or maladaptive is unclear. Here, we analyse a large dataset from a flanker task using two complementary approaches: a multistage drift-diffusion model, and the lateralisation of EEG beta power as a time-resolved index of choice formation. Fitted model parameters and their independently measured neuronal proxies in beta power convergently show a complex interplay of multiple mechanisms initiated after mistakes. Suppression of distracting evidence, response threshold increase, and reduction of evidence accumulation cause slow and accurate post-error responses. This data provides evidence for both adaptive control and maladaptive orienting after errors yielding an adaptive net effect - a decreased likelihood to repeat mistakes. Generally, lateralised beta power provides a non-invasive readout of action selection for the study of speeded cognitive control processes.

Physical Exercise and Spatial Training: A Longitudinal Study of Effects on Cognition, Growth Factors, and Hippocampal Plasticity.

Physical exercise has been suggested to improve cognitive performance through various neurobiological mechanisms, mediated by growth factors such as BDNF, IGF-I, and VEGF. Moreover, animal research has demonstrated that combined physical and cognitive stimulation leads to increased adult neurogenesis as compared to either experimental condition alone. In the present study, we therefore investigated whether a sequential combination of physical and spatial training in young, healthy adults elicits an additive effect on training and transfer gains. To this end, we compared the effects of (i) eight 20-minute sessions of cycling, (ii) sixteen 30-minute sessions of spatial training, (iii) a combination of both, and included (iv) a passive control cohort. We assessed longitudinal changes in cognitive performance, growth factor levels, and T1 relaxation of hippocampal subfields (acquired with 7 T MRI). While substantial physical and spatial training gains were elicited in all trained groups, longitudinal transfer changes did not differ between these groups. Notably, we found no evidence for an additive effect of sequential physical and spatial training. These results challenge the extrapolation from the findings reported in animals to young, healthy adults.

Learning relative values in the striatum induces violations of normative decision making.

To decide optimally between available options, organisms need to learn the values associated with these options. Reinforcement learning models offer a powerful explanation of how these values are learnt from experience. However, human choices often violate normative principles. We suggest that seemingly counterintuitive decisions may arise as a natural consequence of the learning mechanisms deployed by humans. Here, using fMRI and a novel behavioural task, we show that, when suddenly switched to novel choice contexts, participants' choices are incongruent with values learnt by standard learning algorithms. Instead, behaviour is compatible with the decisions of an agent learning how good an option is relative to an option with which it had previously been paired. Striatal activity exhibits the characteristics of a prediction error used to update such relative option values. Our data suggest that choices can be biased by a tendency to learn option values with reference to the available alternatives.

Comparing the error-related negativity across groups: The impact of error- and trial-number differences.

The error-related negativity (ERN or Ne) is increasingly being investigated as a marker discriminating interindividual factors and moves toward a surrogate marker for disorders or interventions. Although reproducibility and validity of neuroscientific and psychological research has been criticized, clear data on how different quantification methods of the ERN and their relation to available trial numbers affect within- and across-participant studies is sparse. Within a large sample of 863 healthy human participants, we demonstrate that, across participants, the number of errors correlates with the amplitude of the ERN independently of the number of errors included in ERN quantification per participant, constituting a possible confound when such variance is unaccounted for. Additionally, we find that ERN amplitudes reach high consistency within participants at lower trial numbers, yet when comparisons between groups of participants are desired, increasing error-trial numbers lead to higher statistical power. We derive concrete suggestions for specific types of analyses, which may help researchers to more effectively design studies and analyze error-related EEG data with the most appropriate measurement technique for the question at hand and trial number available.

Gender Influences on Brain Responses to Errors and Post-Error Adjustments.

Sexual dimorphisms have been observed in many species, including humans, and extend to the prevalence and presentation of important mental disorders associated with performance monitoring malfunctions. However, precisely which underlying differences between genders contribute to the alterations observed in psychiatric diseases is unknown. Here, we compare behavioural and neural correlates of cognitive control functions in 438 female and 436 male participants performing a flanker task while EEG was recorded. We found that males showed stronger performance-monitoring-related EEG amplitude modulations which were employed to predict subjects' genders with ~72% accuracy. Females showed more post-error slowing, but both samples did not differ in regard to response-conflict processing and coupling between the error-related negativity (ERN) and consecutive behavioural slowing. Furthermore, we found that the ERN predicted consecutive behavioural slowing within subjects, whereas its overall amplitude did not correlate with post-error slowing across participants. These findings elucidate specific gender differences in essential neurocognitive functions with implications for clinical studies. They highlight that within- and between-subject associations for brain potentials cannot be interpreted in the same way. Specifically, despite higher general amplitudes in males, it appears that the dynamics of coupling between ERN and post-error slowing between men and women is comparable.

Neural synchrony indexes impaired motor slowing after errors and novelty following white matter damage.

In humans, action errors and perceptual novelty elicit activity in a shared frontostriatal brain network, allowing them to adapt their ongoing behavior to such unexpected action outcomes. Healthy and pathologic aging reduces the integrity of white matter pathways that connect individual hubs of such networks and can impair the associated cognitive functions. Here, we investigated whether structural disconnection within this network because of small-vessel disease impairs the neural processes that subserve motor slowing after errors and novelty (post-error slowing, PES; post-novel slowing, PNS). Participants with intact frontostriatal circuitry showed increased right-lateralized beta-band (12-24 Hz) synchrony between frontocentral and frontolateral electrode sites in the electroencephalogram after errors and novelty, indexing increased neural communication. Importantly, this synchrony correlated with PES and PNS across participants. Furthermore, such synchrony was reduced in participants with frontostriatal white matter damage, in line with reduced PES and PNS. The results demonstrate that behavioral change after errors and novelty result from coordinated neural activity across a frontostriatal brain network and that such cognitive control is impaired by reduced white matter integrity.

Rapid feedback processing in human nucleus accumbens and motor thalamus.

The nucleus accumbens (NAcc) and thalamus are integral parts in models of feedback processing. Deep brain stimulation (DBS) has been successfully employed to alleviate symptoms of psychiatric conditions including obsessive-compulsive disorder (OCD) and Tourette's syndrome (TS). Common target structures are the NAcc and the ventral anterior and ventro-lateral nuclei (VA/VL) of the thalamus, for OCD and TS, respectively. The feedback related negativity (FRN) is an event-related potential associated with feedback processing reflecting posterior medial frontal cortex (pMFC) activity. Here we report on three cases where we recorded scalp EEG and local field potentials (LFP) from externalized electrodes located in the NAcc or thalamus (VA/VL) while patients engaged in a modified time estimation task, known to engage feedback processing and elicit the FRN. Additionally, scalp EEG were recorded from 29 healthy participants (HP) engaged in the same task. The signal in all structures (pMFC, NAcc, and thalamus) was differently modulated by positive and negative feedback. LFP activity in the NAcc showed a biphasic time course after positive feedback during the FRN time interval. Negative feedback elicited a much weaker and later response. In the thalamus a monophasic modulation was recorded during the FRN time interval. Again, this modulation was more pronounced after positive performance feedback compared to negative feedback. In channels outside the target area no modulation was observed. The surface-FRN was reliably elicited on a group level in HP and showed no significant difference following negative feedback between patients and HP. German Clinical Trial Register: Neurocognitive specification of dysfunctions within basal ganglia-cortex loops and their therapeutic modulation by deep brain stimulation in patients with obsessive compulsive disorder and Tourette syndrome, http://www.drks.de/DRKS00005316.

Differential modulation of reinforcement learning by D2 dopamine and NMDA glutamate receptor antagonism.

The firing pattern of midbrain dopamine (DA) neurons is well known to reflect reward prediction errors (PEs), the difference between obtained and expected rewards. The PE is thought to be a crucial signal for instrumental learning, and interference with DA transmission impairs learning. Phasic increases of DA neuron firing during positive PEs are driven by activation of NMDA receptors, whereas phasic suppression of firing during negative PEs is likely mediated by inputs from the lateral habenula. We aimed to determine the contribution of DA D2-class and NMDA receptors to appetitively and aversively motivated reinforcement learning. Healthy human volunteers were scanned with functional magnetic resonance imaging while they performed an instrumental learning task under the influence of either the DA D2 receptor antagonist amisulpride (400 mg), the NMDA receptor antagonist memantine (20 mg), or placebo. Participants quickly learned to select ("approach") rewarding and to reject ("avoid") punishing options. Amisulpride impaired both approach and avoidance learning, while memantine mildly attenuated approach learning but had no effect on avoidance learning. These behavioral effects of the antagonists were paralleled by their modulation of striatal PEs. Amisulpride reduced both appetitive and aversive PEs, while memantine diminished appetitive, but not aversive PEs. These data suggest that striatal D2-class receptors contribute to both approach and avoidance learning by detecting both the phasic DA increases and decreases during appetitive and aversive PEs. NMDA receptors on the contrary appear to be required only for approach learning because phasic DA increases during positive PEs are NMDA dependent, whereas phasic decreases during negative PEs are not.

Lesions to the prefrontal performance-monitoring network disrupt neural processing and adaptive behaviors after both errors and novelty.

Unexpected events can have internal causes (action errors) as well as external causes (perceptual novelty). Both events call for adaptations of ongoing behavior, resulting, amongst other things, in post-error and post-novelty slowing (PES/PNS) of reaction times (RT). Both types of events are processed in prefrontal brain areas, indexed by event-related potentials (ERPs): Errors are followed by a complex of ERPs comprised of the error-related negativity (ERN) and error positivity (Pe), whereas novels are followed by a N2/P3 complex. However, despite those overlapping properties, past neuroscientific studies of both types of events resulted in largely separate branches of research. Only recently have theoretical efforts proposed overlapping neuronal networks for the computation of 'unexpectedness' in general. Crucially, in a recent study, we have shown that both errors and novelty are indeed processed in the same neuronal network in the human brain: the prefrontal-cingulate performance-monitoring network (PCMN) underlying the ERN also explained significant parts of the N2/P3 complex. Here, we attempt to take this research further by investigating the causal role of the PCMN in both error and novelty processing. Eight patients with ischemic lesions to the PCMN and eight control participants performed a version of the flanker task in which they made errors, while also being presented with unexpected action effects on a subset of otherwise correct trials. In line with our predictions, lesions to the PCMN lead to significant reductions in ERP amplitude following both errors and perceptual novelty. Also, while the age-matched control participants showed the expected pattern of adaptive RT slowing to both errors and novelty, patients did not exhibit adaptive slowing behaviors following either event. These results support recent theoretical accounts according to which a general PCMN reacts to surprising events, regardless of valence and/or source of the unexpectedness.

Error awareness and the insula: links to neurological and psychiatric diseases.

Becoming aware of errors that one has committed might be crucial for strategic behavioral and neuronal adjustments to avoid similar errors in the future. This review addresses conscious error perception ("error awareness") in healthy subjects as well as the relationship between error awareness and neurological and psychiatric diseases. We first discuss the main findings on error awareness in healthy subjects. A brain region, that appears consistently involved in error awareness processes, is the insula, which also provides a link to the clinical conditions reviewed here. Then we focus on a neurological condition whose core element is an impaired awareness for neurological consequences of a disease: anosognosia for hemiplegia (AHP). The insular cortex has been implicated in both error awareness and AHP, with anterior insular regions being involved in conscious error processing and more posterior areas being related to AHP. In addition to cytoarchitectonic and connectivity data, this reflects a functional and structural gradient within the insula from anterior to posterior. Furthermore, studies dealing with error awareness and lack of insight in a number of psychiatric diseases are reported. Especially in schizophrenia, attention-deficit hyperactivity disorder, (ADHD) and autism spectrum disorders (ASD) the performance monitoring system seems impaired, thus conscious error perception might be altered.

Continuous theta-burst stimulation (cTBS) over the lateral prefrontal cortex alters reinforcement learning bias.

The prefrontal cortex is known to play a key role in higher-order cognitive functions. Recently, we showed that this brain region is active in reinforcement learning, during which subjects constantly have to integrate trial outcomes in order to optimize performance. To further elucidate the role of the dorsolateral prefrontal cortex (DLPFC) in reinforcement learning, we applied continuous theta-burst stimulation (cTBS) either to the left or right DLPFC, or to the vertex as a control region, respectively, prior to the performance of a probabilistic learning task in an fMRI environment. While there was no influence of cTBS on learning performance per se, we observed a stimulation-dependent modulation of reward vs. punishment sensitivity: Left-hemispherical DLPFC stimulation led to a more reward-guided performance, while right-hemispherical cTBS induced a more avoidance-guided behavior. FMRI results showed enhanced prediction error coding in the ventral striatum in subjects stimulated over the left as compared to the right DLPFC. Both behavioral and imaging results are in line with recent findings that left, but not right-hemispherical stimulation can trigger a release of dopamine in the ventral striatum, which has been suggested to increase the relative impact of rewards rather than punishment on behavior.

Dopamine-mediated reinforcement learning signals in the striatum and ventromedial prefrontal cortex underlie value-based choices.

A large body of evidence exists on the role of dopamine in reinforcement learning. Less is known about how dopamine shapes the relative impact of positive and negative outcomes to guide value-based choices. We combined administration of the dopamine D(2) receptor antagonist amisulpride with functional magnetic resonance imaging in healthy human volunteers. Amisulpride did not affect initial reinforcement learning. However, in a later transfer phase that involved novel choice situations requiring decisions between two symbols based on their previously learned values, amisulpride improved participants' ability to select the better of two highly rewarding options, while it had no effect on choices between two very poor options. During the learning phase, activity in the striatum encoded a reward prediction error. In the transfer phase, in the absence of any outcome, ventromedial prefrontal cortex (vmPFC) continually tracked the learned value of the available options on each trial. Both striatal prediction error coding and tracking of learned value in the vmPFC were predictive of subjects' choice performance in the transfer phase, and both were enhanced under amisulpride. These findings show that dopamine-dependent mechanisms enhance reinforcement learning signals in the striatum and sharpen representations of associative values in prefrontal cortex that are used to guide reinforcement-based decisions.