Temporal scaling of human scalp-recorded potentials.

Much of human behavior is governed by common processes that unfold over varying timescales. Standard event-related potential analysis assumes fixed-duration responses relative to experimental events. However, recent single-unit recordings in animals have revealed neural activity scales to span different durations during behaviors demanding flexible timing. Here, we employed a general linear modeling approach using a combination of fixed-duration and variable-duration regressors to unmix fixed-time and scaled-time components in human magneto-/electroencephalography (M/EEG) data. We use this to reveal consistent temporal scaling of human scalp-recorded potentials across four independent electroencephalogram (EEG) datasets, including interval perception, production, prediction, and value-based decision making. Between-trial variation in the temporally scaled response predicts between-trial variation in subject reaction times, demonstrating the relevance of this temporally scaled signal for temporal variation in behavior. Our results provide a general approach for studying flexibly timed behavior in the human brain.

Reward positivity affects temporal interval production in a continuous timing task.

The neural circuits of reward processing and interval timing (including the perception and production of temporal intervals) are functionally intertwined, suggesting that it might be possible for momentary reward processing to influence subsequent timing behavior. Previous animal and human studies have mainly focused on the effect of reward on interval perception, whereas its impact on interval production is less clear. In this study, we examined whether feedback, as an example of performance-contingent reward, biases interval production. We recorded EEG from 20 participants while they engaged in a continuous drumming task with different realistic tempos (1728 trials per participant). Participants received color-coded feedback after each beat about whether they were correct (on time) or incorrect (early or late). Regression-based EEG analysis was used to unmix the rapid occurrence of a feedback response called the reward positivity (RewP), which is traditionally observed in more slow-paced tasks. Using linear mixed modeling, we found that RewP amplitude predicted timing behavior for the upcoming beat. This performance-biasing effect of the RewP was interpreted as reflecting the impact of fluctuations in reward-related anterior cingulate cortex activity on timing, and the necessity of continuous paradigms to make such observations was highlighted.

Visual Motion and Decision-Making in Dyslexia: Reduced Accumulation of Sensory Evidence and Related Neural Dynamics.

Children with and without dyslexia differ in their behavioral responses to visual information, particularly when required to pool dynamic signals over space and time. Importantly, multiple processes contribute to behavioral responses. Here we investigated which processing stages are affected in children with dyslexia when performing visual motion processing tasks, by combining two methods that are sensitive to the dynamic processes leading to responses. We used a diffusion model which decomposes response time and accuracy into distinct cognitive constructs, and high-density EEG. Fifty children with dyslexia (24 male) and 50 typically developing children (28 male) 6-14 years of age judged the direction of motion as quickly and accurately as possible in two global motion tasks (motion coherence and direction integration), which varied in their requirements for noise exclusion. Following our preregistered analyses, we fitted hierarchical Bayesian diffusion models to the data, blinded to group membership. Unblinding revealed reduced evidence accumulation in children with dyslexia compared with typical children for both tasks. Additionally, we identified a response-locked EEG component which was maximal over centro-parietal electrodes which indicated a neural correlate of reduced drift rate in dyslexia in the motion coherence task, thereby linking brain and behavior. We suggest that children with dyslexia tend to be slower to extract sensory evidence from global motion displays, regardless of whether noise exclusion is required, thus furthering our understanding of atypical perceptual decision-making processes in dyslexia.SIGNIFICANCE STATEMENT Reduced sensitivity to visual information has been reported in dyslexia, with a lively debate about whether these differences causally contribute to reading difficulties. In this large preregistered study with a blind modeling approach, we combine state-of-the art methods in both computational modeling and EEG analysis to pinpoint the stages of processing that are atypical in children with dyslexia in two visual motion tasks that vary in their requirement for noise exclusion. We find reduced evidence accumulation in children with dyslexia across both tasks, and identify a neural marker, allowing us to link brain and behavior. We show that children with dyslexia exhibit general difficulties with extracting sensory evidence from global motion displays, not just in tasks that require noise exclusion.

The neural correlates of continuous feedback processing.

Feedback processing is commonly studied by analyzing the brain's response to discrete rather than continuous events. Such studies have led to the hypothesis that rapid phasic midbrain dopaminergic activity tracks reward prediction errors (RPEs), the effects of which are measurable at the scalp via electroencephalography (EEG). Although studies using continuous feedback are sparse, recent animal work suggests that moment-to-moment changes in reward are tracked by slowly ramping midbrain dopaminergic activity. Some have argued that these ramping signals index state values rather than RPEs. Our goal here was to develop an EEG measure of continuous feedback processing in humans, then test whether its behavior could be accounted for by the RPE hypothesis. Participants completed a stimulus-response learning task in which a continuous reward cue gradually increased or decreased over time. A regression-based unmixing approach revealed EEG activity with a topography and time course consistent with the stimulus-preceding negativity (SPN), a scalp potential previously linked to reward anticipation and tonic dopamine release. Importantly, this reward-related activity depended on outcome expectancy: as predicted by the RPE hypothesis, activity for expected reward cues was reduced compared to unexpected reward cues. These results demonstrate the possibility of using human scalp-recorded potentials to track continuous feedback processing, and test candidate hypotheses of this activity.

Task-level value affects trial-level reward processing.

Despite disagreement about how anterior cingulate cortex (ACC) supports decision making, a recent hypothesis suggests that activity in this region is best understood in the context of a task or series of tasks. One important task-level variable is average reward because it is both a known driver of effortful behaviour and an important determiner of the tasks in which we choose to engage. Here we asked how average task value affects reward-related ACC activity. To answer this question, we measured a reward-related signal said to be generated in ACC called the reward positivity (RewP) while participants gambled in three tasks of differing average value. The RewP was reduced in the high-value task, an effect that was not explainable by either reward magnitude or outcome expectancy. This result suggests that ACC does not evaluate outcomes and cues in isolation, but in the context of the value of the current task.

A win-win situation: Does familiarity with a social robot modulate feedback monitoring and learning?

Social species rely on the ability to modulate feedback-monitoring in social contexts to adjust one's actions and obtain desired outcomes. When being awarded positive outcomes during a gambling task, feedback-monitoring is attenuated when strangers are rewarded, as less value is assigned to the awarded outcome. This difference in feedback-monitoring can be indexed by an event-related potential (ERP) component known as the Reward Positivity (RewP), whose amplitude is enhanced when receiving positive feedback. While the degree of familiarity influences the RewP, little is known about how the RewP and reinforcement learning are affected when gambling on behalf of familiar versus nonfamiliar agents, such as robots. This question becomes increasingly important given that robots may be used as teachers and/or social companions in the near future, with whom children and adults will interact with for short or long periods of time. In the present study, we examined whether feedback-monitoring when gambling on behalf of oneself compared with a robot is impacted by whether participants have familiarized themselves with the robot before the task. We expected enhanced RewP amplitude for self versus other for those who did not familiarize with the robot and that self-other differences in the RewP would be attenuated for those who familiarized with the robot. Instead, we observed that the RewP was larger when familiarization with the robot occurred, which corresponded to overall worse learning outcomes. We additionally observed an enhanced P3 effect for the high-familiarity condition, which suggests an increased motivation to reward. These findings suggest that familiarization with robots may cause a positive motivational effect, which positively affects RewP amplitudes, but interferes with learning.

Thinking theta and alpha: Mechanisms of intuitive and analytical reasoning.

Humans have a unique ability to engage in different modes of thinking. Intuitive thinking (coined System 1, see Kahneman, 2011) is fast, automatic, and effortless whereas analytical thinking (coined System 2) is slow, contemplative, and effortful. We extend seminal pupillometry research examining these modes of thinking by using electroencephalography (EEG) to decipher their respective underlying neural mechanisms. We demonstrate that System 1 thinking is characterized by an increase in parietal alpha EEG power reflecting autonomic access to long-term memory and a release of attentional resources whereas System 2 thinking is characterized by an increase in frontal theta EEG power indicative of the engagement of cognitive control and working memory processes. Consider our results in terms of an example - a child may need cognitive control and working memory when contemplating a mathematics problem yet an adult can drive a car with little to no attention by drawing on easily accessed memories. Importantly, the unravelling of intuitive and analytical thinking mechanisms and their neural signatures will provide insight as to how different modes of thinking drive our everyday lives.

The importance of agency in human reward processing.

Converging evidence suggests that reinforcement learning (RL) signals exist within the human brain and that they play a role in the modification of behaviour. According to RL theory, prediction errors are used to update values associated with actions and/or predictive cues, thus facilitate decision-making. For example, the reward positivity-a feedback-sensitive component of the event-related brain potential (ERP)-is thought to index an RL prediction error. An unresolved question, however, is whether or not action is required to elicit a reward positivity. Reinforcement learning theory would predict that the reward positivity should diminish or disappear in the absence of action, but evidence for this claim is conflicting. To investigate the impact of cue, choice, and action on the amplitude of the reward positivity, we altered a two-armed bandit task by systematically removing these factors. The reward positivity was greatly reduced or absent in the altered versions of the task. This result highlights the key role of agency in producing learning signals, such as the reward positivity.

How we learn to make decisions: rapid propagation of reinforcement learning prediction errors in humans.

Our ability to make decisions is predicated upon our knowledge of the outcomes of the actions available to us. Reinforcement learning theory posits that actions followed by a reward or punishment acquire value through the computation of prediction errors-discrepancies between the predicted and the actual reward. A multitude of neuroimaging studies have demonstrated that rewards and punishments evoke neural responses that appear to reflect reinforcement learning prediction errors [e.g., Krigolson, O. E., Pierce, L. J., Holroyd, C. B., & Tanaka, J. W. Learning to become an expert: Reinforcement learning and the acquisition of perceptual expertise. Journal of Cognitive Neuroscience, 21, 1833-1840, 2009; Bayer, H. M., & Glimcher, P. W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron, 47, 129-141, 2005; O'Doherty, J. P. Reward representations and reward-related learning in the human brain: Insights from neuroimaging. Current Opinion in Neurobiology, 14, 769-776, 2004; Holroyd, C. B., & Coles, M. G. H. The neural basis of human error processing: Reinforcement learning, dopamine, and the error-related negativity. Psychological Review, 109, 679-709, 2002]. Here, we used the brain ERP technique to demonstrate that not only do rewards elicit a neural response akin to a prediction error but also that this signal rapidly diminished and propagated to the time of choice presentation with learning. Specifically, in a simple, learnable gambling task, we show that novel rewards elicited a feedback error-related negativity that rapidly decreased in amplitude with learning. Furthermore, we demonstrate the existence of a reward positivity at choice presentation, a previously unreported ERP component that has a similar timing and topography as the feedback error-related negativity that increased in amplitude with learning. The pattern of results we observed mirrored the output of a computational model that we implemented to compute reward prediction errors and the changes in amplitude of these prediction errors at the time of choice presentation and reward delivery. Our results provide further support that the computations that underlie human learning and decision-making follow reinforcement learning principles.

Perceived ownership impacts reward evaluation within medial-frontal cortex.

Ownership is a powerful construct. Indeed, in a series of recent studies, perceived ownership has been shown to increase attentional capacity, facilitate a memorial advantage, and elicit positive attitudes. Here, we sought to determine whether self-relevance would bias reward evaluation systems within the brain. To accomplish this, we had participants complete a simple gambling task during which they could "win" or "lose" prizes for themselves or for someone else, while electroencephalographic data were recorded. Our results indicated that the amplitude of the feedback error-related negativity, a component of the event-related brain potential sensitive to reward evaluation, was diminished when participants were not gambling for themselves. Furthermore, our data suggest that the ownership cues that indicated who would win or lose a given gamble either were processed as a potential for an increase in utility (i.e., gain: self-gambles) or were processed in a nonutilitarian manner (other-gambles). Importantly, our results suggest that the medial-frontal reward system is sensitive to perceived ownership, to the extent that it may not process changes in utility when they are not directly relevant to self.

Stabilizing expectations when shifting from analytical to intuitive reasoning: The role of prediction errors in reasoning.

As humans, we rely on intuitive reasoning for most of our decisions. However, when there is a novel or atypical decision to be made, we must rely on a slower and more deliberative thought process-analytical reasoning. As we gain experience with these novel or atypical decisions, our reasoning shifts from analytical to intuitive, which parallels a reduction in the need for cognitive control. Here, we sought to confirm this claim by employing electroencephalographic (EEG) measures of cognitive control as participants performed a simple perceptual decision-making task. Specifically, we had participants categorize "blobs" into families based on their visual attributes so we could examine how their reasoning changed with learning. In a key manipulation, halfway through the experiment we introduced novel blob families to categorize, thus temporarily increasing the need for analytical reasoning (i.e., cognitive control). Congruent with past research, we focused our EEG analyses on frontal theta activity as it has been linked to cognitive control and analytical thinking. As hypothesized, we found a transition from analytical to intuitive decision-making systems with learning as indexed by a decrease in frontal theta power. Further, when the novel blobs were introduced at the midpoint of the experiment, we found that decisions about these stimuli recruited analytical reasoning as indicated by increased theta power in comparison to decisions about well-practiced stimuli. We propose our findings to reflect prediction errors to decision demands-a monitoring process that determines whether our expectations of demands are met. Shifting from analytical to intuitive reasoning thus reflects the stabilization of our expectations of decision demands, which can be violated with unexpected demands when encountering novel stimuli.

Novelty, uncertainty, and the looming horizon.

Novelty and uncertainty are powerful drivers of exploration that are often conflated. In this issue of Neuron, Cockburn and colleagues dissociate the two and report a key interaction: close to task termination, novel options appear much more attractive relative to uncertain options.

Behavioural and neural indices of perceptual decision-making in autistic children during visual motion tasks.

Many studies report atypical responses to sensory information in autistic individuals, yet it is not clear which stages of processing are affected, with little consideration given to decision-making processes. We combined diffusion modelling with high-density EEG to identify which processing stages differ between 50 autistic and 50 typically developing children aged 6-14 years during two visual motion tasks. Our pre-registered hypotheses were that autistic children would show task-dependent differences in sensory evidence accumulation, alongside a more cautious decision-making style and longer non-decision time across tasks. We tested these hypotheses using hierarchical Bayesian diffusion models with a rigorous blind modelling approach, finding no conclusive evidence for our hypotheses. Using a data-driven method, we identified a response-locked centro-parietal component previously linked to the decision-making process. The build-up in this component did not consistently relate to evidence accumulation in autistic children. This suggests that the relationship between the EEG measure and diffusion-modelling is not straightforward in autistic children. Compared to a related study of children with dyslexia, motion processing differences appear less pronounced in autistic children. Exploratory analyses also suggest weak evidence that ADHD symptoms moderate perceptual decision-making in autistic children.

Dissociated neural signals of conflict and surprise in effortful decision Making: Theta activity reflects surprise while alpha and beta activity reflect conflict.

What makes a decision difficult? Two key factors are conflict and surprise: conflict emerges with multiple competing responses and surprise occurs with unexpected events. Conflict and surprise, however, are often thought of as parsimonious accounts of decision making rather than an integrated narrative. We sought to determine whether conflict and/or surprise concurrently or independently elicit effortful decision making. Participants made a series of diagnostic decisions from physiological readings while electroencephalographic (EEG) data were recorded. To induce conflict and surprise, we manipulated task difficulty by varying the distance between a presented physiological reading and the category border that separated the two diagnoses. Whereas frontal theta oscillations reflected surprise - when presented readings were far from the expected mean, parietal alpha and beta oscillations indicated conflict - when readings were near the category border. Our findings provide neural evidence that both conflict and surprise engage cognitive control to employ effort in decision making.

The ERP, frequency, and time-frequency correlates of feedback processing: Insights from a large sample study.

Human learning, at least in part, appears to be dependent on the evaluation of how outcomes of our actions align with our expectations. Over the past 23 years, electroencephalography (EEG) has been used to probe the neural signatures of feedback processing. Seminal work demonstrated a difference in the human event-related potential (ERP) dependent on whether people were processing correct or incorrect feedback. Since then, these feedback evoked ERPs have been associated with reinforcement learning and conflict monitoring, tied to subsequent behavioral adaptations, and shown to be sensitive to a wide range of factors (e.g., Parkinson's disease). Recently, research has turned to frequency decomposition techniques to examine how changes in the EEG power spectra are related to underlying learning mechanisms. Although the literature on the neural correlates of feedback processing is vast, there are still methodological discrepancies and differences in results across studies. Here, we provide reference results and an investigation of methodological considerations for the ERP (reward positivity) and frequency (delta and theta power) correlates of feedback evaluation with a large sample size. Specifically, participants (n = 500) performed a two-armed bandit task while we recorded EEG. Our findings provide key information about the data characteristics and relationships that exist between the neural signatures of feedback evaluation. Additionally, we conclude with selected methodological recommendations for standardization of future research. All data and scripts are freely provided to facilitate open science.

Feedback processing is enhanced following exploration in continuous environments.

Decision-making is typically studied by presenting participants with a small set of options. However, real-world behaviour, like foraging, often occurs in continuous environments. The degree to which human decision-making in discrete tasks generalizes to continuous tasks is questionable. For example, successful foraging comprises both exploration (learning about the environment) and exploitation (taking advantage of what is known). Although progress has been made in understanding the neural processes related to this trade-off in discrete tasks, it is currently unclear how, or whether, the same processes are involved in continuous tasks. To address this, we recorded electroencephalographic data while participants "dug for gold" by selecting locations on a map. Participants were cued beforehand that the map contained either a single patch of gold, or many patches of gold. We then used a computational model to classify participant responses as either exploitations, which were driven by previous reward locations and amounts, or explorations. Our participants were able to adjust their strategy based on reward distribution, exploring more in multi-patch environments and less in single-patch environments. We observed an enhancement of the feedback-locked P300, a neural signal previously linked to exploration in discrete tasks, which suggests the presence of a general neural system for managing the explore-exploit trade-off. Furthermore, the P300 was accompanied by an exploration-related enhancement of the late positive potential that was greatest in the multi-patch environment, suggesting a role for motivational processes during exploration.

The impact of wellness on neural learning systems.

Over the past 20 years there has been an increasing push for people to achieve or maintain "wellness" - a state in which one has not only physical but also mental and social well-being. While it may seem obvious that maintaining a state of wellness is beneficial, little research has been done to probe how maintaining a state of wellness impacts our brain. Here, we specifically examined the impact of wellness on a neural system within the medial-frontal cortex responsible for human reinforcement learning. Sixty-two undergraduate students completed the Perceived Wellness Survey after which they completed a computer-based learnable gambling game while electroencephalographic data were recorded. Within the game, participants were presented with a series of choices that either led to financial gains or losses. An analysis of our behavioral data indicated that participants were able to learn the underlying structure of the gambling game given that we observed improvements in performance. Concurrent with this, we observed an electroencephalographic response evoked by the evaluation of gambling outcomes - the reward positivity. Importantly, we found significant relationships between several aspects of wellness and the amplitude of the reward positivity. Given that the reward positivity is thought to reflect the function of a reinforcement learning system within the medial-frontal cortex, our results suggest that wellness impacts neural function - in this instance one of the systems responsible for human learning.

Electroencephalographic evidence for a reinforcement learning advantage during motor skill acquisition.

The feedback that we receive shapes how we learn. Previous research has demonstrated that quantitative feedback results in better performance than qualitative feedback. However, the data supporting a quantitative feedback advantage are not conclusive and further little work has been done to examine the mechanistic neural differences that underlie the relative benefits of quantitative and qualitative feedback. To address these issues, participants learned a simple motor task in quantitative and qualitative feedback conditions while electroencephalographic (EEG) data were recorded. We found that participants were more accurate and had a larger neural response - the feedback related negativity - when qualitative feedback was provided. Our data suggest that qualitative feedback is more advantageous than quantitative feedback during the early stages of skill acquisition. Additionally, our findings support previous work suggesting that a reinforcement learning system within the human medial-frontal cortex plays a key role in motor skill acquisition.