Relationship between reward-related brain activity and opportunities to sit.

The present study tested whether energy-minimizing behaviors evoke reward-related brain activity that promotes the repetition of these behaviors via reinforcement learning processes. Fifty-eight healthy young adults in a standing position performed a task where they could earn a reward either by sitting down or squatting while undergoing electroencephalographic (EEG) recording. Reward-prediction errors were quantified as the amplitude of the EEG-derived reward positivity. Results showed that reward positivity was larger on reward versus no reward trials, confirming the validity of our paradigm to measure evoked reward-related brain activity. However, results showed no evidence that sitting (versus standing and squatting) trials led to larger reward positivity. Moreover, we found no evidence suggesting that this effect was moderated by typical physical activity, physical activity on the day of the study, or energy expenditure during the experiment. However, at the behavioral level, results showed that the probability of choosing the stimulus more likely to lead to sitting than standing increased as the number of trials increased. In addition, results revealed that the probability of changing the selected stimulus was higher when the previous trial was a stand trial relative to a sit trial. In sum, neural results showed no evidence supporting the theory that opportunities to minimize energy expenditure are rewarding. However, behavioral findings suggested participants tend to choose the less effortful behavioral alternative and were therefore consistent with the theory of effort minimization (TEMPA).

A win-win situation: Does familiarity with a social robot modulate feedback monitoring and learning?

Social species rely on the ability to modulate feedback-monitoring in social contexts to adjust one's actions and obtain desired outcomes. When being awarded positive outcomes during a gambling task, feedback-monitoring is attenuated when strangers are rewarded, as less value is assigned to the awarded outcome. This difference in feedback-monitoring can be indexed by an event-related potential (ERP) component known as the Reward Positivity (RewP), whose amplitude is enhanced when receiving positive feedback. While the degree of familiarity influences the RewP, little is known about how the RewP and reinforcement learning are affected when gambling on behalf of familiar versus nonfamiliar agents, such as robots. This question becomes increasingly important given that robots may be used as teachers and/or social companions in the near future, with whom children and adults will interact with for short or long periods of time. In the present study, we examined whether feedback-monitoring when gambling on behalf of oneself compared with a robot is impacted by whether participants have familiarized themselves with the robot before the task. We expected enhanced RewP amplitude for self versus other for those who did not familiarize with the robot and that self-other differences in the RewP would be attenuated for those who familiarized with the robot. Instead, we observed that the RewP was larger when familiarization with the robot occurred, which corresponded to overall worse learning outcomes. We additionally observed an enhanced P3 effect for the high-familiarity condition, which suggests an increased motivation to reward. These findings suggest that familiarization with robots may cause a positive motivational effect, which positively affects RewP amplitudes, but interferes with learning.

Two Interventions Decrease Anxiety Sensitivity Among High Anxiety Sensitive Women: Could Physical Exercise Be the Key?

A brief group-based cognitive behavioral therapy (CBT), with running as an interoceptive exposure (IE) component, was effective in reducing anxiety sensitivity (AS) levels in undergraduate women (Watt, Stewart, Lefaivre, & Uman, 2006). This study investigated whether the CBT/IE intervention would result in decreases in AS and emotional distress that would be maintained over 14 weeks. Female undergraduates, high (n = 81) or low (n = 73) in AS, were randomized to 3-day CBT plus forty-two 10-min running IE trials (n = 83) or 3-day health education control (HEC) with interactive discussions and problem solving on exercise, nutrition, and sleep (n = 71). The CBT/IE intervention led to decreases in AS, depression, and stress symptoms for high AS participants, which were maintained at 14 weeks. Unexpectedly, HEC participants experienced similar and lasting decreases in AS, depression, and anxiety symptoms. Furthermore, there were no post-intervention differences between CBT/IE and HEC participants in any of the outcomes. Low AS participants experienced few sustained changes. Clinical implications and the possible role of aerobic exercise in explaining outcomes of both interventions are discussed.

Using brain potentials to understand prism adaptation: the error-related negativity and the P300.

Prism adaptation (PA) is both a perceptual-motor learning task as well as a promising rehabilitation tool for visuo-spatial neglect (VSN)-a spatial attention disorder often experienced after stroke resulting in slowed and/or inaccurate motor responses to contralesional targets. During PA, individuals are exposed to prism-induced shifts of the visual-field while performing a visuo-guided reaching task. After adaptation, with goggles removed, visuomotor responding is shifted to the opposite direction of that initially induced by the prisms. This visuomotor aftereffect has been used to study visuomotor learning and adaptation and has been applied clinically to reduce VSN severity by improving motor responding to stimuli in contralesional (usually left-sided) space. In order to optimize PA's use for VSN patients, it is important to elucidate the neural and cognitive processes that alter visuomotor function during PA. In the present study, healthy young adults underwent PA while event-related potentials (ERPs) were recorded at the termination of each reach (screen-touch), then binned according to accuracy (hit vs. miss) and phase of exposure block (early, middle, late). Results show that two ERP components were evoked by screen-touch: an error-related negativity (ERN), and a P300. The ERN was consistently evoked on miss trials during adaptation, while the P300 amplitude was largest during the early phase of adaptation for both hit and miss trials. This study provides evidence of two neural signals sensitive to visual feedback during PA that may sub-serve changes in visuomotor responding. Prior ERP research suggests that the ERN reflects an error processing system in medial-frontal cortex, while the P300 is suggested to reflect a system for context updating and learning. Future research is needed to elucidate the role of these ERP components in improving visuomotor responses among individuals with VSN.

Running as Interoceptive Exposure for Decreasing Anxiety Sensitivity: Replication and Extension.

A brief, group cognitive behavioural therapy with running as the interoceptive exposure (IE; exposure to physiological sensations) component was effective in decreasing anxiety sensitivity (AS; fear of arousal sensations) levels in female undergraduates (Watt et al., Anxiety and Substance Use Disorders: The Vicious Cycle of Comorbidity, 201-219, 2008). Additionally, repeated exposure to running resulted in decreases in cognitive (i.e., catastrophic thoughts) and affective (i.e., feelings of anxiety) reactions to running over time for high AS, but not low AS, participants (Sabourin et al., "Physical exercise as interoceptive exposure within a brief cognitive-behavioral treatment for anxiety-sensitive women", Journal of Cognitive Psychotherapy, 22:302-320, 2008). A follow-up study including the above-mentioned intervention with an expanded IE component also resulted in decreases in AS levels (Sabourin et al., under review). The goals of the present process study were (1) to replicate the original process study, with the expanded IE component, and (2) to determine whether decreases in cognitive, affective, and/or somatic (physiological sensations) reactions to running would be related to decreases in AS. Eighteen high AS and 10 low AS participants completed 20 IE running trials following the 3-day group intervention. As predicted, high AS participants, but not low AS participants, experienced decreases in cognitive, affective, and somatic reactions to running over time. Furthermore, decreases in cognitive and affective, but not in somatic, reactions to running were related to decreases in AS levels. These results suggest that the therapeutic effects of repeated exposure to running in decreasing sensitivity to anxiety-related sensations are not related to decreasing the experience of somatic sensations themselves. Rather, they are related to altering the cognitive and affective reactions to these sensations.

The role of visual processing in motor learning and control: Insights from electroencephalography.

Traditionally our understanding of goal-directed action been derived from either behavioral findings or neuroanatomically derived imaging (i.e., fMRI). While both of these approaches have proven valuable, they lack the ability to determine a direct locus of function while concurrently having the necessary temporal precision needed to understand millisecond scale neural interactions respectively. In this review we summarize some seminal behavioral findings across three broad areas (target perturbation, feed-forward control, and feedback processing) and for each discuss the application of electroencephalography (EEG) to the understanding of the temporal nature of visual cue utilization during movement planning, control, and learning using four existing scalp potentials. Specifically, we examine the appropriateness of using the N100 potential as an indicator of corrective behaviors in response to target perturbation, the N200 as an index of movement planning, the P300 potential as a metric of feed-forward processes, and the feedback-related negativity as an index of motor learning. Although these existing components have potential for insight into cognitive contributions and the timing of the neural processes that contribute to motor control further research is needed to expand the control-related potentials and to develop methods to permit their accurate characterization across a wide range of behavioral tasks.

How we learn to make decisions: rapid propagation of reinforcement learning prediction errors in humans.

Our ability to make decisions is predicated upon our knowledge of the outcomes of the actions available to us. Reinforcement learning theory posits that actions followed by a reward or punishment acquire value through the computation of prediction errors-discrepancies between the predicted and the actual reward. A multitude of neuroimaging studies have demonstrated that rewards and punishments evoke neural responses that appear to reflect reinforcement learning prediction errors [e.g., Krigolson, O. E., Pierce, L. J., Holroyd, C. B., & Tanaka, J. W. Learning to become an expert: Reinforcement learning and the acquisition of perceptual expertise. Journal of Cognitive Neuroscience, 21, 1833-1840, 2009; Bayer, H. M., & Glimcher, P. W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron, 47, 129-141, 2005; O'Doherty, J. P. Reward representations and reward-related learning in the human brain: Insights from neuroimaging. Current Opinion in Neurobiology, 14, 769-776, 2004; Holroyd, C. B., & Coles, M. G. H. The neural basis of human error processing: Reinforcement learning, dopamine, and the error-related negativity. Psychological Review, 109, 679-709, 2002]. Here, we used the brain ERP technique to demonstrate that not only do rewards elicit a neural response akin to a prediction error but also that this signal rapidly diminished and propagated to the time of choice presentation with learning. Specifically, in a simple, learnable gambling task, we show that novel rewards elicited a feedback error-related negativity that rapidly decreased in amplitude with learning. Furthermore, we demonstrate the existence of a reward positivity at choice presentation, a previously unreported ERP component that has a similar timing and topography as the feedback error-related negativity that increased in amplitude with learning. The pattern of results we observed mirrored the output of a computational model that we implemented to compute reward prediction errors and the changes in amplitude of these prediction errors at the time of choice presentation and reward delivery. Our results provide further support that the computations that underlie human learning and decision-making follow reinforcement learning principles.

Mind wandering and motor control: off-task thinking disrupts the online adjustment of behavior.

Mind wandering episodes have been construed as periods of "stimulus-independent" thought, where our minds are decoupled from the external sensory environment. In two experiments, we used behavioral and event-related potential (ERP) measures to determine whether mind wandering episodes can also be considered as periods of "response-independent" thought, with our minds disengaged from adjusting our behavioral outputs. In the first experiment, participants performed a motor tracking task and were occasionally prompted to report whether their attention was "on-task" or "mind wandering." We found greater tracking error in periods prior to mind wandering vs. on-task reports. To ascertain whether this finding was due to attenuation in visual perception per se vs. a disruptive effect of mind wandering on performance monitoring, we conducted a second experiment in which participants completed a time-estimation task. They were given feedback on the accuracy of their estimations while we recorded their EEG, and were also occasionally asked to report their attention state. We found that the sensitivity of behavior and the P3 ERP component to feedback signals were significantly reduced just prior to mind wandering vs. on-task attentional reports. Moreover, these effects co-occurred with decreases in the error-related negativity elicited by feedback signals (fERN), a direct measure of behavioral feedback assessment in cortex. Our findings suggest that the functional consequences of mind wandering are not limited to just the processing of incoming stimulation per se, but extend as well to the control and adjustment of behavior.

Reward positivity elicited by predictive cues.

A recent theory holds that a component of the human event-related brain potential called the reward positivity reflects a reward prediction error signal. We investigated this idea in gambling-like task in which, on each trial, a visual stimulus predicted a subsequent rewarding or nonrewarding outcome with 80% probability. Consistent with earlier results, we found that the reward positivity was larger to unexpected than to expected outcomes. In addition, we found that the predictive cues also elicited a reward positivity, as proposed by the theory. These results indicate that the reward positivity reflects the initial assessment of whether a trial will end in success or failure and the reappraisal of that information once the outcome actually occurs.

Observational practice benefits are limited to perceptual improvements in the acquisition of a novel coordination skill.

There is disagreement about the effectiveness of observational practice for the acquisition of novel coordination skills and the type of processes involved in observation of novel movements. In this study, we examined learning of a bimanual 90 degrees phase offset through comparisons of three groups; physical practice, observational practice and no practice (n = 12/group). Groups were compared before and after practice on perception and production scans of the practised pattern. The observation group was yoked to the physical group such that observers watched repeated demonstrations of a learning model. Although there were no positive effects of observational practice for physical performance measures, the observation group did not differ from the physical practice group and was more accurate than controls on perceptual discrimination measures after practice. We concluded that observation of a novel bimanual movement can aid perception but that physical practice is necessary for immediate physical performance benefits. These results are discussed in terms of cognitive mediation models of motor skill learning.

The amount of practice really matters: specificity of practice may be valid only after sufficient practice.

Studies investigating the specificity hypothesis have not always demonstrated that reliance on a specific source offeedback increases with practice. The goal of the present study was to address this inconsistency by having participants practice a throwing task with or without vision at incremental levels (10, 50, 100, or 200 acquisition trials). Following acquisition, all participants in the present experiment performed 10 trials in a no-vision transfer condition. Our results demonstrated that, given a sufficient number of acquisition trials, feedback reliance increased as a function of time engaged in practice. Our results also suggest that increased reliance on a specific source of feedback occurs only after the control strategy for a task is optimized.

Learning to become an expert: reinforcement learning and the acquisition of perceptual expertise.

To elucidate the neural mechanisms underlying the development of perceptual expertise, we recorded ERPs while participants performed a categorization task. We found that as participants learned to discriminate computer generated "blob" stimuli, feedback modulated the amplitude of the error-related negativity (ERN)-an ERP component thought to reflect error evaluation within medial-frontal cortex. As participants improved at the categorization task, we also observed an increase in amplitude of an ERP component associated with object recognition (the N250). The increase in N250 amplitude preceded an increase in amplitude of an ERN component associated with internal error evaluation (the response ERN). Importantly, these electroencephalographic changes were not observed for participants who failed to improve on the categorization task. Our results suggest that the acquisition of perceptual expertise relies on interactions between the posterior perceptual system and the reinforcement learning system involving medial-frontal cortex.

Electroencephalographic correlates of target and outcome errors.

Different neural systems underlie the evaluation of different types of errors. Recent electroencephalographic evidence suggests that outcome errors -- errors indicating the failure to achieve a movement goal -- are evaluated within medial-frontal cortex (Krigolson and Holroyd 2006, 2007a, b). Conversely, evidence from a variety of manual aiming studies has demonstrated that target errors -- discrepancies between the actual and desired motor command brought about by an unexpected change in the movement environment -- are mediated within posterior parietal cortex (e.g., Desmurget et al. 1999, 2001; Diedrichsen et al. 2005). Here, event-related brain potentials (ERP) were recorded to assess medial-frontal and parietal ERP components associated with the evaluation of outcome and target errors during performance of a manual aiming task. In line with previous results (Krigolson and Holroyd 2007a), we found that target perturbations elicited an ERP component with a parietal scalp distribution, the P300. However, the timing of kinematic changes associated with accommodation of the target perturbations relative to the timing of the P300 suggests that the P300 component was not related to the online control of movement. Instead, we believe that the P300 evoked by target perturbations reflects the updating of an internal model of the movement environment. Our results also revealed that an error-related negativity, an ERP component typically associated with the evaluation of speeded response errors and error feedback, was elicited when participants missed the movement target. Importantly, this result suggests that a reinforcement learning system within medial-frontal cortex may play a role in improving subsequent motor output.

Hierarchical error processing: different errors, different systems.

Error processing during motor control involves the evaluation of "high-level" errors (i.e., failures to meet a system goal) by a frontal system involving anterior cingulate cortex and the evaluation of "low-level" errors (i.e., discrepancies between actual and desired motor commands) by a posterior system involving posterior parietal cortex. We have recently demonstrated that high-level errors committed within the context of a continuous tracking task elicited an error-related negativity (ERN) -- a component of the event-related brain potential (ERP) generated within medial-frontal cortex that is sensitive to error commission. The purpose of the present study was to demonstrate that low-level motor errors do not elicit an ERN, but may instead evoke other ERP components associated with visual processing and online motor control. Participants performed a computer aiming task in which they manipulated a joystick to move a cursor from a start to a target position. On a random subset of trials the target jumped to a new position at movement onset, requiring the participants to modify their current motor command. Further, on one half of these "target perturbation" trials the cursor did not respond to corrective movements of the joystick. Consistent with our previous findings, we found that the uncorrectable errors elicited an ERN. We also found that the target perturbations on both correctable and uncorrectable trials did not elicit an ERN, but rather evoked two other ERP components, the N100 and P300. These results suggest that medial-frontal cortex is insensitive to low-level motor errors, and are in line with a recent theory that holds that the P300 reflects stimulus-response optimization by the impact of locus coeruleus activity on posterior cortex.

Predictive information and error processing: the role of medial-frontal cortex during motor control.

We have recently provided evidence that an error-related negativity (ERN), an ERP component generated within medial-frontal cortex, is elicited by errors made during the performance of a continuous tracking task (O.E. Krigolson & C.B. Holroyd, 2006). In the present study we conducted two experiments to investigate the ability of the medial-frontal error system to evaluate predictive error information. In two experiments participants used a joystick to perform a computer-based continuous tracking task in which some tracking errors were inevitable. In both experiments, half of these errors were preceded by a predictive cue. The results of both experiments indicated that an ERN-like waveform was elicited by tracking errors. Furthermore, in both experiments the predicted error waveforms had an earlier peak latency than the unpredicted error waveforms. These results demonstrate that the medial-frontal error system can evaluate predictive error information.