Consensus Paper: Cerebellum and Reward.

Cerebellum is a key-structure for the modulation of motor, cognitive, social and affective functions, contributing to automatic behaviours through interactions with the cerebral cortex, basal ganglia and spinal cord. The predictive mechanisms used by the cerebellum cover not only sensorimotor functions but also reward-related tasks. Cerebellar circuits appear to encode temporal difference error and reward prediction error. From a chemical standpoint, cerebellar catecholamines modulate the rate of cerebellar-based cognitive learning, and mediate cerebellar contributions during complex behaviours. Reward processing and its associated emotions are tuned by the cerebellum which operates as a controller of adaptive homeostatic processes based on interoceptive and exteroceptive inputs. Lobules VI-VII/areas of the vermis are candidate regions for the cortico-subcortical signaling pathways associated with loss aversion and reward sensitivity, together with other nodes of the limbic circuitry. There is growing evidence that the cerebellum works as a hub of regional dysconnectivity across all mood states and that mental disorders involve the cerebellar circuitry, including mood and addiction disorders, and impaired eating behaviors where the cerebellum might be involved in longer time scales of prediction as compared to motor operations. Cerebellar patients exhibit aberrant social behaviour, showing aberrant impulsivity/compulsivity. The cerebellum is a master-piece of reward mechanisms, together with the striatum, ventral tegmental area (VTA) and prefrontal cortex (PFC). Critically, studies on reward processing reinforce our view that a fundamental role of the cerebellum is to construct internal models, perform predictions on the impact of future behaviour and compare what is predicted and what actually occurs.

Area-specific thalamocortical synchronization underlies the transition from motor planning to execution.

We studied correlated firing between motor thalamic and cortical cells in monkeys performing a delayed-response reaching task. Simultaneous recording of thalamocortical activity revealed that around movement onset, thalamic cells were positively correlated with cell activity in the primary motor cortex but negatively correlated with the activity of the premotor cortex. The differences in the correlation contrasted with the average neural responses, which were similar in all three areas. Neuronal correlations reveal functional cooperation and opposition between the motor thalamus and distinct motor cortical areas with specific roles in planning vs. performing movements. Thus, by enhancing and suppressing motor and premotor firing, the motor thalamus can facilitate the transition from a motor plan to execution.

Attenuation of noise correlations in the transformation from the frontal eye field to movement.

Correlated activity between neurons can cause variability in behavior across trials, as trial-by-trial cofluctuations can propagate downstream through the motor system. The extent to which correlated activity affects behavior depends on the properties of the translation of the population activity into movement. A major hurdle in studying the effects of noise correlations on behavior is that in many cases this translation is unknown. Previous research has overcome this by using models that make strong assumptions about the coding of motor variables. We developed a novel method that estimates the contribution of correlations to behavior with minimal assumptions. Our method partitions noise correlations into correlations that are expressed in a specific behavior, termed behavior-related correlations, and correlations that are not. We applied this method to study the relationship between noise correlations in the frontal eye field (FEF) and pursuit eye movements. We defined a distance metric between the pursuit behavior on different trials. Based on this metric, we used a shuffling approach to estimate pursuit-related correlations. Although the correlations were partially linked to variability in the eye movements, even the most constrained shuffle strongly attenuated the correlations. Thus, only a small fraction of FEF correlations is expressed in behavior. We used simulations to validate our approach, show that it captures behavior-related correlations, and demonstrate its generalizability in different models. We show that the attenuation of correlated activity through the motor pathway could stem from the interplay between the structure of the correlations and the decoder of FEF activity.NEW & NOTEWORTHY The effect of noise correlations on neural computations has been studied extensively. However, the degree to which correlations affect downstream areas remains unknown. Here, we take advantage of precise measurement of eye movement behavior to estimate the degree to which correlated variability between neurons in the frontal eye field (FEF) affects subsequent behavior. To achieve this, we developed a novel shuffling-based method and verified it using different models of the FEF.

Encoding of eye movements explains reward-related activity in cerebellar simple spikes.

The cerebellum exhibits both motor and reward-related signals. However, it remains unclear whether reward is processed independently from the motor command or might reflect the motor consequences of the reward drive. To test how reward-related signals interact with sensorimotor processing in the cerebellum, we recorded Purkinje cell simple spike activity in the cerebellar floccular complex while monkeys were engaged in smooth pursuit eye movement tasks. The color of the target signaled the size of the reward the monkeys would receive at the end of the target motion. When the tracking task presented a single target, both pursuit and neural activity were only slightly modulated by the reward size. The reward modulations in single cells were rarely large enough to be detected. These modulations were only significant in the population analysis when we averaged across many neurons. In two-target tasks where the monkey learned to select based on the size of the reward outcome, both behavior and neural activity adapted rapidly. In both the single- and two-target tasks, the size of the reward-related modulation matched the size of the effect of reward on behavior. Thus, unlike cortical activity in eye movement structures, the reward-related signals could not be dissociated from the motor command. These results suggest that reward information is integrated with the eye movement command upstream of the Purkinje cells in the floccular complex. Thus reward-related modulations of the simple spikes are akin to modulations found in motor behavior and not to the central processing of the reward value.NEW & NOTEWORTHY Disentangling sensorimotor and reward signals is only possible if these signals do not completely overlap. We recorded activity in the floccular complex of the cerebellum while monkeys performed tasks designed to separate representations of reward from those of movement. Activity modulation by reward could be accounted for by the coding of eye movement parameters, suggesting that reward information is already integrated into motor commands upstream of the floccular complex.

Encoding of Reward and Decoding Movement from the Frontal Eye Field during Smooth Pursuit Eye Movements.

Expectation of reward potentiates sensorimotor transformations to drive vigorous movements. One of the main challenges in studying reward is to determine how representations of reward interact with the computations that drive behavior. We recorded activity in smooth pursuit neurons in the frontal eye field (FEF) of two male rhesus monkeys while controlling the eye speed by manipulating either reward size or target speed. The neurons encoded the different reward conditions more strongly than the different target speed conditions. This pattern could not be explained by differences in the eye speed, since the eye speed sensitivity of the neurons was also larger for the reward conditions. Pooling the responses by the preferred direction of the neurons attenuated the reward modulation and led to a tighter association between neural activity and behavior. Therefore, a plausible decoder such as the population vector could explain how the FEF both drives behavior and encodes reward beyond behavior.SIGNIFICANCE STATEMENT Motor areas combine sensory and reward information to drive movement. To disambiguate these sources, we manipulated the speed of smooth pursuit eye movements by controlling either the size of the reward or the speed of the visual motion signals. We found that the relationship between activity in frontal eye field and eye kinematics varied: the eye speed sensitivity was larger for the different reward conditions than for the different target speed conditions. Decoders that pooled signals by the preferred direction of the neurons attenuated the reward modulations. These decoders may indicate how reward can be both encoded beyond eye kinematics at the single neuron level and drive movement at the population level.

Dissecting patterns of preparatory activity in the frontal eye fields during pursuit target selection.

We investigated the composition of preparatory activity of frontal eye field (FEF) neurons in monkeys performing a pursuit target selection task. In response to the orthogonal motion of a large and a small reward target, monkeys initiated pursuit biased toward the direction of large reward target motion. FEF neurons exhibited robust preparatory activity preceding movement initiation in this task. Preparatory activity consisted of two components, ramping activity that was constant across target selection conditions, and a flat offset in firing rates that signaled the target selection condition. Ramping activity accounted for 50% of the variance in the preparatory activity and was linked most strongly, on a trial-by-trial basis, to pursuit eye movement latency rather than to its direction or gain. The offset in firing rates that discriminated target selection conditions accounted for 25% of the variance in the preparatory activity and was commensurate with a winner-take-all representation, signaling the direction of large reward target motion rather than a representation that matched the parameters of the upcoming movement. These offer new insights into the role that the frontal eye fields play in target selection and pursuit control. They show that preparatory activity in the FEF signals more strongly when to move rather than where or how to move and suggest that structures outside the FEF augment its contributions to the target selection process.NEW & NOTEWORTHY We used the smooth eye movement pursuit system to link between patterns of preparatory activity in the frontal eye fields and movement during a target selection task. The dominant pattern was a ramping signal that did not discriminate between selection conditions and was linked, on trial-by-trial basis, to movement latency. A weaker pattern was composed of a constant signal that discriminated between selection conditions but was only weakly linked to the movement parameters.

Local vs. volume conductance activity of field potentials in the human subthalamic nucleus.

Subthalamic nucleus field potentials have attracted growing research and clinical interest over the last few decades. However, it is unclear whether subthalamic field potentials represent locally generated neuronal subthreshold activity or volume conductance of the organized neuronal activity generated in the cortex. This study aimed at understanding of the physiological origin of subthalamic field potentials and determining the most accurate method for recording them. We compared different methods of recordings in the human subthalamic nucleus: spikes (300-9,000 Hz) and field potentials (3-100 Hz) recorded by monopolar micro- and macroelectrodes, as well as by differential-bipolar macroelectrodes. The recordings were done outside and inside the subthalamic nucleus during electrophysiological navigation for deep brain stimulation procedures (150 electrode trajectories) in 41 Parkinson's disease patients. We modeled the signal and estimated the contribution of nearby/independent vs. remote/common activity in each recording configuration and area. Monopolar micro- and macroelectrode recordings detect field potentials that are considerably affected by common (probably cortical) activity. However, bipolar macroelectrode recordings inside the subthalamic nucleus can detect locally generated potentials. These results are confirmed by high correspondence between the model predictions and actual correlation of neuronal activity recorded by electrode pairs. Differential bipolar macroelectrode subthalamic field potentials can overcome volume conductance effects and reflect locally generated neuronal activity. Bipolar macroelectrode local field potential recordings might be used as a biological marker of normal and pathological brain functions for future electrophysiological studies and navigation systems as well as for closed-loop deep brain stimulation paradigms.NEW & NOTEWORTHY Our results integrate a new method for human subthalamic recordings with a development of an advanced mathematical model. We found that while monopolar microelectrode and macroelectrode recordings detect field potentials that are considerably affected by common (probably cortical) activity, bipolar macroelectrode recordings inside the subthalamic nucleus (STN) detect locally generated potentials that are significantly different than those recorded outside the STN. Differential bipolar subthalamic field potentials can be used in navigation and closed-loop deep brain stimulation paradigms.

Interactions between target location and reward size modulate the rate of microsaccades in monkeys.

We have studied how rewards modulate the occurrence of microsaccades by manipulating the size of an expected reward and the location of the cue that sets the expectations for future reward. We found an interaction between the size of the reward and the location of the cue. When monkeys fixated on a cue that signaled the size of future reward, the frequency of microsaccades was higher if the monkey expected a large vs. a small reward. When the cue was presented at a site in the visual field that was remote from the position of fixation, reward size had the opposite effect: the frequency of microsaccades was lower when the monkey was expecting a large reward. The strength of pursuit initiation also was affected by reward size and by the presence of microsaccades just before the onset of target motion. The gain of pursuit initiation increased with reward size and decreased when microsaccades occurred just before or after the onset of target motion. The effect of the reward size on pursuit initiation was much larger than any indirect effects reward might cause through modulation of the rate of microsaccades. We found only a weak relationship between microsaccade direction and the location of the exogenous cue relative to fixation position, even in experiments where the location of the cue indicated the direction of target motion. Our results indicate that the expectation of reward is a powerful modulator of the occurrence of microsaccades, perhaps through attentional mechanisms.

Hold your pauses: external globus pallidus neurons respond to behavioural events by decreasing pause activity.

Awareness of its rich structural pathways has earned the external segment of the globus pallidus (GPe) recognition as a central figure within the basal ganglia circuitry. Interestingly, GPe neurons are uniquely identified by the presence of prominent pauses interspersed among a high-frequency discharge rate of 50-80 spikes/s. These pauses have an average pause duration of 620 ms with a frequency of 13/min, yielding an average pause activity (probability of a GPe neuron being in a pause) of (620 × 13)/(60 × 1000) = 0.13. Spontaneous pause activity has been found to be inversely related to arousal state. The relationship of pause activity with behavioural events remains to be elucidated. In the present study, we analysed the electrophysiological activity of 200 well-isolated GPe pauser cells recorded from four non-human primates (Macaque fascicularis) while they were engaged in similar classical conditioning tasks. The isolation quality of the recorded activity and the pauses were determined with objective automatic methods. The results showed that the pause probability decreased by 9.09 and 10.0%, and the discharge rate increased by 2.96 and 1.95%, around cue and outcome presentation, respectively. Analysis of the linear relationship between the changes in pause activity and discharge rate showed r(2)  = 0.46 and r(2)  = 0.66 upon cue onset and outcome presentation, respectively. Thus, pause activity is a pertinent element in short-term encoding of relevant behavioural events, and has a significant, but not exclusive, role in the modulation of GPe discharge rate around these events.

Diversity of neural responses in the brainstem during smooth pursuit eye movements constrains the circuit mechanisms of neural integration.

Neural integration converts transient events into sustained neural activity. In the smooth pursuit eye movement system, neural integration is required to convert cerebellar output into the sustained discharge of extraocular motoneurons. We recorded the expression of integration in the time-varying firing rates of cerebellar and brainstem neurons in the monkey during pursuit of step-ramp target motion. Electrical stimulation with single shocks in the cerebellum identified brainstem neurons that are monosynaptic targets of inhibition from the cerebellar floccular complex. They discharge in relation to eye acceleration, eye velocity, and eye position, with a stronger acceleration signal than found in most other brainstem neurons. The acceleration and velocity signals can be accounted for by opponent contributions from the two sides of the cerebellum, without integration; the position signal implies participation in the integrator. Other neurons in the vestibular nucleus show a wide range of blends of signals related to eye velocity and eye position, reflecting different stages of integration. Neurons in the abducens nucleus discharge homogeneously in relation mainly to eye position, and reflect almost perfect integration of the cerebellar outputs. Average responses of neural populations and the diverse individual responses of large samples of individual neurons are reproduced by a hierarchical neural circuit based on a model suggested the anatomy and physiology of the larval zebrafish brainstem. The model uses a combination of feedforward and feedback connections to support a neural circuit basis for integration in monkeys and other species.

A framework for using signal, noise, and variation to determine whether the brain controls movement synergies or single muscles.

We have used an analysis of signal and variation in motor behavior to elucidate the organization of the cerebellar and brain stem circuits that control smooth pursuit eye movements. We recorded from the abducens nucleus and identified floccular target neurons (FTNs) and other, non-FTN vestibular neurons. First, we assessed neuron-behavior correlations, defined as the trial-by-trial correlation between the variation in neural firing and eye movement, in brain stem neurons. In agreement with prior data from the cerebellum, neuron-behavior correlations during pursuit initiation were large in all neurons. Second, we asked whether movement variation arises upstream from, in parallel to, or downstream from a given site of recording. We developed a model that highlighted two measures: the ratio of the SDs of neural firing rate and eye movement ("SDratio") and the neuron-behavior correlation. The relationship between these measures defines possible sources of variation. During pursuit initiation, SDratio was approximately equal to neuron-behavior correlation, meaning that the source of signal and variation is upstream from the brain stem. During steady-state pursuit, neuron-behavior correlation became somewhat smaller than SDratio for FTNs, meaning that some variation may arise downstream in the brain stem. The data contradicted the model's predictions for sources of variation in pathways that run parallel to the site of recording. Because signal and noise are tightly linked in motor control, we take the source of variation as a proxy for the source of signal, leading us to conclude that the brain controls movement synergies rather than single muscles for eye movements.

Organization of reward and movement signals in the basal ganglia and cerebellum.

The basal ganglia and the cerebellum are major subcortical structures in the motor system. The basal ganglia have been cast as the reward center of the motor system, whereas the cerebellum is thought to be involved in adjusting sensorimotor parameters. Recent findings of reward signals in the cerebellum have challenged this dichotomous view. To compare the basal ganglia and the cerebellum directly, we recorded from oculomotor regions in both structures from the same monkeys. We partitioned the trial-by-trial variability of the neurons into reward and eye-movement signals to compare the coding across structures. Reward expectation and movement signals were the most pronounced in the output structure of the basal ganglia, intermediate in the cerebellum, and the smallest in the input structure of the basal ganglia. These findings suggest that reward and movement information is sharpened through the basal ganglia, resulting in a higher signal-to-noise ratio than in the cerebellum.

High-dimensional encoding of movement by single neurons in basal ganglia output.

The substantia nigra pars reticulata (SNpr), an output structure of the basal ganglia, is hypothesized to gate movement execution. Previous studies in the eye movement system focusing mostly on saccades have reported that SNpr neurons are tonically active and either pause or increase their firing during movements, consistent with the gating role. We recorded activity in the SNpr of two monkeys during smooth pursuit and saccadic eye movements. SNpr neurons exhibited highly diverse reaction patterns during pursuit, including frequent increases and decreases in firing rate, uncorrelated responses in different movement directions and in reward conditions that resulted in the high dimensional activity of single neurons. These diverse temporal patterns surpassed those in other oculomotor areas in the medial-temporal cortex, frontal cortex, basal ganglia, and cerebellum. These findings suggest that temporal properties of the responses enrich the coding capacity of the basal ganglia output beyond gating or permitting movement.

Reward action in the initiation of smooth pursuit eye movements.

Reward has a powerful influence on motor behavior. To probe how and where reward systems alter motor behavior, we studied smooth pursuit eye movements in monkeys trained to associate the color of a visual cue with the size of the reward to be issued at the end of the target motion. When the tracking task presented two different colored targets that moved orthogonally, monkeys biased the initiation of pursuit toward the direction of motion of the target that led to larger reward. The bias was larger than expected given the modest effects of reward size on tracking of single targets. Experiments with three different reward sizes suggested that the bias afforded a given target depends mainly on the size of the larger reward. To analyze the effect of reward on directional learning in pursuit, monkeys tracked a single moving target that changed direction 250 ms after the onset of motion. Expectation of a larger reward led to a larger learned eye movement during the acquisition of the learned response and during subsequent probes of what had been learned, implying that reward influenced the expression rather than the acquisition of learning. The specific effects of reward size on learning and two-target stimuli imply that the site of reward modulation is at a level where multiple target motions compete for control of eye movement, downstream from sensory processing and learning and upstream from final motor processing.

Descending systems translate transient cortical commands into a sustained muscle activation signal.

Controlling motor actions requires online adjustments of time-varying parameters. Although numerous studies have attempted to identify the parameters coded in different motor sites, the relationships between the temporal profile of neuronal responses and the dynamics of motor behavior remain poorly understood in particular because motor parameters such as force and movement direction often change over time. We studied time-dependent coding of cortical and spinal neurons in primates performing an isometric wrist task with an active hold period, which made it possible to segregate motor behavior into its phasic and sustained components. Here, we show that cortical neurons transiently code motor-related parameters when actively acquiring a goal, whereas spinal interneurons provide persistent information regarding maintained torque level and posture. Moreover, motor cortical neurons differed substantially from spinal neurons with regard to the evolvement of parameter-specific coding over the course of a trial. These results suggest that the motor cortex and spinal cord use different control policies: Cortical neurons produce transient motor commands governing ensuing actions, whereas spinal neurons exhibit sustained coding of ongoing motor states. Hence, motor structures downstream to M1 need to integrate cortical commands to produce state-dependent spinal firing.

Rate versus synchrony codes for cerebellar control of motor behavior.

Information transmission between neural populations could occur through either coordinated changes in firing rates or the precise transmission of spike timing. We investigate the code for information transmission from a part of the cerebellar cortex that is crucial for the accurate execution of a quantifiable motor behavior. Simultaneous recordings from Purkinje cell pairs in the cerebellum of rhesus macaques reveal how these cells coordinate their activity to drive smooth pursuit eye movements. Purkinje cells show millisecond-scale coordination of spikes (synchrony), but the level of synchrony is small and insufficient to impact the firing of downstream vestibular nucleus neurons. Analysis of previous metrics that purported to reveal Purkinje cell synchrony demonstrates that these metrics conflate changes in firing rate and neuron-neuron covariance. We conclude that the output of the cerebellar cortex uses primarily a rate rather than a synchrony code to drive the activity of downstream neurons and thus control motor behavior.

Rate versus synchrony codes for cerebellar control of motor behavior.

Control of movement requires the coordination of multiple brain areas, each containing populations of neurons that receive inputs, process these inputs via recurrent dynamics, and then relay the processed information to downstream populations. Information transmission between neural populations could occur through either coordinated changes in firing rates or the precise transmission of spike timing. We investigate the nature of the code for transmission of signals to downstream areas from a part of the cerebellar cortex that is crucial for the accurate execution of a quantifiable motor behavior. Simultaneous recordings from Purkinje cell pairs in the cerebellar flocculus of rhesus macaques revealed how these cells coordinate their activity to drive smooth pursuit eye movements. Purkinje cells show millisecond-scale coordination of spikes (synchrony), but the level of synchrony is small and likely insufficient to impact the firing of downstream neurons in the vestibular nucleus. Further, analysis of previous metrics for assaying Purkinje cell synchrony demonstrates that these metrics conflate changes in firing rate and neuron-neuron covariance. We conclude that the output of the cerebellar cortex uses primarily a rate code rather than synchrony code to drive activity of downstream neurons and thus control motor behavior. Impact statement: Information transmission in the brain can occur via changes in firing rate or via the precise timing of spikes. Simultaneous recordings from pairs of Purkinje cells in the floccular complex reveals that information transmission out of the cerebellar cortex relies almost exclusively on changes in firing rates rather than millisecond-scale coordination of spike timing across the Purkinje cell population.

Sensorimotor-linked reward modulates smooth pursuit eye movements in monkeys.

Reward is essential for shaping behavior. Using sensory cues to imply forthcoming rewards, previous studies have demonstrated powerful effects of rewards on behavior. Nevertheless, the impact of reward on the sensorimotor transformation, particularly when reward is linked to behavior remains uncertain. In this study, we investigated how reward modulates smooth pursuit eye movements in monkeys. Three distinct associations between reward and eye movements were conducted in independent blocks. Results indicated that reward increased eye velocity during the steady-state pursuit, rather than during the initiation. The influence depended on the particular association between behavior and reward: a faster eye velocity was linked with reward. Neither rewarding slower eye movements nor randomizing rewards had a significant effect on behavior. The findings support the existence of distinct mechanisms involved in the initiation and steady-state phases of pursuit, and contribute to a deeper understanding of how reward interacts with these two periods of pursuit.

Fixational drift is driven by diffusive dynamics in central neural circuitry.

During fixation and between saccades, our eyes undergo diffusive random motion called fixational drift. The role of fixational drift in visual coding and inference has been debated in the past few decades, but the mechanisms that underlie this motion remained unknown. In particular, it has been unclear whether fixational drift arises from peripheral sources, or from central sources within the brain. Here we show that fixational drift is correlated with neural activity, and identify its origin in central neural circuitry within the oculomotor system, upstream to the ocular motoneurons (OMNs). We analyzed a large data set of OMN recordings in the rhesus monkey, alongside precise measurements of eye position, and found that most of the variance of fixational eye drifts must arise upstream of the OMNs. The diffusive statistics of the motion points to the oculomotor integrator, a memory circuit responsible for holding the eyes still between saccades, as a likely source of the motion. Theoretical modeling, constrained by the parameters of the primate oculomotor system, supports this hypothesis by accounting for the amplitude as well as the statistics of the motion. Thus, we propose that fixational ocular drift provides a direct observation of diffusive dynamics in a neural circuit responsible for storage of continuous parameter memory in persistent neural activity. The identification of a mechanistic origin for fixational drift is likely to advance the understanding of its role in visual processing and inference.

Passive Motor Learning: Oculomotor Adaptation in the Absence of Behavioral Errors.

Motor adaptation is commonly thought to be a trial-and-error process in which the accuracy of movement improves with repetition of behavior. We challenged this view by testing whether erroneous movements are necessary for motor adaptation. In the eye movement system, the association between movements and errors can be disentangled, since errors in the predicted stimulus trajectory can be perceived even without movements. We modified a smooth pursuit eye movement adaptation paradigm in which monkeys learn to make an eye movement that predicts an upcoming change in target direction. We trained the monkeys to fixate on a target while covertly, an additional target initially moved in one direction and then changed direction after 250 ms. The monkeys showed a learned response to infrequent probe trials in which they were instructed to follow the moving target. Additional experiments confirmed that probing learning or residual eye movements during fixation did not drive learning. These results show that motor adaptation can be elicited in the absence of movement and provide an animal model for studying the implementation of passive motor learning. Current models assume that the interaction between movement and error signals underlies adaptive motor learning. Our results point to other mechanisms that may drive learning in the absence of movement.