Consensus Paper: Cerebellum and Reward.

Cerebellum is a key-structure for the modulation of motor, cognitive, social and affective functions, contributing to automatic behaviours through interactions with the cerebral cortex, basal ganglia and spinal cord. The predictive mechanisms used by the cerebellum cover not only sensorimotor functions but also reward-related tasks. Cerebellar circuits appear to encode temporal difference error and reward prediction error. From a chemical standpoint, cerebellar catecholamines modulate the rate of cerebellar-based cognitive learning, and mediate cerebellar contributions during complex behaviours. Reward processing and its associated emotions are tuned by the cerebellum which operates as a controller of adaptive homeostatic processes based on interoceptive and exteroceptive inputs. Lobules VI-VII/areas of the vermis are candidate regions for the cortico-subcortical signaling pathways associated with loss aversion and reward sensitivity, together with other nodes of the limbic circuitry. There is growing evidence that the cerebellum works as a hub of regional dysconnectivity across all mood states and that mental disorders involve the cerebellar circuitry, including mood and addiction disorders, and impaired eating behaviors where the cerebellum might be involved in longer time scales of prediction as compared to motor operations. Cerebellar patients exhibit aberrant social behaviour, showing aberrant impulsivity/compulsivity. The cerebellum is a master-piece of reward mechanisms, together with the striatum, ventral tegmental area (VTA) and prefrontal cortex (PFC). Critically, studies on reward processing reinforce our view that a fundamental role of the cerebellum is to construct internal models, perform predictions on the impact of future behaviour and compare what is predicted and what actually occurs.

Organization of reward and movement signals in the basal ganglia and cerebellum.

The basal ganglia and the cerebellum are major subcortical structures in the motor system. The basal ganglia have been cast as the reward center of the motor system, whereas the cerebellum is thought to be involved in adjusting sensorimotor parameters. Recent findings of reward signals in the cerebellum have challenged this dichotomous view. To compare the basal ganglia and the cerebellum directly, we recorded from oculomotor regions in both structures from the same monkeys. We partitioned the trial-by-trial variability of the neurons into reward and eye-movement signals to compare the coding across structures. Reward expectation and movement signals were the most pronounced in the output structure of the basal ganglia, intermediate in the cerebellum, and the smallest in the input structure of the basal ganglia. These findings suggest that reward and movement information is sharpened through the basal ganglia, resulting in a higher signal-to-noise ratio than in the cerebellum.

Rate versus synchrony codes for cerebellar control of motor behavior.

Information transmission between neural populations could occur through either coordinated changes in firing rates or the precise transmission of spike timing. We investigate the code for information transmission from a part of the cerebellar cortex that is crucial for the accurate execution of a quantifiable motor behavior. Simultaneous recordings from Purkinje cell pairs in the cerebellum of rhesus macaques reveal how these cells coordinate their activity to drive smooth pursuit eye movements. Purkinje cells show millisecond-scale coordination of spikes (synchrony), but the level of synchrony is small and insufficient to impact the firing of downstream vestibular nucleus neurons. Analysis of previous metrics that purported to reveal Purkinje cell synchrony demonstrates that these metrics conflate changes in firing rate and neuron-neuron covariance. We conclude that the output of the cerebellar cortex uses primarily a rate rather than a synchrony code to drive the activity of downstream neurons and thus control motor behavior.

Attenuation of noise correlations in the transformation from the frontal eye field to movement.

Correlated activity between neurons can cause variability in behavior across trials, as trial-by-trial cofluctuations can propagate downstream through the motor system. The extent to which correlated activity affects behavior depends on the properties of the translation of the population activity into movement. A major hurdle in studying the effects of noise correlations on behavior is that in many cases this translation is unknown. Previous research has overcome this by using models that make strong assumptions about the coding of motor variables. We developed a novel method that estimates the contribution of correlations to behavior with minimal assumptions. Our method partitions noise correlations into correlations that are expressed in a specific behavior, termed behavior-related correlations, and correlations that are not. We applied this method to study the relationship between noise correlations in the frontal eye field (FEF) and pursuit eye movements. We defined a distance metric between the pursuit behavior on different trials. Based on this metric, we used a shuffling approach to estimate pursuit-related correlations. Although the correlations were partially linked to variability in the eye movements, even the most constrained shuffle strongly attenuated the correlations. Thus, only a small fraction of FEF correlations is expressed in behavior. We used simulations to validate our approach, show that it captures behavior-related correlations, and demonstrate its generalizability in different models. We show that the attenuation of correlated activity through the motor pathway could stem from the interplay between the structure of the correlations and the decoder of FEF activity.NEW & NOTEWORTHY The effect of noise correlations on neural computations has been studied extensively. However, the degree to which correlations affect downstream areas remains unknown. Here, we take advantage of precise measurement of eye movement behavior to estimate the degree to which correlated variability between neurons in the frontal eye field (FEF) affects subsequent behavior. To achieve this, we developed a novel shuffling-based method and verified it using different models of the FEF.

Rate versus synchrony codes for cerebellar control of motor behavior.

Control of movement requires the coordination of multiple brain areas, each containing populations of neurons that receive inputs, process these inputs via recurrent dynamics, and then relay the processed information to downstream populations. Information transmission between neural populations could occur through either coordinated changes in firing rates or the precise transmission of spike timing. We investigate the nature of the code for transmission of signals to downstream areas from a part of the cerebellar cortex that is crucial for the accurate execution of a quantifiable motor behavior. Simultaneous recordings from Purkinje cell pairs in the cerebellar flocculus of rhesus macaques revealed how these cells coordinate their activity to drive smooth pursuit eye movements. Purkinje cells show millisecond-scale coordination of spikes (synchrony), but the level of synchrony is small and likely insufficient to impact the firing of downstream neurons in the vestibular nucleus. Further, analysis of previous metrics for assaying Purkinje cell synchrony demonstrates that these metrics conflate changes in firing rate and neuron-neuron covariance. We conclude that the output of the cerebellar cortex uses primarily a rate code rather than synchrony code to drive activity of downstream neurons and thus control motor behavior. Impact statement: Information transmission in the brain can occur via changes in firing rate or via the precise timing of spikes. Simultaneous recordings from pairs of Purkinje cells in the floccular complex reveals that information transmission out of the cerebellar cortex relies almost exclusively on changes in firing rates rather than millisecond-scale coordination of spike timing across the Purkinje cell population.

Sensorimotor-linked reward modulates smooth pursuit eye movements in monkeys.

Reward is essential for shaping behavior. Using sensory cues to imply forthcoming rewards, previous studies have demonstrated powerful effects of rewards on behavior. Nevertheless, the impact of reward on the sensorimotor transformation, particularly when reward is linked to behavior remains uncertain. In this study, we investigated how reward modulates smooth pursuit eye movements in monkeys. Three distinct associations between reward and eye movements were conducted in independent blocks. Results indicated that reward increased eye velocity during the steady-state pursuit, rather than during the initiation. The influence depended on the particular association between behavior and reward: a faster eye velocity was linked with reward. Neither rewarding slower eye movements nor randomizing rewards had a significant effect on behavior. The findings support the existence of distinct mechanisms involved in the initiation and steady-state phases of pursuit, and contribute to a deeper understanding of how reward interacts with these two periods of pursuit.

Fixational drift is driven by diffusive dynamics in central neural circuitry.

During fixation and between saccades, our eyes undergo diffusive random motion called fixational drift. The role of fixational drift in visual coding and inference has been debated in the past few decades, but the mechanisms that underlie this motion remained unknown. In particular, it has been unclear whether fixational drift arises from peripheral sources, or from central sources within the brain. Here we show that fixational drift is correlated with neural activity, and identify its origin in central neural circuitry within the oculomotor system, upstream to the ocular motoneurons (OMNs). We analyzed a large data set of OMN recordings in the rhesus monkey, alongside precise measurements of eye position, and found that most of the variance of fixational eye drifts must arise upstream of the OMNs. The diffusive statistics of the motion points to the oculomotor integrator, a memory circuit responsible for holding the eyes still between saccades, as a likely source of the motion. Theoretical modeling, constrained by the parameters of the primate oculomotor system, supports this hypothesis by accounting for the amplitude as well as the statistics of the motion. Thus, we propose that fixational ocular drift provides a direct observation of diffusive dynamics in a neural circuit responsible for storage of continuous parameter memory in persistent neural activity. The identification of a mechanistic origin for fixational drift is likely to advance the understanding of its role in visual processing and inference.

Area-specific thalamocortical synchronization underlies the transition from motor planning to execution.

We studied correlated firing between motor thalamic and cortical cells in monkeys performing a delayed-response reaching task. Simultaneous recording of thalamocortical activity revealed that around movement onset, thalamic cells were positively correlated with cell activity in the primary motor cortex but negatively correlated with the activity of the premotor cortex. The differences in the correlation contrasted with the average neural responses, which were similar in all three areas. Neuronal correlations reveal functional cooperation and opposition between the motor thalamus and distinct motor cortical areas with specific roles in planning vs. performing movements. Thus, by enhancing and suppressing motor and premotor firing, the motor thalamus can facilitate the transition from a motor plan to execution.

Passive Motor Learning: Oculomotor Adaptation in the Absence of Behavioral Errors.

Motor adaptation is commonly thought to be a trial-and-error process in which the accuracy of movement improves with repetition of behavior. We challenged this view by testing whether erroneous movements are necessary for motor adaptation. In the eye movement system, the association between movements and errors can be disentangled, since errors in the predicted stimulus trajectory can be perceived even without movements. We modified a smooth pursuit eye movement adaptation paradigm in which monkeys learn to make an eye movement that predicts an upcoming change in target direction. We trained the monkeys to fixate on a target while covertly, an additional target initially moved in one direction and then changed direction after 250 ms. The monkeys showed a learned response to infrequent probe trials in which they were instructed to follow the moving target. Additional experiments confirmed that probing learning or residual eye movements during fixation did not drive learning. These results show that motor adaptation can be elicited in the absence of movement and provide an animal model for studying the implementation of passive motor learning. Current models assume that the interaction between movement and error signals underlies adaptive motor learning. Our results point to other mechanisms that may drive learning in the absence of movement.

Encoding of eye movements explains reward-related activity in cerebellar simple spikes.

The cerebellum exhibits both motor and reward-related signals. However, it remains unclear whether reward is processed independently from the motor command or might reflect the motor consequences of the reward drive. To test how reward-related signals interact with sensorimotor processing in the cerebellum, we recorded Purkinje cell simple spike activity in the cerebellar floccular complex while monkeys were engaged in smooth pursuit eye movement tasks. The color of the target signaled the size of the reward the monkeys would receive at the end of the target motion. When the tracking task presented a single target, both pursuit and neural activity were only slightly modulated by the reward size. The reward modulations in single cells were rarely large enough to be detected. These modulations were only significant in the population analysis when we averaged across many neurons. In two-target tasks where the monkey learned to select based on the size of the reward outcome, both behavior and neural activity adapted rapidly. In both the single- and two-target tasks, the size of the reward-related modulation matched the size of the effect of reward on behavior. Thus, unlike cortical activity in eye movement structures, the reward-related signals could not be dissociated from the motor command. These results suggest that reward information is integrated with the eye movement command upstream of the Purkinje cells in the floccular complex. Thus reward-related modulations of the simple spikes are akin to modulations found in motor behavior and not to the central processing of the reward value.NEW & NOTEWORTHY Disentangling sensorimotor and reward signals is only possible if these signals do not completely overlap. We recorded activity in the floccular complex of the cerebellum while monkeys performed tasks designed to separate representations of reward from those of movement. Activity modulation by reward could be accounted for by the coding of eye movement parameters, suggesting that reward information is already integrated into motor commands upstream of the floccular complex.

Cerebellar climbing fibers encode expected reward size.

Climbing fiber inputs to the cerebellum encode error signals that instruct learning. Recently, evidence has accumulated to suggest that the cerebellum is also involved in the processing of reward. To study how rewarding events are encoded, we recorded the activity of climbing fibers when monkeys were engaged in an eye movement task. At the beginning of each trial, the monkeys were cued to the size of the reward that would be delivered upon successful completion of the trial. Climbing fiber activity increased when the monkeys were presented with a cue indicating a large reward, but not a small reward. Reward size did not modulate activity at reward delivery or during eye movements. Comparison between climbing fiber and simple spike activity indicated different interactions for coding of movement and reward. These results indicate that climbing fibers encode the expected reward size and suggest a general role of the cerebellum in associative learning beyond error correction.

Using extracellular low frequency signals to improve the spike sorting of cerebellar complex spikes.

BACKGROUND: The challenge of spike sorting has been addressed by numerous electrophysiological studies. These methods tend to focus on the information conveyed by the high frequencies, but ignore the potentially informative signals at lower frequencies. Activation of Purkinje cells in the cerebellum by input from the climbing fibers results in a large amplitude dendritic spike concurrent with a high-frequency burst known as a complex spike. Due to the variability in the high-frequency component of complex spikes, previous methods have struggled to sort these complex spikes in an accurate and reliable way. However, complex spikes have a prominent extracellular low-frequency signal generated by the input from the climbing fibers, which can be exploited for complex spike sorting. NEW METHOD: We exploited the low-frequency signal (20-400 Hz) to improve complex spike sorting by applying Principal Component Analysis (PCA). RESULTS AND COMPARISONS: The low-frequency first PC achieves a better separation of the complex spikes from noise. The low-frequency data facilitate the detection of events entering into the analysis, and therefore can be harnessed to analyze the data with a larger signal to noise ratio. These advantages make this method more effective for complex spike sorting than methods restricted to the high-frequency signal (> 600 Hz). CONCLUSIONS: Gathering low frequency data can improve spike sorting. This is illustrated for the case of complex spikes in the cerebellum. Our characterization of the dendritic low-frequency components of complex spikes can be applied elsewhere to gain insights into processing in the cerebellum.

Encoding of Reward and Decoding Movement from the Frontal Eye Field during Smooth Pursuit Eye Movements.

Expectation of reward potentiates sensorimotor transformations to drive vigorous movements. One of the main challenges in studying reward is to determine how representations of reward interact with the computations that drive behavior. We recorded activity in smooth pursuit neurons in the frontal eye field (FEF) of two male rhesus monkeys while controlling the eye speed by manipulating either reward size or target speed. The neurons encoded the different reward conditions more strongly than the different target speed conditions. This pattern could not be explained by differences in the eye speed, since the eye speed sensitivity of the neurons was also larger for the reward conditions. Pooling the responses by the preferred direction of the neurons attenuated the reward modulation and led to a tighter association between neural activity and behavior. Therefore, a plausible decoder such as the population vector could explain how the FEF both drives behavior and encodes reward beyond behavior.SIGNIFICANCE STATEMENT Motor areas combine sensory and reward information to drive movement. To disambiguate these sources, we manipulated the speed of smooth pursuit eye movements by controlling either the size of the reward or the speed of the visual motion signals. We found that the relationship between activity in frontal eye field and eye kinematics varied: the eye speed sensitivity was larger for the different reward conditions than for the different target speed conditions. Decoders that pooled signals by the preferred direction of the neurons attenuated the reward modulations. These decoders may indicate how reward can be both encoded beyond eye kinematics at the single neuron level and drive movement at the population level.

Coordinated cerebellar climbing fiber activity signals learned sensorimotor predictions.

The prevailing model of cerebellar learning states that climbing fibers (CFs) are both driven by, and serve to correct, erroneous motor output. However, this model is grounded largely in studies of behaviors that utilize hardwired neural pathways to link sensory input to motor output. To test whether this model applies to more flexible learning regimes that require arbitrary sensorimotor associations, we developed a cerebellar-dependent motor learning task that is compatible with both mesoscale and single-dendrite-resolution calcium imaging in mice. We found that CFs were preferentially driven by and more time-locked to correctly executed movements and other task parameters that predict reward outcome, exhibiting widespread correlated activity in parasagittal processing zones that was governed by these predictions. Together, our data suggest that such CF activity patterns are well-suited to drive learning by providing predictive instructional input that is consistent with an unsigned reinforcement learning signal but does not rely exclusively on motor errors.

Smooth Pursuit Eye Movement of Monkeys Naive to Laboratory Setups With Pictures and Artificial Stimuli.

When animal behavior is studied in a laboratory environment, the animals are often extensively trained to shape their behavior. A crucial question is whether the behavior observed after training is part of the natural repertoire of the animal or represents an outlier in the animal's natural capabilities. This can be investigated by assessing the extent to which the target behavior is manifested during the initial stages of training and the time course of learning. We explored this issue by examining smooth pursuit eye movements in monkeys naïve to smooth pursuit tasks. We recorded the eye movements of monkeys from the 1st days of training on a step-ramp paradigm. We used bright spots, monkey pictures and scrambled versions of the pictures as moving targets. We found that during the initial stages of training, the pursuit initiation was largest for the monkey pictures and in some direction conditions close to target velocity. When the pursuit initiation was large, the monkeys mostly continued to track the target with smooth pursuit movements while correcting for displacement errors with small saccades. Two weeks of training increased the pursuit eye velocity in all stimulus conditions, whereas further extensive training enhanced pursuit slightly more. The training decreased the coefficient of variation of the eye velocity. Anisotropies that grade pursuit across directions were observed from the 1st day of training and mostly persisted across training. Thus, smooth pursuit in the step-ramp paradigm appears to be part of the natural repertoire of monkeys' behavior and training adjusts monkeys' natural predisposed behavior.

Dissecting patterns of preparatory activity in the frontal eye fields during pursuit target selection.

We investigated the composition of preparatory activity of frontal eye field (FEF) neurons in monkeys performing a pursuit target selection task. In response to the orthogonal motion of a large and a small reward target, monkeys initiated pursuit biased toward the direction of large reward target motion. FEF neurons exhibited robust preparatory activity preceding movement initiation in this task. Preparatory activity consisted of two components, ramping activity that was constant across target selection conditions, and a flat offset in firing rates that signaled the target selection condition. Ramping activity accounted for 50% of the variance in the preparatory activity and was linked most strongly, on a trial-by-trial basis, to pursuit eye movement latency rather than to its direction or gain. The offset in firing rates that discriminated target selection conditions accounted for 25% of the variance in the preparatory activity and was commensurate with a winner-take-all representation, signaling the direction of large reward target motion rather than a representation that matched the parameters of the upcoming movement. These offer new insights into the role that the frontal eye fields play in target selection and pursuit control. They show that preparatory activity in the FEF signals more strongly when to move rather than where or how to move and suggest that structures outside the FEF augment its contributions to the target selection process.NEW & NOTEWORTHY We used the smooth eye movement pursuit system to link between patterns of preparatory activity in the frontal eye fields and movement during a target selection task. The dominant pattern was a ramping signal that did not discriminate between selection conditions and was linked, on trial-by-trial basis, to movement latency. A weaker pattern was composed of a constant signal that discriminated between selection conditions but was only weakly linked to the movement parameters.

Local vs. volume conductance activity of field potentials in the human subthalamic nucleus.

Subthalamic nucleus field potentials have attracted growing research and clinical interest over the last few decades. However, it is unclear whether subthalamic field potentials represent locally generated neuronal subthreshold activity or volume conductance of the organized neuronal activity generated in the cortex. This study aimed at understanding of the physiological origin of subthalamic field potentials and determining the most accurate method for recording them. We compared different methods of recordings in the human subthalamic nucleus: spikes (300-9,000 Hz) and field potentials (3-100 Hz) recorded by monopolar micro- and macroelectrodes, as well as by differential-bipolar macroelectrodes. The recordings were done outside and inside the subthalamic nucleus during electrophysiological navigation for deep brain stimulation procedures (150 electrode trajectories) in 41 Parkinson's disease patients. We modeled the signal and estimated the contribution of nearby/independent vs. remote/common activity in each recording configuration and area. Monopolar micro- and macroelectrode recordings detect field potentials that are considerably affected by common (probably cortical) activity. However, bipolar macroelectrode recordings inside the subthalamic nucleus can detect locally generated potentials. These results are confirmed by high correspondence between the model predictions and actual correlation of neuronal activity recorded by electrode pairs. Differential bipolar macroelectrode subthalamic field potentials can overcome volume conductance effects and reflect locally generated neuronal activity. Bipolar macroelectrode local field potential recordings might be used as a biological marker of normal and pathological brain functions for future electrophysiological studies and navigation systems as well as for closed-loop deep brain stimulation paradigms.NEW & NOTEWORTHY Our results integrate a new method for human subthalamic recordings with a development of an advanced mathematical model. We found that while monopolar microelectrode and macroelectrode recordings detect field potentials that are considerably affected by common (probably cortical) activity, bipolar macroelectrode recordings inside the subthalamic nucleus (STN) detect locally generated potentials that are significantly different than those recorded outside the STN. Differential bipolar subthalamic field potentials can be used in navigation and closed-loop deep brain stimulation paradigms.

Signal, Noise, and Variation in Neural and Sensory-Motor Latency.

Analysis of the neural code for sensory-motor latency in smooth pursuit eye movements reveals general principles of neural variation and the specific origin of motor latency. The trial-by-trial variation in neural latency in MT comprises a shared component expressed as neuron-neuron latency correlations and an independent component that is local to each neuron. The independent component arises heavily from fluctuations in the underlying probability of spiking, with an unexpectedly small contribution from the stochastic nature of spiking itself. The shared component causes the latency of single-neuron responses in MT to be weakly predictive of the behavioral latency of pursuit. Neural latency deeper in the motor system is more strongly predictive of behavioral latency. A model reproduces both the variance of behavioral latency and the neuron-behavior latency correlations in MT if it includes realistic neural latency variation, neuron-neuron latency correlations in MT, and noisy gain control downstream of MT.

Hold your pauses: external globus pallidus neurons respond to behavioural events by decreasing pause activity.

Awareness of its rich structural pathways has earned the external segment of the globus pallidus (GPe) recognition as a central figure within the basal ganglia circuitry. Interestingly, GPe neurons are uniquely identified by the presence of prominent pauses interspersed among a high-frequency discharge rate of 50-80 spikes/s. These pauses have an average pause duration of 620 ms with a frequency of 13/min, yielding an average pause activity (probability of a GPe neuron being in a pause) of (620 × 13)/(60 × 1000) = 0.13. Spontaneous pause activity has been found to be inversely related to arousal state. The relationship of pause activity with behavioural events remains to be elucidated. In the present study, we analysed the electrophysiological activity of 200 well-isolated GPe pauser cells recorded from four non-human primates (Macaque fascicularis) while they were engaged in similar classical conditioning tasks. The isolation quality of the recorded activity and the pauses were determined with objective automatic methods. The results showed that the pause probability decreased by 9.09 and 10.0%, and the discharge rate increased by 2.96 and 1.95%, around cue and outcome presentation, respectively. Analysis of the linear relationship between the changes in pause activity and discharge rate showed r(2)  = 0.46 and r(2)  = 0.66 upon cue onset and outcome presentation, respectively. Thus, pause activity is a pertinent element in short-term encoding of relevant behavioural events, and has a significant, but not exclusive, role in the modulation of GPe discharge rate around these events.

Interactions between target location and reward size modulate the rate of microsaccades in monkeys.

We have studied how rewards modulate the occurrence of microsaccades by manipulating the size of an expected reward and the location of the cue that sets the expectations for future reward. We found an interaction between the size of the reward and the location of the cue. When monkeys fixated on a cue that signaled the size of future reward, the frequency of microsaccades was higher if the monkey expected a large vs. a small reward. When the cue was presented at a site in the visual field that was remote from the position of fixation, reward size had the opposite effect: the frequency of microsaccades was lower when the monkey was expecting a large reward. The strength of pursuit initiation also was affected by reward size and by the presence of microsaccades just before the onset of target motion. The gain of pursuit initiation increased with reward size and decreased when microsaccades occurred just before or after the onset of target motion. The effect of the reward size on pursuit initiation was much larger than any indirect effects reward might cause through modulation of the rate of microsaccades. We found only a weak relationship between microsaccade direction and the location of the exogenous cue relative to fixation position, even in experiments where the location of the cue indicated the direction of target motion. Our results indicate that the expectation of reward is a powerful modulator of the occurrence of microsaccades, perhaps through attentional mechanisms.