Differential coding of goals and actions in ventral and dorsal corticostriatal circuits during goal-directed behavior.

Goal-directed behavior requires identifying objects in the environment that can satisfy internal needs and executing actions to obtain those objects. The current study examines ventral and dorsal corticostriatal circuits that support complementary aspects of goal-directed behavior. We analyze activity from the amygdala, ventral striatum, orbitofrontal cortex, and lateral prefrontal cortex (LPFC) while monkeys perform a three-armed bandit task. Information about chosen stimuli and their value is primarily encoded in the amygdala, ventral striatum, and orbitofrontal cortex, while the spatial information is primarily encoded in the LPFC. Before the options are presented, information about the to-be-chosen stimulus is represented in the amygdala, ventral striatum, and orbitofrontal cortex; at the time of choice, the information is passed to the LPFC to direct a saccade. Thus, learned value information specifying behavioral goals is maintained throughout the ventral corticostriatal circuit, and it is routed through the dorsal circuit at the time actions are selected.

A convolutional neural network for estimating synaptic connectivity from spike trains.

The recent increase in reliable, simultaneous high channel count extracellular recordings is exciting for physiologists and theoreticians because it offers the possibility of reconstructing the underlying neuronal circuits. We recently presented a method of inferring this circuit connectivity from neuronal spike trains by applying the generalized linear model to cross-correlograms. Although the algorithm can do a good job of circuit reconstruction, the parameters need to be carefully tuned for each individual dataset. Here we present another method using a Convolutional Neural Network for Estimating synaptic Connectivity from spike trains. After adaptation to huge amounts of simulated data, this method robustly captures the specific feature of monosynaptic impact in a noisy cross-correlogram. There are no user-adjustable parameters. With this new method, we have constructed diagrams of neuronal circuits recorded in several cortical areas of monkeys.

Reward-related choices determine information timing and flow across macaque lateral prefrontal cortex.

Prefrontal cortex is critical for cognition. Although much is known about the representation of cognitive variables in the prefrontal cortex, much less is known about the spatio-temporal neural dynamics that underlie cognitive operations. In the present study, we examined information timing and flow across the lateral prefrontal cortex (LPFC), while monkeys carried out a two-armed bandit reinforcement learning task in which they had to learn to select rewarding actions or rewarding objects. When we analyzed signals independently within subregions of the LPFC, we found a task-specific, caudo-rostral gradient in the strength and timing of signals related to chosen objects and chosen actions. In addition, when we characterized information flow among subregions, we found that information flow from action to object representations was stronger from the dorsal to ventral LPFC, and information flow from object to action representations was stronger from the ventral to dorsal LPFC. The object to action effects were more pronounced in object blocks, and also reflected learning specifically in these blocks. These results suggest anatomical segregation followed by the rapid integration of information within the LPFC.

Inference as a fundamental process in behavior.

In the real world, uncertainty is omnipresent due to incomplete or noisy information. This makes inferring the state-of-the-world difficult. Furthermore, the state-of-the-world often changes over time, though with some regularity. This makes learning and decision-making challenging. Organisms have evolved to take advantage of environmental regularities, that allow organisms to acquire a model of the world and perform model-based inference to robustly make decisions and adjust behavior efficiently under uncertainty. Recent research has shed light on many aspects of model-based inference and its neural underpinnings. Here we review recent progress on hidden-state inference, state transition inference, and hierarchical inference processes.

Dimensionality, information and learning in prefrontal cortex.

Learning leads to changes in population patterns of neural activity. In this study we wanted to examine how these changes in patterns of activity affect the dimensionality of neural responses and information about choices. We addressed these questions by carrying out high channel count recordings in dorsal-lateral prefrontal cortex (dlPFC; 768 electrodes) while monkeys performed a two-armed bandit reinforcement learning task. The high channel count recordings allowed us to study population coding while monkeys learned choices between actions or objects. We found that the dimensionality of neural population activity was higher across blocks in which animals learned the values of novel pairs of objects, than across blocks in which they learned the values of actions. The increase in dimensionality with learning in object blocks was related to less shared information across blocks, and therefore patterns of neural activity that were less similar, when compared to learning in action blocks. Furthermore, these differences emerged with learning, and were not a simple function of the choice of a visual image or action. Therefore, learning the values of novel objects increases the dimensionality of neural representations in dlPFC.

Prefrontal Cortex Predicts State Switches during Reversal Learning.

Reinforcement learning allows organisms to predict future outcomes and to update their beliefs about value in the world. The dorsal-lateral prefrontal cortex (dlPFC) integrates information carried by reward circuits, which can be used to infer the current state of the world under uncertainty. Here, we explored the dlPFC computations related to updating current beliefs during stochastic reversal learning. We recorded the activity of populations up to 1,000 neurons, simultaneously, in two male macaques while they executed a two-armed bandit reversal learning task. Behavioral analyses using a Bayesian framework showed that animals inferred reversals and switched their choice preference rapidly, rather than slowly updating choice values, consistent with state inference. Furthermore, dlPFC neural populations accurately encoded choice preference switches. These results suggest that prefrontal neurons dynamically encode decisions associated with Bayesian subjective values, highlighting the role of the PFC in representing a belief about the current state of the world.

Information-Limiting Correlations in Large Neural Populations.

Understanding the neural code requires understanding how populations of neurons code information. Theoretical models predict that information may be limited by correlated noise in large neural populations. Nevertheless, analyses based on tens of neurons have failed to find evidence of saturation. Moreover, some studies have shown that noise correlations can be very small, and therefore may not affect information coding. To determine whether information-limiting correlations exist, we implanted eight Utah arrays in prefrontal cortex (PFC; area 46) of two male macaque monkeys, recording >500 neurons simultaneously. We estimated information in PFC about saccades as a function of ensemble size. Noise correlations were, on average, small (∼10-3). However, information scaled strongly sublinearly with ensemble size. After shuffling trials, destroying noise correlations, information was a linear function of ensemble size. Thus, we provide evidence for the existence of information-limiting noise correlations in large populations of PFC neurons.SIGNIFICANCE STATEMENT Recent theoretical work has shown that even small correlations can limit information if they are "differential correlations," which are difficult to measure directly. However, they can be detected through decoding analyses on recordings from a large number of neurons over a large number of trials. We have achieved both by collecting neural activity in dorsal-lateral prefrontal cortex of macaques using eight microelectrode arrays (768 electrodes), from which we were able to compute accurate information estimates. We show, for the first time, strong evidence for information-limiting correlations. Despite pairwise correlations being small (on the order of 10-3), they affect information coding in populations on the order of 100 s of neurons.

Primate beta oscillations and rhythmic behaviors.

The study of non-human primates in complex behaviors such as rhythm perception and entrainment is critical to understand the neurophysiological basis of human cognition. Next to reviewing the role of beta oscillations in human beat perception, here we discuss the role of primate putaminal oscillatory activity in the control of rhythmic movements that are guided by a sensory metronome or internally gated. The analysis of the local field potentials of the behaving macaques showed that gamma-oscillations reflect local computations associated with stimulus processing of the metronome, whereas beta-activity involves the entrainment of large putaminal circuits, probably in conjunction with other elements of cortico-basal ganglia-thalamo-cortical circuit, during internally driven rhythmic tapping. Thus, this review emphasizes the need of parametric neurophysiological observations in non-human primates that display a well-controlled behavior during high-level cognitive processes.

High channel count single-unit recordings from nonhuman primate frontal cortex.

BACKGROUND: Single unit recording in behaving nonhuman primates is widely used to study the primate central nervous system. However, certain questions cannot be addressed without recording large numbers of neurons simultaneously. Multiple 96-electrode probes can be implanted at one time, but certain problems must be overcome to make this approach practical. NEW METHOD: We describe a series of innovations and practical guidance for implanting and recording from 8 arrays of 96 electrodes (768 electrodes) in the frontal cortex of Macaca mulatta. The methods include an individualized 3D-printed connector mounting platform, sequencing of assembly and surgical steps to minimize surgery time, and interventions to protect electrical connections of the implant. RESULTS: The methodology is robust and was successful in our hands on the first attempt. On average, we were able to isolate hundreds (535.7 and 806.9 in two animals) of high quality units in each session during one month of recording. COMPARISON WITH EXISTING METHODS: To the best of our knowledge, this technique at least doubles the number of Blackrock arrays that have been successfully implanted in single animals. Although each technological component was pre-existing at the time we developed these methods, their amalgamation to solve the problem of high channel count recording is novel. CONCLUSIONS: The implantation of large numbers of electrodes opens new research possibilities. Refinements could lead to even greater capacity.

Effects of Ventral Striatum Lesions on Stimulus-Based versus Action-Based Reinforcement Learning.

Learning the values of actions versus stimuli may depend on separable neural circuits. In the current study, we evaluated the performance of rhesus macaques with ventral striatum (VS) lesions on a two-arm bandit task that had randomly interleaved blocks of stimulus-based and action-based reinforcement learning (RL). Compared with controls, monkeys with VS lesions had deficits in learning to select rewarding images but not rewarding actions. We used a RL model to quantify learning and choice consistency and found that, in stimulus-based RL, the VS lesion monkeys were more influenced by negative feedback and had lower choice consistency than controls. Using a Bayesian model to parse the groups' learning strategies, we also found that VS lesion monkeys defaulted to an action-based choice strategy. Therefore, the VS is involved specifically in learning the value of stimuli, not actions.SIGNIFICANCE STATEMENT Reinforcement learning models of the ventral striatum (VS) often assume that it maintains an estimate of state value. This suggests that it plays a general role in learning whether rewards are assigned based on a chosen action or stimulus. In the present experiment, we examined the effects of VS lesions on monkeys' ability to learn that choosing a particular action or stimulus was more likely to lead to reward. We found that VS lesions caused a specific deficit in the monkeys' ability to discriminate between images with different values, whereas their ability to discriminate between actions with different values remained intact. Our results therefore suggest that the VS plays a specific role in learning to select rewarded stimuli.

Sensorimotor neural dynamics during isochronous tapping in the medial premotor cortex of the macaque.

We determined the response properties of neurons in the primate medial premotor cortex that were classified as sensory or motor during isochronous tapping to a visual or auditory metronome, using different target intervals and three sequential elements in the task. The cell classification was based on a warping transformation, which determined whether the cell activity was statistically aligned to sensory or motor events, finding a large proportion of cells classified as sensory or motor. Two distinctive clusters of sensory cells were observed, i.e. one cell population with short response-onset latencies to the previous stimulus, and another that was probably predicting the occurrence of the next stimuli. These cells were called sensory-driven and stimulus-predicting neurons, respectively. Sensory-driven neurons showed a clear bias towards the visual modality and were more responsive to the first stimulus, with a decrease in activity for the following sequential elements of the metronome. In contrast, stimulus-predicting neurons were bimodal and showed similar response profiles across serial-order elements. Motor cells showed a consecutive activity onset across discrete neural ensembles, generating a rapid succession of activation patterns between the two taps defining a produced interval. The cyclical configuration in activation profiles engaged more motor cells as the serial-order elements progressed across the task, and the rate of cell recruitment over time decreased as a function of the target interval. Our findings support the idea that motor cells were responsible for the rhythmic progression of taps in the task, gaining more importance as the trial advanced, while, simultaneously, the sensory-driven cells lost their functional impact.

ß oscillations are linked to the initiation of sensory-cued movement sequences and the internal guidance of regular tapping in the monkey.

ß oscillations in the basal ganglia have been associated with interval timing. We recorded the putaminal local field potentials (LFPs) from monkeys performing a synchronization-continuation task (SCT) and a serial reaction-time task (RTT), where the animals produced regularly and irregularly paced tapping sequences, respectively. We compared the activation profile of ß oscillations between tasks and found transient bursts of ß activity in both the RTT and SCT. During the RTT, ß power was higher at the beginning of the task, especially when LFPs were aligned to the stimuli. During the SCT, ß was higher during the internally driven continuation phase, especially for tap-aligned LFPs. Interestingly, a set of LFPs showed an initial burst of ß at the beginning of the SCT, similar to the RTT, followed by a decrease in ß oscillations during the synchronization phase, to finally rebound during the continuation phase. The rebound during the continuation phase of the SCT suggests that the corticostriatal circuit is involved in the control of internally driven motor sequences. In turn, the transient bursts of ß activity at the beginning of both tasks suggest that the basal ganglia produce a general initiation signal that engages the motor system in different sequential behaviors.

Neurophysiology of timing in the hundreds of milliseconds: multiple layers of neuronal clocks in the medial premotor areas.

The precise quantification of time in the subsecond scale is critical for many complex behaviors including music and dance appreciation/execution, speech comprehension/articulation, and the performance of many sports. Nevertheless, its neural underpinnings are largely unknown. Recent neurophysiological experiments from our laboratory have shown that the cell activity in the medial premotor areas (MPC) of macaques can represent different aspects of temporal processing during a synchronization-continuation tapping task (SCT). In this task the rhythmic behavior of monkeys was synchronized to a metronome of isochronous stimuli in the hundreds of milliseconds range (synchronization phase), followed by a period where animals internally temporalized their movements (continuation phase). Overall, we found that the time-keeping mechanism in MPC is governed by different layers of neural clocks. Close to the temporal control of movements are two separate populations of ramping cells that code for elapsed or remaining time for a tapping movement during the SCT. Thus, the sensorimotor loops engaged during the task may depend on the cyclic interplay between two neuronal chronometers that quantify in their instantaneous discharge rate the time passed and the remaining time for an action. In addition, we found MPC neurons that are tuned to the duration of produced intervals during the rhythmic task, showing an orderly variation in the average discharge rate as a function of duration. All the tested durations in the subsecond scale were represented in the preferred intervals of the cell population. Most of the interval-tuned cells were also tuned to the ordinal structure of the six intervals produced sequentially in the SCT. Hence, this next level of temporal processing may work as the notes of a musical score, providing information to the timing network about what duration and ordinal element of the sequence are being executed. Finally, we describe how the timing circuit can use a dynamic neural representation of the passage of time and the context in which the intervals are executed by integrating the time-varying activity of populations of cells. These neural population clocks can be defined as distinct trajectories in the multidimensional cell response-space. We provide a hypothesis of how these different levels of neural clocks can interact to constitute a coherent timing machine that controls the rhythmic behavior during the SCT.

Dynamic representation of the temporal and sequential structure of rhythmic movements in the primate medial premotor cortex.

We determined the encoding properties of single cells and the decoding accuracy of cell populations in the medial premotor cortex (MPC) of Rhesus monkeys to represent in a time-varying fashion the duration and serial order of six intervals produced rhythmically during a synchronization-continuation tapping task. We found that MPC represented the temporal and sequential structure of rhythmic movements by activating small ensembles of neurons that encoded the duration or the serial order in rapid succession, so that the pattern of active neurons changed dramatically within each interval. Interestingly, the width of the encoding or decoding function for serial order increased as a function of duration. Finally, we found that the strength of correlation in spontaneous activity of the individual cells varied as a function of the timing of their recruitment. These results demonstrate the existence of dynamic representations in MPC for the duration and serial order of intervals produced rhythmically and suggest that this dynamic code depends on ensembles of interconnected neurons that provide a strong synaptic drive to the next ensemble in a consecutive chain of neural events.

Information processing in the primate basal ganglia during sensory-guided and internally driven rhythmic tapping.

Gamma (γ) and beta (ß) oscillations seem to play complementary functions in the cortico-basal ganglia-thalamo-cortical circuit (CBGT) during motor behavior. We investigated the time-varying changes of the putaminal spiking activity and the spectral power of local field potentials (LFPs) during a task where the rhythmic tapping of monkeys was guided by isochronous stimuli separated by a fixed duration (synchronization phase), followed by a period of internally timed movements (continuation phase). We found that the power of both bands and the discharge rate of cells showed an orderly change in magnitude as a function of the duration and/or the serial order of the intervals executed rhythmically. More LFPs were tuned to duration and/or serial order in the ß- than the γ-band, although different values of preferred features were represented by single cells and by both bands. Importantly, in the LFPs tuned to serial order, there was a strong bias toward the continuation phase for the ß-band when aligned to movements, and a bias toward the synchronization phase for the γ-band when aligned to the stimuli. Our results suggest that γ-oscillations reflect local computations associated with stimulus processing, whereas ß-activity involves the entrainment of large putaminal circuits, probably in conjunction with other elements of CBGT, during internally driven rhythmic tapping.

Monkeys time their pauses of movement and not their movement-kinematics during a synchronization-continuation rhythmic task.

A critical question in tapping behavior is to understand whether the temporal control is exerted on the duration and trajectory of the downward-upward hand movement or on the pause between hand movements. In the present study, we determined the duration of both the movement execution and pauses of monkeys performing a synchronization-continuation task (SCT), using the speed profile of their tapping behavior. We found a linear increase in the variance of pause-duration as a function of interval, while the variance of the motor implementation was relatively constant across intervals. In fact, 96% of the variability of the duration of a complete tapping cycle (pause + movement) was due to the variability of the pause duration. In addition, we performed a Bayesian model selection to determine the effect of interval duration (450-1,000 ms), serial-order (1-6 produced intervals), task phase (sensory cued or internally driven), and marker modality (auditory or visual) on the duration of the movement-pause and tapping movement. The results showed that the most important parameter used to successfully perform the SCT was the control of the pause duration. We also found that the kinematics of the tapping movements was concordant with a stereotyped ballistic control of the hand pressing the push-button. The present findings support the idea that monkeys used an explicit timing strategy to perform the SCT, where a dedicated timing mechanism controlled the duration of the pauses of movement, while also triggered the execution of fixed movements across each interval of the rhythmic sequence.

Rhesus monkeys (Macaca mulatta) detect rhythmic groups in music, but not the beat.

It was recently shown that rhythmic entrainment, long considered a human-specific mechanism, can be demonstrated in a selected group of bird species, and, somewhat surprisingly, not in more closely related species such as nonhuman primates. This observation supports the vocal learning hypothesis that suggests rhythmic entrainment to be a by-product of the vocal learning mechanisms that are shared by several bird and mammal species, including humans, but that are only weakly developed, or missing entirely, in nonhuman primates. To test this hypothesis we measured auditory event-related potentials (ERPs) in two rhesus monkeys (Macaca mulatta), probing a well-documented component in humans, the mismatch negativity (MMN) to study rhythmic expectation. We demonstrate for the first time in rhesus monkeys that, in response to infrequent deviants in pitch that were presented in a continuous sound stream using an oddball paradigm, a comparable ERP component can be detected with negative deflections in early latencies (Experiment 1). Subsequently we tested whether rhesus monkeys can detect gaps (omissions at random positions in the sound stream; Experiment 2) and, using more complex stimuli, also the beat (omissions at the first position of a musical unit, i.e. the 'downbeat'; Experiment 3). In contrast to what has been shown in human adults and newborns (using identical stimuli and experimental paradigm), the results suggest that rhesus monkeys are not able to detect the beat in music. These findings are in support of the hypothesis that beat induction (the cognitive mechanism that supports the perception of a regular pulse from a varying rhythm) is species-specific and absent in nonhuman primates. In addition, the findings support the auditory timing dissociation hypothesis, with rhesus monkeys being sensitive to rhythmic grouping (detecting the start of a rhythmic group), but not to the induced beat (detecting a regularity from a varying rhythm).

Measuring time with different neural chronometers during a synchronization-continuation task.

Temporal information processing is critical for many complex behaviors including speech and music cognition, yet its neural substrate remains elusive. We examined the neurophysiological properties of medial premotor cortex (MPC) of two Rhesus monkeys during the execution of a synchronization-continuation tapping task that includes the basic sensorimotor components of a variety of rhythmic behaviors. We show that time-keeping in the MPC is governed by separate cell populations. One group encoded the time remaining for an action, showing activity whose duration changed as a function of interval duration, reaching a peak at similar magnitudes and times with respect to the movement. The other cell group showed a response that increased in duration or magnitude as a function of the elapsed time from the last movement. Hence, the sensorimotor loops engaged during the task may depend on the cyclic interplay between different neuronal chronometers that quantify the time passed and the remaining time for an action.

Learning and generalization of time production in humans: rules of transfer across modalities and interval durations.

This article investigated both the ability of naive human subjects to learn interval production, as well as the properties of learning generalization across modalities and interval durations that varied systematically from the over-trained interval. Human subjects trained on a 450-, 650-, or 850-ms single-interval production task, using auditory stimuli to define the intervals, showed a significant decrease in performance variability with intensive training. This learning generalized to the visual modality and to non-trained durations following a Gaussian transfer pattern. However, the learning carryover followed different rules, depending on the duration of the trained interval as follows: (1) the dispersion of the generalization curve increased as a function of the trained interval, (2) the generalization pattern was tilted to the right in the visual condition, and (3) the transfer magnitude for 650 ms was less prominent than for the other two intervals. These findings suggest the existence of neural circuits that are tuned to specific time lengths and that show different temporal processing properties depending on their preferred interval duration.

The context of temporal processing is represented in the multidimensional relationships between timing tasks.

In the present study we determined the performance interrelations of ten different tasks that involved the processing of temporal intervals in the subsecond range, using multidimensional analyses. Twenty human subjects executed the following explicit timing tasks: interval categorization and discrimination (perceptual tasks), and single and multiple interval tapping (production tasks). In addition, the subjects performed a continuous circle-drawing task that has been considered an implicit timing paradigm, since time is an emergent property of the produced spatial trajectory. All tasks could be also classified as single or multiple interval paradigms. Auditory or visual markers were used to define the intervals. Performance variability, a measure that reflects the temporal and non-temporal processes for each task, was used to construct a dissimilarity matrix that quantifies the distances between pairs of tasks. Hierarchical clustering and multidimensional scaling were carried out on the dissimilarity matrix, and the results showed a prominent segregation of explicit and implicit timing tasks, and a clear grouping between single and multiple interval paradigms. In contrast, other variables such as the marker modality were not as crucial to explain the performance between tasks. Thus, using this methodology we revealed a probable functional arrangement of neural systems engaged during different timing behaviors.