Using temperature to analyze the neural basis of a time-based decision.

The basal ganglia are thought to contribute to decision-making and motor control. These functions are critically dependent on timing information, which can be extracted from the evolving state of neural populations in their main input structure, the striatum. However, it is debated whether striatal activity underlies latent, dynamic decision processes or kinematics of overt movement. Here, we measured the impact of temperature on striatal population activity and the behavior of rats, and compared the observed effects with neural activity and behavior collected in multiple versions of a temporal categorization task. Cooling caused dilation, and warming contraction, of both neural activity and patterns of judgment in time, mimicking endogenous decision-related variability in striatal activity. However, temperature did not similarly affect movement kinematics. These data provide compelling evidence that the timecourse of evolving striatal activity dictates the speed of a latent process that is used to guide choices, but not continuous motor control. More broadly, they establish temporal scaling of population activity as a likely neural basis for variability in timing behavior.

A robust role for motor cortex.

The role of motor cortex in non-primate mammals remains unclear. More than a century of stimulation, anatomical and electrophysiological studies has implicated neural activity in this region with all kinds of movement. However, following the removal of motor cortex, rats retain most of their adaptive behaviors, including previously learned skilled movements. Here we revisit these two conflicting views of motor cortex and present a new behavior assay, challenging animals to respond to unexpected situations while navigating a dynamic obstacle course. Surprisingly, rats with motor cortical lesions show clear impairments facing an unexpected collapse of the obstacles, while showing no impairment with repeated trials in many motor and cognitive metrics of performance. We propose a new role for motor cortex: extending the robustness of sub-cortical movement systems, specifically to unexpected situations demanding rapid motor responses adapted to environmental context. The implications of this idea for current and future research are discussed.

Action suppression reveals opponent parallel control via striatal circuits.

The direct and indirect pathways of the basal ganglia are classically thought to promote and suppress action, respectively1. However, the observed co-activation of striatal direct and indirect medium spiny neurons2 (dMSNs and iMSNs, respectively) has challenged this view. Here we study these circuits in mice performing an interval categorization task that requires a series of self-initiated and cued actions and, critically, a sustained period of dynamic action suppression. Although movement produced the co-activation of iMSNs and dMSNs in the sensorimotor, dorsolateral striatum (DLS), fibre photometry and photo-identified electrophysiological recordings revealed signatures of functional opponency between the two pathways during action suppression. Notably, optogenetic inhibition showed that DLS circuits were largely engaged to suppress-and not promote-action. Specifically, iMSNs on a given hemisphere were dynamically engaged to suppress tempting contralateral action. To understand how such regionally specific circuit function arose, we constructed a computational reinforcement learning model that reproduced key features of behaviour, neural activity and optogenetic inhibition. The model predicted that parallel striatal circuits outside the DLS learned the action-promoting functions, generating the temptation to act. Consistent with this, optogenetic inhibition experiments revealed that dMSNs in the associative, dorsomedial striatum, in contrast to those in the DLS, promote contralateral actions. These data highlight how opponent interactions between multiple circuit- and region-specific basal ganglia processes can lead to behavioural control, and establish a critical role for the sensorimotor indirect pathway in the proactive suppression of tempting actions.

Efficient coding of cognitive variables underlies dopamine response and choice behavior.

Reward expectations based on internal knowledge of the external environment are a core component of adaptive behavior. However, internal knowledge may be inaccurate or incomplete due to errors in sensory measurements. Some features of the environment may also be encoded inaccurately to minimize representational costs associated with their processing. In this study, we investigated how reward expectations are affected by features of internal representations by studying behavior and dopaminergic activity while mice make time-based decisions. We show that several possible representations allow a reinforcement learning agent to model animals' overall performance during the task. However, only a small subset of highly compressed representations simultaneously reproduced the co-variability in animals' choice behavior and dopaminergic activity. Strikingly, these representations predict an unusual distribution of response times that closely match animals' behavior. These results inform how constraints of representational efficiency may be expressed in encoding representations of dynamic cognitive variables used for reward-based computations.

Dopamine gives credit where credit is due.

Heterogeneity is an increasingly appreciated feature of dopamine signaling in the striatum. Hamid et al. (2021) leverage a variety of imaging techniques to reveal striking spatiotemporal patterns of dopamine signals in mouse dorsal striatum. Time will tell what this means for reinforcement learning in the brain.

The Neural Basis of Timing: Distributed Mechanisms for Diverse Functions.

Timing is critical to most forms of learning, behavior, and sensory-motor processing. Converging evidence supports the notion that, precisely because of its importance across a wide range of brain functions, timing relies on intrinsic and general properties of neurons and neural circuits; that is, the brain uses its natural cellular and network dynamics to solve a diversity of temporal computations. Many circuits have been shown to encode elapsed time in dynamically changing patterns of neural activity-so-called population clocks. But temporal processing encompasses a wide range of different computations, and just as there are different circuits and mechanisms underlying computations about space, there are a multitude of circuits and mechanisms underlying the ability to tell time and generate temporal patterns.

The many worlds hypothesis of dopamine prediction error: implications of a parallel circuit architecture in the basal ganglia.

Computational models of reinforcement learning (RL) strive to produce behavior that maximises reward, and thus allow software or robots to behave adaptively [1]. At the core of RL models is a learned mapping between 'states'-situations or contexts that an agent might encounter in the world-and actions. A wealth of physiological and anatomical data suggests that the basal ganglia (BG) is important for learning these mappings [2,3]. However, the computations performed by specific circuits are unclear. In this brief review, we highlight recent work concerning the anatomy and physiology of BG circuits that suggest refinements in our understanding of computations performed by the basal ganglia. We focus on one important component of basal ganglia circuitry, midbrain dopamine neurons, drawing attention to data that has been cast as supporting or departing from the RL framework that has inspired experiments in basal ganglia research over the past two decades. We suggest that the parallel circuit architecture of the BG might be expected to produce variability in the response properties of different dopamine neurons, and that variability in response profile may not reflect variable functions, but rather different arguments that serve as inputs to a common function: the computation of prediction error.

High-Yield Methods for Accurate Two-Alternative Visual Psychophysics in Head-Fixed Mice.

Research in neuroscience increasingly relies on the mouse, a mammalian species that affords unparalleled genetic tractability and brain atlases. Here, we introduce high-yield methods for probing mouse visual decisions. Mice are head-fixed, facilitating repeatable visual stimulation, eye tracking, and brain access. They turn a steering wheel to make two alternative choices, forced or unforced. Learning is rapid thanks to intuitive coupling of stimuli to wheel position. The mouse decisions deliver high-quality psychometric curves for detection and discrimination and conform to the predictions of a simple probabilistic observer model. The task is readily paired with two-photon imaging of cortical activity. Optogenetic inactivation reveals that the task requires mice to use their visual cortex. Mice are motivated to perform the task by fluid reward or optogenetic stimulation of dopamine neurons. This stimulation elicits a larger number of trials and faster learning. These methods provide a platform to accurately probe mouse vision and its neural basis.

Distinct Roles for the Amygdala and Orbitofrontal Cortex in Representing the Relative Amount of Expected Reward.

The same reward can possess different motivational meaning depending upon its magnitude relative to other rewards. To study the neurophysiological mechanisms mediating assignment of motivational meaning, we recorded the activity of neurons in the amygdala and orbitofrontal cortex (OFC) of monkeys during a Pavlovian task in which the relative amount of liquid reward associated with one conditioned stimulus (CS) was manipulated by changing the reward amount associated with a second CS. Anticipatory licking tracked relative reward magnitude, implying that monkeys integrated information about recent rewards to adjust the motivational meaning of a CS. Upon changes in relative reward magnitude, neural responses to reward-predictive cues updated more rapidly in OFC than amygdala, and activity in OFC but not the amygdala was modulated by recent reward history. These results highlight a distinction between the amygdala and OFC in assessing reward history to support the flexible assignment of motivational meaning to sensory cues.

Midbrain dopamine neurons control judgment of time.

Our sense of time is far from constant. For instance, time flies when we are having fun, and it slows to a trickle when we are bored. Midbrain dopamine neurons have been implicated in variable time estimation. However, a direct link between signals carried by dopamine neurons and temporal judgments is lacking. We measured and manipulated the activity of dopamine neurons as mice judged the duration of time intervals. We found that pharmacogenetic suppression of dopamine neurons decreased behavioral sensitivity to time and that dopamine neurons encoded information about trial-to-trial variability in time estimates. Last, we found that transient activation or inhibition of dopamine neurons was sufficient to slow down or speed up time estimation, respectively. Dopamine neuron activity thus reflects and can directly control the judgment of time.

Striatal dynamics explain duration judgments.

The striatum is an input structure of the basal ganglia implicated in several time-dependent functions including reinforcement learning, decision making, and interval timing. To determine whether striatal ensembles drive subjects' judgments of duration, we manipulated and recorded from striatal neurons in rats performing a duration categorization psychophysical task. We found that the dynamics of striatal neurons predicted duration judgments, and that simultaneously recorded ensembles could judge duration as well as the animal. Furthermore, striatal neurons were necessary for duration judgments, as muscimol infusions produced a specific impairment in animals' duration sensitivity. Lastly, we show that time as encoded by striatal populations ran faster or slower when rats judged a duration as longer or shorter, respectively. These results demonstrate that the speed with which striatal population state changes supports the fundamental ability of animals to judge the passage of time.

Bonsai: an event-based framework for processing and controlling data streams.

The design of modern scientific experiments requires the control and monitoring of many different data streams. However, the serial execution of programming instructions in a computer makes it a challenge to develop software that can deal with the asynchronous, parallel nature of scientific data. Here we present Bonsai, a modular, high-performance, open-source visual programming framework for the acquisition and online processing of data streams. We describe Bonsai's core principles and architecture and demonstrate how it allows for the rapid and flexible prototyping of integrated experimental designs in neuroscience. We specifically highlight some applications that require the combination of many different hardware and software components, including video tracking of behavior, electrophysiology and closed-loop control of stimulation.

A scalable population code for time in the striatum.

To guide behavior and learn from its consequences, the brain must represent time over many scales. Yet, the neural signals used to encode time in the seconds-to-minute range are not known. The striatum is a major input area of the basal ganglia associated with learning and motor function. Previous studies have also shown that the striatum is necessary for normal timing behavior. To address how striatal signals might be involved in timing, we recorded from striatal neurons in rats performing an interval timing task. We found that neurons fired at delays spanning tens of seconds and that this pattern of responding reflected the interaction between time and the animals' ongoing sensorimotor state. Surprisingly, cells rescaled responses in time when intervals changed, indicating that striatal populations encoded relative time. Moreover, time estimates decoded from activity predicted timing behavior as animals adjusted to new intervals, and disrupting striatal function led to a decrease in timing performance. These results suggest that striatal activity forms a scalable population code for time, providing timing signals that animals use to guide their actions.

Big behavioral data: psychology, ethology and the foundations of neuroscience.

Behavior is a unifying organismal process where genes, neural function, anatomy and environment converge and interrelate. Here we review the current state and discuss the future effect of accelerating advances in technology for behavioral studies, focusing on rodents as an example. We frame our perspective in three dimensions: the degree of experimental constraint, dimensionality of data and level of description. We argue that 'big behavioral data' presents challenges proportionate to its promise and describe how these challenges might be met through opportunities afforded by the two rival conceptual legacies of twentieth century behavioral science, ethology and psychology. We conclude that, although 'more is not necessarily better', copious, quantitative and open behavioral data has the potential to transform and unify these two disciplines and to solidify the foundations of others, including neuroscience, but only if the development of new theoretical frameworks and improved experimental designs matches the technological progress.

Ongoing behavior predicts perceptual report of interval duration.

The ability to estimate the passage of time is essential for adaptive behavior in complex environments. Yet, it is not known how the brain encodes time over the durations necessary to explain animal behavior. Under temporally structured reinforcement schedules, animals tend to develop temporally structured behavior, and interval timing has been suggested to be accomplished by learning sequences of behavioral states. If this is true, trial to trial fluctuations in behavioral sequences should be predictive of fluctuations in time estimation. We trained rodents in an duration categorization task while continuously monitoring their behavior with a high speed camera. Animals developed highly reproducible behavioral sequences during the interval being timed. Moreover, those sequences were often predictive of perceptual report from early in the trial, providing support to the idea that animals may use learned behavioral patterns to estimate the duration of time intervals. To better resolve the issue, we propose that continuous and simultaneous behavioral and neural monitoring will enable identification of neural activity related to time perception that is not explained by ongoing behavior.

Functional circuits and anatomical distribution of response properties in the primate amygdala.

Recent electrophysiological studies on the primate amygdala have advanced our understanding of how individual neurons encode information relevant to emotional processes, but it remains unclear how these neurons are functionally and anatomically organized. To address this, we analyzed cross-correlograms of amygdala spike trains recorded during a task in which monkeys learned to associate novel images with rewarding and aversive outcomes. Using this task, we have recently described two populations of amygdala neurons: one that responds more strongly to images predicting reward (positive value-coding), and another that responds more strongly to images predicting an aversive stimulus (negative value-coding). Here, we report that these neural populations are organized into distinct, but anatomically intermingled, appetitive and aversive functional circuits, which are dynamically modulated as animals used the images to predict outcomes. Furthermore, we report that responses to sensory stimuli are prevalent in the lateral amygdala, and are also prevalent in the medial amygdala for sensory stimuli that are emotionally significant. The circuits identified here could potentially mediate valence-specific emotional behaviors thought to involve the amygdala.

Moment-to-moment tracking of state value in the amygdala.

As an organism interacts with the world, how good or bad things are at the moment, the value of the current state of the organism, is an important parameter that is likely to be encoded in the brain. As the environment changes and new stimuli appear, estimates of state value must be updated to support appropriate responses and learning. Indeed, many models of reinforcement learning posit representations of state value. We examined how the brain mediates this process by recording amygdala neural activity while monkeys performed a trace-conditioning task requiring fixation. The presentation of different stimuli induced state transitions; these stimuli included unconditioned stimuli (USs) (liquid rewards and aversive air puffs), newly learned reinforcement-predictive visual stimuli [conditioned stimuli (CSs)], and familiar stimuli long associated with reinforcement [fixation point (FP)]. The FP had a positive value to monkeys, because they chose to foveate it to initiate trials. Different populations of amygdala neurons tracked the positive or negative value of the current state, regardless of whether state transitions were caused by the FP, CSs, or USs. Positive value-coding neurons increased their firing during the fixation interval and fired more strongly after rewarded CSs and rewards than after punished CSs and air puffs. Negative value-coding neurons did the opposite, decreasing their firing during the fixation interval and firing more strongly after punished CSs and air puffs than after rewarded CSs and rewards. This representation of state value could underlie how the amygdala helps coordinate cognitive, emotional, and behavioral responses depending on the value of one's state.

Flexible neural representations of value in the primate brain.

The amygdala and orbitofrontal cortex (OFC) are often thought of as components of a neural circuit that assigns affective significance--or value--to sensory stimuli so as to anticipate future events and adjust behavioral and physiological responses. Much recent work has been aimed at understanding the distinct contributions of the amygdala and OFC to these processes, but a detailed understanding of the physiological mechanisms underlying learning about value remains lacking. To gain insight into these processes, we have focused initially on characterizing the neural signals of the primate amygdala, and more recently of the primate OFC, during appetitive and aversive reinforcement learning procedures. We have employed a classical conditioning procedure whereby monkeys form associations between visual stimuli and rewards or aversive stimuli. After learning these initial associations, we reverse the stimulus-reinforcement contingencies, and monkeys learn these new associations. We have discovered that separate populations of neurons in the amygdala represent the positive and negative value of conditioned visual stimuli. This representation of value updates rapidly upon image value reversal, as fast as monkeys learn, often within a single trial. We suggest that representations of value in the amygdala may change through multiple interrelated mechanisms: some that arise from fairly simple Hebbian processes, and others that may involve gated inputs from other brain areas, such as the OFC.