Dynamics of striatal action selection and reinforcement learning.

Spiny projection neurons (SPNs) in dorsal striatum are often proposed as a locus of reinforcement learning in the basal ganglia. Here, we identify and resolve a fundamental inconsistency between striatal reinforcement learning models and known SPN synaptic plasticity rules. Direct-pathway (dSPN) and indirect-pathway (iSPN) neurons, which promote and suppress actions, respectively, exhibit synaptic plasticity that reinforces activity associated with elevated or suppressed dopamine release. We show that iSPN plasticity prevents successful learning, as it reinforces activity patterns associated with negative outcomes. However, this pathological behavior is reversed if functionally opponent dSPNs and iSPNs, which promote and suppress the current behavior, are simultaneously activated by efferent input following action selection. This prediction is supported by striatal recordings and contrasts with prior models of SPN representations. In our model, learning and action selection signals can be multiplexed without interference, enabling learning algorithms beyond those of standard temporal difference models.

A groom with a view.

Mapping mouse grooming episodes to neural activity shows that striatal cells deep in the brain collectively represent key aspects of self-grooming.

Hidden behavioral fingerprints in epilepsy.

Epilepsy is a major disorder affecting millions of people. Although modern electrophysiological and imaging approaches provide high-resolution access to the multi-scale brain circuit malfunctions in epilepsy, our understanding of how behavior changes with epilepsy has remained rudimentary. As a result, screening for new therapies for children and adults with devastating epilepsies still relies on the inherently subjective, semi-quantitative assessment of a handful of pre-selected behavioral signs of epilepsy in animal models. Here, we use machine learning-assisted 3D video analysis to reveal hidden behavioral phenotypes in mice with acquired and genetic epilepsies and track their alterations during post-insult epileptogenesis and in response to anti-epileptic drugs. These results show the persistent reconfiguration of behavioral fingerprints in epilepsy and indicate that they can be employed for rapid, automated anti-epileptic drug testing at scale.

Spontaneous behaviour is structured by reinforcement without explicit reward.

Spontaneous animal behaviour is built from action modules that are concatenated by the brain into sequences1,2. However, the neural mechanisms that guide the composition of naturalistic, self-motivated behaviour remain unknown. Here we show that dopamine systematically fluctuates in the dorsolateral striatum (DLS) as mice spontaneously express sub-second behavioural modules, despite the absence of task structure, sensory cues or exogenous reward. Photometric recordings and calibrated closed-loop optogenetic manipulations during open field behaviour demonstrate that DLS dopamine fluctuations increase sequence variation over seconds, reinforce the use of associated behavioural modules over minutes, and modulate the vigour with which modules are expressed, without directly influencing movement initiation or moment-to-moment kinematics. Although the reinforcing effects of optogenetic DLS dopamine manipulations vary across behavioural modules and individual mice, these differences are well predicted by observed variation in the relationships between endogenous dopamine and module use. Consistent with the possibility that DLS dopamine fluctuations act as a teaching signal, mice build sequences during exploration as if to maximize dopamine. Together, these findings suggest a model in which the same circuits and computations that govern action choices in structured tasks have a key role in sculpting the content of unconstrained, high-dimensional, spontaneous behaviour.

Striatal dopamine explains novelty-induced behavioral dynamics and individual variability in threat prediction.

Animals both explore and avoid novel objects in the environment, but the neural mechanisms that underlie these behaviors and their dynamics remain uncharacterized. Here, we used multi-point tracking (DeepLabCut) and behavioral segmentation (MoSeq) to characterize the behavior of mice freely interacting with a novel object. Novelty elicits a characteristic sequence of behavior, starting with investigatory approach and culminating in object engagement or avoidance. Dopamine in the tail of the striatum (TS) suppresses engagement, and dopamine responses were predictive of individual variability in behavior. Behavioral dynamics and individual variability are explained by a reinforcement-learning (RL) model of threat prediction in which behavior arises from a novelty-induced initial threat prediction (akin to "shaping bonus") and a threat prediction that is learned through dopamine-mediated threat prediction errors. These results uncover an algorithmic similarity between reward- and threat-related dopamine sub-systems.

Flexible scaling and persistence of social vocal communication.

Innate vocal sounds such as laughing, screaming or crying convey one's feelings to others. In many species, including humans, scaling the amplitude and duration of vocalizations is essential for effective social communication1-3. In mice, female scent triggers male mice to emit innate courtship ultrasonic vocalizations (USVs)4,5. However, whether mice flexibly scale their vocalizations and how neural circuits are structured to generate flexibility remain largely unknown. Here we identify mouse neurons from the lateral preoptic area (LPOA) that express oestrogen receptor 1 (LPOAESR1 neurons) and, when activated, elicit the complete repertoire of USV syllables emitted during natural courtship. Neural anatomy and functional data reveal a two-step, di-synaptic circuit motif in which primary long-range inhibitory LPOAESR1 neurons relieve a clamp of local periaqueductal grey (PAG) inhibition, enabling excitatory PAG USV-gating neurons to trigger vocalizations. We find that social context shapes a wide range of USV amplitudes and bout durations. This variability is absent when PAG neurons are stimulated directly; PAG-evoked vocalizations are time-locked to neural activity and stereotypically loud. By contrast, increasing the activity of LPOAESR1 neurons scales the amplitude of vocalizations, and delaying the recovery of the inhibition clamp prolongs USV bouts. Thus, the LPOA disinhibition motif contributes to flexible loudness and the duration and persistence of bouts, which are key aspects of effective vocal social communication.

Revealing the structure of pharmacobehavioral space through motion sequencing.

Understanding how genes, drugs and neural circuits influence behavior requires the ability to effectively organize information about similarities and differences within complex behavioral datasets. Motion Sequencing (MoSeq) is an ethologically inspired behavioral analysis method that identifies modular components of three-dimensional mouse body language called 'syllables'. Here, we show that MoSeq effectively parses behavioral differences and captures similarities elicited by a panel of neuroactive and psychoactive drugs administered to a cohort of nearly 700 mice. MoSeq identifies syllables that are characteristic of individual drugs, a finding we leverage to reveal specific on- and off-target effects of both established and candidate therapeutics in a mouse model of autism spectrum disorder. These results demonstrate that MoSeq can meaningfully organize large-scale behavioral data, illustrate the power of a fundamentally modular description of behavior and suggest that behavioral syllables represent a new class of druggable target.

The Striatum Organizes 3D Behavior via Moment-to-Moment Action Selection.

Many naturalistic behaviors are built from modular components that are expressed sequentially. Although striatal circuits have been implicated in action selection and implementation, the neural mechanisms that compose behavior in unrestrained animals are not well understood. Here, we record bulk and cellular neural activity in the direct and indirect pathways of dorsolateral striatum (DLS) as mice spontaneously express action sequences. These experiments reveal that DLS neurons systematically encode information about the identity and ordering of sub-second 3D behavioral motifs; this encoding is facilitated by fast-timescale decorrelations between the direct and indirect pathways. Furthermore, lesioning the DLS prevents appropriate sequence assembly during exploratory or odor-evoked behaviors. By characterizing naturalistic behavior at neural timescales, these experiments identify a code for elemental 3D pose dynamics built from complementary pathway dynamics, support a role for DLS in constructing meaningful behavioral sequences, and suggest models for how actions are sculpted over time.

A fast and accurate zebra finch syllable detector.

The song of the adult male zebra finch is strikingly stereotyped. Efforts to understand motor output, pattern generation, and learning have taken advantage of this consistency by investigating the bird's ability to modify specific parts of song under external cues, and by examining timing relationships between neural activity and vocal output. Such experiments require that precise moments during song be identified in real time as the bird sings. Various syllable-detection methods exist, but many require special hardware, software, and know-how, and details on their implementation and performance are scarce. We present an accurate, versatile, and fast syllable detector that can control hardware at precisely timed moments during zebra finch song. Many moments during song can be isolated and detected with false negative and false positive rates well under 1% and 0.005% respectively. The detector can run on a stock Mac Mini with triggering delay of less than a millisecond and a jitter of σ ≈ 2 milliseconds.

Dynamic illumination of spatially restricted or large brain volumes via a single tapered optical fiber.

Optogenetics promises precise spatiotemporal control of neural processes using light. However, the spatial extent of illumination within the brain is difficult to control and cannot be adjusted using standard fiber optics. We demonstrate that optical fibers with tapered tips can be used to illuminate either spatially restricted or large brain volumes. Remotely adjusting the light input angle to the fiber varies the light-emitting portion of the taper over several millimeters without movement of the implant. We use this mode to activate dorsal versus ventral striatum of individual mice and reveal different effects of each manipulation on motor behavior. Conversely, injecting light over the full numerical aperture of the fiber results in light emission from the entire taper surface, achieving broader and more efficient optogenetic activation of neurons, compared to standard flat-faced fiber stimulation. Thus, tapered fibers permit focal or broad illumination that can be precisely and dynamically matched to experimental needs.

Unstable neurons underlie a stable learned behavior.

Motor skills can be maintained for decades, but the biological basis of this memory persistence remains largely unknown. The zebra finch, for example, sings a highly stereotyped song that is stable for years, but it is not known whether the precise neural patterns underlying song are stable or shift from day to day. Here we demonstrate that the population of projection neurons coding for song in the premotor nucleus, HVC, change from day to day. The most dramatic shifts occur over intervals of sleep. In contrast to the transient participation of excitatory neurons, ensemble measurements dominated by inhibition persist unchanged even after damage to downstream motor nerves. These observations offer a principle of motor stability: spatiotemporal patterns of inhibition can maintain a stable scaffold for motor dynamics while the population of principal neurons that directly drive behavior shift from one day to the next.

Mesoscopic patterns of neural activity support songbird cortical sequences.

Time-locked sequences of neural activity can be found throughout the vertebrate forebrain in various species and behavioral contexts. From "time cells" in the hippocampus of rodents to cortical activity controlling movement, temporal sequence generation is integral to many forms of learned behavior. However, the mechanisms underlying sequence generation are not well known. Here, we describe a spatial and temporal organization of the songbird premotor cortical microcircuit that supports sparse sequences of neural activity. Multi-channel electrophysiology and calcium imaging reveal that neural activity in premotor cortex is correlated with a length scale of 100 µm. Within this length scale, basal-ganglia-projecting excitatory neurons, on average, fire at a specific phase of a local 30 Hz network rhythm. These results show that premotor cortical activity is inhomogeneous in time and space, and that a mesoscopic dynamical pattern underlies the generation of the neural sequences controlling song.

A carbon-fiber electrode array for long-term neural recording.

OBJECTIVE: Chronic neural recording in behaving animals is an essential method for studies of neural circuit function. However, stable recordings from small, densely packed neurons remains challenging, particularly over time-scales relevant for learning. APPROACH: We describe an assembly method for a 16-channel electrode array consisting of carbon fibers (<5 µm diameter) individually insulated with Parylene-C and fire-sharpened. The diameter of the array is approximately 26 µm along the full extent of the implant. MAIN RESULTS: Carbon fiber arrays were tested in HVC (used as a proper name), a song motor nucleus, of singing zebra finches where individual neurons discharge with temporally precise patterns. Previous reports of activity in this population of neurons have required the use of high impedance electrodes on movable microdrives. Here, the carbon fiber electrodes provided stable multi-unit recordings over time-scales of months. Spike-sorting indicated that the multi-unit signals were dominated by one, or a small number of cells. Stable firing patterns during singing confirmed the stability of these clusters over time-scales of months. In addition, from a total of 10 surgeries, 16 projection neurons were found. This cell type is characterized by sparse stereotyped firing patterns, providing unambiguous confirmation of single cell recordings. SIGNIFICANCE: Carbon fiber electrode bundles may provide a scalable solution for long-term neural recordings of densely packed neurons.

Long-range order in canary song.

Bird songs range in form from the simple notes of a Chipping Sparrow to the rich performance of the nightingale. Non-adjacent correlations can be found in the syntax of some birdsongs, indicating that the choice of what to sing next is determined not only by the current syllable, but also by previous syllables sung. Here we examine the song of the domesticated canary, a complex singer whose song consists of syllables, grouped into phrases that are arranged in flexible sequences. Phrases are defined by a fundamental time-scale that is independent of the underlying syllable duration. We show that the ordering of phrases is governed by long-range rules: the choice of what phrase to sing next in a given context depends on the history of the song, and for some syllables, highly specific rules produce correlations in song over timescales of up to ten seconds. The neural basis of these long-range correlations may provide insight into how complex behaviors are assembled from more elementary, stereotyped modules.

The song must go on: resilience of the songbird vocal motor pathway.

Stereotyped sequences of neural activity underlie learned vocal behavior in songbirds; principle neurons in the cortical motor nucleus HVC fire in stereotyped sequences with millisecond precision across multiple renditions of a song. The geometry of neural connections underlying these sequences is not known in detail though feed-forward chains are commonly assumed in theoretical models of sequential neural activity. In songbirds, a well-defined cortical-thalamic motor circuit exists but little is known the fine-grain structure of connections within each song nucleus. To examine whether the structure of song is critically dependent on long-range connections within HVC, we bilaterally transected the nucleus along the anterior-posterior axis in normal-hearing and deafened birds. The disruption leads to a slowing of song as well as an increase in acoustic variability. These effects are reversed on a time-scale of days even in deafened birds or in birds that are prevented from singing post-transection. The stereotyped song of zebra finches includes acoustic details that span from milliseconds to seconds--one of the most precise learned behaviors in the animal kingdom. This detailed motor pattern is resilient to disruption of connections at the cortical level, and the details of song variability and duration are maintained by offline homeostasis of the song circuit.