A virtual rodent predicts the structure of neural activity across behaviours.

Animals have exquisite control of their bodies, allowing them to perform a diverse range of behaviours. How such control is implemented by the brain, however, remains unclear. Advancing our understanding requires models that can relate principles of control to the structure of neural activity in behaving animals. Here, to facilitate this, we built a 'virtual rodent', in which an artificial neural network actuates a biomechanically realistic model of the rat1 in a physics simulator2. We used deep reinforcement learning3-5 to train the virtual agent to imitate the behaviour of freely moving rats, thus allowing us to compare neural activity recorded in real rats to the network activity of a virtual rodent mimicking their behaviour. We found that neural activity in the sensorimotor striatum and motor cortex was better predicted by the virtual rodent's network activity than by any features of the real rat's movements, consistent with both regions implementing inverse dynamics6. Furthermore, the network's latent variability predicted the structure of neural variability across behaviours and afforded robustness in a way consistent with the minimal intervention principle of optimal feedback control7. These results demonstrate how physical simulation of biomechanically realistic virtual animals can help interpret the structure of neural activity across behaviour and relate it to theoretical principles of motor control.

The Role of Variability in Motor Learning.

Trial-to-trial variability in the execution of movements and motor skills is ubiquitous and widely considered to be the unwanted consequence of a noisy nervous system. However, recent studies have suggested that motor variability may also be a feature of how sensorimotor systems operate and learn. This view, rooted in reinforcement learning theory, equates motor variability with purposeful exploration of motor space that, when coupled with reinforcement, can drive motor learning. Here we review studies that explore the relationship between motor variability and motor learning in both humans and animal models. We discuss neural circuit mechanisms that underlie the generation and regulation of motor variability and consider the implications that this work has for our understanding of motor learning.

Geometric deep learning enables 3D kinematic profiling across species and environments.

Comprehensive descriptions of animal behavior require precise three-dimensional (3D) measurements of whole-body movements. Although two-dimensional approaches can track visible landmarks in restrictive environments, performance drops in freely moving animals, due to occlusions and appearance changes. Therefore, we designed DANNCE to robustly track anatomical landmarks in 3D across species and behaviors. DANNCE uses projective geometry to construct inputs to a convolutional neural network that leverages learned 3D geometric reasoning. We trained and benchmarked DANNCE using a dataset of nearly seven million frames that relates color videos and rodent 3D poses. In rats and mice, DANNCE robustly tracked dozens of landmarks on the head, trunk, and limbs of freely moving animals in naturalistic settings. We extended DANNCE to datasets from rat pups, marmosets, and chickadees, and demonstrate quantitative profiling of behavioral lineage during development.

Acute off-target effects of neural circuit manipulations.

Rapid and reversible manipulations of neural activity in behaving animals are transforming our understanding of brain function. An important assumption underlying much of this work is that evoked behavioural changes reflect the function of the manipulated circuits. We show that this assumption is problematic because it disregards indirect effects on the independent functions of downstream circuits. Transient inactivations of motor cortex in rats and nucleus interface (Nif) in songbirds severely degraded task-specific movement patterns and courtship songs, respectively, which are learned skills that recover spontaneously after permanent lesions of the same areas. We resolve this discrepancy in songbirds, showing that Nif silencing acutely affects the function of HVC, a downstream song control nucleus. Paralleling song recovery, the off-target effects resolved within days of Nif lesions, a recovery consistent with homeostatic regulation of neural activity in HVC. These results have implications for interpreting transient circuit manipulations and for understanding recovery after brain lesions.

Mapping the landscape of social behavior.

Social interaction is integral to animal behavior. However, we lack tools to describe it with quantitative rigor, limiting our understanding of its principles and neuropsychiatric disorders, like autism, that perturb it. Here, we present a technique for high-resolution 3D tracking of postural dynamics and social touch in freely interacting animals, solving the challenging subject occlusion and part assignment problems using 3D geometric reasoning, graph neural networks, and semi-supervised learning. We collected over 140 million 3D postures in interacting rodents, featuring new monogenic autism rat lines lacking reports of social behavioral phenotypes. Using a novel multi-scale embedding approach, we identified a rich landscape of stereotyped actions, interactions, synchrony, and body contact. This enhanced phenotyping revealed a spectrum of changes in autism models and in response to amphetamine that were inaccessible to conventional measurements. Our framework and large library of interactions will greatly facilitate studies of social behaviors and their neurobiological underpinnings.

Dissociating the contributions of sensorimotor striatum to automatic and visually guided motor sequences.

The ability to sequence movements in response to new task demands enables rich and adaptive behavior. However, such flexibility is computationally costly and can result in halting performances. Practicing the same motor sequence repeatedly can render its execution precise, fast and effortless, that is, 'automatic'. The basal ganglia are thought to underlie both types of sequence execution, yet whether and how their contributions differ is unclear. We parse this in rats trained to perform the same motor sequence instructed by cues and in a self-initiated overtrained, or 'automatic,' condition. Neural recordings in the sensorimotor striatum revealed a kinematic code independent of the execution mode. Although lesions reduced the movement speed and affected detailed kinematics similarly, they disrupted high-level sequence structure for automatic, but not visually guided, behaviors. These results suggest that the basal ganglia are essential for 'automatic' motor skills that are defined in terms of continuous kinematics, but can be dispensable for discrete motor sequences guided by sensory cues.

Motor cortex is required for flexible but not automatic motor sequences.

How motor cortex contributes to motor sequence execution is much debated, with studies supporting disparate views. Here we probe the degree to which motor cortex's engagement depends on task demands, specifically whether its role differs for highly practiced, or 'automatic', sequences versus flexible sequences informed by external events. To test this, we trained rats to generate three-element motor sequences either by overtraining them on a single sequence or by having them follow instructive visual cues. Lesioning motor cortex revealed that it is necessary for flexible cue-driven motor sequences but dispensable for single automatic behaviors trained in isolation. However, when an automatic motor sequence was practiced alongside the flexible task, it became motor cortex-dependent, suggesting that subcortical consolidation of an automatic motor sequence is delayed or prevented when the same sequence is produced also in a flexible context. A simple neural network model recapitulated these results and explained the underlying circuit mechanisms. Our results critically delineate the role of motor cortex in motor sequence execution, describing the condition under which it is engaged and the functions it fulfills, thus reconciling seemingly conflicting views about motor cortex's role in motor sequence generation.

Catalyzing next-generation Artificial Intelligence through NeuroAI.

Neuroscience has long been an essential driver of progress in artificial intelligence (AI). We propose that to accelerate progress in AI, we must invest in fundamental research in NeuroAI. A core component of this is the embodied Turing test, which challenges AI animal models to interact with the sensorimotor world at skill levels akin to their living counterparts. The embodied Turing test shifts the focus from those capabilities like game playing and language that are especially well-developed or uniquely human to those capabilities - inherited from over 500 million years of evolution - that are shared with all animals. Building models that can pass the embodied Turing test will provide a roadmap for the next generation of AI.

Learning-induced changes in the neural circuits underlying motor sequence execution.

As the old adage goes: practice makes perfect. Yet, the neural mechanisms by which rote repetition transforms a halting behavior into a fluid, effortless, and "automatic" action are not well understood. Here we consider the possibility that well-practiced motor sequences, which initially rely on higher-level decision-making circuits, become wholly specified in lower-level control circuits. We review studies informing this idea, discuss the constraints on such shift in control, and suggest approaches to pinpoint circuit-level changes associated with motor sequence learning.

Distinct roles for motor cortical and thalamic inputs to striatum during motor skill learning and execution.

The acquisition and execution of motor skills are mediated by a distributed motor network, spanning cortical and subcortical brain areas. The sensorimotor striatum is an important cog in this network, yet the roles of its two main inputs, from motor cortex and thalamus, remain largely unknown. To address this, we silenced the inputs in rats trained on a task that results in highly stereotyped and idiosyncratic movement patterns. While striatal-projecting motor cortex neurons were critical for learning these skills, silencing this pathway after learning had no effect on performance. In contrast, silencing striatal-projecting thalamus neurons disrupted the execution of the learned skills, causing rats to revert to species-typical pressing behaviors and preventing them from relearning the task. These results show distinct roles for motor cortex and thalamus in the learning and execution of motor skills and suggest that their interaction in the striatum underlies experience-dependent changes in subcortical motor circuits.

Long-term stability of single neuron activity in the motor system.

How an established behavior is retained and consistently produced by a nervous system in constant flux remains a mystery. One possible solution to ensure long-term stability in motor output is to fix the activity patterns of single neurons in the relevant circuits. Alternatively, activity in single cells could drift over time provided that the population dynamics are constrained to produce the same behavior. To arbitrate between these possibilities, we recorded single-unit activity in motor cortex and striatum continuously for several weeks as rats performed stereotyped motor behaviors-both learned and innate. We found long-term stability in single neuron activity patterns across both brain regions. A small amount of drift in neural activity, observed over weeks of recording, could be explained by concomitant changes in task-irrelevant aspects of the behavior. These results suggest that long-term stable behaviors are generated by single neuron activity patterns that are themselves highly stable.

Retinal adaptation to object motion.

Due to fixational eye movements, the image on the retina is always in motion, even when one views a stationary scene. When an object moves within the scene, the corresponding patch of retina experiences a different motion trajectory than the surrounding region. Certain retinal ganglion cells respond selectively to this condition, when the motion in the cell's receptive field center is different from that in the surround. Here we show that this response is strongest at the very onset of differential motion, followed by gradual adaptation with a time course of several seconds. Different subregions of a ganglion cell's receptive field can adapt independently. The circuitry responsible for differential motion adaptation lies in the inner retina. Several candidate mechanisms were tested, and the adaptation most likely results from synaptic depression at the synapse from bipolar to ganglion cell. Similar circuit mechanisms may act more generally to emphasize novel features of a visual stimulus.

Changes in the neural control of a complex motor sequence during learning.

The acquisition of complex motor sequences often proceeds through trial-and-error learning, requiring the deliberate exploration of motor actions and the concomitant evaluation of the resulting performance. Songbirds learn their song in this manner, producing highly variable vocalizations as juveniles. As the song improves, vocal variability is gradually reduced until it is all but eliminated in adult birds. In the present study we examine how the motor program underlying such a complex motor behavior evolves during learning by recording from the robust nucleus of the arcopallium (RA), a motor cortex analog brain region. In young birds, neurons in RA exhibited highly variable firing patterns that throughout development became more precise, sparse, and bursty. We further explored how the developing motor program in RA is shaped by its two main inputs: LMAN, the output nucleus of a basal ganglia-forebrain circuit, and HVC, a premotor nucleus. Pharmacological inactivation of LMAN during singing made the song-aligned firing patterns of RA neurons adultlike in their stereotypy without dramatically affecting the spike statistics or the overall firing patterns. Removing the input from HVC, on the other hand, resulted in a complete loss of stereotypy of both the song and the underlying motor program. Thus our results show that a basal ganglia-forebrain circuit drives motor exploration required for trial-and-error learning by adding variability to the developing motor program. As learning proceeds and the motor circuits mature, the relative contribution of LMAN is reduced, allowing the premotor input from HVC to drive an increasingly stereotyped song.

The basal ganglia control the detailed kinematics of learned motor skills.

The basal ganglia are known to influence action selection and modulation of movement vigor, but whether and how they contribute to specifying the kinematics of learned motor skills is not understood. Here, we probe this question by recording and manipulating basal ganglia activity in rats trained to generate complex task-specific movement patterns with rich kinematic structure. We find that the sensorimotor arm of the basal ganglia circuit is crucial for generating the detailed movement patterns underlying the acquired motor skills. Furthermore, the neural representations in the striatum, and the control function they subserve, do not depend on inputs from the motor cortex. Taken together, these results extend our understanding of the basal ganglia by showing that they can specify and control the fine-grained details of learned motor skills through their interactions with lower-level motor circuits.

Continuous Whole-Body 3D Kinematic Recordings across the Rodent Behavioral Repertoire.

In mammalian animal models, high-resolution kinematic tracking is restricted to brief sessions in constrained environments, limiting our ability to probe naturalistic behaviors and their neural underpinnings. To address this, we developed CAPTURE (Continuous Appendicular and Postural Tracking Using Retroreflector Embedding), a behavioral monitoring system that combines motion capture and deep learning to continuously track the 3D kinematics of a rat's head, trunk, and limbs for week-long timescales in freely behaving animals. CAPTURE realizes 10- to 100-fold gains in precision and robustness compared with existing convolutional network approaches to behavioral tracking. We demonstrate CAPTURE's ability to comprehensively profile the kinematics and sequential organization of natural rodent behavior, its variation across individuals, and its perturbation by drugs and disease, including identifying perseverative grooming states in a rat model of fragile X syndrome. CAPTURE significantly expands the range of behaviors and contexts that can be quantitatively investigated, opening the door to a new understanding of natural behavior and its neural basis.

The neurobiology of deep reinforcement learning.

In this primer, Ölveczky and Gershman review concepts and advances in deep reinforcement learning and discuss how these can inform the implementation of learning processes in biological neural networks.

Discovering Precise Temporal Patterns in Large-Scale Neural Recordings through Robust and Interpretable Time Warping.

Though the temporal precision of neural computation has been studied intensively, a data-driven determination of this precision remains a fundamental challenge. Reproducible spike patterns may be obscured on single trials by uncontrolled temporal variability in behavior and cognition and may not be time locked to measurable signatures in behavior or local field potentials (LFP). To overcome these challenges, we describe a general-purpose time warping framework that reveals precise spike-time patterns in an unsupervised manner, even when these patterns are decoupled from behavior or are temporally stretched across single trials. We demonstrate this method across diverse systems: cued reaching in nonhuman primates, motor sequence production in rats, and olfaction in mice. This approach flexibly uncovers diverse dynamical firing patterns, including pulsatile responses to behavioral events, LFP-aligned oscillatory spiking, and even unanticipated patterns, such as 7 Hz oscillations in rat motor cortex that are not time locked to measured behaviors or LFP.

A retinal circuit that computes object motion.

Certain ganglion cells in the retina respond sensitively to differential motion between the receptive field center and surround, as produced by an object moving over the background, but are strongly suppressed by global image motion, as produced by the observer's head or eye movements. We investigated the circuit basis for this object motion sensitive (OMS) response by recording intracellularly from all classes of retinal interneurons while simultaneously recording the spiking output of many ganglion cells. Fast, transient bipolar cells respond linearly to motion in the receptive field center. The synaptic output from their terminals is rectified and then pooled by the OMS ganglion cell. A type of polyaxonal amacrine cell is driven by motion in the surround, again via pooling of rectified inputs, but from a different set of bipolar cell terminals. By direct intracellular current injection, we found that these polyaxonal amacrine cells selectively suppress the synaptic input of OMS ganglion cells. A quantitative model of these circuit elements and their interactions explains how an important visual computation is accomplished by retinal neurons and synapses.

Adaptive Regulation of Motor Variability.

Trial-to-trial movement variability can both drive motor learning and interfere with expert performance, suggesting benefits of regulating it in context-specific ways. Here we address whether and how the brain regulates motor variability as a function of performance by training rats to execute ballistic forelimb movements for reward. Behavioral datasets comprising millions of trials revealed that motor variability is regulated by two distinct processes. A fast process modulates variability as a function of recent trial outcomes, increasing it when performance is poor and vice versa. A slower process tunes the gain of the fast process based on the uncertainty in the task's reward landscape. Simulations demonstrated that this regulation strategy optimizes reward accumulation over a wide range of time horizons, while also promoting learning. Our results uncover a sophisticated algorithm implemented by the brain to adaptively regulate motor variability to improve task performance. VIDEO ABSTRACT.