Achieving natural behavior in a robot using neurally inspired hierarchical perceptual control.

Terrestrial locomotion presents tremendous computational challenges on account of the enormous degrees of freedom in legged animals, and complex, unpredictable properties of natural environments, including the body and its effectors, yet the nervous system can achieve locomotion with ease. Here we introduce a quadrupedal robot that is capable of posture control and goal-directed locomotion across uneven terrain. The control architecture is a hierarchical network of simple negative feedback control systems inspired by the organization of the vertebrate nervous system. This robot is capable of robust posture control and locomotion in novel environments with unpredictable disturbances. Unlike current robots, our robot does not use internal inverse and forward models, nor does it require any training in order to perform successfully in novel environments.

Protocol for Recording from Ventral Tegmental Area Dopamine Neurons in Mice while Measuring Force during Head-Fixation.

Many studies in systems neuroscience use head-fixation preparations for in vivo experimentation. While head-fixation confers several advantages, one major limitation is the lack of behavioral measures that quantify whole-body movements. Here, we detail a step-by-step protocol for using a novel head-fixation device that measures the forces exerted by head-fixed mice in multiple dimensions. We further detail how this system can be used in conjunction with in vivo electrophysiology and optogenetics to study dopamine neurons in the ventral tegmental area. For complete details on the use and execution of this protocol, please refer to Hughes et al. (2020a, 2020b).

A Head-Fixation System for Continuous Monitoring of Force Generated During Behavior.

Many studies in neuroscience use head-fixed behavioral preparations, which confer a number of advantages, including the ability to limit the behavioral repertoire and use techniques for large-scale monitoring of neural activity. But traditional studies using this approach use extremely limited behavioral measures, in part because it is difficult to detect the subtle movements and postural adjustments that animals naturally exhibit during head fixation. Here we report a new head-fixed setup with analog load cells capable of precisely monitoring the continuous forces exerted by mice. The load cells reveal the dynamic nature of movements generated not only around the time of task-relevant events, such as presentation of stimuli and rewards, but also during periods in between these events, when there is no apparent overt behavior. It generates a new and rich set of behavioral measures that have been neglected in previous experiments. We detail the construction of the system, which can be 3D-printed and assembled at low cost, show behavioral results collected from head-fixed mice, and demonstrate that neural activity can be highly correlated with the subtle, whole-body movements continuously produced during head restraint.

Using Expectancy Theory to quantitatively dissociate the neural representation of motivation from its influential factors in the human brain: An fMRI study.

Researchers have yet to apply a formal operationalized theory of motivation to neurobiology that would more accurately and precisely define neural activity underlying motivation. We overcome this challenge with the novel application of the Expectancy Theory of Motivation to human fMRI to identify brain activity that explicitly reflects motivation. Expectancy Theory quantitatively describes how individual constructs determine motivation by defining motivation force as the product of three variables: expectancy - belief that effort will better performance; instrumentality - belief that successful performance leads to particular outcome, and valence - outcome desirability. Here, we manipulated information conveyed by reward-predicting cues such that relative cue-evoked activity patterns could be statistically mapped to individual Expectancy Theory variables. The variable associated with activity in any voxel is only reported if it replicated between two groups of healthy participants. We found signals in midbrain, ventral striatum, sensorimotor cortex, and visual cortex that specifically map to motivation itself, rather than other factors. This is important because, for the first time, it empirically clarifies approach motivation neural signals during reward anticipation. It also highlights the effectiveness of the application of Expectancy Theory to neurobiology to more precisely and accurately probe motivation neural correlates than has been achievable previously.

A GABAergic nigrotectal pathway for coordination of drinking behavior.

The contribution of basal ganglia outputs to consummatory behavior remains poorly understood. We recorded from the substantia nigra pars reticulata (SNR), the major basal ganglia output nucleus, during self-initiated drinking in mice. The firing rates of many lateral SNR neurons were time-locked to individual licks. These neurons send GABAergic projections to the deep layers of the orofacial region of the lateral tectum (superior colliculus, SC). Many tectal neurons were also time-locked to licking, but their activity was usually in antiphase with that of SNR neurons, suggesting inhibitory nigrotectal projections. We used optogenetics to selectively activate the GABAergic nigrotectal afferents in the deep layers of the SC. Photo-stimulation of the nigrotectal projections transiently inhibited the activity of the lick-related tectal neurons, disrupted their licking-related oscillatory pattern and suppressed self-initiated drinking. These results demonstrate that GABAergic nigrotectal projections have a crucial role in coordinating drinking behavior.

Luminopsins integrate opto- and chemogenetics by using physical and biological light sources for opsin activation.

Luminopsins are fusion proteins of luciferase and opsin that allow interrogation of neuronal circuits at different temporal and spatial resolutions by choosing either extrinsic physical or intrinsic biological light for its activation. Building on previous development of fusions of wild-type Gaussia luciferase with channelrhodopsin, here we expanded the utility of luminopsins by fusing bright Gaussia luciferase variants with either channelrhodopsin to excite neurons (luminescent opsin, LMO) or a proton pump to inhibit neurons (inhibitory LMO, iLMO). These improved LMOs could reliably activate or silence neurons in vitro and in vivo. Expression of the improved LMO in hippocampal circuits not only enabled mapping of synaptic activation of CA1 neurons with fine spatiotemporal resolution but also could drive rhythmic circuit excitation over a large spatiotemporal scale. Furthermore, virus-mediated expression of either LMO or iLMO in the substantia nigra in vivo produced not only the expected bidirectional control of single unit activity but also opposing effects on circling behavior in response to systemic injection of a luciferase substrate. Thus, although preserving the ability to be activated by external light sources, LMOs expand the use of optogenetics by making the same opsins accessible to noninvasive, chemogenetic control, thereby allowing the same probe to manipulate neuronal activity over a range of spatial and temporal scales.

Beyond reward prediction errors: the role of dopamine in movement kinematics.

We recorded activity of dopamine (DA) neurons in the substantia nigra pars compacta in unrestrained mice while monitoring their movements with video tracking. Our approach allows an unbiased examination of the continuous relationship between single unit activity and behavior. Although DA neurons show characteristic burst firing following cue or reward presentation, as previously reported, their activity can be explained by the representation of actual movement kinematics. Unlike neighboring pars reticulata GABAergic output neurons, which can represent vector components of position, DA neurons represent vector components of velocity or acceleration. We found neurons related to movements in four directions-up, down, left, right. For horizontal movements, there is significant lateralization of neurons: the left nigra contains more rightward neurons, whereas the right nigra contains more leftward neurons. The relationship between DA activity and movement kinematics was found on both appetitive trials using sucrose and aversive trials using air puff, showing that these neurons belong to a velocity control circuit that can be used for any number of purposes, whether to seek reward or to avoid harm. In support of this conclusion, mimicry of the phasic activation of DA neurons with selective optogenetic stimulation could also generate movements. Contrary to the popular hypothesis that DA neurons encode reward prediction errors, our results suggest that nigrostriatal DA plays an essential role in controlling the kinematics of voluntary movements. We hypothesize that DA signaling implements gain adjustment for adaptive transition control, and describe a new model of the basal ganglia (BG) in which DA functions to adjust the gain of the transition controller. This model has significant implications for our understanding of movement disorders implicating DA and the BG.

Basal ganglia outputs map instantaneous position coordinates during behavior.

The basal ganglia (BG) are implicated in many movement disorders, yet how they contribute to movement remains unclear. Using wireless in vivo recording, we measured BG output from the substantia nigra pars reticulata (SNr) in mice while monitoring their movements with video tracking. The firing rate of most nigral neurons reflected Cartesian coordinates (either x- or y-coordinates) of the animal's head position during movement. The firing rates of SNr neurons are either positively or negatively correlated with the coordinates. Using an egocentric reference frame, four types of neurons can be classified: each type increases firing during movement in a particular direction (left, right, up, down), and decreases firing during movement in the opposite direction. Given the high correlation between the firing rate and the x and y components of the position vector, the movement trajectory can be reconstructed from neural activity. Our results therefore demonstrate a quantitative and continuous relationship between BG output and behavior. Thus, a steady BG output signal from the SNr (i.e., constant firing rate) is associated with the lack of overt movement, when a stable posture is maintained by structures downstream of the BG. Any change in SNr firing rate is associated with a change in position (i.e., movement). We hypothesize that the SNr output quantitatively determines the direction, velocity, and amplitude of voluntary movements. By changing the reference signals to downstream position control systems, the BG can produce transitions in body configurations and initiate actions.

Striatal firing rate reflects head movement velocity.

Although the basal ganglia have long been implicated in the initiation of actions, their contribution to movement remains a matter of dispute. Using wireless multi-electrode recording and motion tracking, we examined the relationship between single-unit activity in the sensorimotor striatum and movement kinematics. We recorded single-unit activity from medium spiny projection neurons and fast-spiking interneurons while monitoring the movements of mice using motion tracking. In Experiment 1, we trained mice to generate movements reliably by water-depriving them and giving them periodic cued sucrose rewards. We found high correlations between single-unit activity and movement velocity in particular directions. This correlation was found in both putative medium spiny projection neurons and fast-spiking interneurons. In Experiment 2, to rule out the possibility that the observed correlations were due to reward expectancy, we repeated the same procedure but added trials in which sucrose delivery was replaced by an aversive air puff stimulus. The air puff generated avoidance movements that were clearly different from movements on rewarded trials, but the same neurons that showed velocity correlation on reward trials exhibited a similar correlation on air puff trials. These experiments show for the first time that the firing rate of striatal neurons reflects movement velocity for different types of movements, whether to seek rewards or to avoid harm.

The role of the substantia nigra in posture control.

Disorders implicating the basal ganglia are often characterized by postural deficits, but little is known about the role of the basal ganglia in posture control. Using wireless multi-electrode recording, we measured single unit activity from GABAergic and dopaminergic neurons in the substantia nigra as unrestrained mice stood on an elevated platform while introducing continuous postural disturbances in the roll plane. We found two major types of neurons - those activated by tilt to the left side of the body and suppressed by tilt to the right side, and others activated by tilt to the right side and suppressed by tilt to the left side. Contrary to the prevailing view that the basal ganglia output from the substantia nigra pars reticulata either inhibits or disinhibits downstream structures in an all or none fashion, we showed that it continuously sends anti-phase signals to their downstream targets. We also demonstrated for the first time that nigrostriatal dopaminergic transmission is modulated by postural disturbances.

Bidirectional modulation of substantia nigra activity by motivational state.

A major output nucleus of the basal ganglia is the substantia nigra pars reticulata, which sends GABAergic projections to brainstem and thalamic nuclei. The GABAergic (GABA) neurons are reciprocally connected with nearby dopaminergic neurons, which project mainly to the basal ganglia, a set of subcortical nuclei critical for goal-directed behaviors. Here we examined the impact of motivational states on the activity of GABA neurons in the substantia nigra pars reticulata and the neighboring dopaminergic (DA) neurons in the pars compacta. Both types of neurons show short-latency bursts to a cue predicting a food reward. As mice became sated by repeated consumption of food pellets, one class of neurons reduced cue-elicited firing, whereas another class of neurons progressively increased firing. Extinction or pre-feeding just before the test session dramatically reduced the phasic responses and their motivational modulation. These results suggest that signals related to the current motivational state bidirectionally modulate behavior and the magnitude of phasic response of both DA and GABA neurons in the substantia nigra.

Inhaled oxytocin amplifies both vicarious reinforcement and self reinforcement in rhesus macaques (Macaca mulatta).

People attend not only to their own experiences, but also to the experiences of those around them. Such social awareness profoundly influences human behavior by enabling observational learning, as well as by motivating cooperation, charity, empathy, and spite. Oxytocin (OT), a neurosecretory hormone synthesized by hypothalamic neurons in the mammalian brain, can enhance affiliation or boost exclusion in different species in distinct contexts, belying any simple mechanistic neural model. Here we show that inhaled OT penetrates the CNS and subsequently enhances the sensitivity of rhesus macaques to rewards occurring to others as well as themselves. Roughly 2 h after inhaling OT, monkeys increased the frequency of prosocial choices associated with reward to another monkey when the alternative was to reward no one. OT also increased attention to the recipient monkey as well as the time it took to render such a decision. In contrast, within the first 2 h following inhalation, OT increased selfish choices associated with delivery of reward to self over a reward to the other monkey, without affecting attention or decision latency. Despite the differences in species typical social behavior, exogenous, inhaled OT causally promotes social donation behavior in rhesus monkeys, as it does in more egalitarian and monogamous ones, like prairie voles and humans, when there is no perceived cost to self. These findings potentially implicate shared neural mechanisms.

A wireless multi-channel recording system for freely behaving mice and rats.

To understand the neural basis of behavior, it is necessary to record brain activity in freely moving animals. Advances in implantable multi-electrode array technology have enabled researchers to record the activity of neuronal ensembles from multiple brain regions. The full potential of this approach is currently limited by reliance on cable tethers, with bundles of wires connecting the implanted electrodes to the data acquisition system while impeding the natural behavior of the animal. To overcome these limitations, here we introduce a multi-channel wireless headstage system designed for small animals such as rats and mice. A variety of single unit and local field potential signals were recorded from the dorsal striatum and substantia nigra in mice and the ventral striatum and prefrontal cortex simultaneously in rats. This wireless system could be interfaced with commercially available data acquisition systems, and the signals obtained were comparable in quality to those acquired using cable tethers. On account of its small size, light weight, and rechargeable battery, this wireless headstage system is suitable for studying the neural basis of natural behavior, eliminating the need for wires, commutators, and other limitations associated with traditional tethered recording systems.

Subjective socioeconomic status predicts human ventral striatal responses to social status information.

The enormous influence of hierarchical rank on social interactions [1] suggests that neural mechanisms exist to process status-related information [2] and ascribe value to it. The ventral striatum is prominently implicated in processing value and salience, independent of hedonic properties [3, 4], and a functional magnetic resonance imaging (fMRI) study of social status perception in humans demonstrated that viewing higher-ranked compared to lower-ranked individuals evokes a ventral striatal response [5], indicative of a greater assignment of value/salience to higher status. Consistent with this interpretation, nonhuman primates value information associated with higher-ranked conspecifics more than lower-ranked, as illustrated using a choice paradigm in which monkeys preferentially take the opportunity to view high-status monkeys [6]. Interestingly, this status-related value assignment in nonhuman primates is influenced by one's own hierarchical rank: high-status monkeys preferentially attend to conspecifics of high status, whereas low-status monkeys will also attend to other low-status monkeys [7]. Complementary to these findings, using fMRI and a social status judgment task in humans, we suggest a neurobiological mechanism by which one's own relative hierarchical rank influences the value attributed to particular social status information by demonstrating that one's subjective socioeconomic status differentially influences ventral striatal activity during processing of status-related information.

Inducible nitric oxide synthase, Nos2, does not mediate optic neuropathy and retinopathy in the DBA/2J glaucoma model.

BACKGROUND: Nitric oxide synthase 2 (NOS2) contributes to neural death in some settings, but its role in glaucoma remains controversial. NOS2 is implicated in retinal ganglion cell degeneration in a rat glaucoma model in which intraocular pressure (IOP) is experimentally elevated by blood vessel cauterization, but not in a rat glaucoma model where IOP was elevated by injection of hypertonic saline. To test the importance of NOS2 for an inherited glaucoma, in this study we both genetically and pharmacologically decreased NOS2 activity in the DBA/2J mouse glaucoma model. METHODS: The expression of Nos2 in the optic nerve head was analyzed at both the RNA and protein levels at different stages of disease pathogenesis. To test the involvement of Nos2 in glaucomatous neurodegeneration, a null allele of Nos2 was backcrossed into DBA/2J mice and the incidence and severity of glaucoma was assessed in mice of each Nos2 genotype. Additionally, DBA/2J mice were treated with the NOS2 inhibitor aminoguanidine and the disease compared to untreated mice. RESULTS: Optic nerve head Nos2 RNA levels varied and increased during moderate but decreased at early and severe stages of disease. Despite the presence of a few NOS2 positive cells in the optic nerve head, NOS2 protein was not substantially increased during the glaucoma. Genetic deficiency of Nos2 or aminoguanidine treatment did not alter the IOP profile of DBA/2J mice. Additionally, neither Nos2 deficiency nor aminoguanidine had any detectable affect on the glaucomatous optic nerve damage. CONCLUSION: Glaucomatous neurodegeneration in DBA/2J mice does not require NOS2 activity. Further experiments involving various models are needed to assess the general importance of Nos2 in glaucoma.

Axons of retinal ganglion cells are insulted in the optic nerve early in DBA/2J glaucoma.

Here, we use a mouse model (DBA/2J) to readdress the location of insult(s) to retinal ganglion cells (RGCs) in glaucoma. We localize an early sign of axon damage to an astrocyte-rich region of the optic nerve just posterior to the retina, analogous to the lamina cribrosa. In this region, a network of astrocytes associates intimately with RGC axons. Using BAX-deficient DBA/2J mice, which retain all of their RGCs, we provide experimental evidence for an insult within or very close to the lamina in the optic nerve. We show that proximal axon segments attached to their cell bodies survive to the proximity of the lamina. In contrast, axon segments in the lamina and behind the eye degenerate. Finally, the Wld(s) allele, which is known to protect against insults to axons, strongly protects against DBA/2J glaucoma and preserves RGC activity as measured by pattern electroretinography. These experiments provide strong evidence for a local insult to axons in the optic nerve.