Learning attentional templates for value-based decision-making.

Attention filters sensory inputs to enhance task-relevant information. It is guided by an "attentional template" that represents the stimulus features that are currently relevant. To understand how the brain learns and uses templates, we trained monkeys to perform a visual search task that required them to repeatedly learn new attentional templates. Neural recordings found that templates were represented across the prefrontal and parietal cortex in a structured manner, such that perceptually neighboring templates had similar neural representations. When the task changed, a new attentional template was learned by incrementally shifting the template toward rewarded features. Finally, we found that attentional templates transformed stimulus features into a common value representation that allowed the same decision-making mechanisms to deploy attention, regardless of the identity of the template. Altogether, our results provide insight into the neural mechanisms by which the brain learns to control attention and how attention can be flexibly deployed across tasks.

All-optical physiology resolves a synaptic basis for behavioral timescale plasticity.

Learning has been associated with modifications of synaptic and circuit properties, but the precise changes storing information in mammals have remained largely unclear. We combined genetically targeted voltage imaging with targeted optogenetic activation and silencing of pre- and post-synaptic neurons to study the mechanisms underlying hippocampal behavioral timescale plasticity. In mice navigating a virtual-reality environment, targeted optogenetic activation of individual CA1 cells at specific places induced stable representations of these places in the targeted cells. Optical elicitation, recording, and modulation of synaptic transmission in behaving mice revealed that activity in presynaptic CA2/3 cells was required for the induction of plasticity in CA1 and, furthermore, that during induction of these place fields in single CA1 cells, synaptic input from CA2/3 onto these same cells was potentiated. These results reveal synaptic implementation of hippocampal behavioral timescale plasticity and define a methodology to resolve synaptic plasticity during learning and memory in behaving mammals.

Light links neonatal neurons for learning.

The newborn mouse's retina senses light even before the eye opens, informing the developing brain of the visual world. Without this information, the brain forms fewer connections and the adult mouse learns sluggishly.

Impaired neural replay of inferred relationships in schizophrenia.

An ability to build structured mental maps of the world underpins our capacity to imagine relationships between objects that extend beyond experience. In rodents, such representations are supported by sequential place cell reactivations during rest, known as replay. Schizophrenia is proposed to reflect a compromise in structured mental representations, with animal models reporting abnormalities in hippocampal replay and associated ripple activity during rest. Here, utilizing magnetoencephalography (MEG), we tasked patients with schizophrenia and control participants to infer unobserved relationships between objects by reorganizing visual experiences containing these objects. During a post-task rest session, controls exhibited fast spontaneous neural reactivation of presented objects that replayed inferred relationships. Replay was coincident with increased ripple power in hippocampus. Patients showed both reduced replay and augmented ripple power relative to controls, convergent with findings in animal models. These abnormalities are linked to impairments in behavioral acquisition and subsequent neural representation of task structure.

A neural circuit state change underlying skilled movements.

In motor neuroscience, state changes are hypothesized to time-lock neural assemblies coordinating complex movements, but evidence for this remains slender. We tested whether a discrete change from more autonomous to coherent spiking underlies skilled movement by imaging cerebellar Purkinje neuron complex spikes in mice making targeted forelimb-reaches. As mice learned the task, millimeter-scale spatiotemporally coherent spiking emerged ipsilateral to the reaching forelimb, and consistent neural synchronization became predictive of kinematic stereotypy. Before reach onset, spiking switched from more disordered to internally time-locked concerted spiking and silence. Optogenetic manipulations of cerebellar feedback to the inferior olive bi-directionally modulated neural synchronization and reaching direction. A simple model explained the reorganization of spiking during reaching as reflecting a discrete bifurcation in olivary network dynamics. These findings argue that to prepare learned movements, olivo-cerebellar circuits enter a self-regulated, synchronized state promoting motor coordination. State changes facilitating behavioral transitions may generalize across neural systems.

A genetically defined insula-brainstem circuit selectively controls motivational vigor.

The anterior insular cortex (aIC) plays a critical role in cognitive and motivational control of behavior, but the underlying neural mechanism remains elusive. Here, we show that aIC neurons expressing Fezf2 (aICFezf2), which are the pyramidal tract neurons, signal motivational vigor and invigorate need-seeking behavior through projections to the brainstem nucleus tractus solitarii (NTS). aICFezf2 neurons and their postsynaptic NTS neurons acquire anticipatory activity through learning, which encodes the perceived value and the vigor of actions to pursue homeostatic needs. Correspondingly, aIC → NTS circuit activity controls vigor, effort, and striatal dopamine release but only if the action is learned and the outcome is needed. Notably, aICFezf2 neurons do not represent taste or valence. Moreover, aIC → NTS activity neither drives reinforcement nor influences total consumption. These results pinpoint specific functions of aIC → NTS circuit for selectively controlling motivational vigor and suggest that motivation is subserved, in part, by aIC's top-down regulation of dopamine signaling.

Efficient Inference in Structured Spaces.

Whittington et al. demonstrate how network architectures defined in a spatial context may be useful for inference on different types of relational knowledge. These architectures allow for learning the structure of the environment and then transferring that knowledge to allow prediction of novel transitions.

A Genetically Defined Compartmentalized Striatal Direct Pathway for Negative Reinforcement.

The striosome compartment within the dorsal striatum has been implicated in reinforcement learning and regulation of motivation, but how striosomal neurons contribute to these functions remains elusive. Here, we show that a genetically identified striosomal population, which expresses the Teashirt family zinc finger 1 (Tshz1) and belongs to the direct pathway, drives negative reinforcement and is essential for aversive learning in mice. Contrasting a "conventional" striosomal direct pathway, the Tshz1 neurons cause aversion, movement suppression, and negative reinforcement once activated, and they receive a distinct set of synaptic inputs. These neurons are predominantly excited by punishment rather than reward and represent the anticipation of punishment or the motivation for avoidance. Furthermore, inhibiting these neurons impairs punishment-based learning without affecting reward learning or movement. These results establish a major role of striosomal neurons in behaviors reinforced by punishment and moreover uncover functions of the direct pathway unaccounted for in classic models.

The Geometry of Abstraction in the Hippocampus and Prefrontal Cortex.

The curse of dimensionality plagues models of reinforcement learning and decision making. The process of abstraction solves this by constructing variables describing features shared by different instances, reducing dimensionality and enabling generalization in novel situations. Here, we characterized neural representations in monkeys performing a task described by different hidden and explicit variables. Abstraction was defined operationally using the generalization performance of neural decoders across task conditions not used for training, which requires a particular geometry of neural representations. Neural ensembles in prefrontal cortex, hippocampus, and simulated neural networks simultaneously represented multiple variables in a geometry reflecting abstraction but that still allowed a linear classifier to decode a large number of other variables (high shattering dimensionality). Furthermore, this geometry changed in relation to task events and performance. These findings elucidate how the brain and artificial systems represent variables in an abstract format while preserving the advantages conferred by high shattering dimensionality.

Striosomes Mediate Value-Based Learning Vulnerable in Age and a Huntington's Disease Model.

Learning valence-based responses to favorable and unfavorable options requires judgments of the relative value of the options, a process necessary for species survival. We found, using engineered mice, that circuit connectivity and function of the striosome compartment of the striatum are critical for this type of learning. Calcium imaging during valence-based learning exhibited a selective correlation between learning and striosomal but not matrix signals. This striosomal activity encoded discrimination learning and was correlated with task engagement, which, in turn, could be regulated by chemogenetic excitation and inhibition. Striosomal function during discrimination learning was disturbed with aging and severely so in a mouse model of Huntington's disease. Anatomical and functional connectivity of parvalbumin-positive, putative fast-spiking interneurons (FSIs) to striatal projection neurons was enhanced in striosomes compared with matrix in mice that learned. Computational modeling of these findings suggests that FSIs can modulate the striosomal signal-to-noise ratio, crucial for discrimination and learning.

Human Replay Spontaneously Reorganizes Experience.

Knowledge abstracted from previous experiences can be transferred to aid new learning. Here, we asked whether such abstract knowledge immediately guides the replay of new experiences. We first trained participants on a rule defining an ordering of objects and then presented a novel set of objects in a scrambled order. Across two studies, we observed that representations of these novel objects were reactivated during a subsequent rest. As in rodents, human "replay" events occurred in sequences accelerated in time, compared to actual experience, and reversed their direction after a reward. Notably, replay did not simply recapitulate visual experience, but followed instead a sequence implied by learned abstract knowledge. Furthermore, each replay contained more than sensory representations of the relevant objects. A sensory code of object representations was preceded 50 ms by a code factorized into sequence position and sequence identity. We argue that this factorized representation facilitates the generalization of a previously learned structure to new objects.

Continual Learning in a Multi-Layer Network of an Electric Fish.

Distributing learning across multiple layers has proven extremely powerful in artificial neural networks. However, little is known about how multi-layer learning is implemented in the brain. Here, we provide an account of learning across multiple processing layers in the electrosensory lobe (ELL) of mormyrid fish and report how it solves problems well known from machine learning. Because the ELL operates and learns continuously, it must reconcile learning and signaling functions without switching its mode of operation. We show that this is accomplished through a functional compartmentalization within intermediate layer neurons in which inputs driving learning differentially affect dendritic and axonal spikes. We also find that connectivity based on learning rather than sensory response selectivity assures that plasticity at synapses onto intermediate-layer neurons is matched to the requirements of output neurons. The mechanisms we uncover have relevance to learning in the cerebellum, hippocampus, and cerebral cortex, as well as in artificial systems.

Primate Amygdala Neurons Simulate Decision Processes of Social Partners.

By observing their social partners, primates learn about reward values of objects. Here, we show that monkeys' amygdala neurons derive object values from observation and use these values to simulate a partner monkey's decision process. While monkeys alternated making reward-based choices, amygdala neurons encoded object-specific values learned from observation. Dynamic activities converted these values to representations of the recorded monkey's own choices. Surprisingly, the same activity patterns unfolded spontaneously before partner's choices in separate neurons, as if these neurons simulated the partner's decision-making. These "simulation neurons" encoded signatures of mutual-inhibitory decision computation, including value comparisons and value-to-choice conversions, resulting in accurate predictions of partner's choices. Population decoding identified differential contributions of amygdala subnuclei. Biophysical modeling of amygdala circuits showed that simulation neurons emerge naturally from convergence between object-value neurons and self-other neurons. By simulating decision computations during observation, these neurons could allow primates to reconstruct their social partners' mental states.

Area-Specificity and Plasticity of History-Dependent Value Coding During Learning.

Decision making is often driven by the subjective value of available options, a value which is formed through experience. To support this fundamental behavior, the brain must encode and maintain the subjective value. To investigate the area specificity and plasticity of value coding, we trained mice in a value-based decision task and imaged neural activity in 6 cortical areas with cellular resolution. History- and value-related signals were widespread across areas, but their strength and temporal patterns differed. In expert mice, the retrosplenial cortex (RSC) uniquely encoded history- and value-related signals with persistent population activity patterns across trials. This unique encoding of RSC emerged during task learning with a strong increase in more distant history signals. Acute inactivation of RSC selectively impaired the reward-history-based behavioral strategy. Our results indicate that RSC flexibly changes its history coding and persistently encodes value-related signals to support adaptive behaviors.

A Thalamic Circuit Lights up Mood.

The contributions of areas downstream of retinal ganglion cells involved in the processing and regulation of mood remain largely unspecified. In this issue of Cell, Fernandez et al. (2018) identify a thalamic circuit within the perihabenular region (pHb) linking daily changes of light pattern to mood regulation.

Fear Extinction Requires Reward.

Learning theorists long hypothesized that appetitive and aversive motivational states influence one another antagonistically. Here, Felsenberg et al. show that the activity of neurons in Drosophila known to be important in appetitive conditioning is necessary for the extinction of aversive conditioning, thereby uncovering biological evidence for this opponent-process.

Sunlight Brightens Learning and Memory.

Sunlight can alter mood, behavior, and cognition, but the cellular basis of this phenomenon remains to be fully elucidated. In this issue of Cell, Zhu et al. shed light on a UV-dependent metabolic pathway that leads to increased synaptic release of glutamate and enhanced motor learning and memory in mice.

Light Affects Mood and Learning through Distinct Retina-Brain Pathways.

Light exerts a range of powerful biological effects beyond image vision, including mood and learning regulation. While the source of photic information affecting mood and cognitive functions is well established, viz. intrinsically photosensitive retinal ganglion cells (ipRGCs), the central mediators are unknown. Here, we reveal that the direct effects of light on learning and mood utilize distinct ipRGC output streams. ipRGCs that project to the suprachiasmatic nucleus (SCN) mediate the effects of light on learning, independently of the SCN's pacemaker function. Mood regulation by light, on the other hand, requires an SCN-independent pathway linking ipRGCs to a previously unrecognized thalamic region, termed perihabenular nucleus (PHb). The PHb is integrated in a distinctive circuitry with mood-regulating centers and is both necessary and sufficient for driving the effects of light on affective behavior. Together, these results provide new insights into the neural basis required for light to influence mood and learning.

Fast-Spiking Interneurons Supply Feedforward Control of Bursting, Calcium, and Plasticity for Efficient Learning.

Fast-spiking interneurons (FSIs) are a prominent class of forebrain GABAergic cells implicated in two seemingly independent network functions: gain control and network plasticity. Little is known, however, about how these roles interact. Here, we use a combination of cell-type-specific ablation, optogenetics, electrophysiology, imaging, and behavior to describe a unified mechanism by which striatal FSIs control burst firing, calcium influx, and synaptic plasticity in neighboring medium spiny projection neurons (MSNs). In vivo silencing of FSIs increased bursting, calcium transients, and AMPA/NMDA ratios in MSNs. In a motor sequence task, FSI silencing increased the frequency of calcium transients but reduced the specificity with which transients aligned to individual task events. Consistent with this, ablation of FSIs disrupted the acquisition of striatum-dependent egocentric learning strategies. Together, our data support a model in which feedforward inhibition from FSIs temporally restricts MSN bursting and calcium-dependent synaptic plasticity to facilitate striatum-dependent sequence learning.

Corticoamygdala Transfer of Socially Derived Information Gates Observational Learning.

Observational learning is a powerful survival tool allowing individuals to learn about threat-predictive stimuli without directly experiencing the pairing of the predictive cue and punishment. This ability has been linked to the anterior cingulate cortex (ACC) and the basolateral amygdala (BLA). To investigate how information is encoded and transmitted through this circuit, we performed electrophysiological recordings in mice observing a demonstrator mouse undergo associative fear conditioning and found that BLA-projecting ACC (ACC→BLA) neurons preferentially encode socially derived aversive cue information. Inhibition of ACC→BLA alters real-time amygdala representation of the aversive cue during observational conditioning. Selective inhibition of the ACC→BLA projection impaired acquisition, but not expression, of observational fear conditioning. We show that information derived from observation about the aversive value of the cue is transmitted from the ACC to the BLA and that this routing of information is critically instructive for observational fear conditioning. VIDEO ABSTRACT.