The Geometry of Abstraction in the Hippocampus and Prefrontal Cortex.

The curse of dimensionality plagues models of reinforcement learning and decision making. The process of abstraction solves this by constructing variables describing features shared by different instances, reducing dimensionality and enabling generalization in novel situations. Here, we characterized neural representations in monkeys performing a task described by different hidden and explicit variables. Abstraction was defined operationally using the generalization performance of neural decoders across task conditions not used for training, which requires a particular geometry of neural representations. Neural ensembles in prefrontal cortex, hippocampus, and simulated neural networks simultaneously represented multiple variables in a geometry reflecting abstraction but that still allowed a linear classifier to decode a large number of other variables (high shattering dimensionality). Furthermore, this geometry changed in relation to task events and performance. These findings elucidate how the brain and artificial systems represent variables in an abstract format while preserving the advantages conferred by high shattering dimensionality.

Neural Representations of Unconditioned Stimuli in Basolateral Amygdala Mediate Innate and Learned Responses.

Stimuli that possess inherently rewarding or aversive qualities elicit emotional responses and also induce learning by imparting valence upon neutral sensory cues. Evidence has accumulated implicating the amygdala as a critical structure in mediating these processes. We have developed a genetic strategy to identify the representations of rewarding and aversive unconditioned stimuli (USs) in the basolateral amygdala (BLA) and have examined their role in innate and learned responses. Activation of an ensemble of US-responsive cells in the BLA elicits innate physiological and behavioral responses of different valence. Activation of this US ensemble can also reinforce appetitive and aversive learning when paired with differing neutral stimuli. Moreover, we establish that the activation of US-responsive cells in the BLA is necessary for the expression of a conditioned response. Neural representations of conditioned and unconditioned stimuli therefore ultimately connect to US-responsive cells in the BLA to elicit both innate and learned responses.

The coding of valence and identity in the mammalian taste system.

The ability of the taste system to identify a tastant (what it tastes like) enables animals to recognize and discriminate between the different basic taste qualities1,2. The valence of a tastant (whether it is appetitive or aversive) specifies its hedonic value and elicits the execution of selective behaviours. Here we examine how sweet and bitter are afforded valence versus identity in mice. We show that neurons in the sweet-responsive and bitter-responsive cortex project to topographically distinct areas of the amygdala, with strong segregation of neural projections conveying appetitive versus aversive taste signals. By manipulating selective taste inputs to the amygdala, we show that it is possible to impose positive or negative valence on a neutral water stimulus, and even to reverse the hedonic value of a sweet or bitter tastant. Remarkably, mice with silenced neurons in the amygdala no longer exhibit behaviour that reflects the valence associated with direct stimulation of the taste cortex, or with delivery of sweet and bitter chemicals. Nonetheless, these mice can still identify and discriminate between tastants, just as wild-type controls do. These results help to explain how the taste system generates stereotypic and predetermined attractive and aversive taste behaviours, and support the existence of distinct neural substrates for the discrimination of taste identity and the assignment of valence.

Low-dimensional dynamics for working memory and time encoding.

Our decisions often depend on multiple sensory experiences separated by time delays. The brain can remember these experiences and, simultaneously, estimate the timing between events. To understand the mechanisms underlying working memory and time encoding, we analyze neural activity recorded during delays in four experiments on nonhuman primates. To disambiguate potential mechanisms, we propose two analyses, namely, decoding the passage of time from neural data and computing the cumulative dimensionality of the neural trajectory over time. Time can be decoded with high precision in tasks where timing information is relevant and with lower precision when irrelevant for performing the task. Neural trajectories are always observed to be low-dimensional. In addition, our results further constrain the mechanisms underlying time encoding as we find that the linear "ramping" component of each neuron's firing rate strongly contributes to the slow timescale variations that make decoding time possible. These constraints rule out working memory models that rely on constant, sustained activity and neural networks with high-dimensional trajectories, like reservoir networks. Instead, recurrent networks trained with backpropagation capture the time-encoding properties and the dimensionality observed in the data.

Partitioning variability in animal behavioral videos using semi-supervised variational autoencoders.

Recent neuroscience studies demonstrate that a deeper understanding of brain function requires a deeper understanding of behavior. Detailed behavioral measurements are now often collected using video cameras, resulting in an increased need for computer vision algorithms that extract useful information from video data. Here we introduce a new video analysis tool that combines the output of supervised pose estimation algorithms (e.g. DeepLabCut) with unsupervised dimensionality reduction methods to produce interpretable, low-dimensional representations of behavioral videos that extract more information than pose estimates alone. We demonstrate this tool by extracting interpretable behavioral features from videos of three different head-fixed mouse preparations, as well as a freely moving mouse in an open field arena, and show how these interpretable features can facilitate downstream behavioral and neural analyses. We also show how the behavioral features produced by our model improve the precision and interpretation of these downstream analyses compared to using the outputs of either fully supervised or fully unsupervised methods alone.

The contribution of nonhuman primate research to the understanding of emotion and cognition and its clinical relevance.

Psychiatric disorders are often conceptualized as arising from dysfunctional interactions between neural systems mediating cognitive and emotional processes. Mechanistic insights into these interactions have been lacking in part because most work in emotions has occurred in rodents, often without concurrent manipulations of cognitive variables. Nonhuman primate (NHP) model systems provide a powerful platform for investigating interactions between cognitive operations and emotions due to NHPs' strong homology with humans in behavioral repertoire and brain anatomy. Recent electrophysiological studies in NHPs have delineated how neural signals in the amygdala, a brain structure linked to emotion, predict impending appetitive and aversive stimuli. In addition, abstract conceptual information has also been shown to be represented in the amygdala and in interconnected brain structures such as the hippocampus and prefrontal cortex. Flexible adjustments of emotional behavior require the ability to apply conceptual knowledge and generalize to different, often novel, situations, a hallmark example of interactions between cognitive and emotional processes. Elucidating the neural mechanisms that explain how the brain processes conceptual information in relation to emotional variables promises to provide important insights into the pathophysiology accounting for symptoms in neuropsychiatric disorders.

Emotion, cognition, and mental state representation in amygdala and prefrontal cortex.

Neuroscientists have often described cognition and emotion as separable processes implemented by different regions of the brain, such as the amygdala for emotion and the prefrontal cortex for cognition. In this framework, functional interactions between the amygdala and prefrontal cortex mediate emotional influences on cognitive processes such as decision-making, as well as the cognitive regulation of emotion. However, neurons in these structures often have entangled representations, whereby single neurons encode multiple cognitive and emotional variables. Here we review studies using anatomical, lesion, and neurophysiological approaches to investigate the representation and utilization of cognitive and emotional parameters. We propose that these mental state parameters are inextricably linked and represented in dynamic neural networks composed of interconnected prefrontal and limbic brain structures. Future theoretical and experimental work is required to understand how these mental state representations form and how shifts between mental states occur, a critical feature of adaptive cognitive and emotional behavior.

The amygdala and basal forebrain as a pathway for motivationally guided attention.

Visual stimuli associated with rewards attract spatial attention. Neurophysiological mechanisms that mediate this process must register both the motivational significance and location of visual stimuli. Recent neurophysiological evidence indicates that the amygdala encodes information about both of these parameters. Furthermore, the firing rate of amygdala neurons predicts the allocation of spatial attention. One neural pathway through which the amygdala might influence attention involves the intimate and bidirectional connections between the amygdala and basal forebrain (BF), a brain area long implicated in attention. Neurons in the rhesus monkey amygdala and BF were therefore recorded simultaneously while subjects performed a detection task in which the stimulus-reward associations of visual stimuli modulated spatial attention. Neurons in BF were spatially selective for reward-predictive stimuli, much like the amygdala. The onset of reward-predictive signals in each brain area suggested different routes of processing for reward-predictive stimuli appearing in the ipsilateral and contralateral fields. Moreover, neurons in the amygdala, but not BF, tracked trial-to-trial fluctuations in spatial attention. These results suggest that the amygdala and BF could play distinct yet inter-related roles in influencing attention elicited by reward-predictive stimuli.

Task-dependent spatial selectivity in the primate amygdala.

Humans and other animals routinely encounter visual stimuli that indicate whether future reward delivery depends upon the identity or location of a stimulus, or the performance of a particular action. These reinforcement contingencies can influence how much attention is directed toward a stimulus. Neurons in the primate amygdala encode information about the association between visual stimuli and reinforcement as well as about the location of reward-predictive stimuli. Amygdala neural activity also predicts variability in spatial attention. In principle, the spatial properties of amygdala neurons may be present independent of spatial attention allocation. Alternatively, the encoding of spatial information may require attention. We trained monkeys to perform tasks that engaged spatial attention to varying degrees to understand the genesis of spatial processing in the amygdala. During classical conditioning tasks, conditioned stimuli appeared at different locations; amygdala neurons responded selectively to the location of stimuli. These spatial signals diminished rapidly upon stimulus disappearance and were unrelated to selectivity for expected reward. In contrast, spatial selectivity was sustained in time when monkeys performed a delayed saccade task that required sustained spatial attention. This temporally extended spatial signal was correlated with signals encoding reward expectation. Furthermore, variability in firing rates was correlated with variability in spatial attention, as measured by reaction time. These results reveal two types of spatial signals in the amygdala: one that is tied to initial visual responses and a second that reflects coordination between spatial and reinforcement information and that relates to the engagement of spatial attention.

Functional circuits and anatomical distribution of response properties in the primate amygdala.

Recent electrophysiological studies on the primate amygdala have advanced our understanding of how individual neurons encode information relevant to emotional processes, but it remains unclear how these neurons are functionally and anatomically organized. To address this, we analyzed cross-correlograms of amygdala spike trains recorded during a task in which monkeys learned to associate novel images with rewarding and aversive outcomes. Using this task, we have recently described two populations of amygdala neurons: one that responds more strongly to images predicting reward (positive value-coding), and another that responds more strongly to images predicting an aversive stimulus (negative value-coding). Here, we report that these neural populations are organized into distinct, but anatomically intermingled, appetitive and aversive functional circuits, which are dynamically modulated as animals used the images to predict outcomes. Furthermore, we report that responses to sensory stimuli are prevalent in the lateral amygdala, and are also prevalent in the medial amygdala for sensory stimuli that are emotionally significant. The circuits identified here could potentially mediate valence-specific emotional behaviors thought to involve the amygdala.

Visuospatial information foraging describes search behavior in learning latent environmental features.

In the real world, making sequences of decisions to achieve goals often depends upon the ability to learn aspects of the environment that are not directly perceptible. Learning these so-called latent features requires seeking information about them. Prior efforts to study latent feature learning often used single decisions, used few features, and failed to distinguish between reward-seeking and information-seeking. To overcome this, we designed a task in which humans and monkeys made a series of choices to search for shapes hidden on a grid. On our task, the effects of reward and information outcomes from uncovering parts of shapes could be disentangled. Members of both species adeptly learned the shapes and preferred to select tiles expected to be informative earlier in trials than previously rewarding ones, searching a part of the grid until their outcomes dropped below the average information outcome-a pattern consistent with foraging behavior. In addition, how quickly humans learned the shapes was predicted by how well their choice sequences matched the foraging pattern, revealing an unexpected connection between foraging and learning. This adaptive search for information may underlie the ability in humans and monkeys to learn latent features to support goal-directed behavior in the long run.

The representational geometry of emotional states in basolateral amygdala.

Sensory stimuli associated with aversive outcomes can cause multiple behavioral responses related to an animal's evolving emotional state. We employed chemogenetic inactivation and two-photon imaging to reveal how the basolateral amygdala (BLA) mediates these state changes. Mice were presented stimuli in a virtual burrow, causing two responses reflecting fear and flight to safety: tremble and ingress into the burrow. Inactivation eliminated differential tremble and ingress to aversive and neutral stimuli without eliminating responses themselves. Multiple variables, including stimulus valence and identity, and being in the tremble or ingressed state, typically modulated each neuron's activity (mixed-selectivity). BLA neural ensembles represented these variables even after neurons with apparent specialized selectivity were eliminated from analyses. Thus, implementing different readouts of BLA ensembles comprised of mixed-selectivity neurons can identify distinct emotional states defined by responses, like tremble for fear and ingress for safety. This mechanism relies on BLA's representational geometry, not its circuit specialization.

Abstract representations emerge in human hippocampal neurons during inference behavior.

Humans have the remarkable cognitive capacity to rapidly adapt to changing environments. Central to this capacity is the ability to form high-level, abstract representations that take advantage of regularities in the world to support generalization 1 . However, little is known about how these representations are encoded in populations of neurons, how they emerge through learning, and how they relate to behavior 2,3 . Here we characterized the representational geometry of populations of neurons (single-units) recorded in the hippocampus, amygdala, medial frontal cortex, and ventral temporal cortex of neurosurgical patients who are performing an inferential reasoning task. We find that only the neural representations formed in the hippocampus simultaneously encode multiple task variables in an abstract, or disentangled, format. This representational geometry is uniquely observed after patients learn to perform inference, and consisted of disentangled directly observable and discovered latent task variables. Interestingly, learning to perform inference by trial and error or through verbal instructions led to the formation of hippocampal representations with similar geometric properties. The observed relation between representational format and inference behavior suggests that abstract/disentangled representational geometries are important for complex cognition.

The primate amygdala represents the positive and negative value of visual stimuli during learning.

Visual stimuli can acquire positive or negative value through their association with rewards and punishments, a process called reinforcement learning. Although we now know a great deal about how the brain analyses visual information, we know little about how visual representations become linked with values. To study this process, we turned to the amygdala, a brain structure implicated in reinforcement learning. We recorded the activity of individual amygdala neurons in monkeys while abstract images acquired either positive or negative value through conditioning. After monkeys had learned the initial associations, we reversed image value assignments. We examined neural responses in relation to these reversals in order to estimate the relative contribution to neural activity of the sensory properties of images and their conditioned values. Here we show that changes in the values of images modulate neural activity, and that this modulation occurs rapidly enough to account for, and correlates with, monkeys' learning. Furthermore, distinct populations of neurons encode the positive and negative values of visual stimuli. Behavioural and physiological responses to visual stimuli may therefore be based in part on the plastic representation of value provided by the amygdala.

Prelimbic cortex drives discrimination of non-aversion via amygdala somatostatin interneurons.

The amygdala and prelimbic cortex (PL) communicate during fear discrimination retrieval, but how they coordinate discrimination of a non-threatening stimulus is unknown. Here, we show that somatostatin (SOM) interneurons in the basolateral amygdala (BLA) become active specifically during learned non-threatening cues and desynchronize cell firing by blocking phase reset of theta oscillations during the safe cue. Furthermore, we show that SOM activation and desynchronization of the BLA is PL-dependent and promotes discrimination of non-threat. Thus, fear discrimination engages PL-dependent coordination of BLA SOM responses to non-threatening stimuli.

Expectation modulates neural responses to pleasant and aversive stimuli in primate amygdala.

Animals and humans learn to approach and acquire pleasant stimuli and to avoid or defend against aversive ones. However, both pleasant and aversive stimuli can elicit arousal and attention, and their salience or intensity increases when they occur by surprise. Thus, adaptive behavior may require that neural circuits compute both stimulus valence--or value--and intensity. To explore how these computations may be implemented, we examined neural responses in the primate amygdala to unexpected reinforcement during learning. Many amygdala neurons responded differently to reinforcement depending upon whether or not it was expected. In some neurons, this modulation occurred only for rewards or aversive stimuli, but not both. In other neurons, expectation similarly modulated responses to both rewards and punishments. These different neuronal populations may subserve two sorts of processes mediated by the amygdala: those activated by surprising reinforcements of both valences-such as enhanced arousal and attention-and those that are valence-specific, such as fear or reward-seeking behavior.

The convergence of information about rewarding and aversive stimuli in single neurons.

Neuroscientists, psychologists, clinicians, and economists have long been interested in how individuals weigh information about potential rewarding and aversive stimuli to make decisions and to regulate their emotions. However, we know relatively little about how appetitive and aversive systems interact in the brain, as most prior studies have investigated only one valence of reinforcement. Previous work has suggested that primate orbitofrontal cortex (OFC) represents information about the reward value of stimuli. We therefore investigated whether OFC also represents information about aversive stimuli, and, if so, whether individual neurons process information about both rewarding and aversive stimuli. Monkeys performed a trace conditioning task in which different novel abstract visual stimuli (conditioned stimuli, CSs) predicted the occurrence of one of three unconditioned stimuli (USs): a large liquid reward, a small liquid reward, or an aversive air-puff. Three lines of evidence suggest that information about rewarding and aversive stimuli converges in individual neurons in OFC. First, OFC neurons often responded to both rewarding and aversive USs, despite their different sensory features. Second, OFC neural responses to CSs often encoded information about both potential rewarding and aversive stimuli, even though these stimuli differed in both valence and sensory modality. Finally, OFC neural responses were correlated with monkeys' behavioral use of information about both rewarding and aversive CS-US associations. These data indicate that processing of appetitive and aversive stimuli converges at the single cell level in OFC, providing a possible substrate for executive and emotional processes that require using information from both appetitive and aversive systems.

Attractor concretion as a mechanism for the formation of context representations.

Complex tasks often require the memory of recent events, the knowledge about the context in which they occur, and the goals we intend to reach. All this information is stored in our mental states. Given a set of mental states, reinforcement learning (RL) algorithms predict the optimal policy that maximizes future reward. RL algorithms assign a value to each already-known state so that discovering the optimal policy reduces to selecting the action leading to the state with the highest value. But how does the brain create representations of these mental states in the first place? We propose a mechanism for the creation of mental states that contain information about the temporal statistics of the events in a particular context. We suggest that the mental states are represented by stable patterns of reverberating activity, which are attractors of the neural dynamics. These representations are built from neurons that are selective to specific combinations of external events (e.g. sensory stimuli) and pre-existent mental states. Consistent with this notion, we find that neurons in the amygdala and in orbitofrontal cortex (OFC) often exhibit this form of mixed selectivity. We propose that activating different mixed selectivity neurons in a fixed temporal order modifies synaptic connections so that conjunctions of events and mental states merge into a single pattern of reverberating activity. This process corresponds to the birth of a new, different mental state that encodes a different temporal context. The concretion process depends on temporal contiguity, i.e. on the probability that a combination of an event and mental states follows or precedes the events and states that define a certain context. The information contained in the context thereby allows an animal to assign unambiguously a value to the events that initially appeared in different situations with different meanings.

Moment-to-moment tracking of state value in the amygdala.

As an organism interacts with the world, how good or bad things are at the moment, the value of the current state of the organism, is an important parameter that is likely to be encoded in the brain. As the environment changes and new stimuli appear, estimates of state value must be updated to support appropriate responses and learning. Indeed, many models of reinforcement learning posit representations of state value. We examined how the brain mediates this process by recording amygdala neural activity while monkeys performed a trace-conditioning task requiring fixation. The presentation of different stimuli induced state transitions; these stimuli included unconditioned stimuli (USs) (liquid rewards and aversive air puffs), newly learned reinforcement-predictive visual stimuli [conditioned stimuli (CSs)], and familiar stimuli long associated with reinforcement [fixation point (FP)]. The FP had a positive value to monkeys, because they chose to foveate it to initiate trials. Different populations of amygdala neurons tracked the positive or negative value of the current state, regardless of whether state transitions were caused by the FP, CSs, or USs. Positive value-coding neurons increased their firing during the fixation interval and fired more strongly after rewarded CSs and rewards than after punished CSs and air puffs. Negative value-coding neurons did the opposite, decreasing their firing during the fixation interval and firing more strongly after punished CSs and air puffs than after rewarded CSs and rewards. This representation of state value could underlie how the amygdala helps coordinate cognitive, emotional, and behavioral responses depending on the value of one's state.

Shared neural coding for social hierarchy and reward value in primate amygdala.

The social brain hypothesis posits that dedicated neural systems process social information. In support of this, neurophysiological data have shown that some brain regions are specialized for representing faces. It remains unknown, however, whether distinct anatomical substrates also represent more complex social variables, such as the hierarchical rank of individuals within a social group. Here we show that the primate amygdala encodes the hierarchical rank of individuals in the same neuronal ensembles that encode the rewards associated with nonsocial stimuli. By contrast, orbitofrontal and anterior cingulate cortices lack strong representations of hierarchical rank while still representing reward values. These results challenge the conventional view that dedicated neural systems process social information. Instead, information about hierarchical rank-which contributes to the assessment of the social value of individuals within a group-is linked in the amygdala to representations of rewards associated with nonsocial stimuli.