Flexible decision-making is related to strategy learning, vicarious trial and error, and medial prefrontal rhythms during spatial set-shifting.

Flexible decision-making requires a balance between exploring features of an environment and exploiting prior knowledge. Behavioral flexibility is typically measured by how long it takes subjects to consistently make accurate choices after reward contingencies switch or task rules change. This measure, however, only allows for tracking flexibility across multiple trials, and does not assess the degree of flexibility. Plus, although increases in decision-making accuracy are strong indicators of learning, other decision-making behaviors have also been suggested as markers of flexibility, such as the on-the-fly decision reversals known as vicarious trial and error (VTE) or switches to a different, but incorrect, strategy. We sought to relate flexibility, learning, and neural activity by comparing choice history-derived evaluation of strategy use with changes in decision-making accuracy and VTE behavior while recording from the medial prefrontal cortex (mPFC) in rats. Using a set-shifting task that required rats to repeatedly switch between spatial decision-making strategies, we show that a previously developed strategy likelihood estimation procedure could identify putative learning points based on decision history. We confirm the efficacy of learning point estimation by showing increases in decision-making accuracy aligned to the learning point. Additionally, we show increases in the rate of VTE behavior surrounding identified learning points. By calculating changes in strategy likelihoods across trials, we tracked flexibility on a trial-by-trial basis and show that flexibility scores also increased around learning points. Further, we demonstrate that VTE behaviors could be separated into indecisive and deliberative subtypes depending on whether they occurred during periods of high or low flexibility and whether they led to correct or incorrect choice outcomes. Field potential recordings from the mPFC during decisions exhibited increased beta band activity on trials with VTE compared to non-VTE trials, as well as increased gamma during periods when learned strategies could be exploited compared to prelearning, exploratory periods. This study demonstrates that increased behavioral flexibility and VTE rates are often aligned to task learning. These relationships can break down, however, suggesting that VTE is not always an indicator of deliberative decision-making. Additionally, we further implicate the mPFC in decision-making and learning by showing increased beta-based activity on VTE trials and increased gamma after learning.

The medial prefrontal cortex during flexible decisions: Evidence for its role in distinct working memory processes.

During decisions that involve working memory, task-related information must be encoded, maintained across delays, and retrieved. Few studies have attempted to causally disambiguate how different brain structures contribute to each of these components of working memory. In the present study, we used transient optogenetic disruptions of rat medial prefrontal cortex (mPFC) during a serial spatial reversal learning (SSRL) task to test its role in these specific working memory processes. By analyzing numerous performance metrics, we found: (1) mPFC disruption impaired performance during only the choice epoch of initial discrimination learning of the SSRL task; (2) mPFC disruption impaired performance in dissociable ways across all task epochs (delay, choice, return) during flexible decision-making; (3) mPFC disruption resulted in a reduction of the typical vicarious-trial-and-error rate modulation that was related to changes in task demands. Taken together, these findings suggest that the mPFC plays an outsized role in working memory retrieval, becomes involved in encoding and maintenance when recent memories conflict with task demands, and enables animals to flexibly utilize working memory to update behavior as environments change.

A selective role for the mPFC during choice and deliberation, but not spatial memory retention over short delays.

Important interactions between memory and decision-making processes are required to maintain high-levels of spatial working memory task performance. Past research reveals that the medial prefrontal cortex (mPFC) and hippocampus (HPC) are both vital structures involved in these processes. Recent evidence suggests that interactions between these two structures are dynamic and task dependent. However, there exists uncertainty surrounding the specific conditions that recruit mPFC contributions to these tasks, specifically regarding its role in retaining information online during delay periods. To address this issue, we tested rats on a spatial-delayed alternation task in which we utilized a closed-loop optogenetic system to transiently disrupt mPFC activity during different task epochs (delay, choice, return). By analyzing the effects of mPFC disruption on choice accuracy and a deliberative behavior known as vicarious-trial-and-error (VTE), our study revealed several interesting findings regarding the role of the mPFC in spatial-working memory tasks. The main findings include: (a) choice accuracy in the spatial-delayed alternation (SDA) task was impaired when the mPFC was disrupted during the choice epoch and not delay or return epochs, (b) mPFC disruption resulted in a non-epoch specific reduction in VTE occurrence which correlated with impairments in task performance. Taken together, findings from this study suggest that, during spatial decision-making, contributions made by the mPFC are specific to points of deliberation and choice (not delay), and that VTEs are a deliberative behavior which relies on intact mPFC function.

The Lateral Habenula Circuitry: Reward Processing and Cognitive Control.

There has been a growing interest in understanding the role of the lateral habenula (LHb) in reward processing, affect regulation, and goal-directed behaviors. The LHb gets major inputs from the habenula-projecting globus pallidus and the mPFC, sending its efferents to the dopaminergic VTA and SNc, serotonergic dorsal raphe nuclei, and the GABAergic rostromedial tegmental nucleus. Recent studies have made advances in our understanding of the LHb circuit organization, yet the precise mechanisms of its involvement in complex behaviors are largely unknown. To begin to address this unresolved question, we present here emerging cross-species perspectives with a goal to provide a more refined understanding of the role of the LHb circuits in reward and cognition. We begin by highlighting recent findings from rodent experiments using optogenetics, electrophysiology, molecular, pharmacology, and tracing techniques that reveal diverse neural phenotypes in the LHb circuits that may underlie previously undescribed behavioral functions. We then discuss results from electrophysiological studies in macaques that suggest that the LHb cooperates with the anterior cingulate cortex to monitor action outcomes and signal behavioral adjustment. Finally, we provide an integrated summary of cross-species findings and discuss how further research on the connectivity, neural signaling, and physiology of the LHb circuits can deepen our understanding of the role of the LHb in normal and maladaptive behaviors associated with mental illnesses and drug abuse.

Hippocampal neural activity reflects the economy of choices during goal-directed navigation.

Distinguishing spatial contexts is likely essential for the well-known role of the hippocampus in episodic memory. We studied whether types of hippocampal neural organization thought to underlie context discrimination are impacted by learned economic considerations of choice behavior. Hippocampal place cells and theta activity were recorded as rats performed a maze-based probability discounting task that involved choosing between a small certain reward or a large probabilistic reward. Different spatial distributions of place fields were observed in response to changes in probability, the outcome of the rats' choice, and whether or not rats were free to make that choice. The degree to which the reward location was represented by place cells scaled with the expected probability of rewards. Theta power increased around the goal location also in proportion to the expected probability of signaled rewards. Furthermore, theta power dynamically varied as specific econometric information was obtained "on the fly" during task performance. Such an economic perspective of memory processing by hippocampal place cells expands our view of the nature of context memories retrieved by hippocampus during adaptive navigation.

A framework for understanding and advancing intertemporal choice research using rodent models.

Intertemporal choices are common and consequential to private and public life. Thus, there is considerable interest in understanding the neural basis of intertemporal decision making. In this minireview, we briefly describe conceptual and psychological perspectives on intertemporal choice and then provide a comprehensive evaluation of the neural structures and signals that comprise the underlying cortico-limbic-striatal circuit. Even though great advances have been made, our understanding of the neurobiology of intertemporal choice is still in its infancy because of the complex and dynamic nature of this form of decision making. We close by briefly discussing recommendations for the future study of intertemporal choice research.

Prefrontal Regulation of Neuronal Activity in the Ventral Tegmental Area.

Growing evidence indicates that midbrain dopamine (DA) cells integrate reward expectancy-related information from the prefrontal cortex to properly compute errors in reward prediction. Here we investigated how 2 major prefrontal subregions, the orbitofrontal cortex (OFC) and the medial prefrontal cortex (mPFC), contributed to DAergic prediction errors while rats performed a delay discounting task on a T-maze. Most putative DA cells in the task showed phasic responses to salient cues that predicted delayed rewards, but not to the actual rewards. After temporary inactivation of the OFC, putative DA cells exhibited strikingly reduced phasic responses to reward-predicting cues but increased responses to rewards. In contrast, mPFC inactivation significantly elevated DA responses to both predictive cues and rewards. In addition, OFC, but not mPFC, inactivation disrupted the activity of putative non-DA cells that encoded expected reward values during waiting periods. These results suggest that the 2 prefrontal subregions differentially regulate DAergic prediction errors and the OFC conveys value signals to midbrain dopamine systems.

A role for the lateral dorsal tegmentum in memory and decision neural circuitry.

A role for the hippocampus in memory is clear, although the mechanism for its contribution remains a matter of debate. Converging evidence suggests that hippocampus evaluates the extent to which context-defining features of events occur as expected. The consequence of mismatches, or prediction error, signals from hippocampus is discussed in terms of its impact on neural circuitry that evaluates the significance of prediction errors: Ventral tegmental area (VTA) dopamine cells burst fire to rewards or cues that predict rewards (Schultz, Dayan, & Montague, 1997). Although the lateral dorsal tegmentum (LDTg) importantly controls dopamine cell burst firing (Lodge & Grace, 2006) the behavioral significance of the LDTg control is not known. Therefore, we evaluated LDTg functional activity as rats performed a spatial memory task that generates task-dependent reward codes in VTA (Jo, Lee, & Mizumori, 2013; Puryear, Kim, & Mizumori, 2010) and another VTA afferent, the pedunculopontine nucleus (PPTg, Norton, Jo, Clark, Taylor, & Mizumori, 2011). Reversible inactivation of the LDTg significantly impaired choice accuracy. LDTg neurons coded primarily egocentric information in the form of movement velocity, turning behaviors, and behaviors leading up to expected reward locations. A subset of the velocity-tuned LDTg cells also showed high frequency bursts shortly before or after reward encounters, after which they showed tonic elevated firing during consumption of small, but not large, rewards. Cells that fired before reward encounters showed stronger correlations with velocity as rats moved toward, rather than away from, rewarded sites. LDTg neural activity was more strongly regulated by egocentric behaviors than that observed for PPTg or VTA cells that were recorded by Puryear et al. and Norton et al. While PPTg activity was uniquely sensitive to ongoing sensory input, all three regions encoded reward magnitude (although in different ways), reward expectation, and reward encounters. Only VTA encoded reward prediction errors. LDTg may inform VTA about learned goal-directed movement that reflects the current motivational state, and this in turn may guide VTA determination of expected subjective goal values. When combined it is clear the LDTg and PPTg provide only a portion of the information that dopamine cells need to assess the value of prediction errors, a process that is essential to future adaptive decisions and switches of cognitive (i.e. memorial) strategies and behavioral responses.

Effects of prefrontal cortical inactivation on neural activity in the ventral tegmental area.

Dopamine (DA) cells have been suggested to signal discrepancies between expected and actual rewards in reinforcement learning. DA cells in the ventral tegmental area (VTA) receive direct projections from the medial prefrontal cortex (mPFC), a structure known to be one of the brain areas that represents expected future rewards. To investigate whether the mPFC contributes to generating reward prediction error signals of DA cells, we recorded VTA cells from rats foraging for different amounts of reward in a spatial working memory task. Our results showed that DA cells initially responded after the acquisition of rewards, but over training, they exhibited phasic responses when rats detected sensory cues originating from the rewards before obtaining them. We also observed two separate groups of non-DA cells activated in expectation of upcoming rewards or during reward consumption. Bilateral injections of muscimol, a GABAA agonist, into the mPFC significantly decreased the non-DA activity that encoded reward expectation. By contrast, the same manipulation of the mPFC elevated DA responses to reward-predicting cues. However, neither DA nor non-DA responses elicited after reward acquisition were affected by mPFC inactivation. These results suggest that the mPFC provides information about expected rewards to the VTA, and its functional loss elevates DA responses to reward-predicting cues by altering expectations about forthcoming rewards.

Thriving in neuroscience careers: Three lessons from 12+ years of the BRAINS Program.

The NINDS-funded BRAINS Program for neuroscientists from underrepresented and marginalized groups has positively impacted its participants and the field. We discuss three lessons to advance excellence and diversity: center relationships, provide ongoing engagement, and leverage programmatic expertise.

Homeostatic regulation of memory systems and adaptive decisions.

While it is clear that many brain areas process mnemonic information, understanding how their interactions result in continuously adaptive behaviors has been a challenge. A homeostatic-regulated prediction model of memory is presented that considers the existence of a single memory system that is based on a multilevel coordinated and integrated network (from cells to neural systems) that determines the extent to which events and outcomes occur as predicted. The "multiple memory systems of the brain" have in common output that signals errors in the prediction of events and/or their outcomes, although these signals differ in terms of what the error signal represents (e.g., hippocampus: context prediction errors vs. midbrain/striatum: reward prediction errors). The prefrontal cortex likely plays a pivotal role in the coordination of prediction analysis within and across prediction brain areas. By virtue of its widespread control and influence, and intrinsic working memory mechanisms. Thus, the prefrontal cortex supports the flexible processing needed to generate adaptive behaviors and predict future outcomes. It is proposed that prefrontal cortex continually and automatically produces adaptive responses according to homeostatic regulatory principles: prefrontal cortex may serve as a controller that is intrinsically driven to maintain in prediction areas an experience-dependent firing rate set point that ensures adaptive temporally and spatially resolved neural responses to future prediction errors. This same drive by prefrontal cortex may also restore set point firing rates after deviations (i.e. prediction errors) are detected. In this way, prefrontal cortex contributes to reducing uncertainty in prediction systems. An emergent outcome of this homeostatic view may be the flexible and adaptive control that prefrontal cortex is known to implement (i.e. working memory) in the most challenging of situations. Compromise to any of the prediction circuits should result in rigid and suboptimal decision making and memory as seen in addiction and neurological disease.

Hippocampal beta rhythms as a bridge between sensory learning and memory-guided decision-making.

A pillar of systems neuroscience has been the study of neural oscillations. Research into these oscillations spans brain areas, species, and disciplines, giving us common ground for discussing typically disparate fields of neuroscience. In this review, we aim to strengthen the dialog between sensory systems research and learning and memory systems research by examining a 15-40 Hz oscillation known as the beta rhythm. Starting with foundational observations based largely in olfactory systems neuroscience, we review evidence suggesting beta-based activity may extend across sensory systems generally, as well as into the hippocampus and areas well known for coordinating decisions and memory-guided behaviors. After evaluating this work, we propose a framework wherein the hippocampal beta oscillation and its diverse coupling with other brain areas can support both sensory learning and memory-guided decision-making. Using this framework, we also propose circuitries that may support these processes, and experiments to test our hypothesis.

Connecting Counterspaces and Community Cultural Wealth in a Professional Development Program.

This qualitative study analyzes the relationship between two concepts from critical race theory-counterspaces and community cultural wealth. Counterspaces are supportive, identity-affirming community spaces, while community cultural wealth highlights the importance of the knowledge, skills, and networks used by individuals belonging to marginalized groups to successfully navigate academia. This study investigates the hypothesis that the processes operating within counterspaces serve to strengthen an individual's access to their community cultural wealth. The study site is BRAINS, a U.S.-based professional development program for early-career academic neuroscientists from underrepresented groups. Findings revealed that two types of counterspace processes (narrative identity work and direct relational transactions) and three types of community cultural wealth (aspirational capital, social capital, and navigational capital) are most salient within BRAINS. After examining the complex interactions connecting counterspace processes and community cultural wealth, we offer recommendations for future professional development programs and research designed to broaden participation in academia.

Complimentary roles of the hippocampus and retrosplenial cortex in behavioral context discrimination.

Complex cognitive functions, such as learning and memory, arise from the interaction of multiple brain regions that comprise functional circuits and different components of these circuits make unique contributions to learning. The hippocampus and the retrosplenial cortex (RSC) are anatomically interconnected and both regions are involved in learning and memory. Previous studies indicate that the hippocampus exhibits unique firing patterns for different contexts and that RSC neurons selectively respond to cues that predict reinforcement or the need for a behavioral response, suggesting a hippocampal role in encoding contexts and an RSC role in encoding behaviorally significant cues. To test this, we simultaneously recorded hippocampal and RSC neuronal activity as rats learned to discriminate two behavioral contexts. The rats learned to approach the east arm of a plus maze for reward during the first half of each session and to approach the west arm during the second half. The "go east" and "go west" conditions constitute distinct behavioral contexts, which were cued by the reward location. Neurons in both regions developed highly context-specific responses as subjects learned to discriminate the contexts, but the response patterns differed in the two brain regions. Consistent with a context processing role, hippocampal neurons developed context-specific responses to a variety of task stimuli and events. In contrast, RSC neurons only developed context-specific responses to the reward location, which served as the context identifying cue. These results suggest that the hippocampus and RSC play distinct, but complimentary roles in mediating context appropriate memories and behaviors.

Disruption of NMDAR-dependent burst firing by dopamine neurons provides selective assessment of phasic dopamine-dependent behavior.

Midbrain dopamine (DA) neurons fire in 2 characteristic modes, tonic and phasic, which are thought to modulate distinct aspects of behavior. However, the inability to selectively disrupt these patterns of activity has hampered the precise definition of the function of these modes of signaling. Here, we addressed the role of phasic DA in learning and other DA-dependent behaviors by attenuating DA neuron burst firing and subsequent DA release, without altering tonic neural activity. Disruption of phasic DA was achieved by selective genetic inactivation of NMDA-type, ionotropic glutamate receptors in DA neurons. Disruption of phasic DA neuron activity impaired the acquisition of numerous conditioned behavioral responses, and dramatically attenuated learning about cues that predicted rewarding and aversive events while leaving many other DA-dependent behaviors unaffected.

Ventral tegmental area and substantia nigra neural correlates of spatial learning.

The ventral tegmental area (VTA) and substantia nigra pars compacta (SNc) may provide modulatory signals that, respectively, influence hippocampal (HPC)- and striatal-dependent memory. Electrophysiological studies investigating neural correlates of learning and memory of dopamine (DA) neurons during classical conditioning tasks have found DA neural activity in VTA and SNc to be tightly coupled with reinforcement expectations. Also, VTA integrity and DA in HPC have been found to regulate the encoding of HPC-dependent memories. Therefore, to determine the nature of the neural code HPC may receive from midbrain DA regions, the present study investigated VTA and SNc neural activity as navigating rats engaged in new spatial learning and experienced changes in expected goal locations. VTA and SNc cells were differentially engaged during training to a series of three novel goal locations. During task acquisition, the peak firing rates of VTA neurons decreased at the time of reward and shifted to time points before reward retrieval, whereas the peak firing rates of SNc neurons remained elevated at the time of reward during training to all three goal locations. Both VTA and SNc egocentric coding was strongest during training to the first goal location, which coincided with the time subjects learned the behavioral rules specific to the task. These data imply that VTA and SNc play complementary yet distinct roles in spatial learning to optimize adaptive behavior.

Response Flexibility: The Role of the Lateral Habenula.

The ability to make appropriate decisions that result in an optimal outcome is critical for survival. This process involves assessing the environment as well as integrating prior knowledge about the environment with information about one's current internal state. There are many neural structures that play critical roles in mediating these processes, but it is not yet known how such information coalesces to influence behavioral output. The lateral habenula (LHb) has often been cited as a structure critical for adaptive and flexible responding when environmental contexts and internal state changes. A challenge, however, has been understanding how LHb promotes response flexibility. In this review, we hypothesize that the LHb enables flexible responding following the integration of context memory and internal state information by signaling downstream brainstem structures known to drive hippocampal theta. In this way, animals respond more flexibly in a task situation not because the LHb selects a particular action, but rather because LHb enhances a hippocampal neural state that is often associated with greater attention, arousal, and exploration. In freely navigating animals, these are essential conditions that are needed to discover and implement appropriate alternative choices and behaviors. As a corollary to our hypothesis, we describe short- and intermediate-term functions of the LHb. Finally, we discuss the effects on the behavior of LHb dysfunction in short- and intermediate-timescales, and then suggest that new therapies may act on the LHb to alleviate the behavioral impairments following long-term LHb disruption.

Lateral Habenula Beyond Avoidance: Roles in Stress, Memory, and Decision-Making With Implications for Psychiatric Disorders.

In this Perspective review, we highlight some of the less explored aspects of lateral habenula (LHb) function in contextual memory, sleep, and behavioral flexibility. We provide evidence that LHb is well-situated to integrate different internal state and multimodal sensory information from memory-, stress-, motivational-, and reward-related circuits essential for both survival and decision making. We further discuss the impact of early life stress (ELS) on LHb function as an example of stress-induced hyperactivity and dysregulation of neuromodulatory systems within the LHb that promote anhedonia and motivational deficits following ELS. We acknowledge that recent technological advancements in manipulation and recording of neural circuits in simplified and well-controlled behavioral paradigms have been invaluable in our understanding of the critical role of LHb in motivation and emotional regulation as well as the involvement of LHb dysfunction in stress-induced psychopathology. However, we also argue that the use of ethologically-relevant behaviors with consideration of complex aspects of decision-making is warranted for future studies of LHb contributions in a wide range of psychiatric illnesses. We conclude this Perspective with some of the outstanding issues for the field to consider where a multi-systems approach is needed to investigate the complex nature of LHb circuitry interactions with environmental stimuli that predisposes psychiatric disorders.

Ventral tegmental area disruption selectively affects CA1/CA2 but not CA3 place fields during a differential reward working memory task.

Hippocampus (HPC) receives dopaminergic (DA) projections from the ventral tegmental area (VTA) and substantia nigra. These inputs appear to provide a modulatory signal that influences HPC dependent behaviors and place fields. We examined how efferent projections from VTA to HPC influence spatial working memory and place fields when the reward context changes. CA1 and CA3 process environmental context changes differently and VTA preferentially innervates CA1. Given these anatomical data and electrophysiological evidence that implicate DA in reward processing, we predicted that CA1 place fields would respond more strongly to both VTA disruption and changes in the reward context than CA3 place fields. Rats (N = 9) were implanted with infusion cannula targeting VTA and recording tetrodes aimed at HPC. Then they were tested on a differential reward, win-shift working memory task. One recording session consisted of 5 baseline and 5 manipulation trials during which place cells in CA1/CA2 (N = 167) and CA3 (N = 94) were recorded. Prior to manipulation trials rats were infused with either baclofen or saline and then subjected to control or reward conditions during which the learned locations of large and small reward quantities were reversed. VTA disruption resulted in an increase in errors, and in CA1/CA2 place field reorganization. There were no changes in any measures of CA3 place field stability during VTA disruption. Reward manipulations did not affect performance or place field stability in CA1/CA2 or CA3; however, changes in the reward locations "rescued" performance and place field stability in CA1/CA2 when VTA activity was compromised, perhaps by trigging compensatory mechanisms. These data support the hypothesis that VTA contributes to spatial working memory performance perhaps by maintaining place field stability selectively in CA1/CA2.

Hippocampal episode fields develop with learning.

Several recent studies have shown that hippocampal neurons fire during the delay period in between trials and that these firing patterns differ when different behaviors are required, suggesting that the neuronal responses may be involved in maintaining the memories needed for the upcoming trial. In particular, one study found that hippocampal neurons reliably fired at particular times, referred to as "episode fields" (EFs), during the delay period of a spatial alternation task (Pastalkova et al. (2008) Science 321:1322-1327). The firing of these neurons resulted in distinct sequential firing patterns on left and right turn trials, and these firing patterns could be used to predict the upcoming behavioral response. In this study, we examined neuronal firing during the delay period of a hippocampus-dependent plus maze task, which involved learning to approach two different reward locations (east and west), and we examined the development of these firing patterns with learning. As in the previous study, hippocampal neurons exhibited discrete periods of elevated firing during the delay (EFs) and the firing patterns were distinct on the east and west trials. Moreover, these firing patterns emerged and began to differentiate the east and west conditions during the first training session and continued to develop as the rats learned the task. The finding of similar firing patterns in different tasks suggests that the EFs are a robust phenomenon, which may occur whenever subjects must maintain distinct memory representations during a delay period. Additionally, in the previous study (Pastalkova et al. (2008) Science 321:1322-1327), the distinct firing patterns could have been due to the differing goal locations, behavioral responses (left or right turns), or trajectories. In this study, neuronal firing varied with the goal location regardless of the trajectories or responses, suggesting that the firing patterns encode the behavioral context rather than specific behaviors.