Temporal Dynamics of Uncertainty Cause Anxiety and Avoidance.

Alfred Hitchcock, film director and "Master of Suspense," observed that terror is not driven by a negative event, but "only in the anticipation of it." This observation is not restricted to the movies: Anxiety builds when we anticipate uncertain negative events, and heightened reactivity during uncertain threat anticipation is a transdiagnostic marker of anxiety (Grupe & Nitschke, 2013; Holley & Fox, 2022; Hur et al., 2020; Krain et al., 2008; Simmons et al., 2008; Yassa et al., 2012). Here, we manipulate the temporal dynamics of an uncertain threat to demonstrate how the evolving expectation of threat can lead people to forgo rewards and experience fear/anxiety. Specifically, we show that increased "hazard rate," which can build during periods of uncertainty, promotes a tendency to avoid threatening contexts while increasing fear/anxiety. These results provide insight into why the anticipation of temporally uncertain threats elicits fear/anxiety, and reframe the underlying causes of related psychopathology.

PET-measured human dopamine synthesis capacity and receptor availability predict trading rewards and time-costs during foraging.

Foraging behavior requires weighing costs of time to decide when to leave one reward patch to search for another. Computational and animal studies suggest that striatal dopamine is key to this process; however, the specific role of dopamine in foraging behavior in humans is not well characterized. We use positron emission tomography (PET) imaging to directly measure dopamine synthesis capacity and D1 and D2/3 receptor availability in 57 healthy adults who complete a computerized foraging task. Using voxelwise data and principal component analysis to identify patterns of variation across PET measures, we show that striatal D1 and D2/3 receptor availability and a pattern of mesolimbic and anterior cingulate cortex dopamine function are important for adjusting the threshold for leaving a patch to explore, with specific sensitivity to changes in travel time. These findings suggest a key role for dopamine in trading reward benefits against temporal costs to modulate behavioral adaptions to changes in the reward environment critical for foraging.

Cognitive maps: Constructing a route with your snout.

Humans construct cognitive maps of the physical, imagined, and abstract world around us based on visually sampled information. A new study shows how the human brain can also use olfactory cues to form and use cognitive maps.

Goal-oriented representations in the human hippocampus during planning and navigation.

Recent work in cognitive and systems neuroscience has suggested that the hippocampus might support planning, imagination, and navigation by forming cognitive maps that capture the abstract structure of physical spaces, tasks, and situations. Navigation involves disambiguating similar contexts, and the planning and execution of a sequence of decisions to reach a goal. Here, we examine hippocampal activity patterns in humans during a goal-directed navigation task to investigate how contextual and goal information are incorporated in the construction and execution of navigational plans. During planning, hippocampal pattern similarity is enhanced across routes that share a context and a goal. During navigation, we observe prospective activation in the hippocampus that reflects the retrieval of pattern information related to a key-decision point. These results suggest that, rather than simply representing overlapping associations or state transitions, hippocampal activity patterns are shaped by context and goals.

Neural mechanisms of credit assignment for inferred relationships in a structured world.

Animals abstract compact representations of a task's structure, which supports accelerated learning and flexible behavior. Whether and how such abstracted representations may be used to assign credit for inferred, but unobserved, relationships in structured environments are unknown. We develop a hierarchical reversal-learning task and Bayesian learning model to assess the computational and neural mechanisms underlying how humans infer specific choice-outcome associations via structured knowledge. We find that the medial prefrontal cortex (mPFC) efficiently represents hierarchically related choice-outcome associations governed by the same latent cause, using a generalized code to assign credit for both experienced and inferred outcomes. Furthermore, the mPFC and lateral orbitofrontal cortex track the current "position" within a latent association space that generalizes over stimuli. Collectively, these findings demonstrate the importance of both tracking the current position in an abstracted task space and efficient, generalizable representations in the prefrontal cortex for supporting flexible learning and inference in structured environments.

The hippocampus and orbitofrontal cortex jointly represent task structure during memory-guided decision making.

The hippocampus, well known for its role in episodic memory, might also be an important brain region for extracting structure from our experiences in order to guide future decisions. Recent evidence in rodents suggests that the hippocampus supports decision making by representing task structure in cooperation with the orbitofrontal cortex (OFC). Here, we examine how the human hippocampus and OFC represent task structure during an associative learning task that required learning of both context-determined and context-invariant probabilistic associations. We find that after learning, hippocampal and lateral OFC representations differentiated between context-determined and context-invariant task structures. The degree of this differentiation within the hippocampus and lateral OFC is highly correlated. These results advance our understanding of the hippocampus and suggest that the hippocampus and OFC support goal-directed behavior by representing information that guides the selection of appropriate decision strategies.

Inferences on a multidimensional social hierarchy use a grid-like code.

Generalizing experiences to guide decision-making in novel situations is a hallmark of flexible behavior. Cognitive maps of an environment or task can theoretically afford such flexibility, but direct evidence has proven elusive. In this study, we found that discretely sampled abstract relationships between entities in an unseen two-dimensional social hierarchy are reconstructed into a unitary two-dimensional cognitive map in the hippocampus and entorhinal cortex. We further show that humans use a grid-like code in entorhinal cortex and medial prefrontal cortex for inferred direct trajectories between entities in the reconstructed abstract space during discrete decisions. These grid-like representations in the entorhinal cortex are associated with decision value computations in the medial prefrontal cortex and temporoparietal junction. Collectively, these findings show that grid-like representations are used by the human brain to infer novel solutions, even in abstract and discrete problems, and suggest a general mechanism underpinning flexible decision-making and generalization.

The orbital frontal cortex, task structure, and inference.

The orbital frontal cortex (OFC) has long been linked to goal-directed, flexible behaviors. Recent evidence suggests the OFC plays key roles in representing the abstracted structure of task spaces, and using this representation for flexible inferences during both learning and choice. Here, we review convergent evidence from studies in animal models and humans in support of this view. We begin by considering early accounts of OFC function, then discuss how more recent evidence supports theories that have re-cast OFC's function as representing the structure of a task or environment for flexible inference. Finally, we turn to neural recording studies that provide insights into the underlying representations and computations the OFC may implement in coordination with other brain areas. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Protocol for building a cognitive map of structural knowledge in humans by integrating abstract relationships from separate experiences.

Humans are adept at learning the latent structure of the relationship between abstract concepts and can build a cognitive map from limited experiences. However, examining internal representations of the cognitive map is challenging because they are unobservable and differ across individuals. Here, we introduce a behavioral training protocol designed for human participants to implicitly build a map of two-dimensional social hierarchies while making a series of binary choices and analytic tools for measuring the internal representation of this structural knowledge. For complete details on the use and execution of this protocol, please refer to Park et al. (2020a, 2020b).

Map Making: Constructing, Combining, and Inferring on Abstract Cognitive Maps.

Cognitive maps enable efficient inferences from limited experience that can guide novel decisions. We tested whether the hippocampus (HC), entorhinal cortex (EC), and ventromedial prefrontal cortex (vmPFC)/medial orbitofrontal cortex (mOFC) organize abstract and discrete relational information into a cognitive map to guide novel inferences. Subjects learned the status of people in two unseen 2D social hierarchies, with each dimension learned on a separate day. Although one dimension was behaviorally relevant, multivariate activity patterns in HC, EC, and vmPFC/mOFC were linearly related to the Euclidean distance between people in the mentally reconstructed 2D space. Hubs created unique comparisons between the hierarchies, enabling inferences between novel pairs. We found that both behavior and neural activity in EC and vmPFC/mOFC reflected the Euclidean distance to the retrieved hub, which was reinstated in HC. These findings reveal how abstract and discrete relational structures are represented, are combined, and enable novel inferences in the human brain.

Neural computations underlying strategic social decision-making in groups.

When making decisions in groups, the outcome of one's decision often depends on the decisions of others, and there is a tradeoff between short-term incentives for an individual and long-term incentives for the groups. Yet, little is known about the neurocomputational mechanisms at play when weighing different utilities during repeated social interactions. Here, using model-based fMRI and Public-good-games, we find that the ventromedial prefrontal cortex encodes immediate expected rewards as individual utility while the lateral frontopolar cortex encodes group utility (i.e., pending rewards of alternative strategies beneficial for the group). When it is required to change one's strategy, these brain regions exhibited changes in functional interactions with brain regions engaged in switching strategies. Moreover, the anterior cingulate cortex and the temporoparietal junction updated beliefs about the decision of others during interactions. Together, our findings provide a neurocomputational account of how the brain dynamically computes effective strategies to make adaptive collective decisions.

Value, search, persistence and model updating in anterior cingulate cortex.

Dorsal anterior cingulate cortex (dACC) carries a wealth of value-related information necessary for regulating behavioral flexibility and persistence. It signals error and reward events informing decisions about switching or staying with current behavior. During decision-making, it encodes the average value of exploring alternative choices (search value), even after controlling for response selection difficulty, and during learning, it encodes the degree to which internal models of the environment and current task must be updated. dACC value signals are derived in part from the history of recent reward integrated simultaneously over multiple time scales, thereby enabling comparison of experience over the recent and extended past. Such ACC signals may instigate attentionally demanding and difficult processes such as behavioral change via interactions with prefrontal cortex. However, the signal in dACC that instigates behavioral change need not itself be a conflict or difficulty signal.

Observational learning computations in neurons of the human anterior cingulate cortex.

When learning from direct experience, neurons in the primate brain have been shown to encode a teaching signal used by algorithms in artificial intelligence: the reward prediction error (PE)-the difference between how rewarding an event is, and how rewarding it was expected to be. However, in humans and other species learning often takes place by observing other individuals. Here, we show that, when humans observe other players in a card game, neurons in their rostral anterior cingulate cortex (rACC) encode both the expected value of an observed choice, and the PE after the outcome was revealed. Notably, during the same task neurons recorded in the amygdala (AMY) and the rostromedial prefrontal cortex (rmPFC) do not exhibit this type of encoding. Our results suggest that humans learn by observing others, at least in part through the encoding of observational PEs in single neurons in the rACC.

Two Anatomically and Computationally Distinct Learning Signals Predict Changes to Stimulus-Outcome Associations in Hippocampus.

Complex cognitive processes require sophisticated local processing but also interactions between distant brain regions. It is therefore critical to be able to study distant interactions between local computations and the neural representations they act on. Here we report two anatomically and computationally distinct learning signals in lateral orbitofrontal cortex (lOFC) and the dopaminergic ventral midbrain (VM) that predict trial-by-trial changes to a basic internal model in hippocampus. To measure local computations during learning and their interaction with neural representations, we coupled computational fMRI with trial-by-trial fMRI suppression. We find that suppression in a medial temporal lobe network changes trial-by-trial in proportion to stimulus-outcome associations. During interleaved choice trials, we identify learning signals that relate to outcome type in lOFC and to reward value in VM. These intervening choice feedback signals predicted the subsequent change to hippocampal suppression, suggesting a convergence of signals that update the flexible representation of stimulus-outcome associations.

Unilateral medial frontal cortex lesions cause a cognitive decision-making deficit in rats.

The medial frontal cortex (MFC) is critical for cost-benefit decision-making. Generally, cognitive and reward-based behaviour in rodents is not thought to be lateralised within the brain. In this study, however, we demonstrate that rats with unilateral MFC lesions show a profound change in decision-making on an effort-based decision-making task. Furthermore, unilateral MFC lesions have a greater effect when the rat has to choose to put in more effort for a higher reward when it is on the contralateral side of space to the lesion. Importantly, this could not be explained by motor impairments as these animals did not show a turning bias in separate experiments. In contrast, rats with unilateral dopaminergic midbrain lesions did exhibit a motoric turning bias, but were unimpaired on the effort-based decision-making task. This rare example of a cognitive deficit caused by a unilateral cortical lesion in the rat brain indicates that the MFC may have a specialised and lateralised role in evaluating the costs and benefits of actions directed to specific spatial locations.

The behavioral and neural mechanisms underlying the tracking of expertise.

Evaluating the abilities of others is fundamental for successful economic and social behavior. We investigated the computational and neurobiological basis of ability tracking by designing an fMRI task that required participants to use and update estimates of both people and algorithms' expertise through observation of their predictions. Behaviorally, we find a model-based algorithm characterized subject predictions better than several alternative models. Notably, when the agent's prediction was concordant rather than discordant with the subject's own likely prediction, participants credited people more than algorithms for correct predictions and penalized them less for incorrect predictions. Neurally, many components of the mentalizing network-medial prefrontal cortex, anterior cingulate gyrus, temporoparietal junction, and precuneus-represented or updated expertise beliefs about both people and algorithms. Moreover, activity in lateral orbitofrontal and medial prefrontal cortex reflected behavioral differences in learning about people and algorithms. These findings provide basic insights into the neural basis of social learning.

Ventromedial prefrontal and anterior cingulate cortex adopt choice and default reference frames during sequential multi-alternative choice.

Although damage to the medial frontal cortex causes profound decision-making impairments, it has been difficult to pinpoint the relative contributions of key anatomical subdivisions. Here we use function magnetic resonance imaging to examine the contributions of human ventromedial prefrontal cortex (vmPFC) and dorsal anterior cingulate cortex (dACC) during sequential choices between multiple alternatives--two key features of choices made in ecological settings. By carefully constructing options whose current value at any given decision was dissociable from their longer term value, we were able to examine choices in current and long-term frames of reference. We present evidence showing that activity at choice and feedback in vmPFC and dACC was tied to the current choice and the best long-term option, respectively. vmPFC, mid-cingulate, and posterior cingulate cortex encoded the relative value between the chosen and next best option at each sequential decision, whereas dACC encoded the relative value of adapting choices from the option with the highest value in the longer term. Furthermore, at feedback we identify temporally dissociable effects that predict repetition of the current choice and adaptation away from the long-term best option in vmPFC and dACC, respectively. These functional dissociations at choice and feedback suggest that sequential choices are subject to competing cortical mechanisms.

Counterfactual choice and learning in a neural network centered on human lateral frontopolar cortex.

Decision making and learning in a real-world context require organisms to track not only the choices they make and the outcomes that follow but also other untaken, or counterfactual, choices and their outcomes. Although the neural system responsible for tracking the value of choices actually taken is increasingly well understood, whether a neural system tracks counterfactual information is currently unclear. Using a three-alternative decision-making task, a Bayesian reinforcement-learning algorithm, and fMRI, we investigated the coding of counterfactual choices and prediction errors in the human brain. Rather than representing evidence favoring multiple counterfactual choices, lateral frontal polar cortex (lFPC), dorsomedial frontal cortex (DMFC), and posteromedial cortex (PMC) encode the reward-based evidence favoring the best counterfactual option at future decisions. In addition to encoding counterfactual reward expectations, the network carries a signal for learning about counterfactual options when feedback is available-a counterfactual prediction error. Unlike other brain regions that have been associated with the processing of counterfactual outcomes, counterfactual prediction errors within the identified network cannot be related to regret theory. Furthermore, individual variation in counterfactual choice-related activity and prediction error-related activity, respectively, predicts variation in the propensity to switch to profitable choices in the future and the ability to learn from hypothetical feedback. Taken together, these data provide both neural and behavioral evidence to support the existence of a previously unidentified neural system responsible for tracking both counterfactual choice options and their outcomes.

Frontal cortex and reward-guided learning and decision-making.

Reward-guided decision-making and learning depends on distributed neural circuits with many components. Here we focus on recent evidence that suggests four frontal lobe regions make distinct contributions to reward-guided learning and decision-making: the lateral orbitofrontal cortex, the ventromedial prefrontal cortex and adjacent medial orbitofrontal cortex, anterior cingulate cortex, and the anterior lateral prefrontal cortex. We attempt to identify common themes in experiments with human participants and with animal models, which suggest roles that the areas play in learning about reward associations, selecting reward goals, choosing actions to obtain reward, and monitoring the potential value of switching to alternative courses of action.

Distributed and causal influence of frontal operculum in task control.

It has been suggested that the frontal operculum (fO) is a key node in a network for exerting control over cognitive processes. How it exerts this influence, however, has been unclear. Here, using the complementary approaches of functional MRI and transcranial magnetic stimulation, we have shown that the fO regulates increases and decreases of activity in multiple occipitotemporal cortical areas when task performance depended on directing attention to different classes of stimuli held in memory. Only one region, the fO, was significantly more active when subjects selectively attended to a single stimulus so that it determined task performance. The stimuli that guided task performance could belong to three categories--houses, body parts, and faces--associated with three occipitotemporal regions. On each trial, the pattern of functional correlation between the fO and the three occipitotemporal regions became either positive or negative, depending on which stimulus was to be attended and which ignored. Activation of the fO preceded both activity increases and decreases in the occipitotemporal cortex. The causal dependency of the distributed occipitotemporal pattern of activity increases and decreases on the fO was demonstrated by showing that transcranial magnetic stimulation-mediated interference of the fO diminished top-down selective attentional modulation in the occipitotemporal cortex, but it did not alter bottom-up activation of the same areas to the same stimuli when they were presented in isolation. The fO's prominence in cognitive control may stem from a role in regulating the level of activity of representations in posterior brain areas that are relevant or irrelevant, respectively, for response selection.