Uncertainty in learning and decision-making: Introduction to the special issue.

Uncertainty is a fundamental aspect of the environment. This special issue presents interdisciplinary research on decision-making and learning under uncertainty. Thirty-one research and review papers report the findings of the behavioral, neural, and computational bases of coping with uncertainty, as well as changes of these mechanisms in development, aging, and psychopathology. Taken together, this special issue presents extant research, identifies gaps in our knowledge, and offers paths for future directions.

Evidence for a minimal role of stimulus awareness in reversal of threat learning.

In an ever-changing environment, survival depends on learning which stimuli represent threat, and also on updating such associations when circumstances shift. It has been claimed that humans can acquire physiological responses to threat-associated stimuli even when they are unaware of them, but the role of awareness in updating threat contingencies remains unknown. This complex process-generating novel responses while suppressing learned ones-relies on distinct neural mechanisms from initial learning, and has only been shown with awareness. Can it occur unconsciously? Here, we present evidence that threat reversal may not require awareness. Participants underwent classical threat conditioning to visual stimuli that were suppressed from awareness. One of two images was paired with an electric shock; halfway through the experiment, contingencies were reversed and the shock was paired with the other image. Despite variations in suppression across participants, we found that physiological responses reflected changes in stimulus-threat pairings independently of stimulus awareness. These findings suggest that unconscious affective processing may be sufficiently flexible to adapt to changing circumstances.

Latent cause inference during extinction learning in trauma-exposed individuals with and without PTSD.

BACKGROUND: Problems in learning that sights, sounds, or situations that were once associated with danger have become safe (extinction learning) may explain why some individuals suffer prolonged psychological distress following traumatic experiences. Although simple learning models have been unable to provide a convincing account of why this learning fails, it has recently been proposed that this may be explained by individual differences in beliefs about the causal structure of the environment. METHODS: Here, we tested two competing hypotheses as to how differences in causal inference might be related to trauma-related psychopathology, using extinction learning data collected from clinically well-characterised individuals with varying degrees of post-traumatic stress (N = 56). Model parameters describing individual differences in causal inference were related to multiple post-traumatic stress disorder (PTSD) and depression symptom dimensions via network analysis. RESULTS: Individuals with more severe PTSD were more likely to assign observations from conditioning and extinction stages to a single underlying cause. Specifically, greater re-experiencing symptom severity was associated with a lower likelihood of inferring that multiple causes were active in the environment. CONCLUSIONS: We interpret these results as providing evidence of a primary deficit in discriminative learning in participants with more severe PTSD. Specifically, a tendency to attribute a greater diversity of stimulus configurations to the same underlying cause resulted in greater uncertainty about stimulus-outcome associations, impeding learning both that certain stimuli were safe, and that certain stimuli were no longer dangerous. In the future, better understanding of the role of causal inference in trauma-related psychopathology may help refine cognitive therapies for these disorders.

The social hippocampus.

The concept of cognitive map has been proposed as a way to organize our experiences and guide behavior across all domains of cognition. The hippocampus has been identified as the neural substrate supporting cognitive maps for navigating physical space. Recent evidence is broadening the role of the hippocampus into mapping other manner of spaces. Here we focus on the case of social space as a candidate for hippocampal representation because it combines multiple continuous dimensions and requires dynamic navigation through social contexts. We present evidence for the role of the hippocampus in (1) supporting social memory, (2) representing different dimensions of social space, (3) tracking dynamic social behavior, (4) maintaining a flexible map allowing adaptation to new social contexts, and (5) maladaptive social behavior across psychiatric disorders. To do so, we explore evidence across species including birds, rodents, nonhuman primates and humans, indicating hippocampal involvement in a range of social processes. Review of previous findings in a manner predicted by the cognitive map supports the existence of systematic mapping of social space by the hippocampus. Evidence for hippocampal social maps complements findings from other abstract domains, such as auditory, temporal and conceptual, allowing successful navigation through many domains of everyday life.

Prazosin during threat discrimination boosts memory of the safe stimulus.

The α-1 adrenoreceptor antagonist prazosin has shown promise in the treatment of post-traumatic stress disorder (PTSD) symptoms, but its mechanisms are not well understood. Here we administered prazosin or placebo prior to threat conditioning (day 1) and tested subsequent extinction (day 2) and reextinction (day 3) in healthy human participants. Prazosin did not affect threat conditioning but augmented stimulus discrimination during extinction and reextinction, via lower responding to the safe stimulus. These results suggest that prazosin during threat acquisition may have influenced encoding or consolidation of safety processing in particular, subsequently leading to enhanced discrimination between the safe and threatening stimuli.

Aversive smell associations shape social judgment.

Once associating another person with an unpleasant smell, how do we perceive and judge this person from that moment on? Here, we used aversive olfactory conditioning followed by a social attribution task during functional magnetic resonance imaging to address this question. After conditioning, where one of two faces was repeatedly paired with an aversive smell, the participants reported negative affect when viewing the smell-conditioned but not the neutral face. When subsequently confronted with the smell-conditioned face (without any smell), the participants tended to judge both positive and negative behaviors as indicative of personality traits rather than related to the situation. This effect was predicted by the degree of the preceding olfactory evaluative conditioning. Whole brain analysis of stimulus by stage interaction indicated differential activation of the ventromedial prefrontal cortex and right angular gyrus to the conditioned versus the neutral person during the attribution phase only. These results suggest that negative smell associations do not simply induce a negative perception of the target person but rather bias the attribution style towards trait attributions. The fact that this bias was evident regardless of behavior valence suggests it may reflect enhanced psychological distance. Thus, the known observation of social rejection triggered by aversive smell may be driven by a shift in social attribution style.

Memory and Space: Towards an Understanding of the Cognitive Map.

More than 50 years of research have led to the general agreement that the hippocampus contributes to memory, but there has been a major schism among theories of hippocampal function over this time. Some researchers argue that the hippocampus plays a broad role in episodic and declarative memory, whereas others argue for a specific role in the creation of spatial cognitive maps and navigation. Although both views have merit, neither provides a complete account of hippocampal function. Guided by recent reviews that attempt to bridge between these views, here we suggest that reconciliation can be accomplished by exploring hippocampal function from the perspective of Tolman's (1948) original conception of a cognitive map as organizing experience and guiding behavior across all domains of cognition. We emphasize recent studies in animals and humans showing that hippocampal networks support a broad range of domains of cognitive maps, that these networks organize specific experiences within the contextually relevant map, and that network activity patterns reflect behavior guided through cognitive maps. These results are consistent with a framework that bridges theories of hippocampal function by conceptualizing the hippocampus as organizing incoming information within the context of a multidimensional cognitive map of spatial, temporal, and associational context. SIGNIFICANCE STATEMENT: Research of hippocampal function is dominated by two major views. The spatial view argues that the hippocampus tracks routes through space, whereas the memory view suggests a broad role in declarative memory. Both views rely on considerable evidence, but neither provides a complete account of hippocampal function. Here we review evidence that, in addition to spatial context, the hippocampus encodes a wide variety of information about temporal and situational context, about the systematic organization of events in abstract space, and about routes through maps of cognition and space. We argue that these findings cross the boundaries of the memory and spatial views and offer new insights into hippocampal function as a system supporting a broad range of cognitive maps.

Preventing the return of fear in humans using reconsolidation update mechanisms.

Recent research on changing fears has examined targeting reconsolidation. During reconsolidation, stored information is rendered labile after being retrieved. Pharmacological manipulations at this stage result in an inability to retrieve the memories at later times, suggesting that they are erased or persistently inhibited. Unfortunately, the use of these pharmacological manipulations in humans can be problematic. Here we introduce a non-invasive technique to target the reconsolidation of fear memories in humans. We provide evidence that old fear memories can be updated with non-fearful information provided during the reconsolidation window. As a consequence, fear responses are no longer expressed, an effect that lasted at least a year and was selective only to reactivated memories without affecting others. These findings demonstrate the adaptive role of reconsolidation as a window of opportunity to rewrite emotional memories, and suggest a non-invasive technique that can be used safely in humans to prevent the return of fear.

Extinction during reconsolidation of threat memory diminishes prefrontal cortex involvement.

Controlling learned defensive responses through extinction does not alter the threat memory itself, but rather regulates its expression via inhibitory influence of the prefrontal cortex (PFC) over amygdala. Individual differences in amygdala-PFC circuitry function have been linked to trait anxiety and posttraumatic stress disorder. This finding suggests that exposure-based techniques may actually be least effective in those who suffer from anxiety disorders. A theoretical advantage of techniques influencing reconsolidation of threat memories is that the threat representation is altered, potentially diminishing reliance on this PFC circuitry, resulting in a more persistent reduction of defensive reactions. We hypothesized that timing extinction to coincide with threat memory reconsolidation would prevent the return of defensive reactions and diminish PFC involvement. Two conditioned stimuli (CS) were paired with shock and the third was not. A day later, one stimulus (reminded CS+) but not the other (nonreminded CS+) was presented 10 min before extinction to reactivate the threat memory, followed by extinction training for all CSs. The recovery of the threat memory was tested 24 h later. Extinction of the nonreminded CS+ (i.e., standard extinction) engaged the PFC, as previously shown, but extinction of the reminded CS+ (i.e., extinction during reconsolidation) did not. Moreover, only the nonreminded CS+ memory recovered on day 3. These results suggest that extinction during reconsolidation prevents the return of defensive reactions and diminishes PFC involvement. Reducing the necessity of the PFC-amygdala circuitry to control defensive reactions may help overcome a primary obstacle in the long-term efficacy of current treatments for anxiety disorders.

Taking action in the face of threat: neural synchronization predicts adaptive coping.

The ability to take action in the face of threat is highly diverse across individuals. What are the neural processes that determine individual differences in the ability to cope with danger? We hypothesized that the extent of synchronization between amygdala, striatum, and medial prefrontal cortex (mPFC) would predict successful active coping performance. To test this, we developed a novel computer task based on the principals of Sidman avoidance. Healthy human participants learned through trial and error to move a marker between virtual game board compartments once every 3 s to avoid mild shocks. Behaviorally, participants exhibited large individual differences. Strikingly, both amygdala-mPFC and caudate-mPFC coupling during active coping trials covaried with final active coping performance across participants. These findings indicate that synchronization between mPFC subregions, and both amygdala and caudate predicts whether individuals will achieve successful active coping performance by the end of training. Thus, successful performance of adaptive actions in the face of threat requires functional synchronization of a neural circuit consisting of mPFC, striatum, and amygdala. Malfunction in the crosstalk between these components might underlie anxiety symptoms and impair individuals' ability to actively cope under stress. This opens an array of possibilities for therapeutic targets for fear and anxiety disorders.

Extinction resistant changes in the human auditory association cortex following threat learning.

Research in humans has highlighted the importance of the amygdala for transient modulation of cortical areas for enhanced processing of emotional stimuli. However, non-human animal data has shown that amygdala dependent threat (fear) learning can also lead to long lasting changes in cortical sensitivity, persisting even after extinction of fear responses. The neural mechanisms of long-lasting traces of such conditioning in humans have not yet been explored. We used functional magnetic resonance imaging (fMRI) and assessed skin conductance responses (SCR) during threat acquisition, extinction learning and extinction retrieval. We provide evidence of lasting cortical plasticity in the human brain following threat extinction and show that enhanced blood oxygen level-dependent (BOLD) signal to the learned threat stimulus in the auditory association cortex is resistant to extinction. These findings point to a parallel avenue by which cortical processing of potentially dangerous stimuli can be long lasting, even when immediate threat and the associated amygdala modulation have subsided.

Neural Trajectories of Conceptually Related Events.

In a series of conceptually related episodes, meaning arises from the link between these events rather than from each event individually. How does the brain keep track of conceptually related sequences of events (i.e., conceptual trajectories)? In a particular kind of conceptual trajectory-a social relationship-meaning arises from a specific sequence of interactions. To test whether such abstract sequences are neurally tracked, we had participants complete a naturalistic narrative-based social interaction game, during functional magnetic resonance imaging. We modeled the simulated relationships as trajectories through an abstract affiliation and power space. In two independent samples, we found evidence of individual social relationships being tracked with unique sequences of hippocampal states. The neural states corresponded to the accumulated trial-to-trial affiliation and power relations between the participant and each character, such that each relationship's history was captured by its own neural trajectory. Each relationship had its own sequence of states, and all relationships were embedded within the same manifold. As such, we show that the hippocampus represents social relationships with ordered sequences of low-dimensional neural patterns. The number of distinct clusters of states on this manifold is also related to social function, as measured by the size of real-world social networks. These results suggest that our evolving relationships with others are represented in trajectory-like neural patterns.

The hippocampus dissociates present from past and future goals.

Our brain adeptly navigates goals across time frames, distinguishing between urgent needs and those of the past or future. The hippocampus is a region known for supporting mental time travel and organizing information along its longitudinal axis, transitioning from detailed posterior representations to generalized anterior ones. This study investigates the role of the hippocampus in distinguishing goals over time: whether the hippocampus encodes time regardless of detail or abstraction, and whether the hippocampus preferentially activates its anterior region for temporally distant goals (past and future) and its posterior region for immediate goals. We use a space-themed experiment with 7T functional MRI on 31 participants to examine how the hippocampus encodes the temporal distance of goals. During a simulated Mars mission, we find that the hippocampus tracks goals solely by temporal proximity. We show that past and future goals activate the left anterior hippocampus, while current goals engage the left posterior hippocampus. This suggests that the hippocampus maps goals using timestamps, extending its long axis system to include temporal goal organization.

Phenotypical divergence between self-reported and clinically ascertained autism.

While allowing for rapid recruitment of large samples, online psychiatric and neurodevelopmental research relies heavily on participants' self-report of neuropsychiatric symptoms, foregoing the rigorous clinical characterization of laboratory settings. Autism spectrum disorder (ASD) research is one example where the clinical validity of such an approach remains elusive. Here, we compared participants characterized online via self-reports against in-person participants evaluated by clinicians. Despite having comparable self-reported autism symptoms, the online high-trait group reported significantly more social anxiety and avoidant behavior than in-person ASD subjects. Within the in-person sample, there was no relationship between self-rated and clinician-rated autism symptoms, suggesting these approaches may capture different aspects of ASD. The online high-trait and in-person ASD participants also differed in their behavior in well-validated social decision-making tasks: the in-person group perceived having less social control and acted less affiliative towards virtual characters. Our study aimed to draw comparisons at three levels: methodological platform (online versus in-person), symptom measurement (self- versus clinician-report), and social behavior. We identified a lack of agreement between self- and clinician-rated measures of symptoms and divergent social tendencies in groups ascertained by each method, highlighting the need for differentiation between in-person versus online samples in autism research.

The Human Affectome.

Over the last decades, theoretical perspectives in the interdisciplinary field of the affective sciences have proliferated rather than converged due to differing assumptions about what human affective phenomena are and how they work. These metaphysical and mechanistic assumptions, shaped by academic context and values, have dictated affective constructs and operationalizations. However, an assumption about the purpose of affective phenomena can guide us to a common set of metaphysical and mechanistic assumptions. In this capstone paper, we home in on a nested teleological principle for human affective phenomena in order to synthesize metaphysical and mechanistic assumptions. Under this framework, human affective phenomena can collectively be considered algorithms that either adjust based on the human comfort zone (affective concerns) or monitor those adaptive processes (affective features). This teleologically-grounded framework offers a principled agenda and launchpad for both organizing existing perspectives and generating new ones. Ultimately, we hope the Human Affectome brings us a step closer to not only an integrated understanding of human affective phenomena, but an integrated field for affective research.

Traumatic memory retrieval followed by electroconvulsive therapy as a treatment for posttraumatic stress disorder: A pilot study.

Delivering electroconvulsive therapy (ECT) during the reconsolidation of traumatic memories may enhance the treatment efficacy in posttraumatic stress disorder (PTSD). To test this, 14 patients with severe and refractory PTSD were randomly allocated to receive ECT sessions either after retrieving the traumatic (n=8) or a neutral (n=6) memory. We found that delivering ECT after retrieving the traumatic memory enhanced the improvement of PTSD symptoms and the reduction of subjective reactivity to the traumatic memory. Reduction in anxiety and mood symptoms and physiological reactivity to the traumatic memory were observed in the sample as a whole regardless of memory retrieval.

Hippocampal timestamp for goals.

Our brain must manage multiple goals that differ in their temporal proximity. Some goals require immediate attention, while others have already been accomplished, or will be relevant later in time. Here, we examined how the hippocampus represents the temporal distance to different goals using a novel space-themed paradigm during 7T functional MRI (n=31). The hippocampus has an established role in mental time travel and a system in place to stratify information along its longitudinal axis on the basis of representational granularity. Previous work has documented a functional transformation from fine-grained, detail rich representations in the posterior hippocampus to coarse, gist-like representations in the anterior hippocampus. We tested whether the hippocampus uses this long axis system to dissociate goals based upon their temporal distance from the present. We hypothesized that the hippocampus would distinguish goals relevant for ones' current needs from those that are removed in time along the long axis, with temporally removed past and future goals eliciting increasingly anterior activation. We sent participants on a mission to Mars where they had to track goals that differed in when they needed to be accomplished. We observed a long-axis dissociation, where temporally removed past and future goals activated the left anterior hippocampus and current goals activated the left posterior hippocampus. Altogether, this study demonstrates that the timestamp attached to a goal is a key driver in where the goal is represented in the hippocampus. This work extends the scope of the hippocampus' long axis system to the goal-mapping domain.

Neural patterns differentiate traumatic from sad autobiographical memories in PTSD.

For people with post-traumatic stress disorder (PTSD), recall of traumatic memories often displays as intrusions that differ profoundly from processing of 'regular' negative memories. These mnemonic features fueled theories speculating a unique cognitive state linked with traumatic memories. Yet, to date, little empirical evidence supports this view. Here we examined neural activity of patients with PTSD who were listening to narratives depicting their own memories. An intersubject representational similarity analysis of cross-subject semantic content and neural patterns revealed a differentiation in hippocampal representation by narrative type: semantically similar, sad autobiographical memories elicited similar neural representations across participants. By contrast, within the same individuals, semantically similar trauma memories were not represented similarly. Furthermore, we were able to decode memory type from hippocampal multivoxel patterns. Finally, individual symptom severity modulated semantic representation of the traumatic narratives in the posterior cingulate cortex. Taken together, these findings suggest that traumatic memories are an alternative cognitive entity that deviates from memory per se.