Decoding cognition from spontaneous neural activity.

In human neuroscience, studies of cognition are rarely grounded in non-task-evoked, 'spontaneous' neural activity. Indeed, studies of spontaneous activity tend to focus predominantly on intrinsic neural patterns (for example, resting-state networks). Taking a 'representation-rich' approach bridges the gap between cognition and resting-state communities: this approach relies on decoding task-related representations from spontaneous neural activity, allowing quantification of the representational content and rich dynamics of such activity. For example, if we know the neural representation of an episodic memory, we can decode its subsequent replay during rest. We argue that such an approach advances cognitive research beyond a focus on immediate task demand and provides insight into the functional relevance of the intrinsic neural pattern (for example, the default mode network). This in turn enables a greater integration between human and animal neuroscience, facilitating experimental testing of theoretical accounts of intrinsic activity, and opening new avenues of research in psychiatry.

Influence of surprise on reinforcement learning in younger and older adults.

Surprise is a key component of many learning experiences, and yet its precise computational role, and how it changes with age, remain debated. One major challenge is that surprise often occurs jointly with other variables, such as uncertainty and outcome probability. To assess how humans learn from surprising events, and whether aging affects this process, we studied choices while participants learned from bandits with either Gaussian or bi-modal outcome distributions, which decoupled outcome probability, uncertainty, and surprise. A total of 102 participants (51 older, aged 50-73; 51 younger, 19-30 years) chose between three bandits, one of which had a bimodal outcome distribution. Behavioral analyses showed that both age-groups learned the average of the bimodal bandit less well. A trial-by-trial analysis indicated that participants performed choice reversals immediately following large absolute prediction errors, consistent with heightened sensitivity to surprise. This effect was stronger in older adults. Computational models indicated that learning rates in younger as well as older adults were influenced by surprise, rather than uncertainty, but also suggested large interindividual variability in the process underlying learning in our task. Our work bridges between behavioral economics research that has focused on how outcomes with low probability affect choice in older adults, and reinforcement learning work that has investigated age differences in the effects of uncertainty and suggests that older adults overly adapt to surprising events, even when accounting for probability and uncertainty effects.

Dopamine differentially modulates medial temporal lobe activity and behavior during spatial navigation in young and older adults.

Aging is associated with changes in spatial navigation behavior. In addition to an overall performance decline, older adults tend to rely more on proximal location cue information than on environmental boundary information during spatial navigation compared to young adults. The fact that older adults are more susceptible to errors during spatial navigation might be partly attributed to deficient dopaminergic modulation of hippocampal and striatal functioning. Hence, elevating dopamine levels might differentially modulate spatial navigation and memory performance in young and older adults. In this work, we administered levodopa (L-DOPA) in a double-blind within-subject, placebo-controlled design and recorded functional neuroimaging while young and older adults performed a 3D spatial navigation task in which boundary geometry or the position of a location cue were systematically manipulated. An age by intervention interaction on the neural level revealed an upregulation of brain responses in older adults and a downregulation of responses in young adults within the medial temporal lobe (including hippocampus and parahippocampus) and brainstem, during memory retrieval. Behaviorally, L-DOPA had no effect on older adults' overall memory performance; however, older adults whose spatial memory improved under L-DOPA also showed a shift towards more boundary processing under L-DOPA. In young adults, L-DOPA induced a decline in spatial memory performance in task-naïve participants. These results are consistent with the inverted-U-shaped hypothesis of dopamine signaling and cognitive function and suggest that increasing dopamine availability improves hippocampus-dependent place learning in some older adults.

The dorsomedial prefrontal cortex represents subjective value across effort-based and risky decision-making.

Decisions that require taking effort costs into account are ubiquitous in real life. The neural common currency theory hypothesizes that a particular neural network integrates different costs (e.g., risk) and rewards into a common scale to facilitate value comparison. Although there has been a surge of interest in the computational and neural basis of effort-related value integration, it is still under debate if effort-based decision-making relies on a domain-general valuation network as implicated in the neural common currency theory. Therefore, we comprehensively compared effort-based and risky decision-making using a combination of computational modeling, univariate and multivariate fMRI analyses, and data from two independent studies. We found that effort-based decision-making can be best described by a power discounting model that accounts for both the discounting rate and effort sensitivity. At the neural level, multivariate decoding analyses indicated that the neural patterns of the dorsomedial prefrontal cortex (dmPFC) represented subjective value across different decision-making tasks including either effort or risk costs, although univariate signals were more diverse. These findings suggest that multivariate dmPFC patterns play a critical role in computing subjective value in a task-independent manner and thus extend the scope of the neural common currency theory.

L-DOPA enhances neural direction signals in younger and older adults.

Previous studies indicate a role of dopamine in spatial navigation. Although neural representations of direction are an important aspect of spatial cognition, it is not well understood whether dopamine directly affects these representations, or only impacts other aspects of spatial brain function. Moreover, both dopamine and spatial cognition decline sharply during age, raising the question which effect dopamine has on directional signals in the brain of older adults. To investigate these questions, we used a double-blind cross-over L-DOPA/Placebo intervention design in which 43 younger and 37 older adults navigated in a virtual spatial environment while undergoing functional magnetic resonance imaging (fMRI). We studied the effect of L-DOPA, a dopamine precursor, on fMRI activation patterns that encode spatial walking directions that have previously been shown to lose specificity with age. This was done in predefined regions of interest, including the early visual cortex, retrosplenial cortex, and hippocampus. Classification of brain activation patterns associated with different walking directions was improved across all regions following L-DOPA administration, suggesting that dopamine broadly enhances neural representations of direction. No evidence for differences between regions was found. In the hippocampus these results were found in both age groups, while in the retrosplenial cortex they were only observed in younger adults. Taken together, our study provides evidence for a link between dopamine and the specificity of neural responses during spatial navigation. SIGNIFICANCE STATEMENT: The sense of direction is an important aspect of spatial navigation, and neural representations of direction can be found throughout a large network of space-related brain regions. But what influences how well these representations track someone's true direction? Using a double-blind cross-over L-DOPA/Placebo intervention design, we find causal evidence that the neurotransmitter dopamine impacts the fidelity of direction selective neural representations in the human hippocampus and retrosplenial cortex. Interestingly, the effect of L-DOPA was either equally present or even smaller in older adults, despite the well-known age related decline of dopamine. These results provide novel insights into how dopamine shapes the neural representations that underlie spatial navigation.

Similarities and differences in spatial and non-spatial cognitive maps.

Learning and generalization in spatial domains is often thought to rely on a "cognitive map", representing relationships between spatial locations. Recent research suggests that this same neural machinery is also recruited for reasoning about more abstract, conceptual forms of knowledge. Yet, to what extent do spatial and conceptual reasoning share common computational principles, and what are the implications for behavior? Using a within-subject design we studied how participants used spatial or conceptual distances to generalize and search for correlated rewards in successive multi-armed bandit tasks. Participant behavior indicated sensitivity to both spatial and conceptual distance, and was best captured using a Bayesian model of generalization that formalized distance-dependent generalization and uncertainty-guided exploration as a Gaussian Process regression with a radial basis function kernel. The same Gaussian Process model best captured human search decisions and judgments in both domains, and could simulate realistic learning curves, where we found equivalent levels of generalization in spatial and conceptual tasks. At the same time, we also find characteristic differences between domains. Relative to the spatial domain, participants showed reduced levels of uncertainty-directed exploration and increased levels of random exploration in the conceptual domain. Participants also displayed a one-directional transfer effect, where experience in the spatial task boosted performance in the conceptual task, but not vice versa. While confidence judgments indicated that participants were sensitive to the uncertainty of their knowledge in both tasks, they did not or could not leverage their estimates of uncertainty to guide exploration in the conceptual task. These results support the notion that value-guided learning and generalization recruit cognitive-map dependent computational mechanisms in spatial and conceptual domains. Yet both behavioral and model-based analyses suggest domain specific differences in how these representations map onto actions.

Correction: Similarities and differences in spatial and non-spatial cognitive maps.

[This corrects the article DOI: 10.1371/journal.pcbi.1008149.].

Brain network dynamics during spontaneous strategy shifts and incremental task optimization.

With practice, humans improve their performance in a task by either optimizing a known strategy or discovering a novel, potentially more fruitful strategy. We investigated the neural processes underlying these two fundamental abilities by applying fMRI in a task with two possible alternative strategies. For analysis we combined time-resolved network analysis with Coherence Density Peak Clustering (Allegra et al., 2017), univariate GLM, and multivariate pattern classification. Converging evidence showed that the posterior portion of the default network, i.e. the precuneus and the angular gyrus bilaterally, has a central role in the optimization of the current strategy. These regions encoded the relevant spatial information, increased the strength of local connectivity as well as the long-distance connectivity with other relevant regions in the brain (e.g., visual cortex, dorsal attention network). The connectivity increase was proportional to performance optimization. By contrast, the anterior portion of the default network (i.e. medial prefrontal cortex) and the rostral portion of the fronto-parietal network were associated with new strategy discovery: an early increase of local and long-range connectivity centered on these regions was only observed in the subjects who would later shift to a new strategy. Overall, our findings shed light on the dynamic interactions between regions related to attention and with cognitive control, underlying the balance between strategy exploration and exploitation. Results suggest that the default network, far from being "shut-down" during task performance, has a pivotal role in the background exploration and monitoring of potential alternative courses of action.

Representational structure or task structure? Bias in neural representational similarity analysis and a Bayesian method for reducing bias.

The activity of neural populations in the brains of humans and animals can exhibit vastly different spatial patterns when faced with different tasks or environmental stimuli. The degrees of similarity between these neural activity patterns in response to different events are used to characterize the representational structure of cognitive states in a neural population. The dominant methods of investigating this similarity structure first estimate neural activity patterns from noisy neural imaging data using linear regression, and then examine the similarity between the estimated patterns. Here, we show that this approach introduces spurious bias structure in the resulting similarity matrix, in particular when applied to fMRI data. This problem is especially severe when the signal-to-noise ratio is low and in cases where experimental conditions cannot be fully randomized in a task. We propose Bayesian Representational Similarity Analysis (BRSA), an alternative method for computing representational similarity, in which we treat the covariance structure of neural activity patterns as a hyper-parameter in a generative model of the neural data. By marginalizing over the unknown activity patterns, we can directly estimate this covariance structure from imaging data. This method offers significant reductions in bias and allows estimation of neural representational similarity with previously unattained levels of precision at low signal-to-noise ratio, without losing the possibility of deriving an interpretable distance measure from the estimated similarity. The method is closely related to Pattern Component Model (PCM), but instead of modeling the estimated neural patterns as in PCM, BRSA models the imaging data directly and is suited for analyzing data in which the order of task conditions is not fully counterbalanced. The probabilistic framework allows for jointly analyzing data from a group of participants. The method can also simultaneously estimate a signal-to-noise ratio map that shows where the learned representational structure is supported more strongly. Both this map and the learned covariance matrix can be used as a structured prior for maximum a posteriori estimation of neural activity patterns, which can be further used for fMRI decoding. Our method therefore paves the way towards a more unified and principled analysis of neural representations underlying fMRI signals. We make our tool freely available in Brain Imaging Analysis Kit (BrainIAK).

An Integrated Model of Action Selection: Distinct Modes of Cortical Control of Striatal Decision Making.

Making decisions in environments with few choice options is easy. We select the action that results in the most valued outcome. Making decisions in more complex environments, where the same action can produce different outcomes in different conditions, is much harder. In such circumstances, we propose that accurate action selection relies on top-down control from the prelimbic and orbitofrontal cortices over striatal activity through distinct thalamostriatal circuits. We suggest that the prelimbic cortex exerts direct influence over medium spiny neurons in the dorsomedial striatum to represent the state space relevant to the current environment. Conversely, the orbitofrontal cortex is argued to track a subject's position within that state space, likely through modulation of cholinergic interneurons.

Electrophysiological correlates of observational learning in children.

Observational learning is an important mechanism for cognitive and social development. However, the neurophysiological mechanisms underlying observational learning in children are not well understood. In this study, we used a probabilistic reward-based observational learning paradigm to compare behavioral and electrophysiological markers of individual and observational reinforcement learning in 8- to 10-year-old children. Specifically, we manipulated the amount of observable information as well as children's similarity in age to the observed person (same-aged child vs. adult) to examine the effects of similarity in age on the integration of observed information in children. We show that the feedback-related negativity (FRN) during individual reinforcement learning reflects the valence of outcomes of own actions. Furthermore, we found that the feedback-related negativity during observational reinforcement learning (oFRN) showed a similar distinction between outcome valences of observed actions. This suggests that the oFRN can serve as a measure of observational learning in middle childhood. Moreover, during observational learning children profited from the additional social information and imitated the choices of their own peers more than those of adults, indicating that children have a tendency to conform more with similar others (e.g. their own peers) compared to dissimilar others (adults). Taken together, our results show that children can benefit from integrating observable information and that oFRN may serve as a measure of observational learning in children.

Human aging alters the neural computation and representation of space.

The hippocampus and striatum are core neural circuits involved in spatial learning and memory. Although both neural systems support spatial navigation, experimental and theoretical evidence indicate that they play different roles. In particular, whereas hippocampal place cells generate allocentric neural representations of space that are sensitive to geometric information, striatum-dependent learning is influenced by local landmarks. How human aging affects these different neural representations, however, is still not well understood. In this paper, we combined virtual reality, computational modeling, and neuroimaging to investigate the effects of age upon the neural computation and representation of space in humans. We manipulated the geometry and local landmarks of a virtual environment and examined the effects on memory performance and brain activity during spatial learning. In younger adults, both behavior and brain activity in the medial-temporal lobe were consistent with predictions of a computational model of hippocampus-dependent boundary processing. In contrast, older adults' behavior and medial-temporal lobe activity were primarily influenced by local cue information, and spatial learning was more associated with activity in the caudate nucleus rather than the hippocampus. Together these results point to altered spatial representations and information processing in the hippocampal-striatal circuitry with advancing adult age, which may contribute to spatial learning and memory deficits associated with normal and pathological aging.

Reduced striatal responses to reward prediction errors in older compared with younger adults.

We examined whether older adults differ from younger adults in how they learn from rewarding and aversive outcomes. Human participants were asked to either learn to choose actions that lead to monetary reward or learn to avoid actions that lead to monetary losses. To examine age differences in the neurophysiological mechanisms of learning, we applied a combination of computational modeling and fMRI. Behavioral results showed age-related impairments in learning from reward but not in learning from monetary losses. Consistent with these results, we observed age-related reductions in BOLD activity during learning from reward in the ventromedial PFC. Furthermore, the model-based fMRI analysis revealed a reduced responsivity of the ventral striatum to reward prediction errors during learning in older than younger adults. This age-related reduction in striatal sensitivity to reward prediction errors may result from a decline in phasic dopaminergic learning signals in the elderly.

Grid-like entorhinal representation of an abstract value space during prospective decision making.

How valuable a choice option is often changes over time, making the prediction of value changes an important challenge for decision making. Prior studies identified a cognitive map in the hippocampal-entorhinal system that encodes relationships between states and enables prediction of future states, but does not inherently convey value during prospective decision making. In this fMRI study, participants predicted changing values of choice options in a sequence, forming a trajectory through an abstract two-dimensional value space. During this task, the entorhinal cortex exhibited a grid-like representation with an orientation aligned to the axis through the value space most informative for choices. A network of brain regions, including ventromedial prefrontal cortex, tracked the prospective value difference between options. These findings suggest that the entorhinal grid system supports the prediction of future values by representing a cognitive map, which might be used to generate lower-dimensional value signals to guide prospective decision making.

Aging and KIBRA/WWC1 genotype affect spatial memory processes in a virtual navigation task.

Spatial navigation relies on multiple mnemonic mechanisms and previous work in younger adults has described two separate types of spatial memory. One type uses directional as well as boundary-related information for spatial memory and mainly implicates the hippocampal formation. The other type has been linked to directional and landmark-related information and primarily involves the striatum. Using a virtual reality navigation paradigm, we studied the impacts of aging and a single nucleotide polymorphism (SNP rs17070145) of the KIBRA gene (official name: WWC1) on these memory forms. Our data showed that older adult's spatial learning was preferentially related to processing of landmark information, whereas processing of boundary information played a more prominent role in younger adults. Moreover, among older adults T-allele carriers of the examined KIBRA polymorphism showed better spatial learning compared to C homozygotes. Together these findings provide the first evidence for an effect of the KIBRA rs17070145 polymorphism on spatial memory in humans and age differences in the reliance on landmark and boundary-related spatial information.

Trait anxiety is associated with hidden state inference during aversive reversal learning.

Updating beliefs in changing environments can be driven by gradually adapting expectations or by relying on inferred hidden states (i.e. contexts), and changes therein. Previous work suggests that increased reliance on context could underly fear relapse phenomena that hinder clinical treatment of anxiety disorders. We test whether trait anxiety variations in a healthy population influence how much individuals rely on hidden-state inference. In a Pavlovian learning task, participants observed cues that predicted an upcoming electrical shock with repeatedly changing probability, and were asked to provide expectancy ratings on every trial. We show that trait anxiety is associated with steeper expectation switches after contingency reversals and reduced oddball learning. Furthermore, trait anxiety is related to better fit of a state inference, compared to a gradual learning, model when contingency changes are large. Our findings support previous work suggesting hidden-state inference as a mechanism behind anxiety-related to fear relapse phenomena.

Task state representations in vmPFC mediate relevant and irrelevant value signals and their behavioral influence.

The ventromedial prefrontal-cortex (vmPFC) is known to contain expected value signals that inform our choices. But expected values even for the same stimulus can differ by task. In this study, we asked how the brain flexibly switches between such value representations in a task-dependent manner. Thirty-five participants alternated between tasks in which either stimulus color or motion predicted rewards. We show that multivariate vmPFC signals contain a rich representation that includes the current task state or context (motion/color), the associated expected value, and crucially, the irrelevant value of the alternative context. We also find that irrelevant value representations in vmPFC compete with relevant value signals, interact with task-state representations and relate to behavioral signs of value competition. Our results shed light on vmPFC's role in decision making, bridging between its role in mapping observations onto the task states of a mental map, and computing expected values for multiple states.

Hippocampal spatio-predictive cognitive maps adaptively guide reward generalization.

The brain forms cognitive maps of relational knowledge-an organizing principle thought to underlie our ability to generalize and make inferences. However, how can a relevant map be selected in situations where a stimulus is embedded in multiple relational structures? Here, we find that both spatial and predictive cognitive maps influence generalization in a choice task, where spatial location determines reward magnitude. Mirroring behavior, the hippocampus not only builds a map of spatial relationships but also encodes the experienced transition structure. As the task progresses, participants' choices become more influenced by spatial relationships, reflected in a strengthening of the spatial map and a weakening of the predictive map. This change is driven by orbitofrontal cortex, which represents the degree to which an outcome is consistent with the spatial rather than the predictive map and updates hippocampal representations accordingly. Taken together, this demonstrates how hippocampal cognitive maps are used and updated flexibly for inference.

Goal-seeking compresses neural codes for space in the human hippocampus and orbitofrontal cortex.

Humans can navigate flexibly to meet their goals. Here, we asked how the neural representation of allocentric space is distorted by goal-directed behavior. Participants navigated an agent to two successive goal locations in a grid world environment comprising four interlinked rooms, with a contextual cue indicating the conditional dependence of one goal location on another. Examining the neural geometry by which room and context were encoded in fMRI signals, we found that map-like representations of the environment emerged in both hippocampus and neocortex. Cognitive maps in hippocampus and orbitofrontal cortices were compressed so that locations cued as goals were coded together in neural state space, and these distortions predicted successful learning. This effect was captured by a computational model in which current and prospective locations are jointly encoded in a place code, providing a theory of how goals warp the neural representation of space in macroscopic neural signals.