Active Inference, Curiosity and Insight.

This article offers a formal account of curiosity and insight in terms of active (Bayesian) inference. It deals with the dual problem of inferring states of the world and learning its statistical structure. In contrast to current trends in machine learning (e.g., deep learning), we focus on how people attain insight and understanding using just a handful of observations, which are solicited through curious behavior. We use simulations of abstract rule learning and approximate Bayesian inference to show that minimizing (expected) variational free energy leads to active sampling of novel contingencies. This epistemic behavior closes explanatory gaps in generative models of the world, thereby reducing uncertainty and satisfying curiosity. We then move from epistemic learning to model selection or structure learning to show how abductive processes emerge when agents test plausible hypotheses about symmetries (i.e., invariances or rules) in their generative models. The ensuing Bayesian model reduction evinces mechanisms associated with sleep and has all the hallmarks of "aha" moments. This formulation moves toward a computational account of consciousness in the pre-Cartesian sense of sharable knowledge (i.e., con: "together"; scire: "to know").

Activity in the nucleus accumbens and amygdala underlies individual differences in prosocial and individualistic economic choices.

Much decision-making requires balancing benefits to the self with benefits to the group. There are marked individual differences in this balance such that individualists tend to favor themselves whereas prosocials tend to favor the group. Understanding the mechanisms underlying this difference has important implications for society and its institutions. Using behavioral and fMRI data collected during the performance of the ultimatum game, we show that individual differences in social preferences for resource allocation, so-called "social value orientation," is linked with activity in the nucleus accumbens and amygdala elicited by inequity, rather than activity in insula, ACC, and dorsolateral pFC. Importantly, the presence of cognitive load made prosocials behave more prosocially and individualists more individualistically, suggesting that social value orientation is driven more by intuition than reflection. In parallel, activity in the nucleus accumbens and amygdala, in response to inequity, tracked this behavioral pattern of prosocials and individualists. In addition, we conducted an impunity game experiment with different participants where they could not punish unfair behavior and found that the inequity-correlated activity seen in prosocials during the ultimatum game disappeared. This result suggests that the accumbens and amygdala activity of prosocials encodes "outcome-oriented emotion" designed to change situations (i.e., achieve equity or punish). Together, our results suggest a pivotal contribution of the nucleus accumbens and amygdala to individual differences in sociality.

Federated inference and belief sharing.

This paper concerns the distributed intelligence or federated inference that emerges under belief-sharing among agents who share a common world-and world model. Imagine, for example, several animals keeping a lookout for predators. Their collective surveillance rests upon being able to communicate their beliefs-about what they see-among themselves. But, how is this possible? Here, we show how all the necessary components arise from minimising free energy. We use numerical studies to simulate the generation, acquisition and emergence of language in synthetic agents. Specifically, we consider inference, learning and selection as minimising the variational free energy of posterior (i.e., Bayesian) beliefs about the states, parameters and structure of generative models, respectively. The common theme-that attends these optimisation processes-is the selection of actions that minimise expected free energy, leading to active inference, learning and model selection (a.k.a., structure learning). We first illustrate the role of communication in resolving uncertainty about the latent states of a partially observed world, on which agents have complementary perspectives. We then consider the acquisition of the requisite language-entailed by a likelihood mapping from an agent's beliefs to their overt expression (e.g., speech)-showing that language can be transmitted across generations by active learning. Finally, we show that language is an emergent property of free energy minimisation, when agents operate within the same econiche. We conclude with a discussion of various perspectives on these phenomena; ranging from cultural niche construction, through federated learning, to the emergence of complexity in ensembles of self-organising systems.

Testosterone disrupts human collaboration by increasing egocentric choices.

Collaboration can provide benefits to the individual and the group across a variety of contexts. Even in simple perceptual tasks, the aggregation of individuals' personal information can enable enhanced group decision-making. However, in certain circumstances such collaboration can worsen performance, or even expose an individual to exploitation in economic tasks, and therefore a balance needs to be struck between a collaborative and a more egocentric disposition. Neurohumoral agents such as oxytocin are known to promote collaborative behaviours in economic tasks, but whether there are opponent agents, and whether these might even affect information aggregation without an economic component, is unknown. Here, we show that an androgen hormone, testosterone, acts as such an agent. Testosterone causally disrupted collaborative decision-making in a perceptual decision task, markedly reducing performance benefit individuals accrued from collaboration while leaving individual decision-making ability unaffected. This effect emerged because testosterone engendered more egocentric choices, manifest in an overweighting of one's own relative to others' judgements during joint decision-making. Our findings show that the biological control of social behaviour is dynamically regulated not only by modulators promoting, but also by those diminishing a propensity to collaborate.

Neural correlates of sequence learning with stochastic feedback.

Although much is known about decision making under uncertainty when only a single step is required in the decision process, less is known about sequential decision making. We carried out a stochastic sequence learning task in which subjects had to use noisy feedback to learn sequences of button presses. We compared flat and hierarchical behavioral models and found that although both models predicted the choices of the group of subjects equally well, only the hierarchical model correlated significantly with learning-related changes in the magneto-encephalographic response. The significant modulations in the magneto-encephalographic signal occurred 83 msec before button press and 67 msec after button press. We also localized the sources of these effects and found that the early effect localized to the insula, whereas the late effect localized to the premotor cortex.

How do illusions constrain goal-directed movement: perceptual and visuomotor influences on speed/accuracy trade-off.

Recent research shows that visual processing influences the speed/accuracy trade-off people use when performing goal-directed movement. This raises the question of how this influence is produced in visual cognition. Visual influences on speed/accuracy trade-off could be produced in conscious visual perception, in non-conscious visuomotor transformation, or by some interaction of conscious perceptual and non-conscious visuomotor processes. There is independent evidence showing that both perceptual and visuomotor processes are involved in trading off speed and accuracy; however, the interaction between these processes has yet to be investigated. We present an experiment in which we show that a change in visual consciousness induced by a perceptual illusion affects the speed and accuracy of goal-directed movements, suggesting that perceptual and visuomotor processes do interact in speed/accuracy trade-off. We discuss the consequences of these results for theories of visual function more generally.

From threat to fear: the neural organization of defensive fear systems in humans.

Postencounter and circa-strike defensive contexts represent two adaptive responses to potential and imminent danger. In the context of a predator, the postencounter reflects the initial detection of the potential threat, whereas the circa-strike is associated with direct predatory attack. We used functional magnetic resonance imaging to investigate the neural organization of anticipation and avoidance of artificial predators with high or low probability of capturing the subject across analogous postencounter and circa-strike contexts of threat. Consistent with defense systems models, postencounter threat elicited activity in forebrain areas, including subgenual anterior cingulate cortex (sgACC), hippocampus, and amygdala. Conversely, active avoidance during circa-strike threat increased activity in mid-dorsal ACC and midbrain areas. During the circa-strike condition, subjects showed increased coupling between the midbrain and mid-dorsal ACC and decreased coupling with the sgACC, amygdala, and hippocampus. Greater activity was observed in the right pregenual ACC for high compared with low probability of capture during circa-strike threat. This region showed decreased coupling with the amygdala, insula, and ventromedial prefrontal cortex. Finally, we found that locomotor errors correlated with subjective reports of panic for the high compared with low probability of capture during the circa-strike threat, and these panic-related locomotor errors were correlated with midbrain activity. These findings support models suggesting that higher forebrain areas are involved in early-threat responses, including the assignment and control of fear, whereas imminent danger results in fast, likely "hard-wired," defensive reactions mediated by the midbrain.

Abnormal functional specialization within medial prefrontal cortex in high-functioning autism: a multi-voxel similarity analysis.

Multi-voxel pattern analyses have proved successful in 'decoding' mental states from fMRI data, but have not been used to examine brain differences associated with atypical populations. We investigated a group of 16 (14 males) high-functioning participants with autism spectrum disorder (ASD) and 16 non-autistic control participants (12 males) performing two tasks (spatial/verbal) previously shown to activate medial rostral prefrontal cortex (mrPFC). Each task manipulated: (i) attention towards perceptual versus self-generated information and (ii) reflection on another person's mental state ('mentalizing'versus 'non-mentalizing') in a 2 x 2 design. Behavioral performance and group-level fMRI results were similar between groups. However, multi-voxel similarity analyses revealed strong differences. In control participants, the spatial distribution of activity generalized significantly between task contexts (spatial/verbal) when examining the same function (attention/mentalizing) but not when comparing different functions. This pattern was disrupted in the ASD group, indicating abnormal functional specialization within mrPFC, and demonstrating the applicability of multi-voxel pattern analysis to investigations of atypical populations.

Aberrant effective connectivity is associated with positive symptoms in first-episode schizophrenia.

Schizophrenia is a neurodevelopmental psychiatric disorder thought to result from synaptic dysfunction that affects distributed brain connectivity, rather than any particular brain region. While symptomatology is traditionally divided into positive and negative symptoms, abnormal social cognition is now recognized a key component of schizophrenia. Nonetheless, we are still lacking a mechanistic understanding of effective brain connectivity in schizophrenia during social cognition and how it relates to clinical symptomatology. To address this question, we used fMRI and dynamic causal modelling (DCM) to test for abnormal brain connectivity in twenty-four patients with first-episode schizophrenia (FES) compared to twenty-five matched controls performing the Human Connectome Project (HCP) social cognition paradigm. Patients had not received regular therapeutic antipsychotics, but were not completely drug naïve. Whilst patients were less accurate than controls in judging social stimuli from non-social stimuli, our results revealed an increase in feedforward connectivity from motion-sensitive V5 to posterior superior temporal sulcus (pSTS) in patients compared to matched controls. At the same time, patients with a higher degree of positive symptoms had more disinhibition within pSTS, a region computationally involved in social cognition. We interpret these findings the framework of active inference, where increased feedforward connectivity may encode aberrant prediction errors from V5 to pSTS and local disinhibition within pSTS may reflect aberrant encoding of the precision of cortical representations about social stimuli.

Knowing Ourselves Together: The Cultural Origins of Metacognition.

Metacognition - the ability to represent, monitor and control ongoing cognitive processes - helps us perform many tasks, both when acting alone and when working with others. While metacognition is adaptive, and found in other animals, we should not assume that all human forms of metacognition are gene-based adaptations. Instead, some forms may have a social origin, including the discrimination, interpretation, and broadcasting of metacognitive representations. There is evidence that each of these abilities depends on cultural learning and therefore that cultural selection might shape human metacognition. The cultural origins hypothesis is a plausible and testable alternative that directs us towards a substantial new programme of research.

Choking on the money: reward-based performance decrements are associated with midbrain activity.

A pernicious paradox in human motivation is the occasional reduced performance associated with tasks and situations that involve larger-than-average rewards. Three broad explanations that might account for such performance decrements are attentional competition (distraction theories), inhibition by conscious processes (explicit-monitoring theories), and excessive drive and arousal (overmotivation theories). Here, we report incentive-dependent performance decrements in humans in a reward-pursuit task; subjects were less successful in capturing a more valuable reward in a computerized maze. Concurrent functional magnetic resonance imaging revealed that increased activity in ventral midbrain, a brain area associated with incentive motivation and basic reward responding, correlated with both reduced number of captures and increased number of near-misses associated with imminent high rewards. These data cast light on the neurobiological basis of choking under pressure and are consistent with overmotivation accounts.

Blinking suppresses the neural response to unchanging retinal stimulation.

Blinks profoundly interrupt visual input but are rarely noticed, perhaps because of blink suppression, a visual-sensitivity loss that begins immediately prior to blink onset. Blink suppression is thought to result from an extra-retinal signal that is associated with the blink motor command and may act to attenuate the sensory consequences of the motor action. However, the neural mechanisms underlying this phenomenon remain unclear. They are challenging to study because any brain-activity changes resulting from an extra-retinal signal associated with the blink motor command are potentially masked by profound neural-activity changes caused by the retinal-illumination reduction that results from occlusion of the pupil by the eyelid. Here, we distinguished direct top-down effects of blink-associated motor signals on cortical activity from purely mechanical or optical effects of blinking on visual input by combining pupil-independent retinal stimulation with functional MRI (fMRI) in humans. Even though retinal illumination was kept constant during blinks, we found that blinking nevertheless suppressed activity in visual cortex and in areas of parietal and prefrontal cortex previously associated with awareness of environmental change. Our findings demonstrate active top-down modulation of visual processing during blinking, suggesting a possible mechanism by which blinks go unnoticed.

Atypical recruitment of medial prefrontal cortex in autism spectrum disorders: an fMRI study of two executive function tasks.

Recent studies have suggested an uneven profile of executive dysfunction in autism spectrum disorders (ASD). For example, some authors have reported deficits on newly developed tests of executive function sensitive to rostral prefrontal function, despite spared, or even superior, performance on other tests. We investigated the performance of a group of high-functioning participants with ASD (N=15) and an age- and IQ-matched control group (N=18) on two executive function tests, whilst undergoing functional magnetic resonance imaging (fMRI). Behaviourally, there were no significant differences between the two groups. In a classical test of executive function (random response generation), BOLD signal differed between the groups in the cerebellum but not in the frontal lobes. However, on a new test of executive function (selection between stimulus-oriented and stimulus-independent thought), the ASD group exhibited significantly greater signal-change in medial rostral prefrontal cortex (especially Brodmann Area 10) in the comparison of stimulus-oriented versus stimulus-independent attention. In addition, the new test (but not the classical test) provided evidence for abnormal functional organisation of medial prefrontal cortex in ASD. These results underline the heterogeneity of different tests of executive function, and suggest that executive functioning in ASD is associated with task-specific functional change.

The substantia nigra pars compacta and temporal processing.

The basal ganglia and cerebellum are considered to play a role in timing, although their differential roles in timing remain unclear. It has been proposed that the timing of short milliseconds-range intervals involves the cerebellum, whereas longer seconds-range intervals engage the basal ganglia (Ivry, 1996). We tested this hypothesis using positron emission tomography to measure regional cerebral blood flow in eight right-handed males during estimation and reproduction of long and short intervals. Subjects performed three tasks: (1) reproduction of a short 500 ms interval, (2) reproduction of a long 2 s interval, and (3) a control simple reaction time (RT) task. We compared the two time reproduction tasks with the control RT task to investigate activity associated with temporal processing once additional cognitive, motor, or sensory processing was controlled. We found foci in the left substantia nigra and the left lateral premotor cortex to be significantly more activated in the time reproduction tasks than the control RT task. The left caudate nucleus and right cerebellum were more active in the short relative to the long interval, whereas greater activation of the right putamen and right cerebellum occurred in the long rather than the short interval. These results suggest that the basal ganglia and the cerebellum are engaged by reproduction of both long and short intervals but play different roles. The fundamental role of the substantia nigra in temporal processing is discussed in relation to previous animal lesion studies and evidence for the modulating influence of dopamine on temporal processing.

Differential involvement of regions of rostral prefrontal cortex (Brodmann area 10) in time- and event-based prospective memory.

Rostral prefrontal cortex (approximating Brodmann area 10) has been shown repeatedly to have a role in the maintenance and realization of delayed intentions that are triggered by event cues (i.e., event-based prospective memory). The cerebral organization of the processes associated with the use of time cues (time-based prospective memory) has however received less attention. In two positron emission tomography (PET) studies we therefore examined brain activity associated with time- and event-based prospective memory tasks. In the time-based condition of the first study, young healthy volunteers were asked to make a prospective response based on their self-estimation of the passage of time while engaged in an attention-demanding ongoing activity. In the time-based condition of the second study, participants had a clock available in the ongoing task display and did not need to estimate the time for the prospective response. In the event-based condition of both studies, participants were asked to make a prospective response when prospective cues were presented in ongoing trials. Both studies showed activation differences in rostral prefrontal cortex according to whether the task was time- or event-based. In study one, an area of left superior frontal gyrus was more active in the time-based condition. In study two, three rostral prefrontal regions were more active in the time-based condition: right superior frontal gyrus, anterior medial frontal lobe and anterior cingulate gyrus. A region in left superior frontal gyrus, different from the area found in the first study, was more active in the event-based condition. These results indicate involvement of multiple brain regions of rostral prefrontal cortex in time- and event-based prospective memory. The results are interpreted as reflecting the differing processing demands made by event- or time-based prospective memory tasks, and the differing demands of time-based tasks according to whether a clock is present as an aid.

Performance-related activity in medial rostral prefrontal cortex (area 10) during low-demand tasks.

Neuroimaging studies have frequently observed relatively high activity in medial rostral prefrontal cortex (PFC) during rest or baseline conditions. Some accounts have attributed this high activity to the occurrence of unconstrained stimulus-independent and task-unrelated thought processes during baseline conditions. Here, the authors investigated the alternative possibility that medial rostral PFC supports attention toward the external environment during low-demand conditions. Participants performed a baseline simple reaction time (RT) task, along with 3 other tasks that differed in the requirement to attend to external stimuli versus stimulus-independent thought. Medial rostral PFC activation was observed in the baseline task and in a condition requiring strong engagement with external stimuli, relative to 2 conditions with a greater requirement for stimulus-independent thought. An important finding was that activity in this region was associated with faster RTs in the baseline task, ruling out an explanation in terms of task-unrelated thought processes during this condition. Thus, at least under certain circumstances, medial rostral PFC appears to support attention toward the external environment, facilitating performance in situations that do not require extensive processing of experimental stimuli.

Lateral asymmetry in the hippocampal response to the remoteness of autobiographical memories.

The time scale of hippocampal involvement in retrieving memories, particularly those more remote, is still a matter of debate. Some propose that the hippocampus is not involved in the retrieval of remote memories, whereas others assert that it is necessary for memory retrieval in perpetuity. Functional magnetic resonance imaging was used to examine the effect of remoteness on the neural basis of memory. We used a parametric event-related random-effects design in a large group of subjects to overcome some of the limitations of previous neuroimaging studies. We found that the hippocampi were significantly active during the retrieval of autobiographical memories. Notably, the two hippocampi diverged in their responses to remoteness. The right hippocampus showed a temporal gradient, decreasing in activity the more remote the autobiographical memories. No such effect was apparent in the left hippocampus, suggesting its invariant involvement in remembering autobiographical events throughout the lifespan. The dorsal amygdalas showed a temporal gradient similar to the right hippocampus, but emotional valence and intensity were not directly associated with changes in activity. The current results indicate that consideration of lateral asymmetry may help to broaden the scope of theoretical interpretations concerning hippocampal involvement in remote memory.

Active inference, communication and hermeneutics.

Hermeneutics refers to interpretation and translation of text (typically ancient scriptures) but also applies to verbal and non-verbal communication. In a psychological setting it nicely frames the problem of inferring the intended content of a communication. In this paper, we offer a solution to the problem of neural hermeneutics based upon active inference. In active inference, action fulfils predictions about how we will behave (e.g., predicting we will speak). Crucially, these predictions can be used to predict both self and others--during speaking and listening respectively. Active inference mandates the suppression of prediction errors by updating an internal model that generates predictions--both at fast timescales (through perceptual inference) and slower timescales (through perceptual learning). If two agents adopt the same model, then--in principle--they can predict each other and minimise their mutual prediction errors. Heuristically, this ensures they are singing from the same hymn sheet. This paper builds upon recent work on active inference and communication to illustrate perceptual learning using simulated birdsongs. Our focus here is the neural hermeneutics implicit in learning, where communication facilitates long-term changes in generative models that are trying to predict each other. In other words, communication induces perceptual learning and enables others to (literally) change our minds and vice versa.

Reduced frontotemporal functional connectivity in schizophrenia associated with auditory hallucinations.

BACKGROUND: We used functional magnetic resonance imaging (fMRI) to investigate the frontotemporal disconnection hypothesis of schizophrenia. METHODS: Eight DSM-IV schizophrenia patients and 10 control subjects were studied with fMRI while they thought of the missing last word in 128 visually presented sentences. The fMRI data were analyzed comparing the effect of sentence completion (vs. rest) using a random effects analysis. RESULTS: There were no significant group differences in regional brain responses. Correlation coefficients between left temporal cortex (x = -54, y = -42, z = 3) and left dorsolateral prefrontal cortex (x = -39, y = 12, z = 24) were significantly lower in the schizophrenic group and were negatively correlated with the severity of auditory hallucinations. CONCLUSIONS: Previous demonstrations of hypofrontality in schizophrenia may reflect particular task requirements. Frontotemporal functional connectivity is reduced in schizophrenia and may be associated with auditory hallucinations.