Collective behavior from surprise minimization.

Collective motion is ubiquitous in nature; groups of animals, such as fish, birds, and ungulates appear to move as a whole, exhibiting a rich behavioral repertoire that ranges from directed movement to milling to disordered swarming. Typically, such macroscopic patterns arise from decentralized, local interactions among constituent components (e.g., individual fish in a school). Preeminent models of this process describe individuals as self-propelled particles, subject to self-generated motion and "social forces" such as short-range repulsion and long-range attraction or alignment. However, organisms are not particles; they are probabilistic decision-makers. Here, we introduce an approach to modeling collective behavior based on active inference. This cognitive framework casts behavior as the consequence of a single imperative: to minimize surprise. We demonstrate that many empirically observed collective phenomena, including cohesion, milling, and directed motion, emerge naturally when considering behavior as driven by active Bayesian inference-without explicitly building behavioral rules or goals into individual agents. Furthermore, we show that active inference can recover and generalize the classical notion of social forces as agents attempt to suppress prediction errors that conflict with their expectations. By exploring the parameter space of the belief-based model, we reveal nontrivial relationships between the individual beliefs and group properties like polarization and the tendency to visit different collective states. We also explore how individual beliefs about uncertainty determine collective decision-making accuracy. Finally, we show how agents can update their generative model over time, resulting in groups that are collectively more sensitive to external fluctuations and encode information more robustly.

Incubation of palatable food craving is associated with brain-wide neuronal activation in mice.

Studies using rodent models have shown that relapse to drug or food seeking increases progressively during abstinence, a behavioral phenomenon termed "incubation of craving." Mechanistic studies of incubation of craving have focused on specific neurobiological targets within preselected brain areas. Recent methodological advances in whole-brain immunohistochemistry, clearing, and imaging now allow unbiased brain-wide cellular resolution mapping of regions and circuits engaged during learned behaviors. However, these whole-brain imaging approaches were developed for mouse brains, while incubation of drug craving has primarily been studied in rats, and incubation of food craving has not been demonstrated in mice. Here, we established a mouse model of incubation of palatable food craving and examined food reward seeking after 1, 15, and 60 abstinence days. We then used the neuronal activity marker Fos with intact-brain mapping procedures to identify corresponding patterns of brain-wide activation. Relapse to food seeking was significantly higher after 60 abstinence days than after 1 or 15 days. Using unbiased ClearMap analysis, we identified increased activation of multiple brain regions, particularly corticostriatal structures, following 60 but not 1 or 15 abstinence days. We used orthogonal SMART2 analysis to confirm these findings within corticostriatal and thalamocortical subvolumes and applied expert-guided registration to investigate subdivision and layer-specific activation patterns. Overall, we 1) identified brain-wide activity patterns during incubation of food seeking using complementary analytical approaches and 2) provide a single-cell resolution whole-brain atlas that can be used to identify functional networks and global architecture underlying the incubation of food craving.

Epistemic Communities under Active Inference.

The spread of ideas is a fundamental concern of today's news ecology. Understanding the dynamics of the spread of information and its co-option by interested parties is of critical importance. Research on this topic has shown that individuals tend to cluster in echo-chambers and are driven by confirmation bias. In this paper, we leverage the active inference framework to provide an in silico model of confirmation bias and its effect on echo-chamber formation. We build a model based on active inference, where agents tend to sample information in order to justify their own view of reality, which eventually leads to them to have a high degree of certainty about their own beliefs. We show that, once agents have reached a certain level of certainty about their beliefs, it becomes very difficult to get them to change their views. This system of self-confirming beliefs is upheld and reinforced by the evolving relationship between an agent's beliefs and observations, which over time will continue to provide evidence for their ingrained ideas about the world. The epistemic communities that are consolidated by these shared beliefs, in turn, tend to produce perceptions of reality that reinforce those shared beliefs. We provide an active inference account of this community formation mechanism. We postulate that agents are driven by the epistemic value that they obtain from sampling or observing the behaviours of other agents. Inspired by digital social networks like Twitter, we build a generative model in which agents generate observable social claims or posts (e.g., 'tweets') while reading the socially observable claims of other agents that lend support to one of two mutually exclusive abstract topics. Agents can choose which other agent they pay attention to at each timestep, and crucially who they attend to and what they choose to read influences their beliefs about the world. Agents also assess their local network's perspective, influencing which kinds of posts they expect to see other agents making. The model was built and simulated using the freely available Python package pymdp. The proposed active inference model can reproduce the formation of echo-chambers over social networks, and gives us insight into the cognitive processes that lead to this phenomenon.

Stochastic Chaos and Markov Blankets.

In this treatment of random dynamical systems, we consider the existence-and identification-of conditional independencies at nonequilibrium steady-state. These independencies underwrite a particular partition of states, in which internal states are statistically secluded from external states by blanket states. The existence of such partitions has interesting implications for the information geometry of internal states. In brief, this geometry can be read as a physics of sentience, where internal states look as if they are inferring external states. However, the existence of such partitions-and the functional form of the underlying densities-have yet to be established. Here, using the Lorenz system as the basis of stochastic chaos, we leverage the Helmholtz decomposition-and polynomial expansions-to parameterise the steady-state density in terms of surprisal or self-information. We then show how Markov blankets can be identified-using the accompanying Hessian-to characterise the coupling between internal and external states in terms of a generalised synchrony or synchronisation of chaos. We conclude by suggesting that this kind of synchronisation may provide a mathematical basis for an elemental form of (autonomous or active) sentience in biology.

Memory and Markov Blankets.

In theoretical biology, we are often interested in random dynamical systems-like the brain-that appear to model their environments. This can be formalized by appealing to the existence of a (possibly non-equilibrium) steady state, whose density preserves a conditional independence between a biological entity and its surroundings. From this perspective, the conditioning set, or Markov blanket, induces a form of vicarious synchrony between creature and world-as if one were modelling the other. However, this results in an apparent paradox. If all conditional dependencies between a system and its surroundings depend upon the blanket, how do we account for the mnemonic capacity of living systems? It might appear that any shared dependence upon past blanket states violates the independence condition, as the variables on either side of the blanket now share information not available from the current blanket state. This paper aims to resolve this paradox, and to demonstrate that conditional independence does not preclude memory. Our argument rests upon drawing a distinction between the dependencies implied by a steady state density, and the density dynamics of the system conditioned upon its configuration at a previous time. The interesting question then becomes: What determines the length of time required for a stochastic system to 'forget' its initial conditions? We explore this question for an example system, whose steady state density possesses a Markov blanket, through simple numerical analyses. We conclude with a discussion of the relevance for memory in cognitive systems like us.

Nucleus Accumbens Drd1-Expressing Neurons Control Aggression Self-Administration and Aggression Seeking in Mice.

We recently developed a mouse model of appetitive operant aggression and reported that adult male outbred CD-1 mice lever-press for the opportunity to attack subordinate male mice and relapse to aggression seeking during abstinence. Here we studied the role of nucleus accumbens (NAc) dopamine receptor (Drd)1- and Drd2-expressing neurons in aggression self-administration and aggression seeking. We trained CD-1 mice to self-administer intruders (9 d, 12 trials/d) and tested them for aggression self-administration and aggression seeking on abstinence Day 1. We used immunohistochemistry and in situ hybridization to measure the neuronal activity marker Fos in the NAc, and cell-type-specific colocalization of Fos with Drd1- and Drd2-expressing neurons. To test the causal role of Drd1- and Drd2-expressing neurons, we validated a transgenic hybrid breeding strategy crossing inbred Drd1-Cre and Drd2-Cre transgenic mice with outbred CD-1 mice and used cell-type-specific Cre-DREADD (hM4Di) to inhibit NAc Drd1- and Drd2-expressing neuron activity. We found that aggression self-administration and aggression seeking induced higher Fos expression in NAc shell than in core, that Fos colocalized with Drd1 and Drd2 in both subregions, and that chemogenetic inhibition of Drd1-, but not Drd2-, expressing neurons decreased aggression self-administration and aggression seeking. Results indicate a cell-type-specific role of Drd1-expressing neurons that is critical for both aggression self-administration and aggression seeking. Our study also validates a simple breeding strategy between outbred CD-1 mice and inbred C57-based Cre lines that can be used to study cell-type and circuit mechanisms of aggression reward and relapse.SIGNIFICANCE STATEMENT Aggression is often comorbid with neuropsychiatric diseases, including drug addiction. One form, appetitive aggression, exhibits symptomatology that mimics that of drug addiction and is hypothesized to be due to dysregulation of addiction-related reward circuits. However, our mechanistic understanding of the circuitry modulating appetitive operant aggression is limited. Here we used a novel mouse model of aggression self-administration and relapse, in combination with immunohistochemistry, in situ hybridization, and chemogenetic manipulations to examine how cell types in the nucleus accumbens are recruited for, and control, operant aggression self-administration and aggression seeking on abstinence Day 1. We found that one population, dopamine receptor 1-expressing neurons, act as a critical modulator of operant aggression reward and aggression seeking.

Federated inference and belief sharing.

This paper concerns the distributed intelligence or federated inference that emerges under belief-sharing among agents who share a common world-and world model. Imagine, for example, several animals keeping a lookout for predators. Their collective surveillance rests upon being able to communicate their beliefs-about what they see-among themselves. But, how is this possible? Here, we show how all the necessary components arise from minimising free energy. We use numerical studies to simulate the generation, acquisition and emergence of language in synthetic agents. Specifically, we consider inference, learning and selection as minimising the variational free energy of posterior (i.e., Bayesian) beliefs about the states, parameters and structure of generative models, respectively. The common theme-that attends these optimisation processes-is the selection of actions that minimise expected free energy, leading to active inference, learning and model selection (a.k.a., structure learning). We first illustrate the role of communication in resolving uncertainty about the latent states of a partially observed world, on which agents have complementary perspectives. We then consider the acquisition of the requisite language-entailed by a likelihood mapping from an agent's beliefs to their overt expression (e.g., speech)-showing that language can be transmitted across generations by active learning. Finally, we show that language is an emergent property of free energy minimisation, when agents operate within the same econiche. We conclude with a discussion of various perspectives on these phenomena; ranging from cultural niche construction, through federated learning, to the emergence of complexity in ensembles of self-organising systems.

Emerging technologies for behavioral research in changing environments.

The first response exhibited by animals to changing environments is typically behavioral. Behavior is thus central to predicting, and mitigating, the impacts that natural and anthropogenic environmental changes will have on populations and, consequently, ecosystems. Yet the inherently multiscale nature of behavior, as well as the complexities associated with inferring how animals perceive their world, and make decisions, has constrained the scope of behavioral research. Major technological advances in electronics and in machine learning, however, provide increasingly powerful means to see, analyze, and interpret behavior in its natural complexity. We argue that these disruptive technologies will foster new approaches that will allow us to move beyond quantitative descriptions and reveal the underlying generative processes that give rise to behavior.

Path integrals, particular kinds, and strange things.

This paper describes a path integral formulation of the free energy principle. The ensuing account expresses the paths or trajectories that a particle takes as it evolves over time. The main results are a method or principle of least action that can be used to emulate the behaviour of particles in open exchange with their external milieu. Particles are defined by a particular partition, in which internal states are individuated from external states by active and sensory blanket states. The variational principle at hand allows one to interpret internal dynamics-of certain kinds of particles-as inferring external states that are hidden behind blanket states. We consider different kinds of particles, and to what extent they can be imbued with an elementary form of inference or sentience. Specifically, we consider the distinction between dissipative and conservative particles, inert and active particles and, finally, ordinary and strange particles. Strange particles can be described as inferring their own actions, endowing them with apparent autonomy or agency. In short-of the kinds of particles afforded by a particular partition-strange kinds may be apt for describing sentient behaviour.

On Bayesian mechanics: a physics of and by beliefs.

The aim of this paper is to introduce a field of study that has emerged over the last decade, called Bayesian mechanics. Bayesian mechanics is a probabilistic mechanics, comprising tools that enable us to model systems endowed with a particular partition (i.e. into particles), where the internal states (or the trajectories of internal states) of a particular system encode the parameters of beliefs about external states (or their trajectories). These tools allow us to write down mechanical theories for systems that look as if they are estimating posterior probability distributions over the causes of their sensory states. This provides a formal language for modelling the constraints, forces, potentials and other quantities determining the dynamics of such systems, especially as they entail dynamics on a space of beliefs (i.e. on a statistical manifold). Here, we will review the state of the art in the literature on the free energy principle, distinguishing between three ways in which Bayesian mechanics has been applied to particular systems (i.e. path-tracking, mode-tracking and mode-matching). We go on to examine a duality between the free energy principle and the constrained maximum entropy principle, both of which lie at the heart of Bayesian mechanics, and discuss its implications.

Bayesian mechanics for stationary processes.

This paper develops a Bayesian mechanics for adaptive systems. Firstly, we model the interface between a system and its environment with a Markov blanket. This affords conditions under which states internal to the blanket encode information about external states. Second, we introduce dynamics and represent adaptive systems as Markov blankets at steady state. This allows us to identify a wide class of systems whose internal states appear to infer external states, consistent with variational inference in Bayesian statistics and theoretical neuroscience. Finally, we partition the blanket into sensory and active states. It follows that active states can be seen as performing active inference and well-known forms of stochastic control (such as PID control), which are prominent formulations of adaptive behaviour in theoretical biology and engineering.

What do you know? ERP evidence for immediate use of common ground during online reference resolution.

Recent evidence on the time-course of conversational perspective taking is mixed. Some results suggest that listeners rapidly incorporate an interlocutor's knowledge during comprehension, while other findings suggest that listeners initially interpret language egocentrically. A key finding in support of the egocentric view comes from visual-world eye-tracking studies - listeners systematically look at potential referents that are known to them but unknown to the speaker. An alternative explanation is that eye movements might be driven by attentional processes that are unrelated to referent identification. To address this question, we assessed the time-course of perspective taking using event-related potentials (ERP). Participants were instructed to select a referent from a display of four animals (e.g., "Click on the brontosaurus with the boots") by a speaker who could only see three of the animals. A competitor (e.g., a brontosaurus with a purse) was either mutually visible, visible only to the listener, or absent from the display. Results showed that only the mutually visible competitor elicited an ERP signature of referential ambiguity. Critically, ERPs exhibited no evidence of referential confusion when the listener had privileged access to the competitor. Contra the egocentric hypothesis, this pattern of results indicates that listeners did not consider privileged competitors to be candidates for reference. These findings are consistent with theories of language processing that allow socio-pragmatic information to rapidly influence online language comprehension. The results also suggest that eye-tracking evidence in studies of online reference resolution may include distraction effects driven by privileged competitors and highlight the importance of using multiple measures to investigate perspective use.

Volitional social interaction prevents drug addiction in rat models.

Addiction treatment has not been appreciably improved by neuroscientific research. One problem is that mechanistic studies using rodent models do not incorporate volitional social factors, which play a critical role in human addiction. Here, using rats, we introduce an operant model of choice between drugs and social interaction. Independent of sex, drug class, drug dose, training conditions, abstinence duration, social housing, or addiction score in Diagnostic & Statistical Manual IV-based and intermittent access models, operant social reward prevented drug self-administration. This protection was lessened by delay or punishment of the social reward but neither measure was correlated with the addiction score. Social-choice-induced abstinence also prevented incubation of methamphetamine craving. This protective effect was associated with activation of central amygdala PKCδ-expressing inhibitory neurons and inhibition of anterior insular cortex activity. These findings highlight the need for incorporating social factors into neuroscience-based addiction research and support the wider implantation of socially based addiction treatments.

Compulsive Addiction-like Aggressive Behavior in Mice.

BACKGROUND: Some people are highly motivated to seek aggressive encounters, and among those who have been incarcerated for such behavior, recidivism rates are high. These observations echo two core features of drug addiction: high motivation to seek addictive substances, despite adverse consequences, and high relapse rates. Here we used established rodent models of drug addiction to determine whether they would be sensitive to "addiction-like" features of aggression in CD-1 mice. METHODS: In experiments 1 and 2, we trained older CD-1 mice to lever press for opportunities to attack younger C57BL6/J mice. We then tested them for relapse to aggression seeking after forced abstinence or punishment-induced suppression of aggression self-administration. In experiment 3, we trained a large cohort of CD-1 mice and tested them for choice-based voluntary suppression of aggression seeking, relapse to aggression seeking, progressive ratio responding, and punishment-induced suppression of aggression self-administration. We then used cluster analysis to identify patterns of individual differences in compulsive "addiction-like" aggressive behavior. RESULTS: In experiments 1 and 2, we observed strong motivation to acquire operant self-administration of opportunities to aggress and relapse vulnerability during abstinence. In experiment 3, cluster analysis of the aggression-related measures identified a subset of "addicted" mice (∼19%) that exhibited intense operant-reinforced attack behavior, decreased likelihood to select an alternative reinforcer over aggression, heightened relapse vulnerability and progressive ratio responding, and resilience to punishment-induced suppression of aggressive behavior. CONCLUSIONS: Using procedures established to model drug addiction, we showed that a subpopulation of CD-1 mice demonstrate "addiction-like" aggressive behavior, suggesting an evolutionary origin for compulsive aggression.