Deep kinematic inference affords efficient and scalable control of bodily movements.

Performing goal-directed movements requires mapping goals from extrinsic (workspace-relative) to intrinsic (body-relative) coordinates and then to motor signals. Mainstream approaches based on optimal control realize the mappings by minimizing cost functions, which is computationally demanding. Instead, active inference uses generative models to produce sensory predictions, which allows a cheaper inversion to the motor signals. However, devising generative models to control complex kinematic chains like the human body is challenging. We introduce an active inference architecture that affords a simple but effective mapping from extrinsic to intrinsic coordinates via inference and easily scales up to drive complex kinematic chains. Rich goals can be specified in both intrinsic and extrinsic coordinates using attractive or repulsive forces. The proposed model reproduces sophisticated bodily movements and paves the way for computationally efficient and biologically plausible control of actuated systems.

Needing: An Active Inference Process for Physiological Motivation.

Need states are internal states that arise from deprivation of crucial biological stimuli. They direct motivation, independently of external learning. Despite their separate origin, they interact with reward processing systems that respond to external stimuli. This article aims to illuminate the functioning of the needing system through the lens of active inference, a framework for understanding brain and cognition. We propose that need states exert a pervasive influence on the organism, which in active inference terms translates to a "pervasive surprise"-a measure of the distance from the organism's preferred state. Crucially, we define needing as an active inference process that seeks to reduce this pervasive surprise. Through a series of simulations, we demonstrate that our proposal successfully captures key aspects of the phenomenology and neurobiology of needing. We show that as need states increase, the tendency to occupy preferred states strengthens, independently of external reward prediction. Furthermore, need states increase the precision of states (stimuli and actions) leading to preferred states, suggesting their ability to amplify the value of reward cues and rewards themselves. Collectively, our model and simulations provide valuable insights into the directional and underlying influence of need states, revealing how this influence amplifies the wanting or liking associated with relevant stimuli.

The fear-avoidance model as an embodied prediction of threat.

The fear-avoidance model is a well-established framework in the understanding of persistent pain. It proposes a dichotomous path: either the context is interpreted as safe; there is no fear reaction and, therefore, the individual engages in active (positive) coping; or the context is interpreted as threatening, leading to a self-reinforcing vicious circle of fear and (negative) avoidance. We propose an embodied interpretation of this phenomenon employing the joint framework of predictive coding and active inference. The key idea is that multisensory integration of exteroceptive, proprioceptive, and interoceptive sensory inputs can lead to dysfunctional experiences of threat in nonthreatening situations. Threat inference can promote fear responses, maladaptive strategies (i.e., avoidance) and self-provides evidence for threat in associated or future contexts, or both. Under this treatment, the prediction of nonrealized threat becomes self-evidencing and context-invariant, and hence self-perpetuating. Safety cues are unable to attenuate the interpretation of the negative context as the dominant inference of the context is threatful and gains more precision and becomes resistant over time. Our model provides an explanation for the emergence of a dysfunctional fear response in the clinical setting despite apparent safety based on modern concepts from theoretical (computational) neuroscience.

Humans account for cognitive costs when finding shortcuts: An information-theoretic analysis of navigation.

When faced with navigating back somewhere we have been before we might either retrace our steps or seek a shorter path. Both choices have costs. Here, we ask whether it is possible to characterize formally the choice of navigational plans as a bounded rational process that trades off the quality of the plan (e.g., its length) and the cognitive cost required to find and implement it. We analyze the navigation strategies of two groups of people that are firstly trained to follow a "default policy" taking a route in a virtual maze and then asked to navigate to various known goal destinations, either in the way they want ("Go To Goal") or by taking novel shortcuts ("Take Shortcut"). We address these wayfinding problems using InfoRL: an information-theoretic approach that formalizes the cognitive cost of devising a navigational plan, as the informational cost to deviate from a well-learned route (the "default policy"). In InfoRL, optimality refers to finding the best trade-off between route length and the amount of control information required to find it. We report five main findings. First, the navigational strategies automatically identified by InfoRL correspond closely to different routes (optimal or suboptimal) in the virtual reality map, which were annotated by hand in previous research. Second, people deliberate more in places where the value of investing cognitive resources (i.e., relevant goal information) is greater. Third, compared to the group of people who receive the "Go To Goal" instruction, those who receive the "Take Shortcut" instruction find shorter but less optimal solutions, reflecting the intrinsic difficulty of finding optimal shortcuts. Fourth, those who receive the "Go To Goal" instruction modulate flexibly their cognitive resources, depending on the benefits of finding the shortcut. Finally, we found a surprising amount of variability in the choice of navigational strategies and resource investment across participants. Taken together, these results illustrate the benefits of using InfoRL to address navigational planning problems from a bounded rational perspective.

Skilled motor control of an inverted pendulum implies low entropy of states but high entropy of actions.

The mastery of skills, such as balancing an inverted pendulum, implies a very accurate control of movements to achieve the task goals. Traditional accounts of skilled action control that focus on either routinization or perceptual control make opposite predictions about the ways we achieve mastery. The notion of routinization emphasizes the decrease of the variance of our actions, whereas the notion of perceptual control emphasizes the decrease of the variance of the states we visit, but not of the actions we execute. Here, we studied how participants managed control tasks of varying levels of difficulty, which consisted of controlling inverted pendulums of different lengths. We used information-theoretic measures to compare the predictions of alternative accounts that focus on routinization and perceptual control, respectively. Our results indicate that the successful performance of the control task strongly correlates with the decrease of state variability and the increase of action variability. As postulated by perceptual control theory, the mastery of skilled pendulum control consists in achieving stable control of goals by flexible means.

The traps of adaptation: Addiction as maladaptive referent-dependent evaluation.

Referent-dependent evaluation theories propose that the ongoing context influences how the brain attributes value to stimuli. What are the implications of these theories for understanding addiction? The paper asks this question by casting this disorder as a form of maladaptive referent-dependent evaluation. Specifically, addiction is proposed to arise from the establishment of an excessive reference point following repeated drug consumption. Several key aspects of the disorder emerge from this perspective, including withdrawal, tolerance, enhanced craving, negative mood, and diminished stimulus discriminability. As highlighted in the paper, this formulation has important analogies with classical accounts of addiction, such as set point theories and associative learning theories. Moreover, this picture fits with the pattern of striatal dopaminergic activity observed in addiction, a key neural signature of the disorder. Overall, the referent-dependent evaluation approach emerges as a useful add-on to the theoretical toolkit adopted to interpret addiction. This also supports the idea that referent-dependent evaluation might offer a general framework to understand various disorders characterised by disrupted motivation.

Active inference unifies intentional and conflict-resolution imperatives of motor control.

The field of motor control has long focused on the achievement of external goals through action (e.g., reaching and grasping objects). However, recent studies in conditions of multisensory conflict, such as when a subject experiences the rubber hand illusion or embodies an avatar in virtual reality, reveal the presence of unconscious movements that are not goal-directed, but rather aim at resolving multisensory conflicts; for example, by aligning the position of a person's arm with that of an embodied avatar. This second, conflict-resolution imperative of movement control did not emerge in classical studies of motor adaptation and online corrections, which did not allow movements to reduce the conflicts; and has been largely ignored so far in formal theories. Here, we propose a model of movement control grounded in the theory of active inference that integrates intentional and conflict-resolution imperatives. We present three simulations showing that the active inference model is able to characterize movements guided by the intention to achieve an external goal, by the necessity to resolve multisensory conflict, or both. Furthermore, our simulations reveal a fundamental difference between the (active) inference underlying intentional and conflict-resolution imperatives by showing that it is driven by two different (model and sensory) kinds of prediction errors. Finally, our simulations show that when movement is only guided by conflict resolution, the model incorrectly infers that is velocity is zero, as if it was not moving. This result suggests a novel speculative explanation for the fact that people are unaware of their subtle compensatory movements to avoid multisensory conflict. Furthermore, it can potentially help shed light on deficits of motor awareness that arise in psychopathological conditions.

Risky decisions are influenced by individual attributes as a function of risk preference.

It has long been assumed in economic theory that multi-attribute decisions involving several attributes or dimensions - such as probabilities and amounts of money to be earned during risky choices - are resolved by first combining the attributes of each option to form an overall expected value and then comparing the expected values of the alternative options, using a unique evidence accumulation process. A plausible alternative would be performing independent comparisons between the individual attributes and then integrating the results of the comparisons afterwards. Here, we devise a novel method to disambiguate between these types of models, by orthogonally manipulating the expected value of choice options and the relative salience of their attributes. Our results, based on behavioral measures and drift-diffusion models, provide evidence in favor of the framework where information about individual attributes independently impacts deliberation. This suggests that risky decisions are resolved by running in parallel multiple comparisons between the separate attributes - possibly alongside an additional comparison of expected value. This result stands in contrast with the assumption of standard economic theory that choices require a unique comparison of expected values and suggests that at the cognitive level, decision processes might be more distributed than commonly assumed. Beyond our planned analyses, we also discovered that attribute salience affects people of different risk preference type in different ways: risk-averse participants seem to focus more on probability, except when monetary amount is particularly high; risk-neutral/seeking participants, in contrast, seem to focus more on monetary amount, except when probability is particularly low.

An interoceptive illusion of effort induced by false heart-rate feedback.

Interoception, or the sense of the internal state of the body, is key to the adaptive regulation of our physiological needs. Recent theories contextualize interception within a predictive coding framework, according to which the brain both estimates and controls homeostatic and physiological variables, such as hunger, thirst, and effort levels, by orchestrating sensory, proprioceptive, and interoceptive signals from inside the body. This framework suggests that providing false interoceptive feedback may induce misperceptions of physiological variables, or "interoceptive illusions." Here we ask whether it is possible to produce an illusory perception of effort by giving participants false acoustic feedback about their heart-rate frequency during an effortful cycling task. We found that participants reported higher levels of perceived effort when their heart-rate feedback was faster compared with when they cycled at the same level of intensity with a veridical feedback. However, participants did not report lower effort when their heart-rate feedback was slower, which is reassuring, given that failing to notice one's own effort is dangerous in ecologically valid conditions. Our results demonstrate that false cardiac feedback can produce interoceptive illusions. Furthermore, our results pave the way for novel experimental manipulations that use illusions to study interoceptive processing.

Predictive Processing in Cognitive Robotics: A Review.

Predictive processing has become an influential framework in cognitive sciences. This framework turns the traditional view of perception upside down, claiming that the main flow of information processing is realized in a top-down, hierarchical manner. Furthermore, it aims at unifying perception, cognition, and action as a single inferential process. However, in the related literature, the predictive processing framework and its associated schemes, such as predictive coding, active inference, perceptual inference, and free-energy principle, tend to be used interchangeably. In the field of cognitive robotics, there is no clear-cut distinction on which schemes have been implemented and under which assumptions. In this letter, working definitions are set with the main aim of analyzing the state of the art in cognitive robotics research working under the predictive processing framework as well as some related nonrobotic models. The analysis suggests that, first, research in both cognitive robotics implementations and nonrobotic models needs to be extended to the study of how multiple exteroceptive modalities can be integrated into prediction error minimization schemes. Second, a relevant distinction found here is that cognitive robotics implementations tend to emphasize the learning of a generative model, while in nonrobotics models, it is almost absent. Third, despite the relevance for active inference, few cognitive robotics implementations examine the issues around control and whether it should result from the substitution of inverse models with proprioceptive predictions. Finally, limited attention has been placed on precision weighting and the tracking of prediction error dynamics. These mechanisms should help to explore more complex behaviors and tasks in cognitive robotics research under the predictive processing framework.

I want to believe: delusion, motivated reasoning, and Bayesian decision theory.

Introduction: Several arguments suggest that motivated reasoning (occurring when beliefs are not solely shaped by accuracy, but also by other motives such as promoting self-esteem or self-protection) is important in delusions. However, classical theories of delusion disregard the role of motivated reasoning. Thus, this role remains poorly understood.Methods: To explore the role of motivated reasoning in delusions, here we propose a computational model of delusion based on a Bayesian decision framework. This proposes that beliefs are not only evaluated based on their accuracy (as in classical theories), but also based on the cost (in terms of reward and punishment) of rejecting them.Results: The model proposes that, when the values at stake are high (as often it is the case in the context of delusion), a belief might be endorsed because rejecting it is evaluated as too costly, even if the belief is less accurate. This process might contribute to the genesis of delusions.Conclusions: Our account offers an interpretation of how motivated reasoning might shape delusions. This can inspire research on the affective and motivational processes supporting delusions in clinical conditions such as in psychosis, neurological disorders, and delusional disorder.

Keep your interoceptive streams under control: An active inference perspective on anorexia nervosa.

Anorexia nervosa (AN) is a psychiatric disorder primarily characterized by "restriction of energy intake relative to requirements leading to a significantly low body weight in the context of age, sex, developmental trajectory, and physical health" (DSM-5, n.d.). Here, we propose a novel interpretation of food restriction of AN, which elaborates on recent accounts of the brain as a predictive machine, continuously inferring and controling incoming signals, including bodily signals - i.e., interoceptive (active) inference. In this perspective, the extreme eating restrictions characterizing AN may serve the fundamental role of keeping under control (i.e., reducing excessively high levels of) interoceptive uncertainty, above and beyond controlling and mitigating concerns for body weight. In other words, noisy interoceptive streams may instantiate active strategies (i.e., eating restrictions) that amplify autonomic hunger signals, to minimize interoceptive uncertainty and maintain a more coherent sense of (interoceptive) self.

Social epistemic actions.

We consider the ways humans engage in social epistemic actions, to guide each other's attention, prediction, and learning processes towards salient information, at the timescale of online social interaction and joint action. This parallels the active guidance of other's attention, prediction, and learning processes at the longer timescale of niche construction and cultural practices, as discussed in the target article.

The value of uncertainty: An active inference perspective.

We discuss how uncertainty underwrites exploration and epistemic foraging from the perspective of active inference: a generic scheme that places pragmatic (utility maximization) and epistemic (uncertainty minimization) imperatives on an equal footing - as primary determinants of proximal behavior. This formulation contextualizes the complementary motivational incentives for reward-related stimuli and environmental uncertainty, offering a normative treatment of their trade-off.

Making the Environment an Informative Place: A Conceptual Analysis of Epistemic Policies and Sensorimotor Coordination.

How do living organisms decide and act with limited and uncertain information? Here, we discuss two computational approaches to solving these challenging problems: a "cognitive" and a "sensorimotor" enrichment of stimuli, respectively. In both approaches, the key notion is that agents can strategically modulate their behavior in informative ways, e.g., to disambiguate amongst alternative hypotheses or to favor the perception of stimuli providing the information necessary to later act appropriately. We discuss how, despite their differences, both approaches appeal to the notion that actions must obey both epistemic (i.e., information-gathering or uncertainty-reducing) and pragmatic (i.e., goal- or reward-maximizing) imperatives and balance them. Our computationally-guided analysis reveals that epistemic behavior is fundamental to understanding several facets of cognitive processing, including perception, decision making, and social interaction.

Endogenous Bioelectric Signaling Networks: Exploiting Voltage Gradients for Control of Growth and Form.

Living systems exhibit remarkable abilities to self-assemble, regenerate, and remodel complex shapes. How cellular networks construct and repair specific anatomical outcomes is an open question at the heart of the next-generation science of bioengineering. Developmental bioelectricity is an exciting emerging discipline that exploits endogenous bioelectric signaling among many cell types to regulate pattern formation. We provide a brief overview of this field, review recent data in which bioelectricity is used to control patterning in a range of model systems, and describe the molecular tools being used to probe the role of bioelectrics in the dynamic control of complex anatomy. We suggest that quantitative strategies recently developed to infer semantic content and information processing from ionic activity in the brain might provide important clues to cracking the bioelectric code. Gaining control of the mechanisms by which large-scale shape is regulated in vivo will drive transformative advances in bioengineering, regenerative medicine, and synthetic morphology, and could be used to therapeutically address birth defects, traumatic injury, and cancer.

Avoiding Accidents at the Champagne Reception.

Using a lifting and balancing task, we contrasted two alternative views of planning joint actions: one postulating that joint action involves distinct predictions for self and other, the other postulating that joint action involves coordinated plans between the coactors and reuse of bimanual models. We compared compensatory movements required to keep a tray balanced when 2 participants lifted glasses from each other's trays at the same time (simultaneous joint action) and when they took turns lifting (sequential joint action). Compared with sequential joint action, simultaneous joint action made it easier to keep the tray balanced. Thus, in keeping with the view that bimanual models are reused for joint action, predicting the timing of their own lifting action helped participants compensate for another person's lifting action. These results raise the possibility that simultaneous joint actions do not necessarily require distinguishing between one's own and the coactor's contributions to the action plan and may afford an agent-neutral stance.

Consensus Paper: Towards a Systems-Level View of Cerebellar Function: the Interplay Between Cerebellum, Basal Ganglia, and Cortex.

Despite increasing evidence suggesting the cerebellum works in concert with the cortex and basal ganglia, the nature of the reciprocal interactions between these three brain regions remains unclear. This consensus paper gathers diverse recent views on a variety of important roles played by the cerebellum within the cerebello-basal ganglia-thalamo-cortical system across a range of motor and cognitive functions. The paper includes theoretical and empirical contributions, which cover the following topics: recent evidence supporting the dynamical interplay between cerebellum, basal ganglia, and cortical areas in humans and other animals; theoretical neuroscience perspectives and empirical evidence on the reciprocal influences between cerebellum, basal ganglia, and cortex in learning and control processes; and data suggesting possible roles of the cerebellum in basal ganglia movement disorders. Although starting from different backgrounds and dealing with different topics, all the contributors agree that viewing the cerebellum, basal ganglia, and cortex as an integrated system enables us to understand the function of these areas in radically different ways. In addition, there is unanimous consensus between the authors that future experimental and computational work is needed to understand the function of cerebellar-basal ganglia circuitry in both motor and non-motor functions. The paper reports the most advanced perspectives on the role of the cerebellum within the cerebello-basal ganglia-thalamo-cortical system and illustrates other elements of consensus as well as disagreements and open questions in the field.

Active Inference: A Process Theory.

This article describes a process theory based on active inference and belief propagation. Starting from the premise that all neuronal processing (and action selection) can be explained by maximizing Bayesian model evidence-or minimizing variational free energy-we ask whether neuronal responses can be described as a gradient descent on variational free energy. Using a standard (Markov decision process) generative model, we derive the neuronal dynamics implicit in this description and reproduce a remarkable range of well-characterized neuronal phenomena. These include repetition suppression, mismatch negativity, violation responses, place-cell activity, phase precession, theta sequences, theta-gamma coupling, evidence accumulation, race-to-bound dynamics, and transfer of dopamine responses. Furthermore, the (approximately Bayes' optimal) behavior prescribed by these dynamics has a degree of face validity, providing a formal explanation for reward seeking, context learning, and epistemic foraging. Technically, the fact that a gradient descent appears to be a valid description of neuronal activity means that variational free energy is a Lyapunov function for neuronal dynamics, which therefore conform to Hamilton's principle of least action.

Active Inference, Curiosity and Insight.

This article offers a formal account of curiosity and insight in terms of active (Bayesian) inference. It deals with the dual problem of inferring states of the world and learning its statistical structure. In contrast to current trends in machine learning (e.g., deep learning), we focus on how people attain insight and understanding using just a handful of observations, which are solicited through curious behavior. We use simulations of abstract rule learning and approximate Bayesian inference to show that minimizing (expected) variational free energy leads to active sampling of novel contingencies. This epistemic behavior closes explanatory gaps in generative models of the world, thereby reducing uncertainty and satisfying curiosity. We then move from epistemic learning to model selection or structure learning to show how abductive processes emerge when agents test plausible hypotheses about symmetries (i.e., invariances or rules) in their generative models. The ensuing Bayesian model reduction evinces mechanisms associated with sleep and has all the hallmarks of "aha" moments. This formulation moves toward a computational account of consciousness in the pre-Cartesian sense of sharable knowledge (i.e., con: "together"; scire: "to know").