The empirical status of predictive coding and active inference.

Research on predictive processing models has focused largely on two specific algorithmic theories: Predictive Coding for perception and Active Inference for decision-making. While these interconnected theories possess broad explanatory potential, they have only recently begun to receive direct empirical evaluation. Here, we review recent studies of Predictive Coding and Active Inference with a focus on evaluating the degree to which they are empirically supported. For Predictive Coding, we find that existing empirical evidence offers modest support. However, some positive results can also be explained by alternative feedforward (e.g., feature detection-based) models. For Active Inference, most empirical studies have focused on fitting these models to behavior as a means of identifying and explaining individual or group differences. While Active Inference models tend to explain behavioral data reasonably well, there has not been a focus on testing empirical validity of active inference theory per se, which would require formal comparison to other models (e.g., non-Bayesian or model-free reinforcement learning models). This review suggests that, while promising, a number of specific research directions are still necessary to evaluate the empirical adequacy and explanatory power of these algorithms.

Directed exploration is elevated in affective disorders but reduced by an aversive interoceptive state induction.

Elevated anxiety and uncertainty avoidance are known to exacerbate maladaptive choice in individuals with affective disorders. However, the differential roles of state vs. trait anxiety remain unclear, and underlying computational mechanisms have not been thoroughly characterized. In the present study, we investigated how a somatic (interoceptive) state anxiety induction influences learning and decision-making under uncertainty in individuals with clinically significant levels of trait anxiety. A sample of 58 healthy comparisons (HCs) and 61 individuals with affective disorders (iADs; i.e., depression and/or anxiety) completed a previously validated explore-exploit decision task, with and without an added breathing resistance manipulation designed to induce state anxiety. Computational modeling revealed a pattern in which iADs showed greater information-seeking (i.e., directed exploration; Cohen's d=.39, p=.039) in resting conditions, but that this was reduced by the anxiety induction. The affective disorders group also showed slower learning rates across conditions (Cohen's d=.52, p=.003), suggesting more persistent uncertainty. These findings highlight a complex interplay between trait anxiety and state anxiety. Specifically, while elevated trait anxiety is associated with persistent uncertainty, acute somatic anxiety can paradoxically curtail exploratory behaviors, potentially reinforcing maladaptive decision-making patterns in affective disorders.

Individuals with Methamphetamine Use Disorder Show Reduced Directed Exploration and Learning Rates Independent of an Aversive Interoceptive State Induction.

Methamphetamine Use Disorder (MUD) is associated with substantially reduced quality of life. Yet, decisions to use persist, due in part to avoidance of anticipated withdrawal states. However, the specific cognitive mechanisms underlying this decision process, and possible modulatory effects of aversive states, remain unclear. Here, 56 individuals with MUD and 58 healthy comparisons (HCs) performed a decision task, both with and without an aversive interoceptive state induction. Computational modeling measured the tendency to test beliefs about uncertain outcomes (directed exploration) and the ability to update beliefs in response to outcomes (learning rates). Compared to HCs, those with MUD exhibited less directed exploration and slower learning rates, but these differences were not affected by aversive state induction. These results suggest novel, state-independent computational mechanisms whereby individuals with MUD may have difficulties in testing beliefs about the tolerability of abstinence and in adjusting behavior in response to consequences of continued use.

Planning to Learn: A Novel Algorithm for Active Learning during Model-Based Planning.

Active Inference is a recently developed framework for modeling decision processes under uncertainty. Over the last several years, empirical and theoretical work has begun to evaluate the strengths and weaknesses of this approach and how it might be extended and improved. One recent extension is the "sophisticated inference" (SI) algorithm, which improves performance on multi-step planning problems through a recursive decision tree search. However, little work to date has been done to compare SI to other established planning algorithms in reinforcement learning (RL). In addition, SI was developed with a focus on inference as opposed to learning. The present paper therefore has two aims. First, we compare performance of SI to Bayesian RL schemes designed to solve similar problems. Second, we present and compare an extension of SI - sophisticated learning (SL) - that more fully incorporates active learning during planning. SL maintains beliefs about how model parameters would change under the future observations expected under each policy. This allows a form of counterfactual retrospective inference in which the agent considers what could be learned from current or past observations given different future observations. To accomplish these aims, we make use of a novel, biologically inspired environment that requires an optimal balance between goal-seeking and active learning, and which was designed to highlight the problem structure for which SL offers a unique solution. This setup requires an agent to continually search an open environment for available (but changing) resources in the presence of competing affordances for information gain. Our simulations demonstrate that SL outperforms all other algorithms in this context - most notably, Bayes-adaptive RL and upper confidence bound (UCB) algorithms, which aim to solve multi-step planning problems using similar principles (i.e., directed exploration and counterfactual reasoning about belief updates given different possible actions/observations). These results provide added support for the utility of Active Inference in solving this class of biologically-relevant problems and offer added tools for testing hypotheses about human cognition.