Imprecise neural computations as a source of adaptive behaviour in volatile environments.

In everyday life, humans face environments that feature uncertain and volatile or changing situations. Efficient adaptive behaviour must take into account uncertainty and volatility. Previous models of adaptive behaviour involve inferences about volatility that rely on complex and often intractable computations. Because such computations are presumably implausible biologically, it is unclear how humans develop efficient adaptive behaviours in such environments. Here, we demonstrate a counterintuitive result: simple, low-level inferences confined to uncertainty can produce near-optimal adaptive behaviour, regardless of the environmental volatility, assuming imprecisions in computation that conform to the psychophysical Weber law. We further show empirically that this Weber-imprecision model explains human behaviour in volatile environments better than optimal adaptive models that rely on high-level inferences about volatility, even when considering biologically plausible approximations of such models, as well as non-inferential models like adaptive reinforcement learning.

Computational noise in reward-guided learning drives behavioral variability in volatile environments.

When learning the value of actions in volatile environments, humans often make seemingly irrational decisions that fail to maximize expected value. We reasoned that these 'non-greedy' decisions, instead of reflecting information seeking during choice, may be caused by computational noise in the learning of action values. Here using reinforcement learning models of behavior and multimodal neurophysiological data, we show that the majority of non-greedy decisions stem from this learning noise. The trial-to-trial variability of sequential learning steps and their impact on behavior could be predicted both by blood oxygen level-dependent responses to obtained rewards in the dorsal anterior cingulate cortex and by phasic pupillary dilation, suggestive of neuromodulatory fluctuations driven by the locus coeruleus-norepinephrine system. Together, these findings indicate that most behavioral variability, rather than reflecting human exploration, is due to the limited computational precision of reward-guided learning.