RESUMO
Human reasoning depends on reusing pieces of information by putting them together in new ways. However, very little is known about how compositional computation is implemented in the brain. Here, we ask participants to solve a series of problems that each require constructing a whole from a set of elements. With fMRI, we find that representations of novel constructed objects in the frontal cortex and hippocampus are relational and compositional. With MEG, we find that replay assembles elements into compounds, with each replay sequence constituting a hypothesis about a possible configuration of elements. The content of sequences evolves as participants solve each puzzle, progressing from predictable to uncertain elements and gradually converging on the correct configuration. Together, these results suggest a computational bridge between apparently distinct functions of hippocampal-prefrontal circuitry and a role for generative replay in compositional inference and hypothesis testing.
Assuntos
Hipocampo , Córtex Pré-Frontal , Humanos , Encéfalo , Lobo Frontal , Hipocampo/fisiologia , Imageamento por Ressonância Magnética/métodos , Vias Neurais , Córtex Pré-Frontal/fisiologiaRESUMO
Impaired decision-making is often displayed by individuals suffering from gambling disorder (GD). Since there are a variety of different phenomena influencing decision-making, we focused in this study on the effects of GD on neural and behavioural processes related to loss aversion and choice difficulty. Behavioural responses as well as brain images of 23 patients with GD and 20 controls were recorded while they completed a mixed gambles task, where they had to decide to either accept or reject gambles with different amounts of potential gain and loss. We found no behavioural loss aversion in either group and no group differences regarding loss and gain-related choice behaviour, but there was a weaker relation between choice difficulty and decision time in patients with GD. Similarly, we observed no group differences in processing of losses or gains, but choice difficulty was weaker associated with brain activity in the right anterior insula and anterior cingulate cortex in patients with GD. Our results showed for the first time the effects of GD on neural processes related to choice difficulty. In addition, our findings on choice difficulty give new insights on the psychopathology of GD and on neural processes related to impaired decision-making in GD.
Assuntos
Comportamento de Escolha , Tomada de Decisões , Jogo de Azar , Giro do Cíngulo , Imageamento por Ressonância Magnética , Humanos , Jogo de Azar/fisiopatologia , Jogo de Azar/diagnóstico por imagem , Jogo de Azar/psicologia , Masculino , Adulto , Comportamento de Escolha/fisiologia , Feminino , Giro do Cíngulo/diagnóstico por imagem , Giro do Cíngulo/fisiopatologia , Tomada de Decisões/fisiologia , Estudos de Casos e Controles , Pessoa de Meia-Idade , Encéfalo/diagnóstico por imagem , Encéfalo/fisiopatologia , Mapeamento Encefálico/métodos , Córtex Insular/diagnóstico por imagem , Adulto JovemRESUMO
Distinguishing between meaningful and meaningless sensory information is fundamental to forming accurate representations of the world. Dopamine is thought to play a central role in processing the meaningful information content of observations, which motivates an agent to update their beliefs about the environment. However, direct evidence for dopamine's role in human belief updating is lacking. We addressed this question in healthy volunteers who performed a model-based fMRI task designed to separate the neural processing of meaningful and meaningless sensory information. We modeled participant behavior using a normative Bayesian observer model and used the magnitude of the model-derived belief update following an observation to quantify its meaningful information content. We also acquired PET imaging measures of dopamine function in the same subjects. We show that the magnitude of belief updates about task structure (meaningful information), but not pure sensory surprise (meaningless information), are encoded in midbrain and ventral striatum activity. Using PET we show that the neural encoding of meaningful information is negatively related to dopamine-2/3 receptor availability in the midbrain and dexamphetamine-induced dopamine release capacity in the striatum. Trial-by-trial analysis of task performance indicated that subclinical paranoid ideation is negatively related to behavioral sensitivity to observations carrying meaningful information about the task structure. The findings provide direct evidence implicating dopamine in model-based belief updating in humans and have implications for understating the pathophysiology of psychotic disorders where dopamine function is disrupted.
Assuntos
Dopamina/metabolismo , Motivação/fisiologia , Transtornos Paranoides/metabolismo , Transtornos Paranoides/fisiopatologia , Transtornos Psicóticos/metabolismo , Transtornos Psicóticos/fisiopatologia , Adulto , Teorema de Bayes , Encéfalo/fisiopatologia , Mapeamento Encefálico/métodos , Cultura , Feminino , Humanos , Imageamento por Ressonância Magnética/métodos , Masculino , Receptores Dopaminérgicos/metabolismoRESUMO
This article describes a process theory based on active inference and belief propagation. Starting from the premise that all neuronal processing (and action selection) can be explained by maximizing Bayesian model evidence-or minimizing variational free energy-we ask whether neuronal responses can be described as a gradient descent on variational free energy. Using a standard (Markov decision process) generative model, we derive the neuronal dynamics implicit in this description and reproduce a remarkable range of well-characterized neuronal phenomena. These include repetition suppression, mismatch negativity, violation responses, place-cell activity, phase precession, theta sequences, theta-gamma coupling, evidence accumulation, race-to-bound dynamics, and transfer of dopamine responses. Furthermore, the (approximately Bayes' optimal) behavior prescribed by these dynamics has a degree of face validity, providing a formal explanation for reward seeking, context learning, and epistemic foraging. Technically, the fact that a gradient descent appears to be a valid description of neuronal activity means that variational free energy is a Lyapunov function for neuronal dynamics, which therefore conform to Hamilton's principle of least action.
RESUMO
Dopamine is implicated in a diverse range of cognitive functions including cognitive flexibility, task switching, signalling novel or unexpected stimuli as well as advance information. There is also longstanding line of thought that links dopamine with belief formation and, crucially, aberrant belief formation in psychosis. Integrating these strands of evidence would suggest that dopamine plays a central role in belief updating and more specifically in encoding of meaningful information content in observations. The precise nature of this relationship has remained unclear. To directly address this question we developed a paradigm that allowed us to decompose two distinct types of information content, information-theoretic surprise that reflects the unexpectedness of an observation, and epistemic value that induces shifts in beliefs or, more formally, Bayesian surprise. Using functional magnetic-resonance imaging in humans we show that dopamine-rich midbrain regions encode shifts in beliefs whereas surprise is encoded in prefrontal regions, including the pre-supplementary motor area and dorsal cingulate cortex. By linking putative dopaminergic activity to belief updating these data provide a link to false belief formation that characterises hyperdopaminergic states associated with idiopathic and drug induced psychosis.
Assuntos
Encéfalo/metabolismo , Cultura , Dopamina/metabolismo , Modelos Neurológicos , Adulto , Teorema de Bayes , Mapeamento Encefálico/métodos , Sinais (Psicologia) , Feminino , Humanos , Processamento de Imagem Assistida por Computador , Imageamento por Ressonância Magnética , Masculino , Modelos Teóricos , Adulto JovemRESUMO
Dopamine plays a key role in learning; however, its exact function in decision making and choice remains unclear. Recently, we proposed a generic model based on active (Bayesian) inference wherein dopamine encodes the precision of beliefs about optimal policies. Put simply, dopamine discharges reflect the confidence that a chosen policy will lead to desired outcomes. We designed a novel task to test this hypothesis, where subjects played a "limited offer" game in a functional magnetic resonance imaging experiment. Subjects had to decide how long to wait for a high offer before accepting a low offer, with the risk of losing everything if they waited too long. Bayesian model comparison showed that behavior strongly supported active inference, based on surprise minimization, over classical utility maximization schemes. Furthermore, midbrain activity, encompassing dopamine projection neurons, was accurately predicted by trial-by-trial variations in model-based estimates of precision. Our findings demonstrate that human subjects infer both optimal policies and the precision of those inferences, and thus support the notion that humans perform hierarchical probabilistic Bayesian inference. In other words, subjects have to infer both what they should do as well as how confident they are in their choices, where confidence may be encoded by dopaminergic firing.
Assuntos
Comportamento de Escolha/fisiologia , Dopamina/fisiologia , Substância Negra/fisiologia , Área Tegmentar Ventral/fisiologia , Adulto , Teorema de Bayes , Mapeamento Encefálico , Córtex Cerebral/fisiologia , Conflito Psicológico , Feminino , Humanos , Imageamento por Ressonância Magnética , Masculino , Pessoa de Meia-Idade , Risco , Adulto JovemRESUMO
Multiple features of the environment are often imbued with motivational significance, and the relative importance of these can change across contexts. The ability to flexibly adjust evaluative processes so that currently important features of the environment alone drive behavior is critical to adaptive routines. We know relatively little about the neural mechanisms involved, including whether motivationally significant features are obligatorily evaluated or whether current relevance gates access to value-sensitive regions. We addressed these questions using functional magnetic resonance imaging data and a task design where human subjects had to choose whether to accept or reject an offer indicated by visual and auditory stimuli. By manipulating, on a trial-by-trial basis, which stimulus determined the value of the offer, we show choice activity in the ventral striatum solely reflects the value of the currently relevant stimulus, consistent with a model wherein behavioral relevance modulates the impact of sensory stimuli on value processing. Choice outcome signals in this same region covaried positively with wins on accept trials, and negatively with wins on reject trials, consistent with striatal activity at feedback reflecting correctness of response rather than reward processing per se. We conclude that ventral striatum activity during decision making is dynamically modulated by behavioral context, indexed here by task relevance and action selection.
Assuntos
Mapeamento Encefálico , Comportamento de Escolha/fisiologia , Corpo Estriado/fisiologia , Motivação/fisiologia , Recompensa , Adulto , Feminino , Humanos , Imageamento por Ressonância Magnética , Masculino , Pessoa de Meia-Idade , Adulto JovemRESUMO
Deciding how much evidence to accumulate before making a decision is a problem we and other animals often face, but one that is not completely understood. This issue is particularly important because a tendency to sample less information (often known as reflection impulsivity) is a feature in several psychopathologies, such as psychosis. A formal understanding of information sampling may therefore clarify the computational anatomy of psychopathology. In this theoretical letter, we consider evidence accumulation in terms of active (Bayesian) inference using a generic model of Markov decision processes. Here, agents are equipped with beliefs about their own behavior--in this case, that they will make informed decisions. Normative decision making is then modeled using variational Bayes to minimize surprise about choice outcomes. Under this scheme, different facets of belief updating map naturally onto the functional anatomy of the brain (at least at a heuristic level). Of particular interest is the key role played by the expected precision of beliefs about control, which we have previously suggested may be encoded by dopaminergic neurons in the midbrain. We show that manipulating expected precision strongly affects how much information an agent characteristically samples, and thus provides a possible link between impulsivity and dopaminergic dysfunction. Our study therefore represents a step toward understanding evidence accumulation in terms of neurobiologically plausible Bayesian inference and may cast light on why this process is disordered in psychopathology.
Assuntos
Teorema de Bayes , Encéfalo/fisiologia , Comportamento de Escolha/fisiologia , Cognição , Tomada de Decisões , Modelos Teóricos , Animais , Encéfalo/anatomia & histologia , Encéfalo/efeitos dos fármacos , Simulação por Computador , Tomada de Decisões/efeitos dos fármacos , Dopamina/metabolismo , Dopamina/farmacologia , Entropia , Teoria dos Jogos , Humanos , Cadeias de Markov , Tempo de Reação/efeitos dos fármacos , Tempo de Reação/fisiologiaRESUMO
Adaptive behaviours depend on dynamically updating internal representations of the world based on the ever-changing environmental contingencies. People with a substance use disorder (pSUD) show maladaptive behaviours with high persistence in drug-taking, despite severe negative consequences. We recently proposed a salience misattribution model for addiction (SMMA; Kalhan et al., 2021), arguing that pSUD have aberrations in their updating processes where drug cues are misattributed as strong predictors of positive outcomes, but weaker predictors of negative outcomes. We also argued that conversely, non-drug cues are misattributed as weak predictors of positive outcomes, but stronger predictors of negative outcomes. We tested these hypotheses using a multi-cue reversal learning task, with reversals in whether drug or non-drug cues are relevant in predicting the outcome (monetary win or loss). We show that people with a tobacco use disorder (pTUD), do form misaligned internal representations. We found that pTUD updated less towards learning the drug cue's relevance in predicting a loss. Further, when neither drug nor non-drug cue predicted a win, pTUD updated more towards the drug cue being relevant predictors of that win. Our Bayesian belief updating model revealed that pTUD had a low estimated likelihood of non-drug cues being predictors of wins, compared to drug cues, which drove the misaligned updating. Overall, several hypotheses of the SMMA were supported, but not all. Our results implicate that strengthening the non-drug cue association with positive outcomes may help restore the misaligned internal representation in pTUD, and offers a quantifiable, computational account of these updating processes.
Assuntos
Tabagismo , Humanos , Sinais (Psicologia) , Teorema de Bayes , Aprendizagem , Adaptação PsicológicaRESUMO
The freedom to choose between options is strongly linked to notions of free will. Accordingly, several studies have shown that individuals demonstrate a preference for choice, or the availability of multiple options, over and above utilitarian value. Yet we lack a decision-making framework that integrates preference for choice with traditional utility maximisation in free choice behaviour. Here we test the predictions of an inference-based model of decision-making in which an agent actively seeks states yielding entropy (availability of options) in addition to utility (economic reward). We designed a study in which participants freely navigated a virtual environment consisting of two consecutive choices leading to reward locations in separate rooms. Critically, the choice of one room always led to two final doors while, in the second room, only one door was permissible to choose. This design allowed us to separately determine the influence of utility and entropy on participants' choice behaviour and their self-evaluation of free will. We found that choice behaviour was better predicted by an inference-based model than by expected utility alone, and that both the availability of options and the value of the context positively influenced participants' perceived freedom of choice. Moreover, this consideration of options was apparent in the ongoing motion dynamics as individuals navigated the environment. In a second study, in which participants selected between rooms that gave access to three or four doors, we observed a similar pattern of results, with participants preferring the room that gave access to more options and feeling freer in it. These results suggest that free choice behaviour is well explained by an inference-based framework in which both utility and entropy are optimised and supports the idea that the feeling of having free will is tightly related to options availability.
Assuntos
Comportamento de Escolha , Emoções , Humanos , Entropia , Recompensa , Tomada de DecisõesRESUMO
Replay in the brain has been viewed as rehearsal or, more recently, as sampling from a transition model. Here, we propose a new hypothesis: that replay is able to implement a form of compositional computation where entities are assembled into relationally bound structures to derive qualitatively new knowledge. This idea builds on recent advances in neuroscience, which indicate that the hippocampus flexibly binds objects to generalizable roles and that replay strings these role-bound objects into compound statements. We suggest experiments to test our hypothesis, and we end by noting the implications for AI systems which lack the human ability to radically generalize past experience to solve new problems.
Assuntos
Hipocampo , Aprendizagem , Humanos , Encéfalo , Potenciais de AçãoRESUMO
Humans often face sequential decision-making problems, in which information about the environmental reward structure is detached from rewards for a subset of actions. In the current exploratory study, we introduce an information-selective symmetric reversal bandit task to model such situations and obtained choice data on this task from 24 participants. To arbitrate between different decision-making strategies that participants may use on this task, we developed a set of probabilistic agent-based behavioral models, including exploitative and explorative Bayesian agents, as well as heuristic control agents. Upon validating the model and parameter recovery properties of our model set and summarizing the participants' choice data in a descriptive way, we used a maximum likelihood approach to evaluate the participants' choice data from the perspective of our model set. In brief, we provide quantitative evidence that participants employ a belief state-based hybrid explorative-exploitative strategy on the information-selective symmetric reversal bandit task, lending further support to the finding that humans are guided by their subjective uncertainty when solving exploration-exploitation dilemmas. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s42113-021-00112-3.
RESUMO
Within computational neuroscience, the algorithmic and neural basis of structure learning remains poorly understood. Concept learning is one primary example, which requires both a type of internal model expansion process (adding novel hidden states that explain new observations), and a model reduction process (merging different states into one underlying cause and thus reducing model complexity via meta-learning). Although various algorithmic models of concept learning have been proposed within machine learning and cognitive science, many are limited to various degrees by an inability to generalize, the need for very large amounts of training data, and/or insufficiently established biological plausibility. Using concept learning as an example case, we introduce a novel approach for modeling structure learning-and specifically state-space expansion and reduction-within the active inference framework and its accompanying neural process theory. Our aim is to demonstrate its potential to facilitate a novel line of active inference research in this area. The approach we lay out is based on the idea that a generative model can be equipped with extra (hidden state or cause) "slots" that can be engaged when an agent learns about novel concepts. This can be combined with a Bayesian model reduction process, in which any concept learning-associated with these slots-can be reset in favor of a simpler model with higher model evidence. We use simulations to illustrate this model's ability to add new concepts to its state space (with relatively few observations) and increase the granularity of the concepts it currently possesses. We also simulate the predicted neural basis of these processes. We further show that it can accomplish a simple form of "one-shot" generalization to new stimuli. Although deliberately simple, these simulation results highlight ways in which active inference could offer useful resources in developing neurocomputational models of structure learning. They provide a template for how future active inference research could apply this approach to real-world structure learning problems and assess the added utility it may offer.
RESUMO
BACKGROUND: Substance use disorders (SUDs) are a major public health risk. However, mechanisms accounting for continued patterns of poor choices in the face of negative life consequences remain poorly understood. METHODS: We use a computational (active inference) modeling approach, combined with multiple regression and hierarchical Bayesian group analyses, to examine how treatment-seeking individuals with one or more SUDs (alcohol, cannabis, sedatives, stimulants, hallucinogens, and/or opioids; N = 147) and healthy controls (HCs; N = 54) make choices to resolve uncertainty within a gambling task. A subset of SUDs (N = 49) and HCs (N = 51) propensity-matched on age, sex, and verbal IQ were also compared to replicate larger group findings. RESULTS: Results indicate that: (a) SUDs show poorer task performance than HCs (p = 0.03, Cohen's d = 0.33), with model estimates revealing less precise action selection mechanisms (p = 0.004, d = 0.43), a lower learning rate from losses (p = 0.02, d = 0.36), and a greater learning rate from gains (p = 0.04, d = 0.31); and (b) groups do not differ significantly in goal-directed information seeking. CONCLUSIONS: Findings suggest a pattern of inconsistent behavior in response to positive outcomes in SUDs combined with a tendency to attribute negative outcomes to chance. Specifically, individuals with SUDs fail to settle on a behavior strategy despite sufficient evidence of its success. These learning impairments could help account for difficulties in adjusting behavior and maintaining optimal decision-making during and after treatment.
Assuntos
Transtornos Relacionados ao Uso de Substâncias/psicologia , Adulto , Teorema de Bayes , Feminino , Jogo de Azar , Humanos , Masculino , Aprendizagem Baseada em Problemas , Análise e Desempenho de Tarefas , Incerteza , Adulto JovemRESUMO
Individuals suffering from pathological gambling (PG) show impaired decision making, but it is still not clear how this impairment is related to other traits and neuroanatomical characteristics. In this study, we investigated how the influence of PG on decision making (1) is connected to different impulsivity facets and (2) how it is related to gray matter volume (GMV) in various brain regions. Twenty-eight diagnosed PG patients and 23 healthy controls completed the cups task to measure decision making. In this task, participants had to decide between safe and risky options, which varied in expected value (EV) between risk advantageous, equal EV, and risk disadvantageous choices. A delay discounting task and the Barrant Impulsiveness Scale were applied to assess multiple impulsivity facets. In addition, structural magnetic resonance images were acquired. In comparison to the control group PG patients demonstrated more deficits in decision making, indicated by less EV sensitivity, but there was no significant difference in number of overall risky choices. Also, PG patients showed increased impulsivity in nearly every dimension. Results revealed (1) a positive correlation between decision making impairments and non-planning impulsivity but no significant relation to other impulsivity facets. Although we found no GMV differences between PG patients and controls, (2) a regions of interest analysis showed a correlation between medial orbitofrontal GMV and EV sensitivity in PG patients. Our findings showed that (1) the association between decision making and impulsivity can also be found in PG patients, but only for certain impulsivity facets. This suggests that it is essential to consider measuring different dimensions, when investigating impulsivity in a PG sample. Secondly, our findings revealed that (2) dysfunctional decision making-particularly the component of risk evaluation-is related to decreased GMV in the medial orbitofrontal cortex, a brain region concerned with processing of rewards. Interestingly, we did not find more risky choices for PG patients, and thus, we assume that decision making deficits in PG are primarily related to risk evaluation, not risk seeking, which is in line with our GMV findings.
RESUMO
Older adults struggle in dealing with changeable and uncertain environments across several cognitive domains. This has been attributed to difficulties in forming adequate task representations that help navigate uncertain environments. Here, we investigate how, in older adults, inadequate task representations impact on model-based reversal learning. We combined computational modeling and pupillometry during a novel model-based reversal learning task, which allowed us to isolate the relevance of task representations at feedback evaluation. We find that older adults overestimate the changeability of task states and consequently are less able to converge on unequivocal task representations through learning. Pupillometric measures and behavioral data show that these unreliable task representations in older adults manifest as a reduced ability to focus on feedback that is relevant for updating task representations, and as a reduced metacognitive awareness in the accuracy of their actions. Instead, the data suggested older adults' choice behavior was more consistent with a guidance by uninformative feedback properties such as outcome valence. Our study highlights that an inability to form adequate task representations may be a crucial factor underlying older adults' impaired model-based inference.
Assuntos
Envelhecimento/psicologia , Cognição , Reversão de Aprendizagem , Análise e Desempenho de Tarefas , Adulto , Idoso , Tomada de Decisões , Meio Ambiente , Feminino , Feedback Formativo , Humanos , Pessoa de Meia-Idade , Adulto JovemRESUMO
Successful behaviour depends on the right balance between maximising reward and soliciting information about the world. Here, we show how different types of information-gain emerge when casting behaviour as surprise minimisation. We present two distinct mechanisms for goal-directed exploration that express separable profiles of active sampling to reduce uncertainty. 'Hidden state' exploration motivates agents to sample unambiguous observations to accurately infer the (hidden) state of the world. Conversely, 'model parameter' exploration, compels agents to sample outcomes associated with high uncertainty, if they are informative for their representation of the task structure. We illustrate the emergence of these types of information-gain, termed active inference and active learning, and show how these forms of exploration induce distinct patterns of 'Bayes-optimal' behaviour. Our findings provide a computational framework for understanding how distinct levels of uncertainty systematically affect the exploration-exploitation trade-off in decision-making.
Assuntos
Comportamento Exploratório , Modelos Neurológicos , Algoritmos , Animais , Simulação por Computador , Humanos , AprendizagemRESUMO
Computational psychiatry is a rapidly emerging field that uses model-based quantities to infer the behavioral and neuronal abnormalities that underlie psychopathology. If successful, this approach promises key insights into (pathological) brain function as well as a more mechanistic and quantitative approach to psychiatric nosology-structuring therapeutic interventions and predicting response and relapse. The basic procedure in computational psychiatry is to build a computational model that formalizes a behavioral or neuronal process. Measured behavioral (or neuronal) responses are then used to infer the model parameters of a single subject or a group of subjects. Here, we provide an illustrative overview over this process, starting from the modeling of choice behavior in a specific task, simulating data, and then inverting that model to estimate group effects. Finally, we illustrate cross-validation to assess whether between-subject variables (e.g., diagnosis) can be recovered successfully. Our worked example uses a simple two-step maze task and a model of choice behavior based on (active) inference and Markov decision processes. The procedural steps and routines we illustrate are not restricted to a specific field of research or particular computational model but can, in principle, be applied in many domains of computational psychiatry.
Assuntos
Simulação por Computador , Transtornos Mentais , Teorema de Bayes , Comportamento de Escolha , Biologia Computacional , Humanos , Aprendizagem , Cadeias de Markov , Fenótipo , Psiquiatria , SoftwareRESUMO
This paper offers an active inference account of choice behaviour and learning. It focuses on the distinction between goal-directed and habitual behaviour and how they contextualise each other. We show that habits emerge naturally (and autodidactically) from sequential policy optimisation when agents are equipped with state-action policies. In active inference, behaviour has explorative (epistemic) and exploitative (pragmatic) aspects that are sensitive to ambiguity and risk respectively, where epistemic (ambiguity-resolving) behaviour enables pragmatic (reward-seeking) behaviour and the subsequent emergence of habits. Although goal-directed and habitual policies are usually associated with model-based and model-free schemes, we find the more important distinction is between belief-free and belief-based schemes. The underlying (variational) belief updating provides a comprehensive (if metaphorical) process theory for several phenomena, including the transfer of dopamine responses, reversal learning, habit formation and devaluation. Finally, we show that active inference reduces to a classical (Bellman) scheme, in the absence of ambiguity.
Assuntos
Aprendizagem , Comportamento de Escolha , Dopamina , Hábitos , RecompensaRESUMO
Classical economic models are predicated on the idea that the ultimate aim of choice is to maximize utility or reward. In contrast, an alternative perspective highlights the fact that adaptive behavior requires agents' to model their environment and minimize surprise about the states they frequent. We propose that choice behavior can be more accurately accounted for by surprise minimization compared to reward or utility maximization alone. Minimizing surprise makes a prediction at variance with expected utility models; namely, that in addition to attaining valuable states, agents attempt to maximize the entropy over outcomes and thus 'keep their options open'. We tested this prediction using a simple binary choice paradigm and show that human decision-making is better explained by surprise minimization compared to utility maximization. Furthermore, we replicated this entropy-seeking behavior in a control task with no explicit utilities. These findings highlight a limitation of purely economic motivations in explaining choice behavior and instead emphasize the importance of belief-based motivations.