Ergodicity-breaking reveals time optimal decision making in humans.

Ergodicity describes an equivalence between the expectation value and the time average of observables. Applied to human behaviour, ergodic theories of decision-making reveal how individuals should tolerate risk in different environments. To optimize wealth over time, agents should adapt their utility function according to the dynamical setting they face. Linear utility is optimal for additive dynamics, whereas logarithmic utility is optimal for multiplicative dynamics. Whether humans approximate time optimal behavior across different dynamics is unknown. Here we compare the effects of additive versus multiplicative gamble dynamics on risky choice. We show that utility functions are modulated by gamble dynamics in ways not explained by prevailing decision theories. Instead, as predicted by time optimality, risk aversion increases under multiplicative dynamics, distributing close to the values that maximize the time average growth of in-game wealth. We suggest that our findings motivate a need for explicitly grounding theories of decision-making on ergodic considerations.

Reward signalling in brainstem nuclei under fluctuating blood glucose.

Phasic dopamine release from mid-brain dopaminergic neurons is thought to signal errors of reward prediction (RPE). If reward maximisation is to maintain homeostasis, then the value of primary rewards should be coupled to the homeostatic errors they remediate. This leads to the prediction that RPE signals should be configured as a function of homeostatic state and thus diminish with the attenuation of homeostatic error. To test this hypothesis, we collected a large volume of functional MRI data from five human volunteers on four separate days. After fasting for 12 hours, subjects consumed preloads that differed in glucose concentration. Participants then underwent a Pavlovian cue-conditioning paradigm in which the colour of a fixation-cross was stochastically associated with the delivery of water or glucose via a gustometer. This design afforded computation of RPE separately for better- and worse-than expected outcomes during ascending and descending trajectories of serum glucose fluctuations. In the parabrachial nuclei, regional activity coding positive RPEs scaled positively with serum glucose for both ascending and descending glucose levels. The ventral tegmental area and substantia nigra became more sensitive to negative RPEs when glucose levels were ascending. Together, the results suggest that RPE signals in key brainstem structures are modulated by homeostatic trajectories of naturally occurring glycaemic flux, revealing a tight interplay between homeostatic state and the neural encoding of primary reward in the human brain.

Neurocomputational theories of homeostatic control.

Homeostasis is a problem for all living agents. It entails predictively regulating internal states within the bounds compatible with survival in order to maximise fitness. This can be achieved physiologically, through complex hierarchies of autonomic regulation, but it must also be achieved via behavioural control, both reactive and proactive. Here we briefly review some of the major theories of homeostatic control and their historical cognates, addressing how they tackle the optimisation of both physiological and behavioural homeostasis. We start with optimal control approaches, setting up key concepts, exploring their strengths and limitations. We then concentrate on contemporary neurocomputational approaches to homeostatic control. We primarily focus on a branch of reinforcement learning known as homeostatic reinforcement learning (HRL). A central premise of HRL is that reward optimisation is directly coupled to homeostatic control. A central construct in this framework is the drive function which maps from homeostatic state to motivational drive, where reductions in drive are operationally defined as reward values. We explain HRL's main advantages, empirical applications, and conceptual insights. Notably, we show how simple constraints on the drive function can yield a normative account of predictive control, as well as account for phenomena such as satiety, risk aversion, and interactions between competing homeostatic needs. We illustrate how HRL agents can learn to avoid hazardous states without any need to experience them, and how HRL can be applied in clinical domains. Finally, we outline several challenges to HRL, and how survival constraints and active inference models could circumvent these problems.

Patient profiling for success after weight loss surgery (GO Bypass study): An interdisciplinary study protocol.

Despite substantial research efforts, the mechanisms proposed to explain weight loss after gastric bypass (RYGB) and sleeve gastrectomy (SL) do not explain the large individual variation seen after these treatments. A complex set of factors are involved in the onset and development of obesity and these may also be relevant for the understanding of why success with treatments vary considerably between individuals. This calls for explanatory models that take into account not only biological determinants but also behavioral, affective and contextual factors. In this prospective study, we recruited 47 women and 8 men, aged 25-56 years old, with a BMI of 45.8 ±â€¯7.1 kg/m2 from the waiting list for RYGB and SL at Køge hospital, Denmark. Pre-surgery and 1.5, 6 and 18 months after surgery we assessed various endpoints spanning multiple domains. Endpoints were selected on basis of previous studies and include: physiological measures: anthropometrics, vital signs, biochemical measures and appetite hormones, genetics, gut microbiota, appetite sensation, food and taste preferences, neural sensitivity, sensory perception and movement behaviors; psychological measures: general psychiatric symptom-load, depression, eating disorders, ADHD, personality disorder, impulsivity, emotion regulation, attachment pattern, general self-efficacy, alexithymia, internalization of weight bias, addiction, quality of life and trauma; and sociological and anthropological measures: sociodemographic measures, eating behavior, weight control practices and psycho-social factors.Joining these many endpoints and methodologies from different scientific disciplines and creating a multi-dimensional predictive model has not previously been attempted. Data on the primary endpoint are expected to be published in 2018. TRIAL REGISTRATION: Clinicaltrials. gov ID NCT02070081.

Tuning the Brake While Raising the Stake: Network Dynamics during Sequential Decision-Making.

UNLABELLED: When gathering valued goods, risk and reward are often coupled and escalate over time, for instance, during foraging, trading, or gambling. This escalating frame requires agents to continuously balance expectations of reward against those of risk. To address how the human brain dynamically computes these tradeoffs, we performed whole-brain fMRI while healthy young individuals engaged in a sequential gambling task. Participants were repeatedly confronted with the option to continue with throwing a die to accumulate monetary reward under escalating risk, or the alternative option to stop to bank the current balance. Within each gambling round, the accumulation of gains gradually increased reaction times for "continue" choices, indicating growing uncertainty in the decision to continue. Neural activity evoked by "continue" choices was associated with growing activity and connectivity of a cortico-subcortical "braking" network that positively scaled with the accumulated gains, including pre-supplementary motor area (pre-SMA), inferior frontal gyrus, caudate, and subthalamic nucleus (STN). The influence of the STN on continue-evoked activity in the pre-SMA was predicted by interindividual differences in risk-aversion attitudes expressed during the gambling task. Furthermore, activity in dorsal anterior cingulate cortex (ACC) reflected individual choice tendencies by showing increased activation when subjects made nondefault "continue" choices despite an increasing tendency to stop, but ACC activity did not change in proportion with subjective choice uncertainty. Together, the results implicate a key role of dorsal ACC, pre-SMA, inferior frontal gyrus, and STN in computing the trade-off between escalating reward and risk in sequential decision-making. SIGNIFICANCE STATEMENT: Using a paradigm where subjects experienced increasing potential rewards coupled with increasing risk, this study addressed two unresolved questions in the field of decision-making: First, we investigated an "inhibitory" network of regions that has so far been investigated with externally cued action inhibition. In this study, we show that the dynamics in this network under increasingly risky decisions are predictive of subjects' risk attitudes. Second, we contribute to a currently ongoing debate about the anterior cingulate cortex's role in sequential foraging decisions by showing that its activity is related to making nondefault choices rather than to choice uncertainty.