Nutrient-Sensitive Reinforcement Learning in Monkeys.

In reinforcement learning (RL), animals choose by assigning values to options and learn by updating these values from reward outcomes. This framework has been instrumental in identifying fundamental learning variables and their neuronal implementations. However, canonical RL models do not explain how reward values are constructed from biologically critical intrinsic reward components, such as nutrients. From an ecological perspective, animals should adapt their foraging choices in dynamic environments to acquire nutrients that are essential for survival. Here, to advance the biological and ecological validity of RL models, we investigated how (male) monkeys adapt their choices to obtain preferred nutrient rewards under varying reward probabilities. We found that the nutrient composition of rewards strongly influenced learning and choices. Preferences of the animals for specific nutrients (sugar, fat) affected how they adapted to changing reward probabilities; the history of recent rewards influenced choices of the monkeys more strongly if these rewards contained the their preferred nutrients (nutrient-specific reward history). The monkeys also chose preferred nutrients even when they were associated with lower reward probability. A nutrient-sensitive RL model captured these processes; it updated the values of individual sugar and fat components of expected rewards based on experience and integrated them into subjective values that explained the choices of the monkeys. Nutrient-specific reward prediction errors guided this value-updating process. Our results identify nutrients as important reward components that guide learning and choice by influencing the subjective value of choice options. Extending RL models with nutrient-value functions may enhance their biological validity and uncover nutrient-specific learning and decision variables.SIGNIFICANCE STATEMENT RL is an influential framework that formalizes how animals learn from experienced rewards. Although reward is a foundational concept in RL theory, canonical RL models cannot explain how learning depends on specific reward properties, such as nutrients. Intuitively, learning should be sensitive to the nutrient components of the reward to benefit health and survival. Here, we show that the nutrient (fat, sugar) composition of rewards affects how the monkeys choose and learn in an RL paradigm and that key learning variables including reward history and reward prediction error should be modified with nutrient-specific components to account for the choice behavior observed in the monkeys. By incorporating biologically critical nutrient rewards into the RL framework, our findings help advance the ecological validity of RL models.

Preferences for nutrients and sensory food qualities identify biological sources of economic values in monkeys.

Value is a foundational concept in reinforcement learning and economic choice theory. In these frameworks, individuals choose by assigning values to objects and learn by updating values with experience. These theories have been instrumental for revealing influences of probability, risk, and delay on choices. However, they do not explain how values are shaped by intrinsic properties of the choice objects themselves. Here, we investigated how economic value derives from the biologically critical components of foods: their nutrients and sensory qualities. When monkeys chose nutrient-defined liquids, they consistently preferred fat and sugar to low-nutrient alternatives. Rather than maximizing energy indiscriminately, they seemed to assign subjective values to specific nutrients, flexibly trading them against offered reward amounts. Nutrient-value functions accurately modeled these preferences, predicted choices across contexts, and accounted for individual differences. The monkeys' preferences shifted their daily nutrient balance away from dietary reference points, contrary to ecological foraging models but resembling human suboptimal eating in free-choice situations. To identify the sensory basis of nutrient values, we developed engineering tools that measured food textures on biological surfaces, mimicking oral conditions. Subjective valuations of two key texture parameters-viscosity and sliding friction-explained the monkeys' fat preferences, suggesting a texture-sensing mechanism for nutrient values. Extended reinforcement learning and choice models identified candidate neuronal mechanisms for nutrient-sensitive decision-making. These findings indicate that nutrients and food textures constitute critical reward components that shape economic values. Our nutrient-choice paradigm represents a promising tool for studying food-reward mechanisms in primates to better understand human-like eating behavior and obesity.