Neural substrates of parallel devaluation-sensitive and devaluation-insensitive Pavlovian learning in humans.

We aim to differentiate the brain regions involved in the learning and encoding of Pavlovian associations sensitive to changes in outcome value from those that are not sensitive to such changes by combining a learning task with outcome devaluation, eye-tracking, and functional magnetic resonance imaging in humans. Contrary to theoretical expectation, voxels correlating with reward prediction errors in the ventral striatum and subgenual cingulate appear to be sensitive to devaluation. Moreover, regions encoding state prediction errors appear to be devaluation insensitive. We can also distinguish regions encoding predictions about outcome taste identity from predictions about expected spatial location. Regions encoding predictions about taste identity seem devaluation sensitive while those encoding predictions about an outcome's spatial location seem devaluation insensitive. These findings suggest the existence of multiple and distinct associative mechanisms in the brain and help identify putative neural correlates for the parallel expression of both devaluation sensitive and insensitive conditioned behaviors.

Neural substrates of parallel devaluation-sensitive and devaluation-insensitive Pavlovian learning in humans.

Pavlovian learning depends on multiple and parallel associations leading to distinct classes of conditioned responses that vary in their flexibility following changes in the value of an associated outcome. Here, we aimed to differentiate brain areas involved in learning and encoding associations that are sensitive to changes in the value of an outcome from those that are not sensitive to such changes. To address this question, we combined a Pavlovian learning task with outcome devaluation, eye-tracking and functional magnetic resonance imaging. We used computational modeling to identify brain regions involved in learning stimulus-reward associations and stimulus-stimulus associations, by testing for brain areas correlating with reward-prediction errors and state-prediction errors, respectively. We found that, contrary to theoretical predictions about reward prediction errors being exclusively model-free, voxels correlating with reward prediction errors in the ventral striatum and subgenual anterior cingulate cortex were sensitive to devaluation. On the other hand, brain areas correlating with state prediction errors were found to be devaluation insensitive. In a supplementary analysis, we distinguished brain regions encoding predictions about outcome taste identity from those involved in encoding predictions about its expected spatial location. A subset of regions involved in taste identity predictions were devaluation sensitive while those involved in encoding predictions about spatial location were devaluation insensitive. These findings provide insights into the role of multiple associative mechanisms in the brain in mediating Pavlovian conditioned behavior - illustrating how distinct neural pathways can in parallel produce both devaluation sensitive and devaluation insensitive behaviors.

Neural mechanisms underlying the hierarchical construction of perceived aesthetic value.

Little is known about how the brain computes the perceived aesthetic value of complex stimuli such as visual art. Here, we used computational methods in combination with functional neuroimaging to provide evidence that the aesthetic value of a visual stimulus is computed in a hierarchical manner via a weighted integration over both low and high level stimulus features contained in early and late visual cortex, extending into parietal and lateral prefrontal cortices. Feature representations in parietal and lateral prefrontal cortex may in turn be utilized to produce an overall aesthetic value in the medial prefrontal cortex. Such brain-wide computations are not only consistent with a feature-based mechanism for value construction, but also resemble computations performed by a deep convolutional neural network. Our findings thus shed light on the existence of a general neurocomputational mechanism for rapidly and flexibly producing value judgements across an array of complex novel stimuli and situations.

Using deep reinforcement learning to reveal how the brain encodes abstract state-space representations in high-dimensional environments.

Humans possess an exceptional aptitude to efficiently make decisions from high-dimensional sensory observations. However, it is unknown how the brain compactly represents the current state of the environment to guide this process. The deep Q-network (DQN) achieves this by capturing highly nonlinear mappings from multivariate inputs to the values of potential actions. We deployed DQN as a model of brain activity and behavior in participants playing three Atari video games during fMRI. Hidden layers of DQN exhibited a striking resemblance to voxel activity in a distributed sensorimotor network, extending throughout the dorsal visual pathway into posterior parietal cortex. Neural state-space representations emerged from nonlinear transformations of the pixel space bridging perception to action and reward. These transformations reshape axes to reflect relevant high-level features and strip away information about task-irrelevant sensory features. Our findings shed light on the neural encoding of task representations for decision-making in real-world situations.

Elucidating the underlying components of food valuation in the human orbitofrontal cortex.

The valuation of food is a fundamental component of our decision-making. Yet little is known about how value signals for food and other rewards are constructed by the brain. Using a food-based decision task in human participants, we found that subjective values can be predicted from beliefs about constituent nutritive attributes of food: protein, fat, carbohydrates and vitamin content. Multivariate analyses of functional MRI data demonstrated that, while food value is represented in patterns of neural activity in both medial and lateral parts of the orbitofrontal cortex (OFC), only the lateral OFC represents the elemental nutritive attributes. Effective connectivity analyses further indicate that information about the nutritive attributes represented in the lateral OFC is integrated within the medial OFC to compute an overall value. These findings provide a mechanistic account for the construction of food value from its constituent nutrients.