An efficiency framework for valence processing systems inspired by soft cross-wiring.

Recent experiments suggest that subsecond dopamine delivery to human striatum encodes a combination of reward prediction errors and counterfactual errors thus composing the actual with the possible into one neurochemical signal. Here, we present a model where the counterfactual part of these striatal dopamine fluctuations originates in another valuation system that shadows the dopamine system by acting as its near-antipode in terms of spike-rate encoding yet co-releases dopamine alongside its own native neurotransmitter. We show that such a hypothesis engenders important representational consequences where valence processing appears subject to the efficient encoding considerations common to the visual and auditory systems. This new perspective opens up important computational consequences for understanding how value-predicting information should integrate with sensory processing streams.

Necessary, yet dissociable contributions of the insular and ventromedial prefrontal cortices to norm adaptation: computational and lesion evidence in humans.

Social norms and their enforcement are fundamental to human societies. The ability to detect deviations from norms and to adapt to norms in a changing environment is therefore important to individuals' normal social functioning. Previous neuroimaging studies have highlighted the involvement of the insular and ventromedial prefrontal (vmPFC) cortices in representing norms. However, the necessity and dissociability of their involvement remain unclear. Using model-based computational modeling and neuropsychological lesion approaches, we examined the contributions of the insula and vmPFC to norm adaptation in seven human patients with focal insula lesions and six patients with focal vmPFC lesions, in comparison with forty neurologically intact controls and six brain-damaged controls. There were three computational signals of interest as participants played a fairness game (ultimatum game): sensitivity to the fairness of offers, sensitivity to deviations from expected norms, and the speed at which people adapt to norms. Significant group differences were assessed using bootstrapping methods. Patients with insula lesions displayed abnormally low adaptation speed to norms, yet detected norm violations with greater sensitivity than controls. Patients with vmPFC lesions did not have such abnormalities, but displayed reduced sensitivity to fairness and were more likely to accept the most unfair offers. These findings provide compelling computational and lesion evidence supporting the necessary, yet dissociable roles of the insula and vmPFC in norm adaptation in humans: the insula is critical for learning to adapt when reality deviates from norm expectations, and that the vmPFC is important for valuation of fairness during social exchange.

Cognitive strategies regulate fictive, but not reward prediction error signals in a sequential investment task.

Computational models of reward processing suggest that foregone or fictive outcomes serve as important information sources for learning and augment those generated by experienced rewards (e.g. reward prediction errors). An outstanding question is how these learning signals interact with top-down cognitive influences, such as cognitive reappraisal strategies. Using a sequential investment task and functional magnetic resonance imaging, we show that the reappraisal strategy selectively attenuates the influence of fictive, but not reward prediction error signals on investment behavior; such behavioral effect is accompanied by changes in neural activity and connectivity in the anterior insular cortex, a brain region thought to integrate subjective feelings with high-order cognition. Furthermore, individuals differ in the extent to which their behaviors are driven by fictive errors versus reward prediction errors, and the reappraisal strategy interacts with such individual differences; a finding also accompanied by distinct underlying neural mechanisms. These findings suggest that the variable interaction of cognitive strategies with two important classes of computational learning signals (fictive, reward prediction error) represent one contributing substrate for the variable capacity of individuals to control their behavior based on foregone rewards. These findings also expose important possibilities for understanding the lack of control in addiction based on possibly foregone rewarding outcomes.

Smokers' brains compute, but ignore, a fictive error signal in a sequential investment task.

Addicted individuals pursue substances of abuse even in the clear presence of positive outcomes that may be foregone and negative outcomes that may occur. Computational models of addiction depict the addicted state as a feature of a valuation disease, where drug-induced reward prediction error signals steer decisions toward continued drug use. Related models admit the possibility that valuation and choice are also directed by 'fictive' outcomes (outcomes that have not been experienced) that possess their own detectable error signals. We hypothesize that, in addiction, anomalies in these fictive error signals contribute to the diminished influence of potential consequences. Using a simple investment game and functional magnetic resonance imaging in chronic cigarette smokers, we measured neural and behavioral responses to error signals derived from actual experience and from fictive outcomes. In nonsmokers, both fictive and experiential error signals predicted subjects' choices and possessed distinct neural correlates. In chronic smokers, choices were not guided by error signals derived from what might have happened, despite ongoing and robust neural correlates of these fictive errors. These data provide human neuroimaging support for computational models of addiction and suggest the addition of fictive learning signals to reinforcement learning accounts of drug dependence.