Neuro-computational mechanisms and individual biases in action-outcome learning under moral conflict.

Learning to predict action outcomes in morally conflicting situations is essential for social decision-making but poorly understood. Here we tested which forms of Reinforcement Learning Theory capture how participants learn to choose between self-money and other-shocks, and how they adapt to changes in contingencies. We find choices were better described by a reinforcement learning model based on the current value of separately expected outcomes than by one based on the combined historical values of past outcomes. Participants track expected values of self-money and other-shocks separately, with the substantial individual difference in preference reflected in a valuation parameter balancing their relative weight. This valuation parameter also predicted choices in an independent costly helping task. The expectations of self-money and other-shocks were biased toward the favored outcome but fMRI revealed this bias to be reflected in the ventromedial prefrontal cortex while the pain-observation network represented pain prediction errors independently of individual preferences.

Humans depart from optimal computational models of interactive decision-making during competition under partial information.

Decision making under uncertainty in multiagent settings is of increasing interest in decision science. The degree to which human agents depart from computationally optimal solutions in socially interactive settings is generally unknown. Such understanding provides insight into how social contexts affect human interaction and the underlying contributions of Theory of Mind. In this paper, we adapt the well-known 'Tiger Problem' from artificial-agent research to human participants in solo and interactive settings. Compared to computationally optimal solutions, participants gathered less information before outcome-related decisions when competing than cooperating with others. These departures from optimality were not haphazard but showed evidence of improved performance through learning. Costly errors emerged under conditions of competition, yielding both lower rates of rewarding actions and accuracy in predicting others. Taken together, this work provides a novel approach and insights into studying human social interaction when shared information is partial.

Theory of mind and decision science: Towards a typology of tasks and computational models.

The ability to form a Theory of Mind (ToM), i.e., to theorize about others' mental states to explain and predict behavior in relation to attributed intentional states, constitutes a hallmark of human cognition. These abilities are multi-faceted and include a variety of different cognitive sub-functions. Here, we focus on decision processes in social contexts and review a number of experimental and computational modeling approaches in this field. We provide an overview of experimental accounts and formal computational models with respect to two dimensions: interactivity and uncertainty. Thereby, we aim at capturing the nuances of ToM functions in the context of social decision processes. We suggest there to be an increase in ToM engagement and multiplexing as social cognitive decision-making tasks become more interactive and uncertain. We propose that representing others as intentional and goal directed agents who perform consequential actions is elicited only at the edges of these two dimensions. Further, we argue that computational models of valuation and beliefs follow these dimensions to best allow researchers to effectively model sophisticated ToM-processes. Finally, we relate this typology to neuroimaging findings in neurotypical (NT) humans, studies of persons with autism spectrum (AS), and studies of nonhuman primates.

Semantic Incongruency Interferes With Endogenous Attention in Cross-Modal Integration of Semantically Congruent Objects.

Efficient multisensory integration is often influenced by other cognitive processes including, but not limited to, semantic congruency and focused endogenous attention. Semantic congruency can re-allocate processing resources to the location of a congruent stimulus, while attention can prioritize the integration of multi-sensory stimuli under focus. Here, we explore the robustness of this phenomenon in the context of three stimuli, two of which are in the focus of endogenous attention. Participants completed an endogenous attention task with a stimulus compound consisting of 3 different objects: (1) a visual object (V) in the foreground, (2) an auditory object (A), and (3) a visual background scene object (B). Three groups of participants focused their attention on either the visual object and auditory sound (Group VA, n = 30), the visual object and the background (VB, n = 27), or the auditory sound and the background (AB, n = 30), and judged the semantic congruency of the objects under focus. Congruency varied systematically across all 3 stimuli: All stimuli could be semantically incongruent (e.g., V, ambulance; A, church bell; and B, swimming-pool) or all could be congruent (e.g., V, lion; A, roar; and B, savannah), or two objects could be congruent with the remaining one incongruent to the other two (e.g., V, duck; A, quack; and B, phone booth). Participants exhibited a distinct pattern of errors: when participants attended two congruent objects (e.g., group VA: V, lion; A, roar), in the presence of an unattended, incongruent third object (e.g., B, bath room) they tended to make more errors than in any other stimulus combination. Drift diffusion modeling of the behavioral data revealed a significantly smaller drift rate in two-congruent-attended condition, indicating slower evidence accumulation, which was likely due to interference from the unattended, incongruent object. Interference with evidence accumulation occurred independently of which pair of objects was in the focus of attention, which suggests that the vulnerability of congruency judgments to incongruent unattended distractors is not affected by sensory modalities. A control analysis ruled out the simple explanation of a negative response bias. These findings implicate that our perceptual system is highly sensitive to semantic incongruencies even when they are not endogenously attended.

Rapid face orienting in infants and school-age children with and without autism: Exploring measurement invariance in eye-tracking.

Questions concerning the ontogenetic stability of autism have recently received increased attention as long-term longitudinal studies have appeared in the literature. Most experimental measures are designed for specific ages and functioning levels, yet developing experimental tasks appropriate for a wide range of ages and functioning levels is critical for future long-term longitudinal studies, and treatment studies implemented at different ages. Accordingly, we designed an eye-tracking task to measure preferential orienting to facial features and implemented it with groups of participants with varying levels of functioning: infants, and school-age children with and without autism. All groups fixated eyes first, revealing an early and stable orienting bias. This indicates common bias towards the eyes across participants regardless of age or diagnosis. We also demonstrate that this eye-tracking task can be used with diverse populations who range in age and cognitive functioning. Our developmental approach has conceptual implications for future work focused on task development and particularly new experimental measures that offer measurement equivalence across broad age ranges, intellectual functioning and verbal abilities.

Congruence of Inherent and Acquired Values Facilitates Reward-Based Decision-Making.

UNLABELLED: Most real-life cues exhibit certain inherent values that may interfere with or facilitate the acquisition of new expected values during associative learning. In particular, when inherent and acquired values are congruent, learning may progress more rapidly. Here we investigated such an influence through a 2 × 2 factorial design, using attractiveness (high/low) of the facial picture as a proxy for the inherent value of the cue and its reward probability (high/low) as a surrogate for the acquired value. Each picture was paired with a monetary win or loss either congruently or incongruently. Behavioral results from 32 human participants indicated both faster response time and faster learning rate for value-congruent cue-outcome pairings. Model-based fMRI analysis revealed a fractionation of reinforcement learning (RL) signals in the ventral striatum, including a strong and novel correlation between the cue-specific decaying learning rate and BOLD activity in the ventral caudate. Additionally, we detected a functional link between neural signals of both learning rate and reward prediction error in the ventral striatum, and the signal of expected value in the ventromedial prefrontal cortex, showing a novel confirmation of the mathematical RL model via functional connectivity. SIGNIFICANCE STATEMENT: Most real-world decisions require the integration of inherent value and sensitivity to outcomes to facilitate adaptive learning. Inherent value is drawing increasing interest from decision scientists because it influences decisions in contexts ranging from advertising to investing. This study provides novel insight into how inherent value influences the acquisition of new expected value during associative learning. Specifically, we find that the congruence between the inherent value and the acquired reward influences the neural coding of learning rate. We also show for the first time that neuroimaging signals coding the learning rate, prediction error, and acquired value follow the multiplicative Rescorla-Wagner learning rule, a finding predicted by reinforcement learning theory.

Facial emotion recognition in agenesis of the corpus callosum.

BACKGROUND: Impaired social functioning is a common symptom of individuals with developmental disruptions in callosal connectivity. Among these developmental conditions, agenesis of the corpus callosum provides the most extreme and clearly identifiable example of callosal disconnection. To date, deficits in nonliteral language comprehension, humor, theory of mind, and social reasoning have been documented in agenesis of the corpus callosum. Here, we examined a basic social ability as yet not investigated in this population: recognition of facial emotion and its association with social gaze. METHODS: Nine individuals with callosal agenesis and nine matched controls completed four tasks involving emotional faces: emotion recognition from upright and inverted faces, gender recognition, and passive viewing. Eye-tracking data were collected concurrently on all four tasks and analyzed according to designated facial regions of interest. RESULTS: Individuals with callosal agenesis exhibited impairments in recognizing emotions from upright faces, in particular lower accuracy for fear and anger, and these impairments were directly associated with diminished attention to the eye region. The callosal agenesis group exhibited greater consistency in emotion recognition across conditions (upright vs. inverted), with poorest performance for fear identification in both conditions. The callosal agenesis group also had atypical facial scanning (lower fractional dwell time in the eye region) during gender naming and passive viewing of faces, but they did not differ from controls on gender naming performance. The pattern of results did not differ when taking into account full-scale intelligence quotient or presence of autism spectrum symptoms. CONCLUSIONS: Agenesis of the corpus callosum results in a pattern of atypical facial scanning characterized by diminished attention to the eyes. This pattern suggests that reduced callosal connectivity may contribute to the development and maintenance of emotion processing deficits involving reduced attention to others' eyes.

Primary somatosensory cortex discriminates affective significance in social touch.

Another person's caress is one of the most powerful of all emotional social signals. How much the primary somatosensory cortices (SIs) participate in processing the pleasantness of such social touch remains unclear. Although ample empirical evidence supports the role of the insula in affective processing of touch, here we argue that SI might be more involved in affective processing than previously thought by showing that the response in SI to a sensual caress is modified by the perceived sex of the caresser. In a functional MRI study, we manipulated the perceived affective quality of a caress independently of the sensory properties at the skin: heterosexual males believed they were sensually caressed by either a man or woman, although the caress was in fact invariantly delivered by a female blind to condition type. Independent analyses showed that SI encoded, and was modulated by, the visual sex of the caress, and that this effect is unlikely to originate from the insula. This suggests that current models may underestimate the role played by SI in the affective processing of social touch.

Asymmetrical use of eye information from faces following unilateral amygdala damage.

The human amygdalae are involved in processing visual information about the eyes within faces, and play an essential role in the use of information from the eye region of the face in order to judge emotional expressions, as well as in directing gaze to the eyes in conversations with real people. However, the roles played here by the left and right amygdala individually remain unknown. Here we investigated this question by applying the 'Bubbles' method, which asks viewers to discriminate facial emotions from randomly sampled small regions of a face, to 23 neurological participants with focal, unilateral amygdala damage (10 to the right amygdala). We found a statistically significant asymmetry in the use of eye information when comparing those with unilateral left lesions to those with unilateral right lesions, specifically during emotion judgments. The findings have implications for the amygdala's role in emotion recognition and gaze direction during face processing.

A neural basis for the effect of candidate appearance on election outcomes.

Election outcomes correlate with judgments based on a candidate's visual appearance, suggesting that the attributions viewers make based on appearance, so-called thin-slice judgments, influence voting. Yet, it is not known whether the effect of appearance on voting is more strongly influenced by positive or negative attributions, nor which neural mechanisms subserve this effect. We conducted two independent brain imaging studies to address this question. In Study 1, images of losing candidates elicited greater activation in the insula and ventral anterior cingulate than images of winning candidates. Winning candidates elicited no differential activation at all. This suggests that negative attributions from appearance exert greater influence on voting than do positive. We further tested this hypothesis in Study 2 by asking a separate group of participants to judge which unfamiliar candidate in a pair looked more attractive, competent, deceitful and threatening. When negative attribution processing was enhanced (specifically, under judgment of threat), images of losing candidates again elicited greater activation in the insula and ventral anterior cingulate. Together, these findings support the view that negative attributions play a critical role in mediating the effects of appearance on voter decisions, an effect that may be of special importance when other information is absent.

Distinct face-processing strategies in parents of autistic children.

In his original description of autism, Kanner [1] noted that the parents of autistic children often exhibited unusual social behavior themselves, consistent with what we now know about the high heritability of autism [2]. We investigated this so-called Broad Autism Phenotype in the parents of children with autism, who themselves did not receive a diagnosis of any psychiatric illness. Building on recent quantifications of social cognition in autism [3], we investigated face processing by using the "bubbles" method [4] to measure how viewers make use of information from specific facial features in order to judge emotions. Parents of autistic children who were assessed as socially aloof (N = 15), a key component of the phenotype [5], showed a remarkable reduction in processing the eye region in faces, together with enhanced processing of the mouth, compared to a control group of parents of neurotypical children (N = 20), as well as to nonaloof parents of autistic children (N = 27, whose pattern of face processing was intermediate). The pattern of face processing seen in the Broad Autism Phenotype showed striking similarities to that previously reported to occur in autism [3] and for the first time provides a window into the endophenotype that may result from a subset of the genes that contribute to social cognition.

Amygdala damage impairs eye contact during conversations with real people.

The role of the human amygdala in real social interactions remains essentially unknown, although studies in nonhuman primates and studies using photographs and video in humans have shown it to be critical for emotional processing and suggest its importance for social cognition. We show here that complete amygdala lesions result in a severe reduction in direct eye contact during conversations with real people, together with an abnormal increase in gaze to the mouth. These novel findings from real social interactions are consistent with an hypothesized role for the amygdala in autism and the approach taken here opens up new directions for quantifying social behavior in humans.

Analysis of face gaze in autism using "Bubbles".

One of the components of abnormal social functioning in autism is an impaired ability to direct eye gaze onto other people's faces in social situations. Here, we investigated the relationship between gaze onto the eye and mouth regions of faces, and the visual information that was present within those regions. We used the "Bubbles" method to vary the facial information available on any given trial by revealing only small parts of the face, and measured the eye movements made as participants viewed these stimuli. Compared to ten IQ- and age-matched healthy controls, eight participants with autism showed less fixation specificity to the eyes and mouth, a greater tendency to saccade away from the eyes when information was present in those regions, and abnormal directionality of saccades. The findings provide novel detail to the abnormal way in which people with autism look at faces, an impairment that likely influences all subsequent face processing.

Abnormal use of facial information in high-functioning autism.

Altered visual exploration of faces likely contributes to social cognition deficits seen in autism. To investigate the relationship between face gaze and social cognition in autism, we measured both face gaze and how facial regions were actually used during emotion judgments from faces. Compared to IQ-matched healthy controls, nine high-functioning adults with autism failed to make use of information from the eye region of faces, instead relying primarily on information from the mouth. Face gaze accounted for the increased reliance on the mouth, and partially accounted for the deficit in using information from the eyes. These findings provide a novel quantitative assessment of how people with autism utilize information in faces when making social judgments.

Role of the amygdala in processing visual social stimuli.

We review the evidence implicating the amygdala as a critical component of a neural network of social cognition, drawing especially on research involving the processing of faces and other visual social stimuli. We argue that, although it is clear that social behavioral representations are not stored in the amygdala, the most parsimonious interpretation of the data is that the amygdala plays a role in guiding social behaviors on the basis of socioenvironmental context. Thus, it appears to be required for normal social cognition. We propose that the amygdala plays this role by attentionally modulating several areas of visual and somatosensory cortex that have been implicated in social cognition, and in helping to direct overt visuospatial attention in face gaze. We also hypothesize that the amygdala exerts attentional modulation of simulation in somatosensory cortices such as supramarginal gyrus and insula. Finally, we argue that the term emotion be broadened to include increased attention to bodily responses and their representation in cortex.

Emotional arousal in agenesis of the corpus callosum.

While the processing of verbal and psychophysiological indices of emotional arousal have been investigated extensively in relation to the left and right cerebral hemispheres, it remains poorly understood how both hemispheres normally function together to generate emotional responses to stimuli. Drawing on a unique sample of nine high-functioning subjects with complete agenesis of the corpus callosum (AgCC), we investigated this issue using standardized emotional visual stimuli. Compared to healthy controls, subjects with AgCC showed a larger variance in their cognitive ratings of valence and arousal, and an insensitivity to the emotion category of the stimuli, especially for negatively-valenced stimuli, and especially for their arousal. Despite their impaired cognitive ratings of arousal, some subjects with AgCC showed large skin-conductance responses, and in general skin-conductance responses discriminated emotion categories and correlated with stimulus arousal ratings. We suggest that largely intact right hemisphere mechanisms can support psychophysiological emotional responses, but that the lack of interhemispheric communication between the hemispheres, perhaps together with dysfunction of the anterior cingulate cortex, interferes with normal verbal ratings of arousal, a mechanism in line with some models of alexithymia.

Looking you in the mouth: abnormal gaze in autism resulting from impaired top-down modulation of visual attention.

People with autism are impaired in their social behavior, including their eye contact with others, but the processes that underlie this impairment remain elusive. We combined high-resolution eye tracking with computational modeling in a group of 10 high-functioning individuals with autism to address this issue. The group fixated the location of the mouth in facial expressions more than did matched controls, even when the mouth was not shown, even in faces that were inverted and most noticeably at latencies of 200-400 ms. Comparisons with a computational model of visual saliency argue that the abnormal bias for fixating the mouth in autism is not driven by an exaggerated sensitivity to the bottom-up saliency of the features, but rather by an abnormal top-down strategy for allocating visual attention.

Frequency-specific interaural level difference tuning predicts spatial response patterns of space-specific neurons in the barn owl inferior colliculus.

Space-specific neurons in the barn owl's inferior colliculus have spatial receptive fields (RFs) because of sensitivity to interaural time difference and frequency-specific interaural level difference (ILD). These neurons are assumed to be tuned to the frequency-specific ILDs occurring at their spatial RFs, but attempts to assess this tuning with traditional narrowband stimuli have had limited success. Indeed, tuning assessed in this manner, when processed via a linear model of spectral integration, typically explains only approximately half the variance in spatial response patterns. Here we report our findings that frequency-specific ILD tuning of space-specific neurons, when assessed from responses to broadband stimuli, predicted nearly 75% of the variance in spatial responses, using a linear model of spectral integration (p < 0.0001; n = 97 neurons). Furthermore, when we tested neurons using only those frequencies we found to be spatially relevant, we saw that their responses were similar to those elicited by broadband stimuli. When we used frequencies not identified as spatially relevant, such similarity was lacking. Furthermore, spectral components that elicited high firing rates when presented as narrowband stimuli were found in several cases to be irrelevant for or detrimental to the definition of spatial RFs. Thus, neurons achieved sharp spatial tuning by selecting for ILDs of a subset of spectral components in noise, some of which were not identified using narrowband stimuli.