A shared structure for emotion experiences from narratives, videos, and everyday life.

Our knowledge of the diversity and psychological organization of emotion experiences is based primarily on studies that used a single type of stimulus with an often limited set of rating scales and analyses. Here we take a comprehensive data-driven approach. We surveyed 1,000+ participants on a diverse set of ratings of emotion experiences to a validated set of ca. 150 text narratives, a validated set of ca. 1,000 videos, and over 10,000 personal experiences sampled longitudinally in everyday life, permitting a unique comparison. All three types of emotion experiences were characterized by similar dimensional spaces that included valence and arousal, as well as dimensions related to generalizability. Emotion experiences were distributed along continuous gradients, with no clear clusters even for the so-called basic emotions. Individual differences in personality traits were associated with differences in everyday emotion experiences but not with emotions evoked by narratives or videos.

Resting-State Functional Connectivity of the Amygdala in Autism: A Preregistered Large-Scale Study.

OBJECTIVE: Three leading neurobiological hypotheses about autism spectrum disorder (ASD) propose underconnectivity between brain regions, atypical function of the amygdala, and generally higher variability between individuals with ASD than between neurotypical individuals. Past work has often failed to generalize, because of small sample sizes, unquantified data quality, and analytic flexibility. This study addressed these limitations while testing the above three hypotheses, applied to amygdala functional connectivity. METHODS: In a comprehensive preregistered study, the three hypotheses were tested in a subset (N=488 after exclusions; N=212 with ASD) of the Autism Brain Imaging Data Exchange data sets. The authors analyzed resting-state functional connectivity (FC) from functional MRI data from two anatomically defined amygdala subdivisions, in three hypotheses with respect to magnitude, pattern similarity, and variability, across different anatomical scales ranging from whole brain to specific regions and networks. RESULTS: A Bayesian approach to hypothesis evaluation produced inconsistent evidence in ASD for atypical amygdala FC magnitude, strong evidence that the multivariate pattern of FC was typical, and no consistent evidence of increased interindividual variability in FC. The results strongly depended on analytic choices, including preprocessing pipeline for the neuroimaging data, anatomical specificity, and subject exclusions. CONCLUSIONS: A preregistered set of analyses found no reliable evidence for atypical functional connectivity of the amygdala in autism, contrary to leading hypotheses. Future studies should test an expanded set of hypotheses across multiple processing pipelines, collect deeper data per individual, and include a greater diversity of participants to ensure robust generalizability of findings on amygdala FC in ASD.

Neural dynamics underlying minute-timescale persistent behavior in the human brain.

The ability to pursue long-term goals relies on a representations of task context that can both be maintained over long periods of time and switched flexibly when goals change. Little is known about the neural substrate for such minute-scale maintenance of task sets. Utilizing recordings in neurosurgical patients, we examined how groups of neurons in the human medial frontal cortex and hippocampus represent task contexts. When cued explicitly, task context was encoded in both brain areas and changed rapidly at task boundaries. Hippocampus exhibited a temporally dynamic code with fast decorrelation over time, preventing cross-temporal generalization. Medial frontal cortex exhibited a static code that decorrelated slowly, allowing generalization across minutes of time. When task context needed to be inferred as a latent variable, hippocampus encoded task context with a static code. These findings reveal two possible regimes for encoding minute-scale task-context representations that were engaged differently based on task demands.

Smartphone-based gaze estimation for in-home autism research.

Atypical gaze patterns are a promising biomarker of autism spectrum disorder. To measure gaze accurately, however, it typically requires highly controlled studies in the laboratory using specialized equipment that is often expensive, thereby limiting the scalability of these approaches. Here we test whether a recently developed smartphone-based gaze estimation method could overcome such limitations and take advantage of the ubiquity of smartphones. As a proof-of-principle, we measured gaze while a small sample of well-assessed autistic participants and controls watched videos on a smartphone, both in the laboratory (with lab personnel) and in remote home settings (alone). We demonstrate that gaze data can be efficiently collected, in-home and longitudinally by participants themselves, with sufficiently high accuracy (gaze estimation error below 1° visual angle on average) for quantitative, feature-based analysis. Using this approach, we show that autistic individuals have reduced gaze time on human faces and longer gaze time on non-social features in the background, thereby reproducing established findings in autism using just smartphones and no additional hardware. Our approach provides a foundation for scaling future research with larger and more representative participant groups at vastly reduced cost, also enabling better inclusion of underserved communities.

Commentary on, "The Human Affectome," by Schiller et al.

I suggest that this project could benefit from a relational database of some sort to provide readers with a more formal ontology, and that the authors consider making a distinction between experiential and functional aspects of emotion.

Multimodal single-neuron, intracranial EEG, and fMRI brain responses during movie watching in human patients.

We present a multimodal dataset of intracranial recordings, fMRI, and eye tracking in 20 participants during movie watching. Recordings consist of single neurons, local field potential, and intracranial EEG activity acquired from depth electrodes targeting the amygdala, hippocampus, and medial frontal cortex implanted for monitoring of epileptic seizures. Participants watched an 8-min long excerpt from the video "Bang! You're Dead" and performed a recognition memory test for movie content. 3 T fMRI activity was recorded prior to surgery in 11 of these participants while performing the same task. This NWB- and BIDS-formatted dataset includes spike times, field potential activity, behavior, eye tracking, electrode locations, demographics, and functional and structural MRI scans. For technical validation, we provide signal quality metrics, assess eye tracking quality, behavior, the tuning of cells and high-frequency broadband power field potentials to familiarity and event boundaries, and show brain-wide inter-subject correlations for fMRI. This dataset will facilitate the investigation of brain activity during movie watching, recognition memory, and the neural basis of the fMRI-BOLD signal.

Opinion: Which animals have personality?

Human personality generally refers to coherent individuating patterns in affect, behavior, and cognition. We can only observe and measure behavior, from which we then infer personality and other psychological processes (affect, cognition, etc.). We emphasize that the study of personality always explains or summarizes patterns not only in behavior but also in these other psychological processes inferred from behavior. We thus argue that personality should be attributed only to nonhuman animals with behaviors from which we can infer a sufficiently rich set of psychological processes. The mere inference of a biological trait that explains behavioral variability, on our view, is not sufficient to count as a personality construct and should be given a different term. Methodologically, inferring personality in nonhuman animals entails challenges in characterizing ecologically valid behaviors, doing so across rich and varied environments, and collecting enough data. We suggest that studies should gradually accumulate such corpora of data on a species through well-curated shared databases. A mixture of approaches should include both top-down fit with extant human personality theories (such as the Big Five) as well as bottom-up discovery of species-specific personality dimensions. Adopting the above framework will help us to build a comparative psychology and will provide the most informative models also for understanding human personality, its evolution, and its disorders.

The Future of Affective Science: Introduction to the Special Issue.

Modern affective science-the empirical study of emotional responding and affective experience-has been active for a half-century. The Future of Affective Science special issue considers the history of this field and proposes new directions for the decades ahead. Contributors represent diverse theoretical perspectives, methodological expertise, and domains of study, and the special issue includes both literature reviews and new empirical studies as illustrations. This introductory article synthesizes the contributions, articulating the broader context of the current status of our field and highlighting common themes across articles as well as gaps notable even in this special issue. Sections of the article address theoretical and conceptual issues, research methodology, the questions we ask, and translation of basic affective science to applied domains. We conclude that much has been learned from the first 50 years of affective science, and it is now time for new theories, new research questions, and innovative methods for the decades ahead.

Collective events and individual affect shape autobiographical memory.

How do collective events shape how we remember our lives? We leveraged advances in natural language processing as well as a rich, longitudinal assessment of 1,000 Americans throughout 2020 to examine how memory is influenced by two prominent factors: surprise and emotion. Autobiographical memory for 2020 displayed a unique signature: There was a substantial bump in March, aligning with pandemic onset and lockdowns, consistent across three memory collections 1 y apart. We further investigated how emotion, using both immediate and retrieved measures, predicted the amount and content of autobiographical memory: Negative affect increased recall across all measures, whereas its more clinical indices, depression and posttraumatic stress disorder, selectively increased nonepisodic recall. Finally, in a separate cohort, we found pandemic news to be better remembered, surprising, and negative, while lockdowns compressed remembered time. Our work connects laboratory findings to the real world and delineates the effects of acute versus clinical signatures of negative emotion on memory.

Neural signatures of social inferences predict the number of real-life social contacts and autism severity.

We regularly infer other people's thoughts and feelings from observing their actions, but how this ability contributes to successful social behavior and interactions remains unknown. We show that neural activation patterns during social inferences obtained in the laboratory predict the number of social contacts in the real world, as measured by the social network index, in three neurotypical samples (total n = 126) and one sample of autistic adults (n = 23). We also show that brain patterns during social inference generalize across individuals in these groups. Cross-validated associations between brain activations and social inference localize selectively to the right posterior superior temporal sulcus and were specific for social, but not nonsocial, inference. Activation within this same brain region also predicts autism-like trait scores from questionnaires and autism symptom severity. Thus, neural activations produced while thinking about other people's mental states predict variance in multiple indices of social functioning in the real world.

Trait impressions from faces depend on the goals of the perceiver.

Trait impressions from faces formed in the real world likely depend on the circumstances in which a face is seen, in particular, on the goal of the perceiver in that circumstance. This goal dependency is typically not incorporated into laboratory studies, an omission that has limited our understanding of trait impressions from faces.

Dissociable brain structural asymmetry patterns reveal unique phenome-wide profiles.

Broca reported ~150 years ago that particular lesions of the left hemisphere impair speech. Since then, other brain regions have been reported to show lateralized structure and function. Yet, studies of brain asymmetry have limited their focus to pairwise comparisons between homologous regions. Here, we characterized separable whole-brain asymmetry patterns in grey and white matter structure from n = 37,441 UK Biobank participants. By pooling information on left-right shifts underlying whole-brain structure, we deconvolved signatures of brain asymmetry that are spatially distributed rather than locally constrained. Classically asymmetric regions turned out to belong to more than one asymmetry pattern. Instead of a single dominant signature, we discovered complementary asymmetry patterns that contributed similarly to whole-brain asymmetry at the population level. These asymmetry patterns were associated with unique collections of phenotypes, ranging from early lifestyle factors to demographic status to mental health indicators.

Atypical gaze patterns in autistic adults are heterogeneous across but reliable within individuals.

BACKGROUND: Across behavioral studies, autistic individuals show greater variability than typically developing individuals. However, it remains unknown to what extent this variability arises from heterogeneity across individuals, or from unreliability within individuals. Here, we focus on eye tracking, which provides rich dependent measures that have been used extensively in studies of autism. Autistic individuals have an atypical gaze onto both static visual images and dynamic videos that could be leveraged for diagnostic purposes if the above open question could be addressed. METHODS: We tested three competing hypotheses: (1) that gaze patterns of autistic individuals are less reliable or noisier than those of controls, (2) that atypical gaze patterns are individually reliable but heterogeneous across autistic individuals, or (3) that atypical gaze patterns are individually reliable and also homogeneous among autistic individuals. We collected desktop-based eye tracking data from two different full-length television sitcom episodes, at two independent sites (Caltech and Indiana University), in a total of over 150 adult participants (N = 48 autistic individuals with IQ in the normal range, 105 controls) and quantified gaze onto features of the videos using automated computer vision-based feature extraction. RESULTS: We found support for the second of these hypotheses. Autistic people and controls showed equivalently reliable gaze onto specific features of videos, such as faces, so much so that individuals could be identified significantly above chance using a fingerprinting approach from video epochs as short as 2 min. However, classification of participants into diagnostic groups based on their eye tracking data failed to produce clear group classifications, due to heterogeneity in the autistic group. LIMITATIONS: Three limitations are the relatively small sample size, assessment across only two videos (from the same television series), and the absence of other dependent measures (e.g., neuroimaging or genetics) that might have revealed individual-level variability that was not evident with eye tracking. Future studies should expand to larger samples across longer longitudinal epochs, an aim that is now becoming feasible with Internet- and phone-based eye tracking. CONCLUSIONS: These findings pave the way for the investigation of autism subtypes, and for elucidating the specific visual features that best discriminate gaze patterns-directions that will also combine with and inform neuroimaging and genetic studies of this complex disorder.

Mapping effective connectivity of human amygdala subdivisions with intracranial stimulation.

The primate amygdala is a complex consisting of over a dozen nuclei that have been implicated in a host of cognitive functions, individual differences, and psychiatric illnesses. These functions are implemented through distinct connectivity profiles, which have been documented in animals but remain largely unknown in humans. Here we present results from 25 neurosurgical patients who had concurrent electrical stimulation of the amygdala with intracranial electroencephalography (electrical stimulation tract-tracing; es-TT), or fMRI (electrical stimulation fMRI; es-fMRI), methods providing strong inferences about effective connectivity of amygdala subdivisions with the rest of the brain. We quantified functional connectivity with medial and lateral amygdala, the temporal order of these connections on the timescale of milliseconds, and also detail second-order effective connectivity among the key nodes. These findings provide a uniquely detailed characterization of human amygdala functional connectivity that will inform functional neuroimaging studies in healthy and clinical populations.

The geometry of domain-general performance monitoring in the human medial frontal cortex.

Controlling behavior to flexibly achieve desired goals depends on the ability to monitor one's own performance. It is unknown how performance monitoring can be both flexible, to support different tasks, and specialized, to perform each task well. We recorded single neurons in the human medial frontal cortex while subjects performed two tasks that involve three types of cognitive conflict. Neurons encoding conflict probability, conflict, and error in one or both tasks were intermixed, forming a representational geometry that simultaneously allowed task specialization and generalization. Neurons encoding conflict retrospectively served to update internal estimates of conflict probability. Population representations of conflict were compositional. These findings reveal how representations of evaluative signals can be both abstract and task-specific and suggest a neuronal mechanism for estimating control demand.

Caltech Conte Center, a multimodal data resource for exploring social cognition and decision-making.

This data release of 117 healthy community-dwelling adults provides multimodal high-quality neuroimaging and behavioral data for the investigation of brain-behavior relationships. We provide structural MRI, resting-state functional MRI, movie functional MRI, together with questionnaire-based and task-based psychological variables; many of the participants have multiple datasets from retesting over the course of several years. Our dataset is distinguished by utilizing open-source data formats and processing tools (BIDS, FreeSurfer, fMRIPrep, MRIQC), providing data that is thoroughly quality checked, preprocessed to various extents and available in multiple anatomical spaces. A customizable denoising pipeline is provided as open-source code that includes tools for the generation of functional connectivity matrices and initialization of individual difference analyses. Behavioral data include a comprehensive set of psychological assessments on gold-standard instruments encompassing cognitive function, mood and personality, together with exploratory factor analyses. The dataset provides an in-depth, multimodal resource for investigating associations between individual differences, brain structure and function, with a focus on the domains of social cognition and decision-making.

Video-evoked fMRI BOLD responses are highly consistent across different data acquisition sites.

Naturalistic imaging paradigms, in which participants view complex videos in the scanner, are increasingly used in human cognitive neuroscience. Videos evoke temporally synchronized brain responses that are similar across subjects as well as within subjects, but the reproducibility of these brain responses across different data acquisition sites has not yet been quantified. Here, we characterize the consistency of brain responses across independent samples of participants viewing the same videos in functional magnetic resonance imaging (fMRI) scanners at different sites (Indiana University and Caltech). We compared brain responses collected at these different sites for two carefully matched datasets with identical scanner models, acquisition, and preprocessing details, along with a third unmatched dataset in which these details varied. Our overall conclusion is that for matched and unmatched datasets alike, video-evoked brain responses have high consistency across these different sites, both when compared across groups and across pairs of individuals. As one might expect, differences between sites were larger for unmatched datasets than matched datasets. Residual differences between datasets could in part reflect participant-level variability rather than scanner- or data- related effects. Altogether our results indicate promise for the development and, critically, generalization of video fMRI studies of individual differences in healthy and clinical populations alike.

Ethical commitments, principles, and practices guiding intracranial neuroscientific research in humans.

Leveraging firsthand experience, BRAIN-funded investigators conducting intracranial human neuroscience research propose two fundamental ethical commitments: (1) maintaining the integrity of clinical care and (2) ensuring voluntariness. Principles, practices, and uncertainties related to these commitments are offered for future investigation.

A Cautionary Note on Predicting Social Judgments from Faces with Deep Neural Networks.

People spontaneously infer other people's psychology from faces, encompassing inferences of their affective states, cognitive states, and stable traits such as personality. These judgments are known to be often invalid, but nonetheless bias many social decisions. Their importance and ubiquity have made them popular targets for automated prediction using deep convolutional neural networks (DCNNs). Here, we investigated the applicability of this approach: how well does it generalize, and what biases does it introduce? We compared three distinct sets of features (from a face identification DCNN, an object recognition DCNN, and using facial geometry), and tested their prediction across multiple out-of-sample datasets. Across judgments and datasets, features from both pre-trained DCNNs provided better predictions than did facial geometry. However, predictions using object recognition DCNN features were not robust to superficial cues (e.g., color and hair style). Importantly, predictions using face identification DCNN features were not specific: models trained to predict one social judgment (e.g., trustworthiness) also significantly predicted other social judgments (e.g., femininity and criminal), and at an even higher accuracy in some cases than predicting the judgment of interest (e.g., trustworthiness). Models trained to predict affective states (e.g., happy) also significantly predicted judgments of stable traits (e.g., sociable), and vice versa. Our analysis pipeline not only provides a flexible and efficient framework for predicting affective and social judgments from faces but also highlights the dangers of such automated predictions: correlated but unintended judgments can drive the predictions of the intended judgments. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s42761-021-00075-5.