The representational structure of mental states generalizes across target people and stimulus modalities.

Each individual experiences mental states in their own idiosyncratic way, yet perceivers can accurately understand a huge variety of states across unique individuals. How do they accomplish this feat? Do people think about their own anger in the same ways as another person's anger? Is reading about someone's anxiety the same as seeing it? Here, we test the hypothesis that a common conceptual core unites mental state representations across contexts. Across three studies, participants judged the mental states of multiple targets, including a generic other, the self, a socially close other, and a socially distant other. Participants viewed mental state stimuli in multiple modalities, including written scenarios and images. Using representational similarity analysis, we found that brain regions associated with social cognition expressed stable neural representations of mental states across both targets and modalities. Together, these results suggest that people use stable models of mental states across different people and contexts.

The Social Brain Automatically Predicts Others' Future Mental States.

Social life requires people to predict the future: people must anticipate others' thoughts, feelings, and actions to interact with them successfully. The theory of predictive coding suggests that the social brain may meet this need by automatically predicting others' social futures. If so, when representing others' current mental state, the brain should already start representing their future states. To test this hypothesis, we used fMRI to measure female and male human participants' neural representations of mental states. Representational similarity analysis revealed that neural patterns associated with mental states currently under consideration resembled patterns of likely future states more so than patterns of unlikely future states. This effect manifested in activity across the social brain network and in medial prefrontal cortex in particular. Repetition suppression analysis also supported the social predictive coding hypothesis: considering mental states presented in predictable sequences reduced activity in the precuneus relative to unpredictable sequences. In addition to demonstrating that the brain makes automatic predictions of others' social futures, the results also demonstrate that the brain leverages a 3D representational space to make these predictions. Proximity between mental states on the psychological dimensions of rationality, social impact, and valence explained much of the association between state-specific neural pattern similarity and state transition likelihood. Together, these findings suggest that the way the brain represents the social present gives people an automatic glimpse of the social future.SIGNIFICANCE STATEMENT When you see a ball in flight, your brain calculates, not just its static visual features such as size and shape, but also predicts its future trajectory. Here, we investigated whether the same might hold true in the social world: when we see someone flying into a rage, does our brain automatically predict their social trajectory? In this study, we scanned participants' brain activity while they judged others' mental states. We found that neural activity associated with a given state resembled activity associated with likely future states. Additionally, unpredictable sequences of states evoked more brain activity than predictable sequences, consistent with monitoring for, and updating from, prediction errors. These results suggest that the social brain automatically predicts others' future mental states.

Mental models accurately predict emotion transitions.

Successful social interactions depend on people's ability to predict others' future actions and emotions. People possess many mechanisms for perceiving others' current emotional states, but how might they use this information to predict others' future states? We hypothesized that people might capitalize on an overlooked aspect of affective experience: current emotions predict future emotions. By attending to regularities in emotion transitions, perceivers might develop accurate mental models of others' emotional dynamics. People could then use these mental models of emotion transitions to predict others' future emotions from currently observable emotions. To test this hypothesis, studies 1-3 used data from three extant experience-sampling datasets to establish the actual rates of emotional transitions. We then collected three parallel datasets in which participants rated the transition likelihoods between the same set of emotions. Participants' ratings of emotion transitions predicted others' experienced transitional likelihoods with high accuracy. Study 4 demonstrated that four conceptual dimensions of mental state representation-valence, social impact, rationality, and human mind-inform participants' mental models. Study 5 used 2 million emotion reports on the Experience Project to replicate both of these findings: again people reported accurate models of emotion transitions, and these models were informed by the same four conceptual dimensions. Importantly, neither these conceptual dimensions nor holistic similarity could fully explain participants' accuracy, suggesting that their mental models contain accurate information about emotion dynamics above and beyond what might be predicted by static emotion knowledge alone.

Theories of Person Perception Predict Patterns of Neural Activity During Mentalizing.

Social life requires making inferences about other people. What information do perceivers spontaneously draw upon to make such inferences? Here, we test 4 major theories of person perception, and 1 synthetic theory that combines their features, to determine whether the dimensions of such theories can serve as bases for describing patterns of neural activity during mentalizing. While undergoing functional magnetic resonance imaging, participants made social judgments about well-known public figures. Patterns of brain activity were then predicted using feature encoding models that represented target people's positions on theoretical dimensions such as warmth and competence. All 5 theories of person perception proved highly accurate at reconstructing activity patterns, indicating that each could describe the informational basis of mentalizing. Cross-validation indicated that the theories robustly generalized across both targets and participants. The synthetic theory consistently attained the best performance-approximately two-thirds of noise ceiling accuracy--indicating that, in combination, the theories considered here can account for much of the neural representation of other people. Moreover, encoding models trained on the present data could reconstruct patterns of activity associated with mental state representations in independent data, suggesting the use of a common neural code to represent others' traits and states.

Neural evidence that three dimensions organize mental state representation: Rationality, social impact, and valence.

How do people understand the minds of others? Existing psychological theories have suggested a number of dimensions that perceivers could use to make sense of others' internal mental states. However, it remains unclear which of these dimensions, if any, the brain spontaneously uses when we think about others. The present study used multivoxel pattern analysis (MVPA) of neuroimaging data to identify the primary organizing principles of social cognition. We derived four unique dimensions of mental state representation from existing psychological theories and used functional magnetic resonance imaging to test whether these dimensions organize the neural encoding of others' mental states. MVPA revealed that three such dimensions could predict neural patterns within the medial prefrontal and parietal cortices, temporoparietal junction, and anterior temporal lobes during social thought: rationality, social impact, and valence. These results suggest that these dimensions serve as organizing principles for our understanding of other people.

Consistent Neural Activity Patterns Represent Personally Familiar People.

How does the brain encode and organize our understanding of the people we know? In this study, participants imagined personally familiar others in a variety of contexts while undergoing fMRI. Using multivoxel pattern analysis, we demonstrated that thinking about familiar others elicits consistent fine-grained patterns of neural activity. Person-specific patterns were distributed across many regions previously associated with social cognition, including medial prefrontal, medial parietal, and lateral temporoparietal cortices, as well as other regions including the anterior and mid-cingulate, insula, and precentral gyrus. Analogous context-specific patterns were observed in medial parietal and superior occipital regions. These results suggest that medial parietal cortex may play a particularly central role in simulating familiar others, as this is the only region to simultaneously represent both person and context information. Moreover, within portions of medial parietal cortex, the degree to which person-specific patterns were typically instated on a given trial predicted subsequent judgments of accuracy and vividness in the mental simulation. This suggests that people may access neural representations in this region to form metacognitive judgments of confidence in their mental simulations. In addition to fine-grained patterns within brain regions, we also observed encoding of both familiar people and contexts in coarse-grained patterns spread across the independently defined social brain network. Finally, we found tentative evidence that several established theories of person perception might explain the relative similarity between person-specific patterns within the social brain network.

Discrimination of wine lactic acid bacteria by Raman spectroscopy.

Species of Lactobacillus, Pediococcus, Oenococcus, and Leuconostoc play an important role in winemaking, as either inoculants or contaminants. The metabolic products of these lactic acid bacteria have considerable effects on the flavor, aroma, and texture of a wine. However, analysis of a wine's microflora, especially the bacteria, is rarely done unless spoilage becomes evident, and identification at the species or strain level is uncommon as the methods required are technically difficult and expensive. In this work, we used Raman spectral fingerprints to discriminate 19 strains of Lactobacillus, Pediococcus, and Oenococcus. Species of Lactobacillus and Pediococcus and strains of O. oeni and P. damnosus were classified with high sensitivity: 86-90 and 84-85%, respectively. Our results demonstrate that a simple, inexpensive method utilizing Raman spectroscopy can be used to accurately identify lactic acid bacteria isolated from wine.

Neural representations of situations and mental states are composed of sums of representations of the actions they afford.

Human behavior depends on both internal and external factors. Internally, people's mental states motivate and govern their behavior. Externally, one's situation constrains which actions are appropriate or possible. To predict others' behavior, one must understand the influences of mental states and situations on actions. On this basis, we hypothesize that people represent situations and states in terms of associated actions. To test this, we use functional neuroimaging to estimate neural activity patterns associated with situations, mental states, and actions. We compute sums of the action patterns, weighted by how often each action occurs in each situation and state. We find that these summed action patterns reconstructed the corresponding situation and state patterns. These results suggest that neural representations of situations and mental states are composed of sums of their action affordances. Summed action representations thus offer a biological mechanism by which people can predict actions given internal and external factors.

Deep social neuroscience: the promise and peril of using artificial neural networks to study the social brain.

This review offers an accessible primer to social neuroscientists interested in neural networks. It begins by providing an overview of key concepts in deep learning. It then discusses three ways neural networks can be useful to social neuroscientists: (i) building statistical models to predict behavior from brain activity; (ii) quantifying naturalistic stimuli and social interactions; and (iii) generating cognitive models of social brain function. These applications have the potential to enhance the clinical value of neuroimaging and improve the generalizability of social neuroscience research. We also discuss the significant practical challenges, theoretical limitations and ethical issues faced by deep learning. If the field can successfully navigate these hazards, we believe that artificial neural networks may prove indispensable for the next stage of the field's development: deep social neuroscience.

The Emerging Science of Interacting Minds.

For over a century, psychology has focused on uncovering mental processes of a single individual. However, humans rarely navigate the world in isolation. The most important determinants of successful development, mental health, and our individual traits and preferences arise from interacting with other individuals. Social interaction underpins who we are, how we think, and how we behave. Here we discuss the key methodological challenges that have limited progress in establishing a robust science of how minds interact and the new tools that are beginning to overcome these challenges. A deep understanding of the human mind requires studying the context within which it originates and exists: social interaction.

Individual differences in emotion prediction and implications for social success.

The social world requires people to predict others' thoughts, feelings, and actions. People who successfully predict others' emotions experience significant social advantages. What makes a person good at predicting emotions? To predict others' future emotional states, a person must know how one emotion transitions to the next. People learn how emotions transition from at least two sources: (a) internal information, or one's own emotion experiences, and (b) external information, such as the social cues detected in a person's face. Across five studies collected between 2018 and 2020, we find evidence that both sources of information are related to accurate emotion prediction: individuals with atypical personal emotion transitions, difficulty understanding their own emotional experiences, and impaired emotion perception displayed impaired emotion prediction. This ability to predict others' emotions has real-world social implications. Individuals who make accurate emotion predictions have better relationships with their friends and communities and experience less loneliness. In contrast, disruptions in both internal and external information sources explain prediction inaccuracy in individuals with social difficulties, specifically with social communication difficulties common in autism spectrum disorder. These findings provide evidence that successful emotion prediction, which relies on the perception of accurate internal and external data about how emotions transition, may be key to social success. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Disentangling the effects of similarity, familiarity, and liking on social inference strategies.

People constantly make inferences about others' beliefs and preferences. People can draw on various sources of information to make these inferences, including stereotypes, self-knowledge, and target-specific knowledge. What leads people to use each of these sources of information over others? The current study examined factors that influence the use of these sources of information, focusing on three interpersonal dimensions - the extent to which people feel (a) familiar with, (b) similar to, or (c) like the target. In four studies (total N = 1136), participants inferred the beliefs and preferences of others - celebrities (Studies 1a-1b), constructed fictional targets (Study 2), and actual acquaintances (Study 3). Participants also rated familiarity with, similarity to, and liking of the target. Analyses assessed the use of each source of information by comparing inferences with information provided by those sources. Familiarity was associated with greater use of target-specific knowledge, while similarity and liking were associated with self-knowledge. Low similarity and high liking were associated with increased use of stereotypes. We discuss the implication of these findings and their applicability to unique cases, including inferences about celebrities, public figures, and positively stereotyped groups, in which familiarity, similarity, and liking do not perfectly align.

Working memory for social information: chunking or domain-specific buffer?

Humans possess unique social abilities that set us apart from other species. These abilities may be partially supported by a large capacity for maintaining and manipulating social information. Efficient social working memory might arise from two different sources: chunking of social information or a domain-specific buffer. We test these hypotheses with functional magnetic resonance imaging (fMRI) by manipulating sociality and working memory load in an n-back paradigm. We observe (i) an effect of load in the frontoparietal control network, (ii) an effect of sociality in regions associated with social cognition and face processing, and (iii) an interaction within the frontoparietal network such that social load has a smaller effect than nonsocial load. These results support the hypothesis that working memory is more efficient for social information than for nonsocial information, and suggest that chunking, rather than a domain-specific buffer, is the mechanism of this greater efficiency.

Raman spectroscopy and chemometrics for identification and strain discrimination of the wine spoilage yeasts Saccharomyces cerevisiae, Zygosaccharomyces bailii, and Brettanomyces bruxellensis.

The yeasts Zygosaccharomyces bailii, Dekkera bruxellensis (anamorph, Brettanomyces bruxellensis), and Saccharomyces cerevisiae are the major spoilage agents of finished wine. A novel method using Raman spectroscopy in combination with a chemometric classification tool has been developed for the identification of these yeast species and for strain discrimination of these yeasts. Raman spectra were collected for six strains of each of the yeasts Z. bailii, B. bruxellensis, and S. cerevisiae. The yeasts were classified with high sensitivity at the species level: 93.8% for Z. bailii, 92.3% for B. bruxellensis, and 98.6% for S. cerevisiae. Furthermore, we have demonstrated that it is possible to discriminate between strains of these species. These yeasts were classified at the strain level with an overall accuracy of 81.8%.

Transition dynamics shape mental state concepts.

People have a unique ability to represent other people's internal thoughts and feelings-their mental states. Mental state knowledge has a rich conceptual structure, organized along key dimensions, such as valence. People use this conceptual structure to guide social interactions. How do people acquire their understanding of this structure? Here we investigate an underexplored contributor to this process: observation of mental state dynamics. Mental states-including both emotions and cognitive states-are not static. Rather, the transitions from one state to another are systematic and predictable. Drawing on prior cognitive science, we hypothesize that these transition dynamics may shape the conceptual structure that people learn to apply to mental states. Across nine behavioral experiments (N = 1,439), we tested whether the transition probabilities between mental states causally shape people's conceptual judgments of those states. In each study, we found that observing frequent transitions between mental states caused people to judge them to be conceptually similar. Computational modeling indicated that people translated mental state dynamics into concepts by embedding the states as points within a geometric space. The closer two states are within this space, the greater the likelihood of transitions between them. In three neural network experiments, we trained artificial neural networks to predict real human mental state dynamics. The networks spontaneously learned the same conceptual dimensions that people use to understand mental states. Together these results indicate that mental state dynamics-and the goal of predicting them-shape the structure of mental state concepts. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Advancing Naturalistic Affective Science with Deep Learning.

People express their own emotions and perceive others' emotions via a variety of channels, including facial movements, body gestures, vocal prosody, and language. Studying these channels of affective behavior offers insight into both the experience and perception of emotion. Prior research has predominantly focused on studying individual channels of affective behavior in isolation using tightly controlled, non-naturalistic experiments. This approach limits our understanding of emotion in more naturalistic contexts where different channels of information tend to interact. Traditional methods struggle to address this limitation: manually annotating behavior is time-consuming, making it infeasible to do at large scale; manually selecting and manipulating stimuli based on hypotheses may neglect unanticipated features, potentially generating biased conclusions; and common linear modeling approaches cannot fully capture the complex, nonlinear, and interactive nature of real-life affective processes. In this methodology review, we describe how deep learning can be applied to address these challenges to advance a more naturalistic affective science. First, we describe current practices in affective research and explain why existing methods face challenges in revealing a more naturalistic understanding of emotion. Second, we introduce deep learning approaches and explain how they can be applied to tackle three main challenges: quantifying naturalistic behaviors, selecting and manipulating naturalistic stimuli, and modeling naturalistic affective processes. Finally, we describe the limitations of these deep learning methods, and how these limitations might be avoided or mitigated. By detailing the promise and the peril of deep learning, this review aims to pave the way for a more naturalistic affective science.

Six dimensions describe action understanding: The ACT-FASTaxonomy.

Humans engage in a wide variety of different actions and activities. These range from simple motor actions like reaching for an object, to complex activities like governing a nation. Navigating everyday life requires people to make sense of this diversity of actions. We suggest that the mind simplifies this complex domain by attending primarily to the most essential features of actions. Using a parsimonious set of action dimensions, the mind can organize action knowledge in a low-dimensional representational space. In seven studies, we derive and validate such an action taxonomy. Study 1 uses large-scale text analyses to generate and test potential action dimensions. Study 2 validates interpretable labels for these dimensions. Studies 3-5 demonstrate that these dimensions can explain human judgments about actions. We perform model selection on data from these studies to arrive at the optimal set of six psychological dimensions, together forming the Abstraction, Creation, Tradition, Food, Animacy, Spiritualism Taxonomy (ACT-FAST). Study 6 demonstrates that ACT-FAST can predict socially relevant qualities of actions, including how, when, where, why, and by whom they are performed. Finally, Study 7 shows that ACT-FAST can explain action-related patterns of brain activity using naturalistic functional MRI (MRI). Together, these studies reveal the dimensional structure the mind applies to organize action concepts. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Accurate emotion prediction in dyads and groups and its potential social benefits.

Emotion dynamics vary considerably from individual to individual and from group to group. Successful social interactions require people to track this moving target in order to anticipate the thoughts, feelings, and actions of others. In two studies, we test whether people track others' emotional idiosyncrasies to make accurate, target-specific emotion predictions. In both studies, participants predicted the emotion transitions of a specific target-either a close friend (Study 1) or a first-year college roommate (Study 2)-as well as an average group member. Results demonstrate that people can make highly accurate predictions both for specific individuals and specific groups. Accurate predictions rely on target-specific knowledge; new community members were able to make accurate predictions at zero-acquaintance, but accuracy increased over time as individuals accrued specialized knowledge. Results also suggest that accurate emotion prediction is associated with social success in both individual and communal relationships and that such a relation might emerge over time. Overall, our studies suggest that people accurately make individualized predictions of others' emotion transitions and that doing so fulfills a meaningful function in the social world. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

The 3d Mind Model characterizes how people understand mental states across modern and historical cultures.

Humans rely on social interaction to achieve many important goals. These interactions rely in turn on people's capacity to understand others' mental states: their thoughts and feelings. Do different cultures understand minds in different ways, or do widely shared principles describe how different cultures understand mental states? Extensive data suggest that the mind organizes mental state concepts using the 3d Mind Model, composed of the psychological dimensions: rationality (vs. emotionality), social impact (states which affect others more vs. less), and valence (positive vs. negative states). However, this evidence comes primarily from English-speaking individuals in the United States. Here we investigated mental state representation in 57 contemporary countries, using 163 million English language tweets; in 17 languages, using billions of words of text from internet webpages; and across more than 2000 years of history, using curated texts from four historical societies. We quantified mental state meaning by analyzing the text produced by each culture using word embeddings. We then tested whether the 3d Mind Model could explain which mental states were similar in meaning within each culture. We found that the 3d Mind Model significantly explained mental state meaning in every country, language, and historical society that we examined. These results suggest that rationality, social impact, and valence form a generalizable conceptual backbone for mental state representation.

Perceiving actions before they happen: psychological dimensions scaffold neural action prediction.

The social world buzzes with action. People constantly walk, talk, eat, work, play, snooze and so on. To interact with others successfully, we need to both understand their current actions and predict their future actions. Here we used functional neuroimaging to test the hypothesis that people do both at the same time: when the brain perceives an action, it simultaneously encodes likely future actions. Specifically, we hypothesized that the brain represents perceived actions using a map that encodes which actions will occur next: the six-dimensional Abstraction, Creation, Tradition, Food(-relevance), Animacy and Spiritualism Taxonomy (ACT-FAST) action space. Within this space, the closer two actions are, the more likely they are to precede or follow each other. To test this hypothesis, participants watched a video featuring naturalistic sequences of actions while undergoing functional magnetic resonance imaging (fMRI) scanning. We first use a decoding model to demonstrate that the brain uses ACT-FAST to represent current actions. We then successfully predicted as-yet unseen actions, up to three actions into the future, based on their proximity to the current action's coordinates in ACT-FAST space. This finding suggests that the brain represents actions using a six-dimensional action space that gives people an automatic glimpse of future actions.