Testing theory of mind in large language models and humans.

At the core of what defines us as humans is the concept of theory of mind: the ability to track other people's mental states. The recent development of large language models (LLMs) such as ChatGPT has led to intense debate about the possibility that these models exhibit behaviour that is indistinguishable from human behaviour in theory of mind tasks. Here we compare human and LLM performance on a comprehensive battery of measurements that aim to measure different theory of mind abilities, from understanding false beliefs to interpreting indirect requests and recognizing irony and faux pas. We tested two families of LLMs (GPT and LLaMA2) repeatedly against these measures and compared their performance with those from a sample of 1,907 human participants. Across the battery of theory of mind tests, we found that GPT-4 models performed at, or even sometimes above, human levels at identifying indirect requests, false beliefs and misdirection, but struggled with detecting faux pas. Faux pas, however, was the only test where LLaMA2 outperformed humans. Follow-up manipulations of the belief likelihood revealed that the superiority of LLaMA2 was illusory, possibly reflecting a bias towards attributing ignorance. By contrast, the poor performance of GPT originated from a hyperconservative approach towards committing to conclusions rather than from a genuine failure of inference. These findings not only demonstrate that LLMs exhibit behaviour that is consistent with the outputs of mentalistic inference in humans but also highlight the importance of systematic testing to ensure a non-superficial comparison between human and artificial intelligences.

Flexible Cultural Learning Through Action Coordination.

The cultural transmission of technical know-how has proven vital to the success of our species. The broad diversity of learning contexts and social configurations, as well as the various kinds of coordinated interactions they involve, speaks to our capacity to flexibly adapt to and succeed in transmitting vital knowledge in various learning contexts. Although often recognized by ethnographers, the flexibility of cultural learning has so far received little attention in terms of cognitive mechanisms. We argue that a key feature of the flexibility of cultural learning is that both the models and learners recruit cognitive mechanisms of action coordination to modulate their behavior contingently on the behavior of their partner, generating a process of mutual adaptation supporting the successful transmission of technical skills in diverse and fluctuating learning environments. We propose that the study of cultural learning would benefit from the experimental methods, results, and insights of joint-action research and, complementarily, that the field of joint-action research could expand its scope by integrating a learning and cultural dimension. Bringing these two fields of research together promises to enrich our understanding of cultural learning, its contextual flexibility, and joint action coordination.

Evaluating the relative contributions of copying and reconstruction processes in cultural transmission episodes.

The ability to transmit information between individuals through social learning is a foundational component of cultural evolution. However, how this transmission occurs is still debated. On the one hand, the copying account draws parallels with biological mechanisms for genetic inheritance, arguing that learners copy what they observe and novel variations occur through random copying errors. On the other hand, the reconstruction account claims that, rather than directly copying behaviour, learners reconstruct the information that they believe to be most relevant on the basis of pragmatic inference, environmental and contextual cues. Distinguishing these two accounts empirically is difficult based on data from typical transmission chain studies because the predictions they generate frequently overlap. In this study we present a methodological approach that generates different predictions of these accounts by manipulating the task context between model and learner in a transmission episode. We then report an empirical proof-of-concept that applies this approach. The results show that, when a model introduces context-dependent embedded signals to their actions that are not intended to be transmitted, it is possible to empirically distinguish between competing predictions made by these two accounts. Our approach can therefore serve to understand the underlying cognitive mechanisms at play in cultural transmission and can make important contributions to the debate between preservative and reconstructive schools of thought.

Efficiency is prioritised over fairness when distributing joint actions.

Individuals have a drive towards maximising action efficiency, which is reflected in action choices that minimise movement costs to reach a goal. In joint actions, actors prioritise joint efficiency or coefficiency, maximising the utility of the joint action even if this comes at a cost to themselves. However, it remains an open question whether actors are willing to unilaterally sacrifice their partner's individual efficiency for the greater good, when forcing a partner to incur additional costs may be interpreted as unfair. In two experiments we explored how participants would choose to distribute a motor task that required either a fair or an unfair distribution of labour. We found that, both whether there was opportunity for reciprocity (Experiment 1) or not (Experiment 2), participants maximised the coefficiency of their joint actions, regardless of how unfair this distribution of labour proved to be regarding the individual action costs. Taken together, our results suggest participants use a rational decision-making framework that prioritises overall efficiency over both individual efficiency and a consideration of fairness.

It goes with the territory: Ownership across spatial boundaries.

Previous studies have shown that people are faster to process objects that they own as compared with objects that other people own. Yet object ownership is embedded within a social environment that has distinct and sometimes competing rules for interaction. Here we ask whether ownership of space can act as a filter through which we process what belongs to us. Can a sense of territory modulate the well-established benefits in information processing that owned objects enjoy? In 4 experiments participants categorized their own or another person's objects that appeared in territories assigned either to themselves or to another. We consistently found that faster processing of self-owned than other-owned objects only emerged for objects appearing in the self-territory, with no such advantage in other territories. We propose that knowing whom spaces belong to may serve to define the space in which affordances resulting from ownership lead to facilitated processing. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Investigating the formation and consolidation of incidentally learned trust.

People make inferences about the trustworthiness of others based on their observed gaze behavior. Faces that consistently look toward a target location are rated as more trustworthy than those that look away from the target. Representations of trust are important for future interactions; yet little is known about how they are consolidated in long-term memory. Sleep facilitates memory consolidation for incidentally learned information and may therefore support the retention of trust representations. We investigated the consolidation of trust inferences across periods of sleep or wakefulness. In addition, we employed a memory cueing procedure (targeted memory reactivation [TMR]) in a bid to strengthen certain trust memories over others. We observed no difference in the retention of trust inferences following delays of sleep or wakefulness, and there was no effect of TMR in either condition. Interestingly, trust inferences remained stable 1 week after learning, irrespective of the initial postlearning delay. A second experiment showed that this implicit learning occurs despite participants' being unable to explicitly recall the gaze behavior of specific faces immediately after encoding. Together, these results suggest that gist-like, social inferences are formed at the time of learning without retaining the original episodic memory and thus do not benefit from offline consolidation through replay. We discuss our findings in the context of a novel framework whereby trust judgments reflect an efficient, powerful, and adaptable storage device for social information. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

The role of emotion in the dyad inversion effect.

When observing two individuals, people are faster and better able to identify them as other people if they are facing each other than if they are facing away from each other. This advantage disappears when the images are inverted, suggesting that the visual system is particularly sensitive to dyads in this upright configuration, and perceptually groups socially engaged dyads into a single holistic unit. This dyadic inversion effect was obtained with images of full bodies. Body information was sufficient to elicit this effect even when information about head orientation was absent. However, it has not been tested whether the dyadic inversion effect occurs with face images and whether the emotions displayed by the faces modulate the effect. In three experiments we obtained robust dyadic inversion with face images. Holistic processing of upright face pairs occurred for neutral, happy, and sad faces but not for angry and fearful face pairs. Thus, perceptual grouping of individuals into pairs appears to depend on the emotional expressions of individual faces and the interpersonal relations they imply.

Examining the durability of incidentally learned trust from gaze cues.

In everyday interactions we find our attention follows the eye gaze of faces around us. As this cueing is so powerful and difficult to inhibit, gaze can therefore be used to facilitate or disrupt visual processing of the environment, and when we experience this we infer information about the trustworthiness of the cueing face. However, to date no studies have investigated how long these impressions last. To explore this we used a gaze-cueing paradigm where faces consistently demonstrated either valid or invalid cueing behaviours. Previous experiments show that valid faces are subsequently rated as more trustworthy than invalid faces. We replicate this effect (Experiment 1) and then include a brief interference task in Experiment 2 between gaze cueing and trustworthiness rating, which weakens but does not completely eliminate the effect. In Experiment 3, we explore whether greater familiarity with the faces improves the durability of trust learning and find that the effect is more resilient with familiar faces. Finally, in Experiment 4, we push this further and show that evidence of trust learning can be seen up to an hour after cueing has ended. Taken together, our results suggest that incidentally learned trust can be durable, especially for faces that deceive.

Incidental learning of trust: Examining the role of emotion and visuomotor fluency.

Eye gaze is a powerful directional cue that automatically evokes joint attention states. Even when faces are ignored, there is incidental learning of the reliability of the gaze cueing of another person, such that people who look away from targets are judged less trustworthy. In a series of experiments, we demonstrated further properties of the incidental learning of trust from gaze direction. First, the emotion of the face, whether neutral or smiling, influenced the pattern of trust learning. Second, the effect was specific to judgments of trust; reliability of gaze direction did not influence other emotional judgments of a person, such as liking. And third, visuomotor fluency was not sufficient for learning of trust, whether or not the face served as a target or as a distractor. Taken together, incidental learning of trust is influenced by facial emotion, it is a specific effect that does not generalize to other emotional assessments, and it is not determined solely by processing fluency. (PsycINFO Database Record

Responses in the right posterior superior temporal sulcus show a feature-based response to facial expression.

The face-selective region of the right posterior superior temporal sulcus (pSTS) plays an important role in analysing facial expressions. However, it is less clear how facial expressions are represented in this region. In this study, we used the face composite effect to explore whether the pSTS contains a holistic or feature-based representation of facial expression. Aligned and misaligned composite images were created from the top and bottom halves of faces posing different expressions. In Experiment 1, participants performed a behavioural matching task in which they judged whether the top half of two images was the same or different. The ability to discriminate the top half of the face was affected by changes in the bottom half of the face when the images were aligned, but not when they were misaligned. This shows a holistic behavioural response to expression. In Experiment 2, we used fMR-adaptation to ask whether the pSTS has a corresponding holistic neural representation of expression. Aligned or misaligned images were presented in blocks that involved repeating the same image or in which the top or bottom half of the images changed. Increased neural responses were found in the right pSTS regardless of whether the change occurred in the top or bottom of the image, showing that changes in expression were detected across all parts of the face. However, in contrast to the behavioural data, the pattern did not differ between aligned and misaligned stimuli. This suggests that the pSTS does not encode facial expressions holistically. In contrast to the pSTS, a holistic pattern of response to facial expression was found in the right inferior frontal gyrus (IFG). Together, these results suggest that pSTS reflects an early stage in the processing of facial expression in which facial features are represented independently.