Testing theory of mind in large language models and humans.

At the core of what defines us as humans is the concept of theory of mind: the ability to track other people's mental states. The recent development of large language models (LLMs) such as ChatGPT has led to intense debate about the possibility that these models exhibit behaviour that is indistinguishable from human behaviour in theory of mind tasks. Here we compare human and LLM performance on a comprehensive battery of measurements that aim to measure different theory of mind abilities, from understanding false beliefs to interpreting indirect requests and recognizing irony and faux pas. We tested two families of LLMs (GPT and LLaMA2) repeatedly against these measures and compared their performance with those from a sample of 1,907 human participants. Across the battery of theory of mind tests, we found that GPT-4 models performed at, or even sometimes above, human levels at identifying indirect requests, false beliefs and misdirection, but struggled with detecting faux pas. Faux pas, however, was the only test where LLaMA2 outperformed humans. Follow-up manipulations of the belief likelihood revealed that the superiority of LLaMA2 was illusory, possibly reflecting a bias towards attributing ignorance. By contrast, the poor performance of GPT originated from a hyperconservative approach towards committing to conclusions rather than from a genuine failure of inference. These findings not only demonstrate that LLMs exhibit behaviour that is consistent with the outputs of mentalistic inference in humans but also highlight the importance of systematic testing to ensure a non-superficial comparison between human and artificial intelligences.

Action prediction in psychosis.

Aberrant motor-sensory predictive functions have been linked to symptoms of psychosis, particularly reduced attenuation of self-generated sensations and misattribution of self-generated actions. Building on the parallels between prediction of self- and other-generated actions, this study aims to investigate whether individuals with psychosis also demonstrate abnormal perceptions and predictions of others' actions. Patients with psychosis and matched controls completed a two-alternative object size discrimination task. In each trial, they observed reaching actions towards a small and a large object, with varying levels of temporal occlusion ranging from 10% to 80% of movement duration. Their task was to predict the size of the object that would be grasped. We employed a novel analytic approach to examine how object size information was encoded and read out across progressive levels of occlusion with single-trial resolution. Patients with psychosis exhibited an overall pattern of reduced and discontinuous evidence integration relative to controls, characterized by a period of null integration up to 20% of movement duration, during which they did not read any size information. Surprisingly, this drop in accuracy in the initial integration period was not accompanied by a reduction in confidence. Difficulties in action prediction were correlated with the severity of negative symptoms and impaired functioning in social relationships.

Observational learning of atypical biological kinematics in autism.

Observing and voluntarily imitating the biological kinematics displayed by a model underpins the acquisition of new motor skills via sensorimotor processes linking perception with action. Differences in voluntary imitation in autism could be related to sensorimotor processing activity during action-observation of biological motion, as well as how sensorimotor integration processing occurs across imitation attempts. Using an observational practice protocol, which minimized the active contribution of the peripheral sensorimotor system, we examined the contribution of sensorimotor processing during action-observation. The data showed that autistic participants imitated both the temporal duration and atypical kinematic profile of the observed movement with a similar level of accuracy as neurotypical participants. These findings suggest the lower-level perception-action processes responsible for encoding biological kinematics during the action-observation phase of imitation are operational in autism. As there was no task-specific engagement of the peripheral sensorimotor system during observational practice, imitation difficulties in autism are most likely underpinned by sensorimotor integration issues related to the processing of efferent and (re)afferent sensorimotor information during trial-to-trial motor execution.

Neural network-based Bluetooth synchronization of multiple wearable devices.

Bluetooth-enabled wearables can be linked to form synchronized networks to provide insightful and representative data that is exceptionally beneficial in healthcare applications. However, synchronization can be affected by inevitable variations in the component's performance from their ideal behavior. Here, we report an application-level solution that embeds a Neural network to analyze and overcome these variations. The neural network examines the timing at each wearable node, recognizes time shifts, and fine-tunes a virtual clock to make them operate in unison and thus achieve synchronization. We demonstrate the integration of multiple Kinematics Detectors to provide synchronized motion capture at a high frequency (200 Hz) that could be used for performing spatial and temporal interpolation in movement assessments. The technique presented in this work is general and independent from the physical layer used, and it can be potentially applied to any wireless communication protocol.

Kinematic priming of action predictions.

The ability to anticipate what others will do next is crucial for navigating social, interactive environments. Here, we develop an experimental and analytical framework to measure the implicit readout of prospective intention information from movement kinematics. Using a primed action categorization task, we first demonstrate implicit access to intention information by establishing a novel form of priming, which we term kinematic priming: subtle differences in movement kinematics prime action prediction. Next, using data collected from the same participants in a forced-choice intention discrimination task 1 h later, we quantify single-trial intention readout-the amount of intention information read by individual perceivers in individual kinematic primes-and assess whether it can be used to predict the amount of kinematic priming. We demonstrate that the amount of kinematic priming, as indexed by both response times (RTs) and initial fixations to a given probe, is directly proportional to the amount of intention information read by the individual perceiver at the single-trial level. These results demonstrate that human perceivers have rapid, implicit access to intention information encoded in movement kinematics and highlight the potential of our approach to reveal the computations that permit the readout of this information with single-subject, single-trial resolution.

Morphology, Connectivity, and Encoding Features of Tactile and Motor Representations of the Fingers in the Human Precentral and Postcentral Gyrus.

Despite the tight coupling between sensory and motor processing for fine manipulation in humans, it is not yet totally clear which specific properties of the fingers are mapped in the precentral and postcentral gyrus. We used fMRI to compare the morphology, connectivity, and encoding of the motor and tactile finger representations (FRs) in the precentral and postcentral gyrus of 25 5-fingered participants (8 females). Multivoxel pattern and structural and functional connectivity analyses demonstrated the existence of distinct motor and tactile FRs within both the precentral and postcentral gyrus, integrating finger-specific motor and tactile information. Using representational similarity analysis, we found that the motor and tactile FRs in the sensorimotor cortex were described by the perceived structure of the hand better than by the actual hand anatomy or other functional models (finger kinematics, muscles synergies). We then studied a polydactyly individual (i.e., with a congenital 6-fingered hand) showing superior manipulation abilities and divergent anatomic-functional hand properties. The perceived hand model was still the best model for tactile representations in the precentral and postcentral gyrus, while finger kinematics better described motor representations in the precentral gyrus. We suggest that, under normal conditions (i.e., in subjects with a standard hand anatomy), the sensorimotor representations of the 5 fingers in humans converge toward a model of perceived hand anatomy, deviating from the real hand structure, as the best synthesis between functional and structural features of the hand.SIGNIFICANCE STATEMENT Distinct motor and tactile finger representations exist in both the precentral and postcentral gyrus, supported by a finger-specific pattern of anatomic and functional connectivity across modalities. At the representational level, finger representations reflect the perceived structure of the hand, which might result from an adapting process harmonizing (i.e., uniformizing) the encoding of hand function and structure in the precentral and postcentral gyrus. The same analyses performed in an extremely rare polydactyly subject showed that the emergence of such representational geometry is also found in neuromechanical variants with different hand anatomy and function. However, the harmonization process across the precentral and postcentral gyrus might not be possible because of divergent functional-structural properties of the hand and associated superior manipulation abilities.

Decoding social decisions from movement kinematics.

Decisions, including social decisions, are ultimately expressed through actions. However, very little is known about the kinematics of social decisions, and whether movements might reveal important aspects of social decision-making. We addressed this question by developing a motor version of a widely used behavioral economic game - the Ultimatum Game - and using a multivariate kinematic decoding approach to map parameters of social decisions to the single-trial kinematics of individual responders. Using this approach, we demonstrated that movement contains predictive information about both the fairness of a proposed offer and the choice to either accept or reject that offer. This information is expressed in personalized kinematic patterns that are consistent within a given responder, but that varies from one responder to another. These results provide insights into the relationship between decision-making and sensorimotor control, as they suggest that hand kinematics can reveal hidden parameters of complex, social interactive, choice.

Pursuing occupants' health and well-being in building management: Definition of new metrics based on indoor air parameters.

The spread of COVID-19 has affected the lives of millions of people. Pandemic has made people more sensitive to health issues. In particular, the growing concern for the virus spread in confined spaces has promoted the necessity to improve indoor air quality. Literature is stressing how buildings must be designed and operated pursuing occupants' health and well-being, with a particular attention for indoor air parameters. This poses the challenge of monitoring and assessing these aspects through proper metrics. In this paper the approach towards a multi-step assessment procedure embedding in buildings assessment health and well-being related variables and indicators is elaborated. They are intended to inform a building manager of the potential influence of air conditions on human health and well-being. Moreover, a set of monetary metrics (i.e., impacts) is proposed to translate energy and indoor air related building performances into euros, putting the basis for a comprehensive economic evaluation. The application of the set of proposed metrics to an Italian hotel (i.e., Italian pilot of H2020 MOBISTYLE project), enabled to map some indoor air conditions causing health concerns, and to identify clusters of guests with best and worst indoor air conditions, to be targeted by new management strategies. Despite case study specific limitations, the application exemplified how the methodology can expand the traditional energy-based performance assessment for building management towards indoor air domain and the related economic impacts, with implication on results in terms of overall economic performance of the building from both a private and public perspective.

Intersecting kinematic encoding and readout of intention in autism.

Observers with autism spectrum disorders (ASDs) find it difficult to read intentions from movements. However, the computational bases of these difficulties are unknown. Do these difficulties reflect an intention readout deficit, or are they more likely rooted in kinematic (dis-)similarities between typical and ASD kinematics? We combined motion tracking, psychophysics, and computational analyses to uncover single-trial intention readout computations in typically developing (TD) children (n = 35) and children with ASD (n = 35) who observed actions performed by TD children and children with ASD. Average intention discrimination performance was above chance for TD observers but not for ASD observers. However, single-trial analysis showed that both TD and ASD observers read single-trial variations in movement kinematics. TD readers were better able to identify intention-informative kinematic features during observation of TD actions; conversely, ASD readers were better able to identify intention-informative features during observation of ASD actions. Crucially, while TD observers were generally able to extract the intention information encoded in movement kinematics, those with autism were unable to do so. These results extend existing conceptions of mind reading in ASD by suggesting that intention reading difficulties reflect both an interaction failure, rooted in kinematic dissimilarity between TD and ASD kinematics (at the level of feature identification), and an individual readout deficit (at the level of information extraction), accompanied by an overall reduced sensitivity of intention readout to single-trial variations in movement kinematics.

Costs and benefits of communicating vigor.

Why do we run toward people we love, but only walk toward others? One reason is to let them know we love them. In this commentary, we elaborate on how subjective utility information encoded in vigor is read out by others. We consider the potential implications for understanding and modeling the link between movements and decisions in social environments.

A low-cost stand-alone platform for measuring motor behavior across developmental applications.

Motion tracking provides unique insights into motor, cognitive, and social development by capturing subtle variations into how movements are planned and controlled. Here, we present a low-cost, wearable movement measurement platform, KiD, specifically designed for tracking the movements of infants and children in a variety of natural settings. KiD consists of a small, lightweight sensor containing a nine-axis inertial measurement unit plus an integrated processor for computing rotations. Measurements of three-dimensional acceleration using KiD compare well with those of current state-of-the-art optical motion capture systems. As a proof of concept, we demonstrate successful classification of different types of sinusoidal right arm movements using KiD.

Understanding joint action: Current theoretical and empirical approaches.

Joint actions are omnipresent, ranging from a handshake between two people to the coordination of groups of people playing in an orchestra. We are highly skilled at coordinating our actions with those of others to reach common goals and rely on this ability throughout our daily lives. What are the social, cognitive and neural processes underlying this ability? How do others around us influence our task representations? How does joint action influence interpersonal interactions? How do language and gesture support joint action? What differentiates joint action from individual action? This article forms an introductory editorial to the field of joint action. It accompanies contributions to the special issue entitled "Current Issues in Joint Action Research". The issue brings together conceptual and empirical approaches on different topics, ranging from lower-level issues such as the link between perception and joint action, to higher-level issues such as language as a form of joint action.

Identifying the signature of prospective motor control in children with autism.

Failure to develop prospective motor control has been proposed to be a core phenotypic marker of autism spectrum disorders (ASD). However, whether genuine differences in prospective motor control permit discriminating between ASD and non-ASD profiles over and above individual differences in motor output remains unclear. Here, we combined high precision measures of hand movement kinematics and rigorous machine learning analyses to determine the true power of prospective movement data to differentiate children with autism and typically developing children. Our results show that while movement is unique to each individual, variations in the kinematic patterning of sequential grasping movements genuinely differentiate children with autism from typically developing children. These findings provide quantitative evidence for a prospective motor control impairment in autism and indicate the potential to draw inferences about autism on the basis of movement kinematics.

Transient Disruption of the Inferior Parietal Lobule Impairs the Ability to Attribute Intention to Action.

Although it is well established that fronto-parietal regions are active during action observation, whether they play a causal role in the ability to infer others' intentions from visual kinematics remains undetermined. In the experiments reported here, we combined offline continuous theta burst stimulation (cTBS) with computational modeling to reveal and causally probe single-trial computations in the inferior parietal lobule (IPL) and inferior frontal gyrus (IFG). Participants received cTBS over the left anterior IPL and the left IFG pars orbitalis in separate sessions before completing an intention discrimination task (discriminate intention of observed reach-to-grasp acts) or a kinematic discrimination task unrelated to intention (discriminate peak wrist height of the same acts). We targeted intention-sensitive regions whose fMRI activity, recorded when observing the same reach-to-grasp acts, could accurately discriminate intention. We found that transient disruption of activity of the left IPL, but not the IFG, impaired the observer's ability to attribute intention to action. Kinematic discrimination unrelated to intention, in contrast, was largely unaffected. Computational analyses of how encoding (mapping of intention to movement kinematics) and readout (mapping of kinematics to intention choices) intersect at the single-trial level revealed that IPL cTBS did not diminish the overall sensitivity of intention readout to movement kinematics. Rather, it selectively misaligned intention readout with respect to encoding, deteriorating mapping from informative kinematic features to intention choices. These results provide causal evidence of how the left anterior IPL computes mapping from kinematics to intentions.

A kind of magic: Enhanced detection of pantomimed grasps in professional magicians.

Professional magicians regularly use pantomimed grasps (i.e., movements towards imagined objects) to deceive audiences. To do so, they learn to shape their hands similarly for real and pantomimed grasps. Here we tested whether this form of motor expertise provides them a significant benefit when processing pantomimed grasps. To this aim, in a one-interval discrimination design, we asked 17 professional magicians and 17 naïve controls to watch video clips of reach-to-grasp movements recorded from naïve participants and judge whether the observed movement was real or pantomimed. All video clips were edited to spatially occlude the grasped object (either present or imagined). Data were analysed within a drift diffusion model approach. Fitting different models showed that, whereas magicians and naïve performed similarly when observing real grasps, magicians had a specific advantage compared with naïve at discriminating pantomimed grasps. These findings suggest that motor expertise may be crucial for detecting relevant cues from hand movement during the discrimination of pantomimed grasps. Results are discussed in terms of motor recalibration.

And Yet It Moves: What We Currently Know about Phantom Arm Movements.

What is left over if I subtract the fact that my arm goes up from the fact that I raise my arm? Neurological evidence invites the provocative hypothesis that what is left over is a phantom arm movement-a movement of an arm that has been amputated. After arm/hand amputation, many amputees report that they can generate voluntary movements of the phantom limb; that is, they can move the arm that was amputated. But what is it like to move an arm/hand that is not there? Here, we review what is currently known about phantom limb movements at three descriptive levels: the kinematic level, the muscle level, and the cortical level. We conclude that phantom arm movements are best conceptualized as the real movements of a dematerialized hand.

Modulation of corticospinal output during goal-directed actions: Evidence for a contingent coding hypothesis.

Seeing a person perform an action activates the observer's motor system. The present study aimed at investigating the temporal relationship between execution and observation of goal-directed actions. One possibility is that the corticospinal excitability (CSE) follows the dynamic evolution of the pattern of muscle activity in the executed action. Alternatively, CSE may anticipate the future course of the observed action, prospectively extrapolating future states. Our study was designed to test these alternative hypotheses by directly comparing the time course of muscle recruitment during the execution and observation of reach-to-grasp movements. We found that the time course of CSE during action observation followed the time course of the EMG signal during action execution. This contingent coding was observed despite the outcome of the observed motor act being predictable from the earliest phases of the movement. These findings challenge the view that CSE serves to predict the target of an observed action.

Tracking the Leader: Gaze Behavior in Group Interactions.

Can social gaze behavior reveal the leader during real-world group interactions? To answer this question, we developed a novel tripartite approach combining (1) computer vision methods for remote gaze estimation, (2) a detailed taxonomy to encode the implicit semantics of multi-party gaze features, and (3) machine learning methods to establish dependencies between leadership and visual behaviors. We found that social gaze behavior distinctively identified group leaders. Crucially, the relationship between leadership and gaze behavior generalized across democratic and autocratic leadership styles under conditions of low and high time-pressure, suggesting that gaze can serve as a general marker of leadership. These findings provide the first direct evidence that group visual patterns can reveal leadership across different social behaviors and validate a new promising method for monitoring natural group interactions.