Availability of attention affects time-to-contact estimation.

To estimate the time-to-contact (TTC) of a moving object, numerous studies have focused on the type of information or gaze strategy used by the observer. However, it remains to be determined whether and how attention could affect TTC estimation. In particular, how does TTC estimation operate when less attention is available? To answer this question, we conducted two experiments in which the participants had to perform an absolute (Experiment 1) or relative (Experiment 2) prediction-motion task, either alone (i.e., in single-task condition) or along with a secondary, visual working-memory task (i.e., in dual-task condition). In both experiments, we found that TTC estimation was superior in dual-task condition relative to single-task condition. This finding suggests that the reduction of available attention actually improves TTC estimation. We discuss possible explanations as well as theoretical implications for this seemingly counter-intuitive finding. Further research is needed to investigate if (in)attention facilitates or only shifts TTC estimation.

Predicting where a ball will land: from thrower's body language to ball's motion.

To predict where a thrown ball will land, an observer may use visual information about its trajectory. However, in addition, the thrower's body language (i.e., body movement and facial expression) may contain useful information that could be used by the observer to understand intention and emotional state. Here, we investigated how observers estimated a ball's landing point thrown by a virtual agent with different amounts of information from body language. In addition, occlusion time was varied to examine how it potentiates the use of body-language information. Results showed that body movement and facial expression carry information about thrower's effort. However, once the ball has left the thrower's hand, advance information on facial expression does contribute to judgments only if consistent with the amplitude of the throw. Moreover, as the occlusion time increases, a stronger influence of the body movement is observed for estimating the landing point. The overriding effect of ball's trajectory availability over body language is discussed.

Agency and social affordance shape visual perception.

Research on the sense of agency has shown that being the author of an action changes the way we estimate the timing and the intensity of the action-effect. Yet, there is a dearth of attempts to assess the influence of agency on perception per se. The present study used the Representational Momentum paradigm to measure participants' visual anticipation of movement while manipulating their agency. In line with previous literature emphasizing the impact of social factors on visual anticipation and on the sense of agency, we additionally investigated the modulating power of social affordances on the relationship between agency and visual anticipation. We conducted two experiments where participants viewed a virtual agent directing a handshake gesture toward a second virtual agent. In a first experiment, we addressed the role of agency on visual anticipation by comparing a condition in which participants triggered the virtual agent's gesture with a condition where the computer triggered the gesture. Results showed greater forward movement anticipation when participants triggered the gesture. The second experiment investigated how altering social interaction parameters (interindividual distance and body posture) modulated the relationship between agency and visual anticipation. The outcome contrasted with the first experiment, with participants anticipating a backward movement of the hand when the computer triggered the gesture and displaying a null anticipation when participants triggered the gesture. Those two experiments highlighted how active involvement and environmental affordance interact to shape perception and allowed us to propose an updated model of agency processing.

Adaptation and validation of two annotation scales for assessing social skills in a corpus of multimodal collaborative interactions.

Context: Behavioral observation scales are important for understanding and assessing social skills. In the context of collaborative problem-solving (CPS) skills, considered essential in the 21st century, there are no validated scales in French that can be adapted to different CPS tasks. The aim of this study is to adapt and validate, by annotating a new video corpus of dyadic interactions that we have collected, two observational scales allowing us to qualitatively assess CPS skills: the Social Performance Rating Scale (SPRS) and the Social Skills of Collaboration Scale (SSC). Method: The construct validity of these two scales was assessed by exploratory factor analysis and inter-item correlations. We also checked inter-judge agreement using inter-class correlation coefficients. Internal consistency was determined using Cronbach's alpha and convergent and divergent validity by assessing correlations between the two scales and measures of depression and alexithymia. Finally, the discriminative properties of the two scales were analyzed by comparing the scores obtained by a group of anxious individuals and a non-anxious control group. Results: The results show that our two scales have excellent inter-item correlations. Internal consistency is excellent (alpha SPRS =0.90; SSC = 0.93). Inter-rater agreement ranged from moderate to high. Finally, convergent validity was significant with the alexithymia scale, as was divergent validity with the depression scale. Anxious individuals had lower scores on both scales than non-anxious individuals. Conclusion: Both scales show good psychometric properties for assessing social skills relevant to different collaborative tasks. They also identify individuals with difficulties in social interaction. Thus, they could allow monitoring the effectiveness of training social skills useful in CPS.

Decomposing spontaneous sign language into elementary movements: A principal component analysis-based approach.

Sign Language (SL) is a continuous and complex stream of multiple body movement features. That raises the challenging issue of providing efficient computational models for the description and analysis of these movements. In the present paper, we used Principal Component Analysis (PCA) to decompose SL motion into elementary movements called principal movements (PMs). PCA was applied to the upper-body motion capture data of six different signers freely producing discourses in French Sign Language. Common PMs were extracted from the whole dataset containing all signers, while individual PMs were extracted separately from the data of individual signers. This study provides three main findings: (1) although the data were not synchronized in time across signers and discourses, the first eight common PMs contained 94.6% of the variance of the movements; (2) the number of PMs that represented 94.6% of the variance was nearly the same for individual as for common PMs; (3) the PM subspaces were highly similar across signers. These results suggest that upper-body motion in unconstrained continuous SL discourses can be described through the dynamic combination of a reduced number of elementary movements. This opens up promising perspectives toward providing efficient automatic SL processing tools based on heavy mocap datasets, in particular for automatic recognition and generation.

Machine Learning of Motion Statistics Reveals the Kinematic Signature of the Identity of a Person in Sign Language.

Sign language (SL) motion contains information about the identity of a signer, as does voice for a speaker or gait for a walker. However, how such information is encoded in the movements of a person remains unclear. In the present study, a machine learning model was trained to extract the motion features allowing for the automatic identification of signers. A motion capture (mocap) system recorded six signers during the spontaneous production of French Sign Language (LSF) discourses. A principal component analysis (PCA) was applied to time-averaged statistics of the mocap data. A linear classifier then managed to identify the signers from a reduced set of principal components (PCs). The performance of the model was not affected when information about the size and shape of the signers were normalized. Posture normalization decreased the performance of the model, which nevertheless remained over five times superior to chance level. These findings demonstrate that the identity of a signer can be characterized by specific statistics of kinematic features, beyond information related to size, shape, and posture. This is a first step toward determining the motion descriptors necessary to account for the human ability to identify signers.

Representational momentum in dynamic facial expressions is modulated by the level of expressed pain: Amplitude and direction effects.

Humans have developed a specific capacity to rapidly perceive and anticipate other people's facial expressions so as to get an immediate impression of their emotional state of mind. We carried out two experiments to examine the perceptual and memory dynamics of facial expressions of pain. In the first experiment, we investigated how people estimate other people's levels of pain based on the perception of various dynamic facial expressions; these differ both in terms of the amount and intensity of activated action units. A second experiment used a representational momentum (RM) paradigm to study the emotional anticipation (memory bias) elicited by the same facial expressions of pain studied in Experiment 1. Our results highlighted the relationship between the level of perceived pain (in Experiment 1) and the direction and magnitude of memory bias (in Experiment 2): When perceived pain increases, the memory bias tends to be reduced (if positive) and ultimately becomes negative. Dynamic facial expressions of pain may reenact an "immediate perceptual history" in the perceiver before leading to an emotional anticipation of the agent's upcoming state. Thus, a subtle facial expression of pain (i.e., a low contraction around the eyes) that leads to a significant positive anticipation can be considered an adaptive process-one through which we can swiftly and involuntarily detect other people's pain.

The neuroelectric dynamics of the emotional anticipation of other people's pain.

When we observe a dynamic emotional facial expression, we usually automatically anticipate how that expression will develop. Our objective was to study a neurocognitive biomarker of this anticipatory process for facial pain expressions, operationalized as a mismatch effect. For this purpose, we studied the behavioral and neuroelectric (Event-Related Potential, ERP) correlates, of a match or mismatch, between the intensity of an expression of pain anticipated by the participant, and the intensity of a static test expression of pain displayed with the use of a representational momentum paradigm. Here, the paradigm consisted in displaying a dynamic facial pain expression which suddenly disappeared, and participants had to memorize the final intensity of the dynamic expression. We compared ERPs in response to congruent (intensity the same as the one memorized) and incongruent (intensity different from the one memorized) static expression intensities displayed after the dynamic expression. This paradigm allowed us to determine the amplitude and direction of this intensity anticipation by measuring the observer's memory bias. Results behaviorally showed that the anticipation was backward (negative memory bias) for high intensity expressions of pain (participants expected a return to a neutral state) and more forward (memory bias less negative, or even positive) for less intense expressions (participants expected increased intensity). Detecting mismatch (incongruent intensity) led to faster responses than detecting match (congruent intensity). The neuroelectric correlates of this mismatch effect in response to the testing of expression intensity ranged from P100 to LPP (Late Positive Potential). Path analysis and source localization suggested that the medial frontal gyrus was instrumental in mediating the mismatch effect through top-down influence on both the occipital and temporal regions. Moreover, having the facility to detect incongruent expressions, by anticipating emotional state, could be useful for prosocial behavior and the detection of trustworthiness.

How and why affective and reactive virtual agents will bring new insights on social cognitive disorders in schizophrenia? An illustration with a virtual card game paradigm.

In recent decades, many studies have shown that schizophrenia is associated with severe social cognitive impairments affecting key components, such as the recognition of emotions, theory of mind, attributional style, and metacognition. Most studies investigated each construct separately, precluding analysis of the interactive and immersive nature of real-life situation. Specialized batteries of tests are under investigation to assess social cognition, which is thought now as a link between neurocognitive disorders and impaired functioning. However, this link accounts for a limited part of the variance of real-life functioning. To fill this gap, advances in virtual reality and affective computing have made it possible to carry out experimental investigations of naturalistic social cognition, in controlled conditions, with good reproducibility. This approach is illustrated with the description of a new paradigm based on an original virtual card game in which subjects interpret emotional displays from a female virtual agent, and decipher her helping intentions. Independent variables concerning emotional expression in terms of valence and intensity were manipulated. We show how several useful dependant variables, ranging from classic experimental psychology data to metacognition or subjective experiences records, may be extracted from a single experiment. Methodological issues about the immersion into a simulated intersubjective situation are considered. The example of this new flexible experimental setting, with regards to the many constructs recognized in social neurosciences, constitutes a rationale for focusing on this potential intermediate link between standardized tests and real-life functioning, and also for using it as an innovative media for cognitive remediation.