Does co-presence affect the way we perceive and respond to emotional interactions?

This study compared how two virtual display conditions of human body expressions influenced explicit and implicit dimensions of emotion perception and response behavior in women and men. Two avatars displayed emotional interactions (angry, sad, affectionate, happy) in a "pictorial" condition depicting the emotional interactive partners on a screen within a virtual environment and a "visual" condition allowing participants to share space with the avatars, thereby enhancing co-presence and agency. Subsequently to stimulus presentation, explicit valence perception and response tendency (i.e. the explicit tendency to avoid or approach the situation) were assessed on rating scales. Implicit responses, i.e. postural and autonomic responses towards the observed interactions were measured by means of postural displacement and changes in skin conductance. Results showed that self-reported presence differed between pictorial and visual conditions, however, it was not correlated with skin conductance responses. Valence perception was only marginally influenced by the virtual condition and not at all by explicit response behavior. There were gender-mediated effects on postural response tendencies as well as gender differences in explicit response behavior but not in valence perception. Exploratory analyses revealed a link between valence perception and preferred behavioral response in women but not in men. We conclude that the display condition seems to influence automatic motivational tendencies but not higher level cognitive evaluations. Moreover, intragroup differences in explicit and implicit response behavior highlight the importance of individual factors beyond gender.

Sensory features of mental images in the framework of human actions.

What determines the sensory impression of a self-generated motor image? Motor imagery is a process in which subjects imagine executing a body movement with a strong kinesthetic and/or visual component from a first-person perspective. Both sensory modalities can be combined flexibly to form a motor image. 90 participants of varying ages had to freely generate motor images from a large set of movements. They were asked to rate their kinesthetic as well as their visual impression, the perceived vividness, and their personal experience with the imagined movement. Data were subjected to correlational analyses, linear regressions, and representation similarity analyses. Results showed that both action characteristics and experience drove the sensory impression of motor images with a strong individual component. We conclude that imagining actions that impose varying demands can be considered as reexperiencing actions by using one's own sensorimotor representations that represent not only individual experience but also action demands.

Emotional expressivity of the observer mediates recognition of affective states from human body movements.

Research on human motion perception shows that people are highly adept at inferring emotional states from body movements. Yet, this process is mediated by a number of individual factors and experiences. Within this study, we tackle two questions. Firstly, we ask which part of the body transmits the key information that is used to infer affective states. Secondly, we address how the observer's own emotional expressivity influences the recognition process. We used two types of impoverished point-light displays depicting the same emotional interactions as either arm or trunk movements. Results showed that participants used different sources of information in an emotion-specific manner. Participants with richer self-reported emotional expressivity showed higher recognition accuracies overall but also benefited more from information delivered by arm gestures. We interpret our findings in terms of embodied simulation, suggesting that emotion perception constitutes a function of the expressing body and the individual observer.

Subjective vividness of motor imagery has a neural signature in human premotor and parietal cortex.

Motor imagery (MI) is the process in which subjects imagine executing a body movement with a strong kinesthetic component from a first-person perspective. The individual capacity to elicit such mental images is not universal but varies within and between subjects. Neuroimaging studies have shown that these inter-as well as intra-individual differences in imagery quality mediate the amplitude of neural activity during MI on a group level. However, these analyses were not sensitive to forms of representation that may not map onto a simple modulation of overall amplitude. Therefore, the present study asked how far the subjective impression of motor imagery vividness is reflected by a spatial neural code, and how patterns of neural activation in different motor regions relate to specific imagery impressions. During fMRI scanning, 20 volunteers imagined three different types of right-hand actions. After each imagery trial, subjects were asked to evaluate the perceived vividness of their imagery. A correlation analysis compared the rating differences and neural dissimilarity values of the rating groups separately for each region of interest. Results showed a significant positive correlation in the left vPMC and right IPL, indicating that these regions particularly reflect perceived imagery vividness in that similar rated trials evoke more similar neural patterns. A decoding analysis revealed that the vividness of the motor image related systematically to the action specificity of neural activation patterns in left vPMC and right SPL. Imagined actions accompanied by higher vividness ratings were significantly more distinguishable within these areas. Altogether, results showed that spatial patterns of neural activity within the human motor cortices reflect the individual vividness of imagined actions. Hence, the findings reveal a link between the subjective impression of motor imagery vividness and objective physiological markers.

Imagined and Executed Actions in the Human Motor System: Testing Neural Similarity Between Execution and Imagery of Actions with a Multivariate Approach.

Simulation theory proposes motor imagery (MI) to be a simulation based on representations also used for motor execution (ME). Nonetheless, it is unclear how far they use the same neural code. We use multivariate pattern analysis (MVPA) and representational similarity analysis (RSA) to describe the neural representations associated with MI and ME within the frontoparietal motor network. During functional magnetic resonance imaging scanning, 20 volunteers imagined or executed 3 different types of right-hand actions. Results of MVPA showed that these actions as well as their modality (MI or ME) could be decoded significantly above chance from the spatial patterns of BOLD signals in premotor and posterior parietal cortices. This was also true for cross-modal decoding. Furthermore, representational dissimilarity matrices of frontal and parietal areas showed that MI and ME representations formed separate clusters, but that the representational organization of action types within these clusters was identical. For most ROIs, this pattern of results best fits with a model that assumes a low-to-moderate degree of similarity between the neural patterns associated with MI and ME. Thus, neural representations of MI and ME are neither the same nor totally distinct but exhibit a similar structural geometry with respect to different types of action.

Decoding spatiotemporal features of emotional body language in social interactions.

How are emotions perceived through human body language in social interactions? This study used point-light displays of human interactions portraying emotional scenes (1) to examine quantitative intrapersonal kinematic and postural body configurations, (2) to calculate interaction-specific parameters of these interactions, and (3) to analyze how far both contribute to the perception of an emotion category (i.e. anger, sadness, happiness or affection) as well as to the perception of emotional valence. By using ANOVA and classification trees, we investigated emotion-specific differences in the calculated parameters. We further applied representational similarity analyses to determine how perceptual ratings relate to intra- and interpersonal features of the observed scene. Results showed that within an interaction, intrapersonal kinematic cues corresponded to emotion category ratings, whereas postural cues reflected valence ratings. Perception of emotion category was also driven by interpersonal orientation, proxemics, the time spent in the personal space of the counterpart, and the motion-energy balance between interacting people. Furthermore, motion-energy balance and orientation relate to valence ratings. Thus, features of emotional body language are connected with the emotional content of an observed scene and people make use of the observed emotionally expressive body language and interpersonal coordination to infer emotional content of interactions.

When the timing is right: The link between temporal coupling in dyadic interactions and emotion recognition.

Affective states can be understood as dynamic interpersonal processes developing over time and space. When we observe emotional interactions performed by other individuals, our visual system anticipates how the action will unfold. Thus, it has been proposed that the process of emotion perception is not only a simulative but also a predictive process - a phenomenon described as interpersonal predictive coding. The present study investigated whether the recognition of emotions from dyadic interactions depends on a fixed spatiotemporal coupling of the agents. We used an emotion recognition task to manipulate the actions of two interacting point-light figures by implementing different temporal offsets that delayed the onset of one of the agent's actions (+0 ms, +500 ms, +1000 ms or + 2000 ms). Participants had to determine both the subjective valence and the emotion category (happiness, anger, sadness, affection) of the interaction. Results showed that temporal decoupling had a critical effect on both emotion recognition and the subjective impression of valence intensity: Both measures decreased with increasing temporal offset. However, these effects were dependent on which emotion was displayed. Whereas affection and anger sequences were impacted by the temporal manipulation, happiness and sadness were not. To further investigate these effects, we conducted post-hoc exploratory analyses of interpersonal movement parameters. Our findings complement and extend previous evidence by showing that the complex, noncoincidental coordination of actions within dyadic interactions results in a meaningful movement pattern and might serve as a fundamental factor in both detecting and understanding complex actions during human interaction.

Practice modality of motor sequences impacts the neural signature of motor imagery.

Motor imagery is conceptualized as an internal simulation that uses motor-related parts of the brain as its substrate. Many studies have investigated this sharing of common neural resources between the two modalities of motor imagery and motor execution. They have shown overlapping but not identical activation patterns that thereby result in a modality-specific neural signature. However, it is not clear how far this neural signature depends on whether the imagined action has previously been practiced physically or only imagined. The present study aims to disentangle whether the neural imprint of an imagined manual pointing sequence within cortical and subcortical motor areas is determined by the nature of this prior practice modality. Each participant practiced two sequences physically, practiced two other sequences mentally, and did a behavioural pre-test without any further practice on a third pair of sequences. After a two-week practice intervention, participants underwent fMRI scans while imagining all six sequences. Behavioural data demonstrated practice-related effects as well as very good compliance with instructions. Functional MRI data confirmed the previously known motor imagery network. Crucially, we found that mental and physical practice left a modality-specific footprint during mental motor imagery. In particular, activation within the right posterior cerebellum was stronger when the imagined sequence had previously been practiced physically. We conclude that cerebellar activity is shaped specifically by the nature of the prior practice modality.