Design and Validation of a Virtual Player for Studying Interpersonal Coordination in the Mirror Game.

The mirror game has been recently proposed as a simple, yet powerful paradigm for studying interpersonal interactions. It has been suggested that a virtual partner able to play the game with human subjects can be an effective tool to affect the underlying neural processes needed to establish the necessary connections between the players, and also to provide new clinical interventions for rehabilitation of patients suffering from social disorders. Inspired by the motor processes of the central nervous system (CNS) and the musculoskeletal system in the human body, in this paper we develop a novel interactive cognitive architecture based on nonlinear control theory to drive a virtual player (VP) to play the mirror game with a human player (HP) in different configurations. Specifically, we consider two cases: 1) the VP acts as leader and 2) the VP acts as follower. The crucial problem is to design a feedback control architecture capable of imitating and following or leading an HP in a joint action task. The movement of the end-effector of the VP is modeled by means of a feedback controlled Haken-Kelso-Bunz (HKB) oscillator, which is coupled with the observed motion of the HP measured in real time. To this aim, two types of control algorithms (adaptive control and optimal control) are used and implemented on the HKB model so that the VP can generate a human-like motion while satisfying certain kinematic constraints. A proof of convergence of the control algorithms is presented together with an extensive numerical and experimental validation of their effectiveness. A comparison with other existing designs is also discussed, showing the flexibility and the advantages of our control-based approach.

Interaction patterns and individual dynamics shape the way we move in synchrony.

An important open problem in Human Behaviour is to understand how coordination emerges in human ensembles. This problem has been seldom studied quantitatively in the existing literature, in contrast to situations involving dual interaction. Here we study motor coordination (or synchronisation) in a group of individuals where participants are asked to visually coordinate an oscillatory hand motion. We separately tested two groups of seven participants. We observed that the coordination level of the ensemble depends on group homogeneity, as well as on the pattern of visual couplings (who looked at whom). Despite the complexity of social interactions, we show that networks of coupled heterogeneous oscillators with different structures capture well the group dynamics. Our findings are relevant to any activity requiring the coordination of several people, as in music, sport or at work, and can be extended to account for other perceptual forms of interaction such as sound or feel.

Unravelling socio-motor biomarkers in schizophrenia.

We present novel, low-cost and non-invasive potential diagnostic biomarkers of schizophrenia. They are based on the 'mirror-game', a coordination task in which two partners are asked to mimic each other's hand movements. In particular, we use the patient's solo movement, recorded in the absence of a partner, and motion recorded during interaction with an artificial agent, a computer avatar or a humanoid robot. In order to discriminate between the patients and controls, we employ statistical learning techniques, which we apply to nonverbal synchrony and neuromotor features derived from the participants' movement data. The proposed classifier has 93% accuracy and 100% specificity. Our results provide evidence that statistical learning techniques, nonverbal movement coordination and neuromotor characteristics could form the foundation of decision support tools aiding clinicians in cases of diagnostic uncertainty.

A Novel Computer-Based Set-Up to Study Movement Coordination in Human Ensembles.

Existing experimental works on movement coordination in human ensembles mostly investigate situations where each subject is connected to all the others through direct visual and auditory coupling, so that unavoidable social interaction affects their coordination level. Here, we present a novel computer-based set-up to study movement coordination in human groups so as to minimize the influence of social interaction among participants and implement different visual pairings between them. In so doing, players can only take into consideration the motion of a designated subset of the others. This allows the evaluation of the exclusive effects on coordination of the structure of interconnections among the players in the group and their own dynamics. In addition, our set-up enables the deployment of virtual computer players to investigate dyadic interaction between a human and a virtual agent, as well as group synchronization in mixed teams of human and virtual agents. We show how this novel set-up can be employed to study coordination both in dyads and in groups over different structures of interconnections, in the presence as well as in the absence of virtual agents acting as followers or leaders. Finally, in order to illustrate the capabilities of the architecture, we describe some preliminary results. The platform is available to any researcher who wishes to unfold the mechanisms underlying group synchronization in human ensembles and shed light on its socio-psychological aspects.

Reconstructing the structure of directed and weighted networks of nonlinear oscillators.

The formalism of complex networks is extensively employed to describe the dynamics of interacting agents in several applications. The features of the connections among the nodes in a network are not always provided beforehand, hence the problem of appropriately inferring them often arises. Here, we present a method to reconstruct directed and weighted topologies of networks of heterogeneous nonlinear oscillators. We illustrate the theory on a set of representative examples.

Entrainment and synchronization in networks of Rayleigh-van der Pol oscillators with diffusive and Haken-Kelso-Bunz couplings.

We analyze a network of non-identical Rayleigh-van der Pol (RvdP) oscillators interconnected through either diffusive or nonlinear coupling functions. The work presented here extends existing results on the case of two nonlinearly coupled RvdP oscillators to the problem of considering a network of three or more of them. Specifically, we study synchronization and entrainment in networks of heterogeneous RvdP oscillators and contrast the effects of diffusive linear coupling strategies with the nonlinear Haken-Kelso-Bunz coupling, originally introduced to study human bimanual experiments. We show how convergence of the error among the nodes' trajectories toward a bounded region is possible with both linear and nonlinear coupling functions. Under the assumption that the network is connected, simple, and undirected, analytical results are obtained to prove boundedness of the error when the oscillators are coupled diffusively. All results are illustrated by way of numerical examples and compared with the experimental findings available in the literature on synchronization of people rocking chairs, confirming the effectiveness of the model we propose to capture some of the features of human group synchronization observed experimentally in the previous literature.

Design of a Virtual Player for Joint Improvisation with Humans in the Mirror Game.

Joint improvisation is often observed among humans performing joint action tasks. Exploring the underlying cognitive and neural mechanisms behind the emergence of joint improvisation is an open research challenge. This paper investigates jointly improvised movements between two participants in the mirror game, a paradigmatic joint task example. First, experiments involving movement coordination of different dyads of human players are performed in order to build a human benchmark. No designation of leader and follower is given beforehand. We find that joint improvisation is characterized by the lack of a leader and high levels of movement synchronization. Then, a theoretical model is proposed to capture some features of their interaction, and a set of experiments is carried out to test and validate the model ability to reproduce the experimental observations. Furthermore, the model is used to drive a computer avatar able to successfully improvise joint motion with a human participant in real time. Finally, a convergence analysis of the proposed model is carried out to confirm its ability to reproduce joint movements between the participants.

Dynamic similarity promotes interpersonal coordination in joint action.

Human movement has been studied for decades, and dynamic laws of motion that are common to all humans have been derived. Yet, every individual moves differently from everyone else (faster/slower, harder/smoother, etc.). We propose here an index of such variability, namely an individual motor signature (IMS) able to capture the subtle differences in the way each of us moves. We show that the IMS of a person is time-invariant and that it significantly differs from those of other individuals. This allows us to quantify the dynamic similarity, a measure of rapport between dynamics of different individuals' movements, and demonstrate that it facilitates coordination during interaction. We use our measure to confirm a key prediction of the theory of similarity that coordination between two individuals performing a joint-action task is higher if their motions share similar dynamic features. Furthermore, we use a virtual avatar driven by an interactive cognitive architecture based on feedback control theory to explore the effects of different kinematic features of the avatar motion on coordination with human players.