A guide to inter-joint coordination characterization for discrete movements: a comparative study.

Characterizing human movement is essential for understanding movement disorders, evaluating progress in rehabilitation, or even analyzing how a person adapts to the use of assistive devices. Thanks to the improvement of motion capture technology, recording human movement has become increasingly accessible and easier to conduct. Over the last few years, multiple methods have been proposed for characterizing inter-joint coordination. Despite this, there is no real consensus regarding how these different inter-joint coordination metrics should be applied when analyzing the coordination of discrete movement from kinematic data. In this work, we consider 12 coordination metrics identified from the literature and apply them to a simulated dataset based on reaching movements using two degrees of freedom. Each metric is evaluated according to eight criteria based on current understanding of human motor control physiology, i.e, each metric is graded on how well it fulfills each of these criteria. This comparative analysis highlights that no single inter-joint coordination metric can be considered as ideal. Depending on the movement characteristics that one seeks to understand, one or several metrics among those reviewed here may be pertinent in data analysis. We propose four main factors when choosing a metric (or a group of metrics): the importance of temporal vs. spatial coordination, the need for result explainability, the size of the dataset, and the computational resources. As a result, this study shows that extracting the relevant characteristics of inter-joint coordination is a scientific challenge and requires a methodical choice. As this preliminary study is conducted on a limited dataset, a more comprehensive analysis, introducing more variability, could be complementary to these results.

Identification of inverse kinematic parameters in redundant systems: Towards quantification of inter-joint coordination in the human upper extremity.

Understanding and quantifying inter-joint coordination is valuable in several domains such as neurorehabilitation, robot-assisted therapy, robotic prosthetic arms, and control of supernumerary arms. Inter-joint coordination is often understood as a consistent spatiotemporal relation among kinematically redundant joints performing functional and goal-oriented movements. However, most approaches in the literature to investigate inter-joint coordination are limited to analysis of the end-point trajectory or correlation analysis of the joint rotations without considering the underlying task; e.g., creating a desirable hand movement toward a goal as in reaching motions. This work goes beyond this limitation by taking a model-based approach to quantifying inter-joint coordination. More specifically, we use the weighted pseudo-inverse of the Jacobian matrix and its associated null-space to explain the human kinematics in reaching tasks. We propose a novel algorithm to estimate such Inverse Kinematics weights from observed kinematic data. These estimated weights serve as a quantification for spatial inter-joint coordination; i.e., how costly a redundant joint is in its contribution to creating an end-effector velocity. We apply our estimation algorithm to datasets obtained from two different experiments. In the first experiment, the estimated Inverse Kinematics weights pinpoint how individuals change their Inverse Kinematics strategy when exposed to the viscous field wearing an exoskeleton. The second experiment shows how the resulting Inverse Kinematics weights can quantify a robotic prosthetic arm's contribution (or the level of assistance).

Influence of facial feedback during a cooperative human-robot task in schizophrenia.

Rapid progress in the area of humanoid robots offers tremendous possibilities for investigating and improving social competences in people with social deficits, but remains yet unexplored in schizophrenia. In this study, we examined the influence of social feedbacks elicited by a humanoid robot on motor coordination during a human-robot interaction. Twenty-two schizophrenia patients and twenty-two matched healthy controls underwent a collaborative motor synchrony task with the iCub humanoid robot. Results revealed that positive social feedback had a facilitatory effect on motor coordination in the control participants compared to non-social positive feedback. This facilitatory effect was not present in schizophrenia patients, whose social-motor coordination was similarly impaired in social and non-social feedback conditions. Furthermore, patients' cognitive flexibility impairment and antipsychotic dosing were negatively correlated with patients' ability to synchronize hand movements with iCub. Overall, our findings reveal that patients have marked difficulties to exploit facial social cues elicited by a humanoid robot to modulate their motor coordination during human-robot interaction, partly accounted for by cognitive deficits and medication. This study opens new perspectives for comprehension of social deficits in this mental disorder.

Unravelling socio-motor biomarkers in schizophrenia.

We present novel, low-cost and non-invasive potential diagnostic biomarkers of schizophrenia. They are based on the 'mirror-game', a coordination task in which two partners are asked to mimic each other's hand movements. In particular, we use the patient's solo movement, recorded in the absence of a partner, and motion recorded during interaction with an artificial agent, a computer avatar or a humanoid robot. In order to discriminate between the patients and controls, we employ statistical learning techniques, which we apply to nonverbal synchrony and neuromotor features derived from the participants' movement data. The proposed classifier has 93% accuracy and 100% specificity. Our results provide evidence that statistical learning techniques, nonverbal movement coordination and neuromotor characteristics could form the foundation of decision support tools aiding clinicians in cases of diagnostic uncertainty.

Humanoid robots versus humans: How is emotional valence of facial expressions recognized by individuals with schizophrenia? An exploratory study.

BACKGROUND: The use of humanoid robots to play a therapeutic role in helping individuals with social disorders such as autism is a newly emerging field, but remains unexplored in schizophrenia. As the ability for robots to convey emotion appear of fundamental importance for human-robot interactions, we aimed to evaluate how schizophrenia patients recognize positive and negative facial emotions displayed by a humanoid robot. METHODS: We included 21 schizophrenia outpatients and 17 healthy participants. In a reaction time task, they were shown photographs of human faces and of a humanoid robot (iCub) expressing either positive or negative emotions, as well as a non-social stimulus. Patients' symptomatology, mind perception, reaction time and number of correct answers were evaluated. RESULTS: Results indicated that patients and controls recognized better and faster the emotional valence of facial expressions expressed by humans than by the robot. Participants were faster when responding to positive compared to negative human faces and inversely were faster for negative compared to positive robot faces. Importantly, participants performed worse when they perceived iCub as being capable of experiencing things (experience subscale of the mind perception questionnaire). In schizophrenia patients, negative correlations emerged between negative symptoms and both robot's and human's negative face accuracy. CONCLUSIONS: Individuals do not respond similarly to human facial emotion and to non-anthropomorphic emotional signals. Humanoid robots have the potential to convey emotions to patients with schizophrenia, but their appearance seems of major importance for human-robot interactions.

Role of Gaze Cues in Interpersonal Motor Coordination: Towards Higher Affiliation in Human-Robot Interaction.

BACKGROUND: The ability to follow one another's gaze plays an important role in our social cognition; especially when we synchronously perform tasks together. We investigate how gaze cues can improve performance in a simple coordination task (i.e., the mirror game), whereby two players mirror each other's hand motions. In this game, each player is either a leader or follower. To study the effect of gaze in a systematic manner, the leader's role is played by a robotic avatar. We contrast two conditions, in which the avatar provides or not explicit gaze cues that indicate the next location of its hand. Specifically, we investigated (a) whether participants are able to exploit these gaze cues to improve their coordination, (b) how gaze cues affect action prediction and temporal coordination, and (c) whether introducing active gaze behavior for avatars makes them more realistic and human-like (from the user point of view). METHODOLOGY/PRINCIPAL FINDINGS: 43 subjects participated in 8 trials of the mirror game. Each subject performed the game in the two conditions (with and without gaze cues). In this within-subject study, the order of the conditions was randomized across participants, and subjective assessment of the avatar's realism was assessed by administering a post-hoc questionnaire. When gaze cues were provided, a quantitative assessment of synchrony between participants and the avatar revealed a significant improvement in subject reaction-time (RT). This confirms our hypothesis that gaze cues improve the follower's ability to predict the avatar's action. An analysis of the pattern of frequency across the two players' hand movements reveals that the gaze cues improve the overall temporal coordination across the two players. Finally, analysis of the subjective evaluations from the questionnaires reveals that, in the presence of gaze cues, participants found it not only more human-like/realistic, but also easier to interact with the avatar. CONCLUSION/SIGNIFICANCE: This work confirms that people can exploit gaze cues to predict another person's movements and to better coordinate their motions with their partners, even when the partner is a computer-animated avatar. Moreover, this study contributes further evidence that implementing biological features, here task-relevant gaze cues, enable the humanoid robotic avatar to appear more human-like, and thus increase the user's sense of affiliation.