SmartDetector: Automatic and vision-based approach to point-light display generation for human action perception.

Over the past four decades, point-light displays (PLD) have been integrated into psychology and psychophysics, providing a valuable means to probe human perceptual skills. Leveraging the inherent kinematic information and controllable display parameters, researchers have utilized this technique to examine the mechanisms involved in learning and rehabilitation. However, classical PLD generation methods (e.g., motion capture) are difficult to apply for behavior analysis in real-world situations, such as patient care or sports activities. Therefore, there is a demand for automated and affordable tools that enable efficient and real-world-compatible generation of PLDs for psychological research. In this paper, we propose SmartDetector, a new artificial intelligence (AI)-based tool for automatic PLD creation from RGB videos. To evaluate humans' perceptual skills for processing PLD building with SmartDetector, 126 participants were randomly assigned to recognition, discrimination, or detection tasks. Results demonstrated that, irrespective of the task, PLDs generated by SmartDetector exhibited commendable perceptual performance in terms of accuracy and response times compared to literature findings. Moreover, to enhance usability and broaden accessibility, we developed an intuitive web interface for our method, making it available to a wider audience. The resulting application is available at https://plavimop.prd.fr/index.php/en/automatic-creation-pld . SmartDetector offers interesting possibilities for using PLD in research and makes the use of PLD more accessible for nonacademic applications.

Point-light display: a new tool to improve verb recovery in patients with aphasia? A pilot study.

Some studies have demonstrated that Action Observation (AO) could help patients with aphasia to recover use of verbs. However, the role of kinematics in this effect has remained unknown. The main aim was to assess the effectiveness of a complementary intervention based on the observation of action kinematics in patients with aphasia. Seven aphasic patients (3 males, 4 females) aged between 55 and 88 years participated in the studies. All patients received a classical intervention and an additional, specific intervention based on action observation. This consisted in visualizing a static image or a point-light sequence representing a human action and in trying to name the verb representing the action. In each session, 57 actions were visualized: 19 represented by a static drawing, 19 by a non-focalized point-light sequence, i.e., a point-light display with all dots in white, and 19 by a focalized point-light sequence, i.e., a point-light display (PLD) with the dots corresponding to the main limbs in yellow. Before (pre-test) and after (post-test) the intervention, each patient performed the same denomination task, in which all actions were presented in photographs. The results showed a significant improvement in performance between pre and post-test, but only when the actions were presented in focalized and non-focalized point-light sequences during the intervention. The presentation of action kinematics seems crucial in the recovery of verbs in patients with aphasia. This should be considered by speech therapists in their interventions.

Effects of short-term hand immobilization on anticipatory mechanism for tool use.

The short-term immobilization of a limb such as the right arm can impair sensorimotor mechanisms, which in turn reduces motor control of this arm. However, it is not known whether immobilization also impairs the anticipatory mechanism for tool use without actual enactment. In two experiments, we asked participants to judge how they would use a tool in a particular environment (e.g., "Take the pencil to write on a sheet of paper"). Prior to this tool-use judgment task, some participants had been immobilized (right arm) for 24 h. Results revealed that compared to controls, immobilized participants performed more poorly on the tool-use judgment task (accuracy and response time) as well as in a manual dexterity task. As our tool-use judgment task involved anticipating the expected perceptual effect of using a tool to achieve an environmental goal (e.g., writing on a sheet of paper), our data are discussed in line with theories of motor control (e.g., ideomotor theory) that emphasize the expected perceptual consequences of the action.

The role of implicit motor simulation on action verb memory.

Observation is known to improve memory for action. Previous findings linked such an effect with an easier relation processing of action components following observation compared to mere sentence reading. However, action observation also elicits implicit motor simulation, that is a processing of one's movement through the observer own motor system. We aimed to assess whether encoding of implicit motor simulation can also explain why observation is better than reading for action memory. To prevent influence of item relation processing, two studies about isolated action verbs learning were designed. In Experiment 1, action verbs were encoded with short videos of point-light human movements or with written definitions. Subsequent free recall indicated better memory for the verbs within the video clip condition. Experiment 2 compared two encoding conditions based on point-light human movement videos. Half of the verbs were learned with their normal corresponding movement (biological kinematic). For the other half of the verbs, the velocity of point-light movements was modified to create abnormal nonbiological kinematic actions. We observed better free recall for the verbs learned with biological kinematics. Taken together, those results suggest that action observation is beneficial because it allows the encoding of motor-related information (implicit motor simulation). Semantic resonance between linguistic and motor representations of action could also contribute to memory improvement. Contrary to previous studies, our results cannot be explained by an improvement of items relation processing. However, it suggests that the basic level of action verb memory is sensorimotor perception, such as implicit motor simulation.

PLAViMoP database: A new continuously assessed and collaborative 3D point-light display dataset.

It was more than 45 years ago that Gunnar Johansson invented the point-light display technique. This showed for the first time that kinematics is crucial for action recognition, and that humans are very sensitive to their conspecifics' movements. As a result, many of today's researchers use point-light displays to better understand the mechanisms behind this recognition ability. In this paper, we propose PLAViMoP, a new database of 3D point-light displays representing everyday human actions (global and fine-motor control movements), sports movements, facial expressions, interactions, and robotic movements. Access to the database is free, at https://plavimop.prd.fr/en/motions . Moreover, it incorporates a search engine to facilitate action retrieval. In this paper, we describe the construction, functioning, and assessment of the PLAViMoP database. Each sequence was analyzed according to four parameters: type of movement, movement label, sex of the actor, and age of the actor. We provide both the mean scores for each assessment of each point-light display, and the comparisons between the different categories of sequences. Our results are discussed in the light of the literature and the suitability of our stimuli for research and applications.

The Added Value of Point-Light Display Observation in Total Knee Arthroplasty Rehabilitation Program: A Prospective Randomized Controlled Pilot Study.

Background and Objectives: The present study aimed to assess the potential benefit of the observation of rehabilitation-related point-light display in addition to a conventional 3-week rehabilitation program, the objective being to improve functional capacity in patients having undergone total knee arthroplasty. Materials and Methods: Patients randomized in the control group had conventional rehabilitation treatment with two sessions per day 5 days a week of physical therapy (90 min), whereas patients in the experimental group had a program of conventional rehabilitation combined with a point-light display observation two times per day (5 min) and 3 days a week. Results: The patients of both groups had improved their performances by the end of the program, and the pre- and post-test improvement were superior for the experimental group over the control group concerning the total WOMAC score (p = 0.04), the functional WOMAC score (p = 0.03), and correct recognition of point-light displays (p = 0.003). Conclusions: These findings provide new insight favoring systematic point-light display observation to improve functional recovery in patients with total knee arthroplasty.

Physical and observational practices of unusual actions prime action verb processing.

Numerous studies have highlighted a strong relationship between language and sensorimotor processes, showing, for example, that perceiving an action influences subsequent language processing. Moreover, previous studies have demonstrated that the context in which actions are perceived is crucial to enable this action-language relationship. In particular, action verb processing is facilitated when an action is perceived in its usual context (e.g., someone watering a plant) but not in an unusual context (e.g., someone watering a computer). This difference could be explained in terms of experience; because people always practice actions in accordance with the context, they have no (visual or motor) experience related to the unusual context. The aim of the present study was to test this assumption by assessing and comparing the effect of physical practice and observational learning on the action-language relationship. The results of two experiments showed a facilitation effect of both training methods. Whereas usual actions systematically prime action verb processing, the link between action and language appears for unusual actions only after training by practicing (experiment 1, physical practice) or observing (experiment 2, observational learning). Overall, these findings support the role of experience in the activation of sensorimotor representations during action verb processing.

Short-term upper-limb immobilization alters peripersonal space representation.

Peripersonal space is a multisensory interface between the environment and the body subserving motor interactions with the physical and social world. Although changing body properties has been shown to alter the functional processing of space, little is known about the effect of short-term limb immobilization specifically on the motor representation of peripersonal space. In the present study, we investigated the effect of a right upper-limb immobilization for a duration of 24 h on a reachability judgment task and a brightness judgment task. Analyses of perceptual thresholds revealed a reduction of peripersonal space representation after the immobilization period, which was not observed when there was no immobilization (control group). In contrast, no variation appeared in the brightness judgment task, suggesting no presence of specific visual perception or decisional deficits in the limb immobilization group. Considered together, the results confirm the crucial role of the motor system in the representation of peripersonal space. They also highlight the plasticity of the motor system resulting in a rapid change of its activity following limb immobilization, with a concomitant effect on motor-related perceptual and cognitive processes.

Does watching Han Solo or C-3PO similarly influence our language processing?

Several studies have demonstrated that perceiving an action influences the subsequent processing of action verbs. However, which characteristics of the perceived action are truly determinant to enable this influence is still unknown. The current study investigated the role of the agent executing an action in this action-language relationship. Participants performed a semantic decision task after seeing a video of a human or a robot performing an action. The results of the first study showed that perceiving a human being executing an action as well as perceiving a robot facilitate subsequent language processing, suggesting that the humanness (The term "humanness" is used as meaning "belonging to human race" and not to refer to a personal quality) of the agent is not crucial in the link between action and language. However, this experiment was conducted with Japanese people who are very familiar with robots; thus, an alternative explanation could be that it is the unfamiliarity with the agent that could perturb the action-language relationship. To assess this hypothesis, we carried out two additional experiments with French participants. The results of the second study showed that, unlike the observation of a human agent, the observation of a robot did not influence language processing. Finally, the results of the third study showed that, after a familiarization phase, French participants too were influenced by the observation of a robot. Overall, the outcomes of these studies indicate that, more than the humanness of the agent, it is the familiarity which we have with this agent that is crucial in the action-language relationship.

How Action Context Modulates the Action-Language Relationship: A Topographic ERP Analysis.

The aim of the present study was to investigate how the context in which an action is presented could modulate the effect of action observation on language processing, an effect that is classically observed in the literature. To address this question, we recorded both behavioral (reaction times) and electrophysiological measures (event-related potentials) of participants performing a semantic decision task involving a verb describing an action that was congruent or incongruent with the action presented in a prime picture that had been observed. The prime picture presented an action performed in a usual or an unusual context. The results revealed different behavioral and topographical pattern responses according to the context in which an action is presented. Importantly, only in the usual context, the congruency between the prime picture and the verb stimulus facilitated the semantic processes, leading to shorter response times in this condition compared to the others. Moreover, the topographic analysis revealed that this facilitation was related to reduced processing times for the semantic access to the verb and for the motor preparation for the answer. Taken together, these findings demonstrate that the context of an action is crucial in the link between action and language.

PLAViMoP: How to standardize and simplify the use of point-light displays.

The study of biological point-light displays (PLDs) has fascinated researchers for more than 40 years. However, the mechanisms underlying PLD perception remain unclear, partly due to difficulties with precisely controlling and transforming PLD sequences. Furthermore, little agreement exists regarding how transformations are performed. This article introduces a new free-access program called PLAViMoP (Point-Light Display Visualization and Modification Platform) and presents the algorithms for PLD transformations actually included in the software. PLAViMoP fulfills two objectives. First, it standardizes and makes clear many classical spatial and kinematic transformations described in the PLD literature. Furthermore, given its optimized interface, PLAViMOP makes these transformations easy and fast to achieve. Overall, PLAViMoP could directly help scientists avoid technical difficulties and make possible the use of PLDs for nonacademic applications.

Sentence plausibility influences the link between action words and the perception of biological human movements.

The present study aimed to assess the role of sentence plausibility in the functional link between action words and visual judgments of point-light human actions. Following the oral presentation of action verbs included in a plausible or implausible sentence, participants were asked to detect the presence of congruent or incongruent biological movements. Sentence plausibility was manipulated by inverting the positions of the subject and the complement (e.g., the neighbor is running in the garden vs the garden is running in the neighbor). The results showed that for both plausible and implausible sentences, the detection of human movements is greater following presentation of congruent action verbs. These results suggest that the presentation of action verbs affects the subsequent perception of point-light human movements, regardless of the associated semantic context. However, the link between action verbs and judgment of biological movements is strengthened when plausible sentences are presented, as illustrated by the increase in visual detection capacity in plausible congruent conditions. Concerning the analysis of the detection speed, the performance is only affected in plausible sentences with slower response times associated with the presentation of an incongruent action verb. These findings are discussed in light of an embodied mechanism and the domain of biological movement perception.

Sex Differences in the Neuromagnetic Cortical Response to Biological Motion.

Body motion is a rich source of information for social interaction, and visual biological motion processing may be considered as a hallmark of social cognition. It is unclear, however, whether the social brain is sex specific. Here we assess sex impact on the magnetoencephalographic (MEG) cortical response to point-light human locomotion. Sex differences in the cortical MEG response to biological motion occur mostly over the right brain hemisphere. At early latencies, females exhibit a greater activation than males over the right parietal, left temporal, and right temporal cortex, a core of the social brain. At later latencies, the boosts of activation are greater in males over the right frontal and occipital cortices. The findings deliver the first evidence for gender-dependent modes in the time course and topography of the neural circuitry underpinning visual processing of biological motion. The outcome represents a framework for studying sex differences in the social brain in psychiatric and neurodevelopmental disorders.

Are judgments for action verbs and point-light human actions equivalent?

The aim of the present study was to examine whether the ability to judge action words and the ability to judge human actions share common mechanisms. With this purpose in mind, we proposed both a lexical and an action decision task to twenty-four healthy participants. For both tasks, the participants had to judge whether the stimulus that was presented (a letter string or a point-light sequence) was valid or not (i.e. a word vs. a pseudo-word, an action vs. a pseudo-action). The data analysis showed that the action decision task has common characteristics with the lexical decision task. As for verbal material, judgements of pseudo-actions were slower than judgements for actions. Moreover, we demonstrated that the ability to judge an action verb was positively correlated with the ability to judge a point-light human action, whereas no significant correlation appeared between nouns and point-light judgements abilities. This dissociation supports the argument that the judgement of action words and the judgement of human actions share a common but specific basis through the involvement of motor representations.

Effect of gender matching and attentional focus on the link between action observation and action verb processing.

Several studies have demonstrated the existence of a link between action observation (AO) and language. However, the optimizing parameters for this link have not been explored until now. To answer this question, the present study proposed two experiments for assessing the role of motor repertory and attentional focus. Sixty participants performed a priming task in which they had to decide if a verb was or was not an action verb after they saw a point-light display (PLD) representing an action. Only one difference distinguished the experiments. In experiment one, the PLD was either in accordance or not with the gender kinematics of the observer, whereas in the second experiment, the PLD either focused on or was unfocused on the main limbs implied in the action. The results show that motor repertory affects the link between action observation and action verb processing, whereas attentional focus does not. Implications of these results are highlighted in the discussion, notably to better understand the mechanisms that explain the link between action observation and action verb processing.

The value of action observation in speech and language rehabilitation: A systematic review and meta-analysis.

In this systematic review and meta-analysis, our aim was to identify and quantify evidence of action observation therapy (AOT) efficacy in managing language deficits in patients with aphasia. This study conducts two quantitative investigations: firstly, comparing the effects of AOT and conventional control therapy in different groups, and secondly, analyzing within AOT group to explore potential moderators of AOT effectiveness. Four databases were searched up until August 2023 to find studies utilizing AOT for aphasia management. Seven eligible studies were included. The main analyses revealed moderate evidence of improvement in naming tasks, with a large effect size (Hedge's g = 1.27, 95 %CI [0.44; 2.09], p = 0.003, I2 < 25) following AOT compared to control interventions. Furthermore, to be efficient, AOT should focus on human actions (e.g., running, jumping) rather than non-human actions (e.g., meowing or barking). These findings indicate that AOT is a promising alternative complementary approach for patients with aphasia. Future research should confirm the potential benefits of AOT with more randomized controlled studies and aim to clarify the minimal dose necessary and the possibility of transfer to various language tasks.

Kinematic observation reduces effect of aging on episodic memory performance.

This study investigated the influence of kinematics observation (i.e., observing action from only the motion of the main joints of an actor) on episodic memory performance differences between young and older adults. To this end, 42 young (20-35 years) and 45 older (60-75 years) participants performed a free recall task in two different conditions: either after an encoding phase consisting of the visual presentation and reading of action verbs or after an encoding phase consisting of the visual presentation and naming of point-light displays of humans performing the same actions. Results showed a beneficial effect of point-light display encoding for both young and older participants but with a more pronounced benefit for the older participants compared to young adults. These findings are discussed in relation to the embodied view of memory which considers that memory is directly linked to the sensorimotor experiences and the environmental support hypothesis which postulates that elaborate processing can improve memory performance. In conclusion, kinematic observation could constitute an interesting potential intervention for supporting memory in older adults.

Action observation and motor learning: The role of action observation in learning judo techniques.

Within the theoretical framework of embodied cognition, several experiments have shown the existence of links between action observation and motor learning. Our aim was to assess the effectiveness of an observational learning protocol (action observation training: AOT) of point-light-display (PLD) in judoka. Twenty participants were given 7 days to learn Go-No-Sen. During this time period, all of the participants received conventional kata training consisting of Uchi-komi and Nage-komi (repetition of techniques) on tatami. In addition to this conventional learning, the experimental group watched 5 min of PLD video representing the different kata techniques, whereas the control group watched neutral videos during the same time period. After the learning period, both the qualitative and biomechanical performances on the kata and the transfer abilities were assessed. The results showed better biomechanical performance and transfer ability in the experimental group than in the control group. Therefore, this first experiment suggests that observational learning of PLD may be beneficial for the acquisition of judo techniques. Future experiments will be needed to specify the mechanisms that are involved in this effect.

The Link Between Action Verb Processing and Action Observation: A Developmental Study.

In this work we aimed to assess the typical link in human development between action observation and language. For this, we studied, in 68 children aged 5-11 years of age, how action verbs can prime action representation. While children 7-8 years of age benefited from a congruent action verb prime when they had to judge an image representing an action, this effect was not present in 5-6-year-olds. Thus, the link between language and action observation changes during development at about age 7. We discussed these findings in consideration of current theories proposed to account for the action-language link.

Short-term upper limb immobilisation impairs grasp representation.

The present experiment aimed to gain more information on the effect of limb nonuse on the cognitive level of actions and, more specifically, on the content of the motor programme used for grasping an object. For that purpose, we used a hand-grasping laterality task that is known to contain concrete information on manipulation activity. Two groups participated in the experiment: an immobilised group, including participants whose right hand and arm were fixed with a rigid splint and an immobilisation vest for 24 hr, and a control group, including participants who did not undergo the immobilisation procedure. The main results confirmed the slowdown of sensorimotor processes, which is highlighted in the literature, with slower response times when the participants identified the laterality of hand images that corresponded to the immobilised hand. Importantly, the grip-precision effect, highlighted by slower response times for hands grasping a small sphere versus a large sphere, is impaired by 24 hr of limb nonuse. Overall, this study provided additional evidence of the disengagement of sensorimotor processes due to a short period of limb immobilisation.