SmartDetector: Automatic and vision-based approach to point-light display generation for human action perception.

Over the past four decades, point-light displays (PLD) have been integrated into psychology and psychophysics, providing a valuable means to probe human perceptual skills. Leveraging the inherent kinematic information and controllable display parameters, researchers have utilized this technique to examine the mechanisms involved in learning and rehabilitation. However, classical PLD generation methods (e.g., motion capture) are difficult to apply for behavior analysis in real-world situations, such as patient care or sports activities. Therefore, there is a demand for automated and affordable tools that enable efficient and real-world-compatible generation of PLDs for psychological research. In this paper, we propose SmartDetector, a new artificial intelligence (AI)-based tool for automatic PLD creation from RGB videos. To evaluate humans' perceptual skills for processing PLD building with SmartDetector, 126 participants were randomly assigned to recognition, discrimination, or detection tasks. Results demonstrated that, irrespective of the task, PLDs generated by SmartDetector exhibited commendable perceptual performance in terms of accuracy and response times compared to literature findings. Moreover, to enhance usability and broaden accessibility, we developed an intuitive web interface for our method, making it available to a wider audience. The resulting application is available at https://plavimop.prd.fr/index.php/en/automatic-creation-pld . SmartDetector offers interesting possibilities for using PLD in research and makes the use of PLD more accessible for nonacademic applications.

The value of action observation in speech and language rehabilitation: A systematic review and meta-analysis.

In this systematic review and meta-analysis, our aim was to identify and quantify evidence of action observation therapy (AOT) efficacy in managing language deficits in patients with aphasia. This study conducts two quantitative investigations: firstly, comparing the effects of AOT and conventional control therapy in different groups, and secondly, analyzing within AOT group to explore potential moderators of AOT effectiveness. Four databases were searched up until August 2023 to find studies utilizing AOT for aphasia management. Seven eligible studies were included. The main analyses revealed moderate evidence of improvement in naming tasks, with a large effect size (Hedge's g = 1.27, 95 %CI [0.44; 2.09], p = 0.003, I2 < 25) following AOT compared to control interventions. Furthermore, to be efficient, AOT should focus on human actions (e.g., running, jumping) rather than non-human actions (e.g., meowing or barking). These findings indicate that AOT is a promising alternative complementary approach for patients with aphasia. Future research should confirm the potential benefits of AOT with more randomized controlled studies and aim to clarify the minimal dose necessary and the possibility of transfer to various language tasks.

Kinematic observation reduces effect of aging on episodic memory performance.

This study investigated the influence of kinematics observation (i.e., observing action from only the motion of the main joints of an actor) on episodic memory performance differences between young and older adults. To this end, 42 young (20-35 years) and 45 older (60-75 years) participants performed a free recall task in two different conditions: either after an encoding phase consisting of the visual presentation and reading of action verbs or after an encoding phase consisting of the visual presentation and naming of point-light displays of humans performing the same actions. Results showed a beneficial effect of point-light display encoding for both young and older participants but with a more pronounced benefit for the older participants compared to young adults. These findings are discussed in relation to the embodied view of memory which considers that memory is directly linked to the sensorimotor experiences and the environmental support hypothesis which postulates that elaborate processing can improve memory performance. In conclusion, kinematic observation could constitute an interesting potential intervention for supporting memory in older adults.

Effects of short-term hand immobilization on anticipatory mechanism for tool use.

The short-term immobilization of a limb such as the right arm can impair sensorimotor mechanisms, which in turn reduces motor control of this arm. However, it is not known whether immobilization also impairs the anticipatory mechanism for tool use without actual enactment. In two experiments, we asked participants to judge how they would use a tool in a particular environment (e.g., "Take the pencil to write on a sheet of paper"). Prior to this tool-use judgment task, some participants had been immobilized (right arm) for 24 h. Results revealed that compared to controls, immobilized participants performed more poorly on the tool-use judgment task (accuracy and response time) as well as in a manual dexterity task. As our tool-use judgment task involved anticipating the expected perceptual effect of using a tool to achieve an environmental goal (e.g., writing on a sheet of paper), our data are discussed in line with theories of motor control (e.g., ideomotor theory) that emphasize the expected perceptual consequences of the action.

Point-light display: a new tool to improve verb recovery in patients with aphasia? A pilot study.

Some studies have demonstrated that Action Observation (AO) could help patients with aphasia to recover use of verbs. However, the role of kinematics in this effect has remained unknown. The main aim was to assess the effectiveness of a complementary intervention based on the observation of action kinematics in patients with aphasia. Seven aphasic patients (3 males, 4 females) aged between 55 and 88 years participated in the studies. All patients received a classical intervention and an additional, specific intervention based on action observation. This consisted in visualizing a static image or a point-light sequence representing a human action and in trying to name the verb representing the action. In each session, 57 actions were visualized: 19 represented by a static drawing, 19 by a non-focalized point-light sequence, i.e., a point-light display with all dots in white, and 19 by a focalized point-light sequence, i.e., a point-light display (PLD) with the dots corresponding to the main limbs in yellow. Before (pre-test) and after (post-test) the intervention, each patient performed the same denomination task, in which all actions were presented in photographs. The results showed a significant improvement in performance between pre and post-test, but only when the actions were presented in focalized and non-focalized point-light sequences during the intervention. The presentation of action kinematics seems crucial in the recovery of verbs in patients with aphasia. This should be considered by speech therapists in their interventions.

Action observation and motor learning: The role of action observation in learning judo techniques.

Within the theoretical framework of embodied cognition, several experiments have shown the existence of links between action observation and motor learning. Our aim was to assess the effectiveness of an observational learning protocol (action observation training: AOT) of point-light-display (PLD) in judoka. Twenty participants were given 7 days to learn Go-No-Sen. During this time period, all of the participants received conventional kata training consisting of Uchi-komi and Nage-komi (repetition of techniques) on tatami. In addition to this conventional learning, the experimental group watched 5 min of PLD video representing the different kata techniques, whereas the control group watched neutral videos during the same time period. After the learning period, both the qualitative and biomechanical performances on the kata and the transfer abilities were assessed. The results showed better biomechanical performance and transfer ability in the experimental group than in the control group. Therefore, this first experiment suggests that observational learning of PLD may be beneficial for the acquisition of judo techniques. Future experiments will be needed to specify the mechanisms that are involved in this effect.

PLAViMoP database: A new continuously assessed and collaborative 3D point-light display dataset.

It was more than 45 years ago that Gunnar Johansson invented the point-light display technique. This showed for the first time that kinematics is crucial for action recognition, and that humans are very sensitive to their conspecifics' movements. As a result, many of today's researchers use point-light displays to better understand the mechanisms behind this recognition ability. In this paper, we propose PLAViMoP, a new database of 3D point-light displays representing everyday human actions (global and fine-motor control movements), sports movements, facial expressions, interactions, and robotic movements. Access to the database is free, at https://plavimop.prd.fr/en/motions . Moreover, it incorporates a search engine to facilitate action retrieval. In this paper, we describe the construction, functioning, and assessment of the PLAViMoP database. Each sequence was analyzed according to four parameters: type of movement, movement label, sex of the actor, and age of the actor. We provide both the mean scores for each assessment of each point-light display, and the comparisons between the different categories of sequences. Our results are discussed in the light of the literature and the suitability of our stimuli for research and applications.

The role of implicit motor simulation on action verb memory.

Observation is known to improve memory for action. Previous findings linked such an effect with an easier relation processing of action components following observation compared to mere sentence reading. However, action observation also elicits implicit motor simulation, that is a processing of one's movement through the observer own motor system. We aimed to assess whether encoding of implicit motor simulation can also explain why observation is better than reading for action memory. To prevent influence of item relation processing, two studies about isolated action verbs learning were designed. In Experiment 1, action verbs were encoded with short videos of point-light human movements or with written definitions. Subsequent free recall indicated better memory for the verbs within the video clip condition. Experiment 2 compared two encoding conditions based on point-light human movement videos. Half of the verbs were learned with their normal corresponding movement (biological kinematic). For the other half of the verbs, the velocity of point-light movements was modified to create abnormal nonbiological kinematic actions. We observed better free recall for the verbs learned with biological kinematics. Taken together, those results suggest that action observation is beneficial because it allows the encoding of motor-related information (implicit motor simulation). Semantic resonance between linguistic and motor representations of action could also contribute to memory improvement. Contrary to previous studies, our results cannot be explained by an improvement of items relation processing. However, it suggests that the basic level of action verb memory is sensorimotor perception, such as implicit motor simulation.

The Added Value of Point-Light Display Observation in Total Knee Arthroplasty Rehabilitation Program: A Prospective Randomized Controlled Pilot Study.

Background and Objectives: The present study aimed to assess the potential benefit of the observation of rehabilitation-related point-light display in addition to a conventional 3-week rehabilitation program, the objective being to improve functional capacity in patients having undergone total knee arthroplasty. Materials and Methods: Patients randomized in the control group had conventional rehabilitation treatment with two sessions per day 5 days a week of physical therapy (90 min), whereas patients in the experimental group had a program of conventional rehabilitation combined with a point-light display observation two times per day (5 min) and 3 days a week. Results: The patients of both groups had improved their performances by the end of the program, and the pre- and post-test improvement were superior for the experimental group over the control group concerning the total WOMAC score (p = 0.04), the functional WOMAC score (p = 0.03), and correct recognition of point-light displays (p = 0.003). Conclusions: These findings provide new insight favoring systematic point-light display observation to improve functional recovery in patients with total knee arthroplasty.

The Link Between Action Verb Processing and Action Observation: A Developmental Study.

In this work we aimed to assess the typical link in human development between action observation and language. For this, we studied, in 68 children aged 5-11 years of age, how action verbs can prime action representation. While children 7-8 years of age benefited from a congruent action verb prime when they had to judge an image representing an action, this effect was not present in 5-6-year-olds. Thus, the link between language and action observation changes during development at about age 7. We discussed these findings in consideration of current theories proposed to account for the action-language link.

Short-term upper limb immobilization and the embodied view of memory: A pilot study.

The present study aimed to explore the contribution of the manual sensorimotor system to the memory of graspable objects. Participants in the experimental group underwent a short-term upper limb immobilization design to decrease arousal to their dominant hand. Such designs are known to elicit updating of sensorimotor representations and to hardened use of implicit motor simulation, a process that occurs when observing graspable objects. Subsequently, a free recall and a recognition task of graspable and non-graspable objects took place. We found slower recognition for graspable than for non-graspable objects in the control group, while no differences appeared for the immobilized group. Moreover, the recognition latency for graspable objects tended to be slower for the control than for the immobilized group. These results suggest that a time demanding reactivation of motor simulation is elicited when a graspable object is correctly recognized by control participants. The effect of immobilization could prevent this reactivation, leading to faster recognition. Hence, immobilization selectively affects graspable object memory, showing a close relationship with the manual sphere of the sensorimotor system. We suggest that recognition accuracy would probably be affected in cases of stronger disruption of sensorimotor arousal.

Short-term upper limb immobilisation impairs grasp representation.

The present experiment aimed to gain more information on the effect of limb nonuse on the cognitive level of actions and, more specifically, on the content of the motor programme used for grasping an object. For that purpose, we used a hand-grasping laterality task that is known to contain concrete information on manipulation activity. Two groups participated in the experiment: an immobilised group, including participants whose right hand and arm were fixed with a rigid splint and an immobilisation vest for 24 hr, and a control group, including participants who did not undergo the immobilisation procedure. The main results confirmed the slowdown of sensorimotor processes, which is highlighted in the literature, with slower response times when the participants identified the laterality of hand images that corresponded to the immobilised hand. Importantly, the grip-precision effect, highlighted by slower response times for hands grasping a small sphere versus a large sphere, is impaired by 24 hr of limb nonuse. Overall, this study provided additional evidence of the disengagement of sensorimotor processes due to a short period of limb immobilisation.

Recognition of Emotions From Facial Point-Light Displays.

Facial emotion recognition occupies a prominent place in emotion psychology. How perceivers recognize messages conveyed by faces can be studied in either an explicit or an implicit way, and using different kinds of facial stimuli. In the present study, we explored for the first time how facial point-light displays (PLDs) (i.e., biological motion with minimal perceptual properties) can elicit both explicit and implicit mechanisms of facial emotion recognition. Participants completed tasks of explicit or implicit facial emotion recognition from PLDs. Results showed that point-light stimuli are sufficient to allow facial emotion recognition, be it explicit and implicit. We argue that this finding could encourage the use of PLDs in research on the perception of emotional cues from faces.

The link between language and action in aging.

OBJECTIVE: Many studies have demonstrated the existence of a link between action verb processing and action. However, little is known about the changes in this relationship with aging. METHOD: To assess this point, we compare the performances of younger and older people during a priming task consisting of judging whether an image contains a human after listening to an action verb. RESULTS: In accordance with previous literature, the results showed that younger people were faster to detect the presence of a human in the image in congruent conditions, namely, when the action verb and the image refer to the same action. However, this effect was not present in older adults' participants. CONCLUSION: These findings suggest that the link between action and language decreases with age. We discuss these findings in the context of the embodied view of cognition.

Physical and observational practices of unusual actions prime action verb processing.

Numerous studies have highlighted a strong relationship between language and sensorimotor processes, showing, for example, that perceiving an action influences subsequent language processing. Moreover, previous studies have demonstrated that the context in which actions are perceived is crucial to enable this action-language relationship. In particular, action verb processing is facilitated when an action is perceived in its usual context (e.g., someone watering a plant) but not in an unusual context (e.g., someone watering a computer). This difference could be explained in terms of experience; because people always practice actions in accordance with the context, they have no (visual or motor) experience related to the unusual context. The aim of the present study was to test this assumption by assessing and comparing the effect of physical practice and observational learning on the action-language relationship. The results of two experiments showed a facilitation effect of both training methods. Whereas usual actions systematically prime action verb processing, the link between action and language appears for unusual actions only after training by practicing (experiment 1, physical practice) or observing (experiment 2, observational learning). Overall, these findings support the role of experience in the activation of sensorimotor representations during action verb processing.

Short-term upper-limb immobilization alters peripersonal space representation.

Peripersonal space is a multisensory interface between the environment and the body subserving motor interactions with the physical and social world. Although changing body properties has been shown to alter the functional processing of space, little is known about the effect of short-term limb immobilization specifically on the motor representation of peripersonal space. In the present study, we investigated the effect of a right upper-limb immobilization for a duration of 24 h on a reachability judgment task and a brightness judgment task. Analyses of perceptual thresholds revealed a reduction of peripersonal space representation after the immobilization period, which was not observed when there was no immobilization (control group). In contrast, no variation appeared in the brightness judgment task, suggesting no presence of specific visual perception or decisional deficits in the limb immobilization group. Considered together, the results confirm the crucial role of the motor system in the representation of peripersonal space. They also highlight the plasticity of the motor system resulting in a rapid change of its activity following limb immobilization, with a concomitant effect on motor-related perceptual and cognitive processes.

Does watching Han Solo or C-3PO similarly influence our language processing?

Several studies have demonstrated that perceiving an action influences the subsequent processing of action verbs. However, which characteristics of the perceived action are truly determinant to enable this influence is still unknown. The current study investigated the role of the agent executing an action in this action-language relationship. Participants performed a semantic decision task after seeing a video of a human or a robot performing an action. The results of the first study showed that perceiving a human being executing an action as well as perceiving a robot facilitate subsequent language processing, suggesting that the humanness (The term "humanness" is used as meaning "belonging to human race" and not to refer to a personal quality) of the agent is not crucial in the link between action and language. However, this experiment was conducted with Japanese people who are very familiar with robots; thus, an alternative explanation could be that it is the unfamiliarity with the agent that could perturb the action-language relationship. To assess this hypothesis, we carried out two additional experiments with French participants. The results of the second study showed that, unlike the observation of a human agent, the observation of a robot did not influence language processing. Finally, the results of the third study showed that, after a familiarization phase, French participants too were influenced by the observation of a robot. Overall, the outcomes of these studies indicate that, more than the humanness of the agent, it is the familiarity which we have with this agent that is crucial in the action-language relationship.

How Action Context Modulates the Action-Language Relationship: A Topographic ERP Analysis.

The aim of the present study was to investigate how the context in which an action is presented could modulate the effect of action observation on language processing, an effect that is classically observed in the literature. To address this question, we recorded both behavioral (reaction times) and electrophysiological measures (event-related potentials) of participants performing a semantic decision task involving a verb describing an action that was congruent or incongruent with the action presented in a prime picture that had been observed. The prime picture presented an action performed in a usual or an unusual context. The results revealed different behavioral and topographical pattern responses according to the context in which an action is presented. Importantly, only in the usual context, the congruency between the prime picture and the verb stimulus facilitated the semantic processes, leading to shorter response times in this condition compared to the others. Moreover, the topographic analysis revealed that this facilitation was related to reduced processing times for the semantic access to the verb and for the motor preparation for the answer. Taken together, these findings demonstrate that the context of an action is crucial in the link between action and language.

PLAViMoP: How to standardize and simplify the use of point-light displays.

The study of biological point-light displays (PLDs) has fascinated researchers for more than 40 years. However, the mechanisms underlying PLD perception remain unclear, partly due to difficulties with precisely controlling and transforming PLD sequences. Furthermore, little agreement exists regarding how transformations are performed. This article introduces a new free-access program called PLAViMoP (Point-Light Display Visualization and Modification Platform) and presents the algorithms for PLD transformations actually included in the software. PLAViMoP fulfills two objectives. First, it standardizes and makes clear many classical spatial and kinematic transformations described in the PLD literature. Furthermore, given its optimized interface, PLAViMOP makes these transformations easy and fast to achieve. Overall, PLAViMoP could directly help scientists avoid technical difficulties and make possible the use of PLDs for nonacademic applications.

The kinematics, not the orientation, of an action influences language processing.

Consistent with the embodied view of cognition, several studies have shown a link between action and the processing of action verbs. However, it is largely unknown how action properties can influence semantic activation during word processing. On the basis of the observation of point-light display (PLDs), the present study addressed this issue. Through four experiments, we assessed whether kinematics and orientation, which are two crucial characteristics of human action, can influence the link between action and language. Participants performed a semantic decision task involving action and nonaction verbs after seeing a PLD representing either a biological or a nonbiological human movement, the kinematics (Experiments 1 and 2) or orientation (Experiment 3) of which was modified. Experiment 1 showed that important modifications to the kinematics of actions have a direct influence on the link between action and language. Experiment 2 confirmed this effect and showed that the effect is somatotopic because only modifications to the relevant part of actions affected the link between action and language. In contrast, Experiment 3 showed that modifying the orientation did not disturb the influence of an action on language probably due to the use of a mental rotation strategy. Experiment 4 confirmed this view by demonstrating the use of mental rotation when participants have to recognize rotated point-light actions. These outcomes shed light on how action properties can influence action verbs processing, a crucial step to better understanding the link between action and language. (PsycINFO Database Record (c) 2018 APA, all rights reserved).