The effect of gamified robot-enhanced training on motor performance in chronic stroke survivors.

Task-specific training constitutes a core element for evidence-based rehabilitation strategies targeted at improving upper extremity activity after stroke. Its combination with additional treatment strategies and neurotechnology-based solutions could further improve patients' outcomes. Here, we studied the effect of gamified robot-assisted upper limb motor training on motor performance, skill learning, and transfer with respect to a non-gamified control condition with a group of chronic stroke survivors. The results suggest that a gamified training strategy results in more controlled motor performance during the training phase, which is characterized by a higher accuracy (lower deviance), higher smoothness (lower jerk), but slower speed. The responder analyses indicated that mildly impaired patients benefited most from the gamification approach. In conclusion, gamified robot-assisted motor training, which is personalized to the individual capabilities of a patient, constitutes a promising investigational strategy for further improving motor performance after a stroke.

Envisioning social drones in education.

Education is one of the major application fields in social Human-Robot Interaction. Several forms of social robots have been explored to engage and assist students in the classroom environment, from full-bodied humanoid robots to tabletop robot companions, but flying robots have been left unexplored in this context. In this paper, we present seven online remote workshops conducted with 20 participants to investigate the application area of Education in the Human-Drone Interaction domain; particularly focusing on what roles a social drone could fulfill in a classroom, how it would interact with students, teachers and its environment, what it could look like, and what would specifically differ from other types of social robots used in education. In the workshops we used online collaboration tools, supported by a sketch artist, to help envision a social drone in a classroom. The results revealed several design implications for the roles and capabilities of a social drone, in addition to promising research directions for the development and design in the novel area of drones in education.

10 Years of Human-NAO Interaction Research: A Scoping Review.

The evolving field of human-robot interaction (HRI) necessitates that we better understand how social robots operate and interact with humans. This scoping review provides an overview of about 300 research works focusing on the use of the NAO robot from 2010 to 2020. This study presents one of the most extensive and inclusive pieces of evidence on the deployment of the humanoid NAO robot and its global reach. Unlike most reviews, we provide both qualitative and quantitative results regarding how NAO is being used and what has been achieved so far. We analyzed a wide range of theoretical, empirical, and technical contributions that provide multidimensional insights, such as general trends in terms of application, the robot capabilities, its input and output modalities of communication, and the human-robot interaction experiments that featured NAO (e.g. number and roles of participants, design, and the length of interaction). Lastly, we derive from the review some research gaps in current state-of-the-art and provide suggestions for the design of the next generation of social robots.

"It Is Not the Robot Who Learns, It Is Me." Treating Severe Dysgraphia Using Child-Robot Interaction.

Writing disorders are frequent and impairing. However, social robots may help to improve children's motivation and to propose enjoyable and tailored activities. Here, we have used the Co-writer scenario in which a child is asked to teach a robot how to write via demonstration on a tablet, combined with a series of games we developed to train specifically pressure, tilt, speed, and letter liaison controls. This setup was proposed to a 10-year-old boy with a complex neurodevelopmental disorder combining phonological disorder, attention deficit/hyperactivity disorder, dyslexia, and developmental coordination disorder with severe dysgraphia. Writing impairments were severe and limited his participation in classroom activities despite 2 years of specific support in school and professional speech and motor remediation. We implemented the setup during his occupational therapy for 20 consecutive weekly sessions. We found that his motivation was restored; avoidance behaviors disappeared both during sessions and at school; handwriting quality and posture improved dramatically. In conclusion, treating dysgraphia using child-robot interaction is feasible and improves writing. Larger clinical studies are required to confirm that children with dysgraphia could benefit from this setup.

The transferability of handwriting skills: from the Cyrillic to the Latin alphabet.

Do handwriting skills transfer when a child writes in two different scripts, such as the Latin and Cyrillic alphabets? Are our measures of handwriting skills intrinsically bound to one alphabet or will a child who faces handwriting difficulties in one script experience similar difficulties in the other script? To answer these questions, 190 children from grades 1-4 were asked to copy a short text using both the Cyrillic and Latin alphabets on a digital tablet. A recent change of policy in Kazakhstan gave us an opportunity to measure transfer, as the Latin-based Kazakh alphabet has not yet been introduced. Therefore, pupils in grade 1 had a 6-months experience in Cyrillic, and pupils in grades 2, 3, and 4 had 1.5, 2.5, and 3.5 years of experience in Cyrillic, respectively. This unique situation created a quasi-experimental situation that allowed us to measure the influence of the number of years spent practicing Cyrillic on the quality of handwriting in the Latin alphabet. The results showed that some of the differences between the two scripts were constant across all grades. These differences thus reflect the intrinsic differences in the handwriting dynamics between the two alphabets. For instance, several features related to the pen pressure on the tablet are quite different. Other features, however, revealed decreasing differences between the two scripts across grades. While we found that the quality of Cyrillic writing increased from grades 1-4, due to increased practice, we also found that the quality of the Latin writing increased as well, despite the fact that all of the pupils had the same absence of experience in writing in Latin. We can therefore interpret this improvement in Latin script as an indicator of the transfer of fine motor control skills from Cyrillic to Latin. This result is especially surprising given that one could instead hypothesize a negative transfer, i.e., that the finger controls automated for one alphabet would interfere with those required by the other alphabet. One interesting side-effect of these findings is that the algorithms that we developed for the diagnosis of handwriting difficulties among French-speaking children could be relevant for other alphabets, paving the way for the creation of a cross-lingual model for the detection of handwriting difficulties.

Acquisition of handwriting in children with and without dysgraphia: A computational approach.

Handwriting is a complex skill to acquire and it requires years of training to be mastered. Children presenting dysgraphia exhibit difficulties automatizing their handwriting. This can bring anxiety and can negatively impact education. 280 children were recruited in schools and specialized clinics to perform the Concise Evaluation Scale for Children's Handwriting (BHK) on digital tablets. Within this dataset, we identified children with dysgraphia. Twelve digital features describing handwriting through different aspects (static, kinematic, pressure and tilt) were extracted and used to create linear models to investigate handwriting acquisition throughout education. K-means clustering was performed to define a new classification of dysgraphia. Linear models show that three features only (two kinematic and one static) showed a significant association to predict change of handwriting quality in control children. Most kinematic and statics features interacted with age. Results suggest that children with dysgraphia do not simply differ from ones without dysgraphia by quantitative differences on the BHK scale but present a different development in terms of static, kinematic, pressure and tilt features. The K-means clustering yielded 3 clusters (Ci). Children in C1 presented mild dysgraphia usually not detected in schools whereas children in C2 and C3 exhibited severe dysgraphia. Notably, C2 contained individuals displaying abnormalities in term of kinematics and pressure whilst C3 regrouped children showing mainly tilt problems. The current results open new opportunities for automatic detection of children with dysgraphia in classroom. We also believe that the training of pressure and tilt may open new therapeutic opportunities through serious games.

Gamified Motor Training With Tangible Robots in Older Adults: A Feasibility Study and Comparison With the Young.

Background: An increasing lifespan and the resulting change in our expectations of later life stages are dependent on a good health state. This emphasizes the importance of the development of strategies to further strengthen healthy aging. One important aspect of good health in later life stages is sustained skilled motor function. Objective: Here, we tested the effectiveness of robotic upper limb motor training in a game-like scenario assessing game-based learning and its transfer potential. Methods: Thirty-six healthy participants (n = 18 elderly participants, n = 18 young controls) trained with a Pacman-like game using a hand-held Cellulo robot on 2 consecutive days. The game-related movements were conducted on a printed map displaying a maze and targets that had to be collected. Gradually, the task difficulty was adjusted between games by modifying or adding different game elements (e.g., speed and number of chasing ghosts, additional rules, and haptic feedback). Transfer was assessed by scoring simple robot manipulation on two different trajectories. Results: Elderly participants were able to improve their game performance over time [t (874) = 2.97, p < 0.01]. The applied game elements had similar effects on both age groups. Importantly, the game-based learning was transferable to simple robot manipulation that resembles activities of daily life. Only minor age-related differences were present (smaller overall learning gain and different effects of the wall-crash penalty rule in the elderly group). Conclusions: Gamified motor training with the Cellulo system has the potential to translate into an efficient and relatively low-cost robotic motor training tool for promoting upper limb function to promote healthy aging.

Iterative Design and Evaluation of a Tangible Robot-Assisted Handwriting Activity for Special Education.

In this article we investigate the role of interactive haptic-enabled tangible robots in supporting the learning of cursive letter writing for children with attention and visuomotor coordination issues. We focus on the two principal aspects of handwriting that are linked to these issues: Visual perception and visuomotor coordination. These aspects, respectively, enhance two features of letter representation in the learner's mind in particular, namely the shape (grapheme) and the dynamics (ductus) of the letter, which constitute the central learning goals in our activity. Building upon an initial design tested with 17 healthy children in a preliminary school, we iteratively ported the activity to an occupational therapy context in 2 different therapy centers, in the context of 3 different summer school camps involving a total of 12 children having writing difficulties. The various iterations allowed us to uncover insights about the design of robot-enhanced writing activities for special education, specifically highlighting the importance of ease of modification of the duration of an activity as well as of adaptable frequency, content, flow and game-play and of providing a range of evaluation test alternatives. Results show that the use of robot-assisted handwriting activities could have a positive impact on the learning of the representation of letters in the context of occupational therapy (V = 1, 449, p < 0.001, r = 0.42). Results also highlight how the design changes made across the iterations affected the outcomes of the handwriting sessions, such as the evaluation of the performances, monitoring of the performances, and the connectedness of the handwriting.

A Comparison of Social Robot to Tablet and Teacher in a New Script Learning Context.

This research occurred in a special context where Kazakhstan's recent decision to switch from Cyrillic to the Latin-based alphabet has resulted in challenges connected to teaching literacy, addressing a rare combination of research hypotheses and technical objectives about language learning. Teachers are not necessarily trained to teach the new alphabet, and this could result in a challenge for children with learning difficulties. Prior research studies in Human-Robot Interaction (HRI) have proposed the use of a robot to teach handwriting to children (Hood et al., 2015; Lemaignan et al., 2016). Drawing on the Kazakhstani case, our study takes an interdisciplinary approach by bringing together smart solutions from robotics, computer vision areas, and educational frameworks, language, and cognitive studies that will benefit diverse groups of stakeholders. In this study, a human-robot interaction application is designed to help primary school children learn both a newly-adopted script and also its handwriting system. The setup involved an experiment with 62 children between the ages of 7-9 years old, across three conditions: a robot and a tablet, a tablet only, and a teacher. Based on the paradigm-learning by teaching-the study showed that children improved their knowledge of the Latin script by interacting with a robot. Findings reported that children gained similar knowledge of a new script in all three conditions without gender effect. In addition, children's likeability ratings and positive mood change scores demonstrate significant benefits favoring the robot over a traditional teacher and tablet only approaches.

Towards an Adaptive Upper Limb Rehabilitation Game with Tangible Robots.

A key feature of a successful game is its ability to provide the player with an adequate level of challenge. However, the objective of difficulty adaptation in serious games is not only to maintain the player's motivation by challenging, but also to ensure the completion of training objectives.This paper describes our proposed upper-limb rehabilitation game with tangible robots and investigates the effect of game elements and gameplay on the amount of the performed motion in several planes and percentage of failure by using the data from 33 unimpaired subjects who played 53 games within two consecutive days. In order to provide a more generic adaptation strategy in the future, we discretize the game area to circular zones. We then show the effect of changing these zones during gameplay on the activation of different muscles through EMG data in a pilot study.The study shows that it is possible to increase the challenge level by adding more active agents chasing the player and increasing the speed of these agents. However, only the increase in number of agents significantly increases the users' motion on both planes. Analysis of player behaviors leads us to suggest that by adapting the behaviour of these active agents in specific zones, it is possible to change the trajectory of the user, and to provide a focus on the activation of specific muscles.

Designing Configurable Arm Rehabilitation Games: How Do Different Game Elements Affect User Motion Trajectories?

For successful rehabilitation of a patient after a stroke or traumatic brain injury, it is crucial that rehabilitation activities are motivating, provide feedback and have a high rate of repetitions. Advancements in recent technologies provide solutions to address these aspects where needed. Additionally, through the use of gamification, we are able to increase the motivation for participants. However, many of these systems require complex set-ups, which can be a big challenge when conducting rehabilitation in a home-based setting. To address the lack of simple rehabilitation tools for arm function for a home-based application, we previously developed a system, Cellulo for rehabilitation, that is comprised of paper-supported tangible robots that are orchestrated by applications deployed on consumer tablets. These components enable different features that allow for gamification, easy setup, portability, and scalability. To support the configuration of game elements to patients' level of motor skills and strategies, their motor trajectories need to be classified. In this paper, we investigate the classification of different motor trajectories and how game elements impact these in unimpaired, healthy participants. We show that the manipulation of certain game elements do have an impact on motor trajectories, which might indicate that it is possible to adapt the arm remediation of patients by configuring game elements. These results provide a first step towards providing adaptive rehabilitation based upon patients' measured trajectories.

Automated human-level diagnosis of dysgraphia using a consumer tablet.

The academic and behavioral progress of children is associated with the timely development of reading and writing skills. Dysgraphia, characterized as a handwriting learning disability, is usually associated with dyslexia, developmental coordination disorder (dyspraxia), or attention deficit disorder, which are all neuro-developmental disorders. Dysgraphia can seriously impair children in their everyday life and require therapeutic care. Early detection of handwriting difficulties is, therefore, of great importance in pediatrics. Since the beginning of the 20th century, numerous handwriting scales have been developed to assess the quality of handwriting. However, these tests usually involve an expert investigating visually sentences written by a subject on paper, and, therefore, they are subjective, expensive, and scale poorly. Moreover, they ignore potentially important characteristics of motor control such as writing dynamics, pen pressure, or pen tilt. However, with the increasing availability of digital tablets, features to measure these ignored characteristics are now potentially available at scale and very low cost. In this work, we developed a diagnostic tool requiring only a commodity tablet. To this end, we modeled data of 298 children, including 56 with dysgraphia. Children performed the BHK test on a digital tablet covered with a sheet of paper. We extracted 53 handwriting features describing various aspects of handwriting, and used the Random Forest classifier to diagnose dysgraphia. Our method achieved 96.6% sensibility and 99.2% specificity. Given the intra-rater and inter-rater levels of agreement in the BHK test, our technique has comparable accuracy for experts and can be deployed directly as a diagnostics tool.