Social robots as skilled ignorant peers for supporting learning.

When designing social robots for educational settings, there is often an emphasis on domain knowledge. This presents challenges: 1) Either robots must autonomously acquire domain knowledge, a currently unsolved problem in HRI, or 2) the designers provide this knowledge implying re-programming the robot for new contexts. Recent research explores alternative, relatively easier to port, knowledge areas like student rapport, engagement, and synchrony though these constructs are typically treated as the ultimate goals, when the final goal should be students' learning. Our aim is to propose a shift in how engagement is considered, aligning it naturally with learning. We introduce the notion of a skilled ignorant peer robot: a robot peer that has little to no domain knowledge but possesses knowledge of student behaviours conducive to learning, i.e., behaviours indicative of productive engagement as extracted from student behavioral profiles. We formally investigate how such a robot's interventions manipulate the children's engagement conducive to learning. Specifically, we evaluate two versions of the proposed robot, namely, Harry and Hermione, in a user study with 136 students where each version differs in terms of the intervention strategy. Harry focuses on which suggestions to intervene with from a pool of communication, exploration, and reflection inducing suggestions, while Hermione also carefully considers when and why to intervene. While the teams interacting with Harry have higher productive engagement correlated to learning, this engagement is not affected by the robot's intervention scheme. In contrast, Hermione's well-timed interventions, deemed more useful, correlate with productive engagement though engagement is not correlated to learning. These results highlight the potential of a social educational robot as a skilled ignorant peer and stress the importance of precisely timing the robot interventions in a learning environment to be able to manipulate moderating variable of interest such as productive engagement.

The effect of gamified robot-enhanced training on motor performance in chronic stroke survivors.

Task-specific training constitutes a core element for evidence-based rehabilitation strategies targeted at improving upper extremity activity after stroke. Its combination with additional treatment strategies and neurotechnology-based solutions could further improve patients' outcomes. Here, we studied the effect of gamified robot-assisted upper limb motor training on motor performance, skill learning, and transfer with respect to a non-gamified control condition with a group of chronic stroke survivors. The results suggest that a gamified training strategy results in more controlled motor performance during the training phase, which is characterized by a higher accuracy (lower deviance), higher smoothness (lower jerk), but slower speed. The responder analyses indicated that mildly impaired patients benefited most from the gamification approach. In conclusion, gamified robot-assisted motor training, which is personalized to the individual capabilities of a patient, constitutes a promising investigational strategy for further improving motor performance after a stroke.

Predictive Analysis of Errors During Robot-Mediated Gamified Training.

This paper presents our approach to predicting future error-related events in a robot-mediated gamified physical training activity for stroke patients. The ability to predict future error under such conditions suggests the existence of distinguishable features and separated class characteristics between the casual gameplay state and error prune state in the data. Identifying such features provides valuable insight to creating individually tailored, adaptive games as well as possible ways to increase rehabilitation success by patients. Considering the time-series nature of sensory data created by motor actions of patients we employed a predictive analysis strategy on carefully engineered features of sequenced data. We split the data into fixed time windows and explored logistic regression models, decision trees, and recurrent neural networks to predict the likelihood of a patient making an error based on the features from the time window before the error. We achieved an 84.4% F1-score with a 0.76 ROC value in our best model for predicting motion accuracy related errors. Moreover, we computed the permutation importance of the features to explain which ones are more indicative of future errors.

"It Is Not the Robot Who Learns, It Is Me." Treating Severe Dysgraphia Using Child-Robot Interaction.

Writing disorders are frequent and impairing. However, social robots may help to improve children's motivation and to propose enjoyable and tailored activities. Here, we have used the Co-writer scenario in which a child is asked to teach a robot how to write via demonstration on a tablet, combined with a series of games we developed to train specifically pressure, tilt, speed, and letter liaison controls. This setup was proposed to a 10-year-old boy with a complex neurodevelopmental disorder combining phonological disorder, attention deficit/hyperactivity disorder, dyslexia, and developmental coordination disorder with severe dysgraphia. Writing impairments were severe and limited his participation in classroom activities despite 2 years of specific support in school and professional speech and motor remediation. We implemented the setup during his occupational therapy for 20 consecutive weekly sessions. We found that his motivation was restored; avoidance behaviors disappeared both during sessions and at school; handwriting quality and posture improved dramatically. In conclusion, treating dysgraphia using child-robot interaction is feasible and improves writing. Larger clinical studies are required to confirm that children with dysgraphia could benefit from this setup.

The transferability of handwriting skills: from the Cyrillic to the Latin alphabet.

Do handwriting skills transfer when a child writes in two different scripts, such as the Latin and Cyrillic alphabets? Are our measures of handwriting skills intrinsically bound to one alphabet or will a child who faces handwriting difficulties in one script experience similar difficulties in the other script? To answer these questions, 190 children from grades 1-4 were asked to copy a short text using both the Cyrillic and Latin alphabets on a digital tablet. A recent change of policy in Kazakhstan gave us an opportunity to measure transfer, as the Latin-based Kazakh alphabet has not yet been introduced. Therefore, pupils in grade 1 had a 6-months experience in Cyrillic, and pupils in grades 2, 3, and 4 had 1.5, 2.5, and 3.5 years of experience in Cyrillic, respectively. This unique situation created a quasi-experimental situation that allowed us to measure the influence of the number of years spent practicing Cyrillic on the quality of handwriting in the Latin alphabet. The results showed that some of the differences between the two scripts were constant across all grades. These differences thus reflect the intrinsic differences in the handwriting dynamics between the two alphabets. For instance, several features related to the pen pressure on the tablet are quite different. Other features, however, revealed decreasing differences between the two scripts across grades. While we found that the quality of Cyrillic writing increased from grades 1-4, due to increased practice, we also found that the quality of the Latin writing increased as well, despite the fact that all of the pupils had the same absence of experience in writing in Latin. We can therefore interpret this improvement in Latin script as an indicator of the transfer of fine motor control skills from Cyrillic to Latin. This result is especially surprising given that one could instead hypothesize a negative transfer, i.e., that the finger controls automated for one alphabet would interfere with those required by the other alphabet. One interesting side-effect of these findings is that the algorithms that we developed for the diagnosis of handwriting difficulties among French-speaking children could be relevant for other alphabets, paving the way for the creation of a cross-lingual model for the detection of handwriting difficulties.

Acquisition of handwriting in children with and without dysgraphia: A computational approach.

Handwriting is a complex skill to acquire and it requires years of training to be mastered. Children presenting dysgraphia exhibit difficulties automatizing their handwriting. This can bring anxiety and can negatively impact education. 280 children were recruited in schools and specialized clinics to perform the Concise Evaluation Scale for Children's Handwriting (BHK) on digital tablets. Within this dataset, we identified children with dysgraphia. Twelve digital features describing handwriting through different aspects (static, kinematic, pressure and tilt) were extracted and used to create linear models to investigate handwriting acquisition throughout education. K-means clustering was performed to define a new classification of dysgraphia. Linear models show that three features only (two kinematic and one static) showed a significant association to predict change of handwriting quality in control children. Most kinematic and statics features interacted with age. Results suggest that children with dysgraphia do not simply differ from ones without dysgraphia by quantitative differences on the BHK scale but present a different development in terms of static, kinematic, pressure and tilt features. The K-means clustering yielded 3 clusters (Ci). Children in C1 presented mild dysgraphia usually not detected in schools whereas children in C2 and C3 exhibited severe dysgraphia. Notably, C2 contained individuals displaying abnormalities in term of kinematics and pressure whilst C3 regrouped children showing mainly tilt problems. The current results open new opportunities for automatic detection of children with dysgraphia in classroom. We also believe that the training of pressure and tilt may open new therapeutic opportunities through serious games.

Gamified Motor Training With Tangible Robots in Older Adults: A Feasibility Study and Comparison With the Young.

Background: An increasing lifespan and the resulting change in our expectations of later life stages are dependent on a good health state. This emphasizes the importance of the development of strategies to further strengthen healthy aging. One important aspect of good health in later life stages is sustained skilled motor function. Objective: Here, we tested the effectiveness of robotic upper limb motor training in a game-like scenario assessing game-based learning and its transfer potential. Methods: Thirty-six healthy participants (n = 18 elderly participants, n = 18 young controls) trained with a Pacman-like game using a hand-held Cellulo robot on 2 consecutive days. The game-related movements were conducted on a printed map displaying a maze and targets that had to be collected. Gradually, the task difficulty was adjusted between games by modifying or adding different game elements (e.g., speed and number of chasing ghosts, additional rules, and haptic feedback). Transfer was assessed by scoring simple robot manipulation on two different trajectories. Results: Elderly participants were able to improve their game performance over time [t (874) = 2.97, p < 0.01]. The applied game elements had similar effects on both age groups. Importantly, the game-based learning was transferable to simple robot manipulation that resembles activities of daily life. Only minor age-related differences were present (smaller overall learning gain and different effects of the wall-crash penalty rule in the elderly group). Conclusions: Gamified motor training with the Cellulo system has the potential to translate into an efficient and relatively low-cost robotic motor training tool for promoting upper limb function to promote healthy aging.

Extending the Spectrum of Dysgraphia: A Data Driven Strategy to Estimate Handwriting Quality.

This paper proposes new ways to assess handwriting, a critical skill in any child's school journey. Traditionally, a pen and paper test called the BHK test (Concise Evaluation Scale for Children's Handwriting) is used to assess children's handwriting in French-speaking countries. Any child with a BHK score above a certain threshold is diagnosed as 'dysgraphic', meaning that they are then eligible for financial coverage for therapeutic support. We previously developed a version of the BHK for tablet computers which provides rich data on the dynamics of writing (acceleration, pressure, and so forth). The underlying model was trained on dysgraphic and non-dysgraphic children. In this contribution, we deviate from the original BHK for three reasons. First, in this instance, we are interested not in a binary output but rather a scale of handwriting difficulties, from the lightest cases to the most severe. Therefore, we wish to compute how far a child's score is from the average score of children of the same age and gender. Second, our model analyses dynamic features that are not accessible on paper; hence, the BHK is useful in this instance. Using the PCA (Principal Component Analysis) reduced the set of 53 handwriting features to three dimensions that are independent of the BHK. Nonetheless, we double-checked that, when clustering our data set along any of these three axes, we accurately detected dysgraphic children. Third, dysgraphia is an umbrella concept that embraces a broad variety of handwriting difficulties. Two children with the same global score can have totally different types of handwriting difficulties. For instance, one child could apply uneven pen pressure while another one could have trouble controlling their writing speed. Our new test not only provides a global score, but it also includes four specific score for kinematics, pressure, pen tilt and static features (letter shape). Replacing a global score with a more detailed profile enables the selection of remediation games that are very specific to each profile.

Iterative Design and Evaluation of a Tangible Robot-Assisted Handwriting Activity for Special Education.

In this article we investigate the role of interactive haptic-enabled tangible robots in supporting the learning of cursive letter writing for children with attention and visuomotor coordination issues. We focus on the two principal aspects of handwriting that are linked to these issues: Visual perception and visuomotor coordination. These aspects, respectively, enhance two features of letter representation in the learner's mind in particular, namely the shape (grapheme) and the dynamics (ductus) of the letter, which constitute the central learning goals in our activity. Building upon an initial design tested with 17 healthy children in a preliminary school, we iteratively ported the activity to an occupational therapy context in 2 different therapy centers, in the context of 3 different summer school camps involving a total of 12 children having writing difficulties. The various iterations allowed us to uncover insights about the design of robot-enhanced writing activities for special education, specifically highlighting the importance of ease of modification of the duration of an activity as well as of adaptable frequency, content, flow and game-play and of providing a range of evaluation test alternatives. Results show that the use of robot-assisted handwriting activities could have a positive impact on the learning of the representation of letters in the context of occupational therapy (V = 1, 449, p < 0.001, r = 0.42). Results also highlight how the design changes made across the iterations affected the outcomes of the handwriting sessions, such as the evaluation of the performances, monitoring of the performances, and the connectedness of the handwriting.

A Comparison of Social Robot to Tablet and Teacher in a New Script Learning Context.

This research occurred in a special context where Kazakhstan's recent decision to switch from Cyrillic to the Latin-based alphabet has resulted in challenges connected to teaching literacy, addressing a rare combination of research hypotheses and technical objectives about language learning. Teachers are not necessarily trained to teach the new alphabet, and this could result in a challenge for children with learning difficulties. Prior research studies in Human-Robot Interaction (HRI) have proposed the use of a robot to teach handwriting to children (Hood et al., 2015; Lemaignan et al., 2016). Drawing on the Kazakhstani case, our study takes an interdisciplinary approach by bringing together smart solutions from robotics, computer vision areas, and educational frameworks, language, and cognitive studies that will benefit diverse groups of stakeholders. In this study, a human-robot interaction application is designed to help primary school children learn both a newly-adopted script and also its handwriting system. The setup involved an experiment with 62 children between the ages of 7-9 years old, across three conditions: a robot and a tablet, a tablet only, and a teacher. Based on the paradigm-learning by teaching-the study showed that children improved their knowledge of the Latin script by interacting with a robot. Findings reported that children gained similar knowledge of a new script in all three conditions without gender effect. In addition, children's likeability ratings and positive mood change scores demonstrate significant benefits favoring the robot over a traditional teacher and tablet only approaches.

Towards an Adaptive Upper Limb Rehabilitation Game with Tangible Robots.

A key feature of a successful game is its ability to provide the player with an adequate level of challenge. However, the objective of difficulty adaptation in serious games is not only to maintain the player's motivation by challenging, but also to ensure the completion of training objectives.This paper describes our proposed upper-limb rehabilitation game with tangible robots and investigates the effect of game elements and gameplay on the amount of the performed motion in several planes and percentage of failure by using the data from 33 unimpaired subjects who played 53 games within two consecutive days. In order to provide a more generic adaptation strategy in the future, we discretize the game area to circular zones. We then show the effect of changing these zones during gameplay on the activation of different muscles through EMG data in a pilot study.The study shows that it is possible to increase the challenge level by adding more active agents chasing the player and increasing the speed of these agents. However, only the increase in number of agents significantly increases the users' motion on both planes. Analysis of player behaviors leads us to suggest that by adapting the behaviour of these active agents in specific zones, it is possible to change the trajectory of the user, and to provide a focus on the activation of specific muscles.

Designing Configurable Arm Rehabilitation Games: How Do Different Game Elements Affect User Motion Trajectories?

For successful rehabilitation of a patient after a stroke or traumatic brain injury, it is crucial that rehabilitation activities are motivating, provide feedback and have a high rate of repetitions. Advancements in recent technologies provide solutions to address these aspects where needed. Additionally, through the use of gamification, we are able to increase the motivation for participants. However, many of these systems require complex set-ups, which can be a big challenge when conducting rehabilitation in a home-based setting. To address the lack of simple rehabilitation tools for arm function for a home-based application, we previously developed a system, Cellulo for rehabilitation, that is comprised of paper-supported tangible robots that are orchestrated by applications deployed on consumer tablets. These components enable different features that allow for gamification, easy setup, portability, and scalability. To support the configuration of game elements to patients' level of motor skills and strategies, their motor trajectories need to be classified. In this paper, we investigate the classification of different motor trajectories and how game elements impact these in unimpaired, healthy participants. We show that the manipulation of certain game elements do have an impact on motor trajectories, which might indicate that it is possible to adapt the arm remediation of patients by configuring game elements. These results provide a first step towards providing adaptive rehabilitation based upon patients' measured trajectories.

Multimodal Teaching Analytics: Automated Extraction of Orchestration Graphs from Wearable Sensor Data.

The pedagogical modelling of everyday classroom practice is an interesting kind of evidence, both for educational research and teachers' own professional development. This paper explores the usage of wearable sensors and machine learning techniques to automatically extract orchestration graphs (teaching activities and their social plane over time), on a dataset of 12 classroom sessions enacted by two different teachers in different classroom settings. The dataset included mobile eye-tracking as well as audiovisual and accelerometry data from sensors worn by the teacher. We evaluated both time-independent and time-aware models, achieving median F1 scores of about 0.7-0.8 on leave-one-session-out k-fold cross-validation. Although these results show the feasibility of this approach, they also highlight the need for larger datasets, recorded in a wider variety of classroom settings, to provide automated tagging of classroom practice that can be used in everyday practice across multiple teachers.

Automated human-level diagnosis of dysgraphia using a consumer tablet.

The academic and behavioral progress of children is associated with the timely development of reading and writing skills. Dysgraphia, characterized as a handwriting learning disability, is usually associated with dyslexia, developmental coordination disorder (dyspraxia), or attention deficit disorder, which are all neuro-developmental disorders. Dysgraphia can seriously impair children in their everyday life and require therapeutic care. Early detection of handwriting difficulties is, therefore, of great importance in pediatrics. Since the beginning of the 20th century, numerous handwriting scales have been developed to assess the quality of handwriting. However, these tests usually involve an expert investigating visually sentences written by a subject on paper, and, therefore, they are subjective, expensive, and scale poorly. Moreover, they ignore potentially important characteristics of motor control such as writing dynamics, pen pressure, or pen tilt. However, with the increasing availability of digital tablets, features to measure these ignored characteristics are now potentially available at scale and very low cost. In this work, we developed a diagnostic tool requiring only a commodity tablet. To this end, we modeled data of 298 children, including 56 with dysgraphia. Children performed the BHK test on a digital tablet covered with a sheet of paper. We extracted 53 handwriting features describing various aspects of handwriting, and used the Random Forest classifier to diagnose dysgraphia. Our method achieved 96.6% sensibility and 99.2% specificity. Given the intra-rater and inter-rater levels of agreement in the BHK test, our technique has comparable accuracy for experts and can be deployed directly as a diagnostics tool.