Motor Learning in Robot-Based Haptic Dyads: A Review.

Rehabilitation robots have the potential to alleviate the global burden of neurorehabilitation. Robot-based multiplayer gaming with virtual and haptic interaction may improve motivation, engagement, and implicit learning in robotic therapy. Over the past few years, there has been growing interest in robot mediated haptic dyads, or human-robot-robot-human interaction. The effect of such a paradigm on motor learning in general and specifically for individuals with motor and/or cognitive impairments is an open area of research. We reviewed the literature to investigate the effect of a robot-based haptic dyad on motor learning. Thirty-eight articles met the inclusion criteria for this review. We summarize study characteristics including device, haptic rendering, and experimental task. Our main findings indicate that dyadic training's impact on motor learning is inconsistent in that some studies show significant improvement of motor training while others show no influence. We also find that the relative skill level of the partner and interaction characteristics such as stiffness of connection and availability of visual information influence motor learning outcomes. We discuss implications for neurorehabilitation and conclude that additional research is needed to determine optimal interaction characteristics for motor learning and to extend this research to individuals with cognitive and motor impairments.

Towards an AI-driven soft toy for automatically detecting and classifying infant-toy interactions using optical force sensors.

Introduction: It is crucial to identify neurodevelopmental disorders in infants early on for timely intervention to improve their long-term outcomes. Combining natural play with quantitative measurements of developmental milestones can be an effective way to swiftly and efficiently detect infants who are at risk of neurodevelopmental delays. Clinical studies have established differences in toy interaction behaviors between full-term infants and pre-term infants who are at risk for cerebral palsy and other developmental disorders. Methods: The proposed toy aims to improve the quantitative assessment of infant-toy interactions and fully automate the process of detecting those infants at risk of developing motor delays. This paper describes the design and development of a toy that uniquely utilizes a collection of soft lossy force sensors which are developed using optical fibers to gather play interaction data from infants laying supine in a gym. An example interaction database was created by having 15 adults complete a total of 2480 interactions with the toy consisting of 620 touches, 620 punches-"kick substitute," 620 weak grasps and 620 strong grasps. Results: The data is analyzed for patterns of interaction with the toy face using a machine learning model developed to classify the four interactions present in the database. Results indicate that the configuration of 6 soft force sensors on the face created unique activation patterns. Discussion: The machine learning algorithm was able to identify the distinct action types from the data, suggesting the potential usability of the toy. Next steps involve sensorizing the entire toy and testing with infants.

Quantifying interaction with robotic toys in pre-term and full-term infants.

Infants born pre-term are at an increased risk for developmental, behavioral, and motor delay and subsequent disability. When these problems are detected early, clinical intervention can be effective at improving functional outcomes. Current methods of early clinical assessment are resource intensive, require extensive training, and do not always capture infants' behavior in natural play environments. We developed the Play and Neuro Development Assessment (PANDA) Gym, an affordable, mechatronic, sensor-based play environment that can be used outside clinical settings to capture infant visual and motor behavior. Using a set of classification codes developed from the literature, we analyzed videos from 24 pre-term and full-term infants as they played with each of three robotic toys designed to elicit different types of interactions-a lion, an orangutan, and an elephant. We manually coded for frequency and duration of toy interactions such as kicking, grasping, touching, and gazing. Pre-term infants gazed at the toys with similar frequency as full-term infants, but infants born full-term physically engaged more frequently and for longer durations with the robotic toys than infants born pre-term. While we showed we could detect differences between full-term and pre-term infants, further work is needed to determine whether differences seen were primarily due to age, developmental delays, or a combination.

Reimagining robotic walkers for real-world outdoor play environments with insights from legged robots: a scoping review.

PURPOSE: For children with mobility impairments, without cognitive delays, who want to participate in outdoor activities, existing assistive technology (AT) to support their needs is limited. In this review, we investigate the control and design of a selection of robotic walkers while exploring a selection of legged robots to develop solutions that address this gap in robotic AT. METHOD: We performed a comprehensive literature search from four main databases: PubMed, Google Scholar, Scopus, and IEEE Xplore. The keywords used in the search were the following: "walker", "rollator", "smart walker", "robotic walker", "robotic rollator". Studies were required to discuss the control or design of robotic walkers to be considered. A total of 159 papers were analyzed. RESULTS: From the 159 papers, 127 were excluded since they failed to meet our inclusion criteria. The total number of papers analyzed included publications that utilized the same device, therefore we classified the remaining 32 studies into groups based on the type of robotic walker used. This paper reviewed 15 different types of robotic walkers. CONCLUSIONS: The ability of many-legged robots to negotiate and transition between a range of unstructured substrates suggests several avenues of future consideration whose pursuit could benefit robotic AT, particularly regarding the present limitations of wheeled paediatric robotic walkers for children's daily outside use.IMPLICATIONS FOR REHABILITATIONChildren with lower limb disabilities can benefit from assistive technology designed for daily usage in outdoor surroundings.An extension of existing robotic assistive technology that allows the user to travel safely on irregular surfaces both indoors and outdoors is needed.Approaches used to solve research problems in the field of robotics (outside of the rehabilitation area) can be used to address problems that robotic assistive technology currently faces.There is a need for more research on the development of robotic assistive technology for children with mobility impairments without cognitive delays.

How should robots exercise with people? Robot-mediated exergames win with music, social analogues, and gameplay clarity.

Introduction: The modern worldwide trend toward sedentary behavior comes with significant health risks. An accompanying wave of health technologies has tried to encourage physical activity, but these approaches often yield limited use and retention. Due to their unique ability to serve as both a health-promoting technology and a social peer, we propose robots as a game-changing solution for encouraging physical activity. Methods: This article analyzes the eight exergames we previously created for the Rethink Baxter Research Robot in terms of four key components that are grounded in the video-game literature: repetition, pattern matching, music, and social design. We use these four game facets to assess gameplay data from 40 adult users who each experienced the games in balanced random order. Results: In agreement with prior research, our results show that relevant musical cultural references, recognizable social analogues, and gameplay clarity are good strategies for taking an otherwise highly repetitive physical activity and making it engaging and popular among users. Discussion: Others who study socially assistive robots and rehabilitation robotics can benefit from this work by considering the presented design attributes to generate future hypotheses and by using our eight open-source games to pursue follow-up work on social-physical exercise with robots.

Feasibility and Acceptability of Remote Neuromotor Rehabilitation Interactions Using Social Robot Augmented Telepresence: A Case Study.

There is a growing need to deliver rehabilitation care to patients remotely. Long term demographic changes, geographic shortages of care providers, and now a global pandemic contribute to this need. Telepresence provides an option for delivering this care. However, telepresence using video and audio alone does not provide an interaction of the same quality as in-person. To bridge this gap, we propose the use of social robot augmented telepresence (SRAT). We have constructed a demonstration SRAT system for upper extremity rehab, in which a humanoid, with a head, body, face, and arms, is attached to a mobile telepresence system, to collaborate with the patient and clinicians as an independent social entity. The humanoid can play games with the patient and demonstrate activities. These activities could be used to perform assessments in support of self-directed rehab and to perform exercises.In this paper, we present a case series with six subjects who completed interactions with the robot, three subjects who have previously suffered a stroke and three pediatric subjects who are typically developing. Subjects performed a Simon Says activity and a target touch activity in person, using classical telepresence (CT), and using SRAT. Subjects were able to effectively work with the social robot guiding interactions and 5 of 6 rated SRAT better than CT. This study demonstrates the feasibility of SRAT and some of its benefits.

The Impact of Cognitive Impairment on Robot-Based Upper-Limb Motor Assessment in Chronic Stroke.

BACKGROUND: Chronic upper extremity motor deficits are present in up to 65% of stroke survivors, and cognitive impairment is prevalent in 46-61% of stroke survivors even 10 years after their stroke. Robot-assisted therapy programs tend to focus on motor recovery and do not include stroke patients with cognitive impairment. OBJECTIVE: This study aims to investigate performance on the individual cognitive domains evaluated in the MoCA and their relation to upper-limb motor performance on a robotic system. METHODS: Participants were recruited from the stroke population with a wide range of cognitive and motor levels to complete a trajectory tracking task using the Haptic TheraDrive rehabilitation robot system. Motor performance was evaluated against standard clinical cognitive and motor assessments. Our hypothesis is that the cognitive domains involved in the visuomotor tracking task are significant predictors of performance on the robot-based task and that impairment in these domains results in worse motor performance on the task compared to subjects with no cognitive impairment. RESULTS: Our results support the hypothesis that visuospatial and executive function have a significant impact on motor performance, with differences emerging between different functional groups on the various robot-based metrics. We also show that the kinematic metrics from this task differentiate cognitive-motor functional groups differently. CONCLUSION: This study demonstrates that performance on a motor-based robotic assessment task also involves a significant visuospatial and executive function component and highlights the need to account for cognitive impairment in the assessment of motor performance.

Design of pediatric robot to simulate infant biomechanics for neuro-developmental assessment in a sensorized gym.

Infants at risk for developmental delays often exhibit postures and movements that may provide a window into potential impairment for cerebral palsy and other neuromotor conditions. We developed a simple 4 DOF robot pediatric simulator to help provide insight into how infant kinematic movements may affect the center of pressure (COP), a common measure thought to be sensitive to neuromotor delay when assessed from supine infants at play. We conducted two experiments: 1) we compared changes in COP caused by limb movements to a human infant and 2) we determined if we could predict COP position due to limb movements using simulator kinematic pose retrieved from video and a sensorized mat. Our results indicate that the limb movements alone were not sufficient to mimic the COP in a human infant. In addition, we show that given a robot simulator and a simple camera, we can predict COP measured by a force sensing mat. Future directions suggest a more complex robot is needed such as one that may include trunk DOF.

Infants at risk for physical disability may be identified by measures of postural control in supine.

BACKGROUND: Early detection of delay or impairment in motor function is important to guide clinical management and inform prognosis during a critical window for the development of motor control in children. The purpose of this study was to investigate the ability of biomechanical measures of early postural control to distinguish infants with future impairment in motor control from their typically developing peers. METHODS: We recorded postural control from infants lying in supine in several conditions. We compared various center of pressure metrics between infants grouped by birth status (preterm and full term) and by future motor outcome (impaired motor control and typical motor control). RESULTS: One of the seven postural control metrics-path length-was consistently different between groups for both group classifications and for the majority of conditions. CONCLUSIONS: Quantitative measures of early spontaneous infant movement may have promise to distinguish early in life between infants who are at risk for motor impairment or physical disability and those who will demonstrate typical motor control. Our observation that center of pressure path length may be a potential early marker of postural instability and motor control impairment needs further confirmation and further investigation to elucidate the responsible neuromotor mechanisms. IMPACT: The key message of this article is that quantitative measures of infant postural control in supine may have promise to distinguish between infants who will demonstrate future motor impairment and those who will demonstrate typical motor control. One of seven postural control metrics-path length-was consistently different between groups. This metric may be an early marker of postural instability in infants at risk for physical disability.

Classifying and tracking rehabilitation interventions through machine-learning algorithms in individuals with stroke.

INTRODUCTION: Stroke is the leading cause of disability worldwide. It has been well-documented that rehabilitation (rehab) therapy can aid in regaining health and function for individuals with stroke. Yet, tracking in-home rehab continues to be a challenge because of a lack of resources and population-scale demands. In order to address this gap, we implemented a methodology to classify and track rehab interventions in individuals with stroke. METHODS: We developed personalized classification algorithms, including neural network-based algorithms, to classify four rehab exercises performed by two individuals with stroke who were part of a week-long therapy camp in Jamaica, a low- and middle-income country. Accelerometry-based wearable sensors were placed on each upper and lower limb to collect movement data during therapy. RESULTS: The classification accuracy for traditional and neural network-based algorithms utilizing feature data (e.g., number of peaks) from the sensors ranged from 64 to 94%, respectively. In addition, the study proposes a new method to assess change in bilateral mobility over the camp duration. CONCLUSION: The results of this pilot study indicate that personalized supervised learning algorithms can be used to classify and track rehab activities and functional outcomes in resource limited settings such as LMICs.

The design of Lil'Flo, a socially assistive robot for upper extremity motor assessment and rehabilitation in the community via telepresence.

INTRODUCTION: We present Lil'Flo, a socially assistive robotic telerehabilitation system for deployment in the community. As shortages in rehabilitation professionals increase, especially in rural areas, there is a growing need to deliver care in the communities where patients live, work, learn, and play. Traditional telepresence, while useful, fails to deliver the rich interactions and data needed for motor rehabilitation and assessment. METHODS: We designed Lil'Flo, targeted towards pediatric patients with cerebral palsy and brachial plexus injuries using results from prior usability studies. The system combines traditional telepresence and computer vision with a humanoid, who can play games with patients and guide them in a present and engaging way under the supervision of a remote clinician. We surveyed 13 rehabilitation clinicians in a virtual usability test to evaluate the system. RESULTS: The system is more portable, extensible, and cheaper than our prior iteration, with an expressive humanoid. The virtual usability testing shows that clinicians believe Lil'Flo could be deployed in rural and elder care facilities and is more capable of remote stretching, strength building, and motor assessments than traditional video only telepresence. CONCLUSIONS: Lil'Flo represents a novel approach to delivering rehabilitation care in the community while maintaining the clinician-patient connection.

Perceived Usefulness of a Social Robot Augmented Telehealth Platform by Therapists in the United States.

With the shortage of rehabilitation clinicians in rural areas and the ongoing COVID-19 pandemic, remote rehabilitation (telerehab) fills an important gap in access to rehabilitation, especially for the treatment of adults and children experiencing upper arm disability due to stroke and cerebral palsy. We propose the use of a socially assistive robot with arms, a torso, and a face to play games with and guide patients, coupled with a telepresence platform, to maintain the patient-clinician interaction, and a computer vision system, to aid in automated objective assessments, as a tool for achieving more effective telerehab. In this paper, we outline the design of such a system, Lil'Flo, and present a uniquely large perceived usefulness evaluation of the Lil'Flo platform with 351 practicing therapists in the United States. We analyzed responses to the question of general interest and 5 questions on Lil'Flo's perceived usefulness. Therapists believe that Lil'Flo would significantly improve communication, motivation, and compliance during telerehab interactions when compared to traditional telepresence. 27% of therapists reported that they were interested in using Lil'Flo. Therapists interested in using Lil'Flo perceived it as having significantly higher usefulness across all measured dimensions than those who were not interested in using it.

A First Step Toward the Operationalization of the Learned Non-Use Phenomenon: A Delphi Study.

BACKGROUND: The negative discrepancy between residual functional capacity and reduced use of the contralesional hand, frequently observed after a brain lesion, has been termed Learned Non-Use (LNU) and is thought to depend on the interaction of neuronal mechanisms during recovery and learning-dependent mechanisms. OBJECTIVE: Albeit the LNU phenomenon is generally accepted to exist, currently, no transdisciplinary definition exists. Furthermore, although therapeutic approaches are implemented in clinical practice targeting LNU, no standardized diagnostic routine is described in the available literature. Our objective was to reach consensus regarding a definition as well as synthesize knowledge about the current diagnostic procedures. METHODS: We used a structured group communication following the Delphi method among clinical and scientific experts in the field, knowledge from both, the work with patient populations and with animal models. RESULTS: Consensus was reached regarding a transdisciplinary definition of the LNU phenomenon. Furthermore, the mode and strategy of the diagnostic process, as well as the sources of information and outcome parameters relevant for the clinical decision making, were described with a wide range showing the current lack of a consistent universal diagnostic approach. CONCLUSIONS: The need for the development of a structured diagnostic procedure and its implementation into clinical practice is emphasized. Moreover, it exists a striking gap between the prevailing hypotheses regarding the mechanisms underlying the LNU phenomenon and the actual evidence. Therefore, basic research is needed to bridge between bedside and bench and eventually improve clinical decision making and further development of interventional strategies beyond the field of stroke rehabilitation.

Robot-Based Assessment of HIV-Related Motor and Cognitive Impairment for Neurorehabilitation.

There is a pressing need for strategies to slow or treat the progression of functional decline in people living with HIV. This paper explores a novel rehabilitation robotics approach to measuring cognitive and motor impairment in adults living with HIV, including a subset with stroke. We conducted a cross-sectional study with 21 subjects exhibiting varying levels of cognitive and motor impairment. We tested three robot-based tasks - trajectory tracking, N-back, and spatial span - to assess if metrics derived from these tasks were sensitive to differences in subjects with varying levels of executive function and upper limb motor impairments. We also examined how well these metrics could estimate clinical cognitive and motor scores. The results showed that the average sequence length on the robot-based spatial span task was the most sensitive to differences between various cognitive and motor impairment levels. We observed strong correlations between robot-based measures and clinical cognitive and motor assessments relevant to the HIV population, such as the Color Trails 1 (rho = 0.83), Color Trails 2 (rho = 0.71), Digit Symbol - Coding (rho = 0.81), Montreal Cognitive Assessment - Executive Function subscore (rho = 0.70), and Box and Block Test (rho = 0.74). Importantly, our results highlight that gross motor impairment may be overlooked in the assessment of HIV-related disability. This study shows that rehabilitation robotics can be expanded to new populations beyond stroke, namely to people living with HIV and those with cognitive impairments.

Robot-based assessment of HIV-related motor and cognitive impairment for neurorehabilitation.

There is a pressing need for strategies to slow or treat the progression of functional decline in people living with HIV. This paper explores a novel rehabilitation robotics approach to measuring cognitive and motor impairment in adults living with HIV, including a subset with stroke. We conducted a cross-sectional study with 21 subjects exhibiting varying levels of cognitive and motor impairment. We developed three robot-based tasks trajectory tracking, N-back, and spatial span - to assess if metrics derived from these tasks were sensitive to differences in subjects with varying levels of executive function and upper limb motor impairments. We also examined if these metrics could estimate clinical cognitive and motor scores. The results showed that the average sequence length on the robot-based spatial span task was the most sensitive to differences between subjects' cognitive and motor impairment levels. We observed strong correlations between robot-based measures and clinical cognitive and motor assessments relevant to the HIV population, such as the Color Trails 1 (rho = 0.83), Color Trails 2 (rho = 0.71), Digit Symbol - Coding (rho = 0.81), Montreal Cognitive Assessment - Executive Function subscore (rho = 0.70), and Box and Block Test (rho = 0.74). Importantly, our results highlight that gross motor impairment may be overlooked in the assessment of HIV-related disability. This study shows that rehabilitation robotics can be expanded to new populations beyond stroke, namely to people living with HIV and those with cognitive impairments.

Computer Vision to Automatically Assess Infant Neuromotor Risk.

An infant's risk of developing neuromotor impairment is primarily assessed through visual examination by specialized clinicians. Therefore, many infants at risk for impairment go undetected, particularly in under-resourced environments. There is thus a need to develop automated, clinical assessments based on quantitative measures from widely-available sources, such as videos recorded on a mobile device. Here, we automatically extract body poses and movement kinematics from the videos of at-risk infants (N = 19). For each infant, we calculate how much they deviate from a group of healthy infants (N = 85 online videos) using a Naïve Gaussian Bayesian Surprise metric. After pre-registering our Bayesian Surprise calculations, we find that infants who are at high risk for impairments deviate considerably from the healthy group. Our simple method, provided as an open-source toolkit, thus shows promise as the basis for an automated and low-cost assessment of risk based on video recordings.

Exercising with Baxter: preliminary support for assistive social-physical human-robot interaction.

BACKGROUND: The worldwide population of older adults will soon exceed the capacity of assisted living facilities. Accordingly, we aim to understand whether appropriately designed robots could help older adults stay active at home. METHODS: Building on related literature as well as guidance from experts in game design, rehabilitation, and physical and occupational therapy, we developed eight human-robot exercise games for the Baxter Research Robot, six of which involve physical human-robot contact. After extensive iteration, these games were tested in an exploratory user study including 20 younger adult and 20 older adult users. RESULTS: Only socially and physically interactive games fell in the highest ranges for pleasantness, enjoyment, engagement, cognitive challenge, and energy level. Our games successfully spanned three different physical, cognitive, and temporal challenge levels. User trust and confidence in Baxter increased significantly between pre- and post-study assessments. Older adults experienced higher exercise, energy, and engagement levels than younger adults, and women rated the robot more highly than men on several survey questions. CONCLUSIONS: The results indicate that social-physical exercise with a robot is more pleasant, enjoyable, engaging, cognitively challenging, and energetic than similar interactions that lack physical touch. In addition to this main finding, researchers working in similar areas can build on our design practices, our open-source resources, and the age-group and gender differences that we found.

Towards Automated Emotion Classification of Atypically and Typically Developing Infants.

The World Health Organization estimates that 15 million infants are born preterm every year [1]. This is of concern because these infants have a significant chance of having neuromotor or cognitive developmental delays due to cerebral palsy or other developmental issues [2]. Our long-term goal is to determine the roles emotion and movement play in the diagnosis of atypical infants. In this paper, we examine how automated emotion assessment may have potential to classify typically and atypically developing infants. We compare a custom supervised machine learning algorithm that utilizes individual and grouped facial features for infant emotion classification with a state-of-the-art neural network. Our results show that only three concavity features are needed for the concavity algorithm, and the custom algorithm performed with relatively similar performance to the neural network. Automatic sentiment labels used in tandem with infant movement kinematics would be further investigated to determine if emotion and movement are interdependent and predictive of an infant's neurodevelopmental delay in disorders such as cerebral palsy.

Towards Data-Driven Autonomous Robot-Assisted Physical Rehabilitation Therapy.

Task-oriented therapy consists of three stages: demonstration, observation and assistance. While demonstration using robots has been extensively studied, the other two stages rarely involve robots. This paper focuses on the transition between observation and assistance. More specifically, we tackle the robot's decision making problem of whether to assist a patient or not based on the observation. The proposed method is to train a discrete tunnel shape 3-D decision boundary through correct demonstration to classify motions. Additional conditions such as slow progress, self correction and overshot motions are taken into account of the decision making. Preliminary experiments have been performed on BAXTER robot for a cup reaching task. The BAXTER robot is programmed to react according to the decision boundary. It assists the patient when the patient's hand position is determined by the proposed algorithm to be unacceptable. Multiple cases including correct motion, continuous assistance, overshot, misaim and slow progress are tested. Results have confirmed the feasibility of the proposed method, which can reduce the current shortage of physical rehabilitation therapists.