Tracing curves in the plane: Geometric-invariant learning from human demonstrations.

The empirical laws governing human-curvilinear movements have been studied using various relationships, including minimum jerk, the 2/3 power law, and the piecewise power law. These laws quantify the speed-curvature relationships of human movements during curve tracing using critical speed and curvature as regressors. In this work, we provide a reservoir computing-based framework that can learn and reproduce human-like movements. Specifically, the geometric invariance of the observations, i.e., lateral distance from the closest point on the curve, instantaneous velocity, and curvature, when viewed from the moving frame of reference, are exploited to train the reservoir system. The artificially produced movements are evaluated using the power law to assess whether they are indistinguishable from their human counterparts. The generalisation capabilities of the trained reservoir to curves that have not been used during training are also shown.

Design of a Passive Wearable Device Using an Optimized Mechanical Metamaterial for Mirror Therapy.

Mirror Therapy (MT) is an effective therapeutic method used in the rehabilitation of hemiplegics. The effectiveness of this method is improved by employing a bi-modal approach which requires the synchronous movement of the affected and unaffected arm. For this purpose, we describe the design of a wearable device using a Mechanical Metamaterial (MM) that is optimized for the specific user to provide passive assistance of wrist flexion-extension and enable synchronous motion of the affected and unaffected arm during MT.

Learning Compliant Box-in-Box Insertion through Haptic-Based Robotic Teleoperation.

In modern logistics, the box-in-box insertion task is representative of a wide range of packaging applications, and automating compliant object insertion is difficult due to challenges in modelling the object deformation during insertion. Using Learning from Demonstration (LfD) paradigms, which are frequently used in robotics to facilitate skill transfer from humans to robots, can be one solution for complex tasks that are difficult to mathematically model. In order to automate the box-in-box insertion task for packaging applications, this study makes use of LfD techniques. The proposed framework has three phases. Firstly, a master-slave teleoperated robot system is used in the initial phase to haptically demonstrate the insertion task. Then, the learning phase involves identifying trends in the demonstrated trajectories using probabilistic methods, in this case, Gaussian Mixture Regression. In the third phase, the insertion task is generalised, and the robot adjusts to any object position using barycentric interpolation. This method is novel because it tackles tight insertion by taking advantage of the boxes' natural compliance, making it possible to complete the task even with a position-controlled robot. To determine whether the strategy is generalisable and repeatable, experimental validation was carried out.

How to Train Your Posture: Haptic Feedback Can be Used for Postural Adaptation of the Trunk During Upper-Limb Motor Training.

Poor trunk posture, especially during long periods of sitting, could lead to problems such as Low Back Pain (LBP) and Forward Head Posture (FHP). Typical solutions are based on visual or vibration-based feedback. However, these systems could lead to feedback being ignored by the user and phantom vibration syndrome, respectively. In this study, we propose using haptic feedback for postural adaptation. In this two-part study, twenty-four healthy participants (age 25.87 ± 2.17 years) adapted to three different postural targets in the anterior direction while performing a unimanual reaching task using a robotic device. Results suggest a strong adaptation to the desired postural targets. Mean anterior trunk bending after the intervention is significantly different compared to baseline measurements for all postural targets. Additional analysis of movement straightness and smoothness indicates an absence of any negative interference of posture-based feedback on the performance of reaching movement. Taken together, these results suggest that haptic feedback-based systems could be used for postural adaptation applications. Also, this type of postural adaptation system can be used during the rehabilitation of stroke patients to reduce trunk compensation in lieu of typical physical constraint-based methods.

The Design and Development of Instrumented Toys for the Assessment of Infant Cognitive Flexibility.

The first years of an infant's life represent a sensitive period for neurodevelopment where one can see the emergence of nascent forms of executive function (EF), which are required to support complex cognition. Few tests exist for measuring EF during infancy, and the available tests require painstaking manual coding of infant behaviour. In modern clinical and research practice, human coders collect data on EF performance by manually labelling video recordings of infant behaviour during toy or social interaction. Besides being extremely time-consuming, video annotation is known to be rater-dependent and subjective. To address these issues, starting from existing cognitive flexibility research protocols, we developed a set of instrumented toys to serve as a new type of task instrumentation and data collection tool suitable for infant use. A commercially available device comprising a barometer and an inertial measurement unit (IMU) embedded in a 3D-printed lattice structure was used to detect when and how the infant interacts with the toy. The data collected using the instrumented toys provided a rich dataset that described the sequence of toy interaction and individual toy interaction patterns, from which EF-relevant aspects of infant cognition can be inferred. Such a tool could provide an objective, reliable, and scalable method of collecting early developmental data in socially interactive contexts.

Let the force guide you: a performance-based adaptive algorithm for postural training using haptic feedback.

Motor learning is an essential component of human behavior. Many different factors can influence the process of motor learning, such as the amount of practice and type of feedback. Changes in task difficulty during training can also considerably impact motor learning. Typical motor learning studies include a sequential variation of task difficulty, i.e., easy to challenging, irrespective of user performance. However, many studies have reported the importance of performance-based task difficulty variation for effective motor learning and skill transfer. A performance-based adaptive algorithm for task difficulty variation based on the challenge-point framework is proposed in this study. The algorithm is described for postural adaptation during simultaneous upper-limb training. Ten healthy participants (28 ± 2.44 years) were recruited to validate the algorithm. Participants adapted to a postural target of 20° in the anterior direction from the initial upright posture while performing a unimanual reaching task using a robotic device. Results suggest a significant decrease in postural error after training. The algorithm successfully adapted the task difficulty based on the performance of the user. The proposed algorithm could be modified for different motor skills and can be further evaluated for different applications in order to maximize the potential benefits of rehabilitation sessions.

Haptic feedback system for postural adaptation during robotic rehabilitation of upper limb.

Although trunk compensation during stroke rehabilitation is widely studied, the proposed solutions primarily include a trunk constraint, which has several disadvantages. In this study, we have proposed a haptic feedback-based system for postural training during upper-limb motor rehabilitation. We have tested the proposed system on six healthy people in this preliminary study. Participants performed a simple 1-dimensional reaching task while their posture was being monitored. They received haptic feedback based on their trunk posture. Preliminary results revealed a significant decline in postural error (p<0.05) after the haptic-based training. The reduction in error was maintained even after haptic feedback was turned off. This study shows that haptic feedback could be a viable alternative to the traditional constraint-based methods for postural adaptation. Additional studies need to be conducted to further evaluate the influence of using such feedback strategies.

Fast Kinematic Re-Calibration for Industrial Robot Arms.

Accurate kinematic modelling is pivotal in the safe and reliable execution of both contact and non-contact robotic applications. The kinematic models provided by robot manufacturers are valid only under ideal conditions and it is necessary to account for the manufacturing errors, particularly the joint offsets introduced during the assembling stages, which is identified as the underlying problem for position inaccuracy in more than 90% of the situations. This work was motivated by a very practical need, namely the discrepancy in terms of end-effector kinematics as computed by factory-calibrated internal controller and the nominal kinematic model as per robot datasheet. Even though the problem of robot calibration is not new, the focus is generally on the deployment of external measurement devices (for open loop calibration) or mechanical fixtures (for closed loop calibration). On the other hand, we use the factory-calibrated controller as an 'oracle' for our fast-recalibration approach. This allows extracting calibrated intrinsic parameters (e.g., link lengths) otherwise not directly available from the 'oracle', for use in ad-hoc control strategies. In this process, we minimize the kinematic mismatch between the ideal and the factory-calibrated robot models for a Kinova Gen3 ultra-lightweight robot by compensating for the joint zero position error and the possible variations in the link lengths. Experimental analysis has been presented to validate the proposed method, followed by the error comparison between the calibrated and un-calibrated models over training and test sets.

Towards Haptic-Based Dual-Arm Manipulation.

Vision is the main component of current robotics systems that is used for manipulating objects. However, solely relying on vision for hand-object pose tracking faces challenges such as occlusions and objects moving out of view during robotic manipulation. In this work, we show that object kinematics can be inferred from local haptic feedback at the robot-object contact points, combined with robot kinematics information given an initial vision estimate of the object pose. A planar, dual-arm, teleoperated robotic setup was built to manipulate an object with hands shaped like circular discs. The robot hands were built with rubber cladding to allow for rolling contact without slipping. During stable grasping by the dual arm robot, under quasi-static conditions, the surface of the robot hand and object at the contact interface is defined by local geometric constraints. This allows one to define a relation between object orientation and robot hand orientation. With rolling contact, the displacement of the contact point on the object surface and the hand surface must be equal and opposite. This information, coupled with robot kinematics, allows one to compute the displacement of the object from its initial location. The mathematical formulation of the geometric constraints between robot hand and object is detailed. This is followed by the methodology in acquiring data from experiments to compute object kinematics. The sensors used in the experiments, along with calibration procedures, are presented before computing the object kinematics from recorded haptic feedback. Results comparing object kinematics obtained purely from vision and from haptics are presented to validate our method, along with the future ideas for perception via haptic manipulation.

Robotic Assisted Upper Limb Training Post Stroke: A Randomized Control Trial Using Combinatory Approach Toward Reducing Workforce Demands.

Post stroke upper limb rehabilitation is a challenging problem with poor outcomes as 40% of survivors have functionally useless upper limbs. Robot-aided therapy (RAT) is a potential method to alleviate the effort of intensive, task-specific, repetitive upper limb exercises for both patients and therapists. The present study aims to investigate how a time matched combinatory training scheme that incorporates conventional and RAT, using H-Man, compares with conventional training toward reducing workforce demands. In a randomized control trial (NCT02188628, www.clinicaltrials.gov), 44 subacute to chronic stroke survivors with first-ever clinical stroke and predominant arm motor function deficits were recruited and randomized into two groups of 22 subjects: Robotic Therapy (RT) and Conventional Therapy (CT). Both groups received 18 sessions of 90 min; three sessions per week over 6 weeks. In each session, participants of the CT group received 90 min of 1:1 therapist-supervised conventional therapy while participants of the RT group underwent combinatory training which consisted of 60 min of minimally-supervised H-Man therapy followed by 30 min of conventional therapy. The clinical outcomes [Fugl-Meyer (FMA), Action Research Arm Test and, Grip Strength] and the quantitative measures (smoothness, time efficiency, and task error, derived from two robotic assessment tasks) were independently evaluated prior to therapy intervention (week 0), at mid-training (week 3), at the end of training (week 6), and post therapy (week 12 and 24). Significant differences within group were observed at the end of training for all clinical scales compared with baseline [mean and standard deviation of FMA score changes between baseline and week 6; RT: Δ4.41 (3.46) and CT: Δ3.0 (4.0); p < 0.01]. FMA gains were retained 18 weeks post-training [week 24; RT: Δ5.38 (4.67) and week 24 CT: Δ4.50 (5.35); p < 0.01]. The RT group clinical scores improved similarly when compared to CT group with no significant inter-group at all time points although the conventional therapy time was reduced to one third in RT group. There were no training-related adverse side effects. In conclusion, time matched combinatory training incorporating H-Man RAT produced similar outcomes compared to conventional therapy alone. Hence, this study supports a combinatory approach to improve motor function in post-stroke arm paresis. Clinical Trial Registration: www.ClinicalTrials.gov, identifier: NCT02188628.

A Hessian-based decomposition characterizes how performance in complex motor skills depends on individual strategy and variability.

In complex real-life motor skills such as unconstrained throwing, performance depends on how accurate is on average the outcome of noisy, high-dimensional, and redundant actions. What characteristics of the action distribution relate to performance and how different individuals select specific action distributions are key questions in motor control. Previous computational approaches have highlighted that variability along the directions of first order derivatives of the action-to-outcome mapping affects performance the most, that different mean actions may be associated to regions of the actions space with different sensitivity to noise, and that action covariation in addition to noise magnitude matters. However, a method to relate individual high-dimensional action distribution and performance is still missing. Here we introduce a decomposition of performance into a small set of indicators that compactly and directly characterize the key performance-related features of the distribution of high-dimensional redundant actions. Central to the method is the observation that, if performance is quantified as a mean score, the Hessian (second order derivatives) of the action-to-score function determines how the noise of the action distribution affects performance. We can then approximate the mean score as the sum of the score of the mean action and a tolerance-variability index which depends on both Hessian and action covariance. Such index can be expressed as the product of three terms capturing noise magnitude, noise sensitivity, and alignment of the most variable and most noise sensitive directions. We apply this method to the analysis of unconstrained throwing actions by non-expert participants and show that, consistently across four different throwing targets, each participant shows a specific selection of mean action score and tolerance-variability index as well as specific selection of noise magnitude and alignment indicators. Thus, participants with different strategies may display the same performance because they can trade off suboptimal mean action for better tolerance-variability and higher action variability for better alignment with more tolerant directions in action space.

Neurorehabilitation From a Distance: Can Intelligent Technology Support Decentralized Access to Quality Therapy?

Current neurorehabilitation models primarily rely on extended hospital stays and regular therapy sessions requiring close physical interactions between rehabilitation professionals and patients. The current COVID-19 pandemic has challenged this model, as strict physical distancing rules and a shift in the allocation of hospital resources resulted in many neurological patients not receiving essential therapy. Accordingly, a recent survey revealed that the majority of European healthcare professionals involved in stroke care are concerned that this lack of care will have a noticeable negative impact on functional outcomes. COVID-19 highlights an urgent need to rethink conventional neurorehabilitation and develop alternative approaches to provide high-quality therapy while minimizing hospital stays and visits. Technology-based solutions, such as, robotics bear high potential to enable such a paradigm shift. While robot-assisted therapy is already established in clinics, the future challenge is to enable physically assisted therapy and assessments in a minimally supervized and decentralized manner, ideally at the patient's home. Key enablers are new rehabilitation devices that are portable, scalable and equipped with clinical intelligence, remote monitoring and coaching capabilities. In this perspective article, we discuss clinical and technological requirements for the development and deployment of minimally supervized, robot-assisted neurorehabilitation technologies in patient's homes. We elaborate on key principles to ensure feasibility and acceptance, and on how artificial intelligence can be leveraged for embedding clinical knowledge for safe use and personalized therapy adaptation. Such new models are likely to impact neurorehabilitation beyond COVID-19, by providing broad access to sustained, high-quality and high-dose therapy maximizing long-term functional outcomes.

Haptic Manipulation of 3D Scans for Geometric Feature Enhancement.

Localisation of geometric features like holes, edges, slots, etc. is vital to robotic planning in industrial automation settings. Low-cost 3D scanners are crucial in terms of improving accessibility, but pose a practical challenge to feature localisation because of poorer resolution and consequently affect robotic planning. In this work, we address the possibility of enhancing the quality of a 3D scan by a manual 'touch-up' of task-relevant features, to ensure their automatic detection prior to automation. We propose a framework whereby the operator (i) has access to both the actual work-piece and its 3D scan; (ii) evaluates the missing salient features from the scan; (iii) uses a haptic stylus to physically interact with the actual work-piece, around such specific features; (iv) interactively updates the scan using the position and force information from the haptic stylus. The contribution of this work is the use of haptic mismatch for geometric update. Specifically, the geometry from the 3D scan is used to predict haptic feedback at a point on the work-piece surface. The haptic mismatch is derived as a measure of error between this prediction and the real interaction forces from physical contact at that point on the work-piece. The geometric update is driven until the haptic mismatch is minimised. Convergence of the proposed algorithm is first numerically verified on an analytical surface with simulated physical interaction. Error analysis of the surface position and orientations were also plotted. Experiments were conducted using a motion capture system providing sub-mm accuracy in position and a 6 axis F/T sensor. Missing features are successfully detected after the update of the scan using the proposed method in an experiment.

Elbow Motion Trajectory Prediction Using a Multi-Modal Wearable System: A Comparative Analysis of Machine Learning Techniques.

Motion intention detection is fundamental in the implementation of human-machine interfaces applied to assistive robots. In this paper, multiple machine learning techniques have been explored for creating upper limb motion prediction models, which generally depend on three factors: the signals collected from the user (such as kinematic or physiological), the extracted features and the selected algorithm. We explore the use of different features extracted from various signals when used to train multiple algorithms for the prediction of elbow flexion angle trajectories. The accuracy of the prediction was evaluated based on the mean velocity and peak amplitude of the trajectory, which are sufficient to fully define it. Results show that prediction accuracy when using solely physiological signals is low, however, when kinematic signals are included, it is largely improved. This suggests kinematic signals provide a reliable source of information for predicting elbow trajectories. Different models were trained using 10 algorithms. Regularization algorithms performed well in all conditions, whereas neural networks performed better when the most important features are selected. The extensive analysis provided in this study can be consulted to aid in the development of accurate upper limb motion intention detection models.

Black-Scholes Theory and Diffusion Processes on the Cotangent Bundle of the Affine Group.

The Black-Scholes partial differential equation (PDE) from mathematical finance has been analysed extensively and it is well known that the equation can be reduced to a heat equation on Euclidean space by a logarithmic transformation of variables. However, an alternative interpretation is proposed in this paper by reframing the PDE as evolving on a Lie group. This equation can be transformed into a diffusion process and solved using mean and covariance propagation techniques developed previously in the context of solving Fokker-Planck equations on Lie groups. An extension of the Black-Scholes theory with coupled asset dynamics produces a diffusion equation on the affine group, which is not a unimodular group. In this paper, we show that the cotangent bundle of a Lie group endowed with a semidirect product group operation, constructed in this paper for the case of groups with trivial centers, is always unimodular and considering PDEs as diffusion processes on the unimodular cotangent bundle group allows a direct application of previously developed mean and covariance propagation techniques, thereby offering an alternative means of solution of the PDEs. Ultimately these results, provided here in the context of PDEs in mathematical finance may be applied to PDEs arising in a variety of different fields and inform new methods of solution.

A Complementary Filter Design on SE(3) to IdentifyMicro-Motions during 3D Motion Tracking.

In 3D motion capture, multiple methods have been developed in order to optimize thequality of the captured data. While certain technologies, such as inertial measurement units (IMU),are mostly suitable for 3D orientation estimation at relatively high frequencies, other technologies,such as marker-based motion capture, are more suitable for 3D position estimations at a lower frequencyrange. In this work, we introduce a complementary filter that complements 3D motion capture datawith high-frequency acceleration signals from an IMU. While the local optimization reduces the error ofthe motion tracking, the additional accelerations can help to detect micro-motions that are useful whendealing with high-frequency human motions or robotic applications. The combination of high-frequencyaccelerometers improves the accuracy of the data and helps to overcome limitations in motion capturewhen micro-motions are not traceable with 3D motion tracking system. In our experimental evaluation,we demonstrate the improvements of the motion capture results during translational, rotational,and combined movements.

Estimating Human Wrist Stiffness during a Tooling Task.

In this work, we propose a practical approach to estimate human joint stiffness during tooling tasks for the purpose of programming a robot by demonstration. More specifically, we estimate the stiffness along the wrist radial-ulnar deviation while a human operator performs flexion-extension movements during a polishing task. The joint stiffness information allows to transfer skills from expert human operators to industrial robots. A typical hand-held, abrasive tool used by humans during finishing tasks was instrumented at the handle (through which both robots and humans are attached to the tool) to assess the 3D force/torque interactions between operator and tool during finishing task, as well as the 3D kinematics of the tool itself. Building upon stochastic methods for human arm impedance estimation, the novelty of our approach is that we rely on the natural variability taking place during the multi-passes task itself to estimate (neuro-)mechanical impedance during motion. Our apparatus (hand-held, finishing tool instrumented with motion capture and multi-axis force/torque sensors) and algorithms (for filtering and impedance estimation) were first tested on an impedance-controlled industrial robot carrying out the finishing task of interest, where the impedance could be pre-programmed. We were able to accurately estimate impedance in this case. The same apparatus and algorithms were then applied to the same task performed by a human operators. The stiffness values of the human operator, at different force level, correlated positively with the muscular activity, measured during the same task.

A geometric framework for the estimation of joint stiffness of the human wrist.

Estimating joint stiffness is of paramount importance for studying human motor control and for clinical assessment of neurological diseases. Usually stiffness estimation is performed using cumbersome instrumentations (e.g. robots), and by approximating robot joint angles and torques to the human ones. This paper proposes a methodology and an experimental setup to measure wrist joint stiffness in unstructured environments, with the twofold aim of: 1) providing a geometric framework in order to derive angular displacements and torques at the wrist Flexion/Extension (FE) and Radial/Ulnar Deviation (RUD) axes of rotation, using a subject specific kinematic model; 2) suggesting an experimental setup made of two portable sensors for motion tracking and one load cell, to allow for measurements in out-of-the-lab scenarios. We tested our method on a hardware mockup of wrist kinematics, providing a ground truth for estimated angles and torques at FE and RUD joints. The experimental validation showed average absolute errors in FE and RUD angles of 0.005 rad and 0.0167 rad respectively, and an average error of FE and RUD torques of 0.006 Nm and 0.003 Nm.

A preliminary study on the relationship between proprioceptive deficits and motor functions in chronic stroke patients.

Although motor and sensory impairments of the upper limb after stroke have been widely studied, the relationship between sensory deficits and motor functions has been less thoroughly explored. In this ongoing study, we investigated the relationship between proprioceptive impairments and motor functions with 20 chronic stroke survivors. Their proprioceptive abilities were assessed with a passive joint position matching test using H-Man and their motor functions were assessed with ARAT (Action Research Arm Test) and FMA (Fugl Meyer Upper Extremity Assessment) clinical scores. The assessments were conducted before, during and after the therapy. Results indicated a significant difference between the proprioceptive outcomes of healthy and stroke participants (at baseline) in both matching accuracy (absolute error, p=0.02) and precision (variability of the signed error, p=0.03). Significant correlations were found between the proprioceptive assessment outcomes (assessed before the beginning of the motor rehabilitation) of stroke participants with impaired proprioception and their ARAT clinical scores assessed at the first follow-up (week 12) (rho =- 0.74 and p=0.047 for the absolute error; rho =-0.78 and p= 0.03 for the variability of the signed error). The results from this preliminary study indicated a significant relationship between proprioceptive impairments and motor function performances in proprioceptively impaired chronic stroke participants.

The effect of skill level matching in dyadic interaction on learning of a tracing task.

Dyadic interaction between humans has gained great research interest in the last years. The effects of factors that influence the interaction, as e.g. roles or skill level matching, are still not well understood. In this paper, we further investigated the effect of skill level matching between partners on learning of a visuo-motor task. Understanding the effect of skill level matching is crucial for applications in collaborative rehabilitation. Fifteen healthy participants were asked to trace a path while being subjected to a visuo-motor rotation (Novice). The Novices were paired with a partner, forming one of the three Dyad Types: a) haptic connection to another Novice, b) haptic connection to an Expert (no visuo-motor rotation), or c) no haptic. The intervention consisted of a Familiarization phase, followed by a Training phase, in which the Novices were learning the task in the respective Dyad Type, and a Test phase in which the learning was assessed (haptic connection removed, if any). Results suggest that learning of the task with a haptic connection to an Expert was least beneficial. However, during the Training phase the dyads comprising an Expert clearly outperformed the dyads with matched skill levels. The results point towards the same direction as previous findings in literature and can be explained by current motor-learning theories. Future work needs to corroborate these preliminary results.