User training for machine learning controlled upper limb prostheses: a serious game approach.

BACKGROUND: Upper limb prosthetics with multiple degrees of freedom (DoFs) are still mostly operated through the clinical standard Direct Control scheme. Machine learning control, on the other hand, allows controlling multiple DoFs although it requires separable and consistent electromyogram (EMG) patterns. Whereas user training can improve EMG pattern quality, conventional training methods might limit user potential. Training with serious games might lead to higher quality EMG patterns and better functional outcomes. In this explorative study we compare outcomes of serious game training with conventional training, and machine learning control with the users' own one DoF prosthesis. METHODS: Participants with upper limb absence participated in 7 training sessions where they learned to control a 3 DoF prosthesis with two grips which was fitted. Participants received either game training or conventional training. Conventional training was based on coaching, as described in the literature. Game-based training was conducted using two games that trained EMG pattern separability and functional use. Both groups also trained functional use with the prosthesis donned. The prosthesis system was controlled using a neural network regressor. Outcome measures were EMG metrics, number of DoFs used, the spherical subset of the Southampton Hand Assessment Procedure and the Clothespin Relocation Test. RESULTS: Eight participants were recruited and four completed the study. Training did not lead to consistent improvements in EMG pattern quality or functional use, but some participants improved in some metrics. No differences were observed between the groups. Participants achieved consistently better results using their own prosthesis than the machine-learning controlled prosthesis used in this study. CONCLUSION: Our explorative study showed in a small group of participants that serious game training seems to achieve similar results as conventional training. No consistent improvements were found in either group in terms of EMG metrics or functional use, which might be due to insufficient training. This study highlights the need for more research in user training for machine learning controlled prosthetics. In addition, this study contributes with more data comparing machine learning controlled prosthetics with Direct Controlled prosthetics.

Phantom motor execution facilitated by machine learning and augmented reality as treatment for phantom limb pain: a single group, clinical trial in patients with chronic intractable phantom limb pain.

BACKGROUND: Phantom limb pain is a debilitating condition for which no effective treatment has been found. We hypothesised that re-engagement of central and peripheral circuitry involved in motor execution could reduce phantom limb pain via competitive plasticity and reversal of cortical reorganisation. METHODS: Patients with upper limb amputation and known chronic intractable phantom limb pain were recruited at three clinics in Sweden and one in Slovenia. Patients received 12 sessions of phantom motor execution using machine learning, augmented and virtual reality, and serious gaming. Changes in intensity, frequency, duration, quality, and intrusion of phantom limb pain were assessed by the use of the numeric rating scale, the pain rating index, the weighted pain distribution scale, and a study-specific frequency scale before each session and at follow-up interviews 1, 3, and 6 months after the last session. Changes in medication and prostheses were also monitored. Results are reported using descriptive statistics and analysed by non-parametric tests. The trial is registered at ClinicalTrials.gov, number NCT02281539. FINDINGS: Between Sept 15, 2014, and April 10, 2015, 14 patients with intractable chronic phantom limb pain, for whom conventional treatments failed, were enrolled. After 12 sessions, patients showed statistically and clinically significant improvements in all metrics of phantom limb pain. Phantom limb pain decreased from pre-treatment to the last treatment session by 47% (SD 39; absolute mean change 1·0 [0·8]; p=0·001) for weighted pain distribution, 32% (38; absolute mean change 1·6 [1·8]; p=0·007) for the numeric rating scale, and 51% (33; absolute mean change 9·6 [8·1]; p=0·0001) for the pain rating index. The numeric rating scale score for intrusion of phantom limb pain in activities of daily living and sleep was reduced by 43% (SD 37; absolute mean change 2·4 [2·3]; p=0·004) and 61% (39; absolute mean change 2·3 [1·8]; p=0·001), respectively. Two of four patients who were on medication reduced their intake by 81% (absolute reduction 1300 mg, gabapentin) and 33% (absolute reduction 75 mg, pregabalin). Improvements remained 6 months after the last treatment. INTERPRETATION: Our findings suggest potential value in motor execution of the phantom limb as a treatment for phantom limb pain. Promotion of phantom motor execution aided by machine learning, augmented and virtual reality, and gaming is a non-invasive, non-pharmacological, and engaging treatment with no identified side-effects at present. FUNDING: Promobilia Foundation, VINNOVA, Jimmy Dahlstens Fond, PicoSolve, and Innovationskontor Väst.

Towards Implementation of a Home-Based Phantom Limb Pain Treatment Facilitated by Textile-Electrode System - A Case Study.

This case study reports the use of a new textile-electrode system for self-administered Phantom Motor Execution (PME) treatment at home in one patient with Phantom Limb Pain (PLP). In follow-up interviews, the patient reported reduced pain, increased mobility, and improved mental health, and aspects such as motivation, usability, support, and treatment outcome, could be recognized from an earlier study as crucial for successful implementation and adoption of the home-based long-term treatment. The findings are of interest to developers, providers, users, and researchers planning home-based clinical studies and/or scenarios based on technology-assisted treatment.

Analysis of Neural Network Based Proportional Myoelectric Hand Prosthesis Control.

OBJECTIVE: We show that state-of-the-art deep neural networks achieve superior results in regression-based multi-class proportional myoelectric hand prosthesis control than two common baseline approaches, and we analyze the neural network mapping to explain why this is the case. METHODS: Feedforward neural networks and baseline systems are trained on an offline corpus of 11 able-bodied subjects and 4 prosthesis wearers, using the R2 score as metric. Analysis is performed using diverse qualitative and quantitative approaches, followed by a rigorous evaluation. RESULTS: Our best neural networks have at least three hidden layers with at least 128 neurons per layer; smaller architectures, as used by many prior studies, perform substantially worse. The key to good performance is to both optimally regress the target movement, and to suppress spurious movements. Due to the properties of the underlying data, this is impossible to achieve with linear methods, but can be attained with high exactness using sufficiently large neural networks. CONCLUSION: Neural networks perform significantly better than common linear approaches in the given task, in particular when sufficiently large architectures are used. This can be explained by salient properties of the underlying data, and by theoretical and experimental analysis of the neural network mapping. SIGNIFICANCE: To the best of our knowledge, this work is the first one in the field which not only reports that large and deep neural networks are superior to existing architectures, but also explains this result.

Upper Limb Stroke Rehabilitation Using Surface Electromyography: A Systematic Review and Meta-Analysis.

Background: Upper limb impairment is common after stroke, and many will not regain full upper limb function. Different technologies based on surface electromyography (sEMG) have been used in stroke rehabilitation, but there is no collated evidence on the different sEMG-driven interventions and their effect on upper limb function in people with stroke. Aim: Synthesize existing evidence and perform a meta-analysis on the effect of different types of sEMG-driven interventions on upper limb function in people with stroke. Methods: PubMed, SCOPUS, and PEDro databases were systematically searched for eligible randomized clinical trials that utilize sEMG-driven interventions to improve upper limb function assessed by Fugl-Meyer Assessment (FMA-UE) in stroke. The PEDro scale was used to evaluate the methodological quality and the risk of bias of the included studies. In addition, a meta-analysis utilizing a random effect model was performed for studies comparing sEMG interventions to non-sEMG interventions and for studies comparing different sEMG interventions protocols. Results: Twenty-four studies comprising 808 participants were included in this review. The methodological quality was good to fair. The meta-analysis showed no differences in the total effect, assessed by total FMA-UE score, comparing sEMG interventions to non-sEMG interventions (14 studies, 509 participants, SMD 0.14, P 0.37, 95% CI -0.18 to 0.46, I2 55%). Similarly, no difference in the overall effect was found for the meta-analysis comparing different types of sEMG interventions (7 studies, 213 participants, SMD 0.42, P 0.23, 95% CI -0.34 to 1.18, I2 73%). Twenty out of the twenty-four studies, including participants with varying impairment levels at all stages of stroke recovery, reported statistically significant improvements in upper limb function at post-sEMG intervention compared to baseline. Conclusion: This review and meta-analysis could not discern the effect of sEMG in comparison to a non-sEMG intervention or the most effective type of sEMG intervention for improving upper limb function in stroke populations. Current evidence suggests that sEMG is a promising tool to further improve functional recovery, but randomized clinical trials with larger sample sizes are needed to verify whether the effect on upper extremity function of a specific sEMG intervention is superior compared to other non-sEMG or other type of sEMG interventions.

Exploring the Relationship Between EMG Feature Space Characteristics and Control Performance in Machine Learning Myoelectric Control.

In myoelectric machine learning (ML) based control, it has been demonstrated that control performance usually increases with training, but it remains largely unknown which underlying factors govern these improvements. It has been suggested that the increase in performance originates from changes in characteristics of the Electromyography (EMG) patterns, such as separability or repeatability. However, the relation between these EMG metrics and control performance has hardly been studied. We assessed the relation between three common EMG feature space metrics (separability, variability and repeatability) in 20 able bodied participants who learned ML myoelectric control in a virtual task over 15 training blocks on 5 days. We assessed the change in offline and real-time performance, as well as the change of each EMG metric over the training. Subsequently, we assessed the relation between individual EMG metrics and offline and real-time performance via correlation analysis. Last, we tried to predict real-time performance from all EMG metrics via L2-regularized linear regression. Results showed that real-time performance improved with training, but there was no change in offline performance or in any of the EMG metrics. Furthermore, we only found a very low correlation between separability and real-time performance and no correlation between any other EMG metric and real-time performance. Finally, real-time performance could not be successfully predicted from all EMG metrics employing L2-regularized linear regression. We concluded that the three EMG metrics and real-time performance appear to be unrelated.

Should Hands Be Restricted When Measuring Able-Bodied Participants to Evaluate Machine Learning Controlled Prosthetic Hands?

OBJECTIVE: When evaluating methods for machine-learning controlled prosthetic hands, able-bodied participants are often recruited, for practical reasons, instead of participants with upper limb absence (ULA). However, able-bodied participants have been shown to often perform myoelectric control tasks better than participants with ULA. It has been suggested that this performance difference can be reduced by restricting the wrist and hand movements of able-bodied participants. However, the effect of such restrictions on the consistency and separability of the electromyogram's (EMG) features remains unknown. The present work investigates whether the EMG separability and consistency between unaffected and affected arms differ and whether they change after restricting the unaffected limb in persons with ULA. METHODS: Both arms of participants with unilateral ULA were compared in two conditions: with the unaffected hand and wrist restricted or not. Furthermore, it was tested if the effect of arm and restriction is influenced by arm posture (arm down, arm in front, or arm up). RESULTS: Fourteen participants (two women, age = 53.4±4.05) with acquired transradial limb loss were recruited. We found that the unaffected limb generated more separated EMG than the affected limb. Furthermore, restricting the unaffected hand and wrist lowered the separability of the EMG when the arm was held down. CONCLUSION: Limb restriction is a viable method to make the EMG of able-bodied participants more similar to that of participants with ULA. SIGNIFICANCE: Future research that evaluates methods for machine learning controlled hands in able-bodied participants should restrict the participants' hand and wrist.

Users' and therapists' perceptions of myoelectric multi-function upper limb prostheses with conventional and pattern recognition control.

OBJECTIVE: To describe users' and therapists' opinions on multi-function myoelectric upper limb prostheses with conventional control and pattern recognition control. DESIGN: Qualitative interview study. SETTINGS: Two rehabilitation institutions in the Netherlands and one in Austria. SUBJECTS: The study cohort consisted of 15 prosthesis users (13 males, mean age: 43.7 years, average experience with multi-function prosthesis: 3.15 years) and seven therapists (one male, mean age: 44.1 years, average experience with multi-function prostheses: 6.6 years). Four of these users and one therapist had experience with pattern recognition control. METHOD: This study consisted of semi-structured interviews. The participants were interviewed at their rehabilitation centres or at home by telephone. The thematic framework approach was used for analysis. RESULTS: The themes emerging from prosthesis users and therapists were largely congruent and resulted in one thematic framework with three main themes: control, prosthesis, and activities. The participants mostly addressed (dis-) satisfaction with the control type and the prosthesis itself and described the way they used their prostheses in daily tasks. CONCLUSION: Prosthesis users and therapists described multi-function upper limb prostheses as more functional devices than conventional one-degree-of-freedom prostheses. Nonetheless, the prostheses were seldom used to actively grasp and manipulate objects. Moreover, the participants clearly expressed their dissatisfaction with the mechanical robustness of the devices and with the process of switching prosthesis function under conventional control. Pattern recognition was appreciated as an intuitive control that facilitated fast switching between prosthesis functions, but was reported to be too unreliable for daily use and require extensive training.

The Effect of Feedback During Training Sessions on Learning Pattern-Recognition-Based Prosthesis Control.

Human-machine interfaces have not yet advanced to enable intuitive control of multiple degrees of freedom as offered by modern myoelectric prosthetic hands. Pattern Recognition (PR) control has been proposed to make human-machine interfaces in myoelectric prosthetic hands more intuitive, but it requires the user to generate high-quality, i.e., consistent and separable, electromyogram (EMG) patterns. To generate such patterns, user training is required and has shown promising results. However, how different levels of feedback affect effectivity in training differently, has not been established yet. Furthermore, a correlation between qualities of the EMG patterns (the focus of training) and user performance has not been shown yet. In this study, 37 able-bodied participants (mean age 21 years, 19 males) were recruited and trained PR control over five days. Three levels of feedback were tested for their effectiveness: no external feedback, visual feedback and visual feedback with coaching. Training resulted in improved performance from pre- to post-test with no interaction effect of feedback. Feedback did however affect the quality of the EMG patterns where people who did not receive external feedback generated higher amplitude patterns. A weak correlation was found between a principal component, composed of EMG amplitude and pattern variability, and performance. Our results show that training is highly effective in improving PR control regardless of feedback and that none of the quality metrics correlate with performance. We discuss how different levels of feedback can be leveraged to improve PR control training.

Treatment of phantom limb pain (PLP) based on augmented reality and gaming controlled by myoelectric pattern recognition: a case study of a chronic PLP patient.

A variety of treatments have been historically used to alleviate phantom limb pain (PLP) with varying efficacy. Recently, virtual reality (VR) has been employed as a more sophisticated mirror therapy. Despite the advantages of VR over a conventional mirror, this approach has retained the use of the contralateral limb and is therefore restricted to unilateral amputees. Moreover, this strategy disregards the actual effort made by the patient to produce phantom motions. In this work, we investigate a treatment in which the virtual limb responds directly to myoelectric activity at the stump, while the illusion of a restored limb is enhanced through augmented reality (AR). Further, phantom motions are facilitated and encouraged through gaming. The proposed set of technologies was administered to a chronic PLP patient who has shown resistance to a variety of treatments (including mirror therapy) for 48 years. Individual and simultaneous phantom movements were predicted using myoelectric pattern recognition and were then used as input for VR and AR environments, as well as for a racing game. The sustained level of pain reported by the patient was gradually reduced to complete pain-free periods. The phantom posture initially reported as a strongly closed fist was gradually relaxed, interestingly resembling the neutral posture displayed by the virtual limb. The patient acquired the ability to freely move his phantom limb, and a telescopic effect was observed where the position of the phantom hand was restored to the anatomically correct distance. More importantly, the effect of the interventions was positively and noticeably perceived by the patient and his relatives. Despite the limitation of a single case study, the successful results of the proposed system in a patient for whom other medical and non-medical treatments have been ineffective justifies and motivates further investigation in a wider study.