A direct spinal cord-computer interface enables the control of the paralysed hand in spinal cord injury.

The paralysis of the muscles controlling the hand dramatically limits the quality of life of individuals living with spinal cord injury (SCI). Here, with a non-invasive neural interface, we demonstrate that eight motor complete SCI individuals (C5-C6) are still able to task-modulate in real-time the activity of populations of spinal motor neurons with residual neural pathways. In all SCI participants tested, we identified groups of motor units under voluntary control that encoded various hand movements. The motor unit discharges were mapped into more than 10 degrees of freedom, ranging from grasping to individual hand-digit flexion and extension. We then mapped the neural dynamics into a real-time controlled virtual hand. The SCI participants were able to match the cue hand posture by proportionally controlling four degrees of freedom (opening and closing the hand and index flexion/extension). These results demonstrate that wearable muscle sensors provide access to spared motor neurons that are fully under voluntary control in complete cervical SCI individuals. This non-invasive neural interface allows the investigation of motor neuron changes after the injury and has the potential to promote movement restoration when integrated with assistive devices.

Experimental evaluation of the impact of sEMG interfaces in enhancing embodiment of virtual myoelectric prostheses.

INTRODUCTION: Despite recent technological advances that have led to sophisticated bionic prostheses, attaining embodied solutions still remains a challenge. Recently, the investigation of prosthetic embodiment has become a topic of interest in the research community, which deals with enhancing the perception of artificial limbs as part of users' own body. Surface electromyography (sEMG) interfaces have emerged as a promising technology for enhancing upper-limb prosthetic control. However, little is known about the impact of these sEMG interfaces on users' experience regarding embodiment and their interaction with different functional levels. METHODS: To investigate this aspect, a comparison is conducted among sEMG configurations with different number of sensors (4 and 16 channels) and different time delay. We used a regression algorithm to simultaneously control hand closing/opening and forearm pronation/supination in an immersive virtual reality environment. The experimental evaluation includes 24 able-bodied subjects and one prosthesis user. We assess functionality with the Target Achievement Control test, and the sense of embodiment with a metric for the users perception of self-location, together with a standard survey. RESULTS: Among the four tested conditions, results proved a higher subjective embodiment when participants used sEMG interfaces employing an increased number of sensors. Regarding functionality, significant improvement over time is observed in the same conditions, independently of the time delay implemented. CONCLUSIONS: Our work indicates that a sufficient number of sEMG sensors improves both, functional and subjective embodiment outcomes. This prompts discussion regarding the potential relationship between these two aspects present in bionic integration. Similar embodiment outcomes are observed in the prosthesis user, showing also differences due to the time delay, and demonstrating the influence of sEMG interfaces on the sense of agency.

Simultaneous assessment and training of an upper-limb amputee using incremental machine-learning-based myocontrol: a single-case experimental design.

BACKGROUND: Machine-learning-based myocontrol of prosthetic devices suffers from a high rate of abandonment due to dissatisfaction with the training procedure and with the reliability of day-to-day control. Incremental myocontrol is a promising approach as it allows on-demand updating of the system, thus enforcing continuous interaction with the user. Nevertheless, a long-term study assessing the efficacy of incremental myocontrol is still missing, partially due to the lack of an adequate tool to do so. In this work we close this gap and report about a person with upper-limb absence who learned to control a dexterous hand prosthesis using incremental myocontrol through a novel functional assessment protocol called SATMC (Simultaneous Assessment and Training of Myoelectric Control). METHODS: The participant was fitted with a custom-made prosthetic setup with a controller based on Ridge Regression with Random Fourier Features (RR-RFF), a non-linear, incremental machine learning method, used to build and progressively update the myocontrol system. During a 13-month user study, the participant performed increasingly complex daily-living tasks, requiring fine bimanual coordination and manipulation with a multi-fingered hand prosthesis, in a realistic laboratory setup. The SATMC was used both to compose the tasks and continually assess the participant's progress. Patient satisfaction was measured using Visual Analog Scales. RESULTS: Over the course of the study, the participant progressively improved his performance both objectively, e.g., the time required to complete each task became shorter, and subjectively, meaning that his satisfaction improved. The SATMC actively supported the improvement of the participant by progressively increasing the difficulty of the tasks in a structured way. In combination with the incremental RR-RFF allowing for small adjustments when required, the participant was capable of reliably using four actions of the prosthetic hand to perform all required tasks at the end of the study. CONCLUSIONS: Incremental myocontrol enabled an upper-limb amputee to reliably control a dexterous hand prosthesis while providing a subjectively satisfactory experience. The SATMC can be an effective tool to this aim.

Human-in-the-Loop Assessment of an Ultralight, Low-Cost Body Posture Tracking Device.

In rehabilitation, assistive and space robotics, the capability to track the body posture of a user in real time is highly desirable. In more specific cases, such as teleoperated extra-vehicular activity, prosthetics and home service robotics, the ideal posture-tracking device must also be wearable, light and low-power, while still enforcing the best possible accuracy. Additionally, the device must be targeted at effective human-machine interaction. In this paper, we present and test such a device based upon commercial inertial measurement units: it weighs 575 grams in total, lasts up to 10.5 hours of continual operation, can be donned and doffed in under a minute and costs less than 290 EUR. We assess the attainable performance in terms of error in an online trajectory-tracking task in Virtual Reality using the device through an experiment involving 10 subjects, showing that an average user can attain a precision of 0.66 cm during a static precision task and 6.33 cm while tracking a moving trajectory, when tested in the full peri-personal space of a user.

Evidence of muscle synergies during human grasping.

Motor synergies have been investigated since the 1980s as a simplifying representation of motor control by the nervous system. This way of representing finger positional data is in particular useful to represent the kinematics of the human hand. Whereas, so far, the focus has been on kinematic synergies, that is common patterns in the motion of the hand and fingers, we hereby also investigate their force aspects, evaluated through surface electromyography (sEMG). We especially show that force-related motor synergies exist, i.e. that muscle activation during grasping, as described by the sEMG signal, can be grouped synergistically; that these synergies are largely comparable to one another across human subjects notwithstanding the disturbances and inaccuracies typical of sEMG; and that they are physiologically feasible representations of muscular activity during grasping. Potential applications of this work include force control of mechanical hands, especially when many degrees of freedom must be simultaneously controlled.

Omnidirectional endpoint force control through Functional Electrical Stimulation.

OBJECTIVE: in recent years, Functional Electrical Stimulation has found many applications both within and outside the medical field. However, most available wearable FES devices are not easily adaptable to different users, and most setups rely on task-specific control schemes. APPROACH: in this article, we present a peripheral stimulation prototype featuring a compressive jacket which allows to easily modify the electrode arrangement to better fit any body frame. Coupled with a suitable control system, this device can induce the output of arbitrary forces at the end-effector, which is the basis to facilitate universal, task-independent impedance control of the human limbs. Here, the device is validated by having it provide stimulation currents that should induce a desired force output. The forces exerted by the user as a result of stimulation are measured through a 6-axis force-torque sensor, and compared to the desired forces. Furthermore, here we present the offline analysis of a regression algorithm, trained on the data acquired during the aforementioned validation, which is able to reliably predict the force output based on the stimulation currents. MAIN RESULTS: open-loop control of the output force is possible with correlation coefficients between commanded and measured force output direction up to 0.88. A twitch-based calibration procedure shows significant reduction of the RMS error in the online control. The regression algorithm trained offline is able to predict the force output given the injected stimulation with correlations up to 0.94, and average normalized errors of 0.12 RMS. SIGNIFICANCE: A reliable force output control through FES is the first basis towards higher-level FES force controls. This could eventually provide full, general-purpose control of the human neuromuscular system, which would allow to induce any desired movement in the peri-personal space in individuals affected by e.g. spinal cord injury.

Robot-Inspired Human Impedance Control Through Functional Electrical Stimulation.

Functional Electrical Stimulation is an effective tool to foster rehabilitation of neurological patients suffering from impaired motor functions. It can also serve as an assistive device to compensate for compromised motor functions in the chronic phase occurring after a disease or trauma. In all cases, the dominant paradigm in FES applications is that of aiding specialized, task-specific movements, such as reaching or grasping. Usually this is achieved by targeting specific muscle groups which are associated to the targeted motion by experts. A general purpose, FES-based control theory capable of enabling neurological patients to achieve a wide range of positional goals in their peri-personal space is still missing. In this paper, we present an early analysis of the performance achievable through a muscular impedance control loop employing FES to actuate force and movement. The control is evaluated in a test where the user's upper limb is moved by means of an exonerve to a series of target positions on a plane without providing visual feedback nor requiring volitional effort. The results allow to characterize the performance of such a setup over time and to assess how well can it generalize over different target positions in the user's peri-personal space. The current study population also allows to evaluate the effects of user's experience with FES systems on the overall performance during the test. The results indicate that the proposed control loop can generalize well over different arm poses.

Adaptive Filter for Biosignal-Driven Force Controls Preserves Predictive Powers of sEMG.

Electromyographic controls based on machine learning rely on the stability and repeatability of signals related to muscular activity. However, such algorithms are prone to several issues, making them non-viable in certain applications with low tolerances for delays and signal instability, such as exoskeleton control or teleimpedance. These issues can become dramatic whenever, e.g., muscular activity is present not only when the user is trying to move but also for mere gravity compensation, which generally becomes more prominent the more proximal a muscle is. A substantial part of this instability is attributed to electromyography's inherent heteroscedasticity. In this study, we introduce and characterize an adaptive filter for sEMG features in such applications, which automatically adjusts its own cutoff frequency to suit the current movement intention. The adaptive filter is tested offline and online on a regression-based joint torque predictor. Both the offline and the online test show that the adaptive filter leads to more accurate prediction in terms of root mean square error when compared to the unfiltered prediction and higher responsiveness of the signal in terms of lag when compared to the output of a conventional low-pass filter.

Simultaneous and Proportional Real-Time Myocontrol of Up to Three Degrees of Freedom of the Wrist and Hand.

Achieving robust, intuitive, simultaneous and proportional control over multiple degrees of freedom (DOFs) is an outstanding challenge in the development of myoelectric prosthetic systems. Since the priority in myoelectric prosthesis solutions is robustness and stability, their number of functions is usually limited. OBJECTIVE: Here, we introduce a system for intuitive concurrent hand and wrist control, based on a robust feature-extraction protocol and machine-learning. METHODS: Using the mean absolute value of high-density EMG, we train a ridge-regressor (RR) on only the sustained portions of the single-DOF contractions and leverage the regressor's inherent ability to provide simultaneous multi-DOF estimates. In this way, we robustly capture the amplitude information of the inputs while harnessing the power of the RR to extrapolate otherwise noisy and often overfitted estimations of dynamic portions of movements. RESULTS: The real-time evaluation of the system on 13 able-bodied participants and an amputee shows that almost all single-DOF tasks could be reached (96% success rate), while at the same time users were able to complete most of the two-DOF (62%) and even some of the very challenging three-DOF tasks (37%). To further investigate the translational potential of the approach, we reduced the original 192-channel setup to a 16-channel configuration and the observed performance did not deteriorate. Notably, the amputee performed similarly well to the other participants, according to all considered metrics. CONCLUSION: This is the first real-time operated myocontrol system that consistently provides intuitive simultaneous and proportional control over 3-DOFs of wrist and hand, relying on only surface EMG signals from the forearm. SIGNIFICANCE: Focusing on reduced complexity, a real-time test and the inclusion of an amputee in the study demonstrate the translational potential of the control system for future applications in prosthetic control.

Progressive unsupervised control of myoelectric upper limbs.

Objective.Unsupervised myocontrol methods aim to create control models for myoelectric prostheses while avoiding the complications of acquiring reliable, regular, and sufficient labeled training data. A limitation of current unsupervised methods is that they fix the number of controlled prosthetic functions a priori, thus requiring an initial assessment of the user's motor skills and neglecting the development of novel motor skills over time.Approach.We developed a progressive unsupervised myocontrol (PUM) paradigm in which the user and the control model coadaptively identify distinct muscle synergies, which are then used to control arbitrarily associated myocontrol functions, each corresponding to a hand or wrist movement. The interaction starts with learning a single function and the user may request additional functions after mastering the available ones, which aligns the evolution of their motor skills with an increment in system complexity. We conducted a multi-session user study to evaluate PUM and compare it against a state-of-the-art non-progressive unsupervised alternative. Two participants with congenital upper-limb differences tested PUM, while ten non-disabled control participants tested either PUM or the non-progressive baseline. All participants engaged in myoelectric control of a virtual hand and wrist.Main results.PUM enabled autonomous learning of three myocontrol functions for participants with limb differences, and of all four available functions for non-disabled subjects, using both existing or newly identified muscle synergies. Participants with limb differences achieved similar success rates to non-disabled ones on myocontrol tests, but faced greater difficulties in internalizing new motor skills and exhibited slightly inferior movement quality. The performance was comparable with either PUM or the non-progressive baseline for the group of non-disabled participants.Significance.The PUM paradigm enables users to autonomously learn to operate the myocontrol system, adapts to the users' varied preexisting motor skills, and supports the further development of those skills throughout practice.

Preliminary Assessment of Two Simultaneous and Proportional Myocontrol Methods for 3-DoFs Prostheses Using Incremental Learning.

Despite progressive developments over the last decades, current upper limb prostheses still lack a suitable control able to fully restore the functionalities of the lost arm. Traditional control approaches for prostheses fail when simultaneously actuating multiple Degrees of Freedom (DoFs), thus limiting their usability in daily-life scenarios. Machine learning, on the one hand, offers a solution to this issue through a promising approach for decoding user intentions but fails when input signals change. Incremental learning, on the other hand, reduces sources of error by quickly updating the model on new data rather than training the control model from scratch. In this study, we present an initial evaluation of a position and a velocity control strategy for simultaneous and proportional control over 3-DoFs based on incremental learning. The proposed controls are tested using a virtual Hannes prosthesis on two healthy participants. The performances are evaluated over eight sessions by performing the Target Achievement Control test and administering SUS and NASA-TLX questionnaires. Overall, this preliminary study demonstrates that both control strategies are promising approaches for prosthetic control, offering the potential to improve the usability of prostheses for individuals with limb loss. Further research extended to a wider population of both healthy subjects and amputees will be essential to thoroughly assess these control paradigms.

Unsupervised Myocontrol of a Virtual Hand Based on a Coadaptive Abstract Motor Mapping.

Applications of simultaneous and proportional control for upper-limb prostheses typically rely on supervised machine learning to map muscle activations to prosthesis movements. This scheme often poses problems for individuals with limb differences, as they may not be able to reliably reproduce the training activations required to construct a natural motor mapping. We propose an unsupervised myocontrol paradigm that eliminates the need for labeled data by mapping the most salient muscle synergies in arbitrary order to a number of predefined prosthesis actions. The paradigm is coadaptive, in the sense that while the user learns to control the system via interaction, the system continually refines the identification of the user's muscular synergies. Our evaluation consisted of eight subjects without limb-loss performing target achievement control tasks of four actions of the hand and wrist. The subjects achieved comparable performance using the proposed unsupervised myocontrol paradigm and a supervised benchmark method, despite reporting increased mental load with the former.

A Comprehensive Framework for the Modelling of Cartesian Force Output in Human Limbs.

Neuromuscular functional electrical stimulation represents a valid technique for functional rehabilitation or, in the form of a neuroprosthesis, for the assistance of neurological patients. However, the selected stimulation of single muscles through surface electrodes remains challenging particularly for the upper extremity. In this paper, we present the MyoCeption, a comprehensive setup, which enables intuitive modeling of the user's musculoskeletal system, as well as proportional stimulation of the muscles with 16-bit resolution through up to 10 channels. The system can be used to provide open-loop force control, which, if coupled with an adequate body tracking system, can be used to implement an impedance control where the control loop is closed around the body posture. The system is completely self-contained and can be used in a wide array of scenarios, from rehabilitation to VR to teleoperation. Here, the MyoCeption's control environment has been experimentally validated through comparison with a third-party simulation suite. The results indicate that the musculoskeletal model used for the MyoCeption provides muscle geometries that are qualitatively similar to those computed in the baseline model.

Unobtrusive, natural support control of an adaptive industrial exoskeleton using force myography.

Repetitive or tiring tasks and movements during manual work can lead to serious musculoskeletal disorders and, consequently, to monetary damage for both the worker and the employer. Among the most common of these tasks is overhead working while operating a heavy tool, such as drilling, painting, and decorating. In such scenarios, it is desirable to provide adaptive support in order to take some of the load off the shoulder joint as needed. However, even to this day, hardly any viable approaches have been tested, which could enable the user to control such assistive devices naturally and in real time. Here, we present and assess the adaptive Paexo Shoulder exoskeleton, an unobtrusive device explicitly designed for this kind of industrial scenario, which can provide a variable amount of support to the shoulders and arms of a user engaged in overhead work. The adaptive Paexo Shoulder exoskeleton is controlled through machine learning applied to force myography. The controller is able to determine the lifted mass and provide the required support in real time. Twelve subjects joined a user study comparing the Paexo driven through this adaptive control to the Paexo locked in a fixed level of support. The results showed that the machine learning algorithm can successfully adapt the level of assistance to the lifted mass. Specifically, adaptive assistance can sensibly reduce the muscle activity's sensitivity to the lifted mass, with an observed relative reduction of up to 31% of the muscular activity observed when lifting 2 kg normalized by the baseline when lifting no mass.

Interaction in Assistive Robotics: A Radical Constructivist Design Framework.

Despite decades of research, muscle-based control of assistive devices (myocontrol) is still unreliable; for instance upper-limb prostheses, each year more and more dexterous and human-like, still provide hardly enough functionality to justify their cost and the effort required to use them. In order to try and close this gap, we propose to shift the goal of myocontrol from guessing intended movements to creating new circular reactions in the constructivist sense defined by Piaget. To this aim, the myocontrol system must be able to acquire new knowledge and forget past one, and knowledge acquisition/forgetting must happen on demand, requested either by the user or by the system itself. We propose a unifying framework based upon Radical Constructivism for the design of such a myocontrol system, including its user interface and user-device interaction strategy.

Learning to teleoperate an upper-limb assistive humanoid robot for bimanual daily-living tasks.

Objective.Bimanual humanoid platforms for home assistance are nowadays available, both as academic prototypes and commercially. Although they are usually thought of as daily helpers for non-disabled users, their ability to move around, together with their dexterity, makes them ideal assistive devices for upper-limb disabled persons, too. Indeed, teleoperating a bimanual robotic platform via muscle activation could revolutionize the way stroke survivors, amputees and patients with spinal injuries solve their daily home chores. Moreover, with respect to direct prosthetic control, teleoperation has the advantage of freeing the user from the burden of the prosthesis itself, overpassing several limitations regarding size, weight, or integration, and thus enables a much higher level of functionality.Approach.In this study, nine participants, two of whom suffer from severe upper-limb disabilities, teleoperated a humanoid assistive platform, performing complex bimanual tasks requiring high precision and bilateral arm/hand coordination, simulating home/office chores. A wearable body posture tracker was used for position control of the robotic torso and arms, while interactive machine learning applied to electromyography of the forearms helped the robot to build an increasingly accurate model of the participant's intent over time.Main results.All participants, irrespective of their disability, were uniformly able to perform the demanded tasks. Completion times, subjective evaluation scores, as well as energy- and time- efficiency show improvement over time on short and long term.Significance.This is the first time a hybrid setup, involving myoeletric and inertial measurements, is used by disabled people to teleoperate a bimanual humanoid robot. The proposed setup, taking advantage of interactive machine learning, is simple, non-invasive, and offers a new assistive solution for disabled people in their home environment. Additionnally, it has the potential of being used in several other applications in which fine humanoid robot control is required.

The Merits of Dynamic Data Acquisition for Realistic Myocontrol.

Natural myocontrol is the intuitive control of a prosthetic limb via the user's voluntary muscular activations. This type of control is usually implemented by means of pattern recognition, which uses a set of training data to create a model that can decipher these muscular activations. A consequence of this approach is that the reliability of a myocontrol system depends on how representative this training data is for all types of signal variability that may be encountered when the amputee puts the prosthesis into real use. Myoelectric signals are indeed known to vary according to the position and orientation of the limb, among other factors, which is why it has become common practice to take this variability into account by acquiring training data in multiple body postures. To shed further light on this problem, we compare two ways of collecting data: while the subjects hold their limb statically in several positions one at a time, which is the traditional way, or while they dynamically move their limb at a constant pace through those same positions. Since our interest is to investigate any differences when controlling an actual prosthetic device, we defined an evaluation protocol that consisted of a series of complex, bimanual daily-living tasks. Fourteen intact participants performed these tasks while wearing prosthetic hands mounted on splints, which were controlled via either a statically or dynamically built myocontrol model. In both cases all subjects managed to complete all tasks and participants without previous experience in myoelectric control manifested a significant learning effect; moreover, there was no significant difference in the task completion times achieved with either model. When evaluated in a simulated scenario with traditional offline performance evaluation, on the other hand, the dynamically-trained system showed significantly better accuracy. Regardless of the setting, the dynamic data acquisition was faster, less tiresome, and better accepted by the users. We conclude that dynamic data acquisition is advantageous and confirm the limited relevance of offline analyses for online myocontrol performance.

Online Natural Myocontrol of Combined Hand and Wrist Actions Using Tactile Myography and the Biomechanics of Grasping.

Objective: Despite numerous recent advances in the field of rehabilitation robotics, simultaneous, and proportional control of hand and/or wrist prostheses is still unsolved. In this work we concentrate on myocontrol of combined actions, for instance power grasping while rotating the wrist, by only using training data gathered from single actions. This is highly desirable since gathering data for all possible combined actions would be unfeasibly long and demanding for the amputee. Approach: We first investigated physiologically feasible limits for muscle activation during combined actions. Using these limits we involved 12 intact participants and one amputee in a Target Achievement Control test, showing that tactile myography, i.e., high-density force myography, solves the problem of combined actions to a remarkable extent using simple linear regression. Since real-time usage of many sensors can be computationally demanding, we compare this approach with another one using a reduced feature set. These reduced features are obtained using a fast, spatial first-order approximation of the sensor values. Main results: By using the training data of single actions only, i.e., power grasp or wrist movements, subjects achieved an average success rate of 70.0% in the target achievement test using ridge regression. When combining wrist actions, e.g., pronating and flexing the wrist simultaneously, similar results were obtained with an average of 68.1%. If a power grasp is added to the pool of actions, combined actions are much more difficult to achieve (36.1%). Significance: To the best of our knowledge, for the first time, the effectiveness of tactile myography on single and combined actions is evaluated in a target achievement test. The present study includes 3 DoFs control instead of the two generally used in the literature. Additionally, we define a set of physiologically plausible muscle activation limits valid for most experiments of this kind.

Feedback-aided data acquisition improves myoelectric control of a prosthetic hand.

OBJECTIVE: Pattern-recognition-based myocontrol can be unreliable, which may limit its use in the clinical practice and everyday activities. One cause for this is the poor generalization of the underlying machine learning models to untrained conditions. Acquiring the training data and building the model more interactively can reduce this problem. For example, the user could be encouraged to target the model's instabilities during the data acquisition supported by automatic feedback guidance. Interactivity is an emerging trend in myocontrol of upper-limb electric prostheses: the user should be actively involved throughout the training and usage of the device. APPROACH: In this study, 18 non-disabled participants tested two novel feedback-aided acquisition protocols against a standard one that did not provide any guidance. All the protocols acquired data dynamically in multiple arm positions to counteract the limb position effect. During feedback-aided acquisition, an acoustic signal urged the participant to hover with the arm in specific regions of her peri-personal space, de facto acquiring more data where needed. The three protocols were compared on everyday manipulation tasks performed with a prosthetic hand. MAIN RESULTS: Our results showed that feedback-aided data acquisition outperformed the acquisition routine without guidance, both objectively and subjectively. SIGNIFICANCE: This indicates that the interaction with the user during the data acquisition is fundamental to improve myocontrol.