Tactile feedback is an effective instrument for the training of grasping with a prosthesis at low- and medium-force levels.

Grasping is a complex task routinely performed in an anticipatory (feedforward) manner, where sensory feedback is responsible for learning and updating the internal model of grasp dynamics. This study aims at evaluating whether providing a proportional tactile force feedback during the myoelectric control of a prosthesis facilitates learning a stable internal model of the prosthesis force control. Ten able-bodied subjects controlled a sensorized myoelectric prosthesis performing four blocks of consecutive grasps at three levels of target force (30, 50, and 70%), repeatedly closing the fully opened hand. In the first and third block, the subjects received tactile and visual feedback, respectively, while during the second and fourth block, the feedback was removed. The subjects also performed an additional block with no feedback 1 day after the training (Retest). The median and interquartile range of the generated forces was computed to assess the accuracy and precision of force control. The results demonstrated that the feedback was indeed an effective instrument for the training of prosthesis control. After the training, the subjects were still able to accurately generate the desired force for the low and medium target (30 and 50% of maximum force available in a prosthesis), despite the feedback being removed within the session and during the retest (low target force). However, the training was substantially less successful for high forces (70% of prosthesis maximum force), where subjects exhibited a substantial loss of accuracy as soon as the feedback was removed. The precision of control decreased with higher forces and it was consistent across conditions, determined by an intrinsic variability of repeated myoelectric grasping. This study demonstrated that the subject could rely on the tactile feedback to adjust the motor command to the prosthesis across trials. The subjects adjusted the mean level of muscle activation (accuracy), whereas the precision could not be modulated as it depends on the intrinsic myoelectric variability. They were also able to maintain the feedforward command even after the feedback was removed, demonstrating thereby a stable learning, but the retention depended on the level of the target force. This is an important insight into the role of feedback as an instrument for learning of anticipatory prosthesis force control.

EMG Biofeedback for online predictive control of grasping force in a myoelectric prosthesis.

BACKGROUND: Active hand prostheses controlled using electromyography (EMG) signals have been used for decades to restore the grasping function, lost after an amputation. Although myocontrol is a simple and intuitive interface, it is also imprecise due to the stochastic nature of the EMG recorded using surface electrodes. Furthermore, the sensory feedback from the prosthesis to the user is still missing. In this study, we present a novel concept to close the loop in myoelectric prostheses. In addition to conveying the grasping force (system output), we provided to the user the online information about the system input (EMG biofeedback). METHODS: As a proof-of-concept, the EMG biofeedback was transmitted in the current study using a visual interface (ideal condition). Ten able-bodied subjects and two amputees controlled a state-of-the-art myoelectric prosthesis in routine grasping and force steering tasks using EMG and force feedback (novel approach) and force feedback only (classic approach). The outcome measures were the variability of the generated forces and absolute deviation from the target levels in the routine grasping task, and the root mean square tracking error and the number of sudden drops in the force steering task. RESULTS: During the routine grasping, the novel method when used by able-bodied subjects decreased twofold the force dispersion as well as absolute deviations from the target force levels, and also resulted in a more accurate and stable tracking of the reference force profiles during the force steering. Furthermore, the force variability during routine grasping did not increase for the higher target forces with EMG biofeedback. The trend was similar in the two amputees. CONCLUSIONS: The study demonstrated that the subjects, including the two experienced users of a myoelectric prosthesis, were able to exploit the online EMG biofeedback to observe and modulate the myoelectric signals, generating thereby more consistent commands. This allowed them to control the force predictively (routine grasping) and with a finer resolution (force steering). The future step will be to implement this promising and simple approach using an electrotactile interface. A prosthesis with a reliable response, following faithfully user intentions, would improve the utility during daily-life use and also facilitate the embodiment of the assistive system.

The impact of the stimulation frequency on closed-loop control with electrotactile feedback.

BACKGROUND: Electrocutaneous stimulation can restore the missing sensory information to prosthetic users. In electrotactile feedback, the information about the prosthesis state is transmitted in the form of pulse trains. The stimulation frequency is an important parameter since it influences the data transmission rate over the feedback channel as well as the form of the elicited tactile sensations. METHODS: We evaluated the influence of the stimulation frequency on the subject's ability to utilize the feedback information during electrotactile closed-loop control. Ten healthy subjects performed a real-time compensatory tracking (standard test bench) of sinusoids and pseudorandom signals using either visual feedback (benchmark) or electrocutaneous feedback in seven conditions characterized by different combinations of the stimulation frequency (FSTIM) and tracking error sampling rate (FTE). The tracking error was transmitted using two concentric electrodes placed on the forearm. The quality of tracking was assessed using the Squared Pearson Correlation Coefficient (SPCC), the Normalized Root Mean Square Tracking Error (NRMSTE) and the time delay between the reference and generated trajectories (TDIO). RESULTS: The results demonstrated that FSTIM was more important for the control performance than FTE. The quality of tracking deteriorated with a decrease in the stimulation frequency, SPCC and NRMSTE (mean) were 87.5% and 9.4% in the condition 100/100 (FTE/FSTIM), respectively, and deteriorated to 61.1% and 15.3% in 5/5, respectively, while the TDIO increased from 359.8 ms in 100/100 to 1009 ms in 5/5. However, the performance recovered when the tracking error sampled at a low rate was delivered using a high stimulation frequency (SPCC = 83.6%, NRMSTE = 10.3%, TDIO = 415.6 ms, in 5/100). CONCLUSIONS: The likely reason for the performance decrease and recovery was that the stimulation frequency critically influenced the tactile perception quality and thereby the effective rate of information transfer through the feedback channel. The outcome of this study can facilitate the selection of optimal system parameters for somatosensory feedback in upper limb prostheses. The results imply that the feedback variables (e.g., grasping force) should be transmitted at relatively high frequencies of stimulation (>25 Hz), but that they can be sampled at much lower rates (e.g., 5 Hz).

Toward higher-performance bionic limbs for wider clinical use.

Most prosthetic limbs can autonomously move with dexterity, yet they are not perceived by the user as belonging to their own body. Robotic limbs can convey information about the environment with higher precision than biological limbs, but their actual performance is substantially limited by current technologies for the interfacing of the robotic devices with the body and for transferring motor and sensory information bidirectionally between the prosthesis and the user. In this Perspective, we argue that direct skeletal attachment of bionic devices via osseointegration, the amplification of neural signals by targeted muscle innervation, improved prosthesis control via implanted muscle sensors and advanced algorithms, and the provision of sensory feedback by means of electrodes implanted in peripheral nerves, should all be leveraged towards the creation of a new generation of high-performance bionic limbs. These technologies have been clinically tested in humans, and alongside mechanical redesigns and adequate rehabilitation training should facilitate the wider clinical use of bionic limbs.

Myoelectric control in neurorehabilitation.

A myoelectric signal, or electromyogram (EMG), is the electrical manifestation of a muscle contraction. Through advanced signal processing techniques, information on the neural control of muscles can be extracted from the EMG, and the state of the neuromuscular system can be inferred. Because of its easy accessibility and relatively high signal-to-noise ratio, EMG has been applied as a control signal in several neurorehabilitation devices and applications, such as multi-function prostheses and orthoses, rehabilitation robots, and functional electrical stimulation/therapy. These EMG-based neurorehabilitation modules, which constitute muscle-machine interfaces, are applied for replacement, restoration, or modulation of lost or impaired function in research and clinical settings. The purpose of this review is to discuss the assumptions of EMG-based control and its applications in neurorehabilitation.

Event-related beta EEG-changes during passive and attempted foot movements in paraplegic patients.

A number of electroencephalographic (EEG) studies report on motor event-related desynchronization and synchronization (ERD/ERS) in the beta band, i.e. a decrease and increase of spectral amplitudes of central beta rhythms in the range from 13 to 35 Hz. Following an ERD that occurs shortly before and during the movement, bursts of beta oscillations (beta ERS) appear within a 1-s interval after movement offset. Such a post-movement beta ERS has been reported after voluntary hand movements, passive movements, movement imagination, and also after movements induced by functional electrical stimulation. The present study compares ERD/ERS patterns in paraplegic patients (suffering from a complete spinal cord injury) and healthy subjects during attempted (active) and passive foot movements. The aim of this work is to address the question, whether patients do have the same focal beta ERD/ERS pattern during attempted foot movement as healthy subjects do. The results showed midcentral-focused beta ERD/ERS patterns during passive, active, and imagined foot movements in healthy subjects. This is in contrast to a diffuse and broad distributed ERD/ERS pattern during attempted foot movements in patients. Only one patient showed a similar ERD/ERS pattern. Furthermore, no significant ERD/ERS patterns during passive foot movement in the group of the paraplegics could be found.

GLIMPSE: Google Glass interface for sensory feedback in myoelectric hand prostheses.

OBJECTIVE: Providing sensory feedback to the user of the prosthesis is an important challenge. The common approach is to use tactile stimulation, which is easy to implement but requires training and has limited information bandwidth. In this study, we propose an alternative approach based on augmented reality. APPROACH: We have developed the GLIMPSE, a Google Glass application which connects to the prosthesis via a Bluetooth interface and renders the prosthesis states (EMG signals, aperture, force and contact) using augmented reality (see-through display) and sound (bone conduction transducer). The interface was tested in healthy subjects that used the prosthesis with (FB group) and without (NFB group) feedback during a modified clothespins test that allowed us to vary the difficulty of the task. The outcome measures were the number of unsuccessful trials, the time to accomplish the task, and the subjective ratings of the relevance of the feedback. MAIN RESULTS: There was no difference in performance between FB and NFB groups in the case of a simple task (basic, same-color clothespins test), but the feedback significantly improved the performance in a more complex task (pins of different resistances). Importantly, the GLIMPSE feedback did not increase the time to accomplish the task. Therefore, the supplemental feedback might be useful in the tasks which are more demanding, and thereby less likely to benefit from learning and feedforward control. The subjects integrated the supplemental feedback with the intrinsic sources (vision and muscle proprioception), developing their own idiosyncratic strategies to accomplish the task. SIGNIFICANCE: The present study demonstrates a novel self-contained, ready-to-deploy, wearable feedback interface. The interface was successfully tested and was proven to be feasible and functionally beneficial. The GLIMPSE can be used as a practical solution but also as a general and flexible instrument to investigate closed-loop prosthesis control.

Quantification and visualization of event-related changes in oscillatory brain activity in the time-frequency domain.

In this chapter we review the traditional approach for ERD/ERS quantification and a more recent approach based on wavelet transform. In particular, we address the visualization of these phenomena and the validation of the results through statistical significance testing. Furthermore, we report on preprocessing using independent component analysis (ICA) and introduce a novel ERD/ERS maximization method.

A comparison of common spatial patterns with complex band power features in a four-class BCI experiment.

We report on the offline analysis of four-class brain-computer interface (BCI) data recordings. Although the analysis is done within defined time windows (cue-based BCI), our goal is to work toward an approach which classifies on-going electroencephalogram (EEG) signals without the use of such windows (un-cued BCI). To that end, we provide some elements of that analysis related to timing issues that will become important as we pursue this goal in the future. A new set of features called complex band power (CBP) features which make explicit use of phase are introduced and are shown to produce good results. As reference methods we used traditional band power features and the method of common spatial patterns. We consider also for the first time in the context of a four-class problem the issue of variability of the features over time and how much data is required to give good classification results. We do this in a practical way where training data precedes testing data in time.

Online control of a brain-computer interface using phase synchronization.

Currently, almost all brain-computer interfaces (BCIs) ignore the relationship between phases of electroencephalographic signals detected from different recording sites (i.e., electrodes). The vast majority of BCI systems rely on feature vectors derived from e.g., bandpower or univariate adaptive autoregressive (AAR) parameters. However, ample evidence suggests that additional information is obtained by quantifying the relationship between signals of single electrodes, which might provide innovative features for future BCI systems. This paper investigates one method to extract the degree of phase synchronization between two electroencephalogram (EEG) signals by calculating the so-called phase locking value (PLV). In our offline study, several PLV-based features were acquired and the optimal feature set was selected for each subject individually by a feature selection algorithm. The online sessions with three trained subjects revealed that all subjects were able to control three mental states (motor imagery of left hand, right hand, and foot, respectively) with single-trial accuracies between 60% and 66.7% (33% would be expected by chance) throughout the whole session.

Electrotactile EMG feedback improves the control of prosthesis grasping force.

OBJECTIVE: A drawback of active prostheses is that they detach the subject from the produced forces, thereby preventing direct mechanical feedback. This can be compensated by providing somatosensory feedback to the user through mechanical or electrical stimulation, which in turn may improve the utility, sense of embodiment, and thereby increase the acceptance rate. APPROACH: In this study, we compared a novel approach to closing the loop, namely EMG feedback (emgFB), to classic force feedback (forceFB), using electrotactile interface in a realistic task setup. Eleven intact-bodied subjects and one transradial amputee performed a routine grasping task while receiving emgFB or forceFB. The two feedback types were delivered through the same electrotactile interface, using a mixed spatial/frequency coding to transmit 8 discrete levels of the feedback variable. In emgFB, the stimulation transmitted the amplitude of the processed myoelectric signal generated by the subject (prosthesis input), and in forceFB the generated grasping force (prosthesis output). The task comprised 150 trials of routine grasping at six forces, randomly presented in blocks of five trials (same force). Interquartile range and changes in the absolute error (AE) distribution (magnitude and dispersion) with respect to the target level were used to assess precision and overall performance, respectively. MAIN RESULTS: Relative to forceFB, emgFB significantly improved the precision of myoelectric commands (min/max of the significant levels) for 23%/36% as well as the precision of force control for 12%/32%, in intact-bodied subjects. Also, the magnitude and dispersion of the AE distribution were reduced. The results were similar in the amputee, showing considerable improvements. SIGNIFICANCE: Using emgFB, the subjects therefore decreased the uncertainty of the forward pathway. Since there is a correspondence between the EMG and force, where the former anticipates the latter, the emgFB allowed for predictive control, as the subjects used the feedback to adjust the desired force even before the prosthesis contacted the object. In conclusion, the online emgFB was superior to the classic forceFB in realistic conditions that included electrotactile stimulation, limited feedback resolution (8 levels), cognitive processing delay, and time constraints (fast grasping).

Context-Dependent Upper Limb Prosthesis Control for Natural and Robust Use.

Pattern recognition and regression methods applied to the surface EMG have been used for estimating the user intended motor tasks across multiple degrees of freedom (DOF), for prosthetic control. While these methods are effective in several conditions, they are still characterized by some shortcomings. In this study we propose a methodology that combines these two approaches for mutually alleviating their limitations. This resulted in a control method capable of context-dependent movement estimation that switched automatically between sequential (one DOF at a time) or simultaneous (multiple DOF) prosthesis control, based on an online estimation of signal dimensionality. The proposed method was evaluated in scenarios close to real-life situations, with the control of a physical prosthesis in applied tasks of varying difficulties. Test prostheses were individually manufactured for both able-bodied and transradial amputee subjects. With these prostheses, two amputees performed the Southampton Hand Assessment Procedure test with scores of 58 and 71 points. The five able-bodied individuals performed standardized tests, such as the box&block and clothes pin test, reducing the completion times by up to 30%, with respect to using a state-of-the-art pure sequential control algorithm. Apart from facilitating fast simultaneous movements, the proposed control scheme was also more intuitive to use, since human movements are predominated by simultaneous activations across joints. The proposed method thus represents a significant step towards intelligent, intuitive and natural control of upper limb prostheses.

Phase relationships between different subdural electrode recordings in man.

Almost all brain-computer interfaces (BCIs) ignore information related to the phase coupling between electroencephalogram (EEG) or electrocorticogram (ECoG) recordings from different electrodes. This paper investigates whether additional information can be found when calculating the amount of synchronization between two electrode channels by using a phase locking measurement called the phase locking value (PLV). Special emphasis is put on the beta band (around 20 Hz) as well as the gamma band (high frequencies up to 95 Hz), which can only be used when subdural electrode recordings are available.

A Multi-Class Proportional Myocontrol Algorithm for Upper Limb Prosthesis Control: Validation in Real-Life Scenarios on Amputees.

Functional replacement of upper limbs by means of dexterous prosthetic devices remains a technological challenge. While the mechanical design of prosthetic hands has advanced rapidly, the human-machine interfacing and the control strategies needed for the activation of multiple degrees of freedom are not reliable enough for restoring hand function successfully. Machine learning methods capable of inferring the user intent from EMG signals generated by the activation of the remnant muscles are regarded as a promising solution to this problem. However, the lack of robustness of the current methods impedes their routine clinical application. In this study, we propose a novel algorithm for controlling multiple degrees of freedom sequentially, inherently proportionally and with high robustness, allowing a good level of prosthetic hand function. The control algorithm is based on the spatial linear combinations of amplitude-related EMG signal features. The weighting coefficients in this combination are derived from the optimization criterion of the common spatial patterns filters which allow for maximal discriminability between movements. An important component of the study is the validation of the method which was performed on both able-bodied and amputee subjects who used physical prostheses with customized sockets and performed three standardized functional tests mimicking daily-life activities of varying difficulty. Moreover, the new method was compared in the same conditions with one clinical/industrial and one academic state-of-the-art method. The novel algorithm outperformed significantly the state-of-the-art techniques in both subject groups for tests that required the activation of more than one degree of freedom. Because of the evaluation in real time control on both able-bodied subjects and final users (amputees) wearing physical prostheses, the results obtained allow for the direct extrapolation of the benefits of the proposed method for the end users. In conclusion, the method proposed and validated in real-life use scenarios, allows the practical usability of multifunctional hand prostheses in an intuitive way, with significant advantages with respect to previous systems.

Sensor fusion and computer vision for context-aware control of a multi degree-of-freedom prosthesis.

OBJECTIVE: Myoelectric activity volitionally generated by the user is often used for controlling hand prostheses in order to replicate the synergistic actions of muscles in healthy humans during grasping. Muscle synergies in healthy humans are based on the integration of visual perception, heuristics and proprioception. Here, we demonstrate how sensor fusion that combines artificial vision and proprioceptive information with the high-level processing characteristics of biological systems can be effectively used in transradial prosthesis control. APPROACH: We developed a novel context- and user-aware prosthesis (CASP) controller integrating computer vision and inertial sensing with myoelectric activity in order to achieve semi-autonomous and reactive control of a prosthetic hand. The presented method semi-automatically provides simultaneous and proportional control of multiple degrees-of-freedom (DOFs), thus decreasing overall physical effort while retaining full user control. The system was compared against the major commercial state-of-the art myoelectric control system in ten able-bodied and one amputee subject. All subjects used transradial prosthesis with an active wrist to grasp objects typically associated with activities of daily living. MAIN RESULTS: The CASP significantly outperformed the myoelectric interface when controlling all of the prosthesis DOF. However, when tested with less complex prosthetic system (smaller number of DOF), the CASP was slower but resulted with reaching motions that contained less compensatory movements. Another important finding is that the CASP system required minimal user adaptation and training. SIGNIFICANCE: The CASP constitutes a substantial improvement for the control of multi-DOF prostheses. The application of the CASP will have a significant impact when translated to real-life scenarious, particularly with respect to improving the usability and acceptance of highly complex systems (e.g., full prosthetic arms) by amputees.

Phase coupling between different motor areas during tongue-movement imagery.

Motor imagery can be accompanied by an enhancement of brain oscillations (event-related synchronization, ERS) within specific frequency bands. To characterize the neuronal couplings involved during these prominent power changes, we have chosen a certain coupling measure that bears directly on the issue of transient cortical connections. Specifically, we applied for the first time the phase-locking value to investigate the phase coupling of sensorimotor rhythms in different motor areas during tongue-movement imagery. Most interesting, we showed that robust neuronal couplings within the alpha frequency range are established between the midcentral position and bilateral central electrode positions, overlying the supplementary motor area (SMA) and the right and left primary sensorimotor area, respectively. In contrast, no direct linkage was present between sensorimotor rhythms in both hemispheres. We suggest that the coupling results point at a separate interplay between neural networks within the SMA and lateralized networks in primary sensorimotor areas of each hemisphere during motor imagery.

Continuous EEG classification during motor imagery--simulation of an asynchronous BCI.

Nearly all electroencephalogram (EEG)-based brain-computer interface (BCI) systems operate in a cue-paced or synchronous mode. This means that the onset of mental activity (thought) is externally-paced and the EEG has to be analyzed in predefined time windows. In the near future, BCI systems that allow the user to intend a specific mental pattern whenever she/he wishes to produce such patterns will also become important. An asynchronous BCI is characterized by continuous analyzing and classification of EEG data. Therefore, it is important to maximize the hits (true positive rate) during an intended mental task and to minimize the false positive detections in the resting or idling state. EEG data recorded during right/left motor imagery is used to simulate an asynchronous BCI. To optimize the classification results, a refractory period and a dwell time are introduced.

Detection of movement-related desynchronization patterns in ongoing single-channel electrocorticogram.

Adaptive autoregressive parameters and a linear classifier were used to detect movement related desynchronization and synchronization patterns in single-channel electrocorticogram (ECoG) obtained from implanted electrode grids. The best classification accuracies found had more than 90% hits and less than 10% false positives. The findings show that the detection of event-related desynchronization and synchronization in ECoG data can be used to reliably provide switch control directly by the brain and is therefore very suitable as the basis of a direct brain interface.

Toward a direct brain interface based on human subdural recordings and wavelet-packet analysis.

Highly accurate asynchronous detection of movement related patterns in individual electrocorticogram channels has been shown using detection based on either event-related potentials (ERPs) or event-related desynchronization and synchronization (ERD/ERS). A method using wavelet-packet features selected with a genetic algorithm was proposed to simultaneously detect ERP and ERD/ERS and was tested on data from seven subjects and four motor tasks. The proposed wavelet method performed better than previous methods with perfect detection for four subject/task combinations and hit percentages greater than 90% with false positive percentages less than 15% for at least one task for all seven subjects.

An asynchronously controlled EEG-based virtual keyboard: improvement of the spelling rate.

An improvement of the information transfer rate of brain-computer communication is necessary for the creation of more powerful and convenient applications. This paper presents an asynchronously controlled three-class brain-computer interface-based spelling device [virtual keyboard (VK)], operated by spontaneous electroencephalogram and modulated by motor imagery. Of the first results of three able-bodied subjects operating the VK, two were successful, showing an improvement of the spelling rate sigma, the number of correctly spelled letters/min, up to sigma = 3.38 (average sigma = 1.99).