Neural evidence for functional roles of tactile and visual feedback in the application of myoelectric prosthesis.

Objective. The primary purpose of this study was to investigate the electrophysiological mechanism underlying different modalities of sensory feedback and multi-sensory integration in typical prosthesis control tasks.Approach. We recruited 15 subjects and developed a closed-loop setup for three prosthesis control tasks which covered typical activities in the practical prosthesis application, i.e. prosthesis finger position control (PFPC), equivalent grasping force control (GFC) and box and block control (BABC). All the three tasks were conducted under tactile feedback (TF), visual feedback (VF) and tactile-visual feedback (TVF), respectively, with a simultaneous electroencephalography (EEG) recording to assess the electroencephalogram (EEG) response underlying different types of feedback. Behavioral and psychophysical assessments were also administered in each feedback condition.Results. EEG results showed that VF played a predominant role in GFC and BABC tasks. It was reflected by a significantly lower somatosensory alpha event-related desynchronization (ERD) in TVF than in TF and no significant difference in visual alpha ERD between TVF and VF. In PFPC task, there was no significant difference in somatosensory alpha ERD between TF and TVF, while a significantly lower visual alpha ERD was found in TVF than in VF, indicating that TF was essential in situations related to proprioceptive position perception. Tactile-visual integration was found when TF and VF were congruently implemented, showing an obvious activation over the premotor cortex in the three tasks. Behavioral and psychophysical results were consistent with EEG evaluations.Significance. Our findings could provide neural evidence for multi-sensory integration and functional roles of tactile and VF in a practical setting of prosthesis control, shedding a multi-dimensional insight into the functional mechanisms of sensory feedback.

A soft neuroprosthetic hand providing simultaneous myoelectric control and tactile feedback.

Neuroprosthetic hands are typically heavy (over 400 g) and expensive (more than US$10,000), and lack the compliance and tactile feedback of human hands. Here, we report the design, fabrication and performance of a soft, low-cost and lightweight (292 g) neuroprosthetic hand that provides simultaneous myoelectric control and tactile feedback. The neuroprosthesis has six active degrees of freedom under pneumatic actuation, can be controlled through the input from four electromyography sensors that measure surface signals from residual forearm muscles, and integrates five elastomeric capacitive sensors on the fingertips to measure touch pressure so as to enable tactile feedback by eliciting electrical stimulation on the skin of the residual limb. In a set of standardized tests performed by two individuals with transradial amputations, we show that the soft neuroprosthetic hand outperforms a conventional rigid neuroprosthetic hand in speed and dexterity. We also show that one individual with a transradial amputation wearing the soft neuroprosthetic hand can regain primitive touch sensation and real-time closed-loop control.

Electrotactile Feedback Improves Grip Force Control and Enables Object Stiffness Recognition While Using a Myoelectric Hand.

Current myoelectric hands are limited in their ability to provide effective sensory feedback to the users, which highly affects their functionality and utility. Although the sensory information of a myoelectric hand can be acquired with equipped sensors, transforming the sensory signals into effective tactile sensations on users for functional tasks is a largely unsolved challenge. The purpose of this study aims to demonstrate that electrotactile feedback of the grip force improves the sensorimotor control of a myoelectric hand and enables object stiffness recognition. The grip force of a sensorized myoelectric hand was delivered to its users via electrotactile stimulation based on four kinds of typical encoding strategies, including graded (G), linear amplitude (LA), linear frequency (LF), and biomimetic (B) modulation. Object stiffness was encoded with the change of electrotactile sensations triggered by final grip force, as the prosthesis grasped the objects. Ten able-bodied subjects and two transradial amputees were recruited to participate in a dual-task virtual eggs test (VET) and an object stiffness discrimination test (OSDT) to quantify the prosthesis users' ability to handle fragile objects and recognize object stiffnesses, respectively. The quantified results showed that with electrotactile feedback enabled, the four kinds of encoding strategies allowed subjects to better able to handle fragile objects with similar performance, and the subjects were able to differentiate four levels of object stiffness at favorable accuracies (>86%) and high manual efficiency. Strategy LA presented the best stiffness discrimination performance, while strategy B was able to reduce the discrimination time but the discrimination accuracy was not better than the other three strategies. Electrotactile feedback also enhanced prosthesis embodiment and improved the users' confidence in prosthetic control. Outcomes indicate electrotactile feedback can be effectively exploited by the prosthesis users for grip force control and object stiffness recognition, proving the feasibility of functional sensory restoration of myoelectric prostheses equipped with electrotactile feedback.

Assessing differential representation of hand movements in multiple domains using stereo-electroencephalographic recordings.

Invasive brain-computer interfaces (BCI) have made great progress in the reconstruction of fine hand movement parameters for paralyzed patients, where superficial measurement modalities including electrocorticography (ECoG) and micro-array recordings are mostly used. However, these recording techniques typically focus on the signals from the sensorimotor cortex, leaving subcortical regions and other cortical regions related to the movements largely unexplored. As an intracranial recording technique for the presurgical assessments of brain surgery, stereo-encephalography (SEEG) inserts depth electrodes containing multiple contacts into the brain and thus provides the unique opportunity for investigating movement-related neural representation throughout the brain. Although SEEG samples neural signals with high spatial-temporal resolutions, its potential of being used to build BCIs has just been realized recently, and the decoding of SEEG activity related to hand movements has not been comprehensively investigated yet. Here, we systematically evaluated the factors influencing the performance of movement decoding using SEEG signals recorded from 32 human subjects performing a visually-cued hand movement task. Our results suggest that multiple regions in both lateral and depth directions present significant neural selectivity to the task, whereas the sensorimotor area, including both precentral and postcentral cortex, carries the richest discriminative neural information for the decoding. The posterior parietal and prefrontal cortex contribute gradually less, but still rich sources for extracting movement parameters. The insula, temporal and occipital cortex also contains useful task-related information for decoding. Under the cortex layer, white matter presents decodable neural patterns but yields a lower accuracy (42.0 ± 0.8%) than the cortex on average (44.2 ± 0.8%, p<0.01). Notably, collectively using neural signals from multiple task-related areas can significantly enhance the movement decoding performance by 6.9% (p<0.01) on average compared to using a single region. Among the different spectral components of SEEG activity, the high gamma and delta bands offer the most informative features for hand movements reconstruction. Additionally, the phase-amplitude coupling strength between these two frequency ranges correlates positively with the performance of movement decoding. In the temporal domain, maximum decoding accuracy is first reached around 2 s after the onset of movement commands. In sum, this study provides valuable insights for the future motor BCIs design employing both SEEG recordings and other recording modalities.

Towards optimizing the non-invasive sensory feedback interfaces in a neural prosthetic control.

Objective. The somatotopic interface (SI) and non-somatotopic interface (NI) are commonly used to provide non-invasive sensory feedback. Nevertheless, differences between somatotopic and non-somatotopic feedbacks are rarely reported, and objective evaluations of the corresponding brain response are missing as well. Few studies have reported how to design the stimulation encoding strategy based on the two interfaces in sensory feedback. The objective of this study was to investigate the difference in sensory characteristics between SI and NI, and propose an optimal encoding method for non-invasive feedback interfaces.Approach. We recruited seven amputees and compared the tactile sensitivity to stimulated positions and intensities between SI (phantom finger area) and NI (upper arm) in a tactile discrimination task. Electroencephalography (EEG) evaluation task was subsequently conducted to objectively evaluate the stimulus-evoked brain response for SI and NI. Finally, the two kinds of tactile information (stimulated position and intensity) was applied to an object recognition task in a closed-loop prosthesis control system. Specifically, the object size was reflected by the prosthetic finger position through stimulated position encoding, and the object stiffness was reflected by the contact force of prosthetic fingers through stimulated intensity encoding. We compared the performance under four feedback conditions (combinations between two kinds of tactile information and two interfaces).Main results. Behavioral results showed that NI was more sensitive to position information while SI was more sensitive to intensity information. EEG results were consistent with behavioral results, showing a higher sensitivity of sensory alpha ERD and a stronger correlation between alpha ERD and discrimination accuracies for NI in the position discrimination, while the trend was opposite in the intensity discrimination. The feedback encoding allowed amputees to discriminate the size and stiffness of nine objects with the best performance of 62% overall accuracy (84% for size discrimination, 71% for stiffness discrimination) when position and intensity information was delivered on the NI and SI, respectively.Significance. Our results provided an instructive strategy for sensory feedback via non-invasive solutions.

Detection of human white matter activation and evaluation of its function in movement decoding using stereo-electroencephalography (SEEG).

Objective. White matter tissue takes up approximately 50% of the human brain volume and it is widely known as a messenger conducting information between areas of the central nervous system. However, the characteristics of white matter neural activity and whether white matter neural recordings can contribute to movement decoding are often ignored and still remain largely unknown. In this work, we make quantitative analyses to investigate these two important questions using invasive neural recordings.Approach. We recorded stereo-electroencephalography (SEEG) data from 32 human subjects during a visually-cued motor task, where SEEG recordings can tap into gray and white matter electrical activity simultaneously. Using the proximal tissue density method, we identified the location (i.e. gray or white matter) of each SEEG contact. Focusing on alpha oscillatory and high gamma activities, we compared the activation patterns between gray matter and white matter. Then, we evaluated the performance of such white matter activation in movement decoding.Main results. The results show that white matter also presents activation under the task, in a similar way with the gray matter but at a significantly lower amplitude. Additionally, this work also demonstrates that combing white matter neural activities together with that of gray matter significantly promotes the movement decoding accuracy than using gray matter signals only.Significance. Taking advantage of SEEG recordings from a large number of subjects, we reveal the response characteristics of white matter neural signals under the task and demonstrate its enhancing function in movement decoding. This study highlights the importance of taking white matter activities into consideration in further scientific research and translational applications.

Self-Related Stimuli Decoding With Auditory and Visual Modalities Using Stereo-Electroencephalography.

Name recognition plays important role in self-related cognitive processes and also contributes to a variety of clinical applications, such as autism spectrum disorder diagnosis and consciousness disorder analysis. However, most previous name-related studies usually adopted noninvasive EEG or fMRI recordings, which were limited by low spatial resolution and temporal resolution, respectively, and thus millisecond-level response latencies in precise brain regions could not be measured using these noninvasive recordings. By invasive stereo-electroencephalography (SEEG) recordings that have high resolution in both the spatial and temporal domain, the current study distinguished the neural response to one's own name or a stranger's name, and explored common active brain regions in both auditory and visual modalities. The neural activities were classified using spatiotemporal features of high-gamma, beta, and alpha band. Results showed that different names could be decoded using multi-region SEEG signals, and the best classification performance was achieved at high gamma (60-145 Hz) band. In this case, auditory and visual modality-based name classification accuracies were 84.5 ± 8.3 and 79.9 ± 4.6%, respectively. Additionally, some single regions such as the supramarginal gyrus, middle temporal gyrus, and insula could also achieve remarkable accuracies for both modalities, supporting their roles in the processing of self-related information. The average latency of the difference between the two responses in these precise regions was 354 ± 63 and 285 ± 59 ms in the auditory and visual modality, respectively. This study suggested that name recognition was attributed to a distributed brain network, and the subsets with decoding capabilities might be potential implanted regions for awareness detection and cognition evaluation.

Evaluating User and Machine Learning in Short- and Long-Term Pattern Recognition-Based Myoelectric Control.

Proper training is essential to achieve reliable pattern recognition (PR) based myoelectric control. The amount of training is commonly determined by experience. The purpose of this study is to provide an offline validation method that makes the offline performance transferable to online control and find the proper amount of training that achieves good online performance. In the offline experiment, eight able-bodied subjects and three amputees participated in a ten-day training. Repeatability index (RI) and classification error (CE) were used to evaluate user learning and machine learning, respectively. The performance of cross-validation (CV) and time serial related validation (TSV) was compared. Learning curves were established with different training trials by TSV. In the online experiment, sixteen able-bodied subjects were randomly divided into two groups with one- or five-trial training, respectively, followed by participating in the test with and without classifier-output feedback. The correlation between offline and online tests was analyzed. Results indicated that five-trial training was proper to train the user and the classifier. The long-term retention of skills could not shorten the learning process. The correlation between CEs of TSV and the online test was strong ( r=0.87 ) with five-trial training, while the correlation between CEs of CV and the online test was weak ( r=0.30 ). Outcomes demonstrate that offline performance evaluated by TSV is transferable to online performance and the learning process can guide the user to achieve good online myoelectric control with minimum training.

Contra-lateral desynchronized alpha oscillations linearly correlate with discrimination performance of tactile acuity.

OBJECTIVE: We used EEG to investigate cortical oscillatory activities during the tactile discrimination task and characterize the correlation between the EEG features and subjects' discrimination performance. APPROACH: Transcutaneous electrical nerve stimulation (TENS) was applied on two finger areas (thumb and index for healthy hands, thumb and index-projected areas for disabled hands) to evoke two kinds of tactile sensations (vibration and pressure) with three levels of intensities (low, medium and high). Four forearm amputees and thirteen able-bodied subjects were recruited to discriminate the specific intensity and area of the applied stimulation. We assessed the discrimination performance [discrimination accuracy rate (AR) and response time (RT)] to quantify the tactile acuity. During the stimulation, EEG signals were recorded and the evoked cortical oscillatory activities were analyzed. Linear regression analyses were performed between EEG features and tactile discrimination performance. MAIN RESULTS: Spectral analysis revealed that alpha ERD over somatosensory regions persisted the whole task period and was related to the sensory information processing. Alpha ERD over prefrontal regions was only found during the stimulation judgement period and might reflect advanced cognitive process. There was no linear correlation between prefrontal alpha ERD and tactile discrimination performance. While contralateral somatosensory alpha ERDs exhibited significantly negative correlations with ARs ([Formula: see text]) and positive correlations with RTs ([Formula: see text]). Specifically, the fitting results of higher alpha band (10-13 Hz) were superior to lower alpha band (8-10 Hz). SIGNIFICANCE: Alpha ERD over contralateral somatosensory cortex could be used as an objective index for the evaluation of tactile acuity and might have the potential to be applied in sensory rehabilitation for amputees.

Electrical stimulation-induced SSSEP as an objective index to evaluate the difference of tactile acuity between the left and right hand.

OBJECTIVE: The objective of this study is to propose an objective index to evaluate the difference of tactile acuity between the left and right hand based on steady-state somatosensory evoked potential (SSSEP). APPROACH: Two kinds of tactile sensations (vibration and pressure) with three levels of intensities (low/medium/high) were evoked on two finger areas of the left or right hand (thumb and index for healthy hands, thumb and index-projected areas for disabled hands) via transcutaneous electrical nerve stimulation (TENS). Three forearm amputees and 13 able-bodied subjects were recruited to discriminate the specific level and area of the applied stimulation. Electroencephalography was adopted to simultaneously record the somatosensory cortex response to TENS. We assessed the discrimination performance (discrimination accuracy rate (AR) and response time (RT)) to quantify the tactile acuity, while the evoked SSSEP was synchronously analyzed. Linear regression analyses were performed between the difference of SSSEP amplitudes and the difference of discrimination performance for the left and right hand stimulation. MAIN RESULTS: Frequency domain analysis revealed that SSSEP amplitude increased with the increase of the stimulation intensity. There were positive correlations between the difference of SSSEP amplitudes and the difference of ARs for the left and right hand stimulation in the sensations of vibration (R 2 = 0.6389 for able-bodied subjects, R 2 = 0.5328 for amputees) and pressure (R 2 = 0.6102 for able-bodied subjects, R 2 = 0.5452 for amputees), respectively. Significance The SSSEP amplitude could be used as an objective index to evaluate the difference of the tactile acuity between the left and right hand and has the potential to be applied in sensory rehabilitation for amputees or stroke patients.

Prediction of finger kinematics from discharge timings of motor units: implications for intuitive control of myoelectric prostheses.

OBJECTIVE: The aim of the study was to characterize the accuracy in the identification of motor unit discharges during natural movements using high-density electromyography (EMG) signals and to investigate their correlation with finger kinematics. APPROACH: High-density EMG signals of forearm muscles and finger joint angles were recorded concurrently during hand movements of ten able-bodied subjects. EMG signals were decomposed into motor unit spike trains (MUSTs) with a blind-source separation method. The first principle component (FPC) of the low-pass filtered MUST was correlated with finger joint angles. MAIN RESULTS: On average, [Formula: see text] motor units were identified during each individual finger task with an estimated decomposition accuracy [Formula: see text]85%. The FPC extracted from discharge rates was strongly associated to the joint angles ([Formula: see text]), and preceded the joint angles on average by [Formula: see text] ms. Moreover, the FPC outperformed two time-domain features (the EMG envelop and the root mean square of EMG) in estimating joint angles. SIGNIFICANCE: These results indicated the possibility of identifying individual motor unit behavior in dynamic natural contractions. Moreover, the strong association between motor unit discharge behaviors and kinematics proves the potential of the approach for the simultaneous and proportional control of prostheses.

Electrotactile Feedback with Spatial and Mixed Coding for Object Identification and Closed-loop Control of Grasping Force in Myoelectric Prostheses.

Providing high-quality somatosensory feedback from myoelectric prostheses to an upper-limb amputee user is a long-standing challenge. Various approaches have been investigated for tactile feedback, ranging from direct neural stimulation to noninvasive sensory substitution methods. However, only a few of studies evaluated the closed-loop performance, and real-time movement information of active prostheses still could not be transferred in the form of proprioceptive feedback so far. In current study, an integrated closed-loop prosthesis system consisted of two types of sensors, programmable electrical stimulator and multichannel array electrodes was presented. The grasping angle and corresponding grasping force of the single-freedom myoelectric prosthesis were simultaneously coded with spatial and mixed (spatial and intensity of sensation) coding scheme and tested in 15 able-bodied subjects. The experimental results demonstrated that the subjects were able to discriminate 4 types of object sizes, 3 kinds of different softness and 4 levels of grasping forces in relatively high correct identification rates (CIRs) (size: 87.5%, Softness: 94%, grasping force: 73.8%). The study outcomes and specific conclusions provide valuable guidance for the design of closed-loop myoelectric prostheses equipped with electrotactile feedback.

Neural Modulation By Repetitive Transcranial Magnetic Stimulation (rTMS) for BCI Enhancement in Stroke Patients.

Brain-computer interface (BCI) is a novel method for stroke rehabilitation. However, lacking of sufficient motor-related cortical activity greatly decreases the BCI performance in stroke patients. Interestingly, high-frequency repetitive transcranial magnetic stimulation (rTMS) has been shown to increase the cortical excitability of lesioned hemisphere in stroke patients. This stimulation effect may have benefits on the enhancement of BCI decoding. This study recruited 16 stroke patients to evaluate the stimulation effect on BCI accuracy, with 8 patients were assigned to the TMS-group and the other 8 patients were assigned to the Control-group. Patients in the TMS-group underwent 12 sessions of 10-Hz TMS interventions in four consecutive weeks, whereas no stimulation was applied during this period in the Control-group. Meanwhile, three BCI evaluation sessions were carried out in one day before, one day after, and three days after the TMS intervention, separately. The results showed that the TMS intervention significantly improved the BCI accuracy from 63.5% to 74.3% in motor imagery (MI) tasks, and from 81.9% to 91.1% in motor execution (ME) tasks. This finding provides a novel method for the cure of BCI-inefficiency problem, and may facilitate the clinical application of BCI-based stroke rehabilitation.

Developing Non-Somatotopic Phantom Finger Sensation to Comparable Levels of Somatotopic Sensation through User Training With Electrotactile Stimulation.

Cutaneous electrical stimulation can provide tactile feedback for upper-limb amputees through somatotopic feedback (SF) or non-somatotopic feedback (NF). The SF delivers electrotactile stimulus to projection finger maps (PFMs) on the stumps of amputees, which outperforms NF that transfers stimulus to other human intact skin areas in general. However, the SF areas on stumps are very limited and often occupied by electromyography (EMG) sensors in application of myoelectric prosthesis. This work aims at improving NF performance on human upper arms through user training with electrotactile stimulation. The experiments were conducted over seven consecutive days on nine able-bodied subjects and two forearm amputees. The performance measures of NF/SF included the correct identification rates (CIRs), the response time and the NASA-TLX questionnaire. The between-day CIR s on NF sites increased logarithmically with a mean course of 3-day rapid-improving phase and plateaued in the relative-steady phase. The response time and NASA-TLX scores could also rapidly reduce to the comparable levels of the SF areas during the same mean period of 3-day rapid-improving phase, respectively. These results indicated that the performance of NF could be highly improved to the equivalent level as that of SF through 3-day electrotactile training, which we named as "3-day effect". It provides important insights that intact skin areas without phantom sensations can effectively replace SF sites to transfer tactile feedback after continuous user training, which validates effectiveness of non-invasive interfaces of tactile feedback for upper-limb amputees in practice.

Characterization of evoked tactile sensation in forearm amputees with transcutaneous electrical nerve stimulation.

OBJECTIVE: The goal of this study is to characterize the phenomenon of evoked tactile sensation (ETS) on the stump skin of forearm amputees using transcutaneous electrical nerve stimulation (TENS). APPROACH: We identified the projected finger map (PFM) of ETS on the stump skin in 11 forearm amputees, and compared perceptual attributes of the ETS in nine forearm amputees and eight able-bodied subjects using TENS. The profile of perceptual thresholds at the most sensitive points (MSPs) in each finger-projected area was obtained by modulating current amplitude, pulse width, and frequency of the biphasic, rectangular current stimulus. The long-term stability of the PFM and the perceptual threshold of the ETS were monitored in five forearm amputees for a period of 11 months. MAIN RESULTS: Five finger-specific projection areas can be independently identified on the stump skin of forearm amputees with a relatively long residual stump length. The shape of the PFM was progressively similar to that of the hand with more distal amputation. Similar sensory modalities of touch, pressure, buzz, vibration, and numb below pain sensation could be evoked both in the PFM of the stump skin of amputees and in the normal skin of able-bodied subjects. Sensory thresholds in the normal skin of able-bodied subjects were generally lower than those in the stump skin of forearm amputees, however, both were linearly modulated by current amplitude and pulse width. The variation of the MSPs in the PFM was confined to a small elliptical area with 95% confidence. The perceptual thresholds of thumb-projected areas were found to vary less than 0.99 × 10(-2) mA cm(-2). SIGNIFICANCE: The stable PFM and sensory thresholds of ETS are desirable for a non-invasive neural interface that can feed back finger-specific tactile information from the prosthetic hand to forearm amputees.