Electroencephalography Reflects User Satisfaction in Controlling Robot Hand through Electromyographic Signals.

This study addresses time intervals during robot control that dominate user satisfaction and factors of robot movement that induce satisfaction. We designed a robot control system using electromyography signals. In each trial, participants were exposed to different experiences as the cutoff frequencies of a low-pass filter were changed. The participants attempted to grab a bottle by controlling a robot. They were asked to evaluate four indicators (stability, imitation, response time, and movement speed) and indicate their satisfaction at the end of each trial by completing a questionnaire. The electroencephalography signals of the participants were recorded while they controlled the robot and responded to the questionnaire. Two independent component clusters in the precuneus and postcentral gyrus were the most sensitive to subjective evaluations. For the moment that dominated satisfaction, we observed that brain activity exhibited significant differences in satisfaction not immediately after feeding an input but during the later stage. The other indicators exhibited independently significant patterns in event-related spectral perturbations. Comparing these indicators in a low-frequency band related to the satisfaction with imitation and movement speed, which had significant differences, revealed that imitation covered significant intervals in satisfaction. This implies that imitation was the most important contributing factor among the four indicators. Our results reveal that regardless of subjective satisfaction, objective performance evaluation might more fully reflect user satisfaction.

Multi-Joint Angles Estimation of Forearm Motion Using a Regression Model.

To improve the life quality of forearm amputees, prosthetic hands with high accuracy, and robustness are necessary. The application of surface electromyography (sEMG) signals to control a prosthetic hand is challenging. In this study, we proposed a time-domain CNN model for the regression prediction of joint angles in three degrees of freedom (3-DOFs, include two wrist joint motion and one finger joint motion), and five-fold cross validation was used to evaluate the correlation coefficient (CC). The CC value results of wrist flexion/extension motion obtained from 10 participants was 0.87-0.92, pronation/supination motion was 0.72-0.95, and hand grip/open motion was 0.75-0.94. We backtracked the fully connected layer weights to create a geometry plot for analyzing the motion pattern to investigate the learning of the proposed model. In order to discuss the daily updateability of the model by transfer learning, we performed a second experiment on five of the participants in another day and conducted transfer learning based on smaller amount of dataset. The CC results improved (wrist flexion/extension was 0.90-0.97, pronation/supination was 0.84-0.96, hand grip/open was 0.85-0.92), suggesting the effectiveness of the transfer learning by incorporating the small amounts of sEMG data acquired in different days. We compared our CNN-based model with four conventional regression models, the result illustrates that proposed model significantly outperforms the four conventional models with and without transfer learning. The offline result suggests the reliability of the proposed model in real-time control in different days, it can be applied for real-time prosthetic control in the future.

Brain Activity Reflects Subjective Response to Delayed Input When Using an Electromyography-Controlled Robot.

In various experimental settings, electromyography (EMG) signals have been used to control robots. EMG-based robot control requires intrinsic parameters for control, which makes it difficult for users to understand the input protocol. When a proper input is not provided, the response time of the system varies; as such, the user's subjective delay should be investigated regardless of the actual delay. In this study, we investigated the influence of the subjective perception of delay on brain activation. Brain recordings were taken while subjects used EMG signals to control a robot hand, which requires a basic processing delay. We used muscle synergy for the grip command of the robot hand. After controlling the robot by grasping their hand, one of four additional delay durations (0 ms, 50 ms, 125 ms, and 250 ms) was applied in every trial, and subjects were instructed to answer whether the delay was natural, additional, or whether they were not sure. We compared brain activity based on responses ("sure" and "not sure"). Our results revealed a significant power difference in the theta band of the parietal lobe, and this time range included the interval in which the subjects could not feel the delay. Our study provides important insights that should be considered when constructing an adaptive system and evaluating its usability.

The Effect of ICA and Non-negative Matrix Factorization Analysis for EMG Signals Recorded From Multi-Channel EMG Sensors.

Surface electromyography (EMG) measurements are affected by various noises such as power source and movement artifacts and adjacent muscle activities. Hardware solutions have been found that use multi-channel EMG signal to attenuate noise signals related to sensor positions. However, studies addressing the overcoming of crosstalk from EMG and the division of overlaid superficial and deep muscles are scarce. In this study, two signal decompositions-independent component analysis and non-negative matrix factorization-were used to create a low-dimensional input signal that divides noise, surface muscles, and deep muscles and utilizes them for movement classification based on direction. In the case of index finger movement, it was confirmed that the proposed decomposition method improved the classification performance with the least input dimensions. These results suggest a new method to analyze more dexterous movements of the hand by separating superficial and deep muscles in the future using multi-channel EMG signals.

Muscle Synergy and Musculoskeletal Model-Based Continuous Multi-Dimensional Estimation of Wrist and Hand Motions.

In this study, seven-channel electromyography signal-based two-dimensional wrist joint movement estimation with and without handgrip motions was carried out. Electromyography signals were analyzed using the synergy-based linear regression model and musculoskeletal model; they were subsequently compared with respect to single and combined wrist joint movements and handgrip. Using each one of wrist motion and grip trial as a training set, the synergy-based linear regression model exhibited a statistically significant performance with 0.7891 ± 0.0844 Pearson correlation coefficient (r) value in two-dimensional wrist motion estimation compared with 0.7608 ± 0.1037 r value of the musculoskeletal model. Estimates on the grip force produced 0.8463 ± 0.0503 r value with 0.2559 ± 0.1397 normalized root-mean-square error of the wrist motion range. This continuous wrist and handgrip estimation can be considered when electromyography-based multi-dimensional input signals in the prosthesis, virtual interface, and rehabilitation are needed.

Finger Angle Estimation From Array EMG System Using Linear Regression Model With Independent Component Analysis.

Surface ElectroMyoGraphy (EMG) signals from the forearm used in prosthetic hand and finger control systems require precise anatomy data of finger muscles that are small and located deep within the forearm. The main problem of this method is that the signal quality depends on the placement of EMG sensor, which can significantly affects the accuracy and precision to estimate joint angles or forces. Moreover, in case of amputees, the location of finger muscles is unknown and needed to be identified manually for EMG recording. As a result, most modern prosthetic hands utilize limited number of muscles with pattern recognition to control finger according to pre-defined grip which is unable to mimic natural finger motion. To address such issue, we used array EMG sensors to obtain EMG signals from all possible positions on the forearm and applied regression method to produce natural finger motion. The signals were analyzed using independent component analysis (ICA) to find the best-fitted independent component (IC) that matches the anatomical data taken after the experiment. Next, from the IC and EMG signals, finger angles were estimated using linear regression model (LRM). Each finger was assigned EMG and IC component for flexion and extension muscles, to assess the possibility of controlling each finger angle separately. We compared the joint angles of each finger between calculated from IC and EMG by correlation coefficients (CC) for all fingers. The average CC values were higher than 0.7, demonstrating the strength of the linear relationship. The different between IC and EMG methods suggests that the IC method can reduce noise and increase the signal to noise ratio. The performance of ICA method showed higher CC value at around 0.2 ± 0.10. In order to confirm the performance of ICA method, we also tested mathematical musculoskeletal model (MSM). The result from this study showed that not only array EMG sensors with ICA significantly improve the quality of signal detected from forearm but also reduce problems of conventional EMG sensors and consequently improve the performance of regression method to imitate natural finger motion.