Graph-Driven Simultaneous and Proportional Estimation of Wrist Angle and Grasp Force via High-Density EMG.

Myoelectric prostheses are generally unable to accurately control the position and force simultaneously, prohibiting natural and intuitive human-machine interaction. This issue is attributed to the limitations of myoelectric interfaces in effectively decoding multi-degree-of-freedom (multi-DoF) kinematic and kinetic information. We thus propose a novel multi-task, spatial-temporal model driven by graphical high-density electromyography (HD-EMG) for simultaneous and proportional control of wrist angle and grasp force. Twelve subjects were recruited to perform three multi-DoF movements, including wrist pronation/supination, wrist flexion/extension, and wrist abduction/adduction while varying grasp force. Experimental results demonstrated that the proposed model outperformed five baseline models, with the normalized root mean square error of 13.2% and 9.7% and the correlation coefficient of 89.6% and 91.9% for wrist angle and grasp force estimation, respectively. In addition, the proposed model still maintained comparable accuracy even with a significant reduction in the number of HD-EMG electrodes. To the best of our knowledge, this is the first study to achieve simultaneous and proportional wrist angle and grasp force control via HD-EMG and has the potential to empower prostheses users to perform a broader range of tasks with greater precision and control, ultimately enhancing their independence and quality of life.

3D Knee and Hip Angle Estimation With Reduced Wearable IMUs via Transfer Learning During Yoga, Golf, Swimming, Badminton, and Dance.

Wearable lower-limb joint angle estimation using a reduced inertial measurement unit (IMU) sensor set could enable quick, economical sports injury risk assessment and motion capture; however the vast majority of existing research requires a full IMU set attached to every related body segment and is implemented in only a single movement, typically walking. We thus implemented 3-dimensional knee and hip angle estimation with a reduced IMU sensor set during yoga, golf, swimming (simulated lower body swimming in a seated posture), badminton, and dance movements. Additionally, current deep-learning models undergo an accuracy drop when tested with new and unseen activities, which necessitates collecting large amounts of data for the new activity. However, collecting large datasets for every new activity is time-consuming and expensive. Thus, a transfer learning (TL) approach with long short-term memory neural networks was proposed to enhance the model's generalization ability towards new activities while minimizing the need for a large new-activity dataset. This approach could transfer the generic knowledge acquired from training the model in the source-activity domain to the target-activity domain. The maximum improvement in estimation accuracy (RMSE) achieved by TL is 23.6 degrees for knee flexion/extension and 22.2 degrees for hip flexion/extension compared to without TL. These results extend the application of motion capture with reduced sensor configurations to a broader range of activities relevant to injury prevention and sports training. Moreover, they enhance the capacity of data-driven models in scenarios where acquiring a substantial amount of training data is challenging.

Synthetic EMG Based on Adversarial Style Transfer Can Effectively Attack Biometric-Based Personal Identification Models.

Biometric-based personal identification models are generally considered to be accurate and secure because biological signals are too complex and person-specific to be fabricated, and EMG signals, in particular, have been used as biological identification tokens due to their high dimension and non-linearity. We investigate the possibility of effectively attacking EMG-based identification models with adversarial biological input via a novel EMG signal individual-style transformer based on a generative adversarial network and tiny leaked data segments. Since two same EMG segments do not exist in nature; the leaked data can't be used to attack the model directly or it will be easily detected. Therefore, it is necessary to extract the style with the leaked personal signals and generate the attack signals with different contents. With our proposed method and tiny leaked personal EMG fragments, numerous EMG signals with different content can be generated in that person's style. EMG hand gesture data from eighteen subjects and three well-recognized deep EMG classifiers were used to demonstrate the effectiveness of the proposed attack methods. The proposed methods achieved an average of 99.41% success rate on confusing identification models and an average of 91.51% success rate on manipulating identification models. These results demonstrate that EMG classifiers based on deep neural networks can be vulnerable to synthetic data attacks. The proof-of-concept results reveal that synthetic EMG biological signals must be considered in biological identification system design across a vast array of relevant biometric systems to ensure personal identification security for individuals and institutions.

A Novel PPG-FMG-ACC Wristband for Hand Gesture Recognition.

Wrist-based hand gesture recognition has the potential to unlock naturalistic human-computer interaction for a vast array of virtual and augmented reality applications. Photoplethysmography (PPG), force myography (FMG), and accelerometry (ACC) have generally been proposed as isolated single sensing modalities for gesture recognition, but any of these alone is inherently limited in the amount of biological information it can collect during finger and hand movements. We thus propose a novel, wrist-based, PPG-FMG-ACC combined sensing approach based on a multi-head attention mechanism fusion convolutional neural network (CNN-AF) for gesture recognition. Nine subjects performed twelve hand gestures involving various wrist and finger postures. Experimental results showed that multi-modal fusion improved classification performance significantly ( p 0.01) compared to any single sensing modality, and the F1-score of the combined PPG-FMG-ACC approach was 40.1% higher than PPG alone, 27.4% higher than ACC alone, and 11.9% higher than FMG alone. To the best of our knowledge, this paper is the first to combine wrist-based PPG, FMG, and ACC signals for hand gesture recognition. These results could serve to inform wrist-based gesture recognition design (e.g., via a smartwatch) and thus expand the capabilities of intuitive and ubiquitous human-machine interaction.

Proposal of a Wearable Multimodal Sensing-Based Serious Games Approach for Hand Movement Training After Stroke.

Stroke often leads to hand motor dysfunction, and effective rehabilitation requires keeping patients engaged and motivated. Among the existing automated rehabilitation approaches, data glove-based systems are not easy to wear for patients due to spasticity, and single sensor-based approaches generally provided prohibitively limited information. We thus propose a wearable multimodal serious games approach for hand movement training after stroke. A force myography (FMG), electromyography (EMG), and inertial measurement unit (IMU)-based multi-sensor fusion model was proposed for hand movement classification, which was worn on the user's affected arm. Two movement recognition-based serious games were developed for hand movement and cognition training. Ten stroke patients with mild to moderate motor impairments (Brunnstrom Stage for Hand II-VI) performed experiments while playing interactive serious games requiring 12 activities-of-daily-living (ADLs) hand movements taken from the Fugl Meyer Assessment. Feasibility was evaluated by movement classification accuracy and qualitative patient questionnaires. The offline classification accuracy using combined FMG-EMG-IMU was 81.0% for the 12 movements, which was significantly higher than any single sensing modality; only EMG, only FMG, and only IMU were 69.6, 63.2, and 47.8%, respectively. Patients reported that they were more enthusiastic about hand movement training while playing the serious games as compared to conventional methods and strongly agreed that they subjectively felt that the proposed training could be beneficial for improving upper limb motor function. These results showed that multimodal-sensor fusion improved hand gesture classification accuracy for stroke patients and demonstrated the potential of this proposed approach to be used as upper limb movement training after stroke.

Emerging Wearable Interfaces and Algorithms for Hand Gesture Recognition: A Survey.

Hands are vital in a wide range of fundamental daily activities, and neurological diseases that impede hand function can significantly affect quality of life. Wearable hand gesture interfaces hold promise to restore and assist hand function and to enhance human-human and human-computer communication. The purpose of this review is to synthesize current novel sensing interfaces and algorithms for hand gesture recognition, and the scope of applications covers rehabilitation, prosthesis control, exoskeletons for augmentation, sign language recognition, human-computer interaction, and user authentication. Results showed that electrical, mechanical, acoustical/vibratory, and optical sensing were the primary input modalities in gesture recognition interfaces. Two categories of algorithms were identified: 1) classification algorithms for predefined, fixed hand poses and 2) regression algorithms for continuous finger and wrist joint angles. Conventional machine learning algorithms, including linear discriminant analysis, support vector machines, random forests, and non-negative matrix factorization, have been widely used for a variety of gesture recognition applications, and deep learning algorithms have more recently been applied to further facilitate the complex relationship between sensor signals and multi-articulated hand postures. Future research should focus on increasing recognition accuracy with larger hand gesture datasets, improving reliability and robustness for daily use outside of the laboratory, and developing softer, less obtrusive interfaces.

Transfer Learning Improves Accelerometer-Based Child Activity Recognition via Subject-Independent Adult-Domain Adaption.

Wearable activity recognition can collate the type, intensity, and duration of each child's physical activity profile, which is important for exploring underlying adolescent health mechanisms. Traditional machine-learning-based approaches require large labeled data sets; however, child activity data sets are typically small and insufficient. Thus, we proposed a transfer learning approach that adapts adult-domain data to train a high-fidelity, subject-independent model for child activity recognition. Twenty children and twenty adults wore an accelerometer wristband while performing walking, running, sitting, and rope skipping activities. Activity classification accuracy was determined via the traditional machine learning approach without transfer learning and with the proposed subject-independent transfer learning approach. Results showed that transfer learning increased classification accuracy to 91.4% as compared to 80.6% without transfer learning. These results suggest that subject-independent transfer learning can improve accuracy and potentially reduce the size of the required child data sets to enable physical activity monitoring systems to be adopted more widely, quickly, and economically for children and provide deeper insights into injury prevention and health promotion strategies.

Wrist-Worn Hand Gesture Recognition While Walking via Transfer Learning.

Walking, one of the most common daily activities, causes unwanted movement artifacts which can significantly deteriorate hand gesture recognition accuracy. However, traditional hand gesture recognition algorithms are typically developed and validated with wrist-worn devices only during static human poses, neglecting the critical importance of dynamic effects on gesture accuracy. Thus, we developed and validated a signal decomposition approach via empirical mode decomposition to accurately segment target gestures from coupled raw signals during dynamic walking and a transfer learning method based on distribution adaptation to enable gesture recognition through domain transfer between dynamic walking and static standing scenarios. Ten healthy subjects performed seven hand gestures during both walking and standing experiments while wearing an IMU wrist-worn device. Experimental results showed that the signal decomposition approach reduced the gesture detection error by 83.8%, and the transfer learning approach (20% transfer rate) improved hand gesture recognition accuracy by 15.1%. This ground-breaking work demonstrates the feasibility of hand gesture recognition while walking via wrist-worn sensing. These findings serve to inform real-life and ubiquitous adoption of wrist-worn hand gesture recognition for intuitive human-machine interaction in dynamic walking situations.

Feasibility Validation on Healthy Adults of a Novel Active Vibrational Sensing Based Ankle Band for Ankle Flexion Angle Estimation.

Goal: In this paper, we introduced a novel ankle band with a vibrational sensor that can achieve low-cost ankle flexion angle estimation, which can be potentially used for automated ankle flexion angle estimation in home-based foot drop rehabilitation scenarios. Methods: Previous ankle flexion angle estimation methods require either professional knowledge or specific equipment and lab environment, which is not feasible for foot drop patients to achieve accurate measurement by themselves in a home-based scenario. To solve the above problems, a prototype was developed based on the assumption that the echo of a vibration signal on the tibialis anterior had different acoustic impedance distribution. By analyzing the frequency spectrum of the echo, the ankle flexion angle can be estimated. Therefore, a surface transducer was utilized to generate frequency-varying active vibration, and a contact microphone was utilized to capture the echo. A portable analog signal processing hub drove the transducer, and was used for echo signal collection from the microphone. Finally, a Random Forest regression model was applied to estimate the ankle flexion angle based on the spectrum amplitude of the echo. Results: Five healthy subjects were recruited in the experiment. The regression estimation error is 4.16 degrees, and the R2 is 0.81. Conclusions: These results demonstrate the feasibility of the proposed ankle band for accurate ankle flexion angle estimation.