Reconstructing Synergy-Based Hand Grasp Kinematics from Electroencephalographic Signals.

Brain-machine interfaces (BMIs) have become increasingly popular in restoring the lost motor function in individuals with disabilities. Several research studies suggest that the CNS may employ synergies or movement primitives to reduce the complexity of control rather than controlling each DoF independently, and the synergies can be used as an optimal control mechanism by the CNS in simplifying and achieving complex movements. Our group has previously demonstrated neural decoding of synergy-based hand movements and used synergies effectively in driving hand exoskeletons. In this study, ten healthy right-handed participants were asked to perform six types of hand grasps representative of the activities of daily living while their neural activities were recorded using electroencephalography (EEG). From half of the participants, hand kinematic synergies were derived, and a neural decoder was developed, based on the correlation between hand synergies and corresponding cortical activity, using multivariate linear regression. Using the synergies and the neural decoder derived from the first half of the participants and only cortical activities from the remaining half of the participants, their hand kinematics were reconstructed with an average accuracy above 70%. Potential applications of synergy-based BMIs for controlling assistive devices in individuals with upper limb motor deficits, implications of the results in individuals with stroke and the limitations of the study were discussed.

Dynamical Synergies of Multidigit Hand Prehension.

Hand prehension requires highly coordinated control of contact forces. The high-dimensional sensorimotor system of the human hand operates at ease, but poses several challenges when replicated in artificial hands. This paper investigates how the dynamical synergies, coordinated spatiotemporal patterns of contact forces, contribute to the hand grasp, and whether they could potentially capture the force primitives in a low-dimensional space. Ten right-handed subjects were recruited to grasp and hold mass-varied objects. The contact forces during this multidigit prehension were recorded using an instrumented grip glove. The dynamical synergies were derived using principal component analysis (PCA). The contact force patterns during the grasps were reconstructed using the first few synergies. The significance of the dynamical synergies, the influence of load forces and task configurations on the synergies were explained. This study also discussed the contribution of biomechanical constraints on the first few synergies and the current challenges and possible applications of the dynamical synergies in the design and control of exoskeletons. The integration of the dynamical synergies into exoskeletons will be realized in the near future.

Data Fusion-Based Musculoskeletal Synergies in the Grasping Hand.

The hypothesis that the central nervous system (CNS) makes use of synergies or movement primitives in achieving simple to complex movements has inspired the investigation of different types of synergies. Kinematic and muscle synergies have been extensively studied in the literature, but only a few studies have compared and combined both types of synergies during the control and coordination of the human hand. In this paper, synergies were extracted first independently (called kinematic and muscle synergies) and then combined through data fusion (called musculoskeletal synergies) from 26 activities of daily living in 22 individuals using principal component analysis (PCA) and independent component analysis (ICA). By a weighted linear combination of musculoskeletal synergies, the recorded kinematics and the recorded muscle activities were reconstructed. The performances of musculoskeletal synergies in reconstructing the movements were compared to the synergies reported previously in the literature by us and others. The results indicate that the musculoskeletal synergies performed better than the synergies extracted without fusion. We attribute this improvement in performance to the musculoskeletal synergies that were generated on the basis of the cross-information between muscle and kinematic activities. Moreover, the synergies extracted using ICA performed better than the synergies extracted using PCA. These musculoskeletal synergies can possibly improve the capabilities of the current methodologies used to control high dimensional prosthetics and exoskeletons.

Biomimetic learning of hand gestures in a humanoid robot.

Hand gestures are a natural and intuitive form of communication, and integrating this communication method into robotic systems presents significant potential to improve human-robot collaboration. Recent advances in motor neuroscience have focused on replicating human hand movements from synergies also known as movement primitives. Synergies, fundamental building blocks of movement, serve as a potential strategy adapted by the central nervous system to generate and control movements. Identifying how synergies contribute to movement can help in dexterous control of robotics, exoskeletons, prosthetics and extend its applications to rehabilitation. In this paper, 33 static hand gestures were recorded through a single RGB camera and identified in real-time through the MediaPipe framework as participants made various postures with their dominant hand. Assuming an open palm as initial posture, uniform joint angular velocities were obtained from all these gestures. By applying a dimensionality reduction method, kinematic synergies were obtained from these joint angular velocities. Kinematic synergies that explain 98% of variance of movements were utilized to reconstruct new hand gestures using convex optimization. Reconstructed hand gestures and selected kinematic synergies were translated onto a humanoid robot, Mitra, in real-time, as the participants demonstrated various hand gestures. The results showed that by using only few kinematic synergies it is possible to generate various hand gestures, with 95.7% accuracy. Furthermore, utilizing low-dimensional synergies in control of high dimensional end effectors holds promise to enable near-natural human-robot collaboration.

Dynamical Synergies in Multidigit Hand Prehension.

Hand prehension requires a highly coordinated control of contact forces. The high dimensional sensorimotor system of the human hand although operates at ease, poses several challenges when replicated for prosthetic control. This study investigates how the dynamical synergies, coordinated spatial patterns of contact forces, contribute to the contact forces in a grasp, and whether the dynamical synergies could potentially serve as candidates for feedforward and feedback mechanisms. Ten right-handed subjects were recruited to grasp and hold mass-varied objects. The contact forces during this multidigit prehension were recorded using an instrumented grip glove. The dynamical synergies were derived using principal component analysis (PCA). The contact force patterns during the grasps were reconstructed using the first few synergies. The significance of the dynamical synergies and the current challenges and possible applications of the dynamical synergies were discussed along with the integration of the dynamical synergies into prosthetics and exoskeletons that can possibly enable near-natural control. This research presents dynamical synergies observed in contact forces during hand grasps. These dynamical synergies could help in improving feedforward force control and sensory feedback in hand prosthetics and exoskeletons.

Musculoskeletal Synergies in the Grasping Hand.

Investigations on how the central nervous system (CNS) effortlessly conducts complex hand movements have led to an extensive study of synergies or movement primitives. Of the different types of hand synergies, kinematic and muscle synergies have been widely studied in literature, but only a few studies have fused both. In this paper kinematic and muscle activities recorded from the activities of daily living were first fused and then dimensionally reduced through principal component analysis (PCA). By using these principal components or musculoskeletal synergies in a weighted linear combination, the recorded kinematics and muscle activities were reconstructed. The performance of these musculoskeletal synergies in reconstructing the movements was compared to the kinematic and muscle synergies reported previously in the literature by us and others. The results from these findings indicate that musculoskeletal synergies perform better than the synergies extracted without fusion. These newly demonstrated musculoskeletal synergies might improve neural control of robotics, prosthetics and exoskeletons. Clinical Relevance- In this paper, musculoskeletal synergies were extracted from the fusion of kinematic and muscle activities recorded from the activities of daily living. These newly demonstrated musculoskeletal synergies might enhance our understanding of neural control of robotics, prosthetics and exoskeletons.

Generalizability of Hand Kinematic Synergies derived using Independent Component Analysis.

In this paper, hand synergies were derived using independent component analysis (ICA) and compared against synergies derived from our previous methods using principal component analysis (PCA). For ICA, we used two algorithms - Infomax and entropy bound minimization (EBM). For all the methods, the synergies were extracted from rapid hand grasps. The extracted synergies were then tested for generalizability in reconstructing natural hand grasps and American Sign Language (ASL) postures that were different from rapid grasps. The results indicate that the synergies derived from ICA were able to generalize only marginally better when compared to those from PCA. Among the two ICA methods, Infomax performed slightly better in yielding lower reconstruction error while EBM performed better in sparse selection of synergies. The implications and future scope were discussed.

Design and Implementation of an Instrumented Data Glove that measures Kinematics and Dynamics of Human Hand.

Human hands are versatile biomechanical architectures that can perform simple movements such as grasping to complicated movements such as playing a musical instrument. Such extremely dependable and useful parts of the human body can be debilitated due to movement disorders such as Parkinson's disease, stroke, spinal cord injury, multiple sclerosis and cerebral palsy. In such cases, precisely measuring the residual or abnormal hand function becomes a critical assessment to help clinicians and physical therapists in diagnosis, treatment and in prescribing appropriate prosthetics or rehabilitation therapies. The current methodologies used to measure abnormal or residual hand function are either paperbased scales that are prone to human error or expensive motion tracking systems. The cost and complexity restrict the usability of these methods in clinical environments. In this paper we present a low-cost instrumented glove that can measure kinematics and dynamics of human hand, by leveraging the recent advances in 3D printing technologies and flexible sensors.

Lateralization and Model Transference in a Bilateral Cursor Task.

Post-stroke rehabilitation, occupational and physical therapy, and training for use of assistive prosthetics leverages our current understanding of bilateral motor control to better train individuals. In this study, we examine upper limb lateralization and model transference using a bimanual joystick cursor task with orthogonal controls. Two groups of healthy subjects are recruited into a 2-session study spaced seven days apart. One group uses their left and right hands to control cursor position and rotation respectively, while the other uses their right and left hands. The groups switch control methods in the second session, and a rotational perturbation is applied to the positional controls in the latter half of each session. We find agreement with current lateralization theories when comparing robustness to feedforward perturbations in feedback and feedforward measures. We find no evidence of a transferable model after seven days, and evidence that the brain does not synchronize task completion between the hands.

Myoelectric Control of a Soft Hand Exoskeleton Using Kinematic Synergies.

Soft hand exoskeletons offer a lightweight, low-profile alternative to rigid rehabilitative robotic systems, enabling their use to restore activities of daily living (ADL) in those with hand paresis due to stroke or other conditions. The hand exoskeleton with embedded synergies (HEXOES) is a soft cable-driven hand exoskeleton capable of independently actuating and sensing 10 degrees of freedom (DoF) of the hand. Control of the 10 DoF exoskeleton is dimensionally reduced using three manually defined synergies in software corresponding to thumb, index, and 3-finger flexion and extension. In this paper, five healthy subjects control HEXOES using a neural network which decodes synergy weights from contralateral electromyography (EMG) activity. The three synergies are manipulated in real time to grasp and lift 15 ADL objects of various sizes and weights. The neural network's training and validation mean squared error, object grasp time, and grasp success rate were measured for five healthy subjects. The final training error of the neural network was 4.8 ± 1.8% averaged across subjects and tasks, with 8.3 ± 3.4% validation error. The time to reach, grasp, and lift an object was 11.15 ± 4.35 s on average, with an average success rate of 66.7% across all objects. The complete system demonstrates real time use of biosignals and machine learning to allow subjects to operate kinematic synergies to grasp objects using a wearable hand exoskeleton. Future work and applications are further discussed, including possible design improvements and enrollment of individuals with stroke.

Decoding Synergy-Based Hand Movements using Electroencephalography.

In this paper, scalp electroencephalographic (EEG) signals were recorded from 10 subjects during hand grasping. Six objects that span different grasp types were used. Grasp kinematics were recorded using CyberGlove. From a training subset of the data, kinematic synergies were determined and their reconstruction weights in these grasps were calculated. EEG features (power spectral densities in four low and high frequency bands) were trained on kinematic synergy weights using multivariate linear regression. Using this model, kinematics from testing subset of data were decoded from EEG with 3-fold cross validation. Results are compared to chance level to determine if reconstruction weights are related to EEG features. Results indicate that EEG features can decode synergy-based movement generation. Study implications and future implementations were discussed.