Limb loading enhances skill transfer between augmented and physical reality tasks during limb loss rehabilitation.

BACKGROUND: Virtual and augmented reality (AR) have become popular modalities for training myoelectric prosthesis control with upper-limb amputees. While some systems have shown moderate success, it is unclear how well the complex motor skills learned in an AR simulation transfer to completing the same tasks in physical reality. Limb loading is a possible dimension of motor skill execution that is absent in current AR solutions that may help to increase skill transfer between the virtual and physical domains. METHODS: We implemented an immersive AR environment where individuals could operate a myoelectric virtual prosthesis to accomplish a variety of object relocation manipulations. Intact limb participants were separated into three groups, the load control (CGLD; [Formula: see text]), the AR control (CGAR; [Formula: see text]), and the experimental group (EG; [Formula: see text]). Both the CGAR and EG completed a 5-session prosthesis training protocol in AR while the CGLD performed simple muscle training. The EG attempted manipulations in AR while undergoing limb loading. The CGAR attempted the same manipulations without loading. All participants performed the same manipulations in physical reality while operating a real prosthesis pre- and post-training. The main outcome measure was the change in the number of manipulations completed during the physical reality assessments (i.e. completion rate). Secondary outcomes included movement kinematics and visuomotor behavior. RESULTS: The EG experienced a greater increase in completion rate post-training than both the CGAR and CGLD. This performance increase was accompanied by a shorter motor learning phase, the EG's performance saturating in less sessions of AR training than the CGAR. CONCLUSION: The results demonstrated that limb loading plays an important role in transferring complex motor skills learned in virtual spaces to their physical reality analogs. While participants who did not receive limb loading were able to receive some functional benefit from AR training, participants who received the loading experienced a greater positive change in motor performance with their performance saturating in fewer training sessions.

Sensory stimulation enhances phantom limb perception and movement decoding.

OBJECTIVE: A major challenge for controlling a prosthetic arm is communication between the device and the user's phantom limb. We show the ability to enhance phantom limb perception and improve movement decoding through targeted transcutaneous electrical nerve stimulation in individuals with an arm amputation. APPROACH: Transcutaneous nerve stimulation experiments were performed with four participants with arm amputation to map phantom limb perception. We measured myoelectric signals during phantom hand movements before and after participants received sensory stimulation. Using electroencephalogram (EEG) monitoring, we measured the neural activity in sensorimotor regions during phantom movements and stimulation. In one participant, we also tracked sensory mapping over 2 years and movement decoding performance over 1 year. MAIN RESULTS: Results show improvements in the participants' ability to perceive and move the phantom hand as a result of sensory stimulation, which leads to improved movement decoding. In the extended study with one participant, we found that sensory mapping remains stable over 2 years. Sensory stimulation improves within-day movement decoding while performance remains stable over 1 year. From the EEG, we observed cortical correlates of sensorimotor integration and increased motor-related neural activity as a result of enhanced phantom limb perception. SIGNIFICANCE: This work demonstrates that phantom limb perception influences prosthesis control and can benefit from targeted nerve stimulation. These findings have implications for improving prosthesis usability and function due to a heightened sense of the phantom hand.

Predictive trajectory estimation during rehabilitative tasks in augmented reality using inertial sensors.

This paper presents a wireless kinematic tracking framework used for biomechanical analysis during rehabilitative tasks in augmented and virtual reality. The framework uses low-cost inertial measurement units and exploits the rigid connections of the human skeletal system to provide egocentric position estimates of joints to centimeter accuracy. On-board sensor fusion combines information from three-axis accelerometers, gyroscopes, and magnetometers to provide robust estimates in real-time. Sensor precision and accuracy were validated using the root mean square error of estimated joint angles against ground truth goniometer measurements. The sensor network produced a mean estimate accuracy of 2.81° with 1.06° precision, resulting in a maximum hand tracking error of 7.06 cm. As an application, the network is used to collect kinematic information from an unconstrained object manipulation task in augmented reality, from which dynamic movement primitives are extracted to characterize natural task completion in N = 3 able-bodied human subjects. These primitives are then leveraged for trajectory estimation in both a generalized and a subject-specific scheme resulting in 0.187 cm and 0.161 cm regression accuracy, respectively. Our proposed kinematic tracking network is wireless, accurate, and especially useful for predicting voluntary actuation in virtual and augmented reality applications.

Prosthesis with neuromorphic multilayered e-dermis perceives touch and pain.

The human body is a template for many state-of-the-art prosthetic devices and sensors. Perceptions of touch and pain are fundamental components of our daily lives that convey valuable information about our environment while also providing an element of protection from damage to our bodies. Advances in prosthesis designs and control mechanisms can aid an amputee's ability to regain lost function but often lack meaningful tactile feedback or perception. Through transcutaneous electrical nerve stimulation (TENS) with an amputee, we discovered and quantified stimulation parameters to elicit innocuous (non-painful) and noxious (painful) tactile perceptions in the phantom hand. Electroencephalography (EEG) activity in somatosensory regions confirms phantom hand activation during stimulation. We invented a multilayered electronic dermis (e-dermis) with properties based on the behavior of mechanoreceptors and nociceptors to provide neuromorphic tactile information to an amputee. Our biologically inspired e-dermis enables a prosthesis and its user to perceive a continuous spectrum from innocuous to noxious touch through a neuromorphic interface that produces receptor-like spiking neural activity. In a Pain Detection Task (PDT), we show the ability of the prosthesis and amputee to differentiate non-painful or painful tactile stimuli using sensory feedback and a pain reflex feedback control system. In this work, an amputee can use perceptions of touch and pain to discriminate object curvature, including sharpness. This work demonstrates possibilities for creating a more natural sensation spanning a range of tactile stimuli for prosthetic hands.

Limb Position Tolerant Pattern Recognition for Myoelectric Prosthesis Control with Adaptive Sparse Representations From Extreme Learning.

Myoelectric signals can be used to predict the intended movements of an amputee for prosthesis control. However, untrained effects like limb position changes influence myoelectric signal characteristics, hindering the ability of pattern recognition algorithms to discriminate among motion classes. Despite frequent and long training sessions, these deleterious conditional influences may result in poor performance and device abandonment. GOAL: We present a robust sparsity-based adaptive classification method that is significantly less sensitive to signal deviations resulting from untrained conditions. METHODS: We compare this approach in the offline and online contexts of untrained upper-limb positions for amputee and able-bodied subjects to demonstrate its robustness compared against other myoelectric classification methods. RESULTS: We report significant performance improvements () in untrained limb positions across all subject groups. SIGNIFICANCE: The robustness of our suggested approach helps to ensure better untrained condition performance from fewer training conditions. CONCLUSIONS: This method of prosthesis control has the potential to deliver real-world clinical benefits to amputees: better condition-tolerant performance, reduced training burden in terms of frequency and duration, and increased adoption of myoelectric prostheses.

Limb-position robust classification of myoelectric signals for prosthesis control using sparse representations.

The fundamental objective in non-invasive myoelectric prosthesis control is to determine the user's intended movements from corresponding skin-surface recorded electromyographic (sEMG) activation signals as quickly and accurately as possible. Linear Discriminant Analysis (LDA) has emerged as the de facto standard for real-time movement classification due to its ease of use, calculation speed, and remarkable classification accuracy under controlled training conditions. However, performance of cluster-based methods like LDA for sEMG pattern recognition degrades significantly when real-world testing conditions do not resemble the trained conditions, limiting the utility of myoelectrically controlled prosthesis devices. We propose an enhanced classification method that is more robust to generic deviations from training conditions by constructing sparse representations of the input data dictionary comprised of sEMG time-frequency features. We apply our method in the context of upper-limb position changes to demonstrate pattern recognition robustness and improvement over LDA across discrete positions not explicitly trained. For single position training we report an accuracy improvement in untrained positions of 7.95%, p ≪ .001, in addition to significant accuracy improvements across all multiposition training conditions, p <; .001.