A Multimodal Assistive-Robotic-Arm Control System to Increase Independence After Tetraplegia.

Following tetraplegia, independence for completing essential daily tasks, such as opening doors and eating, significantly declines. Assistive robotic manipulators (ARMs) could restore independence, but typically input devices for these manipulators require functional use of the hands. We created and validated a hands-free multimodal input system for controlling an ARM in virtual reality using combinations of a gyroscope, eye-tracking, and heterologous surface electromyography (sEMG). These input modalities are mapped to ARM functions based on the user's preferences and to maximize the utility of their residual volitional capabilities following tetraplegia. The two participants in this study with tetraplegia preferred to use the control mapping with sEMG button functions and disliked winking commands. Non-disabled participants were more varied in their preferences and performance, further suggesting that customizability is an advantageous component of the control system. Replacing buttons from a traditional handheld controller with sEMG did not substantively reduce performance. The system provided adequate control to all participants to complete functional tasks in virtual reality such as opening door handles, turning stove dials, eating, and drinking, all of which enable independence and improved quality of life for these individuals.

Validity and Impact of Methods for Collecting Training Data for Myoelectric Prosthetic Control Algorithms.

Intuitive regression control of prostheses relies on training algorithms to correlate biological recordings to motor intent. The quality of the training dataset is critical to run-time regression performance, but accurately labeling intended hand kinematics after hand amputation is challenging. In this study, we quantified the accuracy and precision of labeling hand kinematics using two common training paradigms: 1) mimic training, where participants mimic predetermined motions of a prosthesis, and 2) mirror training, where participants mirror their contralateral intact hand during synchronized bilateral movements. We first explored this question in healthy non-amputee individuals where the ground-truth kinematics could be readily determined using motion capture. Kinematic data showed that mimic training fails to account for biomechanical coupling and temporal changes in hand posture. Additionally, mirror training exhibited significantly higher accuracy and precision in labeling hand kinematics. These findings suggest that the mirror training approach generates a more faithful, albeit more complex, dataset. Accordingly, mirror training resulted in significantly better offline regression performance when using a large amount of training data and a non-linear neural network. Next, we explored these different training paradigms online, with a cohort of unilateral transradial amputees actively controlling a prosthesis in real-time to complete a functional task. Overall, we found that mirror training resulted in significantly faster task completion speeds and similar subjective workload. These results demonstrate that mirror training can potentially provide more dexterous control through the utilization of task-specific, user-selected training data. Consequently, these findings serve as a valuable guide for the next generation of myoelectric and neuroprostheses leveraging machine learning to provide more dexterous and intuitive control.

A Multi-User Transradial Functional-Test Socket for Validation of New Myoelectric Prosthetic Control Strategies.

The validation of myoelectric prosthetic control strategies for individuals experiencing upper-limb loss is hindered by the time and cost affiliated with traditional custom-fabricated sockets. Consequently, researchers often rely upon virtual reality or robotic arms to validate novel control strategies, which limits end-user involvement. Prosthetists fabricate diagnostic check sockets to assess and refine socket fit, but these clinical techniques are not readily available to researchers and are not intended to assess functionality for control strategies. Here we present a multi-user, low-cost, transradial, functional-test socket for short-term research use that can be custom-fit and donned rapidly, used in conjunction with various electromyography configurations, and adapted for use with various residual limbs and terminal devices. In this study, participants with upper-limb amputation completed functional tasks in physical and virtual environments both with and without the socket, and they reported on their perceived comfort level over time. The functional-test socket was fabricated prior to participants' arrival, iteratively fitted by the researchers within 10 mins, and donned in under 1 min (excluding electrode placement, which will vary for different use cases). It accommodated multiple individuals and terminal devices and had a total cost of materials under $10 USD. Across all participants, the socket did not significantly impede functional task performance or reduce the electromyography signal-to-noise ratio. The socket was rated as comfortable enough for at least 2 h of use, though it was expectedly perceived as less comfortable than a clinically-prescribed daily-use socket. The development of this multi-user, transradial, functional-test socket constitutes an important step toward increased end-user participation in advanced myoelectric prosthetic research. The socket design has been open-sourced and is available for other researchers.

Portable System for Home Use Enables Closed-Loop, Continuous Control of Multi-Degree-of-Freedom Bionic Arm.

Commercial prosthetic hands are frequently abandoned due to unintuitive control methods and a lack of sensory feedback from the prosthesis. Advanced neuromyoelectric prostheses can restore intuitive control and sensory feedback to prosthesis users and potentially reduce abandonment. However, not all advanced prosthetic systems are deployable for home use on portable systems with limited computational power. In this work, we use a commercially available portable neural interface processor (the Ripple Neuro Nomad), and a multi-degree-of-freedom bionic arm (the DEKA LUKE Arm) to create a closed-loop neuromyoelectric prosthesis. The system restores intuitive, independent, continuous control over the arm's six-degrees-of-freedom and provides sensory feedback for up to 288 neural and six vibrotactile channels. Additionally, the large storage capacity of the system enables high-resolution logging of EMG, hand positions, prosthesis sensors, and stimulation parameters. We developed two GUIs enabling wireless, real-time adjustments to motor control and feedback parameters: one with nearly full control over motor control and feedback parameters for investigators, and one with restricted capabilities enabling end-user safety. We verified the system's closed-loop function through a fragile egg task with vibrotactile sensory feedback. We tested the neural stimulation with an amplifier capable of eliciting transcutaneous percepts. This neuromyoelectric prosthetic system will be used for an extended take-home trial that could provide strong clinical justification for advanced, closed-loop prostheses.Clinical Relevance- This work establishes an advanced, intuitive, sensorized prosthesis that can be used in home and clinical settings.

Bilaterally Mirrored Movements Improve the Accuracy and Precision of Training Data for Supervised Learning of Neural or Myoelectric Prosthetic Control.

Intuitive control of prostheses relies on training algorithms to correlate biological recordings to motor intent. The quality of the training dataset is critical to run-time performance, but it is difficult to label hand kinematics accurately after the hand has been amputated. We quantified the accuracy and precision of labeling hand kinematics for two different training approaches: 1) assuming a participant is perfectly mimicking predetermined motions of a prosthesis (mimicked training), and 2) assuming a participant is perfectly mirroring their contralateral hand during identical bilateral movements (mirrored training). We compared these approaches in non-amputee individuals, using an infrared camera to track eight different joint angles of the hands in real-time. Aggregate data revealed that mimicked training does not account for biomechanical coupling or temporal changes in hand posture. Mirrored training was significantly more accurate and precise at labeling hand kinematics. However, when training a modified Kalman filter to estimate motor intent, the mimicked and mirrored training approaches were not significantly different. The results suggest that the mirrored training approach creates a more faithful but more complex dataset. Advanced algorithms, more capable of learning the complex mirrored training dataset, may yield better run-time prosthetic control.