Integrating Upper-Limb Prostheses with the Human Body: Technology Advances, Readiness, and Roles in Human-Prosthesis Interaction.

Significant advances in bionic prosthetics have occurred in the past two decades. The field's rapid expansion has yielded many exciting technologies that can enhance the physical, functional, and cognitive integration of a prosthetic limb with a human. We review advances in the engineering of prosthetic devices and their interfaces with the human nervous system, as well as various surgical techniques for altering human neuromusculoskeletal systems for seamless human-prosthesis integration. We discuss significant advancements in research and clinical translation, focusing on upper limbprosthetics since they heavily rely on user intent for daily operation, although many discussed technologies have been extended to lower limb prostheses as well. In addition, our review emphasizes the roles of advanced prosthetics technologies in complex interactions with humans and the technology readiness levels (TRLs) of individual research advances. Finally, we discuss current gaps and controversies in the field and point out future research directions, guided by TRLs.

Wrist speed feedback improves elbow compensation and reaching accuracy for myoelectric transradial prosthesis users in hybrid virtual reaching task.

BACKGROUND: Myoelectric prostheses are a popular choice for restoring motor capability following the loss of a limb, but they do not provide direct feedback to the user about the movements of the device-in other words, kinesthesia. The outcomes of studies providing artificial sensory feedback are often influenced by the availability of incidental feedback. When subjects are blindfolded and disconnected from the prosthesis, artificial sensory feedback consistently improves control; however, when subjects wear a prosthesis and can see the task, benefits often deteriorate or become inconsistent. We theorize that providing artificial sensory feedback about prosthesis speed, which cannot be precisely estimated via vision, will improve the learning and control of a myoelectric prosthesis. METHODS: In this study, we test a joint-speed feedback system with six transradial amputee subjects to evaluate how it affects myoelectric control and adaptation behavior during a virtual reaching task. RESULTS: Our results showed that joint-speed feedback lowered reaching errors and compensatory movements during steady-state reaches. However, the same feedback provided no improvement when control was perturbed. CONCLUSIONS: These outcomes suggest that the benefit of joint speed feedback may be dependent on the complexity of the myoelectric control and the context of the task.

Improving internal model strength and performance of prosthetic hands using augmented feedback.

BACKGROUND: The loss of an arm presents a substantial challenge for upper limb amputees when performing activities of daily living. Myoelectric prosthetic devices partially replace lost hand functions; however, lack of sensory feedback and strong understanding of the myoelectric control system prevent prosthesis users from interacting with their environment effectively. Although most research in augmented sensory feedback has focused on real-time regulation, sensory feedback is also essential for enabling the development and correction of internal models, which in turn are used for planning movements and reacting to control variability faster than otherwise possible in the presence of sensory delays. METHODS: Our recent work has demonstrated that audio-augmented feedback can improve both performance and internal model strength for an abstract target acquisition task. Here we use this concept in controlling a robotic hand, which has inherent dynamics and variability, and apply it to a more functional grasp-and-lift task. We assessed internal model strength using psychophysical tests and used an instrumented Virtual Egg to assess performance. RESULTS: Results obtained from 14 able-bodied subjects show that a classifier-based controller augmented with audio feedback enabled stronger internal model (p = 0.018) and better performance (p = 0.028) than a controller without this feedback. CONCLUSIONS: We extended our previous work and accomplished the first steps on a path towards bridging the gap between research and clinical usability of a hand prosthesis. The main goal was to assess whether the ability to decouple internal model strength and motion variability using the continuous audio-augmented feedback extended to real-world use, where the inherent mechanical variability and dynamics in the mechanisms may contribute to a more complicated interplay between internal model formation and motion variability. We concluded that benefits of using audio-augmented feedback for improving internal model strength of myoelectric controllers extend beyond a virtual target acquisition task to include control of a prosthetic hand.

Virtual Constraint Control of a Powered Prosthetic Leg: From Simulation to Experiments with Transfemoral Amputees.

Recent powered (or robotic) prosthetic legs independently control different joints and time periods of the gait cycle, resulting in control parameters and switching rules that can be difficult to tune by clinicians. This challenge might be addressed by a unifying control model used by recent bipedal robots, in which virtual constraints define joint patterns as functions of a monotonic variable that continuously represents the gait cycle phase. In the first application of virtual constraints to amputee locomotion, this paper derives exact and approximate control laws for a partial feedback linearization to enforce virtual constraints on a prosthetic leg. We then encode a human-inspired invariance property called effective shape into virtual constraints for the stance period. After simulating the robustness of the partial feedback linearization to clinically meaningful conditions, we experimentally implement this control strategy on a powered transfemoral leg. We report the results of three amputee subjects walking overground and at variable cadences on a treadmill, demonstrating the clinical viability of this novel control approach.

Towards Biomimetic Virtual Constraint Control of a Powered Prosthetic Leg.

This brief presents a novel control strategy for a powered prosthetic ankle based on a biomimetic virtual constraint. We first derive a kinematic constraint for the "effective shape" of the human ankle-foot complex during locomotion. This shape characterizes ankle motion as a function of the Center of Pressure (COP)-the point on the foot sole where the resultant ground reaction force is imparted. Since the COP moves monotonically from heel to toe during steady walking, we adopt the COP as a mechanical representation of the gait cycle phase in an autonomous feedback controller. We show that our kinematic constraint can be enforced as a virtual constraint by an output linearizing controller that uses only feedback available to sensors onboard a prosthetic leg. Using simulations of a passive walking model with feet, we show that this novel controller exactly enforces the desired effective shape whereas a standard impedance (i.e., proportional-derivative) controller cannot. This work provides a single, biomimetic control law for the entire single-support period during robot-assisted locomotion.

Joint speed feedback improves myoelectric prosthesis adaptation after perturbed reaches in non amputees.

Accurate control of human limbs involves both feedforward and feedback signals. For prosthetic arms, feedforward control is commonly accomplished by recording myoelectric signals from the residual limb to predict the user's intent, but augmented feedback signals are not explicitly provided in commercial devices. Previous studies have demonstrated inconsistent results when artificial feedback was provided in the presence of vision; some studies showed benefits, while others did not. We hypothesized that negligible benefits in past studies may have been due to artificial feedback with low precision compared to vision, which results in heavy reliance on vision during reaching tasks. Furthermore, we anticipated more reliable benefits from artificial feedback when providing information that vision estimates with high uncertainty (e.g. joint speed). In this study, we test an artificial sensory feedback system providing joint speed information and how it impacts performance and adaptation during a hybrid positional-and-myoelectric ballistic reaching task. We found that overall reaching errors were reduced after perturbed control, but did not significantly improve steady-state reaches. Furthermore, we found that feedback about the joint speed of the myoelectric prosthesis control improved the adaptation rate of biological limb movements, which may have resulted from high prosthesis control noise and strategic overreaching with the positional control and underreaching with the myoelectric control. These results provide insights into the relevant factors influencing the improvements conferred by artificial sensory feedback.

An analytical method reduces noise bias in motor adaptation analysis.

When a person makes a movement, a motor error is typically observed that then drives motor planning corrections on subsequent movements. This error correction, quantified as a trial-by-trial adaptation rate, provides insight into how the nervous system is operating, particularly regarding how much confidence a person places in different sources of information such as sensory feedback or motor command reproducibility. Traditional analysis has required carefully controlled laboratory conditions such as the application of perturbations or error clamping, limiting the usefulness of motor analysis in clinical and everyday environments. Here we focus on error adaptation during unperturbed and naturalistic movements. With increasing motor noise, we show that the conventional estimation of trial-by-trial adaptation increases, a counterintuitive finding that is the consequence of systematic bias in the estimate due to noise masking the learner's intention. We present an analytic solution relying on stochastic signal processing to reduce this effect of noise, producing an estimate of motor adaptation with reduced bias. The result is an improved estimate of trial-by-trial adaptation in a human learner compared to conventional methods. We demonstrate the effectiveness of the new method in analyzing simulated and empirical movement data under different noise conditions.

Embodied Cooperation to Promote Forgiving Interactions With Autonomous Machines.

During every waking moment, we must engage with our environments, the people around us, the tools we use, and even our own bodies to perform actions and achieve our intentions. There is a spectrum of control that we have over our surroundings that spans the extremes from full to negligible. When the outcomes of our actions do not align with our goals, we have a tremendous capacity to displace blame and frustration on external factors while forgiving ourselves. This is especially true when we cooperate with machines; they are rarely afforded the level of forgiveness we provide our bodies and often bear much of our blame. Yet, our brain readily engages with autonomous processes in controlling our bodies to coordinate complex patterns of muscle contractions, make postural adjustments, adapt to external perturbations, among many others. This acceptance of biological autonomy may provide avenues to promote more forgiving human-machine partnerships. In this perspectives paper, we argue that striving for machine embodiment is a pathway to achieving effective and forgiving human-machine relationships. We discuss the mechanisms that help us identify ourselves and our bodies as separate from our environments and we describe their roles in achieving embodied cooperation. Using a representative selection of examples in neurally interfaced prosthetic limbs and intelligent mechatronics, we describe techniques to engage these same mechanisms when designing autonomous systems and their potential bidirectional interfaces.

Neurorobotic fusion of prosthetic touch, kinesthesia, and movement in bionic upper limbs promotes intrinsic brain behaviors.

Bionic prostheses have restorative potential. However, the complex interplay between intuitive motor control, proprioception, and touch that represents the hallmark of human upper limb function has not been revealed. Here, we show that the neurorobotic fusion of touch, grip kinesthesia, and intuitive motor control promotes levels of behavioral performance that are stratified toward able-bodied function and away from standard-of-care prosthetic users. This was achieved through targeted motor and sensory reinnervation, a closed-loop neural-machine interface, coupled to a noninvasive robotic architecture. Adding touch to motor control improves the ability to reach intended target grasp forces, find target durometers among distractors, and promote prosthetic ownership. Touch, kinesthesia, and motor control restore balanced decision strategies when identifying target durometers and intrinsic visuomotor behaviors that reduce the need to watch the prosthetic hand during object interactions, which frees the eyes to look ahead to the next planned action. The combination of these three modalities also enhances error correction performance. We applied our unified theoretical, functional, and clinical analyses, enabling us to define the relative contributions of the sensory and motor modalities operating simultaneously in this neural-machine interface. This multiperspective framework provides the necessary evidence to show that bionic prostheses attain more human-like function with effective sensory-motor restoration.

A Review of Sensory Feedback in Upper-Limb Prostheses From the Perspective of Human Motor Control.

This manuscript reviews historical and recent studies that focus on supplementary sensory feedback for use in upper limb prostheses. It shows that the inability of many studies to speak to the issue of meaningful performance improvements in real-life scenarios is caused by the complexity of the interactions of supplementary sensory feedback with other types of feedback along with other portions of the motor control process. To do this, the present manuscript frames the question of supplementary feedback from the perspective of computational motor control, providing a brief review of the main advances in that field over the last 20 years. It then separates the studies on the closed-loop prosthesis control into distinct categories, which are defined by relating the impact of feedback to the relevant components of the motor control framework, and reviews the work that has been done over the last 50+ years in each of those categories. It ends with a discussion of the studies, along with suggestions for experimental construction and connections with other areas of research, such as machine learning.

When Less Is More - Discrete Tactile Feedback Dominates Continuous Audio Biofeedback in the Integrated Percept While Controlling a Myoelectric Prosthetic Hand.

State of the art myoelectric hand prostheses can restore some feedforward motor function to their users, but they cannot yet restore sensory feedback. It has been shown, using psychophysical tests, that multi-modal sensory feedback is readily used in the formation of the users' representation of the control task in their central nervous system - their internal model. Hence, to fully describe the effect of providing feedback to prosthesis users, not only should functional outcomes be assessed, but so should the internal model. In this study, we compare the complex interactions between two different feedback types, as well as a combination of the two, on the internal model, and the functional performance of naïve participants without limb difference. We show that adding complementary audio biofeedback to visual feedback enables the development of a significantly stronger internal model for controlling a myoelectric hand compared to visual feedback alone, but adding discrete vibrotactile feedback to vision does not. Both types of feedback, however, improved the functional grasping abilities to a similar degree. Contrary to our expectations, when both types of feedback are combined, the discrete vibrotactile feedback seems to dominate the continuous audio feedback. This finding indicates that simply adding sensory information may not necessarily enhance the formation of the internal model in the short term. In fact, it could even degrade it. These results support our argument that assessment of the internal model is crucial to understanding the effects of any type of feedback, although we cannot be sure that the metrics used here describe the internal model exhaustively. Furthermore, all the feedback types tested herein have been proven to provide significant functional benefits to the participants using a myoelectrically controlled robotic hand. This article, therefore, proposes a crucial conceptual and methodological addition to the evaluation of sensory feedback for upper limb prostheses - the internal model - as well as new types of feedback that promise to significantly and considerably improve functional prosthesis control.

Modeling and preliminary testing socket-residual limb interface stiffness of above-elbow prostheses.

The interface between the socket and residual limb can have a significant effect on the performance of a prosthesis. Specifically, knowledge of the rotational stiffness of the socket-residual limb (S-RL) interface is extremely useful in designing new prostheses and evaluating new control paradigms, as well as in comparing existing and new socket technologies. No previous studies, however, have examined the rotational stiffness of S-RL interfaces. To address this problem, a math model is compared to a more complex finite element analysis, to see if the math model sufficiently captures the main effects of S-RL interface rotational stiffness. Both of these models are then compared to preliminary empirical testing, in which a series of X-rays, called fluoroscopy, is taken to obtain the movement of the bone relative to the socket. Force data are simultaneously recorded, and the combination of force and movement data are used to calculate the empirical rotational stiffness of elbow S-RL interface. The empirical rotational stiffness values are then compared to the models, to see if values of Young's modulus obtained in other studies at localized points may be used to determine the global rotational stiffness of the S-RL interface. Findings include agreement between the models and empirical results and the ability of persons to significantly modulate the rotational stiffness of their S-RL interface a little less than one order of magnitude. The floor and ceiling of this range depend significantly on socket length and co-contraction levels, but not on residual limb diameter or bone diameter. Measured trans-humeral S-RL interface rotational stiffness values ranged from 24-140 Nm/rad for the four subjects tested in this study.

Audible Feedback Improves Internal Model Strength and Performance of Myoelectric Prosthesis Control.

Myoelectric prosthetic devices are commonly used to help upper limb amputees perform activities of daily living, however amputees still lack the sensory feedback required to facilitate reliable and precise control. Augmented feedback may play an important role in affecting both short-term performance, through real-time regulation, and long-term performance, through the development of stronger internal models. In this work, we investigate the potential tradeoff between controllers that enable better short-term performance and those that provide sufficient feedback to develop a strong internal model. We hypothesize that augmented feedback may be used to mitigate this tradeoff, ultimately improving both short and long-term control. We used psychometric measures to assess the internal model developed while using a filtered myoelectric controller with augmented audio feedback, imitating classification-based control but with augmented regression-based feedback. In addition, we evaluated the short-term performance using a multi degree-of-freedom constrained-time target acquisition task. Results obtained from 24 able-bodied subjects show that an augmented feedback control strategy using audio cues enables the development of a stronger internal model than the filtered control with filtered feedback, and significantly better path efficiency than both raw and filtered control strategies. These results suggest that the use of augmented feedback control strategies may improve both short-term and long-term performance.

Evaluating Internal Model Strength and Performance of Myoelectric Prosthesis Control Strategies.

On-going developments in myoelectric prosthesis control have provided prosthesis users with an assortment of control strategies that vary in reliability and performance. Many studies have focused on improving performance by providing feedback to the user but have overlooked the effect of this feedback on internal model development, which is key to improve long-term performance. In this paper, the strength of internal models developed for two commonly used myoelectric control strategies: raw control with raw feedback (using a regression-based approach) and filtered control with filtered feedback (using a classifier-based approach), were evaluated using two psychometric measures: trial-by-trial adaptation and just-noticeable difference. The performance of both strategies was also evaluated using Schmidt's style target acquisition task. Results obtained from 24 able-bodied subjects showed that although filtered control with filtered feedback had better short-term performance in path efficiency ( ), raw control with raw feedback resulted in stronger internal model development ( ), which may lead to better long-term performance. Despite inherent noise in the control signals of the regression controller, these findings suggest that rich feedback associated with regression control may be used to improve human understanding of the myoelectric control system.

Validation of a constrained-time movement task for use in rehabilitation outcome measures.

Current motor assessment tools can provide numerical indicators of performance but do not provide actionable information to target further improvement in rehabilitation interventions. Psychophysics-based outcome measures show promise to provide more useful information in the laboratory environment but have been limited in clinical implementation. Here we present a constrained-time task to assess paced and non-rhythmic movements. The task's output metrics include trial-by-trial adaptation rate and the just noticeable difference of a perturbation. We show that the task's metrics are reliable (i.e. high test-retest reliability) and are responsive to changes in feedback type and experience. We also discuss the task's versatility to be used for other types of movements including grasping. The consistent, sensitive and flexible time-constrained movement task we present provides a foundation from which to develop advanced outcome measures for prosthesis users and for other rehabilitation contexts.

The effect of myoelectric prosthesis control strategies and feedback level on adaptation rate for a target acquisition task.

The long-term performance of myoelectric prostheses is related not only to the short-term performance of the controller, but also to the user's ability to learn and adapt to the system. Different control architectures may have inherent tradeoffs between their short-term performance and the amount of relevant feedback that informs this adaptation. In this study we focused on the ability of two common types of myoelectric control interfaces: raw control with raw feedback, such as a regression, and filtered control with filtered feedback, such as a classifier, to affect user adaptation. We evaluated trial-by-trial adaptation to self-generated errors during a multi degree-of-freedom target acquisition task by fitting a linear regression model to data collected from 24 able-bodied subjects. Subjects showed significantly higher adaptation behavior to self-generated errors when using raw control with a raw feedback strategy than when using filtered control with a filtered feedback strategy, which suggests that control strategies with more feedback allow for higher adaptation. These results support our hypothesis that feedback-rich control strategies allow users to better understand the myoelectric control system, which may enable better long-term performance.

EMG Versus Torque Control of Human-Machine Systems: Equalizing Control Signal Variability Does not Equalize Error or Uncertainty.

In this paper we asked the question: if we artificially raise the variability of torque control signals to match that of EMG, do subjects make similar errors and have similar uncertainty about their movements? We answered this question using two experiments in which subjects used three different control signals: torque, torque+noise, and EMG. First, we measured error on a simple target-hitting task in which subjects received visual feedback only at the end of their movements. We found that even when the signal-to-noise ratio was equal across EMG and torque+noise control signals, EMG resulted in larger errors. Second, we quantified uncertainty by measuring the just-noticeable difference of a visual perturbation. We found that for equal errors, EMG resulted in higher movement uncertainty than both torque and torque+noise. The differences suggest that performance and confidence are influenced by more than just the noisiness of the control signal, and suggest that other factors, such as the user's ability to incorporate feedback and develop accurate internal models, also have significant impacts on the performance and confidence of a person's actions. We theorize that users have difficulty distinguishing between random and systematic errors for EMG control, and future work should examine in more detail the types of errors made with EMG control.

Adaptation to random and systematic errors: Comparison of amputee and non-amputee control interfaces with varying levels of process noise.

The objective of this study was to understand how people adapt to errors when using a myoelectric control interface. We compared adaptation across 1) non-amputee subjects using joint angle, joint torque, and myoelectric control interfaces, and 2) amputee subjects using myoelectric control interfaces with residual and intact limbs (five total control interface conditions). We measured trial-by-trial adaptation to self-generated errors and random perturbations during a virtual, single degree-of-freedom task with two levels of feedback uncertainty, and evaluated adaptation by fitting a hierarchical Kalman filter model. We have two main results. First, adaptation to random perturbations was similar across all control interfaces, whereas adaptation to self-generated errors differed. These patterns matched predictions of our model, which was fit to each control interface by changing the process noise parameter that represented system variability. Second, in amputee subjects, we found similar adaptation rates and error levels between residual and intact limbs. These results link prosthesis control to broader areas of motor learning and adaptation and provide a useful model of adaptation with myoelectric control. The model of adaptation will help us understand and solve prosthesis control challenges, such as providing additional sensory feedback.

Comparing functional EMG characteristics between zero-order and first-order interface dynamics.

The optimal control scheme for powered prostheses can be determined using simulation experiments, for which an accurate model of prosthesis control is essential. This paper focuses on electromyographic (EMG) control signal characteristics across two different control schemes. We constructed a functional EMG model comprising three EMG signal characteristics-standard deviation, kurtosis, and median power frequency-using data collected under realistic conditions for prosthesis control (closed-loop, dynamic, anisometric contractions). We examined how the model changed when subjects used zero-order or first-order control. Control order had a statistically significant effect on EMG characteristics, but the effect size was small and generally did not exceed inter-subject variability. Therefore, we suggest that this functional EMG model remains valid across different control schemes.

Does EMG control lead to distinct motor adaptation?

Powered prostheses are controlled using electromyographic (EMG) signals, which may introduce high levels of uncertainty even for simple tasks. According to Bayesian theories, higher uncertainty should influence how the brain adapts motor commands in response to perceived errors. Such adaptation may critically influence how patients interact with their prosthetic devices; however, we do not yet understand adaptation behavior with EMG control. Models of adaptation can offer insights on movement planning and feedback correction, but we first need to establish their validity for EMG control interfaces. Here we created a simplified comparison of prosthesis and able-bodied control by studying adaptation with three control interfaces: joint angle, joint torque, and EMG. Subjects used each of the control interfaces to perform a target-directed task with random visual perturbations. We investigated how control interface and visual uncertainty affected trial-by-trial adaptation. As predicted by Bayesian models, increased errors and decreased visual uncertainty led to faster adaptation. The control interface had no significant effect beyond influencing error sizes. This result suggests that Bayesian models are useful for describing prosthesis control and could facilitate further investigation to characterize the uncertainty faced by prosthesis users. A better understanding of factors affecting movement uncertainty will guide sensory feedback strategies for powered prostheses and clarify what feedback information best improves control.