Integrating Upper-Limb Prostheses with the Human Body: Technology Advances, Readiness, and Roles in Human-Prosthesis Interaction.

Significant advances in bionic prosthetics have occurred in the past two decades. The field's rapid expansion has yielded many exciting technologies that can enhance the physical, functional, and cognitive integration of a prosthetic limb with a human. We review advances in the engineering of prosthetic devices and their interfaces with the human nervous system, as well as various surgical techniques for altering human neuromusculoskeletal systems for seamless human-prosthesis integration. We discuss significant advancements in research and clinical translation, focusing on upper limbprosthetics since they heavily rely on user intent for daily operation, although many discussed technologies have been extended to lower limb prostheses as well. In addition, our review emphasizes the roles of advanced prosthetics technologies in complex interactions with humans and the technology readiness levels (TRLs) of individual research advances. Finally, we discuss current gaps and controversies in the field and point out future research directions, guided by TRLs.

Wrist speed feedback improves elbow compensation and reaching accuracy for myoelectric transradial prosthesis users in hybrid virtual reaching task.

BACKGROUND: Myoelectric prostheses are a popular choice for restoring motor capability following the loss of a limb, but they do not provide direct feedback to the user about the movements of the device-in other words, kinesthesia. The outcomes of studies providing artificial sensory feedback are often influenced by the availability of incidental feedback. When subjects are blindfolded and disconnected from the prosthesis, artificial sensory feedback consistently improves control; however, when subjects wear a prosthesis and can see the task, benefits often deteriorate or become inconsistent. We theorize that providing artificial sensory feedback about prosthesis speed, which cannot be precisely estimated via vision, will improve the learning and control of a myoelectric prosthesis. METHODS: In this study, we test a joint-speed feedback system with six transradial amputee subjects to evaluate how it affects myoelectric control and adaptation behavior during a virtual reaching task. RESULTS: Our results showed that joint-speed feedback lowered reaching errors and compensatory movements during steady-state reaches. However, the same feedback provided no improvement when control was perturbed. CONCLUSIONS: These outcomes suggest that the benefit of joint speed feedback may be dependent on the complexity of the myoelectric control and the context of the task.

Neurorobotic fusion of prosthetic touch, kinesthesia, and movement in bionic upper limbs promotes intrinsic brain behaviors.

Bionic prostheses have restorative potential. However, the complex interplay between intuitive motor control, proprioception, and touch that represents the hallmark of human upper limb function has not been revealed. Here, we show that the neurorobotic fusion of touch, grip kinesthesia, and intuitive motor control promotes levels of behavioral performance that are stratified toward able-bodied function and away from standard-of-care prosthetic users. This was achieved through targeted motor and sensory reinnervation, a closed-loop neural-machine interface, coupled to a noninvasive robotic architecture. Adding touch to motor control improves the ability to reach intended target grasp forces, find target durometers among distractors, and promote prosthetic ownership. Touch, kinesthesia, and motor control restore balanced decision strategies when identifying target durometers and intrinsic visuomotor behaviors that reduce the need to watch the prosthetic hand during object interactions, which frees the eyes to look ahead to the next planned action. The combination of these three modalities also enhances error correction performance. We applied our unified theoretical, functional, and clinical analyses, enabling us to define the relative contributions of the sensory and motor modalities operating simultaneously in this neural-machine interface. This multiperspective framework provides the necessary evidence to show that bionic prostheses attain more human-like function with effective sensory-motor restoration.

Embodied Cooperation to Promote Forgiving Interactions With Autonomous Machines.

During every waking moment, we must engage with our environments, the people around us, the tools we use, and even our own bodies to perform actions and achieve our intentions. There is a spectrum of control that we have over our surroundings that spans the extremes from full to negligible. When the outcomes of our actions do not align with our goals, we have a tremendous capacity to displace blame and frustration on external factors while forgiving ourselves. This is especially true when we cooperate with machines; they are rarely afforded the level of forgiveness we provide our bodies and often bear much of our blame. Yet, our brain readily engages with autonomous processes in controlling our bodies to coordinate complex patterns of muscle contractions, make postural adjustments, adapt to external perturbations, among many others. This acceptance of biological autonomy may provide avenues to promote more forgiving human-machine partnerships. In this perspectives paper, we argue that striving for machine embodiment is a pathway to achieving effective and forgiving human-machine relationships. We discuss the mechanisms that help us identify ourselves and our bodies as separate from our environments and we describe their roles in achieving embodied cooperation. Using a representative selection of examples in neurally interfaced prosthetic limbs and intelligent mechatronics, we describe techniques to engage these same mechanisms when designing autonomous systems and their potential bidirectional interfaces.

An analytical method reduces noise bias in motor adaptation analysis.

When a person makes a movement, a motor error is typically observed that then drives motor planning corrections on subsequent movements. This error correction, quantified as a trial-by-trial adaptation rate, provides insight into how the nervous system is operating, particularly regarding how much confidence a person places in different sources of information such as sensory feedback or motor command reproducibility. Traditional analysis has required carefully controlled laboratory conditions such as the application of perturbations or error clamping, limiting the usefulness of motor analysis in clinical and everyday environments. Here we focus on error adaptation during unperturbed and naturalistic movements. With increasing motor noise, we show that the conventional estimation of trial-by-trial adaptation increases, a counterintuitive finding that is the consequence of systematic bias in the estimate due to noise masking the learner's intention. We present an analytic solution relying on stochastic signal processing to reduce this effect of noise, producing an estimate of motor adaptation with reduced bias. The result is an improved estimate of trial-by-trial adaptation in a human learner compared to conventional methods. We demonstrate the effectiveness of the new method in analyzing simulated and empirical movement data under different noise conditions.

Joint speed feedback improves myoelectric prosthesis adaptation after perturbed reaches in non amputees.

Accurate control of human limbs involves both feedforward and feedback signals. For prosthetic arms, feedforward control is commonly accomplished by recording myoelectric signals from the residual limb to predict the user's intent, but augmented feedback signals are not explicitly provided in commercial devices. Previous studies have demonstrated inconsistent results when artificial feedback was provided in the presence of vision; some studies showed benefits, while others did not. We hypothesized that negligible benefits in past studies may have been due to artificial feedback with low precision compared to vision, which results in heavy reliance on vision during reaching tasks. Furthermore, we anticipated more reliable benefits from artificial feedback when providing information that vision estimates with high uncertainty (e.g. joint speed). In this study, we test an artificial sensory feedback system providing joint speed information and how it impacts performance and adaptation during a hybrid positional-and-myoelectric ballistic reaching task. We found that overall reaching errors were reduced after perturbed control, but did not significantly improve steady-state reaches. Furthermore, we found that feedback about the joint speed of the myoelectric prosthesis control improved the adaptation rate of biological limb movements, which may have resulted from high prosthesis control noise and strategic overreaching with the positional control and underreaching with the myoelectric control. These results provide insights into the relevant factors influencing the improvements conferred by artificial sensory feedback.

A Review of Sensory Feedback in Upper-Limb Prostheses From the Perspective of Human Motor Control.

This manuscript reviews historical and recent studies that focus on supplementary sensory feedback for use in upper limb prostheses. It shows that the inability of many studies to speak to the issue of meaningful performance improvements in real-life scenarios is caused by the complexity of the interactions of supplementary sensory feedback with other types of feedback along with other portions of the motor control process. To do this, the present manuscript frames the question of supplementary feedback from the perspective of computational motor control, providing a brief review of the main advances in that field over the last 20 years. It then separates the studies on the closed-loop prosthesis control into distinct categories, which are defined by relating the impact of feedback to the relevant components of the motor control framework, and reviews the work that has been done over the last 50+ years in each of those categories. It ends with a discussion of the studies, along with suggestions for experimental construction and connections with other areas of research, such as machine learning.

When Less Is More - Discrete Tactile Feedback Dominates Continuous Audio Biofeedback in the Integrated Percept While Controlling a Myoelectric Prosthetic Hand.

State of the art myoelectric hand prostheses can restore some feedforward motor function to their users, but they cannot yet restore sensory feedback. It has been shown, using psychophysical tests, that multi-modal sensory feedback is readily used in the formation of the users' representation of the control task in their central nervous system - their internal model. Hence, to fully describe the effect of providing feedback to prosthesis users, not only should functional outcomes be assessed, but so should the internal model. In this study, we compare the complex interactions between two different feedback types, as well as a combination of the two, on the internal model, and the functional performance of naïve participants without limb difference. We show that adding complementary audio biofeedback to visual feedback enables the development of a significantly stronger internal model for controlling a myoelectric hand compared to visual feedback alone, but adding discrete vibrotactile feedback to vision does not. Both types of feedback, however, improved the functional grasping abilities to a similar degree. Contrary to our expectations, when both types of feedback are combined, the discrete vibrotactile feedback seems to dominate the continuous audio feedback. This finding indicates that simply adding sensory information may not necessarily enhance the formation of the internal model in the short term. In fact, it could even degrade it. These results support our argument that assessment of the internal model is crucial to understanding the effects of any type of feedback, although we cannot be sure that the metrics used here describe the internal model exhaustively. Furthermore, all the feedback types tested herein have been proven to provide significant functional benefits to the participants using a myoelectrically controlled robotic hand. This article, therefore, proposes a crucial conceptual and methodological addition to the evaluation of sensory feedback for upper limb prostheses - the internal model - as well as new types of feedback that promise to significantly and considerably improve functional prosthesis control.

Improving internal model strength and performance of prosthetic hands using augmented feedback.

BACKGROUND: The loss of an arm presents a substantial challenge for upper limb amputees when performing activities of daily living. Myoelectric prosthetic devices partially replace lost hand functions; however, lack of sensory feedback and strong understanding of the myoelectric control system prevent prosthesis users from interacting with their environment effectively. Although most research in augmented sensory feedback has focused on real-time regulation, sensory feedback is also essential for enabling the development and correction of internal models, which in turn are used for planning movements and reacting to control variability faster than otherwise possible in the presence of sensory delays. METHODS: Our recent work has demonstrated that audio-augmented feedback can improve both performance and internal model strength for an abstract target acquisition task. Here we use this concept in controlling a robotic hand, which has inherent dynamics and variability, and apply it to a more functional grasp-and-lift task. We assessed internal model strength using psychophysical tests and used an instrumented Virtual Egg to assess performance. RESULTS: Results obtained from 14 able-bodied subjects show that a classifier-based controller augmented with audio feedback enabled stronger internal model (p = 0.018) and better performance (p = 0.028) than a controller without this feedback. CONCLUSIONS: We extended our previous work and accomplished the first steps on a path towards bridging the gap between research and clinical usability of a hand prosthesis. The main goal was to assess whether the ability to decouple internal model strength and motion variability using the continuous audio-augmented feedback extended to real-world use, where the inherent mechanical variability and dynamics in the mechanisms may contribute to a more complicated interplay between internal model formation and motion variability. We concluded that benefits of using audio-augmented feedback for improving internal model strength of myoelectric controllers extend beyond a virtual target acquisition task to include control of a prosthetic hand.

Audible Feedback Improves Internal Model Strength and Performance of Myoelectric Prosthesis Control.

Myoelectric prosthetic devices are commonly used to help upper limb amputees perform activities of daily living, however amputees still lack the sensory feedback required to facilitate reliable and precise control. Augmented feedback may play an important role in affecting both short-term performance, through real-time regulation, and long-term performance, through the development of stronger internal models. In this work, we investigate the potential tradeoff between controllers that enable better short-term performance and those that provide sufficient feedback to develop a strong internal model. We hypothesize that augmented feedback may be used to mitigate this tradeoff, ultimately improving both short and long-term control. We used psychometric measures to assess the internal model developed while using a filtered myoelectric controller with augmented audio feedback, imitating classification-based control but with augmented regression-based feedback. In addition, we evaluated the short-term performance using a multi degree-of-freedom constrained-time target acquisition task. Results obtained from 24 able-bodied subjects show that an augmented feedback control strategy using audio cues enables the development of a stronger internal model than the filtered control with filtered feedback, and significantly better path efficiency than both raw and filtered control strategies. These results suggest that the use of augmented feedback control strategies may improve both short-term and long-term performance.

Evaluating Internal Model Strength and Performance of Myoelectric Prosthesis Control Strategies.

On-going developments in myoelectric prosthesis control have provided prosthesis users with an assortment of control strategies that vary in reliability and performance. Many studies have focused on improving performance by providing feedback to the user but have overlooked the effect of this feedback on internal model development, which is key to improve long-term performance. In this paper, the strength of internal models developed for two commonly used myoelectric control strategies: raw control with raw feedback (using a regression-based approach) and filtered control with filtered feedback (using a classifier-based approach), were evaluated using two psychometric measures: trial-by-trial adaptation and just-noticeable difference. The performance of both strategies was also evaluated using Schmidt's style target acquisition task. Results obtained from 24 able-bodied subjects showed that although filtered control with filtered feedback had better short-term performance in path efficiency ( ), raw control with raw feedback resulted in stronger internal model development ( ), which may lead to better long-term performance. Despite inherent noise in the control signals of the regression controller, these findings suggest that rich feedback associated with regression control may be used to improve human understanding of the myoelectric control system.

The effect of myoelectric prosthesis control strategies and feedback level on adaptation rate for a target acquisition task.

The long-term performance of myoelectric prostheses is related not only to the short-term performance of the controller, but also to the user's ability to learn and adapt to the system. Different control architectures may have inherent tradeoffs between their short-term performance and the amount of relevant feedback that informs this adaptation. In this study we focused on the ability of two common types of myoelectric control interfaces: raw control with raw feedback, such as a regression, and filtered control with filtered feedback, such as a classifier, to affect user adaptation. We evaluated trial-by-trial adaptation to self-generated errors during a multi degree-of-freedom target acquisition task by fitting a linear regression model to data collected from 24 able-bodied subjects. Subjects showed significantly higher adaptation behavior to self-generated errors when using raw control with a raw feedback strategy than when using filtered control with a filtered feedback strategy, which suggests that control strategies with more feedback allow for higher adaptation. These results support our hypothesis that feedback-rich control strategies allow users to better understand the myoelectric control system, which may enable better long-term performance.

Validation of a constrained-time movement task for use in rehabilitation outcome measures.

Current motor assessment tools can provide numerical indicators of performance but do not provide actionable information to target further improvement in rehabilitation interventions. Psychophysics-based outcome measures show promise to provide more useful information in the laboratory environment but have been limited in clinical implementation. Here we present a constrained-time task to assess paced and non-rhythmic movements. The task's output metrics include trial-by-trial adaptation rate and the just noticeable difference of a perturbation. We show that the task's metrics are reliable (i.e. high test-retest reliability) and are responsive to changes in feedback type and experience. We also discuss the task's versatility to be used for other types of movements including grasping. The consistent, sensitive and flexible time-constrained movement task we present provides a foundation from which to develop advanced outcome measures for prosthesis users and for other rehabilitation contexts.

Adaptation to random and systematic errors: Comparison of amputee and non-amputee control interfaces with varying levels of process noise.

The objective of this study was to understand how people adapt to errors when using a myoelectric control interface. We compared adaptation across 1) non-amputee subjects using joint angle, joint torque, and myoelectric control interfaces, and 2) amputee subjects using myoelectric control interfaces with residual and intact limbs (five total control interface conditions). We measured trial-by-trial adaptation to self-generated errors and random perturbations during a virtual, single degree-of-freedom task with two levels of feedback uncertainty, and evaluated adaptation by fitting a hierarchical Kalman filter model. We have two main results. First, adaptation to random perturbations was similar across all control interfaces, whereas adaptation to self-generated errors differed. These patterns matched predictions of our model, which was fit to each control interface by changing the process noise parameter that represented system variability. Second, in amputee subjects, we found similar adaptation rates and error levels between residual and intact limbs. These results link prosthesis control to broader areas of motor learning and adaptation and provide a useful model of adaptation with myoelectric control. The model of adaptation will help us understand and solve prosthesis control challenges, such as providing additional sensory feedback.

EMG Versus Torque Control of Human-Machine Systems: Equalizing Control Signal Variability Does not Equalize Error or Uncertainty.

In this paper we asked the question: if we artificially raise the variability of torque control signals to match that of EMG, do subjects make similar errors and have similar uncertainty about their movements? We answered this question using two experiments in which subjects used three different control signals: torque, torque+noise, and EMG. First, we measured error on a simple target-hitting task in which subjects received visual feedback only at the end of their movements. We found that even when the signal-to-noise ratio was equal across EMG and torque+noise control signals, EMG resulted in larger errors. Second, we quantified uncertainty by measuring the just-noticeable difference of a visual perturbation. We found that for equal errors, EMG resulted in higher movement uncertainty than both torque and torque+noise. The differences suggest that performance and confidence are influenced by more than just the noisiness of the control signal, and suggest that other factors, such as the user's ability to incorporate feedback and develop accurate internal models, also have significant impacts on the performance and confidence of a person's actions. We theorize that users have difficulty distinguishing between random and systematic errors for EMG control, and future work should examine in more detail the types of errors made with EMG control.

Does EMG control lead to distinct motor adaptation?

Powered prostheses are controlled using electromyographic (EMG) signals, which may introduce high levels of uncertainty even for simple tasks. According to Bayesian theories, higher uncertainty should influence how the brain adapts motor commands in response to perceived errors. Such adaptation may critically influence how patients interact with their prosthetic devices; however, we do not yet understand adaptation behavior with EMG control. Models of adaptation can offer insights on movement planning and feedback correction, but we first need to establish their validity for EMG control interfaces. Here we created a simplified comparison of prosthesis and able-bodied control by studying adaptation with three control interfaces: joint angle, joint torque, and EMG. Subjects used each of the control interfaces to perform a target-directed task with random visual perturbations. We investigated how control interface and visual uncertainty affected trial-by-trial adaptation. As predicted by Bayesian models, increased errors and decreased visual uncertainty led to faster adaptation. The control interface had no significant effect beyond influencing error sizes. This result suggests that Bayesian models are useful for describing prosthesis control and could facilitate further investigation to characterize the uncertainty faced by prosthesis users. A better understanding of factors affecting movement uncertainty will guide sensory feedback strategies for powered prostheses and clarify what feedback information best improves control.

Comparison of body-powered voluntary opening and voluntary closing prehensor for activities of daily life.

Persons with an upper-limb amputation who use a body-powered prosthesis typically control the prehensor through contralateral shoulder movement, which is transmitted through a Bowden cable. Increased cable tension either opens or closes the prehensor; when tension is released, some passive element, such as a spring, returns the prehensor to the default state (closed or open). In this study, we used the Southampton Hand Assessment Procedure to examine functional differences between these two types of prehensors in 29 nondisabled subjects (who used a body-powered bypass prosthesis) and 2 persons with unilateral transradial amputations (who used a conventional body-powered device). We also administered a survey to determine whether subjects preferred one prehensor or the other for specific tasks, with a long-term goal of assessing whether a prehensor that could switch between both modes would be advantageous. We found that using the voluntary closing prehensor was 1.3 s faster (p = 0.02) than using the voluntary opening prehensor, across tasks, and that there was consensus among subjects on which types of tasks they preferred to do with each prehensor type. Twenty-five subjects wanted a device that could switch between the two modes in order to perform particular tasks.

Comparing functional EMG characteristics between zero-order and first-order interface dynamics.

The optimal control scheme for powered prostheses can be determined using simulation experiments, for which an accurate model of prosthesis control is essential. This paper focuses on electromyographic (EMG) control signal characteristics across two different control schemes. We constructed a functional EMG model comprising three EMG signal characteristics-standard deviation, kurtosis, and median power frequency-using data collected under realistic conditions for prosthesis control (closed-loop, dynamic, anisometric contractions). We examined how the model changed when subjects used zero-order or first-order control. Control order had a statistically significant effect on EMG characteristics, but the effect size was small and generally did not exceed inter-subject variability. Therefore, we suggest that this functional EMG model remains valid across different control schemes.

Evidence for a time-invariant phase variable in human ankle control.

Human locomotion is a rhythmic task in which patterns of muscle activity are modulated by state-dependent feedback to accommodate perturbations. Two popular theories have been proposed for the underlying embodiment of phase in the human pattern generator: a time-dependent internal representation or a time-invariant feedback representation (i.e., reflex mechanisms). In either case the neuromuscular system must update or represent the phase of locomotor patterns based on the system state, which can include measurements of hundreds of variables. However, a much simpler representation of phase has emerged in recent designs for legged robots, which control joint patterns as functions of a single monotonic mechanical variable, termed a phase variable. We propose that human joint patterns may similarly depend on a physical phase variable, specifically the heel-to-toe movement of the Center of Pressure under the foot. We found that when the ankle is unexpectedly rotated to a position it would have encountered later in the step, the Center of Pressure also shifts forward to the corresponding later position, and the remaining portion of the gait pattern ensues. This phase shift suggests that the progression of the stance ankle is controlled by a biomechanical phase variable, motivating future investigations of phase variables in human locomotor control.

Towards Biomimetic Virtual Constraint Control of a Powered Prosthetic Leg.

This brief presents a novel control strategy for a powered prosthetic ankle based on a biomimetic virtual constraint. We first derive a kinematic constraint for the "effective shape" of the human ankle-foot complex during locomotion. This shape characterizes ankle motion as a function of the Center of Pressure (COP)-the point on the foot sole where the resultant ground reaction force is imparted. Since the COP moves monotonically from heel to toe during steady walking, we adopt the COP as a mechanical representation of the gait cycle phase in an autonomous feedback controller. We show that our kinematic constraint can be enforced as a virtual constraint by an output linearizing controller that uses only feedback available to sensors onboard a prosthetic leg. Using simulations of a passive walking model with feet, we show that this novel controller exactly enforces the desired effective shape whereas a standard impedance (i.e., proportional-derivative) controller cannot. This work provides a single, biomimetic control law for the entire single-support period during robot-assisted locomotion.