Improved prosthetic hand control with concurrent use of myoelectric and inertial measurements.

BACKGROUND: Myoelectric pattern recognition systems can decode movement intention to drive upper-limb prostheses. Despite recent advances in academic research, the commercial adoption of such systems remains low. This limitation is mainly due to the lack of classification robustness and a simultaneous requirement for a large number of electromyogram (EMG) electrodes. We propose to address these two issues by using a multi-modal approach which combines surface electromyography (sEMG) with inertial measurements (IMs) and an appropriate training data collection paradigm. We demonstrate that this can significantly improve classification performance as compared to conventional techniques exclusively based on sEMG signals. METHODS: We collected and analyzed a large dataset comprising recordings with 20 able-bodied and two amputee participants executing 40 movements. Additionally, we conducted a novel real-time prosthetic hand control experiment with 11 able-bodied subjects and an amputee by using a state-of-the-art commercial prosthetic hand. A systematic performance comparison was carried out to investigate the potential benefit of incorporating IMs in prosthetic hand control. RESULTS: The inclusion of IM data improved performance significantly, by increasing classification accuracy (CA) in the offline analysis and improving completion rates (CRs) in the real-time experiment. Our findings were consistent across able-bodied and amputee subjects. Integrating the sEMG electrodes and IM sensors within a single sensor package enabled us to achieve high-level performance by using on average 4-6 sensors. CONCLUSIONS: The results from our experiments suggest that IMs can form an excellent complimentary source signal for upper-limb myoelectric prostheses. We trust that multi-modal control solutions have the potential of improving the usability of upper-extremity prostheses in real-life applications.

On the origins of suboptimality in human probabilistic inference.

Humans have been shown to combine noisy sensory information with previous experience (priors), in qualitative and sometimes quantitative agreement with the statistically-optimal predictions of Bayesian integration. However, when the prior distribution becomes more complex than a simple Gaussian, such as skewed or bimodal, training takes much longer and performance appears suboptimal. It is unclear whether such suboptimality arises from an imprecise internal representation of the complex prior, or from additional constraints in performing probabilistic computations on complex distributions, even when accurately represented. Here we probe the sources of suboptimality in probabilistic inference using a novel estimation task in which subjects are exposed to an explicitly provided distribution, thereby removing the need to remember the prior. Subjects had to estimate the location of a target given a noisy cue and a visual representation of the prior probability density over locations, which changed on each trial. Different classes of priors were examined (Gaussian, unimodal, bimodal). Subjects' performance was in qualitative agreement with the predictions of Bayesian Decision Theory although generally suboptimal. The degree of suboptimality was modulated by statistical features of the priors but was largely independent of the class of the prior and level of noise in the cue, suggesting that suboptimality in dealing with complex statistical features, such as bimodality, may be due to a problem of acquiring the priors rather than computing with them. We performed a factorial model comparison across a large set of Bayesian observer models to identify additional sources of noise and suboptimality. Our analysis rejects several models of stochastic behavior, including probability matching and sample-averaging strategies. Instead we show that subjects' response variability was mainly driven by a combination of a noisy estimation of the parameters of the priors, and by variability in the decision process, which we represent as a noisy or stochastic posterior.

Internal representations of temporal statistics and feedback calibrate motor-sensory interval timing.

Humans have been shown to adapt to the temporal statistics of timing tasks so as to optimize the accuracy of their responses, in agreement with the predictions of Bayesian integration. This suggests that they build an internal representation of both the experimentally imposed distribution of time intervals (the prior) and of the error (the loss function). The responses of a Bayesian ideal observer depend crucially on these internal representations, which have only been previously studied for simple distributions. To study the nature of these representations we asked subjects to reproduce time intervals drawn from underlying temporal distributions of varying complexity, from uniform to highly skewed or bimodal while also varying the error mapping that determined the performance feedback. Interval reproduction times were affected by both the distribution and feedback, in good agreement with a performance-optimizing Bayesian observer and actor model. Bayesian model comparison highlighted that subjects were integrating the provided feedback and represented the experimental distribution with a smoothed approximation. A nonparametric reconstruction of the subjective priors from the data shows that they are generally in agreement with the true distributions up to third-order moments, but with systematically heavier tails. In particular, higher-order statistical features (kurtosis, multimodality) seem much harder to acquire. Our findings suggest that humans have only minor constraints on learning lower-order statistical properties of unimodal (including peaked and skewed) distributions of time intervals under the guidance of corrective feedback, and that their behavior is well explained by Bayesian decision theory.

Dynamic obstacle avoidance using Bayesian Occupancy Filter and approximate inference.

The goal of this paper is to solve the problem of dynamic obstacle avoidance for a mobile platform using the stochastic optimal control framework to compute paths that are optimal in terms of safety and energy efficiency under constraints. We propose a three-dimensional extension of the Bayesian Occupancy Filter (BOF) (Coué et al. Int. J. Rob. Res. 2006, 25, 19-30) to deal with the noise in the sensor data, improving the perception stage. We reduce the computational cost of the perception stage by estimating the velocity of each obstacle using optical flow tracking and blob filtering. While several obstacle avoidance systems have been presented in the literature addressing safety and optimality of the robot motion separately, we have applied the approximate inference framework to this problem to combine multiple goals, constraints and priors in a structured way. It is important to remark that the problem involves obstacles that can be moving, therefore classical techniques based on reactive control are not optimal from the point of view of energy consumption. Some experimental results, including comparisons against classical algorithms that highlight the advantages, are presented.

Adaptive assistive robotics: a framework for triadic collaboration between humans and robots.

Robots and other assistive technologies have a huge potential to help society in domains ranging from factory work to healthcare. However, safe and effective control of robotic agents in these environments is complex, especially when it involves close interactions and multiple actors. We propose an effective framework for optimizing the behaviour of robots and complementary assistive technologies in systems comprising a mix of human and technological agents with numerous high-level goals. The framework uses a combination of detailed biomechanical modelling and weighted multi-objective optimization to allow for the fine tuning of robot behaviours depending on the specification of the task at hand. We illustrate our framework via two case studies across assisted living and rehabilitation scenarios, and conduct simulations and experiments of triadic collaboration in practice. Our results indicate a marked benefit to the triadic approach, showing the potential to improve outcome measures for human agents in robot-assisted tasks.

A Haptic Sleeve as a Method of Mechanotactile Feedback Restoration for Myoelectric Hand Prosthesis Users.

Current myoelectric upper limb prostheses do not restore sensory feedback, impairing fine motor control. Mechanotactile feedback restoration with a haptic sleeve may rectify this problem. This randomised crossover within-participant controlled study aimed to assess a prototype haptic sleeve's effect on routine grasping tasks performed by eight able-bodied participants. Each participant completed 15 repetitions of the three tasks: Task 1-normal grasp, Task 2-strong grasp and Task 3-weak grasp, using visual, haptic, or combined feedback All data were collected in April 2021 in the Scottish Microelectronics Centre, Edinburgh, UK. Combined feedback correlated with significantly higher grasp success rates compared to the vision alone in Task 1 (p < 0.0001), Task 2 (p = 0.0057), and Task 3 (p = 0.0170). Similarly, haptic feedback was associated with significantly higher grasp success rates compared to vision in Task 1 (p < 0.0001) and Task 2 (p = 0.0015). Combined feedback correlated with significantly lower energy expenditure compared to visual feedback in Task 1 (p < 0.0001) and Task 3 (p = 0.0003). Likewise, haptic feedback was associated with significantly lower energy expenditure compared to the visual feedback in Task 1 (p < 0.0001), Task 2 (p < 0.0001), and Task 3 (p < 0.0001). These results suggest that mechanotactile feedback provided by the haptic sleeve effectively augments grasping and reduces its energy expenditure.

The role of feed-forward and feedback processes for closed-loop prosthesis control.

BACKGROUND: It is widely believed that both feed-forward and feed-back mechanisms are required for successful object manipulation. Open-loop upper-limb prosthesis wearers receive no tactile feedback, which may be the cause of their limited dexterity and compromised grip force control. In this paper we ask whether observed prosthesis control impairments are due to lack of feedback or due to inadequate feed-forward control. METHODS: Healthy subjects were fitted with a closed-loop robotic hand and instructed to grasp and lift objects of different weights as we recorded trajectories and force profiles. We conducted three experiments under different feed-forward and feed-back configurations to elucidate the role of tactile feedback (i) in ideal conditions, (ii) under sensory deprivation, and (iii) under feed-forward uncertainty. RESULTS: (i) We found that subjects formed economical grasps in ideal conditions. (ii) To our surprise, this ability was preserved even when visual and tactile feedback were removed. (iii) When we introduced uncertainty into the hand controller performance degraded significantly in the absence of either visual or tactile feedback. Greatest performance was achieved when both sources of feedback were present. CONCLUSIONS: We have introduced a novel method to understand the cognitive processes underlying grasping and lifting. We have shown quantitatively that tactile feedback can significantly improve performance in the presence of feed-forward uncertainty. However, our results indicate that feed-forward and feed-back mechanisms serve complementary roles, suggesting that to improve on the state-of-the-art in prosthetic hands we must develop prostheses that empower users to correct for the inevitable uncertainty in their feed-forward control.

HapFIC: An Adaptive Force/Position Controller for Safe Environment Interaction in Articulated Systems.

Haptic interaction is essential for the dynamic dexterity of animals, which seamlessly switch from an impedance to an admittance behaviour using the force feedback from their proprioception. However, this ability is extremely challenging to reproduce in robots, especially when dealing with complex interaction dynamics, distributed contacts, and contact switching. Current model-based controllers require accurate interaction modelling to account for contacts and stabilise the interaction. In this manuscript, we propose an adaptive force/position controller that exploits the fractal impedance controller's passivity and non-linearity to execute a finite search algorithm using the force feedback signal from the sensor at the end-effector. The method is computationally inexpensive, opening the possibility to deal with distributed contacts in the future. We evaluated the architecture in physics simulation and showed that the controller can robustly control the interaction with objects of different dynamics without violating the maximum allowable target forces or causing numerical instability even for very rigid objects. The proposed controller can also autonomously deal with contact switching and may find application in multiple fields such as legged locomotion, rehabilitation and assistive robotics.

Information about complex fingertip parameters in individual human tactile afferent neurons.

Although information in tactile afferent neurons represented by firing rates has been studied extensively over nearly a century, recent studies suggest that precise spike timing might be more important than firing rates. Here, we used information theory to compare the information content in the discharges of 92 tactile afferents distributed over the entire terminal segment of the fingertip when it was contacted by surfaces with different curvatures and force directions representative of everyday manipulations. Estimates of the information content with regard to curvature and force direction based on the precise timing of spikes were at least 2.2 times and 1.6 times, respectively, larger than that of spike counts during a 125 ms period of force increase. Moreover, the information regarding force direction based on the timing of the very first elicited spike was comparable with that provided by spike counts and more than twice as large with respect to object shape. For all encoding schemes, afferents terminating close to the stimulation site tended to convey more information about surface curvature than more remote afferents that tended to convey more information about force direction. Finally, coding schemes based on spike timing and spike counts overall contributed mostly independent information. We conclude that information about tactile stimuli in timing of spikes in primary afferents, even if limited to the first spikes, surpasses that contained in firing rates and that these measures of afferents' responses might capture different aspects of the stimulus.

Efficient learning and feature selection in high-dimensional regression.

We present a novel algorithm for efficient learning and feature selection in high-dimensional regression problems. We arrive at this model through a modification of the standard regression model, enabling us to derive a probabilistic version of the well-known statistical regression technique of backfitting. Using the expectation-maximization algorithm, along with variational approximation methods to overcome intractability, we extend our algorithm to include automatic relevance detection of the input features. This variational Bayesian least squares (VBLS) approach retains its simplicity as a linear model, but offers a novel statistically robust black-box approach to generalized linear regression with high-dimensional inputs. It can be easily extended to nonlinear regression and classification problems. In particular, we derive the framework of sparse Bayesian learning, the relevance vector machine, with VBLS at its core, offering significant computational and robustness advantages for this class of methods. The iterative nature of VBLS makes it most suitable for real-time incremental learning, which is crucial especially in the application domain of robotics, brain-machine interfaces, and neural prosthetics, where real-time learning of models for control is needed. We evaluate our algorithm on synthetic and neurophysiological data sets, as well as on standard regression and classification benchmark data sets, comparing it with other competitive statistical approaches and demonstrating its suitability as a drop-in replacement for other generalized linear regression techniques.

Multi-Grip Classification-Based Prosthesis Control With Two EMG-IMU Sensors.

In the field of upper-limb myoelectric prosthesis control, the use of statistical and machine learning methods has been long proposed as a means of enabling intuitive grip selection and actuation. Recently, this paradigm has found its way toward commercial adoption. Machine learning-based prosthesis control typically relies on the use of a large number of electrodes. Here, we propose an end-to-end strategy for multi-grip, classification-based prosthesis control using only two sensors, comprising electromyography (EMG) electrodes and inertial measurement units (IMUs). We emphasize the importance of accurately estimating posterior class probabilities and rejecting predictions made with low confidence, so as to minimize the rate of unintended prosthesis activations. To that end, we propose a confidence-based error rejection strategy using grip-specific thresholds. We evaluate the efficacy of the proposed system with real-time pick and place experiments using a commercial multi-articulated prosthetic hand and involving 12 able-bodied and two transradial (i.e., below-elbow) amputee participants. Results promise the potential for deploying intuitive, classification-based multi-grip control in existing upper-limb prosthetic systems subject to small modifications.

Implications of different classes of sensorimotor disturbance for cerebellar-based motor learning models.

The exact role of the cerebellum in motor control and learning is not yet fully understood. The structure, connectivity and plasticity within cerebellar cortex has been extensively studied, but the patterns of connectivity and interaction with other brain structures, and the computational significance of these patterns, is less well known and a matter of debate. Two contrasting models of the role of the cerebellum in motor adaptation have previously been proposed. Most commonly, the cerebellum is employed in a purely feedforward pathway, with its output contributing directly to the outgoing motor command. The cerebellum must then learn an inverse model of the motor apparatus in order to achieve accurate control. More recently, Porrill et al. (Proc Biol Sci 271(1541):789-796, 2004) and Porrill et al. (PLoS Comput Biol 3:1935-1950, 2007a) and Porrill et al. (Neural Comput 19(1), 170-193, 2007b) have highlighted the potential importance of these recurrent connections by proposing an alternative architecture in which the cerebellum is embedded in a recurrent loop with brainstem control circuitry. In this framework, the feedforward connections are not necessary at all. The cerebellum must learn a forward model of the motor apparatus for accurate motor commands to be generated. We show here how these two models exhibit contrasting yet complimentary learning capabilities. Central to the differences in performance between architectures is that there are two distinct kinds of disturbance to which a motor system may need to adapt (1) changes in the relationship between the motor command and the observed outcome and (2) changes in the relationship between the stimulus and the desired outcome. The computational distinction between these two kinds of transformation is subtle and has therefore often been overlooked. However, the implications for learning turn out to be significant: learning with a feedforward architecture is robust following changes in the stimulus-desired outcome mapping but not necessarily the motor command-outcome mapping, while learning with a recurrent architecture is robust under changes in the motor command-outcome mapping but not necessarily the stimulus-desired outcome mapping. We first analyse these differences theoretically and through simulations in the vestibulo-ocular reflex (VOR), then illustrate how these same concepts apply more generally with a model of reaching movements.

Analytic Model for Quadruped Locomotion Task-Space Planning.

Despite the extensive presence of the legged locomotion in animals, it is extremely challenging to be reproduced with robots. Legged locomotion is an dynamic task which benefits from a planning that takes advantage of the gravitational pull on the system. However, the computational cost of such optimization rapidly increases with the complexity of kinematic structures, rendering impossible real-time deployment in unstructured environments. This paper proposes a simplified method that can generate desired centre of mass and feet trajectory for quadrupeds. The model describes a quadruped as two bipeds connected via their centres of mass, and it is based on the extension of an algebraic bipedal model that uses the topology of the gravitational attractor to describe bipedal locomotion strategies. The results show that the model generates trajectories that agrees with previous studies. The model will be deployed in the future as seed solution for whole-body trajectory optimization in the attempt to reduce the computational cost and obtain real-time planning of complex action in challenging environments.

Effect of User Practice on Prosthetic Finger Control With an Intuitive Myoelectric Decoder.

Machine learning-based myoelectric control is regarded as an intuitive paradigm, because of the mapping it creates between muscle co-activation patterns and prosthesis movements that aims to simulate the physiological pathways found in the human arm. Despite that, there has been evidence that closed-loop interaction with a classification-based interface results in user adaptation, which leads to performance improvement with experience. Recently, there has been a focus shift toward continuous prosthesis control, yet little is known about whether and how user adaptation affects myoelectric control performance in dexterous, intuitive tasks. We investigate the effect of short-term adaptation with independent finger position control by conducting real-time experiments with 10 able-bodied and two transradial amputee subjects. We demonstrate that despite using an intuitive decoder, experience leads to significant improvements in performance. We argue that this is due to the lack of an utterly natural control scheme, which is mainly caused by differences in the anatomy of human and artificial hands, movement intent decoding inaccuracies, and lack of proprioception. Finally, we extend previous work in classification-based and wrist continuous control by verifying that offline analyses cannot reliably predict real-time performance, thereby reiterating the importance of validating myoelectric control algorithms with real-time experiments.

Bio-inspired design and validation of the Efficient Lockable Spring Ankle (ELSA) prosthesis.

Over the last decade, active lower-limb prostheses demonstrated their ability to restore a physiological gait for transfemoral amputees by supplying the required positive energy balance during daily life locomotion activities. However, the added-value of such devices is significantly impacted by their limited energetic autonomy, excessive weight and cost, thus preventing their full appropriation by the users. There is thus a strong incentive to produce active yet affordable, lightweight and energy efficient devices. To address these issues, we developed the ELSA (Efficient Lockable Spring Ankle) prosthesis embedding both a lockable parallel spring and a series elastic actuator, tailored to the walking dynamics of a sound ankle. The first contribution of this paper concerns the developement of a bio-inspired, lightweight and stiffness-adjustable parallel spring, comprising an energy efficient ratchet and pawl mechanism with servo actuation. The second contribution is the addition of a complementary rope-driven series elastic actuator to generate the active push-off. The system produces a sound ankle torque pattern during flat ground walking. Up to 50% of the peak torque is generated passively at a negligible energetic cost (0.1 J/stride). By design, the total system is lightweight (1.2kg) and low cost.

Structure inference for Bayesian multisensory scene understanding.

We investigate a solution to the problem of multi-sensor scene understanding by formulating it in the framework of Bayesian model selection and structure inference. Humans robustly associate multimodal data as appropriate, but previous modelling work has focused largely on optimal fusion, leaving segregation unaccounted for and unexploited by machine perception systems. We illustrate a unifying, Bayesian solution to multi-sensor perception and tracking which accounts for both integration and segregation by explicit probabilistic reasoning about data association in a temporal context. Such explicit inference of multimodal data association is also of intrinsic interest for higher level understanding of multisensory data. We illustrate this using a probabilistic implementation of data association in a multi-party audio-visual scenario, where unsupervised learning and structure inference is used to automatically segment, associate and track individual subjects in audiovisual sequences. Indeed, the structure inference based framework introduced in this work provides the theoretical foundation needed to satisfactorily explain many confounding results in human psychophysics experiments involving multimodal cue integration and association.

Continuous Versus Discrete Simultaneous Control of Prosthetic Fingers.

Modern, commercially available hand prostheses offer the potential of individual digit control. However, this feature is often not utilized due to the lack of a robust scheme for finger motion estimation from surface electromyographic (EMG) measurements. Regression methods have been proposed to achieve closed-loop finger position, velocity, or force control. In this paper, we propose an alternative approach, based on open-loop action-based control, which could be achieved through simultaneous finger motion classification. We compare the efficacy of continuous closed-loop and discrete open-loop control on the task of controlling the five degrees of actuation (DOAs) of a dexterous robotic hand. Eight normally-limbed subjects were instructed to teleoperate the hand using a data glove and the two control schemes under investigation in order to match target postures presented to them on a screen as closely as possible. Results indicate that, firstly, the performance of the two control methods is comparable and, secondly, that experience can lead to significant performance improvement over time, regardless of the method used. These results suggest that prosthetic finger control in a continuous space can be potentially achieved by means of myoelectric classification and discrete, action-based control and hence encourage further research in this direction.

Causes of Performance Degradation in Non-invasive Electromyographic Pattern Recognition in Upper Limb Prostheses.

Surface Electromyography (EMG)-based pattern recognition methods have been investigated over the past years as a means of controlling upper limb prostheses. Despite the very good reported performance of myoelectric controlled prosthetic hands in lab conditions, real-time performance in everyday life conditions is not as robust and reliable, explaining the limited clinical use of pattern recognition control. The main reason behind the instability of myoelectric pattern recognition control is that EMG signals are non-stationary in real-life environments and present a lot of variability over time and across subjects, hence affecting the system's performance. This can be the result of one or many combined changes, such as muscle fatigue, electrode displacement, difference in arm posture, user adaptation on the device over time and inter-subject singularity. In this paper an extensive literature review is performed to present the causes of the drift of EMG signals, ways of detecting them and possible techniques to counteract for their effects in the application of upper limb prostheses. The suggested techniques are organized in a table that can be used to recognize possible problems in the clinical application of EMG-based pattern recognition methods for upper limb prosthesis applications and state-of-the-art methods to deal with such problems.

Effectively Quantifying the Performance of Lower-Limb Exoskeletons Over a Range of Walking Conditions.

Exoskeletons and other wearable robotic devices have a wide range of potential applications, including assisting patients with walking pathologies, acting as tools for rehabilitation, and enhancing the capabilities of healthy humans. However, applying these devices effectively in a real-world setting can be challenging, as the optimal design features and control commands for an exoskeleton are highly dependent on the current user, task and environment. Consequently, robust metrics and methods for quantifying exoskeleton performance are required. This work presents an analysis of walking data collected for healthy subjects walking with an active pelvis exoskeleton over three assistance scenarios and five walking contexts. Spatial and temporal, kinematic, kinetic and other novel dynamic gait metrics were compared to identify which metrics exhibit desirable invariance properties, and so are good candidates for use as a stability metric over varying walking conditions. Additionally, using a model-based approach, the average metabolic power consumption was calculated for a subset of muscles crossing the hip, knee and ankle joints, and used to analyse how the energy-reducing properties of an exoskeleton are affected by changes in walking context. The results demonstrated that medio-lateral centre of pressure displacement and medio-lateral margin of stability exhibit strong invariance to changes in walking conditions. This suggests that these dynamic gait metrics are optimised in human gait and are potentially suitable metrics for optimising in an exoskeleton control paradigm. The effectiveness of the exoskeleton at reducing human energy expenditure was observed to increase when walking on an incline, where muscles aiding in hip flexion were assisted, but decrease when walking at a slow speed. These results underline the need for adaptive control algorithms for exoskeletons if they are to be used in varied environments.

Target Uncertainty Mediates Sensorimotor Error Correction.

Human movements are prone to errors that arise from inaccuracies in both our perceptual processing and execution of motor commands. We can reduce such errors by both improving our estimates of the state of the world and through online error correction of the ongoing action. Two prominent frameworks that explain how humans solve these problems are Bayesian estimation and stochastic optimal feedback control. Here we examine the interaction between estimation and control by asking if uncertainty in estimates affects how subjects correct for errors that may arise during the movement. Unbeknownst to participants, we randomly shifted the visual feedback of their finger position as they reached to indicate the center of mass of an object. Even though participants were given ample time to compensate for this perturbation, they only fully corrected for the induced error on trials with low uncertainty about center of mass, with correction only partial in trials involving more uncertainty. The analysis of subjects' scores revealed that participants corrected for errors just enough to avoid significant decrease in their overall scores, in agreement with the minimal intervention principle of optimal feedback control. We explain this behavior with a term in the loss function that accounts for the additional effort of adjusting one's response. By suggesting that subjects' decision uncertainty, as reflected in their posterior distribution, is a major factor in determining how their sensorimotor system responds to error, our findings support theoretical models in which the decision making and control processes are fully integrated.