Recovery of Distal Arm Movements in Spinal Cord Injured Patients with a Body-Machine Interface: A Proof-of-Concept Study.

BACKGROUND: The recovery of upper limb mobility and functions is essential for people with cervical spinal cord injuries (cSCI) to maximize independence in daily activities and ensure a successful return to normality. The rehabilitative path should include a thorough neuromotor evaluation and personalized treatments aimed at recovering motor functions. Body-machine interfaces (BoMI) have been proven to be capable of harnessing residual joint motions to control objects like computer cursors and virtual or physical wheelchairs and to promote motor recovery. However, their therapeutic application has still been limited to shoulder movements. Here, we expanded the use of BoMI to promote the whole arm's mobility, with a special focus on elbow movements. We also developed an instrumented evaluation test and a set of kinematic indicators for assessing residual abilities and recovery. METHODS: Five inpatient cSCI subjects (four acute, one chronic) participated in a BoMI treatment complementary to their standard rehabilitative routine. The subjects wore a BoMI with sensors placed on both proximal and distal arm districts and practiced for 5 weeks. The BoMI was programmed to promote symmetry between right and left arms use and the forearms' mobility while playing games. To evaluate the effectiveness of the treatment, the subjects' kinematics were recorded while performing an evaluation test that involved functional bilateral arms movements, before, at the end, and three months after training. RESULTS: At the end of the training, all subjects learned to efficiently use the interface despite being compelled by it to engage their most impaired movements. The subjects completed the training with bilateral symmetry in body recruitment, already present at the end of the familiarization, and they increased the forearm activity. The instrumental evaluation confirmed this. The elbow motion's angular amplitude improved for all subjects, and other kinematic parameters showed a trend towards the normality range. CONCLUSION: The outcomes are preliminary evidence supporting the efficacy of the proposed BoMI as a rehabilitation tool to be considered for clinical practice. It also suggests an instrumental evaluation protocol and a set of indicators to assess and evaluate motor impairment and recovery in cSCI.

The dynamics of motor learning through the formation of internal models.

A medical student learning to perform a laparoscopic procedure or a recently paralyzed user of a powered wheelchair must learn to operate machinery via interfaces that translate their actions into commands for an external device. Since the user's actions are selected from a number of alternatives that would result in the same effect in the control space of the external device, learning to use such interfaces involves dealing with redundancy. Subjects need to learn an externally chosen many-to-one map that transforms their actions into device commands. Mathematically, we describe this type of learning as a deterministic dynamical process, whose state is the evolving forward and inverse internal models of the interface. The forward model predicts the outcomes of actions, while the inverse model generates actions designed to attain desired outcomes. Both the mathematical analysis of the proposed model of learning dynamics and the learning performance observed in a group of subjects demonstrate a first-order exponential convergence of the learning process toward a particular state that depends only on the initial state of the inverse and forward models and on the sequence of targets supplied to the users. Noise is not only present but necessary for the convergence of learning through the minimization of the difference between actual and predicted outcomes.

Guiding functional reorganization of motor redundancy using a body-machine interface.

BACKGROUND: Body-machine interfaces map movements onto commands to external devices. Redundant motion signals derived from inertial sensors are mapped onto lower-dimensional device commands. Then, the device users face two problems, a) the structural problem of understanding the operation of the interface and b) the performance problem of controlling the external device with high efficiency. We hypothesize that these problems, while being distinct are connected in that aligning the space of body movements with the space encoded by the interface, i.e. solving the structural problem, facilitates redundancy resolution towards increasing efficiency, i.e. solving the performance problem. METHODS: Twenty unimpaired volunteers practiced controlling the movement of a computer cursor by moving their arms. Eight signals from four inertial sensors were mapped onto the two cursor's coordinates on a screen. The mapping matrix was initialized by asking each user to perform free-form spontaneous upper-limb motions and deriving the two main principal components of the motion signals. Participants engaged in a reaching task for 18 min, followed by a tracking task. One group of 10 participants practiced with the same mapping throughout the experiment, while the other 10 with an adaptive mapping that was iteratively updated by recalculating the principal components based on ongoing movements. RESULTS: Participants quickly reduced reaching time while also learning to distribute most movement variance over two dimensions. Participants with the fixed mapping distributed movement variance over a subspace that did not match the potent subspace defined by the interface map. In contrast, participant with the adaptive map reduced the difference between the two subspaces, resulting in a smaller amount of arm motions distributed over the null space of the interface map. This, in turn, enhanced movement efficiency without impairing generalization from reaching to tracking. CONCLUSIONS: Aligning the potent subspace encoded by the interface map to the user's movement subspace guides redundancy resolution towards increasing movement efficiency, with implications for controlling assistive devices. In contrast, in the pursuit of rehabilitative goals, results would suggest that the interface must change to drive the statistics of user's motions away from the established pattern and toward the engagement of movements to be recovered. TRIAL REGISTRATION: ClinicalTrials.gov, NCT01608438, Registered 16 April 2012.

Effects of visuomotor delays on the control of movement and on perceptual localization in the presence and absence of visual targets.

The sensory system constantly deals with delayed feedback. Recent studies showed that playing a virtual game of pong with delayed feedback caused hypermetric reaching movements. We investigated whether this effect is associated with a perceptual bias. In addition, we examined the importance of the target in causing hypermetric movements. In a first experiment, participants played a delayed pong game and blindly reached to presented targets. Following each reaching movement, they assessed the position of the invisible cursor. We found that participants performed hypermetric movements but reported that the invisible cursor reached the target, suggesting that they were unaware of the hypermetria and that their perception was biased toward the target rather than toward their hand position. In a second experiment, we removed the visual target, and strikingly, the hypermetria vanished. Moreover, participants reported that the invisible cursor was located with their hand. Taking these results together, we conclude that the adaptation to the visuomotor delay during the pong game selectively affected the execution of goal directed movements, resulting in hypermetria and perceptual bias when movements are directed toward visual targets but not when such targets are absent.NEW & NOTEWORTHY Recent studies showed that adaptation to visuomotor delays causes hypermetric movements in the absence of visual feedback, suggesting that visuomotor delay is represented using current state information. We report that this adaptation also affects perception. Importantly, both the motor and perceptual effects are selective to the representations that are used in the execution of goal-directed movements toward visual targets.

Using noise to shape motor learning.

Each of our movements is selected from any number of alternative movements. Some studies have shown evidence that the central nervous system (CNS) chooses to make the specific movements that are least affected by motor noise. Previous results showing that the CNS has a natural tendency to minimize the effects of noise make the direct prediction that if the relationship between movements and noise were to change, the specific movements people learn to make would also change in a predictable manner. Indeed, this has been shown for well-practiced movements such as reaching. Here, we artificially manipulated the relationship between movements and visuomotor noise by adding noise to a motor task in a novel redundant geometry such that there arose a single control policy that minimized the noise. This allowed us to see whether, for a novel motor task, people could learn the specific control policy that minimized noise or would need to employ other compensation strategies to overcome the added noise. As predicted, subjects were able to learn movements that were biased toward the specific ones that minimized the noise, suggesting not only that the CNS can learn to minimize the effects of noise in a novel motor task but also that artificial visuomotor noise can be a useful tool for teaching people to make specific movements. Using noise as a teaching signal promises to be useful for rehabilitative therapies and movement training with human-machine interfaces. NEW & NOTEWORTHY: Many theories argue that we choose to make the specific movements that minimize motor noise. Here, by changing the relationship between movements and noise, we show that people actively learn to make movements that minimize noise. This not only provides direct evidence for the theories of noise minimization but presents a way to use noise to teach specific movements to improve rehabilitative therapies and human-machine interface control.

Representing delayed force feedback as a combination of current and delayed states.

To adapt to deterministic force perturbations that depend on the current state of the hand, internal representations are formed to capture the relationships between forces experienced and motion. However, information from multiple modalities travels at different rates, resulting in intermodal delays that require compensation for these internal representations to develop. To understand how these delays are represented by the brain, we presented participants with delayed velocity-dependent force fields, i.e., forces that depend on hand velocity either 70 or 100 ms beforehand. We probed the internal representation of these delayed forces by examining the forces the participants applied to cope with the perturbations. The findings showed that for both delayed forces, the best model of internal representation consisted of a delayed velocity and current position and velocity. We show that participants relied initially on the current state, but with adaptation, the contribution of the delayed representation to adaptation increased. After adaptation, when the participants were asked to make movements with a higher velocity for which they had not previously experienced with the delayed force field, they applied forces that were consistent with current position and velocity as well as delayed velocity representations. This suggests that the sensorimotor system represents delayed force feedback using current and delayed state information and that it uses this representation when generalizing to faster movements.NEW & NOTEWORTHY The brain compensates for forces in the body and the environment to control movements, but it is unclear how it does so given the inherent delays in information transmission and processing. We examined how participants cope with delayed forces that depend on their arm velocity 70 or 100 ms beforehand. After adaptation, participants applied opposing forces that revealed a partially correct representation of the perturbation using the current and the delayed information.

Sensory Agreement Guides Kinetic Energy Optimization of Arm Movements during Object Manipulation.

The laws of physics establish the energetic efficiency of our movements. In some cases, like locomotion, the mechanics of the body dominate in determining the energetically optimal course of action. In other tasks, such as manipulation, energetic costs depend critically upon the variable properties of objects in the environment. Can the brain identify and follow energy-optimal motions when these motions require moving along unfamiliar trajectories? What feedback information is required for such optimal behavior to occur? To answer these questions, we asked participants to move their dominant hand between different positions while holding a virtual mechanical system with complex dynamics (a planar double pendulum). In this task, trajectories of minimum kinetic energy were along curvilinear paths. Our findings demonstrate that participants were capable of finding the energy-optimal paths, but only when provided with veridical visual and haptic information pertaining to the object, lacking which the trajectories were executed along rectilinear paths.

Learning redundant motor tasks with and without overlapping dimensions: facilitation and interference effects.

Prior learning of a motor skill creates motor memories that can facilitate or interfere with learning of new, but related, motor skills. One hypothesis of motor learning posits that for a sensorimotor task with redundant degrees of freedom, the nervous system learns the geometric structure of the task and improves performance by selectively operating within that task space. We tested this hypothesis by examining if transfer of learning between two tasks depends on shared dimensionality between their respective task spaces. Human participants wore a data glove and learned to manipulate a computer cursor by moving their fingers. Separate groups of participants learned two tasks: a prior task that was unique to each group and a criterion task that was common to all groups. We manipulated the mapping between finger motions and cursor positions in the prior task to define task spaces that either shared or did not share the task space dimensions (x-y axes) of the criterion task. We found that if the prior task shared task dimensions with the criterion task, there was an initial facilitation in criterion task performance. However, if the prior task did not share task dimensions with the criterion task, there was prolonged interference in learning the criterion task due to participants finding inefficient task solutions. These results show that the nervous system learns the task space through practice, and that the degree of shared task space dimensionality influences the extent to which prior experience transfers to subsequent learning of related motor skills.

Adaptation to visual feedback delay in a redundant motor task.

The goal of this study was to examine the reorganization of hand movements during adaptation to delayed visual feedback in a novel and redundant environment. In most natural behaviors, the brain must learn to invert a many-to-one map from high-dimensional joint movements and muscle forces to a low-dimensional goal. This spatial "inverse map" is learned by associating motor commands to their low-dimensional consequences. How is this map affected by the presence of temporal delays? A delay presents the brain with a new set of kinematic data, and, because of redundancy, the brain may use these data to form a new inverse map. We consider two possible responses to a novel visuomotor delay. In one case, the brain updates the previously learned spatial map, building a new association between motor commands and visual feedback of their effects. In the alternative case, the brain preserves the original map and learns to compensate the delay by a temporal shift of the motor commands. To test these alternative possibilities, we developed a virtual reality game in which subjects controlled the two-dimensional coordinates of a cursor by continuous hand gestures. Two groups of subjects tracked a target along predictable paths by wearing an instrumented data glove that recorded finger motions. The 19-dimensional glove signals controlled a cursor on a 2-dimensional computer display. The experiment was performed on 2 consecutive days. On the 1st day, subjects practiced tracking movements without delay. On the 2nd day, the test group performed the same task with a delay of 300 ms between the glove signals and the cursor display, whereas the control group continued practicing the nondelayed trials. We found evidence that to compensate for the delay, the test group relied on the coordination patterns established during the baseline, e.g., their hand-to-cursor inverse map was robust to the delay perturbation, which was counteracted by an anticipation of the motor command.

The coordinate system for force control.

The primary objective of this study was to establish the coordinate frame for force control by observing how parameters of force that are not explicitly specified by a motor task vary across the workspace. We asked subjects to apply a force of a specific magnitude with their hand. Subjects could complete the task by applying forces in any direction of their choice in the transverse plane. They were tested with the arm in seven different configurations. To estimate whether contact forces are represented in extrinsic or intrinsic coordinates, we applied the parallel transport method of differential geometry to the net joint torques applied during the task. This approach allowed us to compare the force variability observed at different arm configurations with the force variability that would be expected if the control system were applying an invariant pattern of joint torques at the tested configurations. The results indicate that for the majority of the subjects, the predominant pattern was consistent with an invariant representation in joint coordinates. However, two out of eleven subjects also demonstrated a preference for extrinsic representation. These findings suggest that the central nervous system can represent contact forces in both coordinate frames, with a prevalence toward intrinsic representations.

Learning to be lazy: exploiting redundancy in a novel task to minimize movement-related effort.

A key issue in motor control is to understand how the motor system chooses a solution from the multiple solutions that exist to achieve any particular task goal. One hypothesis is that redundancy may be resolved by minimizing movement-related costs. However, testing this prediction in motor learning has been problematic in simple laboratory tasks, like reaching, because the motor system already has extensive prior knowledge about redundancy in these tasks. Here, we used a novel task where healthy human participants performed finger movements to guide a computer cursor to different targets on the screen. Through training, all participants learned to perform successful goal-directed movements. Our findings showed that subjects did not develop a single inverse map from target to hand posture. Instead, they learned to use distinct hand postures to get to a single target, using a strategy in which the final hand posture at the target depended on the starting hand posture. Furthermore, postures chosen also depended upon the information content of visual feedback, with precise visual feedback resulting in postures that minimized movement-related costs. These results reinforce the idea that redundancy is exploited to minimize movement-related costs and that feedback plays a critical role in modulating this ability to effectively take advantage of the abundance of degrees of freedom.

White matter microstructure changes induced by motor skill learning utilizing a body machine interface.

The purpose of this study is to identify white matter microstructure changes following bilateral upper extremity motor skill training to increase our understanding of learning-induced structural plasticity and enhance clinical strategies in physical rehabilitation. Eleven healthy subjects performed two visuo-spatial motor training tasks over 9 sessions (2-3 sessions per week). Subjects controlled a cursor with bilateral simultaneous movements of the shoulders and upper arms using a body machine interface. Before the start and within 2days of the completion of training, whole brain diffusion tensor MR imaging data were acquired. Motor training increased fractional anisotropy (FA) values in the posterior and anterior limbs of the internal capsule, the corona radiata, and the body of the corpus callosum by 4.19% on average indicating white matter microstructure changes induced by activity-dependent modulation of axon number, axon diameter, or myelin thickness. These changes may underlie the functional reorganization associated with motor skill learning.

The influence of visual motion on motor learning.

How does visual perception shape the way we coordinate movements? Recent studies suggest that the brain organizes movements based on minimizing reaching errors in the presence of motor and sensory noise. We present an alternative hypothesis in which movement trajectories also result from acquired knowledge about the geometrical properties of the object that the brain is controlling. To test this hypothesis, we asked human subjects to control a simulated kinematic linkage by continuous finger motion, a completely novel experience. This paradigm removed all biases arising from influences of limb dynamics and past experience. Subjects were exposed to two different types of visual feedback; some saw the entire simulated linkage and others saw only the moving extremity. Consistent with our hypothesis, subjects learned to move the simulated linkage along geodesic lines corresponding to the geometrical structure of the observed motion. Thus, optimizing final accuracy is not the unique determinant of trajectory formation.

Shaping the dynamics of a bidirectional neural interface.

Progress in decoding neural signals has enabled the development of interfaces that translate cortical brain activities into commands for operating robotic arms and other devices. The electrical stimulation of sensory areas provides a means to create artificial sensory information about the state of a device. Taken together, neural activity recording and microstimulation techniques allow us to embed a portion of the central nervous system within a closed-loop system, whose behavior emerges from the combined dynamical properties of its neural and artificial components. In this study we asked if it is possible to concurrently regulate this bidirectional brain-machine interaction so as to shape a desired dynamical behavior of the combined system. To this end, we followed a well-known biological pathway. In vertebrates, the communications between brain and limb mechanics are mediated by the spinal cord, which combines brain instructions with sensory information and organizes coordinated patterns of muscle forces driving the limbs along dynamically stable trajectories. We report the creation and testing of the first neural interface that emulates this sensory-motor interaction. The interface organizes a bidirectional communication between sensory and motor areas of the brain of anaesthetized rats and an external dynamical object with programmable properties. The system includes (a) a motor interface decoding signals from a motor cortical area, and (b) a sensory interface encoding the state of the external object into electrical stimuli to a somatosensory area. The interactions between brain activities and the state of the external object generate a family of trajectories converging upon a selected equilibrium point from arbitrary starting locations. Thus, the bidirectional interface establishes the possibility to specify not only a particular movement trajectory but an entire family of motions, which includes the prescribed reactions to unexpected perturbations.

Critical damping conditions for third order muscle models: implications for force control.

Experimental results presented in the literature suggest that humans use a position control strategy to indirectly control force rather than direct force control. Modeling the muscle-tendon system as a third-order linear model, we provide an explanation of why an indirect force control strategy is preferred. We analyzed a third-order muscle system and verified that it is required for a faithful representation of muscle-tendon mechanics, especially when investigating critical damping conditions. We provided numerical examples using biomechanical properties of muscles and tendons reported in the literature. We demonstrated that at maximum isotonic contraction, for muscle and tendon stiffness within physiologically compatible ranges, a third-order muscle-tendon system can be under-damped. Over-damping occurs for values of the damping coefficient included within a finite interval defined by two separate critical limits (such interval is a semi-infinite region in second-order models). An increase in damping beyond the larger critical value would lead the system to mechanical instability. We proved the existence of a theoretical threshold for the ratio between tendon and muscle stiffness above which critical damping can never be achieved; thus resulting in an oscillatory free response of the system, independently of the value of the damping. Under such condition, combined with high muscle activation, oscillation of the system can be compensated only by active control.

Autoencoder-based myoelectric controller for prosthetic hands.

In the past, linear dimensionality-reduction techniques, such as Principal Component Analysis, have been used to simplify the myoelectric control of high-dimensional prosthetic hands. Nonetheless, their nonlinear counterparts, such as Autoencoders, have been shown to be more effective at compressing and reconstructing complex hand kinematics data. As a result, they have a potential of being a more accurate tool for prosthetic hand control. Here, we present a novel Autoencoder-based controller, in which the user is able to control a high-dimensional (17D) virtual hand via a low-dimensional (2D) space. We assess the efficacy of the controller via a validation experiment with four unimpaired participants. All the participants were able to significantly decrease the time it took for them to match a target gesture with a virtual hand to an average of 6.9s and three out of four participants significantly improved path efficiency. Our results suggest that the Autoencoder-based controller has the potential to be used to manipulate high-dimensional hand systems via a myoelectric interface with a higher accuracy than PCA; however, more exploration needs to be done on the most effective ways of learning such a controller.

A Non-Linear Body Machine Interface for Controlling Assistive Robotic Arms.

OBJECTIVE: Body machine interfaces (BoMIs) enable individuals with paralysis to achieve a greater measure of independence in daily activities by assisting the control of devices such as robotic manipulators. The first BoMIs relied on Principal Component Analysis (PCA) to extract a lower dimensional control space from information in voluntary movement signals. Despite its widespread use, PCA might not be suited for controlling devices with a large number of degrees of freedom, as because of PCs' orthonormality the variance explained by successive components drops sharply after the first. METHODS: Here, we propose an alternative BoMI based on non-linear autoencoder (AE) networks that mapped arm kinematic signals into joint angles of a 4D virtual robotic manipulator. First, we performed a validation procedure that aimed at selecting an AE structure that would allow to distribute the input variance uniformly across the dimensions of the control space. Then, we assessed the users' proficiency practicing a 3D reaching task by operating the robot with the validated AE. RESULTS: All participants managed to acquire an adequate level of skill when operating the 4D robot. Moreover, they retained the performance across two non-consecutive days of training. CONCLUSION: While providing users with a fully continuous control of the robot, the entirely unsupervised nature of our approach makes it ideal for applications in a clinical context since it can be tailored to each user's residual movements. SIGNIFICANCE: We consider these findings as supporting a future implementation of our interface as an assistive tool for people with motor impairments.

Learning to operate a high-dimensional hand via a low-dimensional controller.

Dimensionality reduction techniques have proven useful in simplifying complex hand kinematics. They may allow for a low-dimensional kinematic or myoelectric interface to be used to control a high-dimensional hand. Controlling a high-dimensional hand, however, is difficult to learn since the relationship between the low-dimensional controls and the high-dimensional system can be hard to perceive. In this manuscript, we explore how training practices that make this relationship more explicit can aid learning. We outline three studies that explore different factors which affect learning of an autoencoder-based controller, in which a user is able to operate a high-dimensional virtual hand via a low-dimensional control space. We compare computer mouse and myoelectric control as one factor contributing to learning difficulty. We also compare training paradigms in which the dimensionality of the training task matched or did not match the true dimensionality of the low-dimensional controller (both 2D). The training paradigms were a) a full-dimensional task, in which the user was unaware of the underlying controller dimensionality, b) an implicit 2D training, which allowed the user to practice on a simple 2D reaching task before attempting the full-dimensional one, without establishing an explicit connection between the two, and c) an explicit 2D training, during which the user was able to observe the relationship between their 2D movements and the higher-dimensional hand. We found that operating a myoelectric interface did not pose a big challenge to learning the low-dimensional controller and was not the main reason for the poor performance. Implicit 2D training was found to be as good, but not better, as training directly on the high-dimensional hand. What truly aided the user's ability to learn the controller was the 2D training that established an explicit connection between the low-dimensional control space and the high-dimensional hand movements.

An Exploratory Multi-Session Study of Learning High-Dimensional Body-Machine Interfacing for Assistive Robot Control.

Individuals who suffer from severe paralysis often lose the capacity to perform fundamental body movements and everyday activities. Empowering these individuals with the ability to operate robotic arms, in high degrees-of-freedom (DoFs), can help to maximize both functional utility and independence. However, robot teleoperation in high DoFs currently lacks accessibility due to the challenge in capturing high-dimensional control signals from the human, especially in the face of motor impairments. Body-machine interfacing is a viable option that offers the necessary high-dimensional motion capture, and it moreover is noninvasive, affordable, and promotes movement and motor recovery. Nevertheless, to what extent body-machine interfacing is able to scale to high-DoF robot control, and whether it is feasible for humans to learn, remains an open question. In this exploratory multi-session study, we demonstrate the feasibility of human learning to operate a body-machine interface to control a complex, assistive robotic arm. We use a sensor net of four inertial measurement unit sensors, bilaterally placed on the scapulae and humeri. Ten uninjured participants are familiarized, trained, and evaluated in reaching and Activities of Daily Living tasks, using the body- machine interface. Our results suggest the manner of control space mapping (joint-space control versus task-space control), from interface to robot, plays a critical role in the evolution of human learning. Though joint-space control shows to be more intuitive initially, task-space control is found to have a greater capacity for longer-term improvement and learning.

How soft is that pillow? The perceptual localization of the hand and the haptic assessment of contact rigidity.

A new haptic illusion is described, in which the location of the mobile object affects the perception of its rigidity. There is theoretical and experimental support for the notion that limb position sense results from the brain combining ongoing sensory information with expectations arising from prior experience. How does this probabilistic state information affect one's tactile perception of the environment mechanics? In a simple estimation process, human subjects were asked to report the relative rigidity of two simulated virtual objects. One of the objects remained fixed in space and had various coefficients of stiffness. The other virtual object had constant stiffness but moved with respect to the subjects. Earlier work suggested that the perception of an object's rigidity is consistent with a process of regression between the contact force and the perceived amount of penetration inside the object's boundary. The amount of penetration perceived by the subject was affected by varying the position of the object. This, in turn, had a predictable effect on the perceived rigidity of the contact. Subjects' reports on the relative rigidity of the object are best accounted for by a probabilistic model in which the perceived boundary of the object is estimated based on its current location and on past observations. Therefore, the perception of contact rigidity is accounted for by a stochastic process of state estimation underlying proprioceptive localization of the hand.