Reward actively engages both implicit and explicit components in dual force field adaptation.

Motor learning occurs through multiple mechanisms, including unsupervised, supervised (error-based) and reinforcement (reward-based) learning. Although studies have shown that reward leads to an overall better motor adaptation, the specific processes by which reward influences adaptation are still unclear. Here, we examine how the presence of reward affects dual-adaptation to novel dynamics, and distinguish its influence on implicit and explicit learning. Participants adapted to two opposing force fields in an adaptation/de-adaptation/error-clamp paradigm, where five levels of reward (a score and a digital face) were provided as participants reduced their lateral error. Both reward and control (no reward provided) groups simultaneously adapted to both opposing force fields, exhibiting a similar final level of adaptation, which was primarily implicit. Triple-rate models fit to the adaptation process found higher learning rates in the fast and slow processes, and a slightly increased fast retention rate for the reward group. While differences in the slow learning rate were only driven by implicit learning, the large difference in the fast learning rate was mainly explicit. Overall, we confirm previous work showing that reward increases learning rates, extending this to dual-adaptation experiments, and demonstrating that reward influences both implicit and explicit adaptation. Specifically, we show that reward acts primarily explicitly on the fast learning rate and implicitly on the slow learning rates.

Learning context shapes bimanual control strategy and generalization of novel dynamics.

Bimanual movements are fundamental components of everyday actions, yet the underlying mechanisms coordinating adaptation of the two hands remain unclear. Although previous studies highlighted the contextual effect of kinematics of both arms on internal model formation, we do not know how the sensorimotor control system associates the learned memory with the experienced states in bimanual movements. More specifically, can, and if so, how, does the sensorimotor control system combine multiple states from different effectors to create and adapt a motor memory? Here, we tested motor memory formation in two groups with a novel paradigm requiring the encoding of the kinematics of the right hand to produce the appropriate predictive force on the left hand. While one group was provided with training movements in which this association was evident, the other group was trained on conditions in which this association was ambiguous. After adaptation, we tested the encoding of the learned motor memory by measuring the generalization to new movement combinations. While both groups adapted to the novel dynamics, the evident group showed a weighted encoding of the learned motor memory based on movements of the other (right) hand, whereas the ambiguous group exhibited mainly same (left) hand encoding in bimanual trials. Despite these differences, both groups demonstrated partial generalization to unimanual movements of the left hand. Our results show that motor memories can be encoded depending on the motion of other limbs, but that the training conditions strongly shape the encoding of the motor memory formation and determine the generalization to novel contexts.

Explicit learning based on reward prediction error facilitates agile motor adaptations.

Error based motor learning can be driven by both sensory prediction error and reward prediction error. Learning based on sensory prediction error is termed sensorimotor adaptation, while learning based on reward prediction error is termed reward learning. To investigate the characteristics and differences between sensorimotor adaptation and reward learning, we adapted a visuomotor paradigm where subjects performed arm movements while presented with either the sensory prediction error, signed end-point error, or binary reward. Before each trial, perturbation indicators in the form of visual cues were presented to inform the subjects of the presence and direction of the perturbation. To analyse the interconnection between sensorimotor adaptation and reward learning, we designed a computational model that distinguishes between the two prediction errors. Our results indicate that subjects adapted to novel perturbations irrespective of the type of prediction error they received during learning, and they converged towards the same movement patterns. Sensorimotor adaptations led to a pronounced aftereffect, while adaptation based on reward consequences produced smaller aftereffects suggesting that reward learning does not alter the internal model to the same degree as sensorimotor adaptation. Even though all subjects had learned to counteract two different perturbations separately, only those who relied on explicit learning using reward prediction error could timely adapt to the randomly changing perturbation. The results from the computational model suggest that sensorimotor and reward learning operate through distinct adaptation processes and that only sensorimotor adaptation changes the internal model, whereas reward learning employs explicit strategies that do not result in aftereffects. Additionally, we demonstrate that when humans learn motor tasks, they utilize both learning processes to successfully adapt to the new environments.

Behavioral Motor Performance.

The human sensorimotor control system has exceptional abilities to perform skillful actions. We easily switch between strenuous tasks that involve brute force, such as lifting a heavy sewing machine, and delicate movements such as threading a needle in the same machine. Using a structure with different control architectures, the motor system is capable of updating its ability to perform through our daily interaction with the fluctuating environment. However, there are issues that make this a difficult computational problem for the brain to solve. The brain needs to control a nonlinear, nonstationary neuromuscular system, with redundant and occasionally undesired degrees of freedom, in an uncertain environment using a body in which information transmission is subject to delays and noise. To gain insight into the mechanisms of motor control, here we survey movement laws and invariances that shape our everyday motion. We then examine the major solutions to each of these problems in the three parts of the sensorimotor control system, sensing, planning, and acting. We focus on how the sensory system, the control architectures, and the structure and operation of the muscles serve as complementary mechanisms to overcome deviations and disturbances to motor behavior and give rise to skillful motor performance. We conclude with possible future research directions based on suggested links between the operation of the sensorimotor system across the movement stages. © 2024 American Physiological Society. Compr Physiol 14:5179-5224, 2024.

Goal-directed modulation of stretch reflex gains is reduced in the non-dominant upper limb.

Most individuals experience their dominant arm as being more dexterous than the non-dominant arm, but the neural mechanisms underlying this asymmetry in motor behaviour are unclear. Using a delayed-reach task, we have recently demonstrated strong goal-directed tuning of stretch reflex gains in the dominant upper limb of human participants. Here, we used an equivalent experimental paradigm to address the neural mechanisms that underlie the preparation for reaching movements with the non-dominant upper limb. There were consistent effects of load, preparatory delay duration and target direction on the long latency stretch reflex. However, by comparing stretch reflex responses in the non-dominant arm with those previously documented in the dominant arm, we demonstrate that goal-directed tuning of short and long latency stretch reflexes is markedly weaker in the non-dominant limb. The results indicate that the motor performance asymmetries across the two upper limbs are partly due to the more sophisticated control of reflexive stiffness in the dominant limb, likely facilitated by the superior goal-directed control of muscle spindle receptors. Our findings therefore suggest that fusimotor control may play a role in determining performance of complex motor behaviours and support existing proposals that the dominant arm is better supplied than the non-dominant arm for executing more complex tasks, such as trajectory control.

Congruent visual cues speed dynamic motor adaptation.

Motor adaptation to novel dynamics occurs rapidly using sensed errors to update the current motor memory. This adaption is strongly driven by proprioceptive and visual signals that indicate errors in the motor memory. Here, we extend this previous work by investigating whether the presence of additional visual cues could increase the rate of motor adaptation, specifically when the visual motion cue is congruent with the dynamics. Six groups of participants performed reaching movements while grasping the handle of a robotic manipulandum. A visual cue (small red circle) was connected to the cursor (representing the hand position) via a thin red bar. After a baseline, a unidirectional (3 groups) or bidirectional (3 groups) velocity-dependent force field was applied during the reach. For each group, the movement of the red object relative to the cursor was either congruent with the force field dynamics, incongruent with the force field dynamics, or constant (fixed distance from the cursor). Participants adapted more to the unidirectional force fields than to the bidirectional force field groups. However, across both force fields, groups in which the visual cues matched the type of force field (congruent visual cue) exhibited higher final adaptation level at the end of learning than the control or incongruent conditions. In all groups, we observed that an additional congruent cue assisted the formation of the motor memory of the external dynamics. We then demonstrate that a state estimation-based model that integrates proprioceptive and visual information can successfully replicate the experimental data.NEW & NOTEWORTHY We demonstrate that adaptation to novel dynamics is stronger when additional online visual cues that are congruent with the dynamics are presented during adaptation, compared with either a constant or incongruent visual cue. This effect was found regardless of whether a bidirectional or unidirectional velocity-dependent force field was presented to the participants. We propose that this effect might arise through the inclusion of this additional visual cue information within the state estimation process.

Musculotendon Parameters in Lower Limb Models: Simplifications, Uncertainties, and Muscle Force Estimation Sensitivity.

Musculotendon parameters are key factors in the Hill-type muscle contraction dynamics, determining the muscle force estimation accuracy of a musculoskeletal model. Their values are mostly derived from muscle architecture datasets, whose emergence has been a major impetus for model development. However, it is often not clear if such parameter update indeed improves simulation accuracy. Our goal is to explain to model users how these parameters are derived and how accurate they are, as well as to what extent errors in parameter values might influence force estimation. We examine in detail the derivation of musculotendon parameters in six muscle architecture datasets and four prominent OpenSim models of the lower limb, and then identify simplifications which could add uncertainties to the derived parameter values. Finally, we analyze the sensitivity of muscle force estimation to these parameters both numerically and analytically. Nine typical simplifications in parameter derivation are identified. Partial derivatives of the Hill-type contraction dynamics are derived. Tendon slack length is determined as the musculotendon parameter that muscle force estimation is most sensitive to, whereas pennation angle is the least impactful. Anatomical measurements alone are not enough to calibrate musculotendon parameters, and the improvement on muscle force estimation accuracy will be limited if the source muscle architecture datasets are the only main update. Model users may check if a dataset or model is free of concerning factors for their research or application requirements. The derived partial derivatives may be used as the gradient for musculotendon parameter calibration. For model development, we demonstrate that it is more promising to focus on other model parameters or components and seek alternative strategies to further increase simulation accuracy.

Assistive Loading Promotes Goal-Directed Tuning of Stretch Reflex Gains.

Voluntary movements are prepared before they are executed. Preparatory activity has been observed across the CNS and recently documented in first-order neurons of the human PNS (i.e., in muscle spindles). Changes seen in sensory organs suggest that independent modulation of stretch reflex gains may represent an important component of movement preparation. The aim of the current study was to further investigate the preparatory modulation of short-latency stretch reflex responses (SLRs) and long-latency stretch reflex responses (LLRs) of the dominant upper limb of human subjects. Specifically, we investigated how different target parameters (target distance and direction) affect the preparatory tuning of stretch reflex gains in the context of goal-directed reaching, and whether any such tuning depends on preparation duration and the direction of background loads. We found that target distance produced only small variations in reflex gains. In contrast, both SLR and LLR gains were strongly modulated as a function of target direction, in a manner that facilitated the upcoming voluntary movement. This goal-directed tuning of SLR and LLR gains was present or enhanced when the preparatory delay was sufficiently long (>250 ms) and the homonymous muscle was unloaded [i.e., when a background load was first applied in the direction of homonymous muscle action (assistive loading)]. The results extend further support for a relatively slow-evolving process in reach preparation that functions to modulate reflexive muscle stiffness, likely via the independent control of fusimotor neurons. Such control can augment voluntary goal-directed movement and is triggered or enhanced when the homonymous muscle is unloaded.

Stability of inverted pendulum reveals transition between predictive control and impedance control in grip force modulation.

During object manipulation, our sensorimotor sys-tem needs to represent the objects dynamics in order to better control it. This is especially important in the case of grip force control where small forces can cause the object to slip from our fingers, and excessive forces can cause fatigue or even damage the object. While the tradeoff between these two constraints is clear for stable objects, such as lifting a soda can, it is less clear how the sensorimotor system adjusts the grip force for unstable objects. For this purpose, we measured the change in the grip force of individual human participants while they stabilize five different lengths of an inverted pendulum. These lengths set different dynamics of the pendulum, ranging in their degree of controllability. We observed two main states during such manipulation, a marginally stable state of the pendulum and a stabilization state in which participants acted to stabilize the system. While during the stabilization state participants increased their applied grip force, for the stable state we observed a mixed behaviour. For small and less controllable pendulums, grip force increased while for larger pendulums, participants could modulate the the grip force according to the anticipated load forces. Based on these results, we suggest that the pendulum dynamics change the control strategy between predictive control and impedance control.

Learning of Dexterous Object Manipulation in a Virtual Reality Environment.

Humans have unrivalled abilities to perform dexterous object manipulation. This requires the sensorimotor system to quickly adapt to environmental changes and predictively counter act the external disturbances. Many studies have focused on the anticipatory control of digits with real-world experiments. However, examining manipulation using virtual reality with haptic devices expands the possibilities of investigation. In this work, participants grasped and lifted an inverted T-shaped object in a virtual reality setup. The graspable surface of the object was either constrained to a small area or unconstrained. The position of the object's center of mass changed between blocks, and the participants were asked to minimize the rotation of the object during the lift. Our results show that, consistent with the results of real-world experiments, participants gradually learn to adjust the digit positions and forces to predictively compensate for the torque due to the shifted center of mass prior to liftoff. The only major difference found was that the length of trials needed during the adaptation phase to each condition increased from 3 in real-world to 5 in virtual environment.

Bimanual Manipulation of a Complex Object with Internal Dynamics.

Object manipulation often requires coordination between hands and adaption to the dynamic characteristics of the object. When manipulating the same object, the two hands can have either symmetric or asymmetric impact on the object's trajectory. In this work, we used a bimanual manipulation task of a complex object with internal dynamics to examine how symmetric or scaled-down control of one of the hands affects the coordination between hands. Our result shows that participants are able to quickly adapt to different conditions but the coordination between the two hands changes very little.

Contextual cues are not unique for motor learning: Task-dependant switching of feedback controllers.

The separation of distinct motor memories by contextual cues is a well known and well studied phenomenon of feedforward human motor control. However, there is no clear evidence of such context-induced separation in feedback control. Here we test both experimentally and computationally if context-dependent switching of feedback controllers is possible in the human motor system. Specifically, we probe visuomotor feedback responses of our human participants in two different tasks-stop and hit-and under two different schedules. The first, blocked schedule, is used to measure the behaviour of stop and hit controllers in isolation, showing that it can only be described by two independent controllers with two different sets of control gains. The second, mixed schedule, is then used to compare how such behaviour evolves when participants regularly switch from one task to the other. Our results support our hypothesis that there is contextual switching of feedback controllers, further extending the accumulating evidence of shared features between feedforward and feedback control.

Motor Learning in Response to Different Experimental Pain Models Among Healthy Individuals: A Systematic Review.

Learning new movement patterns is a normal part of daily life, but of critical importance in both sport and rehabilitation. A major question is how different sensory signals are integrated together to give rise to motor adaptation and learning. More specifically, there is growing evidence that pain can give rise to alterations in the learning process. Despite a number of studies investigating the role of pain on the learning process, there is still no systematic review to summarize and critically assess investigations regarding this topic in the literature. Here in this systematic review, we summarize and critically evaluate studies that examined the influence of experimental pain on motor learning. Seventeen studies that exclusively assessed the effect of experimental pain models on motor learning among healthy human individuals were included for this systematic review, carried out based on the preferred reporting items for systematic reviews and meta-analyses (PRISMA) statement. The results of the review revealed there is no consensus regarding the effect of pain on the skill learning acquisition and retention. However, several studies demonstrated that participants who experienced pain continued to express a changed motor strategy to perform a motor task even 1 week after training under the pain condition. The results highlight a need for further studies in this area of research, and specifically to investigate whether pain has different effects on motor learning depending on the type of motor task.

Active strategies for multisensory conflict suppression in the virtual hand illusion.

The perception of our body in space is flexible and manipulable. The predictive brain hypothesis explains this malleability as a consequence of the interplay between incoming sensory information and our body expectations. However, given the interaction between perception and action, we might also expect that actions would arise due to prediction errors, especially in conflicting situations. Here we describe a computational model, based on the free-energy principle, that forecasts involuntary movements in sensorimotor conflicts. We experimentally confirm those predictions in humans using a virtual reality rubber-hand illusion. Participants generated movements (forces) towards the virtual hand, regardless of its location with respect to the real arm, with little to no forces produced when the virtual hand overlaid their physical hand. The congruency of our model predictions and human observations indicates that the brain-body is generating actions to reduce the prediction error between the expected arm location and the new visual arm. This observed unconscious mechanism is an empirical validation of the perception-action duality in body adaptation to uncertain situations and evidence of the active component of predictive processing.

Direct and indirect cues can enable dual adaptation, but through different learning processes.

Switching between motor tasks requires accurate adjustments for changes in dynamics (grasping a cup) or sensorimotor transformations (moving a computer mouse). Dual-adaptation studies have investigated how learning of context-dependent dynamics or transformations is enabled by sensory cues. However, certain cues, such as color, have shown mixed results. We propose that these mixed results may arise from two major classes of cues: "direct" cues, which are part of the dynamic state and "indirect" cues, which are not. We hypothesized that explicit strategies would primarily account for the adaptation of an indirect color cue but would be limited to simple tasks, whereas a direct visual separation cue would allow implicit adaptation regardless of task complexity. To test this idea, we investigated the relative contribution of implicit and explicit learning in relation to contextual cue type (colored or visually shifted workspace) and task complexity (1 or 8 targets) in a dual-adaptation task. We found that the visual workspace location cue enabled adaptation across conditions primarily through implicit adaptation. In contrast, we found that the color cue was largely ineffective for dual adaptation, except in a small subset of participants who appeared to use explicit strategies. Our study suggests that the previously inconclusive role of color cues in dual adaptation may be explained by differential contribution of explicit strategies across conditions.NEW & NOTEWORTHY We present evidence that learning of context-dependent dynamics proceeds via different processes depending on the type of sensory cue used to signal the context. Visual workspace location enabled learning different dynamics implicitly, presumably because it directly enters the dynamic state estimate. In contrast, a color cue was only successful where learners were apparently able to leverage explicit strategies to account for changed dynamics. This suggests a unification for the previously inconclusive role of color cues.

Nanomesh pressure sensor for monitoring finger manipulation without sensory interference.

Monitoring of finger manipulation without disturbing the inherent functionalities is critical to understand the sense of natural touch. However, worn or attached sensors affect the natural feeling of the skin. We developed nanomesh pressure sensors that can monitor finger pressure without detectable effects on human sensation. The effect of the sensor on human sensation was quantitatively investigated, and the sensor-applied finger exhibits comparable grip forces with those of the bare finger, even though the attachment of a 2-micrometer-thick polymeric film results in a 14% increase in the grip force after adjusting for friction. Simultaneously, the sensor exhibits an extreme mechanical durability against cyclic shearing and friction greater than hundreds of kilopascals.

Timescales of motor memory formation in dual-adaptation.

The timescales of adaptation to novel dynamics are well explained by a dual-rate model with slow and fast states. This model can predict interference, savings and spontaneous recovery, but cannot account for adaptation to multiple tasks, as each new task drives unlearning of the previously learned task. Nevertheless, in the presence of appropriate contextual cues, humans are able to adapt simultaneously to opposing dynamics. Consequently this model was expanded, suggesting that dual-adaptation occurs through a single fast process and multiple slow processes. However, such a model does not predict spontaneous recovery within dual-adaptation. Here we assess the existence of multiple fast processes by examining the presence of spontaneous recovery in two experimental variations of an adaptation-de-adaptation-error-clamp paradigm within dual-task adaptation in humans. In both experiments, evidence for spontaneous recovery towards the initially learned dynamics (A) was found in the error-clamp phase, invalidating the one-fast-two-slow dual-rate model. However, as adaptation is not only constrained to two timescales, we fit twelve multi-rate models to the experimental data. BIC model comparison again supported the existence of two fast processes, but extended the timescales to include a third rate: the ultraslow process. Even within our single day experiment, we found little evidence for decay of the learned memory over several hundred error-clamp trials. Overall, we show that dual-adaptation can be best explained by a two-fast-triple-rate model over the timescales of adaptation studied here. Longer term learning may require even slower timescales, explaining why we never forget how to ride a bicycle.

Time-to-Target Simplifies Optimal Control of Visuomotor Feedback Responses.

Visuomotor feedback responses vary in intensity throughout a reach, commonly explained by optimal control. Here, we show that the optimal control for a range of movements with the same goal can be simplified to a time-to-target dependent control scheme. We measure our human participants' visuomotor responses in five reaching conditions, each with different hand or cursor kinematics. Participants only produced different feedback responses when these kinematic changes resulted in different times-to-target. We complement our experimental data with a range of finite and non-finite horizon optimal feedback control (OFC) models, finding that the model with time-to-target as one of the input parameters best replicates the experimental data. Overall, this suggests that time-to-target is a critical control parameter in online feedback control. Moreover, we propose that for a specific task and known dynamics, humans can instantly produce a control signal without any additional online computation allowing rapid response onset and close to optimal control.

Asymmetry in kinematic generalization between visual and passive lead-in movements are consistent with a forward model in the sensorimotor system.

In our daily life we often make complex actions comprised of linked movements, such as reaching for a cup of coffee and bringing it to our mouth to drink. Recent work has highlighted the role of such linked movements in the formation of independent motor memories, affecting the learning rate and ability to learn opposing force fields. In these studies, distinct prior movements (lead-in movements) allow adaptation of opposing dynamics on the following movement. Purely visual or purely passive lead-in movements exhibit different angular generalization functions of this motor memory as the lead-in movements are modified, suggesting different neural representations. However, we currently have no understanding of how different movement kinematics (distance, speed or duration) affect this recall process and the formation of independent motor memories. Here we investigate such kinematic generalization for both passive and visual lead-in movements to probe their individual characteristics. After participants adapted to opposing force fields using training lead-in movements, the lead-in kinematics were modified on random trials to test generalization. For both visual and passive modalities, recalled compensation was sensitive to lead-in duration and peak speed, falling off away from the training condition. However, little reduction in force was found with increasing lead-in distance. Interestingly, asymmetric transfer between lead-in movement modalities was also observed, with partial transfer from passive to visual, but very little vice versa. Overall these tuning effects were stronger for passive compared to visual lead-ins demonstrating the difference in these sensory inputs in regulating motor memories. Our results suggest these effects are a consequence of state estimation, with differences across modalities reflecting their different levels of sensory uncertainty arising as a consequence of dissimilar feedback delays.

A Technique for Measuring Visuomotor Feedback Contributions to the Control of an Inverted Pendulum.

We developed a new technique to measure the contributions of rapid visuomotor feedback responses to the stabilization of a simulated inverted pendulum. Human participants balanced an inverted pendulum simulated on a robotic manipulandum. At a random time during the balancing task, the visual representation of the tip of the pendulum was shifted by a small displacement to the left or right while the motor response was measured. This response was either the exerted force against a fixation position, or the motion to re-stabilize the pendulum in the free condition. Our results demonstrate that rapid involuntary visuomotor feedback responses contribute to the stabilization of the pendulum.