De novo sensorimotor learning through reuse of movement components.

From tying one's shoelaces to driving a car, complex skills involving the coordination of multiple muscles are common in everyday life; yet relatively little is known about how these skills are learned. Recent studies have shown that new sensorimotor skills involving re-mapping familiar body movements to unfamiliar outputs cannot be learned by adjusting pre-existing controllers, and that new task-specific controllers must instead be learned "de novo". To date, however, few studies have investigated de novo learning in scenarios requiring continuous and coordinated control of relatively unpractised body movements. In this study, we used a myoelectric interface to investigate how a novel controller is learned when the task involves an unpractised combination of relatively untrained continuous muscle contractions. Over five sessions on five consecutive days, participants learned to trace a series of trajectories using a computer cursor controlled by the activation of two muscles. The timing of the generated cursor trajectory and its shape relative to the target improved for conditions trained with post-trial visual feedback. Improvements in timing transferred to all untrained conditions, but improvements in shape transferred less robustly to untrained conditions requiring the trained order of muscle activation. All muscle outputs in the final session could already be generated during the first session, suggesting that participants learned the new task by improving the selection of existing motor commands. These results suggest that the novel controllers acquired during de novo learning can, in some circumstances, be constructed from components of existing controllers.

Implicit Adaptation Is Modulated by the Relevance of Feedback.

Given that informative and relevant feedback in the real world is often intertwined with distracting and irrelevant feedback, we asked how the relevancy of visual feedback impacts implicit sensorimotor adaptation. To tackle this question, we presented multiple cursors as visual feedback in a center-out reaching task and varied the task relevance of these cursors. In other words, participants were instructed to hit a target with a specific task-relevant cursor, while ignoring the other cursors. In Experiment 1, we found that reach aftereffects were attenuated by the mere presence of distracting cursors, compared with reach aftereffects in response to a single task-relevant cursor. The degree of attenuation did not depend on the position of the distracting cursors. In Experiment 2, we examined the interaction between task relevance and attention. Participants were asked to adapt to a task-relevant cursor/target pair, while ignoring the task-irrelevant cursor/target pair. Critically, we jittered the location of the relevant and irrelevant target in an uncorrelated manner, allowing us to index attention via how well participants tracked the position of target. We found that participants who were better at tracking the task-relevant target/cursor pair showed greater aftereffects, and interestingly, the same correlation applied to the task-irrelevant target/cursor pair. Together, these results highlight a novel role of task relevancy on modulating implicit adaptation, perhaps by giving greater attention to informative sources of feedback, increasing the saliency of the sensory prediction error.

Kinematic markers of skill in first-person shooter video games.

Video games present a unique opportunity to study motor skill. First-person shooter (FPS) games have particular utility because they require visually guided hand movements that are similar to widely studied planar reaching tasks. However, there is a need to ensure the tasks are equivalent if FPS games are to yield their potential as a powerful scientific tool for investigating sensorimotor control. Specifically, research is needed to ensure that differences in visual feedback of a movement do not affect motor learning between the two contexts. In traditional tasks, a movement will translate a cursor across a static background, whereas FPS games use movements to pan and tilt the view of the environment. To this end, we designed an online experiment where participants used their mouse or trackpad to shoot targets in both visual contexts. Kinematic analysis showed player movements were nearly identical between contexts, with highly correlated spatial and temporal metrics. This similarity suggests a shared internal model based on comparing predicted and observed displacement vectors rather than primary sensory feedback. A second experiment, modeled on FPS-style aim-trainer games, found movements exhibited classic invariant features described within the sensorimotor literature. We found the spatial metrics tested were significant predictors of overall task performance. More broadly, these results show that FPS games offer a novel, engaging, and compelling environment to study sensorimotor skill, providing the same precise kinematic metrics as traditional planar reaching tasks.

A double dissociation between savings and long-term memory in motor learning.

Memories are easier to relearn than learn from scratch. This advantage, known as savings, has been widely assumed to result from the reemergence of stable long-term memories. In fact, the presence of savings has often been used as a marker for whether a memory has been consolidated. However, recent findings have demonstrated that motor learning rates can be systematically controlled, providing a mechanistic alternative to the reemergence of a stable long-term memory. Moreover, recent work has reported conflicting results about whether implicit contributions to savings in motor learning are present, absent, or inverted, suggesting a limited understanding of the underlying mechanisms. To elucidate these mechanisms, we investigate the relationship between savings and long-term memory by experimentally dissecting the underlying memories based on short-term (60-s) temporal persistence. Components of motor memory that are temporally-persistent at 60 s might go on to contribute to stable, consolidated long-term memory, whereas temporally-volatile components that have already decayed away by 60 s cannot. Surprisingly, we find that temporally-volatile implicit learning leads to savings, whereas temporally-persistent learning does not, but that temporally-persistent learning leads to long-term memory at 24 h, whereas temporally-volatile learning does not. This double dissociation between the mechanisms for savings and long-term memory formation challenges widespread assumptions about the connection between savings and memory consolidation. Moreover, we find that temporally-persistent implicit learning not only fails to contribute to savings, but also that it produces an opposite, anti-savings effect, and that the interplay between this temporally-persistent anti-savings and temporally-volatile savings provides an explanation for several seemingly conflicting recent reports about whether implicit contributions to savings are present, absent, or inverted. Finally, the learning curves we observed for the acquisition of temporally-volatile and temporally-persistent implicit memories demonstrate the coexistence of implicit memories with distinct time courses, challenging the assertion that models of context-based learning and estimation should supplant models of adaptive processes with different learning rates. Together, these findings provide new insight into the mechanisms for savings and long-term memory formation.

Measuring motion-to-photon latency for sensorimotor experiments with virtual reality systems.

Consumer virtual reality (VR) systems are increasingly being deployed in research to study sensorimotor behaviors, but properties of such systems require verification before being used as scientific tools. The 'motion-to-photon' latency (the lag between a user making a movement and the movement being displayed within the display) is a particularly important metric as temporal delays can degrade sensorimotor performance. Extant approaches to quantifying this measure have involved the use of bespoke software and hardware and produce a single measure of latency and ignore the effect of the motion prediction algorithms used in modern VR systems. This reduces confidence in the generalizability of the results. We developed a novel, system-independent, high-speed camera-based latency measurement technique to co-register real and virtual controller movements, allowing assessment of how latencies change through a movement. We applied this technique to measure the motion-to-photon latency of controller movements in the HTC Vive, Oculus Rift, Oculus Rift S, and Valve Index, using the Unity game engine and SteamVR. For the start of a sudden movement, all measured headsets had mean latencies between 21 and 42 ms. Once motion prediction could account for the inherent delays, the latency was functionally reduced to 2-13 ms, and our technique revealed that this reduction occurs within ~25-58 ms of movement onset. Our findings indicate that sudden accelerations (e.g., movement onset, impacts, and direction changes) will increase latencies and lower spatial accuracy. Our technique allows researchers to measure these factors and determine the impact on their experimental design before collecting sensorimotor data from VR systems.

Sensorimotor ability and inhibitory control independently predict attainment in mathematics in children and adolescents.

We previously linked interceptive timing performance to mathematics attainment in 5- to 11-yr-old children, which we attributed to the neural overlap between spatiotemporal and numerical operations. This explanation implies that the relationship should persist through the teenage years. Here, we replicated this finding in adolescents (n = 200, 11-15 yr). However, an alternative explanation is that sensorimotor proficiency and academic attainment are both consequences of executive function. To assess this competing hypothesis, we developed a measure of a core executive function, inhibitory control, from the kinematic data. We combined our new adolescent data with the original children's data (total n = 568), performing a novel analysis controlling for our marker of executive function. We found that the relationship between mathematics and interceptive timing persisted at all ages. These results suggest a distinct functional link between interceptive timing and mathematics that operates independently of our measure of executive function.NEW & NOTEWORTHY Previous research downplays the role of sensorimotor skills in the development of higher-order cognitive domains such as mathematics: using inadequate sensorimotor measures, differences in "executive function" account for any shared variance. Utilizing a high-resolution, kinematic measure of a sensorimotor skill previously linked to mathematics attainment, we show that inhibitory control alone cannot account for this relationship. The practical implication is that the development of children's sensorimotor skills must be considered in their intellectual development.

Reexposure to a sensorimotor perturbation produces opposite effects on explicit and implicit learning processes.

The motor system demonstrates an exquisite ability to adapt to changes in the environment and to quickly reset when these changes prove transient. If similar environmental changes are encountered in the future, learning may be faster, a phenomenon known as savings. In studies of sensorimotor learning, a central component of savings is attributed to the explicit recall of the task structure and appropriate compensatory strategies. Whether implicit adaptation also contributes to savings remains subject to debate. We tackled this question by measuring, in parallel, explicit and implicit adaptive responses in a visuomotor rotation task, employing a protocol that typically elicits savings. While the initial rate of learning was faster in the second exposure to the perturbation, an analysis decomposing the 2 processes showed the benefit to be solely associated with explicit re-aiming. Surprisingly, we found a significant decrease after relearning in aftereffect magnitudes during no-feedback trials, a direct measure of implicit adaptation. In a second experiment, we isolated implicit adaptation using clamped visual feedback, a method known to eliminate the contribution of explicit learning processes. Consistent with the results of the first experiment, participants exhibited a marked reduction in the adaptation function, as well as an attenuated aftereffect when relearning from the clamped feedback. Motivated by these results, we reanalyzed data from prior studies and observed a consistent, yet unappreciated pattern of attenuation of implicit adaptation during relearning. These results indicate that explicit and implicit sensorimotor processes exhibit opposite effects upon relearning: Explicit learning shows savings, while implicit adaptation becomes attenuated.

Invariant errors reveal limitations in motor correction rather than constraints on error sensitivity.

Implicit sensorimotor adaptation is traditionally described as a process of error reduction, whereby a fraction of the error is corrected for with each movement. Here, in our study of healthy human participants, we characterize two constraints on this learning process: the size of adaptive corrections is only related to error size when errors are smaller than 6°, and learning functions converge to a similar level of asymptotic learning over a wide range of error sizes. These findings are problematic for current models of sensorimotor adaptation, and point to a new theoretical perspective in which learning is constrained by the size of the error correction, rather than sensitivity to error.

Characteristics of Implicit Sensorimotor Adaptation Revealed by Task-irrelevant Clamped Feedback.

Sensorimotor adaptation occurs when there is a discrepancy between the expected and actual sensory consequences of a movement. This learning can be precisely measured, but its source has been hard to pin down because standard adaptation tasks introduce two potential learning signals: task performance errors and sensory prediction errors. Here we employed a new method that induces sensory prediction errors without task performance errors. This method combines the use of clamped visual feedback that is angularly offset from the target and independent of the direction of motion, along with instructions to ignore this feedback while reaching to targets. Despite these instructions, participants unknowingly showed robust adaptation of their movements. This adaptation was similar to that observed with standard methods, showing sign dependence, local generalization, and cerebellar dependency. Surprisingly, adaptation rate and magnitude were invariant across a large range of offsets. Collectively, our results challenge current models of adaptation and demonstrate that behavior observed in many studies of adaptation reflect the composite effects of task performance and sensory prediction errors.

Savings upon Re-Aiming in Visuomotor Adaptation.

Sensorimotor adaptation has traditionally been viewed as a purely error-based process. There is, however, growing appreciation for the idea that performance changes in these tasks can arise from the interplay of error-based adaptation with other learning processes. The challenge is to specify constraints on these different processes, elucidating their respective contributions to performance, as well as the manner in which they interact. We address this question by exploring constraints on savings, the phenomenon in which people show faster performance gains when the same learning task is repeated. In a series of five experiments, we demonstrate that error-based learning associated with sensorimotor adaptation does not contribute to savings. Instead, savings reflects improvements in action selection, rather than motor execution. SIGNIFICANCE STATEMENT: Savings is the phenomenon in which people show faster relearning of a previously forgotten memory. In the motor learning domain, this phenomenon has been a puzzle for learning models that operate exclusively on error-based learning processes. We demonstrate, in a series of experiments, that savings selectively reflects improvements in action selection: Participants are more adept in invoking an appropriate aiming strategy when presented with a previously experienced perturbation. Indeed, improvements in action selection appear to be the sole source of savings in visuomotor adaptation tasks. We observe no evidence of savings in implicit error-based adaptation.

Prediction during statistical learning, and implications for the implicit/explicit divide.

Accounts of statistical learning, both implicit and explicit, often invoke predictive processes as central to learning, yet practically all experiments employ non-predictive measures during training. We argue that the common theoretical assumption of anticipation and prediction needs clearer, more direct evidence for it during learning. We offer a novel experimental context to explore prediction, and report results from a simple sequential learning task designed to promote predictive behaviors in participants as they responded to a short sequence of simple stimulus events. Predictive tendencies in participants were measured using their computer mouse, the trajectories of which served as a means of tapping into predictive behavior while participants were exposed to very short and simple sequences of events. A total of 143 participants were randomly assigned to stimulus sequences along a continuum of regularity. Analysis of computer-mouse trajectories revealed that (a) participants almost always anticipate events in some manner, (b) participants exhibit two stable patterns of behavior, either reacting to vs. predicting future events, (c) the extent to which participants predict relates to performance on a recall test, and (d) explicit reports of perceiving patterns in the brief sequence correlates with extent of prediction. We end with a discussion of implicit and explicit statistical learning and of the role prediction may play in both kinds of learning.