Sensorimotor prediction is used to direct gaze toward task-relevant locations in a goal-directed throwing task.

Previous research has shown that action effects of self-generated movements are internally predicted before outcome feedback becomes available. To test whether these sensorimotor predictions are used to facilitate visual information uptake for feedback processing, we measured eye movements during the execution of a goal-directed throwing task. Participants could fully observe the effects of their throwing actions (ball trajectory, and either hitting or missing a target) in most of the trials. In a portion of the trials, the ball trajectory was not visible, and participants only received static information about the outcome. We observed a large proportion of predictive saccades, shifting gaze toward the goal region before the ball arrived and outcome feedback became available. Fixation locations after predictive saccades systematically covaried with future ball positions in trials with continuous ball flight information, but notably also in trials with static outcome feedback and only efferent and proprioceptive information about the movement that could be used for predictions. Fixation durations at the chosen positions after feedback onset were modulated by action outcome (longer durations for misses than for hits) and outcome uncertainty (longer durations for narrow vs. clear outcomes). Combining both effects, durations were longest for narrow errors and shortest for clear hits, indicating that the chosen locations offer informational value for feedback processing. Thus, humans are able to use sensorimotor predictions to direct their gaze toward task-relevant feedback locations. Outcome-dependent saccade latency differences (miss vs. hit) indicate that also predictive valuation processes are involved in planning predictive saccades.

A primitive-based representation of dance: modulations by experience and perceptual validity.

The generation of complex movements such as dance might be possible due to the utilization of movement building blocks, i.e., movement primitives. However, it is largely unexplored how the temporal structure of a movement sequence and the recruitment of these primitives change with experience. Therefore, we obtained a representation of primitives with the temporal movement primitive model from the motion capture data of dancers with varying experiences, both for improvised and choreographed movements (elements from contemporary/modern/jazz) with different qualitative expressions. We analyzed differences between movement conditions regarding the number of temporal segments and the number of primitives, as well as their association with dance experience. Especially for the choreography with a neutral expression, the results indicate a negative association between experience and the number of segments and a positive association between experience and the number of primitives. The variation in the recruitment of these primitives suggests an increased consistency of modular control with experience, particularly for improvised dance. A prerequisite for the meaningful interpretation of these results regarding human movement production is that the model can generate perceptually valid dance movements. This was confirmed in a subsequent experiment, although the validity was slightly impaired for improvised movements. Overall, the results of the choreographed movement sequences suggest that experience is associated with an increase in motor repertoire that might facilitate fewer and longer temporal segments.NEW & NOTEWORTHY This study demonstrates that a temporal movement primitive model, trained with movements performed by dancers with different levels of experience, is able to generate natural-looking dance movements. The results suggest that motor experience in dance is associated not only with fewer temporal segments but also with an increase in the number of underlying movement building blocks. The recruitment of these primitives, which might be used to simplify movement production, additionally seems to become more consistent with experience.

The multisensory nature of human action imagery.

Imagination can appeal to all our senses and may, therefore, manifest in very different qualities (e.g., visual, tactile, proprioceptive, or kinesthetic). One line of research addresses action imagery that refers to a process by which people imagine the execution of an action without actual body movements. In action imagery, visual and kinesthetic aspects of the imagined action are particularly important. However, other sensory modalities may also play a role. The purpose of the paper will be to address issues that include: (i) the creation of an action image, (ii) how the brain generates images of movements and actions, (iii) the richness and vividness of action images. We will further address possible causes that determine the sensory impression of an action image, like task specificity, instruction and experience. In the end, we will outline open questions and future directions.

Direct and indirect cues can enable dual adaptation, but through different learning processes.

Switching between motor tasks requires accurate adjustments for changes in dynamics (grasping a cup) or sensorimotor transformations (moving a computer mouse). Dual-adaptation studies have investigated how learning of context-dependent dynamics or transformations is enabled by sensory cues. However, certain cues, such as color, have shown mixed results. We propose that these mixed results may arise from two major classes of cues: "direct" cues, which are part of the dynamic state and "indirect" cues, which are not. We hypothesized that explicit strategies would primarily account for the adaptation of an indirect color cue but would be limited to simple tasks, whereas a direct visual separation cue would allow implicit adaptation regardless of task complexity. To test this idea, we investigated the relative contribution of implicit and explicit learning in relation to contextual cue type (colored or visually shifted workspace) and task complexity (1 or 8 targets) in a dual-adaptation task. We found that the visual workspace location cue enabled adaptation across conditions primarily through implicit adaptation. In contrast, we found that the color cue was largely ineffective for dual adaptation, except in a small subset of participants who appeared to use explicit strategies. Our study suggests that the previously inconclusive role of color cues in dual adaptation may be explained by differential contribution of explicit strategies across conditions.NEW & NOTEWORTHY We present evidence that learning of context-dependent dynamics proceeds via different processes depending on the type of sensory cue used to signal the context. Visual workspace location enabled learning different dynamics implicitly, presumably because it directly enters the dynamic state estimate. In contrast, a color cue was only successful where learners were apparently able to leverage explicit strategies to account for changed dynamics. This suggests a unification for the previously inconclusive role of color cues.

Assessing explicit strategies in force field adaptation.

In recent years, it has become increasingly clear that a number of learning processes are at play in visuomotor adaptation tasks. In addition to implicitly adapting to a perturbation, learners can develop explicit knowledge allowing them to select better actions in responding to it. Advances in visuomotor rotation experiments have underscored the important role of such "explicit learning" in shaping adaptation to kinematic perturbations. Yet, in adaptation to dynamic perturbations, its contribution has been largely overlooked. We therefore sought to approach the assessment of explicit learning in adaptation to dynamic perturbations, by developing two novel modifications of a force field experiment. First, we asked learners to abandon any cognitive strategy before selected force channel trials to expose consciously accessible parts of overall learning. Here, learners indeed reduced compensatory force compared with standard Catch channels. Second, we instructed a group of learners to mimic their right hand's adaptation by moving with their naïve left hand. While a control group displayed negligible left hand force compensation, the mimicking group reported forces that approximated right hand adaptation but appeared to under-report the velocity component of the force field in favor of a more position-based component. Our results highlight the viability of explicit learning as a potential contributor to force field adaptation, though the fraction of learning under participants' deliberate control on average remained considerably smaller than that of implicit learning, despite task conditions favoring explicit learning. The methods we employed provide a starting point for investigating the contribution of explicit strategies to force field adaptation.NEW & NOTEWORTHY While the contribution of explicit learning has been increasingly studied in visuomotor adaptation, its contribution to force field adaptation has not been studied extensively. We employed two novel methods to assay explicit learning in a force field adaptation task and found that learners can voluntarily control aspects of compensatory force production and manually report it with their untrained limb. This supports the general viability of the contribution of explicit learning also in force field adaptation.

Predictive error processing distinguishes between relevant and irrelevant errors after visuomotor learning.

Error processing is an important aspect of learning. The detection and online correction of an error as well as error-based adaptation of subsequent movements enables humans to improve behavior. For this improvement, it is necessary to differentiate between relevant and irrelevant errors. Behavioral adaptations are only reasonable when an error is attributed to one's own behavior and therefore regarded as relevant for subsequent adjustments, whereas irrelevant errors caused by unsystematic external influences should be disregarded. Here, we ask whether error predictions as indexed by the error-related negativity (Ne/ERN) can be used to differentiate relevant and irrelevant errors in movements with a complex visuomotor mapping. Using event-related potentials, we compared the neural activation between relevant (self-induced/internal) errors and irrelevant (externally manipulated) errors in a virtual goal-oriented throwing task. Results show that the Ne/ERN responds more strongly to self-induced errors, while the feedback-related negativity (FRN) more strongly correlates with externally manipulated errors. Moreover, subsequent behavioral adjustments were larger in the relevant compared to the irrelevant error trials. We conclude that predictive processes, marked by the Ne/ERN, can subserve error attribution in naturalistic, complex visuomotor tasks like throwing.

Plan-based generalization shapes local implicit adaptation to opposing visuomotor transformations.

The human ability to use different tools demonstrates our capability of forming and maintaining multiple, context-specific motor memories. Experimentally, this has been investigated in dual adaptation, where participants adjust their reaching movements to opposing visuomotor transformations. Adaptation in these paradigms occurs by distinct processes, such as strategies for each transformation or the implicit acquisition of distinct visuomotor mappings. Although distinct, transformation-dependent aftereffects have been interpreted as support for the latter, they could reflect adaptation of a single visuomotor map, which is locally adjusted in different regions of the workspace. Indeed, recent studies suggest that explicit aiming strategies direct where in the workspace implicit adaptation occurs, thus potentially serving as a cue to enable dual adaptation. Disentangling these possibilities is critical to understanding how humans acquire and maintain motor memories for different skills and tools. We therefore investigated generalization of explicit and implicit adaptation to untrained movement directions after participants practiced two opposing cursor rotations, which were associated with the visual display being presented in the left or right half of the screen. Whereas participants learned to compensate for opposing rotations by explicit strategies specific to this visual workspace cue, aftereffects were not cue sensitive. Instead, aftereffects displayed bimodal generalization patterns that appeared to reflect locally limited learning of both transformations. By varying target arrangements and instructions, we show that these patterns are consistent with implicit adaptation that generalizes locally around movement plans associated with opposing visuomotor transformations. Our findings show that strategies can shape implicit adaptation in a complex manner. NEW & NOTEWORTHY Visuomotor dual adaptation experiments have identified contextual cues that enable learning of separate visuomotor mappings, but the underlying representations of learning are unclear. We report that visual workspace separation as a contextual cue enables the compensation of opposing cursor rotations by a combination of explicit and implicit processes: Learners developed context-dependent explicit aiming strategies, whereas an implicit visuomotor map represented dual adaptation independent from arbitrary context cues by local adaptation around the explicit movement plan.

Brain negativity as an indicator of predictive error processing: the contribution of visual action effect monitoring.

The error (related) negativity (Ne/ERN) is an event-related potential in the electroencephalogram (EEG) correlating with error processing. Its conditions of appearance before terminal external error information suggest that the Ne/ERN is indicative of predictive processes in the evaluation of errors. The aim of the present study was to specifically examine the Ne/ERN in a complex motor task and to particularly rule out other explaining sources of the Ne/ERN aside from error prediction processes. To this end, we focused on the dependency of the Ne/ERN on visual monitoring about the action outcome after movement termination but before result feedback (action effect monitoring). Participants performed a semi-virtual throwing task by using a manipulandum to throw a virtual ball displayed on a computer screen to hit a target object. Visual feedback about the ball flying to the target was masked to prevent action effect monitoring. Participants received a static feedback about the action outcome (850 ms) after each trial. We found a significant negative deflection in the average EEG curves of the error trials peaking at ~250 ms after ball release, i.e., before error feedback. Furthermore, this Ne/ERN signal did not depend on visual ball-flight monitoring after release. We conclude that the Ne/ERN has the potential to indicate error prediction in motor tasks and that it exists even in the absence of action effect monitoring.NEW & NOTEWORTHY In this study, we are separating different kinds of possible contributors to an electroencephalogram (EEG) error correlate (Ne/ERN) in a throwing task. We tested the influence of action effect monitoring on the Ne/ERN amplitude in the EEG. We used a task that allows us to restrict movement correction and action effect monitoring and to control the onset of result feedback. We ascribe the Ne/ERN to predictive error processing where a conscious feeling of failure is not a prerequisite.

Feedback delay attenuates implicit but facilitates explicit adjustments to a visuomotor rotation.

We examined the effects of delaying terminal visual feedback on the relative contribution of explicit and implicit components of adaptation to a visuomotor rotation. Participants practiced a 30° rotation while receiving terminal visual feedback with either a short (0ms), medium (200ms), or long (1500ms) delay. Explicit and implicit adjustments were dissociated by a series of posttests. While overall adaptation did not differ significantly between groups, aftereffects progressively decreased with increasing feedback delay. Moreover, explicit knowledge of the rotation increased in both the medium and high delay groups relative to the short delay group, but did not differ between the former two. This finding of feedback delay differentially affecting implicit adjustments as indexed by aftereffects and conscious strategic corrections based on explicit knowledge of the transformation substantiates the importance of distinguishing implicit and explicit components of adaptation even with rotations of smaller size and emphasizes the need to consider time delays in the interpretation of adaptation experiments and potentially in the design of training environments.

Impact of task difficulty on gaze behavior in a sequential object manipulation task.

Task difficulty affects both gaze behavior and hand movements. Therefore, the present study aimed to investigate how task difficulty modulates gaze behaviour with respect to the balance between visually monitoring the ongoing action and prospectively collecting visual information about the future course of the ongoing action. For this, we examined sequences of reach and transport movements of water glasses that differed in task difficulty using glasses filled to different levels. Participants had to grasp water glasses with different filling levels (100, 94, 88, 82, and 76%) and transport them to a target. Subsequently, they had to grasp the next water glass and transport it to a target on the opposite side. Results showed significant differences in both gaze and movement kinematics for higher filling levels. However, there were no relevant differences between the 88, 82, and 76% filling levels. Results revealed a significant influence of task difficulty on the interaction between gaze and kinematics during transport and a strong influence of task difficulty on gaze during the release phase between different grasp-to-place movements. In summary, we found a movement and gaze pattern revealing an influence of task difficulty that was especially evident for the later phases of transport and release.

Perceiving animacy from shape.

Superordinate visual classification-for example, identifying an image as "animal," "plant," or "mineral"-is computationally challenging because radically different items (e.g., "octopus," "dog") must be grouped into a common class ("animal"). It is plausible that learning superordinate categories teaches us not only the membership of particular (familiar) items, but also general features that are shared across class members, aiding us in classifying novel (unfamiliar) items. Here, we investigated visual shape features associated with animate and inanimate classes. One group of participants viewed images of 75 unfamiliar and atypical items and provided separate ratings of how much each image looked like an animal, plant, and mineral. Results show systematic tradeoffs between the ratings, indicating a class-like organization of items. A second group rated each image in terms of 22 midlevel shape features (e.g., "symmetrical," "curved"). The results confirm that superordinate classes are associated with particular shape features (e.g., "animals" generally have high "symmetry" ratings). Moreover, linear discriminant analysis based on the 22-D feature vectors predicts the perceived classes approximately as well as the ground truth classification. This suggests that a generic set of midlevel visual shape features forms the basis for superordinate classification of novel objects along the animacy continuum.

Violating instructed human agency: An fMRI study on ocular tracking of biological and nonbiological motion stimuli.

Previous studies have shown that beliefs about the human origin of a stimulus are capable of modulating the coupling of perception and action. Such beliefs can be based on top-down recognition of the identity of an actor or bottom-up observation of the behavior of the stimulus. Instructed human agency has been shown to lead to superior tracking performance of a moving dot as compared to instructed computer agency, especially when the dot followed a biological velocity profile and thus matched the predicted movement, whereas a violation of instructed human agency by a nonbiological dot motion impaired oculomotor tracking (Zwickel et al., 2012). This suggests that the instructed agency biases the selection of predictive models on the movement trajectory of the dot motion. The aim of the present fMRI study was to examine the neural correlates of top-down and bottom-up modulations of perception-action couplings by manipulating the instructed agency (human action vs. computer-generated action) and the observable behavior of the stimulus (biological vs. nonbiological velocity profile). To this end, participants performed an oculomotor tracking task in an MRI environment. Oculomotor tracking activated areas of the eye movement network. A right-hemisphere occipito-temporal cluster comprising the motion-sensitive area V5 showed a preference for the biological as compared to the nonbiological velocity profile. Importantly, a mismatch between instructed human agency and a nonbiological velocity profile primarily activated medial-frontal areas comprising the frontal pole, the paracingulate gyrus, and the anterior cingulate gyrus, as well as the cerebellum and the supplementary eye field as part of the eye movement network. This mismatch effect was specific to the instructed human agency and did not occur in conditions with a mismatch between instructed computer agency and a biological velocity profile. Our results support the hypothesis that humans activate a specific predictive model for biological movements based on their own motor expertise. A violation of this predictive model causes costs as the movement needs to be corrected in accordance with incoming (nonbiological) sensory information.

Explicit and implicit components of visuo-motor adaptation: An analysis of individual differences.

Adaptation to visuo-motor rotations embraces implicit and explicit components. We contrast this two-component model with a three-component model by means of an individual-differences approach. Adaptive changes were tested under four conditions: (1) closed-loop test, presence of the rotation cued (initial adaptive shift), (2) open-loop test, presence of the rotation cued (adaptive shift), (3) open-loop test, absence of the rotation cued (after-effect), (4) test of explicit knowledge (explicit shift). After-effects and explicit shifts were uncorrelated. After regression on after-effects and explicit shifts, the residuals of the initial adaptive shifts and the adaptive shifts remained correlated, suggesting an additional implicit component of adaptation found only in the cued presence of the visuo-motor rotation. The two implicit components are consistent with the distinction between a change of the body schema giving rise to after-effects, and the development of an internal model of a tool that is applied only when the transformation is present.

Adaptation of motor control strategies to environmental cues in a pursuit-tracking task.

Visually guided tracking paradigms can provide insight into the adaptability of motor control strategies. We argue that the question of whether a sensorimotor process, or its absence, is beneficial or detrimental for performance is one that needs to be answered relative to the sensory cues available in the environment and the given task constraints. In this paper, we describe how and when environmental cues have task-dependent benefits. We used a new pursuit-tracking paradigm and added, removed, or replaced cues within the same tracking task in either predictable or unpredictable environments to investigate the use and adaptability of different control strategies. Participants were invited to perform a tracking task over six blocks in six different conditions. Compared to a condition where both target and control cursor (cues) were visible, performance was maintained when a cue was added and decreased when a cue was removed. Our results show that participants only learned to use new cues if the old one was removed and the replacement was valid. This means that the sensorimotor system adapts only if forced to do so instead of constantly exploiting optimization strategies.

Acting in Temporal Contexts: On the Behavioral and Neurophysiological Consequences of Feedback Delays.

Feedback on success or failure is critical to increase rewards through behavioral adaptation or learning of dependencies from trial and error. Learning from reward feedback is thereby treated as embedded in a reinforcement learning framework. Due to temporal discounting of reward, learning in this framework is suspected to be vulnerable to feedback delay. Together, investigations of reinforcement learning in learned decision making tasks show that performance and learning impairments due to feedback delay vary as a function of task type. Performance in tasks that require implicit processing is affected by the delayed availability of feedback compared to tasks that can be accomplished with explicit processing. At the same time, the feedback related negativity, an event related potential component in the electroencephalogram that is associated with feedback processing, is affected by feedback delay similarly independent of task type. With the idea of fully implicit or explicit processing as opposite endpoints of a continuum of reciprocal shares of the implicit and explicit processing systems with feedback delay as the determinant of where on this continuum processing can be located, a common explanatory approach of both, behavioral and electrophysiological findings, is suggested.

Can Stephen Curry really know?-Conscious access to outcome prediction of motor actions.

The NBA player Stephen Curry has a habit of turning away from the basket right after taking three-point shots even before the ball reaches the basket, suggesting that he can reliably predict whether the just released shot will hit or not. In order to use this "knowledge" to deliberately decide which action to take next, Stephen Curry needs conscious access to the results of internal processes of outcome prediction and valuation. In general, computational simulations and empirical data suggest that the quality of such internal predictions is related to motor skill level. Whether the results of internal predictions can reliably be consciously accessed, however, is less clear. In the current study, 30 participants each practiced a virtual goal-oriented throwing task for 1000 trials. Every second trial, they were required to verbally predict the success of the current throw. Results showed that on average, verbal prediction accuracy was above an individually computed chance level, taking into account individual success rates and response strategies. Furthermore, prediction accuracy was related to task skill level. Participants with better performances predicted the success of their throws more accurately than participants with poorer performances. For the poorer performing individuals, movement execution was negatively affected when the verbalized predictions were required. They also showed no noticeable modulation of speech characteristics (response latency) for correct and incorrect predictions as observed in the high performers.

Additional cognitive load decreases performance but not adaptation to a visuomotor transformation.

Dual-task paradigms are procedures for investigating interference with two tasks performed simultaneously. Studies that previously addressed dual-task paradigms within a visuomotor reaching task yielded mixed results. While some of the studies found evidence of cognitive interference, called dual-task costs, other studies did not. We assume that dual-task costs only manifest themselves within the explicit component of adaptation, as it involves cognitive resources for processing. We suspect the divergent findings to be due to the lack of differentiation between the explicit and implicit component. In this study, we aimed to investigate how a cognitive secondary task affects visuomotor adaptation overall and its different components, both during and after adaptation. In a series of posttests, we examined the explicit and implicit components separately. Eighty participants performed a center-outward reaching movement with a 30° cursor perturbation. Participants were either assigned to a single task group (ST) or a dual-task group (DT) with an additional auditory 1-back task. To further enhance our predicted effect of dual-task interference on the explicit component, we added a visual feedback delay condition to both groups (ST/DTDEL). In the other condition, participants received visual feedback immediately after movement termination (ST/DTNoDEL). While there were clear dual-task costs during the practice phase, there were no dual-task effects on any of the posttest measures. On one hand, our findings suggest that dual-task costs in visuomotor adaptation tasks can occur with sufficient cognitive demand, and on the other hand, that cognitive constraints may affect motor performance but not necessarily motor adaptation.

Relevance of Predictive and Postdictive Error Information in the Course of Motor Learning.

The prediction of the sensory consequences of physical movements is a fundamental feature of the human brain. This function is attributed to a forward model, which generates predictions based on sensory and efferent information. The neural processes underlying such predictions have been studied using the error-related negativity (ERN) as a fronto-central event-related potential in electroencephalogram (EEG) tracings. In this experiment, 16 participants practiced a novel motor task for 4000 trials over ten sessions. Neural correlates of error processing were recorded in sessions one, five, and ten. Along with significant improvements in task performance, the ERN amplitude increased over the sessions. Simultaneously, the feedback-related negativity (FRN), a neural marker corresponding to the processing of movement-outcome feedback, attenuated with learning. The findings suggest that early in learning, the motor control system relies more on information from external feedback about terminal outcome. With increasing task performance, the forward model is able to generate more accurate outcome predictions, which, as a result, increasingly contributes to error processing. The data also suggests a complementary relationship between the ERN and the FRN over motor learning.

Generalization of implicit and explicit adjustments to visuomotor rotations across the workspace in younger and older adults.

We examined the generalization of adjustment to a visuomotor rotation across the workspace in younger and older adults. Participants practiced in the right workspace with a single target direction and were tested in both the right and left workspace with eight different target directions. A set of tests served to identify implicit and explicit components of adjustment. Explicit, but not implicit, components were stronger at younger than at older adult age. Explicit components generalized across all target directions, whereas implicit components were restricted to the target direction during practice and clockwise adjacent ones. Generalization to the contralateral workspace was found only for explicit components of adjustment. These findings expand the list of functional differences between implicit and explicit components of adjustment to visuomotor transformations.

Prolonged response time helps eliminate residual errors in visuomotor adaptation.

One persistent curiosity in visuomotor adaptation tasks is that participants often do not reach maximal performance. This incomplete asymptote has been explained as a consequence of obligatory computations within the implicit adaptation system, such as an equilibrium between learning and forgetting. A body of recent work has shown that in standard adaptation tasks, cognitive strategies operate alongside implicit learning. We reasoned that incomplete learning in adaptation tasks may primarily reflect a speed-accuracy tradeoff on time-consuming motor planning. Across three experiments, we find evidence supporting this hypothesis, showing that hastened motor planning may primarily lead to under-compensation. When an obligatory waiting period was administered before movement start, participants were able to fully counteract imposed perturbations (Experiment 1). Inserting the same delay between trials - rather than during movement planning - did not induce full compensation, suggesting that the motor planning interval influences the learning asymptote (Experiment 2). In the last experiment (Experiment 3), we asked participants to continuously report their movement intent. We show that emphasizing explicit re-aiming strategies (and concomitantly increasing planning time) also lead to complete asymptotic learning. Findings from all experiments support the hypothesis that incomplete adaptation is, in part, the result of an intrinsic speed-accuracy tradeoff, perhaps related to cognitive strategies that require parametric attentional reorienting from the visual target to the goal.