Multisensory perception depends on the reliability of the type of judgment.

The brain engages the processes of multisensory integration and recalibration to deal with discrepant multisensory signals. These processes consider the reliability of each sensory input, with the more reliable modality receiving the stronger weight. Sensory reliability is typically assessed via the variability of participants' judgments, yet these can be shaped by factors both external and internal to the nervous system. For example, motor noise and participant's dexterity with the specific response method contribute to judgment variability, and different response methods applied to the same stimuli can result in different estimates of sensory reliabilities. Here we ask how such variations in reliability induced by variations in the response method affect multisensory integration and sensory recalibration, as well as motor adaptation, in a visuomotor paradigm. Participants performed center-out hand movements and were asked to judge the position of the hand or rotated visual feedback at the movement end points. We manipulated the variability, and thus the reliability, of repeated judgments by asking participants to respond using either a visual or a proprioceptive matching procedure. We find that the relative weights of visual and proprioceptive signals, and thus the asymmetry of multisensory integration and recalibration, depend on the reliability modulated by the judgment method. Motor adaptation, in contrast, was insensitive to this manipulation. Hence, the outcome of multisensory binding is shaped by the noise introduced by sensorimotor processing, in line with perception and action being intertwined.NEW & NOTEWORTHY Our brain tends to combine multisensory signals based on their respective reliability. This reliability depends on sensory noise in the environment, noise in the nervous system, and, as we show here, variability induced by the specific judgment procedure.

Both stimulus-specific and configurational features of multiple visual stimuli shape the spatial ventriloquism effect.

Studies on multisensory perception often focus on simplistic conditions in which one single stimulus is presented per modality. Yet, in everyday life, we usually encounter multiple signals per modality. To understand how multiple signals within and across the senses are combined, we extended the classical audio-visual spatial ventriloquism paradigm to combine two visual stimuli with one sound. The individual visual stimuli presented in the same trial differed in their relative timing and spatial offsets to the sound, allowing us to contrast their individual and combined influence on sound localization judgements. We find that the ventriloquism bias is not dominated by a single visual stimulus but rather is shaped by the collective multisensory evidence. In particular, the contribution of an individual visual stimulus to the ventriloquism bias depends not only on its own relative spatio-temporal alignment to the sound but also the spatio-temporal alignment of the other visual stimulus. We propose that this pattern of multi-stimulus multisensory integration reflects the evolution of evidence for sensory causal relations during individual trials, calling for the need to extend established models of multisensory causal inference to more naturalistic conditions. Our data also suggest that this pattern of multisensory interactions extends to the ventriloquism aftereffect, a bias in sound localization observed in unisensory judgements following a multisensory stimulus.

Imagery practice of motor skills without conscious awareness?: a commentary to Frank et al.

Modifications of imagined sensory consequences will not benefit overt performance when they cannot be transformed into motor outflow that produces them. With physical practice, the acquisition of internal models of motor transformations is largely based on prediction errors that are absent in imagery practice. What can imagery practice nevertheless contribute to transformation learning? Explicit, strategic adjustments to novel transformations should be possible. This appears less likely for implicit adjustments. Are there variants of imagery practice that can produce adjustments without conscious awareness of the transformation and/or the resultant movement changes?

Short-term effects of visuomotor discrepancies on multisensory integration, proprioceptive recalibration, and motor adaptation.

Information about the position of our hand is provided by multisensory signals that are often not perfectly aligned. Discrepancies between the seen and felt hand position or its movement trajectory engage the processes of 1) multisensory integration, 2) sensory recalibration, and 3) motor adaptation, which adjust perception and behavioral responses to apparently discrepant signals. To foster our understanding of the coemergence of these three processes, we probed their short-term dependence on multisensory discrepancies in a visuomotor task that has served as a model for multisensory perception and motor control previously. We found that the well-established integration of discrepant visual and proprioceptive signals is tied to the immediate discrepancy and independent of the outcome of the integration of discrepant signals in immediately preceding trials. However, the strength of integration was context dependent, being stronger in an experiment featuring stimuli that covered a smaller range of visuomotor discrepancies (±15°) compared with one covering a larger range (±30°). Both sensory recalibration and motor adaptation for nonrepeated movement directions were absent after two bimodal trials with same or opposite visuomotor discrepancies. Hence our results suggest that short-term sensory recalibration and motor adaptation are not an obligatory consequence of the integration of preceding discrepant multisensory signals.NEW & NOTEWORTHY The functional relation between multisensory integration and recalibration remains debated. We here refute the notion that they coemerge in an obligatory manner and support the hypothesis that they serve distinct goals of perception.

Different time scales of common-cause evidence shape multisensory integration, recalibration and motor adaptation.

Perceptual coherence in the face of discrepant multisensory signals is achieved via the processes of multisensory integration, recalibration and sometimes motor adaptation. These supposedly operate on different time scales, with integration reducing immediate sensory discrepancies and recalibration and motor adaptation reflecting the cumulative influence of their recent history. Importantly, whether discrepant signals are bound during perception is guided by the brains' inference of whether they originate from a common cause. When combined, these two notions lead to the hypothesis that the time scales on which integration and recalibration (or motor adaptation) operate are associated with different time scales of evidence about a common cause underlying two signals. We tested this prediction in a well-established visuo-motor paradigm, in which human participants performed visually guided hand movements. The kinematic correlation between hand and cursor movements indicates their common origin, which allowed us to manipulate the common-cause evidence by titrating this correlation. Specifically, we dissociated hand and cursor signals during individual movements while preserving their correlation across the series of movement endpoints. Following our hypothesis, this manipulation reduced integration compared with a condition in which visual and proprioceptive signals were perfectly correlated. In contrast, recalibration and motor adaption were not affected by this manipulation. This supports the notion that multisensory integration and recalibration deal with sensory discrepancies on different time scales guided by common-cause evidence: Integration is prompted by local common-cause evidence and reduces immediate discrepancies, whereas recalibration and motor adaptation are prompted by global common-cause evidence and reduce persistent discrepancies.

From Psychologische Forschung to psychological research: a rough journey through a century.

Psychologische Forschung started as a journal "für Psychologie und ihre Grenzgebiete" and became strongly associated with the Berlin school of Gestalt psychology. Parallel to the fate of that school, the Journal was discontinued after 1938 and re-appeared only 1949. A number of years with variable and broad editorial boards and without a clear profile followed. In 1974 the Journal switched to English as the first German psychology journal and became Psychological Research. Gradually and without any abrupt changes-as indicated e.g. by analyses of citing and cited journals-the current profile as "An International Journal of Perception, Attention, Memory, and Action" was developed. During the last one to two decades the number of papers, the number of contributing countries, and the impact increase.

Explicit knowledge of sensory non-redundancy can reduce the strength of multisensory integration.

The brain integrates incoming sensory signals to a degree that depends on the signals' redundancy. Redundancy-which is commonly high when signals originate from a common physical object or event-is estimated by the brain from the signals' spatial and/or temporal correspondence. Here we tested whether verbally instructed knowledge of non-redundancy can also be used to reduce the strength of the sensory integration. We used a cursor-control task in which cursor motions in the frontoparallel plane were controlled by hand movements in the horizontal plane, yet with a small and randomly varying visuomotor rotation that created spatial discrepancies between hand and cursor positions. Consistent with previous studies, we found mutual biases in the hand and cursor position judgments, indicating partial sensory integration. The integration was reduced in strength, but not eliminated, after participants were verbally informed about the non-redundancy (i.e., the spatial discrepancies) in the hand and cursor positions. Comparisons with model predictions excluded confounding bottom-up effects of the non-redundancy instruction. Our findings thus show that participants have top-down control over the degree to which they integrate sensory information. Additionally, we found that the magnitude of this top-down modulatory capability is a reliable individual trait. A comparison between participants with and without video-gaming experience tentatively suggested a relation between top-down modulation of integration strength and attentional control.

A serial-position curve in high-performance darts: The effect of visuomotor calibration on throwing accuracy.

The aim of the present research was to test if the fine-tuning of skilled motor actions benefits from proximate previous actions via a visuomotor calibration process. In professional darts, each player cycles through different activities: three darts are thrown with a rather smooth sequence of movements, the darts are retrieved from the dartboard, the other player throws his or her darts and retrieves them, the next three darts are thrown, retrieved, etc. We hypothesized that these cycles give rise to a serial-position curve for the precision of darts as a result of a particular kind of warm-up decrement. Even though the interruptions of actually throwing darts are only in the order of seconds, walking away from the throw line should lead to a loss of fine-tuning of the calibration of movement parameters with respect to targets defined in the external frame of reference of the dartboard. For the players of the 2017 Professional Darts Corporation World Darts Championship (N = 36,168 scores) we confirmed that the first dart of a series of three is indeed less accurate than the subsequent two. This warm-up decrement is particularly pronounced for vertical errors, for which the relation to movement parameters is more complex than for horizontal errors. Fine-tuning of visuomotor calibration is a neglected facet of warm-up that is also important for various other sports such as tennis, basketball, handball, and football.

Visual and proprioceptive recalibrations after exposure to a visuomotor rotation.

Adaptation to a visuomotor rotation in a cursor-control task is accompanied by proprioceptive recalibration, whereas the existence of visual recalibration is uncertain and has even been doubted. In the present study, we tested both visual and proprioceptive recalibration; proprioceptive recalibration was not only assessed by means of psychophysical judgments of the perceived position of the hand, but also by an indirect procedure based on movement characteristics. Participants adapted to a gradually introduced visuomotor rotation of 30° by making center-out movements to remembered targets. In subsequent test trials, they made center-out movements without visual feedback or observed center-out motions of a cursor without moving the hand. In each test trial, they judged the endpoint of hand or cursor by matching the position of the hand or of a visual marker, respectively, moving along a semicircular path. This path ran through all possible endpoints of the center-out movements. We observed proprioceptive recalibration of 7.3° (3.1° with the indirect procedure) and a smaller, but significant, visual recalibration of 1.3°. Total recalibration of 8.6° was about half as strong as motor adaptation, the adaptive shift of the movement direction. The evidence of both proprioceptive and visual recalibration was obtained with a judgment procedure that suggests that recalibration is restricted to the type of movement performed during exposure to a visuomotor rotation. Consequently, identical physical positions of the hand can be perceived differently depending on how they have been reached, and similarly identical positions of a cursor on a monitor can be perceived differently.

Contrasting effects of adaptation to a visuomotor rotation on explicit and implicit measures of sensory coupling.

We previously investigated sensory coupling of the sensed positions of cursor and hand in a cursor-control task and found differential characteristics of implicit and explicit measures of the bias of sensed hand position toward the position of the cursor. The present study further tested whether adaptation to a visuomotor rotation differentially affects these two measures. Participants made center-out reaching movements to remembered targets while looking at a rotated feedback cursor. After sets of practice trials with constant (adaptation condition) or random (control condition) visuomotor rotations, test trials served to assess sensory coupling. In these trials, participants judged the position of the hand at the end of the center-out movement, and the deviation of these judgments from the physical hand positions served as explicit measure of the bias of sensed hand position toward the position of the cursor, whereas the implicit measure was based on the direction of the return movement. The results showed that inter-individual variability of explicitly assessed biases of sensed hand position toward the cursor position was less in the adaptation condition than in the control condition. Conversely, no such changes were observed for the implicit measure of the bias of sensed hand position, revealing contrasting effects of adaptation on the explicit and implicit measures. These results suggest that biases of explicitly sensed hand position reflect sensory coupling of neural representations that are altered by visuomotor adaptation. In contrast, biases of implicitly sensed hand position reflect sensory coupling of neural representations that are unaffected by adaptation.

The impact of anatomical and spatial distance between responses on response conflict.

Different features of objects can be associated with different responses, so that their concurrent presence results in conflict. The Simon effect is a prominent example of this type of response conflict. In two experiments, we ask whether it is modulated by the anatomical or spatial relation between responses. Predictions were derived from an extended variant of the leaky, competing accumulator (LCA) model proposed by Usher and McClelland (Psychological Review, 108, 550-592, 2001). The relation between responses was represented by the lateral-inhibition parameter of the model. For the anatomical distance between responses the expectations were largely confirmed, but not for spatial distance. First, the Simon effect was stronger when responses were performed with two fingers of the same hand than with different hands. Second, the Simon effect was larger only for responses with different hands at short reaction times and disappeared at long ones, whereas for responses with fingers of the same hand, the Simon effect was essentially the same for shorter and longer reaction times. This difference resulted in smaller variability of reaction times in noncorresponding than in corresponding conditions. The dependence of decision processes, as modelled by the LCA model, on the anatomical relation between responses supports the broad hypothesis that the accumulation of evidence on the state of the world is intricately linked with the activation of response codes, that is, the selection of the appropriate actions.

Sensory integration of movements and their visual effects is not enhanced by spatial proximity.

Spatial proximity enhances the sensory integration of exafferent position information, likely because it indicates whether the information comes from a single physical source. Does spatial proximity also affect the integration of position information regarding an action (here a hand movement) with that of its visual effect (here a cursor motion), that is, when the sensory information comes from physically distinct objects? In this study, participants made out-and-back hand movements whereby the outward movements were accompanied by corresponding cursor motions on a monitor. Their subsequent judgments of hand or cursor movement endpoints are typically biased toward each other, consistent with an underlying optimal integration mechanism. To study the effect of spatial proximity, we presented the hand and cursor either in orthogonal planes (horizontal and frontal, respectively) or we aligned them in the horizontal plane. We did not find the expected enhanced integration strength in the latter spatial condition. As a secondary question we asked whether spatial transformations required for the position judgments (i.e., horizontal to frontal or vice versa) could be the origin of previously observed suboptimal variances of the integrated hand and cursor position judgments. We found, however, that the suboptimality persisted when spatial transformations were omitted (i.e., with the hand and cursor in the same plane). Our findings thus clearly show that the integration of actions with their visual effects is, at least for cursor control, independent of spatial proximity.

Perceptual attraction in tool use: evidence for a reliability-based weighting mechanism.

Humans are well able to operate tools whereby their hand movement is linked, via a kinematic transformation, to a spatially distant object moving in a separate plane of motion. An everyday example is controlling a cursor on a computer monitor. Despite these separate reference frames, the perceived positions of the hand and the object were found to be biased toward each other. We propose that this perceptual attraction is based on the principles by which the brain integrates redundant sensory information of single objects or events, known as optimal multisensory integration. That is, 1) sensory information about the hand and the tool are weighted according to their relative reliability (i.e., inverse variances), and 2) the unisensory reliabilities sum up in the integrated estimate. We assessed whether perceptual attraction is consistent with optimal multisensory integration model predictions. We used a cursor-control tool-use task in which we manipulated the relative reliability of the unisensory hand and cursor position estimates. The perceptual biases shifted according to these relative reliabilities, with an additional bias due to contextual factors that were present in experiment 1 but not in experiment 2 The biased position judgments' variances were, however, systematically larger than the predicted optimal variances. Our findings suggest that the perceptual attraction in tool use results from a reliability-based weighting mechanism similar to optimal multisensory integration, but that certain boundary conditions for optimality might not be satisfied.NEW & NOTEWORTHY Kinematic tool use is associated with a perceptual attraction between the spatially separated hand and the effective part of the tool. We provide a formal account for this phenomenon, thereby showing that the process behind it is similar to optimal integration of sensory information relating to single objects.

Kinematic cross-correlation induces sensory integration across separate objects.

In a basic cursor-control task, the perceived positions of the hand and the cursor are biased towards each other. We recently found that this phenomenon conforms to the reliability-based weighting mechanism of optimal multisensory integration. This indicates that optimal integration is not restricted to sensory signals originating from a single source, as is the prevailing view, but that it also applies to separate objects that are connected by a kinematic relation (i.e. hand and cursor). In the current study, we examined which aspects of the kinematic relation are crucial for eliciting the sensory integration: (i) the cross-correlation between kinematic variables of the hand and cursor trajectories, and/or (ii) an internal model of the hand-cursor kinematic transformation. Participants made out-and-back movements from the centre of a semicircular workspace to its boundary, after which they judged the position where either their hand or the cursor hit the boundary. We analysed the position biases and found that the integration was strong in a condition with high kinematic correlations (a straight hand trajectory was mapped to a straight cursor trajectory), that it was significantly reduced for reduced kinematic correlations (a straight hand trajectory was transformed into a curved cursor trajectory) and that it was not affected by the inability to acquire an internal model of the kinematic transformation (i.e. by the trial-to-trial variability of the cursor curvature). These findings support the idea that correlations play a crucial role in multisensory integration irrespective of the number of sensory sources involved.

Financial incentives enhance adaptation to a sensorimotor transformation.

Adaptation to sensorimotor transformations has received much attention in recent years. However, the role of motivation and its relation to the implicit and explicit processes underlying adaptation has been neglected thus far. Here, we examine the influence of extrinsic motivation on adaptation to a visuomotor rotation by way of providing financial incentives for accurate movements. Participants in the experimental group "bonus" received a defined amount of money for high end-point accuracy in a visuomotor rotation task; participants in the control group "no bonus" did not receive a financial incentive. Results showed better overall adaptation to the visuomotor transformation in participants who were extrinsically motivated. However, there was no beneficial effect of financial incentives on the implicit component, as assessed by the after-effects, and on separately assessed explicit knowledge. These findings suggest that the positive influence of financial incentives on adaptation is due to a component which cannot be measured by after-effects or by our test of explicit knowledge. A likely candidate is model-free learning based on reward-prediction errors, which could be enhanced by the financial bonuses.

Explicit and implicit components of visuo-motor adaptation: An analysis of individual differences.

Adaptation to visuo-motor rotations embraces implicit and explicit components. We contrast this two-component model with a three-component model by means of an individual-differences approach. Adaptive changes were tested under four conditions: (1) closed-loop test, presence of the rotation cued (initial adaptive shift), (2) open-loop test, presence of the rotation cued (adaptive shift), (3) open-loop test, absence of the rotation cued (after-effect), (4) test of explicit knowledge (explicit shift). After-effects and explicit shifts were uncorrelated. After regression on after-effects and explicit shifts, the residuals of the initial adaptive shifts and the adaptive shifts remained correlated, suggesting an additional implicit component of adaptation found only in the cued presence of the visuo-motor rotation. The two implicit components are consistent with the distinction between a change of the body schema giving rise to after-effects, and the development of an internal model of a tool that is applied only when the transformation is present.

The coding of repetitions and alternations in action sequences: spatial or relational?

We used variants of the Simon task to investigate whether repetitions and alternations in short keypress sequences are represented by spatial or relational codes. With spatial coding, either absolute or relative location would be used for coding the second response in a sequence. With relational coding, the second response would be coded in terms of a non-spatial relation to the first one (e.g., left response-same response, for a repetition). In three experiments with different imperative stimuli, we compared Simon effects across three experimental conditions, a single-response condition, a response-repetition condition (each stimulus required two keypresses on the same side, e.g., left-left), and a response-alternation condition (each stimulus required a keypress on each side, e.g., left-right). When compared to the single-response condition, spatial coding of the second response should modulate the Simon effect (i.e., response conflict) in selecting the first response because spatial coding of the second response produces additional dimensional overlap of stimulus and the second-response code. We observed Simon effects in the response times of first responses in each condition, and they increased in the response-alternation condition, but not in the response-repetition condition. The findings suggest relational coding of response repetitions, and spatial coding of response alternations.

Haptic guidance interferes with learning to make movements at an angle to stimulus direction.

Haptic guidance has been shown to interfere with learning a novel visuo-motor rotation. Here, we ask whether this interference is specific to the learning of visuo-motor transformations or whether it is a more generalized phenomenon of learning spatial movement characteristics. Participants practiced to make movements at an angle of 75° relative to target directions with and without haptic path guidance. Test trials without guidance and without visual feedback revealed poorer performance after haptic guidance practice than in the no-guidance control group, even though during guided practice movement directions were more accurate than during unguided practice. The more precise directions during guided practice were associated with shorter movement latencies than in the control group, suggesting an incomplete rotation of the movement direction relative to the direction of the visual target. A detailed analysis of the first few millimeters of each movement revealed that the initial movement direction was indeed rotated too little, and the correct direction was adopted only gradually under the influence of haptic guidance. These findings support the perspective on motor learning as a process that is driven by consequent improvements of performance and that is compromised by factors that serve to improve performance even without learning. Haptic guidance is one such factor.

Motor learning with fading and growing haptic guidance.

Haptic guidance has been shown to have both facilitatory and interfering effects on motor learning. Interfering effects have been hypothesized to result from the particular dynamic environment, which supports a passive role of the learner, and they should be attenuated by fading guidance. Facilitatory effects, in particular for dynamic movement characteristics, have been hypothesized to result from the high-quality information provided by haptic demonstration. If haptic demonstration provides particularly precise information about target movements, the motor system's need for such information should more likely increase in the course of motor learning, in which case growing guidance should be more beneficial for learning. We contrasted fading and growing guidance in the course of learning a spatio-temporal motor pattern. To stimulate an active role of the learner, practice trials consisted of three phases, a visual demonstration of the target movement, a guided reproduction, and a reproduction without haptic guidance. Performance was assessed in terms of variable duration errors, relative-timing errors, variable path-length errors, and shape errors. Motor learning with growing and fading guidance turned out to be largely equivalent, so that the notion of an increasing optimal precision of haptic demonstrations, which matches a demand of increasingly precise information on the target movement, found no support. Duration errors declined only with fading, but not with growing guidance. Relative timing revealed a benefit of immediately preceding haptic demonstration, but learning was not different between the two practice protocols. This contrast between absolute and relative timing adds to other evidence according to which acquisition of these two aspects of motor timing involves different learning mechanisms. Whereas relative timing gained from immediately preceding haptic demonstration, but revealed no practice-related improvement in the presence of haptic guidance, the opposite pattern of results was found for the shape error. This finding is consistent with the claim that haptic demonstration is particularly efficient with respect to relative timing, but not with respect to spatial movement characteristics.

Movement paths in operating hand-held tools: tests of distal-shift hypotheses.

Extending the body with a tool could imply that characteristics of hand movements become characteristics of the movement of the effective part of the tool. Recent research suggests that such distal shifts are subject to boundary conditions. Here we propose the existence of three constraints: a strategy constraint, a constraint of movement characteristics, and a constraint of mode of control. We investigate their validity for the curvature of transverse movements aimed at a target while using a sliding first-order lever. Participants moved the tip of the effort arm of a real or virtual lever to control a cursor representing movements of the tip of the load arm of the lever on a monitor. With this tool, straight transverse hand movements are associated with concave curvature of the path of the tip of the tool. With terminal visual feedback and when targets were presented for the hand, hand paths were slightly concave in the absence of the dynamic transformation of the tool and slightly convex in its presence. When targets were presented for the tip of the lever, both the concave and convex curvatures of the hand paths became stronger. Finally, with continuous visual feedback of the tip of the lever, curvature of hand paths became convex and concave curvature of the paths of the tip of the lever was reduced. In addition, the effect of the dynamic transformation on curvature was attenuated. These findings support the notion that distal shifts are subject to at least the three proposed constraints.