Perception of the dynamic visual vertical during sinusoidal linear motion.

The vestibular system provides information for spatial orientation. However, this information is ambiguous: because the otoliths sense the gravitoinertial force, they cannot distinguish gravitational and inertial components. As a consequence, prolonged linear acceleration of the head can be interpreted as tilt, referred to as the somatogravic effect. Previous modeling work suggests that the brain disambiguates the otolith signal according to the rules of Bayesian inference, combining noisy canal cues with the a priori assumption that prolonged linear accelerations are unlikely. Within this modeling framework the noise of the vestibular signals affects the dynamic characteristics of the tilt percept during linear whole-body motion. To test this prediction, we devised a novel paradigm to psychometrically characterize the dynamic visual vertical-as a proxy for the tilt percept-during passive sinusoidal linear motion along the interaural axis (0.33 Hz motion frequency, 1.75 m/s2 peak acceleration, 80 cm displacement). While subjects (n=10) kept fixation on a central body-fixed light, a line was briefly flashed (5 ms) at different phases of the motion, the orientation of which had to be judged relative to gravity. Consistent with the model's prediction, subjects showed a phase-dependent modulation of the dynamic visual vertical, with a subject-specific phase shift with respect to the imposed acceleration signal. The magnitude of this modulation was smaller than predicted, suggesting a contribution of nonvestibular signals to the dynamic visual vertical. Despite their dampening effect, our findings may point to a link between the noise components in the vestibular system and the characteristics of dynamic visual vertical.NEW & NOTEWORTHY A fundamental question in neuroscience is how the brain processes vestibular signals to infer the orientation of the body and objects in space. We show that, under sinusoidal linear motion, systematic error patterns appear in the disambiguation of linear acceleration and spatial orientation. We discuss the dynamics of these illusory percepts in terms of a dynamic Bayesian model that combines uncertainty in the vestibular signals with priors based on the natural statistics of head motion.

Decisions in motion: vestibular contributions to saccadic target selection.

The natural world continuously presents us with many opportunities for action, and thus a process of target selection must precede action execution. While there has been considerable progress in understanding target selection in stationary environments, little is known about target selection when we are in motion. Here we investigated the effect of self-motion signals on saccadic target selection in a dynamic environment. Human subjects were sinusoidally translated (f = 0.6 Hz, 30-cm peak-to-peak displacement) along an interaural axis with a vestibular sled. During the motion two visual targets were presented asynchronously but equidistantly on either side of fixation. Subjects had to look at one of these targets as quickly as possible. With an adaptive approach, the time delay between these targets was adjusted until the subject selected both targets equally often. We determined this balanced time delay for different phases of the motion in order to distinguish the effects of body acceleration and velocity on saccadic target selection. Results show that acceleration (or position, as these are indistinguishable during sinusoidal motion), but not velocity, affects target selection for saccades. Subjects preferred to look at targets in the direction of the acceleration-the leftward target was preferred when the sled accelerated to the left, and vice versa. Saccadic reaction times mimicked this selection bias by being reliably shorter to targets in the direction of acceleration. Our results provide evidence that saccade target selection mechanisms are modulated by self-motion signals, which could be derived directly from the otolith system.

Parallel updating and weighting of multiple spatial maps for visual stability during whole body motion.

It is known that the brain uses multiple reference frames to code spatial information, including eye-centered and body-centered frames. When we move our body in space, these internal representations are no longer in register with external space, unless they are actively updated. Whether the brain updates multiple spatial representations in parallel, or whether it restricts its updating mechanisms to a single reference frame from which other representations are constructed, remains an open question. We developed an optimal integration model to simulate the updating of visual space across body motion in multiple or single reference frames. To test this model, we designed an experiment in which participants had to remember the location of a briefly presented target while being translated sideways. The behavioral responses were in agreement with a model that uses a combination of eye- and body-centered representations, weighted according to the reliability in which the target location is stored and updated in each reference frame. Our findings suggest that the brain simultaneously updates multiple spatial representations across body motion. Because both representations are kept in sync, they can be optimally combined to provide a more precise estimate of visual locations in space than based on single-frame updating mechanisms.

Parallax-sensitive remapping of visual space in occipito-parietal alpha-band activity during whole-body motion.

Despite the constantly changing retinal image due to eye, head, and body movements, we are able to maintain a stable representation of the visual environment. Various studies on retinal image shifts caused by saccades have suggested that occipital and parietal areas correct for these perturbations by a gaze-centered remapping of the neural image. However, such a uniform, rotational, remapping mechanism cannot work during translations when objects shift on the retina in a more complex, depth-dependent fashion due to motion parallax. Here we tested whether the brain's activity patterns show parallax-sensitive remapping of remembered visual space during whole-body motion. Under continuous recording of electroencephalography (EEG), we passively translated human subjects while they had to remember the location of a world-fixed visual target, briefly presented in front of or behind the eyes' fixation point prior to the motion. Using a psychometric approach we assessed the quality of the memory update, which had to be made based on vestibular feedback and other extraretinal motion cues. All subjects showed a variable amount of parallax-sensitive updating errors, i.e., the direction of the errors depended on the depth of the target relative to fixation. The EEG recordings show a neural correlate of this parallax-sensitive remapping in the alpha-band power at occipito-parietal electrodes. At parietal electrodes, the strength of these alpha-band modulations correlated significantly with updating performance. These results suggest that alpha-band oscillatory activity reflects the time-varying updating of gaze-centered spatial information during parallax-sensitive remapping during whole-body motion.

Generalization and transfer of contextual cues in motor learning.

We continuously adapt our movements in daily life, forming new internal models whenever necessary and updating existing ones. Recent work has suggested that this flexibility is enabled via sensorimotor cues, serving to access the correct internal model whenever necessary and keeping new models apart from previous ones. While research to date has mainly focused on identifying the nature of such cue representations, here we investigated whether and how these cue representations generalize, interfere, and transfer within and across effector systems. Subjects were trained to make two-stage reaching movements: a premovement that served as a cue, followed by a targeted movement that was perturbed by one of two opposite curl force fields. The direction of the premovement was uniquely coupled to the direction of the ensuing force field, enabling simultaneous learning of the two respective internal models. After training, generalization of the two premovement cues' representations was tested at untrained premovement directions, within both the trained and untrained hand. We show that the individual premovement representations generalize in a Gaussian-like pattern around the trained premovement direction. When the force fields are of unequal strengths, the cue-dependent generalization skews toward the strongest field. Furthermore, generalization patterns transfer to the nontrained hand, in an extrinsic reference frame. We conclude that contextual cues do not serve as discrete switches between multiple internal models. Instead, their generalization suggests a weighted contribution of the associated internal models based on the angular separation from the trained cues to the net motor output.

Functional magnetic resonance imaging adaptation reveals the cortical networks for processing grasp-relevant object properties.

Grasping behaviors require the selection of grasp-relevant object dimensions, independent of overall object size. Previous neuroimaging studies found that the intraparietal cortex processes object size, but it is unknown whether the graspable dimension (i.e., grasp axis between selected points on the object) or the overall size of objects triggers activation in that region. We used functional magnetic resonance imaging adaptation to investigate human brain areas involved in processing the grasp-relevant dimension of real 3-dimensional objects in grasping and viewing tasks. Trials consisted of 2 sequential stimuli in which the object's grasp-relevant dimension, its global size, or both were novel or repeated. We found that calcarine and extrastriate visual areas adapted to object size regardless of the grasp-relevant dimension during viewing tasks. In contrast, the superior parietal occipital cortex (SPOC) and lateral occipital complex of the left hemisphere adapted to the grasp-relevant dimension regardless of object size and task. Finally, the dorsal premotor cortex adapted to the grasp-relevant dimension in grasping, but not in viewing, tasks, suggesting that motor processing was complete at this stage. Taken together, our results provide a complete cortical circuit for progressive transformation of general object properties into grasp-related responses.

Reorganization of oscillatory activity in human parietal cortex during spatial updating.

Single-neuron recordings have shown that the posterior parietal cortex (PPC) processes spatial information in many frames of reference, including gaze-centered, head-centered, body-centered, and intermediate coding frames. At the population level, rhythmic neuronal synchronization may provide a mechanism by which PPC could selectively emphasize the task-relevant reference frame in spatial processing. Using magnetoencephalography, we tested this hypothesis by studying the modulations in oscillatory activity in a spatial updating task. Human subjects had to remember the location of a target, briefly flashed left or right of central fixation. Next, they refixated and then, after a further memory delay, made a saccade to the memorized target location. We observed gamma-band (>40 Hz) synchronization and alpha-band (8-12 Hz) desychronization in contralateral occipital and parietal areas, both showing updating in a gaze-centered reference frame but with fast and slow time courses, respectively. Furthermore, after updating, ipsilateral areas showed less alpha desynchronization when they had been contralateral to the target before updating. Taken together, our results suggest that power in the gamma band is instantly reorganized to encode task-relevant visuomotor space in a gaze-centered reference frame, while power in the alpha band reflects a regulatory mechanism actively facilitating the gating of the saccade target and inhibiting the original stimulus representation.

Vestibular benefits to task savings in motor adaptation.

In everyday life, we seamlessly adapt our movements and consolidate them to multiple behavioral contexts. This natural flexibility seems to be contingent on the presence of movement-related sensorimotor cues and cannot be reproduced when static visual or haptic cues are given to signify different behavioral contexts. So far, only sensorimotor cues that dissociate the sensorimotor plans prior to force field exposure have been successful in learning two opposing perturbations. Here we show that vestibular cues, which are only available during the perturbation, improve the formation and recall of multiple control strategies. We exposed subjects to inertial forces by accelerating them laterally on a vestibular platform. The coupling between reaching movement (forward-backward) and acceleration direction (leftward-rightward) switched every 160 trials, resulting in two opposite force environments. When exposed for a second time to the same environment, with the opposite environment in between, subjects showed retention resulting in an ∼3 times faster adaptation rate compared with the first exposure. Our results suggest that vestibular cues provide contextual information throughout the reach, which is used to facilitate independent learning and recall of multiple motor memories. Vestibular cues provide feedback about the underlying cause of reach errors, thereby disambiguating the various task environments and reducing interference of motor memories.

Writer's cramp: increased dorsal premotor activity during intended writing.

Simple writer's cramp (WC) is a task-specific form of dystonia, characterized by abnormal movements and postures of the hand during writing. It is extremely task-specific, since dystonic symptoms can occur when a patient uses a pencil for writing, but not when it is used for sharpening. Maladaptive plasticity, loss of inhibition, and abnormal sensory processing are important pathophysiological elements of WC. However, it remains unclear how those elements can account for its task-specificity. We used fMRI to isolate cerebral alterations associated with the task-specificity of simple WC. Subjects (13 simple WC patients, 20 matched controls) imagined grasping a pencil to either write with it or sharpen it. On each trial, we manipulated the pencil's position and the number of imagined movements, while monitoring variations in motor output with electromyography. We show that simple WC is characterized by abnormally increased activity in the dorsal premotor cortex (PMd) when imagined actions are specifically related to writing. This cerebral effect was independent from the known deficits in dystonia in generating focal motor output and in processing somatosensory feedback. This abnormal activity of the PMd suggests that the task-specific element of simple WC is primarily due to alterations at the planning level, in the computations that transform a desired action outcome into the motor commands leading to that action. These findings open the way for testing the therapeutic value of interventions that take into account the computational substrate of task-specificity in simple WC, e.g. modulations of PMd activity during the planning phase of writing.

Influence of panoramic cues during prolonged roll-tilt adaptation on the percept of vertical.

The percept of vertical, which mainly relies on vestibular and visual cues, is known to be affected after sustained whole-body roll tilt, mostly at roll positions adjacent to the position of adaptation. Here we ask whether the viewing of panoramic visual cues during the adaptation further influences the percept of the visual vertical. Participants were rotated in the frontal plane to a 90° clockwise tilt position, which was maintained for 4-minutes. During this period, the subject was either kept in darkness, or viewed panoramic pictures that were either veridical (aligned with gravity) or oriented along the body longitudinal axis. Errors of the subsequent subjective visual vertical (SVV), measured at various tilt angles, showed that the adaptation effect of panoramic cues is local, i.e. for a narrow range of tilts in the direction of the adaptation angle. This distortion was found irrespective of the orientation of the panoramic cues. We conclude that sustained exposure to panoramic and vestibular cues does not adapt the subsequent percept of vertical to the direction of the panoramic cue. Rather, our results suggest that sustained panoramic cues affect the SVV by an indirect effect on head orientation, with a 90° periodicity, that interacts with a vestibular cue to determine the percept of vertical.

Reference frames for reach planning in human parietofrontal cortex.

To plan a reaching movement, the brain must integrate information about the spatial goal of the reach with positional information about the selected hand. Recent monkey neurophysiological evidence suggests that a mixture of reference frames is involved in this process. Here, using 3T functional magnetic resonance imaging (fMRI), we tested the role of gaze-centered and body-centered reference frames in reach planning in the human brain. Fourteen human subjects planned and executed arm movements to memorized visual targets, while hand starting position and gaze direction were monitored and varied on a trial-by-trial basis. We further introduced a variable delay between target presentation and movement onset to dissociate cerebral preparatory activity from stimulus- and movement-related responses. By varying the position of the target and hand relative to the gaze line, we distinguished cerebral responses that increased for those movements requiring the integration of peripheral target and hand positions in a gaze-centered frame. Posterior parietal and dorsal premotor areas showed such gaze-centered integration effects. In regions closer to the primary motor cortex, body-centered hand position effects were found. These results suggest that, in humans, spatially contiguous neuronal populations operate in different frames of reference, supporting sensorimotor transformations according to gaze-centered or body-centered coordinates. The former appears suited for calculating a difference vector between target and hand location, whereas the latter may be related to the implementation of a joint-based motor command.

Accuracy-precision trade-off in visual orientation constancy.

Using the subjective visual vertical task (SVV), previous investigations on the maintenance of visual orientation constancy during lateral tilt have found two opposite bias effects in different tilt ranges. The SVV typically shows accurate performance near upright but severe undercompensation at tilts beyond 60 deg (A-effect), frequently with slight overcompensation responses (E-effect) in between. Here we investigate whether a Bayesian spatial-perception model can account for this error pattern. The model interprets A- and E-effects as the drawback of a computational strategy, geared at maintaining visual stability with optimal precision at small tilt angles. In this study, we test whether these systematic errors can be seen as the consequence of a precision-accuracy trade-off when combining a veridical but noisy signal about eye orientation in space with the visual signal. To do so, we used a psychometric approach to assess both precision and accuracy of the SVV in eight subjects laterally tilted at 9 different tilt angles (-120 degrees to 120 degrees). Results show that SVV accuracy and precision worsened with tilt angle, according to a pattern that could be fitted quite adequately by the Bayesian model. We conclude that spatial vision essentially follows the rules of Bayes' optimal observer theory.

Alpha-band transcranial alternating current stimulation modulates precision, but not gain during whole-body spatial updating.

Spatial updating is essential to maintain an accurate representation of our visual environment when we move. A neural mechanism that contributes to this ability is called remapping: the transfer of visual information from neural populations that code a location before the motion to those that encode it after the motion. While there is ample evidence for neural remapping in conjunction with eye movements, only recent findings suggest a role of this mechanism for whole-body motion updating, based on the observation that alpha band (10Hz) activity is selectively reorganized during remapping. This study tested whether alpha oscillations directly contribute to whole-body motion updating using transcranial alternating current stimulation (tACS). In a double blind sham controlled design, healthy volunteers received 10Hz tACS at an intensity of 1mA over either the left or right posterior hemisphere during a whole-body motion updating task. Updating performance was assessed psychometrically and indices of gain and precision were obtained. No tACS-related effects on updating gain were found, irrespective of whether the remapping was across or within the hemispheres. In contrast, updating precision was enhanced when a target representation had to be internally remapped to the stimulated hemisphere, but not in other remapping conditions. Our observations suggest that alpha band oscillations do not directly affect the transfer of target representations during remapping, but increase the fidelity of the updated representation by attenuating interference of afferent information.

Flexible Visuomotor Associations in Touchscreen Control.

To move real objects, our hand needs to get in direct physical contact with the object. However, this is not necessarily the case when interacting with virtual objects, for example when displacing objects on tablets by swipe movements. Here, we performed two experiments to study the behavioral strategies of these movements, examining how visual information about the virtual object is mapped into a swipe that moves the object into a goal location. In the first experiment, we investigated how swiping behavior depends on whether objects were located within or outside the swiping workspace. Results show that participants do not start the swipe movement by placing their finger on the virtual object, as they do when reaching to real objects, but rather keep a systematic distance between the object location and the initial swipe location. This mismatch, which was experimentally imposed by placing the object outside the workspace, also occurred when the object was within the workspace. In the second experiment, we investigated which factors determine this mismatch by systematically manipulating the initial hand location, the location of the object and the location of the goal. Dimensionality reduction of the data showed that three factors are taken into account when participants choose the initial swipe location: the expected total movement distance, the distance between their finger on the screen and the object, and a preference not to cover the object. The weight given to each factor differed among individuals. These results delineate, for the first time, the flexibility of visuomotor associations in the virtual world.

Role of Alpha-Band Oscillations in Spatial Updating across Whole Body Motion.

When moving around in the world, we have to keep track of important locations in our surroundings. In this process, called spatial updating, we must estimate our body motion and correct representations of memorized spatial locations in accordance with this motion. While the behavioral characteristics of spatial updating across whole body motion have been studied in detail, its neural implementation lacks detailed study. Here we use electroencephalography (EEG) to distinguish various spectral components of this process. Subjects gazed at a central body-fixed point in otherwise complete darkness, while a target was briefly flashed, either left or right from this point. Subjects had to remember the location of this target as either moving along with the body or remaining fixed in the world while being translated sideways on a passive motion platform. After the motion, subjects had to indicate the remembered target location in the instructed reference frame using a mouse response. While the body motion, as detected by the vestibular system, should not affect the representation of body-fixed targets, it should interact with the representation of a world-centered target to update its location relative to the body. We show that the initial presentation of the visual target induced a reduction of alpha band power in contralateral parieto-occipital areas, which evolved to a sustained increase during the subsequent memory period. Motion of the body led to a reduction of alpha band power in central parietal areas extending to lateral parieto-temporal areas, irrespective of whether the targets had to be memorized relative to world or body. When updating a world-fixed target, its internal representation shifts hemispheres, only when subjects' behavioral responses suggested an update across the body midline. Our results suggest that parietal cortex is involved in both self-motion estimation and the selective application of this motion information to maintaining target locations as fixed in the world or fixed to the body.

Visuomotor transformations for eye-hand coordination.

In recent years the scientific community has come to appreciate that the early cortical representations for visually guided arm movements are probably coded in a visual frame, i.e. relative to retinal landmarks. While this scheme accounts for many behavioral and neurophysiological observations, it also poses certain problems for manual control. For example, how are these oculocentric representations updated across eye movements, and how are they then transformed into useful commands for accurate movements of the arm relative to the body? Also, since we have two eyes, which is used as the reference point in eye-hand alignment tasks like pointing? We show that patterns of errors in human pointing suggest that early oculocentric representations for arm movement are remapped relative to the gaze direction during each saccade. To then transform these oculocentric representations into useful commands for accurate movements of the arm relative to the body, the brain correctly incorporates the three-dimensional, rotary geometry of the eyes when interpreting retinal images. We also explore the possibility that the eye-hand coordination system uses a strategy like ocular dominance, but switches alignment between the left and right eye in order to maximize eye-hand coordination in the best field of view. Finally, we describe the influence of eye position on eye-hand alignment, and then consider how head orientation influences the linkage between oculocentric visual frames and bodycentric motor frames. These findings are framed in terms of our 'conversion-on-demand' model, which suggests a virtual representation of egocentric space, i.e. one in which only those representations selected for action are put through the complex visuomotor transformations required for interaction with actual objects in personal space.

The subjective vertical and the sense of self orientation during active body tilt.

Previous testing of the ability to set a luminous line to the direction of gravity in passively-tilted subjects, in darkness, has revealed a remarkable pattern of systematic errors at tilts beyond 60 degrees, as if body tilt is undercompensated or underestimated (Aubert or A-effect). We investigated whether these consistent deviations from orientation constancy can be avoided during active body tilt, where more potential cues about body tilt (e.g. proprioception and efference copy) are available. The effects of active body tilt on the subjective vertical and on the perception of self tilt were studied in six subjects. After adopting a laterally-tilted posture, while standing in a dark room, they indicated the subjective vertical by adjusting a visual line and gave their verbal estimate of head orientation, expressed on a clock scale. Head roll tilts covered the range from -150 degrees to +150 degrees. The subjective vertical results showed no sign of improvement. Actively-tilted subjects still exhibited the same pattern of systematic errors that characterised their performance during passive tilt. Random errors in this task showed a steep monotonic increase with tilt angle, as in earlier passive tilt experiments. By contrast, verbal head-tilt estimates in the active experiments showed a clear improvement and were now almost devoid of systematic errors, but the noise level remained high. Various models are discussed in an attempt to clarify how these task-related differences and the selective improvement of the self-tilt estimates in the active experiments may have come about.

Ocular kinematics and eye-hand coordination.

Eye-hand coordination is complicated by the fact that the eyes are constantly in motion relative to the head. This poses problems in interpreting the spatial information gathered from the retinas and using this to guide hand motion. In particular, eye-centered visual information must somehow be spatially updated across eye movements to be useful for future actions, and these representations must then be transformed into commands appropriate for arm motion. In this review, we present evidence that early visuomotor representations for arm movement are remapped relative to the gaze direction during each saccade. We find that this mechanism holds for targets in both far and near visual space. We then show how the brain incorporates the three-dimensional, rotary geometry of the eyes when interpreting retinal images and transforming these into commands for arm movement. Next, we explore the possibility that hand-eye alignment is optimized for the eye with the best field of view. Finally, we describe how head orientation influences the linkage between oculocentric visual frames and bodycentric motor frames. These findings are framed in terms of our 'conversion-on-demand' model, in which only those representations selected for action are put through the complex visuomotor transformations required for interaction with objects in personal space, thus providing a virtual on-line map of visuomotor space.

Saccadic updating of object orientation for grasping movements.

Reach and grasp movements are a fundamental part of our daily interactions with the environment. This spatially-guided behavior is often directed to memorized objects because of intervening eye movements that caused them to disappear from sight. How does the brain store and maintain the spatial representations of objects for future reach and grasp movements? We had subjects (n=8) make reach and two-digit grasp movements to memorized objects, briefly presented before an intervening saccade. Grasp errors, characterizing the spatial representation of object orientation, depended on current gaze position, with and without intervening saccade. This suggests that the orientation information of the object is coded and updated relative to gaze during intervening saccades, and that the grasp errors arose after the updating stage, during the later transformations involved in grasping. The pattern of reach errors also revealed a gaze-centered updating of object location, consistent with previous literature on updating of single-point targets. Furthermore, grasp and reach errors correlated strongly, but their relationship had a non-unity slope, which may suggest that the gaze-centered spatial updates were made in separate channels. Finally, the errors of the two digits were strongly correlated, supporting the notion that these were not controlled independently to form the grip in these experimental conditions. Taken together, our results suggest that the visuomotor system dynamically represents the short-term memory of location and orientation information for reach-and-grasp movements.