Action-dependent processing of self-motion in parietal cortex of macaque monkeys.

Successful interaction with the environment requires the dissociation of self-induced from externally induced sensory stimulation. Temporal proximity of action and effect is hereby often used as an indicator of whether an observed event should be interpreted as a result of own actions or not. We tested how the delay between an action (press of a touch bar) and an effect (onset of simulated self-motion) influences the processing of visually simulated self-motion in the ventral intraparietal area (VIP) of macaque monkeys. We found that a delay between the action and the start of the self-motion stimulus led to a rise of activity above the baseline activity before motion onset in a subpopulation of 21% of the investigated neurons. In the responses to the stimulus, we found a significantly lower sustained activity when the press of a touch bar and the motion onset were contiguous compared to the condition when the motion onset was delayed. We speculate that this weak inhibitory effect might be part of a mechanism that sharpens the tuning of VIP neurons during self-induced motion and thus has the potential to increase the precision of heading information that is required to adjust the orientation of self-motion in everyday navigational tasks.NEW & NOTEWORTHY Neurons in macaque ventral intraparietal area (VIP) are responding to sensory stimulation related to self-motion, e.g. visual optic flow. Here, we found that self-motion induced activation depends on the sense of agency, i.e., it differed when optic flow was perceived as self- or externally induced. This demonstrates that area VIP is well suited for study of the interplay between active behavior and sensory processing during self-motion.

Integration of visual and tactile information in reproduction of traveled distance.

In the natural world, self-motion always stimulates several different sensory modalities. Here we investigated the interplay between a visual optic flow stimulus simulating self-motion and a tactile stimulus (air flow resulting from self-motion) while human observers were engaged in a distance reproduction task. We found that adding congruent tactile information (i.e., speed of the air flow and speed of visual motion are directly proportional) to the visual information significantly improves the precision of the actively reproduced distances. This improvement, however, was smaller than predicted for an optimal integration of visual and tactile information. In contrast, incongruent tactile information (i.e., speed of the air flow and speed of visual motion are inversely proportional) did not improve subjects' precision indicating that incongruent tactile information and visual information were not integrated. One possible interpretation of the results is a link to properties of neurons in the ventral intraparietal area that have been shown to have spatially and action-congruent receptive fields for visual and tactile stimuli.NEW & NOTEWORTHY This study shows that tactile and visual information can be integrated to improve the estimates of the parameters of self-motion. This, however, happens only if the two sources of information are congruent-as they are in a natural environment. In contrast, an incongruent tactile stimulus is still used as a source of information about self-motion but it is not integrated with visual information.

Perisaccadic perception of visual space in people with schizophrenia.

Corollary discharge signals are found in the nervous systems of many animals, where they serve a large variety of functions related to the integration of sensory and motor signals. In humans, an important corollary discharge signal is generated by oculomotor structures and communicated to sensory systems in concert with the execution of each saccade. This signal is thought to serve a number of purposes related to the maintenance of accurate visual perception. The properties of the oculomotor corollary discharge can be probed by asking subjects to localize stimuli that are flashed briefly around the time of a saccade. The results of such experiments typically reveal large errors in localization. Here, we have exploited these well-known psychophysical effects to assess the potential dysfunction of corollary discharge signals in people with schizophrenia. In a standard perisaccadic localization task, we found that, compared with controls, patients with schizophrenia exhibited larger errors in localizing visual stimuli. The pattern of errors could be modeled as an overdamped corollary discharge signal that encodes instantaneous eye position. The dynamics of this signal predicted symptom severity among patients, suggesting a possible mechanistic basis for widely observed behavioral manifestations of schizophrenia.

Spatiotemporal structure of visual receptive fields in macaque superior colliculus.

Saccades are useful for directing the high-acuity fovea to visual targets that are of behavioral relevance. The selection of visual targets for eye movements involves the superior colliculus (SC), where many neurons respond to visual stimuli. Many of these neurons are also activated before and during saccades of specific directions and amplitudes. Although the role of the SC in controlling eye movements has been thoroughly examined, far less is known about the nature of the visual responses in this area. We have, therefore, recorded from neurons in the intermediate layers of the macaque SC, while using a sparse-noise mapping procedure to obtain a detailed characterization of the spatiotemporal structure of visual receptive fields. We find that SC responses to flashed visual stimuli start roughly 50 ms after the onset of the stimulus and last for on average ~70 ms. About 50% of these neurons are strongly suppressed by visual stimuli flashed at certain locations flanking the excitatory center, and the spatiotemporal pattern of suppression exerts a predictable influence on the timing of saccades. This suppression may, therefore, contribute to the filtering of distractor stimuli during target selection. We also find that saccades affect the processing of visual stimuli by SC neurons in a manner that is quite similar to the saccadic suppression and postsaccadic enhancement that has been observed in the cortex and in perception. However, in contrast to what has been observed in the cortex, decreased visual sensitivity was generally associated with increased firing rates, while increased sensitivity was associated with decreased firing rates. Overall, these results suggest that the processing of visual stimuli by SC receptive fields can influence oculomotor behavior and that oculomotor signals originating in the SC can shape perisaccadic visual perception.

Influence of Tactile Flow on Visual Heading Perception.

The integration of information from different sensory modalities is crucial for successful navigation through an environment. Among others, self-motion induces distinct optic flow patterns on the retina, vestibular signals and tactile flow, which contribute to determine traveled distance (path integration) or movement direction (heading). While the processing of combined visual-vestibular information is subject to a growing body of literature, the processing of visuo-tactile signals in the context of self-motion has received comparatively little attention. Here, we investigated whether visual heading perception is influenced by behaviorally irrelevant tactile flow. In the visual modality, we simulated an observer's self-motion across a horizontal ground plane (optic flow). Tactile self-motion stimuli were delivered by air flow from head-mounted nozzles (tactile flow). In blocks of trials, we presented only visual or tactile stimuli and subjects had to report their perceived heading. In another block of trials, tactile and visual stimuli were presented simultaneously, with the tactile flow within ±40° of the visual heading (bimodal condition). Here, importantly, participants had to report their perceived visual heading. Perceived self-motion direction in all conditions revealed a centripetal bias, i.e., heading directions were perceived as compressed toward straight ahead. In the bimodal condition, we found a small but systematic influence of task-irrelevant tactile flow on visually perceived headings as function of their directional offset. We conclude that tactile flow is more tightly linked to self-motion perception than previously thought.

Context dependence of receptive field remapping in superior colliculus.

Our perception of the positions of objects in our surroundings is surprisingly unaffected by movements of the eyes, head, and body. This suggests that the brain has a mechanism for maintaining perceptual stability, based either on the spatial relationships among visible objects or internal copies of its own motor commands. Strong evidence for the latter mechanism comes from the remapping of visual receptive fields that occurs around the time of a saccade. Remapping occurs when a single neuron responds to visual stimuli placed presaccadically in the spatial location that will be occupied by its receptive field after the completion of a saccade. Although evidence for remapping has been found in many brain areas, relatively little is known about how it interacts with sensory context. This interaction is important for understanding perceptual stability more generally, as the brain may rely on extraretinal signals or visual signals to different degrees in different contexts. Here, we have studied the interaction between visual stimulation and remapping by recording from single neurons in the superior colliculus of the macaque monkey, using several different visual stimulus conditions. We find that remapping responses are highly sensitive to low-level visual signals, with the overall luminance of the visual background exerting a particularly powerful influence. Specifically, although remapping was fairly common in complete darkness, such responses were usually decreased or abolished in the presence of modest background illumination. Thus the brain might make use of a strategy that emphasizes visual landmarks over extraretinal signals whenever the former are available.

Perceptual compression of visual space during eye-head gaze shifts.

In primates, inspection of a visual scene is typically interrupted by frequent gaze shifts, occurring at an average rate of three to five times per second. Perceptually, these gaze shifts are accompanied by a compression of visual space toward the saccade target, which may be attributed to an oculomotor signal that transiently influences visual processing. While previous studies of compression have focused exclusively on saccadic eye movements made with the head artificially immobilized, many brain structures involved in saccade generation also encode combined eye-head gaze shifts. Thus, in order to understand the interaction between gaze motor and visual signals, we studied perception during eye-head gaze shifts and found a powerful compression of visual space that was spatially directed toward the intended gaze (and not the eye movement) target location. This perceptual compression was nearly constant in duration across gaze shift amplitudes, suggesting that the signal that triggers compression is largely independent of the size and kinematics of the gaze shift. The spatial pattern of results could be captured by a model that involves interactions, on a logarithmic map of visual space, between two loci of neural activity that encode the gaze shift vector and visual stimulus position relative to the fovea.

Coding of interceptive saccades in parietal cortex of macaque monkeys.

The oculomotor system can initiate remarkably accurate saccades towards moving targets (interceptive saccades) the processing of which is still under debate. The generation of these saccades requires the oculomotor centers to have information about the motion parameters of the target that then must be extrapolated to bridge the inherent processing delays. We investigated to what degree the information about motion of a saccade target is available in the lateral intra-parietal area (area LIP) of macaque monkeys for generation of accurate interceptive saccades. When a multi-layer neural network was trained based on neural discharges from area LIP around the time of saccades towards stationary targets, it was also able to predict the end points of saccades directed towards moving targets. This prediction, however, lagged behind the actual post-saccadic position of the moving target by ~ 80 ms when the whole neuronal sample of 105 neurons was used. We further found that single neurons differentially code for the motion of the target. Selecting neurons with the strongest representation of target motion reduced this lag to ~ 30 ms which represents the position of the moving target approximately at the onset of the interceptive saccade. We conclude that-similarly to recent findings from the Superior Colliculus (Goffart et al. J Neurophysiol 118(5):2890-2901)-there is a continuum of contributions of individual LIP neurons to the accuracy of interceptive saccades. A contribution of other gaze control centers (like the cerebellum or the frontal eye field) that further increase the saccadic accuracy is, however, likely.

Preattentive processing of visually guided self-motion in humans and monkeys.

The visually-based control of self-motion is a challenging task, requiring - if needed - immediate adjustments to keep on track. Accordingly, it would appear advantageous if the processing of self-motion direction (heading) was predictive, thereby accelerating the encoding of unexpected changes, and un-impaired by attentional load. We tested this hypothesis by recording EEG in humans and macaque monkeys with similar experimental protocols. Subjects viewed a random dot pattern simulating self-motion across a ground plane in an oddball EEG paradigm. Standard and deviant trials differed only in their simulated heading direction (forward-left vs. forward-right). Event-related potentials (ERPs) were compared in order to test for the occurrence of a visual mismatch negativity (vMMN), a component that reflects preattentive and likely also predictive processing of sensory stimuli. Analysis of the ERPs revealed signatures of a prediction mismatch for deviant stimuli in both humans and monkeys. In humans, a MMN was observed starting 110 ms after self-motion onset. In monkeys, peak response amplitudes following deviant stimuli were enhanced compared to the standard already 100 ms after self-motion onset. We consider our results strong evidence for a preattentive processing of visual self-motion information in humans and monkeys, allowing for ultrafast adjustments of their heading direction.

The geometry of perisaccadic visual perception.

Our ability to explore our surroundings requires a combination of high-resolution vision and frequent rotations of the visual axis toward objects of interest. Such gaze shifts are themselves a source of powerful retinal stimulation, and so the visual system appears to have evolved mechanisms to maintain perceptual stability during movements of the eyes in space. The mechanisms underlying this perceptual stability can be probed in the laboratory by briefly presenting a stimulus around the time of a saccadic eye movement and asking subjects to report its position. Under such conditions, there is a systematic misperception of the probes toward the saccade end point. This perisaccadic compression of visual space has been the subject of much research, but few studies have attempted to relate it to specific brain mechanisms. Here, we show that the magnitude of perceptual compression for a wide variety of probe stimuli and saccade amplitudes is quantitatively predicted by a simple heuristic model based on the geometry of retinotopic representations in the primate brain. Specifically, we propose that perisaccadic compression is determined by the distance between the probe and saccade end point on a map that has a logarithmic representation of visual space, similar to those found in numerous cortical and subcortical visual structures. Under this assumption, the psychophysical data on perisaccadic compression can be appreciated intuitively by imagining that, around the time of a saccade, the brain confounds nearby oculomotor and sensory signals while attempting to localize the position of objects in visual space.

Interaction of spatial and temporal factors in psychophysical estimates of surround suppression.

Recent psychophysical work has shown that performance on a direction discrimination task decreases with increasing stimulus size, provided the stimulus is high in contrast. This psychophysical surround suppression has been linked to the inhibitory spatial surrounds that have been observed throughout the primate visual system. In this work we have examined a temporal factor that may also contribute to psychophysical surround suppression. Consistent with previous work, we found that psychophysical surround suppression is strongest when a high-contrast motion stimulus is presented very briefly so that the appearance of the stimulus coincided with its motion. However, when a brief delay was inserted between the stimulus onset and the onset of motion, the counterintuitive effects of stimulus size disappeared. The effect of the motion onset asynchrony (MOA) was strongest when the stationary stimulus immediately preceded the stimulus motion and when stimulus orientation during the MOA was very similar to that during the motion presentation. We conclude that psychophysical surround suppression is partially linked to the temporal structure of the stimulus, more precisely to a masking effect caused by sudden stimulus onsets (and to a smaller degree stimulus offsets).

Eye movements during path integration.

Self-motion induces spontaneous eye movements which serve the purpose of stabilizing the visual image on the retina. Previous studies have mainly focused on their reflexive nature and how the perceptual system disentangles visual flow components caused by eye movements and self-motion. Here, we investigated the role of eye movements in distance reproduction (path integration). We used bimodal (visual-auditory)-simulated self-motion: visual optic flow was paired with an auditory stimulus whose frequency was scaled with simulated speed. The task of the subjects in each trial was, first, to observe the simulated self-motion over a certain distance (Encoding phase) and, second, to actively reproduce the observed distance using only visual, only auditory, or bimodal feedback (Reproduction phase). We found that eye positions and eye speeds were strongly correlated between the Encoding and the Reproduction phases. This was the case even when reproduction relied solely on auditory information and thus no visual stimulus was presented. We believe that these correlations are indicative of a contribution of eye movements to path integration.

Comparison of the precision of smooth pursuit in humans and head unrestrained monkeys.

Direct comparison of results of humans and monkeys is often complicated by differences in experimental conditions. We replicated in head unrestrained macaques experiments of a recent study comparing human directional precision during smooth pursuit eye movements (SPEM) and saccades to moving targets (Braun & Gegenfurtner, 2016). Directional precision of human SPEM follows an exponential decay function reaching optimal values of 1.5°-3° within 300 ms after target motion onset, whereas precision of initial saccades to moving targets is slightly better. As in humans, we found general agreement in the devel-opment of directional precision of SPEM over time and in the differences between direc-tional precision of initial saccades and SPEM initiation. However, monkeys showed over-all lower precision in SPEM compared to humans. This was most likely due to differences in experimental conditions, such as in the stabilization of the head, which was by a chin and a head rest in human subjects and unrestrained in monkeys.

Impaired time perception and motor timing in stimulant-dependent subjects.

Stimulant-dependent individuals (SDI) have abnormal brain metabolism and structural changes involving dopaminergic target areas important for the processing of time. These individuals are also more impulsive and impaired in working memory and attention. The current study tested whether SDI show altered temporal processing in relation to impulsivity or impaired prefrontal cortex functioning. We employed a series of timing tasks aimed to examine time processing from the milliseconds to multiple seconds range and assessed cognitive function in 15 male SDI and 15 stimulant-naïve individuals. A mediation analysis determined the degree to which impulsivity or executive dysfunctions contributed to group differences in time processing. SDI showed several abnormal time processing characteristics. SDI needed larger time differences for effective duration discrimination, particularly for intervals of around 1s. SDI also accelerated finger tapping during a continuation period after a 1Hz pacing stimulus was removed. In addition, SDI overestimated the duration of a relatively long time interval, an effect which was attributable to higher impulsivity. Taken together, these data show for the first time that SDI exhibit altered time processing in several domains, one which can be explained by increased impulsivity. Altered time processing in SDI could explain why SDI have difficulty delaying gratification.

Temporal reproduction: further evidence for two processes.

Some authors have suggested separate mechanisms for the processing of temporal intervals above versus below 2-3s. Given that the evidence is mixed, the present experiment was carried out as a critical test of the separate-mechanism hypothesis. Subjects reproduced five standard durations of 1-5s presented in the auditory and visual modalities. The Corsi-block test was used to assess effects of working-memory span on different interval lengths. Greater working-memory span was associated with longer reproductions of intervals of 3-5s. A factor analysis run on mean reproduced intervals revealed one modality-unspecific factor for durations of 1-2s and two modality-specific factors for longer intervals. These results are interpreted as further indications that two different processes underlie temporal reproductions of shorter and longer intervals.

Heading representations in primates are compressed by saccades.

Perceptual illusions help to understand how sensory signals are decoded in the brain. Here we report that the opposite approach is also applicable, i.e., results from decoding neural activity from monkey extrastriate visual cortex correctly predict a hitherto unknown perceptual illusion in humans. We record neural activity from monkey medial superior temporal (MST) and ventral intraparietal (VIP) area during presentation of self-motion stimuli and concurrent reflexive eye movements. A heading-decoder performs veridically during slow eye movements. During fast eye movements (saccades), however, the decoder erroneously reports compression of heading toward straight ahead. Functional equivalents of macaque areas MST and VIP have been identified in humans, implying a perceptual correlate (illusion) of this perisaccadic decoding error. Indeed, a behavioral experiment in humans shows that perceived heading is perisaccadically compressed toward the direction of gaze. Response properties of primate areas MST and VIP are consistent with being the substrate of the newly described visual illusion.Macaque higher visual areas MST and VIP encode heading direction based on self-motion stimuli. Here the authors show that, while making saccades, the heading direction decoded from the neural responses is compressed toward straight-ahead, and independently demonstrate a perceptual illusion in humans based on this perisaccadic decoding error.

Temporal processing and context dependency of phoneme discrimination in patients with aphasia.

Standard diagnostic procedures for assessing temporal-processing abilities of adult patients with aphasia have so far not been developed. In our study, temporal-order measurements were conducted using two different experimental procedures to identify a suitable measure for clinical studies. Additionally, phoneme-discrimination abilities were tested on the word, as well as on the sentence level, as a relationship between temporal processing and phoneme-discrimination abilities is assumed. Patients with aphasia displayed significantly higher temporal-order thresholds than control subjects. The detection of an association between temporal processing and speech processing, however, depended on the stimuli and the phoneme-discrimination tasks used. Our results also suggest top-down feedback on phonemic processing.

Decoding Target Distance and Saccade Amplitude from Population Activity in the Macaque Lateral Intraparietal Area (LIP).

Primates perform saccadic eye movements in order to bring the image of an interesting target onto the fovea. Compared to stationary targets, saccades toward moving targets are computationally more demanding since the oculomotor system must use speed and direction information about the target as well as knowledge about its own processing latency to program an adequate, predictive saccade vector. In monkeys, different brain regions have been implicated in the control of voluntary saccades, among them the lateral intraparietal area (LIP). Here we asked, if activity in area LIP reflects the distance between fovea and saccade target, or the amplitude of an upcoming saccade, or both. We recorded single unit activity in area LIP of two macaque monkeys. First, we determined for each neuron its preferred saccade direction. Then, monkeys performed visually guided saccades along the preferred direction toward either stationary or moving targets in pseudo-randomized order. LIP population activity allowed to decode both, the distance between fovea and saccade target as well as the size of an upcoming saccade. Previous work has shown comparable results for saccade direction (Graf and Andersen, 2014a,b). Hence, LIP population activity allows to predict any two-dimensional saccade vector. Functional equivalents of macaque area LIP have been identified in humans. Accordingly, our results provide further support for the concept of activity from area LIP as neural basis for the control of an oculomotor brain-machine interface.

Assessment of auditory temporal-order thresholds - a comparison of different measurement procedures and the influences of age and gender.

PURPOSE: The relationship between auditory temporal-order perception and phoneme discrimination has been discussed for several years, based on findings, showing that patients with cerebral damage in the left hemisphere and aphasia, as well as children with specific language impairments, show deficits in temporal-processing and phoneme discrimination. Over the last years several temporal-order measurement procedures and training batteries have been developed. However, there exists no standard diagnostic tool for adults that could be applied to patients with aphasia. Therefore, our study aimed at identifying a feasible, reliable and efficient measurement procedure to test for auditory-temporal processing in healthy young and elderly adults, which in a further step can be applied to patients with aphasia. METHODS: The tasks varied according to adaptive procedures (staircase vs. maximum-likelihood), stimuli (tones vs. clicks) and stimulation modes (binaural- vs. alternating monaural) respectively. A phoneme-discrimination task was also employed to assess the relationship between temporal and language processing. RESULTS: The results show that auditory temporal-order thresholds are stimulus dependent, age related, and influenced by gender. Furthermore, the cited relationship between temporal-order threshold and phoneme discrimination can only be confirmed for measurements with pairs of tones. CONCLUSION: Our results indicate, that different norms have to be established for different gender and age groups. Furthermore, temporal-order measurements with tones seem to be more suitable for clinical intervention studies than measurements with clicks, as they show higher re-test reliabilities, and only for measurements with tones an association with phoneme-discrimination abilities was found.