The perceived position of moving objects: transcranial magnetic stimulation of area MT+ reduces the flash-lag effect.

How does the visual system assign the perceived position of a moving object? This question is surprisingly complex, since sluggish responses of photoreceptors and transmission delays along the visual pathway mean that visual cortex does not have immediate information about a moving object's position. In the flash-lag effect (FLE), a moving object is perceived ahead of an aligned flash. Psychophysical work on this illusion has inspired models for visual localization of moving objects. However, little is known about the underlying neural mechanisms. Here, we investigated the role of neural activity in areas MT+ and V1/V2 in localizing moving objects. Using short trains of repetitive Transcranial Magnetic Stimulation (TMS) or single pulses at different time points, we measured the influence of TMS on the perceived location of a moving object. We found that TMS delivered to MT+ significantly reduced the FLE; single pulse timings revealed a broad temporal tuning with maximum effect for TMS pulses, 200 ms after the flash. Stimulation of V1/V2 did not significantly influence perceived position. Our results demonstrate that area MT+ contributes to the perceptual localization of moving objects and is involved in the integration of position information over a long time window.

Going, going, gone: localizing abrupt offsets of moving objects.

When a moving object abruptly disappears, this profoundly influences its localization by the visual system. In Experiment 1, 2 aligned objects moved across the screen, and 1 of them abruptly disappeared. Observers reported seeing the objects misaligned at the time of the offset, with the continuing object leading. Experiment 2 showed that the perceived forward displacement of the moving object depended on speed and that offsets were localized accurately. Two competing representations of position for moving objects are proposed: 1 based on a spatially extrapolated internal model, and the other based on transient signals elicited by sudden changes in the object trajectory that can correct the forward-shifted position. Experiment 3 measured forward displacements for moving objects that disappeared only for a short time or abruptly reduced contrast by various amounts. Manipulating the relative strength of the 2 position representations in this way resulted in intermediate positions being perceived, with weaker motion signals or stronger transients leading to less forward displacement. This 2-process mechanism is advantageous because it uses available information about object position to maximally reduce spatio-temporal localization errors.

Motion extrapolation into the blind spot.

The flash-lag effect, in which a moving object is perceived ahead of a colocalized flash, has led to keen empirical and theoretical debates. To test the proposal that a predictive mechanism overcomes neural delays in vision by shifting objects spatially, we asked observers to judge the final position of a bar moving into the retinal blind spot. The bar was perceived to disappear in positions well inside the unstimulated area. Given that photoreceptors are absent in the blind spot, the perceived shift must be based on the history of the moving object. Such predictive overshoots are suppressed when a moving object disappears abruptly from the retina, triggering retinal transient signals. No such transient-driven suppression occurs when the object disappears by virtue of moving into the blind spot. The extrapolated position of the moving bar revealed in this manner provides converging support for visual prediction.

Perceiving the present and a systematization of illusions.

Over the history of the study of visual perception there has been great success at discovering countless visual illusions. There has been less success in organizing the overwhelming variety of illusions into empirical generalizations (much less explaining them all via a unifying theory). Here, this article shows that it is possible to systematically organize more than 50 kinds of illusion into a 7 × 4 matrix of 28 classes. In particular, this article demonstrates that (1) smaller sizes, (2) slower speeds, (3) greater luminance contrast, (4) farther distance, (5) lower eccentricity, (6) greater proximity to the vanishing point, and (7) greater proximity to the focus of expansion all tend to have similar perceptual effects, namely, to (A) increase perceived size, (B) increase perceived speed, (C) decrease perceived luminance contrast, and (D) decrease perceived distance. The detection of these empirical regularities was motivated by a hypothesis, called "perceiving the present," that the visual system possesses mechanisms for compensating neural delay during forward motion. This article shows how this hypothesis predicts the empirical regularity.

Behavioral significance of motion direction causes anisotropic flash-lag, flash-drag, flash-repulsion, and movement-mislocalization effects.

Motion from periphery to central vision (foveopetal motion) causes a greater flash-lag effect than motion in the opposite direction (foveofugal motion). In order to examine the factors that contribute to the motion direction-based anisotropic flash-lag effect, we investigated the mislocalization of the flash caused by motion and the mislocalization of the moving object per se. We observed that for foveofugal motion, flashes were perceived shifted in the direction of motion but mislocalized in the opposite direction for foveopetal motion. Additionally the mislocalization of the moving object was larger in foveopetal motion than in foveofugal motion. Thus, both factors contribute to the anisotropic flash-lag effect. We interpret these findings in terms of greater behavioral significance of foveopetal motion in relation to foveofugal motion.

Visual prediction: psychophysics and neurophysiology of compensation for time delays.

A necessary consequence of the nature of neural transmission systems is that as change in the physical state of a time-varying event takes place, delays produce error between the instantaneous registered state and the external state. Another source of delay is the transmission of internal motor commands to muscles and the inertia of the musculoskeletal system. How does the central nervous system compensate for these pervasive delays? Although it has been argued that delay compensation occurs late in the motor planning stages, even the earliest visual processes, such as phototransduction, contribute significantly to delays. I argue that compensation is not an exclusive property of the motor system, but rather, is a pervasive feature of the central nervous system (CNS) organization. Although the motor planning system may contain a highly flexible compensation mechanism, accounting not just for delays but also variability in delays (e.g., those resulting from variations in luminance contrast, internal body temperature, muscle fatigue, etc.), visual mechanisms also contribute to compensation. Previous suggestions of this notion of "visual prediction" led to a lively debate producing re-examination of previous arguments, new analyses, and review of the experiments presented here. Understanding visual prediction will inform our theories of sensory processes and visual perception, and will impact our notion of visual awareness.

Analogous mechanisms compensate for neural delays in the sensory and the motor pathways: evidence from motor flash-lag.

Motor behaviors require animals to coordinate neural activity across different areas within their motor system. In particular, the significant processing delays within the motor system must somehow be compensated for. Internal models of the motor system, in particular the forward model, have emerged as important potential mechanisms for compensation. For motor responses directed at moving visual objects, there is, additionally, a problem of delays within the sensory pathways carrying crucial position information. The visual phenomenon known as the flash-lag effect has led to a motion-extrapolation model for compensation of sensory delays. In the flash-lag effect, observers see a flashed item colocalized with a moving item as lagging behind the moving item. Here, we explore the possibility that the internal forward model and the motion-extrapolation model are analogous mechanisms compensating for neural delays in the motor and the visual system, respectively. In total darkness, observers moved their right hand gripping a rod while a visual flash was presented at various positions in relation to the rod. When the flash was aligned with the rod, observers perceived it in a position lagging behind the instantaneous felt position of the invisible rod. These results suggest that compensation of neural delays for time-varying motor behavior parallels compensation of delays for time-varying visual stimulation.

Forward displacements of fading objects in motion: the role of transient signals in perceiving position.

Visual motion causes mislocalisation phenomena in a variety of experimental paradigms. For many displays objects are perceived as displaced 'forward' in the direction of motion. However, in some cases involving the abrupt stopping or reversal of motion the forward displacements are not observed. We propose that the transient neural signals at the offset of a moving object play a crucial role in accurate localisation. In the present study, we eliminated the transient signals at motion offset by gradually reducing the luminance of the moving object. Our results show that the 'disappearance threshold' for a moving object is lower than the detection threshold for the same object without a motion history. In units of time this manipulation led to a forward displacement of the disappearance point by 175 ms. We propose an explanation of our results in terms of two processes: Forward displacements are caused by internal models predicting positions of moving objects. The usually observed correct localisation of stopping positions, however, is based on transient inputs that retroactively attenuate errors that internal models might otherwise cause. Both processes are geared to reducing localisation errors for moving objects.

Cyclopean flash-lag illusion.

Possible physiological mechanisms to explain the flash-lag effect, in which subjects perceive a flashed item that is co-localized with a moving item as trailing behind the moving item, have been found within the retina of lower species, and in the motor pathways of humans. Here, we demonstrate flash-lag employing "second-order" moving and flashed stimuli, defined solely by their binocular-disparity, to circumvent any possible "early" contributions to the effect. A significant flash-lag effect was measured with cyclopean stimuli composed entirely of correlated random dot patterns. When the disparity-defined moving stimulus was replaced with a luminance-defined one, potentially engaging retinal mechanisms, the magnitude of the measured effect showed no significant change. Thus, in primates, though retinal mechanisms may contribute, flash-lag must be explained through cortical processes.

Flash-lag chimeras: the role of perceived alignment in the composite face effect.

Spatial alignment of different face halves results in a configuration that mars the recognition of the identity of either face half (). What would happen to the recognition performance for face halves that were aligned on the retina but were perceived as misaligned, or were misaligned on the retina but were perceived as aligned? We used the 'flash-lag' effect () to address these questions. We created chimeras consisting of a stationary top half-face initially aligned with a moving bottom half-face. Flash-lag chimeras were better recognized than their stationary counterparts. However when flashed face halves were presented physically ahead of moving halves thereby nulling the flash-lag effect, recognition was impaired. This counters the notion that relative movement between the two face halves per se is sufficient to explain better recognition of flash-lag chimeras. Thus, the perceived spatial alignment of face halves (despite retinal misalignment) impairs recognition, while perceived misalignment (despite retinal alignment) does not.

Perceptual organization of moving stimuli modulates the flash-lag effect.

When a visual stimulus is flashed at a given location the moment a second moving stimulus arrives at the same location, observers report the flashed stimulus as spatially lagging behind the moving stimulus (the flash-lag effect). The authors investigated whether the global configuration (perceptual organization) of the moving stimulus influences the magnitude of the flash-lag effect. The results indicate that a flash presented near the leading portion of a moving stimulus lags significantly more than a flash presented near the trailing portion. This result also holds for objects consisting of several elements that group to form a unitary percept of an object in motion. The present study demonstrates a novel interaction between the global configuration of moving objects and the representation of their spatial position and may provide a new and useful tool for the study of perceptual organization.

Shifts in perceived position of flashed stimuli by illusory object motion.

Moving stimuli cause the position of flashed stimuli to appear shifted in the direction of motion (position capture). To examine whether position capture depends on low-level motion interactions or perception of integrated object motion, we employed a slit-view display. Two line-drawn diamonds translated horizontally in opposite directions, one above and one below the fixation cross, either behind an occluding surface with a narrow slit or without occluding surface. When the diamonds were in vertical alignment, two vertical bars were flashed, one in the center of each diamond. In the slit-view condition, the diamonds were visible through a 4-, 2-, or 1-pixel vertical slit; the width of the flashed bars always matched the width of the slit. Even though the horizontal component of physical motion was greatly reduced or absent in the slit-view conditions, observers perceived diamonds moving behind the occluding surface. Furthermore, the position of the flashed bar was captured by the moving diamonds such that each bar appeared shifted in the direction of perceived motion. We conclude that the position capture reported here has a component based on high-level motion processing that is responsible for dynamically integrating object motion and shape.

Motion extrapolation in the central fovea.

Neural transmission latency would introduce a spatial lag when an object moves across the visual field, if the latency was not compensated. A visual predictive mechanism has been proposed, which overcomes such spatial lag by extrapolating the position of the moving object forward. However, a forward position shift is often absent if the object abruptly stops moving (motion-termination). A recent "correction-for-extrapolation" hypothesis suggests that the absence of forward shifts is caused by sensory signals representing 'failed' predictions. Thus far, this hypothesis has been tested only for extra-foveal retinal locations. We tested this hypothesis using two foveal scotomas: scotoma to dim light and scotoma to blue light. We found that the perceived position of a dim dot is extrapolated into the fovea during motion-termination. Next, we compared the perceived position shifts of a blue versus a green moving dot. As predicted the extrapolation at motion-termination was only found with the blue moving dot. The results provide new evidence for the correction-for-extrapolation hypothesis for the region with highest spatial acuity, the fovea.

Does Area V3A Predict Positions of Moving Objects?

A gradually fading moving object is perceived to disappear at positions beyond its luminance detection threshold, whereas abrupt offsets are usually localized accurately. What role does retinotopic activity in visual cortex play in this motion-induced mislocalization of the endpoint of fading objects? Using functional magnetic resonance imaging (fMRI), we localized regions of interest (ROIs) in retinotopic maps abutting the trajectory endpoint of a bar moving either toward or away from this position while gradually decreasing or increasing in luminance. Area V3A showed predictive activity, with stronger fMRI responses for motion toward versus away from the ROI. This effect was independent of the change in luminance. In Area V1 we found higher activity for high-contrast onsets and offsets near the ROI, but no significant differences between motion directions. We suggest that perceived final positions of moving objects are based on an interplay of predictive position representations in higher motion-sensitive retinotopic areas and offset transients in primary visual cortex.

Compensating time delays with neural predictions: are predictions sensory or motor?

Neural delays are a general property of computations carried out by neural circuits. Delays are a natural consequence of temporal summation and coding used by the nervous system to integrate information from multiple resources. For adaptive behaviour, however, these delays must be compensated. In order to sense and interact with moving objects, for example, the visual system must predict the future position of the object to compensate for delays. In this paper, we address two critical questions concerning the implementation of the compensation mechanisms in the brain, namely, where does compensation occur and how is it realized. We present evidence showing that compensation can happen in both the motor and sensory systems, and that compensation using 'diagonal neural pathways' is a suitable strategy for implementing compensation in the visual system. In this strategy, neural signals in the early stage of information processing are sent to the future cortical positions that correspond to the distance the object will travel in the period of transmission delay. We propose a computational model to elucidate this using the retinal visual information pathway.

Neural delays, visual motion and the flash-lag effect.

In the primate visual system, there is a significant delay in the arrival of photoreceptor signals in visual cortical areas. Since Helmholtz, scientists have pondered over the implications of these delays for human perception. Do visual delays cause the ' position of a moving object to lag its 'real' position? This question has recently been re-evaluated in the context of the flash-lag phenomenon, in which a flashed object appears to lag behind a moving object, when physically the two objects are co-localized at the instant of the flash. This article critically examines recent accounts of this phenomenon, assesses its biological significance, and offers new hypotheses.