Cortical potentials in humans reflecting the direction of object motion.

In the monkey, cortical areas can be localized which are specific for the processing of form, colour, or motion, and it is expected that the human visual cortex is organized in a similar way. The recording of scalp potentials generated by neural activity of underlying cortical areas is a non-invasive method which can be used to study the functional organization of the visual cortex with a high temporal resolution. In the present study we recorded slow cortical potentials from normal subjects to investigate how motion stimuli of variable complexity are processed in human visual cortex. The results show that the pattern of cortical activation is dependent on the type of stimulus. When random dots were moving within the entire stimulus field, or during counterphase flicker, maximal activation occurred over occipital electrode sites. During object motion a pronounced activation is recorded at parietal locations, with the direction of object motion being reflected by the time course of this activation.

The cortical representation of object motion in man is interindividually variable.

Cortical areas processing visual motion have been well investigated in monkeys, but comparatively little is known about these areas in man. In order to define such cortical areas in the brains of individuals, the magnetic field was recorded while subjects were watching motion-defined static and moving objects. The magnetic response showed a transient component with a clear dipolar magnetic field followed by a sustained component which exhibited some variation in magnetic field structure over time. For the transient component, the single equivalent current dipoles superimposed upon magnetic resonance images for individual subjects were clearly localized outside the primary visual areas. In most cases the neural generator was found in the region of the temporo-parieto-occipital junction of the lateral cortex. The results also suggest that the activated cortical area show interindividual variations in location.

Theta motion: a paradoxical stimulus to explore higher order motion extraction.

Apparent motion stimuli of increasing complexity have been applied to analyse the mechanisms underlying visual motion perception. In the present paper it is investigated how motion detectors respond to three classes of stimuli which are realized as random-dot kinematograms. (i) In the most conventional stimuli, Fourier motion, a group of dots is displaced coherently in a random-dot pattern. (ii) In drift-balanced motion stimuli a bar made of static random dots is shifted in front of another random-dot pattern. (iii) In the novel class of stimuli, theta motion, an object which is exclusively defined by dot motion into one direction, is moving itself into the opposite direction. It is shown in psychophysical experiments that human observers perceive the direction of object motion in all three classes of stimuli. Simple motion detectors, however, only extract the motion direction of the object in the case of Fourier stimuli, and in the case of drift-balanced stimuli, if a nonlinear preprocessing is assumed. Any of the model alternatives discussed so far just detects the moving dots but not the object in a theta-stimulus, as is illustrated by a combinatorial analysis using a simplified version of a motion detector of the correlation type, which operates on a discrete time scale and takes only discrete values. In order to account for the detection of theta-motion, a model consisting of two hierarchical layers of motion detectors is developed, and simulated for conditions as used in the psychophysical experiments. The perception of theta-motion and the two-layer model is discussed in relation to psychophysical data and theoretical considerations from the literature, to try to incorporate the proposed two-layer model into a general scheme of visual motion processing.

Is facilitation responsible for the "motion induction" effect.

When a horizontal bar is presented after a single dot is shown at one of its ends, an illusory motion is seen which has been dubbed "motion induction" in the literature. The phenomenon has been attributed to a facilitation process which asymmetrically modulates the inputs to motion detectors, for instance by some sort of changes in processing speed. Computer simulations of motion detector arrays show, however, that this basic effects has to be expected from the properties of simple motion detectors. It has been recently reported that the strength of the illusory motion increases with the subjective salience of the inducing element. New computer simulations demonstrate that this observation can be related to the control of the local gain of motion detector input signals by the feature contrast in a particular region of the stimulus. High-level attentional mechanisms or changes in transmission speed are not required to explain these phenomena. The implications of such local gain-control mechanisms for our understanding of second-order motion perception are discussed.

Perceptual learning in primary and secondary motion vision.

Specific improvements of perceptual capabilities with practise are thought to give some clues about cortical plasticity and the localisation of cortical processing. In the present study, perceptual learning is used as a paradigm to separate mechanisms underlying the perception of different classes of motion stimuli. Primary motion stimuli (phi-motion), are characterised by displacements of the luminance distribution. However, for secondary motion stimuli the movement is not accompanied by a corresponding luminance shift. Instead, moving objects are defined by their temporal frequency composition (mu-motion) or by motion itself (theta-motion). On theoretical grounds, the perception of secondary motion requires a higher degree of nonlinearity in the processing stream than the perception of primary motion but debate continues as to whether there might be a unique mechanism underlying the perception of both motion classes. In a large group of subjects, coherence thresholds for direction discrimination in random dot kinematograms of phi-, mu-, and theta-motion were repeatedly measured in a staircase paradigm. Training effects were found on different timescales, within short sessions containing multiple staircases and over training periods of several months. They were fairly stable over long breaks without testing. When subjects were trained with two different motion stimuli in a sequence, an asymmetry in the transfer of perceptual learning was revealed: sensitivity increases achieved during practise of theta-motion are largely transferred to phi-motion, but theta-motion perception does not profit from prior exposure to phi-motion. This finding supports the view derived from modelling of motion processing that there must be at least partially separate systems. A primary motion detection mechanism falls short of discriminating direction in secondary motion stimuli, whereas a mechanism able to extract secondary motion will be inherently sensitive to primary motion.

Interaction between primary and secondary mechanisms in human motion perception.

Two layers of information processing can be distinguished as being involved in human motion perception. The primary motion detection stage processes displacements of the luminance distribution across space, such as experienced in natural scenes during the pursuit of moving targets. Primary motion detection is often investigated with artificial motion stimuli realized as random-dot kinematograms (RDKs). Such stimuli belong to the class of "Fourier motion", and their perception can be easily explained by means of elementary motion detectors (EMDs) of the correlation type. Other tasks require the comparison of motion signals from neighbouring areas in the visual field. The perception of the displacement of the motion distribution, for instance, has been accounted for by a secondary motion processing stage. In order to understand the principles of interaction between the motion in neighbouring areas of the visual field, we investigated the sensitivity of the human visual system for moving objects which are defined by moving dots in variable directions. These experiments lead to "secondary tuning curves" of direction discrimination for secondary motion as function of primary motion direction. A base level of sensitivity for all dot motion directions without a velocity component in the same direction of the object movement is enhanced when the object and the dots have a common velocity component. Thus primary motion in any direction can be exploited by the secondary stage, and primary and secondary system both feed into the object motion percept. Furthermore it is suggested from the shape of the secondary tuning curve that the outputs from the two layers of motion processing do not superimpose linearly, but are combined by some sort of veto-like mechanism which increases the directional sensitivity when the two processing layers experience movement along the same direction.

A glimpse into crabworld.

Almost all known arthropod compound eyes exhibit regional variations of resolving power, absolute light, spectral and polarisation sensitivity which are likely to be matched to the probability of significant events and the availability of cues in the visual world. To understand the signal processing requirements that have led to the evolution of matched sensory and neural filters, we thus need a detailed description of the input signals to a visual system and of the tasks to be performed under natural operating conditions. We report here on the first steps we took in an attempt to reconstruct an animal's specific visual world with emphasis on the motion domain. Fiddler crabs (genus Uca) live in burrows on sand- and mudflats and are active during low tide. They carry their eyes on long, vertically oriented stalks and use vision to detect predators and conspecific signals generated by males waving one massively enlarged claw. The crabs sit on the ground plane of a flat world, where significant events are most likely to occur in a narrow band around the horizon. We recorded scenes in a crab colony with a video camera at crab eye height. The salience of relevant features in the spatial, spectral and polarisation domains was analysed in digitised video images and short sequences of film were processed by a two-dimensional network of motion detectors at various spatial scales. The output of the network provides us with histograms of the direction and strength of motion signals in various spatio-temporal frequency bands. We discuss our results in terms of detection problems, predictability of events, global vs local information content and higher level motion processing involved in intraspecific communication.

On temporal hyperacuity in the human visual system.

The spatial grain of the human visual system has always been a central topic for visual sciences, and the optical and physiological basis of perceptual limitations are well described. In particular, we have thorough accounts of spatial hyperacuity, which refers to a precision in the spatial localisation of stimulus contours that is better than the photoreceptor grain that determines spatial resolution. However, although the temporal resolution of the human visual system is comparably well described, we have almost no direct knowledge about the precision of localising visual stimuli in time in the absence of correlated spatial cues. The present study addresses this question by comparing directly the temporal resolution of human observers with their temporal acuity as measured in a temporal bisection task. Despite some improvement with practice, temporal acuity in this task does not fall below 20-30 ms in the best case, which is similar to the temporal resolution limit, and performance does not improve for comparison tasks with multiple stimulus presentations. The absence of visual hyperacuity for purely temporal modulations as tested here contrasts with processing limitations for other types of visual information in comparable tasks, and with other sensory modalities, in particular to those of the auditory system. Such differences can be interpreted in the context of the ecological requirements for organising behaviour, and the functional design of nervous systems.

How does noise influence the estimation of speed?

Local motion signals have to be combined in space and time, to yield a coherent motion percept as it is involved in a variety of visual tasks. This combination necessarily means to trade-off between loosing spatio-temporal resolution by pooling local signals and maintaining perceptually significant segmentation between them. When signals are pooled to detect the presence of coherent motion in large amounts of random noise, the question raised is how the noise affects the perceived quality, in particular speed, of the coherent motion. Is there an analogy to the well-known reduction in the perceived speed of moving gratings at low contrast? Using a two-interval forced-choice procedure, we have investigated the assessment of speed in random-dot kinematograms containing different proportions of noise. Under the conditions investigated, there is no strong reduction of perceived speed with increasing noise, as long as coherence levels remain well above the thresholds for directional judgements. This basic result, which could suggest considerable but not perfect segregation of signal and noise motion components in the pooling process leading to speed estimation, is discussed in relation to a model that is designed to decode speed from a population of elementary motion detectors (EMDs) of the correlation type. A strategy to estimate speed from a set of EMDs with a variety of spatio-temporal tuning does not only provide a velocity predictor unambiguous with the spatial structure of the stimulus, but also is largely independent of noise.

Hyperacuity for spatial localization of contrast-modulated patterns.

The acuity for localizing the position of a grating and other first order patterns which are defined directly by the luminance distribution, is much higher than the resolution for such gratings. This well-described phenomenon usually is referred to as hyperacuity, and is regarded as a cortical function which is not limited by the optics and the sampling properties of the eye. Second order patterns which can be defined by the distribution of local contrast gained some interest because they require more complex processing mechanisms than first order patterns. We investigated how well gratings and bars which are exclusively defined by the variation of the local contrast of static random dot patterns can be localized in space. In this case localization acuity does not reach the precision which is known for first order patterns. However, the localization of contrast-modulated patterns can be almost one order of magnitude better than second order grating resolution, and therefore reaches into the hyperacuity range. In combination with findings for motion-defined or stereo-defined patterns it is concluded that the brain mechanisms responsible for the localization of features in the visual scene have not only access to first order information which is available immediately from the retinal image, but in addition, to second order information which has to be extracted from the retinal intensity distribution by some sort of nonlinear processing.

Movement-induced motion signal distributions in outdoor scenes.

The movement of an observer generates a characteristic field of velocity vectors on the retina (Gibson 1950). Because such optic flow-fields are useful for navigation, many theoretical, psychophysical and physiological studies have addressed the question how ego-motion parameters such as direction of heading can be estimated from optic flow. Little is known, however, about the structure of optic flow under natural conditions. To address this issue, we recorded sequences of panoramic images along accurately defined paths in a variety of outdoor locations and used these sequences as input to a two-dimensional array of correlation-based motion detectors (2DMD). We find that (a) motion signal distributions are sparse and noisy with respect to local motion directions; (b) motion signal distributions contain patches (motion streaks) which are systematically oriented along the principal flow-field directions; (c) motion signal distributions show a distinct, dorso-ventral topography, reflecting the distance anisotropy of terrestrial environments; (d) the spatiotemporal tuning of the local motion detector we used has little influence on the structure of motion signal distributions, at least for the range of conditions we tested; and (e) environmental motion is locally noisy throughout the visual field, with little spatial or temporal correlation; it can therefore be removed by temporal averaging and is largely over-ridden by image motion caused by observer movement. Our results suggest that spatial or temporal integration is important to retrieve reliable information on the local direction and size of motion vectors, because the structure of optic flow is clearly detectable in the temporal average of motion signal distributions. Ego-motion parameters can be reliably retrieved from such averaged distributions under a range of environmental conditions. These observations raise a number of questions about the role of specific environmental and computational constraints in the processing of natural optic flow.

Does motion perception follow Weber's law?

The subjective strength of a percept often depends on the stimulus intensity in a nonlinear way. Such coding is often reflected by the observation that the just-noticeable difference between two stimulus intensities (JND) is proportional to the absolute stimulus intensity. This behaviour, which is usually referred to as Weber's Law, can be interpreted as a compressive nonlinearity extending the operating range of a sensory system. When the noise superimposed on a motion stimulus is increased along a logarithmic scale (in order to provide linear steps in subjective difference) in motion-coherency measurements, observers often report that the subjective differences between the various noise levels increase together with the absolute level. This observation could indicate a deviation from Weber's Law for variation of motion strength as obtained by changing the signal-to-noise ratio in random-dot kinematograms. Thus JNDs were measured for the superposition of uncorrelated random-dot patterns on static random-dot patterns and three types of motion stimuli realised as random-dot kinematograms, namely large-field and object 'Fourier' motion (all or a group of dots move coherently), 'drift-balanced' motion (a travelling region of static dots), and paradoxical 'theta' motion (the dots on the surface of an object move in opposite direction to the object itself). For all classes of stimuli, the JNDs when expressed as differences in signal-to-noise ratio turned out to increase with the signal-to-noise ratio, whereas the JNDs given as percentage of superimposed noise appear to be similar for all tested noise levels. Thus motion perception is in accordance with Weber's Law when the signal-to-noise ratio is regarded as stimulus intensity, which in turn appears to be coded in a nonlinear fashion. In general the Weber fractions are very large, indicating a poor differential sensitivity in signal-to-noise measurements.

Second-order motion perception in the peripheral visual field.

In motion perception, luminance-defined stimuli (first-order motion) are distinguished from stimuli defined by more complex attributes (second-order motion), because they differ in their processing requirements. For instance, a two-layer model with the output of an array of elementary motion detectors (EMD's) feeding into a second array of EMD's has been proposed to account for seeing the movement of motion-defined objects. The question is raised whether this processing scheme is operating across the whole visual field or whether second-order motion perception is restricted to the fovea. The detection, orientation discrimination, and motion direction discrimination of oblique, vertically moving bars was tested at horizontal eccentricities between 0 degree and 16 degrees. Bars were defined on a dynamic noise background by an area of static dots (drift-balanced motion) or by coherent dot motion either in the direction of the bar motion (Fourier motion) or in the orthogonal direction (theta motion). Coherence thresholds for direction discrimination are severely impaired in the periphery for both types of second-order motion but not for Fourier motion, whereas orientation discrimination and detection marginally decline for all three bar types when the stimuli are presented further out in the periphery. In a control experiment it is shown that this result cannot be due entirely to the changes in spatial scale of the peripheral visual system. The facts that motion-defined objects can be detected in the periphery and that their orientation can be detected, but not their direction of motion, supports the view that the two-layer system suggested for the processing of theta motion is restricted to the central region of the visual field.

Limiting factors for the detection of orientation.

First steps of visual-information processing in primates are characterised by a highly ordered representation of the outside world on the cortex. Two prominent features of cortical organisation are the retinotopic mapping of position in the visual field on the first stages of the visual stream, and the systematic variation of orientation preference in the same areas. In an attempt to understand the relation of position and orientation representation, we need to know the minimum spatial requirements for orientation detection. In the present paper, the spatial limits for detecting orientation are analysed by simulating simple orientation filters and testing the ability of human observers to detect the orientation of small lines at various positions in the visual field. At sufficiently high contrast levels, the minimum physical length of a line to discriminate orientation differences of 45 degrees-90 degrees is not constant when presented at various eccentricities, but covaries inversely with the cortical magnification factor. In consequence, a line needs to correspond to about 0.2 mm of cortical surface, independently of the actual eccentricity at which the stimulus is presented, in order to allow observers to recognise its orientation. This has consequences for our understanding of orientation detection. (i) In combination with simulation experiments, it becomes clear that the elementary process underlying orientation detection is a local operation, which seems to focus on small regions compared with cortical receptive fields. (ii) With respect to the number of inputs to the visual cortex, the performance of this local operation approaches the physical limits, requiring hardly more than three-four input LGN axons to be activated for detecting the orientation of a highly visible line segment. Comparing these spatial characteristics with the receptive fields of orientation-sensitive neurons in the primate visual system could suggest new insights into the neuronal circuits underlying orientation mapping in the human cortex.

On the elementary mechanism underlying secondary motion processing.

The movement of luminance-defined targets can be easily extracted by elementary motion detectors (EMDs) of the correlation type which often are referred to as Reichardt-detectors. In contrast to such 'primary motion', in 'secondary motion' the moving target is defined by more complex features, like changes in texture, flicker, or local contrast. Such stimulus attributes have to be extracted from the retinal intensity distribution by some nonlinear preprocessing, before they are fed into motion detectors. An intriguing case is the perception of the movement of the motion signal, as is present in theta motion, where an object moves in a different direction than the texture on its surface. A two-layer model of hierarchically organised EMDs has been postulated to account for such motion extraction. Other than for the first layer, the computational nature of the mechanism underlying motion processing in the second layer so far is a matter of speculation, and is therefore characterized here by means of computer simulations and psychophysical experiments. Random dot kinematograms were generated in which sinusoidally modulated vertical dot motion defined gratings, and coherence thresholds were measured for the direction discrimination of a horizontally travelling modulation function. This was done for a variety of spatial frequencies and speeds of the modulation sinusoid. Thresholds turn out to be lowest not for a particular speed, but for a fixed temporal frequency of the modulation function (about 1 cycle per second), when various combinations of fine and coarse, and fast and slow secondary gratings are tested. This result favours a correlation-type mechanism over a gradient-type scheme which should lead to a speed-optimum independent of spatial frequency.

Determining the visibility of noise in motion signals.

A procedure is described for generating stimuli to study the detection of noise components in motion signals. By using random dots with intensities distributed according to a Gaussian probability function, a temporally and spatially continuous mixture of signal and noise components can be realized in random dot kinematograms. These stimuli were used in a noise detection task, a signal detection task and a direction discrimination task. Signal-to-noise ratio ('coherence') thresholds for the signal detection and direction discrimination tasks were consistent with previous research. Noise can be detected at levels of approximately 0.5-2.5%, depending on the size of the motion stimulus. We argue that the noise in the motion stimulus becomes detectable when it exceeds the noise intrinsic to the various stages of motion processing. Therefore,the method provides a simple procedure for obtaining measures of equivalent input noise and can be used for estimating internal noise levels of motion processing mechanisms.

The directional tuning of the barber-pole illusion.

In order to study the integration of local motion signals in the human visual system, we measured directional tuning curves for the barber-pole illusion by varying two crucial aspects of the stimulus layout independently across a wide a range in the same experiment. These were the orientation of the grating presented behind the rectangular aperture and the aspect ratio of the aperture, which in combination determine the relative contributions of local motion signals perpendicular to the gratings and parallel to the aperture borders, respectively. The strength of the illusion, ie the tendency to perceive motion along the major axis of the aperture, obviously depends on the spatial layout of the aperture, but also on grating orientation. Subjects were asked which direction they perceived and how compelling their motion percept was, revealing different strategies of the visual system to deal with the barber-pole stimulus. Some individuals respond strongly to the unambiguous motion information at the boundaries, leading to multistable percepts and multimodal distributions of responses. Others tend to report intermediate directions, apparently being less influenced by the actual boundaries. The general pattern of deviations from the motion direction perpendicular to grating orientation--a decrease with aspect ratio approaching unity (ie square-shaped apertures) and with gratings approaching parallel orientation to the shorter aperture boundary--is discussed in the context of simple phenomenological models of motion integration. The best fit between model predictions and experimental data is found for an interaction between two stimulus parameters: (i) cycle ratio, which is the sine-wave gratings equivalent of the terminator ratio for line gratings, describing the effects from the aperture boundaries, and (ii) the grating orientation, responsible for perpendicular motion components, which describes the influence of motion signals from inside the aperture. This suggests that the most simple cycle (terminator) ratio explanation cannot fully account for the quantitative properties of the barber-pole illusion.

What determines the perceived speed of dots moving within apertures?

Whereas it is a well known fact that objects appear to move faster in smaller stimulus fields, the reason for such a misjudgement of speed is still a matter of debate. We present four experiments to characterise the stimulus parameters that are important for the apparent speed increase of dots moving behind small apertures. In these experiments we varied the size and the shape of the aperture and its location in the visual field, as well as the stimulus duration. We report that the field-size effect does not depend on the overall duration of the stimulus, which does influence the typical path length of individual dots in the display. It is, however, affected by the shape of the aperture in such a way that the aperture size along the motion path is crucial for the speed misjudgement. The field-size effect furthermore depends on the location of the stimulus in the visual field. Our combined results are best described as an increase in perceived speed that is consistently elicited when a motion sink, i.e. a boundary of disappearing dots, is located close to the fovea. Such a description of the relevant stimulus parameters is discussed with respect to possible high-level mechanisms, relating back to classic Gestalt psychology explanations of the field-size effect, and with respect to well-known aspects of neuronal processing that may underlie speed perception and motion integration.

Interaction of first- and second-order direction in motion-defined motion.

Motion-defined motion can play a special role in the discussion of whether one or two separate systems are required to process first- and second-order information because, in contrast to other second-order stimuli, such as contrast-modulated contours, motion detection cannot be explained by a simple input nonlinearity but requires preprocessing by motion detectors. Furthermore, the perceptual quality that defines an object (motion on the object surface) is identical to that which is attributed to the object as an emergent feature (motion of the object), raising the question of how these two object properties are linked. The interaction of first- and second-order information in such stimuli has been analyzed previously in a direction-discrimination task, revealing some cooperativity. Because any comprehensive integration of these two types of motion information should be reflected in the most fundamental property of a moving object, i.e., the direction in which it moves, we now investigate how motion direction is estimated in motion-defined objects. Observers had to report the direction of moving objects that were defined by luminance contrast or in random-dot kinematograms by differences in the spatiotemporal properties between the object region and the random-noise background. When the dots were moving coherently with the object (Fourier motion), direction sensitivity resembled that for luminance-defined objects, but performance deteriorated when the dots in the object region were static (drift-balanced motion). When the dots on the object surface were moving diagonally relative to the object direction (theta motion), the general level of accuracy declined further, and the perceived direction was intermediate between the veridical object motion direction and the direction of dot motion, indicating that the first- and second-order velocity vectors are somehow pooled. The inability to separate first- and second-order directional information suggests that the two corresponding subsystems of motion processing are not producing independent percepts and provides clues for possible implementations of the two-layer motion-processing network.

Detecting the orientation of short lines in the periphery.

PURPOSE: Visual information processing in the human cortex is based on a highly ordered representation of the surrounding world. In addition to the retinotopic mapping of the visual field, systematic variations of the orientation tuning of neurons have been described in the primary visual cortex. As a step to understanding the relationship between position and orientation representation, we investigated psychophysically the minimum spatial requirements for the determination of orientation at various positions across the visual field. We know that the shortest line whose orientation can be resolved varies with eccentricity, such that its length corresponds to slightly less than 0.2 mm projected onto the cortical surface. Along the horizontal meridian horizontal lines are detected with higher precision than vertical or oblique lines. In the present experiments, we tested whether this is a preference for horizontal lines or for lines that are orientated radially away from the fovea. METHODS: Human observers were tested with lines positioned at one vertical, two horizontal and two oblique meridians at eccentricities between 5 and 25 degrees. RESULTS/CONCLUSION: Three of the four subjects were most sensitive for targets aligned with the meridian of presentation. This suggests that the visual system has the highest resolution in directions radiating from the fovea, which may be particularly useful for the analysis of flow fields resulting from forward translation.