Perceptual separation of transparent motion components: the interaction of motion, luminance and shape cues.

Transparency is perceived when two or more objects or surfaces can be separated by the visual system whilst they are presented in the same region of the visual field at the same time. This segmentation of distinct entities on the basis of overlapping local visual cues poses an interesting challenge for the understanding of cortical information processing. In psychophysical experiments, we studied stimuli that contained randomly positioned disc elements, moving at two different speeds in the same direction, to analyse the interaction of cues during the perception of motion transparency. The current work extends findings from previous experiments with sine wave luminance gratings which only vary in one spatial dimension. The reported experiments manipulate low-level cues, like differences in speed or luminance, and what are likely to be higher level cues such as the relative size of the elements or the superposition rules that govern overlapping regions. The mechanism responsible for separation appears to be mediated by combination of the relevant and available cues. Where perceived transparency is stronger, the neural representations of components are inferred to be more distinguishable from each other across what appear to be multiple cue dimensions. The disproportionally large effect on transparency strength of the type of superposition of disc suggests that with this manipulation, there may be enhanced separation above what might be expected from the linear combination of low-level cues in a process we term labelling. A mechanism for transparency perception consistent with the current results would require a minimum of three stages; in addition to the local motion detection and global pooling and separation of motion signals, findings suggest a powerful additional role of higher level separation cues.

Variation in the local motion statistics of real-life optic flow scenes.

Optic flow motion patterns can be a rich source of information about our own movement and about the structure of the environment we are moving in. We investigate the information available to the brain under real operating conditions by analyzing video sequences generated by physically moving a camera through various typical human environments. We consider to what extent the motion signal maps generated by a biologically plausible, two-dimensional array of correlation-based motion detectors (2DMD) not only depend on egomotion, but also reflect the spatial setup of such environments. We analyzed the local motion outputs by extracting the relative amounts of detected directions and comparing the spatial distribution of the motion signals to that of idealized optic flow. Using a simple template matching estimation technique, we are able to extract the focus of expansion and find relatively small errors that are distributed in characteristic patterns in different scenes. This shows that all types of scenes provide suitable motion information for extracting ego motion despite the substantial levels of noise affecting the motion signal distributions, attributed to the sparse nature of optic flow and the presence of camera jitter. However, there are large differences in the shape of the direction distributions between different types of scenes; in particular, man-made office scenes are heavily dominated by directions in the cardinal axes, which is much less apparent in outdoor forest scenes. Further examination of motion magnitudes at different scales and the location of motion information in a scene revealed different patterns across different scene categories. This suggests that self-motion patterns are not only relevant for deducing heading direction and speed but also provide a rich information source for scene structure and could be important for the rapid formation of the gist of a scene under normal human locomotion.

Manipulating the content of dynamic natural scenes to characterize response in human MT/MST.

Optic flow is one of the most important sources of information for enabling human navigation through the world. A striking finding from single-cell studies in monkeys is the rapid saturation of response of MT/MST areas with the density of optic flow type motion information. These results are reflected psychophysically in human perception in the saturation of motion aftereffects. We began by comparing responses to natural optic flow scenes in human visual brain areas to responses to the same scenes with inverted contrast (photo negative). This changes scene familiarity while preserving local motion signals. This manipulation had no effect; however, the response was only correlated with the density of local motion (calculated by a motion correlation model) in V1, not in MT/MST. To further investigate this, we manipulated the visible proportion of natural dynamic scenes and found that areas MT and MST did not increase in response over a 16-fold increase in the amount of information presented, i.e., response had saturated. This makes sense in light of the sparseness of motion information in natural scenes, suggesting that the human brain is well adapted to exploit a small amount of dynamic signal and extract information important for survival.

Similarity of gaze patterns across physical and virtual versions of an installation artwork.

An experiment was conducted to compare museum visitors' gaze patterns using mobile eye-trackers, whilst they were engaging with a physical and a virtual reality (VR) installation of Piet Mondrian's Neo-plasticist room design. Visitors' eye movements produced approximately 25,000 fixations and were analysed using linear mixed-effects models. Absolute and area-normalized dwell time analyses yielded mostly non-significant main effects of the environment, indicating similarity of visual exploration patterns between physical and VR settings. One major difference observed was the decrease of average fixation duration in VR, where visitors tended to more rapidly switch focus in this environment with shorter bursts of attentional focus. The experiment demonstrated the ability to compare gaze data between physical and virtual environments as a proxy to measure the similarity of aesthetic experience. Similarity of viewing patterns along with questionnaire results suggested that virtual galleries can be treated as ecologically valid environments that are parallel to physical art galleries.

Animal communication: reading lizards' body language in context.

A recent study has shown that Jacky lizards adjust their movement-based visual signaling in response to the varying environmental conditions; the results indicate that this species has highly sophisticated communication and sensory processing strategies.

Microsaccades and preparatory set: a comparison between delayed and immediate, exogenous and endogenous pro- and anti-saccades.

When we fixate an object, our eyes are not entirely still, but undergo small displacements such as microsaccades. Here, we investigate whether these microsaccades are sensitive to the preparatory processes involved in programming a saccade. We show that the frequency of microsaccades depends in a specific manner on the intention where to move the eyes (towards a target location or away from it), when to move (immediately after the onset of the target or after a delay), and what type of cue is followed (a peripheral onset or a centrally presented symbolic cue). In particular, in the preparatory interval before and early after target onset, more microsaccades were found when a delayed saccade towards a peripheral target was prepared than when a saccade away was programmed. However, no such difference in the frequency of microsaccades was observed when saccades were initiated immediately after the onset of the target or when the saccades were programmed on the basis of a centrally presented arrow cue. The results are discussed in the context of the neural correlates of response preparation, known as preparatory set.

Quantifying and modeling the strength of motion illusions perceived in static patterns.

The origin of motion illusions in simple black and white patterns such those as used by Op artists has been at the center of a lively scientific debate, relating motion processing mechanisms to involuntary eye movements that generate characteristic motion patterns. To overcome the limitations of using subjective ratings as a measure of illusory effects, we developed a new method to quantify the strength of the illusion for synthetic 'riloids' that were inspired by Bridget Riley's 'Fall'. In a 2AFC paradigm, test stimuli were compared to a reference set of patterns that elicit illusory motion of variable strength. We found that pattern parameters influencing the distribution of local orientation in the riloids (the amplitude and the spatial period of the line undulation) systematically affect illusion strength, whereas other parameters, such as the spatial period of the lines themselves, the duration of the stimulus, or fixation conditions, have little effect. These behavioral data are compared in computer simulations to the predicted activity generated by motion detector networks for displacements of the riloids that reflect small eye movements. The match between predicted illusion strength and experimental data support an explanation of the motion illusion in terms of retinal image shifts.

The combined effect of eye movements improve head centred local motion information during walking.

Eye movements play multiple roles in human behaviour-small stabilizing movements are important for keeping the image of the scene steady during locomotion, whilst large scanning movements search for relevant information. It has been proposed that eye movement induced retinal motion interferes with the estimation of self-motion based on optic flow. We investigated the effect of eye movements on retinal motion information during walking. Observers walked towards a target, wearing eye tracking glasses that simultaneously recorded the scene ahead and tracked the movements of both eyes. By realigning the frames of the recording from the scene ahead, relative to the centre of gaze, we could mimic the input received by the retina (retinocentric coordinates) and compare this to the input received by the scene camera (head centred coordinates). We asked which of these coordinate frames resulted in the least noisy motion information. Motion noise was calculated by finding the error in between the optic flow signal and a noise-free motion expansion pattern. We found that eye movements improved the optic flow information available, even when large diversions away from target were made.

Speed encoding in correlation motion detectors as a consequence of spatial structure.

For animals to carry out a wide range of detection, recognition and navigation tasks, visual motion signals are crucial. The encoding of motion information has therefore, attracted much attention in the experimental and computational study of brain function. Two main alternative mechanisms have been proposed on the basis of behavioural and physiological experiments. On one hand, correlation-type and motion energy detectors are simple and efficient in the design of their basic mechanism but are tuned to temporal frequency rather than to speed. On other hand, gradient-type motion detectors directly represent an estimate of speed, but may require more demanding processing mechanisms. We demonstrate here how the temporal frequency dependence observed for sine-wave gratings can disappear for less constrained stimuli, to be replaced by responses reflecting speed for stimuli like square waves when a phase-sensitive detection mechanism is employed. We conclude from these observations that temporal frequency tuning is not necessarily a limitation for motion vision based on correlation detectors, and more generally demonstrate in view of the typical Fourier composition of natural scenes, that correlation detectors operating in such environments can encode image speed. In the context of our results, we discuss the implications of the loss of phase sensitivity inherent in using a linear system approach to describe neural processing.

Characterizing motion contour detection mechanisms and equivalent mechanisms in the luminance domain.

Motion-defined contours are ecologically important cues to object boundaries in complex fields of optic flow. We designed a novel stimulus in which the velocities of randomly positioned dots are defined by a 2D Gabor function, resulting in a motion-defined pattern with a clear orientation. We found that the number of correct responses in a vertical/horizontal orientation discrimination task increases and saturates with size of the Gabor envelope at around 4-5 degrees full width at half height. The number of correct responses decreases with higher spatial frequency of the Gabor patterns. The best performance occurs at 0.1 cycles/degree, when only a single contour is visible. Using elliptical Gabor stimuli, we found that accuracy is higher if the patch is elongated along the contours (rather than orthogonal to them), confirming the existence of an elongated detector mechanism for a single contour. We compared tuning properties for motion-defined Gabor patterns with sparsely defined luminance Gabor patterns and found similar results, but only at low sampling densities. The nature of the information and the strength of the signal influence the properties of luminance contour detection mechanisms, whereas motion contour detection may be limited by the sparse visual representation of the motion field.

Prey Capture: Becoming Invisible When You Move.

Conspicuous skin patterns attract the attention of predators, but are thought also to act as protective mimicry during movement. A new behavioural study of mantises has found that the responses of these insect predators to a striped dummy target are reduced when the target is moving at high speed.

Combining direction and speed for the localisation of visual motion defined contours.

Detecting discontinuities in motion signal distributions is an essential operation of visual systems, contributing to perception and visuo-motor control. Discontinuities can be signalled by a difference in speed, direction or both. We measured how localisation accuracy for a motion defined contour depends on the velocity differences that define it. A vertical motion contour was defined by two fields of random dots with systematically varied combinations of speed and direction. We find that our data is best explained by assuming that localisation precision is inversely proportional to direction and speed differences that are linearly summed and weighted according to reliability, the optimal solution for combining independent estimates.

Brain mechanisms of recovery from pure alexia: A single case study with multiple longitudinal scans.

Pure alexia is an acquired reading disorder, typically due to a left occipito-temporal lesion affecting the Visual Word Form Area (VWFA). It is unclear whether the VWFA acts as a unique bottleneck for reading, or whether alternative routes are available for recovery. Here, we address this issue through the single-case longitudinal study of a neuroscientist who experienced pure alexia and participated in 17 behavioral, 9 anatomical, and 9 fMRI assessment sessions over a period of two years. The origin of the impairment was assigned to a small left fusiform lesion, accompanied by a loss of VWFA responsivity and by the degeneracy of the associated white matter pathways. fMRI experiments allowed us to image longitudinally the visual perception of words, as compared to other classes of stimuli, as well as the mechanisms of letter-by-letter reading. The progressive improvement of reading was not associated with the re-emergence of a new area selective to words, but with increasing responses in spared occipital cortex posterior to the lesion and in contralateral right occipital cortex. Those regions showed a non-specific increase of activations over time and an increase in functional correlation with distant language areas. Those results confirm the existence of an alternative occipital route for reading, bypassing the VWFA, but they also point to its key limitation: the patient remained a slow letter-by-letter reader, thus supporting the critical importance of the VWFA for the efficient parallel recognition of written words.

A Gaze-Driven Evolutionary Algorithm to Study Aesthetic Evaluation of Visual Symmetry.

Empirical work has shown that people like visual symmetry. We used a gaze-driven evolutionary algorithm technique to answer three questions about symmetry preference. First, do people automatically evaluate symmetry without explicit instruction? Second, is perfect symmetry the best stimulus, or do people prefer a degree of imperfection? Third, does initial preference for symmetry diminish after familiarity sets in? Stimuli were generated as phenotypes from an algorithmic genotype, with genes for symmetry (coded as deviation from a symmetrical template, deviation-symmetry, DS gene) and orientation (0° to 90°, orientation, ORI gene). An eye tracker identified phenotypes that were good at attracting and retaining the gaze of the observer. Resulting fitness scores determined the genotypes that passed to the next generation. We recorded changes to the distribution of DS and ORI genes over 20 generations. When participants looked for symmetry, there was an increase in high-symmetry genes. When participants looked for the patterns they preferred, there was a smaller increase in symmetry, indicating that people tolerated some imperfection. Conversely, there was no increase in symmetry during free viewing, and no effect of familiarity or orientation. This work demonstrates the viability of the evolutionary algorithm approach as a quantitative measure of aesthetic preference.

Attention modulates perception of transparent motion.

Human observers can extract a given motion direction from sets of random dots moving simultaneously in two or more directions in the same region of the visual field, a phenomenon referred to as motion transparency. As a necessary condition for separating transparent motion directions, low level encoding of local motion signals must generate frequency distributions of local directions with separable peaks corresponding to these directions--this process would be constrained by local stimulus attributes and the properties of local motion detectors. Furthermore, a representation of multiple directions is needed for simultaneous retrieval of several directions in a psychophysical task--this operation would be limited by higher level processes, such as attention selecting a particular direction to rise into awareness. Preliminary observations suggest that the number of directions that can be seen simultaneously is rather limited and the question arises whether this could be related to limitations of low-level encoding or higher level representations. To study specifically the effect of attention on transparent motion perception, observers were presented with sets of dots moving coherently in a variable number of directions, and were asked after the presentation whether one particular direction was present in the set. When the direction of motion was not known before stimulus onset (uncued condition), observers detected a particular motion direction among no more than 3 other directions. When direction of motion was indicated prior to stimulus onset (precued condition), however, this limit increased up to 6 directions. This attentional effect showed some inter-individual variability and appeared to benefit from spatiotemporal integration of the motion signals. A corresponding effect became apparent when observers were tested in the same paradigm whether they could separate two motion directions with variable angular difference between them. In the precued condition a typical minimum direction difference was about 60 degrees, whereas in the uncued condition this was about 120 degrees, suggesting that the performance in detecting one direction in a multiple direction stimulus might be limited by the ability to separate adjacent motion directions. This pattern of results suggests that attention can reliably improve transparent motion processing by affecting the separability of directional signals in low level encoding mechanisms.

Motion camouflage induced by zebra stripes.

The functional significance of the zebra coat stripe pattern is one of the oldest questions in evolutionary biology, having troubled scientists ever since Charles Darwin and Alfred Russel Wallace first disagreed on the subject. While different theories have been put forward to address this question, the idea that the stripes act to confuse or 'dazzle' observers remains one of the most plausible. However, the specific mechanisms by which this may operate have not been investigated in detail. In this paper, we investigate how motion of the zebra's high contrast stripes creates visual effects that may act as a form of motion camouflage. We simulated a biologically motivated motion detection algorithm to analyse motion signals generated by different areas on a zebra's body during displacements of their retinal images. Our simulations demonstrate that the motion signals that these coat patterns generate could be a highly misleading source of information. We suggest that the observer's visual system is flooded with erroneous motion signals that correspond to two well-known visual illusions: (i) the wagon-wheel effect (perceived motion inversion due to spatiotemporal aliasing); and (ii) the barber-pole illusion (misperceived direction of motion due to the aperture effect), and predict that these two illusory effects act together to confuse biting insects approaching from the air, or possibly mammalian predators during the hunt, particularly when two or more zebras are observed moving together as a herd.

Investigating preferences for color-shape combinations with gaze driven optimization method based on evolutionary algorithms.

Studying aesthetic preference is notoriously difficult because it targets individual experience. Eye movements provide a rich source of behavioral measures that directly reflect subjective choice. To determine individual preferences for simple composition rules we here use fixation duration as the fitness measure in a Gaze Driven Evolutionary Algorithm (GDEA), which has been demonstrated as a tool to identify aesthetic preferences (Holmes and Zanker, 2012). In the present study, the GDEA was used to investigate the preferred combination of color and shape which have been promoted in the Bauhaus arts school. We used the same three shapes (square, circle, triangle) used by Kandinsky (1923), with the three color palette from the original experiment (A), an extended seven color palette (B), and eight different shape orientation (C). Participants were instructed to look for their preferred circle, triangle or square in displays with eight stimuli of different shapes, colors and rotations, in an attempt to test for a strong preference for red squares, yellow triangles and blue circles in such an unbiased experimental design and with an extended set of possible combinations. We Tested six participants extensively on the different conditions and found consistent preferences for color-shape combinations for individuals, but little evidence at the group level for clear color/shape preference consistent with Kandinsky's claims, apart from some weak link between yellow and triangles. Our findings suggest substantial inter-individual differences in the presence of stable individual associations of color and shapes, but also that these associations are robust within a single individual. These individual differences go some way toward challenging the claims of the universal preference for color/shape combinations proposed by Kandinsky, but also indicate that a much larger sample size would be needed to confidently reject that hypothesis. Moreover, these experiments highlight the vast potential of the GDEA methodology in experimental aesthetics and beyond.

Using an oculomotor signature as an indicator of aesthetic preference.

Eye movements are strongly influenced by the task given to an observer. The immediacy of such eye movements, which are difficult to control consciously, offers the potential to explore highly variable subjective evaluations, such as aesthetic preference, with reliable objective measures. We presented a variety of images in sets of 2, 4, or 8 items for different durations and analyzed oculomotor statistics such as cumulative fixation duration, refixations, and the sequence of fixations while participants searched for their preferred image, after which participants indicated their preference using a button press. The total amount of time spent looking at any image correlates with selection preference and does so increasingly well with longer presentation duration. For short presentations, the first and last fixations correlate better with image preference. All response measures become increasingly variable as the number and complexity of presented images are increased. A weighted combination of these measures can significantly improve the correlation with preference, suggesting a "signature" which could be used as a reliable indicator for task-free subjective evaluation of stimuli in visual psychophysics. Its role as an improved fitness function in visually driven evolutionary algorithms is discussed.

The movement of motion-defined contours can bias perceived position.

Illusory position shifts induced by motion suggest that motion processing can interfere with perceived position. This may be because accurate position representation is lost during successive visual processing steps. We found that complex motion patterns, which can only be extracted at a global level by pooling and segmenting local motion signals and integrating over time, can influence perceived position. We used motion-defined Gabor patterns containing motion-defined boundaries, which themselves moved over time. This 'motion-defined motion' induced position biases of up to 0.5 degrees , much larger than has been found with luminance-defined motion. The size of the shift correlated with how detectable the motion-defined motion direction was, suggesting that the amount of bias increased with the magnitude of this complex directional signal. However, positional shifts did occur even when participants were not aware of the direction of the motion-defined motion. The size of the perceptual position shift was greatly reduced when the position judgement was made relative to the location of a static luminance-defined square, but not eliminated. These results suggest that motion-induced position shifts are a result of general mechanisms matching dynamic object properties with spatial location.

Perceiving motion transparency in the absence of component direction differences.

The simultaneous perception of multiple motion components within the same region in the visual field is a difficult processing task, which can be solved by human observers for a range of transparently moving stimuli. We use transparently moving gratings to study this phenomenon psychophysically, focussing on configurations in which individual components move in the same direction and can only be discriminated by speed differences. We first demonstrate that the stimuli are perceived as transparent and then proceed to quantify how the strength of motion transparency changes while component grating parameters such as fundamental spatial frequency, speed and luminance are varied. The results were consistent with perception resolving a signal detection task of separating two superimposed global motion signals corresponding to each of the components. We also identify the importance of broadband stimuli containing edges, both for perceiving transparency with the same direction stimulus configuration, and for static transparency. The local density of edges has a direct influence on the strength of perceived transparency, suggesting that local motion detection at the edges of the stimuli, which is sensitive to speed differences, may be critical to solve the task. The work suggests that there may be a simultaneous retinotopic representation of the two speeds of motion analogous to that accomplished by the motion direction tuned neurons found across regions of visual cortex.