Paradoxical stabilization of relative position in moving frames.

To capture where things are and what they are doing, the visual system may extract the position and motion of each object relative to its surrounding frame of reference [K. Duncker, Routledge and Kegan Paul, London 161-172 (1929) and G. Johansson, Acta Psychol (Amst.) 7, 25-79 (1950)]. Here we report a particularly powerful example where a paradoxical stabilization is produced by a moving frame. We first take a frame that moves left and right and we flash its right edge before, and its left edge after, the frame's motion. For all frame displacements tested, the two edges are perceived as stabilized, with the left edge on the left and right edge on the right, separated by the frame's width as if the frame were not moving. This stabilization is paradoxical because the motion of the frame itself remains visible, albeit much reduced. A second experiment demonstrated that unlike other motion-induced position shifts (e.g., flash lag, flash grab, flash drag, or Fröhlich), the illusory shift here is independent of speed and is set instead by the distance of the frame's travel. In this experiment, two probes are flashed inside the frame at the same physical location before and after the frame moves. Despite being physically superimposed, the probes are perceived widely separated, again as if they were seen in the frame's coordinates and the frame were stationary. This paradoxical stabilization suggests a link to visual stability across eye movements where the displacement of the entire visual scene may act as a frame to stabilize the perception of relative locations.

Influence of frame and probe paths on the frame effect.

Moving frames produce large displacements in the perceived location of flashed and continuously moving probes. In a series of experiments, we test the contributions of the probe's displacement and the frame's displacement on the strength of the frame's effect. In the first experiment, we find a dramatic position shift of flashed probes whereas the effect on a continuously moving probe is only one-third as strong. In Experiment 2, we show that the absence of an effect for the static probe is a consequence of its perceptual grouping with the static background. As long as the continuously present probe has some motion, it appears to group to some extent with the frame and show an illusory shift of intermediate magnitude. Finally, we informally explored the illusory shifts seen for a continuously moving probe when the frame itself has a more complex path. In this case, the probe appears to group more strongly with the frame. Overall, the effects of the frame on the probe demonstrate the outcome of a competition between the frame and the static background in determining the frame of reference for the probe's perceived position.

Different extrapolation of moving object locations in perception, smooth pursuit, and saccades.

The ability to accurately perceive and track moving objects is crucial for many everyday activities. In this study, we use a "double-drift stimulus" to explore the processing of visual motion signals that underlie perception, pursuit, and saccade responses to a moving object. Participants were presented with peripheral moving apertures filled with noise that either drifted orthogonally to the aperture's direction or had no net motion. Participants were asked to saccade to and track these targets with their gaze as soon as they appeared and then to report their direction. In the trials with internal motion, the target disappeared at saccade onset so that the first 100 ms of the postsaccadic pursuit response was driven uniquely by peripheral information gathered before saccade onset. This provided independent measures of perceptual, pursuit, and saccadic responses to the double-drift stimulus on a trial-by-trial basis. Our analysis revealed systematic differences between saccadic responses, on one hand, and perceptual and pursuit responses, on the other. These differences are unlikely to be caused by differences in the processing of motion signals because both saccades and pursuits seem to rely on shared target position and velocity information. We conclude that our results are instead due to a difference in how the processing mechanisms underlying perception, pursuit, and saccades combine motor signals with target position. These findings advance our understanding of the mechanisms underlying dissociation in visual processing between perception and eye movements.

Sustained attention and the flash grab effect.

When a stationary target is briefly presented on top of a moving background as it reverses direction, the target is displaced perceptually in the direction of the upcoming motion (the flash grab effect). To determine the role of attention in this effect, we investigated whether the predictability of the location of the flash grab target modulates the illusion. First, we established that effect was weaker for spatially predictable targets. Next, we showed that the flash grab effect decreased for a narrower spatial spread of attention before the onset of the target and that it was smaller for left hemifield presentations than right. Finally, we demonstrated that diverting attention away from the target and the background motion decreases the flash grab effect. In the first two experiments, the decrease in the illusion could be attributed to either increased attention to the target or decreased attention to the motion; we assume that increasing attention to the target necessarily decreases attention to the motion. However, in the final experiment, the central task decreases attention to both the target and the motion. The results show a decrease in the illusion and that reveals that attention to the motion is the primary causal factor.

Motion-induced distortion of shape.

Motion, position, and form are intricately intertwined in perception. Motion distorts visual space, resulting in illusory position shifts such as flash-drag and flash-grab effects. The flash-grab displaces a test by up to several times its size. This lets us use it to investigate where the motion-induced shift operates in the processing stream from photoreceptor activation to feature activation to object recognition. We present several canonical, highly familiar forms and ask whether the motion-induced shift operates uniformly across the form. If it did, we could conclude that the effect occurred after the elements of the form are bound. However, we find that motion-induced distortion affects not only the position, but also the appearance of briefly presented, canonical shapes (square, circle, and letter T). Features of the flashed target that were closest to its center were shifted in the direction of motion more than those further from its center. Outline shapes were affected more than filled shapes, and the strength of the distortion increased with the contrast of the moving background. This not only supports a nonuniform spatial profile for the motion-induced shift but also indicates that the shift operates before the shape is established, even for highly familiar shapes like squares, circles, and letters.

The art of the float.

For more than 2000 years, artists have exploited cast shadows to influence how objects appear to be positioned in a scene. A contact cast shadow can anchor an object to the ground and a detached cast shadow can make an object appear to float. However, there is a period of approximately 1000 years when there were virtually no cast shadows in art. How were states of contact versus floating depicted by artists without cast shadows? Here, we survey various techniques used by artists to anchor relative position with and without cast shadows. We then conduct experimental tests of the hypothesized surface attraction principles that underlie these techniques. In the absence of cast shadows, an object (a wooden box) was often seen as resting on a surface as long as that surface offered information about ground orientation and support (a tiled floor). When the ground surface was ambiguous and cloud-like (1/f noise), the box was more likely to be seen to float. The presence of cast shadows made the box appear to contact the ground whether it was well-defined or ambiguous. Both shadows and surface support also increased the accuracy with which participants detected when the box was tilted up from the ground. These results indicate that artists long ago discovered the important power of support relationships to anchor objects to surfaces in the absence of shadows.

The double-drift illusion biases the marmoset oculomotor system.

The double-drift illusion has two unique characteristics: The error between the perceived and physical position of the stimulus grows over time, and saccades to the moving target land much closer to the physical than the perceived location. These results suggest that the perceptual and saccade targeting systems integrate visual information over different time scales. Functional imaging studies in humans have revealed several potential cortical areas of interest, including the prefrontal cortex. However, we currently lack an animal model to study the neural mechanisms of location perception that underlie the double-drift illusion. To fill this gap, we trained two marmoset monkeys to fixate and then saccade to the double-drift stimulus. In line with human observers for radial double-drift trajectories with fast internal motion, we find that saccade endpoints show a significant bias that is, nevertheless, smaller than the bias seen in human perceptual reports. This bias is modulated by changes in the external and internal speeds of the stimulus. These results demonstrate that the saccade targeting system of the marmoset monkey is influenced by the double-drift illusion.

Mansfieldite (AlAsO4·2H2O): a new white pigment in Himalayan artwork.

An unusual arsenate mineral, mansfieldite (AlAsO4·2H2O), was identified as a pigment for the first time as the principal white colorant on two Himalayan thangka paintings at the Indianapolis Museum of Art at Newfields. The co-occurrence of this unusual mineral pigment provides support for the belief that the two artworks are members of a cycle of paintings originating from the same workshop, perhaps from Chamdo, Tibet. The complete palettes of both artworks are identical, including the use of mansfieldite, brochantite, malachite, azurite, vermilion, gold, orpiment, and a carbon-based black in a glue binder on a calcite and gypsum-primed cotton fabric.

Measuring the double-drift illusion and its resets with hand trajectories.

If a Gabor pattern drifts in one direction while its internal texture drifts in the orthogonal direction, its perceived position deviates further and further away from its true path. We first evaluated the illusion using manual tracking. Participants followed the Gabor with a stylus on a drawing tablet that coincided optically with the horizontal monitor surface. Their hand and the stylus were not visible during the tracking. The magnitude of the tracking illusion corresponded closely to previous perceptual and pointing measures indicating that manual tracking is a valid measure for the illusion. This allowed us to use it in a second experiment to capture the behavior of the illusion as it eventually degrades and breaks down in single trials. Specifically, the deviation of the Gabor stops accumulating at some point and either stays at a fixed offset or resets toward the veridical position. To report the perceived trajectory of the Gabor, participants drew it after the Gabor was removed from the monitor. Resets were detected and analyzed and their distribution matches neither a temporal nor a spatial limit, but rather a broad gamma distribution over time. This suggests that resets are triggered randomly, about once per 1.3 seconds, possible by extraneous distractions or eye movements.

Endogenous attention biases transformational apparent motion based on high-level shape representations.

When two pre-existing, separated squares are connected by the sudden onset of a bar between them, viewers do not perceive the bar to appear all at once. Instead, they see an illusory morphing of the original squares over time. The direction of this transformational apparent motion (TAM) can be influenced by endogenous attention deployed before the appearance of the connecting bar. Here, we investigated whether the influence of endogenous attention on TAM results from operations over high-level feature-independent shape representations, or instead over lower level shape representations defined by specific visual features. To do so, we tested the influence of endogenous attention on TAM in first- and second-order displays, which shared common shapes but had different shape-defining attributes (luminance and texture contrast, respectively). In terms of both the magnitude of directional bias and timing, we found that endogenous attention exerted a similar influence on both first- and second-order objects. These results imply that endogenous attention biases the perceived direction of TAM by operating on high-level shape representations that are invariant to the low-level visual features that define them. Our results support a four-stage model of TAM, where a feature encoding stage passes a features-specific layout to a parsing stage that forms discrete, high-level meta-featural shapes, which are then matched and visually interpolated over time.

A motion-induced position shift that depends on motion both before and after the test probe.

Two versions of the flash grab illusion were used to examine the relative contributions of motion before and motion after the test flash to the illusory position shift. The stimulus in the first two experiments was a square pattern that expanded and contracted with an outline square flashed each time the motion reversed producing a dramatic difference in perceived size between the two reversals. Experiment 1 showed a strong illusion when motion was present before and after the flashed tests or just after the flashes, but no significant effect when only the pre-flash motion was present. In Experiment 2, motion always followed the flash, and the duration of the pre-flash motion was varied. The results showed a significant increase in illusion strength with the duration of pre-flash motion and the effect of the pre-flash motion was almost 50% that of the post-flash motion. Finally, Experiment 3 tested the position shifts when the linear motion of a disk before the flash was orthogonal to its motion after the flash. Here, the results again showed that the pre-flash motion made a significant contribution, about 32% that of the post-flash motion. Several models are considered and even though all fail to some degree, they do offer insights into the nature of the illusion. Finally, we show that the empirical measure of the relative contribution of motion before and after the flash can be used to distinguish the mechanisms underlying different illusions.

Exploring the frame effect.

Probes flashed within a moving frame are dramatically displaced (Özkan, Anstis, 't Hart, Wexler, & Cavanagh, 2021; Wong & Mack, 1981). The effect is much larger than that seen on static or moving probes (induced motion, Duncker, 1929; Wallach, Bacon, & Schulman, 1978). These flashed probes are often perceived with the separation they have in frame coordinates-a 100% effect (Özkan et al., 2021). Here, we explore this frame effect on flashed tests with several versions of the standard stimulus. We find that the frame effect holds for smoothly or abruptly displacing frames, even when the frame changed shape or orientation between the end points of its travel. The path could be nonlinear, even circular. The effect was driven by perceived not physical motion. When there were competing overlapping frames, the effect was determined by which frame was attended. There were a number of constraints that limited the effect. A static anchor near the flashes suppressed the effect but an extended static texture did not. If the probes were continuous rather than flashed, the effect was abolished. The observational reports of 30 online participants suggest that the frame effect is robust to many variations in its shape and path and leads to a perception of flashed tests in their locations relative to the frame as if the frame were stationary. Our results highlight the role of frame continuity and of the grouping of the flashes with the frame in generating the frame effect.

The Language of Vision.

The descriptions of surfaces, objects, and events computed by visual processes are not solely for consumption in the visual system but are meant to be passed on to other brain centers. Clearly, the description of the visual scene cannot be sent in its entirety, like a picture or movie, to other centers, as that would require that each of them have their own visual system to decode the description. Some very compressed, annotated, or labeled version must be constructed that can be passed on in a format that other centers-memory, language, planning-can understand. If this is a "visual language," what is its grammar? In a first pass, we see, among other things, differences in processing of visual "nouns," visual "verbs," and visual "prepositions." Then we look at recursion and errors of visual grammar. Finally, the possibility of a visual language also raises the question of the acquisition of its grammar from the visual environment and the chance that this acquisition process was borrowed and adapted for spoken language.

A position anchor sinks the double-drift illusion.

When the internal texture of a Gabor patch moves orthogonally to its envelope's motion, the perceived path, viewed in the periphery, shifts dramatically in position, and direction relative to the true path (the double-drift illusion). Here, we examine positional uncertainty as a critical factor underlying this illusory shift. We presented participants with an anchoring line at different distances from the drifting Gabor's physical path. Our results indicate that placing an anchor (a fixed line) close to the Gabor's path halved the magnitude of the illusion. This suppression was symmetrical for anchors placed on either side of the Gabor. In a second experiment, we used crowding to degrade the anchoring line's position information by embedding it in a set of parallel lines. In this case, despite the presence of the same lines that reduced the illusion when presented in isolation, the illusory shift was now largely restored. We suggest that the adjacent lines crowded each other, reducing their positional certainty, and thus their ability to anchor the location of the moving Gabor. These findings indicate that the positional uncertainty of the equiluminant Gabor patch is critical for the illusory position offset.

Dynamic presentation boosts the Ebbinghaus illusion but reduces the Müller-Lyer and orientation contrast illusions.

Mruczek et al. (2015) showed that a moving version of the Ebbinghaus illusion almost doubles in strength compared to the standard version. In their stimulus, the size of the surrounding inducers was modulated between large and small and the whole stimulus was made to drift during the surround modulation. We first replicated the original dynamic Ebbinghaus illusion and then explored dynamic presentations for other simultaneous contrast and geometric illusions. We found no increase in illusion strength in any that we sampled. Here we report the results for the Müller-Lyer illusion and the orientation contrast illusion. Surprisingly, when these two illusions were presented dynamically, their effects were greatly reduced for the Müller-Lyer illusion and eliminated for the orientation contrast illusion.

Scaling depth from shadow offset.

When an object casts a shadow on a background surface, both the offset of the shadow and the blur of its penumbra are potential cues to the distance between the object and the background. However, the shadow offset and blur are also affected by the direction and angular extent of the light source and these are often unknown. This means that the observer must make some assumptions about the illumination, the expected distribution of depth, or the relation between offset and depth in order to use shadows to make distance judgments. Here, we measure human judgments of perceived depth over a range of shadow offsets, blurs, and lighting directions to gain insight into this internal model. We find that distance judgments are relatively unaffected by blur or light direction, whereas the shadow offset has a strong and linear effect. The data are consistent with two models, a generic shadow-to-depth model and a Bayesian model.

Effects of internal and external velocity on the perceived direction of the double-drift illusion.

In the double-drift illusion, the combination of the internal and external motion vectors produces large misperceptions of both position and direction of motion. Here, we investigate the role that speed plays in determining how these two sources of motion are combined to produce the double-drift illusion. To address this question, we measure the size of the illusion at seven internal speeds combined with six external speeds. We find that the illusion increases with increasing internal speed and decreases with increasing external speed. We model this by combining the external and internal vectors to produce the resulting, illusory direction (Tse & Hsieh, 2006). The relative effect of the two vectors is specified by a constant K in this model and the data reveal that K decreases linearly as external speed increases. This critical role of external speed in modulating the vector combination uncovers new details about how the visual system combines different sources of motion information to produce a global motion percept.

Smooth pursuit operates over perceived not physical positions of the double-drift stimulus.

The double-drift illusion produces a large deviation in perceived direction that strongly dissociates physical position from perceived position. Surprisingly, saccades do not seem to be affected by the illusion (Lisi & Cavanagh, 2015). When targeting a double-drift stimulus, the saccade system is driven by retinal rather than perceived position. Here, using paired double-drift targets, we test whether the smooth pursuit system is driven by perceived or physical position. Participants (n = 7) smoothly pursued the inferred midpoint (Steinbach, 1976) between two horizontally aligned Gabor patches that were separated by 20° and moving on parallel, oblique paths. On the first half of each trial, the Gabors' internal textures were static while both drifted obliquely downward. On the second half of each trial, while the envelope moved obliquely upward, the internal texture drifted orthogonally to the envelope's motion, producing a large perceived deviation from the downward path even though the upward and downward trajectories always followed the same physical path but in opposite directions. We find that smooth pursuit eye movements accurately followed the nonillusory downward path of the midpoint between the two Gabors, but then followed the illusory rather than the physical trajectory on the upward return. Thus, virtual targets for smooth pursuit are derived from perceived rather than retinal coordinates.

Variations in crowding, saccadic precision, and spatial localization reveal the shared topology of spatial vision.

Visual sensitivity varies across the visual field in several characteristic ways. For example, sensitivity declines sharply in peripheral (vs. foveal) vision and is typically worse in the upper (vs. lower) visual field. These variations can affect processes ranging from acuity and crowding (the deleterious effect of clutter on object recognition) to the precision of saccadic eye movements. Here we examine whether these variations can be attributed to a common source within the visual system. We first compared the size of crowding zones with the precision of saccades using an oriented clock target and two adjacent flanker elements. We report that both saccade precision and crowded-target reports vary idiosyncratically across the visual field with a strong correlation across tasks for all participants. Nevertheless, both group-level and trial-by-trial analyses reveal dissociations that exclude a common representation for the two processes. We therefore compared crowding with two measures of spatial localization: Landolt-C gap resolution and three-dot bisection. Here we observe similar idiosyncratic variations with strong interparticipant correlations across tasks despite considerably finer precision. Hierarchical regression analyses further show that variations in spatial precision account for much of the variation in crowding, including the correlation between crowding and saccades. Altogether, we demonstrate that crowding, spatial localization, and saccadic precision show clear dissociations, indicative of independent spatial representations, whilst nonetheless sharing idiosyncratic variations in spatial topology. We propose that these topological idiosyncrasies are established early in the visual system and inherited throughout later stages to affect a range of higher-level representations.

Decoding Trans-Saccadic Memory.

We examine whether peripheral information at a planned saccade target affects immediate postsaccadic processing at the fovea on saccade landing. Current neuroimaging research suggests that presaccadic stimulation has a late effect on postsaccadic processing, in contrast to the early effect seen in behavioral studies. Human participants (both male and female) were instructed to saccade toward a face or a house that, on different trials, remained the same, changed, or disappeared during the saccade. We used a multivariate pattern analysis of electroencephalography data to decode face versus house processing directly after the saccade. The classifier was trained on separate trials without a saccade, where a house or face was presented at the fovea. When the saccade target remained the same across the saccade, we could reliably decode the target 123 ms after saccade offset. In contrast, when the target was changed during the saccade, the new target was decoded at a later time-point, 151 ms after saccade offset. The "same" condition advantage suggests that congruent presaccadic information facilitates processing of the postsaccadic stimulus compared with incongruent information. Finally, the saccade target could be decoded above chance even when it had been removed during the saccade, albeit with a slower time course (162 ms) and poorer signal strength. These findings indicate that information about the (peripheral) presaccadic stimulus is transferred across the saccade so that it becomes quickly available and influences processing at its expected new retinal position (the fovea).SIGNIFICANCE STATEMENT Here we provide neural evidence for early information transfer across saccades. Specifically, we examined the effect of presaccadic sensory information on the initial neuronal processing of a postsaccadic stimuli. Using electroencephalography and multivariate pattern analysis, we found the following: (1) that the identity of the presaccadic stimulus modulated the postsaccadic latency of stimulus relevant information; and (2) that a saccadic neural marker for a saccade target stimulus could be detected even when the stimulus had been removed during saccade. These results demonstrate that information about the peripheral presaccadic stimulus was transferred across the saccade and influenced processing at a new retinal position (the fovea) directly after the saccade landed.