Using Illusions to Track the Emergence of Visual Perception.

Everybody loves illusions. At times, the content on the internet seems to be mostly about illusions-shoes, dresses, straight lines looking bent. This attraction has a long history. Almost 2,000 years ago, Ptolemy marveled at how the sail of a distant boat could appear convex or concave. This sense of marvel continues to drive our fascination with illusions; indeed, few other corners of science can boast of such a large reach. However, illusions not only draw in the crowds; they also offer insights into visual processes. This review starts with a simple definition of illusions as conflicts between perception and cognition, where what we see does not agree with what we believe we should see. This mismatch can be either because cognition has misunderstood how perception works or because perception has misjudged the visual input. It is the perceptual errors that offer the chance to track the development of perception across visual regions. Unfortunately, the effects of illusions in different brain regions cannot be isolated in any simple way: Top-down projections from attention broadcast the expected perceptual properties everywhere, obscuring the critical evidence of where the illusion and perception emerge. The second part of this review then highlights the roadblocks to research raised by attention and describes current solutions for accessing what illusions can offer.

The word superiority effect overcomes crowding.

Crowding and the word superiority effect are two perceptual phenomena that influence reading. The identification of the inner letters of a word can be hindered by crowding from adjacent letters, but it can be facilitated by the word context itself (the word superiority effect). In the present study, strings of four-letters (words and non-words) with different inter-letter spacings (ranging from an optimal spacing to produce crowding to a spacing too large to produce crowding) were presented briefly in the periphery and participants were asked to identify the third letter of the string. Each word had a partner word that was identical except for its third letter (e.g., COLD, CORD) so that guessing as the source of the improved performance for words could be ruled out. Unsurprisingly, letter identification accuracy for words was better than non-words. For non-words, it was lowest at closer spacings, confirming crowding. However, for words, accuracy remained high at all inter-letter spacings showing that crowding did not prevent identification of the inner letters. This result supports models of "holistic" word recognition where partial cues can lead to recognition without first identifying individual letters. Once the word is recognized, its inner letters can be recovered, despite their feature loss produced by crowding.

Different extrapolation of moving object locations in perception, smooth pursuit, and saccades.

The ability to accurately perceive and track moving objects is crucial for many everyday activities. In this study, we use a "double-drift stimulus" to explore the processing of visual motion signals that underlie perception, pursuit, and saccade responses to a moving object. Participants were presented with peripheral moving apertures filled with noise that either drifted orthogonally to the aperture's direction or had no net motion. Participants were asked to saccade to and track these targets with their gaze as soon as they appeared and then to report their direction. In the trials with internal motion, the target disappeared at saccade onset so that the first 100 ms of the postsaccadic pursuit response was driven uniquely by peripheral information gathered before saccade onset. This provided independent measures of perceptual, pursuit, and saccadic responses to the double-drift stimulus on a trial-by-trial basis. Our analysis revealed systematic differences between saccadic responses, on one hand, and perceptual and pursuit responses, on the other. These differences are unlikely to be caused by differences in the processing of motion signals because both saccades and pursuits seem to rely on shared target position and velocity information. We conclude that our results are instead due to a difference in how the processing mechanisms underlying perception, pursuit, and saccades combine motor signals with target position. These findings advance our understanding of the mechanisms underlying dissociation in visual processing between perception and eye movements.

Sustained attention and the flash grab effect.

When a stationary target is briefly presented on top of a moving background as it reverses direction, the target is displaced perceptually in the direction of the upcoming motion (the flash grab effect). To determine the role of attention in this effect, we investigated whether the predictability of the location of the flash grab target modulates the illusion. First, we established that effect was weaker for spatially predictable targets. Next, we showed that the flash grab effect decreased for a narrower spatial spread of attention before the onset of the target and that it was smaller for left hemifield presentations than right. Finally, we demonstrated that diverting attention away from the target and the background motion decreases the flash grab effect. In the first two experiments, the decrease in the illusion could be attributed to either increased attention to the target or decreased attention to the motion; we assume that increasing attention to the target necessarily decreases attention to the motion. However, in the final experiment, the central task decreases attention to both the target and the motion. The results show a decrease in the illusion and that reveals that attention to the motion is the primary causal factor.

Assessing perceptual chromatic equiluminance using a reflexive pupillary response.

Equiluminant stimuli help assess the integrity of colour perception and the relationship of colour to other visual features. As a result of individual variation, it is necessary to calibrate experimental visual stimuli to suit each individual's unique equiluminant ratio. Most traditional methods rely on training observers to report their subjective equiluminance point. Such paradigms cannot easily be implemented on pre-verbal or non-verbal observers. Here, we present a novel Pupil Frequency-Tagging Method (PFTM) for detecting a participant's unique equiluminance point without verbal instruction and with minimal training. PFTM analyses reflexive pupil oscillations induced by slow (< 2 Hz) temporal alternations between coloured stimuli. Two equiluminant stimuli will induce a similar pupil dilation response regardless of colour; therefore, an observer's equiluminant point can be identified as the luminance ratio between two colours for which the oscillatory amplitude of the pupil at the tagged frequency is minimal. We compared pupillometry-based equiluminance ratios to those obtained with two established techniques in humans: minimum flicker and minimum motion. In addition, we estimated the equiluminance point in non-human primates, demonstrating that this new technique can be successfully employed in non-verbal subjects.

The role of memory color in visual attention.

The expected color of an object influences how it is perceived. For example, a banana in a greyscale photo may appear slightly yellow because bananas are expected to be yellow. This phenomenon is known as the memory color effect (MCE), and the objects with a memory color are called "color-diagnostic." The MCE is theorized to be a top-down influence of color knowledge on visual perception. However, its validity has been questioned because most evidence for the MCE is based on subjective reports. Here a change detection task is used as an objective measure of the effect and the results show that change detection differs for color-diagnostic objects. Specifically, it was predicted and found that unnaturally colored color-diagnostic objects (e.g., a blue banana) would attract attention and thus be discovered more quickly and accurately. In the experiment, two arrays alternated with the target present in one array and absent in the other while all other objects remained unchanged. Participants had to find the target as quickly and accurately as possible. In the experimental condition, the targets were color-diagnostic objects (e.g., a banana) presented in either their natural (yellow) or an unnatural (blue) color. In the control condition, non-color-diagnostic objects (e.g., a mug) were presented with the same colors as the color-diagnostic objects. Unnaturally colored color-diagnostic objects were found more quickly, which suggests that the MCE is a top-down, preattentive process that can influence a nonsubjective visual perceptual task such as change detection.

Motion-induced distortion of shape.

Motion, position, and form are intricately intertwined in perception. Motion distorts visual space, resulting in illusory position shifts such as flash-drag and flash-grab effects. The flash-grab displaces a test by up to several times its size. This lets us use it to investigate where the motion-induced shift operates in the processing stream from photoreceptor activation to feature activation to object recognition. We present several canonical, highly familiar forms and ask whether the motion-induced shift operates uniformly across the form. If it did, we could conclude that the effect occurred after the elements of the form are bound. However, we find that motion-induced distortion affects not only the position, but also the appearance of briefly presented, canonical shapes (square, circle, and letter T). Features of the flashed target that were closest to its center were shifted in the direction of motion more than those further from its center. Outline shapes were affected more than filled shapes, and the strength of the distortion increased with the contrast of the moving background. This not only supports a nonuniform spatial profile for the motion-induced shift but also indicates that the shift operates before the shape is established, even for highly familiar shapes like squares, circles, and letters.

The double-drift illusion biases the marmoset oculomotor system.

The double-drift illusion has two unique characteristics: The error between the perceived and physical position of the stimulus grows over time, and saccades to the moving target land much closer to the physical than the perceived location. These results suggest that the perceptual and saccade targeting systems integrate visual information over different time scales. Functional imaging studies in humans have revealed several potential cortical areas of interest, including the prefrontal cortex. However, we currently lack an animal model to study the neural mechanisms of location perception that underlie the double-drift illusion. To fill this gap, we trained two marmoset monkeys to fixate and then saccade to the double-drift stimulus. In line with human observers for radial double-drift trajectories with fast internal motion, we find that saccade endpoints show a significant bias that is, nevertheless, smaller than the bias seen in human perceptual reports. This bias is modulated by changes in the external and internal speeds of the stimulus. These results demonstrate that the saccade targeting system of the marmoset monkey is influenced by the double-drift illusion.

The art of the float.

For more than 2000 years, artists have exploited cast shadows to influence how objects appear to be positioned in a scene. A contact cast shadow can anchor an object to the ground and a detached cast shadow can make an object appear to float. However, there is a period of approximately 1000 years when there were virtually no cast shadows in art. How were states of contact versus floating depicted by artists without cast shadows? Here, we survey various techniques used by artists to anchor relative position with and without cast shadows. We then conduct experimental tests of the hypothesized surface attraction principles that underlie these techniques. In the absence of cast shadows, an object (a wooden box) was often seen as resting on a surface as long as that surface offered information about ground orientation and support (a tiled floor). When the ground surface was ambiguous and cloud-like (1/f noise), the box was more likely to be seen to float. The presence of cast shadows made the box appear to contact the ground whether it was well-defined or ambiguous. Both shadows and surface support also increased the accuracy with which participants detected when the box was tilted up from the ground. These results indicate that artists long ago discovered the important power of support relationships to anchor objects to surfaces in the absence of shadows.

Up is best.

Ambiguous patterns have a tendency to appear to point up. This bias makes sense as most objects are on the ground, pointing up. However, we discover that the source of the up bias is the preference for seeing depth receding from the lower to the upper visual field.

The Architecture of Object-Based Attention.

The allocation of attention to objects raises several intriguing questions: What are objects, how does attention access them, what anatomical regions are involved? Here, we review recent progress in the field to determine the mechanisms underlying object-based attention. First, findings from unconscious priming and cueing suggest that the preattentive targets of object-based attention can be fully developed object representations that have reached the level of identity. Next, the control of object-based attention appears to come from ventral visual areas specialized in object analysis that project downward to early visual areas. How feedback from object areas can accurately target the object's specific locations and features is unknown but recent work in autoencoding has made this plausible. Finally, we suggest that the three classic modes of attention may not be as independent as is commonly considered, and instead could all rely on object-based attention. Specifically, studies show that attention can be allocated to the separated members of a group-without affecting the space between them-matching the defining property of feature-based attention. At the same time, object-based attention directed to a single small item has the properties of space-based attention. We outline the architecture of object-based attention, the novel predictions it brings, and discuss how it works in parallel with other attention pathways.

Exploring the frame effect.

Probes flashed within a moving frame are dramatically displaced (Özkan, Anstis, 't Hart, Wexler, & Cavanagh, 2021; Wong & Mack, 1981). The effect is much larger than that seen on static or moving probes (induced motion, Duncker, 1929; Wallach, Bacon, & Schulman, 1978). These flashed probes are often perceived with the separation they have in frame coordinates-a 100% effect (Özkan et al., 2021). Here, we explore this frame effect on flashed tests with several versions of the standard stimulus. We find that the frame effect holds for smoothly or abruptly displacing frames, even when the frame changed shape or orientation between the end points of its travel. The path could be nonlinear, even circular. The effect was driven by perceived not physical motion. When there were competing overlapping frames, the effect was determined by which frame was attended. There were a number of constraints that limited the effect. A static anchor near the flashes suppressed the effect but an extended static texture did not. If the probes were continuous rather than flashed, the effect was abolished. The observational reports of 30 online participants suggest that the frame effect is robust to many variations in its shape and path and leads to a perception of flashed tests in their locations relative to the frame as if the frame were stationary. Our results highlight the role of frame continuity and of the grouping of the flashes with the frame in generating the frame effect.

Endogenous attention biases transformational apparent motion based on high-level shape representations.

When two pre-existing, separated squares are connected by the sudden onset of a bar between them, viewers do not perceive the bar to appear all at once. Instead, they see an illusory morphing of the original squares over time. The direction of this transformational apparent motion (TAM) can be influenced by endogenous attention deployed before the appearance of the connecting bar. Here, we investigated whether the influence of endogenous attention on TAM results from operations over high-level feature-independent shape representations, or instead over lower level shape representations defined by specific visual features. To do so, we tested the influence of endogenous attention on TAM in first- and second-order displays, which shared common shapes but had different shape-defining attributes (luminance and texture contrast, respectively). In terms of both the magnitude of directional bias and timing, we found that endogenous attention exerted a similar influence on both first- and second-order objects. These results imply that endogenous attention biases the perceived direction of TAM by operating on high-level shape representations that are invariant to the low-level visual features that define them. Our results support a four-stage model of TAM, where a feature encoding stage passes a features-specific layout to a parsing stage that forms discrete, high-level meta-featural shapes, which are then matched and visually interpolated over time.

A motion-induced position shift that depends on motion both before and after the test probe.

Two versions of the flash grab illusion were used to examine the relative contributions of motion before and motion after the test flash to the illusory position shift. The stimulus in the first two experiments was a square pattern that expanded and contracted with an outline square flashed each time the motion reversed producing a dramatic difference in perceived size between the two reversals. Experiment 1 showed a strong illusion when motion was present before and after the flashed tests or just after the flashes, but no significant effect when only the pre-flash motion was present. In Experiment 2, motion always followed the flash, and the duration of the pre-flash motion was varied. The results showed a significant increase in illusion strength with the duration of pre-flash motion and the effect of the pre-flash motion was almost 50% that of the post-flash motion. Finally, Experiment 3 tested the position shifts when the linear motion of a disk before the flash was orthogonal to its motion after the flash. Here, the results again showed that the pre-flash motion made a significant contribution, about 32% that of the post-flash motion. Several models are considered and even though all fail to some degree, they do offer insights into the nature of the illusion. Finally, we show that the empirical measure of the relative contribution of motion before and after the flash can be used to distinguish the mechanisms underlying different illusions.

Mansfieldite (AlAsO4·2H2O): a new white pigment in Himalayan artwork.

An unusual arsenate mineral, mansfieldite (AlAsO4·2H2O), was identified as a pigment for the first time as the principal white colorant on two Himalayan thangka paintings at the Indianapolis Museum of Art at Newfields. The co-occurrence of this unusual mineral pigment provides support for the belief that the two artworks are members of a cycle of paintings originating from the same workshop, perhaps from Chamdo, Tibet. The complete palettes of both artworks are identical, including the use of mansfieldite, brochantite, malachite, azurite, vermilion, gold, orpiment, and a carbon-based black in a glue binder on a calcite and gypsum-primed cotton fabric.

Measuring the double-drift illusion and its resets with hand trajectories.

If a Gabor pattern drifts in one direction while its internal texture drifts in the orthogonal direction, its perceived position deviates further and further away from its true path. We first evaluated the illusion using manual tracking. Participants followed the Gabor with a stylus on a drawing tablet that coincided optically with the horizontal monitor surface. Their hand and the stylus were not visible during the tracking. The magnitude of the tracking illusion corresponded closely to previous perceptual and pointing measures indicating that manual tracking is a valid measure for the illusion. This allowed us to use it in a second experiment to capture the behavior of the illusion as it eventually degrades and breaks down in single trials. Specifically, the deviation of the Gabor stops accumulating at some point and either stays at a fixed offset or resets toward the veridical position. To report the perceived trajectory of the Gabor, participants drew it after the Gabor was removed from the monitor. Resets were detected and analyzed and their distribution matches neither a temporal nor a spatial limit, but rather a broad gamma distribution over time. This suggests that resets are triggered randomly, about once per 1.3 seconds, possible by extraneous distractions or eye movements.

Scaling depth from shadow offset.

When an object casts a shadow on a background surface, both the offset of the shadow and the blur of its penumbra are potential cues to the distance between the object and the background. However, the shadow offset and blur are also affected by the direction and angular extent of the light source and these are often unknown. This means that the observer must make some assumptions about the illumination, the expected distribution of depth, or the relation between offset and depth in order to use shadows to make distance judgments. Here, we measure human judgments of perceived depth over a range of shadow offsets, blurs, and lighting directions to gain insight into this internal model. We find that distance judgments are relatively unaffected by blur or light direction, whereas the shadow offset has a strong and linear effect. The data are consistent with two models, a generic shadow-to-depth model and a Bayesian model.

Smooth pursuit operates over perceived not physical positions of the double-drift stimulus.

The double-drift illusion produces a large deviation in perceived direction that strongly dissociates physical position from perceived position. Surprisingly, saccades do not seem to be affected by the illusion (Lisi & Cavanagh, 2015). When targeting a double-drift stimulus, the saccade system is driven by retinal rather than perceived position. Here, using paired double-drift targets, we test whether the smooth pursuit system is driven by perceived or physical position. Participants (n = 7) smoothly pursued the inferred midpoint (Steinbach, 1976) between two horizontally aligned Gabor patches that were separated by 20° and moving on parallel, oblique paths. On the first half of each trial, the Gabors' internal textures were static while both drifted obliquely downward. On the second half of each trial, while the envelope moved obliquely upward, the internal texture drifted orthogonally to the envelope's motion, producing a large perceived deviation from the downward path even though the upward and downward trajectories always followed the same physical path but in opposite directions. We find that smooth pursuit eye movements accurately followed the nonillusory downward path of the midpoint between the two Gabors, but then followed the illusory rather than the physical trajectory on the upward return. Thus, virtual targets for smooth pursuit are derived from perceived rather than retinal coordinates.