Paradoxical stabilization of relative position in moving frames.

To capture where things are and what they are doing, the visual system may extract the position and motion of each object relative to its surrounding frame of reference [K. Duncker, Routledge and Kegan Paul, London 161-172 (1929) and G. Johansson, Acta Psychol (Amst.) 7, 25-79 (1950)]. Here we report a particularly powerful example where a paradoxical stabilization is produced by a moving frame. We first take a frame that moves left and right and we flash its right edge before, and its left edge after, the frame's motion. For all frame displacements tested, the two edges are perceived as stabilized, with the left edge on the left and right edge on the right, separated by the frame's width as if the frame were not moving. This stabilization is paradoxical because the motion of the frame itself remains visible, albeit much reduced. A second experiment demonstrated that unlike other motion-induced position shifts (e.g., flash lag, flash grab, flash drag, or Fröhlich), the illusory shift here is independent of speed and is set instead by the distance of the frame's travel. In this experiment, two probes are flashed inside the frame at the same physical location before and after the frame moves. Despite being physically superimposed, the probes are perceived widely separated, again as if they were seen in the frame's coordinates and the frame were stationary. This paradoxical stabilization suggests a link to visual stability across eye movements where the displacement of the entire visual scene may act as a frame to stabilize the perception of relative locations.

Exploring the frame effect.

Probes flashed within a moving frame are dramatically displaced (Özkan, Anstis, 't Hart, Wexler, & Cavanagh, 2021; Wong & Mack, 1981). The effect is much larger than that seen on static or moving probes (induced motion, Duncker, 1929; Wallach, Bacon, & Schulman, 1978). These flashed probes are often perceived with the separation they have in frame coordinates-a 100% effect (Özkan et al., 2021). Here, we explore this frame effect on flashed tests with several versions of the standard stimulus. We find that the frame effect holds for smoothly or abruptly displacing frames, even when the frame changed shape or orientation between the end points of its travel. The path could be nonlinear, even circular. The effect was driven by perceived not physical motion. When there were competing overlapping frames, the effect was determined by which frame was attended. There were a number of constraints that limited the effect. A static anchor near the flashes suppressed the effect but an extended static texture did not. If the probes were continuous rather than flashed, the effect was abolished. The observational reports of 30 online participants suggest that the frame effect is robust to many variations in its shape and path and leads to a perception of flashed tests in their locations relative to the frame as if the frame were stationary. Our results highlight the role of frame continuity and of the grouping of the flashes with the frame in generating the frame effect.

Direct coupling of haptic signals between hands.

Although motor actions can profoundly affect the perceptual interpretation of sensory inputs, it is not known whether the combination of sensory and movement signals occurs only for sensory surfaces undergoing movement or whether it is a more general phenomenon. In the haptic modality, the independent movement of multiple sensory surfaces poses a challenge to the nervous system when combining the tactile and kinesthetic signals into a coherent percept. When exploring a stationary object, the tactile and kinesthetic signals come from the same hand. Here we probe the internal structure of haptic combination by directing the two signal streams to separate hands: one hand moves but receives no tactile stimulation, while the other hand feels the consequences of the first hand's movement but remains still. We find that both discrete and continuous tactile and kinesthetic signals are combined as if they came from the same hand. This combination proceeds by direct coupling or transfer of the kinesthetic signal from the moving to the feeling hand, rather than assuming the displacement of a mediating object. The combination of signals is due to perception rather than inference, because a small temporal offset between the signals significantly degrades performance. These results suggest that the brain simplifies the complex coordinate transformation task of remapping sensory inputs to take into account the movements of multiple body parts in haptic perception, and they show that the effects of action are not limited to moving sensors.

Persistent states in vision break universality and time invariance.

Studies of perception usually emphasize processes that are largely universal across observers and--except for short-term fluctuations--stationary over time. Here we test the universality and stationarity assumptions with two families of ambiguous visual stimuli. Each stimulus can be perceived in two different ways, parameterized by two opposite directions from a continuous circular variable. A large-sample study showed that almost all observers have preferred directions or biases, with directions lying within 90 degrees of the bias direction nearly always perceived and opposite directions almost never perceived. The biases differ dramatically from one observer to the next, and although nearly every bias direction occurs in the population, the population distributions of the biases are nonuniform, featuring asymmetric peaks in the cardinal directions. The biases for the two families of stimuli are independent and have distinct population distributions. Following external perturbations and spontaneous fluctuations, the biases decay over tens of seconds toward their initial values. Persistent changes in the biases are found on time scales of several minutes to 1 hour. On scales of days to months, the biases undergo a variety of dynamical processes such as drifts, jumps, and oscillations. The global statistics of a majority of these long-term time series are well modeled as random walk processes. The measurable fluctuations of these hitherto unknown degrees of freedom show that the assumptions of universality and stationarity in perception may be unwarranted and that models of perception must include both directly observable variables as well as covert, persistent states.

Multidimensional internal dynamics underlying the perception of motion.

When ambiguous visual stimuli are presented continuously, they often lead to oscillations between usually two perceptions. Because of these oscillations, it has been thought that the underlying neural dynamics also arises from a binary or two-state system. Contradicting the binary assumption, it has been shown recently that the perception of some ambiguous stimuli is governed by continuously varying internal states, measured as biases that differ considerably from one observer to the next and that can also evolve over time (Wexler, Duyck, & Mamassian, 2015). Here I study bias patterns in the motion quartet, an ambiguous apparent motion stimulus, as the quartet's orientation is varied. The bias patterns are robustly idiosyncratic, and are even more complex than those that have been described previously. There are two qualitatively different bias types: Some observers prefer a translation axis, while others show preference for a rotation direction. Each type also varies parametrically: the orientation of the preferred axis, and the direction of preferred rotation. There are also clear cases of combination of the two bias types. When measured repeatedly over 9 hr, the bias patterns usually remain stable, but also sometimes evolve both parametrically (e.g., change of preferred axis), as well as across bias type (change from axial to rotational bias). Control experiments revealed that the variety of bias patterns observed across subjects, and their changes over time, are not due to voluntary decisions. Overall, these results exhibit the multidimensional complexity of internal states underlying the perception of even simple stimuli.

Default perception of high-speed motion.

When human observers are exposed to even slight motion signals followed by brief visual transients--stimuli containing no detectable coherent motion signals--they perceive large and salient illusory jumps. This visually striking effect, which we call "high phi," challenges well-entrenched assumptions about the perception of motion, namely the minimal-motion principle and the breakdown of coherent motion perception with steps above an upper limit called dmax. Our experiments with transients, such as texture randomization or contrast reversal, show that the magnitude of the jump depends on spatial frequency and transient duration--but not on the speed of the inducing motion signals--and the direction of the jump depends on the duration of the inducer. Jump magnitude is robust across jump directions and different types of transient. In addition, when a texture is actually displaced by a large step beyond the upper step size limit of dmax, a breakdown of coherent motion perception is expected; however, in the presence of an inducer, observers again perceive coherent displacements at or just above dmax. In summary, across a large variety of stimuli, we find that when incoherent motion noise is preceded by a small bias, instead of perceiving little or no motion--as suggested by the minimal-motion principle--observers perceive jumps whose amplitude closely follows their own dmax limits.

Masking the saccadic smear.

Static visual stimuli are smeared across the retina during saccades, but in normal conditions this smear is not perceived. Instead, we perceive the visual scene as static and sharp. However, retinal smear is perceived if stimuli are shown only intrasaccadically, but not if the stimulus is additionally shown before a saccade begins, or after the saccade ends (Campbell & Wurtz, 1978). This inhibition has been compared to forward and backward metacontrast masking, but with spatial relations between stimulus and mask that are different from ordinary metacontrast during fixation. Previous studies of smear masking have used subjective measures of smear perception. Here we develop a new, objective technique for measuring smear masking, based on the spatial localization of a gap in the smear created by very quickly blanking the stimulus at various points during the saccade. We apply this technique to show that smear masking survives dichoptic presentation (suggesting that it is therefore cortical in origin), as well as separations of as much as 6° between smear and mask.

Orthogonal steps relieve saccadic suppression.

Although the retinal position of objects changes with each saccadic eye movement, we perceive the visual world to be stable. How this visual stability or constancy arises is debated. Cancellation accounts propose that the retinal consequences of eye movements are compensated for by an equal-but-opposite eye movement signal. Assumption accounts propose that saccade-induced retinal displacements are ignored because we have a prior belief in a stable world. Saccadic suppression of displacement-the fact that small displacements of the visual targets during saccades go unnoticed-argues in favor of assumption accounts. Extinguishing the target before the displacement unmasks it, arguing in favor of cancellation accounts. We show that an irrelevant displacement of the target orthogonal to saccade direction unmasks displacements parallel to saccade direction, and therefore relieves saccadic suppression of displacement. This result suggests that visual stability arises from the interplay between cancellation and assumption mechanisms: When the post-saccadic target position falls within an elliptic region roughly equivalent to habitual saccadic variability, displacements are not seen and stability is assumed. When the displacements fall outside this region, as with our orthogonal steps, displacements are seen and positions are remapped.

Method to Quickly Map Multifocal Pupillary Response Fields (mPRF) Using Frequency Tagging.

We present a method for mapping multifocal Pupillary Response Fields in a short amount of time using a visual stimulus covering 40° of the visual angle divided into nine contiguous sectors simultaneously modulated in luminance at specific, incommensurate, temporal frequencies. We test this multifocal Pupillary Frequency Tagging (mPFT) approach with young healthy participants (N = 36) and show that the spectral power of the sustained pupillary response elicited by 45 s of fixation of this multipartite stimulus reflects the relative contribution of each sector/frequency to the overall pupillary response. We further analyze the phase lag for each temporal frequency as well as several global features related to pupil state. Test/retest performed on a subset of participants indicates good repeatability. We also investigate the existence of structural (RNFL)/functional (mPFT) relationships. We then summarize the results of clinical studies conducted with mPFT on patients with neuropathies and retinopathies and show that the features derived from pupillary signal analyses, the distribution of spectral power in particular, are homologous to disease characteristics and allow for sorting patients from healthy participants with excellent sensitivity and specificity. This method thus appears as a convenient, objective, and fast tool for assessing the integrity of retino-pupillary circuits as well as idiosyncrasies and permits to objectively assess and follow-up retinopathies or neuropathies in a short amount of time.

Motion perception by a moving observer in a three-dimensional environment.

Perceiving three-dimensional object motion while moving through the world is hard: not only must optic flow be segmented and parallax resolved into shape and motion, but also observer motion needs to be taken into account in order to perceive absolute, rather than observer-relative motion. In order to simplify the last step, it has recently been suggested that if the visual background is stationary, then foreground object motion, computed relative to the background, directly yields absolute motion. A series of studies with immobile observers and optic flow simulating observer movement have provided evidence that observers actually utilize this so-called "flow parsing" strategy (Rushton & Warren, 2005). We test this hypothesis by using mobile observers (as well as immobile ones) who judge the motion in depth of a foreground object in the presence of a stationary or moving background. We find that background movement does influence motion perception but not as much as predicted by the flow-parsing hypothesis. Thus, we find evidence that, in order to perceive absolute motion, observers partly use flow-parsing but also compensate egocentric motion by a global self-motion estimate.

Temporal dynamics of remapping captured by peri-saccadic continuous motion.

Different attention and saccade control areas contribute to space constancy by remapping target activity onto their expected post-saccadic locations. To visualize this dynamic remapping, we used a technique developed by Honda (2006) where a probe moved vertically while participants made a saccade across the motion path. Observers do not report any large excursions of the trace at the time of the saccade that would correspond to the classical peri-saccadic mislocalization effect. Instead, they reported that the motion trace appeared to be broken into two separate segments with a shift of approximately one-fifth of the saccade amplitude representing an overcompensation of the expected retinal displacement caused by the saccade. To measure the timing of this break in the trace, we introduced a second, physical shift that was the same size but opposite in direction to the saccade-induced shift. The trace appeared continuous most frequently when the physical shift was introduced at the midpoint of the saccade, suggesting that the compensation is in place when the saccade lands. Moreover, this simple linear shift made the combined traces appear continuous and linear, with no curvature. In contrast, Honda (2006) had reported that the pre- and post-saccadic portion of the trace appeared aligned and that there was often a small, visible excursion of the trace at the time of the saccade. To compare our results more directly, we increased the contrast of our moving probe in a third experiment. Now some observers reported seeing a deviation in the motion path but the misalignment remained present. We conclude that the large deviations at the time of saccade are generally masked for a continuously moving target but that there is nevertheless a residual misalignment between pre- and post-saccadic coordinates of approximately 20% of the saccade amplitude that normally goes unnoticed.

Structure of visual biases revealed by individual differences.

Multiple studies have shown that certain visual stimuli are perceived in accordance with strong biases that are both robust within individuals and highly variable from one individual to the next. These biases undergo small changes over time that demonstrate that they constitute latent states of the visual system. The literature to date indicates that the individual biases for different stimulus classes are independent of each other. Here we asked whether some of these biases are nonetheless related to one another. We measured individual biases for five classes of stimuli in 1000 participants. The stimuli were two different versions of two-dimensional apparent motion, smooth motion in Glass patterns, and two different structure-from-motion stimuli. There were pronounced individual biases in all stimuli, and these biases varied in direction and strength across individuals. Some biases were not independent: the two biases for apparent motion direction were most strongly correlated, and they were both correlated, but less strongly, to the bias direction for smooth motion. While all other pairs of biases had unrelated directions, the strengths of all biases were correlated. The correlation of bias strengths may be due to either a common factor across the stimulus types, or an attentional effect. Only a tiny fraction of the between-participant variance can be explained by age and gender. These results show that latent states of the visual system that we measure as individual biases are organized in a structured way, and call out for further study of this under-explored aspect of visual perception.

Visual continuity during blinks and alterations in time perception.

Eye blinks strongly attenuate visual input, yet we perceive the world as continuous. How this visual continuity is achieved remains a fundamental and unsolved problem. A decrease in luminance sensitivity has been proposed as a mechanism but is insufficient to mask the even larger decrease in luminance because of blinks. Here we put forward a different hypothesis: visual continuity can be achieved through shortening of perceived durations of the sensory consequences of blinks. Here we probed the perceived durations of the blackouts caused by blinks and visual stimuli interrupted by blinks. We found that the perceived durations of blackouts because of blinks are about half as long as artificial blackouts immediately preceding or following the blink. Stimuli interrupted by blinks were perceived as briefer than uninterrupted stimuli, by about the same duration as the interruption-but so were stimuli interrupted by optically simulated blinks. There was a difference between real and simulated blinks, however: The decrease in perceived duration depended on the duration of the interruption for simulated, but not for real, blinks. These profound modifications in time perception during blinks show a way in which temporal processing contributes to the solution of an essential perceptual problem. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Bifurcation in brain dynamics reveals a signature of conscious processing independent of report.

An outstanding challenge for consciousness research is to characterize the neural signature of conscious access independently of any decisional processes. Here we present a model-based approach that uses inter-trial variability to identify the brain dynamics associated with stimulus processing. We demonstrate that, even in the absence of any task or behavior, the electroencephalographic response to auditory stimuli shows bifurcation dynamics around 250-300 milliseconds post-stimulus. Namely, the same stimulus gives rise to late sustained activity on some trials, and not on others. This late neural activity is predictive of task-related reports, and also of reports of conscious contents that are randomly sampled during task-free listening. Source localization further suggests that task-free conscious access recruits the same neural networks as those associated with explicit report, except for frontal executive components. Studying brain dynamics through variability could thus play a key role for identifying the core signatures of conscious access, independent of report.

Observer movement and size constancy.

It is commonly assumed that size constancy-invariance of perceived size of objects as they change retinal size because of changes in distance-depends solely on retinal stimulation and vergence, but on no other action-related signals. Distance to an object can change through displacement of either the observer or the object. The common assumption predicts that the two types of displacement should lead to the same degree of size constancy. We measured size constancy while observers viewed stationary stimuli at different distances. Changes in distance between trials were either actively produced by the observer or generated by real or simulated object displacement, with retinal stimulation held constant across the movement conditions. Responses were always closer to perfect constancy for observer than for object movement. Thus, size constancy is enhanced by information from observer displacement, and, more generally, processes thought to be purely perceptual may have unexpected components related to action.

The reference frame of the tilt aftereffect.

Perceptual aftereffects provide a sensitive tool to investigate the influence of eye and head position on visual processing. There have been recent indications that the TAE is remapped around the time of a saccade to remain aligned to the adapting location in the world. Here, we investigate the spatial frame of reference of the TAE by independently manipulating retinal position, gaze orientation, and head orientation between adaptation and test. The results show that the critical factor in the TAE is the correspondence between the adaptation and test locations in a retinotopic frame of reference, whereas world- and head-centric frames of reference do not play a significant role. Our results confirm that adaptation to orientation takes place at retinotopic levels of visual processing. We suggest that the remapping process that plays a role in visual stability does not transfer feature gain information around the time of eye (or head) movements.

The nonlinear structure of motion perception during smooth eye movements.

To perceive object motion when the eyes themselves undergo smooth movement, we can either perceive motion directly-by extracting motion relative to a background presumed to be fixed-or through compensation, by correcting retinal motion by information about eye movement. To isolate compensation, we created stimuli in which, while the eye undergoes smooth movement due to inertia, only one object is visible-and the motion of this stimulus is decoupled from that of the eye. Using a wide variety of stimulus speeds and directions, we rule out a linear model of compensation, in which stimulus velocity is estimated as a linear combination of retinal and eye velocities multiplied by a constant gain. In fact, we find that when the stimulus moves in the same direction as the eyes, there is little compensation, but when movement is in the opposite direction, compensation grows in a nonlinear way with speed. We conclude that eye movement is estimated from a combination of extraretinal and retinal signals, the latter based on an assumption of stimulus stationarity. Two simple models, in which the direction of eye movement is computed from the extraretinal signal and the speed from the retinal signal, account well for our results.

Radial trunk-centred reference frame in haptic perception.

The shape of objects is typically identified through active touch. The accrual of spatial information by the hand over time requires the continuous integration of tactile and movement information. Sensory inputs arising from one single sensory source gives rise to an infinite number of possible touched locations in space. This observation raises the question of the determination of a common reference frame that might be employed by humans to resolve spatial ambiguity. Here, we employ a paradigm where observers reconstruct the spatial attributes of a triangle from tactile inputs applied to a stationary hand correlated with the voluntary movements of the other hand. We varied the orientation of the hands with respect to one another and to the trunk, and tested three distinct hypotheses regarding a reference frame used for integration: a hand-centred, a trunk-centred or an allocentric reference frame. The results indicated strongly that the integration of movement information and tactile inputs was performed in a radial trunk-centred reference frame.