The execution of saccadic eye movements suppresses visual processing of both color and luminance in the early visual cortex of humans.

Our eyes execute rapid, directional movements known as saccades, occurring several times per second, to focus on objects of interest in our environment. During these movements, visual sensitivity is temporarily reduced. Despite numerous studies on this topic, the underlying mechanism remains elusive, including a lingering debate on whether saccadic suppression affects the parvocellular visual pathway. To address this issue, we conducted a study employing steady-state visual evoked potentials (SSVEPs) elicited by chromatic and luminance stimuli while observers performed saccadic eye movements. We also employed an innovative analysis pipeline to enhance the signal-to-noise ratio, yielding superior results compared to the previous method. Our findings revealed a clear suppression effect on SSVEP signals during saccades compared to fixation periods. Notably, this suppression effect was comparable for both chromatic and luminance stimuli. We went further to measure the suppression effect across various contrast levels, which enabled us to model SSVEP responses with contrast response functions. The results suggest that saccades primarily reduce response gain without significantly affecting contrast gain and that this reduction applies uniformly to both chromatic and luminance pathways. In summary, our study provides robust evidence that saccades similarly suppress visual processing in both the parvocellular and magnocellular pathways within the human early visual cortex, as indicated by SSVEP responses. The observation that saccadic eye movements impact response gain rather than contrast gain implies that they influence visual processing through a multiplicative mechanism.NEW & NOTEWORTHY The present study demonstrates that saccadic eye movements reduce the processing of both luminance and chromatic stimuli in the early visual cortex of humans. By modeling the contrast response function, the study further shows that saccades affect visual processing by reducing the response gain rather than altering the contrast gain, suggesting that a multiplicative mechanism of visual attenuation affects both parvocellular and magnocellular pathways.

Oculomotor routines for perceptual judgments.

In everyday life we frequently make simple visual judgments about object properties, for example, how big or wide is a certain object? Our goal is to test whether there are also task-specific oculomotor routines that support perceptual judgments, similar to the well-established exploratory routines for haptic perception. In a first study, observers saw different scenes with two objects presented in a photorealistic virtual reality environment. Observers were asked to judge which of two objects was taller or wider while gaze was tracked. All tasks were performed with the same set of virtual objects in the same scenes, so that we can compare spatial characteristics of exploratory gaze behavior to quantify oculomotor routines for each task. Width judgments showed fixations around the center of the objects with larger horizontal spread. In contrast, for height judgments, gaze was shifted toward the top of the objects with larger vertical spread. These results suggest specific strategies in gaze behavior that presumably are used for perceptual judgments. To test the causal link between oculomotor behavior and perception, in a second study, observers could freely gaze at the object or we introduced a gaze-contingent setup forcing observers to fixate specific positions on the object. Discrimination performance was similar between free-gaze and the gaze-contingent conditions for width and height judgments. These results suggest that although gaze is adapted for different tasks, performance seems to be based on a perceptual strategy, independent of potential cues that can be provided by the oculomotor system.

Color constancy in real-world settings.

Color constancy denotes the ability to assign a particular and stable color percept to an object, irrespective of its surroundings and illumination. The light reaching the eye confounds illumination and spectral reflectance of the object, making the recovery of constant object color an ill-posed problem. How good the visual system is at accomplishing this task is still a matter of heated debate, despite more than a 100 years of research. Depending on the laboratory task and the specific cues available to observers, color constancy was found to be at levels ranging between 15% and 80%, which seems incompatible with the relatively stable color appearance of objects around us and the consistent usage of color names in real life. Here, we show close-to-perfect color constancy using real objects in a natural task and natural environmental conditions, chosen to mimic the role of color constancy in everyday life. Participants had to identify the color of a (non-present) item familiar to them in an office room under five different experimental illuminations. They mostly selected the same colored Munsell chip as their match to the absent object, even though the light reaching the eye in each case differed substantially. Our results demonstrate that color constancy under ideal conditions in the real world can indeed be exceptionally good. We found it to be as good as visual memory permits and not generally compromised by sensory uncertainty.

Color constancy mechanisms in virtual reality environments.

Prior research has demonstrated high levels of color constancy in real-world scenarios featuring single light sources, extensive fields of view, and prolonged adaptation periods. However, exploring the specific cues humans rely on becomes challenging, if not unfeasible, with actual objects and lighting conditions. To circumvent these obstacles, we employed virtual reality technology to craft immersive, realistic settings that can be manipulated in real time. We designed forest and office scenes illuminated by five colors. Participants selected a test object most resembling a previously shown achromatic reference. To study color constancy mechanisms, we modified scenes to neutralize three contributors: local surround (placing a uniform-colored leaf under test objects), maximum flux (keeping the brightest object constant), and spatial mean (maintaining a neutral average light reflectance), employing two methods for the latter: changing object reflectances or introducing new elements. We found that color constancy was high in conditions with all cues present, aligning with past research. However, removing individual cues led to varied impacts on constancy. Local surrounds significantly reduced performance, especially under green illumination, showing strong interaction between greenish light and rose-colored contexts. In contrast, the maximum flux mechanism barely affected performance, challenging assumptions used in white balancing algorithms. The spatial mean experiment showed disparate effects: Adding objects slightly impacted performance, while changing reflectances nearly eliminated constancy, suggesting human color constancy relies more on scene interpretation than pixel-based calculations.

Laplacian reference is optimal for steady-state visual-evoked potentials.

Steady-state visual-evoked potentials (SSVEPs) are widely used in human neuroscience studies and applications such as brain-computer interfaces (BCIs). Surprisingly, no previous study has systematically evaluated different reference methods for SSVEP analysis, despite that signal reference is crucial for the proper assessment of neural activities. In the present study, using four datasets from our previous SSVEP studies (Chen J, Valsecchi M, Gegenfurtner KR. J Neurophysiol 118: 749-754, 2017; Chen J, Valsecchi M, Gegenfurtner KR. Neuropsychologia 102: 206-216, 2017; Chen J, McManus M, Valsecchi M, Harris LR, Gegenfurtner KR. J Vis 19: 8, 2019) and three public datasets from other studies (Baker DH, Vilidaite G, Wade AR. PLoS Comput Biol 17: e1009507, 2021; Lygo FA, Richard B, Wade AR, Morland AB, Baker DH. NeuroImage 230: 117780, 2021; Vilidaite G, Norcia AM, West RJH, Elliott CJH, Pei F, Wade AR, Baker DH. Proc R Soc B 285: 20182255, 2018), we compared four reference methods: monopolar reference, common average reference, averaged-mastoids reference, and Laplacian reference. The quality of the resulting SSVEP signals was compared in terms of both signal-to-noise ratios (SNRs) and reliability. The results showed that Laplacian reference, which uses signals at the maximally activated electrode after subtracting the average of the nearby electrodes to reduce common noise, gave rise to the highest SNRs. Furthermore, the Laplacian reference resulted in SSVEP signals that were highly reliable across recording sessions or trials. These results suggest that Laplacian reference is optimal for SSVEP studies and applications. Laplacian reference is especially advantageous for SSVEP experiments where short preparation time is preferred as it requires only data from the maximally activated electrode and a few surrounding electrodes.NEW & NOTEWORTHY The present study provides a comprehensive evaluation of the use of different reference methods for steady-state visual-evoked potentials (SSVEPs) and has found that Laplacian reference increases signal-to-noise ratios (SNRs) and enhances reliabilities of SSVEP signals. Thus, the results suggest that Laplacian reference is optimal for SSVEP analysis.

Object-based color constancy in a deep neural network.

Color constancy refers to our capacity to see consistent colors under different illuminations. In computer vision and image processing, color constancy is often approached by explicit estimation of the scene's illumination, followed by an image correction. In contrast, color constancy in human vision is typically measured as the capacity to extract color information about objects and materials in a scene consistently throughout various illuminations, which goes beyond illumination estimation and might require some degree of scene and color understanding. Here, we pursue an approach with deep neural networks that tries to assign reflectances to individual objects in the scene. To circumvent the lack of massive ground truth datasets labeled with reflectances, we used computer graphics to render images. This study presents a model that recognizes colors in an image pixel by pixel under different illumination conditions.

Perception of saturation in natural objects.

The distribution of colors across a surface depends on the interaction between its surface properties, its shape, and the lighting environment. Shading, chroma, and lightness are positively correlated: points on the object that have high luminance also have high chroma. Saturation, typically defined as the ratio of chroma to lightness, is therefore relatively constant across an object. Here we explored to what extent this relationship affects perceived saturation of an object. Using images of hyperspectral fruit and rendered matte objects, we manipulated the lightness-chroma correlation (positive or negative) and asked observers which of two objects appeared more saturated. Despite the negative-correlation stimulus having greater mean and maximum chroma, lightness, and saturation than the positive, observers overwhelmingly chose the positive as more saturated. This suggests that simple colorimetric statistics do not accurately represent perceived saturation of objects-observers likely base their judgments on interpretations about the cause of the color distribution.

Cues for predictive eye movements in naturalistic scenes.

We previously compared following of the same trajectories with eye movements, but either as an isolated targets or embedded in a naturalistic scene-in this case, the movement of a puck in an ice hockey game. We observed that the oculomotor system was able to leverage the contextual cues available in the naturalistic scene to produce predictive eye movements. In this study, we wanted to assess which factors are critical for achieving this predictive advantage by manipulating four factors: the expertise of the viewers, the amount of available peripheral information, and positional and kinematic cues. The more peripheral information became available (by manipulating the area of the video that was visible), the better the predictions of all observers. However, expert ice hockey fans were consistently better at predicting than novices and used peripheral information more effectively for predictive saccades. Artificial cues about player positions did not lead to a predictive advantage, whereas impairing the causal structure of kinematic cues by playing the video in reverse led to a severe impairment. When videos were flipped vertically to introduce more difficult kinematic cues, predictive behavior was comparable to watching the original videos. Together, these results demonstrate that, when contextual information is available in naturalistic scenes, the oculomotor system is successfully integrating them and is not relying only on low-level information about the target trajectory. Critical factors for successful prediction seem to be the amount of available information, experience with the stimuli, and the availability of intact kinematic cues for player movements.

Context effects on the perception of saturation of fruit colors in still-life paintings.

Still-life painters, especially of the so-called Golden Age (17th century) in the Netherlands, are famous for their masterful techniques of rendering reality. Their amazing abilities to depict different material properties of fruits and flowers are stunning. But how important are these careful arrangements of different objects for the perception of an individual item? Is the perceived color saturation of a single fruit influenced by its surrounding context? We selected fruits in still-life paintings as stimuli to investigate whether and how perceived saturations of fruits were affected by their original contexts. In our study, we focused especially on effects of five context properties: complementary colors, chromatic and luminance contrast, object overlap, and surround variance. Six fruit varieties depicted in high-quality digital reproductions of 48 classic and eight varieties in 64 more recent, modern still-life paintings were selected. In a single trial, eight images of fruits of the same variety appeared on a neutral gray background; half were single fruit cutouts, and the other half were the same fruits embedded in their circular contexts. Fifteen participants ranked all eight images according to perceived color saturations of the fruits. Saturation ratings showed a high agreement of 77%. Surrounding contexts led to an increase in perceived saturation of central fruits. This effect was mainly driven by object overlap, the presence of the central fruit type also in the context, and surround variance. Chroma contrast between fruits and contexts decreased saturation significantly. No significant context effects were found for complementary colors or luminance contrast. Our results show that in paintings, many of the cues that are usually experimentally isolated occur in interesting combinations and lead to an increase in perceived saturation that makes fruit objects more appealing and convincing.

The time course of chromatic adaptation in human early visual cortex revealed by SSVEPs.

Previous studies have identified at least two components of chromatic adaptation: a rapid component with a time scale between tens of milliseconds to a few seconds, and a slow component with a half-life of about 10 to 30 seconds. The basis of the rapid adaptation probably lies in receptor adaptation at the retina. The neural substrate for the slow adaptation remains unclear, although previous psychophysical results hint at the early visual cortex. A promising approach to investigate adaptation effects in the visual cortex is to analyze steady-state visual evoked potentials (SSVEPs) elicited by chromatic stimuli, which typically use long durations of stimulation. Here, we re-analyzed the data from two previous pattern-reversal SSVEP studies. In these experiments (N = 49 observers in total), SSVEPs were elicited by counter-phase flickering color- or luminance-defined grating stimuli for 150 seconds in each trial. By analyzing SSVEPs with short time windows, we found that chromatic SSVEP responses decreased with increasing stimulation duration and reached a lower asymptote within a minute of stimulation. The luminance SSVEPs did not show any systematic adaptation. The time course of chromatic SSVEPs can be well described by an exponential decay function with a half-life of about 20 seconds, which is very close to previous psychophysical reports. Despite the difference in stimuli between the current and previous studies, the coherent time course may indicate a more general adaptation mechanism in the early visual cortex. In addition, the current result also provides a guide for future color SSVEP studies in terms of either avoiding or exploiting this adaptation effect.

Color and gloss constancy under diverse lighting environments.

When we look at an object, we simultaneously see how glossy or matte it is, how light or dark, and what color. Yet, at each point on the object's surface, both diffuse and specular reflections are mixed in different proportions, resulting in substantial spatial chromatic and luminance variations. To further complicate matters, this pattern changes radically when the object is viewed under different lighting conditions. The purpose of this study was to simultaneously measure our ability to judge color and gloss using an image set capturing diverse object and illuminant properties. Participants adjusted the hue, lightness, chroma, and specular reflectance of a reference object so that it appeared to be made of the same material as a test object. Critically, the two objects were presented under different lighting environments. We found that hue matches were highly accurate, except for under a chromatically atypical illuminant. Chroma and lightness constancy were generally poor, but these failures correlated well with simple image statistics. Gloss constancy was particularly poor, and these failures were only partially explained by reflection contrast. Importantly, across all measures, participants were highly consistent with one another in their deviations from constancy. Although color and gloss constancy hold well in simple conditions, the variety of lighting and shape in the real world presents significant challenges to our visual system's ability to judge intrinsic material properties.

A deep new look at color.

Bowers et al. counter deep neural networks (DNNs) as good models of human visual perception. From our color perspective we feel their view is based on three misconceptions: A misrepresentation of the state-of-the-art of color perception; the type of model required to move the field forward; and the attribution of shortcomings to DNN research that are already being resolved.

Individual differences in visual salience vary along semantic dimensions.

What determines where we look? Theories of attentional guidance hold that image features and task demands govern fixation behavior, while differences between observers are interpreted as a "noise-ceiling" that strictly limits predictability of fixations. However, recent twin studies suggest a genetic basis of gaze-trace similarity for a given stimulus. This leads to the question of how individuals differ in their gaze behavior and what may explain these differences. Here, we investigated the fixations of >100 human adults freely viewing a large set of complex scenes containing thousands of semantically annotated objects. We found systematic individual differences in fixation frequencies along six semantic stimulus dimensions. These differences were large (>twofold) and highly stable across images and time. Surprisingly, they also held for first fixations directed toward each image, commonly interpreted as "bottom-up" visual salience. Their perceptual relevance was documented by a correlation between individual face salience and face recognition skills. The set of reliable individual salience dimensions and their covariance pattern replicated across samples from three different countries, suggesting they reflect fundamental biological mechanisms of attention. Our findings show stable individual differences in salience along a set of fundamental semantic dimensions and that these differences have meaningful perceptual implications. Visual salience reflects features of the observer as well as the image.

Deep neural models for color classification and color constancy.

Color constancy is our ability to perceive constant colors across varying illuminations. Here, we trained deep neural networks to be color constant and evaluated their performance with varying cues. Inputs to the networks consisted of two-dimensional images of simulated cone excitations derived from three-dimensional (3D) rendered scenes of 2,115 different 3D shapes, with spectral reflectances of 1,600 different Munsell chips, illuminated under 278 different natural illuminations. The models were trained to classify the reflectance of the objects. Testing was done with four new illuminations with equally spaced CIEL*a*b* chromaticities, two along the daylight locus and two orthogonal to it. High levels of color constancy were achieved with different deep neural networks, and constancy was higher along the daylight locus. When gradually removing cues from the scene, constancy decreased. Both ResNets and classical ConvNets of varying degrees of complexity performed well. However, DeepCC, our simplest sequential convolutional network, represented colors along the three color dimensions of human color vision, while ResNets showed a more complex representation.

Execution of saccadic eye movements affects speed perception.

Due to the foveal organization of our visual system we have to constantly move our eyes to gain precise information about our environment. Doing so massively alters the retinal input. This is problematic for the perception of moving objects, because physical motion and retinal motion become decoupled and the brain has to discount the eye movements to recover the speed of moving objects. Two different types of eye movements, pursuit and saccades, are combined for tracking. We investigated how the way we track moving targets can affect the perceived target speed. We found that the execution of corrective saccades during pursuit initiation modifies how fast the target is perceived compared with pure pursuit. When participants executed a forward (catch-up) saccade they perceived the target to be moving faster. When they executed a backward saccade they perceived the target to be moving more slowly. Variations in pursuit velocity without corrective saccades did not affect perceptual judgments. We present a model for these effects, assuming that the eye velocity signal for small corrective saccades gets integrated with the retinal velocity signal during pursuit. In our model, the execution of corrective saccades modulates the integration of these two signals by giving less weight to the retinal information around the time of corrective saccades.

Electrophysiological evidence for higher-level chromatic mechanisms in humans.

Color vision in humans starts with three types of cones (short [S], medium [M], and long [L] wavelengths) in the retina and three retinal and subcortical cardinal mechanisms, which linearly combine cone signals into the luminance channel (L + M), the red-green channel (L - M), and the yellow-blue channel (S-(L + M)). Chromatic mechanisms at the cortical level, however, are less well characterized. The present study investigated such higher-order chromatic mechanisms by recording electroencephalograms (EEGs) on human observers in a noise masking paradigm. Observers viewed colored stimuli that consisted of a target embedded in noise. Color directions of the target and noise varied independently and systematically in an isoluminant plane of color space. The target was flickering on-off at 3 Hz, eliciting steady-state visual evoked potential (SSVEP) responses. As a result, the masking strength could be estimated from the SSVEP amplitude in the presence of 6 Hz noise. Masking was strongest (i.e. target eliciting smallest SSVEPs) when the target and noise were along the same color direction, and was weakest (i.e. target eliciting highest SSVEPs) when the target and noise were along orthogonal directions. This pattern of results was observed both when the target color varied along the cardinal and intermediate directions, which is evidence for higher-order chromatic mechanisms tuned to intermediate axes. The SSVEP result can be well predicted by a model with multiple broadly tuned chromatic mechanisms. In contrast, a model with only cardinal mechanisms failed to account for the data. These results provide strong electrophysiological evidence for multiple chromatic mechanisms in the early visual cortex of humans.

Age effects on saccadic suppression of luminance and color.

Saccadic eye movements modulate visual perception: they initiate and terminate high acuity vision at a certain location in space, but before and during their execution visual contrast sensitivity is strongly attenuated for 100 to 200 ms. Transient perisaccadic perceptual distortions are assumed to be an important mechanism to maintain visual stability. Little is known about age effects on saccadic suppression, even though for healthy adults other major age-related changes are well documented, like a decrease of visual contrast sensitivity for intermediate and high spatial frequencies or an increase of saccade latencies. Here, we tested saccadic suppression of luminance and isoluminant chromatic flashes in 100 participants from eight to 78 years. To estimate the effect of saccadic suppression on contrast sensitivity, we used a two-alternative forced choice (2AFC) design and an adaptive staircase procedure to modulate the luminance or chromatic contrast of a flashed detection target during fixation and 15 ms after saccade onset. The target was a single horizontal luminance or chromatic line flashed 2° above or below the fixation or saccade target. Compared to fixation, average perisaccadic contrast sensitivity decreased significantly by 66% for luminance and by 36% for color. A significant correlation was found for the strength of saccadic suppression of luminance and color. However, a small age effect was found only for the strength of saccadic suppression of luminance, which increased from 64% to 70% from young to old age. We conclude that saccadic suppression for luminance and color is present in most participants independent of their age and that mechanisms of suppression stay relatively stable during healthy aging.

Color consistency in the appearance of bleached fabrics.

Human observers are remarkably good at perceiving constant object color across illumination changes. However, there are numerous other factors that can modulate surface appearance, such as aging, bleaching, staining, or soaking. Despite this, we are often able to identify material properties across such transformations. Little is known about how and to what extent we can compensate for the accompanying color transformations. Here we investigated whether humans could reproduce the original color of bleached fabrics. We treated 12 different fabric samples with a commercial bleaching product. Bleaching increased luminance and decreased saturation. We presented photographs of the original and bleached samples on a computer screen and asked observers to match the fabric colors to an adjustable matching disk. Different groups of observers produced matches for original and bleached samples. One group of observers were instructed to match the color of the bleached samples as they were before bleaching (i.e., compensate for the effects of bleaching); another, to accurately match color appearance. Observers did compensate significantly for the effects of bleaching when instructed to do so, but not in the appearance match condition. Results of a second experiment suggest that observers achieve color consistency, at least in part, through a strategy based on local spatial differences within the bleached samples. According to the results of a third experiment, these local spatial differences are likely to be the perceptual image cues that allow participants to determine whether a sample is bleached. When the effect of bleaching was limited or uniformly distributed across a sample's surface, observers were uncertain about the bleaching magnitude and seemed to apply cognitive strategies to achieve color consistency.

A comparison of the temporal and spatial properties of trans-saccadic perceptual recalibration and saccadic adaptation.

Repeated exposure to a consistent trans-saccadic step in the position of the saccadic target reliably produces a change of saccadic gain, a well-established trans-saccadic motor learning phenomenon known as saccadic adaptation. Trans-saccadic changes can also produce perceptual effects. Specifically, a systematic increase or decrease in the size of the object that is being foveated changes the perceptually equivalent size between fovea and periphery. Previous studies have shown that this recalibration of perceived size can be established within a few dozen trials, persists overnight, and generalizes across hemifields. In the current study, we use a novel adjustment paradigm to characterize both temporally and spatially the learning process that subtends this form of recalibration, and directly compare its properties to those of saccadic adaptation. We observed that sinusoidal oscillations in the amplitude of the trans-saccadic change produce sinusoidal oscillations in the reported peripheral size, with a lag of under 10 trials. This is qualitatively similar to what has been observed in the case of saccadic adaptation. We also tested whether learning is generalized to the mirror location on the opposite hemifield for both size recalibration and saccade adaptation. Here the results were markedly different, showing almost complete generalization for recalibration and no generalization for saccadic adaptation. We conclude that perceptual and visuomotor consequences of trans-saccadic changes rely on learning mechanisms that are distinct but develop on similar time scales.

From Gaussian blobs to naturalistic videos: Comparison of oculomotor behavior across different stimulus complexities.

Research on eye movements has primarily been performed in two distinct ways: (1) under highly controlled conditions using simple stimuli such as dots on a uniform background, or (2) under free-viewing conditions with complex images, real-world movies, or even with observers moving around in the world. Although both approaches offer important insights, the generalizability among eye movement behaviors observed under these different conditions is unclear. Here, we compared eye movement responses to video clips showing moving objects within their natural context with responses to simple Gaussian blobs on a blank screen. Importantly, for both conditions, the targets moved along the same trajectories at the same speed. We measured standard oculometric measures for both stimulus complexities, as well as the effect of the relative angle between saccades and pursuit, and compared them across conditions. In general, eye movement responses were qualitatively similar, especially with respect to pursuit gain. For both types of stimuli, the accuracy of saccades and subsequent pursuit was highest when both eye movements were collinear. We also found interesting differences; for example, latencies of initial saccades to moving Gaussian blob targets were significantly faster compared to saccades to moving objects in video scenes, whereas pursuit accuracy was significantly higher in video scenes. These findings suggest a lower processing demand for simple target conditions during saccade preparation and an advantage for tracking behavior in natural scenes due to higher predictability provided by the context information.