Nonlinear cortical encoding of color predicts enhanced McCollough effects in anomalous trichromats.

Nonlinear encoding of chromatic contrast by the early visual cortex predicts that anomalous trichromats will show a larger McCollough effect than normal trichromats. In Experiment 1 we employed the McCollough effect to probe the cortical representation of saturation in normal trichromats, and used the results to predict enhanced McCollough effects for anomalous trichromats, which we measured in Experiment 2. In Experiment 1 three participants adapted to red and green orthogonal gratings of four different saturations. Using nulling to measure aftereffect strength, we found that halving the saturation of the inducing gratings decreased aftereffect strength only slightly, consistent with a compressive coding of saturation in early visual cortex. In anomalous trichromats, cone contrasts between red and green are greatly decreased from those of normal trichromats, but induced aftereffects are only slightly decreased, because of the non-linearity in the cortical encoding of saturation. To null the aftereffect, however, the retinal color deficiency must be overcome by adding more color to the null than required by normal trichromats. We confirmed this prediction in Experiment 2 where four anomalous trichromats required nulling stimuli approximately four times more saturated than did normal trichromats. We consider two competing models to explain our results: in a 'pigment swap' model anomalous trichromats have an altered photopigment but process color postreceptorally in the same way as normal trichromats; in a 'postreceptoral compensation' model the cortical representation of red-green contrasts is amplified to compensate for reduced cone contrasts. The latter provided a better fit to our data.

The channel for detecting contrast modulation also responds to density modulation (or vice versa).

In textures composed of black and white dots, we modulated dot density and/or dot contrast in one direction of visual space. Just as Mulligan and MacLeod (Vision Research 28 (1988) 503-519) found a strong reciprocity between density and luminance for dots viewed against a darker background, we found a strong reciprocity between density and contrast: detection thresholds for in-phase modulations of density and contrast were 30% - 55% lower than detection thresholds for density and contrast modulations that were 180° out of phase. These findings support the existence of at least one psychophysical channel that is excited by both density modulations and contrast modulations. A good, quantitative fit to our data can be obtained with a two-channel model.

Color discrimination in anomalous trichromacy: Experiment and theory.

In anomalous trichromacy, the color signals available from comparing the activities of the two classes of cone sensitive in the medium and long wavelength parts of the spectrum are much reduced from those available in normal trichromacy, and color discrimination thresholds along the red-green axis are correspondingly elevated. Yet there is evidence that suprathreshold color perception is relatively preserved; this has led to the suggestion that anomalous trichromats post-receptorally amplify their impoverished red-green signals. To test this idea, we measured chromatic discrimination from white and from saturated red and green pedestals. If there is no post-receptoral compensation, the anomalous trichromat's loss of chromatic contrast will apply equally to the pedestal and to the test color. Coupled with a compressively nonlinear neural representation of saturation, this means that a given pedestal contrast will cause a smaller than normal modulation of discrimination sensitivity. We examined cases where chromatic pedestals impair the color discrimination of normal trichromatic observers. As predicted, anomalous observers experienced less impairment than normal trichromats, though they remained less sensitive than normal trichromats. Although the effectiveness of chromatic pedestals in impairing color discrimination was less for anomalous than for normal trichromats, the chromatic pedestals were more effective for anomalous observers than would be expected if the anomalous post-receptoral visual system were the same as in normal trichromacy; the hypothesis of zero compensation can be rejected. This might suggest that the effective contrast of the pedestal is post-receptorally amplified. But on closer analysis, the results do not support candidate simple models involving post-receptoral compensation either.

Color Compensation in Anomalous Trichromats Assessed with fMRI.

Anomalous trichromacy is a common form of congenital color deficiency resulting from a genetic alteration in the photopigments of the eye's light receptors. The changes reduce sensitivity to reddish and greenish hues, yet previous work suggests that these observers may experience the world to be more colorful than their altered receptor sensitivities would predict, potentially indicating an amplification of post-receptoral signals. However, past evidence suggesting such a gain adjustment rests on subjective measures of color appearance or salience. We directly tested for neural amplification by using fMRI to measure cortical responses in color-anomalous and normal control observers. Color contrast response functions were measured in two experiments with different tasks to control for attentional factors. Both experiments showed a predictable reduction in chromatic responses for anomalous trichromats in primary visual cortex. However, in later areas V2v and V3v, chromatic responses in the two groups were indistinguishable. Our results provide direct evidence for neural plasticity that compensates for the deficiency in the initial receptor color signals and suggest that the site of this compensation is in early visual cortex.

Are hue and saturation carried in different neural channels?

Chromatic discrimination data show that a smaller physical stimulus change is required to detect a change in hue than to detect a change in saturation [Palette30, 21 (1968); Proc. R. Soc. London Ser. B283, 20160164 (2016)PRLBA40080-464910.1098/rspb.2016.0164], and, on this basis, it has been suggested that hue and saturation are carried in different neural channels [Color Space and Its Divisions: Color Order from Antiquity to the Present (Wiley, 2003), p. 311]. We used an adaptation paradigm to test explicitly for separate mechanisms, measuring hue and saturation detection thresholds before and after adaptation to hue and saturation stimuli. Within-condition adaptation did not elevate detection thresholds significantly more than between-condition adaptation. We therefore did not find psychophysical evidence for a neural channel that extracts hue thresholds more effectively than the neural channel or channels that determine saturation thresholds.

Mesopic luminance assessed with minimally distinct border perception.

In photopic vision, the border between two fields is minimally distinct when the two fields are isoluminant; that is, when the achromatic luminance of the two fields is equal. The distinctness of a border between extrafoveal reference and comparison fields was used here as an isoluminance criterion under a variety of adaptation conditions ranging from photopic to scotopic. The adjustment was done by trading off the amount of blue against the amount of red in the comparison field. Results show that isoluminant border settings are linear under all constant adaptation conditions, though varying with state of adaptation. The relative contribution of rods and cones to luminance was modeled such that the linear sum of the suitably weighted scotopic and photopic luminance is constant for the mesopic isoluminant conditions. The relative weights change with adapting intensity in a sigmoid fashion and also depend strongly on the position of the border in the visual field.

A lack of experience-dependent plasticity after more than a decade of recovered sight.

In 2000, monocular vision was restored to M. M., who had been blind between the ages of 3 and 46 years. Tests carried out over 2 years following the surgery revealed impairments of 3-D form, object, and face processing and an absence of object- and face-selective blood-oxygen-level-dependent responses in ventral visual cortex. In the present research, we reexamined M. M. to test for experience-dependent recovery of visual function. Behaviorally, M. M. remains impaired in 3-D form, object, and face processing. Accordingly, we found little to no evidence of the category-selective organization within ventral visual cortex typically associated with face, body, scene, or object processing. We did observe remarkably normal object selectivity within lateral occipital cortex, consistent with M. M.'s previously reported shape-discrimination performance. Together, these findings provide little evidence for recovery of high-level visual function after more than a decade of visual experience in adulthood.

Depth and luminance edges attract.

The spatial resolution of disparity perception is poor compared to luminance perception, yet we do not notice that depth edges are more blurry than luminance edges. Is this because the two cues are combined by the visual system? Subjects judged the locations of depth-defined or luminance-defined edges, which were separated by up to 5.6 min of arc. The perceived edge location was a function of the depth-defined edge and the luminance-defined edge, with the luminance edge tending to play a larger role. Our data are compatible with but not completely explained by an optimal cue-combination model that gives more reliable cues a heavier weight. Both edge cues (depth and luminance) contribute to the final percept, with an adaptive weighting depending on the task and the acuity with which each cue is perceived.

Mechanisms of the dimming and brightening aftereffects.

Dimming and brightening aftereffects occur after exposure to a temporal luminance sawtooth stimulus: A subsequently presented steady test field appears to become progressively dimmer or brighter, depending on the polarity of the adapting sawtooth. Although described as "dimming" and "brightening," it is plausible that a component of the aftereffects is based on contrast changes rather than on luminance changes. We conducted two experiments to reveal any contrast component. In the first we investigated whether the aftereffects result from the same mechanism that causes a polarity-selective loss in contrast sensitivity following luminance sawtooth adaptation. We manipulated test contrast: If a component of the aftereffect results from a polarity selective loss of contrast sensitivity we would expect that the aftereffects would differ in magnitude depending on the contrast polarity of the test fields. We found no effect of test-field polarity. In the second experiment we used an adapting sawtooth with a polarity consistent in contrast but alternating in luminance in order to induce a potential equivalent aftereffect of contrast. Again, we found no evidence that the aftereffects result from contrast adaptation. In a third experiment, we used S-cone isolating stimuli to discover whether there are S-cone dimming and brightening aftereffects. We found no aftereffects. However, in a fourth experiment we replicated Krauskopf and Zaidi's (1986) finding that adaptation to S-cone sawtooth stimuli affects thresholds for increment and decrement detection. The mechanism underlying the dimming and brightening aftereffects thus seems to be independent of the mechanism underlying the concurrent polarity selective reductions in contrast sensitivity.

Estimating illuminant color based on luminance balance of surfaces.

To accomplish color constancy the illuminant color needs to be discounted from the light reflected from surfaces. Some strategies for discounting the illuminant color use statistics of luminance and chromaticity distribution in natural scenes. In this study we showed whether color constancy exploits the potential cue that was provided by the luminance balance of differently colored surfaces. In our experiments we used six colors: bright and dim red, green, and blue, as surrounding stimulus colors. In most cases, bright colors were set to be optimal colors. They were arranged among 60 hexagonal elements in close-packed structure. The center element served as the test stimulus. The observer adjusted the chromaticity of the test stimulus to obtain a perceptually achromatic surface. We used simulated black body radiations of 3000 (or 4000), 6500, and 20000 K as test illuminants. The results showed that the luminance balance of surfaces with no chromaticity shift had clear effects on the observer's achromatic setting, which was consistent with our hypothesis on estimating the scene illuminant based on optimal colors.

Mesopic luminance assessed with minimum motion photometry.

We measured the relative contribution of rods and cones to luminance across a range of photopic, mesopic, and scotopic adaptation levels and at various retinal eccentricities. We isolated the luminance channel by setting motion-based luminance nulls (minimum motion photometry) using annular stimuli. Luminance nulls between differently colored stimuli require equality in a weighted sum of rod and cone excitations. The relative cone weight increases smoothly from the scotopic range, where rods dominate, to photopic levels, where rod influence becomes negligible. The change from rod to cone vision does not occur uniformly over the visual field. The more peripheral the stimulus location, the higher is the light level required for cones to participate strongly. The relative cone contribution can be described by a sigmoid function of intensity, with two parameters that each depend on the eccentricity and spatial frequency of the stimulus. One parameter determines the "meso-mesopic" luminance--the center of the mesopic range, at which rod and cone contributions are balanced. This increases with eccentricity, reflecting an increase in the meso-mesopic luminance from 0.04 scotopic cd/m(2) at 2° eccentricity to 0.44 scotopic cd/m(2) at 18°. The second parameter represents the slope of the log-log threshold-versus-intensity curve (TVI curve) for rod vision. This parameter inversely scales the width of the mesopic range and increases only slightly with eccentricity (from 0.73 at 2° to 0.78 for vision at 18° off-axis).

Spatiotemporal averaging of perceived brightness along an apparent motion trajectory.

Objects are critical functional units for many aspects of visual perception and recognition. Many psychophysical experiments support the concept of an "object file" consisting of characteristics attributed to a single object on the basis of successive views of it, but there has been little evidence that object identity influences apparent brightness and color. In this study, we investigated whether the perceptual identification of successive flashed stimuli as views of a single moving object could affect brightness perception. Our target stimulus was composed of eight wedge-shaped sectors. The sectors were presented successively at different inter-flash intervals along an annular trajectory. At inter-flash intervals of around 100 ms, the impression was of a single moving object undergoing long-range apparent motion. By modulating the luminance between successive views, we measured the perception of luminance modulation along the trajectory of this long-range apparent motion. At the inter-flash intervals where the motion perception was strongest, the luminance difference was perceptually underestimated, and forced-choice luminance discrimination thresholds were elevated. Moreover, under such conditions, it became difficult for the observer to correctly associate or "bind" spatial positions and wedge luminances. These results indicate that the different luminances of wedges that were perceived as a single object were averaged along its apparent motion trajectory. The large spatial step size of our stimulus makes it unlikely that the results could be explained by averaging in a low-level mechanism that has a compact spatiotemporal receptive field (such as V1 and V2 neurons); higher level global motion or object mechanisms must be invoked to account for the averaging effect. The luminance averaging and the ambiguity of position-luminance "binding" suggest that the visual system may evade some of the costs of rapidly computing apparent brightness by adopting the assumption that the characteristics of an object are invariant over successive views.

Visual adaptation and face perception.

The appearance of faces can be strongly affected by the characteristics of faces viewed previously. These perceptual after-effects reflect processes of sensory adaptation that are found throughout the visual system, but which have been considered only relatively recently in the context of higher level perceptual judgements. In this review, we explore the consequences of adaptation for human face perception, and the implications of adaptation for understanding the neural-coding schemes underlying the visual representation of faces. The properties of face after-effects suggest that they, in part, reflect response changes at high and possibly face-specific levels of visual processing. Yet, the form of the after-effects and the norm-based codes that they point to show many parallels with the adaptations and functional organization that are thought to underlie the encoding of perceptual attributes like colour. The nature and basis for human colour vision have been studied extensively, and we draw on ideas and principles that have been developed to account for norms and normalization in colour vision to consider potential similarities and differences in the representation and adaptation of faces.

Seeing faces as objects: no face inversion effect with geometrical discrimination.

Inversion dramatically impairs face perception, recognition, and discrimination. Yet it does not interfere with the ability to make precise estimates of facial feature distances. To investigate this discontinuity between facial feature distance estimation and general perception and recognition, we assessed the effect of inversion on the discrimination of differences in facial compression and elongation or expansion using geometrically distorted faces. The results clearly showed that geometrical face discrimination is not subject to the traditional face inversion effect and did not show a benefit for natural faces. Although discrimination thresholds were not affected by inversion, response times to the distance judgments were faster with inversion, especially when the inverted faces contained natural configurations. Based on these counterintuitive results, we suggest that participants used analytical processing to do the discrimination task. Moreover, we suggest that the depth with which a face is holistically encoded depends on the nature of the task, face orientation, and similarity between a face and the prototypical face template.

Imperceptibly rapid contrast modulations processed in cortex: Evidence from psychophysics.

Rapid fluctuations in contrast are common in our modern visual environment. They arise, for example, in a room lit by a fluorescent light, when viewing a CRT computer monitor and when watching a movie in a cinema. As we are unconscious of the rapid changes, it has been assumed that they do not affect the operation of our visual systems. By periodically reversing the contrast of a fixed pattern at a rapid rate we render the pattern itself, as well as the modulations, invisible to observers. We show that exposure to these rapidly contrast-modulated patterns alters the way subsequent stationary patterns are processed; patterns similar to the contrast-modulated pattern require more contrast to be detected than dissimilar patterns. We present evidence that the changes are cortically mediated. Taken together, our findings suggest that cortical stages of the visual system respond to the individual frames of a contrast-reversed sequence, even at rates as high as 160 frames per second.

Functional adaptive sequential testing.

The study of cognition, perception, and behavior often requires the estimation of thresholds as a function of continuous independent variables (e.g., contrast threshold as a function of spatial frequency, subjective value as a function of reward delay, tracking speed as a function of the number of objects tracked). Unidimensional adaptive testing methods make estimation of single threshold values faster and more efficient, but substantial efficiency can be further gained by taking into account the relationship between thresholds at different values of an independent variable. Here we present a generic method--1functional adaptive sequential testing (FAST)--for estimating thresholds as a function of another variable. This method allows efficient estimation of parameters relating an independent variable (e.g., stimulus spatial frequency; or reward delay) to the measured threshold along a stimulus strength dimension (e.g., contrast; or present monetary value). We formally describe the FAST algorithm and introduce a Matlab toolbox implementation thereof; we then evaluate several possible sampling and estimation algorithms for such two-dimensional functions. Our results demonstrate that efficiency can be substantially increased by considering the functional relationship between thresholds at different values of the independent variable of interest.

The effect of notched noise on flicker detection and discrimination.

Flicker perception was investigated using two-alternative forced-choice detection and discrimination tasks with four different types of external noise: (1) broadband noise, (2) 5-Hz notched-noise--broadband noise with a 5-Hz band centered on the signal frequency removed, (3) 10-Hz notched-noise, and (4) no external noise. The signal was a burst of 10-Hz sinusoidal flicker presented in one of two observation intervals. In discrimination experiments, a pedestal--sinusoidal flicker with the same frequency, duration, and phase as the signal--was added to both observation intervals. With no noise, observers' performance first improved with increasing pedestal modulation, before deteriorating in accordance with Weber's Law, producing the typical "dipper" shaped plot of signal versus pedestal modulation. Noise affects performance, but the dipper effect persisted in each type of noise. The results exclude three models: the ideal-observer in which the pedestal improves performance by specifying the signal exactly; off-frequency-looking models in which the dipper depends on detection by channels tuned to temporal frequencies different from that of the signal; and strict energy detectors. Our data are consistent with signal processing by a single mechanism with an expansive non-linearity for near-threshold signal modulations (with an exponent of six) and a compressive "Weberian" non-linearity for high modulations.

The McCollough effect reflects permanent and transient adaptation in early visual cortex.

The brain encounters input varying with many different time courses. Given such temporal variability, it would seem practical for adaptation to operate at multiple timescales. Indeed, to account for peculiar effects such as spacing, savings, and spontaneous recovery, many recent models of learning and adaptation have postulated multiple mechanisms operating at different timescales. However, despite this assumption, and compelling modelling results, different timescales of cortical adaptation and learning are rarely isolated in behaving animals. Here we demonstrate in a series of experiments that early visual cortex adapts at two distinct and separable timescales: fast (saturating with a time constant of roughly 30 seconds) and infinite (a perfect integrator: exhibiting no signs of decay or diminishing returns within the range of intervals tested). We further demonstrate that these two timescales sum linearly and appear to be operating independently and in parallel.

Contingent aftereffects distinguish conscious and preconscious color processing.

The brain can process input without perception, but what distinguishes conscious from preconscious processing? Using aftereffects induced by quickly alternating images, we show that cortical mechanisms track color much faster than perception, responding well to color alternations that are too rapid to be perceptible. The more restricted frequency response of the conscious perception of color suggests that extra integrative steps give conscious color perception a time course substantially slower than that of early cortical mechanisms.

Flexibility of spatial averaging in visual perception.

The classical receptive field (RF) concept-the idea that a visual neuron responds to fixed parts and properties of a stimulus-has been challenged by a series of recent physiological results. Here, we extend these findings to human vision, demonstrating that the extent of spatial averaging in contrast perception is also flexible, depending strongly on stimulus contrast and uniformity. At low contrast, spatial averaging is greatest (about 11 min of arc) within uniform regions such as edges, as expected if the relevant neurons have orientation-selective RFs. At high contrast, spatial averaging is minimal. These results can be understood if the visual system is balancing a trade-off between noise reduction, which favours large areas of averaging, and detail preservation, which favours minimal averaging. Two distinct populations of neurons with hard-wired RFs could account for our results, as could the more intriguing possibility of dynamic, contrast-dependent RFs.