Gromov-Wasserstein unsupervised alignment reveals structural correspondences between the color similarity structures of humans and large language models.

Large Language Models (LLMs), such as the General Pre-trained Transformer (GPT), have shown remarkable performance in various cognitive tasks. However, it remains unclear whether these models have the ability to accurately infer human perceptual representations. Previous research has addressed this question by quantifying correlations between similarity response patterns of humans and LLMs. Correlation provides a measure of similarity, but it relies pre-defined item labels and does not distinguish category- and item- level similarity, falling short of characterizing detailed structural correspondence between humans and LLMs. To assess their structural equivalence in more detail, we propose the use of an unsupervised alignment method based on Gromov-Wasserstein optimal transport (GWOT). GWOT allows for the comparison of similarity structures without relying on pre-defined label correspondences and can reveal fine-grained structural similarities and differences that may not be detected by simple correlation analysis. Using a large dataset of similarity judgments of 93 colors, we compared the color similarity structures of humans (color-neurotypical and color-atypical participants) and two GPT models (GPT-3.5 and GPT-4). Our results show that the similarity structure of color-neurotypical participants can be remarkably well aligned with that of GPT-4 and, to a lesser extent, to that of GPT-3.5. These results contribute to the methodological advancements of comparing LLMs with human perception, and highlight the potential of unsupervised alignment methods to reveal detailed structural correspondences.

"Warm," "cool," and the colors.

Participants judged affective cooler/warmer gradients around a 12-step color circle. Each pair of adjacent colors was presented twice (left-right reversed), all in random order. Participants readily performed the task, but their settings do not correlate very well. Individual responses were compared with a small number of canonical templates. For a little less than one-half of the participants responses or judgements correlate with such a template. We find a warm pole (in the orange environment) and a cool pole (in the teal environment) connected with two tracks that tend to have one or more gaps or weak, even inverted links. We conclude that the common artistic cool-warm polarity is only weakly reflected in responses of our observers. If it does, the observers apparently use categorical warm and cool poles and may be uncertain in relating adjacent hue steps along the 12-step color circle.

The human brain deals with violating general color or depth knowledge in different time courses.

Utilizing the high temporal resolution of event-related potentials (ERPs), we compared the time course of processing incongruent color versus 3D-depth information. Participants were asked to judge whether the food color (color condition) or 3D structure (3D-depth condition) was congruent or incongruent with their previous knowledge and experience. The behavioral results showed that the reaction times in the congruent 3D-depth condition were slower than those in the congruent color condition. The reaction times in the incongruent 3D-depth condition were slower than those in the incongruent color condition. The ERP results showed that incongruent color stimuli induced a larger N270, larger P300, and smaller N400 components in the fronto-central region than the congruent color stimuli. Incongruent 3D-depth stimuli induced a smaller N1 in the occipital region, larger P300 and smaller N400 in the parietal-occipital region than congruent 3D-depth stimuli. The time-frequency analysis found that incongruent color stimuli induced a larger theta band (360-580 ms) activation in the fronto-central region than congruent color stimuli. Incongruent 3D-depth stimuli induced larger alpha and beta bands (240-350 ms) activation in the parietal region than congruent 3D-depth stimuli. Our results suggest that the human brain deals with violating general color or depth knowledge in different time courses. We speculate that the depth perception conflict was dominated by solving the problem with visual processing, whereas the color perception conflict was dominated by solving the problem with semantic violation.

Watercolor spreading in Bridget Riley's and Piet Mondrian's op-art placed in the context of recent watercolor studies.

The watercolor effect (WCE) is a striking visual illusion elicited by a bichromatic double contour, such as a light orange and a dark purple, hugging each other on a white background. Color assimilation, emanating from the lighter contour, spreads onto the enclosed surface area, thereby tinting it with a chromatic veil, not unlike a weak but real color. Map makers in the 17th century utilized the WCE to better demarcate the shape of adjoining states, while 20th-century artist Bridget Riley created illusory watercolor as part of her op-art. Today's visual scientists study the WCE for its filling-in properties and strong figure-ground segregation. This review emphasizes the superior strength of the WCE for grouping and figure-ground organization vis-à-vis the classical Gestalt factors of Max Wertheimer (1923), thereby inspiring a notion of form from induced color. It also demonstrates that a thin chromatic line, flanking the inside of a black Mondrian-type pattern, induces the WCE across a large white surface area. Phenomenological, psychophysical, and neurophysiological approaches are reviewed.

Unique yellow shifts for small and brief stimuli in the central retina.

The spectral locus of unique yellow was determined for flashes of different sizes (<11 arcmin) and durations (<500 ms) presented in and near the fovea. An adaptive optics scanning laser ophthalmoscope was used to minimize the effects of higher-order aberrations during simultaneous stimulus delivery and retinal imaging. In certain subjects, parafoveal cones were classified as L, M, or S, which permitted the comparison of unique yellow measurements with variations in local L/M ratios within and between observers. Unique yellow shifted to longer wavelengths as stimulus size or duration was reduced. This effect is most pronounced for changes in size and more apparent in the fovea than in the parafovea. The observed variations in unique yellow are not entirely predicted from variations in L/M ratio and therefore implicate neural processes beyond photoreception.

Division and spreading of attention across color.

Biological systems must allocate limited perceptual resources to relevant elements in their environment. This often requires simultaneous selection of multiple elements from the same feature dimension (e.g. color). To establish the determinants of divided attentional selection of color, we conducted an experiment that used multicolored displays with four overlapping random dot kinematograms that differed only in hue. We manipulated (i) requirement to focus attention to a single color or divide it between two colors; (ii) distances of distractor hues from target hues in a perceptual color space. We conducted a behavioral and an electroencephalographic experiment, in which each color was tagged by a specific flicker frequency and driving its own steady-state visual evoked potential. Behavioral and neural indices of attention showed several major consistencies. Concurrent selection halved the neural signature of target enhancement observed for single targets, consistent with an approximately equal division of limited resources between two hue-selective foci. Distractors interfered with behavioral performance in a context-dependent fashion but their effects were asymmetric, indicating that perceptual distance did not adequately capture attentional distance. These asymmetries point towards an important role of higher-level mechanisms such as categorization and grouping-by-color in determining the efficiency of attentional allocation in complex, multicolored scenes.

Multistage maturation optimizes vision.

Late development of color vision improves object recognition.

Transparent layer constancy improves with increased naturalness of the scene.

The extent to which hue, saturation, and transmittance of thin light-transmitting layers are perceived as constant when the illumination changes (transparent layer constancy, TLC) has previously been investigated with simple stimuli in asymmetric matching tasks. In this task, a target filter is presented under one illumination and a second filter is matched under a second illumination. Although two different illuminations are applied in the stimulus generation, there is no guarantee that the stimulus will be interpreted appropriately by the visual system. In previous work, we found a higher degree of TLC when both illuminations were presented alternately than when they were presented simultaneously, which could be explained, for example, by an increased plausibility of an illumination change. In this work, we test whether TLC can also be increased in simultaneous presentation when the filter's belonging to a particular illumination context is made more likely by additional cues. To this end, we presented filters in differently lit areas of complex, naturalistically rendered 3D scenes containing different types of cues to the prevailing illumination, such as scene geometry, object shading, and cast shadows. We found higher degrees of TLC in such complex scenes than in colorimetrically similar simple 2D color mosaics, which is consistent with the results of similar studies in the area of color constancy. To test which of the illumination cues available in the scenes are actually used, the different types of cues were successively removed from the naturalistically rendered complex scene. A total of eight levels of scene complexity were examined. As expected, TLC decreased the more cues were removed. Object shading and illumination gradients due to shadow cast were both found to have a positive effect on TLC. A second filter had a small positive effect on TLC when added in strongly reduced scenes, but not in the complex scenes that already provide many cues about the illumination context of the filter.

Flexible social monitoring as revealed by eye movements: Spontaneous mental state updating triggered by others' unexpected actions.

Successful interactions require not only representing others' mental states but also flexibly updating them, whenever one's original inferences may no longer hold. Such situations arise, for instance, when a partner's behavior is incongruent with one's expectations. Although these situations are rather common, the question whether people update others' mental states spontaneously upon encountering unexpected behaviors and whether they use the updated mental states in novel contexts, has been largely unexplored. We addressed these issues in two experiments. In each experiment participants first performed an anticipatory looking task, reacting to a virtual 'partner', who categorized pictures based on their ambiguous or non-ambiguous color. Importantly, to perform the task participants did not have to track their partner's perspective. Following a correct categorization phase, the 'partner' started to systematically miscategorize one of the ambiguous colors (e.g., as if she would now believe that the greenish blue is green). We measured how participants' anticipatory looking preceding the partner's categorization changed across trials. Afterward, we asked whether participants implicitly transferred their knowledge about the partner's updated perspective to a new task. Finally, they performed an explicit perspective-taking task, to test whether they selectively updated the partner's perspective, but not their own. Results revealed that correct anticipations started to emerge only after a few miscategorizations, indicating the spontaneous updating of the other's perspective regarding the miscategorized color. Signatures of updating emerged somewhat earlier when the partner made similarity judgments (Experiment 2), highlighting the subjective nature of her decisions, compared to when following an explicit color-categorization rule (Experiment 1). In the explicit perspective-taking task of both experiments, roughly half of the participants could categorize items according to the partner's (spontaneously updated) perspective and also used their partner's updated perspective in the implicit transfer task to some degree, while they were the ones who displayed more pronounced anticipatory patterns as well. Such data provides strong evidence that the observed changes in anticipatory looking reflect spontaneous and flexible mental state updating. In addition, the findings also point to a high individual variability both in the updating of attributed mental states and the use of the updated mental state content.

Feature-invariant processing of spatial segregation based on temporal asynchrony.

Temporal asynchrony is a cue for the perceptual segregation of spatial regions. Past research found attribute invariance of this phenomenon such that asynchrony induces perceptual segmentation regardless of the changing attribute type, and it does so even when asynchrony occurs between different attributes. To test the generality of this finding and obtain insights into the underlying computational mechanism, we compared the segmentation performance for changes in luminance, color, motion direction, and their combinations. Our task was to detect the target quadrant in which a periodic alternation in attribute was phase-delayed compared to the remaining quadrants. When stimulus elements made a square-wave attribute change, target detection was not clearly attribute invariant, being more difficult for motion direction change than for luminance or color changes and nearly impossible for the combination of motion direction and luminance or color. We suspect that waveform mismatch might cause anomalous behavior of motion direction since a square-wave change in motion direction is a triangular-wave change in the spatial phase (i.e., a second-order change in the direction of the spatial phase change). In agreement with this idea, we found that the segregation performance was strongly affected by the waveform type (square wave, triangular wave, or their combination), and when this factor was controlled, the performance was nearly, though not perfectly, invariant against attribute type. The results were discussed with a model in which different visual attributes share a common asynchrony-based segmentation mechanism.

Hue selectivity from recurrent circuitry in Drosophila.

In the perception of color, wavelengths of light reflected off objects are transformed into the derived quantities of brightness, saturation and hue. Neurons responding selectively to hue have been reported in primate cortex, but it is unknown how their narrow tuning in color space is produced by upstream circuit mechanisms. We report the discovery of neurons in the Drosophila optic lobe with hue-selective properties, which enables circuit-level analysis of color processing. From our analysis of an electron microscopy volume of a whole Drosophila brain, we construct a connectomics-constrained circuit model that accounts for this hue selectivity. Our model predicts that recurrent connections in the circuit are critical for generating hue selectivity. Experiments using genetic manipulations to perturb recurrence in adult flies confirm this prediction. Our findings reveal a circuit basis for hue selectivity in color vision.

Chromatic visual evoked potentials: A review of physiology, methods and clinical applications.

Objective assessment of the visual system can be performed electrophysiologically using the visual evoked potential (VEP). In many clinical circumstances, this is performed using high contrast achromatic patterns or diffuse flash stimuli. These methods are clinically valuable but they may only assess a subset of possible physiological circuitries within the visual system, particularly those involved in achromatic (luminance) processing. The use of chromatic VEPs (cVEPs) in addition to standard VEPs can inform us of the function or dysfunction of chromatic pathways. The chromatic VEP has been well studied in human health and disease. Yet, to date our knowledge of their underlying mechanisms and applications remains limited. This likely reflects a heterogeneity in the methodology, analysis and conclusions of different works, which leads to ambiguity in their clinical use. This review sought to identify the primary methodologies employed for recording cVEPs. Furthermore cVEP maturation and application in understanding the function of the chromatic system under healthy and diseased conditions are reviewed. We first briefly describe the physiology of normal colour vision, before describing the methodologies and historical developments which have led to our understanding of cVEPs. We thereafter describe the expected maturation of the cVEP, followed by reviewing their application in several disorders: congenital colour vision deficiencies, retinal disease, glaucoma, optic nerve and neurological disorders, diabetes, amblyopia and dyslexia. We finalise the review with recommendations for testing and future directions.

Facial expressions affect the memory of facial colors.

Facial color influences the perception of facial expressions, and emotional expressions bias how facial color is remembered. However, it remains unclear whether facial expressions affect daily facial color memory. The memory color effect demonstrates that knowledge about typical colors affects the perception of the actual color of given objects. To investigate the effect of facial color memory, we examined whether the memory color effect for faces varies depending on facial expression. We calculated the subjective achromatic point of the facial expression image stimulus and compared the degree to which it was shifted from the actual achromatic point between facial expression conditions. We hypothesized that if the memory of facial color is influenced by the facial expression color (e.g., anger is a warm color, fear is a cold color), then the subjective achromatic point would vary with facial expression. In Experiment 1, we recruited 13 participants who adjusted the color of facial expression stimuli (anger, neutral, and fear) and a banana stimulus to be achromatic. No significant differences in the subjective achromatic point between facial expressions were observed. Subsequently, we conducted Experiment 2 with 23 participants because Experiment 1 did not account for the sensitivity to color changes on the face; humans perceive greater color differences in faces than in non-faces. Participants selected which facial color they believed the expression stimulus appeared to be, choosing one of two options provided to them. The results indicated that the subjective achromatic points of anger and fear faces significantly shifted toward the opposite color direction compared with neutral faces in the brief presentation condition. This research suggests that the memory color of faces differs depending on facial expressions and supports the idea that the perception of emotional expressions can bias facial color memory.

The object as the unit for state switching in visual working memory.

This study aimed to determine the unit for switching representational states in visual working memory (VWM). Two opposing hypotheses were investigated: (a) the unit of switching being a feature (feature-based hypothesis), and (b) the unit of switching being an object (object-based hypothesis). Participants (N = 180) were instructed to hold two features from either one or two objects in their VWM. The memory-driven attentional capture effect, suggesting that actively held information in VWM can cause attention to be drawn towards matched distractors, was employed to assess representational states of the first and second probed colors (indicated by a retro-cue). The results showed that only the feature indicated to be probed first could elicit memory related capture for the condition of separate objects. Importantly, features from an integrated object could guide attention regardless of the probe order. These findings were observed across three experiments involving features of different dimensions, same dimensions, or perceptual objects defined by Gestalt principles. They provide convergent evidence supporting the object-based hypothesis by indicating that features within a single object cannot exist in different states.

Should absolute pitch be considered as a unique kind of absolute sensory judgment in humans? A systematic and theoretical review of the literature.

Absolute pitch is the name given to the rare ability to identify a musical note in an automatic and effortless manner without the need for a reference tone. Those individuals with absolute pitch can, for example, name the note they hear, identify all of the tones of a given chord, and/or name the pitches of everyday sounds, such as car horns or sirens. Hence, absolute pitch can be seen as providing a rare example of absolute sensory judgment in audition. Surprisingly, however, the intriguing question of whether such an ability presents unique features in the domain of sensory perception, or whether instead similar perceptual skills also exist in other sensory domains, has not been explicitly addressed previously. In this paper, this question is addressed by systematically reviewing research on absolute pitch using the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) method. Thereafter, we compare absolute pitch with two rare types of sensory experience, namely synaesthesia and eidetic memory, to understand if and how these phenomena exhibit similar features to absolute pitch. Furthermore, a common absolute perceptual ability that has been often compared to absolute pitch, namely colour perception, is also discussed. Arguments are provided supporting the notion that none of the examined abilities can be considered like absolute pitch. Therefore, we conclude by suggesting that absolute pitch does indeed appear to constitute a unique kind of absolute sensory judgment in humans, and we discuss some open issues and novel directions for future research in absolute pitch.

Age-related differences in saccadic indices of top-down guidance via short-term memory during visual search.

Aging has been associated with significant declines in the speed and accuracy of visual search. These effects have been attributed partly to low-level (bottom-up) factors including reductions in sensory acuity and general processing speed. Aging is also associated with changes in top-down attentional control, but the impact of these on search is less well-understood. The present study investigated age-related differences in top-down attentional control by comparing the speed and accuracy of saccadic sampling in the presence and absence of top-down information about target color in young (YA) and older (OA) observers. Displays contained an equal number of red and blue Landholt stimuli. Targets were distinguished from distractors by a unique orientation, and observers reported the direction of the target's gap on each trial. Single-target cues signaled the color of the target with 100% validity. Dual-target cues indicated the target could be present in either colored subgroup. The results revealed reliable group differences in the benefits associated with top-down information on single-target cues compared to dual-target cues. On single-target searches, OA made significantly more saccades than YA to stimuli in the uncued color subset. Single-target cues also produced a smaller advantage in the time taken to fixate the target in OA compared to YA. These results support an age-related decline in observers' use of top-down information to restrict sequences of saccades to a task-relevant subset of objects during visual search. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Influence of flavor information on visual search: Attentional capture by and suppression of flavor-associated colors.

Numerous studies have demonstrated the impact of flavor cues on visual search, yet the underlying mechanisms remain elusive. In this experiment, we used event-related potentials (ERPs) to examine whether, and if so, how flavor information could lead to attentional capture by, and suppression of, flavor-associated colors. The participants were asked to taste certain flavored beverages and subsequently complete a shape-based visual search task, while their neural activities were simultaneously recorded. The behavioral results revealed that the participants made slower responses when a distractor in the flavor-associated color (DFAC) was present, suggesting an attentional bias toward the flavor-associated color. The ERP results revealed that the N2pc was detected if the target and the DFAC were shown in the same visual field (e.g. both target and DFCA on the right side of the screen), when the pairings between flavor cues and target colors were incongruent. However, the N2pc was not observed if the target and the DFAC were shown in the opposite visual fields (e.g. target on the right and DFCA on the left side of the screen) for the incongruent color-flavor pairings. Moreover, the distractor positivity (Pd) was observed if the target and the DFAC were shown in the opposite visual field for the congruent color-flavor pairings. These results suggest that both attentional capture and suppression are involved in the influence of flavor information on visual search. Collectively, these findings provide initial electrophysiological evidence on the mechanisms of the crossmodal influence of flavor cues on visual search.

Non-invasive electrical brain stimulation modulates human conscious perception of mental representation.

Mental representation is a key concept in cognitive science; nevertheless, its neural foundations remain elusive. We employed non-invasive electrical brain stimulation and functional magnetic resonance imaging to address this. During this process, participants perceived flickering red and green visual stimuli, discerning them either as distinct, non-fused colours or as a mentally generated, fused colour (orange). The application of transcranial alternating current stimulation to the medial prefrontal region (a key node of the default-mode network) suppressed haemodynamic activation in higher-order subthalamic and central executive networks associated with the perception of fused colours. This implies that higher-order thalamocortical and default-mode networks are crucial in humans' conscious perception of mental representation.

Color constancy mechanisms in virtual reality environments.

Prior research has demonstrated high levels of color constancy in real-world scenarios featuring single light sources, extensive fields of view, and prolonged adaptation periods. However, exploring the specific cues humans rely on becomes challenging, if not unfeasible, with actual objects and lighting conditions. To circumvent these obstacles, we employed virtual reality technology to craft immersive, realistic settings that can be manipulated in real time. We designed forest and office scenes illuminated by five colors. Participants selected a test object most resembling a previously shown achromatic reference. To study color constancy mechanisms, we modified scenes to neutralize three contributors: local surround (placing a uniform-colored leaf under test objects), maximum flux (keeping the brightest object constant), and spatial mean (maintaining a neutral average light reflectance), employing two methods for the latter: changing object reflectances or introducing new elements. We found that color constancy was high in conditions with all cues present, aligning with past research. However, removing individual cues led to varied impacts on constancy. Local surrounds significantly reduced performance, especially under green illumination, showing strong interaction between greenish light and rose-colored contexts. In contrast, the maximum flux mechanism barely affected performance, challenging assumptions used in white balancing algorithms. The spatial mean experiment showed disparate effects: Adding objects slightly impacted performance, while changing reflectances nearly eliminated constancy, suggesting human color constancy relies more on scene interpretation than pixel-based calculations.

The color of fruits in photographs and still life paintings.

Still life paintings comprise a wealth of data on visual perception. Prior work has shown that the color statistics of objects show a marked bias for warm colors. Here, we ask about the relative chromatic contrast of these object-associated colors compared with background colors in still life paintings. We reasoned that, owing to the memory color effect, where the color of familiar objects is perceived more saturated, warm colors will be relatively more saturated than cool colors in still life paintings as compared with photographs. We analyzed color in 108 slides of still life paintings of fruit from the teaching slide collection of the Fogg University Art Museum and 41 color-calibrated photographs of fruit from the McGill data set. The results show that the relatively higher chromatic contrast of warm colors was greater for paintings compared with photographs, consistent with the hypothesis.