Physicochemical features partially explain olfactory crossmodal correspondences.

During the olfactory perception process, our olfactory receptors are thought to recognize specific chemical features. These features may contribute towards explaining our crossmodal perception. The physicochemical features of odors can be extracted using an array of gas sensors, also known as an electronic nose. The present study investigates the role that the physicochemical features of olfactory stimuli play in explaining the nature and origin of olfactory crossmodal correspondences, which is a consistently overlooked aspect of prior work. Here, we answer the question of whether the physicochemical features of odors contribute towards explaining olfactory crossmodal correspondences and by how much. We found a similarity of 49% between the perceptual and the physicochemical spaces of our odors. All of our explored crossmodal correspondences namely, the angularity of shapes, smoothness of textures, perceived pleasantness, pitch, and colors have significant predictors for various physicochemical features, including aspects of intensity and odor quality. While it is generally recognized that olfactory perception is strongly shaped by context, experience, and learning, our findings show that a link, albeit small (6-23%), exists between olfactory crossmodal correspondences and their underlying physicochemical features.

Decoding of EEG signals reveals non-uniformities in the neural geometry of colour.

The idea of colour opponency maintains that colour vision arises through the comparison of two chromatic mechanisms, red versus green and yellow versus blue. The four unique hues, red, green, blue, and yellow, are assumed to appear at the null points of these the two chromatic systems. Here we hypothesise that, if unique hues represent a tractable cortical state, they should elicit more robust activity compared to other, non-unique hues. We use a spatiotemporal decoding approach to report that electroencephalographic (EEG) responses carry robust information about the tested isoluminant unique hues within a 100-350 ms window from stimulus onset. Decoding is possible in both passive and active viewing tasks, but is compromised when concurrent high luminance contrast is added to the colour signals. For large hue-differences, the efficiency of hue decoding can be predicted by mutual distance in a nominally uniform perceptual colour space. However, for small perceptual neighbourhoods around unique hues, the encoding space shows pivotal non-uniformities which suggest that anisotropies in neurometric hue-spaces may reflect perceptual unique hues.

Narrowing of the Audiovisual Temporal Binding Window Due To Perceptual Training Is Specific to High Visual Intensity Stimuli.

The temporal binding window (TBW), which reflects the range of temporal offsets in which audiovisual stimuli are combined to form a singular percept, can be reduced through training. Our research aimed to investigate whether training-induced reductions in TBW size transfer across stimulus intensities. A total of 32 observers performed simultaneity judgements at two visual intensities with a fixed auditory intensity, before and after receiving audiovisual TBW training at just one of these two intensities. We show that training individuals with a high visual intensity reduces the size of the TBW for bright stimuli, but this improvement did not transfer to dim stimuli. The reduction in TBW can be explained by shifts in decision criteria. Those trained with the dim visual stimuli, however, showed no reduction in TBW. Our main finding is that perceptual improvements following training are specific for high-intensity stimuli, potentially highlighting limitations of proposed TBW training procedures.

The effect of induced fusional demand on static and dynamic stereoacuity thresholds: the digital Synoptophore.

BACKGROUND/AIMS: The ability to extract depth from disparity may be hindered under fusional stress, as alignment of the eyes may be more difficult to maintain consistently. Therefore we aim to determine the effect of fusional demand on stereoacuity in individuals with no known binocular vision impairments. METHODS: A novel static and dynamic binocular depth detection task, capable of assessing many discrete levels of stereoacuity, was presented on digital displays attached to each tube of the Synoptophore. Stereoacuity was measured with any latent deviation fully corrected and compared to that measured at the 'recovery' angle. This recovery angle is where single vision is restored after decompensation to diplopia, during vergence range assessment. RESULTS: Seventy-two subjects (50 Female, 22 Male; mean (SD) age 22 (6) years) were assessed. The amount of fusional demand was between 1 and 26 prism dioptres (PD), with a mean (SD) of 8(6)PD. Under zero fusion demand the mean (SD) static and dynamic depth detection thresholds were 322(53)" and 69(23)". Under fusional stress these were 224(40)" and 77(21)". There was no significant difference between thresholds in stressed and zero demand fusion (p = 0.08). Dynamic depth detection thresholds were significantly lower than static (P < 0.01). CONCLUSION: Fusional stress does not appear to impact on stereoacuity. The numerical value of the recovery point varied amongst individuals, but this represents a common point, where single vision is easily restored and binocularity well established. Due to individual differences in the ability to control a certain amount of fusional stress (e.g. vergences stress of 10PD, when recovery is 8PD, will perturb binocularity more than a person with a recovery of 20PD), previous reports may not accurately represent the effect of fusional stress. Whilst our findings are contrary to previous reports, we did not stress fusion beyond the recovery point and used a more accurate/repeatable method to measure stereoacuity.

Neural mechanisms of divided feature-selective attention to colour.

Feature-based attentional selection of colour is challenging to investigate due to the multidimensional nature of colour-space. When attending concurrently to features from different feature dimensions (e.g. red and horizontal), the attentional selections of the separate dimensions are largely independent. Therefore, if colour constitutes multiple independent feature dimensions for attentional purposes, concurrently attending to two colours should be effective and independent of the specific configuration of target and distractor colours. Here, observers attended concurrently to two out of four fully overlapping random dot kinematograms of different colours, and the allocation of attention to each colour was assessed separately by recordings of steady-state visual evoked potentials. The magnitude of attention effects depended on colour proximity and was well described by a simple model which suggested that colour space is rescaled in an adaptive manner to achieve attentional selection. In conclusion, different spatially overlaid colours can be attended concurrently with an efficiency that is determined by their configuration in colour space, supporting the idea that (at least in terms of hue) colour acts as a single dimension for attentional purposes.

A Systematic Comparison of Static and Dynamic Cues for Depth Perception.

PURPOSE: A clinical diagnosis of stereoblindness does not necessarily preclude compelling depth perception. Qualitative observations suggest that this may be due to the dynamic nature of the stimuli. The purpose of this study was to systematically investigate the effectiveness of static and dynamic stereoscopic stimuli. METHODS: Stereoscopic stimuli were presented on a passive polarized stereoscopic monitor and were manipulated as follows: static disparity (baseline condition), dynamic disparity (change in z-location), change in stimulus pattern, change in z-location with pattern change, change in x-location (horizontal shift), a control (nil-disparity signal). All depth-detection thresholds were measured simultaneously using an adaptive four-alternative-forced-choice (4AFC) paradigm with all six conditions randomly interleaved. RESULTS: A total of 127 participants (85 women, 42 men; mean [SD] age, 21 [5] years) with visual acuity better than 0.22 logMAR in both eyes were assessed. In comparison to the static disparity condition, depth-detection thresholds were up to 50% lower for the dynamic disparity conditions, with and without pattern change (P < 0.001). The presence of a changing pattern in isolation (P = 0.71) or a horizontal shift (P = 0.41) did not affect the thresholds. CONCLUSIONS: Dynamic disparity information facilitates the extraction of depth in comparison to static disparity signals. This finding may account for the compelling perception of depth reported in individuals with no measurable static stereoacuity. Our findings challenge the traditional definition of stereoblindness and suggest that current diagnostic tests using static stimuli may be suboptimal. We argue that both static and dynamic stimuli should be employed to fully assess the binocular potential of patients when considering management options.

The effects of stereo disparity on the behavioural and electrophysiological correlates of perception of audio-visual motion in depth.

Motion is represented by low-level signals, such as size-expansion in vision or loudness changes in the auditory modality. The visual and auditory signals from the same object or event may be integrated and facilitate detection. We explored behavioural and electrophysiological correlates of congruent and incongruent audio-visual depth motion in conditions where auditory level changes, visual expansion, and visual disparity cues were manipulated. In Experiment 1 participants discriminated auditory motion direction whilst viewing looming or receding, 2D or 3D, visual stimuli. Responses were faster and more accurate for congruent than for incongruent audio-visual cues, and the congruency effect (i.e., difference between incongruent and congruent conditions) was larger for visual 3D cues compared to 2D cues. In Experiment 2, event-related potentials (ERPs) were collected during presentation of the 2D and 3D, looming and receding, audio-visual stimuli, while participants detected an infrequent deviant sound. Our main finding was that audio-visual congruity was affected by retinal disparity at an early processing stage (135-160ms) over occipito-parietal scalp. Topographic analyses suggested that similar brain networks were activated for the 2D and 3D congruity effects, but that cortical responses were stronger in the 3D condition. Differences between congruent and incongruent conditions were observed between 140-200ms, 220-280ms, and 350-500ms after stimulus onset.

The IAT shows no evidence for Kandinsky's color-shape associations.

In the early twentieth century, the Bauhaus revolutionized art and design by using simple colors and forms. Wassily Kandinsky was especially interested in the relationship of these two visual attributes and postulated a fundamental correspondence between color and form: yellow triangle, red square and blue circle. Subsequent empirical studies used preference judgments to test Kandinsky's original color-form combinations, usually yielding inconsistent results. We have set out to test the validity of these postulated associations by using the Implicit Association Test. Participants pressed one of two buttons on each trial. On some trials they classified shapes (e.g., circle or triangle). On interleaved trials they classified colors (e.g., blue or yellow). Response times should theoretically be faster when the button mapping follows Kandinsky's associations: For example, when the left key is used to report blue or circle and the right is used for yellow and triangle, than when the response mapping is the opposite of this (blue or triangle, yellow or circle). Our findings suggest that there is no implicit association between the original color-form combinations. Of the three combinations we tested, there was only a marginal effect in one case. It can be concluded that the IAT does not support Kandinsky's postulated color-form associations, and that these are probably not a universal property of the visual system.

The time course of auditory-visual processing of speech and body actions: evidence for the simultaneous activation of an extended neural network for semantic processing.

An extensive network of cortical areas is involved in multisensory object and action recognition. This network draws on inferior frontal, posterior temporal, and parietal areas; activity is modulated by familiarity and the semantic congruency of auditory and visual component signals even if semantic incongruences are created by combining visual and auditory signals representing very different signal categories, such as speech and whole body actions. Here we present results from a high-density ERP study designed to examine the time-course and source location of responses to semantically congruent and incongruent audiovisual speech and body actions to explore whether the network involved in action recognition consists of a hierarchy of sequentially activated processing modules or a network of simultaneously active processing sites. We report two main results:1) There are no significant early differences in the processing of congruent and incongruent audiovisual action sequences. The earliest difference between congruent and incongruent audiovisual stimuli occurs between 240 and 280 ms after stimulus onset in the left temporal region. Between 340 and 420 ms, semantic congruence modulates responses in central and right frontal areas. Late differences (after 460 ms) occur bilaterally in frontal areas.2) Source localisation (dipole modelling and LORETA) reveals that an extended network encompassing inferior frontal, temporal, parasaggital, and superior parietal sites are simultaneously active between 180 and 420 ms to process auditory­visual action sequences. Early activation (before 120 ms) can be explained by activity in mainly sensory cortices. . The simultaneous activation of an extended network between 180 and 420 ms is consistent with models that posit parallel processing of complex action sequences in frontal, temporal and parietal areas rather than models that postulate hierarchical processing in a sequence of brain regions.

Low-level and high-level modulations of fixational saccades and high frequency oscillatory brain activity in a visual object classification task.

Until recently induced gamma-band activity (GBA) was considered a neural marker of cortical object representation. However, induced GBA in the electroencephalogram (EEG) is susceptible to artifacts caused by miniature fixational saccades. Recent studies have demonstrated that fixational saccades also reflect high-level representational processes. Do high-level as opposed to low-level factors influence fixational saccades? What is the effect of these factors on artifact-free GBA? To investigate this, we conducted separate eye tracking and EEG experiments using identical designs. Participants classified line drawings as objects or non-objects. To introduce low-level differences, contours were defined along different directions in cardinal color space: S-cone-isolating, intermediate isoluminant, or a full-color stimulus, the latter containing an additional achromatic component. Prior to the classification task, object discrimination thresholds were measured and stimuli were scaled to matching suprathreshold levels for each participant. In both experiments, behavioral performance was best for full-color stimuli and worst for S-cone isolating stimuli. Saccade rates 200-700 ms after stimulus onset were modulated independently by low and high-level factors, being higher for full-color stimuli than for S-cone isolating stimuli and higher for objects. Low-amplitude evoked GBA and total GBA were observed in very few conditions, showing that paradigms with isoluminant stimuli may not be ideal for eliciting such responses. We conclude that cortical loops involved in the processing of objects are preferentially excited by stimuli that contain achromatic information. Their activation can lead to relatively early exploratory eye movements even for foveally-presented stimuli.

Evidence for auditory-visual processing specific to biological motion.

Biological motion is usually associated with highly correlated sensory signals from more than one modality: an approaching human walker will not only have a visual representation, namely an increase in the retinal size of the walker's image, but also a synchronous auditory signal since the walker's footsteps will grow louder. We investigated whether the multisensorial processing of biological motion is subject to different constraints than ecologically invalid motion. Observers were presented with a visual point-light walker and/or synchronised auditory footsteps; the walker was either approaching the observer (looming motion) or walking away (receding motion). A scrambled point-light walker served as a control. Observers were asked to detect the walker's motion as quickly and as accurately as possible. In Experiment 1 we tested whether the reaction time advantage due to redundant information in the auditory and visual modality is specific for biological motion. We found no evidence for such an effect: the reaction time reduction was accounted for by statistical facilitation for both biological and scrambled motion. In Experiment 2, we dissociated the auditory and visual information and tested whether inconsistent motion directions across the auditory and visual modality yield longer reaction times in comparison to consistent motion directions. Here we find an effect specific to biological motion: motion incongruency leads to longer reaction times only when the visual walker is intact and recognisable as a human figure. If the figure of the walker is abolished by scrambling, motion incongruency has no effect on the speed of the observers' judgments. In conjunction with Experiment 1 this suggests that conflicting auditory-visual motion information of an intact human walker leads to interference and thereby delaying the response.

Premotor cortex is sensitive to auditory-visual congruence for biological motion.

The auditory and visual perception systems have developed special processing strategies for ecologically valid motion stimuli, utilizing some of the statistical properties of the real world. A well-known example is the perception of biological motion, for example, the perception of a human walker. The aim of the current study was to identify the cortical network involved in the integration of auditory and visual biological motion signals. We first determined the cortical regions of auditory and visual coactivation (Experiment 1); a conjunction analysis based on unimodal brain activations identified four regions: middle temporal area, inferior parietal lobule, ventral premotor cortex, and cerebellum. The brain activations arising from bimodal motion stimuli (Experiment 2) were then analyzed within these regions of coactivation. Auditory footsteps were presented concurrently with either an intact visual point-light walker (biological motion) or a scrambled point-light walker; auditory and visual motion in depth (walking direction) could either be congruent or incongruent. Our main finding is that motion incongruency (across modalities) increases the activity in the ventral premotor cortex, but only if the visual point-light walker is intact. Our results extend our current knowledge by providing new evidence consistent with the idea that the premotor area assimilates information across the auditory and visual modalities by comparing the incoming sensory input with an internal representation.

Event-related potentials reveal an early advantage for luminance contours in the processing of objects.

Detection and identification of objects are the most crucial goals of visual perception. We studied the role of luminance and chromatic information for object processing by comparing performance of familiar, meaningful object contours with those of novel, non-object contours. Comparisons were made between full-color and reduced-color object (or non-object) contours. Full-color stimuli contained both chromatic and luminance information, whereas luminance information was absent in the reduced-color stimuli. All stimuli were made equally salient by fixing them at multiples of discrimination threshold contrast. In a subsequent electroencephalographic experiment observers were asked to classify contours as objects or non-objects. An advantage in accuracy was found for full-color stimuli over the reduced-color stimuli but only if the contours depicted objects as opposed to non-objects. Event-related potentials revealed the neural correlate of this object-specific luminance advantage. The amplitude of the centro-occipital N1 component was modulated by stimulus class with the effect being driven by the presence of luminance information. We conclude that high-level discrimination processes in the cortex start relatively early and exhibit object-selective effects only in the presence of luminance information. This is consistent with the superiority of luminance in subserving object identification processes.

The integration of local chromatic motion signals is sensitive to contrast polarity.

Global motion integration mechanisms can utilize signals defined by purely chromatic information. Is global motion integration sensitive to the polarity of such color signals? To answer this question, we employed isoluminant random dot kinematograms (RDKs) that contain a single chromatic contrast polarity or two different polarities. Single-polarity RDKs consisted of local motion signals with either a positive or a negative S or L-M component, while in the different-polarity RDKs, half the dots had a positive S or L-M component, and the other half had a negative S or L-M component. In all RDKs, the polarity and the motion direction of the local signals were uncorrelated. Observers discriminated between 50% coherent motion and random motion, and contrast thresholds were obtained for 81% correct responses. Contrast thresholds were obtained for three different dot densities (50, 100, and 200 dots). We report two main findings: (1) dependence on dot density is similar for both contrast polarities (+S vs. -S, +LM vs. -LM) but slightly steeper for S in comparison to LM and (2) thresholds for different-polarity RDKs are significantly higher than for single-polarity RDKs, which is inconsistent with a polarity-blind integration mechanism. We conclude that early motion integration mechanisms are sensitive to the polarity of the local motion signals and do not automatically integrate information across different polarities.

Reaction time facilitation for horizontally moving auditory-visual stimuli.

For moving targets, bimodal facilitation of reaction time has been observed for motion in the depth plane (C. Cappe, G. Thut, B. Romei, & M. M. Murray, 2009), but it is unclear whether analogous RT facilitation is observed for auditory-visual motion stimuli in the horizontal plane, as perception of horizontal motion relies on very different cues. Here we found that bimodal motion cues resulted in significant RT facilitation at threshold level, which could not be explained using an independent decisions model (race model). Bimodal facilitation was observed at suprathreshold levels when the RTs for suprathreshold unimodal stimuli were roughly equated, and significant RT gains were observed for direction-discrimination tasks with abrupt-onset motion stimuli and with motion preceded by a stationary phase. We found no speeded responses for bimodal signals when a motion signal in one modality was paired with a spatially co-localized stationary signal in the other modality, but faster response times could be explained by statistical facilitation when the motion signals traveled in opposite directions. These results strongly suggest that integration of motion cues led to the speeded bimodal responses. Finally, our results highlight the importance of matching the unimodal reaction times to obtain response facilitation for bimodal motion signals in the linear plane.

S-cone signals invisible to the motion system can improve motion extraction via grouping by color.

The purpose of this study was to test whether color-motion correlations carried by a pure color difference (S-cone component only) can be used to improve global motion extraction. We also examined the neural markers of color-motion correlation processing in event-related potentials. Color and motion information was dissociated using a two-colored random dot kinematogram, wherein coherent motion and motion noise differed from each other only in their S-cone component, with spatial and temporal parameters set so that global motion processing relied solely on a constant L-M component. Hence, when color and the local motion direction are correlated, more efficient segregation of coherent motion can only be brought about by the S-cone difference, and crucially, this S-cone component does not provide any effective input to a global motion mechanism but only changes the color appearance of the moving dots. The color contrasts (vector length in the S vs. L-M plane) of both the dots carrying coherent motion and the dots moving randomly were fixed at motion discrimination threshold to ensure equal effectiveness for motion extraction. In the behavioral experiment, participants were asked to discriminate between coherent and random motion, and d' was determined for three different conditions: uncorrelated, uncued correlated, and cued correlated. In the electroencephalographic experiment, participants discriminated direction of motion for uncued correlated and cued correlated conditions. Color-motion correlations were found to improve performance. Cueing a specific color also modulated the N1 component of the event-related potential, with sources in visual area middle temporal. We conclude that S-cone signals "invisible" to the motion system can influence the analysis by direction-selective motion mechanisms through grouping of local motion signals by color. This grouping mechanism must precede motion processing and is likely to be under attentional control.

Multivoxel fMRI analysis of color tuning in human primary visual cortex.

We use multivoxel pattern analysis (MVPA) to study the spatial clustering of color-selective neurons in the human brain. Our main objective was to investigate whether MVPA reveals the spatial arrangements of color-selective neurons in human primary visual cortex (V1). We measured the distributed fMRI activation patterns for different color stimuli (Experiment 1: cardinal colors (to which the LGN is known to be tuned), Experiment 2: perceptual hues) in V1. Our two main findings were that (i) cone-opponent cardinal color modulations produce highly reproducible patterns of activity in V1, but these were not unique to each color. This suggests that V1 neurons with tuning characteristics similar to those found in LGN are not spatially clustered. (ii) Unique activation patterns for perceptual hues in V1 support current evidence for a spatially clustered hue map. We believe that our work is the first to show evidence of spatial clustering of neurons with similar color preferences in human V1.

Integration of ordinal and metric cues in depth processing.

J. Burge, M. A. Peterson, and S. E. Palmer (2005) reported that ordinal, configural cues of familiarity and convexity influence perceived depth even when unambiguous metric information in the form of binocular disparity is available. In their study, a shape that was both convex and familiar (i.e., a face) increased perceived depth in random dot stereograms if the shape was shown in the foreground and decreased perceived depth if it was shown in the background. It is generally assumed that luminance cues are necessary for pre-figural shape representation to influence figure-ground computations in this way (M. A. Peterson & B. S. Gibson, 1993); thus, Burge et al. (2005) had used a luminance edge. In this research, we asked whether configural cues need to be defined by luminance, contrast, or neither. For a sufficiently large disparity pedestal (about 2.5 arcmin), configural cues influenced perceived depth both for second-order contours and for contours defined only by disparity. The integration of ordinal and metric cues seems to be driven by the general saliency of the contours and not only by luminance information. This challenges the notion that the integration of such cues always needs to arise during figure-ground organization through early combinations of luminance-defined shape and binocular disparity.

When S-cones contribute to chromatic global motion processing.

There is common consensus now that color-defined motion can be perceived by the human visual system. For global motion integration tasks based on isoluminant random dot kinematograms conflicting evidence exists, whether observers can (Ruppertsberg et al., 2003) or cannot (Bilodeau & Faubert, 1999) extract a common motion direction for stimuli modulated along the isoluminant red-green axis. Here we report conditions, in which S-cones contribute to chromatic global motion processing. When the display included extra-foveal regions, the individual elements were large ( approximately 0.3 degrees ) and the displacement was large ( approximately 1 degrees ), stimuli modulated along the yellowish-violet axis proved to be effective in a global motion task. The color contrast thresholds for detection for both color axes were well below the contrasts required for global motion integration, and therefore the discrimination-to-detection ratio was >1. We conclude that there is significant S-cone input to chromatic global motion processing and the extraction of global motion is not mediated by the same mechanism as simple detection. Whether the koniocellular or the magnocellular pathway is involved in transmitting S-cone signals is a topic of current debate (Chatterjee & Callaway, 2002).

Low-level integration of auditory and visual motion signals requires spatial co-localisation.

It is well known that the detection thresholds for stationary auditory and visual signals are lower if the signals are presented bimodally rather than unimodally, provided the signals coincide in time and space. Recent work on auditory-visual motion detection suggests that the facilitation seen for stationary signals is not seen for motion signals. We investigate the conditions under which motion perception also benefits from the integration of auditory and visual signals. We show that the integration of cross-modal local motion signals that are matched in position and speed is consistent with thresholds predicted by a neural summation model. If the signals are presented in different hemi-fields, move in different directions, or both, then behavioural thresholds are predicted by a probability-summation model. We conclude that cross-modal signals have to be co-localised and co-incident for effective motion integration. We also argue that facilitation is only seen if the signals contain all localisation cues that would be produced by physical objects.