Flexible time course of spatial frequency use during scene categorization.

Human observers can quickly and accurately categorize scenes. This remarkable ability is related to the usage of information at different spatial frequencies (SFs) following a coarse-to-fine pattern: Low SFs, conveying coarse layout information, are thought to be used earlier than high SFs, representing more fine-grained information. Alternatives to this pattern have rarely been considered. Here, we probed all possible SF usage strategies randomly with high resolution in both the SF and time dimensions at two categorization levels. We show that correct basic-level categorizations of indoor scenes are linked to the sampling of relatively high SFs, whereas correct outdoor scene categorizations are predicted by an early use of high SFs and a later use of low SFs (fine-to-coarse pattern of SF usage). Superordinate-level categorizations (indoor vs. outdoor scenes) rely on lower SFs early on, followed by a shift to higher SFs and a subsequent shift back to lower SFs in late stages. In summary, our results show no consistent pattern of SF usage across tasks and only partially replicate the diagnostic SFs found in previous studies. We therefore propose that SF sampling strategies of observers differ with varying stimulus and task characteristics, thus favouring the notion of flexible SF usage.

The influence of scene and object orientation on the scene consistency effect.

Contextual regularities help us make sense of our visual environment. In scenes, semantically consistent objects are typically better recognized than inconsistent ones (e.g., a toaster vs. printer in a kitchen). What is the role of object and scene orientation in this so-called scene consistency effect? We presented consistent and inconsistent objects either upright (Experiment 1) or inverted (rotated 180°; Experiment 2) on upright, inverted, and scrambled background scenes. In Experiment 1, on upright scenes, consistent objects were recognized with higher accuracy than inconsistent ones, and we observed N300/N400 event-related potentials (ERPs) reflecting object-scene semantic processing. No such effects were observed for inverted or scrambled scenes. In Experiment 2, on both upright and inverted scenes, consistent objects were recognized with higher accuracy than inconsistent ones. Moreover, inconsistent objects on upright scenes triggered N300/N400 responses. Interestingly, no N300 but only an N400 deflection was found for inconsistent objects on inverted scenes. No effects were observed for scrambled scenes. These data suggest that while upright scenes modulate recognition irrespective of object orientation, inverted scenes only modulate the recognition of inverted objects. In ERPs, we found further evidence that inverted scenes can affect semantic object processing, with contextual influences occurring later in time, possibly driven by delayed or impaired scene gist processing. Mere object inversion does not seem to explain the later emergence of contextual influences. Taken together, the results suggest that the orientation of objects and scenes as well as their relationship to each other can influence ongoing object identification.

The role of scene summary statistics in object recognition.

Objects that are semantically related to the visual scene context are typically better recognized than unrelated objects. While context effects on object recognition are well studied, the question which particular visual information of an object's surroundings modulates its semantic processing is still unresolved. Typically, one would expect contextual influences to arise from high-level, semantic components of a scene but what if even low-level features could modulate object processing? Here, we generated seemingly meaningless textures of real-world scenes, which preserved similar summary statistics but discarded spatial layout information. In Experiment 1, participants categorized such textures better than colour controls that lacked higher-order scene statistics while original scenes resulted in the highest performance. In Experiment 2, participants recognized briefly presented consistent objects on scenes significantly better than inconsistent objects, whereas on textures, consistent objects were recognized only slightly more accurately. In Experiment 3, we recorded event-related potentials and observed a pronounced mid-central negativity in the N300/N400 time windows for inconsistent relative to consistent objects on scenes. Critically, inconsistent objects on textures also triggered N300/N400 effects with a comparable time course, though less pronounced. Our results suggest that a scene's low-level features contribute to the effective processing of objects in complex real-world environments.

Does Extensive Training at Individuating Novel Objects in Adulthood Lead to Visual Expertise? The Role of Facelikeness.

Human adults have a rich visual experience thanks to seeing human faces since birth, which may contribute to the acquisition of perceptual processes that rapidly and automatically individuate faces. According to a generic visual expertise hypothesis, extensive experience with nonface objects may similarly lead to efficient processing of objects at the individual level. However, whether extensive training in adulthood leads to visual expertise remains debated. One key issue is the extent to which the acquisition of visual expertise depends on the resemblance of objects to faces in terms of the spatial configuration of parts. We therefore trained naive human adults to individuate a large set of novel parametric multipart objects. Critically, one group of participants trained with the objects in a "facelike" stimulus orientation, whereas a second group trained with the same objects but with the objects rotated 180° in the picture plane into a "nonfacelike" orientation. We used a fast periodic visual stimulation EEG protocol to objectively quantify participants' ability to discriminate untrained exemplars before and after training. EEG responses associated with the frequency of identity change in a fast stimulation sequence, which reflects rapid and automatic perceptual processes, were observed over lateral occipital sites for both groups before training. There was a significant, albeit small, increase in these responses after training but only for the facelike group and only to facelike stimuli. Our findings indicate that perceived facelikeness plays a role in visual expertise and highlight how the adult perceptual system exploits familiar spatial configurations when learning new object categories.

The influence of natural contour and face size on the spatial frequency tuning for identifying upright and inverted faces.

It has previously been proposed that holistic face processing is based on low spatial frequencies (SFs) whereas featural processing relies on higher SFs, a hypothesis still widespread in the face processing literature today (e.g. Peters et al. in Eur J Neurosci 37(9):1448-1457, 2013). Since upright faces are supposedly recognized through holistic processing and inverted faces, using features, it is easy to take the leap to suggest a qualitatively different SF tuning for the identification of upright and vs. inverted faces. However, two independent studies (e.g. Gaspar et al. in Vision Res 48(28):2817-2826, 2008; Willenbockel et al. in J Exp Psychol Human 36(1):122-135, 2010a) found the same SF tuning for both stimulus presentations. Since these authors used relatively small faces hiding the natural facial contour, it is possible that differences in the SF tuning for identifying upright and inverted faces were missed. The present study thus revisits the SF tuning for upright and inverted faces face identification using the SF Bubbles technique. Our results still indicate that the same SFs are involved in both upright and inverted face recognition regardless of these additional parameters (contour and size), thus contrasting with previous data obtained using different methods (e.g. Oruc and Barton in J Vis 10(12):20, 1-12, 2010). The possible reasons subtending this divergence are discussed.

The informational correlates of conscious and nonconscious face-gender perception.

We used a face-gender repetition priming paradigm to precisely map the spatial frequencies (SFs) that influence observers' responses under different prime awareness conditions. A visible prime condition was set up by presenting the stimulus sequence mask-blank-prime-blank-mask-target and an invisible prime condition by switching the order of the masks and the blanks (see also Dehaene et al., 2001). The prime faces (~4.6° × 3.1°) were randomly filtered trial-by-trial according to the SF bubbles technique (Willenbockel, Fiset et al., 2010). Classification vectors, derived by summing the SF filters from each trial weighted by observers' transformed response times, revealed that SFs around 12 cycles per face width modulated responses in both prime awareness conditions. The significant SFs closely matched those optimal for accurate performance in a direct face-gender classification paradigm. Surprisingly, the significant SFs facilitated observers' responses in the visible prime condition, whereas they slowed responses in the invisible prime condition. Our findings suggest that SF tuning per se remains robust under different prime awareness conditions but that diagnostic visual cues might be utilized in a qualitatively different fashion as a function of awareness.

Spatial Frequency Tuning during the Conscious and Non-Conscious Perception of Emotional Facial Expressions - An Intracranial ERP Study.

Previous studies have shown that complex visual stimuli, such as emotional facial expressions, can influence brain activity independently of the observers' awareness. Little is known yet, however, about the "informational correlates" of consciousness - i.e., which low-level information correlates with brain activation during conscious vs. non-conscious perception. Here, we investigated this question in the spatial frequency (SF) domain. We examined which SFs in disgusted and fearful faces modulate activation in the insula and amygdala over time and as a function of awareness, using a combination of intracranial event-related potentials (ERPs), SF Bubbles (Willenbockel et al., 2010a), and Continuous Flash Suppression (CFS; Tsuchiya and Koch, 2005). Patients implanted with electrodes for epilepsy monitoring viewed face photographs (13° × 7°) that were randomly SF filtered on a trial-by-trial basis. In the conscious condition, the faces were visible; in the non-conscious condition, they were rendered invisible using CFS. The data were analyzed by performing multiple linear regressions on the SF filters from each trial and the transformed ERP amplitudes across time. The resulting classification images suggest that many SFs are involved in the conscious and non-conscious perception of emotional expressions, with SFs between 6 and 10 cycles per face width being particularly important early on. The results also revealed qualitative differences between the awareness conditions for both regions. Non-conscious processing relied on low SFs more and was faster than conscious processing. Overall, our findings are consistent with the idea that different pathways are employed for the processing of emotional stimuli under different degrees of awareness. The present study represents a first step to mapping how SF information "flows" through the emotion-processing network with a high temporal resolution and to shedding light on the informational correlates of consciousness in general.

Relative influences of lightness and facial morphology on perceived race.

In a recent study (Brooks and Gwinn, 2010 Perception 39 1142-1145), the lightness contrast illusion was employed to study the influences of skin tone and facial morphology on race perception. The findings were rather counterintuitive: they suggested that skin tone does not play a major role in racial categorisation. To investigate this further, we used a parametric paradigm including five lightness levels, five morphing levels, and two face orientations. In accordance with Brooks and Gwinn, we found that race categorisation of African-American and Caucasian faces by Caucasian participants relied mainly on morphological cues. However, the relative influence of lightness increased when morphological information was ambiguous and when the faces were upside down. Overall, the results point to a flexible multicue-based mechanism underlying race perception.

The neural correlates of memory encoding and recognition for own-race and other-race faces.

People are generally better at recognizing faces from their own race than from a different race, as has been shown in numerous behavioral studies. Here we use event-related potentials (ERPs) to investigate how differences between own-race and other-race faces influence the neural correlates of memory encoding and recognition. ERPs of Asian and Caucasian participants were recorded during the study and test phases of a Remember-Know paradigm with Chinese and Caucasian faces. A behavioral other-race effect was apparent in both groups, neither of which recognized other-race faces as well as own-race faces; however, Caucasian subjects showed stronger behavioral other-race effects. In the study phase, memory encoding was assessed with the ERP difference due to memory (Dm). Other-race effects in memory encoding were only found for Caucasian subjects. For subsequently "recollected" items, Caucasian subjects showed less positive mean amplitudes for own-race than other-race faces indicating that less neural activation was required for successful memory encoding of own-race faces. For the comparison of subsequently "recollected" and "familiar" items, Caucasian subjects showed similar brain activation only for own-race faces suggesting that subsequent familiarity and recollection of own-race faces arose from similar memory encoding processes. Experience with a race also influenced old/new effects, which are ERP correlates of recollection measured during recognition testing. Own-race faces elicited a typical parietal old/new effect, whereas old/new effects for other-race faces were prolonged and dominated by activity in frontal brain regions, suggesting a stronger involvement of post-retrieval monitoring processes. These results indicate that the other-race effect is a memory encoding- and recognition-based phenomenon.

Beyond correlation: do color features influence attention in rainforest?

Recent research indicates a direct relationship between low-level color features and visual attention under natural conditions. However, the design of these studies allows only correlational observations and no inference about mechanisms. Here we go a step further to examine the nature of the influence of color features on overt attention in an environment in which trichromatic color vision is advantageous. We recorded eye-movements of color-normal and deuteranope human participants freely viewing original and modified rainforest images. Eliminating red-green color information dramatically alters fixation behavior in color-normal participants. Changes in feature correlations and variability over subjects and conditions provide evidence for a causal effect of red-green color-contrast. The effects of blue-yellow contrast are much smaller. However, globally rotating hue in color space in these images reveals a mechanism analyzing color-contrast invariant of a specific axis in color space. Surprisingly, in deuteranope participants we find significantly elevated red-green contrast at fixation points, comparable to color-normal participants. Temporal analysis indicates that this is due to compensatory mechanisms acting on a slower time scale. Taken together, our results suggest that under natural conditions red-green color information contributes to overt attention at a low-level (bottom-up). Nevertheless, the results of the image modifications and deuteranope participants indicate that evaluation of color information is done in a hue-invariant fashion.

Controlling low-level image properties: the SHINE toolbox.

Visual perception can be influenced by top-down processes related to the observer's goals and expectations, as well as by bottom-up processes related to low-level stimulus attributes, such as luminance, contrast, and spatial frequency. When using different physical stimuli across psychological conditions, one faces the problem of disentangling the contributions of low- and high-level factors. Here, we make available the SHINE (spectrum, histogram, and intensity normalization and equalization) toolbox for MATLAB, which we have found useful for controlling a number of image properties separately or simultaneously. The toolbox features functions for specifying the (rotational average of the) Fourier amplitude spectra, for normalizing and scaling mean luminance and contrast, and for exact histogram specification optimized for perceptual visual quality. SHINE can thus be employed for parametrically modifying a number of image properties or for equating them across stimuli to minimize potential low-level confounds in studies on higher level processes.

Does face inversion change spatial frequency tuning?

The authors examined spatial frequency (SF) tuning of upright and inverted face identification using an SF variant of the Bubbles technique (F. Gosselin & P. G. Schyns, 2001). In Experiment 1, they validated the SF Bubbles technique in a plaid detection task. In Experiments 2a-c, the SFs used for identifying upright and inverted inner facial features were investigated. Although a clear inversion effect was present (mean accuracy was 24% higher and response times 455 ms shorter for upright faces), SF tunings were remarkably similar in both orientation conditions (mean r = .98; an SF band of 1.9 octaves centered at 9.8 cycles per face width for faces of about 6 degrees ). In Experiments 3a and b, the authors demonstrated that their technique is sensitive to both subtle bottom-up and top-down induced changes in SF tuning, suggesting that the null results of Experiments 2a-c are real. The most parsimonious explanation of the findings is provided by the quantitative account of the face inversion effect: The same information is used for identifying upright and inverted inner facial features, but processing has greater sensitivity with the former.

Sensitivity to spatial frequency and orientation content is not specific to face perception.

Prior work using a matching task between images that were complementary in spatial frequency and orientation information suggested that the representation of faces, but not objects, retains low-level spatial frequency (SF) information [Biederman, I., & Kalocsai, P. (1997). Neurocomputational bases of object and face recognition. Philosophical Transactions of the Royal Society of London, Series B Biological Sciences, 352, 1203-1219]. In two experiments, we reexamine the claim that face perception is uniquely sensitive to changes in SF. In contrast to prior work, we used a design allowing the computation of sensitivity and response criterion for each category, and in one experiment, equalized low-level image properties across object categories. In both experiments, we find that observers are sensitive to SF and orientation changes for upright and inverted faces and non-face objects. Differential response biases across categories contributed to a larger sensitivity for faces, but even sensitivity showed a larger effect for faces, especially when faces were upright and in a front-facing view. However, when objects were inverted, or upright but shown in a three-quarter view, the matching of objects and faces was equally sensitive to SF changes. Accordingly, face perception does not appear to be uniquely affected by changes in spatial filter components.