Visual periodicity reveals distinct attentional signatures for face and non-face categories.

Observers can selectively deploy attention to regions of space, moments in time, specific visual features, individual objects, and even specific high-level categories-for example, when keeping an eye out for dogs while jogging. Here, we exploited visual periodicity to examine how category-based attention differentially modulates selective neural processing of face and non-face categories. We combined electroencephalography with a novel frequency-tagging paradigm capable of capturing selective neural responses for multiple visual categories contained within the same rapid image stream (faces/birds in Exp 1; houses/birds in Exp 2). We found that the pattern of attentional enhancement and suppression for face-selective processing is unique compared to other object categories: Where attending to non-face objects strongly enhances their selective neural signals during a later stage of processing (300-500 ms), attentional enhancement of face-selective processing is both earlier and comparatively more modest. Moreover, only the selective neural response for faces appears to be actively suppressed by attending towards an alternate visual category. These results underscore the special status that faces hold within the human visual system, and highlight the utility of visual periodicity as a powerful tool for indexing selective neural processing of multiple visual categories contained within the same image sequence.

Statistical learning of distractor co-occurrences facilitates visual search.

Visual search is facilitated by knowledge of the relationship between the target and the distractors, including both where the target is likely to be among the distractors and how it differs from the distractors. Whether the statistical structure among distractors themselves, unrelated to target properties, facilitates search is less well understood. Here, we assessed the benefit of distractor structure using novel shapes whose relationship to each other was learned implicitly during visual search. Participants searched for target items in arrays of shapes that comprised either four pairs of co-occurring distractor shapes (structured scenes) or eight distractor shapes randomly partitioned into four pairs on each trial (unstructured scenes). Across five online experiments (N = 1,140), we found that after a period of search training, participants were more efficient when searching for targets in structured than unstructured scenes. This structure benefit emerged independently of whether the position of the shapes within each pair was fixed or variable and despite participants having no explicit knowledge of the structured pairs they had seen. These results show that implicitly learned co-occurrence statistics between distractor shapes increases search efficiency. Increased efficiency in the rejection of regularly co-occurring distractors may contribute to the efficiency of visual search in natural scenes, where such regularities are abundant.

The Influence of Object-Color Knowledge on Emerging Object Representations in the Brain.

The ability to rapidly and accurately recognize complex objects is a crucial function of the human visual system. To recognize an object, we need to bind incoming visual features, such as color and form, together into cohesive neural representations and integrate these with our preexisting knowledge about the world. For some objects, typical color is a central feature for recognition; for example, a banana is typically yellow. Here, we applied multivariate pattern analysis on time-resolved neuroimaging (MEG) data to examine how object-color knowledge affects emerging object representations over time. Our results from 20 participants (11 female) show that the typicality of object-color combinations influences object representations, although not at the initial stages of object and color processing. We find evidence that color decoding peaks later for atypical object-color combinations compared with typical object-color combinations, illustrating the interplay between processing incoming object features and stored object knowledge. Together, these results provide new insights into the integration of incoming visual information with existing conceptual object knowledge.SIGNIFICANCE STATEMENT To recognize objects, we have to be able to bind object features, such as color and shape, into one coherent representation and compare it with stored object knowledge. The MEG data presented here provide novel insights about the integration of incoming visual information with our knowledge about the world. Using color as a model to understand the interaction between seeing and knowing, we show that there is a unique pattern of brain activity for congruently colored objects (e.g., a yellow banana) relative to incongruently colored objects (e.g., a red banana). This effect of object-color knowledge only occurs after single object features are processed, demonstrating that conceptual knowledge is accessed relatively late in the visual processing hierarchy.

Critical information thresholds underlying generic and familiar face categorisation at the same face encounter.

Seeing a face in the real world provokes a host of automatic categorisations related to sex, emotion, identity, and more. Such individual facets of human face recognition have been extensively examined using overt categorisation judgements, yet their relative informational dependencies during the same face encounter are comparatively unknown. Here we used EEG to assess how increasing access to sensory input governs two ecologically relevant brain functions elicited by seeing a face: Distinguishing faces and nonfaces, and recognising people we know. Observers viewed a large set of natural images that progressively increased in either image duration (experiment 1) or spatial frequency content (experiment 2). We show that in the absence of an explicit categorisation task, the human brain requires less sensory input to categorise a stimulus as a face than it does to recognise whether that face is familiar. Moreover, where sensory thresholds for distinguishing faces/nonfaces were remarkably consistent across observers, there was high inter-individual variability in the lower informational bound for familiar face recognition, underscoring the neurofunctional distinction between these categorisation functions. By i) indexing a form of face recognition that goes beyond simple low-level differences between categories, and ii) tapping multiple recognition functions elicited by the same face encounters, the information minima we report bear high relevance to real-world face encounters, where the same stimulus is categorised along multiple dimensions at once. Thus, our finding of lower informational requirements for generic vs. familiar face recognition constitutes some of the strongest evidence to date for the intuitive notion that sensory input demands should be lower for recognising face category than face identity.

Contextual and Spatial Associations Between Objects Interactively Modulate Visual Processing.

Much of what we know about object recognition arises from the study of isolated objects. In the real world, however, we commonly encounter groups of contextually associated objects (e.g., teacup and saucer), often in stereotypical spatial configurations (e.g., teacup above saucer). Here we used electroencephalography to test whether identity-based associations between objects (e.g., teacup-saucer vs. teacup-stapler) are encoded jointly with their typical relative positioning (e.g., teacup above saucer vs. below saucer). Observers viewed a 2.5-Hz image stream of contextually associated object pairs intermixed with nonassociated pairs as every fourth image. The differential response to nonassociated pairs (measurable at 0.625 Hz in 28/37 participants) served as an index of contextual integration, reflecting the association of object identities in each pair. Over right occipitotemporal sites, this signal was larger for typically positioned object streams, indicating that spatial configuration facilitated the extraction of the objects' contextual association. This high-level influence of spatial configuration on object identity integration arose ~ 320 ms post-stimulus onset, with lower-level perceptual grouping (shared with inverted displays) present at ~ 130 ms. These results demonstrate that contextual and spatial associations between objects interactively influence object processing. We interpret these findings as reflecting the high-level perceptual grouping of objects that frequently co-occur in highly stereotyped relative positions.

Ultra-coarse, single-glance human face detection in a dynamic visual stream.

Effective human interaction depends on our ability to rapidly detect faces in dynamic visual environments. Here we asked how basic units of visual information (spatial frequencies, or SF) contribute to this fundamental brain function. Human observers viewed initially blurry, unrecognizable natural object images presented at a fast 12 Hz rate and parametrically increasing in SF content over the course of 1 minute. By inserting highly variable natural face images as every 8th stimulus, we captured an objective neural index of face detection in participants' electroencephalogram (EEG) at exactly 1.5 Hz. This face-selective signal emerged over the right occipito-temporal cortex at <5 cycles/image, suggesting that the brain can - at a single glance - discriminate vastly different faces from multiple unsegmented object categories using only very coarse visual information. Local features (e.g., eyes) are not yet discernable at this threshold, indicating that fast face detection critically relies on global facial configuration. Interestingly, the face-selective neural response continued to increase with additional higher SF content until saturation around >50 cycles/image, potentially supporting higher-level recognition functions (e.g., facial identity recognition).

Fast periodic visual stimulation to study tool-selective processing in the human brain.

Because tools are manipulated for the purpose of action, they are often considered to be a specific object category that associates perceptual and motor properties. Their neural processing has been studied extensively by comparing the cortical activity elicited by the separate presentation of tool and non-tool objects, assuming that observed differences are solely due to activity selective for processing tools. Here, using a fast periodic visual stimulation (FPVS) paradigm, we isolated EEG activity selectively related to the processing of tool objects embedded in a stream of non-tool objects. Participants saw a continuous sequence of tool and non-tool images at a 3.7 Hz presentation rate, arranged as a repeating pattern of four non-tool images followed by one tool image. We expected the stimulation to generate an EEG response at the frequency of image presentation (3.7 Hz) and its harmonics, reflecting activity common to the processing of tool and non-tool images. Most importantly, if tool and non-tool images evoked different neural responses, we expected this differential activity to generate an additional response at the frequency of tool images (3.7 Hz/5 = 0.74 Hz). To ensure that this response was not due to unaccounted for systematic differences in low-level visual features, we also tested a phase-scrambled version of the sequence. The periodic insertion of tool stimuli within a stream of non-tool stimuli elicited a significant EEG response at the tool-selective frequency and its harmonics. This response was reduced when the images were phase-scrambled. We conclude that FPVS is a promising technique to selectively measure tool-related activity.

Face-sex categorization is better above fixation than below: Evidence from the reach-to-touch paradigm.

The masked congruence effect (MCE) elicited by nonconsciously presented faces in a sex-categorization task has recently been shown to be sensitive to the effects of attention. Here we investigated how spatial location along the vertical meridian modulates the MCE for face-sex categorization. Participants made left and right reaching movements to classify the sex of a target face that appeared either immediately above or below central fixation. The target was preceded by a masked prime face that was either congruent (i.e., same sex) or incongruent (i.e., opposite sex) with the target. In the reach-to-touch paradigm, participants typically classify targets more efficiently (i.e., their finger heads in the correct direction earlier and faster) on congruent than on incongruent trials. We observed an upper-hemifield advantage in the time course of this MCE, such that primes affected target classification sooner when they were presented in the upper visual field (UVF) rather than the lower visual field (LVF). Moreover, we observed a differential benefit of attention between the vertical hemifields, in that the MCE was dependent on the appropriate allocation of spatial attention in the LVF, but not the UVF. Taken together, these behavioral findings suggest that the processing of faces qua faces (e.g., sex-categorization) is more robust in upper-hemifield locations.

Movement trajectories as a window into the dynamics of emerging neural representations.

The rapid transformation of sensory inputs into meaningful neural representations is critical to adaptive human behaviour. While non-invasive neuroimaging methods are the de-facto method for investigating neural representations, they remain expensive, not widely available, time-consuming, and restrictive. Here we show that movement trajectories can be used to measure emerging neural representations with fine temporal resolution. By combining online computer mouse-tracking and publicly available neuroimaging data via representational similarity analysis (RSA), we show that movement trajectories track the unfolding of stimulus- and category-wise neural representations along key dimensions of the human visual system. We demonstrate that time-resolved representational structures derived from movement trajectories overlap with those derived from M/EEG (albeit delayed) and those derived from fMRI in functionally-relevant brain areas. Our findings highlight the richness of movement trajectories and the power of the RSA framework to reveal and compare their information content, opening new avenues to better understand human perception.

Visual Representations: Insights from Neural Decoding.

Patterns of brain activity contain meaningful information about the perceived world. Recent decades have welcomed a new era in neural analyses, with computational techniques from machine learning applied to neural data to decode information represented in the brain. In this article, we review how decoding approaches have advanced our understanding of visual representations and discuss efforts to characterize both the complexity and the behavioral relevance of these representations. We outline the current consensus regarding the spatiotemporal structure of visual representations and review recent findings that suggest that visual representations are at once robust to perturbations, yet sensitive to different mental states. Beyond representations of the physical world, recent decoding work has shone a light on how the brain instantiates internally generated states, for example, during imagery and prediction. Going forward, decoding has remarkable potential to assess the functional relevance of visual representations for human behavior, reveal how representations change across development and during aging, and uncover their presentation in various mental disorders.

Visual mismatch responses index surprise signalling but not expectation suppression.

The ability to distinguish between commonplace and unusual sensory events is critical for efficient learning and adaptive behaviour. This has been investigated using oddball designs in which sequences of often-appearing (i.e., expected) stimuli are interspersed with rare (i.e., surprising) deviants. Resulting differences in electrophysiological responses following surprising compared to expected stimuli are known as visual mismatch responses (VMRs). VMRs are thought to index co-occurring contributions of stimulus repetition effects, expectation suppression (that occurs when one's expectations are fulfilled), and expectation violation (i.e., surprise) responses; however, these different effects have been conflated in existing oddball designs. To better isolate and quantify effects of expectation suppression and surprise, we adapted an oddball design based on Fast Periodic Visual Stimulation (FPVS) that controls for stimulus repetition effects. We recorded electroencephalography (EEG) while participants (N = 48) viewed stimulation sequences in which a single face identity was periodically presented at 6 Hz. Critically, one of two different face identities (termed oddballs) appeared as every 7th image throughout the sequence. The presentation probabilities of each oddball image within a sequence varied between 10 and 90%, such that participants could form expectations about which oddball face identity was more likely to appear within each sequence. We also included 'expectation neutral' 50% probability sequences, whereby consistently biased expectations would not be formed for either oddball face identity. We found that VMRs indexed surprise responses, and effects of expectation suppression were absent. That is, ERPs were more negative-going at occipitoparietal electrodes for surprising compared to neutral oddballs, but did not differ between expected and neutral oddballs. Surprising oddball-evoked ERPs were also highly similar across the 10-40% appearance probability conditions. Our findings indicate that VMRs which are not accounted for by repetition effects are best described as an all-or-none surprise response, rather than a minimisation of prediction error responses associated with expectation suppression.

Object Vision in a Structured World.

In natural vision, objects appear at typical locations, both with respect to visual space (e.g., an airplane in the upper part of a scene) and other objects (e.g., a lamp above a table). Recent studies have shown that object vision is strongly adapted to such positional regularities. In this review we synthesize these developments, highlighting that adaptations to positional regularities facilitate object detection and recognition, and sharpen the representations of objects in visual cortex. These effects are pervasive across various types of high-level content. We posit that adaptations to real-world structure collectively support optimal usage of limited cortical processing resources. Taking positional regularities into account will thus be essential for understanding efficient object vision in the real world.

Immediate stimulus repetition abolishes stimulus expectation and surprise effects in fast periodic visual oddball designs.

Oddball designs are widely used to investigate the sensitivity of the visual system to statistical regularities in sensory environments. However, the underlying mechanisms that give rise to visual mismatch responses remain unknown. Much research has focused on identifying separable, additive effects of stimulus repetition and stimulus appearance probability (expectation/surprise) but findings from non-oddball designs indicate that these effects also interact. We adapted the fast periodic visual stimulation (FPVS) unfamiliar face identity oddball design (Liu-Shuang et al., 2014) to test for both additive and interactive effects of stimulus repetition and stimulus expectation. In two experiments, a given face identity was presented at a 6 Hz periodic rate; a different identity face (the oddball) appeared as every 7th image in the sequence (i.e., at 0.857 Hz). Electroencephalographic (EEG) activity was recorded during these stimulation sequences. In Experiment 1, we tested for surprise responses evoked by unexpected face image repetitions by replacing 10% of the commonly-presented oddball faces with exact repetitions of the base rate face identity image. In Experiment 2, immediately repeated or unrepeated face identity oddballs were presented in high and low presentation probability contexts (i.e., expected or surprising contexts), allowing assessment of expectation effects on responses to both repeated and unrepeated stimuli. Across both experiments objective (i.e., frequency-locked) visual mismatch responses driven by stimulus expectation were only found for oddball faces of a different identity to base rate faces (i.e., unrepeated identity oddballs). Our results show that immediate stimulus repetition (i.e., repetition suppression) can reduce or abolish expectation effects as indexed by EEG responses in visual oddball designs.

Category-selective human brain processes elicited in fast periodic visual stimulation streams are immune to temporal predictability.

Recording direct neural activity when periodically inserting exemplars of a particular category in a rapid visual stream of other objects offers an objective and efficient way to quantify perceptual categorization and characterize its spatiotemporal dynamics. However, since periodicity entails predictability, perceptual categorization processes identified within this framework may be partly generated or modulated by temporal expectations. Here we present a stringent test of the hypothesis that temporal predictability generates or modulates category-selective neural processes as measured in a rapid periodic visual stimulation stream. In Experiment 1, we compare neurophysiological responses to periodic and nonperiodic (i.e., unpredictable) variable face stimuli in a fast (12Hz) visual stream of nonface objects. In Experiment 2, we assess potential responses to rare (10%) omissions of periodic face events (i.e., violations of periodicity) in the same fast visual stream. Overall, our observations indicate that category(face)-selective processes elicited in a fast periodic stream of visual objects are immune to temporal predictability. These observations do not support a predictive coding framework interpretation of category-change detection in the human brain and have important implications for understanding automatic human perceptual categorization in a rapidly changing (i.e., dynamic) visual scene.

The upper-hemifield advantage for masked face processing: Not just an attentional bias.

Recent evidence suggests that face processing may be more robust in the upper visual field (UVF) than in the lower visual field (LVF). We asked whether this UVF advantage is due to an upward bias in participants' visuospatial attention. Participants classified the sex of a UVF or LVF target face that was preceded by a congruent or incongruent masked prime face. We manipulated spatial attention within subjects by varying the predictability of target location across sessions (UVF:LVF ratio of 50:50 on Day 1 and 20:80 on Day 2). When target location was unpredictable, priming emerged earlier in the UVF (~165 ms) than the LVF (~195 ms). This UVF advantage was reversed when targets were more likely to be presented in the LVF. Here priming arose earlier for LVF targets (~53 ms) than UVF targets (~165 ms). Critically, however, UVF primes were processed to the same degree regardless of whether spatial attention was diffuse (Day 1) or deployed elsewhere (Day 2). We conclude that, while voluntarily directed spatial attention is sufficient to modulate the processing of masked faces in the LVF, it is not sufficient to explain the UVF advantage for masked face processing.

Gaining the upper hand: evidence of vertical asymmetry in sex-categorisation of human hands.

Visual perception is characterised by asymmetries arising from the brain's preferential response to particular stimulus types at different retinal locations. Where the lower visual field (LVF) holds an advantage over the upper visual field (UVF) for many tasks (e.g., hue discrimination, contrast sensitivity, motion processing), face-perception appears best supported at above-fixation locations (Quek & Finkbeiner, 2014a). This finding is consistent with Previc's (1990) suggestion that vision in the UVF has become specialised for object recognition processes often required in "extrapersonal" space. Outside of faces, however, there have been very few investigations of vertical asymmetry effects for higher-level objects. Our aim in the present study was, thus, to determine whether the UVF advantage reported for face-perception would extend to a nonface object - human hands. Participants classified the sex of hand images presented above or below central fixation by reaching out to touch a left or right response panel. On each trial, a briefly presented spatial cue captured the participant's spatial attention to either the location where the hand was about to appear (valid cue) or the opposite location (invalid cue). We observed that cue validity only modulated the efficiency of the sex-categorisation response for targets in the LVLVF and not the UVF, just as we have reported previously for face-sex categorisation (Quek & Finkbeiner, 2014a). Taken together, the data from these studies provide some empirical support for Previc's (1990) speculation that object recognition processes may enjoy an advantage in the upper-hemifield.

Spatial and temporal attention modulate the early stages of face processing: behavioural evidence from a reaching paradigm.

A presently unresolved question within the face perception literature is whether attending to the location of a face modulates face processing (i.e. spatial attention). Opinions on this matter diverge along methodological lines - where neuroimaging studies have observed that the allocation of spatial attention serves to enhance the neural response to a face, findings from behavioural paradigms suggest face processing is carried out independently of spatial attention. In the present study, we reconcile this divide by using a continuous behavioural response measure that indexes face processing at a temporal resolution not available in discrete behavioural measures (e.g. button press). Using reaching trajectories as our response measure, we observed that although participants were able to process faces both when attended and unattended (as others have found), face processing was not impervious to attentional modulation. Attending to the face conferred clear benefits on sex-classification processes at less than 350ms of stimulus processing time. These findings constitute the first reliable demonstration of the modulatory effects of both spatial and temporal attention on face processing within a behavioural paradigm.