Early Electrophysiological Markers of Navigational Affordances in Scenes.

Scene perception and spatial navigation are interdependent cognitive functions, and there is increasing evidence that cortical areas that process perceptual scene properties also carry information about the potential for navigation in the environment (navigational affordances). However, the temporal stages by which visual information is transformed into navigationally relevant information are not yet known. We hypothesized that navigational affordances are encoded during perceptual processing and therefore should modulate early visually evoked ERPs, especially the scene-selective P2 component. To test this idea, we recorded ERPs from participants while they passively viewed computer-generated room scenes matched in visual complexity. By simply changing the number of doors (0 doors, 1 door, 2 doors, 3 doors), we were able to systematically vary the number of pathways that afford movement in the local environment, while keeping the overall size and shape of the environment constant. We found that rooms with 0 doors evoked a higher P2 response than rooms with three doors, consistent with prior research reporting higher P2 amplitude to closed relative to open scenes. Moreover, we found P2 amplitude scaled linearly with the number of doors in the scenes. Navigability effects on the ERP waveform were also observed in a multivariate analysis, which showed significant decoding of the number of doors and their location at earlier time windows. Together, our results suggest that navigational affordances are represented in the early stages of scene perception. This complements research showing that the occipital place area automatically encodes the structure of navigable space and strengthens the link between scene perception and navigation.

The Case of the Cognitive (Opti)miser: Electrophysiological Correlates of Working Memory Maintenance Predict Demand Avoidance.

People are often considered cognitive misers. When given a free choice between two tasks, people tend to choose tasks requiring less cognitive effort. Such demand avoidance (DA) is associated with cognitive control, but it is still not clear to what extent individual differences in cognitive control can account for variations in DA. We sought to elucidate the relation between cognitive control and cognitive effort preferences by investigating the extent to which sustained neural activity in a task requiring cognitive control is correlated with DA. We hypothesized that neural measures of efficient filtering will predict individual variations in demand preferences. To test this hypothesis, we had participants perform a delayed-match-to-sample paradigm with their ERPs recorded, as well as a separate behavioral demand-selection task. We focused on the ERP correlates of cognitive filtering efficiency (CFE)-the ability to ignore task-irrelevant distractors during working memory maintenance-as it manifests in a modulation of the contralateral delay activity, an ERP correlate of cognitive control. As predicted, we found a significant positive correlation between CFE and DA. Individuals with high CFE tended to be significantly more demand avoidant than their low-CFE counterparts. Low-CFE individuals, in comparison, did not form distinct cognitive effort preferences. Overall, our results suggest that cognitive control over the contents of visual working memory contribute to individual differences in the expression of cognitive effort preferences. This further implies that these observed preferences are the product of sensitivity to cognitive task demands.

Children are sensitive to mutual information in intermediate-complexity face and non-face features.

Understanding developmental changes in children's use of specific visual information for recognizing object categories is essential for understanding how experience shapes recognition. Research on the development of face recognition has focused on children's use of low-level information (e.g. orientation sub-bands), or high-level information. In face categorization tasks, adults also exhibit sensitivity to intermediate complexity features that are diagnostic of the presence of a face. Do children also use intermediate complexity features for categorizing faces and objects, and, if so, how does their sensitivity to such features change during childhood? Intermediate-complexity features bridge the gap between low- and high-level processing: they have computational benefits for object detection and segmentation, and are known to drive neural responses in the ventral visual system. Here, we have investigated the developmental trajectory of children's sensitivity to diagnostic category information in intermediate-complexity features. We presented children (5-10 years old) and adults with image fragments of faces (Experiment 1) and cars (Experiment 2) varying in their mutual information, which quantifies a fragment's diagnosticity of a specific category. Our goal was to determine whether children were sensitive to the amount of mutual information in these fragments, and if their information usage is different from adults. We found that despite better overall categorization performance in adults, all children were sensitive to fragment diagnosticity in both categories, suggesting that intermediate representations of appearance are established early in childhood. Moreover, children's usage of mutual information was not limited to face fragments, suggesting the extracting intermediate-complexity features is a process that is not specific only to faces. We discuss the implications of our findings for developmental theories of face and object recognition.

Pre-stimulus brain state predicts auditory pattern identification accuracy.

Recent studies show that pre-stimulus band-specific power and phase in the electroencephalogram (EEG) can predict accuracy on tasks involving the detection of near-threshold stimuli. However, results in the auditory modality have been mixed, and few works have examined pre-stimulus features when more complex decisions are made (e.g. identifying supra-threshold sounds). Further, most auditory studies have used background sounds known to induce oscillatory EEG states, leaving it unclear whether phase predicts accuracy without such background sounds. To address this gap in knowledge, the present study examined pre-stimulus EEG as it relates to accuracy in a tone pattern identification task. On each trial, participants heard a triad of 40-ms sinusoidal tones (separated by 40-ms intervals), one of which was at a different frequency than the other two. Participants' task was to indicate the tone pattern (low-low-high, low-high-low, etc.). No background sounds were employed. Using a phase opposition measure based on inter-trial phase consistencies, pre-stimulus 7-10 Hz phase was found to differ between correct and incorrect trials ∼200 to 100 ms prior to tone-pattern onset. After sorting trials into bins based on phase, accuracy was found to be lowest at around π-+ relative to individuals' most accurate phase bin. No significant effects were found for pre-stimulus power. In the context of the literature, findings suggest an important relationship between the complexity of task demands and pre-stimulus activity within the auditory domain. Results also raise interesting questions about the role of induced oscillatory states or rhythmic processing modes in obtaining pre-stimulus effects of phase in auditory tasks.

Establishing reference scales for scene naturalness and openness : Naturalness and openness scales.

A key question in the field of scene perception is what information people use when making decisions about images of scenes. A significant body of evidence has indicated the importance of global properties of a scene image. Ideally, well-controlled, real-world images would be used to examine the influence of these properties on perception. Unfortunately, real-world images are generally complex and impractical to control. In the current research, we elicit ratings of naturalness and openness from a large number of subjects using Amazon Mechanic Turk. Subjects were asked to indicate which of a randomly chosen pair of scene images was more representative of a global property. A score and rank for each image was then estimated based on those comparisons using the Bradley-Terry-Luce model. These ranked images offer the opportunity to exercise control over the global scene properties in stimulus set drawn from complex real-world images. This will allow a deeper exploration of the relationship between global scene properties and behavioral and neural responses.

Task context impacts visual object processing differentially across the cortex.

Perception reflects an integration of "bottom-up" (sensory-driven) and "top-down" (internally generated) signals. Although models of visual processing often emphasize the central role of feed-forward hierarchical processing, less is known about the impact of top-down signals on complex visual representations. Here, we investigated whether and how the observer's goals modulate object processing across the cortex. We examined responses elicited by a diverse set of objects under six distinct tasks, focusing on either physical (e.g., color) or conceptual properties (e.g., man-made). Critically, the same stimuli were presented in all tasks, allowing us to investigate how task impacts the neural representations of identical visual input. We found that task has an extensive and differential impact on object processing across the cortex. First, we found task-dependent representations in the ventral temporal and prefrontal cortex. In particular, although object identity could be decoded from the multivoxel response within task, there was a significant reduction in decoding across tasks. In contrast, the early visual cortex evidenced equivalent decoding within and across tasks, indicating task-independent representations. Second, task information was pervasive and present from the earliest stages of object processing. However, although the responses of the ventral temporal, prefrontal, and parietal cortex enabled decoding of both the type of task (physical/conceptual) and the specific task (e.g., color), the early visual cortex was not sensitive to type of task and could only be used to decode individual physical tasks. Thus, object processing is highly influenced by the behavioral goal of the observer, highlighting how top-down signals constrain and inform the formation of visual representations.

Deconstructing visual scenes in cortex: gradients of object and spatial layout information.

Real-world visual scenes are complex cluttered, and heterogeneous stimuli engaging scene- and object-selective cortical regions including parahippocampal place area (PPA), retrosplenial complex (RSC), and lateral occipital complex (LOC). To understand the unique contribution of each region to distributed scene representations, we generated predictions based on a neuroanatomical framework adapted from monkey and tested them using minimal scenes in which we independently manipulated both spatial layout (open, closed, and gradient) and object content (furniture, e.g., bed, dresser). Commensurate with its strong connectivity with posterior parietal cortex, RSC evidenced strong spatial layout information but no object information, and its response was not even modulated by object presence. In contrast, LOC, which lies within the ventral visual pathway, contained strong object information but no background information. Finally, PPA, which is connected with both the dorsal and the ventral visual pathway, showed information about both objects and spatial backgrounds and was sensitive to the presence or absence of either. These results suggest that 1) LOC, PPA, and RSC have distinct representations, emphasizing different aspects of scenes, 2) the specific representations in each region are predictable from their patterns of connectivity, and 3) PPA combines both spatial layout and object information as predicted by connectivity.

Neuroanatomical correlates of visual car expertise.

Expertise in non-visual domains such as musical performance is associated with differences in gray matter volume of particular regions of the human brain. Whether this is also the case for expertise in visual object recognition is unknown. Here we tested whether individual variability in the ability to recognize car models, from novice performance to high level of expertise, is associated with specific structural changes in gray matter volume. We found that inter-individual variability in expertise with cars was significantly and selectively correlated with gray matter volume in prefrontal cortex. Inter-individual differences in the recognition of airplanes, that none of the participants had expertise with, were correlated with structural variability of regions bordering the visual cortex. These results highlight the role of prefrontal regions outside the visual cortex in accessing and processing visual knowledge about objects from the domain of expertise and suggest that expertise in visual object recognition may entail structural changes in regions associated with semantic knowledge.

Top-down engagement modulates the neural expressions of visual expertise.

Perceptual expertise is traditionally associated with enhanced brain activity in response to objects of expertise in category-selective visual cortex, primarily face-selective regions. We reevaluated this view by investigating whether the brain activity associated with expertise in object recognition is limited to category-selective cortex and specifically whether the extent of expertise-related activity manifests automatically or whether it can be top-down modulated. We conducted 2 functional magnetic resonance imaging studies comparing changes in hemodynamic activity associated with car expertise in a conventional 1-back task (Experiment 1) and when the task relevance of cars was explicitly manipulated (Experiment 2). Whole-brain analysis unveiled extensive expertise-related activity throughout the visual cortex, starting as early as V1 and extending into nonvisual areas. However, when the cars were task irrelevant, the expertise-related activity drastically diminished, indeed, becoming similar to the activity elicited by cars in novices. We suggest that expertise entails voluntary top-down engagement of multiple neural networks in addition to stimulus-driven activation associated with perceptual mechanisms.

Basic-level categorization of intermediate complexity fragments reveals top-down effects of expertise in visual perception.

Visual expertise is usually defined as the superior ability to distinguish between exemplars of a homogeneous category. Here, we ask how real-world expertise manifests at basic-level categorization and assess the contribution of stimulus-driven and top-down knowledge-based factors to this manifestation. Car experts and novices categorized computer-selected image fragments of cars, airplanes, and faces. Within each category, the fragments varied in their mutual information (MI), an objective quantifiable measure of feature diagnosticity. Categorization of face and airplane fragments was similar within and between groups, showing better performance with increasing MI levels. Novices categorized car fragments more slowly than face and airplane fragments, while experts categorized car fragments as fast as face and airplane fragments. The experts' advantage with car fragments was similar across MI levels, with similar functions relating RT with MI level for both groups. Accuracy was equal between groups for cars as well as faces and airplanes, but experts' response criteria were biased toward cars. These findings suggest that expertise does not entail only specific perceptual strategies. Rather, at the basic level, expertise manifests as a general processing advantage arguably involving application of top-down mechanisms, such as knowledge and attention, which helps experts to distinguish between object categories.

Artificially-generated scenes demonstrate the importance of global scene properties for scene perception.

Recent electrophysiological research highlights the significance of global scene properties (GSPs) for scene perception. However, since real-world scenes span a range of low-level stimulus properties and high-level contextual semantics, GSP effects may also reflect additional processing of such non-global factors. We examined this question by asking whether Event-Related Potentials (ERPs) to GSPs will still be observed when specific low- and high-level scene properties are absent from the scene. We presented participants with computer-based artificially-manipulated scenes varying in two GSPs (spatial expanse and naturalness) which minimized other sources of scene information (color and semantic object detail). We found that the peak amplitude of the P2 component was sensitive to the spatial expanse and naturalness of the artificially-generated scenes: P2 amplitude was higher to closed than open scenes, and in response to manmade than natural scenes. A control experiment showed that the effect of Naturalness on the P2 is not driven by local texture information, while earlier effects of naturalness, expressed as a modulation of the P1 and N1 amplitudes, are sensitive to texture information. Our results demonstrate that GSPs are processed robustly around 220 ms and that P2 can be used as an index of global scene perception.

Stimulus type, level of categorization, and spatial-frequencies utilization: implications for perceptual categorization hierarchies.

The type of visual information needed for categorizing faces and nonface objects was investigated by manipulating spatial frequency scales available in the image during a category verification task addressing basic and subordinate levels. Spatial filtering had opposite effects on faces and airplanes that were modulated by categorization level. The absence of low frequencies impaired the categorization of faces similarly at both levels, whereas the absence of high frequencies was inconsequential throughout. In contrast, basic-level categorization of airplanes was equally impaired by the absence of either low or high frequencies, whereas at the subordinate level, the absence of high frequencies had more deleterious effects. These data suggest that categorization of faces either at the basic level or by race is based primarily on their global shape but also on the configuration of details. By contrast, basic-level categorization of objects is based on their global shape, whereas category-specific diagnostic details determine the information needed for their subordinate categorization. The authors conclude that the entry point in visual recognition is flexible and determined conjointly by the stimulus category and the level of categorization, which reflects the observer's recognition goal.

Neural Sensitivity to Mutual Information in Intermediate-Complexity Face Features Changes during Childhood.

One way in which face recognition develops during infancy and childhood is with regard to the visual information that contributes most to recognition judgments. Adult face recognition depends on critical features spanning a hierarchy of complexity, including low-level, intermediate, and high-level visual information. To date, the development of adult-like information biases for face recognition has focused on low-level features, which are computationally well-defined but low in complexity, and high-level features, which are high in complexity, but not defined precisely. To complement this existing literature, we examined the development of children's neural responses to intermediate-level face features characterized using mutual information. Specifically, we examined children's and adults' sensitivity to varying levels of category diagnosticity at the P100 and N170 components. We found that during middle childhood, sensitivity to mutual information shifts from early components to later ones, which may indicate a critical restructuring of face recognition mechanisms that takes place over several years. This approach provides a useful bridge between the study of low- and high-level visual features for face recognition and suggests many intriguing questions for further investigation.

Investigating Neural Sensorimotor Mechanisms Underlying Flight Expertise in Pilots: Preliminary Data From an EEG Study.

Over the last decade, the efforts toward unraveling the complex interplay between the brain, body, and environment have set a promising line of research that utilizes neuroscience to study human performance in natural work contexts such as aviation. Thus, a relatively new discipline called neuroergonomics is holding the promise of studying the neural mechanisms underlying human performance in pursuit of both theoretical and practical insights. In this work, we utilized a neuroergonomic approach by combining insights from ecological psychology and embodied cognition to study flight expertise. Specifically, we focused on the Mirror Neuron system as a key correlate for understanding the interaction between an individual and the environment, suggesting that it can be used to index changes in the coupling of perception-action associated with skill development. In this study, we measured the EEG mu suppression as a proxy of the Mirror Neuron system in experts (pilots) and novices while performing a distance estimation task in a landing scenario. To survey the specificity of this measure, we considered central, parietal and occipital electrode pools and analyzed alpha (8-13 Hz) and beta (18-25 Hz) rhythm bands. We hypothesized that in experts vs. novices, specific neural sensorimotor brain activity would underpin the connection between perception and action in an in-flight context. Preliminary results indicate that alpha and beta rhythm suppression was area-specific irrespective of groups, present in the central electrodes placed over the motor areas. Group analysis revealed that specifically alpha mu rhythm, but not beta, was significantly more suppressed in pilots vs. novices. Complementing these findings we found a trend in which the strength of mu suppression increased with the sense of presence experienced by the pilots. Such sensorimotor activation is in line with the idea that for a pilot, a distance judgment is intimately associated with the function of landing. This reflects the ability to use optical invariants to see the world in terms of the capabilities of the aircraft (e.g., reachability and glide angle). These preliminary findings support the role of embodied simulation mechanisms in visual perception and add important insights into a practical understanding of flight expertise, suggesting sensorimotor mechanisms as potential neuro-markers.

The influence of behavioral relevance on the processing of global scene properties: An ERP study.

Recent work studying the temporal dynamics of visual scene processing (Harel et al., 2016) has found that global scene properties (GSPs) modulate the amplitude of early Event-Related Potentials (ERPs). It is still not clear, however, to what extent the processing of these GSPs is influenced by their behavioral relevance, determined by the goals of the observer. To address this question, we investigated how behavioral relevance, operationalized by the task context impacts the electrophysiological responses to GSPs. In a set of two experiments we recorded ERPs while participants viewed images of real-world scenes, varying along two GSPs, naturalness (manmade/natural) and spatial expanse (open/closed). In Experiment 1, very little attention to scene content was required as participants viewed the scenes while performing an orthogonal fixation-cross task. In Experiment 2 participants saw the same scenes but now had to actively categorize them, based either on their naturalness or spatial expense. We found that task context had very little impact on the early ERP responses to the naturalness and spatial expanse of the scenes: P1, N1, and P2 could distinguish between open and closed scenes and between manmade and natural scenes across both experiments. Further, the specific effects of naturalness and spatial expanse on the ERP components were largely unaffected by their relevance for the task. A task effect was found at the N1 and P2 level, but this effect was manifest across all scene dimensions, indicating a general effect rather than an interaction between task context and GSPs. Together, these findings suggest that the extraction of global scene information reflected in the early ERP components is rapid and very little influenced by top-down observer-based goals.

The representational dynamics of task and object processing in humans.

Despite the importance of an observer's goals in determining how a visual object is categorized, surprisingly little is known about how humans process the task context in which objects occur and how it may interact with the processing of objects. Using magnetoencephalography (MEG), functional magnetic resonance imaging (fMRI) and multivariate techniques, we studied the spatial and temporal dynamics of task and object processing. Our results reveal a sequence of separate but overlapping task-related processes spread across frontoparietal and occipitotemporal cortex. Task exhibited late effects on object processing by selectively enhancing task-relevant object features, with limited impact on the overall pattern of object representations. Combining MEG and fMRI data, we reveal a parallel rise in task-related signals throughout the cerebral cortex, with an increasing dominance of task over object representations from early to higher visual areas. Collectively, our results reveal the complex dynamics underlying task and object representations throughout human cortex.

Mutual information of image fragments predicts categorization in humans: electrophysiological and behavioral evidence.

Computational models suggest that features of intermediate complexity (IC) play a central role in object categorization [Ullman, S., Vidal-Naquet, M., & Sali, E. (2002). Visual features of intermediate complexity and their use in classification. Nature Neuroscience, 5, 682-687.]. The critical aspect of these features is the amount of mutual information (MI) they deliver. We examined the relation between MI, human categorization and an electrophysiological response to IC features. Categorization performance correlated with MI level as well as with the amplitude of a posterior temporal potential, peaking around 270 ms. Hence, an objective MI measure predicts human object categorization performance and its underlying neural activity. These results demonstrate that informative IC features serve as categorization features in human vision.

How Configural Is the Configural Superiority Effect? A Neuroimaging Investigation of Emergent Features in Visual Cortex.

The perception of a visual stimulus is dependent not only upon local features, but also on the arrangement of those features. When stimulus features are perceptually well organized (e.g., symmetric or parallel), a global configuration with a high degree of salience emerges from the interactions between these features, often referred to as emergent features. Emergent features can be demonstrated in the Configural Superiority Effect (CSE): presenting a stimulus within an organized context relative to its presentation in a disarranged one results in better performance. Prior neuroimaging work on the perception of emergent features regards the CSE as an "all or none" phenomenon, focusing on the contrast between configural and non-configural stimuli. However, it is still not clear how emergent features are processed between these two endpoints. The current study examined the extent to which behavioral and neuroimaging markers of emergent features are responsive to the degree of configurality in visual displays. Subjects were tasked with reporting the anomalous quadrant in a visual search task while being scanned. Degree of configurality was manipulated by incrementally varying the rotational angle of low-level features within the stimulus arrays. Behaviorally, we observed faster response times with increasing levels of configurality. These behavioral changes were accompanied by increases in response magnitude across multiple visual areas in occipito-temporal cortex, primarily early visual cortex and object-selective cortex. Our findings suggest that the neural correlates of emergent features can be observed even in response to stimuli that are not fully configural, and demonstrate that configural information is already present at early stages of the visual hierarchy.

What is special about expertise? Visual expertise reveals the interactive nature of real-world object recognition.

Ever since Diamond and Carey (1986. J. Exp. Psychol.: Gen., vol. 115, pp. 107-117) seminal work, the main model for studying expertise in visual object recognition ("visual expertise") has been face perception. The underlying assumption was that since faces may be considered the ultimate domain of visual expertise, any face-processing signature might actually be a general characteristic of visual expertise. However, while humans are clearly experts in face recognition, visual expertise is not restricted to faces and can be observed in a variety of domains. This raises the question of whether face recognition is in fact the right model to study visual expertise, and if not, what are the common cognitive and neural characteristics of visual expertise. The current perspective article addresses this question by revisiting past and recent neuroimaging and behavioural works on visual expertise. The view of visual expertise that emerges from these works is that expertise is a unique phenomenon, with distinctive neural and cognitive characteristics. Specifically, visual expertise is a controlled, interactive process that develops from the reciprocal interactions between the visual system and multiple top-down factors, including semantic knowledge, top-down attentional control, and task relevance. These interactions enable the ability to flexibly access domain-specific information at multiple scales and levels guided by multiple recognition goals. Extensive visual experience with a given object category culminates in the recruitment of these multiple systems, and is reflected in widespread neural activity, extending well beyond visual cortex, to include higher-level cortical areas.