Assessing the effectiveness of spatial PCA on SVM-based decoding of EEG data.

Principal component analysis (PCA) has been widely employed for dimensionality reduction prior to multivariate pattern classification (decoding) in EEG research. The goal of the present study was to provide an evaluation of the effectiveness of PCA on decoding accuracy (using support vector machines) across a broad range of experimental paradigms. We evaluated several different PCA variations, including group-based and subject-based component decomposition and the application of Varimax rotation or no rotation. We also varied the numbers of PCs that were retained for the decoding analysis. We evaluated the resulting decoding accuracy for seven common event-related potential components (N170, mismatch negativity, N2pc, P3b, N400, lateralized readiness potential, and error-related negativity). We also examined more challenging decoding tasks, including decoding of face identity, facial expression, stimulus location, and stimulus orientation. The datasets also varied in the number and density of electrode sites. Our findings indicated that none of the PCA approaches consistently improved decoding performance related to no PCA, and the application of PCA frequently reduced decoding performance. Researchers should therefore be cautious about using PCA prior to decoding EEG data from similar experimental paradigms, populations, and recording setups.

Using multivariate pattern analysis to increase effect sizes for event-related potential analyses.

Multivariate pattern analysis (MVPA) approaches can be applied to the topographic distribution of event-related potential (ERP) signals to "decode" subtly different stimulus classes, such as different faces or different orientations. These approaches are extremely sensitive, and it seems possible that they could also be used to increase effect sizes and statistical power in traditional paradigms that ask whether an ERP component differs in amplitude across conditions. To assess this possibility, we leveraged the open-source ERP CORE data set and compared the effect sizes resulting from conventional univariate analyses of mean amplitude with two MVPA approaches (support vector machine decoding and the cross-validated Mahalanobis distance, both of which are easy to compute using open-source software). We assessed these approaches across seven widely studied ERP components (N170, N400, N2pc, P3b, lateral readiness potential, error related negativity, and mismatch negativity). Across all components, we found that multivariate approaches yielded effect sizes that were as large or larger than the effect sizes produced by univariate approaches. These results indicate that researchers could obtain larger effect sizes, and therefore greater statistical power, by using multivariate analysis of topographic voltage patterns instead of traditional univariate analyses in many ERP studies.

Using Multivariate Pattern Analysis to Increase Effect Sizes for Event-Related Potential Analyses.

Multivariate pattern analysis approaches can be applied to the topographic distribution of event-related potential (ERP) signals to 'decode' subtly different stimulus classes, such as different faces or different orientations. These approaches are extremely sensitive, and it seems possible that they could also be used to increase effect sizes and statistical power in traditional paradigms that ask whether an ERP component differs in amplitude across conditions. To assess this possibility, we leveraged the open-source ERP CORE dataset and compared the effect sizes resulting from conventional univariate analyses of mean amplitude with two multivariate pattern analysis approaches (support vector machine decoding and the cross-validated Mahalanobis distance, both of which are easy to compute using open-source software). We assessed these approaches across seven widely studied ERP components (N170, N400, N2pc, P3b, lateral readiness potential, error related negativity, and mismatch negativity). Across all components, we found that multivariate approaches yielded effect sizes that were as large or larger than the effect sizes produced by univariate approaches. These results indicate that researchers could obtain larger effect sizes, and therefore greater statistical power, by using multivariate analysis of topographic voltage patterns instead of traditional univariate analyses in many ERP studies.

Guidance of attention from visual working memory is feature-based, not object-based: Implications for models of feature binding.

A classic question in visual working memory (VWM) research is whether features from the same object are bound directly in an integrated representation or are maintained separately and bound only indirectly though shared location. Here, we examined this question using a novel method that probed the effects of VWM on the guidance of attention (rather than requiring explicit access to VWM content, as has typically been used). Participants remembered two color-shape conjunction objects. During a retention-interval search task, they searched for a target letter among distractor letters superimposed over color-shape conjunction items. There were two critical conditions. In the same-object-match condition, one search item matched both the color and shape of a single remembered object. In the different-object-match condition, one search item matched the color from one remembered object and the shape from the other. Robust effects of VWM-based guidance were observed, both when probing the incidental guidance of attention (Experiments 1 and 2) and the strategic guidance of attention (Experiment 3). Critically, in none of the experiments was the magnitude of guidance greater for same-object-match than for different-object-match. The results indicate that the representational units of guidance from VWM are individual features rather than integrated objects. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Categorical cuing: Object categories structure the acquisition of statistical regularities to guide visual search.

Recent statistical regularities have been demonstrated to influence visual search across a wide variety of learning mechanisms and search features. To function in the guidance of real-world search, however, such learning must be contingent on the context in which the search occurs and the object that is the target of search. The former has been studied extensively under the rubric of contextual cuing. Here, we examined, for the first time, categorical cuing: The role of object categories in structuring the acquisition of statistical regularities used to guide visual search. After an exposure session in which participants viewed six exemplars with the same general color in each of 40 different real-world categories, they completed a categorical search task, in which they searched for any member of a category based on a label cue. Targets that matched recent within-category regularities were found faster than targets that did not (Experiment 1). Such categorical cuing was also found to span multiple recent colors within a category (Experiment 2). It was observed to influence both the guidance of search to the target object (Experiment 3) and the basic operation of assigning single exemplars to categories (Experiment 4). Finally, the rapid acquisition of category-specific regularities was also quickly modified, with the benefit rapidly decreasing during the search session as participants were exposed equally to the two possible colors in each category. The results demonstrate that object categories organize the acquisition of perceptual regularities and that this learning exerts strong control over the instantiation of the category representation as a template for visual search. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Feature-based guidance of attention by visual working memory is applied independently of remembered object location.

Visual working memory (VWM) has been implicated both in the online representation of object tokens (in the object-file framework) and in the top-down guidance of attention during visual search, implementing a feature template. It is well established that object representations in VWM are structured by location, with access to the content of VWM modulated by position consistency. In the present study, we examined whether this property generalizes to the guidance of attention. Specifically, in two experiments, we probed whether the guidance of spatial attention from features in VWM is modulated by the position of the object from which these features were encoded. Participants remembered an object with an incidental color. Items in a subsequent search array could match either the color of the remembered object, the location, or both. Robust benefits of color match (when the matching item was the target) and costs (when the matching items was a distractor) were observed. Critically, the magnitude of neither effect was influenced by spatial correspondence. The results demonstrate that features in VWM influence attentional priority maps in a manner that does not necessarily inherit the spatial structure of the object representations in which those features are maintained.

The architecture of working memory: Features from multiple remembered objects produce parallel, coactive guidance of attention in visual search.

Theories of working memory (WM) differ in their claims about the number of items that can be maintained in a state that directly interacts with other, ongoing cognitive operations (termed the focus of attention). A similar debate has arisen in the literature on visual working memory (VWM), focused on the number of items that can simultaneously interact with attentional priority. In 3 experiments, we used a redundancy-gain paradigm to provide a comprehensive test of the latter question. Participants searched for 2 cued features (e.g., a color and a shape) within a search array. The cued feature values changed on a trial-by-trial basis, requiring VWM. The target (when present) could match 1 of the cued features (single-target trials) or both cued features (redundant-target trials). We tested whether response time distributions contained a substantial proportion of trials with redundant-target responses that were faster than predicted by 2 independent guidance processes operating in parallel (i.e., violations of the race-model inequality). Violations are consistent with a coactive architecture in which both cued values guide attention in parallel and sum on the priority map. Robust violations were observed in all cases predicted by the hypothesis that multiple items in VWM can guide attention simultaneously, and these results were inconsistent with the hypothesis that guidance is limited to a single item simultaneously. When considered in the larger context of the literature on VWM and attention, the present results are consistent with a model of WM architecture in which the focus of attention can maintain multiple, independent representations. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Contrasting episodic and template-based guidance during search through natural scenes.

Visual search through natural scenes can be guided by knowledge of where a target object has been observed previously (episodic guidance) and knowledge of that object's visual properties (template guidance). In the present experiments, we compared the relative contributions of these two sources of guidance. Episodic guidance was implemented in a contextual cuing task: participants searched multiple times through a set of scenes for a target letter that appeared in a consistent location within each scene. Template guidance was implemented by the color match between a critical distractor in each scene and a secondary visual working memory (VWM) load. There were four main findings. First, search time decreased with increasing scene repetition; episodic memory guided search. Second, the critical distractor was fixated more frequently on match compared with mismatch trials, consistent with automatic template guidance. Third, the VWM-match effect persisted in blocks with strong episodic guidance. Finally, VWM-match effects were observed from the first saccade during search, whereas episodic guidance to the target developed only later in the trial. The results support a view of natural search in which template-based mechanisms operate early during search in a manner that is not strongly constrained by scene-based forms of guidance, such as episodic knowledge. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Feature-based statistical regularities of distractors modulate attentional capture.

Ignoring salient distracting information is paramount to efficiently guiding attention during visual search. Learning to reject or suppress these strong sources of distraction leads to more effective visual search for targets. Participants can learn to overcome salient distractors if given reliable search regularities. If salient distractors appear in 1 location more frequently than any other, the visual system can use this environmental regularity to reduce attentional capture at the more frequent location (Wang & Theeuwes, 2018). We asked if reduced attentional capture is limited to location-based regularities, or, if the visual attentional system is configured to use feature-based regularities in reducing attentional capture as well. In 4 experiments examining attentional capture by task-irrelevant color singletons, participants searched for a shape singleton target among homogenously colored distractors. Critically, on a proportion of trials, a salient, color singleton distractor was presented. Color singleton distractors that appeared at a frequent location captured attention less than color singleton distractors that appeared at infrequent locations, replicating previous findings. In subsequent experiments we manipulated the frequency of the colors of the color singleton distractors and observed robust increases in capture based on color feature regularities. Despite strong location information, we observed reliable attentional capture attenuation by frequently presented distractor colors. Our results suggest that attentional capture is attenuated by both location and feature information. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Contrasting gist-based and template-based guidance during real-world visual search.

Visual search through real-world scenes is guided both by a representation of target features and by knowledge of the sematic properties of the scene (derived from scene gist recognition). In 3 experiments, we compared the relative roles of these 2 sources of guidance. Participants searched for a target object in the presence of a critical distractor object. The color of the critical distractor either matched or mismatched (a) the color of an item maintained in visual working memory for a secondary task (Experiment 1), or (b) the color of the target, cued by a picture before search commenced (Experiments 2 and 3). Capture of gaze by a matching distractor served as an index of template guidance. There were 4 main findings: (a) The distractor match effect was observed from the first saccade on the scene, (b) it was independent of the availability of scene-level gist-based guidance, (c) it was independent of whether the distractor appeared in a plausible location for the target, and (d) it was preserved even when gist-based guidance was available before scene onset. Moreover, gist-based, semantic guidance of gaze to target-plausible regions of the scene was delayed relative to template-based guidance. These results suggest that feature-based template guidance is not limited to plausible scene regions after an initial, scene-level analysis. (PsycINFO Database Record

The architecture of interaction between visual working memory and visual attention.

In five experiments, we examined whether a task-irrelevant item in visual working memory (VWM) interacts with perceptual selection when VWM must also be used to maintain a template representation of a search target. This question is critical to distinguishing between competing theories specifying the architecture of interaction between VWM and attention. The single-item template hypothesis (SIT) posits that only a single item in VWM can be maintained in a state that interacts with attention. Thus, the secondary item should be inert with respect to attentional guidance. The multiple-item template hypothesis (MIT) posits that multiple items can be maintained in a state that interacts with attention; thus, both the target representation and the secondary item should be capable of guiding selection. This question has been addressed previously in attention capture studies, but the results have been ambiguous. Here, we modified these earlier paradigms to optimize sensitivity to capture. Capture by a distractor matching the secondary item in VWM was observed consistently across multiple types of search task (abstract arrays and natural scenes), multiple dependent measures (search reaction time (RT) and oculomotor capture), multiple memory dimensions (color and shape), and multiple search stimulus dimensions (color, shape, common objects), providing strong support for the MIT. (PsycINFO Database Record

Human classifier: Observers can deduce task solely from eye movements.

Computer classifiers have been successful at classifying various tasks using eye movement statistics. However, the question of human classification of task from eye movements has rarely been studied. Across two experiments, we examined whether humans could classify task based solely on the eye movements of other individuals. In Experiment 1, human classifiers were shown one of three sets of eye movements: Fixations, which were displayed as blue circles, with larger circles meaning longer fixation durations; Scanpaths, which were displayed as yellow arrows; and Videos, in which a neon green dot moved around the screen. There was an additional Scene manipulation in which eye movement properties were displayed either on the original scene where the task (Search, Memory, or Rating) was performed or on a black background in which no scene information was available. Experiment 2 used similar methods but only displayed Fixations and Videos with the same Scene manipulation. The results of both experiments showed successful classification of Search. Interestingly, Search was best classified in the absence of the original scene, particularly in the Fixation condition. Memory also was classified above chance with the strongest classification occurring with Videos in the presence of the scene. Additional analyses on the pattern of correct responses in these two conditions demonstrated which eye movement properties successful classifiers were using. These findings demonstrate conditions under which humans can extract information from eye movement characteristics in addition to providing insight into the relative success/failure of previous computer classifiers.