Parafoveal and foveal N400 effects in natural reading: A timeline of semantic processing from fixation-related potentials.

The depth at which parafoveal words are processed during reading is an ongoing topic of debate. Recent studies using RSVP-with-flanker paradigms have shown that implausible words within sentences elicit an N400 component while they are still in parafoveal vision, suggesting that the semantics of parafoveal words can be accessed to rapidly update the sentence representation. To study this effect in natural reading, we combined the coregistration of eye movements and EEG with the deconvolution modeling of fixation-related potentials (FRPs) to test whether semantic plausibility is processed parafoveally during Chinese sentence reading. For one target word per sentence, both its parafoveal and foveal plausibility were orthogonally manipulated using the boundary paradigm. Consistent with previous eye movement studies, we observed a delayed effect of parafoveal plausibility on fixation durations that only emerged on the foveal word. Crucially, in FRPs aligned to the pretarget fixation, a clear N400 effect emerged already based on parafoveal plausibility, with more negative voltages for implausible previews. Once participants fixated the target, we again observed an N400 effect of foveal plausibility. Interestingly, this foveal N400 was absent whenever the preview had been implausible, indicating that when a word's (im)plausibility is already processed in parafoveal vision, this information is not revised anymore upon direct fixation. Implausible words also elicited a late positive component (LPC), but exclusively when in foveal vision. Our results not only provide convergent neural and behavioral evidence for the parafoveal uptake of semantic information, but also indicate different contributions of parafoveal versus foveal information toward higher level sentence processing.

Post-Saccadic Face Processing Is Modulated by Pre-Saccadic Preview: Evidence from Fixation-Related Potentials.

Humans actively sample their environment with saccadic eye movements to bring relevant information into high-acuity foveal vision. Despite being lower in resolution, peripheral information is also available before each saccade. How the pre-saccadic extrafoveal preview of a visual object influences its post-saccadic processing is still an unanswered question. The current study investigated this question by simultaneously recording behavior and fixation-related brain potentials while human subjects made saccades to face stimuli. We manipulated the relationship between pre-saccadic "previews" and post-saccadic images to explicitly isolate the influences of the former. Subjects performed a gender discrimination task on a newly foveated face under three preview conditions: scrambled face, incongruent face (different identity from the foveated face), and congruent face (same identity). As expected, reaction times were faster after a congruent-face preview compared with a scrambled-face preview. Importantly, intact face previews (either incongruent or congruent) resulted in a massive reduction of post-saccadic neural responses. Specifically, we analyzed the classic face-selective N170 component at occipitotemporal electroencephalogram electrodes, which was still present in our experiments with active looking. However, the post-saccadic N170 was strongly attenuated following intact-face previews compared with the scrambled condition. This large and long-lasting decrease in evoked activity is consistent with a trans-saccadic mechanism of prediction that influences category-specific neural processing at the start of a new fixation. These findings constrain theories of visual stability and show that the extrafoveal preview methodology can be a useful tool to investigate its underlying mechanisms.SIGNIFICANCE STATEMENT Neural correlates of object recognition have traditionally been studied by flashing stimuli to the central visual field. This procedure differs in fundamental ways from natural vision, where viewers actively sample the environment with eye movements and also obtain a low-resolution preview of soon-to-be-fixated objects. Here we show that the N170, a classic electrophysiological marker of the structural encoding of faces, also occurs during a more natural viewing condition but is strongly reduced due to extrafoveal preprocessing (preview benefit). Our results therefore highlight the importance of peripheral vision during trans-saccadic processing in building a coherent and stable representation of the world around us.

Regression-based analysis of combined EEG and eye-tracking data: Theory and applications.

Fixation-related potentials (FRPs), neural responses aligned to the end of saccades, are a promising tool for studying the dynamics of attention and cognition under natural viewing conditions. In the past, four methodological problems have complicated the analysis of such combined eye-tracking/electroencephalogram experiments: (1) the synchronization of data streams, (2) the removal of ocular artifacts, (3) the condition-specific temporal overlap between the brain responses evoked by consecutive fixations, and (4) the fact that numerous low-level stimulus and saccade properties also influence the postsaccadic neural responses. Although effective solutions exist for the first two problems, the latter two are only beginning to be addressed. In the current paper, we present and review a unified regression-based framework for FRP analysis that allows us to deconvolve overlapping potentials while also controlling for both linear and nonlinear confounds on the FRP waveform. An open software implementation is provided for all procedures. We then demonstrate the advantages of this proposed (non)linear deconvolution modeling approach for data from three commonly studied paradigms: face perception, scene viewing, and reading. First, for a traditional event-related potential (ERP) face recognition experiment, we show how this technique can separate stimulus ERPs from overlapping muscle and brain potentials produced by small (micro)saccades on the face. Second, in natural scene viewing, we model and isolate multiple nonlinear effects of saccade parameters on the FRP. Finally, for a natural sentence reading experiment using the boundary paradigm, we show how it is possible to study the neural correlates of parafoveal preview after removing spurious overlap effects caused by the associated difference in average fixation time. Our results suggest a principal way of measuring reliable eye movement-related brain activity during natural vision.

Fixation-related Brain Potentials during Semantic Integration of Object-Scene Information.

In vision science, a particularly controversial topic is whether and how quickly the semantic information about objects is available outside foveal vision. Here, we aimed at contributing to this debate by coregistering eye movements and EEG while participants viewed photographs of indoor scenes that contained a semantically consistent or inconsistent target object. Linear deconvolution modeling was used to analyze the ERPs evoked by scene onset as well as the fixation-related potentials (FRPs) elicited by the fixation on the target object (t) and by the preceding fixation (t - 1). Object-scene consistency did not influence the probability of immediate target fixation or the ERP evoked by scene onset, which suggests that object-scene semantics was not accessed immediately. However, during the subsequent scene exploration, inconsistent objects were prioritized over consistent objects in extrafoveal vision (i.e., looked at earlier) and were more effortful to process in foveal vision (i.e., looked at longer). In FRPs, we demonstrate a fixation-related N300/N400 effect, whereby inconsistent objects elicit a larger frontocentral negativity than consistent objects. In line with the behavioral findings, this effect was already seen in FRPs aligned to the pretarget fixation t - 1 and persisted throughout fixation t, indicating that the extraction of object semantics can already begin in extrafoveal vision. Taken together, the results emphasize the usefulness of combined EEG/eye movement recordings for understanding the mechanisms of object-scene integration during natural viewing.

Optimizing the ICA-based removal of ocular EEG artifacts from free viewing experiments.

Combining EEG with eye-tracking is a promising approach to study neural correlates of natural vision, but the resulting recordings are also heavily contaminated by activity of the eye balls, eye lids, and extraocular muscles. While Independent Component Analysis (ICA) is commonly used to suppress these ocular artifacts, its performance under free viewing conditions has not been systematically evaluated and many published reports contain residual artifacts. Here I evaluated and optimized ICA-based correction for two tasks with unconstrained eye movements: visual search in images and sentence reading. In a first step, four parameters of the ICA pipeline were varied orthogonally: the (1) high-pass and (2) low-pass filter applied to the training data, (3) the proportion of training data containing myogenic saccadic spike potentials (SP), and (4) the threshold for eye tracker-based component rejection. In a second step, the eye-tracker was used to objectively quantify the correction quality of each ICA solution, both in terms of undercorrection (residual artifacts) and overcorrection (removal of neurogenic activity). As a benchmark, results were compared to those obtained with an alternative spatial filter, Multiple Source Eye Correction (MSEC). With commonly used settings, Infomax ICA not only left artifacts in the data, but also distorted neurogenic activity during eye movement-free intervals. However, correction results could be strongly improved by training the ICA on optimally filtered data in which SPs were massively overweighted. With optimized procedures, ICA removed virtually all artifacts, including the SP and its associated spectral broadband artifact from both viewing paradigms, with little distortion of neural activity. It also outperformed MSEC in terms of SP correction. Matlab code is provided.

The peripheral preview effect with faces: Combined EEG and eye-tracking suggests multiple stages of trans-saccadic predictive and non-predictive processing.

The world appears stable despite saccadic eye-movements. One possible explanation for this phenomenon is that the visual system predicts upcoming input across saccadic eye-movements based on peripheral preview of the saccadic target. We tested this idea using concurrent electroencephalography (EEG) and eye-tracking. Participants made cued saccades to peripheral upright or inverted face stimuli that changed orientation (invalid preview) or maintained orientation (valid preview) while the saccade was completed. Experiment 1 demonstrated better discrimination performance and a reduced fixation-locked N170 component (fN170) with valid than with invalid preview, demonstrating integration of pre- and post-saccadic information. Moreover, the early fixation-related potentials (FRP) showed a preview face inversion effect suggesting that some pre-saccadic input was represented in the brain until around 170 ms post fixation-onset. Experiment 2 replicated Experiment 1 and manipulated the proportion of valid and invalid trials to test whether the preview effect reflects context-based prediction across trials. A whole-scalp Bayes factor analysis showed that this manipulation did not alter the fN170 preview effect but did influence the face inversion effect before the saccade. The pre-saccadic inversion effect declined earlier in the mostly invalid block than in the mostly valid block, which is consistent with the notion of pre-saccadic expectations. In addition, in both studies, we found strong evidence for an interaction between the pre-saccadic preview stimulus and the post-saccadic target as early as 50 ms (Experiment 2) or 90 ms (Experiment 1) into the new fixation. These findings suggest that visual stability may involve three temporal stages: prediction about the saccadic target, integration of pre-saccadic and post-saccadic information at around 50-90 ms post fixation onset, and post-saccadic facilitation of rapid categorization.

Neural Correlates of Word Recognition: A Systematic Comparison of Natural Reading and Rapid Serial Visual Presentation.

Neural correlates of word recognition are commonly studied with (rapid) serial visual presentation (RSVP), a condition that eliminates three fundamental properties of natural reading: parafoveal preprocessing, saccade execution, and the fast changes in attentional processing load occurring from fixation to fixation. We combined eye-tracking and EEG to systematically investigate the impact of all three factors on brain-electric activity during reading. Participants read lists of words either actively with eye movements (eliciting fixation-related potentials) or maintained fixation while the text moved passively through foveal vision at a matched pace (RSVP-with-flankers paradigm, eliciting ERPs). The preview of the upcoming word was manipulated by changing the number of parafoveally visible letters. Processing load was varied by presenting words of varying lexical frequency. We found that all three factors have strong interactive effects on the brain's responses to words: Once a word was fixated, occipitotemporal N1 amplitude decreased monotonically with the amount of parafoveal information available during the preceding fixation; hence, the N1 component was markedly attenuated under reading conditions with preview. Importantly, this preview effect was substantially larger during active reading (with saccades) than during passive RSVP with flankers, suggesting that the execution of eye movements facilitates word recognition by increasing parafoveal preprocessing. Lastly, we found that the N1 component elicited by a word also reflects the lexical processing load imposed by the previously inspected word. Together, these results demonstrate that, under more natural conditions, words are recognized in a spatiotemporally distributed and interdependent manner across multiple eye fixations, a process that is mediated by active motor behavior.

Microsaccade-related brain potentials signal the focus of visuospatial attention.

Covert shifts of visuospatial attention are traditionally assumed to occur in the absence of oculomotor behavior. In contrast, recent behavioral studies have linked attentional cueing effects to the occurrence of microsaccades, small eye movements executed involuntarily during attempted fixation. Here we used a new type of electrophysiological marker to explore the attention-microsaccade relationship, the visual brain activity evoked by the microsaccade itself. By shifting the retinal image, microsaccades frequently elicit neural responses throughout the visual pathway, scalp-recordable in the human EEG as a microsaccade-related potential (mSRP). Although mSRPs contain similar signal components (P1/N1) as traditional visually-evoked potentials (VEPs), it is unknown whether they are also influenced by cognition. Based on established findings that VEPs are amplified for visual inputs at currently attended locations, we expected a selective gain-modulation also for mSRPs. Eye movements and EEG were coregistered in a classic spatial cueing task with an endogenous cue. Replicating behavioral findings, the direction of early microsaccades 200-400ms after cue onset was biased towards the cued side. However, for microsaccades throughout the cue-target interval, mSRPs were systematically enhanced at occipital scalp sites contralateral to the cued hemifield. This attention effect resembled that in a control condition with VEPs and did not interact with the direction of the underlying microsaccade, suggesting that mSRPs reflect the focus of sustained visuospatial attention, which remains fixed at the cued location, despite microsaccades. Microsaccades are not merely an artifact source in the EEG; instead, they are followed by cognitively modulated brain potentials that can serve as non-intrusive electrophysiological probes of attention.

(Micro)saccade-related potentials during face recognition: A study combining EEG, eye-tracking, and deconvolution modeling.

Under natural viewing conditions, complex stimuli such as human faces are typically looked at several times in succession, implying that their recognition may unfold across multiple eye fixations. Although electrophysiological (EEG) experiments on face recognition typically prohibit eye movements, participants still execute frequent (micro)saccades on the face, each of which generates its own visuocortical response. This finding raises the question of whether the fixation-related potentials (FRPs) evoked by these tiny gaze shifts also contain psychologically valuable information about face processing. Here, we investigated this question by corecording EEG and eye movements in an experiment with emotional faces (happy, angry, neutral). Deconvolution modeling was used to separate the stimulus ERPs to face onset from the FRPs generated by subsequent microsaccades-induced refixations on the face. As expected, stimulus ERPs exhibited typical emotion effects, with a larger early posterior negativity (EPN) for happy/angry compared with neutral faces. Eye tracking confirmed that participants made small saccades in 98% of the trials, which were often aimed at the left eye of the stimulus face. However, while each saccade produced a strong response over visual areas, this response was unaffected by the face's emotional expression, both for the first and for subsequent (micro)saccades. This finding suggests that the face's affective content is rapidly evaluated after stimulus onset, leading to only a short-lived sensory enhancement by arousing stimuli that does not repeat itself during immediate refixations. Methodologically, our work demonstrates how eye tracking and deconvolution modeling can be used to extract several brain responses from each EEG trial, providing insights into neural processing at different latencies after stimulus onset.

Neural mechanism underlying preview effects and masked priming effects in visual word processing.

Two classic experimental paradigms - masked repetition priming and the boundary paradigm - have played a pivotal role in understanding the process of visual word recognition. Traditionally, these paradigms have been employed by different communities of researchers, with their own long-standing research traditions. Nevertheless, a review of the literature suggests that the brain-electric correlates of word processing established with both paradigms may show interesting similarities, in particular with regard to the location, timing, and direction of N1 and N250 effects. However, as of yet, no direct comparison has been undertaken between the two paradigms. In the current study, we used combined eye-tracking/EEG to perform such a within-subject comparison using the same materials (single Chinese characters) as stimuli. To facilitate direct comparisons, we used a simplified version of the boundary paradigm - the single word boundary paradigm. Our results show the typical early repetition effects of N1 and N250 for both paradigms. However, repetition effects in N250 (i.e., a reduced negativity following identical-word primes/previews as compared to different-word primes/previews) were larger with the single word boundary paradigm than with masked priming. For N1 effects, repetition effects were similar across the two paradigms, showing a larger N1 after repetitions as compared to alternations. Therefore, the results indicate that at the neural level, a briefly presented and masked foveal prime produces qualitatively similar facilitatory effects on visual word recognition as a parafoveal preview before a single saccade, although such effects appear to be stronger in the latter case.

The impact of emotional facial expressions on reflexive attention depends on the aim of dynamic gaze changes: An ERP study.

The emotional expression and gaze direction of a face are important cues for human social interactions. However, the interplay of these factors and their neural correlates are only partially understood. In the current study, we investigated ERP correlates of gaze and emotion processing following the initial presentation of faces with different emotional expressions (happy, neutral, angry) and an averted or direct gaze direction as well as following a subsequent change in gaze direction that occurred in half of the trials. We focused on the time course and scalp topography of the N170 and EPN components. The N170 amplitude was larger to averted than direct gaze for the initial face presentation and larger to gaze changes from direct to averted than from averted to direct in response to the gaze change. For the EPN component in response to the initial face presentation, we replicate classic effects of emotion, which did not interact with gaze direction. As a major new finding, changes from direct to averted gaze elicited an EPN-like effect when the face showed a happy expression. No such effect was seen for angry expressions. We conclude that happy faces reflexively attract attention when they look at the observer rather than away from the observer. These results for happy expressions are in line with the shared signal hypothesis that posits a better processing of expressions if their approach or avoidance tendency is consistent with gaze direction. However, the shared signal hypothesis is not supported by the present results for angry faces.

Trans-saccadic parafoveal preview benefits in fluent reading: a study with fixation-related brain potentials.

During natural reading, a parafoveal preview of the upcoming word facilitates its subsequent recognition (e.g., shorter fixation durations compared to masked preview) but nothing is known about the neural correlates of this so-called preview benefit. Furthermore, while the evidence is strong that readers preprocess orthographic features of upcoming words, it is controversial whether word meaning can also be accessed parafoveally. We investigated the timing, scope, and electrophysiological correlates of parafoveal information use in reading by simultaneously recording eye movements and fixation-related brain potentials (FRPs) while participants read word lists fluently from left to right. For one word-the target-(e.g., "blade") parafoveal information was manipulated by showing an identical ("blade"), semantically related ("knife"), or unrelated ("sugar") word as preview. In boundary trials, the preview was shown parafoveally but changed to the correct target word during the incoming saccade. Replicating classic findings, target words were fixated shorter after identical previews. In the EEG, this benefit was reflected in an occipitotemporal preview positivity between 200 and 280 ms. In contrast, there was no facilitation from related previews. In parafoveal-on-foveal trials, preview and target were embedded at neighboring list positions without a display change. Consecutive fixation of two related words produced N400 priming effects, but only shortly (160 ms) after the second word was directly fixated. Results demonstrate that neural responses to words are substantially altered by parafoveal preprocessing under normal reading conditions. We found no evidence that word meaning contributes to these effects. Saccade-contingent display manipulations can be combined with EEG recordings to study extrafoveal perception in vision.

Eye movements and brain electric potentials during reading.

The development of theories and computational models of reading requires an understanding of processing constraints, in particular of timelines related to word recognition and oculomotor control. Timelines of word recognition are usually determined with event-related potentials (ERPs) recorded under conditions of serial visual presentation (SVP) of words; timelines of oculomotor control are derived from parameters of eye movements (EMs) during natural reading. We describe two strategies to integrate these approaches. One is to collect ERPs and EMs in separate SVP and natural reading experiments for the same experimental material (but different subjects). The other strategy is to co-register EMs and ERPs during natural reading from the same subjects. Both strategies yield data that allow us to determine how lexical properties influence ERPs (e.g., the N400 component) and EMs (e.g., fixation durations) across neighboring words. We review our recent research on the effects of frequency and predictability of words on both EM and ERP measures with reference to current models of eye-movement control during reading. Results are in support of the proposition that lexical access is distributed across several fixations and across brain-electric potentials measured on neighboring words.

Parafoveal words can modulate sentence meaning: Electrophysiological evidence from an RSVP-with-flanker task.

During natural reading, readers can take up some visual information from not-yet-fixated words to the right of the current fixation and it is well-established that this parafoveal preview facilitates the subsequent foveal processing of the word. However, the extraction and integration of word meaning from parafoveal words and their possible influence on the semantic content of the sentence are controversial. In the current study, we recorded event-related potentials (ERPs) in the RSVP-with-flankers paradigm to test whether and how updates of sentential meaning, based only on parafoveal information, may influence the subsequent foveal processing. In Chinese sentences, the congruency of parafoveal and foveal target words with the sentence was orthogonally manipulated. In contrast to previous research, we also controlled for potentially confounding effects of parafoveal-to-foveal repetition priming (identity preview effects) on the N400. Crucially, we found that the classic effect of foveal congruency on the N400 component only appeared when the word in preview had been congruent with sentence meaning; in contrast, there was no N400 as a function of foveal incongruency when the preview word had also been incongruent. These results indicate that sentence meaning rapidly adapts to parafoveal preview, altering the semantic context for the subsequently fixated word. We also show that correct parafoveal preview generally attenuates the N400 once a word is fixated, regardless of congruency. Taken together, our findings underline the highly generative and adaptive framework of language comprehension.

Is there magnocellular facilitation of early neural processes underlying visual word recognition? Evidence from masked repetition priming with ERPs.

An influential theory in the field of visual object recognition proposes that it is the fast magnocellular (M) system that facilitates neural processing of spatially more fine-grained information rather the slower parvocellular (P) system. While written words can be considered as a special type of visual objects, it is unknown whether magnocellular facilitation also plays a role in reading. We used a masked priming paradigm that has been shown to result in neural facilitation in visual word processing and tested whether these facilitating effects are mediated by the magnocellular system. In two experiments, we manipulated the influence of magnocellular and parvocellular systems on visual processing of a contextually predictable target character by contrasting high versus low spatial frequency and luminance versus color contrast, respectively. In addition, unchanged (normal) primes were included in both experiments as a manipulation check. As expected, unchanged primes elicited typical repetition effects in the N1, N250 and P3 components of the ERP in both experiments. In the experiment manipulating spatial contrast, we obtained repetition effects only for the N1 component for both M- and P-biased primes. In the luminance versus color contrast experiment, repetition effects were found in N1 and N250 for both M- and P- biased primes. Furthermore, no interactions were found between M-vs. P-biased prime types and repetition. Together these results indicate that M- and P- information contributes jointly to early neural processes underlying visual word recognition.

Human microsaccade-related visual brain responses.

Microsaccades are very small, involuntary flicks in eye position that occur on average once or twice per second during attempted visual fixation. Microsaccades give rise to EMG eye muscle spikes that can distort the spectrum of the scalp EEG and mimic increases in gamma band power. Here we demonstrate that microsaccades are also accompanied by genuine and sizeable cortical activity, manifested in the EEG. In three experiments, high-resolution eye movements were corecorded with the EEG: during sustained fixation of checkerboard and face stimuli and in a standard visual oddball task that required the counting of target stimuli. Results show that microsaccades as small as 0.15 degrees generate a field potential over occipital cortex and midcentral scalp sites 100-140 ms after movement onset, which resembles the visual lambda response evoked by larger voluntary saccades. This challenges the standard assumption of human brain imaging studies that saccade-related brain activity is precluded by fixation, even when fully complied with. Instead, additional cortical potentials from microsaccades were present in 86% of the oddball task trials and of similar amplitude as the visual response to stimulus onset. Furthermore, microsaccade probability varied systematically according to the proportion of target stimuli in the oddball task, causing modulations of late stimulus-locked event-related potential (ERP) components. Microsaccades present an unrecognized source of visual brain signal that is of interest for vision research and may have influenced the data of many ERP and neuroimaging studies.

Emotions in cognitive conflicts are not aversive but are task specific.

It has been suggested that cognitive conflicts require effortful processing and, therefore, are aversive (Botvinick, 2007). In the present study, we compared conflicts emerging from the inhibition of a predominant response tendency in a go/no-go task with those between incompatible response activations in a Simon task in a within-subjects design, using the same type of stimuli. Whereas no-go trials elicited reduced skin conductance and pupillometric responses, but prolonged corrugator muscle activity, as compared with go trials, incompatible and compatible Simon trials were indistinguishable with respect to these parameters. Furthermore, the conflict-sensitive N2 components of the event-related brain potential were similar in amplitude, but showed significantly different scalp distributions, indicating dissociable neural generator systems. The present findings suggest the involvement of different emotional and cognitive processes in both types of cognitive conflicts-none being aversive, however. In addition, the N2 findings call into question claims of common monitoring systems for all kinds of cognitive conflicts.

Unfold: an integrated toolbox for overlap correction, non-linear modeling, and regression-based EEG analysis.

Electrophysiological research with event-related brain potentials (ERPs) is increasingly moving from simple, strictly orthogonal stimulation paradigms towards more complex, quasi-experimental designs and naturalistic situations that involve fast, multisensory stimulation and complex motor behavior. As a result, electrophysiological responses from subsequent events often overlap with each other. In addition, the recorded neural activity is typically modulated by numerous covariates, which influence the measured responses in a linear or non-linear fashion. Examples of paradigms where systematic temporal overlap variations and low-level confounds between conditions cannot be avoided include combined electroencephalogram (EEG)/eye-tracking experiments during natural vision, fast multisensory stimulation experiments, and mobile brain/body imaging studies. However, even "traditional," highly controlled ERP datasets often contain a hidden mix of overlapping activity (e.g., from stimulus onsets, involuntary microsaccades, or button presses) and it is helpful or even necessary to disentangle these components for a correct interpretation of the results. In this paper, we introduce unfold, a powerful, yet easy-to-use MATLAB toolbox for regression-based EEG analyses that combines existing concepts of massive univariate modeling ("regression-ERPs"), linear deconvolution modeling, and non-linear modeling with the generalized additive model into one coherent and flexible analysis framework. The toolbox is modular, compatible with EEGLAB and can handle even large datasets efficiently. It also includes advanced options for regularization and the use of temporal basis functions (e.g., Fourier sets). We illustrate the advantages of this approach for simulated data as well as data from a standard face recognition experiment. In addition to traditional and non-conventional EEG/ERP designs, unfold can also be applied to other overlapping physiological signals, such as pupillary or electrodermal responses. It is available as open-source software at http://www.unfoldtoolbox.org.

Interaction of facial expressions and familiarity: ERP evidence.

There is mounting evidence that under some conditions the processing of facial identity and facial emotional expressions may not be independent; however, the nature of this interaction remains to be established. By using event-related brain potentials (ERP) we attempted to localize these interactions within the information processing system. During an expression discrimination task (Experiment 1) categorization was faster for portraits of personally familiar vs. unfamiliar persons displaying happiness. The peak latency of the P300 (trend) and the onset of the stimulus-locked LRP were shorter for familiar than unfamiliar faces. This implies a late perceptual but pre-motoric locus of the facilitating effect of familiarity on expression categorization. In Experiment 2 participants performed familiarity decisions about portraits expressing different emotions. Results revealed an advantage of happiness over disgust specifically for familiar faces. The facilitation was localized in the response selection stage as suggested by a shorter onset of the LRP. Both experiments indicate that familiarity and facial expression may not be independent processes. However, depending on the kind of decision different processing stages may be facilitated for happy familiar faces.

How microsaccades relate to lateralized ERP components of spatial attention: A co-registration study.

Covert shifts of attention that follow the presentation of a cue are associated with lateralized components in the event-related potential (ERP): the "early directing attention negativity" (EDAN) and the "anterior directing attention negativity" (ADAN). Traditionally, these shifts are thought to take place while gaze is fixated and, thus, in the absence of saccades. However, microsaccades of small amplitude (<1°) occur frequently and involuntarily also during fixation and are closely correlated with spatial attention. To investigate potential links between microsaccades and lateralized ERP components, we simultaneously recorded eye movements and ERPs in a spatial cueing task. As a first major result, we show that both the posterior EDAN and the orientation of microsaccades align more strongly with the location of the task-relevant part of the cue stimulus than with the direction of the attention shift indicated by that cue. A coupling between microsaccades and EDAN was also present on the single-trial level: The EDAN was largest when microsaccades were oriented toward the relevant cue, but absent when microsaccades were oriented away from it, suggesting that EDAN and microsaccades are generated by the same neural network, which selects relevant stimuli and orients behavior toward them. As a second major result, we show that small corneoretinal artifacts from microsaccades, which fall below conventional EOG rejection thresholds, contaminate the measurement of the ADAN. After correcting the EEG for microsaccade-related artifacts with an optimized variant of independent component analysis, ADAN was abolished at frontal sites, but a genuine ADAN was still present at central sites. Thus, the combined measurement of microsaccades and lateralized ERPs sheds new light onto cue-elicited shifts of covert attention.