Parafoveal and foveal N400 effects in natural reading: A timeline of semantic processing from fixation-related potentials.

The depth at which parafoveal words are processed during reading is an ongoing topic of debate. Recent studies using RSVP-with-flanker paradigms have shown that implausible words within sentences elicit an N400 component while they are still in parafoveal vision, suggesting that the semantics of parafoveal words can be accessed to rapidly update the sentence representation. To study this effect in natural reading, we combined the coregistration of eye movements and EEG with the deconvolution modeling of fixation-related potentials (FRPs) to test whether semantic plausibility is processed parafoveally during Chinese sentence reading. For one target word per sentence, both its parafoveal and foveal plausibility were orthogonally manipulated using the boundary paradigm. Consistent with previous eye movement studies, we observed a delayed effect of parafoveal plausibility on fixation durations that only emerged on the foveal word. Crucially, in FRPs aligned to the pretarget fixation, a clear N400 effect emerged already based on parafoveal plausibility, with more negative voltages for implausible previews. Once participants fixated the target, we again observed an N400 effect of foveal plausibility. Interestingly, this foveal N400 was absent whenever the preview had been implausible, indicating that when a word's (im)plausibility is already processed in parafoveal vision, this information is not revised anymore upon direct fixation. Implausible words also elicited a late positive component (LPC), but exclusively when in foveal vision. Our results not only provide convergent neural and behavioral evidence for the parafoveal uptake of semantic information, but also indicate different contributions of parafoveal versus foveal information toward higher level sentence processing.

(Micro)saccade-related potentials during face recognition: A study combining EEG, eye-tracking, and deconvolution modeling.

Under natural viewing conditions, complex stimuli such as human faces are typically looked at several times in succession, implying that their recognition may unfold across multiple eye fixations. Although electrophysiological (EEG) experiments on face recognition typically prohibit eye movements, participants still execute frequent (micro)saccades on the face, each of which generates its own visuocortical response. This finding raises the question of whether the fixation-related potentials (FRPs) evoked by these tiny gaze shifts also contain psychologically valuable information about face processing. Here, we investigated this question by corecording EEG and eye movements in an experiment with emotional faces (happy, angry, neutral). Deconvolution modeling was used to separate the stimulus ERPs to face onset from the FRPs generated by subsequent microsaccades-induced refixations on the face. As expected, stimulus ERPs exhibited typical emotion effects, with a larger early posterior negativity (EPN) for happy/angry compared with neutral faces. Eye tracking confirmed that participants made small saccades in 98% of the trials, which were often aimed at the left eye of the stimulus face. However, while each saccade produced a strong response over visual areas, this response was unaffected by the face's emotional expression, both for the first and for subsequent (micro)saccades. This finding suggests that the face's affective content is rapidly evaluated after stimulus onset, leading to only a short-lived sensory enhancement by arousing stimuli that does not repeat itself during immediate refixations. Methodologically, our work demonstrates how eye tracking and deconvolution modeling can be used to extract several brain responses from each EEG trial, providing insights into neural processing at different latencies after stimulus onset.

The impact of emotional facial expressions on reflexive attention depends on the aim of dynamic gaze changes: An ERP study.

The emotional expression and gaze direction of a face are important cues for human social interactions. However, the interplay of these factors and their neural correlates are only partially understood. In the current study, we investigated ERP correlates of gaze and emotion processing following the initial presentation of faces with different emotional expressions (happy, neutral, angry) and an averted or direct gaze direction as well as following a subsequent change in gaze direction that occurred in half of the trials. We focused on the time course and scalp topography of the N170 and EPN components. The N170 amplitude was larger to averted than direct gaze for the initial face presentation and larger to gaze changes from direct to averted than from averted to direct in response to the gaze change. For the EPN component in response to the initial face presentation, we replicate classic effects of emotion, which did not interact with gaze direction. As a major new finding, changes from direct to averted gaze elicited an EPN-like effect when the face showed a happy expression. No such effect was seen for angry expressions. We conclude that happy faces reflexively attract attention when they look at the observer rather than away from the observer. These results for happy expressions are in line with the shared signal hypothesis that posits a better processing of expressions if their approach or avoidance tendency is consistent with gaze direction. However, the shared signal hypothesis is not supported by the present results for angry faces.

Parafoveal words can modulate sentence meaning: Electrophysiological evidence from an RSVP-with-flanker task.

During natural reading, readers can take up some visual information from not-yet-fixated words to the right of the current fixation and it is well-established that this parafoveal preview facilitates the subsequent foveal processing of the word. However, the extraction and integration of word meaning from parafoveal words and their possible influence on the semantic content of the sentence are controversial. In the current study, we recorded event-related potentials (ERPs) in the RSVP-with-flankers paradigm to test whether and how updates of sentential meaning, based only on parafoveal information, may influence the subsequent foveal processing. In Chinese sentences, the congruency of parafoveal and foveal target words with the sentence was orthogonally manipulated. In contrast to previous research, we also controlled for potentially confounding effects of parafoveal-to-foveal repetition priming (identity preview effects) on the N400. Crucially, we found that the classic effect of foveal congruency on the N400 component only appeared when the word in preview had been congruent with sentence meaning; in contrast, there was no N400 as a function of foveal incongruency when the preview word had also been incongruent. These results indicate that sentence meaning rapidly adapts to parafoveal preview, altering the semantic context for the subsequently fixated word. We also show that correct parafoveal preview generally attenuates the N400 once a word is fixated, regardless of congruency. Taken together, our findings underline the highly generative and adaptive framework of language comprehension.

Is there magnocellular facilitation of early neural processes underlying visual word recognition? Evidence from masked repetition priming with ERPs.

An influential theory in the field of visual object recognition proposes that it is the fast magnocellular (M) system that facilitates neural processing of spatially more fine-grained information rather the slower parvocellular (P) system. While written words can be considered as a special type of visual objects, it is unknown whether magnocellular facilitation also plays a role in reading. We used a masked priming paradigm that has been shown to result in neural facilitation in visual word processing and tested whether these facilitating effects are mediated by the magnocellular system. In two experiments, we manipulated the influence of magnocellular and parvocellular systems on visual processing of a contextually predictable target character by contrasting high versus low spatial frequency and luminance versus color contrast, respectively. In addition, unchanged (normal) primes were included in both experiments as a manipulation check. As expected, unchanged primes elicited typical repetition effects in the N1, N250 and P3 components of the ERP in both experiments. In the experiment manipulating spatial contrast, we obtained repetition effects only for the N1 component for both M- and P-biased primes. In the luminance versus color contrast experiment, repetition effects were found in N1 and N250 for both M- and P- biased primes. Furthermore, no interactions were found between M-vs. P-biased prime types and repetition. Together these results indicate that M- and P- information contributes jointly to early neural processes underlying visual word recognition.

Regression-based analysis of combined EEG and eye-tracking data: Theory and applications.

Fixation-related potentials (FRPs), neural responses aligned to the end of saccades, are a promising tool for studying the dynamics of attention and cognition under natural viewing conditions. In the past, four methodological problems have complicated the analysis of such combined eye-tracking/electroencephalogram experiments: (1) the synchronization of data streams, (2) the removal of ocular artifacts, (3) the condition-specific temporal overlap between the brain responses evoked by consecutive fixations, and (4) the fact that numerous low-level stimulus and saccade properties also influence the postsaccadic neural responses. Although effective solutions exist for the first two problems, the latter two are only beginning to be addressed. In the current paper, we present and review a unified regression-based framework for FRP analysis that allows us to deconvolve overlapping potentials while also controlling for both linear and nonlinear confounds on the FRP waveform. An open software implementation is provided for all procedures. We then demonstrate the advantages of this proposed (non)linear deconvolution modeling approach for data from three commonly studied paradigms: face perception, scene viewing, and reading. First, for a traditional event-related potential (ERP) face recognition experiment, we show how this technique can separate stimulus ERPs from overlapping muscle and brain potentials produced by small (micro)saccades on the face. Second, in natural scene viewing, we model and isolate multiple nonlinear effects of saccade parameters on the FRP. Finally, for a natural sentence reading experiment using the boundary paradigm, we show how it is possible to study the neural correlates of parafoveal preview after removing spurious overlap effects caused by the associated difference in average fixation time. Our results suggest a principal way of measuring reliable eye movement-related brain activity during natural vision.

Post-Saccadic Face Processing Is Modulated by Pre-Saccadic Preview: Evidence from Fixation-Related Potentials.

Humans actively sample their environment with saccadic eye movements to bring relevant information into high-acuity foveal vision. Despite being lower in resolution, peripheral information is also available before each saccade. How the pre-saccadic extrafoveal preview of a visual object influences its post-saccadic processing is still an unanswered question. The current study investigated this question by simultaneously recording behavior and fixation-related brain potentials while human subjects made saccades to face stimuli. We manipulated the relationship between pre-saccadic "previews" and post-saccadic images to explicitly isolate the influences of the former. Subjects performed a gender discrimination task on a newly foveated face under three preview conditions: scrambled face, incongruent face (different identity from the foveated face), and congruent face (same identity). As expected, reaction times were faster after a congruent-face preview compared with a scrambled-face preview. Importantly, intact face previews (either incongruent or congruent) resulted in a massive reduction of post-saccadic neural responses. Specifically, we analyzed the classic face-selective N170 component at occipitotemporal electroencephalogram electrodes, which was still present in our experiments with active looking. However, the post-saccadic N170 was strongly attenuated following intact-face previews compared with the scrambled condition. This large and long-lasting decrease in evoked activity is consistent with a trans-saccadic mechanism of prediction that influences category-specific neural processing at the start of a new fixation. These findings constrain theories of visual stability and show that the extrafoveal preview methodology can be a useful tool to investigate its underlying mechanisms.SIGNIFICANCE STATEMENT Neural correlates of object recognition have traditionally been studied by flashing stimuli to the central visual field. This procedure differs in fundamental ways from natural vision, where viewers actively sample the environment with eye movements and also obtain a low-resolution preview of soon-to-be-fixated objects. Here we show that the N170, a classic electrophysiological marker of the structural encoding of faces, also occurs during a more natural viewing condition but is strongly reduced due to extrafoveal preprocessing (preview benefit). Our results therefore highlight the importance of peripheral vision during trans-saccadic processing in building a coherent and stable representation of the world around us.

Fixation-related Brain Potentials during Semantic Integration of Object-Scene Information.

In vision science, a particularly controversial topic is whether and how quickly the semantic information about objects is available outside foveal vision. Here, we aimed at contributing to this debate by coregistering eye movements and EEG while participants viewed photographs of indoor scenes that contained a semantically consistent or inconsistent target object. Linear deconvolution modeling was used to analyze the ERPs evoked by scene onset as well as the fixation-related potentials (FRPs) elicited by the fixation on the target object (t) and by the preceding fixation (t - 1). Object-scene consistency did not influence the probability of immediate target fixation or the ERP evoked by scene onset, which suggests that object-scene semantics was not accessed immediately. However, during the subsequent scene exploration, inconsistent objects were prioritized over consistent objects in extrafoveal vision (i.e., looked at earlier) and were more effortful to process in foveal vision (i.e., looked at longer). In FRPs, we demonstrate a fixation-related N300/N400 effect, whereby inconsistent objects elicit a larger frontocentral negativity than consistent objects. In line with the behavioral findings, this effect was already seen in FRPs aligned to the pretarget fixation t - 1 and persisted throughout fixation t, indicating that the extraction of object semantics can already begin in extrafoveal vision. Taken together, the results emphasize the usefulness of combined EEG/eye movement recordings for understanding the mechanisms of object-scene integration during natural viewing.

Optimizing the ICA-based removal of ocular EEG artifacts from free viewing experiments.

Combining EEG with eye-tracking is a promising approach to study neural correlates of natural vision, but the resulting recordings are also heavily contaminated by activity of the eye balls, eye lids, and extraocular muscles. While Independent Component Analysis (ICA) is commonly used to suppress these ocular artifacts, its performance under free viewing conditions has not been systematically evaluated and many published reports contain residual artifacts. Here I evaluated and optimized ICA-based correction for two tasks with unconstrained eye movements: visual search in images and sentence reading. In a first step, four parameters of the ICA pipeline were varied orthogonally: the (1) high-pass and (2) low-pass filter applied to the training data, (3) the proportion of training data containing myogenic saccadic spike potentials (SP), and (4) the threshold for eye tracker-based component rejection. In a second step, the eye-tracker was used to objectively quantify the correction quality of each ICA solution, both in terms of undercorrection (residual artifacts) and overcorrection (removal of neurogenic activity). As a benchmark, results were compared to those obtained with an alternative spatial filter, Multiple Source Eye Correction (MSEC). With commonly used settings, Infomax ICA not only left artifacts in the data, but also distorted neurogenic activity during eye movement-free intervals. However, correction results could be strongly improved by training the ICA on optimally filtered data in which SPs were massively overweighted. With optimized procedures, ICA removed virtually all artifacts, including the SP and its associated spectral broadband artifact from both viewing paradigms, with little distortion of neural activity. It also outperformed MSEC in terms of SP correction. Matlab code is provided.

Unfold: an integrated toolbox for overlap correction, non-linear modeling, and regression-based EEG analysis.

Electrophysiological research with event-related brain potentials (ERPs) is increasingly moving from simple, strictly orthogonal stimulation paradigms towards more complex, quasi-experimental designs and naturalistic situations that involve fast, multisensory stimulation and complex motor behavior. As a result, electrophysiological responses from subsequent events often overlap with each other. In addition, the recorded neural activity is typically modulated by numerous covariates, which influence the measured responses in a linear or non-linear fashion. Examples of paradigms where systematic temporal overlap variations and low-level confounds between conditions cannot be avoided include combined electroencephalogram (EEG)/eye-tracking experiments during natural vision, fast multisensory stimulation experiments, and mobile brain/body imaging studies. However, even "traditional," highly controlled ERP datasets often contain a hidden mix of overlapping activity (e.g., from stimulus onsets, involuntary microsaccades, or button presses) and it is helpful or even necessary to disentangle these components for a correct interpretation of the results. In this paper, we introduce unfold, a powerful, yet easy-to-use MATLAB toolbox for regression-based EEG analyses that combines existing concepts of massive univariate modeling ("regression-ERPs"), linear deconvolution modeling, and non-linear modeling with the generalized additive model into one coherent and flexible analysis framework. The toolbox is modular, compatible with EEGLAB and can handle even large datasets efficiently. It also includes advanced options for regularization and the use of temporal basis functions (e.g., Fourier sets). We illustrate the advantages of this approach for simulated data as well as data from a standard face recognition experiment. In addition to traditional and non-conventional EEG/ERP designs, unfold can also be applied to other overlapping physiological signals, such as pupillary or electrodermal responses. It is available as open-source software at http://www.unfoldtoolbox.org.

The peripheral preview effect with faces: Combined EEG and eye-tracking suggests multiple stages of trans-saccadic predictive and non-predictive processing.

The world appears stable despite saccadic eye-movements. One possible explanation for this phenomenon is that the visual system predicts upcoming input across saccadic eye-movements based on peripheral preview of the saccadic target. We tested this idea using concurrent electroencephalography (EEG) and eye-tracking. Participants made cued saccades to peripheral upright or inverted face stimuli that changed orientation (invalid preview) or maintained orientation (valid preview) while the saccade was completed. Experiment 1 demonstrated better discrimination performance and a reduced fixation-locked N170 component (fN170) with valid than with invalid preview, demonstrating integration of pre- and post-saccadic information. Moreover, the early fixation-related potentials (FRP) showed a preview face inversion effect suggesting that some pre-saccadic input was represented in the brain until around 170 ms post fixation-onset. Experiment 2 replicated Experiment 1 and manipulated the proportion of valid and invalid trials to test whether the preview effect reflects context-based prediction across trials. A whole-scalp Bayes factor analysis showed that this manipulation did not alter the fN170 preview effect but did influence the face inversion effect before the saccade. The pre-saccadic inversion effect declined earlier in the mostly invalid block than in the mostly valid block, which is consistent with the notion of pre-saccadic expectations. In addition, in both studies, we found strong evidence for an interaction between the pre-saccadic preview stimulus and the post-saccadic target as early as 50 ms (Experiment 2) or 90 ms (Experiment 1) into the new fixation. These findings suggest that visual stability may involve three temporal stages: prediction about the saccadic target, integration of pre-saccadic and post-saccadic information at around 50-90 ms post fixation onset, and post-saccadic facilitation of rapid categorization.

How microsaccades relate to lateralized ERP components of spatial attention: A co-registration study.

Covert shifts of attention that follow the presentation of a cue are associated with lateralized components in the event-related potential (ERP): the "early directing attention negativity" (EDAN) and the "anterior directing attention negativity" (ADAN). Traditionally, these shifts are thought to take place while gaze is fixated and, thus, in the absence of saccades. However, microsaccades of small amplitude (<1°) occur frequently and involuntarily also during fixation and are closely correlated with spatial attention. To investigate potential links between microsaccades and lateralized ERP components, we simultaneously recorded eye movements and ERPs in a spatial cueing task. As a first major result, we show that both the posterior EDAN and the orientation of microsaccades align more strongly with the location of the task-relevant part of the cue stimulus than with the direction of the attention shift indicated by that cue. A coupling between microsaccades and EDAN was also present on the single-trial level: The EDAN was largest when microsaccades were oriented toward the relevant cue, but absent when microsaccades were oriented away from it, suggesting that EDAN and microsaccades are generated by the same neural network, which selects relevant stimuli and orients behavior toward them. As a second major result, we show that small corneoretinal artifacts from microsaccades, which fall below conventional EOG rejection thresholds, contaminate the measurement of the ADAN. After correcting the EEG for microsaccade-related artifacts with an optimized variant of independent component analysis, ADAN was abolished at frontal sites, but a genuine ADAN was still present at central sites. Thus, the combined measurement of microsaccades and lateralized ERPs sheds new light onto cue-elicited shifts of covert attention.

Lateralization of posterior alpha EEG reflects the distribution of spatial attention during saccadic reading.

Visuospatial attention is an important mechanism in reading that governs the uptake of information from foveal and parafoveal regions of the visual field. However, the spatiotemporal dynamics of how attention is allocated during eye fixations are not completely understood. The current study explored the use of EEG alpha-band oscillations to investigate the spatial distribution of attention during reading. We reanalyzed two data sets, focusing on the lateralization of alpha activity at posterior scalp sites. In each experiment, participants read short lists of German nouns in two paradigms: either by freely moving their eyes (saccadic reading) or by fixating the screen center while the text moved passively from right to left at the same average speed (RSVP paradigm). In both paradigms, upcoming words were either visible or masked, and foveal processing load was manipulated by varying the words' lexical frequencies. Posterior alpha lateralization revealed a sustained rightward bias of attention during saccadic reading, but not in the RSVP paradigm. Interestingly, alpha lateralization was not influenced by word frequency (foveal load) or preview during the preceding fixation. Hence, alpha did not reflect transient attention shifts within a given fixation. However, in both experiments, we found that in the saccadic reading condition a stronger alpha lateralization shortly before a saccade predicted shorter fixations on the subsequently fixated word. These results indicate that alpha lateralization can serve as a measure of attention deployment and its link to oculomotor behavior in reading.

Dissociating parafoveal preview benefit and parafovea-on-fovea effects during reading: A combined eye tracking and EEG study.

During reading, the parafoveal processing of an upcoming word n+1 can influence word recognition in two ways: It can affect fixation behavior during the preceding fixation on word n (parafovea-on-fovea effect, POF), and it can facilitate subsequent foveal processing once word n+1 is fixated (preview benefit). While preview benefits are established, evidence for POF effects is mixed. Recently, it has been suggested that POF effects exist, but have a delayed impact on saccade planning and thus coincide with preview benefits measured on word n+1. We combined eye movement and EEG recordings to investigate and separate neural correlates of POF and preview benefit effects. Participants read lists of nouns either in a boundary paradigm or the RSVP-with-flankers paradigm, while we recorded fixation- or event-related potentials (FRPs/ERPs), respectively. The validity and lexical frequency of the word shown as preview for the upcoming word n+1 were orthogonally manipulated. Analyses focused on the first fixation on word n+1. Preview validity (correct vs. incorrect preview) strongly modulated fixation times and electrophysiological N1 amplitudes, replicating previous findings. Importantly, gaze durations and FRPs measured on word n+1 were also affected by the frequency of the word shown as preview, with low-frequency previews eliciting a sustained, N400-like centroparietal negativity. Results support the idea that POF effects exist but affect word recognition with a delay. Lastly, once word n+1 was fixated, its frequency also modulated N1 amplitudes in ERPs and FRPs. Taken together, we separated immediate and delayed effects of parafoveal processing on brain correlates of word recognition.

Neural Correlates of Word Recognition: A Systematic Comparison of Natural Reading and Rapid Serial Visual Presentation.

Neural correlates of word recognition are commonly studied with (rapid) serial visual presentation (RSVP), a condition that eliminates three fundamental properties of natural reading: parafoveal preprocessing, saccade execution, and the fast changes in attentional processing load occurring from fixation to fixation. We combined eye-tracking and EEG to systematically investigate the impact of all three factors on brain-electric activity during reading. Participants read lists of words either actively with eye movements (eliciting fixation-related potentials) or maintained fixation while the text moved passively through foveal vision at a matched pace (RSVP-with-flankers paradigm, eliciting ERPs). The preview of the upcoming word was manipulated by changing the number of parafoveally visible letters. Processing load was varied by presenting words of varying lexical frequency. We found that all three factors have strong interactive effects on the brain's responses to words: Once a word was fixated, occipitotemporal N1 amplitude decreased monotonically with the amount of parafoveal information available during the preceding fixation; hence, the N1 component was markedly attenuated under reading conditions with preview. Importantly, this preview effect was substantially larger during active reading (with saccades) than during passive RSVP with flankers, suggesting that the execution of eye movements facilitates word recognition by increasing parafoveal preprocessing. Lastly, we found that the N1 component elicited by a word also reflects the lexical processing load imposed by the previously inspected word. Together, these results demonstrate that, under more natural conditions, words are recognized in a spatiotemporally distributed and interdependent manner across multiple eye fixations, a process that is mediated by active motor behavior.

Parafoveal processing in reading Chinese sentences: Evidence from event-related brain potentials.

Natural reading involves the preprocessing of upcoming words, resulting in shorter fixations on words visible in the parafovea during preceding fixations. While this preview benefit is established in behavior, its brain-electric correlates have only recently been investigated. Using fixation-related potentials, an attenuation of the occipitotemporal N1 component for words that were parafoveally visible during preceding fixations has been demonstrated. In contrast, another study, using an RSVP paradigm with parafoveal flanker words, observed no such general preview benefit in ERPs, but instead reported N400 effects triggered by semantically incongruous parafoveal words. To follow up on these discrepant findings and to extend them to a nonalphabetic writing system, we conducted two ERP experiments with Chinese readers using the RSVP-with-flankers paradigm and rigorous fixation control via eye tracking. We replicate robust parafoveal N400 semantic congruency effects in Chinese participants. Additionally, we found that, once a word was directly looked at, words after a valid preview elicited a smaller N1 and a weaker N400 than those after an invalid preview. Results underline the importance of considering parafoveal vision in ERP studies on reading.

Are individual differences in reading speed related to extrafoveal visual acuity and crowding?

Readers differ considerably in their speed of self-paced reading. One factor known to influence fixation durations in reading is the preprocessing of words in parafoveal vision. Here we investigated whether individual differences in reading speed or the amount of information extracted from upcoming words (the preview benefit) can be explained by basic differences in extrafoveal vision--i.e., the ability to recognize peripheral letters with or without the presence of flanking letters. Forty participants were given an adaptive test to determine their eccentricity thresholds for the identification of letters presented either in isolation (extrafoveal acuity) or flanked by other letters (crowded letter recognition). In a separate eye-tracking experiment, the same participants read lists of words from left to right, while the preview of the upcoming words was manipulated with the gaze-contingent moving window technique. Relationships between dependent measures were analyzed on the observational level and with linear mixed models. We obtained highly reliable estimates both for extrafoveal letter identification (acuity and crowding) and measures of reading speed (overall reading speed, size of preview benefit). Reading speed was higher in participants with larger uncrowded windows. However, the strength of this relationship was moderate and it was only observed if other sources of variance in reading speed (e.g., the occurrence of regressive saccades) were eliminated. Moreover, the size of the preview benefit--an important factor in normal reading--was larger in participants with better extrafoveal acuity. Together, these results indicate a significant albeit moderate contribution of extrafoveal vision to individual differences in reading speed.

Microsaccade-related brain potentials signal the focus of visuospatial attention.

Covert shifts of visuospatial attention are traditionally assumed to occur in the absence of oculomotor behavior. In contrast, recent behavioral studies have linked attentional cueing effects to the occurrence of microsaccades, small eye movements executed involuntarily during attempted fixation. Here we used a new type of electrophysiological marker to explore the attention-microsaccade relationship, the visual brain activity evoked by the microsaccade itself. By shifting the retinal image, microsaccades frequently elicit neural responses throughout the visual pathway, scalp-recordable in the human EEG as a microsaccade-related potential (mSRP). Although mSRPs contain similar signal components (P1/N1) as traditional visually-evoked potentials (VEPs), it is unknown whether they are also influenced by cognition. Based on established findings that VEPs are amplified for visual inputs at currently attended locations, we expected a selective gain-modulation also for mSRPs. Eye movements and EEG were coregistered in a classic spatial cueing task with an endogenous cue. Replicating behavioral findings, the direction of early microsaccades 200-400ms after cue onset was biased towards the cued side. However, for microsaccades throughout the cue-target interval, mSRPs were systematically enhanced at occipital scalp sites contralateral to the cued hemifield. This attention effect resembled that in a control condition with VEPs and did not interact with the direction of the underlying microsaccade, suggesting that mSRPs reflect the focus of sustained visuospatial attention, which remains fixed at the cued location, despite microsaccades. Microsaccades are not merely an artifact source in the EEG; instead, they are followed by cognitively modulated brain potentials that can serve as non-intrusive electrophysiological probes of attention.

A model of microsaccade-related neural responses induced by short-term depression in thalamocortical synapses.

Microsaccades during fixation have been suggested to counteract visual fading. Recent experiments have also observed microsaccade-related neural responses from cellular record, scalp electroencephalogram (EEG), and functional magnetic resonance imaging (fMRI). The underlying mechanism, however, is not yet understood and highly debated. It has been proposed that the neural activity of primary visual cortex (V1) is a crucial component for counteracting visual adaptation. In this paper, we use computational modeling to investigate how short-term depression (STD) in thalamocortical synapses might affect the neural responses of V1 in the presence of microsaccades. Our model not only gives a possible synaptic explanation for microsaccades in counteracting visual fading, but also reproduces several features in experimental findings. These modeling results suggest that STD in thalamocortical synapses plays an important role in microsaccade-related neural responses and the model may be useful for further investigation of behavioral properties and functional roles of microsaccades.

Order Patterns Networks (ORPAN)-a method to estimate time-evolving functional connectivity from multivariate time series.

Complex networks provide an excellent framework for studying the function of the human brain activity. Yet estimating functional networks from measured signals is not trivial, especially if the data is non-stationary and noisy as it is often the case with physiological recordings. In this article we propose a method that uses the local rank structure of the data to define functional links in terms of identical rank structures. The method yields temporal sequences of networks which permits to trace the evolution of the functional connectivity during the time course of the observation. We demonstrate the potentials of this approach with model data as well as with experimental data from an electrophysiological study on language processing.