Multisensory temporal binding induces an illusory gap/overlap that reduces the expected audiovisual interactions on saccades but not manual responses.

Throughout the day, humans react to multisensory events conveying both visual and auditory signals by rapidly reorienting their gaze. Several studies showed that sounds can impact the latency of visually guided saccades depending on when and where they are delivered. We found that unlocalized beeps delivered near the onset time of a visual target reduce latencies, more for early beeps and less for late beeps, however, this modulation is far weaker than for perceptual temporal judgments. Here we tested our previous assumption that beeps shift the perceived timing of target onset and result in two competing effects on saccade latencies: a multisensory modulation in line with the expected perceptual effect and an illusory gap/overlap effect, resulting from target appearance being perceived later/closer in time than fixation offset and shortening/lengthening saccade latencies. Gap/overlap effects involve an oculomotor component associated with neuronal activity in the superior colliculus (SC), a multisensory subcortical structure devoted to sensory-motor transformation. We therefore predicted that the interfering illusory gap/overlap effect would be weaker for manual responses, which involve distinct multisensory areas. In three experiments we manipulated the delay between target onset and an irrelevant auditory beep (stimulus onset asynchrony; SOA) and between target onset and fixation offset (real gap/overlap). Targets appeared left/right of fixation and participants were instructed to make quick saccades or button presses towards the targets. Adding a real overlap/gap (50% of SOA) compensated for the illusory gap/overlap by increasing the beep-related modulation of saccade latencies across the entire SOA range, whereas it barely affected manual responses. However, although auditory and gap/overlap effects modulated saccade latencies in similar ways, these were additive and could saturate, suggesting that they reflect independent mechanisms. Therefore, multisensory temporal binding affects perception and oculomotor control differently, likely due to the implication of the SC in saccade programming and multisensory integration.

Linguistic processes do not beat visuo-motor constraints, but they modulate where the eyes move regardless of word boundaries: Evidence against top-down word-based eye-movement control during reading.

Where readers move their eyes, while proceeding forward along lines of text, has long been assumed to be determined in a top-down word-based manner. According to this classical view, readers of alphabetic languages would invariably program their saccades towards the center of peripheral target words, as selected based on the (expected) needs of ongoing (word-identification) processing, and the variability in within-word landing positions would exclusively result from systematic and random errors. Here we put this predominant hypothesis to a strong test by estimating the respective influences of language-related variables (word frequency and word predictability) and lower-level visuo-motor factors (word length and saccadic launch-site distance to the beginning of words) on both word-skipping likelihood and within-word landing positions. Our eye-movement data were collected while forty participants read 316 pairs of sentences, that differed only by one word, the prime; this was either semantically related or unrelated to a following test word of variable frequency and length. We found that low-level visuo-motor variables largely predominated in determining which word would be fixated next, and where in a word the eye would land. In comparison, language-related variables only had tiny influences. Yet, linguistic variables affected both the likelihood of word skipping and within-word initial landing positions, all depending on the words' length and how far on average the eye landed from the word boundaries, but pending the word could benefit from peripheral preview. These findings provide a strong case against the predominant word-based account of eye-movement guidance during reading, by showing that saccades are primarily driven by low-level visuo-motor processes, regardless of word boundaries, while being overall subject to subtle, one-off, language-based modulations. Our results also suggest that overall distributions of saccades' landing positions, instead of truncated within-word landing-site distributions, should be used for a better understanding of eye-movement guidance during reading.

The magnification factor accounts for the greater hypometria and imprecision of larger saccades: Evidence from a parametric human-behavioral study.

Saccades quite systematically undershoot a peripheral visual target by about 10% of its eccentricity while becoming more variable, mainly in amplitude, as the target becomes more peripheral. This undershoot phenomenon has been interpreted as the strategic adjustment of saccadic gain downstream of the superior colliculus (SC), where saccades are programmed. Here, we investigated whether the eccentricity-related increase in saccades' hypometria and imprecision might not instead result from overrepresentation of space closer to the fovea in the SC and visual-cortical areas. To test this magnification-factor (MF) hypothesis, we analyzed four parametric eye-movement data sets, collected while humans made saccades to single eccentric stimuli. We first established that the undershoot phenomenon generalizes to ordinary saccade amplitudes (0.5°-15°) and directions (0°-90°) and that landing-position distributions become not only increasingly elongated but also more skewed toward the fovea as target eccentricity increases. Moreover, we confirmed the MF hypothesis by showing (a) that the linear eccentricity-related increase in undershoot error and negative skewness canceled out when landing positions were log-scaled according to the MF in monkeys' SC and (b) that the spread, proportional to eccentricity outside an extended, 5°, foveal region, became circular and invariant in size in SC space. Yet the eccentricity-related increase in variability, slower near the fovea, yielded progressively larger and more elongated clusters toward foveal and vertical-meridian SC representations. What causes this latter, unexpected, pattern remains undetermined. Nevertheless, our findings clearly suggest that the undershoot phenomenon, and related variability, originate in, or upstream of, the SC, rather than reflecting downstream, adaptive, strategies.

A Model of the Superior Colliculus Predicts Fixation Locations during Scene Viewing and Visual Search.

Modern computational models of attention predict fixations using saliency maps and target maps, which prioritize locations for fixation based on feature contrast and target goals, respectively. But whereas many such models are biologically plausible, none have looked to the oculomotor system for design constraints or parameter specification. Conversely, although most models of saccade programming are tightly coupled to underlying neurophysiology, none have been tested using real-world stimuli and tasks. We combined the strengths of these two approaches in MASC, a model of attention in the superior colliculus (SC) that captures known neurophysiological constraints on saccade programming. We show that MASC predicted the fixation locations of humans freely viewing naturalistic scenes and performing exemplar and categorical search tasks, a breadth achieved by no other existing model. Moreover, it did this as well or better than its more specialized state-of-the-art competitors. MASC's predictive success stems from its inclusion of high-level but core principles of SC organization: an over-representation of foveal information, size-invariant population codes, cascaded population averaging over distorted visual and motor maps, and competition between motor point images for saccade programming, all of which cause further modulation of priority (attention) after projection of saliency and target maps to the SC. Only by incorporating these organizing brain principles into our models can we fully understand the transformation of complex visual information into the saccade programs underlying movements of overt attention. With MASC, a theoretical footing now exists to generate and test computationally explicit predictions of behavioral and neural responses in visually complex real-world contexts.SIGNIFICANCE STATEMENT The superior colliculus (SC) performs a visual-to-motor transformation vital to overt attention, but existing SC models cannot predict saccades to visually complex real-world stimuli. We introduce a brain-inspired SC model that outperforms state-of-the-art image-based competitors in predicting the sequences of fixations made by humans performing a range of everyday tasks (scene viewing and exemplar and categorical search), making clear the value of looking to the brain for model design. This work is significant in that it will drive new research by making computationally explicit predictions of SC neural population activity in response to naturalistic stimuli and tasks. It will also serve as a blueprint for the construction of other brain-inspired models, helping to usher in the next generation of truly intelligent autonomous systems.

Integration of parafoveal orthographic information during foveal word reading: beyond the sub-lexical level?

Prior research has shown that processing of a given target word is facilitated by the simultaneous presentation of orthographically related stimuli in the parafovea. Here we investigate the nature of such spatial integration processes by presenting orthographic neighbours of target words in the parafovea, considering that neighbours have been shown to inhibit, rather than facilitate, recognition of target words in foveal masked priming research. In Experiment 1, we used the gaze-contingent boundary paradigm to manipulate the parafoveal information subjects received while they fixated a target word within a sentence. In Experiment 2, we used the Flanking Letters Lexical Decision paradigm to manipulate parafoveal information while subjects read isolated words. Parafoveal words were either a higher-frequency orthographic neighbour of targets words (e.g., blue-blur) or a high-frequency unrelated word (e.g., hand-blur). We found that parafoveal orthographic neighbours facilitated, rather than inhibited, processing of the target. Thus, the present findings provide further evidence that orthographic information is integrated across multiple words and suggest that either the integration process does not enable simultaneous access to those words' lexical representations, or that lexical representations activated by spatially distinct stimuli do not compete for recognition.

No Evidence for a Saccadic Range Effect for Visually Guided and Memory-Guided Saccades in Simple Saccade-Targeting Tasks.

Saccades to single targets in peripheral vision are typically characterized by an undershoot bias. Putting this bias to a test, Kapoula [1] used a paradigm in which observers were presented with two different sets of target eccentricities that partially overlapped each other. Her data were suggestive of a saccadic range effect (SRE): There was a tendency for saccades to overshoot close targets and undershoot far targets in a block, suggesting that there was a response bias towards the center of eccentricities in a given block. Our Experiment 1 was a close replication of the original study by Kapoula [1]. In addition, we tested whether the SRE is sensitive to top-down requirements associated with the task, and we also varied the target presentation duration. In Experiments 1 and 2, we expected to replicate the SRE for a visual discrimination task. The simple visual saccade-targeting task in Experiment 3, entailing minimal top-down influence, was expected to elicit a weaker SRE. Voluntary saccades to remembered target locations in Experiment 3 were expected to elicit the strongest SRE. Contrary to these predictions, we did not observe a SRE in any of the tasks. Our findings complement the results reported by Gillen et al. [2] who failed to find the effect in a saccade-targeting task with a very brief target presentation. Together, these results suggest that unlike arm movements, saccadic eye movements are not biased towards making saccades of a constant, optimal amplitude for the task.

On the optimal viewing position for object processing.

Numerous studies have shown that a visually presented word is processed most easily when participants initially fixate just to the left of the word's center. Fixating on this optimal viewing position (OVP) results in shorter response times and a lower probability of making additional within-word refixations (OVP effects), but also longer initial-fixation durations (an inverted-OVP or I-OVP effect), as compared to initially fixating at the beginning or the end of the word. Thus, typical curves are u-shaped (or inverted-u-shaped), with a leftward bias. Most researchers explain the u-shape in terms of visual constraints, and the leftward bias in terms of language constraints. Previous studies have demonstrated that (I)-OVP effects are not specific to words, but generalize to object viewing. We further investigated this by comparing the strength and (a)symmetry of (I-)OVP effects for words and objects. To this purpose, we gave participants an object- versus word-naming task in which we manipulated the position at which they initially fixated the stimulus (i.e., a line drawing or the written name of an object). Our results showed that object viewing, just as word viewing, resulted in u-shaped (I-)OVP curves. However, the effect was weaker than for words. Furthermore, for words, the curves were biased to the left, whereas they were symmetrical for objects. This might indicate that part of the (I-)OVP effect for words is language specific, and that (I-)OVP effects for objects are a purer measure of the effect of visual constraints.

The role of object affordances and center of gravity in eye movements toward isolated daily-life objects.

The purpose of the current study was to investigate to what extent low-level versus high-level effects determine where the eyes land on isolated daily-life objects. We operationalized low-level effects as eye movements toward an object's center of gravity (CoG) or the absolute object center (OC) and high-level effects as visuomotor priming by object affordances. In two experiments, we asked participants to make saccades toward peripherally presented photographs of graspable objects (e.g., a hammer) and to either categorize them (Experiment 1) or to discriminate them from visually matched nonobjects (Experiment 2). Objects were rotated such that their graspable part (e.g., the hammer's handle) pointed toward either the left or the right whereas their action-performing part (e.g., the hammer's head) pointed toward the other side. We found that early-triggered saccades were neither biased toward the object's graspable part nor toward its action-performing part. Instead, participants' eyes landed near the CoG/OC. Only longer-latency initial saccades and refixations were subject to high-level influences, being significantly biased toward the object's action-performing part. Our comparison with eye movements toward visually matched nonobjects revealed that the latter was not merely the consequence of a low-level effect of shape, texture, asymmetry, or saliency. Instead, we interpret it as a higher-level, object-based affordance effect that requires time, and to some extent also foveation, in order to build up and to overcome default saccadic-programming mechanisms.

Large pupils predict goal-driven eye movements.

Here we report that large pupils predict fixations of the eye on low-salient, inconspicuous parts of a visual scene. We interpret this as showing that mental effort, reflected by a dilation of the pupil, is required to guide gaze toward objects that are relevant to current goals, but that may not be very salient. When mental effort is low, reflected by a constriction of the pupil, the eyes tend to be captured by high-salient parts of the image, irrespective of top-down goals. The relationship between pupil size and visual saliency was not driven by luminance or a range of other factors that we considered. Crucially, the relationship was strongest when mental effort was invested exclusively in eye-movement control (i.e., reduced in a dual-task setting), which suggests that it is not due to general effort or arousal. Our finding illustrates that goal-driven control during scene viewing requires mental effort, and that pupil size can be used as an online measure to track the goal-drivenness of behavior.

The pupillary light response reflects eye-movement preparation.

When the eyes are exposed to an increased influx of light, the pupils constrict. The pupillary light response (PLR) is traditionally believed to be purely reflexive and not susceptible to cognitive influences. In contrast to this traditional view, we report that preparation of a PLR occurs in parallel with preparation of a saccadic eye movement toward a bright (or dark) stimulus, even before the eyes set in motion. Participants fixated a central gray area and made a saccade toward a peripheral target. Using gaze-contingent display changes, we manipulated whether or not the brightness of the target background was the same during and after saccade preparation. More specifically, on some trials we changed the brightness of the target background during the saccade, thus dissociating the preparatory PLR (i.e., to the brightness of the target background before the saccade) from the regular PLR (i.e., to the brightness after the saccade). We show that preparation triggers a pupillary response to the brightness of a to-be-fixated target background already before the eyes have landed on it. We link our findings to the presaccadic shift of attention: The pupil prepares to adjust its size to the brightness of a to-be-fixated stimulus as soon as attention covertly shifts toward that stimulus. Our findings illustrate that the PLR is a dynamic movement that is tightly linked to visual attention and eye-movement preparation.

On the reduced influence of contour on saccade metrics and its competition with stimulus size.

It is well known that the metrical properties of saccadic eye movements are strongly influenced by the extraction of low-level visual features (e.g., luminance). Higher-level visual features (e.g., contour) also play a role, but their relative contribution and time course remain undetermined. Here, we investigated this issue, by testing the influence of contour on saccade metrics. We used a saccade-targeting task in which a peripheral target was, on some trials, simultaneously displayed with a less eccentric distractor. This paradigm is known to yield a global effect, that is a deviation of the eyes towards an intermediate location between the stimuli. The novelty was to test whether this effect would vary with the alignment of the distractor's elementary features. Distractors were of high vs. low luminance, and composed of 16 pixels that were either aligned or misaligned by 0.23° or 0.43°. Our prediction, under the hypothesis that contour intervenes, was that aligned distractors, which formed a definite contour, would deviate the eyes more strongly than misaligned distractors. On the contrary, we found that distractors of high luminance produced greater eye deviations when they were misaligned, and hence more largely spread, than when they were aligned. Furthermore, low-luminance distractors deviated the eyes to the same extent irrespective of their alignment, though showing a reversed, contour-like, effect of alignment for early-triggered saccades. We proposed that contour has only limited influence on saccade metrics, when other, lower-level and more salient visual features, such as the extent of the stimulus pattern, are available.

Tests of a model of multi-word reading: effects of parafoveal flanking letters on foveal word recognition.

We used the "flanking letters lexical decision" paradigm of Dare and Shillcock (2013) in order to test a model of multi-word reading. In the model, multiple words (on fixation, and to the left and right of fixation) are processed in parallel by a bank of location-specific letter detectors. These letter detectors feed information forward to a "bag of bigrams" that represents location-invariant sublexical orthographic information for all words processed in parallel. Bigrams are only formed within words (i.e., between spaces) but activate all compatible word representations. The model accounts for a finding reported by Dare and Shillcock (2013): Word recognition is facilitated when flanking letter pairs are present in the target (e.g. RO ROCK CK) compared with different letter flankers (ST ROCK EN), but independently of the position of the flanking bigrams (e.g., CK ROCK RO). In the present study we replicate this key finding and show that, as predicted by the model, although bigram position does not matter, within-bigram letter position does. Word recognition is harder when the position of letters within bigram flankers is reversed (e.g., OR ROCK KC/KC ROCK OR), but these conditions still facilitate with respect to a different letter flanker condition.

The pupillary light response reveals the focus of covert visual attention.

The pupillary light response is often assumed to be a reflex that is not susceptible to cognitive influences. In line with recent converging evidence, we show that this reflexive view is incomplete, and that the pupillary light response is modulated by covert visual attention: Covertly attending to a bright area causes a pupillary constriction, relative to attending to a dark area under identical visual input. This attention-related modulation of the pupillary light response predicts cuing effects in behavior, and can be used as an index of how strongly participants attend to a particular location. Therefore, we suggest that pupil size may offer a new way to continuously track the focus of covert visual attention, without requiring a manual response from the participant. The theoretical implication of this finding is that the pupillary light response is neither fully reflexive, nor under complete voluntary control, but is instead best characterized as a stereotyped response to a voluntarily selected target. In this sense, the pupillary light response is similar to saccadic and smooth pursuit eye movements. Together, eye movements and the pupillary light response maximize visual acuity, stabilize visual input, and selectively filter visual information as it enters the eye.

On the limited effect of stimulus boundaries on saccade metrics.

How the endpoint of saccadic eye movements is determined out of many potential peripheral locations is a crucial issue in the field of vision. Models of saccade generation account for this seemingly selective process in terms of competitive interactions between populations of neurons that encode respectively for different saccade amplitudes and directions. However, these models do not specify which visual stimulus properties other than the relative location of the stimuli are involved and how these properties contribute to ultimately determine a single saccade endpoint. We addressed this issue by contrasting the respective contributions of the 2-D spatial extent of the stimuli and the location of their boundaries in a global-effect paradigm. Participants were presented a to-be-looked-at peripheral target stimulus with or without a less eccentric visually invariant distractor. The extent of the target stimulus was manipulated in either one or two dimensions, such that targets differed either by their 2-D spatial extent (small, medium, or large circle) or the location of their boundaries (circle vs. horizontal or vertical ellipse of medium size). Results showed that the distractor deviated the eyes away from the target with the deviation varying with the 2-D spatial extent of the target but not the location of its boundaries. This finding suggests that the spatial distribution of luminance contrast and/or the number of elementary features that compose the stimuli prevails over visual boundaries in specifying the saccade endpoint. Implications for models of saccade generation are discussed.

Early dynamics of the semantic priming shift.

Semantic processing of sequences of words requires the cognitive system to keep several word meanings simultaneously activated in working memory with limited capacity. The real- time updating of the sequence of word meanings relies on dynamic changes in the associates to the words that are activated. Protocols involving two sequential primes report a semantic priming shift from larger priming of associates to the first prime to larger priming of associates to the second prime, in a range of long SOAs (stimulus-onset asynchronies) between the second prime and the target. However, the possibility for an early semantic priming shift is still to be tested, and its dynamics as a function of association strength remain unknown. Three multiple priming experiments are proposed that cross-manipulate association strength between each of two successive primes and a target, for different values of short SOAs and prime durations. Results show an early priming shift ranging from priming of associates to the first prime only to priming of strong associates to the first prime and all of the associates to the second prime. We investigated the neural basis of the early priming shift by using a network model of spike frequency adaptive cortical neurons (e.g., Deco & Rolls, 2005), able to code different association strengths between the primes and the target. The cortical network model provides a description of the early dynamics of the priming shift in terms of pro-active and retro-active interferences within populations of excitatory neurons regulated by fast and unselective inhibitory feedback.

Stimulus properties and saccade metrics: when local features are more critical than global features.

One major issue in the field of vision is how stimulus properties contribute to determine where the eyes move. Here, we examined the role of local versus global visual features in the computation of saccade metrics in the light of the well-known tendency for saccade to vary with the size of the stimuli. We used a saccade-target task in which we varied the properties of a visual distractor simultaneously displayed with the target stimulus. Both the size and the luminance contrast of the distractor were varied but the number of elementary features that composed the distractor was held constant. Our results showed that under such controls mean saccades' landing position remained unaffected by stimulus size irrespective of the level of luminance contrast. These findings suggest that the local visual features of a stimulus may be more critical than global features to specify a particular location to look at. These results are consistent with the notion that local features contribute to determine the amplitude and the width of the neuronal activity patterns associated with the visual stimulation and hence the computation of saccade metrics.

The Optimal Viewing Position effect in the lower visual field.

The Optimal Viewing Position (OVP) effect shows that word identification is best when the eyes first fixate near the centre of words. While this effect has been extensively studied in normal reading conditions, it has not been much investigated for words in the periphery. Here, we compared, in a perceptual identification task, the OVP effect for words presented either on the line of sight or in the lower visual field. Results confirmed the existence of an OVP effect for both central and vertically-shifted words but this effect was significantly weaker in the lower visual field. This finding provides further evidence for an important role of letter visibility in determining the shape of the OVP phenomenon. It also indicates that aligning the eyes with the centre of words is not as critical for vertically-shifted words. Implications for patients with central field loss who are forced to read in the periphery are discussed.

On the effect of remote and proximal distractors on saccadic behavior: a challenge to neural-field models.

Two proposals have been made to account for the generation of saccadic eye movements. The first assumes that when the eyes move is under the control of a fixation gating system. The second attributes the decisions of both when and where the eyes move to the interplay between short-range excitatory and long-range inhibitory interactions within the motor map of the superior colliculus (SC). To distinguish both views, three behavioral experiments conducted on human participants tested the respective contributions of stimulus eccentricity and interstimulus distance on the effects of remote and proximal distractors on the latency and accuracy of saccades. Experiment 1 showed that the saccade-latency increase that results from the presentation of a remote distractor in the contralateral, nontarget hemifield varies with the ratio of distractor-to-target eccentricity, but not the interstimulus distance in visual or collicular space, thus indicating that the effect is not due to long-range inhibition. Experiments 2a and 2b showed that short-range excitation does not underlie the effect of proximal, ipsilateral distractors. Proximal distractors do not systematically shorten saccade latency, but rather show a range of effects (from a latency increase to no effect and then facilitation) as the ratio of distractor-to-target eccentricity increases, while deviating the eyes to gradually larger extents. The present findings strongly challenge the neural-field account, while suggesting that when a saccade is initiated depends mainly on the activity of a fixation gating system.

Fixation positions after skipping saccades: a single space makes a large difference.

During reading, saccadic eye movements are generated to shift words into the center of the visual field for lexical processing. Recently, Krügel and Engbert (Vision Research 50:1532-1539, 2010) demonstrated that within-word fixation positions are largely shifted to the left after skipped words. However, explanations of the origin of this effect cannot be drawn from normal reading data alone. Here we show that the large effect of skipped words on the distribution of within-word fixation positions is primarily based on rather subtle differences in the low-level visual information acquired before saccades. Using arrangements of "x" letter strings, we reproduced the effect of skipped character strings in a highly controlled single-saccade task. Our results demonstrate that the effect of skipped words in reading is the signature of a general visuomotor phenomenon. Moreover, our findings extend beyond the scope of the widely accepted range-error model, which posits that within-word fixation positions in reading depend solely on the distances of target words. We expect that our results will provide critical boundary conditions for the development of visuomotor models of saccade planning during reading.

When larger visual distractors become less disruptive: behavioral evidence for lateral inhibition in saccade generation.

How neuronal activity is integrated over time may largely rely on excitatory and inhibitory mechanisms. Dynamic neural field models assume that local excitation and lateral inhibition (i.e., the "Mexican hat") shape the output of neural networks. Most models of saccade generation assume that such interactions in the superior colliculus play a key role in determining both the metrics and the latency of saccades. Here, we investigated the role of lateral inhibition in saccade metrics in humans. We used a saccade target task in which a visual distractor line was presented close to a peripheral visual target (i.e., a small circle). Models assuming lateral inhibition predict that beyond a critical size larger distractors induce less perturbation than smaller ones. To assess this prediction, we varied the length of the distractor. Results confirmed that a distractor presented along with the target deviated the saccade's landing position away from the target. This perturbation increased with distractor length but only up to a critical size as the effect reversed for larger distractors, leading to a reduced perturbation on saccade metrics. These results suggest that larger distractors induce a neuronal activity pattern wide enough to involve lateral inhibition, thereby decreasing the distractor's weight in the spatial integration of distractor and target locations. They are consistent with a critical role of lateral inhibition in the computation of saccade metrics.