Common structure of saccades and microsaccades in visual perception.

We obtain large amounts of external information through our eyes, a process often considered analogous to picture mapping onto a camera lens. However, our eyes are never as still as a camera lens, with saccades occurring between fixations and microsaccades occurring within a fixation. Although saccades are agreed to be functional for information sampling in visual perception, it remains unknown if microsaccades have a similar function when eye movement is restricted. Here, we demonstrated that saccades and microsaccades share common spatiotemporal structures in viewing visual objects. Twenty-seven adults viewed faces and houses in free-viewing and fixation-controlled conditions. Both saccades and microsaccades showed distinctive spatiotemporal patterns between face and house viewing that could be discriminated by pattern classifications. The classifications based on saccades and microsaccades could also be mutually generalized. Importantly, individuals who showed more distinctive saccadic patterns between faces and houses also showed more distinctive microsaccadic patterns. Moreover, saccades and microsaccades showed a higher structure similarity for face viewing than house viewing and a common orienting preference for the eye region over the mouth region. These findings suggested a common oculomotor program that is used to optimize information sampling during visual object perception.

Behavioral Bias for Exploration Is Associated with Enhanced Signaling in the Lateral and Medial Frontopolar Cortex.

Should we keep doing what we know works for us, or should we risk trying something new as it could work even better? The exploration-exploitation dilemma is ubiquitous in daily life decision-making, and balancing between the two is crucial for adaptive behavior. Yet, we only have started to unravel the neurocognitive mechanisms that help us to find this balance in practice. Analyzing BOLD signals of healthy young adults during virtual foraging, we could show that a behavioral tendency for prolonged exploitation was associated with weakened signaling during exploration in central node points of the frontoparietal attention network, plus the frontopolar cortex. These results provide an important link between behavioral heuristics that we use to balance between exploitation and exploration and the brain function that supports shifts from one tendency to the other. Importantly, they stress that interindividual differences in behavioral strategies are reflected in differences in brain activity during exploration and should thus be more in the focus of basic research that aims at delineating general laws governing visual attention.

Saccadic re-referencing training with gaze-contingent FRL-'fixation': Effects of scotoma type and size adaptation.

Foveal vision loss makes the fovea as saccadic reference point maladaptive. Training programs have been proposed that shift the saccadic reference point from the fovea to an extrafoveal location, just outside the area of vision loss. We used a visual search task to train normal-sighted participants to fixate target items with a predetermined 'forced retinal location' (FRL) adjacent to a simulated central scotoma. We found that training was comparatively successful for scotomata that had either a sharp or blurry demarcation from the background. Completing the task with sharp-edged scotoma resulted in overall higher training gains. Training with blurry-edged scotoma, however, yielded overall better results when scotoma size was increased after training and participants needed to adapt to a more eccentric FRL, as may be necessary in patients with progressive degenerative eye diseases.

Efficient versus inefficient visual search as training for saccadic re-referencing to an extrafoveal location.

Central vision loss is one of the leading causes of visual impairment in the elderly and its frequency is increasing. Without formal training, patients adopt an unaffected region of the retina as a new fixation location, a preferred retinal locus (PRL). However, learning to use the PRL as a reference location for saccades, that is, saccadic re-referencing, is protracted and time-consuming. Recent studies showed that training with visual search tasks can expedite this process. However, visual search can be driven by salient external features - leading to efficient search, or by internal goals, usually leading to inefficient, attention-demanding search. We compared saccadic re-referencing training in the presence of a simulated central scotoma with either an efficient or an inefficient visual search task. Participants had to respond by fixating the target with an experimenter-defined retinal location in the lower visual field. We observed that comparable relative training gains were obtained in both tasks for a number of behavioral parameters, with higher training gains for the trained task, compared to the untrained task. The transfer to the untrained task was only observed for some parameters. Our findings thus confirm and extend previous research showing comparable efficiency for exogenously and endogenously driven visual search tasks for saccadic re-referencing training. Our results also show that transfer of training gains to related tasks may be limited and needs to be tested for saccadic re-referencing-training paradigms to assess its suitability as a training tool for patients.

A gaze-contingent saccadic re-referencing training with simulated central vision loss.

Patients with central vision loss (CVL) adopt an eccentric retinal location for fixation, a preferred retinal location (PRL), to compensate for vision loss at the fovea. Although most patients with CVL are able to rapidly use a PRL instead of the fovea, saccadic re-referencing to a PRL develops slowly. Without re-referencing, saccades land the saccade target in the scotoma. This results in corrective saccades and leads to inefficient visual exploration. Here, we tested a new method to train saccadic re-referencing. Healthy participants performed gaze-contingent visual search tasks with simulated central scotoma in which participants had to fixate targets with an experimenter-defined forced retinal location (FRL). In experiment 1, we compared single-target search and foraging search tasks in the course of five training sessions. Results showed that both tasks improved the efficiency of gaze sequences and led to saccadic re-referencing to the FRL. In experiment 2, we trained participants extensively for 25 sessions, both with and without a gaze-contingent FRL-marker visible during training. After extensive training, observers' performance approached that of foveal vision. Thus, gaze-contingent FRL-fixation may become an efficient tool for saccadic re-referencing training in patients with central vision loss.

Right-dominant contextual cueing for global configuration cues, but not local position cues.

Contextual cueing can depend on global configuration or local item position. We investigated the role of these two kinds of cues in the lateralization of contextual cueing effects. Cueing by item position was tested by recombining two previously learned displays, keeping the individual item locations intact, but destroying the global configuration. In contrast, cueing by configuration was investigated by rotating learned displays, thereby keeping the configuration intact but changing all item positions. We observed faster search for targets in the left display half, both for repeated and new displays, along with more first fixation locations on the left. Both position and configuration cues led to faster search, but the search time reduction compared to new displays due to position cues was comparable in the left and right display half. In contrast, configural cues led to increased search time reduction for right half targets. We conclude that only configural cues enabled memory-guided search for targets across the whole search display, whereas position cueing guided search only to targets in the vicinity of the fixation. The right-biased configural cueing effect is a consequence of the initial leftward search bias and does not indicate hemispheric dominance for configural cueing.

Working memory and active sampling of the environment: Medial temporal contributions.

Working memory (WM) refers to the ability to maintain and actively process information-either derived from perception or long-term memory (LTM)-for intelligent thought and action. This chapter focuses on the contributions of the temporal lobe, particularly medial temporal lobe (MTL) to WM. First, neuropsychological evidence for the involvement of MTL in WM maintenance is reviewed, arguing for a crucial role in the case of retaining complex relational bindings between memorized features. Next, MTL contributions at the level of neural mechanisms are covered-with a focus on WM encoding and maintenance, including interactions with ventral temporal cortex. Among WM use processes, we focus on active sampling of environmental information, a key input source to capacity-limited WM. MTL contributions to the bidirectional relationship between active sampling and memory are highlighted-WM control of active sampling and sampling as a way of selecting input to WM. Memory-based sampling studies relying on scene and object inspection, visual-based exploration behavior (e.g., vicarious behavior), and memory-guided visual search are reviewed. The conclusion is that MTL serves an important function in the selection of information from perception and transfer from LTM to capacity-limited WM.

Frontopolar activity carries feature information of novel stimuli during unconscious reweighting of selective attention.

Adapting to novelty is essential for an organism's survival in an uncertain world. Neuroimaging evidence consistently links the anterior prefrontal, specifically the frontopolar cortex (FPC; BA10), to exploratory reweighting of attentional weights thereby underscoring the role of the FPC in responding to environmental changes that are often complex and may occur very rapidly. Here we report new evidence showing that the FPC serves a role in attentional reallocation even in the absence of conscious awareness. Both mass-univariate and multivariate pattern analyses of fMRI data revealed that the right FPC and other attention-related areas not only are sensitive to unaware changes in the relevant stimulus dimension, but also that unconsciously processed information of the novel stimulus was globally represented across these regions. Our results indicate that unconsciously processed information can reach a global level of representation outside the occipitotemporal cortex, and that the FPC is crucial for the reweighting of selection biases in the absence of visual awareness.

Perceptual Learning of Object Recognition in Simulated Retinal Implant Perception - The Effect of Video Training.

Purpose: Retinal implants (RIs) provide new vision for patients suffering from photoreceptor degeneration in the retina. The limited vision gained by RI, however, leaves room for improvement by training regimes. Methods: Two groups of normal-sighted participants were respectively trained with videos or still images of daily objects in a labeling task. Object appearance was simulated to resemble RI perception. In Experiment 1, the training effect was measured as the change in performance during the training, and the same labeling task was conducted after 1 week to test the retention. In Experiment 2 with a different pool of participants, a reverse labeling task was included before (pre-test) and after the training (post-test) to show if the training effect could be generalized into a different task context. Results: Both groups showed improved object recognition through training that was maintained for a week, and the video group showed better improvement (Experiment 1). Both groups showed improved object recognition in a different task that was maintained for a week, but the video group did not show better retention than the image group (Experiment 2). Conclusions: Training with video materials leads to more improvement than training with still images in simulated RI perception, but this better improvement was specific to the trained task. Translational Relevance: We recommend videos as better training materials than still images for patients with RIs to improve object recognition when the task-goal is highly specific. We also propose here that achieving highly specific training goals runs the risk of limiting the generalization of the training effects.

Egocentric and Allocentric Reference Frames Can Flexibly Support Contextual Cueing.

We investigated if contextual cueing can be guided by egocentric and allocentric reference frames. Combinations of search configurations and external frame orientations were learned during a training phase. In Experiment 1, either the frame orientation or the configuration was rotated, thereby disrupting either the allocentric or egocentric and allocentric predictions of the target location. Contextual cueing survived both of these manipulations, suggesting that it can overcome interference from both reference frames. In contrast, when changed orientations of the external frame became valid predictors of the target location in Experiment 2, we observed contextual cueing as long as one reference frame was predictive of the target location, but contextual cueing was eliminated when both reference frames were invalid. Thus, search guidance in repeated contexts can be supported by both egocentric and allocentric reference frames as long as they contain valid information about the search goal.

The interactive effects of reward expectation and emotional interference on cognitive conflict control: An ERP study.

The effects of reward expectation and task-irrelevant emotional content on performance and event-related potential (ERP) recordings in a cognitive conflict control task were investigated using the face-word Stroop paradigm. A precue indicating additional monetary rewards for fast and accurate responses during the upcoming trial (incentive condition; relative to a cue indicating no additional reward, i.e., nonincentive condition) was followed by the presentation of target Chinese words (male vs. female) superimposed on background emotional faces (happy vs. fearful). The face's gender was congruent or incongruent with the target Chinese words. ERP results revealed that incentive cues elicited larger P1, P3, and CNV responses compared to nonincentive cues. There was a significant three-way interaction of reward expectation, emotional content, and congruency during the target processing stage such that emotionality and congruency interacted to affect the N170 and N2 component responses during the nonincentive condition but not during the incentive condition. These results indicate that reward-induced motivation reduces the interference effect of task-irrelevant emotional information, leading to better conflict resolution.

Feature-Based Attentional Weighting and Re-weighting in the Absence of Visual Awareness.

Visual attention evolved as an adaptive mechanism allowing us to cope with a rapidly changing environment. It enables the facilitated processing of relevant information, often automatically and governed by implicit motives. However, despite recent advances in understanding the relationship between consciousness and visual attention, the functional scope of unconscious attentional control is still under debate. Here, we present a novel masking paradigm in which volunteers were to distinguish between varying orientations of a briefly presented, masked grating stimulus. Combining signal detection theory and subjective measures of awareness, we show that performance on unaware trials was consistent with visual selection being weighted towards repeated orientations of Gabor patches and reallocated in response to a novel unconsciously processed orientation. This was particularly present in trials in which the prior feature was strongly weighted and only if the novel feature was invisible. Thus, our results provide evidence that invisible orientation stimuli can trigger the reallocation of history-guided visual selection weights.

Anomalous visual experience is linked to perceptual uncertainty and visual imagery vividness.

An imbalance between top-down and bottom-up processing on perception (specifically, over-reliance on top-down processing) can lead to anomalous perception, such as illusions. One factor that may be involved in anomalous perception is visual mental imagery, which is the experience of "seeing" with the mind's eye. There are vast individual differences in self-reported imagery vividness, and more vivid imagery is linked to a more sensory-like experience. We, therefore, hypothesized that susceptibility to anomalous perception is linked to individual imagery vividness. To investigate this, we adopted a paradigm that is known to elicit the perception of faces in pure visual noise (pareidolia). In four experiments, we explored how imagery vividness contributes to this experience under different response instructions and environments. We found strong evidence that people with more vivid imagery were more likely to see faces in the noise, although removing suggestive instructions weakened this relationship. Analyses from the first two experiments led us to explore confidence as another factor in pareidolia proneness. We, therefore, modulated environment noise and added a confidence rating in a novel design. We found strong evidence that pareidolia proneness is correlated with uncertainty about real percepts. Decreasing perceptual ambiguity abolished the relationship between pareidolia proneness and both imagery vividness and confidence. The results cannot be explained by incidental face-like patterns in the noise, individual variations in response bias, perceptual sensitivity, subjective perceptual thresholds, viewing distance, testing environments, motivation, gender, or prosopagnosia. This indicates a critical role of mental imagery vividness and perceptual uncertainty in anomalous perceptual experience.

Preserved Contextual Cueing in Realistic Scenes in Patients with Age-Related Macular Degeneration.

Foveal vision loss has been shown to reduce efficient visual search guidance due to contextual cueing by incidentally learned contexts. However, previous studies used artificial (T- among L-shape) search paradigms that prevent the memorization of a target in a semantically meaningful scene. Here, we investigated contextual cueing in real-life scenes that allow explicit memory of target locations in semantically rich scenes. In contrast to the contextual cueing deficits in artificial scenes, contextual cueing in patients with age-related macular degeneration (AMD) did not differ from age-matched normal-sighted controls. We discuss this in the context of visuospatial working-memory demands for which both eye movement control in the presence of central vision loss and memory-guided search may compete. Memory-guided search in semantically rich scenes may depend less on visuospatial working memory than search in abstract displays, potentially explaining intact contextual cueing in the former but not the latter. In a practical sense, our findings may indicate that patients with AMD are less deficient than expected after previous lab experiments. This shows the usefulness of realistic stimuli in experimental clinical research.

Intact Contextual Cueing for Search in Realistic Scenes with Simulated Central or Peripheral Vision Loss.

Purpose: Search in repeatedly presented visual search displays can benefit from implicit learning of the display items' spatial configuration. This effect has been named contextual cueing. Previously, contextual cueing was found to be reduced in observers with foveal or peripheral vision loss. Whereas this previous work used symbolic (T among L-shape) search displays with arbitrary configurations, here we investigated search in realistic scenes. Search in meaningful realistic scenes may benefit much more from explicit memory of the target location. We hypothesized that this explicit recall of the target location reduces visuospatial working memory demands on search considerably, thereby enabling efficient search guidance by learnt contextual cues in observers with vision loss. Methods: Two experiments with gaze-contingent scotoma simulation (Experiment 1: central scotoma, Experiment 2: peripheral scotoma) were carried out with normal-sighted observers (total n = 39/40). Observers had to find a cup in pseudorealistic indoor scenes and discriminate the direction of the cup's handle. Results: With both central and peripheral scotoma simulation, contextual cueing was observed in repeatedly presented configurations. Conclusions: The data show that patients suffering from central or peripheral vision loss may benefit more from memory-guided visual search than would be expected from scotoma simulation and patient studies using abstract symbolic search displays. Translational Relevance: In the assessment of visual search in patients with vision loss, semantically meaningless abstract search displays may gain insights into deficient search functions, but more realistic meaningful search scenes are needed to assess whether search deficits can be compensated.

Contextual-Cueing beyond the Initial Field of View-A Virtual Reality Experiment.

In visual search, participants can incidentally learn spatial target-distractor configurations, leading to shorter search times for repeated compared to novel configurations. Usually, this is tested within the limited visual field provided by a computer monitor. While contextual cueing is typically investigated on two-dimensional screens, we present for the first time an implementation of a classic contextual cueing task (search for a T-shape among L-shapes) in a three-dimensional virtual environment. This enabled us to test if the typical finding of incidental learning of repeated search configurations, manifested by shorter search times, would hold in a three-dimensional virtual reality (VR) environment. One specific aspect that was tested by combining virtual reality and contextual cueing was if contextual cueing would hold for targets outside the initial field of view (FOV), requiring head movements to be found. In keeping with two-dimensional search studies, reduced search times were observed after the first epoch and remained stable in the remaining experiment. Importantly, comparable search time reductions were observed for targets both within and outside of the initial FOV. The results show that a repeated distractors-only configuration in the initial FOV can guide search for target locations requiring a head movement to be seen.

Not scene learning, but attentional processing is superior in team sport athletes and action video game players.

We tested if high-level athletes or action video game players have superior context learning skills. Incidental context learning was tested in a spatial contextual cueing paradigm. We found comparable contextual cueing of visual search in repeated displays in high-level amateur handball players, dedicated action video game players and normal controls. In contrast, both handball players and action video game players showed faster search than controls, measured as search time per display item, independent of display repetition. Thus, our data do not indicate superior context learning skills in athletes or action video game players. Rather, both groups showed more efficient visual search in abstract displays that were not related to sport-specific situations.

Individual face- and house-related eye movement patterns distinctively activate FFA and PPA.

We investigated if the fusiform face area (FFA) and the parahippocampal place area (PPA) contain a representation of fixation sequences that are typically used when looking at faces or houses. Here, we instructed observers to follow a dot presented on a uniform background. The dot's movements represented gaze paths acquired separately from observers looking at face or house pictures. Even when gaze dispersion differences were controlled, face- and house-associated gaze patterns could be discriminated by fMRI multivariate pattern analysis in FFA and PPA, more so for the current observer's own gazes than for another observer's gaze. The discrimination of the observer's own gaze patterns was not observed in early visual areas (V1 - V4) or superior parietal lobule and frontal eye fields. These findings indicate a link between perception and action-the complex gaze patterns that are used to explore faces and houses-in the FFA and PPA.

The contribution of spatial position and rotated global configuration to contextual cueing.

Spatial information can incidentally guide attention to the likely location of a target. This contextual cueing was even observed if only the relative configuration, but not the individual locations of distractor items were repeated or vice versa (Jiang & Wagner in Perception & Psychophysics, 66(3), 454-463, 2004). The present study investigated the contribution of global configuration and individual spatial location to contextual cueing. Participants repeatedly searched 12 visual search displays in a learning session. In a subsequent transfer session, there were four conditions: fully repeated configurations (same as the displays in the learning session), recombined configurations from two learned configurations with the same target location (preserving distractor locations but not configuration), rotated configurations (preserving configuration but not distractor locations), and new configurations. We could show that contextual cueing occurred if only distractor locations or relative configuration, randomly intermixed, was preserved in a single experiment. Beyond replicating the results of Jiang and Wagner, we made an adjustment to a particular type of transformation - that may have occurred in separate experiments - unlikely. Moreover, contextual cueing in rotated configurations showed that repeated configurations can serve as context cues even without preserved azimuth.

Frontal cortex differentiates between free and imposed target selection in multiple-target search.

Cognitive control can involve proactive (preparatory) and reactive (corrective) mechanisms. Using a gaze-contingent eye tracking paradigm combined with fMRI, we investigated the involvement of these different modes of control and their underlying neural networks, when switching between different targets in multiple-target search. Participants simultaneously searched for two possible targets presented among distractors, and selected one of them. In one condition, only one of the targets was available in each display, so that the choice was imposed, and reactive control would be required. In the other condition, both targets were present, giving observers free choice over target selection, and allowing for proactive control. Switch costs emerged only when targets were imposed and not when target selection was free. We found differential levels of activity in the frontoparietal control network depending on whether target switches were free or imposed. Furthermore, we observed core regions of the default mode network to be active during target repetitions, indicating reduced control on these trials. Free and imposed switches jointly activated parietal and posterior frontal cortices, while free switches additionally activated anterior frontal cortices. These findings highlight unique contributions of proactive and reactive control during visual search.