Behavioral Bias for Exploration Is Associated with Enhanced Signaling in the Lateral and Medial Frontopolar Cortex.

Should we keep doing what we know works for us, or should we risk trying something new as it could work even better? The exploration-exploitation dilemma is ubiquitous in daily life decision-making, and balancing between the two is crucial for adaptive behavior. Yet, we only have started to unravel the neurocognitive mechanisms that help us to find this balance in practice. Analyzing BOLD signals of healthy young adults during virtual foraging, we could show that a behavioral tendency for prolonged exploitation was associated with weakened signaling during exploration in central node points of the frontoparietal attention network, plus the frontopolar cortex. These results provide an important link between behavioral heuristics that we use to balance between exploitation and exploration and the brain function that supports shifts from one tendency to the other. Importantly, they stress that interindividual differences in behavioral strategies are reflected in differences in brain activity during exploration and should thus be more in the focus of basic research that aims at delineating general laws governing visual attention.

Common structure of saccades and microsaccades in visual perception.

We obtain large amounts of external information through our eyes, a process often considered analogous to picture mapping onto a camera lens. However, our eyes are never as still as a camera lens, with saccades occurring between fixations and microsaccades occurring within a fixation. Although saccades are agreed to be functional for information sampling in visual perception, it remains unknown if microsaccades have a similar function when eye movement is restricted. Here, we demonstrated that saccades and microsaccades share common spatiotemporal structures in viewing visual objects. Twenty-seven adults viewed faces and houses in free-viewing and fixation-controlled conditions. Both saccades and microsaccades showed distinctive spatiotemporal patterns between face and house viewing that could be discriminated by pattern classifications. The classifications based on saccades and microsaccades could also be mutually generalized. Importantly, individuals who showed more distinctive saccadic patterns between faces and houses also showed more distinctive microsaccadic patterns. Moreover, saccades and microsaccades showed a higher structure similarity for face viewing than house viewing and a common orienting preference for the eye region over the mouth region. These findings suggested a common oculomotor program that is used to optimize information sampling during visual object perception.

Efficient versus inefficient visual search as training for saccadic re-referencing to an extrafoveal location.

Central vision loss is one of the leading causes of visual impairment in the elderly and its frequency is increasing. Without formal training, patients adopt an unaffected region of the retina as a new fixation location, a preferred retinal locus (PRL). However, learning to use the PRL as a reference location for saccades, that is, saccadic re-referencing, is protracted and time-consuming. Recent studies showed that training with visual search tasks can expedite this process. However, visual search can be driven by salient external features - leading to efficient search, or by internal goals, usually leading to inefficient, attention-demanding search. We compared saccadic re-referencing training in the presence of a simulated central scotoma with either an efficient or an inefficient visual search task. Participants had to respond by fixating the target with an experimenter-defined retinal location in the lower visual field. We observed that comparable relative training gains were obtained in both tasks for a number of behavioral parameters, with higher training gains for the trained task, compared to the untrained task. The transfer to the untrained task was only observed for some parameters. Our findings thus confirm and extend previous research showing comparable efficiency for exogenously and endogenously driven visual search tasks for saccadic re-referencing training. Our results also show that transfer of training gains to related tasks may be limited and needs to be tested for saccadic re-referencing-training paradigms to assess its suitability as a training tool for patients.

A gaze-contingent saccadic re-referencing training with simulated central vision loss.

Patients with central vision loss (CVL) adopt an eccentric retinal location for fixation, a preferred retinal location (PRL), to compensate for vision loss at the fovea. Although most patients with CVL are able to rapidly use a PRL instead of the fovea, saccadic re-referencing to a PRL develops slowly. Without re-referencing, saccades land the saccade target in the scotoma. This results in corrective saccades and leads to inefficient visual exploration. Here, we tested a new method to train saccadic re-referencing. Healthy participants performed gaze-contingent visual search tasks with simulated central scotoma in which participants had to fixate targets with an experimenter-defined forced retinal location (FRL). In experiment 1, we compared single-target search and foraging search tasks in the course of five training sessions. Results showed that both tasks improved the efficiency of gaze sequences and led to saccadic re-referencing to the FRL. In experiment 2, we trained participants extensively for 25 sessions, both with and without a gaze-contingent FRL-marker visible during training. After extensive training, observers' performance approached that of foveal vision. Thus, gaze-contingent FRL-fixation may become an efficient tool for saccadic re-referencing training in patients with central vision loss.

Anomalous visual experience is linked to perceptual uncertainty and visual imagery vividness.

An imbalance between top-down and bottom-up processing on perception (specifically, over-reliance on top-down processing) can lead to anomalous perception, such as illusions. One factor that may be involved in anomalous perception is visual mental imagery, which is the experience of "seeing" with the mind's eye. There are vast individual differences in self-reported imagery vividness, and more vivid imagery is linked to a more sensory-like experience. We, therefore, hypothesized that susceptibility to anomalous perception is linked to individual imagery vividness. To investigate this, we adopted a paradigm that is known to elicit the perception of faces in pure visual noise (pareidolia). In four experiments, we explored how imagery vividness contributes to this experience under different response instructions and environments. We found strong evidence that people with more vivid imagery were more likely to see faces in the noise, although removing suggestive instructions weakened this relationship. Analyses from the first two experiments led us to explore confidence as another factor in pareidolia proneness. We, therefore, modulated environment noise and added a confidence rating in a novel design. We found strong evidence that pareidolia proneness is correlated with uncertainty about real percepts. Decreasing perceptual ambiguity abolished the relationship between pareidolia proneness and both imagery vividness and confidence. The results cannot be explained by incidental face-like patterns in the noise, individual variations in response bias, perceptual sensitivity, subjective perceptual thresholds, viewing distance, testing environments, motivation, gender, or prosopagnosia. This indicates a critical role of mental imagery vividness and perceptual uncertainty in anomalous perceptual experience.

Not scene learning, but attentional processing is superior in team sport athletes and action video game players.

We tested if high-level athletes or action video game players have superior context learning skills. Incidental context learning was tested in a spatial contextual cueing paradigm. We found comparable contextual cueing of visual search in repeated displays in high-level amateur handball players, dedicated action video game players and normal controls. In contrast, both handball players and action video game players showed faster search than controls, measured as search time per display item, independent of display repetition. Thus, our data do not indicate superior context learning skills in athletes or action video game players. Rather, both groups showed more efficient visual search in abstract displays that were not related to sport-specific situations.

Frontal cortex differentiates between free and imposed target selection in multiple-target search.

Cognitive control can involve proactive (preparatory) and reactive (corrective) mechanisms. Using a gaze-contingent eye tracking paradigm combined with fMRI, we investigated the involvement of these different modes of control and their underlying neural networks, when switching between different targets in multiple-target search. Participants simultaneously searched for two possible targets presented among distractors, and selected one of them. In one condition, only one of the targets was available in each display, so that the choice was imposed, and reactive control would be required. In the other condition, both targets were present, giving observers free choice over target selection, and allowing for proactive control. Switch costs emerged only when targets were imposed and not when target selection was free. We found differential levels of activity in the frontoparietal control network depending on whether target switches were free or imposed. Furthermore, we observed core regions of the default mode network to be active during target repetitions, indicating reduced control on these trials. Free and imposed switches jointly activated parietal and posterior frontal cortices, while free switches additionally activated anterior frontal cortices. These findings highlight unique contributions of proactive and reactive control during visual search.

Differential brain mechanisms for processing distracting information in task-relevant and -irrelevant dimensions in visual search.

A crucial function of our goal-directed behavior is to select task-relevant targets among distractor stimuli, some of which may share properties with the target and thus compete for attentional selection. Here, by applying functional magnetic resonance imaging (fMRI) to a visual search task in which a target was embedded in an array of distractors that were homogeneous or heterogeneous along the task-relevant (orientation or form) and/or task-irrelevant (color) dimensions, we demonstrate that for both (orientation) feature search and (form) conjunction search, the fusiform gyrus is involved in processing the task-irrelevant color information, while the bilateral frontal eye fields (FEF), the cortex along the left intraparietal sulcus (IPS), and the left junction of intraparietal and transverse occipital sulci (IPTO) are involved in processing task-relevant distracting information, especially for target-absent trials. Moreover, in conjunction (but not in feature) search, activity in these frontoparietal regions is affected by stimulus heterogeneity along the task-irrelevant dimension: heterogeneity of the task-irrelevant information increases the activity in these regions only when the task-relevant information is homogeneous, not when it is heterogeneous. These findings suggest that differential neural mechanisms are involved in processing task-relevant and task-irrelevant dimensions of the searched-for objects. In addition, they show that the top-down task set plays a dominant role in determining whether or not task-irrelevant information can affect the processing of the task-relevant dimension in the frontoparietal regions.

Gradual acquisition of visuospatial associative memory representations via the dorsal precuneus.

Activation of parietal cortex structures like the precuneus is commonly observed during explicit memory retrieval, but the role of parietal cortices in encoding has only recently been appreciated and is still poorly understood. Considering the importance of the precuneus in human visual attention and imagery, we aimed to assess a potential role for the precuneus in the encoding of visuospatial representations into long-term memory. We therefore investigated the acquisition of constant versus repeatedly shuffled configurations of icons on background images over five subsequent days in 32 young, healthy volunteers. Functional magnetic resonance imaging was conducted on Days 1, 2, and 5, and persistent memory traces were assessed by a delayed memory test after another 5 days. Constant compared to shuffled configurations were associated with significant improvement of position recognition from Day 1 to 5 and better delayed memory performance. Bilateral dorsal precuneus activations separated constant from shuffled configurations from Day 2 onward, and coactivation of the precuneus and hippocampus dissociated recognized and forgotten configurations, irrespective of condition. Furthermore, learning of constant configurations elicited increased functional coupling of the precuneus with dorsal and ventral visual stream structures. Our results identify the precuneus as a key brain structure in the acquisition of detailed visuospatial information by orchestrating a parieto-occipito-temporal network.

Cross-task perceptual learning of object recognition in simulated retinal implant perception.

The perception gained by retina implants (RI) is limited, which asks for a learning regime to improve patients' visual perception. Here we simulated RI vision and investigated if object recognition in RI patients can be improved and maintained through training. Importantly, we asked if the trained object recognition can be generalized to a new task context, and to new viewpoints of the trained objects. For this purpose, we adopted two training tasks, a labelling task where participants had to choose the correct label out of other distracting labels for the presented object, and a reverse labelling task where participants had to choose the correct object out of other distracting objects to match the presented label. Our results showed that, despite of the task order, recognition performance was improved in both tasks and lasted at least for a week. The improved object recognition, however, can be transferred only from the labelling task to the reverse labelling task but not vice versa. Additionally, the trained object recognition can be transferred to new viewpoints of the trained objects only in the labelling task but not in the reverse labelling task. Training with the labelling task is therefore recommended for RI patients to achieve persistent and flexible visual perception.

The effect of acquisition resolution on orientation decoding from V1 BOLD fMRI at 7T.

A decade after it was shown that the orientation of visual grating stimuli can be decoded from human visual cortex activity by means of multivariate pattern classification of BOLD fMRI data, numerous studies have investigated which aspects of neuronal activity are reflected in BOLD response patterns and are accessible for decoding. However, it remains inconclusive what the effect of acquisition resolution on BOLD fMRI decoding analyses is. The present study is the first to provide empirical ultra high-field fMRI data recorded at four spatial resolutions (0.8mm, 1.4mm, 2mm, and 3mm isotropic voxel size) on this topic - in order to test hypotheses on the strength and spatial scale of orientation discriminating signals. We present detailed analysis, in line with predictions from previous simulation studies, about how the performance of orientation decoding varies with different acquisition resolutions. Moreover, we also examine different spatial filtering procedures and its effects on orientation decoding. Here we show that higher-resolution scans with subsequent down-sampling or low-pass filtering yield no benefit over scans natively recorded in the corresponding lower resolution regarding decoding accuracy. The orientation-related signal in the BOLD fMRI data is spatially broadband in nature, includes both high spatial frequency components, as well as large-scale biases previously proposed in the literature. Moreover, we found above chance-level contribution from large draining veins to orientation decoding. Acquired raw data were publicly released to facilitate further investigation.

Neural structures involved in visual search guidance by reward-enhanced contextual cueing of the target location.

Spatial contextual cueing reflects an incidental form of learning that occurs when spatial distractor configurations are repeated in visual search displays. Recently, it was reported that the efficiency of contextual cueing can be modulated by reward. We replicated this behavioral finding and investigated its neural basis with fMRI. Reward value was associated with repeated displays in a learning session. The effect of reward value on context-guided visual search was assessed in a subsequent fMRI session without reward. Structures known to support explicit reward valuation, such as ventral frontomedial cortex and posterior cingulate cortex, were modulated by incidental reward learning. Contextual cueing, leading to more efficient search, went along with decreased activation in the visual search network. Retrosplenial cortex played a special role in that it showed both a main effect of reward and a reward×configuration interaction and may thereby be a central structure for the reward modulation of context-guided visual search.

Impairment of visual memory for objects in natural scenes by simulated central scotomata.

Because of the close link between foveal vision and the spatial deployment of attention, typically only objects that have been foveated during scene exploration may form detailed and persistent memory representations. In a recent study on patients suffering from age-related macular degeneration, however, we found surprisingly accurate visual long-term memory for objects in scenes. Normal exploration patterns that the patients had learned to rereference saccade targets to an extrafoveal retinal location. This rereferencing may allow use of an extrafoveal location as a focus of attention for efficient object encoding into long-term memory. Here, we tested this hypothesis in normal-sighted observers with gaze-contingent central scotoma simulations. As these observers were inexperienced in scene exploration with central vision loss and had not developed saccadic rereferencing, we expected deficits in long-term memory for objects. We used the same change detection task as in our patient study, probing sensitivity to object changes after a period of free scene exploration. Change detection performance was significantly reduced for two types of scotoma simulation diminishing foveal and parafoveal vision--a visible gray disc and a more subtle image warping--compared with unimpaired controls, confirming our hypothesis. The impact of a smaller scotoma covering specifically foveal vision was less distinct, leading to a marginally significant decrease of long-term memory performance compared with controls. We conclude that attentive encoding of objects is deficient when central vision is lost as long as successful saccadic rereferencing has not yet developed.

The right temporo-parietal junction contributes to visual feature binding.

We investigated the neural basis of conjoined processing of color and spatial frequency with functional magnetic resonance imaging (fMRI). A multivariate classification algorithm was trained to differentiate between either isolated color or spatial frequency differences, or between conjoint differences in both feature dimensions. All displays were presented in a singleton search task, avoiding confounds between conjunctive feature processing and search difficulty that arose in previous studies contrasting single feature and conjunction search tasks. Based on patient studies, we expected the right temporo-parietal junction (TPJ) to be involved in conjunctive feature processing. This hypothesis was confirmed in that only conjoined color and spatial frequency differences, but not isolated feature differences could be classified above chance level in this area. Furthermore, we could show that the accuracy of a classification of differences in both feature dimensions was superadditive compared to the classification accuracies of isolated color or spatial frequency differences within the right TPJ. These data provide evidence for the processing of feature conjunctions, here color and spatial frequency, in the right TPJ.

A universal role of the ventral striatum in reward-based learning: evidence from human studies.

Reinforcement learning enables organisms to adjust their behavior in order to maximize rewards. Electrophysiological recordings of dopaminergic midbrain neurons have shown that they code the difference between actual and predicted rewards, i.e., the reward prediction error, in many species. This error signal is conveyed to both the striatum and cortical areas and is thought to play a central role in learning to optimize behavior. However, in human daily life rewards are diverse and often only indirect feedback is available. Here we explore the range of rewards that are processed by the dopaminergic system in human participants, and examine whether it is also involved in learning in the absence of explicit rewards. While results from electrophysiological recordings in humans are sparse, evidence linking dopaminergic activity to the metabolic signal recorded from the midbrain and striatum with functional magnetic resonance imaging (fMRI) is available. Results from fMRI studies suggest that the human ventral striatum (VS) receives valuation information for a diverse set of rewarding stimuli. These range from simple primary reinforcers such as juice rewards over abstract social rewards to internally generated signals on perceived correctness, suggesting that the VS is involved in learning from trial-and-error irrespective of the specific nature of provided rewards. In addition, we summarize evidence that the VS can also be implicated when learning from observing others, and in tasks that go beyond simple stimulus-action-outcome learning, indicating that the reward system is also recruited in more complex learning tasks.

Prediction of higher visual function in macular degeneration with multifocal electroretinogram and multifocal visual evoked potential.

OBJECTIVE: Visual search can be guided by past experience of regularities in our visual environment. This search guidance by contextual memory cues is impaired by foveal vision loss. Here we compared retinal and cortical visually evoked responses in their predictive value for contextual cueing impairment and visual acuity. METHODS: Multifocal electroretinograms to flash stimulation (mfERGs; 103 locations; 55.8° diameter) and visual evoked potentials to pattern-reversal stimulation (mfVEPs; 60 locations; 48.6° diameter) were recorded monocularly in participants with age-related macular degeneration (n = 14 and 16, respectively). Response magnitudes were calculated as the respective signal-to-noise ratios for each eccentricity. Visual acuities (logMAR, range: 0.0-1.2) and contextual cueing effects on visual search (reaction time gain, range: -0.14-0.15) were correlated with the signal-to-noise ratios. A step-wise regression analysis was applied separately to the mfERG- and mfVEP-dataset to determine the eccentricity range and the processing stage that is critical for these visual functions. RESULTS: Central mfERGs (1.0-3.2°) were the sole predictor of contextual cueing of visual search (p = 0.006), but they were not significant predictors of visual acuity. In contrast, central mfVEPs (1.3-3.2°) were the sole predictor of visual acuity (p < 0.001), but they were not significant predictors of contextual cueing. CONCLUSIONS: Contextual cueing is more dependent on parafoveal mfERG magnitude while visual acuity is more dependent on parafoveal mfVEP magnitude. The relation of contextual cueing to parafoveal mfERG magnitudes indicates the predictive value of retinal bipolar cell activity for this advanced level of visual function.

Saccadic re-referencing training with gaze-contingent FRL-'fixation': Effects of scotoma type and size adaptation.

Foveal vision loss makes the fovea as saccadic reference point maladaptive. Training programs have been proposed that shift the saccadic reference point from the fovea to an extrafoveal location, just outside the area of vision loss. We used a visual search task to train normal-sighted participants to fixate target items with a predetermined 'forced retinal location' (FRL) adjacent to a simulated central scotoma. We found that training was comparatively successful for scotomata that had either a sharp or blurry demarcation from the background. Completing the task with sharp-edged scotoma resulted in overall higher training gains. Training with blurry-edged scotoma, however, yielded overall better results when scotoma size was increased after training and participants needed to adapt to a more eccentric FRL, as may be necessary in patients with progressive degenerative eye diseases.

Differential patch-leaving behavior during probabilistic foraging in humans and gerbils.

Foraging confronts animals, including humans, with the need to balance exploration and exploitation: exploiting a resource until it depletes and then deciding when to move to a new location for more resources. Research across various species has identified rules for when to leave a depleting patch, influenced by environmental factors like patch quality. Here we compare human and gerbil patch-leaving behavior through two analogous tasks: a visual search for humans and a physical foraging task for gerbils, both involving patches with randomly varying initial rewards that decreased exponentially. Patch-leaving decisions of humans but not gerbils follow an incremental mechanism based on reward encounters that is considered optimal for maximizing reward yields in variable foraging environments. The two species also differ in their giving-up times, and some human subjects tend to overharvest. However, gerbils and individual humans who do not overharvest are equally sensitive to declining collection rates in accordance with the marginal value theorem. Altogether this study introduces a paradigm for a between-species comparison on how to resolve the exploitation-exploration dilemma.

Dorsal and ventral working memory-related brain areas support distinct processes in contextual cueing.

Behavioral evidence suggests that the use of implicitly learned spatial contexts for improved visual search may depend on visual working memory resources. Working memory may be involved in contextual cueing in different ways: (1) for keeping implicitly learned working memory contents available during search or (2) for the capture of attention by contexts retrieved from memory. We mapped brain areas that were modulated by working memory capacity. Within these areas, activation was modulated by contextual cueing along the descending segment of the intraparietal sulcus, an area that has previously been related to maintenance of explicit memories. Increased activation for learned displays, but not modulated by the size of contextual cueing, was observed in the temporo-parietal junction area, previously associated with the capture of attention by explicitly retrieved memory items, and in the ventral visual cortex. This pattern of activation extends previous research on dorsal versus ventral stream functions in memory guidance of attention to the realm of attentional guidance by implicit memory.

Evidence for feature binding in the superior parietal lobule.

The neural substrates of feature binding are an old, yet still not completely resolved problem. While patient studies suggest that posterior parietal cortex is necessary for feature binding, imaging evidence has been inconclusive in the past. These studies compared visual feature and conjunction search to investigate the neural substrate of feature conjunctions. However, a common problem of these comparisons was a confound with search difficulty. To circumvent this confound, we directly investigated the localized representation of features (color and spatial frequency) and feature conjunctions in a single search task by using multivariate pattern analysis at high field strength (7T). In right superior parietal lobule, we found evidence for the representation of feature conjunctions that could not be explained by the summation of individual feature representations and thus indicates conjoined processing of color and spatial frequency.