Taking a Popular Science Approach to the Field of Attention.

Over the past few years, I have been actively engaged in informing the public, including policy makers and teachers, about the latest findings in attention research. Despite certain challenges in meeting expectations of both the public and peers, engaging with the public and sharing scientific knowledge not only inspired new research avenues but also highlighted the importance of supporting popular science activities within academia.

Steady-state Visual Evoked Potentials Reveal Dynamic (Re)allocation of Spatial Attention during Maintenance and Utilization of Visual Working Memory.

Visual working memory (VWM) allows storing goal-relevant information to guide future behavior. Prior work suggests that VWM is spatially organized and relies on spatial attention directed toward locations at which memory items were encoded, even if location is task-irrelevant. Importantly, attention often needs to be dynamically redistributed between locations, for example, in preparation for an upcoming probe. Very little is known about how attentional resources are distributed between multiple locations during a VWM task and even less about the dynamic changes governing such attentional shifts over time. This is largely due to the inability to use behavioral outcomes to reveal fast dynamic changes within trials. We here demonstrated that EEG steady-state visual evoked potentials (SSVEPs) successfully track the dynamic allocation of spatial attention during a VWM task. Participants were presented with to-be-memorized gratings and distractors at two distinct locations, tagged with flickering discs. This allowed us to dynamically track attention allocated to memory and distractor items via their coupling with space by quantifying the amplitude and coherence of SSVEP responses in the EEG signal to flickering stimuli at the former memory and distractor locations. SSVEP responses did not differ between memory and distractor locations during early maintenance. However, shortly before probe comparison, we observed a decrease in SSVEP coherence over distractor locations indicative of a reallocation of spatial attentional resources. RTs were shorter when preceded by stronger decreases in SSVEP coherence at distractor locations, likely reflecting attentional shifts from the distractor to the probe or memory location. We demonstrate that SSVEPs can inform about dynamic processes in VWM, even if location does not have to be reported by participants. This finding not only supports the notion of a spatially organized VWM but also reveals that SSVEPs betray a dynamic prioritization process of working memory items and locations over time that is directly predictive of memory performance.

The impact of acute asymmetric hearing loss on multisensory integration.

Humans have the remarkable ability to integrate information from different senses, which greatly facilitates the detection, localization and identification of events in the environment. About 466 million people worldwide suffer from hearing loss. Yet, the impact of hearing loss on how the senses work together is rarely investigated. Here, we investigate how a common sensory impairment, asymmetric conductive hearing loss (AHL), alters the way our senses interact by examining human orienting behaviour with normal hearing (NH) and acute AHL. This type of hearing loss disrupts auditory localization. We hypothesized that this creates a conflict between auditory and visual spatial estimates and alters how auditory and visual inputs are integrated to facilitate multisensory spatial perception. We analysed the spatial and temporal properties of saccades to auditory, visual and audiovisual stimuli before and after plugging the right ear of participants. Both spatial and temporal aspects of multisensory integration were affected by AHL. Compared with NH, AHL caused participants to make slow, inaccurate and unprecise saccades towards auditory targets. Surprisingly, increased weight on visual input resulted in accurate audiovisual localization with AHL. This came at a cost: saccade latencies for audiovisual targets increased significantly. The larger the auditory localization errors, the less participants were able to benefit from audiovisual integration in terms of saccade latency. Our results indicate that observers immediately change sensory weights to effectively deal with acute AHL and preserve audiovisual accuracy in a way that cannot be fully explained by statistical models of optimal cue integration.

Zooming in on abnormal local and global processing biases after stroke: Frequency, lateralization, and associations with cognitive functions.

OBJECTIVES: The 'attentional spotlight' can be adjusted depending on the task requirements, resulting in processing information at either the local or global level. Stroke can lead to local or global processing biases, or the inability to simultaneously attend both levels. In this study, we assessed the (1) prevalence of abnormal local and global biases following stroke, (2) differences between left- and right-sided brain damaged patients, and (3) relations between local and global interference, the ability to attend local and global levels simultaneously, and lateralized attention, search organization, search speed, visuo-construction, executive functioning, and verbal (working) memory. METHODS: Stroke patients admitted for inpatient rehabilitation completed directed (N = 192 total; N = 46 left-sided/N = 48 right-sided lesion) and divided (N = 258 total; N = 67 left-sided/N = 66 right-sided lesion) local-global processing tasks, as well as a conventional neuropsychological assessment. Processing biases and interference effects were separately computed for directed and divided tasks. RESULTS: On the local-global tasks, 7.8-10.9% of patients showed an abnormal local bias and 6.3-8.3% an abnormal global bias for directed attention, and 5.4-10.1% an abnormal local bias and 6.6-15.9% an abnormal global bias for divided attention. There was no significant difference between patients with left- and right-sided brain damage. There was a moderate positive relation between local interference and search speed, and a small positive relation between global interference and neglect. CONCLUSIONS: Abnormal local and global biases can occur after stroke and might relate to a range of cognitive functions. A specific bias might require a different approach in assessment, psycho-education, and treatment.

A matter of availability: sharper tuning for memorized than for perceived stimulus features.

Our visual environment is relatively stable over time. An optimized visual system could capitalize on this by devoting less representational resources to objects that are physically present. The vividness of subjective experience, however, suggests that externally available (perceived) information is more strongly represented in neural signals than memorized information. To distinguish between these opposing predictions, we use EEG multivariate pattern analysis to quantify the representational strength of task-relevant features in anticipation of a change-detection task. Perceptual availability was manipulated between experimental blocks by either keeping the stimulus available on the screen during a 2-s delay period (perception) or removing it shortly after its initial presentation (memory). We find that task-relevant (attended) memorized features are more strongly represented than irrelevant (unattended) features. More importantly, we find that task-relevant features evoke significantly weaker representations when they are perceptually available compared with when they are unavailable. These findings demonstrate that, contrary to what subjective experience suggests, vividly perceived stimuli elicit weaker neural representations (in terms of detectable multivariate information) than the same stimuli maintained in visual working memory. We hypothesize that an efficient visual system spends little of its limited resources on the internal representation of information that is externally available anyway.

When the central integrator disintegrates: A review of the role of the thalamus in cognition and dementia.

The thalamus is a complex neural structure with numerous anatomical subdivisions and intricate connectivity patterns. In recent decades, the traditional view of the thalamus as a relay station and "gateway to the cortex" has expanded in recognition of its role as a central integrator of inputs from sensory systems, cortex, basal ganglia, limbic systems, brain stem nuclei, and cerebellum. As such, the thalamus is critical for numerous aspects of human cognition, mood, and behavior, as well as serving sensory processing and motor functions. Thalamus pathology is an important contributor to cognitive and functional decline, and it might be argued that the thalamus has been somewhat overlooked as an important player in dementia. In this review, we provide a comprehensive overview of thalamus anatomy and function, with an emphasis on human cognition and behavior, and discuss emerging insights on the role of thalamus pathology in dementia.

More (corrective) consecutive saccades after a lesion to the posterior parietal cortex.

To reach a target, primary saccades (S1s) are often followed by (corrective) consecutive saccades (S2, and potentially S3, S4, S5), which are based on retinal and extraretinal feedback. Processing these extraretinal signals was found to be significantly impaired by lesions to the posterior parietal cortex (PPC). Recent studies, however, added a more nuanced view to the role of the PPC, where patients with PPC lesions still used extraretinal signals for S2s and perceptual judgements (Fabius et al., 2020; Rath-Wilson & Guitton, 2015). Hence, it seems that a PPC lesion is not disrupting extraretinal processing per se. Yet, a lesion might still result in less reliable processing of extraretinal signals. Here, we investigated whether this lower reliability manifests as decreased or delayed S2 initiation. Patients with PPC lesions (n = 7) and controls (n = 26) performed a prosaccade task where the target either remained visible or was removed after S1 onset. When S1 is removed, accurate S2s (corrections of S1 error) rely solely on extraretinal signals. We analysed S2 quantity and timing using linear mixed-effects modelling and additive hazards analyses. Patients demonstrated slower S1 execution and lower S1 amplitudes than controls, but their S2s still compensated the S1 undershoot, also when they only relied on extraretinal information. Surprisingly, patients showed an increased amount of S2s. This deviation from control behaviour can be seen as suboptimal, but given the decreased accuracy of the primary saccade, it could be optimal for patients to employ more (corrective) consecutive saccades to overcome this inaccuracy.

Don't hide the instruction manual: A dynamic trade-off between using internal and external templates during visual search.

Visual search is typically studied by requiring participants to memorize a template initially, for which they subsequently search in a crowded display. Search in daily life, however, often involves templates that remain accessible externally, and may therefore be (re)attended for just-in-time encoding or to refresh internal template representations. Here, we show that participants indeed use external templates during search when given the chance. This behavior was observed during both simple and complex search, scaled with task difficulty, and was associated with improved performance. Furthermore, we show that participants used external sampling not only to offload memory, but also as a means of verifying whether the template was remembered correctly at the end of trials. We conclude that the external world may not only provide the challenge (e.g., distractors), but may dynamically ease search. These results argue for extensions of state-of-the-art models of search, because external sampling seems to be used frequently, in at least two ways and is actually beneficial for task performance. Our findings support a model of visual working memory that emphasizes a resource-efficient trade-off between storing and (re)attending external information.

Differential aspects of attention predict the depth of visual working memory encoding: Evidence from pupillometry.

What determines how much one encodes into visual working memory? Traditionally, encoding depth is considered to be indexed by spatiotemporal properties of gaze, such as gaze position and dwell time. Although these properties inform about where and how long one looks, they do not necessarily inform about the current arousal state or how strongly attention is deployed to facilitate encoding. Here, we found that two types of pupillary dynamics predict how much information is encoded during a copy task. The task involved encoding a spatial pattern of multiple items for later reproduction. Results showed that smaller baseline pupil sizes preceding and stronger pupil orienting responses during encoding predicted that more information was encoded into visual working memory. Additionally, we show that pupil size reflects not only how much but also how precisely material is encoded. We argue that a smaller pupil size preceding encoding is related to increased exploitation, whereas larger pupil constrictions signal stronger attentional (re)orienting to the to-be-encoded pattern. Our findings support the notion that the depth of visual working memory encoding is the integrative outcome of differential aspects of attention: how alert one is, how much attention one deploys, and how long it is deployed. Together, these factors determine how much information is encoded into visual working memory.

Open-DPSM: An open-source toolkit for modeling pupil size changes to dynamic visual inputs.

Pupil size change is a widely adopted, sensitive indicator for sensory and cognitive processes. However, the interpretation of these changes is complicated by the influence of multiple low-level effects, such as brightness or contrast changes, posing challenges to applying pupillometry outside of extremely controlled settings. Building on and extending previous models, we here introduce Open Dynamic Pupil Size Modeling (Open-DPSM), an open-source toolkit to model pupil size changes to dynamically changing visual inputs using a convolution approach. Open-DPSM incorporates three key steps: (1) Modeling pupillary responses to both luminance and contrast changes; (2) Weighing of the distinct contributions of visual events across the visual field on pupil size change; and (3) Incorporating gaze-contingent visual event extraction and modeling. These steps improve the prediction of pupil size changes beyond the here-evaluated benchmarks. Open-DPSM provides Python functions, as well as a graphical user interface (GUI), enabling the extension of its applications to versatile scenarios and adaptations to individualized needs. By obtaining a predicted pupil trace using video and eye-tracking data, users can mitigate the effects of low-level features by subtracting the predicted trace or assess the efficacy of the low-level feature manipulations a priori by comparing estimated traces across conditions.

Infant walking experience is related to the development of selective attention.

Previous studies have shown that the way in which infants perceive and explore the world changes as they transition from crawling to walking. Infant walking onset generally precedes advances in cognitive development such as accelerated language growth. However, the underlying mechanism explaining this association between walking experience and cognition is largely unknown. Selective attention is a key factor underlying learning across multiple domains. We propose that the altered visual input that infants obtain as they transition to walking relates to selective attention development and that advances in selective attention may potentially explain previously reported advances in other cognitive domains. As a first step in testing this hypothesis, we investigated how walking experience relates to selective attention. In Study 1, performance of 14-month-old crawlers, novice walkers, and expert walkers was compared on a visual search eye-tracking task (N = 47), including feature and conjunction (effortful) items. Walkers outperformed crawlers on the task in general, and effortful search was enhanced in expert walkers as compared with novice walkers, after controlling for crawling onset and general developmental differences occurring before walking onset. In Study 2, earlier walking onset was related to better visual search performance in 2-year-olds (N = 913). The association appeared to be due to the difference between the 10% latest walkers and the early/average walkers. Taken together, the results of these studies show that walking experience relates to advances in selective attention. This association shows a specific timing in development; it is mainly seen relatively close to the age of walking onset.

Time course of spatiotopic updating across saccades.

Humans move their eyes several times per second, yet we perceive the outside world as continuous despite the sudden disruptions created by each eye movement. To date, the mechanism that the brain employs to achieve visual continuity across eye movements remains unclear. While it has been proposed that the oculomotor system quickly updates and informs the visual system about the upcoming eye movement, behavioral studies investigating the time course of this updating suggest the involvement of a slow mechanism, estimated to take more than 500 ms to operate effectively. This is a surprisingly slow estimate, because both the visual system and the oculomotor system process information faster. If spatiotopic updating is indeed this slow, it cannot contribute to perceptual continuity, because it is outside the temporal regime of typical oculomotor behavior. Here, we argue that the behavioral paradigms that have been used previously are suboptimal to measure the speed of spatiotopic updating. In this study, we used a fast gaze-contingent paradigm, using high phi as a continuous stimulus across eye movements. We observed fast spatiotopic updating within 150 ms after stimulus onset. The results suggest the involvement of a fast updating mechanism that predictively influences visual perception after an eye movement. The temporal characteristics of this mechanism are compatible with the rate at which saccadic eye movements are typically observed in natural viewing.

Low-Level Visual Information Is Maintained across Saccades, Allowing for a Postsaccadic Handoff between Visual Areas.

Experience seems continuous and detailed despite saccadic eye movements changing retinal input several times per second. There is debate whether neural signals related to updating across saccades contain information about stimulus features, or only location pointers without visual details. We investigated the time course of low-level visual information processing across saccades by decoding the spatial frequency of a stationary stimulus that changed from one visual hemifield to the other because of a horizontal saccadic eye movement. We recorded magnetoencephalography while human subjects (both sexes) monitored the orientation of a grating stimulus, making spatial frequency task irrelevant. Separate trials, in which subjects maintained fixation, were used to train a classifier, whose performance was then tested on saccade trials. Decoding performance showed that spatial frequency information of the presaccadic stimulus remained present for ∼200 ms after the saccade, transcending retinotopic specificity. Postsaccadic information ramped up rapidly after saccade offset. There was an overlap of over 100 ms during which decoding was significant from both presaccadic and postsaccadic processing areas. This suggests that the apparent richness of perception across saccades may be supported by the continuous availability of low-level information with a "soft handoff" of information during the initial processing sweep of the new fixation.SIGNIFICANCE STATEMENT Saccades create frequent discontinuities in visual input, yet perception appears stable and continuous. How is this discontinuous input processed resulting in visual stability? Previous studies have focused on presaccadic remapping. Here we examined the time course of processing of low-level visual information (spatial frequency) across saccades with magnetoencephalography. The results suggest that spatial frequency information is not predictively remapped but also is not discarded. Instead, they suggest a soft handoff over time between different visual areas, making this information continuously available across the saccade. Information about the presaccadic stimulus remains available, while the information about the postsaccadic stimulus has also become available. The simultaneous availability of both the presaccadic and postsaccadic information could enable rich and continuous perception across saccades.

When two worlds collide: the influence of an obstacle in peripersonal space on multisensory encoding.

Multisensory coding of the space surrounding our body, the peripersonal space, is crucial for motor control. Recently, it has been proposed that an important function of multisensory coding is that it allows anticipation of the tactile consequences of contact with a nearby object. Indeed, performing goal-directed actions (i.e. pointing and grasping) induces a continuous visuotactile remapping as a function of on-line sensorimotor requirements. Here, we investigated whether visuotactile remapping can be induced by obstacles, e.g. objects that are not the target of the grasping movement. In the current experiment, we used a cross-modal obstacle avoidance paradigm, in which participants reached past an obstacle to grasp a second object. Participants indicated the location of tactile targets delivered to the hand during the grasping movement, while a visual cue was sometimes presented simultaneously on the to-be-avoided object. The tactile and visual stimulation was triggered when the reaching hand passed a position that was drawn randomly from a continuous set of predetermined locations (between 0 and 200 mm depth at 5 mm intervals). We observed differences in visuotactile interaction during obstacle avoidance dependent on the location of the stimulation trigger: visual interference was enhanced for tactile stimulation that occurred when the hand was near the to-be-avoided object. We show that to-be-avoided obstacles, which are relevant for action but are not to-be-interacted with (as the terminus of an action), automatically evoke the tactile consequences of interaction. This shows that visuotactile remapping extends to obstacle avoidance and that this process is flexible.

How retaining objects containing multiple features in visual working memory regulates the priority for access to visual awareness.

The content of visual working memory influences the access to visual awareness. Thus far, research has focused on retention of a single feature, whereas memoranda in real life typically contain multiple features. Here, we intermixed a delayed match-to-sample task to manipulate VWM content, and a breaking Continuous Flash Suppression (b-CFS) task to measure prioritization for visual awareness. Observers memorized either the color (Exp. 1), the shape (Exp. 2) or both the features (Exp. 3) of an item and indicated the location of a suppressed target. We observed that color-matching targets broke suppression faster than color-mismatching targets both when color was memory relevant or irrelevant. Shape only impacted priority for visual awareness through an interaction with color. We conclude that: (1) VWM can regulate the priority of visual information to access visual awareness along a single feature dimension; (2) different features of a memorandum vary in their potency to impact access to visual awareness, and the more dominant feature may even suppress the effect of the less dominant feature; (3) even stimuli that match an irrelevant feature dimension of the memorandum can be prioritized for visual awareness.

The development of retro-cue benefits with extensive practice: Implications for capacity estimation and attentional states in visual working memory.

Accessing the contents of visual short-term memory (VSTM) is compromised by information bottlenecks and visual interference between memorization and recall. Retro-cues, displayed after the offset of a memory stimulus and prior to the onset of a probe stimulus, indicate the test item and improve performance in VSTM tasks. It has been proposed that retro-cues aid recall by transferring information from a high-capacity memory store into visual working memory (multiple-store hypothesis). Alternatively, retro-cues could aid recall by redistributing memory resources within the same (low-capacity) working memory store (single-store hypothesis). If retro-cues provide access to a memory store with a capacity exceeding the set size, then, given sufficient training in the use of the retro-cue, near-ceiling performance should be observed. To test this prediction, 10 observers each performed 12 hours across 8 sessions in a retro-cue change-detection task (40,000+ trials total). The results provided clear support for the single-store hypothesis: retro-cue benefits (difference between a condition with and without retro-cues) emerged after a few hundred trials and then remained constant throughout the testing sessions, consistently improving performance by two items, rather than reaching ceiling performance. Surprisingly, we also observed a general increase in performance throughout the experiment in conditions with and without retro-cues, calling into question the generalizability of change-detection tasks in assessing working memory capacity as a stable trait of an observer (data and materials are available at osf.io/9xr82 and github.com/paulzerr/retrocues). In summary, the present findings suggest that retro-cues increase capacity estimates by redistributing memory resources across memoranda within a low-capacity working memory store.

Decoding binary decisions under differential target probabilities from pupil dilation: A random forest approach.

Although our pupils slightly dilate when we look at an intended target, they do not when we look at irrelevant distractors. This finding suggests that it may be possible to decode the intention of an observer, understood as the outcome of implicit covert binary decisions, from the pupillary dynamics over time. However, few previous works have investigated the feasibility of this approach and the few that did, did not control for possible confounds such as motor-execution, changes in brightness, or target and distractor probability. We report on our efforts to decode intentions from pupil dilation obtained under strict experimental control on a single trial basis using a machine learning approach. The basis for our analyses are data of 69 participants who looked at letters that needed to be selected with stimulus probabilities that varied systematically in a blockwise manner (n = 19,417 trials). We confirm earlier findings that pupil dilation is indicative of intentions and show that these can be decoded with a classification performance of up to 76% area under the curve for receiver operating characteristic curves if targets are rarer than distractors. To better understand which characteristics of the pupillary signal are most informative, we finally compare relative feature importances. The first derivative of pupil size changes was found to be most relevant, allowing us to decode intention within only about 800 ms of trial onset. Taken together, our results provide credible insights into the potential of decoding intentions from pupil dilation and may soon form the basis for new applications in visual search, gaze-based interaction, or human-robot interaction.

No evidence for mnemonic modulation of interocularly suppressed visual input.

Visual working memory (VWM) allows for keeping visual information available for upcoming goal-directed behavior, while new visual input is processed concurrently. Interactions between the mnemonic and perceptual systems cause VWM to affect the processing of visual input in a content-specific manner: visual input that is initially suppressed from consciousness is detected faster when it matches rather than mismatches the content of VWM. It is currently under debate whether such mnemonic influences on perception occur prior to or after conscious access. To address this issue, we investigated whether VWM content modulates the neural response to visual input that remains suppressed from consciousness. We measured fMRI responses to interocularly suppressed stimuli in 20 human participants performing a delayed match-to-sample task: Participants were retro-cued to memorize one of two geometrical shapes for subsequent recognition. During retention, an interocularly suppressed peripheral stimulus (the probe) was briefly presented, which was either of the cued (memorized) or uncued (not memorized) shape category. We found no evidence that VWM content modulated the neural response to the probe. Substantial evidence for the absence of this modulation was found despite leveraging a highly liberal analysis approach: (1) selecting regions of interest that were particularly prone to detecting said modulation, and (2) using directional Bayesian tests favoring the presence of the hypothesized modulation. We did observe faster detection of memory-matching compared to memory-mismatching probes in a behavioral control experiment, thus validating the stimulus set. We conclude that VWM impacts the processing of visual input only once suppression is mostly alleviated.

Two hands are better than one: Perceptual benefits by bimanual movements.

Before looking at or reaching for an object, the focus of attention is first allocated to the movement object. Here we investigated whether the strength of these pre-motor shifts of attention cumulates if an object is targeted by multiple effectors (eyes and hands). A total of 29 participants were tested on a visuomotor task. They were cued to move gaze, the left hand, right hand, or both (one to three effectors) to a common object or to different peripheral objects. Before the movements, eight possible objects briefly changed form, of which one was a distinct probe. Results showed that the average recognition of the probe's identity change increased as more effectors targeted this object. For example, performance was higher when two hands as compared to one hand were moved to the probe. This effect remained evident despite the detrimental effect on performance of the increase in motor task complexity of moving two hands as compared to one hand. The accumulation of recognition improvements as a function of the number of effectors that successfully target the probe points at parallel and presumably independent mechanisms for hand- and eye-coordination that evoke pre-motor shifts of attention.