Dissociable event-related potential modulations of intrinsic and extrinsic factors in temporal integration.

We investigated visual temporal integration, by which multiple stimuli appearing in rapid succession are perceived as a single event. Temporal integration not only depends intrinsically on the passage of time but also, extrinsically, on the number and distribution of successive stimuli that are presented across that time interval. Here, we used a missing element task to investigate intrinsic and extrinsic factors in temporal integration, by manipulating stimulus duration and number, respectively. We found that both contributed interactively to integration performance and that varying the information rate over time did not further modulate this pattern. Intrinsic and extrinsic factors had dissociable effects on the N1, N2, N2pc, and P3 components of the event-related potential, implicating unique contributions to perceptual discrimination, spatio-temporal grouping, attention, and response decision-making. Stimulus number-induced effects on the event-related potential also generally arose later than those of stimulus duration. The latter already modulated the amplitude of the N1 and the early phase of the N2pc, while the former did not. The collective results suggest that while both intrinsic and extrinsic factors drive temporal integration, they do so in different ways. This difference during integration may eventually be reflected in the way in which we perceive longer, episodic events.

Attentional blur and blink: Effects of adaptive attentional scaling on visual awareness.

Attentional scaling is a crucial mechanism that enables us to flexibly allocate our attention to larger or smaller regions in the visual field. Although previous studies have demonstrated the critical role of attentional scaling in visual processing, its impact on modulating visual awareness is not yet fully understood. This study investigates the adaptive control of attentional scaling and its influence on visual awareness in an attentional blink paradigm. Participants were required to attend to the first target's location, which was manipulated either session-wise, trial-wise, or such that it could be learned across a block of trials. Discrete, all-or-none, awareness was expected when attention was allocated to a narrow area, while gradual awareness was expected when attention was allocated to a larger area. We used mixture modeling to assess second target awareness across these different attentional scales. The results revealed that participants could adaptively control their attentional scale both across stable sessions, and through (implicit) statistical learning in blocks of successive trials. This produced gradual perceptual awareness when the participants adopted a broad attentional scale, causing an attentional "blur". However, trial-wise cues did not allow for attentional scaling, resulting in more discrete target perception overall, and an attentional "blink". We conclude that the attentional scale is to some extent under adaptive control during the attentional blink/blur, where it can produce qualitatively different modes of perceptual awareness.

Impulse perturbation reveals cross-modal access to sensory working memory through learned associations.

We investigated if learned associations between visual and auditory stimuli can afford full cross-modal access to working memory. Previous research using the impulse perturbation technique has shown that cross-modal access to working memory is one-sided; visual impulses reveal both auditory and visual memoranda, but auditory impulses do not seem to reveal visual memoranda (Wolff et al., 2020b). Our participants first learned to associate six auditory pure tones with six visual orientation gratings. Next, a delayed match-to-sample task for the orientations was completed, while EEG was recorded. Orientation memories were recalled either via their learned auditory counterpart, or were visually presented. We then decoded the orientation memories from the EEG responses to both auditory and visual impulses presented during the memory delay. Working memory content could always be decoded from visual impulses. Importantly, through recall of the learned associations, the auditory impulse also evoked a decodable response from the visual WM network, providing evidence for full cross-modal access. We also observed that after a brief initial dynamic period, the representational codes of the memory items generalized across time, as well as between perceptual maintenance and long-term recall conditions. Our results thus demonstrate that accessing learned associations in long-term memory provides a cross-modal pathway to working memory that seems to be based on a common coding scheme.

Adaptive Encoding Speed in Working Memory.

Humans can adapt when complex patterns unfold at a faster or slower pace, for instance when remembering a grocery list that is dictated at an increasingly fast rate. Integrating information over such timescales crucially depends on working memory, but although recent findings have shown that working memory capacity can be flexibly adapted, such adaptations have not yet been demonstrated for encoding speed. In a series of experiments, we found that young adults encoded at a faster rate when they were adapted to overall and recent stimulus duration. Interestingly, our participants were unable to use explicit cues to speed up encoding, even though these cues were objectively more informative than statistical information. Our findings suggest that adaptive tuning of encoding speed in working memory is a fundamental but largely implicit mechanism underlying our ability to keep up with the pace of our surroundings.

Drifting codes within a stable coding scheme for working memory.

Working memory (WM) is important to maintain information over short time periods to provide some stability in a constantly changing environment. However, brain activity is inherently dynamic, raising a challenge for maintaining stable mental states. To investigate the relationship between WM stability and neural dynamics, we used electroencephalography to measure the neural response to impulse stimuli during a WM delay. Multivariate pattern analysis revealed representations were both stable and dynamic: there was a clear difference in neural states between time-specific impulse responses, reflecting dynamic changes, yet the coding scheme for memorised orientations was stable. This suggests that a stable subcomponent in WM enables stable maintenance within a dynamic system. A stable coding scheme simplifies readout for WM-guided behaviour, whereas the low-dimensional dynamic component could provide additional temporal information. Despite having a stable subspace, WM is clearly not perfect-memory performance still degrades over time. Indeed, we find that even within the stable coding scheme, memories drift during maintenance. When averaged across trials, such drift contributes to the width of the error distribution.

Object-based visual working memory: an object benefit for equidistant memory items presented within simple contours.

Previous research has shown that more information can be stored in visual working memory (VWM) when multiple items belong to the same object. Here, in four experiments, we investigated the object effect on memory for spatially equidistant features by manipulating simple, task-irrelevant contours that combined these features. In Experiments 1, 3, and, 4, three grating orientations, and in Experiment 2, one color and two orientations, were presented simultaneously to be memorized. Mixture modeling was applied to estimate both the precision and the guess rates of recall errors. Overall results showed that two target features were remembered more accurately when both were part of the same object. Further analysis showed that the probability of recall increased in particular when both features were extracted from the same object. In Experiment 2, we found that the object effect was greater for features from orthogonal dimensions, but this came at the cost of lower memory precision. In Experiment 3, when we kept the locations of the features perfectly consistent over trials so that the participants could attend to these locations rather than the contour, we still found object benefits. Finally, in Experiment 4 when we manipulated the temporal order of the object and the memory features presentations, it was confirmed that the object benefit is unlikely to stem from the strategical usage of object information. These results suggested that the object benefit arises automatically, likely at an early perceptual level.

Acute effects of cocoa flavanols on visual working memory: maintenance and updating.

BACKGROUND: Consumption of cocoa flavanols may have acute physiological effects on the brain due to their ability to activate nitric oxide synthesis. Nitric oxide mediates vasodilation, which increases cerebral blood flow, and can also act as a neurotransmitter. OBJECTIVES: This study aimed to examine whether cocoa flavanols have an acute influence on visual working memory (WM). METHODS: Two separate randomised, double-blind, placebo-controlled, counterbalanced crossover experiments were conducted on normal healthy young adult volunteers (NExp1 = 48 and NExp2 = 32, gender-balanced). In these experiments, 415 mg of cocoa flavanols were administered to test their acute effects on visual working memory. In the first experiment, memory recall precision was measured in a task that required only passive maintenance of grating orientations in WM. In the second experiment, recall was measured after active updating (mental rotation) of WM contents. Habitual daily flavanols intake, body mass index, and gender were also considered in the analysis. RESULTS: The results suggested that neither passive maintenance in visual WM nor active updating of WM were acutely enhanced by consumption of cocoa flavanols. Exploratory analyses with covariates (body mass index and daily flavanols intake), and the between-subjects factor of gender also showed no evidence for effects of cocoa flavanols, neither in terms of reaction time, nor accuracy. CONCLUSIONS: Overall, cocoa flavanols did not improve visual working memory recall performance during maintenance, nor did it improve recall accuracy after memory updating.

Unimodal and Bimodal Access to Sensory Working Memories by Auditory and Visual Impulses.

It is unclear to what extent sensory processing areas are involved in the maintenance of sensory information in working memory (WM). Previous studies have thus far relied on finding neural activity in the corresponding sensory cortices, neglecting potential activity-silent mechanisms, such as connectivity-dependent encoding. It has recently been found that visual stimulation during visual WM maintenance reveals WM-dependent changes through a bottom-up neural response. Here, we test whether this impulse response is uniquely visual and sensory-specific. Human participants (both sexes) completed visual and auditory WM tasks while electroencephalography was recorded. During the maintenance period, the WM network was perturbed serially with fixed and task-neutral auditory and visual stimuli. We show that a neutral auditory impulse-stimulus presented during the maintenance of a pure tone resulted in a WM-dependent neural response, providing evidence for the auditory counterpart to the visual WM findings reported previously. Interestingly, visual stimulation also resulted in an auditory WM-dependent impulse response, implicating the visual cortex in the maintenance of auditory information, either directly or indirectly, as a pathway to the neural auditory WM representations elsewhere. In contrast, during visual WM maintenance, only the impulse response to visual stimulation was content-specific, suggesting that visual information is maintained in a sensory-specific neural network, separated from auditory processing areas.SIGNIFICANCE STATEMENT Working memory is a crucial component of intelligent, adaptive behavior. Our understanding of the neural mechanisms that support it has recently shifted: rather than being dependent on an unbroken chain of neural activity, working memory may rely on transient changes in neuronal connectivity, which can be maintained efficiently in activity-silent brain states. Previous work using a visual impulse stimulus to perturb the memory network has implicated such silent states in the retention of line orientations in visual working memory. Here, we show that auditory working memory similarly retains auditory information. We also observed a sensory-specific impulse response in visual working memory, while auditory memory responded bimodally to both visual and auditory impulses, possibly reflecting visual dominance of working memory.

A functional spiking-neuron model of activity-silent working memory in humans based on calcium-mediated short-term synaptic plasticity.

In this paper, we present a functional spiking-neuron model of human working memory (WM). This model combines neural firing for encoding of information with activity-silent maintenance. While it used to be widely assumed that information in WM is maintained through persistent recurrent activity, recent studies have shown that information can be maintained without persistent firing; instead, information can be stored in activity-silent states. A candidate mechanism underlying this type of storage is short-term synaptic plasticity (STSP), by which the strength of connections between neurons rapidly changes to encode new information. To demonstrate that STSP can lead to functional behavior, we integrated STSP by means of calcium-mediated synaptic facilitation in a large-scale spiking-neuron model and added a decision mechanism. The model was used to simulate a recent study that measured behavior and EEG activity of participants in three delayed-response tasks. In these tasks, one or two visual gratings had to be maintained in WM, and compared to subsequent probes. The original study demonstrated that WM contents and its priority status could be decoded from neural activity elicited by a task-irrelevant stimulus displayed during the activity-silent maintenance period. In support of our model, we show that it can perform these tasks, and that both its behavior as well as its neural representations are in agreement with the human data. We conclude that information in WM can be effectively maintained in activity-silent states by means of calcium-mediated STSP.

Visual and auditory temporal integration in healthy younger and older adults.

As people age, they tend to integrate successive visual stimuli over longer intervals than younger adults. It may be expected that temporal integration is affected similarly in other modalities, possibly due to general, age-related cognitive slowing of the brain. However, the previous literature does not provide convincing evidence that this is the case in audition. One hypothesis is that the primacy of time in audition attenuates the degree to which temporal integration in that modality extends over time as a function of age. We sought to settle this issue by comparing visual and auditory temporal integration in younger and older adults directly, achieved by minimizing task differences between modalities. Participants were presented with a visual or an auditory rapid serial presentation task, at 40-100 ms/item. In both tasks, two subsequent targets were to be identified. Critically, these could be perceptually integrated and reported by the participants as such, providing a direct measure of temporal integration. In both tasks, older participants integrated more than younger adults, especially when stimuli were presented across longer time intervals. This difference was more pronounced in vision and only marginally significant in audition. We conclude that temporal integration increases with age in both modalities, but that this change might be slightly less pronounced in audition.

Supplementation of gamma-aminobutyric acid (GABA) affects temporal, but not spatial visual attention.

In a randomized, double-blind, and placebo-controlled experiment, the acute effects of gamma-aminobutyric acid (GABA) supplementation on temporal and spatial attention in young healthy adults were investigated. A hybrid two-target rapid serial visual presentation task was used to measure temporal attention and integration. Additionally, a visual search task was used to measure the speed and accuracy of spatial attention. While temporal attention depends primarily on the distribution of limited attentional resources across time, spatial attention represents the engagement and disengagement by relevant and irrelevant stimuli across the visual field. Although spatial attention was unaffected by GABA supplementation altogether, we found evidence supporting improved performance in the temporal attention task. The attentional blink was numerically, albeit not significantly, attenuated at Lag 3, and significantly fewer order errors were committed at Lag 1, compared to the placebo condition. No effect was found on temporal integration rates. Although there is controversy about whether oral GABA can cross the blood-brain barrier, our results offer preliminary evidence that GABA intake might help to distribute limited attentional resources more efficiently, and can specifically improve the identification and ordering of visual events that occur in close temporal succession.

What You See Is What You Remember: Visual Chunking by Temporal Integration Enhances Working Memory.

Human memory benefits from information clustering, which can be accomplished by chunking. Chunking typically relies on expertise and strategy, and it is unknown whether perceptual clustering over time, through temporal integration, can also enhance working memory. The current study examined the attentional and working memory costs of temporal integration of successive target stimulus pairs embedded in rapid serial visual presentation. ERPs were measured as a function of behavioral reports: One target, two separate targets, or two targets reported as a single integrated target. N2pc amplitude, reflecting attentional processing, depended on the actual number of successive targets. The memory-related CDA and P3 components instead depended on the perceived number of targets irrespective of their actual succession. The report of two separate targets was associated with elevated amplitude, whereas integrated as well as actual single targets exhibited lower amplitude. Temporal integration thus provided an efficient means of processing sensory input, offloading working memory so that the features of two targets were consolidated and maintained at a cost similar to that of a single target.

Distortions of temporal integration and perceived order caused by the interplay between stimulus contrast and duration.

Stimulus contrast and duration effects on visual temporal integration and order judgment were examined in a unified paradigm. Stimulus onset asynchrony was governed by the duration of the first stimulus in Experiment 1, and by the interstimulus interval in Experiment 2. In Experiment 1, integration and order uncertainty increased when a low contrast stimulus followed a high contrast stimulus, but only when the second stimulus was 20 or 30ms. At 10ms duration of the second stimulus, integration and uncertainty decreased. Temporal order judgments at all durations of the second stimulus were better for a low contrast stimulus following a high contrast one. By contrast, in Experiment 2, a low contrast stimulus following a high contrast stimulus consistently produced higher integration rates, order uncertainty, and lower order accuracy. Contrast and duration thus interacted, breaking correspondence between integration and order perception. The results are interpreted in a tentative conceptual framework.

Illusions of integration are subjectively impenetrable: Phenomenological experience of Lag 1 percepts during dual-target RSVP.

We investigated the relationship between different kinds of target reports in a rapid serial visual presentation task, and their associated perceptual experience. Participants reported the identity of two targets embedded in a stream of stimuli and their associated subjective visibility. In our task, target stimuli could be combined together to form more complex ones, thus allowing participants to report temporally integrated percepts. We found that integrated percepts were associated with high subjective visibility scores, whereas reports in which the order of targets was reversed led to a poorer perceptual experience. We also found a reciprocal relationship between the chance of the second target not being reported correctly and the perceptual experience associated with the first one. Principally, our results indicate that integrated percepts are experienced as a unique, clear perceptual event, whereas order reversals are experienced as confused, similar to cases in which an entirely wrong response was given.

Adaptive forgetting speed in working memory.

Working memory is known to be capacity-limited and is therefore selective not only for what it encodes but also what it forgets. Explicit forgetting cues can be used effectively to free up capacity, but it is not clear how working memory adaptively forgets in the absence of explicit cues. An important implicit cue that may tune forgetting in working memory is the passage of time. When information becomes irrelevant more quickly, working memory should also forget information more quickly. In three delayed-estimation experiments, we systematically manipulated how probing probability changed as time passed on after encoding an item (i.e., the "probing hazard"). In some blocks, probing hazard decreased after encoding an item, requiring participants to only briefly retain the memory item. In other blocks, the probing hazard increased or stayed flat, as the retention interval was lengthened. In line with our hypothesis, we found that participants adapted their forgetting rate to the probing dynamics of the working memory task. When the memory item quickly became irrelevant ("decreasing" probing hazard), forgetting rate was higher than in blocks where probing hazard increased or stayed flat. The time course of these adaptations in forgetting implies a fast and flexible mechanism. Interestingly, participants could not explicitly report the order of conditions, suggesting forgetting is implicitly sped up. These findings suggest that implicit adaptations to the temporal structure of our environment tune forgetting speed in working memory, possibly contributing to the flexible allocation of limited working memory resources.

Maintenance of color memoranda in activity-quiescent working memory states: Evidence from impulse perturbation.

In the present study, we used an impulse perturbation method to probe working memory maintenance of colors in neurally active and activity-quiescent states, focusing on a set of pre-registered analyses. We analyzed the electroencephalograph (EEG) data of 30 participants who completed a delayed match-to-sample working memory task, in which one of the two items that were presented was retro-cued as task relevant. The analyses revealed that both cued and uncued colors were decodable from impulse-evoked activity, the latter in contrast to previous reports of working memory for orientation gratings. Decoding of colors from oscillations in the alpha band showed that cued items could be decoded therein whereas uncued items could not. Overall, the outcomes suggest that subtle differences exist between the representation of colors, and that of stimuli with spatial properties, but the present results also demonstrate that regardless of their specific neural state, both are accessible through visual impulse perturbation.

Attentional control and competition between episodic representations.

The relationship between attentional control and episodic representation was investigated in six experiments that employed a variant of the classic attentional blink paradigm. We introduced a task-irrelevant (unpredictive) color match between the first and second target stimulus in a three-stream rapid serial visual presentation task. When this match was present, the first target reliably elicited a priming benefit to the identification of the second, lateralized target. However, this was only the case when the identities of the targets did not belong to the same category (digits, letters, or symbols). When targets did belong to the same category, interference was observed instead of priming, particularly at Lag 1. Furthermore, when color was the target-defining feature, interference at Lag 1 gave way to priming at longer lags. The interference effect is attributed to partial overlap between competing episodic target representations, which affects the availability of their overlapping features for successive attentional selection in rapid serial visual presentation.

Task set flexibility and feature specificity modulate the limits of temporal attention.

The consequences of maintaining a task set in the context of the (speeded) attentional blink were investigated in a series of experiments. Observers were asked to either attend or ignore the first of two target stimuli (T1 and T2). The results showed that when T1 and T2 shared a task relevant feature that was unique to T2, but not to T1, a shallow attentional blink was observed, as well as a lack of Lag 1 sparing. In comparison, when the targets shared a feature that was uniquely task relevant to both targets, the blink could not be avoided. Conversely, when no feature was shared between targets, ignoring T1 was successful and virtually no attentional costs were apparent. A similar lack of costs was also observed when targets shared a task relevant feature that was unique to T1 but not to T2. Finally, matching the feature dimension of a target feature that was unique to T2, but not T1, also strongly attenuated the blink. However, it did not completely abolish Lag 1 sparing. The results are interpreted in the context of current models of the attentional blink.

Concealed identity information detection with pupillometry in rapid serial visual presentation.

The concealed information test (CIT) relies on bodily reactions to stimuli that are hidden in mind. However, people can use countermeasures, such as purposely focusing on irrelevant things, to confound the CIT. A new method designed to prevent countermeasures uses rapid serial visual presentation (RSVP) to present stimuli on the fringe of awareness. Previous studies that used RSVP in combination with electroencephalography (EEG) showed that participants exhibit a clear reaction to their real first name, even when they try to prevent such a reaction (i.e., when their name is concealed information). Because EEG is not easily applicable outside the laboratory, we investigated here whether pupil size, which is easier to measure, can also be used to detect concealed identity information. In our first study, participants adopted a fake name, and searched for this name in an RSVP task, while their pupil sizes were recorded. Apart from this fake name, their real name and a control name also appeared in the task. We found pupil dilation in response to the task-irrelevant real name, as compared to control names. However, while most participants showed this effect qualitatively, it was not statistically significant for most participants individually. In a second study, we preregistered the proof-of-concept methodology and replicated the original findings. Taken together, our results show that the current RSVP task with pupillometry can detect concealed identity information at a group level. Further development of the method is needed to create a valid and reliable concealed identity information detector at the individual level.

Effects of gamma-aminobutyric acid on working memory and attention: A randomized, double-blinded, placebo-controlled, crossover trial.

BACKGROUND: γ-Aminobutyric acid (GABA) is a primary inhibitory neurotransmitter that plays a significant role in the central nervous system. Studies on both animals and humans show that GABA has the pharmacological potential for reducing the impact of cognitive disorders, as well as enhancing cognitive functions and mood. However, its specific effects on human attention and working memory have not yet been extensively studied. AIMS: In this randomized, double-blind, placebo-controlled, and crossover trial, we aimed to test whether the administration of 800 mg GABA, dissolved in a drink, acutely affected visual working memory (VWM) maintenance, as well as temporal and spatial attention in healthy adults. METHODS: The participants were 32 young adults (16 females and 16 males). Working memory recall precision, spatial attention and temporal attention were measured by a delayed match-to-sample task, a visual search (VS) task and a speeded rapid serial visual presentation task, respectively. Participants completed two experimental sessions (GABA and Placebo) in randomized and counterbalanced order. In each session, 45 min after administration of the drink, they completed all three aforementioned cognitive tasks. RESULTS: Linear mixed model analysis results showed that GABA increased VS time, compared to the placebo, but did not affect VS accuracy, temporal attention, nor VWM precision. CONCLUSIONS: The results suggest that GABA increases VS time but does not affect temporal attention and memory, and that previously reported effects on cognition might rely on other functions.