Action consequences guide the use of visual working memory.

Visual working memory (VWM) is a store for temporary maintenance of visual information. It is often disregarded, though, that information is typically stored to enable actions. Therefore, the context of these actions is of great importance for how VWM is used. Here, we questioned whether the severity of the consequence of an action might affect how well information is memorized, and how cautiously it is utilized. We employed an (online) copying task, in which participants reproduced an example display comprised of six items in a grid, using a pool of items. Crucially, we manipulated the severity of penalties: participants had to wait 5 (high) or 0.5 (low error cost) s after an error. Additionally, we manipulated the accessibility of task-relevant information (a well-studied manipulation in this paradigm): participants had to wait 5 (high) or 0.5 (low sampling cost) s to inspect the example. Our results show that with higher error cost the number of inspections remained comparable, but the number of errors decreased. Furthermore, they show that with higher sampling cost the number of inspections halved, and the number of errors increased. Thus, more severe action consequences increase the reluctance to act on uncertain information in VWM, but do not lead to more attempts to store information in VWM. We conclude that, in contrast to the effect of the accessibility of information, action consequences do not affect how well information is memorized, but affect how cautiously this stored information is utilized. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Mountains of memory in a sea of uncertainty: Sampling the external world despite useful information in visual working memory.

A large part of research on visual working memory (VWM) has traditionally focused on estimating its maximum capacity. Yet, humans rarely need to load up their VWM maximally during natural behavior, since visual information often remains accessible in the external world. Recent work, using paradigms that take into account the accessibility of information in the outside world, has indeed shown that observers utilize only one or two items in VWM before sampling from the external world again. One straightforward interpretation of this finding is that, in daily behavior, much fewer items are memorized than the typically reported capacity limits. Here, we first investigate whether this lower reliance on VWM when information is externally accessible might instead reflect resampling before VWM is actually depleted. To this aim we devised an online task, in which participants copied a model (six items in a 4x4 grid; always accessible) in an adjacent empty 4x4 grid. A key aspect of our paradigm is that we (unpredictably) interrupted participants just before inspection of the model with a 2-alternative-forced-choice (2-AFC) question, probing their VWM content. Critically, we observed above-chance performance on probes appearing just before model inspection. This finding shows that the external world was resampled, despite VWM still containing relevant information. We then asked whether increasing the cost of sampling causes participants to load up more information in VWM or, alternatively, to squeeze out more information from VWM (at the cost of making more errors). To manipulate the cost of resampling, we made it more difficult (specifically, more time-consuming) to access the model. We show that with increased cost of accessing the model (which lead to fewer, but longer model inspections), participants could place more items correctly immediately after sampling, and they kept attempting to place items for longer after their first error. These findings demonstrate that participants both encoded more information in VWM and made attempts to squeeze out more information from VWM when sampling became more costly. We argue that human observers constantly evaluate how certain they are of their VWM contents, and only use that VWM content of which their certainty exceeds a context-dependent "action threshold". This threshold, in turn, depends on the trade-off between the cost of resampling and the benefits of making an action. We argue that considering the interplay between the available VWM contents and a context-dependent action threshold, is key for reconciling the traditional VWM literature with VWM use in our day-to-day behavior.

Prioritization of emotional faces is not driven by emotional content.

Emotional faces have prioritized access to visual awareness. However, studies concerned with what expressions are prioritized most are inconsistent and the source of prioritization remains elusive. Here we tested the predictive value of spatial frequency-based image-features and emotional content, the sub-part of the image content that signals the emotional expression of the actor in the image as opposed to the image content irrelevant for the emotional expression, for prioritization for awareness. Participants reported which of two faces (displaying a combination of angry, happy, and neutral expressions), that were temporarily suppressed from awareness, was perceived first. Even though the results show that happy expressions were prioritized for awareness, this prioritization was driven by the contrast energy of the images. In fact, emotional content could not predict prioritization at all. Our findings show that the source of prioritization for awareness is not the information carrying the emotional content. We argue that the methods used here, or similar approaches, should become standard practice to break the chain of inconsistent findings regarding emotional superiority effects that have been part of the field for decades.

Vestibular and active self-motion signals drive visual perception in binocular rivalry.

Multisensory integration helps the brain build reliable models of the world and resolve ambiguities. Visual interactions with sound and touch are well established but vestibular influences on vision are less well studied. Here, we test the vestibular influence on vision using horizontally opposed motions presented one to each eye so that visual perception is unstable and alternates irregularly. Passive, whole-body rotations in the yaw plane stabilized visual alternations, with perceived direction oscillating congruently with rotation (leftward motion during leftward rotation, and vice versa). This demonstrates a purely vestibular signal can resolve ambiguous visual motion and determine visual perception. Active self-rotation following the same sinusoidal profile also entrained vision to the rotation cycle - more strongly and with a lesser time lag, likely because of efference copy and predictive internal models. Both experiments show that visual ambiguity provides an effective paradigm to reveal how vestibular and motor inputs can shape visual perception.

Unpredictive linguistic verbal cues accelerate congruent visual targets into awareness in a breaking continuous flash suppression paradigm.

One of the most influential ideas within the domain of cognition is that of embodied cognition, in which the experienced world is the result of an interplay between an organism's physiology, sensorimotor system, and its environment. An aspect of this idea is that linguistic information activates sensory representations automatically. For example, hearing the word 'red' would automatically activate sensory representations of this color. But does linguistic information prioritize access to awareness of congruent visual information? Here, we show that linguistic verbal cues accelerate matching visual targets into awareness by using a breaking continuous flash suppression paradigm. In a speeded reaction time task, observers heard spoken color labels (e.g., red) followed by colored targets that were either congruent (red), incongruent (green), or neutral (a neutral noncolor word) with respect to the labels. Importantly, and in contrast to previous studies investigating a similar question, the incidence of congruent trials was not higher than that of incongruent trials. Our results show that RTs were selectively shortened for congruent verbal-visual pairings, and that this shortening occurred over a wide range of cue-target intervals. We suggest that linguistic verbal information preactivates sensory representations, so that hearing the word 'red' preactivates (visual) sensory information internally.

The priority for access to awareness of information matching VWM is mirror-invariant.

Previous studies suggest that 1) storing a visual representation of an item in visual working memory (VWM) prioritizes access to visual awareness for this item and that 2) VWM can contain representations of bound items instead of separate features. It is currently unclear whether VWM affects access to visual awareness at the individual feature level, the conjunction of multiple features level or the object level. To investigate this question, we conducted a series of experiments in which we combined a delayed match to sample task with a breaking Continuous Flash Suppression (b-CFS) task. On each trial, subjects memorized an object consisting of a disk with two halves with different colors for the later recall test and, between them, had to detect the location of a target initially presented under suppression. We varied the congruence in colors between the memory representation and to-be-detected target. Our results show that memory congruent objects (consisting of a conjunction of features) break CFS faster than memory incongruent objects. Interestingly, we also observe this congruence effect when we presented the memorized object in a horizontally-mirrored configuration of colors. However, we do not observe a faster effect when the target shares only a single feature of a memorized object (semi-congruent) or when the memory congruent target is rotated by 90°. Our results suggest that VWM prioritizes access to visual awareness for complex visual memoranda for which the spatial lay-out of the individual features does not need to exactly match the lay-out of the memoranda.

How retaining objects containing multiple features in visual working memory regulates the priority for access to visual awareness.

The content of visual working memory influences the access to visual awareness. Thus far, research has focused on retention of a single feature, whereas memoranda in real life typically contain multiple features. Here, we intermixed a delayed match-to-sample task to manipulate VWM content, and a breaking Continuous Flash Suppression (b-CFS) task to measure prioritization for visual awareness. Observers memorized either the color (Exp. 1), the shape (Exp. 2) or both the features (Exp. 3) of an item and indicated the location of a suppressed target. We observed that color-matching targets broke suppression faster than color-mismatching targets both when color was memory relevant or irrelevant. Shape only impacted priority for visual awareness through an interaction with color. We conclude that: (1) VWM can regulate the priority of visual information to access visual awareness along a single feature dimension; (2) different features of a memorandum vary in their potency to impact access to visual awareness, and the more dominant feature may even suppress the effect of the less dominant feature; (3) even stimuli that match an irrelevant feature dimension of the memorandum can be prioritized for visual awareness.

Saccades reset the priority of visual information to access awareness.

Subjectively, we experience a stable representation of the outside world across saccades. Although previous studies have reported that presaccadically acquired visual information influences postsaccadic perception, whether such information's priority to access visual awareness is either reset by each saccade or continuous across saccades remains unclear. To investigate this issue, we combined a breaking continuous flash suppression (b-CFS) with a saccade task. Before each saccade, a grating was presented in the peripheral visual field under suppression. After the saccade, the same grating was again presented under suppression at either the retinotopically matched, the spatiotopically matched, or a control location. By measuring the duration of the grating to break through CFS into awareness after a saccade, we could compare the breakthrough times across stimuli presented at the different locations. No difference in the reaction times between the spatiotopic and control location was observed, indicating that a saccade resets the buildup of an object's priority to access visual awareness. However, a longer breakthrough time was observed for the retinotopic as compared to the control location, suggesting that a form of retinotopic adaptation to the grating suppressed the priority to access visual awareness after a saccade.

Adaptation to transients disrupts spatial coherence in binocular rivalry.

When one eye is presented with an image that is distinct from the image presented to the other eye, the eyes start to rival and suppress each other's image. Binocular rivalry leads to perceptual alternations between the images of each eye, during which only one of the images is perceived at a time. However, when the eyes exert weak and shallow suppression, participants tend to perceive both images intermixed more often. A recent study proposed that the precedence of mixed percepts positively correlates with the degree of adaptation to conflict between the eyes. However, this study neglected the role of visual transients, which covaried with the degree of conflict in the stimulus design. Here we report that not the conflict between the eyes but prolonged and repeated observations of strong visual transients cause participants to report more mixed percepts. We conclude that visual transients, such as sudden changes in contrast, draw attention, strengthen both eyes' image representations, and facilitate the adaptation to interocular suppression, which consequentially disrupts the spatial coherence in binocular rivalry. This finding is relevant to virtual- and augmented reality for which it is crucial to design stereoscopic environments in which binocular rivalry is limited.

No evidence for mnemonic modulation of interocularly suppressed visual input.

Visual working memory (VWM) allows for keeping visual information available for upcoming goal-directed behavior, while new visual input is processed concurrently. Interactions between the mnemonic and perceptual systems cause VWM to affect the processing of visual input in a content-specific manner: visual input that is initially suppressed from consciousness is detected faster when it matches rather than mismatches the content of VWM. It is currently under debate whether such mnemonic influences on perception occur prior to or after conscious access. To address this issue, we investigated whether VWM content modulates the neural response to visual input that remains suppressed from consciousness. We measured fMRI responses to interocularly suppressed stimuli in 20 human participants performing a delayed match-to-sample task: Participants were retro-cued to memorize one of two geometrical shapes for subsequent recognition. During retention, an interocularly suppressed peripheral stimulus (the probe) was briefly presented, which was either of the cued (memorized) or uncued (not memorized) shape category. We found no evidence that VWM content modulated the neural response to the probe. Substantial evidence for the absence of this modulation was found despite leveraging a highly liberal analysis approach: (1) selecting regions of interest that were particularly prone to detecting said modulation, and (2) using directional Bayesian tests favoring the presence of the hypothesized modulation. We did observe faster detection of memory-matching compared to memory-mismatching probes in a behavioral control experiment, thus validating the stimulus set. We conclude that VWM impacts the processing of visual input only once suppression is mostly alleviated.

Hide and seek: Directing top-down attention is not sufficient for accelerating conscious access.

At any moment in time, we have a single conscious visual experience representing a minute part of our visual world. As such, the visual input stimulating our retinae is in continuous competition for reaching conscious access. Many complex cognitive operations can only be applied to consciously accessible visual information, thereby raising the question whether humans have the ability to select which parts of their visual input reaches consciousness. Top-down attention allows humans to flexibly assign more processing resources to certain parts of our visual input, making it a likely mechanism to volitionally bias conscious access. Here, we investigated whether directing top-down attention to a particular location or feature accelerates conscious access of an initially suppressed visual stimulus at the attended location, or of the attended feature. We instructed participants to attend a spatial location (Experiment 1) or color (Experiment 2) for a speeded discrimination task, using a highly predictive cue. The predictive cues were highly effective in prompting sustained attention towards the cued location or color, as evidenced by faster discrimination of cued relative to uncued targets. We simultaneously measured detection times to interocularly suppressed probes that were either of the cued (i.e., attended) color/location or not, and were visually distinct from the targets used for the discrimination task. Despite our successful manipulation of top-down attention, suppressed probes were not released from suppression faster when they were presented at the attended location, or in the attended color. In contrast, when observers were cued to attend a color for locating targets of an ill-defined shape (inciting a broader attentional template), we did observe faster conscious access of probes in the attended color (Experiment 3). We discuss our findings in light of the specificity of attentional templates, and the inherent limitations that this poses for top-down attentional biases on conscious access.

Visual working memory and saliency independently influence the priority for access to visual awareness.

Both visual working memory (VWM) and visual saliency influence sensory processing, as is evident from research on visual attention and visual awareness. It is generally observed that items that are memorized or salient receive priority in visual search and in the access to awareness. Here we investigate whether these two factors interact and together boost access to visual awareness more than each factor independently. In the present experiment, we manipulated the VWM relevance and saliency of an item through a color memorization task and color uniqueness, respectively. We applied continuous flash suppression (CFS) to suppress items from visual awareness. The color of the suppressed items could either be congruent or incongruent with the memorized color, and either stood out from its surrounding distractors (salient pop out) or not. The item's priority for visual awareness was measured by measuring the time it took for an item to "break" into awareness. We first show that VWM relevance and visual saliency each shortened the time needed for an item to access awareness. More interestingly, the combined effect of VWM and visual salience was additive; that is, items that were both congruent and salient broke into visual awareness even faster. A race model further suggests that the interaction between these two mechanisms can be explained by statistical facilitation. Thus, VWM and saliency influence the priority to access visual awareness independently.

Prospectively reinstated memory drives conscious access of matching visual input.

Maintaining information in visual working memory (VWM) biases attentional selection of concurrent visual input, by favoring VWM-matching over VWM-mismatching visual input. Recently, it was shown that this bias disappears when the same item is memorized on consecutive occasions (as memoranda presumably transit from VWM to long-term memory), but reemerges when observers anticipate to memorize a novel item on a subsequent trial. Here, we aimed to conceptually replicate and extend this intriguing finding, by investigating whether prospectively reinstated memory drives conscious access of memory-matching visual input. We measured the time it took for participants to detect interocularly suppressed target stimuli, which were either from the same color category as a concurrently memorized color or not. Our results showed that the advantage of memory-matching targets in overcoming suppression progresses non-monotonically across consecutive memorizations of the same color ('repetitions'): the advantage for memory-matching visual input initially declined to asymptote, before being fully revived on the last repetition. This revival was not observed in a control experiment in which targets were not interocularly suppressed. The results suggest that, as observers anticipate to memorize a novel item imminently, VWM usage is prospectively reinstated, causing memory-matching visual input to gain accelerated access to consciousness again.

The duration aftereffect does not reflect adaptation to perceived duration.

Recent studies have provided evidence for a role of duration-tuned channels in the encoding of duration. Duration encoding in these channels is thought to reflect the time between responses to the onset and offset of an event. This notion is in apparent conflict with studies that demonstrate that the perceived duration of an event can vary independently from the time separating its perceived onset and offset. Instead, these studies suggest that duration encoding is sensitive to other temporal aspects of a sensory event. In the current study, we investigated whether duration-tuned channels encode duration based on the time between the on- and offset of an event (onset-offset duration), or if they encode a duration corresponding to the perceived duration of that event. We used a duration illusion to dissociate onset-offset duration and perceived duration and measured whether repeated exposure to illusion-inducing stimuli caused adaptation to the onset-offset duration or the perceived duration of these illusion-inducing stimuli. We report clear evidence for adaptation to the onset-offset duration of illusion-inducing stimuli. This finding supports the notion that duration-tuned mechanisms respond to the time between the onset and offset of an event, without necessarily reflecting the duration perceived, and eventually reported by the participant. Implications for the duration channel model and the mechanisms underlying duration illusions are discussed.

Assessing the generalizability of eye dominance across binocular rivalry, onset rivalry, and continuous flash suppression.

It is commonly assumed that one eye is dominant over the other eye. Eye dominance is most frequently determined by using the hole-in-the-card test. However, it is currently unclear whether eye dominance as determined by the hole-in-the-card test (so-called sighting eye dominance) generalizes to tasks involving interocular conflict (engaging sensory eye dominance). We therefore investigated whether sighting eye dominance is linked to sensory eye dominance in several frequently used paradigms that involve interocular conflict. Eye dominance was measured by the hole-in-the-card test, binocular rivalry, and breaking continuous flash suppression (b-CFS). Relationships between differences in eye dominance were assessed using Bayesian statistics. Strikingly, none of the three interocular conflict tasks yielded a difference in perceptual report between eyes when comparing the dominant eye with the nondominant eye as determined by the hole-in-the-card test. From this, we conclude that sighting eye dominance is different from sensory eye dominance. Interestingly, eye dominance of onset rivalry correlated with that of ongoing rivalry but not with that of b-CFS. Hence, we conclude that b-CFS reflects a different form of eye dominance than onset and ongoing rivalry. In sum, eye dominance seems to be a multifaceted phenomenon, which is differently expressed across interocular conflict paradigms. Finally, we highly discourage using tests measuring sighting eye dominance to determine the dominant eye in a subsequent experiment involving interocular conflict. Rather, we recommend that whenever experimental manipulations require a priori knowledge of eye dominance, eye dominance should be determined using pretrials of the same task that will be used in the main experiment.

Cortical depth dependent population receptive field attraction by spatial attention in human V1.

Visual spatial attention concentrates neural resources at the attended location. Recently, we demonstrated that voluntary spatial attention attracts population receptive fields (pRFs) toward its location throughout the visual hierarchy. Theoretically, both a feed forward or feedback mechanism could underlie pRF attraction in a given cortical area. Here, we use sub-millimeter ultra-high field functional MRI to measure pRF attraction across cortical depth and assess the contribution of feed forward and feedback signals to pRF attraction. In line with previous findings, we find consistent attraction of pRFs with voluntary spatial attention in V1. When assessed as a function of cortical depth, we find pRF attraction in every cortical portion (deep, center and superficial), although the attraction is strongest in deep cortical portions (near the gray-white matter boundary). Following the organization of feed forward and feedback processing across V1, we speculate that a mixture of feed forward and feedback processing underlies pRF attraction in V1. Specifically, we propose that feedback processing contributes to the pRF attraction in deep cortical portions.

Attention-based perceptual learning does not affect access to awareness.

Visual information that is relevant for an observer gains prioritized access to awareness (Gayet, Van der Stigchel, & Paffen, 2014). Here we investigate whether information that was relevant for an extended duration is prioritized for access to awareness when it is no longer relevant. We applied a perceptual-learning paradigm, in which observers were trained for 3 days on a speed-discrimination task. This task used a stimulus consisting of two motion directions, of which one was relevant to the task and one irrelevant. Before and after training, we applied a motion-coherence task to validate whether perceptual learning took place, and a breaking continuous flash-suppression (b-CFS) paradigm to assess how training affected access to awareness. The results reveal that motion-coherence thresholds for the task-relevant motion direction selectively decreased after compared to before training, revealing that task-relevant perceptual learning took place. The results of the b-CFS task, however, reveal that access to awareness was not affected by task-relevant learning: Instead, detection times for motion undergoing CFS decreased, irrespective of its direction, after compared to before training. A follow-up experiment showed that the time to detect visual motion also decreased after 3 days without training, revealing that perceptual learning did not cause the general decrease in detection times. The latter is in line with results by Mastropasqua, Tse, and Turatto (2015) and has important consequences for studies applying b-CFS to assess access to awareness: Studies that intend to apply measurements involving b-CFS on different testing days should consider that breakthrough times will dramatically decrease from pre- to postmeasurement.

Attention Gates the Selective Encoding of Duration.

The abundance of temporal information in our environment calls for the effective selection and utilization of temporal information that is relevant for our behavior. Here we investigated whether visual attention gates the selective encoding of relevant duration information when multiple sources of duration information are present. We probed the encoding of duration by using a duration-adaptation paradigm. Participants adapted to two concurrently presented streams of stimuli with different durations, while detecting oddballs in one of the streams. We measured the resulting duration after-effect (DAE) and found that the DAE reflects stronger relative adaptation to attended durations, compared to unattended durations. Additionally, we demonstrate that unattended durations do not contribute to the measured DAE. These results suggest that attention plays a crucial role in the selective encoding of duration: attended durations are encoded, while encoding of unattended durations is either weak or absent.

Competitive interactions in visual working memory drive access to awareness.

Models of biased competition assume that pre-activating a visual representation in visual working memory (VWM) biases perception towards memory-matching objects. Consistent with this, it has been shown that targets suppressed by interocular competition gain prioritized access to awareness when they match VWM content. Thus far, these VWM biases during interocular suppression have been investigated with minimal levels of competition, as there was always only one target stimulus and observers only held a single item in VWM. In the current study we investigated how VWM-based modulation of access to awareness is influenced by a) multiple-item competition within the stimulus display and b) multiple-item competition within VWM. Using the method of breaking continuous flash suppression (b-CFS), we replicated the finding that information matching the content of VWM is released from interocular suppression faster than non-matching information. This VWM-based facilitation was significantly reduced, though still present, when VWM load increased from one to two items, demonstrating a clear competitive constraint on the top-down modulation by VWM. Furthermore, we manipulated inter-stimulus competition by varying the presence of distractors. When distractors were present, VWM-based facilitation was no longer specific to interocular suppression, but also occurred for monocular displays. The results demonstrate that VWM-based visual biases occur in response to competition, whether between or within the eyes, and reconcile findings from different paradigms.