No evidence for mnemonic modulation of interocularly suppressed visual input.

Visual working memory (VWM) allows for keeping visual information available for upcoming goal-directed behavior, while new visual input is processed concurrently. Interactions between the mnemonic and perceptual systems cause VWM to affect the processing of visual input in a content-specific manner: visual input that is initially suppressed from consciousness is detected faster when it matches rather than mismatches the content of VWM. It is currently under debate whether such mnemonic influences on perception occur prior to or after conscious access. To address this issue, we investigated whether VWM content modulates the neural response to visual input that remains suppressed from consciousness. We measured fMRI responses to interocularly suppressed stimuli in 20 human participants performing a delayed match-to-sample task: Participants were retro-cued to memorize one of two geometrical shapes for subsequent recognition. During retention, an interocularly suppressed peripheral stimulus (the probe) was briefly presented, which was either of the cued (memorized) or uncued (not memorized) shape category. We found no evidence that VWM content modulated the neural response to the probe. Substantial evidence for the absence of this modulation was found despite leveraging a highly liberal analysis approach: (1) selecting regions of interest that were particularly prone to detecting said modulation, and (2) using directional Bayesian tests favoring the presence of the hypothesized modulation. We did observe faster detection of memory-matching compared to memory-mismatching probes in a behavioral control experiment, thus validating the stimulus set. We conclude that VWM impacts the processing of visual input only once suppression is mostly alleviated.

Visual working memory and saliency independently influence the priority for access to visual awareness.

Both visual working memory (VWM) and visual saliency influence sensory processing, as is evident from research on visual attention and visual awareness. It is generally observed that items that are memorized or salient receive priority in visual search and in the access to awareness. Here we investigate whether these two factors interact and together boost access to visual awareness more than each factor independently. In the present experiment, we manipulated the VWM relevance and saliency of an item through a color memorization task and color uniqueness, respectively. We applied continuous flash suppression (CFS) to suppress items from visual awareness. The color of the suppressed items could either be congruent or incongruent with the memorized color, and either stood out from its surrounding distractors (salient pop out) or not. The item's priority for visual awareness was measured by measuring the time it took for an item to "break" into awareness. We first show that VWM relevance and visual saliency each shortened the time needed for an item to access awareness. More interestingly, the combined effect of VWM and visual salience was additive; that is, items that were both congruent and salient broke into visual awareness even faster. A race model further suggests that the interaction between these two mechanisms can be explained by statistical facilitation. Thus, VWM and saliency influence the priority to access visual awareness independently.

Visual Working Memory Enhances the Neural Response to Matching Visual Input.

Visual working memory (VWM) is used to maintain visual information available for subsequent goal-directed behavior. The content of VWM has been shown to affect the behavioral response to concurrent visual input, suggesting that visual representations originating from VWM and from sensory input draw upon a shared neural substrate (i.e., a sensory recruitment stance on VWM storage). Here, we hypothesized that visual information maintained in VWM would enhance the neural response to concurrent visual input that matches the content of VWM. To test this hypothesis, we measured fMRI BOLD responses to task-irrelevant stimuli acquired from 15 human participants (three males) performing a concurrent delayed match-to-sample task. In this task, observers were sequentially presented with two shape stimuli and a retro-cue indicating which of the two shapes should be memorized for subsequent recognition. During the retention interval, a task-irrelevant shape (the probe) was briefly presented in the peripheral visual field, which could either match or mismatch the shape category of the memorized stimulus. We show that this probe stimulus elicited a stronger BOLD response, and allowed for increased shape-classification performance, when it matched rather than mismatched the concurrently memorized content, despite identical visual stimulation. Our results demonstrate that VWM enhances the neural response to concurrent visual input in a content-specific way. This finding is consistent with the view that neural populations involved in sensory processing are recruited for VWM storage, and it provides a common explanation for a plethora of behavioral studies in which VWM-matching visual input elicits a stronger behavioral and perceptual response.SIGNIFICANCE STATEMENT Humans heavily rely on visual information to interact with their environment and frequently must memorize such information for later use. Visual working memory allows for maintaining such visual information in the mind's eye after termination of its retinal input. It is hypothesized that information maintained in visual working memory relies on the same neural populations that process visual input. Accordingly, the content of visual working memory is known to affect our conscious perception of concurrent visual input. Here, we demonstrate for the first time that visual input elicits an enhanced neural response when it matches the content of visual working memory, both in terms of signal strength and information content.

Cortical depth dependent population receptive field attraction by spatial attention in human V1.

Visual spatial attention concentrates neural resources at the attended location. Recently, we demonstrated that voluntary spatial attention attracts population receptive fields (pRFs) toward its location throughout the visual hierarchy. Theoretically, both a feed forward or feedback mechanism could underlie pRF attraction in a given cortical area. Here, we use sub-millimeter ultra-high field functional MRI to measure pRF attraction across cortical depth and assess the contribution of feed forward and feedback signals to pRF attraction. In line with previous findings, we find consistent attraction of pRFs with voluntary spatial attention in V1. When assessed as a function of cortical depth, we find pRF attraction in every cortical portion (deep, center and superficial), although the attraction is strongest in deep cortical portions (near the gray-white matter boundary). Following the organization of feed forward and feedback processing across V1, we speculate that a mixture of feed forward and feedback processing underlies pRF attraction in V1. Specifically, we propose that feedback processing contributes to the pRF attraction in deep cortical portions.

Assessing the generalizability of eye dominance across binocular rivalry, onset rivalry, and continuous flash suppression.

It is commonly assumed that one eye is dominant over the other eye. Eye dominance is most frequently determined by using the hole-in-the-card test. However, it is currently unclear whether eye dominance as determined by the hole-in-the-card test (so-called sighting eye dominance) generalizes to tasks involving interocular conflict (engaging sensory eye dominance). We therefore investigated whether sighting eye dominance is linked to sensory eye dominance in several frequently used paradigms that involve interocular conflict. Eye dominance was measured by the hole-in-the-card test, binocular rivalry, and breaking continuous flash suppression (b-CFS). Relationships between differences in eye dominance were assessed using Bayesian statistics. Strikingly, none of the three interocular conflict tasks yielded a difference in perceptual report between eyes when comparing the dominant eye with the nondominant eye as determined by the hole-in-the-card test. From this, we conclude that sighting eye dominance is different from sensory eye dominance. Interestingly, eye dominance of onset rivalry correlated with that of ongoing rivalry but not with that of b-CFS. Hence, we conclude that b-CFS reflects a different form of eye dominance than onset and ongoing rivalry. In sum, eye dominance seems to be a multifaceted phenomenon, which is differently expressed across interocular conflict paradigms. Finally, we highly discourage using tests measuring sighting eye dominance to determine the dominant eye in a subsequent experiment involving interocular conflict. Rather, we recommend that whenever experimental manipulations require a priori knowledge of eye dominance, eye dominance should be determined using pretrials of the same task that will be used in the main experiment.

Attention-based perceptual learning does not affect access to awareness.

Visual information that is relevant for an observer gains prioritized access to awareness (Gayet, Van der Stigchel, & Paffen, 2014). Here we investigate whether information that was relevant for an extended duration is prioritized for access to awareness when it is no longer relevant. We applied a perceptual-learning paradigm, in which observers were trained for 3 days on a speed-discrimination task. This task used a stimulus consisting of two motion directions, of which one was relevant to the task and one irrelevant. Before and after training, we applied a motion-coherence task to validate whether perceptual learning took place, and a breaking continuous flash-suppression (b-CFS) paradigm to assess how training affected access to awareness. The results reveal that motion-coherence thresholds for the task-relevant motion direction selectively decreased after compared to before training, revealing that task-relevant perceptual learning took place. The results of the b-CFS task, however, reveal that access to awareness was not affected by task-relevant learning: Instead, detection times for motion undergoing CFS decreased, irrespective of its direction, after compared to before training. A follow-up experiment showed that the time to detect visual motion also decreased after 3 days without training, revealing that perceptual learning did not cause the general decrease in detection times. The latter is in line with results by Mastropasqua, Tse, and Turatto (2015) and has important consequences for studies applying b-CFS to assess access to awareness: Studies that intend to apply measurements involving b-CFS on different testing days should consider that breakthrough times will dramatically decrease from pre- to postmeasurement.

The Uniformity Illusion.

Vision in the fovea, the center of the visual field, is much more accurate and detailed than vision in the periphery. This is not in line with the rich phenomenology of peripheral vision. Here, we investigated a visual illusion that shows that detailed peripheral visual experience is partially based on a reconstruction of reality. Participants fixated on the center of a visual display in which central stimuli differed from peripheral stimuli. Over time, participants perceived that the peripheral stimuli changed to match the central stimuli, so that the display seemed uniform. We showed that a wide range of visual features, including shape, orientation, motion, luminance, pattern, and identity, are susceptible to this uniformity illusion. We argue that the uniformity illusion is the result of a reconstruction of sparse visual information (from the periphery) based on more readily available detailed visual information (from the fovea), which gives rise to a rich, but illusory, experience of peripheral vision.

Larger Stimuli Require Longer Processing Time for Perception.

The time it takes for a stimulus to reach awareness is often assessed by measuring reaction times (RTs) or by a temporal order judgement (TOJ) task in which perceived timing is compared against a reference stimulus. Dissociations of RT and TOJ have been reported earlier in which increases in stimulus intensity such as luminance intensity results in a decrease of RT, whereas perceived perceptual latency in a TOJ task is affected to a lesser degree. Here, we report that a simple manipulation of stimulus size has stronger effects on perceptual latency measured by TOJ than on motor latency measured by RT tasks. When participants were asked to respond to the appearance of a simple stimulus such as a luminance blob, the perceptual latency measured against a standard reference stimulus was up to 40 ms longer for a larger stimulus. In other words, the smaller stimulus was perceived to occur earlier than the larger one. RT on the other hand was hardly affected by size. The TOJ results were further replicated in a simultaneity judgement task, suggesting that the effects of size are not due to TOJ-specific response biases but more likely reflect an effect on perceived timing.

Predicting bias in perceived position using attention field models.

Attention is the mechanism through which we select relevant information from our visual environment. We have recently demonstrated that attention attracts receptive fields across the visual hierarchy (Klein, Harvey, & Dumoulin, 2014). We captured this receptive field attraction using an attention field model. Here, we apply this model to human perception: We predict that receptive field attraction results in a bias in perceived position, which depends on the size of the underlying receptive fields. We instructed participants to compare the relative position of Gabor stimuli, while we manipulated the focus of attention using exogenous cueing. We varied the eccentric position and spatial frequency of the Gabor stimuli to vary underlying receptive field size. The positional biases as a function of eccentricity matched the predictions by an attention field model, whereas the bias as a function of spatial frequency did not. As spatial frequency and eccentricity are encoded differently across the visual hierarchy, we speculate that they might interact differently with the attention field that is spatially defined.

Visual input that matches the content of visual working memory requires less (not faster) evidence sampling to reach conscious access.

The content of visual working memory (VWM) affects the processing of concurrent visual input. Recently, it has been demonstrated that stimuli are released from interocular suppression faster when they match rather than mismatch a color that is memorized for subsequent recall. In order to investigate the nature of the interaction between visual representations elicited by VWM and visual representations elicited by retinal input, we modeled the perceptual processes leading up to this difference in suppression durations. We replicated the VWM modulation of suppression durations, and fitted sequential sampling models (linear ballistic accumulators) to the response time data. Model comparisons revealed that the data was best explained by a decrease in threshold for visual input that matches the content of VWM. Converging evidence was obtained by fitting similar sequential sampling models (shifted Wald model) to published datasets. Finally, to confirm that the previously observed threshold difference reflected processes occurring before rather than after the stimuli were released from suppression, we applied the same procedure to the data of an experiment in which stimuli were not interocularly suppressed. Here, we found no decrease in threshold for stimuli that match the content of VWM. We discuss our findings in light of a preactivation hypothesis, proposing that matching visual input taps into the same neural substrate that is already activated by a representation concurrently maintained in VWM, thereby reducing its threshold for reaching visual awareness.

Cross-modal, bidirectional priming in grapheme-color synesthesia.

Grapheme-color synesthetes perceive achromatic graphemes to be inherently colored. In this study grapheme-color synesthetes and non-synesthetes discriminated (1) the color of visual targets presented along with aurally presented digit primes, and (2) the identity of aurally presented digit targets presented with visual color primes. Reaction times to visual color targets were longer when the color of the target was incongruent with the synesthetic percept reported for the prime. Likewise, discriminating aurally presented digit targets took longer when the color of the prime was incongruent with the synesthetic percept for the target. These priming effects were absent in non-synesthetes. We conclude that binding between digits and colors in grapheme-color synesthetes can occur bidirectionally across senses. The results are in line with the idea that synesthesia is the result of linking inducing stimuli (e.g. digits) to synesthetic percepts (colors) at an abstract - supra-modal - conceptual level of processing.

Cogito ergo video: Task-relevant information is involuntarily boosted into awareness.

Only part of the visual information that impinges on our retinae reaches visual awareness. In a series of three experiments, we investigated how the task relevance of incoming visual information affects its access to visual awareness. On each trial, participants were instructed to memorize one of two presented hues, drawn from different color categories (e.g., red and green), for later recall. During the retention interval, participants were presented with a differently colored grating in each eye such as to elicit binocular rivalry. A grating matched either the task-relevant (memorized) color category or the task-irrelevant (nonmemorized) color category. We found that the rivalrous stimulus that matched the task-relevant color category tended to dominate awareness over the rivalrous stimulus that matched the task-irrelevant color category. This effect of task relevance persisted when participants reported the orientation of the rivalrous stimuli, even though in this case color information was completely irrelevant for the task of reporting perceptual dominance during rivalry. When participants memorized the shape of a colored stimulus, however, its color category did not affect predominance of rivalrous stimuli during retention. Taken together, these results indicate that the selection of task-relevant information is under volitional control but that visual input that matches this information is boosted into awareness irrespective of whether this is useful for the observer.

Action consequences guide the use of visual working memory.

Visual working memory (VWM) is a store for temporary maintenance of visual information. It is often disregarded, though, that information is typically stored to enable actions. Therefore, the context of these actions is of great importance for how VWM is used. Here, we questioned whether the severity of the consequence of an action might affect how well information is memorized, and how cautiously it is utilized. We employed an (online) copying task, in which participants reproduced an example display comprised of six items in a grid, using a pool of items. Crucially, we manipulated the severity of penalties: participants had to wait 5 (high) or 0.5 (low error cost) s after an error. Additionally, we manipulated the accessibility of task-relevant information (a well-studied manipulation in this paradigm): participants had to wait 5 (high) or 0.5 (low sampling cost) s to inspect the example. Our results show that with higher error cost the number of inspections remained comparable, but the number of errors decreased. Furthermore, they show that with higher sampling cost the number of inspections halved, and the number of errors increased. Thus, more severe action consequences increase the reluctance to act on uncertain information in VWM, but do not lead to more attempts to store information in VWM. We conclude that, in contrast to the effect of the accessibility of information, action consequences do not affect how well information is memorized, but affect how cautiously this stored information is utilized. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Information matching the content of visual working memory is prioritized for conscious access.

Visual working memory (VWM) is used to retain relevant information for imminent goal-directed behavior. In the experiments reported here, we found that VWM helps to prioritize relevant information that is not yet available for conscious experience. In five experiments, we demonstrated that information matching VWM content reaches visual awareness faster than does information not matching VWM content. Our findings suggest a functional link between VWM and visual awareness: The content of VWM is recruited to funnel down the vast amount of sensory input to that which is relevant for subsequent behavior and therefore requires conscious access.

Mountains of memory in a sea of uncertainty: Sampling the external world despite useful information in visual working memory.

A large part of research on visual working memory (VWM) has traditionally focused on estimating its maximum capacity. Yet, humans rarely need to load up their VWM maximally during natural behavior, since visual information often remains accessible in the external world. Recent work, using paradigms that take into account the accessibility of information in the outside world, has indeed shown that observers utilize only one or two items in VWM before sampling from the external world again. One straightforward interpretation of this finding is that, in daily behavior, much fewer items are memorized than the typically reported capacity limits. Here, we first investigate whether this lower reliance on VWM when information is externally accessible might instead reflect resampling before VWM is actually depleted. To this aim we devised an online task, in which participants copied a model (six items in a 4x4 grid; always accessible) in an adjacent empty 4x4 grid. A key aspect of our paradigm is that we (unpredictably) interrupted participants just before inspection of the model with a 2-alternative-forced-choice (2-AFC) question, probing their VWM content. Critically, we observed above-chance performance on probes appearing just before model inspection. This finding shows that the external world was resampled, despite VWM still containing relevant information. We then asked whether increasing the cost of sampling causes participants to load up more information in VWM or, alternatively, to squeeze out more information from VWM (at the cost of making more errors). To manipulate the cost of resampling, we made it more difficult (specifically, more time-consuming) to access the model. We show that with increased cost of accessing the model (which lead to fewer, but longer model inspections), participants could place more items correctly immediately after sampling, and they kept attempting to place items for longer after their first error. These findings demonstrate that participants both encoded more information in VWM and made attempts to squeeze out more information from VWM when sampling became more costly. We argue that human observers constantly evaluate how certain they are of their VWM contents, and only use that VWM content of which their certainty exceeds a context-dependent "action threshold". This threshold, in turn, depends on the trade-off between the cost of resampling and the benefits of making an action. We argue that considering the interplay between the available VWM contents and a context-dependent action threshold, is key for reconciling the traditional VWM literature with VWM use in our day-to-day behavior.

Prioritization of emotional faces is not driven by emotional content.

Emotional faces have prioritized access to visual awareness. However, studies concerned with what expressions are prioritized most are inconsistent and the source of prioritization remains elusive. Here we tested the predictive value of spatial frequency-based image-features and emotional content, the sub-part of the image content that signals the emotional expression of the actor in the image as opposed to the image content irrelevant for the emotional expression, for prioritization for awareness. Participants reported which of two faces (displaying a combination of angry, happy, and neutral expressions), that were temporarily suppressed from awareness, was perceived first. Even though the results show that happy expressions were prioritized for awareness, this prioritization was driven by the contrast energy of the images. In fact, emotional content could not predict prioritization at all. Our findings show that the source of prioritization for awareness is not the information carrying the emotional content. We argue that the methods used here, or similar approaches, should become standard practice to break the chain of inconsistent findings regarding emotional superiority effects that have been part of the field for decades.

Suppressed images selectively affect the dominant percept during binocular rivalry.

During binocular rivalry, perception alternates between dissimilar images that are presented dichoptically. It has been argued that perception during the dominance phase of rivalry is unaffected by the suppressed image. Recent evidence suggests, however, that the suppressed image does affect perception of the dominant image, yet the extent and nature of this interaction remain elusive. We hypothesize that this interaction depends on the difference in feature content between the rivaling images. Here, we investigate how sensitivity to probes presented in the image that is currently dominant in perception is affected by the suppressed image. Observers performed a 2AFC discrimination task on oriented probes (Experiment 1) or probes with different motion directions (Experiment 2). Our results show that performance on both orientation and motion direction discrimination was affected by the content of the suppressed image. The strength of interference depended specifically on the difference in feature content (e.g., the difference in orientation) between the probe and the suppressed image. Moreover, the pattern of interference by the suppressed image is qualitatively similar to the situation where this image and the probe are simultaneously visible. We conclude that perception during the dominance phase of rivalry is affected by a suppressed image as if it were visible.

Vestibular and active self-motion signals drive visual perception in binocular rivalry.

Multisensory integration helps the brain build reliable models of the world and resolve ambiguities. Visual interactions with sound and touch are well established but vestibular influences on vision are less well studied. Here, we test the vestibular influence on vision using horizontally opposed motions presented one to each eye so that visual perception is unstable and alternates irregularly. Passive, whole-body rotations in the yaw plane stabilized visual alternations, with perceived direction oscillating congruently with rotation (leftward motion during leftward rotation, and vice versa). This demonstrates a purely vestibular signal can resolve ambiguous visual motion and determine visual perception. Active self-rotation following the same sinusoidal profile also entrained vision to the rotation cycle - more strongly and with a lesser time lag, likely because of efference copy and predictive internal models. Both experiments show that visual ambiguity provides an effective paradigm to reveal how vestibular and motor inputs can shape visual perception.

Unpredictive linguistic verbal cues accelerate congruent visual targets into awareness in a breaking continuous flash suppression paradigm.

One of the most influential ideas within the domain of cognition is that of embodied cognition, in which the experienced world is the result of an interplay between an organism's physiology, sensorimotor system, and its environment. An aspect of this idea is that linguistic information activates sensory representations automatically. For example, hearing the word 'red' would automatically activate sensory representations of this color. But does linguistic information prioritize access to awareness of congruent visual information? Here, we show that linguistic verbal cues accelerate matching visual targets into awareness by using a breaking continuous flash suppression paradigm. In a speeded reaction time task, observers heard spoken color labels (e.g., red) followed by colored targets that were either congruent (red), incongruent (green), or neutral (a neutral noncolor word) with respect to the labels. Importantly, and in contrast to previous studies investigating a similar question, the incidence of congruent trials was not higher than that of incongruent trials. Our results show that RTs were selectively shortened for congruent verbal-visual pairings, and that this shortening occurred over a wide range of cue-target intervals. We suggest that linguistic verbal information preactivates sensory representations, so that hearing the word 'red' preactivates (visual) sensory information internally.