ERPs and alpha oscillations track the encoding and maintenance of object-based representations in visual working memory.

When memorizing an integrated object such as a Kanizsa figure, the completion of parts into a coherent whole is attained by grouping processes which render a whole-object representation in visual working memory (VWM). The present study measured event-related potentials (ERPs) and oscillatory amplitudes to track these processes of encoding and representing multiple features of an object in VWM. To this end, a change detection task was performed, which required observers to memorize both the orientations and colors of six "pacman" items while inducing configurations of the pacmen that systematically varied in terms of their grouping strength. The results revealed an effect of object configuration in VWM despite physically constant visual input: change detection for both orientation and color features was more accurate with increased grouping strength. At the electrophysiological level, the lateralized ERPs and alpha activity mirrored this behavioral pattern. Perception of the orientation features gave rise to the encoding of a grouped object as reflected by the amplitudes of the Ppc. The grouped object structure, in turn, modulated attention to both orientation and color features as indicated by the enhanced N1pc and N2pc. Finally, during item retention, the representation of individual objects and the concurrent allocation of attention to these memorized objects were modulated by grouping, as reflected by variations in the CDA amplitude and a concurrent lateralized alpha suppression, respectively. These results indicate that memorizing multiple features of grouped, to-be-integrated objects involves multiple, sequential stages of processing, providing support for a hierarchical model of object representations in VWM.

Learning-Induced Plasticity Enhances the Capacity of Visual Working Memory.

Visual working memory (VWM) is limited in capacity, though memorizing meaningful objects may refine this limitation. However, meaningful and meaningless stimuli typically differ perceptually, and objects' associations with meaning are usually already established outside the laboratory, potentially confounding experimental findings. Here, in two experiments with young adults (N = 45 and N = 20), we controlled for these influences by having observers actively learn associations of (for them) initially meaningless stimuli: Chinese characters, half of which were consistently paired with pictures of animals or everyday objects in a learning phase. This phase was preceded and followed by a (pre- and postlearning) change-detection task to assess VWM performance. The results revealed that short-term retention was enhanced after learning, particularly for meaning-associated characters, although participants did not quite reach the accuracy level attained by native Chinese observers (young adults, N = 20). These results thus provide direct experimental evidence that participants' VWM of objects is boosted by them having acquired a long-term-memory association with meaning.

Stimulus-driven updating of long-term context memories in visual search.

Visual search for a target is faster when the spatial layout of nontarget items is repeatedly encountered, illustrating that learned contextual invariances can improve attentional selection (contextual cueing). This type of contextual learning is usually relatively efficient, but relocating the target to an unexpected location (within otherwise unchanged layouts) typically abolishes contextual cueing. Here, we explored whether bottom-up attentional guidance can mediate the efficient contextual adaptation after the change. Two experiments presented an initial learning phase, followed by a subsequent relocation phase that introduced target location changes. This location change was accompanied by transient attention-guiding signals that either up-modulated the changed target location (Experiment 1), or which provided an inhibitory tag to down-modulate the initial target location (Experiment 2). The results from these two experiments showed reliable contextual cueing both before and after the target location change. By contrast, an additional control experiment (Experiment 3) that did not present any attention-guiding signals together with the changed target showed no reliable cueing in the relocation phase, thus replicating previous findings. This pattern of results suggests that attentional guidance (by transient stimulus-driven facilitatory and inhibitory signals) enhances the flexibility of long-term contextual learning.

Feedback from lateral occipital cortex to V1/V2 triggers object completion: Evidence from functional magnetic resonance imaging and dynamic causal modeling.

Illusory figures demonstrate the visual system's ability to integrate disparate parts into coherent wholes. We probed this object integration process by either presenting an integrated diamond shape or a comparable ungrouped configuration that did not render a complete object. Two tasks were used that either required localization of a target dot (relative to the presented configuration) or discrimination of the dot's luminance. The results showed that only when the configuration was task relevant (in the localization task), performance benefited from the presentation of an integrated object. Concurrent functional magnetic resonance imaging was performed and analyzed using dynamic causal modeling to investigate the (causal) relationship between regions that are associated with illusory figure completion. We found object-specific feedback connections between the lateral occipital cortex (LOC) and early visual cortex (V1/V2). These modulatory connections persisted across task demands and hemispheres. Our results thus provide direct evidence that interactions between mid-level and early visual processing regions engage in illusory figure perception. These data suggest that LOC first integrates inputs from multiple neurons in lower-level cortices, generating a global shape representation while more fine-graded object details are then determined via feedback to early visual areas, independently of the current task demands.

Tracking the completion of parts into whole objects: Retinotopic activation in response to illusory figures in the lateral occipital complex.

Illusory figures demonstrate the visual system's ability to integrate separate parts into coherent, whole objects. The present study was performed to track the neuronal object construction process in human observers, by incrementally manipulating the grouping strength within a given configuration until the emergence of a whole-object representation. Two tasks were employed: First, in the spatial localization task, object completion could facilitate performance and was task-relevant, whereas it was irrelevant in the second, luminance discrimination task. Concurrent functional magnetic resonance imaging (fMRI) used spatial localizers to locate brain regions representing task-critical illusory-figure parts to investigate whether the step-wise object construction process would modulate neural activity in these localized brain regions. The results revealed that both V1 and the lateral occipital complex (LOC, with sub-regions LO1 and LO2) were involved in Kanizsa figure processing. However, completion-specific activations were found predominantly in LOC, where neural activity exhibited a modulation in accord with the configuration's grouping strength, whether or not the configuration was relevant to performing the task at hand. Moreover, right LOC activations were confined to LO2 and responded primarily to surface and shape completions, whereas left LOC exhibited activations in both LO1 and LO2 and was related to encoding shape structures with more detail. Together, these results demonstrate that various grouping properties within a visual scene are integrated automatically in LOC, with sub-regions located in different hemispheres specializing in the component sub-processes that render completed objects.

Automatic Guidance (and Misguidance) of Visuospatial Attention by Acquired Scene Memory: Evidence From an N1pc Polarity Reversal.

Visual search is facilitated when the target is repeatedly encountered at a fixed position within an invariant (vs. randomly variable) distractor layout-that is, when the layout is learned and guides attention to the target, a phenomenon known as contextual cuing. Subsequently changing the target location within a learned layout abolishes contextual cuing, which is difficult to relearn. Here, we used lateralized event-related electroencephalogram (EEG) potentials to explore memory-based attentional guidance (N = 16). The results revealed reliable contextual cuing during initial learning and an associated EEG-amplitude increase for repeated layouts in attention-related components, starting with an early posterior negativity (N1pc, 80-180 ms). When the target was relocated to the opposite hemifield following learning, contextual cuing was effectively abolished, and the N1pc was reversed in polarity (indicative of persistent misguidance of attention to the original target location). Thus, once learned, repeated layouts trigger attentional-priority signals from memory that proactively interfere with contextual relearning after target relocation.

Distributed attention beats the down-side of statistical context learning in visual search.

Spatial attention can be deployed with a narrower focus to process individual items or distributed relatively broadly to process larger parts of a scene. This study investigated how focused- versus distributed-attention modes contribute to the adaptation of context-based memories that guide visual search. In two experiments, participants were either required to fixate the screen center and use peripheral vision for search ("distributed attention"), or they could freely move their eyes, enabling serial scanning of the search array ("focused attention"). Both experiments consisted of an initial learning phase and a subsequent test phase. During learning, participants searched for targets presented either among repeated (invariant) or nonrepeated (randomly generated) spatial layouts of distractor items. Prior research showed that repeated encounters of invariant display arrangements lead to long-term context memory about these arrays, which can then come to guide search (contextual-cueing effect). The crucial manipulation in the test phase was a change of the target location within an otherwise constant distractor layout, which has previously been shown to abolish the cueing effect. The current results replicated these findings, although importantly only when attention was focused. By contrast, with distributed attention, the cueing effect recovered rapidly and attained a level comparable to the initial effect (before the target location change). This indicates that contextual cueing can adapt more easily when attention is distributed, likely because a broad attentional set facilitates the flexible updating of global (distractor-distractor), as compared to more local (distractor-target), context representations-allowing local changes to be incorporated more readily.

Taking Attention Out of Context: Frontopolar Transcranial Magnetic Stimulation Abolishes the Formation of New Context Memories in Visual Search.

This study investigates the causal contribution of the left frontopolar cortex (FPC) to the processing of violated expectations from learned target-distractor spatial contingencies during visual search. The experiment consisted of two phases: learning and test. Participants searched for targets presented either among repeated or nonrepeated target-distractor configurations. Prior research showed that repeated encounters of identically arranged displays lead to memory about these arrays, which then can come to guide search (contextual cueing effect). The crucial manipulation was a change of the target location, in a nevertheless constant distractor layout, at the transition from learning to test. In addition to this change, we applied repetitive transcranial magnetic stimulation (rTMS) over the left lateral FPC, over a posterior control site, or no rTMS at all (baseline; between-group manipulation) to see how FPC rTMS influences the ability of observers to adapt context-based memories acquired in the training phase. The learning phase showed expedited search in repeated relative to nonrepeated displays, with this context-based facilitation being comparable across all experimental groups. For the test phase, the recovery of cueing was critically dependent on the stimulation site: Although there was evidence of context adaptation toward the end of the experiment in the occipital and no-rTMS conditions, observers with FPC rTMS showed no evidence of relearning at all after target location changes. This finding shows that FPC plays an important role in the regulation of prediction errors in statistical context learning, thus contributing to an update of the spatial target-distractor contingencies after target position changes in learned spatial arrays.

Object maintenance beyond their visible parts in working memory.

Completion of a partially occluded object requires that a representation of the whole is constructed based on the information provided by the physically specified parts of the stimulus. Such processes of amodal completion rely on the generation and maintenance of a mental image that renders the completed object in visual working memory (VWM). The present study examined this relationship between VWM storage and processes of object completion. We recorded event-related potentials to track VWM maintenance by means of the contralateral delay activity (CDA) during a change detection task in which composite objects (notched shapes abutting an occluding shape) to be memorized were primed to induce either a globally completed object or a noncompleted, mosaic representation. The results revealed an effect of completion in VWM despite physically identical visual input: change detection was more accurate for completed compared with mosaic representations when observers were required to memorize two objects, and these differences were reduced with four memorized items. At the electrophysiological level, globally completed (vs. mosaic) objects gave rise to a corresponding increase in CDA amplitudes. These results indicate that although incorporating the occluded portions of the presented shapes requires mnemonic resources, the complete object representations thus formed in VWM improve change detection performance by providing a more simple, regular shape. Overall, these findings demonstrate that mechanisms of object completion modulate VWM, with the memory load being determined by the structured representations of the memorized stimuli. NEW & NOTEWORTHY This study shows that completion of partially occluded objects requires visual working memory (VWM) resources. In the experiment reported, we induced observers to memorize a given visual input either as completed or as noncompleted objects. The results revealed both a behavioral performance advantage for completed vs. noncompleted objects despite physically identical input, and an associated modulation of an electrophysiological component that reflects VWM object retention, thus indicating that constructing an integrated object consumes mnemonic resources.

Probing the Cognitive Mechanism of Mental Representational Change During Chunk Decomposition: A Parametric fMRI Study.

Chunk decomposition plays an important role in cognitive flexibility in particular with regards to representational change, which is critical for insight problem solving and creative thinking. In this study, we investigated the cognitive mechanism of decomposing Chinese character chunks through a parametric fMRI design. Our results from this parametric manipulation revealed widely distributed activations in frontal, parietal, and occipital cortex and negative activations in parietal and visual areas in response to chunk tightness during decomposition. To mentally manipulate the element of a given old chunk, superior parietal lobe appears to support element restructuring in a goal-directed way, whereas the negatively activated inferior parietal lobe may support preventing irrelevant objects from being attended. Moreover, determining alternative ways of restructuring requires a constellation of frontal areas in the cognitive control network, such as the right lateral prefrontal cortex in inhibiting the predominant chunk representations, the presupplementary motor area in initiating a transition of mental task set, and the inferior frontal junction in establishing task sets. In conclusion, this suggests that chunk decomposition reflects mental transformation of problem representation from an inappropriate state to a new one alongside with an evaluation of novel and insightful solutions by the caudate in the dorsal striatum.

Contextual cueing in 3D visual search depends on representations in planar-, not depth-defined space.

Learning of spatial inter-item associations can speed up visual search in everyday life, an effect referred to as contextual cueing (Chun & Jiang, 1998). Whereas previous studies investigated contextual cueing primarily using 2D layouts, the current study examined how 3D depth influences contextual learning in visual search. In two experiments, the search items were presented evenly distributed across front and back planes in an initial training session. In the subsequent test session, the search items were either swapped between the front and back planes (Experiment 1) or between the left and right halves (Experiment 2) of the displays. The results showed that repeated spatial contexts were learned efficiently under 3D viewing conditions, facilitating search in the training sessions, in both experiments. Importantly, contextual cueing remained robust and virtually unaffected following the swap of depth planes in Experiment 1, but it was substantially reduced (to nonsignificant levels) following the left-right side swap in Experiment 2. This result pattern indicates that spatial, but not depth, inter-item variations limit effective contextual guidance. Restated, contextual cueing (even under 3D viewing conditions) is primarily based on 2D inter-item associations, while depth-defined spatial regularities are probably not encoded during contextual learning. Hence, changing the depth relations does not impact the cueing effect.

Predictive distractor context facilitates attentional selection of high, but not intermediate and low, salience targets.

It is well established that we can focally attend to a specific region in visual space without shifting our eyes, so as to extract action-relevant sensory information from covertly attended locations. The underlying mechanisms that determine how fast we engage our attentional spotlight in visual-search scenarios, however, remain controversial. One dominant view advocated by perceptual decision-making models holds that the times taken for focal-attentional selection are mediated by an internal template that biases perceptual coding and selection decisions exclusively through target-defining feature coding. This notion directly predicts that search times remain unaffected whether or not participants can anticipate the upcoming distractor context. Here we tested this hypothesis by employing an illusory-figure localization task that required participants to search for an invariant target amongst a variable distractor context, which gradually changed--either randomly or predictably--as a function of distractor-target similarity. We observed a graded decrease in internal focal-attentional selection times--correlated with external behavioral latencies--for distractor contexts of higher relative to lower similarity to the target. Critically, for low but not intermediate and high distractor-target similarity, these context-driven effects were cortically and behaviorally amplified when participants could reliably predict the type of distractors. This interactive pattern demonstrates that search guidance signals can integrate information about distractor, in addition to target, identities to optimize distractor-target competition for focal-attentional selection.

Environmental regularities mitigate attentional misguidance in contextual cueing of visual search.

Visual search is faster when a fixed target location is paired with a spatially invariant (vs. randomly changing) distractor configuration, thus indicating that repeated contexts are learned, thereby guiding attention to the target (contextual cueing [CC]). Evidence for memory-guided attention has also been revealed with electrophysiological (electroencephalographic [EEG]) recordings, starting with an enhanced early posterior negativity (N1pc), which signals a preattentive bias toward the target, and, subsequently, attentional and postselective components, such as the posterior contralateral negativity (PCN) and contralateral delay activity (CDA), respectively. Despite effective learning, relearning of previously acquired contexts is inflexible: The CC benefits disappear when the target is relocated to a new position within an otherwise invariant context and corresponding EEG correlates are diminished. The present study tested whether global statistical properties that induce predictions going beyond the immediate invariant layout can facilitate contextual relearning. Global statistical regularities were implemented by presenting repeated and nonrepeated displays in separate streaks (mini blocks) of trials in the relocation phase, with individual displays being presented in a fixed and thus predictable order. Our results revealed a significant CC effect (and an associated modulation of the N1pc, PCN, and CDA components) during initial learning. Critically, the global statistical regularities in the relocation phase also resulted in a reliable CC effect, thus revealing effective relearning with predictive streaks. Moreover, this relearning was reflected in an enhanced PCN amplitude for repeated relative to nonrepeated contexts. Temporally ordered contexts may thus adapt memory-based guidance of attention, particularly the allocation of covert attention in the visual display. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Changes in attentional breadth scale with the demands of Kanizsa-figure object completion-evidence from pupillometry.

The present study investigated whether the integration of separate parts into a whole-object representation varies with the amount of available attentional resources. To this end, two experiments were performed, which required observers to maintain central fixation while searching in peripheral vision for a target among various distractor configurations. The target could either be a "grouped" whole-object Kanizsa figure, or an "ungrouped" configuration of identical figural parts, but which do not support object completion processes to the same extent. In the experiments, accuracies and changes in pupil size were assessed, with the latter reflecting a marker of the covert allocation of attention in the periphery. Experiment 1 revealed a performance benefit for grouped (relative to ungrouped) targets, which increased with decreasing distance from fixation. By contrast, search for ungrouped targets was comparably poor in accuracy without revealing any eccentricity-dependent variation. Moreover, measures of pupillary dilation mirrored this eccentricity-dependent advantage in localizing grouped targets. Next, in Experiment 2, an additional attention-demanding foveal task was introduced in order to further reduce the availability of attentional resources for the peripheral detection task. This additional task hampered performance overall, alongside with corresponding pupil size changes. However, there was still a substantial benefit for grouped over ungrouped targets in both the behavioral and the pupillometric data. This shows that perceptual grouping scales with the allocation of attention even when only residual attentional resources are available to trigger the representation of a complete (target) object, thus illustrating that object completion operates in the "near absence" of attention.

Mission impossible? Spatial context relearning following a target relocation event depends on cue predictiveness.

Visual search for a target is faster when the spatial layout of distractors is repeatedly encountered, illustrating that statistical learning of contextual invariances facilitates attentional guidance (contextual cueing; Chun & Jiang, 1998, Cognitive Psychology, 36, 28-71). While contextual learning is usually relatively efficient, relocating the target to an unexpected location (within an otherwise unchanged search layout) typically abolishes contextual cueing and the benefits deriving from invariant contexts recover only slowly with extensive training (Zellin et al., 2014, Psychonomic Bulletin & Review, 21(4), 1073-1079). However, a recent study by Peterson et al. (2022, Attention, Perception, & Psychophysics, 84(2), 474-489) in fact reported rather strong adaptation of spatial contextual memories following target position changes, thus contrasting with prior work. Peterson et al. argued that previous studies may have been underpowered to detect a reliable recovery of contextual cueing after the change. However, their experiments also used a specific display design that frequently presented the targets at the same locations, which might reduce the predictability of the contextual cues thereby facilitating its flexible relearning (irrespective of statistical power). The current study was a (high-powered) replication of Peterson et al., taking into account both statistical power and target overlap in context-memory adaptation. We found reliable contextual cueing for the initial target location irrespective of whether the targets shared their location across multiple displays, or not. However, contextual adaptation following a target relocation event occurred only when target locations were shared. This suggests that cue predictability modulates contextual adaptation, over and above a possible (yet negligible) influence of statistical power.

When experience with scenes foils attentional orienting: ERP evidence against flexible target-context mapping in visual search.

Visual search is speeded when a target is repeatedly presented in an invariant scene context of nontargets (contextual cueing), demonstrating observers' capability for using statistical long-term memory (LTM) to make predictions about upcoming sensory events, thus improving attentional orienting. In the current study, we investigated whether expectations arising from individual, learned environmental structures can encompass multiple target locations. We recorded event-related potentials (ERPs) while participants performed a contextual cueing search task with repeated and non-repeated spatial item configurations. Notably, a given search display could be associated with either a single target location (standard contextual cueing) or two possible target locations. Our result showed that LTM-guided attention was always limited to only one target position in single- but also in the dual-target displays, as evidenced by expedited reaction times (RTs) and enhanced N1pc and N2pc deflections contralateral to one ("dominant") target of up to two repeating target locations. This contrasts with the processing of non-learned ("minor") target positions (in dual-target displays), which revealed slowed RTs alongside an initial N1pc "misguidance" signal that then vanished in the subsequent N2pc. This RT slowing was accompanied by enhanced N200 and N400 waveforms over fronto-central electrodes, suggesting that control mechanisms regulate the competition between dominant and minor targets. Our study thus reveals a dissociation in processing dominant versus minor targets: While LTM templates guide attention to dominant targets, minor targets necessitate control processes to overcome the automatic bias towards previously learned, dominant target locations.

Selective manipulation of target identification demands in visual search: the role of stimulus contrast in CDA activations.

In classic visual pop-out search, response times are slowed remarkably when participants are required to precisely identify (e.g., vertical vs. horizontal orientation) as compared to simply localize (e.g., left vs. right position) a feature singleton target. This cost associated with stimulus identification has been recently proposed (Töllner, Rangelov, & Müller, 2012) to derive from the engagement of postselective recurrent processes that via feedback connections extract the information required for motor-response selection. Here, we examined whether the contralateral delay activity (CDA), an asymmetric neural marker generally assumed to reflect active maintenance of stimulus information in visual short-term memory (vSTM), may further index the degree of postselective processing requirements in visual search. Employing a compound-search task, we selectively manipulated the ease/difficulty of identifying the response-critical target orientation attribute (horizontal vs. vertical)--irrespective of the target-defining color feature (red vs. green)--by introducing different levels of stimulus-background contrast. As expected, we found a monotonic reaction time increase to be associated with gradually increasing CDA magnitudes as the stimulus contrast decreased. Thus, our findings provide direct evidence that CDA activations provide a useful tool to estimate the amount of postselective recurrent processing recruited to extract detailed object information from vSTM.

Object-based implicit learning in visual search: perceptual segmentation constrains contextual cueing.

In visual search, detection of a target is faster when it is presented within a spatial layout of repeatedly encountered nontarget items, indicating that contextual invariances can guide selective attention (contextual cueing; Chun & Jiang, 1998). However, perceptual regularities may interfere with contextual learning; for instance, no contextual facilitation occurs when four nontargets form a square-shaped grouping, even though the square location predicts the target location (Conci & von Mühlenen, 2009). Here, we further investigated potential causes for this interference-effect: We show that contextual cueing can reliably occur for targets located within the region of a segmented object, but not for targets presented outside of the object's boundaries. Four experiments demonstrate an object-based facilitation in contextual cueing, with a modulation of context-based learning by relatively subtle grouping cues including closure, symmetry, and spatial regularity. Moreover, the lack of contextual cueing for targets located outside the segmented region was due to an absence of (latent) learning of contextual layouts, rather than due to an attentional bias towards the grouped region. Taken together, these results indicate that perceptual segmentation provides a basic structure within which contextual scene regularities are acquired. This in turn argues that contextual learning is constrained by object-based selection.

Statistical learning in the past modulates contextual cueing in the future.

Observers' capability to extract statistical regularities from the visual world can facilitate attentional orienting. For instance, visual search benefits from the repetition of target locations by means of probability learning. Furthermore, repeated (old) contexts of nontargets contribute to faster visual search in comparison to random (new) arrangements of nontargets. Chun and Jiang (1998) called this effect "contextual cueing" because old contexts provide spatial cues to repeated target locations. In the present study, we investigated how probability learning modulates the adaptation of contextual cueing to a change in target location. After an initial learning phase, targets were relocated within their respective contexts to new positions that were, however, familiar from previous presentations in other spatial contexts. Contextual cueing was observed for relocated targets that originated from old contexts, but it turned into costs when relocated targets had previously been presented in new contexts. Thus, probability learning was not sufficient to observe adaptive contextual cueing for relocated targets. Instead, the contextual past of target locations--whether they had been cued or not--modulated the integration of relocated targets into a learned context. These findings imply that observers extract multiple levels of available statistical information and use them to infer hypotheses about future occurrences of familiar stimuli.

Contextual remapping in visual search after predictable target-location changes.

Invariant spatial context can facilitate visual search. For instance, detection of a target is faster if it is presented within a repeatedly encountered, as compared to a novel, layout of nontargets, demonstrating a role of contextual learning for attentional guidance ('contextual cueing'). Here, we investigated how context-based learning adapts to target location (and identity) changes. Three experiments were performed in which, in an initial learning phase, observers learned to associate a given context with a given target location. A subsequent test phase then introduced identity and/or location changes to the target. The results showed that contextual cueing could not compensate for target changes that were not 'predictable' (i.e. learnable). However, for predictable changes, contextual cueing remained effective even immediately after the change. These findings demonstrate that contextual cueing is adaptive to predictable target location changes. Under these conditions, learned contextual associations can be effectively 'remapped' to accommodate new task requirements.