Neural Signatures of Competition between Voluntary and Involuntary Influences over the Focus of Attention in Visual Working Memory.

Adaptive behavior relies on the selection and prioritization of relevant sensory inputs from the external environment as well as from among internal sensory representations held in working memory. Recent behavioral evidence suggests that the classic distinction between voluntary (goal-driven) and involuntary (stimulus-driven) influences over attentional allocation also applies to the selection of internal representations held in working memory. In the current EEG study, we set out to investigate the neural dynamics associated with the competition between voluntary and involuntary control over the focus of attention in visual working memory. We show that when voluntary and involuntary factors compete for the internal focus of attention, prioritization of the appropriate item is delayed-as reflected both in delayed gaze biases that track internal selection and in delayed neural beta (15-25 Hz) dynamics that track the planning for the upcoming memory-guided manual action. We further show how this competition is paralleled-possibly resolved-by an increase in frontal midline theta (4-8 Hz) activity that, moreover, predicts the speed of ensuing memory-guided behavior. Finally, because theta increased following retrocues that effectively reduced working-memory load, our data unveil how frontal theta activity during internal attentional focusing tracks demands on cognitive control over and above working-memory load. Together, these data yield new insight into the neural dynamics that govern the focus of attention in visual working memory, and disentangle the contributions of frontal midline theta activity to the processes of control versus retention in working memory.

Single-pulse Transcranial Magnetic Stimulation Affects Working-memory Performance via Posterior Beta-band Oscillations.

A single pulse of TMS (spTMS) during the delay period of a double serial retrocuing working-memory task can briefly rescue decodability of an unprioritized memory item (UMI). This physiological phenomenon, which is paralleled in behavior by involuntary retrieval of the UMI, is carried by the beta frequency band, implicating beta-band dynamics in priority coding in working memory. We decomposed EEG data from 12 participants performing double serial retrocuing with concurrent delivery of spTMS using Spatially distributed PhAse Coupling Extraction. This procedure decomposes the scalp-level signal into a set of discrete coupled oscillators, each with a component strength that can vary over time. The decomposition revealed a diversity of low-frequency components, a subset of them strengthening with the onset of the task, and the majority declining in strength across the trial, as well as within each delay period. Results with spTMS revealed no evidence that it works by activating previously "silent" sources; instead, it had the effect of modulating ongoing activity, specifically by exaggerating the within-delay decrease in strength of posterior beta components. Furthermore, the magnitude of the effect of spTMS on the loading strength of a posterior beta component correlated with the disruptive effect of spTMS on performance, a pattern also seen when analyses were restricted to trials with "UMI-lure" memory probes. Rather than reflecting the "activation" of a putatively "activity silent" UMI, these results implicate beta-band dynamics in a mechanism that distinguishes prioritized from unprioritized, and suggest that the effect of spTMS is to disrupt this code.

Representing Context and Priority in Working Memory.

The ability to prioritize among contents in working memory (WM) is critical for successful control of thought and behavior. Recent work has demonstrated that prioritization in WM can be implemented by representing different states of priority in different representational formats. Here, we explored the mechanisms underlying WM prioritization by simulating the double serial retrocuing task with recurrent neural networks. Visualization of stimulus representational dynamics using principal component analysis revealed that the network represented trial context (order of presentation) and priority via different mechanisms. Ordinal context, a stable property lasting the duration of the trial, was accomplished by segregating representations into orthogonal subspaces. Priority, which changed multiple times during a trial, was accomplished by separating representations into different strata within each subspace. We assessed the generality of these mechanisms by applying dimensionality reduction and multiclass decoding to fMRI and EEG data sets and found that priority and context are represented differently along the dorsal visual stream and that behavioral performance is sensitive to trial-by-trial variability of priority coding, but not context coding.

Strategic control of location and ordinal context in visual working memory.

Working memory (WM) requires encoding stimulus identity and context (e.g. where or when stimuli were encountered). To explore the neural bases of the strategic control of context binding in WM, we acquired fMRI while subjects performed delayed recognition of 3 orientation patches presented serially and at different locations. The recognition probe was an orientation patch with a superimposed digit, and pretrial instructions directed subjects to respond according to its location ("location-relevant"), to the ordinal position corresponding to its digit ("order-relevant"), or to just its orientation (relative to all three samples; "context-irrelevant"). Delay period signal in PPC was greater for context-relevant than for "context-irrelevant" trials, and multivariate decoding revealed strong sensitivity to context binding requirements (relevant vs. "irrelevant") and to context domain ("location-" vs. "order-relevant") in both occipital cortex and PPC. At recognition, multivariate inverted encoding modeling revealed markedly different patterns in these 2 regions, suggesting different context-processing functions. In occipital cortex, an active representation of the location of each of the 3 samples was reinstated regardless of the trial type. The pattern in PPC, by contrast, suggested a trial type-dependent filtering of sample information. These results indicate that PPC exerts strategic control over the representation of stimulus context in visual WM.

Spectral Distribution Dynamics across Different Attentional Priority States.

Anticipatory covert spatial attention improves performance on tests of visual detection and discrimination, and shifts are accompanied by decreases and increases of α band power at electroencephalography (EEG) electrodes corresponding to the attended and unattended location, respectively. Although the increase at the unattended location is often interpreted as an active mechanism (e.g., inhibiting processing at the unattended location), most experiments cannot rule out the alternative possibility that it is a secondary consequence of selection elsewhere. To adjudicate between these accounts, we designed a Posner-style visual cueing task in which male and female human participants made orientation judgments of targets appearing at one of four locations: up, down, right, or left. Critically, trials were blocked such that within a block the locations along one meridian alternated in status between attended and unattended, and targets never appeared at the other two, making them irrelevant. Analyses of the concurrently measured EEG signal were conducted on "traditional" narrowband α (8-14 Hz), as well as on two components resulting from the decomposition of this signal: "periodic" α; and the slope of the aperiodic 1/f-like component. Although data from right-left blocks replicated the familiar pattern of lateralized asymmetry in narrowband α power, with neither α signal could we find evidence for any difference in the time course at unattended versus irrelevant locations, an outcome consistent with the secondary-consequence interpretation of attention-related dynamics in the α band. Additionally, 1/f slope was shallower at attended and unattended locations, relative to irrelevant, suggesting a tonic adjustment of physiological state.SIGNIFICANCE STATEMENT Visual spatial attention, the prioritization of one location in the visual field, is critical for guiding behavior in cluttered environments. Although influential theories posit an important role for α band oscillations in the inhibition of processing at unattended locations, we used a novel procedure to find evidence for an alternative interpretation: selection of one location may simply result in a return to physiological baseline at all others. In addition to determining one way that attention does not work (important for future progress in this field), we also discovered novel evidence for one way that it does work: by modifying the tonic physiological state (indexed by an aperiodic component of the electroencephalography (EEG)] at locations where spatial selection is likely to occur.

Different states of priority recruit different neural representations in visual working memory.

We used functional magnetic resonance imaging (fMRI) to investigate the neural codes for representing stimulus information held in different states of priority in working memory. Human participants (male and female) performed delayed recall for 2 oriented gratings that could appear in any of several locations. Priority status was manipulated by a retrocue, such that one became the prioritized memory item (PMI) and another the unprioritized memory item (UMI). Using inverted encoding models (IEMs), we found that, in early visual cortex, the orientation of the UMI was represented in a neural representation that was rotated relative to the PMI. In intraparietal sulcus (IPS), we observed the analogous effect for the representation of the location of the UMI. Taken together, these results provide evidence for a common remapping mechanism that may be responsible for representing stimulus identity and stimulus context with different levels of priority in working memory.

Delay-period activity in frontal, parietal, and occipital cortex tracks noise and biases in visual working memory.

Working memory is imprecise, and these imprecisions can be explained by the combined influences of random diffusive error and systematic drift toward a set of stable states ("attractors"). However, the neural correlates of diffusion and drift remain unknown. Here, we investigated how delay-period activity in frontal and parietal cortex, which is known to correlate with the decline in behavioral memory precision observed with increasing memory load, might relate to diffusion and drift. We analyzed data from an existing experiment in which subjects performed delayed recall for line orientation, at different loads, during functional magnetic resonance imaging (fMRI) scanning. To quantify the influence of drift and diffusion, we modeled subjects' behavior using a discrete attractor model and calculated within-subject correlation between frontal and parietal delay-period activity and whole-trial estimates of drift and diffusion. We found that although increases in frontal and parietal activity were associated with increases in both diffusion and drift, diffusion explained the most variance in frontal and parietal delay-period activity. In comparison, a subsequent whole-brain regression analysis showed that drift, rather than diffusion, explained the most variance in delay-period activity in lateral occipital cortex. These results are consistent with a model of the differential recruitment of general frontoparietal mechanisms in response to diffusive noise and of stimulus-specific biases in occipital cortex.

Priority-based transformations of stimulus representation in visual working memory.

How does the brain prioritize among the contents of working memory (WM) to appropriately guide behavior? Previous work, employing inverted encoding modeling (IEM) of electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) datasets, has shown that unprioritized memory items (UMI) are actively represented in the brain, but in a "flipped", or opposite, format compared to prioritized memory items (PMI). To acquire independent evidence for such a priority-based representational transformation, and to explore underlying mechanisms, we trained recurrent neural networks (RNNs) with a long short-term memory (LSTM) architecture to perform a 2-back WM task. Visualization of LSTM hidden layer activity using Principal Component Analysis (PCA) confirmed that stimulus representations undergo a representational transformation-consistent with a flip-while transitioning from the functional status of UMI to PMI. Demixed (d)PCA of the same data identified two representational trajectories, one each within a UMI subspace and a PMI subspace, both undergoing a reversal of stimulus coding axes. dPCA of data from an EEG dataset also provided evidence for priority-based transformations of the representational code, albeit with some differences. This type of transformation could allow for retention of unprioritized information in WM while preventing it from interfering with concurrent behavior. The results from this initial exploration suggest that the algorithmic details of how this transformation is carried out by RNNs, versus by the human brain, may differ.

Spontaneous alpha-band amplitude predicts subjective visibility but not discrimination accuracy during high-level perception.

Near-threshold perception is a paradigm case of awareness diverging from reality - the perception of an unchanging stimulus can vacillate from undetected to clearly perceived. The amplitude of low-frequency brain oscillations - particularly in the alpha-band (8-13 Hz) - has emerged as a reliable predictor of trial-to-trial variability in perceptual decisions based on simple, low-level stimuli. Here, we addressed the question of how spontaneous oscillatory amplitude impacts subjective and objective aspects of perception using high-level visual stimuli. Human observers completed a near-threshold face/house discrimination task with subjective visibility ratings while electroencephalograms (EEG) were recorded. Using single-trial multiple regression analysis, we found that spontaneous fluctuations in prestimulus alpha-band amplitude were negatively related to visibility judgments but did not predict trial-by-trial accuracy. These results extend previous findings that indicate that strong prestimulus alpha diminishes subjective perception without affecting the accuracy or sensitivity (d') of perceptual decisions into the domain of high-level perception.

Flexible top-down control in the interaction between working memory and perception.

Successful goal-directed behavior often requires continuous sensory processing while simultaneously maintaining task-related information in working memory (WM). Although WM and perception are known to interact, little is known about how their interactions are controlled. Here, we tested the hypothesis that WM perception interactions engage two distinct modes of control - proactive and reactive - in a manner similar to classic conflict-adaptation tasks (e.g. Stroop, flanker, and Simon). Participants performed a delayed recall-of-orientation WM task, plus a standalone visual discrimination-of-orientation task the occurred during the delay period, and with the congruity in orientation between the tasks manipulated. Proactive control was seen in the sensitivity of task performance to the previous trial's congruity (i.e. a Gratton effect). Reactive control was observed in a repulsive serial-dependence produced by incongruent discriminanda. Quantitatively, these effects were explained by parameters from a reinforcement learning-based model that tracks trial-to-trial fluctuations in control demand: reactive control by a phasic control prediction error (control PE), and proactive control by a tonic level of predicted conflict updated each trial by the control PE. Thus, WM-perception interactions may be controlled by the same mechanisms that govern conflict in other domains of cognition, such as response selection.

The Neural Codes Underlying Internally Generated Representations in Visual Working Memory.

Humans can construct rich subjective experience even when no information is available in the external world. Here, we investigated the neural representation of purely internally generated stimulus-like information during visual working memory. Participants performed delayed recall of oriented gratings embedded in noise with varying contrast during fMRI scanning. Their trialwise behavioral responses provided an estimate of their mental representation of the to-be-reported orientation. We used multivariate inverted encoding models to reconstruct the neural representations of orientation in reference to the response. We found that response orientation could be successfully reconstructed from activity in early visual cortex, even on 0% contrast trials when no orientation information was actually presented, suggesting the existence of a purely internally generated neural code in early visual cortex. In addition, cross-generalization and multidimensional scaling analyses demonstrated that information derived from internal sources was represented differently from typical working memory representations, which receive influences from both external and internal sources. Similar results were also observed in intraparietal sulcus, with slightly different cross-generalization patterns. These results suggest a potential mechanism for how externally driven and internally generated information is maintained in working memory.

Gender (Im)balance in Citation Practices in Cognitive Neuroscience.

In the field of neuroscience, despite the fact that the proportion of peer-reviewed publications authored by women has increased in recent decades, the proportion of citations of women-led publications has not seen a commensurate increase: In five broad-scope journals, citations of papers first- and/or last-authored by women have been shown to be fewer than would be expected if gender was not a factor in citation decisions [Dworkin, J. D., Linn, K. A., Teich, E. G., Zurn, P., Shinohara, R. T., & Bassett, D. S. The extent and drivers of gender imbalance in neuroscience reference lists. Nature Neuroscience, 23, 918-926, 2020]. Given the important implications that such underrepresentation may have on the careers of women researchers, it is important to determine whether this same trend is true in subdisciplines of the field, where interventions might be more targeted. Here, we report the results of an extension of the analyses carried out by Dworkin et al. (2020) to citation patterns in the Journal of Cognitive Neuroscience. The results indicate that the underrepresentation of women-led publications in reference sections is also characteristic of papers published in Journal of Cognitive Neuroscience over the past decade. Furthermore, this pattern of citation imbalances is present regardless of author gender, implicating systemic factors. These results contribute to the growing body of evidence that intentional action is needed to address inequities in the way that we carry out and communicate our science.

Attention Biases Competition for Visual Representation via Dissociable Influences from Frontal and Parietal Cortex.

What mechanisms underlie the prioritization of neural representations of visually perceived information to guide behavior? We assessed the dynamics whereby attention biases competition for representation of visual stimuli by enhancing representations of relevant information and suppressing the irrelevant. Multivariate pattern analysis (MVPA) classifiers were trained to discriminate patterns of fMRI activity associated with each of three stimuli, within several predefined ROIs. Participants performed a change-detection task wherein two of three presented items flashed at 1 Hz, one to each side of central fixation. Both flashing stimuli would unpredictably change state, but participants covertly counted the number of changes only for the cued item. In the ventral occipito-temporal ROI, MVPA evidence (a proxy for representational fidelity) was dynamically enhanced for attended stimuli and suppressed for unattended stimuli, consistent with a mechanism of biased competition between stimulus representations. Frontal and parietal ROIs displayed a qualitatively distinct, more "source-like" profile, wherein MVPA evidence for only the attended stimulus could be observed above baseline levels. To assess how attentional modulation of ventral occipito-temporal representations might relate to signals originating in the frontal and/or parietal ROIs, we analyzed informational connectivity (IC), which indexes time-varying covariation between regional levels of MVPA evidence. Parietal-posterior IC was elevated during the task, but did not differ for cued versus uncued items. Frontal-posterior IC, in contrast, was sensitive to an item's priority status. Thus, although regions of frontal and parietal cortex act as sources of top-down attentional control, their precise functions likely differ.

The Neural Consequences of Attentional Prioritization of Internal Representations in Visual Working Memory.

Although humans can hold multiple items in mind simultaneously, the contents of working memory (WM) can be selectively prioritized to guide future behavior. We explored whether the "same-object" benefits in visual processing may also be observed in visual WM. fMRI data were collected while participants performed a multistep serial retrocuing task in which they first viewed two 2-D objects (coherently moving colored dots). During retention, an initial relevance cue then indicated whether only the first or only the second object ("object-relevant"), or only the color of both objects or only their direction of motion would be relevant for the remainder of the trial ("feature-relevant"). On "object-relevant" trials, the ensuing priority cues selected either one of the features ("color" or "direction") bound to the relevance-cued object, whereas on "feature-relevant" trials, the priority cues selected one of the two relevance-cued features. Using multivariate inverted encoding models, we found a same-object benefit on object-relevant trials in occipitotemporal regions: On feature-relevant trials, the first priority cue triggered a strengthening of the neural representation of the cued feature and a concomitant weakening to baseline of the uncued feature, whereas on object-relevant trials, the cued item remained active but did not increase in strength and the uncued item weakened but remained significantly elevated throughout the delay period. Although the stimulus-specific representation in frontoparietal regions was weak and uneven, these regions closely tracked the higher order information of which stimulus category was relevant for behavior throughout the trial, suggesting an important role in controlling the prioritization of information in visual WM.

Parietal-Occipital Interactions Underlying Control- and Representation-Related Processes in Working Memory for Nonspatial Visual Features.

Although the manipulation of load is popular in visual working memory research, many studies confound general attentional demands with context binding by drawing memoranda from the same stimulus category. In this fMRI study of human observers (both sexes), we created high- versus low-binding conditions, while holding load constant, by comparing trials requiring memory for the direction of motion of one random dot kinematogram (RDK; 1M trials) versus for three RDKs (3M), or versus one RDK and two color patches (1M2C). Memory precision was highest for 1M trials and comparable for 3M and 1M2C trials. And although delay-period activity in occipital cortex did not differ between the three conditions, returning to baseline for all three, multivariate pattern analysis decoding of a remembered RDK from occipital cortex was also highest for 1M trials and comparable for 3M and 1M2C trials. Delay-period activity in intraparietal sulcus (IPS), although elevated for all three conditions, displayed more sensitivity to demands on context binding than to load per se. The 1M-to-3M increase in IPS signal predicted the 1M-to-3M declines in both behavioral and neural estimates of working memory precision. These effects strengthened along a caudal-to-rostral gradient, from IPS0 to IPS5. Context binding-independent load sensitivity was observed when analyses were lateralized and extended into PFC, with trend-level effects evident in left IPS and strong effects in left lateral PFC. These findings illustrate how visual working memory capacity limitations arise from multiple factors that each recruit dissociable brain systems.SIGNIFICANCE STATEMENT Visual working memory capacity predicts performance on a wide array of cognitive and real-world outcomes. At least two theoretically distinct factors are proposed to influence visual working memory capacity limitations: an amodal attentional resource that must be shared across remembered items; and the demands on context binding. We unconfounded these two factors by varying load with items drawn from the same stimulus category ("high demands on context binding") versus items drawn from different stimulus categories ("low demands on context binding"). The results provide evidence for the dissociability, and the neural bases, of these two theorized factors, and they specify that the functions of intraparietal sulcus may relate more strongly to the control of representations than to the general allocation of attention.

Overlapping and distinct contributions of stimulus location and of spatial context to nonspatial visual short-term memory.

Stimulus location is not always informative during visual short-term memory (VSTM) for nonspatial features. Nevertheless, there is considerable evidence for the automatic encoding and retention of location information, regardless of its task relevance. To explore the functional and neural bases of the representation of spatial context in VSTM for nonspatial information, functional magnetic resonance imaging was performed while subjects performed delayed recall for the orientation of individual stimuli. Stimulus location varied across trials, and although this information was irrelevant for task performance, multivariate pattern analysis decoding of stimulus location sustained across trials, and also the decoding strength, predicted the precision of the recall of orientation. The influence of spatial context on the representation of orientation was operationalized by comparing the orientation reconstructions with multivariate inverted encoding models (IEM) trained in location context-dependent vs. -independent data. Although orientation reconstructions were robust for both location-dependent and location-independent IEMs, they were markedly stronger for the former. Furthermore, the functional relevance of location context was demonstrated by the fact that only the location-dependent neural representations of stimulus orientation predicted recall precision. NEW & NOTEWORTHY Neural representation strength of stimulus location predicts the precision of visual short-term memory (VSTM) recall of nonspatial stimulus, even when this information is task irrelevant. Neural representations of nonspatial stimuli that incorporate location context are stronger than those that do not, and only the former representations are strongly linked to behavior. The contributions to nonspatial VSTM performance of the representation of location context are at least partly distinct from those of the representation of stimulus content.

Confidence boosts serial dependence in orientation estimation.

In the absence of external feedback, a decision maker must rely on a subjective estimate of their decision accuracy in order to appropriately guide behavior. Normative models of perceptual decision-making relate subjective estimates of internal signal quality (e.g., confidence) directly to the internal signal quality itself, thereby making it unknowable whether the subjective estimate or the underlying signal is what drives behavior. We constructed stimuli that dissociated the human observer's performance on a visual estimation task from their subjective estimates of confidence in their performance, thus violating normative principles. To understand whether confidence influences future decision-making, we examined serial dependence in observer's responses, a phenomenon whereby the estimate of a stimulus on the current trial can be biased toward the stimulus from the previous trial. We found that when decisions were made with high confidence, they conferred stronger biases upon the following trial, suggesting that confidence may enhance serial dependence. Critically, this finding was true also when confidence was experimentally dissociated from task performance, indicating that subjective confidence, independent of signal quality, can amplify serial dependence. These findings demonstrate an effect of confidence on future behavior, independent of task performance, and suggest that perceptual decisions incorporate recent history in an uncertainty-weighted manner, but where the uncertainty carried forward is a subjectively estimated and possibly suboptimal readout of objective sensory uncertainty.

Distinct Oscillatory Frequencies Underlie Excitability of Human Occipital and Parietal Cortex.

Transcranial magnetic stimulation (TMS) of human occipital and posterior parietal cortex can give rise to visual sensations called phosphenes. We used near-threshold TMS with concurrent EEG recordings to measure how oscillatory brain dynamics covary, on single trials, with the perception of phosphenes after occipital and parietal TMS. Prestimulus power and phase, predominantly in the alpha band (8-13 Hz), predicted occipital TMS phosphenes, whereas higher-frequency beta-band (13-20 Hz) power (but not phase) predicted parietal TMS phosphenes. TMS-evoked responses related to phosphene perception were similar across stimulation sites and were characterized by an early (200 ms) posterior negativity and a later (>300 ms) parietal positivity in the time domain and an increase in low-frequency (∼5-7 Hz) power followed by a broadband decrease in alpha/beta power in the time-frequency domain. These correlates of phosphene perception closely resemble known electrophysiological correlates of conscious perception of near-threshold visual stimuli. The regionally differential pattern of prestimulus predictors of phosphene perception suggests that distinct frequencies may reflect cortical excitability in occipital versus posterior parietal cortex, calling into question the broader assumption that the alpha rhythm may serve as a general index of cortical excitability.SIGNIFICANCE STATEMENT Alpha-band oscillations are thought to reflect cortical excitability and are therefore ascribed an important role in gating information transmission across cortex. We probed cortical excitability directly in human occipital and parietal cortex and observed that, whereas alpha-band dynamics indeed reflect excitability of occipital areas, beta-band activity was most predictive of parietal cortex excitability. Differences in the state of cortical excitability predicted perceptual outcomes (phosphenes), which were manifest in both early and late patterns of evoked activity, revealing the time course of phosphene perception. Our findings prompt revision of the notion that alpha activity reflects excitability across all of cortex and suggest instead that excitability in different regions is reflected in distinct frequency bands.

Are the Neural Correlates of Consciousness in the Front or in the Back of the Cerebral Cortex? Clinical and Neuroimaging Evidence.

The role of the frontal cortex in consciousness remains a matter of debate. In this Perspective, we will critically review the clinical and neuroimaging evidence for the involvement of the front versus the back of the cortex in specifying conscious contents and discuss promising research avenues.Dual Perspectives Companion Paper: Should a Few Null Findings Falsify Prefrontal Theories of Conscious Perception?, by Brian Odegaard, Robert T. Knight, and Hakwan Lau.