Effective connectivity of working memory performance: a DCM study of MEG data.

Visual working memory (WM) engages several nodes of a large-scale network that includes frontal, parietal, and visual regions; however, little is understood about how these regions interact to support WM behavior. In particular, it is unclear whether network dynamics during WM maintenance primarily represent feedforward or feedback connections. This question has important implications for current debates about the relative roles of frontoparietal and visual regions in WM maintenance. In the current study, we investigated the network activity supporting WM using MEG data acquired while healthy subjects performed a multi-item delayed estimation WM task. We used computational modeling of behavior to discriminate correct responses (high accuracy trials) from two different types of incorrect responses (low accuracy and swap trials), and dynamic causal modeling of MEG data to measure effective connectivity. We observed behaviorally dependent changes in effective connectivity in a brain network comprising frontoparietal and early visual areas. In comparison with high accuracy trials, frontoparietal and frontooccipital networks showed disrupted signals depending on type of behavioral error. Low accuracy trials showed disrupted feedback signals during early portions of WM maintenance and disrupted feedforward signals during later portions of maintenance delay, while swap errors showed disrupted feedback signals during the whole delay period. These results support a distributed model of WM that emphasizes the role of visual regions in WM storage and where changes in large scale network configurations can have important consequences for memory-guided behavior.

Alpha phase-coding supports feature binding during working memory maintenance.

The ability to successfully retain and manipulate information in working memory (WM) requires that objects' individual features are bound into cohesive representations; yet, the mechanisms supporting feature binding remain unclear. Binding (or swap) errors, where memorized features are erroneously associated with the wrong object, can provide a window into the intrinsic limits in capacity of WM that represent a key bottleneck in our cognitive ability. We tested the hypothesis that binding in WM is accomplished via neural phase synchrony and that swap errors result from perturbations in this synchrony. Using magnetoencephalography data collected from human subjects in a task designed to induce swap errors, we showed that swaps are characterized by reduced phase-locked oscillatory activity during memory retention, as predicted by an attractor model of spiking neural networks. Further, we found that this reduction arises from increased phase-coding variability in the alpha-band over a distributed network of sensorimotor areas. Our findings demonstrate that feature binding in WM is accomplished through phase-coding dynamics that emerge from the competition between different memories.

Probabilistic and rich individual working memories revealed by a betting game.

When asked to remember a color, do people remember a point estimate (e.g., a particular shade of red), a point estimate plus an uncertainty estimate, or are memory representations rich probabilistic distributions over feature space? We asked participants to report the color of a circle held in working memory. Rather than collecting a single report per trial, we had participants place multiple bets to create trialwise uncertainty distributions. Bet dispersion correlated with performance, indicating that internal uncertainty guided bet placement. While the first bet was on average the most precisely placed, the later bets systematically shifted the distribution closer to the target, resulting in asymmetrical distributions about the first bet. This resulted in memory performance improvements when averaging across bets, and overall suggests that memory representations contain more information than can be conveyed by a single response. The later bets contained target information even when the first response would generally be classified as a guess or report of an incorrect item, suggesting that such failures are not all-or-none. This paradigm provides multiple pieces of evidence that memory representations are rich and probabilistic. Crucially, standard discrete response paradigms underestimate the amount of information in memory representations.

Mnemonic representations in human lateral geniculate nucleus.

There is a growing appreciation for the role of the thalamus in high-level cognition. Motivated by findings that internal cognitive state drives activity in feedback layers of primary visual cortex (V1) that target the lateral geniculate nucleus (LGN), we investigated the role of LGN in working memory (WM). Specifically, we leveraged model-based neuroimaging approaches to test the hypothesis that human LGN encodes information about spatial locations temporarily encoded in WM. First, we localized and derived a detailed topographic organization in LGN that accords well with previous findings in humans and non-human primates. Next, we used models constructed on the spatial preferences of LGN populations in order to reconstruct spatial locations stored in WM as subjects performed modified memory-guided saccade tasks. We found that population LGN activity faithfully encoded the spatial locations held in memory in all subjects. Importantly, our tasks and models allowed us to dissociate the locations of retinal stimulation and the motor metrics of memory-guided saccades from the maintained spatial locations, thus confirming that human LGN represents true WM information. These findings add LGN to the growing list of subcortical regions involved in WM, and suggest a key pathway by which memories may influence incoming processing at the earliest levels of the visual hierarchy.

Common Neural Mechanisms Control Attention and Working Memory.

Although previous studies point to qualitative similarities between working memory (WM) and attention, the degree to which these two constructs rely on shared neural mechanisms remains unknown. Focusing on one such potentially shared mechanism, we tested the hypothesis that selecting an item within WM utilizes similar neural mechanisms as selecting a visible item via a shift of attention. We used fMRI and machine learning to decode both the selection among items visually available and the selection among items stored in WM in human subjects (both sexes). Patterns of activity in visual, parietal, and to a lesser extent frontal cortex predicted the locations of the selected items. Critically, these patterns were strikingly interchangeable; classifiers trained on data during attentional selection predicted selection from WM, and classifiers trained on data during selection from memory predicted attentional selection. Using models of voxel receptive fields, we visualized topographic population activity that revealed gain enhancements at the locations of the externally and internally selected items. Our results suggest that selecting among perceived items and selecting among items in WM share a common mechanism. This common mechanism, analogous to a shift of spatial attention, controls the relative gains of neural populations that encode behaviorally relevant information.SIGNIFICANCE STATEMENT How we allocate our attention to external stimuli that we see and to internal representations of stimuli stored in memory might rely on a common mechanism. Supporting this hypothesis, we demonstrated that not only could patterns of human brain activity predict which items were selected during perception and memory, but that these patterns were interchangeable during external and internal selection. Additionally, this generalized selection mechanism operates by changes in the gains of the neural populations both encoding attended sensory representations and storing relevant memory representations.

Across-Area Synchronization Supports Feature Integration in a Biophysical Network Model of Working Memory.

Working memory function is severely limited. One key limitation that constrains the ability to maintain multiple items in working memory simultaneously is so-called swap errors. These errors occur when an inaccurate response is in fact accurate relative to a non-target stimulus, reflecting the failure to maintain the appropriate association or "binding" between the features that define one object (e.g., color and location). The mechanisms underlying feature binding in working memory remain unknown. Here, we tested the hypothesis that features are bound in memory through synchrony across feature-specific neural assemblies. We built a biophysical neural network model composed of two one-dimensional attractor networks - one for color and one for location - simulating feature storage in different cortical areas. Within each area, gamma oscillations were induced during bump attractor activity through the interplay of fast recurrent excitation and slower feedback inhibition. As a result, different memorized items were held at different phases of the network's oscillation. These two areas were then reciprocally connected via weak cortico-cortical excitation, accomplishing binding between color and location through the synchronization of pairs of bumps across the two areas. Encoding and decoding of color-location associations was accomplished through rate coding, overcoming a long-standing limitation of binding through synchrony. In some simulations, swap errors arose: "color bumps" abruptly changed their phase relationship with "location bumps." This model, which leverages the explanatory power of similar attractor models, specifies a plausible mechanism for feature binding and makes specific predictions about swap errors that are testable at behavioral and neurophysiological levels.

Working memory representations in visual cortex mediate distraction effects.

Although the contents of working memory can be decoded from visual cortex activity, these representations may play a limited role if they are not robust to distraction. We used model-based fMRI to estimate the impact of distracting visual tasks on working memory representations in several visual field maps in visual and frontoparietal association cortex. Here, we show distraction causes the fidelity of working memory representations to briefly dip when both the memorandum and distractor are jointly encoded by the population activities. Distraction induces small biases in memory errors which can be predicted by biases in neural decoding in early visual cortex, but not other regions. Although distraction briefly disrupts working memory representations, the widespread redundancy with which working memory information is encoded may protect against catastrophic loss. In early visual cortex, the neural representation of information in working memory and behavioral performance are intertwined, solidifying its importance in visual memory.

Spatially Specific Working Memory Activity in the Human Superior Colliculus.

Theoretically, working memory (WM) representations are encoded by population activity of neurons with distributed tuning across the stored feature. Here, we leverage computational neuroimaging approaches to map the topographic organization of human superior colliculus (SC) and model how population activity in SC encodes WM representations. We first modeled receptive field properties of voxels in SC, deriving a detailed topographic organization resembling that of the primate SC. Neural activity within human (5 male and 1 female) SC persisted throughout a retention interval of several types of modified memory-guided saccade tasks. Assuming an underlying neural architecture of the SC based on its retinotopic organization, we used an encoding model to show that the pattern of activity in human SC represents locations stored in WM. Our tasks and models allowed us to dissociate the locations of visual targets and the motor metrics of memory-guided saccades from the spatial locations stored in WM, thus confirming that human SC represents true WM information. These data have several important implications. They add the SC to a growing number of cortical and subcortical brain areas that form distributed networks supporting WM functions. Moreover, they specify a clear neural mechanism by which topographically organized SC encodes WM representations.SIGNIFICANCE STATEMENT Using computational neuroimaging approaches, we mapped the topographic organization of human superior colliculus (SC) and modeled how population activity in SC encodes working memory (WM) representations, rather than simpler visual or motor properties that have been traditionally associated with the laminar maps in the primate SC. Together, these data both position the human SC into a distributed network of brain areas supporting WM and elucidate the neural mechanisms by which the SC supports WM.

The what, where and how of delay activity.

Working memory is characterized by neural activity that persists during the retention interval of delay tasks. Despite the ubiquity of this delay activity across tasks, species and experimental techniques, our understanding of this phenomenon remains incomplete. Although initially there was a narrow focus on sustained activation in a small number of brain regions, methodological and analytical advances have allowed researchers to uncover previously unobserved forms of delay activity various parts of the brain. In light of these new findings, this Review reconsiders what delay activity is, where in the brain it is found, what roles it serves and how it may be generated.

Effects of conflict and strategic processing on neural responses to errors in schizophrenia.

The error-related negativity (ERN) and error-positivity (Pe) are commonly linked to error-detection and strategic processing. Studies have documented the influence of conflict probability on ERN amplitude. However, the influence of conflict probability on ERN/Pe in schizophrenia, where such components are reduced, is unknown. A modified flanker paradigm was used to examine how the probability of conflict modulates ERN and Pe amplitudes in patients with schizophrenia (n = 33) and healthy controls (n = 25). Increased ERN was observed in response to errors on low probability, incongruent trials. No such differences were observed in Pe. While ERN and Pe showed significantly reduced amplitudes in patients relative to controls, patients showed normal condition-dependent ERN and reaction-time modulation. This suggests that while the neural mechanisms generating the ERN and Pe are compromised in schizophrenia, those modulating task performance strategy and neurophysiological responses to errors based on conflict probability are intact.

Flexible Coding of Visual Working Memory Representations during Distraction.

Visual working memory (VWM) recruits a broad network of brain regions, including prefrontal, parietal, and visual cortices. Recent evidence supports a "sensory recruitment" model of VWM, whereby precise visual details are maintained in the same stimulus-selective regions responsible for perception. A key question in evaluating the sensory recruitment model is how VWM representations persist through distracting visual input, given that the early visual areas that putatively represent VWM content are susceptible to interference from visual stimulation.To address this question, we used a functional magnetic resonance imaging inverted encoding model approach to quantitatively assess the effect of distractors on VWM representations in early visual cortex and the intraparietal sulcus (IPS), another region previously implicated in the storage of VWM information. This approach allowed us to reconstruct VWM representations for orientation, both before and after visual interference, and to examine whether oriented distractors systematically biased these representations. In our human participants (both male and female), we found that orientation information was maintained simultaneously in early visual areas and IPS in anticipation of possible distraction, and these representations persisted in the absence of distraction. Importantly, early visual representations were susceptible to interference; VWM orientations reconstructed from visual cortex were significantly biased toward distractors, corresponding to a small attractive bias in behavior. In contrast, IPS representations did not show such a bias. These results provide quantitative insight into the effect of interference on VWM representations, and they suggest a dynamic tradeoff between visual and parietal regions that allows flexible adaptation to task demands in service of VWM.SIGNIFICANCE STATEMENT Despite considerable evidence that stimulus-selective visual regions maintain precise visual information in working memory, it remains unclear how these representations persist through subsequent input. Here, we used quantitative model-based fMRI analyses to reconstruct the contents of working memory and examine the effects of distracting input. Although representations in the early visual areas were systematically biased by distractors, those in the intraparietal sulcus appeared distractor-resistant. In contrast, early visual representations were most reliable in the absence of distraction. These results demonstrate the dynamic, adaptive nature of visual working memory processes, and provide quantitative insight into the ways in which representations can be affected by interference. Further, they suggest that current models of working memory should be revised to incorporate this flexibility.

Short-term retention of visual information: Evidence in support of feature-based attention as an underlying mechanism.

Retention of features in visual short-term memory (VSTM) involves maintenance of sensory traces in early visual cortex. However, the mechanism through which this is accomplished is not known. Here, we formulate specific hypotheses derived from studies on feature-based attention to test the prediction that visual cortex is recruited by attentional mechanisms during VSTM of low-level features. Functional magnetic resonance imaging (fMRI) of human visual areas revealed that neural populations coding for task-irrelevant feature information are suppressed during maintenance of detailed spatial frequency memory representations. The narrow spectral extent of this suppression agrees well with known effects of feature-based attention. Additionally, analyses of effective connectivity during maintenance between retinotopic areas in visual cortex show that the observed highlighting of task-relevant parts of the feature spectrum originates in V4, a visual area strongly connected with higher-level control regions and known to convey top-down influence to earlier visual areas during attentional tasks. In line with this property of V4 during attentional operations, we demonstrate that modulations of earlier visual areas during memory maintenance have behavioral consequences, and that these modulations are a result of influences from V4.

Distributed and dynamic storage of working memory stimulus information in extrastriate cortex.

The predominant neurobiological model of working memory (WM) posits that stimulus information is stored via stable, elevated activity within highly selective neurons. On the basis of this model, which we refer to as the canonical model, the storage of stimulus information is largely associated with lateral PFC (lPFC). A growing number of studies describe results that cannot be fully explained by the canonical model, suggesting that it is in need of revision. In this study, we directly tested key elements of the canonical model. We analyzed fMRI data collected as participants performed a task requiring WM for faces and scenes. Multivariate decoding procedures identified patterns of activity containing information about the items maintained in WM (faces, scenes, or both). Although information about WM items was identified in extrastriate visual cortex (EC) and lPFC, only EC exhibited a pattern of results consistent with a sensory representation. Information in both regions persisted even in the absence of elevated activity, suggesting that elevated population activity may not represent the storage of information in WM. Additionally, we observed that WM information was distributed across EC neural populations that exhibited a broad range of selectivity for the WM items rather than restricted to highly selective EC populations. Finally, we determined that activity patterns coding for WM information were not stable, but instead varied over the course of a trial, indicating that the neural code for WM information is dynamic rather than static. Together, these findings challenge the canonical model of WM.

Revisiting the role of persistent neural activity during working memory.

What are the neural mechanisms underlying working memory (WM)? One influential theory posits that neurons in the lateral prefrontal cortex (lPFC) store WM information via persistent activity. In this review, we critically evaluate recent findings that together indicate that this model of WM needs revision. We argue that sensory cortex, not the lPFC, maintains high-fidelity representations of WM content. By contrast, the lPFC simultaneously maintains representations of multiple goal-related variables that serve to bias stimulus-specific activity in sensory regions. This work highlights multiple neural mechanisms supporting WM, including temporally dynamic population coding in addition to persistent activity. These new insights focus the question on understanding how the mechanisms that underlie WM are related, interact, and are coordinated in the lPFC and sensory cortex.

Evidence for working memory storage operations in perceptual cortex.

Isolating the short-term storage component of working memory (WM) from the myriad of associated executive processes has been an enduring challenge. Recent efforts have identified patterns of activity in visual regions that contain information about items being held in WM. However, it remains unclear (1) whether these representations withstand intervening sensory input and (2) how communication between multimodal association cortex and the unimodal perceptual regions supporting WM representations is involved in WM storage. We present evidence that the features of a face held in WM are stored within face-processing regions, that these representations persist across subsequent sensory input, and that information about the match between sensory input and a memory representation is relayed forward from perceptual to prefrontal regions. Participants were presented with a series of probe faces and indicated whether each probe matched a target face held in WM. We parametrically varied the feature similarity between the probe and target faces. Activity within face-processing regions scaled linearly with the degree of feature similarity between the probe face and the features of the target face, suggesting that the features of the target face were stored in these regions. Furthermore, directed connectivity measures revealed that the direction of information flow that was optimal for performance was from sensory regions that stored the features of the target face to dorsal prefrontal regions, supporting the notion that sensory input is compared to representations stored within perceptual regions and is subsequently relayed forward. Together, these findings indicate that WM storage operations are carried out within perceptual cortex.

Correspondence between stimulus encoding- and maintenance-related neural processes underlies successful working memory.

The ability to actively maintain information in working memory (WM) is vital for goal-directed behavior, but the mechanisms underlying this process remain elusive. We hypothesized that successful WM relies upon a correspondence between the neural processes associated with stimulus encoding and the neural processes associated with maintenance. Using functional magnetic resonance imaging, we identified regional activity and inter-regional connectivity during stimulus encoding and the maintenance of those stimuli when they were no longer present. We compared correspondence in these neural processes across encoding and maintenance epochs with WM performance. Critically, greater correspondence between encoding and maintenance in 1) regional activity in the lateral prefrontal cortex (PFC) and 2) connectivity between lateral PFC and extrastriate cortex was associated with increased performance. These findings suggest that the conservation of neural processes across encoding and maintenance supports the integrity of representations in WM.

Attention selectively modifies the representation of individual faces in the human brain.

Attention modifies neural tuning for low-level features, but it is unclear how attention influences tuning for complex stimuli. We investigated this question in humans using fMRI and face stimuli. Participants were shown six faces (F1-F6) along a morph continuum, and selectivity was quantified by constructing tuning curves for individual voxels. Face-selective voxels exhibited greater responses to their preferred face than to nonpreferred faces, particularly in posterior face areas. Anterior face areas instead displayed tuning for face categories: voxels in these areas preferred either the first (F1-F3) or second (F4-F6) half of the morph continuum. Next, we examined the effects of attention on voxel tuning by having subjects direct attention to one of the superimposed images of F1 and F6. We found that attention selectively enhanced responses in voxels preferring the attended face. Together, our results demonstrate that single voxels carry information about individual faces and that the nature of this information varies across cortical face areas. Additionally, we found that attention selectively enhances these representations. Our findings suggest that attention may act via a unitary principle of selective enhancement of responses to both simple and complex stimuli across multiple stages of the visual hierarchy.

Working memory templates are maintained as feature-specific perceptual codes.

Working memory (WM) representations serve as templates that guide behavior, but the neural basis of these templates remains elusive. We tested the hypothesis that WM templates are maintained by biasing activity in sensoriperceptual neurons that code for features of items being held in memory. Neural activity was recorded using event-related potentials (ERPs) as participants viewed a series of faces and responded when a face matched a target face held in WM. Our prediction was that if activity in neurons coding for the features of the target is preferentially weighted during maintenance of the target, then ERP activity evoked by a nontarget probe face should be commensurate with the visual similarity between target and probe. Visual similarity was operationalized as the degree of overlap in visual features between target and probe. A face-sensitive ERP response was modulated by target-probe similarity. Amplitude was largest for probes that were similar to the target, and decreased monotonically as a function of decreasing target-probe similarity. These results indicate that neural activity is weighted in favor of visual features that comprise an actively held memory representation. As such, our findings support the notion that WM templates rely on neural populations involved in forming percepts of memory items.

Attention to faces modulates early face processing during low but not high face discriminability.

In the present study, we investigated whether attention to faces results in sensory gain modulation. Participants were cued to attend either to faces or to scenes in superimposed face-scene images for which face discriminability was manipulated parametrically. The face-sensitive N170 event-related potential component was used as a measure of early face processing. Attention to faces modulated N170 amplitude, but only when faces were not highly discriminable. Additionally, directing attention to faces modulated later processing (~230-300 msec) for all discriminability levels. These results demonstrate that attention to faces can modulate perceptual processing of faces at multiple stages of processing, including early sensory levels. Critically, the early attentional benefit is present only when the "face signal" (i.e., the perceptual quality of the face) in the environment is suboptimal.

Temporal characteristics of top-down modulations during working memory maintenance: an event-related potential study of the N170 component.

We investigated the top-down influence of working memory (WM) maintenance on feedforward perceptual processing within occipito-temporal face processing structures. During event-related potential (ERP) recordings, subjects performed a delayed-recognition task requiring WM maintenance of faces or houses. The face-sensitive N170 component elicited by delay-spanning task-irrelevant grayscale noise probes was examined. If early feedforward perceptual activity is biased by maintenance requirements, the N170 ERP component elicited by probes should have a greater N170 amplitude response during face relative to house WM trials. Consistent with this prediction, N170 elicited by probes presented at the beginning, middle, and end of the delay interval was greater in amplitude during face relative to house WM. Thus, these results suggest that WM maintenance demands may modulate early feedforward perceptual processing for the entirety of the delay duration. We argue based on these results that temporally early biasing of domain-specific perceptual processing may be a critical mechanism by which WM maintenance is achieved.