Single-pulse Transcranial-magnetic Stimulation Affects Working-memory Performance via Posterior Beta-band Oscillations.

A single pulse of TMS (spTMS) during the delay period of a double serial retrocuing working-memory task can briefly rescue decodability of an unprioritized memory item (UMI). This physiological phenomenon, which is paralleled in behavior by involuntary retrieval of the UMI, is carried by the beta frequency band, implicating beta-band dynamics in priority coding in working memory. We decomposed EEG data from 12 participants performing double serial retrocuing with concurrent delivery of spTMS using Spatially distributed PhAse Coupling Extraction. This procedure decomposes the scalp-level signal into a set of discrete coupled oscillators, each with a component strength that can vary over time. The decomposition revealed a diversity of low-frequency components, a subset of them strengthening with the onset of the task, and the majority declining in strength across the trial, as well as within each delay period. Results with spTMS revealed no evidence that it works by activating previously "silent" sources; instead, it had the effect of modulating ongoing activity, specifically by exaggerating the within-delay decrease in strength of posterior beta components. Furthermore, the magnitude of the effect of spTMS on the loading strength of a posterior beta component correlated with the disruptive effect of spTMS on performance, a pattern also seen when analyses were restricted to trials with "UMI-lure" memory probes. Rather than reflecting the "activation" of a putatively "activity silent" UMI, these results implicate beta-band dynamics in a mechanism that distinguishes prioritized from unprioritized, and suggest that the effect of spTMS is to disrupt this code.

Representing Context and Priority in Working Memory.

The ability to prioritize among contents in working memory (WM) is critical for successful control of thought and behavior. Recent work has demonstrated that prioritization in WM can be implemented by representing different states of priority in different representational formats. Here, we explored the mechanisms underlying WM prioritization by simulating the double serial retrocuing task with recurrent neural networks. Visualization of stimulus representational dynamics using principal component analysis revealed that the network represented trial context (order of presentation) and priority via different mechanisms. Ordinal context, a stable property lasting the duration of the trial, was accomplished by segregating representations into orthogonal subspaces. Priority, which changed multiple times during a trial, was accomplished by separating representations into different strata within each subspace. We assessed the generality of these mechanisms by applying dimensionality reduction and multiclass decoding to fMRI and EEG data sets and found that priority and context are represented differently along the dorsal visual stream and that behavioral performance is sensitive to trial-by-trial variability of priority coding, but not context coding.

Task feedback suggests a post-perceptual component to serial dependence.

Decisions across a range of perceptual tasks are biased toward past stimuli. Such serial dependence is thought to be an adaptive low-level mechanism that promotes perceptual stability across time. However, recent studies suggest post-perceptual mechanisms may also contribute to serially biased responses, calling into question a single locus of serial dependence and the nature of integration of past and present sensory inputs. We measured serial dependence in the context of a three-dimensional (3D) motion perception task where uncertainty in the sensory information varied substantially from trial to trial. We found that serial dependence varied with stimulus properties that impact sensory uncertainty on the current trial. Reduced stimulus contrast was associated with an increased bias toward the stimulus direction of the previous trial. Critically, performance feedback, which reduced sensory uncertainty, abolished serial dependence. These results provide clear evidence for a post-perceptual locus of serial dependence in 3D motion perception and support the role of serial dependence as a response strategy in the face of substantial sensory uncertainty.

Strategic control of location and ordinal context in visual working memory.

Working memory (WM) requires encoding stimulus identity and context (e.g. where or when stimuli were encountered). To explore the neural bases of the strategic control of context binding in WM, we acquired fMRI while subjects performed delayed recognition of 3 orientation patches presented serially and at different locations. The recognition probe was an orientation patch with a superimposed digit, and pretrial instructions directed subjects to respond according to its location ("location-relevant"), to the ordinal position corresponding to its digit ("order-relevant"), or to just its orientation (relative to all three samples; "context-irrelevant"). Delay period signal in PPC was greater for context-relevant than for "context-irrelevant" trials, and multivariate decoding revealed strong sensitivity to context binding requirements (relevant vs. "irrelevant") and to context domain ("location-" vs. "order-relevant") in both occipital cortex and PPC. At recognition, multivariate inverted encoding modeling revealed markedly different patterns in these 2 regions, suggesting different context-processing functions. In occipital cortex, an active representation of the location of each of the 3 samples was reinstated regardless of the trial type. The pattern in PPC, by contrast, suggested a trial type-dependent filtering of sample information. These results indicate that PPC exerts strategic control over the representation of stimulus context in visual WM.

Context Binding in Visual Working Memory Is Reflected in Bilateral Event-Related Potentials, But Not in Contralateral Delay Activity.

Successful retrieval of a specific item from visual working memory (VWM) depends on the binding of that item to its unique context. Recent functional magnetic resonance imaging studies of VWM manipulating memory set homogeneity have identified an important role for the intraparietal sulcus in context binding, independent of any role in representing stimulus identity. The current study explored whether the contralateral delay activity (CDA), which is an event-related potential (ERP) component derived from posterior electrodes that tracks the amount of information held in VWM, might also be sensitive to context-binding demands. In experiment 1, human participants performed lateralized delayed recognition with memory sets containing one, three, or five items that were drawn from the same category (orientations: "homogeneous") or from different categories (orientation, color, and luminance: "heterogeneous"). Because the location and identity of the memory probe indicated the item to be retrieved, homogeneous trials placed higher context-binding demands. VWM capacity was higher in heterogeneous trials. ERPs contralateral (contra) and ipsilateral (ipsi) to the remembered stimuli were higher for homogeneous trials, but these differences were removed in the contra - ipsi subtraction that produced the CDA. In experiment 2, human participants performed lateralized delayed recall with memory sets of one or three items (homogeneous or heterogeneous). Behavior was superior for three-item heterogeneous trials than for homogeneous trials, with modeling revealing context-binding errors in the latter. Bilateral ERPs and CDA results replicated experiment 1. These results support that the CDA tracks the number of object files engaged by VWM and establish that it is not sensitive to context-binding demands.

Flexible top-down control in the interaction between working memory and perception.

Successful goal-directed behavior often requires continuous sensory processing while simultaneously maintaining task-related information in working memory (WM). Although WM and perception are known to interact, little is known about how their interactions are controlled. Here, we tested the hypothesis that WM perception interactions engage two distinct modes of control - proactive and reactive - in a manner similar to classic conflict-adaptation tasks (e.g. Stroop, flanker, and Simon). Participants performed a delayed recall-of-orientation WM task, plus a standalone visual discrimination-of-orientation task the occurred during the delay period, and with the congruity in orientation between the tasks manipulated. Proactive control was seen in the sensitivity of task performance to the previous trial's congruity (i.e. a Gratton effect). Reactive control was observed in a repulsive serial-dependence produced by incongruent discriminanda. Quantitatively, these effects were explained by parameters from a reinforcement learning-based model that tracks trial-to-trial fluctuations in control demand: reactive control by a phasic control prediction error (control PE), and proactive control by a tonic level of predicted conflict updated each trial by the control PE. Thus, WM-perception interactions may be controlled by the same mechanisms that govern conflict in other domains of cognition, such as response selection.

Head jitter enhances three-dimensional motion perception.

Motion perception is a critical function of the visual system. In a three-dimensional environment, multiple sensory cues carry information about an object's motion trajectory. Previous work has quantified the contribution of binocular motion cues, such as interocular velocity differences and changing disparities over time, as well as monocular motion cues, such as size and density changes. However, even when these cues are presented in concert, observers will systematically misreport the direction of motion-in-depth. Although in the majority of laboratory experiments head position is held fixed using a chin or head rest, an observer's head position is subject to involuntary small movements under real-world viewing conditions. Here, we considered the potential impact of such "head jitter" on motion-in-depth perception. We presented visual stimuli in a head-mounted virtual reality device that facilitated low latency head tracking and asked observers to judge 3D object motion. We found performance improved when we updated the visual display consistent with the small changes in head position. When we disrupted or delayed head movement-contingent updating of the visual display, the proportion of motion-in-depth misreports again increased, reflected in both a reduction in sensitivity and an increase in bias. Our findings identify a critical function of head jitter in visual motion perception, which has been obscured in most (head-fixed and non-head jitter contingent) laboratory experiments.

Gender (Im)balance in Citation Practices in Cognitive Neuroscience.

In the field of neuroscience, despite the fact that the proportion of peer-reviewed publications authored by women has increased in recent decades, the proportion of citations of women-led publications has not seen a commensurate increase: In five broad-scope journals, citations of papers first- and/or last-authored by women have been shown to be fewer than would be expected if gender was not a factor in citation decisions [Dworkin, J. D., Linn, K. A., Teich, E. G., Zurn, P., Shinohara, R. T., & Bassett, D. S. The extent and drivers of gender imbalance in neuroscience reference lists. Nature Neuroscience, 23, 918-926, 2020]. Given the important implications that such underrepresentation may have on the careers of women researchers, it is important to determine whether this same trend is true in subdisciplines of the field, where interventions might be more targeted. Here, we report the results of an extension of the analyses carried out by Dworkin et al. (2020) to citation patterns in the Journal of Cognitive Neuroscience. The results indicate that the underrepresentation of women-led publications in reference sections is also characteristic of papers published in Journal of Cognitive Neuroscience over the past decade. Furthermore, this pattern of citation imbalances is present regardless of author gender, implicating systemic factors. These results contribute to the growing body of evidence that intentional action is needed to address inequities in the way that we carry out and communicate our science.

The Role of Location-Context Binding in Nonspatial Visual Working Memory.

Successful retrieval of an item from visual working memory (VWM) often requires an associated representation of the trial-unique context in which that item was presented. In experiment 1, fMRI of 16 male and female humans replicated a previous dissociation of the effects of manipulating memory load in comparison to the effects of manipulating context binding, by comparing VWM for one oriented line versus for three lines individuated by their location versus for three "heterogeneous" items drawn from different categories (orientation, color, and luminance): delay-period fMRI signal in frontal cortex and intraparietal sulcus (IPS) was sensitive to stimulus homogeneity rather than to memory load per se. Additionally, inspection of behavioral performance revealed a broad range of individual differences in the probability of responses to nontargets (also known as "swap errors"), and a post hoc comparison of high swap-error versus low swap-error groups generated several intriguing results: at recall, high swap-error subjects were seen to represent both the orientation and the location of the probed item less strongly, and with less differentiation from nonprobed items, and delay-period signal in IPS predicted behavioral and neural correlates of context binding at recall. In experiment 2, which was a preregistered replication, the 27 male and female humans were grouped into low and high swap-error groups by median split, and the results were broadly consistent with experiment 1. These results present a neural correlate of swap errors, and suggest that delay-period activity of the IPS may be more important for the operation of context binding than for representation per se of stimulus identity.

Cognitive Control, Not Time, Determines the Status of Items in Working Memory.

Despite the fact that multiple items can be held in working memory (WM), it is often the case that only one of these is relevant for guiding in-the-moment behavior. Therefore, understanding how priority is established and controlled in WM is an important problem. Data from Rose et al. (2016) have provided evidence that although neuroimaging evidence for an active trace of an "unprioritized memory item" (UMI) held in WM drops to baseline levels, evidence for its retention in WM can be "reactivated" by a single pulse of transcranial magnetic stimulation (TMS). Critically, this TMS-reactivation effect was specific to the first delay period of a dual serial retrocue (DSR) task, when the UMI could be needed for the trial's second memory probe, and was not observed during the second delay period, when the uncued item was no longer needed (i.e., when it is an "irrelevant memory item" [IMI]). A problem for the interpretation of these results, however, is that the status of the UMI/IMI was confounded with time spent in WM, as well as with the number of intervening cognitive operations. Here, we report data from a follow-up study designed to replicate the findings Rose et al. (2016) and to add a condition that unconfounds time-since-sample-presentation and UMI/IMI status. The results indicate that the TMS-reactivation effect is, indeed, an index of status in WM (UMI vs. IMI), and not a mere consequence of time elapsed since sample presentation.

Cue-dependent effects of VR experience on motion-in-depth sensitivity.

The visual system exploits multiple signals, including monocular and binocular cues, to determine the motion of objects through depth. In the laboratory, sensitivity to different three-dimensional (3D) motion cues varies across observers and is often weak for binocular cues. However, laboratory assessments may reflect factors beyond inherent perceptual sensitivity. For example, the appearance of weak binocular sensitivity may relate to extensive prior experience with two-dimensional (2D) displays in which binocular cues are not informative. Here we evaluated the impact of experience on motion-in-depth (MID) sensitivity in a virtual reality (VR) environment. We tested a large cohort of observers who reported having no prior VR experience and found that binocular cue sensitivity was substantially weaker than monocular cue sensitivity. As expected, sensitivity was greater when monocular and binocular cues were presented together than in isolation. Surprisingly, the addition of motion parallax signals appeared to cause observers to rely almost exclusively on monocular cues. As observers gained experience in the VR task, sensitivity to monocular and binocular cues increased. Notably, most observers were unable to distinguish the direction of MID based on binocular cues above chance level when tested early in the experiment, whereas most showed statistically significant sensitivity to binocular cues when tested late in the experiment. This result suggests that observers may discount binocular cues when they are first encountered in a VR environment. Laboratory assessments may thus underestimate the sensitivity of inexperienced observers to MID, especially for binocular cues.

Systematic misperceptions of 3-D motion explained by Bayesian inference.

People make surprising but reliable perceptual errors. Here, we provide a unified explanation for systematic errors in the perception of three-dimensional (3-D) motion. To do so, we characterized the binocular retinal motion signals produced by objects moving through arbitrary locations in 3-D. Next, we developed a Bayesian model, treating 3-D motion perception as optimal inference given sensory noise in the measurement of retinal motion. The model predicts a set of systematic perceptual errors, which depend on stimulus distance, contrast, and eccentricity. We then used a virtual-reality headset as well as a standard 3-D desktop stereoscopic display to test these predictions in a series of perceptual experiments. As predicted, we found evidence that errors in 3-D motion perception depend on the contrast, viewing distance, and eccentricity of a stimulus. These errors include a lateral bias in perceived motion direction and a surprising tendency to misreport approaching motion as receding and vice versa. In sum, we present a Bayesian model that provides a parsimonious account for a range of systematic misperceptions of motion in naturalistic environments.

Probability Learning: Changes in Behavior Across Time and Development.

Individuals track probabilities, such as associations between events in their environments, but less is known about the degree to which experience-within a learning session and over development-influences people's use of incoming probabilistic information to guide behavior in real time. In two experiments, children (4-11 years) and adults searched for rewards hidden in locations with predetermined probabilities. In Experiment 1, children (n = 42) and adults (n = 32) changed strategies to maximize reward receipt over time. However, adults demonstrated greater strategy change efficiency. Making the predetermined probabilities more difficult to learn (Experiment 2) delayed effective strategy change for children (n = 39) and adults (n = 33). Taken together, these data characterize how children and adults alike react flexibly and change behavior according to incoming information.

Use of cues in virtual reality depends on visual feedback.

3D motion perception is of central importance to daily life. However, when tested in laboratory settings, sensitivity to 3D motion signals is found to be poor, leading to the view that heuristics and prior assumptions are critical for 3D motion perception. Here we explore an alternative: sensitivity to 3D motion signals is context-dependent and must be learned based on explicit visual feedback in novel environments. The need for action-contingent visual feedback is well-established in the developmental literature. For example, young kittens that are passively moved through an environment, but unable to move through it themselves, fail to develop accurate depth perception. We find that these principles also obtain in adult human perception. Observers that do not experience visual consequences of their actions fail to develop accurate 3D motion perception in a virtual reality environment, even after prolonged exposure. By contrast, observers that experience the consequences of their actions improve performance based on available sensory cues to 3D motion. Specifically, we find that observers learn to exploit the small motion parallax cues provided by head jitter. Our findings advance understanding of human 3D motion processing and form a foundation for future study of perception in virtual and natural 3D environments.

Sensory uncertainty leads to systematic misperception of the direction of motion in depth.

Although we have made major advances in understanding motion perception based on the processing of lateral (2D) motion signals on computer displays, the majority of motion in the real (3D) world occurs outside of the plane of fixation, and motion directly toward or away from observers has particular behavioral relevance. Previous work has reported a systematic lateral bias in the perception of 3D motion, such that an object on a collision course with an observer's head is frequently judged to miss it, with obvious negative consequences. To better understand this bias, we systematically investigated the accuracy of 3D motion perception while manipulating sensory noise by varying the contrast of a moving target and its position in depth relative to fixation. Inconsistent with previous work, we found little bias under low sensory noise conditions. With increased sensory noise, however, we revealed a novel perceptual phenomenon: observers demonstrated a surprising tendency to confuse the direction of motion-in-depth, such that approaching objects were reported to be receding and vice versa. Subsequent analysis revealed that the lateral and motion-in-depth components of observers' reports are similarly affected, but that the effects on the motion-in-depth component (i.e., the motion-in-depth confusions) are much more apparent than those on the lateral component. In addition to revealing this novel visual phenomenon, these results shed new light on errors that can occur in motion perception and provide a basis for continued development of motion perception models. Finally, our findings suggest methods to evaluate the effectiveness of 3D visualization environments, such as 3D movies and virtual reality devices.

Revealing individual differences in strategy selection through visual motion extrapolation.

Humans are constantly challenged to make use of internal models to fill in missing sensory information. We measured human performance in a simple motion extrapolation task where no feedback was provided in order to elucidate the models of object motion incorporated into observers' extrapolation strategies. There was no "right" model for extrapolation in this task. Observers consistently adopted one of two models, linear or quadratic, but different observers chose different models. We further demonstrate that differences in motion sensitivity impact the choice of internal models for many observers. These results demonstrate that internal models and individual differences in those models can be elicited by unconstrained, predictive-based psychophysical tasks.

Task-specific response strategy selection on the basis of recent training experience.

The goal of training is to produce learning for a range of activities that are typically more general than the training task itself. Despite a century of research, predicting the scope of learning from the content of training has proven extremely difficult, with the same task producing narrowly focused learning strategies in some cases and broadly scoped learning strategies in others. Here we test the hypothesis that human subjects will prefer a decision strategy that maximizes performance and reduces uncertainty given the demands of the training task and that the strategy chosen will then predict the extent to which learning is transferable. To test this hypothesis, we trained subjects on a moving dot extrapolation task that makes distinct predictions for two types of learning strategy: a narrow model-free strategy that learns an input-output mapping for training stimuli, and a general model-based strategy that utilizes humans' default predictive model for a class of trajectories. When the number of distinct training trajectories is low, we predict better performance for the mapping strategy, but as the number increases, a predictive model is increasingly favored. Consonant with predictions, subject extrapolations for test trajectories were consistent with using a mapping strategy when trained on a small number of training trajectories and a predictive model when trained on a larger number. The general framework developed here can thus be useful both in interpreting previous patterns of task-specific versus task-general learning, as well as in building future training paradigms with certain desired outcomes.

An experimental criterion for consistency in interpolation of partly occluded contours.

We develop an experimental measure of consistency of interpolation of partly occluded contours based entirely on observers' interpolation performance. We first describe the measure, which is based on a two-probe task that compares an observer's interpolation settings at a particular location with vs. without the observer's own setting presented at a nearby location. We then report two experiments aimed at investigating the behavior of the measure. The first compares the proposed measure to the predictions of contour completion models. The second investigates its performance in the Poggendorff and related configurations. We find that consistency covaries with relatability and cocircularity (both interpreted in graded terms) and, sensibly, yields a low measure for interpolation in the Poggendorff configuration. We conclude that the proposed measure of consistency operationalizes an important aspect of what is meant by the "strength" of a partly occluded contour.

Precision and consistency of contour interpolation.

We investigated conditions under which observers can interpolate occluded contours by a single, stable, smooth contour. Observers viewed partly-occluded contours defined by linear segments and estimated the position and tangent orientation of the contour at multiple locations within the occluded region. We measured the precision and consistency of observers' settings as indices of successful interpolation. We found that although increasing the relative offset between inducers led to a decrease in both precision and consistency, increasing turning angle affected primarily precision. We discuss conditions under which interpolation settings are consistent with a single, stable smooth contour.