Flexible Coding of Visual Working Memory Representations during Distraction.

Visual working memory (VWM) recruits a broad network of brain regions, including prefrontal, parietal, and visual cortices. Recent evidence supports a "sensory recruitment" model of VWM, whereby precise visual details are maintained in the same stimulus-selective regions responsible for perception. A key question in evaluating the sensory recruitment model is how VWM representations persist through distracting visual input, given that the early visual areas that putatively represent VWM content are susceptible to interference from visual stimulation.To address this question, we used a functional magnetic resonance imaging inverted encoding model approach to quantitatively assess the effect of distractors on VWM representations in early visual cortex and the intraparietal sulcus (IPS), another region previously implicated in the storage of VWM information. This approach allowed us to reconstruct VWM representations for orientation, both before and after visual interference, and to examine whether oriented distractors systematically biased these representations. In our human participants (both male and female), we found that orientation information was maintained simultaneously in early visual areas and IPS in anticipation of possible distraction, and these representations persisted in the absence of distraction. Importantly, early visual representations were susceptible to interference; VWM orientations reconstructed from visual cortex were significantly biased toward distractors, corresponding to a small attractive bias in behavior. In contrast, IPS representations did not show such a bias. These results provide quantitative insight into the effect of interference on VWM representations, and they suggest a dynamic tradeoff between visual and parietal regions that allows flexible adaptation to task demands in service of VWM.SIGNIFICANCE STATEMENT Despite considerable evidence that stimulus-selective visual regions maintain precise visual information in working memory, it remains unclear how these representations persist through subsequent input. Here, we used quantitative model-based fMRI analyses to reconstruct the contents of working memory and examine the effects of distracting input. Although representations in the early visual areas were systematically biased by distractors, those in the intraparietal sulcus appeared distractor-resistant. In contrast, early visual representations were most reliable in the absence of distraction. These results demonstrate the dynamic, adaptive nature of visual working memory processes, and provide quantitative insight into the ways in which representations can be affected by interference. Further, they suggest that current models of working memory should be revised to incorporate this flexibility.

Induction and Relief of Curiosity Elicit Parietal and Frontal Activity.

Curiosity is a basic biological drive, but little is known about its behavioral and neural mechanisms. We can be curious about several types of information. On the one hand, curiosity is a function of the expected value of information, serving primarily to help us maximize reward. On the other hand, curiosity can be a function of the uncertainty of information, helping us to update what we know. In the current studies, we aimed to disentangle the contribution of information uncertainty and expected value of rewards to curiosity in humans. To this end, we designed a lottery task in which uncertainty and expected value of trial outcomes were manipulated independently and examined how neural activity and behavioral measures of curiosity were modulated by these factors. Curiosity increased linearly with increased outcome uncertainty, both when curiosity was explicitly probed as well as when it was implicitly tested by people's willingness to wait. Increased expected value, however, did not strongly relate to these curiosity measures. Neuroimaging results showed greater BOLD response with increasing outcome uncertainty in parietal cortex at the time of curiosity induction. Outcome updating when curiosity was relieved resulted in an increased signal in the insula, orbitofrontal cortex, and parietal cortex. Furthermore, the insula showed a linear increase corresponding to the size of the information update. These results suggest that curiosity is monotonically related to the uncertainty about one's current world model, the induction and relief of which are associated with activity in parietal and insular cortices, respectively.SIGNIFICANCE STATEMENT Humans are curious by nature. When you hear your phone beep, you probably feel the urge to check the message right away, even though the message itself likely does not give you a direct reward. In this study, we demonstrated that curiosity can be driven by outcome uncertainty, over and above of reward. The induction of curiosity was accompanied by increased activity in the parietal cortex, whereas the information update at the time of curiosity relief was associated with activity in insular cortex. These findings advance our understanding of the behavioral and neural constituents of curiosity, which lies at the core of human information seeking and serves to optimize the individual's current world model.

Neural Correlates of Visual Short-term Memory Dissociate between Fragile and Working Memory Representations.

Evidence is accumulating that the classic two-stage model of visual STM (VSTM), comprising iconic memory (IM) and visual working memory (WM), is incomplete. A third memory stage, termed fragile VSTM (FM), seems to exist in between IM and WM [Vandenbroucke, A. R. E., Sligte, I. G., & Lamme, V. A. F. Manipulations of attention dissociate fragile visual STM from visual working memory. Neuropsychologia, 49, 1559-1568, 2011; Sligte, I. G., Scholte, H. S., & Lamme, V. A. F. Are there multiple visual STM stores? PLoS One, 3, e1699, 2008]. Although FM can be distinguished from IM using behavioral and fMRI methods, the question remains whether FM is a weak expression of WM or a separate form of memory with its own neural signature. Here, we tested whether FM and WM in humans are supported by dissociable time-frequency features of EEG recordings. Participants performed a partial-report change detection task, from which individual differences in FM and WM capacity were estimated. These individual FM and WM capacities were correlated with time-frequency characteristics of the EEG signal before and during encoding and maintenance of the memory display. FM capacity showed negative alpha correlations over peri-occipital electrodes, whereas WM capacity was positively related, suggesting increased visual processing (lower alpha) to be related to FM capacity. Furthermore, FM capacity correlated with an increase in theta power over central electrodes during preparation and processing of the memory display, whereas WM did not. In addition to a difference in visual processing characteristics, a positive relation between gamma power and FM capacity was observed during both preparation and maintenance periods of the task. On the other hand, we observed that theta-gamma coupling was negatively correlated with FM capacity, whereas it was slightly positively correlated with WM. These data show clear differences in the neural substrates of FM versus WM and suggest that FM depends more on visual processing mechanisms compared with WM. This study thus provides novel evidence for a dissociation between different stages in VSTM.

Seeing without knowing: neural signatures of perceptual inference in the absence of report.

Every day, we experience a rich and complex visual world. Our brain constantly translates meaningless fragmented input into coherent objects and scenes. However, our attentional capabilities are limited, and we can only report the few items that we happen to attend to. So what happens to items that are not cognitively accessed? Do these remain fragmentary and meaningless? Or are they processed up to a level where perceptual inferences take place about image composition? To investigate this, we recorded brain activity using fMRI while participants viewed images containing a Kanizsa figure, an illusion in which an object is perceived by means of perceptual inference. Participants were presented with the Kanizsa figure and three matched nonillusory control figures while they were engaged in an attentionally demanding distractor task. After the task, one group of participants was unable to identify the Kanizsa figure in a forced-choice decision task; hence, they were "inattentionally blind." A second group had no trouble identifying the Kanizsa figure. Interestingly, the neural signature that was unique to the processing of the Kanizsa figure was present in both groups. Moreover, within-subject multivoxel pattern analysis showed that the neural signature of unreported Kanizsa figures could be used to classify reported Kanizsa figures and that this cross-report classification worked better for the Kanizsa condition than for the control conditions. Together, these results suggest that stimuli that are not cognitively accessed are processed up to levels of perceptual interpretation.

Accurate metacognition for visual sensory memory representations.

The capacity to attend to multiple objects in the visual field is limited. However, introspectively, people feel that they see the whole visual world at once. Some scholars suggest that this introspective feeling is based on short-lived sensory memory representations, whereas others argue that the feeling of seeing more than can be attended to is illusory. Here, we investigated this phenomenon by combining objective memory performance with subjective confidence ratings during a change-detection task. This allowed us to compute a measure of metacognition--the degree of knowledge that subjects have about the correctness of their decisions--for different stages of memory. We show that subjects store more objects in sensory memory than they can attend to but, at the same time, have similar metacognition for sensory memory and working memory representations. This suggests that these subjective impressions are not an illusion but accurate reflections of the richness of visual perception.

Confuse your illusion: feedback to early visual cortex contributes to perceptual completion.

A striking example of the constructive nature of visual perception is how the human visual system completes contours of occluded objects. To date, it is unclear whether perceptual completion emerges during early stages of visual processing or whether higher-level mechanisms are necessary. To answer this question, we used transcranial magnetic stimulation to disrupt signaling in V1/V2 and in the lateral occipital (LO) area at different moments in time while participants performed a discrimination task involving a Kanizsa-type illusory figure. Results show that both V1/V2 and higher-level visual area LO are critically involved in perceptual completion. However, these areas seem to be involved in an inverse hierarchical fashion, in which the critical time window for V1/V2 follows that for LO. These results are in line with the growing evidence that feedback to V1/V2 contributes to perceptual completion.

Understanding the Dynamics of the Developing Adolescent Brain Through Team Science.

One of the major goals for research on adolescent development is to identify the optimal conditions for adolescents to grow up in a complex social world and to understand individual differences in these trajectories. Based on influential theoretical and empirical work in this field, achieving this goal requires a detailed understanding of the social context in which neural and behavioral development takes place, along with longitudinal measurements at multiple levels (e.g., genetic, hormonal, neural, behavioral). In this perspectives article, we highlight the promising role of team science in achieving this goal. To illustrate our point, we describe meso (peer relations) and micro (social learning) approaches to understand social development in adolescence as crucial aspects of adolescent mental health. Finally, we provide an overview of how our team has extended our collaborations beyond scientific partners to multiple societal partners for the purpose of informing and including policymakers, education and health professionals, as well as adolescents themselves when conducting and communicating research.

Integrating Cognitive Developmental Neuroscience in Society: Lessons Learned From a Multidisciplinary Research Project on Education and Social Safety of Youth.

Integrating fundamental science in society, with the goal to translate research findings to daily practice, comes with certain challenges. Successfully integrating research projects into society requires (1) good collaboration between scientists and societal stakeholders, (2) collaboration partners with common expectations and goals, and (3) investment in clear communication. Here we describe an integrative research project conducted by a large Dutch consortium that consisted of neuroscientists, psychologists, sociologists, ethicists, teachers, health care professionals and policy makers, focusing on applying cognitive developmental neuroscience for the benefit of youth in education and social safety. We argue that to effectively integrate cognitive developmental neuroscience in society, (1) it is necessary to invest in a well-functioning, diverse and multidisciplinary team involving societal stakeholders and youth themselves from the start of the project. This aids to build a so-called productive interactive network that increases the chances to realize societal impact in the long-term. Additionally, we propose that to integrate knowledge, (2) a different than standard research approach should be taken. When focusing on integration, the ultimate goal of research is not solely to understand the world better, but also to intervene with real-life situations, such as education or (forensic) youth care. To accomplish this goal, we propose an approach in which integration is not only started after the research has been conducted, but taken into account throughout the entire project. This approach helps to create common expectations and goals between different stakeholders. Finally, we argue that (3) dedicating sufficient resources to effective communication, both within the consortium and between scientists and society, greatly benefits the integration of cognitive developmental neuroscience in society.

Dissociable neural mechanisms underlie currently-relevant, future-relevant, and discarded working memory representations.

In daily life, we use visual working memory (WM) to guide our actions. While attending to currently-relevant information, we must simultaneously maintain future-relevant information, and discard information that is no longer relevant. However, the neural mechanisms by which unattended, but future-relevant, information is maintained in working memory, and future-irrelevant information is discarded, are not well understood. Here, we investigated representations of these different information types, using functional magnetic resonance imaging in combination with multivoxel pattern analysis and computational modeling based on inverted encoding model simulations. We found that currently-relevant WM information in the focus of attention was maintained through representations in visual, parietal and posterior frontal brain regions, whereas deliberate forgetting led to suppression of the discarded representations in early visual cortex. In contrast, future-relevant information was neither inhibited nor actively maintained in these areas. These findings suggest that different neural mechanisms underlie the WM representation of currently- and future-relevant information, as compared to information that is discarded from WM.

Non-attended representations are perceptual rather than unconscious in nature.

Introspectively we experience a phenomenally rich world. In stark contrast, many studies show that we can only report on the few items that we happen to attend to. So what happens to the unattended objects? Are these consciously processed as our first person perspective would have us believe, or are they - in fact - entirely unconscious? Here, we attempt to resolve this question by investigating the perceptual characteristics of visual sensory memory. Sensory memory is a fleeting, high-capacity form of memory that precedes attentional selection and working memory. We found that memory capacity benefits from figural information induced by the Kanizsa illusion. Importantly, this benefit was larger for sensory memory than for working memory and depended critically on the illusion, not on the stimulus configuration. This shows that pre-attentive sensory memory contains representations that have a genuinely perceptual nature, suggesting that non-attended representations are phenomenally experienced rather than unconscious.

Manipulations of attention dissociate fragile visual short-term memory from visual working memory.

People often rely on information that is no longer in view, but maintained in visual short-term memory (VSTM). Traditionally, VSTM is thought to operate on either a short time-scale with high capacity - iconic memory - or a long time scale with small capacity - visual working memory. Recent research suggests that in addition, an intermediate stage of memory in between iconic memory and visual working memory exists. This intermediate stage has a large capacity and a lifetime of several seconds, but is easily overwritten by new stimulation. We therefore termed it fragile VSTM. In previous studies, fragile VSTM has been dissociated from iconic memory by the characteristics of the memory trace. In the present study, we dissociated fragile VSTM from visual working memory by showing a differentiation in their dependency on attention. A decrease in attention during presentation of the stimulus array greatly reduced the capacity of visual working memory, while this had only a small effect on the capacity of fragile VSTM. We conclude that fragile VSTM is a separate memory store from visual working memory. Thus, a tripartite division of VSTM appears to be in place, comprising iconic memory, fragile VSTM and visual working memory.

Detailed sensory memory, sloppy working memory.

Visual short-term memory (VSTM) enables us to actively maintain information in mind for a brief period of time after stimulus disappearance. According to recent studies, VSTM consists of three stages - iconic memory, fragile VSTM, and visual working memory - with increasingly stricter capacity limits and progressively longer lifetimes. Still, the resolution (or amount of visual detail) of each VSTM stage has remained unexplored and we test this in the present study. We presented people with a change detection task that measures the capacity of all three forms of VSTM, and we added an identification display after each change trial that required people to identify the "pre-change" object. Accurate change detection plus pre-change identification requires subjects to have a high-resolution representation of the "pre-change" object, whereas change detection or identification only can be based on the hunch that something has changed, without exactly knowing what was presented before. We observed that people maintained 6.1 objects in iconic memory, 4.6 objects in fragile VSTM, and 2.1 objects in visual working memory. Moreover, when people detected the change, they could also identify the pre-change object on 88% of the iconic memory trials, on 71% of the fragile VSTM trials and merely on 53% of the visual working memory trials. This suggests that people maintain many high-resolution representations in iconic memory and fragile VSTM, but only one high-resolution object representation in visual working memory.