Inducing representational change in the hippocampus through real-time neurofeedback.

When you perceive or remember something, other related things come to mind, affecting how these competing items are subsequently perceived and remembered. Such behavioural consequences are believed to result from changes in the overlap of neural representations of these items, especially in the hippocampus. According to multiple theories, hippocampal overlap should increase (integration) when there is high coactivation between cortical representations. However, prior studies used indirect proxies for coactivation by manipulating stimulus similarity or task demands. Here, we induce coactivation in visual cortex more directly using closed-loop neurofeedback from real-time functional magnetic resonance imaging (fMRI). While viewing one object, participants were rewarded for activating the representation of another object as strongly as possible. Across multiple real-time fMRI sessions, participants succeeded in using this neurofeedback to increase coactivation. Compared with a baseline of untrained objects, this protocol led to memory integration in behaviour and the brain: the trained objects became harder for participants to discriminate behaviourally in a categorical perception task and harder to discriminate neurally from patterns of fMRI activity in their hippocampus as a result of losing unique features. These findings demonstrate that neurofeedback can be used to alter and combine memories.This article is part of the theme issue 'Neurofeedback: new territories and neurocognitive mechanisms of endogenous neuromodulation'.

Slow and steady: Validating the rhythmic visual response task as a marker for attentional states.

A principal goal of attention research is to develop tasks with clear behavioral signatures of attentional fluctuations. Measures that index attentional states often fall under two broad umbrellas: decision tasks, in which participants make responses based on the changing requirements of each trial, and rhythm tasks, in which participants respond rhythmically to a uniform stimulus (e.g., a metronome tone). In the former, response speeding typically precedes errors (indicative of attention failures). In the latter, increased response variability precedes subjective reports of off-task states. We developed and validated the rhythmic visual response task (RVRT); a rhythm task incorporating trial-unique scene stimuli. The RVRT incorporates two important advances from both task categories: (1) it is free from the influence that differential decision-making has on fluctuations in attentional states, and (2) trial-unique stimuli enable later cognitive judgments to be mapped to specific moments in the task. These features allow a relatively unobtrusive measure of mind wandering that facilitates the downstream assessment of its consequences. Participants completed 900 trials of the RVRT, interrupted periodically by thought probes that assessed their attentional state. We found that both response time variance and speed predicted depth of mind wandering. Encouraged by these findings, we used the same analysis approach on archival data to demonstrate that the combination of variance and speed best predicted attentional states in several rhythm and decision task datasets. We discuss the implications of these findings and suggest future research that uses the RVRT to investigate the impact of spontaneous mind wandering on memory, decision-making, and perception.

Foreground bias: Semantic consistency effects modulated when searching across depth.

When processing visual scenes, we tend to prioritize information in the foreground, often at the expense of background information. The foreground bias has been supported by data demonstrating that there are more fixations to foreground, and faster and more accurate detection of targets embedded in foreground. However, it is also known that semantic consistency is associated with more efficient search. Here, we examined whether semantic context interacts with foreground prioritization, either amplifying or mitigating the effect of target semantic consistency. For each scene, targets were placed in the foreground or background and were either semantically consistent or inconsistent with the context of immediately surrounding depth region. Results indicated faster response times (RTs) for foreground and semantically consistent targets, replicating established effects. More importantly, we found the magnitude of the semantic consistency effect was significantly smaller in the foreground than background region. To examine the robustness of this effect, in Experiment 2, we strengthened the reliability of semantics by increasing the proportion of targets consistent with the scene region to 80%. We found the overall results pattern to replicate the incongruous effect of semantic consistency across depth observed in Experiment 1. This suggests foreground bias modulates the effects of semantics so that performance is less impacted in near space.

Examining the effect of expected test format and test difficulty on the frequency and mnemonic costs of mind wandering.

Mind wandering, generally defined as task-unrelated thought, has been shown to constitute between 30% and 50% of individuals' thoughts during almost every activity in which they are engaged. Critically, however, previous research has shown that the demands of a given task can lead to either the up- or down-regulation of mind wandering and that engagement in mind wandering may be differentially detrimental to future memory performance depending on learning conditions. The goal of the current research was to gain a better understanding of how the circumstances surrounding a learning episode affect the frequency with which individuals engage in off-task thought, and the extent to which these differences differentially affect memory performance across different test formats. Specifically, while prior work has manipulated the conditions of encoding, we focused on the anticipated characteristics of the retrieval task, thereby examining whether the anticipation of later demands imposed by the expected test format/difficulty would influence the frequency or performance costs of mind wandering during encoding. Across three experiments, we demonstrate that the anticipation of future test demands, as modelled by expected test format/difficulty, does not affect rates of mind wandering. However, the costs associated with mind wandering do appear to scale with the difficulty of the test. These findings provide important new insights into the impact of off-task thought on future memory performance and constrain our understanding of the strategic regulation of inattention in the context of learning and memory.

INDUCING REPRESENTATIONAL CHANGE IN THE HIPPOCAMPUS THROUGH REAL-TIME NEUROFEEDBACK.

When you perceive or remember one thing, other related things come to mind. This competition has consequences for how these items are later perceived, attended, or remembered. Such behavioral consequences result from changes in how much the neural representations of the items overlap, especially in the hippocampus. These changes can reflect increased (integration) or decreased (differentiation) overlap; previous studies have posited that the amount of coactivation between competing representations in cortex determines which will occur: high coactivation leads to hippocampal integration, medium coactivation leads to differentiation, and low coactivation is inert. However, those studies used indirect proxies for coactivation, by manipulating stimulus similarity or task demands. Here we induce coactivation of competing memories in visual cortex more directly using closed-loop neurofeedback from real-time fMRI. While viewing one object, participants were rewarded for implicitly activating the representation of another object as strongly as possible. Across multiple real-time fMRI training sessions, they succeeded in using the neurofeedback to induce coactivation. Compared with untrained objects, this coactivation led to behavioral and neural integration: The trained objects became harder for participants to discriminate in a categorical perception task and harder to decode from patterns of fMRI activity in the hippocampus.

Attention in hindsight: Using stimulated recall to capture dynamic fluctuations in attentional engagement.

Attentional engagement is known to vary on a moment-to-moment basis. However, few self-report methods can effectively capture dynamic fluctuations in attentional engagement over time. In the current paper, we evaluated the utility of stimulated recall, a method wherein individuals are asked to remember their subjective states while using a mnemonic cue, for the measurement of temporal changes in attentional engagement. Participants were asked to watch a video lecture, during which we assessed their in-the-moment levels of attentional engagement using intermittent thought probes. Then, we used stimulated recall by cueing participants with short video clips from the lecture to retrospectively assess the levels of attentional engagement they had experienced when they first watched those clips within the lecture. Experiment 1 assessed the statistical overlap between in-the-moment and video-stimulated ratings. Experiment 2 assessed the generalizability of video-stimulated recall across different types of lectures. Experiment 3 assessed the impact of presenting video-stimulated probe clips in non-chronological order. Experiment 4 assessed the effect of video-stimulated recall on its own. Across all experiments, we found statistically robust correspondence between in-the-moment and video-stimulated ratings of attentional engagement, illustrating a strong convergence between these two methods of assessment. Taken together, our findings indicate that stimulated recall provides a new and practical methodological approach that can accurately capture dynamic fluctuations in subjective attentional states over time.

Patterns of ongoing thought in the real world.

Health and well-being are impacted by our thoughts and the things we do. In the laboratory, studies suggest specific task contexts impact thought processes. More broadly, this suggests the people we are with, the places we are in, and the activities we perform may influence our thought patterns. In our study, participants completed experience sampling surveys for five days in daily life. Principal component analysis decomposed this data to identify common "patterns of thought," and linear mixed modelling related these patterns to the participants' activities. Our study replicated the influence of socializing on patterns of thought and established that this is part of a broader set of relationships linking activities to how thoughts are organized in daily life. Our study suggests sampling thinking in the real world may help map thoughts to activities, and these "thought-activity" mappings could be useful to researchers and health care professionals interested in health and well-being.

Drawing your way to an A: Long-lasting improvements in classroom quiz performance following drawing.

Generating a visual representation of a concept through drawing has been established as a valuable strategy for improving memory. While this has been demonstrated in both the laboratory and the real world, the findings are mixed in educational settings, perhaps due to variable operationalization of the drawing task. Participants are often provided additional scaffolding for their drawings, including instructions, training, or pre-drawn backgrounds. In the current work, we investigated whether the drawing advantage persists when participants in an introductory university class (N = 145) were permitted to create free-form drawings without additional scaffolding, akin to the technique often studied in the laboratory. In addition, because the effect has only been established at relatively short timescales, we tested whether benefits persist over multiple weeks. In the tutorial sessions of an undergraduate Biology course, we asked participants to either draw or copy a set of definitions for terms derived from the course textbook. One and 3 weeks later we quizzed participants by showing them the definition and asking them to recall the associated term. The evidence indicated not only that there was better recall for drawn than copied items after 1 week (even with no training or scaffolding), but also that this effect persisted after a 3-week delay. Our results establish that no external guidance is necessary for drawing to enhance memory, and that the benefits are robust over time, providing a foundation for further research on the durability of the benefits of drawing in the laboratory and the real world.

The relation between trait flow and engagement, understanding, and grades in undergraduate lectures.

BACKGROUND: Much work has focused on inattention in the classroom, examining how episodes of task-unrelated thought (i.e., mind wandering) and engagement with various forms of media (e.g., media multitasking, smartphone use) influence retention of lecture material. However, considerably less work has examined factors that may positively influence attentiveness in lectures. AIMS: We aimed to explore whether the trait-level tendency to experience 'flow'-defined here as the subjective experience of deep and effortless concentration-is related to in-class reports of engagement and understanding during undergraduate lectures, as well as academic performance. SAMPLE: Participants were undergraduate students in Psychology at a University in Ontario, Canada. METHODS: We measured trait flow (i.e., deep, effortless concentration) at the beginning of each semester, and assessed engagement and understanding during lectures via experience sampling probes throughout two semesters in several university courses. Experience sampling probes were presented intermittently using a laptop application. We also measured students' trait mind wandering and grit, and collected students' course grades. RESULTS: The general tendency to experience deep, effortless concentration predicted engagement and understanding in lectures throughout the term, as well as final course grades, over and above students' grittiness and tendency to mind wander. CONCLUSIONS: These findings suggest that the everyday tendency to experience flow extends to a classroom environment and has implications for academic success.

Drawing as a versatile cognitive tool.

Drawing is a cognitive tool that makes the invisible contents of mental life visible. Humans use this tool to produce a remarkable variety of pictures, from realistic portraits to schematic diagrams. Despite this variety and the prevalence of drawn images, the psychological mechanisms that enable drawings to be so versatile have yet to be fully explored. In this Review, we synthesize contemporary work in multiple areas of psychology, computer science and neuroscience that examines the cognitive processes involved in drawing production and comprehension. This body of findings suggests that the balance of contributions from perception, memory and social inference during drawing production varies depending on the situation, resulting in some drawings that are more realistic and other drawings that are more abstract. We also consider the use of drawings as a research tool for investigating various aspects of cognition, as well as the role that drawing has in facilitating learning and communication. Taken together, information about how drawings are used in different contexts illuminates the central role of visually grounded abstractions in human thought and behaviour.

Drawing and memory: Using visual production to alleviate concreteness effects.

Countless experiments have been devoted to understanding techniques through which memory might be improved. Many strategies uncovered in the literature are thought to act via the integration of contextual information from multiple distinct codes. However, the mnemonic benefits of these strategies often do not remain when there is no clear link between a word and its multisensory referent (e.g., in abstract words). To test the importance of this link, we asked participants to encode target words (ranging from concrete to abstract) either by drawing them, an encoding strategy recently proven to be reliable in improving memory, or writing them. Drawing provides a compelling test case because while other strategies (e.g., production, generation) shift focus to existing aspects of to-be-remembered information, drawing may forge a link with novel multisensory information, circumventing shortcomings of other memory techniques. Results indicated that while drawing's benefit was slightly larger for concrete stimuli, the effect was present across the spectrum from abstract to concrete. These findings demonstrate that even for highly abstract concepts without a clear link to a visual referent, memory is reliably improved through drawing. An exploratory analysis using a deep convolutional neural network also provided preliminary evidence that in abstract words, drawings that were most distinctive were more likely to be remembered, whereas concrete items benefited from prototypicality. Together, these results indicate that while the advantageous effects of drawing exist across all levels of concreteness, the memory benefit is larger when words are concrete, suggesting a tight coupling between the drawing benefit and visual code.

Relating Visual Production and Recognition of Objects in Human Visual Cortex.

Drawing is a powerful tool that can be used to convey rich perceptual information about objects in the world. What are the neural mechanisms that enable us to produce a recognizable drawing of an object, and how does this visual production experience influence how this object is represented in the brain? Here we evaluate the hypothesis that producing and recognizing an object recruit a shared neural representation, such that repeatedly drawing the object can enhance its perceptual discriminability in the brain. We scanned human participants (N = 31; 11 male) using fMRI across three phases of a training study: during training, participants repeatedly drew two objects in an alternating sequence on an MR-compatible tablet; before and after training, they viewed these and two other control objects, allowing us to measure the neural representation of each object in visual cortex. We found that: (1) stimulus-evoked representations of objects in visual cortex are recruited during visually cued production of drawings of these objects, even throughout the period when the object cue is no longer present; (2) the object currently being drawn is prioritized in visual cortex during drawing production, while other repeatedly drawn objects are suppressed; and (3) patterns of connectivity between regions in occipital and parietal cortex supported enhanced decoding of the currently drawn object across the training phase, suggesting a potential neural substrate for learning how to transform perceptual representations into representational actions. Together, our study provides novel insight into the functional relationship between visual production and recognition in the brain.SIGNIFICANCE STATEMENT Humans can produce simple line drawings that capture rich information about their perceptual experiences. However, the mechanisms that support this behavior are not well understood. Here we investigate how regions in visual cortex participate in the recognition of an object and the production of a drawing of it. We find that these regions carry diagnostic information about an object in a similar format both during recognition and production, and that practice drawing an object enhances transmission of information about it to downstream regions. Together, our study provides novel insight into the functional relationship between visual production and recognition in the brain.

Drawing improves memory: The importance of multimodal encoding context.

Memory is critical to the human experience, and more than a century of empirical work has been devoted to understanding its function and various means by which it can be strengthened. This literature has uncovered a long list of encoding techniques that reliably improve memory and yet little is known about the driving cognitive processes that are common to these techniques. To better understand how these diverse encoding techniques might enhance memory, we examine the mnemonic benefits of drawing, along with several related encoding techniques. Specifically, in two experiments, we disassembled drawing into a few of its component parts - elaborative, motoric, and pictorial - to systematically measure whether (1) memory is improved simply by adding information from additional modalities, and (2) particular components might be more influential to memory. We found that not all components are equal, and that the addition of active aspects - most prominently motoric information - substantially enhanced long-term retention. Together, our findings demonstrate the beneficial outcomes of including multiple distinct cognitive processes or sensory modalities during encoding. Moreover, our work highlights the potential role of these sources in improving memory, and points toward drawing as a fruitful path for applications in everyday life and in educational settings.

Comparing the influence of doodling, drawing, and writing at encoding on memory.

The purpose of the present study was to determine the extent to which doodling, which we define as drawing that is semantically unrelated to to-be-remembered information, enhances memory performance. In Experiment 1, participants heard auditorily presented lists of categorized words. They were asked to either doodle, draw a picture of, or write out, each item while listening to the target words. Participants showed poorer free recall for words encoded while free-form doodling compared to words that were drawn or written, with drawing resulting in the best performance. In Experiment 2, target words were embedded in a narrative story to better resemble a real-world situation in which one might doodle. Participants monitored each auditorily presented narrative while either free-form doodling, drawing, or writing in response to the target words. As in Experiment 1, doodling led to the poorest subsequent recall for targets compared to drawing or writing during encoding. In Experiment 3, we used a structured doodling task at encoding, such that participants shaded in geometric shapes printed on paper rather than create their own doodles. Structured doodling led to similar levels of recall compared to simply writing. Creating a drawing of the words at encoding, rather than doodling, once again enhanced recall significantly. Taken together, these findings indicate that unlike task-relevant drawing, structured doodling during study provides no benefits to free recall, and free-form doodling leads to memory costs. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Drawing enhances item information but undermines sequence information in memory.

Drawing a picture of the referent of a word produces considerably better recall and recognition of that word than does a baseline condition, such as repeatedly writing the word, a phenomenon referred to as the drawing effect. Although the drawing effect has been the focus of much recent research, it is not yet clear what underlies the beneficial effects of drawing to memory. In 3 experiments, we explored the roles of item and order information following drawing versus silent reading and produced 2 important findings. First, the drawing effect in recall was substantially larger when the 2 conditions were intermixed in a single list compared to appearing in separate lists-in other words, drawing produced a design effect. Second, the studied order was better retained for silent reading compared to drawing in pure lists. These findings are consistent with the item-order account: Memory for the order of drawn lists is poorer because the elaborative act of drawing disrupts the encoding of interitem associations (i.e., relational information) between all list items. Thus, the selection of an encoding task should be informed by how that memory might be used in the future; if one wants to remember individual items, then an elaborative task such as drawing is recommended; if one wants to remember their sequence, it is likely better to read silently. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Drawing as an Encoding Tool: Memorial Benefits in Younger and Older Adults.

Background/Study Context. In a recent study, drawing pictures relative to writing words at encoding has been shown to benefit later memory performance in young adults. In the current study, we sought to test whether older adults' memory might also benefit from drawing as an encoding strategy. Our prediction was that drawing would serve as a particularly effective form of environmental support at encoding as it encourages a more detailed perceptual representation. METHODS: Participants were presented 30 nouns, one at a time, and asked to either draw a picture or repeatedly write out the word, which was followed by a free recall test for all words (Experiment 1). In Experiment 2, we added an elaborative processing task in which we asked participants to list physical characteristics of the objects. In Experiment 3, we probed recognition memory for the words. RESULTS: Of the words recalled in Experiment 1, a larger proportion had been drawn than written at encoding, and this effect was larger in older relative to younger adults. In Experiment 2, we demonstrated that drawing improves memory in both younger and older adults more than does an elaborative encoding task consisting of listing descriptive characteristics of the target nouns. In Experiment 3, older and younger adults drew or wrote out words at encoding, and subsequently provided Remember-Know-New recognition memory decisions. We showed that drawing reduced age-related differences in Remember responses. CONCLUSIONS: We suggest that incorporating visuo-perceptual information into the memory trace, by drawing pictures at study, increases reliance of the memory trace on visual sensory regions, which are relatively intact in normal aging, relative to simply writing out or elaborately encoding words. Overall, results indicate that drawing is a highly valuable form of environmental support that can significantly enhance memory performance in older adults.

Task preparation as a mnemonic: The benefits of drawing (and not drawing).

Creating a visual representation of an item through drawing affords that item a substantive memory benefit, relative to several control tasks. Recent findings demonstrate the robustness of this drawing effect across several stimulus classes, irrespective of encoding time, setting, age group, or memory measure. The advantage for drawn information has been attributed to the integrated contributions of at least three components of visual production through drawing, which can independently facilitate memory: elaborative, motoric, and pictorial. In the current work, we investigated the importance of the elaborative process one must engage in while preparing to draw, and directly tested whether this generative period alone was sufficient to improve memory. Participants were prompted to either draw or write out presented words, and were provided with a 1- or 2-s preparatory period prior to completing the prompted task. Critically, on a subset of the trials, participants were prevented from completing the prompted task. There was strong evidence in support of better memory for drawn items, which replicates the drawing effect commonly observed in prior work. Interestingly, following prompts to draw, there was also a reliable memory benefit of the preparatory period alone. In other words, simply engaging in the elaborative process of preparing to draw (i.e., without completing the drawing) was enough to produce a reliable increase in later memory relative to actually writing.

On the Clock: Evidence for the Rapid and Strategic Modulation of Mind Wandering.

We examined the hypothesis that people can modulate their mind wandering on the basis of their expectations of upcoming challenges in a task. To this end, we developed a novel paradigm in which participants were presented with an analog clock, via a computer monitor, and asked to push a button every time the clock's hand was pointed at 12:00. Importantly, the time at which the clock's hand was pointed at 12:00 was completely predictable and occurred at 20-s intervals. During some of the 20-s intervals, we presented thought probes to index participants' rates of mind wandering. Results indicated that participants decreased their levels of mind wandering as they approached the predictable upcoming target. Critically, these results suggest that people can and do modulate their mind wandering in anticipation of changes in task demands.

Creating a recollection-based memory through drawing.

Drawing a picture of to-be-remembered information substantially boosts memory performance in free-recall tasks. In the current work, we sought to test the notion that drawing confers its benefit to memory performance by creating a detailed recollection of the encoding context. In Experiments 1 and 2, we demonstrated that for both pictures and words, items that were drawn by the participant at encoding were better recognized in a later test than were words that were written out. Moreover, participants' source memory (in this experiment, correct identification of whether the word was drawn or written) was superior for items drawn relative to written at encoding. In Experiments 3A and 3B, we used a remember-know paradigm to demonstrate again that drawn words were better recognized than written words, and further showed that this effect was driven by a greater proportion of recollection-, rather than familiarity-based responses. Lastly, in Experiment 4 we implemented a response deadline procedure, and showed that when recognition responses were speeded, thereby reducing participants' capacity for recollection, the benefit of drawing was substantially smaller. Taken together, our findings converge on the idea that drawing improves memory as a result of providing vivid contextual information which can be later called upon to aid retrieval. (PsycINFO Database Record