Probabilistic and rich individual working memories revealed by a betting game.

When asked to remember a color, do people remember a point estimate (e.g., a particular shade of red), a point estimate plus an uncertainty estimate, or are memory representations rich probabilistic distributions over feature space? We asked participants to report the color of a circle held in working memory. Rather than collecting a single report per trial, we had participants place multiple bets to create trialwise uncertainty distributions. Bet dispersion correlated with performance, indicating that internal uncertainty guided bet placement. While the first bet was on average the most precisely placed, the later bets systematically shifted the distribution closer to the target, resulting in asymmetrical distributions about the first bet. This resulted in memory performance improvements when averaging across bets, and overall suggests that memory representations contain more information than can be conveyed by a single response. The later bets contained target information even when the first response would generally be classified as a guess or report of an incorrect item, suggesting that such failures are not all-or-none. This paradigm provides multiple pieces of evidence that memory representations are rich and probabilistic. Crucially, standard discrete response paradigms underestimate the amount of information in memory representations.

Meaningful objects avoid attribute amnesia due to incidental long-term memories.

Attribute amnesia describes the failure to unexpectedly report the attribute of an attended stimulus, likely reflecting a lack of working memory consolidation. Previous studies have shown that unique meaningful objects are immune to attribute amnesia. However, these studies used highly dissimilar foils to test memory, raising the possibility that good performance at the surprise test was based on an imprecise (gist-like) form of long-term memory. In Experiment 1, we explored whether a more sensitive memory test would reveal attribute amnesia in meaningful objects. We used a four-alternative-forced-choice test with foils having mis-matched exemplar (e.g., apple pie/pumpkin pie) and/or state (e.g., cut/full) information. Errors indicated intact exemplar, but not state information. Thus, meaningful objects are vulnerable to attribute amnesia under the right conditions. In Experiments 2A-2D, we manipulated the familiarity signals of test items by introducing a critical object as a pre-surprise target. In the surprise trial, this critical item matched one of the foil choices. Participants selected the critical object more often than other items. By demonstrating that familiarity influences responses in this paradigm, we suggest that meaningful objects are not immune to attribute amnesia but instead side-step the effects of attribute amnesia.

Perception, performance, and detectability of conversational artificial intelligence across 32 university courses.

The emergence of large language models has led to the development of powerful tools such as ChatGPT that can produce text indistinguishable from human-generated work. With the increasing accessibility of such technology, students across the globe may utilize it to help with their school work-a possibility that has sparked ample discussion on the integrity of student evaluation processes in the age of artificial intelligence (AI). To date, it is unclear how such tools perform compared to students on university-level courses across various disciplines. Further, students' perspectives regarding the use of such tools in school work, and educators' perspectives on treating their use as plagiarism, remain unknown. Here, we compare the performance of the state-of-the-art tool, ChatGPT, against that of students on 32 university-level courses. We also assess the degree to which its use can be detected by two classifiers designed specifically for this purpose. Additionally, we conduct a global survey across five countries, as well as a more in-depth survey at the authors' institution, to discern students' and educators' perceptions of ChatGPT's use in school work. We find that ChatGPT's performance is comparable, if not superior, to that of students in a multitude of courses. Moreover, current AI-text classifiers cannot reliably detect ChatGPT's use in school work, due to both their propensity to classify human-written answers as AI-generated, as well as the relative ease with which AI-generated text can be edited to evade detection. Finally, there seems to be an emerging consensus among students to use the tool, and among educators to treat its use as plagiarism. Our findings offer insights that could guide policy discussions addressing the integration of artificial intelligence into educational frameworks.

Common Neural Mechanisms Control Attention and Working Memory.

Although previous studies point to qualitative similarities between working memory (WM) and attention, the degree to which these two constructs rely on shared neural mechanisms remains unknown. Focusing on one such potentially shared mechanism, we tested the hypothesis that selecting an item within WM utilizes similar neural mechanisms as selecting a visible item via a shift of attention. We used fMRI and machine learning to decode both the selection among items visually available and the selection among items stored in WM in human subjects (both sexes). Patterns of activity in visual, parietal, and to a lesser extent frontal cortex predicted the locations of the selected items. Critically, these patterns were strikingly interchangeable; classifiers trained on data during attentional selection predicted selection from WM, and classifiers trained on data during selection from memory predicted attentional selection. Using models of voxel receptive fields, we visualized topographic population activity that revealed gain enhancements at the locations of the externally and internally selected items. Our results suggest that selecting among perceived items and selecting among items in WM share a common mechanism. This common mechanism, analogous to a shift of spatial attention, controls the relative gains of neural populations that encode behaviorally relevant information.SIGNIFICANCE STATEMENT How we allocate our attention to external stimuli that we see and to internal representations of stimuli stored in memory might rely on a common mechanism. Supporting this hypothesis, we demonstrated that not only could patterns of human brain activity predict which items were selected during perception and memory, but that these patterns were interchangeable during external and internal selection. Additionally, this generalized selection mechanism operates by changes in the gains of the neural populations both encoding attended sensory representations and storing relevant memory representations.

Perception is rich and probabilistic.

When we see a stimulus, e.g. a star-shaped object, our intuition is that we should perceive a single, coherent percept (even if it is inaccurate). But the neural processes that support perception are complex and probabilistic. Simple lines cause orientation-selective neurons across a population to fire in a probabilistic-like manner. Does probabilistic neural firing lead to non-probabilistic perception, or are the representations behind perception richer and more complex than intuition would suggest? To test this, we briefly presented a complex shape and had participants report the correct shape from a set of options. Rather than reporting a single value, we used a paradigm designed to encourage to directly report a representation over shape space-participants placed a series of Gaussian bets. We found that participants could report more than point-estimates of shape. The spread of responses was correlated with accuracy, suggesting that participants can convey a notion of relative imprecision. Critically, as participants placed more bets, the mean of responses show increased precision. The later bets were systematically biased towards the target rather than haphazardly placed around bet 1. These findings strongly indicate that participants were aware of more than just a point-estimate; Perceptual representations are rich and likely probabilistic.

No evidence that the retro-cue benefit requires reallocation of memory resources.

Selective mechanisms allow us to prioritize items held in working memory. Does this reflect reallocation of working memory resources? We examined a critical prediction of this account-that reallocating more resources from one item to another should provide a greater benefit. We used a reward manipulation to create variable allocation of resources. Subsequently, a retro-cue instructed participants to drop a memory item. This retro-cue improved performance for the prioritized items relative to a neutral baseline. However, in contrast to the prevailing reallocation account, we found no difference between dropping a higher versus lower reward item. Importantly, removal of high versus low reward items led to better encoding of subsequently presented items, demonstrating that our reward manipulation was successful. While allocation of resources can influence the encoding and storage of new information into working memory, reallocation does not appear to be essential for selection effects in working memory.

Training modulates memory-driven capture.

Attention is captured by information matching the contents of working memory. Though many factors modulate the amount of capture, there is surprising resistance to cognitive control. Capture occurs even when participants are instructed either that an item would never be a target or to drop that item from memory. Does the persistence of capture under these conditions reflect a rigidity in capture, or can properly motivated participants learn to completely suppress distractors and/or completely drop items from memory? Surprisingly, no studies have looked at the influence of extensive training of involuntary capture from working memory items. Here, we addressed whether training leads to a reduction or even elimination of memory-driven capture. After memorizing a single object, participants were cued to remember or to forget this object. Subsequently, they were asked to execute a search task. To measure capture, we compared search performances in displays that did and did not contain a distractor matching the earlier memorized object. Participants completed multiple experimental sessions over four days. The results showed that attentional capture by to-be-remembered distractors was reduced, but not eliminated in subsequent sessions compared with the first session. Training did not impact capture by to-be-forgotten objects. The results suggest observable, but limited, cognitive control over memory-driven capture.

How do expectations change behavior? Investigating the contributions at encoding versus decision-making.

Expectations about the environment play a large role in shaping behavior, but how does this occur? Do expectations change the way we perceive the world, or just our decisions based on unbiased perceptions? We investigated the relative contributions of priors to these 2 stages by manipulating when information about expected color was provided. We compared cases where the prior could affect encoding into perceptual/working memory representations (e.g., when provided prestimulus) against cases where it could not (e.g., when given at response after a delay). Although priors had a minor influence on encoding, the bulk of the effects were at decision-making. Furthermore, these effects appeared to be distinct. The effect on decision-making was Bayesian-like, with priors inducing bias while improving precision. In contrast, the same priors at encoding improved precision without causing changes in bias. Priors do not just affect encoding or decision-making, but appear to affect both, via distinct mechanisms. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

How selection in the mind is different from attention to the world.

Attentional mechanisms allow us to focus on objects that would help us achieve our goals while ignoring those that would distract us. Attention can also be focused internally toward specific items in memory. But does selection within memory work similarly to selection within perception? Perceptual attention is fast and effective at selecting regions of space. Across five experiments, we used a memory search task to investigate whether spatial selection is also efficient for selection in memory. Participants remembered four items on a grid before being asked to access their memory of one item and update one of its features. We found that it took longer to access an item when referenced by its spatial location than by its color, despite memory accuracy for location being superior. We conclude that there must be multiple, distinct memory representations in the brain and that selection in memory is different from perceptual selection. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Object-based selection in visual working memory.

Attentional mechanisms in perception can operate over locations, features, or objects. However, people direct attention not only towards information in the external world, but also to information maintained in working memory. To what extent do perception and memory draw on similar selection properties? Here we examined whether principles of object-based attention can also hold true in visual working memory. Experiment 1 examined whether object structure guides selection independently of spatial distance. In a memory updating task, participants encoded two rectangular bars with colored ends before updating two colors during maintenance. Memory updates were faster for two equidistant colors on the same object than on different objects. Experiment 2 examined whether selection of a single object feature spreads to other features within the same object. Participants memorized two sequentially presented Gabors, and a retro-cue indicated which object and feature dimension (color or orientation) would be most relevant to the memory test. We found stronger effects of object selection than feature selection: accuracy was higher for the uncued feature in the same object than the cued feature in the other object. Together these findings demonstrate effects of object-based attention on visual working memory, at least when object-based representations are encouraged, and suggest shared attentional mechanisms across perception and memory.

Selection in working memory is resource-demanding: Concurrent task effects on the retro-cue effect.

In a retro-cue paradigm, after memorizing a set of objects, people are cued to remember only a subset. Improved memory from the retro-cue suggests that selection processes can benefit items stored in working memory. Does selection in working memory require attention? If so, an attention-demanding task should disrupt retro-cue effects. Studies using a dual-task paradigm have found mixed results, with only one study (Janczyk & Berryhill, Attention, Perception, and Psychophysics, 76 (3), 715-724, 2014) showing a decreased retro-cue effect by a secondary task. Here we explore a potential issue in that study - the temporal overlap of the secondary task response with the memory test presentation. This raises questions about whether the secondary task was impairing selection processes in memory or was impacting the memory response. We replicated their paradigm by inserting a tone discrimination task at the retro-cue offset, but we also included a condition in which the tone task and the memory test were temporally separated. In Experiment 1, performing the tone task did not impair the retro-cue effect. In Experiment 2, we added an articulatory suppression task as in Janczyk and Berryhill's study, and we found that the requirement to execute the tone task impaired retro-cue effects. This impairment was independent of whether the tone and memory tasks overlapped. These findings suggest that internal prioritization can be impaired by dual-task interference, but may only occur when such interference is robust enough, for example, due to switching between multiple tasks.

The road to long-term memory: Top-down attention is more effective than bottom-up attention for forming long-term memories.

Does the strength of representations in long-term memory (LTM) depend on which type of attention is engaged? We tested participants' memory for objects seen during visual search. We compared implicit memory for two types of objects-related-context nontargets that grabbed attention because they matched the target defining feature (i.e., color; top-down attention) and salient distractors that captured attention only because they were perceptually distracting (bottom-up attention). In Experiment 1, the salient distractor flickered, while in Experiment 2, the luminance of the salient distractor was alternated. Critically, salient and related-context nontargets produced equivalent attentional capture, yet related-context nontargets were remembered far better than salient distractors (and salient distractors were not remembered better than unrelated distractors). These results suggest that LTM depends not only on the amount of attention but also on the type of attention. Specifically, top-down attention is more effective in promoting the formation of memory traces than bottom-up attention.

Memory-driven capture occurs for individual features of an object.

Items held in working memory (WM) capture attention (memory-driven capture). People can selectively prioritize specific object features in WM. Here, we examined whether feature-specific prioritization within WM modulates memory-driven capture. In Experiment 1, after remembering the color and orientation of a triangle, participants were instructed, via retro-cue, whether the color, the orientation, or both features were relevant. To measure capture, we asked participants to execute a subsequent search task, and we compared performance in displays that did and did not contain the memory-matching feature. Color attracted attention only when it was relevant. No capture by orientation was found. In Experiment 2, we presented the retro-cue at one of the four locations of the search display to direct attention to specific objects. We found capture by color and this capture was larger when it was indicated as relevant. Crucially, orientation also attracted attention, but only when it was relevant. These findings provide evidence for reciprocal interaction between internal prioritization and external attention on the features level. Specifically, internal feature-specific prioritization modulates memory-driven capture but this capture also depends on the salience of the features.

Working memory is corrupted by strategic changes in search templates.

When searching for a specific object, we often form an image of the target, which we use as a search template. This template is thought to be maintained in working memory, primarily because of evidence that the contents of working memory influences search behavior. However, it is unknown whether this interaction applies in both directions. Here, we show that changes in search templates influence working memory. Participants were asked to remember the orientation of a line that changed every trial, and on some trials (75%) search for that orientation, but on remaining trials recall the orientation. Critically, we manipulated the target template by introducing a predictable context-distractors in the visual search task were always counterclockwise (or clockwise) from the search target. The predictable context produced a large bias in search. Importantly, we also found a similar bias in orientation memory reports, demonstrating that working memory and target templates were not held as completely separate, isolated representations. However, the memory bias was considerably smaller than the search bias, suggesting that, although there is a common source, the two may not be driven by a single, shared process.

Humans incorporate trial-to-trial working memory uncertainty into rewarded decisions.

Working memory (WM) plays an important role in action planning and decision making; however, both the informational content of memory and how that information is used in decisions remain poorly understood. To investigate this, we used a color WM task in which subjects viewed colored stimuli and reported both an estimate of a stimulus color and a measure of memory uncertainty, obtained through a rewarded decision. Reported memory uncertainty is correlated with memory error, showing that people incorporate their trial-to-trial memory quality into rewarded decisions. Moreover, memory uncertainty can be combined with other sources of information; after inducing expectations (prior beliefs) about stimuli probabilities, we found that estimates became shifted toward expected colors, with the shift increasing with reported uncertainty. The data are best fit by models in which people incorporate their trial-to-trial memory uncertainty with potential rewards and prior beliefs. Our results suggest that WM represents uncertainty information, and that this can be combined with prior beliefs. This highlights the potential complexity of WM representations and shows that rewarded decision can be a powerful tool for examining WM and informing and constraining theoretical, computational, and neurobiological models of memory.

Visual search within working memory.

Attention and working memory are 2 key pillars of cognition. Despite much research, there are important aspects about the relationship between the 2 constructs that are not well understood. Here we explore the similarity in the mechanisms that select and update working memory to those that guide attention during perception, such as in visual search. We use a memory search task where participants memorize a display of objects on a grid. During memory maintenance, participants are instructed to update the spatial positions of a subset of objects. The speed of the updating process should reflect the accessibility of the to-be-updated subset. Using this task, we explored whether landmark findings in visual search would hold true for memory search. In Experiment 1, we found a search asymmetry-it was easier to access memory representations defined by a feature than defined by the lack of a feature. In Experiment 2, we found target-distractor similarity effects-updating a single target was easier when the distractors were farther away in feature space. In Experiment 3, we found a feature versus conjunction benefit-access times were much faster for instructions to move objects defined by only 1 feature (e.g., all triangles) as opposed to a conjunction of features (e.g., all red triangles). In Experiment 4, we find a set-size effect-update times increased with the number of items in memory, particularly for conjunctive stimuli. Taken together, our results suggest a common coding and selection scheme for working memory and perceptual representations. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

A common source of attention for auditory and visual tracking.

Tasks that require tracking visual information reveal the severe limitations of our capacity to attend to multiple objects that vary in time and space. Although these limitations have been extensively characterized in the visual domain, very little is known about tracking information in other sensory domains. Does tracking auditory information exhibit characteristics similar to those of tracking visual information, and to what extent do these two tracking tasks draw on the same attention resources? We addressed these questions by asking participants to perform either single or dual tracking tasks from the same (visual-visual) or different (visual-auditory) perceptual modalities, with the difficulty of the tracking tasks being manipulated across trials. The results revealed that performing two concurrent tracking tasks, whether they were in the same or different modalities, affected tracking performance as compared to performing each task alone (concurrence costs). Moreover, increasing task difficulty also led to increased costs in both the single-task and dual-task conditions (load-dependent costs). The comparison of concurrence costs between visual-visual and visual-auditory dual-task performance revealed slightly greater interference when two visual tracking tasks were paired. Interestingly, however, increasing task difficulty led to equivalent costs for visual-visual and visual-auditory pairings. We concluded that visual and auditory tracking draw largely, though not exclusively, on common central attentional resources.

How humans react to changing rewards during visual foraging.

Much is known about the speed and accuracy of search in single-target search tasks, but less attention has been devoted to understanding search in multiple-target foraging tasks. These tasks raise and answer important questions about how individuals decide to terminate searches in cases in which the number of targets in each display is unknown. Even when asked to find every target, individuals quit before exhaustively searching a display. Because a failure to notice targets can have profound effects (e.g., missing a malignant tumor in an X-ray), it is important to develop strategies that could limit such errors. Here, we explored the impact of different reward patterns on these failures. In the Neutral condition, reward for finding a target was constant over time. In the Increasing condition, reward increased for each successive target in a display, penalizing early departure from a display. In the Decreasing condition, reward decreased for each successive target in a display. The experimental results demonstrate that observers will forage for longer (and find more targets) when the value of successive targets increases (and the opposite when value decreases). The data indicate that observers were learning to utilize knowledge of the reward pattern and to forage optimally over the course of the experiment. Simulation results further revealed that human behavior could be modeled with a variant of Charnov's Marginal Value Theorem (MVT) (Charnov, 1976) that includes roles for reward and learning.

Looking inward and back: Real-time monitoring of visual working memories.

Confidence in our memories is influenced by many factors, including beliefs about the perceptibility or memorability of certain kinds of objects and events, as well as knowledge about our skill sets, habits, and experiences. Notoriously, our knowledge and beliefs about memory can lead us astray, causing us to be overly confident in eyewitness testimony or to overestimate the frequency of recent experiences. Here, using visual working memory as a case study, we stripped away all these potentially misleading cues, requiring observers to make confidence judgments by directly assessing the quality of their memory representations. We show that individuals can monitor the status of information in working memory as it degrades over time. Our findings suggest that people have access to information reflecting the existence and quality of their working memories, and furthermore, that they can use this information to guide their behavior. (PsycINFO Database Record