Why do valence asymmetries emerge in value learning? A reinforcement learning account.

The Value Learning Task (VLT; e.g., Raymond & O'Brien, 2009) is widely used to investigate how acquired value impacts how we perceive and process stimuli. The task consists of a series of trials in which participants attempt to maximize accumulated winnings as they make choices from a pair of presented images associated with probabilistic win, loss, or no-change outcomes. The probabilities and outcomes are initially unknown to the participant and thus the task involves decision making and learning under uncertainty. Despite the symmetric outcome structure for win and loss pairs, people learn win associations better than loss associations (Lin, Cabrera-Haro, & Reuter-Lorenz, 2020). This learning asymmetry could lead to differences when the stimuli are probed in subsequent tasks, compromising inferences about how acquired value affects downstream processing. We investigate the nature of the asymmetry using a standard error-driven reinforcement learning model with a softmax choice rule. Despite having no special role for valence, the model yields the learning asymmetry observed in human behavior, whether the model parameters are set to maximize empirical fit, or task payoff. The asymmetry arises from an interaction between a neutral initial value estimate and a choice policy that exploits while exploring, leading to more poorly discriminated value estimates for loss stimuli. We also show how differences in estimated individual learning rates help to explain individual differences in the observed win-loss asymmetries, and how the final value estimates produced by the model provide a simple account of a post-learning explicit value categorization task.

How well do ordinary Americans forecast the growth of COVID-19?

Across three experiments (N = 1565), we investigated how forecasts about the spread of COVID 19 are impacted by data trends, and whether patterns of misestimation predict adherence to social-distancing guidelines. We also investigated how mode of data presentation influences forecasting of future cases by showing participants data on the number of COVID-19 cases from a 5-week period in either graphical, tabular, or text-only form. We consistently found that people shown tables produced more accurate forecasts compared to people shown line-graphs of the same data; yet people shown line-graphs were more confident in their estimates. These findings suggest that graphs engender false-confidence in the accuracy of forecasts, that people's forecasts of future cases have important implications for their attitudes concerning social distancing, and that tables may be better than graphs for informing the public about the trajectory of COVID-19.

Cognitively bounded rational analyses and the crucial role of theories of subjective utility.

We agree that combining rational analysis with cognitive bounds, what we previously introduced as Cognitively Bounded Rational Analysis, is a promising and under-used methodology in psychology. We further situate the framework in the literature, and highlight the important issue of a theory of subjective utility, which is not addressed sufficiently clearly in the framework or related previous work.

The role of (bounded) optimization in theory testing and prediction.

We argue that a radically increased emphasis on (bounded) optimality can contribute to cognitive science by supporting prediction. Bounded optimality (computational rationality), an idea that borrowed from artificial intelligence, supports a priori behavioral prediction from constrained generative models of cognition. Bounded optimality thereby addresses serious failings with the logic and testing of descriptive models of perception and action.

Icon arrays reduce concern over COVID-19 vaccine side effects: a randomized control study.

On April 13, 2021, the CDC announced that the administration of Johnson and Johnson's COVID-19 vaccine would be paused due to a rare blood clotting side effect in ~ 0.0001% of people given the vaccine. Most people who are hesitant to get a COVID-19 vaccine list potential side effects as their main concern (PEW, 2021); thus, it is likely that this announcement increased vaccine hesitancy among the American public. Two days after the CDC's announcement, we administered a survey to a group of 2,046 Americans to assess their changes in attitudes toward COVID-19 vaccines. The aim of this study was to investigate whether viewing icon arrays of side effect risk would prevent increases in COVID-19 vaccine hesitancy due to the announcement. We found that using icon arrays to illustrate the small chance of experiencing the blood clotting side effect significantly prevented increases in aversion toward the Johnson and Johnson vaccine as well as all other COVID-19 vaccines.

Rapid adaptation of predictive models during language comprehension: Aperiodic EEG slope, individual alpha frequency and idea density modulate individual differences in real-time model updating.

Predictive coding provides a compelling, unified theory of neural information processing, including for language. However, there is insufficient understanding of how predictive models adapt to changing contextual and environmental demands and the extent to which such adaptive processes differ between individuals. Here, we used electroencephalography (EEG) to track prediction error responses during a naturalistic language processing paradigm. In Experiment 1, 45 native speakers of English listened to a series of short passages. Via a speaker manipulation, we introduced changing intra-experimental adjective order probabilities for two-adjective noun phrases embedded within the passages and investigated whether prediction error responses adapt to reflect these intra-experimental predictive contingencies. To this end, we calculated a novel measure of speaker-based, intra-experimental surprisal ("speaker-based surprisal") as defined on a trial-by-trial basis and by clustering together adjectives with a similar meaning. N400 amplitude at the position of the critical second adjective was used as an outcome measure of prediction error. Results showed that N400 responses attuned to speaker-based surprisal over the course of the experiment, thus indicating that listeners rapidly adapt their predictive models to reflect local environmental contingencies (here: the probability of one type of adjective following another when uttered by a particular speaker). Strikingly, this occurs in spite of the wealth of prior linguistic experience that participants bring to the laboratory. Model adaptation effects were strongest for participants with a steep aperiodic (1/f) slope in resting EEG and low individual alpha frequency (IAF), with idea density (ID) showing a more complex pattern. These results were replicated in a separate sample of 40 participants in Experiment 2, which employed a highly similar design to Experiment 1. Overall, our results suggest that individuals with a steep aperiodic slope adapt their predictive models most strongly to context-specific probabilistic information. Steep aperiodic slope is thought to reflect low neural noise, which in turn may be associated with higher neural gain control and better cognitive control. Individuals with a steep aperiodic slope may thus be able to more effectively and dynamically reconfigure their prediction-related neural networks to meet current task demands. We conclude that predictive mechanisms in language are highly malleable and dynamic, reflecting both the affordances of the present environment as well as intrinsic information processing capabilities of the individual.

Resolving semantic and proactive interference in memory over the short-term.

Interference is a major source of short-term errors of memory. The present investigation explores the relationship between two important forms of interference: proactive interference (PI), induced by the need to reject recently studied items no longer relevant to task performance, and semantic interference (SI), induced by the need to reject lures sharing a meaningful relationship with current memoranda. We explore the possibility that shared cognitive control processes are recruited to resolve both forms of interference. In Experiment 1, we find that the requirement to engage in articulatory suppression during the retention interval of tasks that induce either PI or SI increases both forms of interference similarly and selectively. In Experiment 2, we develop a task to examine PI and SI within the same experimental context. The results show interactive effects between factors that lead to the two forms of interference. Taken together, these findings support contextual-cuing models of short-term remembering (Nairne, Annual Review of Psychology, 53, 53-81 2002), where the context in which retrieval occurs can influence susceptibility to interference. Lastly, we discuss several theoretical hypotheses concerning the cognitive control processes that are recruited to resolve SI and PI in short-term remembering.

Rational use of episodic and working memory: A normative account of prospective memory.

Humans often simultaneously pursue multiple plans at different time scales, a capacity known as prospective memory (PM). The successful realization of non-immediate plans (e.g., post package after work) requires keeping track of a future plan while accomplishing other intermediate tasks (e.g., write a paper). Prospective memory capacity requires the integration of noisy evidence from perceptual input with evidence from both short-term working memory (WM) and long-term or episodic memory (LTM/EM). Here we formulate a set of empirical studies of prospective memory, all dual-task problems, as problems of computational rationality. We ask how a rational model should integrate noisy perceptual evidence and memory to maximize payoffs in these PM studies. The model combines reinforcement learning (optimal action selection) with evidence accumulation (optimal inference) in order to derive good decision parameters for optimal task performance (i.e., performing an ongoing task while monitoring for a cue that triggers executing a second prospective task). We compare model behavior to human behavioral evidence of key accuracy and reaction time phenomena in PM. Notably, our normative approach to theorizing and modeling these phenomena makes no assumptions about mechanisms of attention or retrieval. This approach can be extended to study the learning and use of meta-parameters governing the boundedly rational use of memory in planned action in health and disease. A computational psychiatry extension of the model can capture compensatory mnemonic strategies in neuropsychiatric disorders that may be rational responses to disturbances of inference, memory, and action selection.

Grammatical predictions in Spanish-English bilinguals and Spanish-language learners.

Although bilingual individuals know 2 languages, research suggests that the languages are not separate in the mind. This is especially evident when a bilingual individual switches languages midsentence, indicating that mental representations are, to some degree, overlapping or integrated across the 2 languages. In 2 eye-tracking experiments, we investigated the nature of this integration during reading to examine whether incremental grammatical predictions generated by Spanish-English bilinguals (Experiment 1, N = 50) and Spanish-as-a-second-language learners (Experiment 2, N = 50) are language-specific or language-independent. As participants in same-language and mixed-language pairs performed a 2-string lexical-decision task, we measured eye fixation times on nouns in grammatical (determiner-noun) and ungrammatical (adverb-noun) contexts. In Experiment 1, bilingual participants read nouns faster following determiners than they read adverbs in both same- and mixed-language pairs, indicating that grammatical predictability in this context is language-independent. Surface-string bigram frequencies are unlikely to account for the results because the grammatical predictability effect was just as large for mixed-language (very low bigram frequency) as same-language (higher bigram frequency) pairs, and the effect was not modulated by the code-switching experience of participants. Experiment 2 found a similar, though nonsignificant, pattern for Spanish-language learners. When the data for Experiments 1 and 2 were combined, the effect of grammaticality did not interact with language congruency, participant group, or language proficiency, suggesting that both bilingual participants and language learners generated language-independent predictions. Our results support a bilingual model in which language-independent syntactic representations are involved in word-by-word, incremental syntactic processing, even within the most basic grammatical constituents. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Rational adaptation under task and processing constraints: implications for testing theories of cognition and action.

The authors assume that individuals adapt rationally to a utility function given constraints imposed by their cognitive architecture and the local task environment. This assumption underlies a new approach to modeling and understanding cognition-cognitively bounded rational analysis-that sharpens the predictive acuity of general, integrated theories of cognition and action. Such theories provide the necessary computational means to explain the flexible nature of human behavior but in doing so introduce extreme degrees of freedom in accounting for data. The new approach narrows the space of predicted behaviors through analysis of the payoff achieved by alternative strategies, rather than through fitting strategies and theoretical parameters to data. It extends and complements established approaches, including computational cognitive architectures, rational analysis, optimal motor control, bounded rationality, and signal detection theory. The authors illustrate the approach with a reanalysis of an existing account of psychological refractory period (PRP) dual-task performance and the development and analysis of a new theory of ordered dual-task responses. These analyses yield several novel results, including a new understanding of the role of strategic variation in existing accounts of PRP and the first predictive, quantitative account showing how the details of ordered dual-task phenomena emerge from the rational control of a cognitive system subject to the combined constraints of internal variance, motor interference, and a response selection bottleneck.

In search of decay in verbal short-term memory.

Is forgetting in the short term due to decay with the mere passage of time, interference from other memoranda, or both? Past research on short-term memory has revealed some evidence for decay and a plethora of evidence showing that short-term memory is worsened by interference. However, none of these studies has directly contrasted decay and interference in short-term memory in a task that rules out the use of rehearsal processes. In this article the authors present a series of studies using a novel paradigm to address this problem directly, by interrogating the operation of decay and interference in short-term memory without rehearsal confounds. The results of these studies indicate that short-term memories are subject to very small decay effects with the mere passage of time but that interference plays a much larger role in their degradation. The authors discuss the implications of these results for existing models of memory decay and interference.

In Search of the Factors Behind Naive Sentence Judgments: A State Trace Analysis of Grammaticality and Acceptability Ratings.

We present a state-trace analysis of sentence ratings elicited by asking participants to evaluate the overall acceptability of a sentence and those elicited by asking participants to focus on structural well-formedness only. Appealing to literature on "grammatical illusion" sentences, we anticipated that a simple instruction manipulation might prompt people to apply qualitatively different kinds of judgment in the two conditions. Although differences consistent with the subjective experience of grammatical illusion dissociations were observed, the state trace analysis of the rating data indicates that responses were still consistent with both judgment types accessing a single underlying factor. These results add to the existing comparisons between analytic and probabilistic modeling approaches to predicting rating judgments.

Processing polarity: how the ungrammatical intrudes on the grammatical.

A central question in online human sentence comprehension is, "How are linguistic relations established between different parts of a sentence?" Previous work has shown that this dependency resolution process can be computationally expensive, but the underlying reasons for this are still unclear. This article argues that dependency resolution is mediated by cue-based retrieval, constrained by independently motivated working memory principles defined in a cognitive architecture. To demonstrate this, this article investigates an unusual instance of dependency resolution, the processing of negative and positive polarity items, and confirms a surprising prediction of the cue-based retrieval model: Partial-cue matches-which constitute a kind of similarity-based interference-can give rise to the intrusion of ungrammatical retrieval candidates, leading to both processing slow-downs and even errors of judgment that take the form of illusions of grammaticality in patently ungrammatical structures. A notable achievement is that good quantitative fits are achieved without adjusting the key model parameters.

Computational principles of working memory in sentence comprehension.

Understanding a sentence requires a working memory of the partial products of comprehension, so that linguistic relations between temporally distal parts of the sentence can be rapidly computed. We describe an emerging theoretical framework for this working memory system that incorporates several independently motivated principles of memory: a sharply limited attentional focus, rapid retrieval of item (but not order) information subject to interference from similar items, and activation decay (forgetting over time). A computational model embodying these principles provides an explanation of the functional capacities and severe limitations of human processing, as well as accounts of reading times. The broad implication is that the detailed nature of cross-linguistic sentence processing emerges from the interaction of general principles of human memory with the specialized task of language comprehension.

Retrieval Interference in Syntactic Processing: The Case of Reflexive Binding in English.

It has been proposed that in online sentence comprehension the dependency between a reflexive pronoun such as himself/herself and its antecedent is resolved using exclusively syntactic constraints. Under this strictly syntactic search account, Principle A of the binding theory-which requires that the antecedent c-command the reflexive within the same clause that the reflexive occurs in-constrains the parser's search for an antecedent. The parser thus ignores candidate antecedents that might match agreement features of the reflexive (e.g., gender) but are ineligible as potential antecedents because they are in structurally illicit positions. An alternative possibility accords no special status to structural constraints: in addition to using Principle A, the parser also uses non-structural cues such as gender to access the antecedent. According to cue-based retrieval theories of memory (e.g., Lewis and Vasishth, 2005), the use of non-structural cues should result in increased retrieval times and occasional errors when candidates partially match the cues, even if the candidates are in structurally illicit positions. In this paper, we first show how the retrieval processes that underlie the reflexive binding are naturally realized in the Lewis and Vasishth (2005) model. We present the predictions of the model under the assumption that both structural and non-structural cues are used during retrieval, and provide a critical analysis of previous empirical studies that failed to find evidence for the use of non-structural cues, suggesting that these failures may be Type II errors. We use this analysis and the results of further modeling to motivate a new empirical design that we use in an eye tracking study. The results of this study confirm the key predictions of the model concerning the use of non-structural cues, and are inconsistent with the strictly syntactic search account. These results present a challenge for theories advocating the infallibility of the human parser in the case of reflexive resolution, and provide support for the inclusion of agreement features such as gender in the set of retrieval cues.

Predicting Short-Term Remembering as Boundedly Optimal Strategy Choice.

It is known that, on average, people adapt their choice of memory strategy to the subjective utility of interaction. What is not known is whether an individual's choices are boundedly optimal. Two experiments are reported that test the hypothesis that an individual's decisions about the distribution of remembering between internal and external resources are boundedly optimal where optimality is defined relative to experience, cognitive constraints, and reward. The theory makes predictions that are tested against data, not fitted to it. The experiments use a no-choice/choice utility learning paradigm where the no-choice phase is used to elicit a profile of each participant's performance across the strategy space and the choice phase is used to test predicted choices within this space. They show that the majority of individuals select strategies that are boundedly optimal. Further, individual differences in what people choose to do are successfully predicted by the analysis. Two issues are discussed: (a) the performance of the minority of participants who did not find boundedly optimal adaptations, and (b) the possibility that individuals anticipate what, with practice, will become a bounded optimal strategy, rather than what is boundedly optimal during training.

Why contextual preference reversals maximize expected value.

Contextual preference reversals occur when a preference for one option over another is reversed by the addition of further options. It has been argued that the occurrence of preference reversals in human behavior shows that people violate the axioms of rational choice and that people are not, therefore, expected value maximizers. In contrast, we demonstrate that if a person is only able to make noisy calculations of expected value and noisy observations of the ordinal relations among option features, then the expected value maximizing choice is influenced by the addition of new options and does give rise to apparent preference reversals. We explore the implications of expected value maximizing choice, conditioned on noisy observations, for a range of contextual preference reversal types-including attraction, compromise, similarity, and phantom effects. These preference reversal types have played a key role in the development of models of human choice. We conclude that experiments demonstrating contextual preference reversals are not evidence for irrationality. They are, however, a consequence of expected value maximization given noisy observations. (PsycINFO Database Record

An activation-based model of sentence processing as skilled memory retrieval.

We present a detailed process theory of the moment-by-moment working-memory retrievals and associated control structure that subserve sentence comprehension. The theory is derived from the application of independently motivated principles of memory and cognitive skill to the specialized task of sentence parsing. The resulting theory construes sentence processing as a series of skilled associative memory retrievals modulated by similarity-based interference and fluctuating activation. The cognitive principles are formalized in computational form in the Adaptive Control of Thought-Rational (ACT-R) architecture, and our process model is realized in ACT-R. We present the results of 6 sets of simulations: 5 simulation sets provide quantitative accounts of the effects of length and structural interference on both unambiguous and garden-path structures. A final simulation set provides a graded taxonomy of double center embeddings ranging from relatively easy to extremely difficult. The explanation of center-embedding difficulty is a novel one that derives from the model' complete reliance on discriminating retrieval cues in the absence of an explicit representation of serial order information. All fits were obtained with only 1 free scaling parameter fixed across the simulations; all other parameters were ACT-R defaults. The modeling results support the hypothesis that fluctuating activation and similarity-based interference are the key factors shaping working memory in sentence processing. We contrast the theory and empirical predictions with several related accounts of sentence-processing complexity.

A computational approach to control in complex cognition.

Cognitive deficits associated with dorsolateral prefrontal cortex (DLPFC) damage are often most apparent in higher cognitive tasks that involve problem solving and managing multiple goals. However, computational models of prefrontal deficits on such tasks are difficult to construct. Problem solving is most naturally modeled with symbolic systems (e.g. production systems), but the effects of lesions are most naturally modeled with subsymbolic systems (neural networks). We show that when we adopt a simple and plausible model of neural computation, there is a natural and explicit mapping from symbolic, goal-driven cognition onto neural computation. We exploit this mapping to construct a neural network model that is capable of solving complex problems in the Tower of London task. The model leads to a specific hypothesis about the role of DLPFC in such tasks, namely, that DLPFC represents internally generated subgoals that modulate competition among posterior representations. When intact, the model accurately simulates the behavior of college students even on the most difficult problems. Furthermore, when the subgoal component is lesioned, it accurately simulates the behavior of prefrontal patients, including the fact that their deficits are most apparent on the most difficult tasks and that they have special difficulty with tasks that require inhibiting a prepotent response.

Utility maximization and bounds on human information processing.

Utility maximization is a key element of a number of theoretical approaches to explaining human behavior. Among these approaches are rational analysis, ideal observer theory, and signal detection theory. While some examples of these approaches define the utility maximization problem with little reference to the bounds imposed by the organism, others start with, and emphasize approaches in which bounds imposed by the information processing architecture are considered as an explicit part of the utility maximization problem. These latter approaches are the topic of this issue of the journal.