What makes a stimulus worthy of attention: Cue-outcome correlation and choice relevance in the learned predictiveness effect.

The learned predictiveness effect refers to the tendency for predictive cues to attract greater attention and show faster learning in subsequent tasks. However, in typical designs, the predictiveness of each cue (its objective cue-outcome correlation) is confounded with the degree to which it is informative for making the correct response on each trial (a feature we term choice relevance). In four experiments, we tested the unique contributions of cue-outcome correlation and choice relevance to the learned predictiveness effect by manipulating the outcome choices available on each trial. Experiments 1A and 1B compared two sets of partially predictive cues and found that participants learned more in a transfer phase about the set of cues that were previously choice-relevant. Experiments 2A and 2B used a design in which the cue-outcome correlation was stronger for one set of cues (perfect predictors) than the other set (imperfect predictors). Manipulating the choice relevance of the imperfect predictors in this design did not influence the magnitude of the learning bias toward the perfect predictor. Unlike cue-outcome correlation, choice relevance did not seem to correspond to biases in eye-gaze, suggesting that it operates via a distinct mechanism. Simulations with a modified EXIT model successfully predicted cue-outcome correlation and choice relevance effects by assuming that participants update learning for present outcomes only, but incorrectly predicted additive effects. We conclude that cue-outcome correlation and choice relevance are important factors that can lead to biases in future learning; both were individually sufficient but neither was necessary. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Aversion, interpretation and determinability: Three factors of uncertainty that may play a role in psychopathology.

This opinion piece considers the construct of tolerance of uncertainty and suggests that it should be viewed in the context of three psychological factors: uncertainty aversion, uncertainty interpretation, and uncertainty determinability. Uncertainty aversion refers to a dislike of situations in which the outcomes are not deterministic and is similar to conventional conceptions of (in)tolerance of uncertainty. Uncertainty interpretation refers to the extent to which variability in an observed outcome is interpreted as random fluctuation around a relatively stable base-rate versus frequent and rapid changes in the base-rate. Uncertainty determinability refers to the (actual or perceived) capacity of the individual to generate any meaningful expectancy of the uncertain outcome, which may be undeterminable if predictions are updated too quickly. We argue that uncertainty interpretation and determinability are psychological responses to the experience of probabilistic events that vary among individuals and can moderate negative affect experienced in response to uncertainty. We describe how individual differences in basic parameters of associative learning (modelled by a simple learning window) could lead to this variation. To explain these hypotheses, we utilise the distinction between aleatory uncertainty (the inherent unpredictability of individual stochastic events) and epistemic uncertainty (obtainable knowledge that the individual lacks or perceives to be lacking). We argue that when expectancies are updated quickly, epistemic uncertainty will dominate the individual's representation of the events around them, leading to a subjective experience of the world as one that is volatile and unpredictable.

Do partial and distributed tests enhance new learning?

Testing facilitates subsequent learning of new information, a phenomenon known as the forward testing effect. The effect is often investigated in multilist procedures, where studied lists are followed by a retrieval test, or a control task such as restudying, and learning is compared on the final list. In most studies of the effect, tests include all material from the preceding list. We report four experiments, three of which were preregistered, to determine whether tests that are partial (not including all studied items) and distributed (including retrieval of items from earlier lists) are effective in enhancing new learning. The results show that testing of all studied material is not necessary to produce beneficial effects on new learning or to reduce intrusions. The beneficial effects of testing were substantially mediated by reduced proactive interference. Importantly, there was minimal evidence that the forward learning benefits of partial and distributed tests are offset by a cost to untested items via retrieval-induced forgetting. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Frequency effects in action versus value learning.

Recent work in reinforcement learning has demonstrated a choice preference for an option that has a lower probability of reward (A) when paired with an alternative option that has a higher probability of reward (C), if A has been experienced more frequently than C (the frequency effect). This finding is critical as it is inconsistent with widespread assumptions that expected value is based on average reward, and instead suggests that value is based on cumulative instances of reward. However, option frequency may also affect instrumental reinforcement of choosing A during training, which may then transfer to choice on AC trials. This study therefore aimed to assess the contribution of action reinforcement and option value to the frequency-effect across 2 experiments. In both experiments we included an additional test phase in which participants were asked to rate the likelihood of reward for each choice option, a response that should be unaffected by action reinforcement. In Experiment 1, participants completed the original choice training phase. In Experiment 2, participants were presented with each option individually, thus removing reinforcement of choice during training. Single cue training reduced the strength of the preference for A compared to choice training, suggesting a contributing role of action reinforcement. However, frequency effects were still evident in both experiments. We found that the pattern of reward likelihood ratings was consistent with the pattern of choice preferences in both experiments, suggesting that action reinforcement may also influence judgements about the likelihood of receiving reward. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Neural regions associated with gain-loss frequency and average reward in older and younger adults.

Research on the biological basis of reinforcement-learning has focused on how brain regions track expected value based on average reward. However, recent work suggests that humans are more attuned to reward frequency. Furthermore, older adults are less likely to use expected values to guide choice than younger adults. This raises the question of whether brain regions assumed to be sensitive to average reward, like the medial and lateral PFC, also track reward frequency, and whether there are age-based differences. Older (60-81 years) and younger (18-30 years) adults performed the Soochow Gambling task, which separates reward frequency from average reward, while undergoing fMRI. Overall, participants preferred options that provided negative net payoffs, but frequent gains. Older adults improved less over time, were more reactive to recent negative outcomes, and showed greater frequency-related activation in several regions, including DLPFC. We also found broader recruitment of prefrontal and parietal regions associated with frequency value and reward prediction errors in older adults, which may indicate compensation. The results suggest greater reliance on average reward for younger adults than older adults.

Hearing hooves, thinking zebras: A review of the inverse base-rate effect.

People often fail to use base-rate information appropriately in decision-making. This is evident in the inverse base-rate effect, a phenomenon in which people tend to predict a rare outcome for a new and ambiguous combination of cues. While the effect was first reported in 1988, it has recently seen a renewed interest from researchers concerned with learning, attention and decision-making. However, some researchers have raised concerns that the effect arises in specific circumstances and is unlikely to provide insight into general learning and decision-making processes. In this review, we critically evaluate the evidence for and against the main explanations that have been proposed to explain the effect, and identify where this evidence is currently weak. We argue that concerns about the effect are not well supported by the data. Instead, the evidence supports the conclusion that the effect is a result of general mechanisms that provides a useful opportunity to understand the processes involved in learning and decision making. We discuss gaps in our knowledge and some promising avenues for future research, including the relevance of the effect to models of attentional change in learning, an area where the phenomenon promises to contribute new insights.

Attention biases in the inverse base-rate effect persist into new learning.

The inverse base-rate effect is a tendency to predict the rarer of two outcomes when presented with cues that make conflicting predictions. Attention-based accounts of the effect appeal to prioritised attention to predictors of rare outcomes. Changes in the processing of these cues are predicted to increase the rate at which they are learned about in the future (i.e., their associability). Our previous work has shown that the development of the inverse base-rate effect is accompanied by greater overt attention to the rare predictor while participants made predictions, and during feedback, and these biases change in different ways depending on the stage of training and global base-rate differences. It is unknown whether these gaze patterns reflect the manner in which cues are prioritised for learning or are merely a consequence of learning what the cues predict. This study tested whether the associability of common and rare predictors differed, and if so, how this difference changed as a function of training length and the presence of base-rate differences in the outcomes. Experiment 1 tested cue associability using a second learning task presented after either short or long training. The results suggest an associability advantage for rare predictors that weakens with extended training and is not strongly affected by the presence of global base-rate differences. However, Experiment 2 showed a clear effect of base-rate differences on choice after very brief training, indicating that attention biases as measured by associability change are not sufficient to produce the inverse base-rate effect.

Does learning history shape the associability of outcomes? Further tests of the outcome predictability effect.

In recent years, several studies of human predictive learning demonstrated better learning about outcomes that have previously been experienced as consistently predictable compared to outcomes previously experienced as less predictable, namely the outcome predictability effect. As this effect may have wide-reaching implications for current theories of associative learning, the present study aimed to examine the generality of the effect with a human goal-tracking paradigm, employing three different designs to manipulate the predictability of outcomes in an initial training phase. In contrast to the previous studies, learning in a subsequent phase, when every outcome was equally predictable by novel cues, was not reliably affected by the outcomes' predictability in the first phase. This lack of an outcome predictability effect provides insights into the parameters of the effect and its underlying mechanisms.

Relational rule discovery in complex discrimination learning.

Failure to learn and generalize abstract relational rules has critical implications for education. In this study, we aimed to determine which training conditions facilitate relational transfer in a relatively simple (patterning) discrimination versus a relatively complex (biconditional) discrimination. The amount of training participants received had little influence on rates of relational transfer. Instead, trial-sequencing of the training contingencies influenced relational transfer in different ways depending on the complexity of the discrimination. Clustering instances of relational rules together during training improved transfer of both simpler patterning and more difficult biconditional rules, regardless of individual differences in cognitive reflection. However, blocking all trials of the same type together improved rule transfer only for biconditional discriminations. Individual differences in cognitive reflection were also more predictive of relational rule use under suboptimal training conditions. The results highlight the need for comprehensive accounts of relational learning to consider how learning conditions and individual differences affect the likelihood of engaging in learning relational structures. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Learning reward frequency over reward probability: A tale of two learning rules.

Learning about the expected value of choice alternatives associated with reward is critical for adaptive behavior. Although human choice preferences are affected by the presentation frequency of reward-related alternatives, this may not be captured by some dominant models of value learning, such as the delta rule. In this study, we examined whether reward learning is driven more by learning the probability of reward provided by each option, or how frequently each option has been rewarded, and assess how well models based on average reward (e.g. the delta model) and models based on cumulative reward (e.g. the decay model) can account for choice preferences. In a binary-outcome choice task, participants selected between pairs of options that had reward probabilities of 0.65 (A) versus 0.35 (B) or 0.75 (C) versus 0.25 (D). Crucially, during training there were twice the number of AB trials as CD trials, such that option A was associated with higher cumulative reward, while option C gave higher average reward. Participants then decided between novel combinations of options (e.g., AC). Most participants preferred option A over C, a result predicted by the Decay model, but not the Delta model. We also compared the Delta and Decay models to both more simplified as well as more complex models that assumed additional mechanisms, such as representation of uncertainty. Overall, models that assume learning about cumulative reward provided the best account of the data.

Learned predictiveness models predict opposite attention biases in the inverse base-rate effect.

Several attention-based models of associative learning are built upon the learned predictiveness principle, whereby learning is optimized by attending to the most predictive features and ignoring the least predictive features. Despite their functional similarity, these models differ in their formal mechanisms and thus may produce very different predictions in some circumstances. As we demonstrate, this is particularly evident in the inverse base-rate effect. Using simulations with a modified Mackintosh model and the EXIT model, we found that models based on the learned predictiveness principle can account for rare-outcome choice biases associated with the inverse base-rate effect, despite making opposite predictions for relative attention to rare versus common predictors. The models also make different predictions regarding changes in attention across training, and effects of context associations on attention to cues. Using a human causal learning task, we replicated the inverse base-rate effect and a recently reported reduction in this effect when the context is not predictive of the common outcome and used eye-tracking to test model predictions about changes in attention both prior to making a decision, and during feedback. The results support the predictions made by EXIT, where the rare predictor commands greater attention than the common predictor throughout training. In addition, patterns of attention prior to making a decision differed to those during feedback, where effects of using a partially predictive context were evident only prior to making a prediction. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Transfer of associability and relational structure in human associative learning.

A wealth of recent studies have demonstrated that predictive cues involved in a linearly solvable component discrimination gain associability in subsequent learning relative to nonpredictive cues. In contrast, contradictory findings have been reported about the fate of cues involved in learning biconditional discriminations in which the cues are relevant but none are individually predictive of a specific outcome. In 3 experiments we examined the transfer of learning from component and biconditional discriminations in a within-subjects design. The results show a greater benefit in associability for cues that had previously served as predictive cues in a component discrimination than cues previously used in a biconditional discrimination. Further, new biconditional discriminations were learned faster when they were composed of cues that were previously trained in separate biconditional discriminations. Similarly, new component discriminations were learned faster when they were composed of cues that were previously trained in a separate component discriminations irrespective of whether they were previously predictive or previously nonpredictive. These results provide novel evidence that cue-specific learning of relational structure affects subsequent learning, suggesting changes in cue processing that go beyond simple changes in cue associability based on learned predictiveness. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Relational discovery in category learning.

Learning categories defined by the relations among objects supports the transfer of knowledge from initial learning contexts to novel contexts that share few surface similarities. Often relational categories have correlated (but nonessential) surface features, which can be a distraction from discovering the category-defining relations, preventing knowledge transfer. This is one explanation for "the inert knowledge problem" in education wherein many students fail to spontaneously apply their learning outside the classroom. Here we present a series of experiments using artificial categories that correlate surface features and relational patterns during learning. Our goal was to determine what task parameters and individual differences in learners shift focus to the relational aspect of the category and foster transfer to novel disparate exemplars. We consistently showed that the effectiveness of task structure manipulations (e.g., the sequence of learning exemplars) depended on the learners' strategies (e.g., whether learners are oriented toward discovering rules or focusing on exemplars). Further, we found support that "inference-learning," wherein learners are presented with incomplete exemplars and learn how to infer the missing pieces, is an effective way to promote relational discovery and transfer, even for learners who are not predisposed to make such discoveries. (PsycINFO Database Record

A Case of Divergent Predictions Made by Delta and Decay Rule Learning Models.

The Delta and Decay rules are two learning rules used to update expected values in reinforcement learning (RL) models. The delta rule learns average rewards, whereas the decay rule learns cumulative rewards for each option. Participants learned to select between pairs of options that had reward probabilities of .65 (option A) versus .35 (option B) or .75 (option C) versus .25 (option D) on separate trials in a binary-outcome choice task. Crucially, during training there were twice as AB trials as CD trials, therefore participants experienced more cumulative reward from option A even though option C had a higher average reward rate (.75 versus .65). Participants then decided between novel combinations of options (e.g, A versus C). The Decay model predicted more A choices, but the Delta model predicted more C choices, because those respective options had higher cumulative versus average reward values. Results were more in line with the Decay model's predictions. This suggests that people may retrieve memories of cumulative reward to compute expected value instead of learning average rewards for each option.

Effects of outcome and trial frequency on the inverse base-rate effect.

The inverse base-rate effect is a bias in contingency learning in which participants tend to predict a rare outcome for a conflicting set of perfectly predictive cues. Although the effect is often explained by attention biases during learning, inferential strategies at test may also contribute substantially to the effect. In three experiments, we manipulated the frequencies of outcomes and trial types to determine the critical conditions for the effect, thereby providing novel tests of the reasoning processes that could contribute to it. The rare bias was substantially reduced when the outcomes were experienced at equal rates in the presence of predictive-cue frequency differences (Exp. 2), and when the predictive cues were experienced at equal rates in the presence of outcome frequency differences (Exp. 3). We also found a consistent common-outcome bias for novel cue compounds. The results indicate the importance of both cue and outcome frequencies to the inverse base-rate effect, and reveal a combination of necessary conditions that are not well captured by appealing to inferential strategies at test. Although both attention-based and inferential theories explain some aspects of these data, no existing theory fully accounts for these effects of relative novelty.

Rule abstraction, model-based choice, and cognitive reflection.

Numerous tasks in learning and cognition have demonstrated differences in response patterns that may reflect the operation of two distinct systems. For example, causal and reinforcement learning tasks each show responding that considers abstract structure as well as responding based on simple associations. Nevertheless, there has been little attempt to verify whether these tasks are measuring related processes. The current study therefore investigated the relationship between rule- and feature-based generalization in a causal learning task, and model-based and model-free responding in a reinforcement learning task, including cognitive reflection as a predictor of individual tendencies to use controlled, deliberative processes in these tasks. We found that the use of rule-based generalization in a patterning task was a significant predictor of model-based, but not model-free, choice. Individual differences in cognitive reflection were significantly correlated with performance in both tasks, although this did not predict variation in model-based choice independently of rule-based generalization. Thus, although there is evidence of stable individual differences in the use of higher order processes across tasks, there may also be differences in mechanisms that these tasks reveal.

Resistance to instructed reversal of the learned predictiveness effect.

The learned predictiveness effect is a widely observed bias towards previously predictive cues in novel situations. Although the effect is generally attributed to an automatic attentional shift, it has recently been explained as the product of controlled inferences about the predictive value of cues. This view is supported by the susceptibility of learned predictiveness to instruction manipulation. However, recent research has shown conflicting results. Three experiments investigated the parameters of the instructed reversal effect in a human causal learning task, to determine the relative contribution of automatic and controlled attention processes. Experiment 1 showed that reversal instructions abolished, but did not reverse, the learned predictiveness effect, although length of initial training had no effect on the extent to which predictive cues subsequently captured attention. Experiment 2 explored whether particular causal scenarios lend themselves more readily to instructed reversal, but still failed to establish a significant reversal effect. Experiment 3 demonstrated a significant reversal effect when nonpredictive cues were explicitly and individually identified as the causes of outcomes. However, this effect was considerably weaker than the learned predictiveness effect when predictive cues were identified in the same way. Taken together, the results are inconsistent with a purely controlled account of learned predictiveness and provide support for dual-process theories of learning and attention.