The effect of prediction error on overt attention and learning in humans.

It has been suggested that attention modulates the speed at which cues come to predict contingent outcomes, and that attention changes with the prediction errors generated by cues. Evidence for this interaction in humans is inconsistent, with divergent findings depending on whether attention was measured with eye fixations or learning speed. We included both measures in our experiment. Initially, predictive cues (A and B) were consistently followed by one outcome (o1), while nonpredictive cues (X and Y) were followed by two randomly alternating outcomes (o1 and o2). Consistent with an effect of prediction error, participants' fixated for longer on the nonpredictive cues than on the predictive ones. Then, the cues were combined in three pairs: AX, followed by o1, and AY and BX, followed by o2. Discrimination of AX and AY depended on the previously nonpredictive cues and, given that these received more attention during initial training, it should proceed faster than discrimination of AX and BX, which depended on the previously predictive cues. However, participants learned to predict the outcomes of AY and BX at a similar rate. The fixation times were similar for the previously predictive and previously nonpredictive cues. We discuss reasons that could explain these findings.

An attentional perspective on differential fear conditioning in chronic pain: The informational value of safety cues.

Differences in fear conditioning between individuals suffering from chronic pain and healthy controls may indicate a learning bias that contributes to the acquisition and persistence of chronic pain. However, evidence from lab-controlled conditioning studies is sparse and previous experiments have produced inconsistent findings. Twenty-five participants suffering from chronic back pain and twenty-five controls not reporting chronic pain took part in a differential fear conditioning experiment measuring attention (eye tracking) and autonomic arousal (pupil dilation and skin conductance) elicited by visual cues predicting the presence or absence of electric shock. In contrast to the healthy control group, participants with chronic pain did not acquire differential autonomic responding to cues of threat and safety and specifically failed to acquire any attentional preference for the safety cue over irrelevant contextual cues (while such preference was intact for the threat cue). We present simulations of a reinforcement learning model to show how the pattern of data can be explained by assuming that participants with chronic pain might have experienced less positive emotion (relief) when the electric shock was absent following safety cues. Our model shows how this assumption can explain both, reduced differential responding to cues of threat and safety as well as less selective attention to the safety cue.

Outcome unpredictability affects outcome-specific motivation to learn.

Outcome predictability effects in associative learning paradigms describe better learning about outcomes with a history of greater predictability in a similar but unrelated task compared with outcomes with a history of unpredictability. Inspired by the similarities between this phenomenon and the effect of uncontrollability in learned helplessness paradigms, here, we investigate whether learning about unpredictability decreases outcome-specific motivation to learn. We used a modified version of the allergy task, in which participants first observe the foods eaten by a fictitious patient, followed by allergic reactions that he subsequently suffers, some of which are perfectly predictable and others unpredictable. We then implemented an active learning method in a second task in which participants could only learn about either the previously predictable or unpredictable outcomes on each trial. At the beginning of each trial, participants had to decide whether they wanted to learn about one outcome category or the other. Participants at the beginning of the second task chose to learn about the previously predictable outcomes first and to learn about the previously unpredictable outcomes in later trials. This showed that unpredictability affects future motivation to learn in other circumstances. Interestingly, we did not find any sign of outcome predictability effect at the end of the second phase, suggesting that participants compensate for biased outcome sampling when making overt choices in ways that they may not when learning about both outcome categories simultaneously.

Evidence for two attentional mechanisms during learning.

We sought to provide evidence for a combined effect of two attentional mechanisms during associative learning. Participants' eye movements were recorded as they predicted the outcomes following different pairs of cues. Across the trials of an initial stage, a relevant cue in each pair was consistently followed by one of two outcomes, while an irrelevant cue was equally followed by either of them. Thus, the relevant cue should have been associated with small relative prediction errors, compared to the irrelevant cue. In a later stage, each pair came to be followed by one outcome on a random half of the trials and by the other outcome on the remaining half, and thus there should have been a rise in the overall prediction error. Consistent with an attentional mechanism based on relative prediction error, an attentional advantage for the relevant cue was evident in the first stage. However, in accordance with a mechanism linked to overall prediction error, the attention paid to both types of cues increased at the beginning of the second stage. These results showed up in both dwell times and within-trial patterns of fixations, and they were predicted by a hybrid model of attention.

Emergence of complex dynamics of choice due to repeated exposures to extinction learning.

Extinction learning, the process of ceasing an acquired behavior in response to altered reinforcement contingencies, is not only essential for survival in a changing environment, but also plays a fundamental role in the treatment of pathological behaviors. During therapy and other forms of training involving extinction, subjects are typically exposed to several sessions with a similar structure. The effects of this repeated exposure are not well understood. Here, we studied the behavior of pigeons across several sessions of a discrimination-learning task in context A, extinction in context B, and a return to context A to test the context-dependent return of the learned responses (ABA renewal). By focusing on individual learning curves across animals, we uncovered a session-dependent variability of behavior: (1) during extinction, pigeons preferred the unrewarded alternative choice in one-third of the sessions, predominantly during the first one. (2) In later sessions, abrupt transitions of behavior at the onset of context B emerged, and (3) the renewal effect decayed as sessions progressed. We show that the observed results can be parsimoniously accounted for by a computational model based only on associative learning between stimuli and actions. Our work thus demonstrates the critical importance of studying the trial-by-trial dynamics of learning in individual sessions, and the power of "simple" associative learning processes.

A rise in prediction error increases attention to irrelevant cues.

We investigated whether a sudden rise in prediction error widens an individual's focus of attention by increasing ocular fixations on cues that otherwise tend to be ignored. To this end, we used a discrimination learning task including cues that were either relevant or irrelevant for predicting the outcomes. Half of participants experienced contingency reversal once they had learned to predict the outcomes (reversal group, n = 30). The other half experienced the same contingencies throughout the task (control group, n = 30). As participants' prediction accuracy increased, they showed a decrease in the number of fixations directed to the irrelevant cues. Following contingency reversal, participants in the reversal group showed a drop in accuracy, indicating a rise in prediction error, and fixated on the irrelevant cues more often than participants in the control group. We discuss the results in the context of attentional theories of associative learning.

Does learning history shape the associability of outcomes? Further tests of the outcome predictability effect.

In recent years, several studies of human predictive learning demonstrated better learning about outcomes that have previously been experienced as consistently predictable compared to outcomes previously experienced as less predictable, namely the outcome predictability effect. As this effect may have wide-reaching implications for current theories of associative learning, the present study aimed to examine the generality of the effect with a human goal-tracking paradigm, employing three different designs to manipulate the predictability of outcomes in an initial training phase. In contrast to the previous studies, learning in a subsequent phase, when every outcome was equally predictable by novel cues, was not reliably affected by the outcomes' predictability in the first phase. This lack of an outcome predictability effect provides insights into the parameters of the effect and its underlying mechanisms.

Adding or removing context components equally disrupts extinction in human predictive learning.

In two human predictive learning experiments, we investigated the impact of adding or removing context components on extinction performance toward a cue. In each experiment, participants initially received repeated pairings of a cue and an outcome in a context composed of two distinctive components (context AB). Initial training was followed by a series of trials in which the cue was no longer followed by the outcome. This extinction treatment was conducted in the presence of a different pair of distinctive context components (context CD). During a final test, we observed that changing the extinction context CD disrupted extinction performance toward the cue regardless of whether the context was changed by adding or removing context components. We discuss implications of our results for theories of associative learning.

The role of common elements in the redundancy effect.

In 2 experiments, participants received a predictive learning task in which the presence of 1 or 2 food items signaled the onset or absence of stomachache in a hypothetical patient. Their task was to identify the cues that signaled the occurrence, or nonoccurrence of this ailment. The 2 groups in Experiment 1 and the single group in Experiment 2 received a blocking treatment, where Cue A and a combination of Cues A and X both signaled stomachache, A+ AX+. These groups also received a simple discrimination where the outcome was signaled by one compound but not another, BY+ CY-. Subsequent test trials revealed the so-called redundancy effect, where X was regarded as a more reliable predictor of the outcome than Y. This result occurred when the trials with A+ preceded those with AX+ (Group E, Experiment 1 and Experiment 2), and when the trials with A+ and AX+ were intermixed (Group C, Experiment 1). The results challenge theories based on the assumption that cues presented together must compete for a limited pool of associative strength. Rather, they are said to support theories that assume changes in attention determine what is learned when two or more cues are presented together. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

How competitive is cue competition?

Cue competition refers to phenomena indicating that learning about the relationship between a cue and an outcome is influenced by learning about the predictive significance of other cues that are concurrently present. In two autoshaping experiments with pigeons, we investigated the strength of competition among cues for predictive value. In each experiment, animals received an overexpectation training (A+, D+ followed by AD+). In addition, the training schedule of each experiment comprised two control conditions-one condition to evaluate the presence of overexpectation (B+ followed by BY+) and a second one to assess the strength of competition among cues (C+ followed by CZ-). Training trials were followed by a test with individual stimuli (A, B, C). Experiment 1 revealed no evidence for cue competition as responding during the test mirrored the individual cue-outcome contingencies. The test results from Experiment 2, which included an outcome additivity training, showed cue competition in form of an overexpectation effect as responding was weaker for Stimulus A than Stimulus B. However, the test results from Experiment 2 also revealed that responding to Stimulus A was stronger than to Stimulus C, which indicates that competition among cues was not as strong as predicted by some influential theories of associative learning.

The fate of redundant cues in human predictive learning: The outcome ratio effect.

In four experiments, participants were shown a sequence of pairs of pictures of food and asked to predict whether each pair signalled an allergic reaction in a hypothetical patient. The pairs of pictures were used to present two simple discriminations that differed in their outcome ratio. A rich discrimination, 3AX+ BX-, involved three trials in which the compound of two foods, AX, was followed by a reaction, for every trial in which the compound BX was not followed by the outcome. A lean discrimination, CY+ 3DY- was based on the opposite outcome ratio. Upon the completion of this training, participants were asked to rate how likely an individual food would be followed by the allergic reaction. In each experiment, the rating for X was stronger than for Y. This outcome ratio effect poses a challenge for theories of learning that assume changes in associative strength are governed by a common error term, based on the significance of all the cues present on a trial. Instead, the results are consistent with the assumption that changes in associative strength are governed by an individual error term, based on the significance of a single cue.

Attention toward contexts modulates context-specificity of behavior in human predictive learning: Evidence from the n-back task.

According to the attentional theory of context processing (ATCP), learning becomes context specific when acquired under conditions that promote attention toward contextual stimuli regardless of whether attention deployment is guided by learning experience or by other factors unrelated to learning. In one experiment with humans, we investigated whether performance in a predictive learning task can be brought under contextual control by means of a secondary task that was unrelated to predictive learning, but supposed to modulate participants' attention toward contexts. Initially, participants acquired cue-outcome relationships presented in contexts that were each composed of two elements from two dimensions. Acquisition training in the predictive learning task was combined with a one-back task that required participants to match across consecutive trials context elements belonging to one of the two dimensions. During a subsequent test, we observed that acquisition behavior in the predictive learning task was disrupted by changing the acquisition context along the dimension that was relevant for the one-back task, while there was no evidence for context specificity of predictive learning when the acquisition context was changed along the dimension that was irrelevant for the one-back task. Our results support the generality of the principles advocated by ATCP.

Pupil dilation indicates the coding of past prediction errors: Evidence for attentional learning theory.

The attentional learning theory of Pearce and Hall () predicts more attention to uncertain cues that have caused a high prediction error in the past. We examined how the cue-elicited pupil dilation during associative learning was linked to such error-driven attentional processes. In three experiments, participants were trained to acquire associations between different cues and their appetitive (Experiment 1), motor (Experiment 2), or aversive (Experiment 3) outcomes. All experiments were designed to examine differences in the processing of continuously reinforced cues (consistently followed by the outcome) versus partially reinforced, uncertain cues (randomly followed by the outcome). We measured the pupil dilation elicited by the cues in anticipation of the outcome and analyzed how this conditioned pupil response changed over the course of learning. In all experiments, changes in pupil size complied with the same basic pattern: During early learning, consistently reinforced cues elicited greater pupil dilation than uncertain, randomly reinforced cues, but this effect gradually reversed to yield a greater pupil dilation for uncertain cues toward the end of learning. The pattern of data accords with the changes in prediction error and error-driven attention formalized by the Pearce-Hall theory.

Context modulation of learned attention deployment.

In three experiments, we investigated the contextual control of attention in human discrimination learning. In each experiment, participants initially received discrimination training in which the cues from Dimension A were relevant in Context 1 but irrelevant in Context 2, whereas the cues from Dimension B were irrelevant in Context 1 but relevant in Context 2. In Experiment 1, the same cues from each dimension were used in Contexts 1 and 2, whereas in Experiments 2 and 3, the cues from each dimension were changed across contexts. In each experiment, participants were subsequently shifted to a transfer discrimination involving novel cues from either dimension, to assess the contextual control of attention. In Experiment 1, measures of eye gaze during the transfer discrimination revealed that Dimension A received more attention than Dimension B in Context 1, whereas the reverse occurred in Context 2. Corresponding results indicating the contextual control of attention were found in Experiments 2 and 3, in which we used the speed of learning (associability) as an indirect marker of learned attentional changes. Implications of our results for current theories of learning and attention are discussed.

Learned predictiveness and outcome predictability effects are not simply two sides of the same coin.

The Learned Predictiveness effect refers to the observation that learning about the relationship between a cue and an outcome is influenced by the predictive relevance of the cue for other outcomes. Similarly, the Outcome Predictability effect refers to a recent observation that the previous predictability of an outcome affects learning about this outcome in new situations, too. We hypothesize that both effects may be two manifestations of the same phenomenon and stimuli that have been involved in highly predictive relationships may be learned about faster when they are involved in new relationships regardless of their functional role in predictive learning as cues and outcomes. Four experiments manipulated both the relationships and the function of the stimuli. While we were able to replicate the standard effects, they did not survive a transfer to situations where the functional role of the stimuli changed, that is the outcome of the first phase becomes a cue in the second learning phase or the cue of the first phase becomes the outcome of the second phase. Furthermore, unlike learned predictiveness, there was little indication that the distribution of overt attention in the second phase was influenced by previous predictability. The results suggest that these 2 very similar effects are not manifestations of a more general phenomenon but rather independent from each other. (PsycINFO Database Record

Fear acquisition and liking of out-group and in-group members: Learning bias or attention?

The present study explores the notion of an out-group fear learning bias that is characterized by facilitated fear acquisition toward harm-doing out-group members. Participants were conditioned with two in-group and two out-group faces as conditioned stimuli. During acquisition, one in-group and one out-group face was paired with an aversive shock whereas the other in-group and out-group face was presented without shock. Psychophysiological measures of fear conditioning (skin conductance and pupil size) and explicit and implicit liking exhibited increased differential responding to out-group faces compared to in-group faces. However, the results did not clearly indicate that harm-doing out-group members were more readily associated with fear than harm-doing in-group members. In contrast, the out-group face not paired with shock decreased conditioned fear and disliking at least to the same extent that the shock-associated out-group face increased these measures. Based on these results, we suggest an account of the out-group fear learning bias that relates to an attentional bias to process in-group information.

Reward Draws the Eye, Uncertainty Holds the Eye: Associative Learning Modulates Distractor Interference in Visual Search.

Stimuli in our sensory environment differ with respect to their physical salience but moreover may acquire motivational salience by association with reward. If we repeatedly observed that reward is available in the context of a particular cue but absent in the context of another cue the former typically attracts more attention than the latter. However, we also may encounter cues uncorrelated with reward. A cue with 50% reward contingency may induce an average reward expectancy but at the same time induces high reward uncertainty. In the current experiment we examined how both values, reward expectancy and uncertainty, affected overt attention. Two different colors were established as predictive cues for low reward and high reward respectively. A third color was followed by high reward on 50% of the trials and thus induced uncertainty. Colors then were introduced as distractors during search for a shape target, and we examined the relative potential of the color distractors to capture and hold the first fixation. We observed that capture frequency corresponded to reward expectancy while capture duration corresponded to uncertainty. The results may suggest that within trial reward expectancy is represented at an earlier time window than uncertainty.

Attentional Bias for Uncertain Cues of Shock in Human Fear Conditioning: Evidence for Attentional Learning Theory.

We conducted a human fear conditioning experiment in which three different color cues were followed by an aversive electric shock on 0, 50, and 100% of the trials, and thus induced low (L), partial (P), and high (H) shock expectancy, respectively. The cues differed with respect to the strength of their shock association (L < P < H) and the uncertainty of their prediction (L < P > H). During conditioning we measured pupil dilation and ocular fixations to index differences in the attentional processing of the cues. After conditioning, the shock-associated colors were introduced as irrelevant distracters during visual search for a shape target while shocks were no longer administered and we analyzed the cues' potential to capture and hold overt attention automatically. Our findings suggest that fear conditioning creates an automatic attention bias for the conditioned cues that depends on their correlation with the aversive outcome. This bias was exclusively linked to the strength of the cues' shock association for the early attentional processing of cues in the visual periphery, but additionally was influenced by the uncertainty of the shock prediction after participants fixated on the cues. These findings are in accord with attentional learning theories that formalize how associative learning shapes automatic attention.

Extinction in multiple contexts: Effects on the rate of extinction and the strength of response recovery.

In two human predictive-learning experiments, we investigated the effects of extinction in multiple contexts on the rate of extinction and the strength of response recovery. In each experiment, participants initially received acquisition training with a target cue in one context, followed by extinction either in a different context (extinction in a single context) or in three different contexts (extinction in multiple contexts). The results of both experiments showed that conducting extinction in multiple contexts led to higher levels of responding during extinction than did extinction in a single context. Additionally, Experiment 2 showed that extinction in multiple contexts prevented ABC renewal but had no detectable impact on ABA renewal. Our results are discussed within the framework of contemporary learning theories of contextual control and extinction.

Cross-cultural adaption of the German Quebec Back Pain Disability Scale: an exposure-specific measurement for back pain patients.

STUDY DESIGN: Cross-cultural translation and psychometric testing. OBJECTIVE: The purpose of the present study was to examine reliability and validity of a cross-cultural adaption of the German Quebec Back Pain Disability Scale (QBPDS) in a context of a randomized controlled trial evaluating the effectiveness of graded in vivo exposure in chronic low back pain patients. BACKGROUND: The QBPDS is one of the most widely used disease-specific disability questionnaires. In particular, for cognitive behavioral treatments with a clear focus on behavioral aspects such as graded in vivo exposure, the QBPDS provides an ascertained strategy with a sound conceptual basis and excellent quality criteria. Nevertheless, there is conflicting evidence concerning factor structure and a German adaption is missing. METHODS: The cross-cultural adaption followed international guidelines. Psychometric testing was performed using data from 180 participants with chronic low back pain. The psychometric analyses included internal consistency, convergent, and divergent validity. Exploratory factor analysis was used to determine the underlying factor structure. RESULTS: The QBPDS showed strong psychometric properties, with high internal consistency for the full scale (α=0.94) and good convergent and divergent validity. The factor analysis revealed a four-factor solution (bending, ambulation, brief effortful movements, and long-lasting postures). CONCLUSION: The translation and cross-cultural adaption of the QBPDS into German was successful. The German version proved to be a valid and reliable instrument and is well suited for use in the context of an exposure-based psychological treatment.