Outcome unpredictability affects outcome-specific motivation to learn.

Outcome predictability effects in associative learning paradigms describe better learning about outcomes with a history of greater predictability in a similar but unrelated task compared with outcomes with a history of unpredictability. Inspired by the similarities between this phenomenon and the effect of uncontrollability in learned helplessness paradigms, here, we investigate whether learning about unpredictability decreases outcome-specific motivation to learn. We used a modified version of the allergy task, in which participants first observe the foods eaten by a fictitious patient, followed by allergic reactions that he subsequently suffers, some of which are perfectly predictable and others unpredictable. We then implemented an active learning method in a second task in which participants could only learn about either the previously predictable or unpredictable outcomes on each trial. At the beginning of each trial, participants had to decide whether they wanted to learn about one outcome category or the other. Participants at the beginning of the second task chose to learn about the previously predictable outcomes first and to learn about the previously unpredictable outcomes in later trials. This showed that unpredictability affects future motivation to learn in other circumstances. Interestingly, we did not find any sign of outcome predictability effect at the end of the second phase, suggesting that participants compensate for biased outcome sampling when making overt choices in ways that they may not when learning about both outcome categories simultaneously.

Time in Associative Learning: A Review on Temporal Maps.

Ability to recall the timing of events is a crucial aspect of associative learning. Yet, traditional theories of associative learning have often overlooked the role of time in learning association and shaping the behavioral outcome. They address temporal learning as an independent and parallel process. Temporal Coding Hypothesis is an attempt to bringing together the associative and non-associative aspects of learning. This account proposes temporal maps, a representation that encodes several aspects of a learned association, but attach considerable importance to the temporal aspect. A temporal map helps an agent to make inferences about missing information by applying an integration mechanism over a common element present in independently acquired temporal maps. We review the empirical evidence demonstrating the construct of temporal maps and discuss the importance of this concept in clinical and behavioral interventions.

Does learning history shape the associability of outcomes? Further tests of the outcome predictability effect.

In recent years, several studies of human predictive learning demonstrated better learning about outcomes that have previously been experienced as consistently predictable compared to outcomes previously experienced as less predictable, namely the outcome predictability effect. As this effect may have wide-reaching implications for current theories of associative learning, the present study aimed to examine the generality of the effect with a human goal-tracking paradigm, employing three different designs to manipulate the predictability of outcomes in an initial training phase. In contrast to the previous studies, learning in a subsequent phase, when every outcome was equally predictable by novel cues, was not reliably affected by the outcomes' predictability in the first phase. This lack of an outcome predictability effect provides insights into the parameters of the effect and its underlying mechanisms.

Adding or removing context components equally disrupts extinction in human predictive learning.

In two human predictive learning experiments, we investigated the impact of adding or removing context components on extinction performance toward a cue. In each experiment, participants initially received repeated pairings of a cue and an outcome in a context composed of two distinctive components (context AB). Initial training was followed by a series of trials in which the cue was no longer followed by the outcome. This extinction treatment was conducted in the presence of a different pair of distinctive context components (context CD). During a final test, we observed that changing the extinction context CD disrupted extinction performance toward the cue regardless of whether the context was changed by adding or removing context components. We discuss implications of our results for theories of associative learning.

Testing the deductive inferential account of blocking in causal learning.

The sensitivity of the blocking effect to outcome additivity pretraining has been used to argue that the phenomenon is the result of deductive inference, and to draw general conclusions about the nature of human causal learning. In two experiments, we manipulated participants' assumptions about the additivity of the outcome using pretraining before a typical blocking procedure. Ratings measuring causal judgments, confidence, and expected severity of the outcome were used concurrently to investigate how pretraining affected assumptions of outcome additivity and blocking. In Experiment 1, additive pretraining led to lower causal ratings and higher confidence ratings of the blocked cue, relative to control cues, consistent with the notion that additive pretraining encourages deductive reasoning. However, Experiments 1 and 2 showed that removing additivity assumptions through nonadditive pretraining had no impact on a statistically reliable blocking effect observed in a blocking procedure with no pretraining. We found no evidence that the blocking effect in the absence of pretraining was related to the participants' assumptions about the additivity of the outcome. Although additive pretraining may enhance blocking by encouraging deductive reasoning about the blocked cue, the evidence suggests that blocking in causal learning is not reliant on this reasoning and that humans do not readily engage in deduction merely because they possess the assumptions that permit its use.

Transfer of associability and relational structure in human associative learning.

A wealth of recent studies have demonstrated that predictive cues involved in a linearly solvable component discrimination gain associability in subsequent learning relative to nonpredictive cues. In contrast, contradictory findings have been reported about the fate of cues involved in learning biconditional discriminations in which the cues are relevant but none are individually predictive of a specific outcome. In 3 experiments we examined the transfer of learning from component and biconditional discriminations in a within-subjects design. The results show a greater benefit in associability for cues that had previously served as predictive cues in a component discrimination than cues previously used in a biconditional discrimination. Further, new biconditional discriminations were learned faster when they were composed of cues that were previously trained in separate biconditional discriminations. Similarly, new component discriminations were learned faster when they were composed of cues that were previously trained in a separate component discriminations irrespective of whether they were previously predictive or previously nonpredictive. These results provide novel evidence that cue-specific learning of relational structure affects subsequent learning, suggesting changes in cue processing that go beyond simple changes in cue associability based on learned predictiveness. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Learned biases in the processing of outcomes: A brief review of the outcome predictability effect.

Much empirical work and theoretical discussion in the associative learning literature has focused on when and how a cue changes in its associability. A series of new findings in human learning preparations (collectively referred to as the "outcome predictability" effect) appear to show that outcomes vary in their capacity to enter into novel associations as a product of their associative history. This effect is reminiscent of how cues change in associability as a consequence of their reinforcement history. We review the new findings within a broader associative literature that has previously investigated how conditioning can modify the effectiveness of outcome events to motivate new learning. A variety of explanations arising from this review are then critically considered. The article concludes by identifying novel questions brought into focus by the outcome predictability effect. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Learned predictiveness and outcome predictability effects are not simply two sides of the same coin.

The Learned Predictiveness effect refers to the observation that learning about the relationship between a cue and an outcome is influenced by the predictive relevance of the cue for other outcomes. Similarly, the Outcome Predictability effect refers to a recent observation that the previous predictability of an outcome affects learning about this outcome in new situations, too. We hypothesize that both effects may be two manifestations of the same phenomenon and stimuli that have been involved in highly predictive relationships may be learned about faster when they are involved in new relationships regardless of their functional role in predictive learning as cues and outcomes. Four experiments manipulated both the relationships and the function of the stimuli. While we were able to replicate the standard effects, they did not survive a transfer to situations where the functional role of the stimuli changed, that is the outcome of the first phase becomes a cue in the second learning phase or the cue of the first phase becomes the outcome of the second phase. Furthermore, unlike learned predictiveness, there was little indication that the distribution of overt attention in the second phase was influenced by previous predictability. The results suggest that these 2 very similar effects are not manifestations of a more general phenomenon but rather independent from each other. (PsycINFO Database Record

Summation effects in human learning: evidence from patterning discriminations in goal-tracking.

Participants in two human goal-tracking experiments were simultaneously trained with negative patterning (NP) and positive patterning (PP) discriminations (A+, B+, AB-, C-, D-, CD+). Both elemental and configural models of associative learning predict a PP advantage, such that NP is solved less readily than PP. However, elemental models like the unique cue approach additionally predict responding in AB- trials to be initially stronger than that in A+ and B+ trials due to summation of associative strength. Both experiments revealed a PP advantage and a strong summation effect in AB- trials in the first half of the experiments, irrespective of whether the same US was used for both discriminations (Experiment 1) or two different USs (Experiment 2). We discuss that the correct predictions of the unique cue approach are based on its assumptions of non-normalized and context-independent stimulus processing rather than elemental processing per se.

Why Harmless Sensations Might Hurt in Individuals with Chronic Pain: About Heightened Prediction and Perception of Pain in the Mind.

In individuals with chronic pain harmless bodily sensations can elicit anticipatory fear of pain resulting in maladaptive responses such as taking pain medication. Here, we aim to broaden the perspective taking into account recent evidence that suggests that interoceptive perception is largely a construction of beliefs, which are based on past experience and that are kept in check by the actual state of the body. Taking a Bayesian perspective, we propose that individuals with chronic pain display a heightened prediction of pain [prior probability p(pain)], which results in heightened pain perception [posterior probability p(pain|sensation)] due to an assumed link between pain and a harmless bodily sensation [p(sensation|pain)]. This pain perception emerges because their mind infers pain as the most likely cause for the sensation. When confronted with a mismatch between predicted pain and a (harmless bodily) sensation, individuals with chronic pain try to minimize the mismatch most likely by active inference of pain or alternatively by an attentional shift away from the sensation. The active inference results in activities that produce a stronger sensation that will match with the prediction, allowing subsequent perceptual inference of pain. Here, we depict heightened pain perception in individuals with chronic pain by reformulating and extending the assumptions of the interoceptive predictive coding model from a Bayesian perspective. The review concludes with a research agenda and clinical considerations.

Extinction in multiple contexts: Effects on the rate of extinction and the strength of response recovery.

In two human predictive-learning experiments, we investigated the effects of extinction in multiple contexts on the rate of extinction and the strength of response recovery. In each experiment, participants initially received acquisition training with a target cue in one context, followed by extinction either in a different context (extinction in a single context) or in three different contexts (extinction in multiple contexts). The results of both experiments showed that conducting extinction in multiple contexts led to higher levels of responding during extinction than did extinction in a single context. Additionally, Experiment 2 showed that extinction in multiple contexts prevented ABC renewal but had no detectable impact on ABA renewal. Our results are discussed within the framework of contemporary learning theories of contextual control and extinction.

Three Ways That Non-associative Knowledge May Affect Associative Learning Processes.

Associative learning theories offer one account of the way animals and humans assess the relationship between events and adapt their behavior according to resulting expectations. They assume knowledge about event relations is represented in associative networks, which consist of mental representations of cues and outcomes and the associative links that connect them. However, in human causal and contingency learning, many researchers have found that variance in standard learning effects is controlled by "non-associative" factors that are not easily captured by associative models. This has given rise to accounts of learning based on higher-order cognitive processes, some of which reject altogether the notion that humans learn in the manner described by associative networks. Despite the renewed focus on this debate in recent years, few efforts have been made to consider how the operations of associative networks and other cognitive operations could potentially interact in the course of learning. This paper thus explores possible ways in which non-associative knowledge may affect associative learning processes: (1) via changes to stimulus representations, (2) via changes to the translation of the associative expectation into behavior (3) via a shared source of expectation of the outcome that is sensitive to both the strength of associative retrieval and evaluation from non-associative influences.

Expectancies as core features of mental disorders.

PURPOSE OF REVIEW: Expectancies are core features of mental disorders, and change in expectations is therefore one of the core mechanisms of treatment in psychiatry. We aim to improve our understanding of expectancies by summarizing factors that contribute to their development, persistence, and modification. We pay particular attention to the issue of persistence of expectancies despite experiences that contradict them. RECENT FINDINGS: Based on recent research findings, we propose a new model for expectation persistence and expectation change. When expectations are established, effects are evident in neural and other biological systems, for example, via anticipatory reactions, different biological reactions to expected versus unexpected stimuli, etc. Psychological 'immunization' and 'assimilation', implicit self-confirming processes, and stability of biological processes help us to better understand why expectancies persist even in the presence of expectation violations. SUMMARY: Learning theory, attentional processes, social influences, and biological determinants contribute to the development, persistence, and modification of expectancies. Psychological interventions should focus on optimizing expectation violation to achieve optimal treatment outcome and to avoid treatment failures.

Indicators of early and late processing reveal the importance of within-trial-time for theories of associative learning.

In four human learning experiments (Pavlovian skin conductance, causal learning, speeded classification task), we evaluated several associative learning theories that assume either an elemental (modified unique cue model and Harris' model) or a configural (Pearce's configural theory and an extension of it) form of stimulus processing. The experiments used two modified patterning problems (A/B/C+, AB/BC/AC+ vs. ABC-; A+, BC+ vs. ABC-). Pearce's configural theory successfully predicted all of our data reflecting early stimulus processing, while the predictions of the elemental theories were in accord with all of our data reflecting later stages of stimulus processing. Our results suggest that the form of stimulus representation depends on the amount of time available for stimulus processing. Our findings highlight the necessity to investigate stimulus processing during conditioning on a finer time scale than usually done in contemporary research.

Normalization between stimulus elements in a model of Pavlovian conditioning: showjumping on an elemental horse.

Harris and Livesey. Learning & Behavior, 38, 1-26, (2010) described an elemental model of associative learning that implements a simple learning rule that produces results equivalent to those proposed by Rescorla and Wagner (1972), and additionally modifies in "real time" the strength of the associative connections between elements. The novel feature of this model is that stimulus elements interact by suppressively normalizing one another's activation. Because of the normalization process, element activity is a nonlinear function of sensory input strength, and the shape of the function changes depending on the number and saliences of all stimuli that are present. The model can solve a range of complex discriminations and account for related empirical findings that have been taken as evidence for configural learning processes. Here we evaluate the model's performance against the host of conditioning phenomena that are outlined in the companion article, and we present a freely available computer program for use by other researchers to simulate the model's behavior in a variety of conditioning paradigms.

Comparing positive and negative patterning in human learning.

In human causal learning, positive patterning (PP) and negative patterning (NP) discriminations are often acquired at roughly the same rate, whereas PP is learned faster than NP in most experiments with nonhuman animals. One likely reason for this discrepancy is that most causal learning scenarios encourage participants to treat the presentation and omission of the relevant outcome as two events of comparable significance and likelihood. To investigate this, the current experiments compared PP and NP using a predictive learning paradigm based on a mock gambling task. In Experiment 1, one outcome (winning) was made more salient by being less frequent than the alternative outcome (losing). Under these circumstances, PP was learned faster than NP. In Experiment 2, subjects learned two PP and two NP discriminations, one involved win versus no change outcomes, the other involved lose versus no change outcomes. The subjects learned PP faster than NP, but only when discriminating win from no change. We argue that a difference in difficulty between PP and NP relies on a difference in the salience of the outcomes, consistent with the predictions of a relatively simple model of associative learning.

Comparing learned predictiveness effects within and across compound discriminations.

In four human learning experiments, we examined the extent to which learned predictiveness depends upon direct comparison between relatively good and poor predictors. Participants initially solved (a) linear compound discriminations in which one or both of the stimuli in each compound were predictive of the correct outcome, (b) biconditional discriminations where only the configurations of the stimuli were predictive of the correct outcome, or (c) pseudodiscriminations in which no stimulus features were predictive. In each experiment, subsequent learning and test stages were used to assay changes in the associability of each stimulus brought about by its role in the initial discriminations. Although learned predictiveness effects were observed in all experiments (i.e., previously predictive cues were more readily associated with a new outcome than previously nonpredictive cues), the same changes in associability were observed regardless of whether the stimulus was initially learned about in the presence of an equally predictive, more predictive, or less predictive stimulus. The results suggest that learned associability is not controlled by competitive allocation of attention, but rather by the absolute predictiveness of each individual cue.

Generalization decrements: further support for flexibility in stimulus processing.

Thorwart and Lachnit (2009) found reliable symmetrical decrements in two generalization tasks: Removing an already trained component from a compound did not result in larger decrements than adding a new one did. In two contingency learning experiments, we investigated first whether time pressure during stimulus processing, as well as the degree of perceptual grouping, was effective in controlling the symmetry of the decrements (Experiment 1); and second, whether the symmetry was affected by the causal versus predictive nature of the relationship between the cue and the outcome (Experiment 2). The experiments generated unexpected results, since both revealed asymmetrical decrements independent of the manipulations introduced. They therefore demonstrate that more research is needed in order to understand the variables influencing stimulus representation in human associative learning.