Serial pattern learning: The anticipation of worsening conditions by pigeons.

In general, animals are known to be sensitive to the immediacy of reinforcers. That is, they are generally impulsive and outcomes that occur in the future are generally heavily discounted. Furthermore, they should prefer alternatives that provide reinforcers that require less rather than greater effort to obtain. In the present research, pigeons were given a choice between (1) obtaining reinforcers on a progressively more difficult schedule of reinforcement; starting with four pecks, then eight pecks, then 16 pecks, then 32 pecks, and finally 64 pecks on each trial, and (2) a color signaling a number of pecks for a single reinforcer: red = six, green = 11, blue = 23, or yellow = 45. If pigeons choose optimally, most of the time they should choose the progressive schedule to obtain five reinforcers rather than switch to a color to receive only one. However, if they are sensitive primarily to the number of pecks to the next reinforcer, they should choose the progressive schedule once before switching to red, twice before switching to green, three times before switching to blue, and four times before switching to yellow. Instead, they systematically switched too early. Rather than choose based on the rate of reinforcement or even based on the time or effort to the next reinforcer, they appear to anticipate that the progressive schedule is going to get more difficult, and they base their choice suboptimally on the serial pattern of the worsening progressive schedule.

How language and agriculture promote culture- and peace-promoting norms.

Humans are predisposed to form in-groups and out-groups that are remarkably flexible in their definition due largely to the complex language that has evolved in them. Language has allowed for the creation of shared "background stories" that can unite people who do not know each other. Second, the discovery of agriculture has resulted in the critical need to negotiate boundaries, a process that can lead to peace (but also war).

"Distractor" effects in delay discounting of probability by pigeons.

Impulsive behavior can be measured by performance on a successive delay-discounting task, in which a response to a stimulus provides a small reinforcer sooner (SS), but in the absence of a response, a larger reinforcer later (LL). Previous research suggests that the presence of a concurrent "distractor" stimulus, to which responding has no programed consequence, can result in increased LL reinforcers. In the present experiments, we used differences in the probability of reinforcement between SS and LL (rather than magnitude of reinforcement) and tested the hypothesis that the concurrent stimulus may become a Pavlovian conditioned stimulus. For the Red-Only group, a response to the SS stimulus resulted in a reinforcer with a low probability (SS), whereas the absence of a response resulted in a reinforcer with a high probability (LL). For the Red-Green group, (analogous to the more typical simultaneous choice between an SS and LL stimulus) the absence of a response to the SS stimulus replaced the SS stimulus with the LL stimulus and a response to the LL stimulus resulted in the reinforcer. Thus, for the Red-Green group, the concurrent stimulus should have been less effective because responding to the concurrent stimulus was not immediately followed by the reinforcer. In Experiment 1, the concurrent stimulus was a yellow key-light; in Experiment 2, it was a houselight. In both experiments, the concurrent stimulus was effective in increasing the number of LL reinforcers and the effect was larger for the Red-Only group than for the Red-Green group.

Memory for where and when: pigeons use single-code/default strategy.

Memory for what, where, and when an event took place has been interpreted as playing a critical role in episodic memory. Moreover, such memory is likely to be important to an animal's ability to efficiently forage for food. In Experiment 1 of the present study, pigeons were trained on a task in which on each trial, one lit stimulus color and location was presented and then another. A cue presented after the last stimulus location signaled that the pigeon was to choose either the first location presented, or the last location presented, to receive a reinforcer. After learning this task, in Experiment 2, the color cue was removed, requiring the pigeons to choose based on location and order alone. In Experiment 3, when a delay was inserted between presentation of the two locations, it had little effect on task accuracy. Results suggested that the pigeons had acquired the task using a single-code/default rule. When presented with the cue indicating that the last location was correct, pigeons selected the location just presented. When presented with the cue indicating that the first location was correct, pigeons chose the other location, by default. In support of this hypothesis, in Experiment 4, when a delay was inserted, prior to receiving the instructional cue, it had a disruptive effect on task accuracy proportional to the delay. Although the present results do not provide evidence for episodic memory, they do suggest that the pigeons have developed a single-code/default strategy that appears to be an efficient means of performing this task.

Pigeons learn two matching tasks, two nonmatching tasks, or one of each.

When pigeons learn matching-to-sample or nonmatching-to-sample there is good evidence that they can transfer that learning to novel stimuli. But early evidence suggests that in the rate of task acquisition, there is no benefit from a matching relation between the sample and the correct or incorrect comparison stimulus. In the present research we trained three groups of pigeons, each on two two-stimulus tasks simultaneously, matching-matching, nonmatching-nonmatching, or matching-nonmatching. If a common matching or nonmatching relationship benefits acquisition, the first two groups should acquire their tasks faster than the third group, for which the two tasks ought to be incompatible. The results indicated that all three groups acquired their tasks at about the same rate. A secondary goal of the experiment was to determine the basis of learning for the each of the three groups. During testing, for each task, there were test trials in which one of the stimuli from the other task replaced either the correct or the incorrect comparison stimulus. Surprisingly, neither comparison stimulus appeared to show complete control over comparison choice. Although replacing either comparison stimulus resulted in a decrement in task accuracy from about 90% to 70% correct, independent of which comparison stimulus was replaced, the pigeons chose correctly at well above chance accuracy. Suggestions to explain this unexpected outcome are discussed.

1-Back reinforcement symbolic-matching by humans: How do they learn it?

For humans, a distinction has been made between implicit and explicit learning. Implicit learning is thought to involve automatic processes of the kind involved in much Pavlovian conditioning, while explicit learning is thought to involve conscious hypothesis testing and rule formation, in which the subject's statement of the rule has been taken as evidence of explicit learning. Various methods have been used to determine if nonverbal animals are able to learn a task explicitly - among these is the 1-back reinforcement task in which feedback from performance on the current conditional discrimination trial is provided only after completion of the following trial. We propose that it is not whether an organism can learn the task, but whether they learn it rapidly, all-or-none, that provides a better distinction between the two kinds of learning. We had humans learn a symbolic matching, 1-back reinforcement task. Almost half of the subjects failed to learn the task, and of those who did, none described the 1-back rule. Thus, it is possible to learn this task without learning the 1-back rule. Furthermore, the backward learning functions for humans differ from those of pigeons. Human subjects who learned the task did so all-or-none, suggesting explicit learning. In earlier research with pigeons, they too showed significant learning of this task; however, backward learning functions suggested that they did so gradually over the course of several sessions of training and to a lower level of asymptotic accuracy than the humans, a result suggesting implicit learning was involved.

Decision making under risk: framing effects in pigeon risk preferences.

When humans face probabilistic outcomes, their choices often depend on whether the choice is framed in terms of losses or gains. In the present research, we gave pigeons a choice between risky (variable) outcomes and safe (constant) outcomes that resulted in the same net reward. In Experiment 1, in which the outcomes represented a loss, the pigeons preferred the risky alternative. In Experiment 2, in which the outcomes represented a gain, the pigeons were indifferent between the two alternatives. In Experiment 3, in which the outcomes represented neither a gain nor a loss, the pigeons strongly preferred the risky alternative. The results were interpreted in terms of the relative value of gains and losses given to the alternatives by pigeons in the context of a risky and safe choice. In Experiment 4 we tested that hypothesis by giving pigeons a choice between a risky and safe alternative with the same net outcome, in the context of a gain associated with the safe alternative, but no gain or loss associated with the risky alternative. In support of the interpretation of the first three experiments, with the safe alternative associated with a gain, the pigeons now preferred the safe alternative. These results were discussed in terms of economic and foraging theories and were contrasted with the aversion to uncertainty (risk) more typically shown by humans.

Pavlovian processes may produce contrast leading to bias and suboptimal choice.

Pavlovian processes are likely responsible for the varied contexts in which contrast occurs between what is expected and what is obtained. Such contrast effects result in paradoxical biases and even suboptimal choice by animals. For example, pigeons prefer a suboptimal alternative that results in a stimulus signaling a low probability (20%) high reward (ten pellets) over an optimal alternative that results in a stimulus signaling a high probability (100%) of a smaller reward (three pellets). This effect is analogous to human unskilled gambling. In another case, pigeons prefer a stimulus that has required many pecks to obtain over one the has required one peck to obtain (a so-called justification of effort effect). In a third line of research that investigated the preference for risky choice over safe choice, pigeons chose between two alternatives, a safe choice that resulted in two pellets of food, or a risky choice that resulted in either one or three pellets of food. In general, the pigeons preferred the risky alternative, but importantly, their choice was influenced by whether the choice reflected a gain or a loss - the difference between what was shown to them (one, two, or three pellets) and what they received. Each of these lines of research suggest the importance of contrast effects produced by Pavlovian processes that result in biases or suboptimal behavior.

Is there a need to distinguish instrumental copying behavior from traditions?

The authors make a distinction between instrumental copying behavior in which there is a clear reward for the copying behavior and social copying (traditions) in which the rewards for copying are less clear. However, I see no reason to distinguish between the two. We are social animals, for whom copying traditions have important rewards, those of affiliation.

Biases and suboptimal choice by animals suggest that framing effects may be ubiquitous.

Framing effects attributed to "quasi-cyclical" irrational complex human preferences are ubiquitous biases resulting from simpler mechanisms that can be found in other animals. Examples of such framing effects vary from simple learning contexts, to an analog of human gambling behavior, to the value added to a reinforcer by the effort that went into obtaining it.

Pigeons are attracted to a perceived gain without an actual gain.

Reference dependence refers to the reduced value of a reward that is less than expected, or the added value of a reward that is greater than expected. There is evidence that when pigeons are offered an alternative that has 1 pellet versus an alternative that has 2 pellets, but one of the two pellets offered will be removed, the pigeons prefer the originally presented 1 pellet (loss aversion). In the present research, we tested for the opposite effect (gain attraction). In Experiments 1 and 2, pigeons could choose between 2 pellets, each one on a distinctive background. If they chose the optimal alternative, they received a second pellet. In Experiment 2, the second pellet obtained was the one not initially chosen (a task sometimes referred to as the ephemeral reward task). Pigeons learned to choose optimally in both experiments. In Experiment 3, we tested the pigeons for reference dependence. Pigeons were given an alternative that offered them one pellet or two pellets, if they chose the one-pellet alternative, they received an additional pellet, and if they chose the two-pellet alternative, they received the two pellets. In keeping with the reference dependence hypothesis, the pigeons preferred the 1-pellet alternative that gave them an extra pellet. These effects are related to similar findings with humans, including the endowment effect.

The paradoxical performance by different species on the ephemeral reward task.

The ephemeral reward task consists of giving an animal a choice between two distinctive stimuli, A and B (e.g., black and white), on each of which is placed a bit of food. If the animal chooses the food on A, it gets that reinforcer, but the other stimulus, B, is removed, and the trial is over. If it chooses the food on B, however, it gets that food and the stimulus A remains, so it can have that food as well. Thus, choice of stimulus B gives the animal two reinforcers rather than one. Wrasse (cleaner fish) easily learn to choose optimally, whereas surprisingly, most non-human primates do not. Parrots, however, appear to learn this task as easily as the fish. To test the hypothesis that animals that choose with their mouth can learn it, we tested pigeons and found that they show no evidence of optimal learning with this task (with either the manual presentation of the stimuli or the operant presentation of the stimuli). Similarly, rats show no evidence of optimal learning. However, if a 20-s delay (fixed-interval schedule) is inserted between stimulus choice and reinforcement, both pigeons and rats learn to perform optimally. The ephemeral reward task appears to promote impulsive choice in several species, but with this task (and others), inserting a delay between choice and reinforcement promotes more careful choice, leading to optimal performance.

Pigeons' midsession reversal: Greater magnitude of reinforcement on the first half of the session leads to improved accuracy.

In the midsession reversal task, pigeons are trained on a simultaneous two-alternative discrimination in which S1 is correct for the first half of the session and S2 is correct for the second half of the session. Optimally, pigeons should choose S1 until it stops being correct and choose S2 afterward. Instead, pigeons anticipate S2 too early and continue choosing S1 even after the reversal. Research suggests that they attempt to time the reversal rather than use the feedback from the preceding response(s). Recently, there is evidence that performance is almost optimized by generating an asymmetry between S1 and S2. For example, pigeons' accuracy improves if correct S1 responses are reinforced 100% of the time, but correct S2 responses are reinforced only 20% of the time. Similarly, accuracy improves if S1 requires one peck but S2 requires 10 pecks. Accuracy does not improve, however, if the value of S1 is less than the value of S2. In the current experiment, we manipulated the magnitude of reinforcement. For the experimental group, correct responses to S1 were reinforced with five pellets of food and correct responses to S2 were reinforced with one pellet. For the control group, all correct responses were reinforced with three pellets. Consistent with the earlier findings, results indicated that there was a significant reduction in anticipatory errors in the experimental group compared with the control, and there was no significant increase in perseverative errors.

Pigeons acquire the 1-back task: Implications for implicit versus explicit learning?

In humans, a distinction can be made between implicit or procedural learning (involving stimulus-response associations) and explicit or declarative learning (involving verbalizable rules) that is relatively easy to make in verbal humans. According to several investigators, it is also possible to make such a distinction in nonverbal animals. One way is by training them on a conditional discrimination task (e.g., matching-to-sample) in which reinforcement for correct choice on the current trial is delayed until after a choice is made on the next trial - a method known as the 1-back procedure. According to Smith, Jackson, and Church ( Journal of Comparative Psychology, 134(4), 423-434, 2020), the delay between the sample-correct-comparison response on one trial and reinforcement obtained on the next trial is too long for implicit (associative) learning. Thus, according to this theory, learning must be explicit. In the present experiments we trained pigeons using the 1-back procedure. In Experiment 1, pigeons were trained on red/green 1-back matching using a non-correction procedure. Some of the pigeons showed significant learning. When a correction procedure was introduced, all the pigeons showed evidence of learning. In Experiment 2, new pigeons learned red/green 1-back matching with the correction procedure. In Experiment 3, new pigeons learned symbolic 1-back matching with yellow and blue conditional stimuli and red/green choice stimuli. Thus, pigeons can learn using 1-back reinforcement. Although it would appear that the pigeons acquired this task explicitly, we believe that this procedure does not adequately distinguish between implicit and explicit learning.

Visual alternation by pigeons: Learning to select or learning to avoid.

In the visual alternation task, pigeons learn to alternate between two stimuli (e.g., red and green) that vary randomly in location from trial to trial. The task is inherently difficult because animals tend to return to a stimulus to which they have just received reinforcement for responding. Williams (1971, Journal of the Experimental Analysis of Behavior, 15, 129-140) suggested that pigeons learn this task by learning to avoid the stimulus most recently chosen. The present experiment tested this hypothesis by involving three groups. The Standard Group replicated Williams' design. For the New Correct Group, following a correct (reinforced) response, on the next trial, the color of the new correct stimulus changed. For example, if it had been green, it changed to either blue or yellow, but the color of the new incorrect stimulus (the one that was just correct) remained the same (i.e., red). For the New Incorrect Group, following a correct response, on the next trial, the color of the new incorrect stimulus changed. For example, if it had been red, it changed to blue or yellow, but the color of the new correct stimulus remained (i.e., green). The Standard Group replicated Williams's finding that pigeons can learn the alternation task. Consistent with Williams's hypothesis, pigeons in the New Correct Group showed evidence of learning the alternation task, whereas pigeons in the New Incorrect Group showed little evidence of learning. Acquisition of the visual alternation task suggests that pigeons are cognitively flexible enough to overcome their natural tendency to repeat their most recently reinforced response to a stimulus.

Pigeons can learn a difficult discrimination if reinforcement is delayed following choice.

Delaying reinforcement typically has been thought to retard the rate of acquisition of an association, but there is evidence that it may facilitate acquisition of some difficult simultaneous discriminations. After describing several cases in which delaying reinforcement can facilitate acquisition, we suggest that under conditions in which the magnitude of reinforcement is difficult to discriminate, the introduction of a delay between choice and reinforcement can facilitate the discrimination. In the present experiment, we tested the hypothesis that the discrimination between one pellet of food for choice of one alternative and two pellets of food for choice of another may be a difficult discrimination when choice consists of a single peck. If a 10-s delay occurs between choice and reinforcement, however, the discrimination is significantly easier. It is suggested that when discrimination between the outcomes of a choice is difficult and impulsive choice leads to immediate reinforcement, acquisition may be retarded. Under these conditions, the introduction of a brief delay may facilitate acquisition.

Enhancing "self-control": The paradoxical effect of delay of reinforcement.

Delay of reinforcement is generally thought to be inversely correlated with speed of acquisition. However, in the case of simultaneous discrimination learning, in which choice results in immediate reinforcement, delay of reinforcement can improve acquisition. For example, in the ephemeral reward task, animals are given a choice between two alternatives, A and B. Choice of A provides reinforcement, and the trial is over. Choice of B provides reinforcement and access to alternative A (thus, two reinforcements). Many animals appear unable to learn to choose B consistently, but inserting a 20-s delay between choice and outcome has been shown to facilitate optimal choice. Similarly, pigeons given a choice between a signal for one pellet and a signal for two pellets (each occurring without a delay) have difficulty learning to choose the two-pellet alternative, unless the reinforcement is delayed. In a version of object permanence, food is placed in one of two containers, and the pigeon must choose the container with the food. Pigeons have difficulty reliably choosing the correct container unless a brief delay is inserted between baiting and choice. Finally, pigeons have been shown to prefer a suboptimal alternative (a 20% chance of getting a cue for reinforcement) over an optimal alternative (a 100% chance of getting a cue for 50% reinforcement). However, if pigeons are forced to wait 20 s following their choice to receive the cues, no preference for the suboptimal alternative is found. Thus, impulsive choice may be reduced by delaying the consequence of that choice.

The midsession reversal task: A theoretical analysis.

In the midsession reversal task, choice of one stimulus (S1) is correct for the first half of each session and choice of the other stimulus (S2) is correct for the last half of each session. Although humans and rats develop very close to what has been called a win-stay/lose-shift response strategy, pigeons do not. Pigeons start choosing S2 before the reversal, making anticipatory errors, and they keep choosing S1 after the reversal, making perseverative errors. Research suggests that the pigeons are timing the reversal from the start of the session. However, making the reversal unpredictable does not discourage the pigeons from timing. Curiously, pigeons' accuracy improves if one decreases the value of the S2 stimulus relative to the S1 stimulus. Another form of asymmetry between S1 and S2 can be found by varying, over trials, the number of S1 or S2 stimuli. Counterintuitively, if the number of S2 stimuli varies, it results in a large increase in anticipatory errors but little increase in perseverative errors. However, if the number of S1 stimuli varies over trials, it results in a large increase in perseverative errors but no increase in anticipatory errors. These last two effects suggest that in the original midsession reversal task, the pigeons are learning to reject S2 during the first half of each session and learning to reject S1 during the last half of each session. These results suggest that reject learning may also play an important role in the learning of simple simultaneous discriminations.

Animal procrastination: Pigeons choose to defer experiencing an aversive gap or a peck requirement.

When humans procrastinate, they delay completing a required relatively aversive task. In the present experiments with pigeons, we considered the possibility that completing the task close to the deadline results in the formation of a stronger conditioned reinforcer. In Experiment 1, pigeons were given a choice between two chains: (a) a signaled long period, followed by a dark gap, followed by a signaled short conditioned reinforcer, and food and (b) a signaled short period, followed by a dark gap, followed by a signaled long conditioned reinforcer, and food. We found a reliable preference for the delayed gap. In Experiment 2, we let pigeons choose between two chains: (a) walking to a near panel to peck a key, followed by a long walk to peck a key for reinforcement and (b) walking to a far panel to peck a key followed by a short walk to peck a key for reinforcement. When a single peck was required to either key, the pigeons were indifferent. When ten pecks were required to the near key but only one peck to the far key, the pigeons preferred the far key. When ten pecks were required to either key, the pigeons preferred the far key. The results of both experiments suggest that pigeons prefer to defer a relatively aversive event but, in keeping with Fantino's Delay Reduction Theory, this effect may result from the development of a strong conditioner reinforcer that occurs when the event (the gap or required pecking) comes close to reinforcement.

Transitive inference in pigeons may result from differential tendencies to reject the test stimuli acquired during training.

In the five-term, transitive inference task used with animals, pigeons are trained on four simultaneous discrimination premise pairs: A + B -, B + C -, C + D -, D + E -. Typically, when tested with the BD pair, most pigeons show a transitive inference effect, choosing B over D. Two non-inferential hypotheses have been proposed to account for this effect but neither has been reliably supported by research. Here we test a third non-inferential hypothesis that the preference for B arises because the animals have not had as much experience with B - in the A + B - discrimination as they have had with the D - in the C + D - discrimination. To test this hypothesis we trained the Experimental Group with the A + B - discrimination in which, over trials, there were four possible A + stimuli that could appear. This was done to encourage the pigeons to learn to reject the B - stimulus. For the Control Group there was only one A + stimulus over trials, as is typically the case. We also varied the nature of the stimuli between groups, such that colors served as the stimuli for half of the pigeons, whereas flags of different counties served as stimuli for the remaining pigeons. In both stimulus conditions, for the Experiment Group, we found little preference for stimulus B over stimulus D, whereas for the Control Group we found the typical preference for stimulus B. Thus, we propose that it is not necessary to attribute the transitive inference effect to an inferential process.