Mechanisms of within-session sequential behavior in pigeons.

Correctly and efficiently selecting among options is critical to the organization of behavior across different time scales (minutes, days, seasons). As a result, understanding the mechanisms underlying the sequential behavior of animals has been a long-standing aim. In three experiments, four pigeons were tested in a four-choice simultaneous color discrimination. Across a session, they had to sequentially select a colored stimulus, and the correct color changed over four 24-trial phases (A→B→C→D). After learning this ABCD within-session sequence, tests identified that both timing and outcome feedback mechanisms contributed to the organization of pigeons' behavior. Different representational mechanisms are considered as accounts for the pigeons' observed sequential behavior.

Within-session reversal learning in rhesus macaques (Macaca mulatta).

In a midsession reversal (MSR) task, animals are typically presented with a simple, simultaneous discrimination (S1+, S2-) where contingencies are reversed (S1-, S2+) half-way through each session. This paradigm creates multiple, relevant cues that can aid in maximizing overall reinforcement. Recent research has shown that pigeons show systematic anticipatory and perseverative errors across the session, which increase as a function of proximity to the reversal trial. This behavior has been theorized to indicate primary control by temporal cues across the session, instead of the cues provided by recent reinforcement history that appear to control behavior shown by humans. Rats, however, appear to be guided by recent reinforcement history when tested in an operant context, thereby demonstrating behavior that parallels that seen in humans, but they appear to be guided by temporal cues when tested in an open-field apparatus, showing behavior more akin to that seen in pigeons. We tested rhesus macaques (Macaca mulatta) on the MSR with a computerized simultaneous visual discrimination to assess whether they would show errors indicative of control by time or by recent reinforcement history. When a single reversal point occurred midsession, rhesus macaques showed no anticipation of the reversal and a similar level of perseveration to rats tested in an operant setting. Nearly identical results also were observed when the monkeys were trained with a single, variable reversal point or with multiple, variable reversal points within a session. These results indicate that temporal cues are not guiding response flexibility in rhesus macaque visual discrimination.

Who are the real bird brains? Qualitative differences in behavioral flexibility between dogs (Canis familiaris) and pigeons (Columba livia).

Pigeons given a simultaneous spatial discrimination reversal, in which a single reversal occurs at the midpoint of each session, consistently show anticipation prior to the reversal as well as perseveration after the reversal, suggesting that they use a less effective cue (time or trial number into the session) than what would be optimal to maximize reinforcement (local feedback from the most recent trials). In contrast, rats (Rattus norvegicus) and humans show near-optimal reversal learning on this task. To determine whether this is a general characteristic of mammals, in the present research, pigeons (Columba livia) and dogs (Canis familiaris) were tested with a simultaneous spatial discrimination mid-session reversal. Overall, dogs performed the task more poorly than pigeons. Interestingly, both pigeons and dogs employed what resembled a timing strategy. However, dogs showed greater perseverative errors, suggesting that they may have relatively poorer working memory and inhibitory control with this task. The greater efficiency shown by pigeons with this task suggests they are better able to time and use the feedback from their preceding choice as the basis of their future choice, highlighting what may be a qualitative difference between the species.

Pigeons' use of cues in a repeated five-trial-sequence, single-reversal task.

We studied behavioral flexibility, or the ability to modify one's behavior in accordance with the changing environment, in pigeons using a reversal-learning paradigm. In two experiments, each session consisted of a series of five-trial sequences involving a simple simultaneous color discrimination in which a reversal could occur during each sequence. The ideal strategy would be to start each sequence with a choice of S1 (the first correct stimulus) until it was no longer correct, and then to switch to S2 (the second correct stimulus), thus utilizing cues provided by local reinforcement (feedback from the preceding trial). In both experiments, subjects showed little evidence of using local reinforcement cues, but instead used the mean probabilities of reinforcement for S1 and S2 on each trial within each sequence. That is, subjects showed remarkably similar behavior, regardless of where (or, in Exp. 2, whether) a reversal occurred during a given sequence. Therefore, subjects appeared to be relatively insensitive to the consequences of responses (local feedback) and were not able to maximize reinforcement. The fact that pigeons did not use the more optimal feedback afforded by recent reinforcement contingencies to maximize their reinforcement has implications for their use of flexible response strategies under reversal-learning conditions.

Midsession reversal learning: why do pigeons anticipate and perseverate?

Past research has shown that when given a simultaneous visual-discrimination midsession reversal task, pigeons typically anticipate the reversal well before it occurs and perseverate after it occurs. It appears that they use the estimation of time (or trial number) into the session, rather than (or in addition to) the more reliable cue, the outcome from the previous trial (i.e., a win-stay/lose-shift response rule), to determine which stimulus they should choose. In the present research, we investigated several variables that we thought might encourage pigeons to use a more efficient response strategy. In Experiment 1, we used a treadle-stepping response, rather than key pecking, to test the hypothesis that reflexive key pecking may have biased pigeons to estimate the time (or trial number) into the session at which the reversal would occur. In Experiment 2, we attempted to make the point of reversal in the session more salient by inserting irrelevant trials with stimuli different from the original discriminative stimuli, and for a separate group, we added a 5-s time-out penalty following incorrect choices. The use of a treadle-stepping response did not improve reversal performance, and although we found some improvement in reversal performance when the reversal was signaled and when errors resulted in a time-out, we found little evidence for performance that approached the win-stay/lose-shift accuracy shown by rats.

Simultaneous discrimination reversal learning in pigeons and humans: anticipatory and perseverative errors.

Pigeons were trained on a two-choice simultaneous discrimination (red vs. green) that reversed midway through each session. After considerable training, they consistently made both anticipatory errors prior to the reversal and perseverative errors after the reversal, suggesting that time (or number of trials) into the session served as a cue for reversal. In Experiment 2, to discourage the use of time as a cue, we varied the location of the reversal point within the session such that it occurred semirandomly after Trial 10, 25, 40, 55, or 70. Pigeons still tended both to anticipate and to perseverate. In Experiment 3, we required 20 pecks to a stimulus on each trial to facilitate memory for the preceding response and sensitivity to local reinforcement contingencies, but the results were similar to those of Experiment 2. We then tested humans on a similar task with a constant (Experiment 4) or variable (Experiment 5) reversal location. When the reversal occurred consistently at the midpoint of the session, humans, like pigeons, showed a tendency to anticipate the reversal; however, they did not show perseverative errors. When the reversal location varied between sessions, unlike pigeons, humans adopted a win-stay/lose-shift strategy, making only a single error on the first trial of the reversal.

Within-session dynamics of categorical and memory mechanisms in pigeons.

The current experiments used categorical mid-session reversal (MSR) to examine how eight pigeons utilized categorical and item-specific mechanisms to learn and solve a novel variation of this task. Employing a fixed order of trial-unique pictorial items from two categories (flowers and cars) on each simultaneous discrimination trial, categorical and item-specific information was available during each session's 80 trials. Choices to one category were rewarded for the first 40 trials, after which the correct category was reversed (e.g., car correct early → flower correct late). This procedure selectively impacts the time-modulated utility of categorical identification, but leaves exclusively item-specific information intact. Results revealed that categorical control emerged rapidly and before item-specific memorization, which came after extended experience. Both types of control occurred within a session, with control modulated by their time-based relative utility. The implications for the timing, ordering, and attention by animals to categorical and item-specific information is considered.

Spatial midsession reversal learning in rats: Effects of egocentric Cue use and memory.

The midsession reversal task has been used to investigate behavioral flexibility and cue use in non-human animals, with results indicating differences in the degree of control by environmental cues across species. For example, time-based control has been found in rats only when tested in a T-maze apparatus and under specific conditions in which position and orientation (i.e., egocentric) cues during the intertrial interval could not be used to aid performance. Other research in an operant setting has shown that rats often produce minimal errors around the reversal location, demonstrating response patterns similar to patterns exhibited by humans and primates in this task. The current study aimed to reduce, but not eliminate, the ability for rats to utilize egocentric cues by placing the response levers on the opposite wall of the chamber in relation to the pellet dispenser. Results showed that rats made minimal errors prior to the reversal, suggesting time-based cues were not controlling responses, and that they switched to the second correct stimulus within a few trials after the reversal event. Video recordings also revealed highly structured patterns of behavior by the majority of rats, which often differed depending on which response was reinforced. We interpret these findings as evidence that rats are adept at utilizing their own egocentric cues and that these cues, along with memory for the recent response-reinforcement contingencies, aid in maximizing reinforcement over the session.

Dynamic cue use in pigeon mid-session reversal.

The systematic anticipation and preservation errors produced by pigeons around the reversal point in midsession reversal (MSR) learning experiments suggest that an internal time estimation cue, instead of a more efficient external cue provided by reinforcement, controls behavior over the course of a session. The current experiments examined the role and effectiveness of other external cues in the MSR task. In Experiment 1, providing differential outcomes based on response key location produced fewer errors prior to, but not after, the reversal as compared with a non-differential outcomes condition. Experiment 2a used alternating differentially colored ITIs (cued sessions) or dark ITIs (un-cued sessions) during each half of the session. The ITI cues improved switch efficiency both prior to and after the reversal. Experiment 2b introduced probe trials around the reversal, testing ITI color cues added to un-cued sessions or removed from cued sessions. Results showed control by the ITI cues when they were available and control by the time-based cue when they were unavailable. This suggests both cues were being simultaneously processed when available and that the cues could also independently provide sufficient information about future reinforcement. In Experiment 2c, ITI cues were inserted as probe trials in the opposite half of the session (miscues). The closer such miscue trials were to the reversal, the more the ITI cues exerted control over behavior. Together, these results indicate that as the utility of internal temporal cues is reduced, the use of external visual cues increases. These results have implications for the way in which cues dynamically shift in controlling behavior over time based on their relative rates of utility, and are discussed in light of an occasion setting perspective.

The Organization of Behavior Over Time: Insights from Mid-Session Reversal.

What are the mechanisms by which behavior is organized sequentially over time? The recently developed mid-session reversal (MSR) task offers new insights into this fundamental question. The typical MSR task is arranged to have a single reversed discrimination occurring in a consistent location within each session and across sessions. In this task, we examine the relevance of time, reinforcement, and other factors as the switching cue in the sequential modulation of control in MSR. New analyses also highlight some of the potential mechanisms underlying this serially organized behavior. MSR provides new evidence and we offer some ideas about how cues interact to compete for the control of behavior within and across sessions. We suggest that MSR is an excellent preparation for studying the competition among psychological states and their resolution toward action.

Pigeons show near-optimal win-stay/lose-shift performance on a simultaneous-discrimination, midsession reversal task with short intertrial intervals.

Discrimination reversal tasks have been used as a measure of species flexibility in dealing with changes in reinforcement contingency. The simultaneous-discrimination, midsession reversal task is one in which one stimulus (S1) is correct for the first 40 trials of an 80-trial session and the other stimulus (S2) is correct for the remaining trials. After many sessions of training with this task, pigeons show a curious pattern of choices. They begin to respond to S2 well before the reversal point (they make anticipatory errors) and they continue to respond to S1 well after the reversal (they make perseverative errors). That is, they appear to be using the passage of time or number of trials into the session as a cue to reverse. We tested the hypothesis that these errors resulted in part from a memory deficit (the inability to remember over the intertrial interval, ITI, both the choice on the preceding trial and the outcome of that choice) by manipulating the duration of the ITI (1.5, 5, and 10 s). We found support for the hypothesis as pigeons with a short 1.5-s ITI showed close to optimal win-stay/lose-shift accuracy.

Reversal learning in rats (Rattus norvegicus) and pigeons (Columba livia): qualitative differences in behavioral flexibility.

Research has shown that pigeons given a simultaneous visually based discrimination reversal, in which a single reversal occurs at the midpoint of each session, consistently show anticipation prior to the reversal as well as perseveration after the reversal, suggesting that they use a less effective cue (time or trial number into the session) than what would be optimal to maximize reinforcement (local feedback from the most recent trials). In the present research, pigeons (Columba livia) and rats (Rattus norvegicus) were tested with a simultaneous spatial discrimination midsession reversal. Pigeons showed remarkably similar errors in anticipation and perseveration as with visual stimuli, thereby continuing to show the suboptimal use of time as a cue, whereas rats showed no anticipatory errors and very few perseverative errors, suggesting that they used local feedback as a cue, thus more nearly optimizing reinforcement. To further test the rats' flexibility, they were then tested with a variable point of reversal and then with multiple points of reversal within a session. Results showed that the rats effectively maximized reinforcement by developing an approximation to a win-stay/lose-shift rule. The greater efficiency shown by rats with this task suggests that they are better able to use the feedback from their preceding choice as the basis of their future choice. The difference in cue preference further suggests a qualitative difference in acquisition of the midsession reversal task between pigeons and rats.

"Counting" by pigeons: discrimination of the number of biologically relevant sequential events.

Numerical competence has been studied in animals under a variety of conditions, but only a few experiments have reported animals' ability to detect absolute number. Capaldi and Miller (1988) tested rats' ability to detect absolute number by using biologically important events--the number of reinforced runs followed by a nonreinforced run--and found that the rats ran significantly slower on the nonreinforced run. In the present experiments, we used a similar procedure. Pigeons were given a sequence of trials in which responding on the first three trials ended in reinforcement but responding on the fourth trial did not (RRRN). When the response requirement on each trial was a single peck (Experiment 1), we found no significant increase in latency to peck on the fourth trial. When the response requirement was increased to 10 pecks (Experiment 2), however, the time to complete the peck requirement was significantly longer on the nonreinforced trial than on the reinforced trials. Tests for control by time, number of responses, and amount of food consumed indicated that the pigeons were using primarily the number of reinforcements obtained in each sequence as a cue for nonreinforcement. This procedure represents a sensitive and efficient method for studying numerical competence in animals.