The active time model of concurrent choice.

The following paper describes a steady-state model of concurrent choice, termed the active time model (ATM). ATM is derived from maximization principles and is characterized by a semi-Markov process. The model proposes that the controlling stimulus in concurrent variable-interval (VI) VI schedules of reinforcement is the time interval since the most recent response, termed here "the active interresponse time" or simply "active time." In the model after a response is generated, it is categorized by a function that relates active times to switch/stay probabilities. In the paper the output of ATM is compared with predictions made by three other models of operant conditioning: melioration, a version of scalar expectancy theory (SET), and momentary maximization. Data sets considered include preferences in multiple-concurrent VI VI schedules, molecular choice patterns, correlations between switching and perseveration, and molar choice proportions. It is shown that ATM can account for all of these data sets, while the other models produce more limited fits. However, rather than argue that ATM is the singular model for concurrent VI VI choice, a consideration of its concept space leads to the conclusion that operant choice is multiply-determined, and that an adaptive viewpoint-one that considers experimental procedures both as selecting mechanisms for animal choice as well as tests of the controlling variables of that choice-is warranted.

Proprioceptive stimuli and habit formation: Interresponse time mediated behavior in CD-1 mice.

The consolidation of behavioral sequences into relatively ballistic habits is thought to involve the formation of stimulus - response associations. Typically, the stimuli in these associations are assumed to be exteroceptive, i.e., external to the organism. However, responses, themselves, also possess stimulus properties that can mediate behavior. Indeed, these "proprioceptive cues" have long been hypothesized to underlie habit formation (Hull, 1934a, 1934b). One such stimulus involves the time durations between responses - a stimulus termed interresponse time (IRT). We hypothesize that IRTs can come to serve as stimuli that differentially control response elements during habit formation. To examine this hypothesis we report on two experiments that asked whether CD-1 mice utilize IRTs to structure behavior in a two-choice environment. In experiment 1, eight mice were exposed to a free-operant concurrent variable-interval (VI) 30-s VI 60-s reinforcement schedule. We found that switch and stay responses were differentially correlated with IRT durations. In Experiment 2 we directly and differentially reinforced stay/switch responses based on IRT durations in a two-lever procedure. For four of the subjects, the probability of receiving reinforcement after switch responses was proportional to IRT duration. For five of the subjects, these reinforcement probabilities were inversely proportional to IRT duration. Regardless, all of our subjects learned to emit IRT-mediated switching behavior that matched the reinforcement contingencies. Together, Experiments 1 and 2 provide the first evidence of which we are aware that IRTs can come to control sequential choice behavior in mice.

Preference as a function of active interresponse times: a test of the active time model.

In this article, we describe a test of the active time model for concurrent variable interval (VI) choice. The active time model (ATM) suggests that the time since the most recent response is one of the variables controlling choice in concurrent VI VI schedules of reinforcement. In our experiment, pigeons were trained in a multiple concurrent similar to that employed by Belke (1992), with VI 20-s and VI 40-s schedules in one component, and VI 40-s and VI 80-s schedules in the other component. However, rather than use a free-operant design, we used a discrete-trial procedure that restricted interresponse times to a range of 0.5-9.0 s. After 45 sessions of training, unreinforced probe periods were mixed with reinforced training periods. These probes paired the two stimuli associated with the VI 40-s schedules. Further, the probes were defined such that during their occurrence, interresponse times were either "short" (0.5-3.0 s) or "long" (7.5-9.0 s). All pigeons showed a preference for the stimulus associated with the relatively rich VI 40-s schedule--a result mirroring that of Belke. We also observed, though, that this preference was more extreme during long probes than during short probes--a result predicted by ATM.

An effect of inter-trial duration on the gambler's fallacy choice bias.

The gambler's fallacy is defined as the avoidance of a winning outcome in a stochastic environment with a constant probability. We tested the possibility that the gambler's fallacy in humans is responsive to the amount of time between choice allocations. Two groups of subjects were placed in a six-choice betting game in which the choices were clustered into two "patches." Groups were defined by the length of time - 2s or 6s - between trials. On any given trial subjects allocated six points among the alternatives, and retained any points that were bet on the winning alternative. Both groups showed evidence of the gambler's fallacy bias. However, the bias was stronger in the 6-s ITI group than in the 2-s ITI group. This difference was found primarily to be due to differences in the number of subjects showing an opposing bias to the gambler's fallacy, namely a preference for the most recent winning alternative. This choice bias is termed the hot hand fallacy. Our findings contradict predictions derived from a foraging heuristic and from traditional accounts of the gambler's fallacy.

A further application of the active time model to multiple concurrent variable-interval schedules.

In this experiment we show that the active time model (ATM) accurately predicts probe data from multiple concurrent VI VI schedules. Subjects were trained under a concurrent VI 30-s VI 60-s and a concurrent VI 60-s VI 120-s schedule. Two types of unreinforced probes were then conducted. The first paired the two VI 60-s stimuli. These stimuli, while equivalent in their associated absolute rates of reinforcement, differed in their relative rates of reinforcement. The second probe paired the VI 30-s stimulus with the relatively rich VI 60-s stimulus. In contrast with the first probe, these stimuli differed in their absolute rates of reinforcement, while being similar in their relative rates. During the first set of probes, birds preferred the VI 60-s stimulus trained with the VI 120-s schedule. During the second set of probes, birds were indifferent to the two stimuli. These results are less extreme than others reported in the literature. Nonetheless, we found that ATM accurately fit individual subject data in both sets of probes. In contrast a variant of scalar expectancy theory did not fit the data at either the individual or group level.

An application of the active time model to multiple concurrent variable-interval schedules.

The current experiment investigates whether an active time model can account for anomalous results that have emerged from multiple schedule, concurrent variable-interval (VI) VI experiments. The model assumes that (1) during concurrent VI VI training pigeons learn a function that relates time since the most immediate response, i.e., active time, to changeover probabilities and (2) that molar preference is the result of an interaction between inter-response time frequencies and the learned active time changeover functions. Pigeons were trained under a concurrent VI 30-s VI 30-s schedule and a concurrent VI 60-s VI 60-s schedule. Probes were conducted in which VI 30-s and VI 60-s stimuli were paired. During these probes, birds allocated choices equally to the stimuli. The active time model accurately fit individual subject data. In contrast data were not fit by a variant of scalar expectancy theory proposed by Gibbon [Gibbon, J., 1995. Dynamics of time matching: arousal makes better seem worse. Psychon. Bull. Rev. 2, 208-215].

Ontogeny has a phylogeny: background to adjunctive behaviors in pigeons and budgerigars.

Animals coping with operant conditioning tasks often show behaviors that are not recorded by keys, levers and similar response transducers. Nevertheless, these adjunctive behaviors should not be disposed of by classifying them as incidental. Often they are found to be at least partially influenced by the experimentally programmed contingencies, and under certain conditions they can in turn influence conditioned behaviors. Here we describe the occurrence and characteristics of two such behaviors, stimulus grasping in operantly key-pecking pigeons and intra-delay stereotypies in a delayed matching-to-sample task with budgerigars. It is argued that for a proper account of these behaviors it is necessary to refer to a behavioral systems approach that appeals to longer ranging ontogenetic and phylogenetic histories than is usually considered in the psychological literature. The gaping towards on-key stimuli by pigeons is attributed to the hypothesis that operantly conditioned key-pecks probably relate to a grasp-pecking response that is normally executed towards non-edible items covering food. The intra-delay behaviors shown by the budgerigars are assumed to have originated from stress-induced displacement responses that adventitiously came under the influence of differential reinforcement contingencies. Finally, we discuss what kinds of evidence are needed to put these hypothetical explanations on a more certain footing.