On sundials, springs, and atoms.

The orderly behavior that occurs when animals are required to deal with time requirements suggests the possibility that they have an internal clock that provides information about the duration of events. After discussing questions inherent in the concept of an internal clock and suggesting criteria that such a device should meet if it exists, data are reviewed involving several different types of experimental procedures. Every procedure produced different conclusions about the nature of timing. Such results, together with observations of behavior outside of the laboratory, suggest that an internal clock has not evolved and, furthermore, is not even necessary for animals to display temporal regularities in their behavior or to respond to temporal demands.

To wait or to respond?

Emitting a certain response and waiting for a specified time without making that response had the same consequence. In Experiment 1, food-deprived pigeons were as likely to wait as to respond only if waiting provided food at a much higher frequency than did pecking. In Experiment 2, the consequence for humans was a brief light flash and tone. People were not biased for responding over waiting. Instead, their choices suggested crude payoff maximization. In Experiment 3, pigeons again obtained food, but they were not food deprived and could eat freely at each opportunity. Their behavior was more like that of the humans of Experiment 2 than that of food-deprived pigeons given small quantities of food at each feeding opportunity. The three experiments together showed that biases for responding over waiting were neither inherent characteristics of species nor inevitable outcomes of particular schedules. Choice between active search and waiting depended on ecological-motivational factors even when species and schedules were held constant.

On immediate function.

Behavior is a property of living organisms, not of inanimate matter. The problems of physical science are to understand how a phenomenon works; biological science adds the questions of what a phenomenon does and how something that does such things came to be. Exclusive dedication to cause-effect explanations ignores how behavior helps creatures cope with their internal and external environments. Laws of causation describe the precursors to behavior; laws of function describe the effects of behavior. The numerous instances of learning reflect the many ways that selective pressure for altering behavior on the basis of experience has been manifested. Little basis exists for assuming that the various forms of learning reflect either common functions or common processes. Instead, it seems that evolutionary processes have resulted in domain-specific learning. The rules of learning must be understood in terms of the function that the particular manifestation of learning serves for the organism. Evolutionary theory provides the framework for understanding function as well as relations between function and causal mechanisms.

Reinforcing the absence of fixed-ratio performance.

Pigeons received food for key pecking according to a fixed-ratio schedule, while, at the same time, food also was available for not pecking for a specified time. With a fixed ratio of 60, responding was not affected by not-pecking times of 80 or 40 seconds, and was eliminated completely at 10 seconds. With ratios of 180, pecking stopped with not-pecking times of 80 seconds or less; with ratios of 300, it stopped at 120 seconds or less. Not-responding schedules produced steady-state performance immediately following contact with the schedule. With return to the fixed-ratio schedule alone, response rate sometimes was elevated temporarily. When response-independent food presentations replaced the not-pecking schedule, response rate often was enhanced, and the ratio pattern was lost. Only the highest densities of food delivery eliminated responding, even with a fixed ratio of 300. In general, the effects corresponded to those of punishment, except that contrast had appeared both during and after punishment, and now appeared only after the response elimination procedure was suspended.

Sources of timulus control during multiple discriminations.

Pigeons learned to respond to the middle-sized member of six or seven sets of three stimuli differing in size. The sets were used successively, each serving as the discrimination problem from 10 to 16 times. After attaining criterion with one set, the birds received the others as probes. The number of responses in probes was related to the similarity of the probes to the prevailing discrimination problem. The birds responded either to the probe stimulus to which responding had most recently been reinforced, or to the probe stimulus closest in size to the positive member (S+) of the prevailing discrimination problem. Responses to a middle-sized probe-set stimulus occurred when it was the probe-set member most recently correlated with reinforcement, when it was one of two stimuli closest in size to S+, and when the stimulus closest in size to S+ was a negative member of the discrimination problem. All of the behavior could be explained in terms of control by the absolute sizes of the various stimuli.

Stimulus control with fixed-ratio reinforcement.

Pigeons were trained on a three-stimulus simultaneous discrimination with reinforcements given on fixed-ratio schedules ranging from one response per reinforcement to 205 responses per reinforcement. Responses to any of the three stimuli advanced the ratio, but, when it was completed, a reinforcement followed a response to one designated stimulus. Throughout the ratio, the birds responded mainly to the designated stimulus. The relatively few responses to the other stimuli usually occurred immediately after postreinforcement pauses. The distribution of responses to the various stimuli was not affected by omitting reinforcements.

Fixed and variable schedules of response-independent reinforcement.

After response-dependent reinforcement established key-pecking as the predominant response, pigeons received schedules in which reinforcements occurred without reference to responding. These response-independent schedules involved either a reinforcement every 5 min, or reinforcements at irregular intervals that averaged 5 min. The response-independent schedules generated characteristic patterns of responding. The fixed schedule produced positively accelerated responding between reinforcements, and the variable schedule produced either steady rates, erratic, or negatively accelerated patterns. The patterns developed independent of the distribution of responses existing when the schedule was first imposed. The rate of responding varied for the three birds, but, for all, response-independent schedules decreased the rates below the level maintained by response-dependent reinforcement. Although the rate of responding was affected primarily by the events contiguous with reinforcement, the pattern of responding appeared to be determined mainly by the presentation of reinforcements in relation to time.

Repeated measurements of reinforcement effects on gradients of stimulus control.

Two experiments studied the effects of reinforcement schedules on generalization gradients. In Exp. 1, after pigeons' responding to a vertical line was reinforced, the pigeons were tested with 10 lines differing in orientation. Reconditioning and the redetermination of generalization gradients were repeated from 8 to 11 times with the schedule of reinforcement varied in the reconditioning phase. Stable gradients could not be observed because the successive reconditionings and tests steepened the gradients and reduced responding. Experiment 2 over-came these effects by first training the birds to respond to all of the stimuli. Then, brief periods of reinforced responding to the stimulus correlated with reinforcement alternated with the presentation of the 10 lines in extinction. The development of stimulus control was studied eight times with each bird, twice with each of four schedules of reinforcement. Gradients were similar each time a schedule was imposed; the degree of control by the stimulus correlated with reinforcement varied with particular schedules. Behavioral contrast occurred when periods of reinforcement and extinction alternated and was more durable with fixed-interval, variable-interval, and variable-ratio schedules than with fixed-ratio or differential-reinforcement-of-low-rate schedules.

Eliminating behavior with reinforcement.

Responding produced food according to a fixed-ratio schedule while the prevailing key-color alternated between red and blue. Stimulus durations were varied until a period was found that maintained equal rates of responding in the presence of both colors. Then, food presentation was discontinued in the presence of one stimulus and made dependent on not responding in the presence of the other. Food presentation dependent on not responding reduced the rate of responding faster than did extinction. Spontaneous recovery occurred only during the stimulus correlated with extinction.

The sleeping giant: Reinforcement schedules.

Schedule research has been the core of operant conditioning, but it is no longer an active area, at least with respect to its traditional focus of describing and explaining moment-to-moment behavior. Yet schedules are central in psychology: Not only do they establish lawful behavior, but they also play a major role in determining the effects of other variables. The reason for the decline appears to be primarily theoretical, in that the work seems not to have led to meaningful integration. The search for controlling variables brought into play by schedule specification has proven unsuccessful, and a catalog of all possible schedule effects is of limited interest. The paper reviews the reasons for the contemporary state of affairs. One prediction about future developments is that instead of revealing component variables and their modes of interaction, schedule effects will be treated as basic empirical laws. Theory will take the form of abstract statements that integrate these separate laws by reference to higher-order principles rather than by reduction to supposedly simpler component variables.

Time limits for completing fixed ratios.

Two experiments investigated how the addition of time limits affected fixed-ratio behavior. In Exp. 1, pigeons obtained food only if they completed the ratio within a specified time after the end of the preceding ratio. In Exp. 2, they obtained food only if they took longer than a specified time. Failures to meet the time criteria produced brief timeouts. The times taken depended on the requirements in both experiments. In Exp. 1, progressively briefer time criteria resulted in faster ratios, and in Exp. 2, longer time criteria increased the time taken in each ratio. The pigeon's sensitivity to the temporal variable, a property of the entire period extending from the first opportunity to respond to the end of the ratio, indicated that performance involved a behavioral unit encompassing both the post-reinforcement pause and the responses comprising the ratio.

Fixed-interval stimulus control.

Pigeons were exposed to three stimuli simultaneously, with responses to one of them, the positive stimulus, followed by food presented according to a fixed-interval schedule (FI 2-min, FI 4-min, FI 8-min, or FI 16-min). Over 90% of the total responses emitted were to the positive stimulus within a few sessions. When the birds were then studied under each of four fixed intervals, responding continued to be confined primarily to the same stimulus independent of interval size. In subsequent conditions, the three stimuli changed positions after each quarter of the interval. If the position changes did not require a response, response rate and the percentage of responses occurring to the positive stimulus decreased. If the changes did depend on a response, the complete interval appeared to be divided into four smaller intervals with a pause and then positively accelerated responding following each position change. Position changes produced by a response to any stimulus decreased control by the positive stimulus, and changes produced only by a response to it increased the percentage of responses made to that stimulus. All of the data suggested that the stimulus conditions contiguous with reinforcement controlled behavior.

Positive reinforcement and the elimination of reinforced responses.

Key pecking was maintained on a fixed-interval schedule while either a differential-reinforcement-of-not-responding or a fixed-time schedule was imposed simultaneously. The lower the time parameter of the not-responding schedule, the lower was the response rate. Similar effects occurred with the fixed-time schedule, if the pigeons had experience with reinforcement for not responding. Otherwise the effects were less orderly, to the extent that rate could reach maximum with the lowest-valued fixed-time schedule. The not-responding and the response-independent schedules had similar effects on rate in experienced pigeons only when the time parameter or nominal frequency of food presentation was considered. When considered in terms of obtained frequency of food presentation, reinforcement of not responding produced larger decrements in rate than did the fixed-time schedule.

Conjunctive schedules of response-dependent and response-independent reinforcement.

Pigeons received food when they emitted the number of responses specified by a fixed-ratio schedule, and the time specified by a fixed-time schedule had elapsed. The order of meeting the response and time requirements was irrelevant. In different conditions, stimuli signalled completion of one, both, or neither requirement. Ratio size interacted with stimulus condition to determine performance. When a stimulus signalled the end of the fixed-time period, under all ratios the birds tended to respond after the stimulus appeared. When stimuli followed both components, small ratios produced responding during the fixed-time period, and other ratios resulted in responses after the time period had elapsed. With either no stimulus changes, or with a stimulus correlated with completion of the ratio alone, responding first increased and then decreased as the ratio increased. Low and high ratios produced stable response frequencies and patterns in successive intervals. Intermediate ratios resulted in two types of performance. Intervals with long initial pauses and few responses during the fixed-time period were followed by intervals with short pauses and numerous responses and vice versa. The source of these dynamic effects was hypothesized to be number of responses per reinforcer in one condition and response-reinforcer contiguity in the other.

Fixed-interval behavior: effects of percentage reinforcement.

The percentage of fixed intervals terminating with food presentation was varied parametrically. Intervals that did not end with food were terminated by a stimulus uncorrelated with food presentation (a timeout stimulus). In Experiment I, the pigeons' response rates were an inverted U-shaped function of the percentage of food presentations: decreasing the percentage from 100% to 90%, 70%, or 50% produced an increase in response rates; lower percentages decreased the rates. The patterns of responding in the 100% condition differed from those of the other conditions. In Experiment II, the chamber was darkened after food presentations and timeouts. Response rate was directly related to the percentage of food presentations: decreasing the percentage decreased the response rate. Characteristic fixed-interval patterns of responding were maintained as long as there were occasional food presentations; pausing followed by positively-accelerated responding occurred in percentage conditions ranging from 7% to 100%. The ability to maintain fixed-interval performance with percentage reinforcement suggested that the behavioral sequences occurring in each interval may operate as unitary responses.

Time limits for completing fixed ratios. II. Stimulus specificity.

Pigeons were exposed to two stimuli that were each correlated with the same fixed-ratio schedule. During one stimulus, the ratio had to be completed in more than a specified time for food to be presented. During the other, the ratio had to be completed in less than a specified time. Failures to meet the time criterion produced a timeout and reset the ratio. Treating the entire behavioral sequence generated by a fixed-ratio schedule as a unitary response and scheduling differential reinforcement with respect to the duration of the sequence produced effects comparable to those observed previously with individual responses. The times taken to complete the ratios conformed to the time criteria under both stimulus conditions. Similar effects have been observed with differential reinforcement of the duration of individual responses and in experiments specifying running speed criteria under discrete trial conditions.

Reinforcement of spaced responding in a simultaneous discrimination.

Pigeons were exposed to three stimuli simultaneously with responses reinforced according to differential-reinforcement-of-low-rate schedules. Responses to one stimulus (the positive stimulus) that were spaced appropriately resulted in food presentation. The variables manipulated were the time parameter of the schedule (5, 10, 20, 30 sec) and the consequences of responding to the other two stimuli (the negative stimuli). The percentage of the total responses that occurred to each stimulus was independent of the schedule value but was dependent on the consequences of responding to the negative stimuli. If responses to both reset the schedule timer, responding was confined largely to the positive stimulus. If responses to neither had scheduled effects, the birds were more likely to respond to those stimuli. Responding to one negative stimulus could be selectively attenuated by having responses to that stimulus alone reset the timer. With the schedule time value held constant, the absolute rate of responding to the positive stimulus was either stable or decreased with maintained exposure; it did not change as a function of increases or decreases in responding to the negative stimuli. Rather than interacting and affecting each other, responses to the three stimuli were controlled independently by their relation to reinforcement. There was no evidence that responses to the negative stimuli mediated the spacing of responses to the positive stimulus.

Elimination of reinforced behavior: intermittent schedules of not-responding.

Pigeons' key pecking resulted in food according to either a variable-ratio or a variable-interval schedule. At the same time, food was available for not pecking for a specified time. The required time of not-pecking was segmented into not-responding units, and these units were followed by food according to a fixed-ratio schedule. Both unit duration and the number required were varied. In general, the shorter the time unit or the smaller the ratio, the lower was response rate. When total required not-responding time was constant, but changes in unit duration and the number required altered how the total was achieved, shorter units produced lower rates. Other conditions involved substitution of food delivered independent of responding for the not-responding schedule. With low and moderate total times to food presentation, the not-responding schedule produced lower rates; with the longest times, the response-independent schedule generated less responding. When considered in terms of relative frequency of food presentation available from a source other than pecking, the not-responding schedule reduced rate more effectively than did the response-independent schedule. Comparisons with other research suggested that food presented dependent on not responding compared favorably with punishment as a procedure for reducing response rate. Transient effects differed. Although punishment temporarily depresses rate when first imposed and temporarily enhances it when first removed, food given for not responding quickly generated steady-state rates.

Pure timing in temporal differentiation.

Temporal control appears to depend on whether the critical durations are those of stimuli or those of responses. Stimulus timing (temporal discrimination) supports Weber's law, whereas response timing (temporal differentiation) indicates decreasing relative sensitivity with longer time intervals. The two types of procedure also yield different conclusions in scaling experiments designed to study the functional midpoint of two or more durations (temporal bisection procedures). In addition, the fractional-exponent power relation between emitted and required duration usually found with animals in differentiation experiments conflicts with deductions from formal analyses. The experiment reported here derived from considering differentiation arrangements as schedules of reinforcement. When analyzed from this perspective, the procedures are tandem schedules involving a required pause followed by a response, and it is the pause alone that involves temporal control. A choice procedure separated timing from responding, and enabled observations of pause timing in isolation. Pure temporal control in differentiation consisted of linear overestimation of the standard duration, and Weber's law described sensitivity. These results indicate that the two problems, the fractional-exponent power relations and the apparently different nature of sensitivity in differentiation and discrimination, disappear when temporal control is observed alone in differentiation.

Temporal control in fixed-interval schedules.

The peak procedure was used to study temporal control in pigeons exposed to seven fixed-interval schedules ranging from 7.5 to 480 s. The focus was on behavior in individual intervals. Quantitative properties of temporal control depended on whether the aspect of behavior considered was initial pause duration, the point of maximum acceleration in responding, the point of maximum deceleration, the point at which responding stopped, or several different statistical derivations of a point of maximum responding. Each aspect produced different conclusions about the nature of temporal control, and none conformed to what was known previously about the way ongoing responding was controlled by time under conditions of differential reinforcement. Existing theory does not explain why Weber's law so rarely fit the results or why each type of behavior seemed unique. These data fit with others suggesting that principles of temporal control may depend on the role played by the particular aspect of behavior in particular situations.