Does the context of reinforcement affect resistance to change?

Eight pigeons were trained on multiple schedules of reinforcement where pairs of components alternated in blocks on different keys to define 2 local contexts. On 1 key, components arranged 160 and 40 reinforcers/hr; on the other, components arranged 40 and 10 reinforcers/hr. Response rates in the 40/hr component were higher in the latter pair. Within pairs, resistance to prefeeding and resistance to extinction were generally greater in the richer component. The two 40/hr components did not differ in resistance to prefeeding, but the 40/hr component that alternated with 10/hr was more resistant to extinction. This discrepancy was interpreted by an algebraic model relating response strength to component reinforcer rate, including generalization decrement. According to this model, strength is independent of context, consistent with research on schedule preference.

The effects of context and feedback on age differences in spoken word recognition.

We investigated the hypothesis that age differences in speech discrimination would be reduced by enhancing the distinctiveness of the speech processing event in terms of both the context of encoding and the response outcome. Younger and older adults performed an auditory lexical decision task in which the degree of semantic constraint (context) and type of feedback were manipulated. Main effects of age indicated that older adults generally showed lower discriminability (D) and greater bias (B) toward reporting signals to be words. Consistent with the environmental support hypothesis, older adults were differentially facilitated in discriminability by feedback, but only when semantic context was provided. Also, for both younger and older adults, feedback and context each had the effect of reducing bias and facilitating the speed of rejecting nonwords. Contrary to one suggestion in the literature that aging brings an insensitivity to environmental contingency, older adults were at least as capable as the young in taking advantage of feedback to normalize the speech signal so as to increase discriminability and decrease bias.

Timing and choice in concurrent chains.

To investigate the role of timing processes in choice, we used a new procedure that provided simultaneous measures of ongoing choice and timing behavior. Pigeons responded in a peak procedure in which the delays to reinforcement signaled by red and green center-key stimuli were 10 and 20, or 20 and 40 s. After 25 sessions of training, the peak procedure was embedded within concurrent chains: The inter-trial interval was replaced by a choice phase in which the two side keys were illuminated white; responses to the left and right keys occasionally changed the center-key to red or green, respectively; and the terminal links signaled by the center-key stimuli were identical to the trials of the peak procedure. The temporal control of responding on no-food trials was the same regardless of whether the no-food trials occurred in the peak procedure or as the terminal links of concurrent chains. After an intervening condition with the peak procedure in which the delay for the 10 s stimulus was changed to 40 s (or vice versa), the pigeons were returned to concurrent chains. Choice responding did not reflect the changed delay, despite the fact that the pigeons timed the delays in both terminal links accurately as indexed by responding on no-food trials. This result challenges current accounts of choice based on timing processes, such as scalar expectancy theory, which assume that choice responding is mediated by a representation of terminal link delays to reinforcement. Apparently, pigeons' choice and timing behavior in a single session can be controlled by temporal information from different temporal epochs.

On the form of the relation between response rates in a multiple schedule.

Three pigeons received training on multiple variable-interval schedules with brief alternating components, concurrently with a fixed-interval schedule of food reinforcement on a second key. Fixed-interval performance exhibited typical increases in rate within the interval, and was independent of multiple-schedule responding. Responding on the multiple-schedule key decreased as a function of proximity to reinforcement on the fixed-interval key. The overall relative rate of responding in one component of the multiple schedule roughly matched the overall relative rate of reinforcement. Within the fixed interval, response rate during one multiple-schedule component was a monotonic, negatively accelerated function of response rate during the other component. To a first approximation, the data were described by a power function, where the exponent depended on the relative rate of reinforcement obtained in the two components. The relative rate of responding in one component of the multiple schedule increased as a function of proximity to fixed-interval reinforcement, and often exceeded the overall obtained relative rate of reinforcement. The form of the function relating response rates is discussed in relation to findings on rate-dependent effects of drugs, chaining, and the relation between response rate and reinforcement rate in single-schedule conditions.

Response strength in multiple schedules.

In several different experiments, pigeons were trained with one schedule or condition of food reinforcement for pecking in the presence of one key color, and a different schedule or condition in the presence of a second key color. After responding in both of these multiple schedule components stabilized, response-independent food was presented during dark-key periods between components, and the rates of pecking in both schedule components decreased. The decrease in responding relative to baseline depended on the frequency, magnitude, delay, or response-rate contingencies of reinforcement prevailing in that component. When reinforcement was terminated, decreases in responding relative to baseline rates were ordered in the same way as with response-independent food. The relations between component response rates were power functions. Internal consistencies in the data, in conjunction with parallel findings in the literature, suggest that the concept of response strength summarizes the effects of diverse procedures, where response strength is identified with relative resistance to change. The exponent of the power function relating response rates may provide the basis for scaling response strength.

Quantitative analysis.

Quantitative analysis permits the isolation of invariant relations in the study of behavior. The parameters of these relations can serve as higher-order dependent variables in more extensive analyses. These points are illustrated by reference to quantitative descriptions of performance maintained by concurrent schedules, multiple schedules, and signal-detection procedures. Such quantitative descriptions of empirical data may be derived from mathematical theories, which in turn can lead to novel empirical analyses so long as their terms refer to behavioral and environmental events. Thus, quantitative analysis is an integral aspect of the experimental analysis of behavior.

Behavioral economics and behavioral momentum.

Some relations between elasticity of demand and the conditions of reinforcement are reanalyzed in terms of resistance to change, in ways suggested by the metaphor of behavioral momentum; some relations between resistance to change and the conditions of reinforcement are reanalyzed in terms of elasticity of demand, in ways suggested by behavioral economics. In addition, some data on labor supply in relation to variable-ratio schedules and alternative reinforcement are reanalyzed in terms of resistance to change and compared with steady-state resistance data for performance on multiple and concurrent interval schedules. The results of these studies can be summarized by two functions based on the behavioral momentum approach, relating relative behavioral mass to relative reinforcement per response or per unit time. The former is a relation between relative unit price and relative behavioral mass, suggesting the possibility of convergent measurement of a theoretical construct common to both approaches. However, the momentum and economic approaches differ fundamentally on whether it is preferable to construe discriminated operant behavior as selected and strengthened by its consequences or as part of a behavior-consequence bundle that maximizes utility.

An integrative model for the study of behavioral momentum.

Behavioral momentum is the product of response rate and resistance to change. The data on relative resistance to change are summarized for pigeons responding on single-key two-component multiple schedules, in the initial links of two-key multiple chained schedules, and in equivalent components of two-key serial schedules. For single-key procedures, the ratio of resistance to change in two schedule components is shown to depend on the ratio of reinforcer rates obtained in the presence of the component stimuli. For two-key procedures, the ratio of resistance to change in equivalent components is shown to depend on the ratio of reinforcer rates correlated with key locations. A model based on stimulus-reinforcer contingencies that combines the reinforcer rates in schedule components summed over key locations and reinforcer rates correlated with key locations summed over components, each expressed relative to the session average reinforcer rate, gives a good account of the data. An extension of the relative law of effect for multiple schedules fails to provide a complete account of resistance to change, but both approaches are needed for a comprehensive understanding of behavioral momentum.

Effects of reinforcement scheduling on simultaneous discrimination performance.

Pigeons were trained on a discrete-trials, simultaneous discrimination procedure, with confusable stimuli such that asymptotic performance was about 85% correct. Trials were terminated if no response occurred within 2 sec of stimulus onset, so that probability of responding was free to vary. The schedule of reinforcement for correct responses was varied, with the following results: (1) there was no relation between frequency of reinforcement and accuracy of responding. (2) In extinction, the probability of responding fell to low levels, but accuracy remained roughly constant. (3) When reinforcement was available after a fixed number of trials or after a fixed number of correct responses, the probability of responding increased with successive trials after reinforcement, but accuracy was generally constant. (4) When every fifth correct response was reinforced, accuracy decreased immediately after reinforcement if the birds were required to respond on every trial.

Rates and patterns of responding with concurrent fixed-interval and variable-interval reinforcement.

Pigeons were exposed to concurrent fixed-interval and variable-interval schedules of food reinforcement on two keys. The times between reinforcement were varied systematically on both keys. The overall relative frequency of responding on the fixed-interval key depended on the relative frequency of reinforcement, but did not match it. Instead, the ratio of responses on the fixed-interval key to responses on the variable-interval key was a power function of the ratio of reinforcements, with an exponent of 0.5. Patterns of responding between reinforcements on the fixed-interval key depended on both relative and absolute values of the reinforcement schedules. Similar overall relative responding was obtained at different absolute schedule values with equal relative reinforcement, despite some differences in patterns of responding.

Interval reinforcement of choice behavior in discrete trials.

Pigeons were trained to peck at red or green keys presented simultaneously in discrete trials. In one experiment, reinforcements were arranged by concurrent variable-interval schedules. The proportion of responses to green approximately matched the proportion of reinforcements produced by pecking green. Detailed analysis of responding revealed a systematic decrease in the probability of switching from green to red within sequences of trials after reinforcement. This trend corresponded to sequential changes in the relative frequency of reinforcement, and not to sequential changes in probability of reinforcement. In a second experiment, reinforcements were scheduled probabilistically every seventh trial. Even though there were no contingencies on pecking during the first six post-reinforcement trials, choices of green on the first response after reinforcement matched the proportion of reinforcements for pecking green. These results extend the generality of overall matching under concurrent reinforcement.

Differential reinforcement and stimulus control of not responding.

Pigeons were trained to respond with equal variable-interval reinforcement in the presence of a white key and also a white key with a vertical line. They were then trained not to respond to the vertical line by extinguishing the response or by reinforcing its non-occurrence at various frequencies. During training, the rate of key-pecking in the presence of the white key, maintained by a constant variable-interval schedule of reinforcement, depended on the frequency of reinforcement in the presence of the line. When lines of different orientations were presented in a generalization test, birds trained with extinction responded more to other orientations than to the vertical line, whereas those trained with high frequencies of reinforcement for not responding tended to respond equally at all line orientations. Intermediate frequencies of reinforcement gave mixed results.

Transfer of hue matching in pigeons.

Pigeons were trained on a modified three-key matching-to-sample procedure, in which only one comparison key (rather than two) was lighted after an observing response to the center-key standard. Pecks on keys of matching comparison hues were reinforced. When non-matching hues appeared as the initially lighted comparisons, the nonmatching hue terminated and the matching hue appeared on the other side key only if the pigeon did not peck the nonmatching comparison for 4.8 sec. Pecks to the nonmatching hue reset the 4.8-sec delay interval. Three hues were used during acquisition. During transfer tests, two novel hues were substituted individually or together for one or two of the training hues. Latencies to the novel side-key hue were shortest when a novel matching hue appeared as the standard on the center key, and were essentially identical to baseline matching latencies. In contrast, when a novel hue appeared as either a standard or comparison in a nonmatching combination, latencies increased with increasing separation between the noevel hue and the nonmatching hue. These transfer data demonstrate the concept of hue matching.

Choice, time allocation, and response rate during stimulus generalization.

Six pigeons were trained to discriminate between two noise intensities using a procedure that assessed choice, time allocation, and response rate simultaneously and independently. Responses on the left or right key (R1 or R2) were respectively correct in the presence of two different intensities, S1 and S2. After a correct response, reinforcement became available for pecks on the center key. Reinforcement density for R1|S1 relative to R2|S2 was varied across experimental conditions. Generalization tests followed extensive training at each condition. As a function of stimulus intensity, proportions of initial choices of R2, of time spent in R2-initiated components, and of center-key responses emitted in R2-initiated components all yielded sigmoidal gradients of similar slope, which shifted slightly in location when relative reinforcement density changed. Changeovers were maximal where initial choice proportions approximated 0.5. Gradients relating the absolute number of center-key responses to stimulus intensity were also roughly sigmoidal, but were more sensitive to changes in reinforcement density. Gradients of momentary response rate also depended on reinforcement density. During training, large but transitory shifts in choice responding occurred when reinforcement density changed, while differences in momentary response rate developed slowly, suggesting separate control of choice and response rate by the contingencies of reinforcement.

Signal detection methods for measurement of utility in animals.

Analytic methods of signal detection theory were employed to assess the utility of reinforcers. Four pigeons were trained to detect the presence or absence of a stimulus by pecking one of two side keys in a trial-by-trial choice paradigm. The relative rate of positive reinforcement for correct choices was varied to offset the biasing effects of electric shock for incorrect right side-key choices. The effects of relative rate of reinforcement on bias were similar at all shock intensities even though the subjects' sensitivity changed during the course of the experiment. The relative rate of reinforcement required to produce equal bias was calculated and plotted against shock intensity to generate utility functions. The relative rate of reinforcement necessary to offset the bias induced by shock was an increasing function of shock intensity.

Preference and resistance to change with constant-duration schedule components.

Previous research on preference between variable-interval terminal links in concurrent chains has most often used variable-duration terminal links ending with a single reinforcer. By contrast, most research on resistance to change in multiple schedules has used constant-duration components that include variable numbers of reinforcers in each presentation. Grace and Nevin (1997) examined both preference and resistance in variable-duration components; here, preference and resistance were examined in constant-duration components. Reinforcer rates were varied across eight conditions, and a generalized-matching-law analysis showed that initial-link preference strongly over-matched terminal-link reinforcer ratios. In multiple schedules, baseline response rates were unaffected by reinforcer rates, but resistance to intercomponent food, to extinction, and to intercomponent food plus extinction was greater in the richer component. The between-component difference in resistance to change exhibited additive effects for the three resistance tests, and was systematically related to reinforcer ratios. However, resistance was less sensitive to reinforcer ratios than was preference. Resistance to intercomponent food and to intercomponent food plus extinction was more sensitive to reinforcer ratios in the present study than in Grace and Nevin (1997). Thus, relative to variable-duration components, constant-duration components increased the sensitivity of both preference and relative resistance, supporting the proposition that these are independent and convergent measures of the effects of a history of reinforcement.

Feedback functions for variable-interval reinforcement.

On a given variable-interval schedule, the average obtained rate of reinforcement depends on the average rate of responding. An expression for this feedback effect is derived from the assumptions that free-operant responding occurs in bursts with a constant tempo, alternating with periods of engagement in other activities; that the durations of bursts and other activities are exponentially distributed; and that the rates of initiating and terminating bursts are inversely related. The expression provides a satisfactory account of the data of three experiments.

Conditioned reinforcement and choice.

In a series of three experiments, rats were exposed to successive schedule components arranged on two levers, in which lever pressing produced a light, and nose-key pressing produced water in 50% of the light periods. When one auditory signal was presented only during those light periods correlated with water on one lever, and a different signal was presented only during those light periods correlated with nonreinforcement on the other lever, the former lever was preferred in choice trials, and higher rates of responding were maintained on the former lever in nonchoice (forced) trials. Thus, the rats preferred a schedule component that included a conditioned reinforcer over one that did not, with the schedules of primary reinforcement and the information value of the signals equated. Preferences were maintained when one or the other of the auditory signals was deleted, but were not established in naive subjects when training began with either the positive or negative signal only. Discriminative control of nose-key pressing by the auditory signals was highly variable across subjects and was not correlated with choice.

Comparing preference and resistance to change in constant- and variable-duration schedule components.

Two experiments explored preference and resistance to change in concurrent chains in which the terminal links were variable-interval schedules that ended either after a single reinforcer had been delivered (variable duration) or after a fixed period of access to the schedule (constant duration). In Experiment 1, pigeons' preference between the same pair of terminal links overmatched relative reinforcement rate when the terminal links were of constant duration, but not when they were of variable duration. Responding during the richer terminal link decreased less, relative to baseline, when response-independent food was presented during the initial links according to a variable-time schedule. In Experiment 2, all subjects consistently preferred a terminal link that consisted of 20-s access to a variable-interval 20-s schedule over a terminal link that ended after one reinforcer had been delivered by the same schedule. Results of resistance-to-change tests corresponded to preference, as responding during the constant-duration terminal link decreased less, relative to baseline, when disrupted by both response-independent food during the initial links and prefeeding. Overall, these data extend the general covariation of preference and resistance to change seen in previous studies. However, they suggest that reinforcement numerosity, including variability in the number of reinforcers per terminal-link entry, may sometimes affect preference and resistance to change in ways that are difficult to explain in terms of current models.