The Role of Contingencies and Stimuli in a Human Laboratory Model of Treatment of Problem Behavior.

Behavioral momentum theory posits a paradoxical implication for behavioral interventions in clinical situations using Differential Reinforcement of Alternative Behavior (DRA): When alternative reinforcers are presented within the same context as the problem behavior, the added reinforcers may decrease the frequency of the behavior but also increase its persistence when the intervention ends. Providing alternative reinforcers in a setting that is distinctively different from that in which the target behavior occurs may avoid or reduce this increase in persistence. The present experiment compared behavioral persistence following standard DRA versus DRA in a different context that was available after refraining from target behavior (Differential Reinforcement of Other Behavior, DRO). We arranged a human laboratory model of treatment intervention using computer games and token reinforcement. Participants were five individuals with intellectual disabilities. Experimental phases included (a) an initial multiple-schedule baseline with token reinforcement for target behaviors A and B, (b) an intervention phase with alternative reinforcement using a conventional DRA procedure for A and a DRO-DRA procedure for B, and (c) an extinction phase with no interventions and no tokens. Response rates as proportion of baseline in the initial extinction phase were greater for A than for B for three of five participants. Four participants whose response rates remained relatively high during the extinction phase then received a second extinction-plus-distraction test with leisure items available. Response rates were greater for A than for B in three of four participants. The results indicate that DRO-DRA contingencies may contribute to reduced post-intervention persistence of problem behavior.

Delayed matching to sample: reinforcement has opposite effects on resistance to change in two related procedures.

The effects of reinforcement on delayed matching to sample (DMTS) have been studied in two within-subjects procedures. In one, reinforcer magnitudes or probabilities vary from trial to trial and are signaled within trials (designated signaled DMTS trials). In the other, reinforcer probabilities are consistent for a series of trials produced by responding on variable-interval (VI) schedules within multiple-schedule components (designated multiple VI DMTS). In both procedures, forgetting functions in rich trials or components are higher than and roughly parallel to those in lean trials or components. However, during disruption, accuracy has been found to decrease more in rich than in lean signaled DMTS trials and, conversely, to decrease more in lean than in rich multiple VI DMTS components. In the present study, we compared these procedures in two groups of pigeons. In baseline, forgetting functions in rich trials or components were higher than and roughly parallel to those in lean trials or components, and were similar between the procedures. During disruption by prefeeding or extinction, accuracy decreased more in rich signaled DMTS trials, whereas accuracy decreased more in lean multiple VI DMTS components. These results replicate earlier studies and are predicted by a model of DMTS from Nevin, Davison, Odum, and Shahan (2007).

Differential outcomes enhance accuracy of delayed matching to sample but not resistance to change.

Three experiments assessed the relation between the differential outcomes effect and resistance to change of delayed matching-to-sample performance. Pigeons produced delayed matching-to-sample trials by responding on variable interval schedules in two components of a multiple schedule. In the same-outcome component, the probability of reinforcement was the same for both samples (.9 in Experiments 1 and 2, .5 in Experiment 3); in the different-outcomes component, the probability of reinforcement was .9 for one sample and .1 for the other. In all three experiments, the forgetting functions in the different-outcomes component were higher and shallower than in the same-outcomes component. When total reinforcement was greater in the same-outcomes component (Experiments 1 and 2), resistance to disruption by prefeeding, intercomponent food, extinction, or flashing lights typically was greater in that component. In Experiment 3, when total reinforcement was equated, resistance to disruption was similar across components. Thus, the level and slope of forgetting functions depended on differential reinforcement correlated with the samples, but the resistance to change of forgetting functions depended on total reinforcement in a component. Both aspects of the results can be explained by a model of delayed matching to sample performance.

Stimuli, reinforcers, and the persistence of behavior.

This article reviews evidence from basic and translational research with pigeons and humans suggesting that the persistence of operant behavior depends on the contingency between stimuli and reinforcers, and considers some implications for clinical interventions.

Control, prediction, order, and the joys of research.

JEAB's first decade featured control of individual behavior by operant contingencies, where the experimenter could interact with a subject in real time and obtain fairly immediate evidence of control, with the possibility of direct extension to applied settings. In subsequent decades, emphasis shifted toward long-term parametric studies with quantitative analyses of individual and group data. As a result, the reinforcers for the experimenter shifted from controlling behavior to uncovering and describing order, often using mathematical expressions. The same sorts of reinforcers are available for quantitative descriptions of aggregate behavioral data that may inform public policy.

The basis of behavioral momentum in the nonlinearity of strength.

The persistence of operant responding in the context of distractors and opposing forces is of central importance to the success of behavioral interventions. It has been successfully analyzed with Behavioral Momentum Theory. Key data from the research inspired by that theory are reanalyzed in terms of more molecular behavioral mechanisms: the demotivational effects of disruptors, and their differential impacts on the target response and other responses that interact with them. Behavioral momentum is regrounded as a nonlinear effect of motivation and reinforcement rate on response probability and persistence. When response probabilities are high, more energy is required to further increase or to decrease them than when they are low. Classic Behavioral Momentum Theory effects are reproduced with this account. Finally, it is shown how the new account involving motivation and competition is closely related to the metaphor of force and action that is at the core of Behavioral Momentum Theory.

Delivering alternative reinforcement in a distinct context reduces its counter-therapeutic effects on relapse.

Delivery of alternative reinforcers in the presence of stimuli previously associated with reinforcement for target behavior increases the susceptibility of target behavior to relapse. To explore contingencies that might mitigate this counter-therapeutic effect, we trained pigeons on a procedure that entailed extinction of previously reinforced target-key pecking, access to a distinct stimulus context contingently on refraining from target behavior (differential-reinforcement-of-other-behavior; DRO), and reinforcement of alternative-key pecks (differential-reinforcement of alternative behavior; DRA) in that context. This DRO-DRA treatment was compared with standard DRA in successive conditions, counterbalanced across pigeons. Target behavior extinguished more rapidly in the Standard-DRA condition. When alternative reinforcement was discontinued, however, there was less resurgence after DRO-DRA than after Standard DRA. In a third condition, the DRO contingency was suspended so that the former DRA stimuli were not presented (DRO-NAC), and resurgence was greater than in the Standard-DRA and DRO-DRA conditions. Reinstatement produced by response-independent reinforcers was small and similar across conditions. Subsequent reacquisition of target-key pecking under baseline reinforcement conditions was faster following DRO-NAC than Standard-DRA or DRO-DRA. These findings suggest that DRO-DRA might serve as a useful method in clinical settings for reducing problem behavior while minimizing the threat of posttreatment relapse.

A theory of attending, remembering, and reinforcement in delayed matching to sample.

A theory of attending and reinforcement in conditional discriminations is extended to working memory in delayed matching to sample by adding terms for disruption of attending during the retention interval. Like its predecessor, the theory assumes that reinforcers and disruptors affect the independent probabilities of attending to sample and comparison stimuli in the same way as the rate of overt free-operant responding as suggested by Nevin and Grace, and that attending is translated into discriminative performance by the model of Davison and Nevin. The theory accounts for the effects of sample-stimulus discriminability and retention-interval disruption on the levels and slopes of forgetting functions, and for the diverse relations between accuracy and sensitivity to reinforcement reported in the literature. It also accounts for the effects of reinforcer probability in multiple schedules on the levels and resistance to change of forgetting functions; for the effects of reinforcer probabilities signaled within delayed-matching trials; and for the effects of reinforcer delay, sample duration, and intertrial-interval duration. The model accounts for some data that have been problematic for previous theories, and makes testably different predictions of the effects of reinforcer probabilities and disruptors on forgetting functions in multiple schedules and signaled trials.

Comparing positive and negative reinforcement: A fantasy experiment.

We propose quantitative experimental approaches to the question of whether positive and negative reinforcement are functionally different, and discuss scientific and ethical concerns that would arise if these approaches were pursued.

Quantitative models of persistence and relapse from the perspective of behavioral momentum theory: Fits and misfits.

We review quantitative accounts of behavioral momentum theory (BMT), its application to clinical treatment, and its extension to post-intervention relapse of target behavior. We suggest that its extension can account for relapse using reinstatement and renewal models, but that its application to resurgence is flawed both conceptually and in its failure to account for recent data. We propose that the enhanced persistence of target behavior engendered by alternative reinforcers is limited to their concurrent availability within a distinctive stimulus context. However, a failure to find effects of stimulus-correlated reinforcer rates in a Pavlovian-to-Instrumental Transfer (PIT) paradigm challenges even a straightforward Pavlovian account of alternative reinforcer effects. BMT has been valuable in understanding basic research findings and in guiding clinical applications and accounting for their data, but alternatives are needed that can account more effectively for resurgence while encompassing basic data on resistance to change as well as other forms of relapse.

Effects of signaled and unsignaled alternative reinforcement on persistence and relapse in children and pigeons.

Three experiments explored the impact of different reinforcer rates for alternative behavior (DRA) on the suppression and post-DRA relapse of target behavior, and the persistence of alternative behavior. All experiments arranged baseline, intervention with extinction of target behavior concurrently with DRA, and post-treatment tests of resurgence or reinstatement, in two- or three-component multiple schedules. Experiment 1, with pigeons, arranged high or low baseline reinforcer rates; both rich and lean DRA schedules reduced target behavior to low levels. When DRA was discontinued, the magnitude of relapse depended on both baseline reinforcer rate and the rate of DRA. Experiment 2, with children exhibiting problem behaviors, arranged an intermediate baseline reinforcer rate and rich or lean signaled DRA. During treatment, both rich and lean DRA rapidly reduced problem behavior to low levels, but post-treatment relapse was generally greater in the DRA-rich than the DRA-lean component. Experiment 3, with pigeons, repeated the low-baseline condition of Experiment 1 with signaled DRA as in Experiment 2. Target behavior decreased to intermediate levels in both DRA-rich and DRA-lean components. Relapse, when it occurred, was directly related to DRA reinforcer rate as in Experiment 2. The post-treatment persistence of alternative behavior was greater in the DRA-rich component in Experiment 1, whereas it was the same or greater in the signaled-DRA-lean component in Experiments 2 and 3. Thus, infrequent signaled DRA may be optimal for effective clinical treatment.

Resistance to extinction in the steady state and in transition.

Three experiments with pigeons explored the constancy of reinforcer omission during extinction conjectured by rate estimation theory. Experiment 1 arranged 3-component multiple variable-interval (VI) schedules with a mixture of food and extinction trials within each session. Reinforcers omitted to an extinction criterion increased with food-trial reinforcer rate. Experiment 2 arranged 3-component multiple VI schedules where components differed in rate or number of reinforcers. Resistance to extinction depended on the training reinforcer rate but not on the number of reinforcers omitted. Experiment 3 replicated the partial-reinforcement extinction effect within subjects in a discrete-trial procedure and found that more reinforcers were omitted in continuous- than in partial-reinforcement trials. A model of extinction based on behavioral momentum theory accounted for all the data.

On science and the discriminative law of effect.

This article considers the process of the dissemination of scientific findings from the point of view of the discriminative law of effect. We assume that the purpose of science is to describe the state of the world in an unbiased and accurate manner. We then consider a number of challenges to the unbiased consensual development of science that arise from differences between science that is done, submitted for publication, and published. These challenges arise from the differential reinforcers for both research and publication delivered by journals and editors for novel results, the undervaluation of systematic replication and findings of invariance, and general lack of reinforcers for failed replications. All these challenges bias science toward searching for, reporting, and valuing novel results and consequently lead to a biased and erroneous view of the world. We suggest that science should be approached more conservatively, and that a reevaluation of the value of replication, and especially failed replication, is in order.

A theory of attending and reinforcement in conditional discriminations.

A model of conditional discrimination performance (Davison & Nevin, 1999) is combined with the notion that unmeasured attending to the sample and comparison stimuli, in the steady state and during disruption, depends on reinforcement in the same way as predicted for overt free-operant responding by behavioral momentum theory (Nevin & Grace, 2000). The rate of observing behavior, a measurable accompaniment of attending, is well described by an equation for steady-state responding derived from momentum theory, and the resistance to change of observing conforms to predictions of momentum theory, supporting a key assumption of the model. When probabilities of attending are less than 1.0, the model accounts for some aspects of conditional-discrimination performance that posed problems for the Davison-Nevin model: (a) the effects of differential reinforcement on the allocation of responses to the comparison stimuli and on accuracy in several matching-to-sample and signal-detection tasks where the differences between the stimuli or responses were varied across conditions, (b) the effects of overall reinforcer rate on the asymptotic level and resistance to change of both response rate and accuracy of matching to sample in multiple schedules, and (c) the effects of fixed-ratio reinforcement on accuracy. Some tests and extensions of the model are suggested, and the role of unmeasured events in behavior theory is considered.

Resistance to change of forgetting functions and response rates.

This experiment examined the effects of reinforcement probability on resistance to change of remembering and response rate. Pigeons responded on a two-component multiple schedule in which completion of a variable-interval 20-s schedule produced delayed matching-to-sample trials in both components. Each session included four delays (0.1 s, 2 s, 4 s, and 8 s) between sample termination and presentation of comparison stimuli in both components. The two components differed in the probability of reinforcement arranged for correct matches (i.e., rich, p = .9; lean, p = .1). Response rates during the variable-interval portion of the procedure were higher in the rich component during baseline and more resistant to the disruptive effects of intercomponent food and extinction. Forgetting functions were constructed by examining matching accuracy as a function of delay duration. Baseline accuracy was higher in the rich component than in the lean component as measured by differences in the gamma-intercept of the forgetting functions (i.e., initial discrimination), rather than from differences in the slope of the forgetting function (i.e., rate of forgetting). Intercomponent food increased the rate of forgetting relatively more in the lean component than in the rich component, but initial discrimination was not systematically affected. Extinction reduced initial discrimination relatively more in the lean component than in the rich component, but did not systematically affect rate of forgetting. These results are consistent with our previous data suggesting that, as for response rate, accuracy and resistance to change of discriminating are positively related to rate of reinforcement. These data also suggest that the disruptability of remembering depends on the conditions of reinforcement, but the way in which remembering is disrupted depends on the nature of the disruptor.

Accuracy of discrimination, rate of responding, and resistance to change.

Pigeons were trained on multiple schedules in which responding on a center key produced matching-to-sample trials according to the same variable-interval 30-s schedules in both components. Matching trials consisted of a vertical or tilted line sample on the center key followed by vertical and tilted comparisons on the side keys. Correct responses to comparison stimuli were reinforced with probability .80 in the rich component and .20 in the lean component. Baseline response rates and matching accuracies generally were higher in the rich component, consistent with previous research. When performance was disrupted by prefeeding, response-independent food during intercomponent intervals, intrusion of a delay between sample and comparison stimuli, or extinction, both response rates and matching accuracies generally decreased. Proportions of baseline response rate were greater in the rich component for all disrupters except delay, which had relatively small and inconsistent effects on response rate. By contrast, delay had large and consistent effects on matching accuracy, and proportions of baseline matching accuracy were greater in the rich component for all four disrupters. The dissociation of response rate and accuracy with delay reflects the localized impact of delay on matching performance. The similarity of the data for response rate and accuracy with prefeeding, response-independent food, and extinction shows that matching performance, like response rate, is more resistant to change in a rich than in a lean component. This result extends resistance to change analyses from the frequency of response emission to the degree of stimulus control, and suggests that the strength of discriminating, like the strength of responding, is positively related to rate of reinforcement.

Preference and resistance to change with constant- and variable-duration terminal links: independence of reinforcement rate and magnitude.

Pigeons responded in a three-component multiple concurrent-chains procedure in which the variable-interval reinforcement schedules were the same across components but magnitudes differed across components. The terminal links were arranged either as a variable delay followed by presentation of a reinforcer ("variable duration") or as a fixed period of access to the schedule during which a variable number of reinforcers could be earned ("constant duration"). Relative reinforcement rate was varied parametrically across both types of conditions. After baseline training in each condition, resistance to change of terminal-link responding was assessed by delivering food during the initial links according to a variable-time schedule. Both preference and resistance to change were more sensitive to reinforcement-rate differences in the constant-duration conditions. Sensitivities of preference and resistance to change to relative reinforcement rate did not change depending on relative reinforcement magnitude. Taken together, these results confirm and extend those of prior studies, and suggest that reinforcement rate and magnitude combine additively to determine preference and resistance to change. A single structural relation linking preference and resistance to change describes all the data from this and several related studies.

Modeling the effects of sensory reinforcers on behavioral persistence with alternative reinforcement.

Problem behavior often has sensory consequences that cannot be separated from the target response, even if external, social reinforcers are removed during treatment. Because sensory reinforcers that accompany socially mediated problem behavior may contribute to persistence and relapse, research must develop analog sensory reinforcers that can be experimentally manipulated. In this research, we devised analogs to sensory reinforcers in order to control for their presence and determine how sensory reinforcers may impact treatment efficacy. Experiments 1 and 2 compared the efficacy of differential reinforcement of alternative behavior (DRA) versus noncontingent reinforcement (NCR) with and without analog sensory reinforcers in a multiple schedule. Experiment 1 measured the persistence of key pecking in pigeons, whereas Experiment 2 measured the persistence of touchscreen responses in children with intellectual and developmental disabilities. Across both experiments, the presence of analog sensory reinforcers increased the levels, persistence, and variability of responding relative to when analog sensory reinforcers were absent. Also in both experiments, target responding was less persistent under conditions of DRA compared to NCR regardless of the presence or absence of analog sensory reinforcers.

Comment on Gallistel: behavior theory and information theory: some parallels.

In this article, Gallistel proposes information theory as an approach to some enduring problems in the study of operant and classical conditioning.

Resistance to extinction and behavioral momentum.

In the metaphor of behavioral momentum, reinforcement is assumed to strengthen discriminated operant behavior in the sense of increasing its resistance to disruption, and extinction is viewed as disruption by contingency termination and reinforcer omission. In multiple schedules of intermittent reinforcement, resistance to extinction is an increasing function of reinforcer rate, consistent with a model based on the momentum metaphor. The partial-reinforcement extinction effect, which opposes the effects of reinforcer rate, can be explained by the large disruptive effect of terminating continuous reinforcement despite its strengthening effect during training. Inclusion of a term for the context of reinforcement during training allows the model to account for a wide range of multiple-schedule extinction data and makes contact with other formulations. The relation between resistance to extinction and reinforcer rate on single schedules of intermittent reinforcement is exactly opposite to that for multiple schedules over the same range of reinforcer rates; however, the momentum model can give an account of resistance to extinction in single as well as multiple schedules. An alternative analysis based on the number of reinforcers omitted to an extinction criterion supports the conclusion that response strength is an increasing function of reinforcer rate during training.