Delivering alternative reinforcement in a distinct context reduces its counter-therapeutic effects on relapse.

Delivery of alternative reinforcers in the presence of stimuli previously associated with reinforcement for target behavior increases the susceptibility of target behavior to relapse. To explore contingencies that might mitigate this counter-therapeutic effect, we trained pigeons on a procedure that entailed extinction of previously reinforced target-key pecking, access to a distinct stimulus context contingently on refraining from target behavior (differential-reinforcement-of-other-behavior; DRO), and reinforcement of alternative-key pecks (differential-reinforcement of alternative behavior; DRA) in that context. This DRO-DRA treatment was compared with standard DRA in successive conditions, counterbalanced across pigeons. Target behavior extinguished more rapidly in the Standard-DRA condition. When alternative reinforcement was discontinued, however, there was less resurgence after DRO-DRA than after Standard DRA. In a third condition, the DRO contingency was suspended so that the former DRA stimuli were not presented (DRO-NAC), and resurgence was greater than in the Standard-DRA and DRO-DRA conditions. Reinstatement produced by response-independent reinforcers was small and similar across conditions. Subsequent reacquisition of target-key pecking under baseline reinforcement conditions was faster following DRO-NAC than Standard-DRA or DRO-DRA. These findings suggest that DRO-DRA might serve as a useful method in clinical settings for reducing problem behavior while minimizing the threat of posttreatment relapse.

The basis of behavioral momentum in the nonlinearity of strength.

The persistence of operant responding in the context of distractors and opposing forces is of central importance to the success of behavioral interventions. It has been successfully analyzed with Behavioral Momentum Theory. Key data from the research inspired by that theory are reanalyzed in terms of more molecular behavioral mechanisms: the demotivational effects of disruptors, and their differential impacts on the target response and other responses that interact with them. Behavioral momentum is regrounded as a nonlinear effect of motivation and reinforcement rate on response probability and persistence. When response probabilities are high, more energy is required to further increase or to decrease them than when they are low. Classic Behavioral Momentum Theory effects are reproduced with this account. Finally, it is shown how the new account involving motivation and competition is closely related to the metaphor of force and action that is at the core of Behavioral Momentum Theory.

The Role of Contingencies and Stimuli in a Human Laboratory Model of Treatment of Problem Behavior.

Behavioral momentum theory posits a paradoxical implication for behavioral interventions in clinical situations using Differential Reinforcement of Alternative Behavior (DRA): When alternative reinforcers are presented within the same context as the problem behavior, the added reinforcers may decrease the frequency of the behavior but also increase its persistence when the intervention ends. Providing alternative reinforcers in a setting that is distinctively different from that in which the target behavior occurs may avoid or reduce this increase in persistence. The present experiment compared behavioral persistence following standard DRA versus DRA in a different context that was available after refraining from target behavior (Differential Reinforcement of Other Behavior, DRO). We arranged a human laboratory model of treatment intervention using computer games and token reinforcement. Participants were five individuals with intellectual disabilities. Experimental phases included (a) an initial multiple-schedule baseline with token reinforcement for target behaviors A and B, (b) an intervention phase with alternative reinforcement using a conventional DRA procedure for A and a DRO-DRA procedure for B, and (c) an extinction phase with no interventions and no tokens. Response rates as proportion of baseline in the initial extinction phase were greater for A than for B for three of five participants. Four participants whose response rates remained relatively high during the extinction phase then received a second extinction-plus-distraction test with leisure items available. Response rates were greater for A than for B in three of four participants. The results indicate that DRO-DRA contingencies may contribute to reduced post-intervention persistence of problem behavior.

Quantitative models of persistence and relapse from the perspective of behavioral momentum theory: Fits and misfits.

We review quantitative accounts of behavioral momentum theory (BMT), its application to clinical treatment, and its extension to post-intervention relapse of target behavior. We suggest that its extension can account for relapse using reinstatement and renewal models, but that its application to resurgence is flawed both conceptually and in its failure to account for recent data. We propose that the enhanced persistence of target behavior engendered by alternative reinforcers is limited to their concurrent availability within a distinctive stimulus context. However, a failure to find effects of stimulus-correlated reinforcer rates in a Pavlovian-to-Instrumental Transfer (PIT) paradigm challenges even a straightforward Pavlovian account of alternative reinforcer effects. BMT has been valuable in understanding basic research findings and in guiding clinical applications and accounting for their data, but alternatives are needed that can account more effectively for resurgence while encompassing basic data on resistance to change as well as other forms of relapse.

Comparing positive and negative reinforcement: A fantasy experiment.

We propose quantitative experimental approaches to the question of whether positive and negative reinforcement are functionally different, and discuss scientific and ethical concerns that would arise if these approaches were pursued.

Effects of signaled and unsignaled alternative reinforcement on persistence and relapse in children and pigeons.

Three experiments explored the impact of different reinforcer rates for alternative behavior (DRA) on the suppression and post-DRA relapse of target behavior, and the persistence of alternative behavior. All experiments arranged baseline, intervention with extinction of target behavior concurrently with DRA, and post-treatment tests of resurgence or reinstatement, in two- or three-component multiple schedules. Experiment 1, with pigeons, arranged high or low baseline reinforcer rates; both rich and lean DRA schedules reduced target behavior to low levels. When DRA was discontinued, the magnitude of relapse depended on both baseline reinforcer rate and the rate of DRA. Experiment 2, with children exhibiting problem behaviors, arranged an intermediate baseline reinforcer rate and rich or lean signaled DRA. During treatment, both rich and lean DRA rapidly reduced problem behavior to low levels, but post-treatment relapse was generally greater in the DRA-rich than the DRA-lean component. Experiment 3, with pigeons, repeated the low-baseline condition of Experiment 1 with signaled DRA as in Experiment 2. Target behavior decreased to intermediate levels in both DRA-rich and DRA-lean components. Relapse, when it occurred, was directly related to DRA reinforcer rate as in Experiment 2. The post-treatment persistence of alternative behavior was greater in the DRA-rich component in Experiment 1, whereas it was the same or greater in the signaled-DRA-lean component in Experiments 2 and 3. Thus, infrequent signaled DRA may be optimal for effective clinical treatment.

Modeling the effects of sensory reinforcers on behavioral persistence with alternative reinforcement.

Problem behavior often has sensory consequences that cannot be separated from the target response, even if external, social reinforcers are removed during treatment. Because sensory reinforcers that accompany socially mediated problem behavior may contribute to persistence and relapse, research must develop analog sensory reinforcers that can be experimentally manipulated. In this research, we devised analogs to sensory reinforcers in order to control for their presence and determine how sensory reinforcers may impact treatment efficacy. Experiments 1 and 2 compared the efficacy of differential reinforcement of alternative behavior (DRA) versus noncontingent reinforcement (NCR) with and without analog sensory reinforcers in a multiple schedule. Experiment 1 measured the persistence of key pecking in pigeons, whereas Experiment 2 measured the persistence of touchscreen responses in children with intellectual and developmental disabilities. Across both experiments, the presence of analog sensory reinforcers increased the levels, persistence, and variability of responding relative to when analog sensory reinforcers were absent. Also in both experiments, target responding was less persistent under conditions of DRA compared to NCR regardless of the presence or absence of analog sensory reinforcers.

Resistance to extinction and behavioral momentum.

In the metaphor of behavioral momentum, reinforcement is assumed to strengthen discriminated operant behavior in the sense of increasing its resistance to disruption, and extinction is viewed as disruption by contingency termination and reinforcer omission. In multiple schedules of intermittent reinforcement, resistance to extinction is an increasing function of reinforcer rate, consistent with a model based on the momentum metaphor. The partial-reinforcement extinction effect, which opposes the effects of reinforcer rate, can be explained by the large disruptive effect of terminating continuous reinforcement despite its strengthening effect during training. Inclusion of a term for the context of reinforcement during training allows the model to account for a wide range of multiple-schedule extinction data and makes contact with other formulations. The relation between resistance to extinction and reinforcer rate on single schedules of intermittent reinforcement is exactly opposite to that for multiple schedules over the same range of reinforcer rates; however, the momentum model can give an account of resistance to extinction in single as well as multiple schedules. An alternative analysis based on the number of reinforcers omitted to an extinction criterion supports the conclusion that response strength is an increasing function of reinforcer rate during training.

Comment on Gallistel: behavior theory and information theory: some parallels.

In this article, Gallistel proposes information theory as an approach to some enduring problems in the study of operant and classical conditioning.

Response strength in extreme multiple schedules.

Four pigeons were trained in a series of two-component multiple schedules. Reinforcers were scheduled with random-interval schedules. The ratio of arranged reinforcer rates in the two components was varied over 4 log units, a much wider range than previously studied. When performance appeared stable, prefeeding tests were conducted to assess resistance to change. Contrary to the generalized matching law, logarithms of response ratios in the two components were not a linear function of log reinforcer ratios, implying a failure of parameter invariance. Over a 2 log unit range, the function appeared linear and indicated undermatching, but in conditions with more extreme reinforcer ratios, approximate matching was observed. A model suggested by McLean (1991), originally for local contrast, predicts these changes in sensitivity to reinforcer ratios somewhat better than models by Herrnstein (1970) and by Williams and Wixted (1986). Prefeeding tests of resistance to change were conducted at each reinforcer ratio, and relative resistance to change was also a nonlinear function of log reinforcer ratios, again contrary to conclusions from previous work. Instead, the function suggests that resistance to change in a component may be determined partly by the rate of reinforcement and partly by the ratio of reinforcers to responses.

Delayed matching to sample: reinforcement has opposite effects on resistance to change in two related procedures.

The effects of reinforcement on delayed matching to sample (DMTS) have been studied in two within-subjects procedures. In one, reinforcer magnitudes or probabilities vary from trial to trial and are signaled within trials (designated signaled DMTS trials). In the other, reinforcer probabilities are consistent for a series of trials produced by responding on variable-interval (VI) schedules within multiple-schedule components (designated multiple VI DMTS). In both procedures, forgetting functions in rich trials or components are higher than and roughly parallel to those in lean trials or components. However, during disruption, accuracy has been found to decrease more in rich than in lean signaled DMTS trials and, conversely, to decrease more in lean than in rich multiple VI DMTS components. In the present study, we compared these procedures in two groups of pigeons. In baseline, forgetting functions in rich trials or components were higher than and roughly parallel to those in lean trials or components, and were similar between the procedures. During disruption by prefeeding or extinction, accuracy decreased more in rich signaled DMTS trials, whereas accuracy decreased more in lean multiple VI DMTS components. These results replicate earlier studies and are predicted by a model of DMTS from Nevin, Davison, Odum, and Shahan (2007).

An evaluation of persistence of treatment effects during long-term treatment of destructive behavior.

Eight young children who displayed destructive behavior maintained, at least in part, by negative reinforcement received long-term functional communication training (FCT). During FCT, the children completed a portion of a task and then touched a communication card attached to a microswitch to obtain brief breaks. Prior to and intermittently throughout FCT, extinction probes were conducted within a withdrawal design in which task completion, manding, and destructive behavior were placed on extinction to evaluate the relative persistence of appropriate and destructive behavior over the course of treatment. FCT continued until appropriate behavior persisted and destructive behavior failed to recur at baseline levels during extinction probes. The completion of FCT was followed by four challenges to the persistence of treatment effects conducted within mixed- or multiple-schedule designs: (a) extended extinction sessions (from 5 to 15 min), (b) introduction of a novel task, (c) removal of the microswitch and communication card, and (d) a mixed schedule of reinforcement in which both appropriate and destructive behavior produced reinforcement. The results showed that although FCT often resulted in quick reductions in destructive behavior and increases in appropriate behavior, destructive behavior often recurred during the extinction probes conducted during the initial treatment. When the effects of treatment persisted during the extinction probes, the remaining challenges to treatment effects resulted in only mild to moderate disruptions in behavior. These results are consistent with the quantitative predictions of behavioral momentum theory and may provide an alternative definition of maintenance as constituting behavioral persistence.

Behavioral momentum theory: equations and applications.

Behavioral momentum theory provides a quantitative account of how reinforcers experienced within a discriminative stimulus context govern the persistence of behavior that occurs in that context. The theory suggests that all reinforcers obtained in the presence of a discriminative stimulus increase resistance to change, regardless of whether those reinforcers are contingent on the target behavior, are noncontingent, or are even contingent on an alternative behavior. In this paper, we describe the equations that constitute the theory and address their application to issues of particular importance in applied settings. The theory provides a framework within which to consider the effects of interventions such as extinction, noncontingent reinforcement, differential reinforcement of alternative behavior, and other phenomena (e.g., resurgence). Finally, the theory predicts some counterintuitive and potentially counterproductive effects of alternative reinforcement, and can serve as an integrative guide for intervention when its terms are identified with the relevant conditions of applied settings.

The effects of constant versus varied reinforcers on preference and resistance to change.

Previous research has demonstrated that factors such as reinforcer frequency, amount, and delay have similar effects on resistance to change and preference. In the present study, 4 boys with autism made choices between a constant reinforcer (one that was the same food item every trial) and a varied food reinforcer (one that varied randomly between three possible food items). For all 4 boys, varied reinforcers were preferred over constant reinforcers, and they maintained higher response rates than constant reinforcers. In addition, when a distraction (a video clip) was introduced, responding maintained by varied reinforcers was more resistant to distraction than responding maintained by constant reinforcers. Thus, the present experiment extended the generality of the relation between preference and resistance to change to variation in reinforcer quality.

Differential outcomes enhance accuracy of delayed matching to sample but not resistance to change.

Three experiments assessed the relation between the differential outcomes effect and resistance to change of delayed matching-to-sample performance. Pigeons produced delayed matching-to-sample trials by responding on variable interval schedules in two components of a multiple schedule. In the same-outcome component, the probability of reinforcement was the same for both samples (.9 in Experiments 1 and 2, .5 in Experiment 3); in the different-outcomes component, the probability of reinforcement was .9 for one sample and .1 for the other. In all three experiments, the forgetting functions in the different-outcomes component were higher and shallower than in the same-outcomes component. When total reinforcement was greater in the same-outcomes component (Experiments 1 and 2), resistance to disruption by prefeeding, intercomponent food, extinction, or flashing lights typically was greater in that component. In Experiment 3, when total reinforcement was equated, resistance to disruption was similar across components. Thus, the level and slope of forgetting functions depended on differential reinforcement correlated with the samples, but the resistance to change of forgetting functions depended on total reinforcement in a component. Both aspects of the results can be explained by a model of delayed matching to sample performance.

Stimuli, reinforcers, and the persistence of behavior.

This article reviews evidence from basic and translational research with pigeons and humans suggesting that the persistence of operant behavior depends on the contingency between stimuli and reinforcers, and considers some implications for clinical interventions.

Contrast effects in response rate and accuracy of delayed matching to sample.

Behavioural contrast is an inverse relation between the response rate in one component of a multiple schedule and the reinforcer rate in an alternated component. To explore possible contrast effects in accuracy as well as response rate, four pigeons were trained in multiple schedules where key pecking produced delayed matching-to-sample trials on a variable-interval schedule. Reinforcer probability for correct matches was constant at .3 in one component, and the conditions of reinforcement were varied in the second component. In Experiment 1, the varied component arranged the same contingencies as the constant component but with reinforcer probabilities of .9 or .1 across conditions. In the varied component, both response rate and accuracy of delayed matching were directly related to reinforcer probability; in the constant component, however, contrast effects on response rate were weak, and there was no evidence of contrast in accuracy of matching. In Experiment 2, the varied component was either variable interval with immediate food reinforcement or extinction. Reliable contrast effects were obtained in both response rate and in accuracy of matching in the constant component, and their magnitudes were correlated within and between subjects. The results of Experiment 2 join previous findings of covariation in the effects of reinforcement on free-operant responding and accuracy of discrimination.

Control, prediction, order, and the joys of research.

JEAB's first decade featured control of individual behavior by operant contingencies, where the experimenter could interact with a subject in real time and obtain fairly immediate evidence of control, with the possibility of direct extension to applied settings. In subsequent decades, emphasis shifted toward long-term parametric studies with quantitative analyses of individual and group data. As a result, the reinforcers for the experimenter shifted from controlling behavior to uncovering and describing order, often using mathematical expressions. The same sorts of reinforcers are available for quantitative descriptions of aggregate behavioral data that may inform public policy.

Stimuli, reinforcers, and private events.

Radical behaviorism considers private events to be a part of ongoing observable behavior and to share the properties of public events. Although private events cannot be measured directly, their roles in overt action can be inferred from mathematical models that relate private responses to external stimuli and reinforcers according to the same quantitative relations that characterize public operant behavior. This approach is illustrated by a model of attending to stimuli and to anticipated reinforcers in delayed matching to sample, in which the probabilities of attending are related to reinforcer rates by an expression derived from research on behavioral momentum.