Boundary conditions of post-retrieval extinction: A direct comparison of low and high partial reinforcement.

Research has shown that a single presentation of the conditioned stimulus prior to extinction training can diminish conditioned responses. However, replication has proven difficult and appears to be limited by boundary conditions. Here we tested the boundary condition of memory strength by comparing the effect of reinforcement rate to assess its role in post-retrieval extinction. Eighty university students had undergone a three-day fear conditioning experiment in which two partial reinforcement schedules (40%, 80%) were applied. The findings indicated that both low and high partial reinforcement groups did not demonstrate recovery of conditioned responses after post-retrieval extinction. In contrast, both groups demonstrated significant recovery to standard extinction with significantly greater recovery in the 80% group relative to the 40% group. Additionally, we found that greater physiological arousal during memory retrieval significantly predicted recovery of fear at test phase. We conclude that when compared to a lower partial reinforcement schedule, a higher partial reinforcement resulted in the formation of a stronger memory as indicated by greater physiological arousal during memory reactivation and recovery of conditioned responses after standard extinction, but that it does not function as a boundary condition of post-retrieval extinction. These data are significant because it is the first study to investigate the effect of varying partial reinforcement schedules on fear recovery and add to the body of literature that continue to identify sources of failure in the application of post-retrieval extinction.

Effects of string length on the organization of rat string-pulling behavior.

The string-pulling paradigm has been adapted to investigate many psychological phenomena across a range of animal species. Although varying string length has been shown to influence performance, the nature of the representation remains to be determined. Across three experiments, rats were shaped to pull string to receive food reinforcement. Either string length or reinforcement rate was manipulated to examine the influence on string-pulling behavior. Experiment 1 demonstrated that varied string length was sufficient to elicit an odor discrimination. Subsequent experiments provided evidence that varying string length (Experiment 2) and reinforcement rate (Experiment 3) produced qualitatively distinct patterns of string-pulling behavior. In Experiment 2 rats that received a long string were more likely to pull in the probe string to the end, yet no differences were observed in approach time between short and long groups. However, in Experiment 3 rats that received low reinforcement were less likely to pull in the probe string to the end and were slower to approach the string to begin pulling. These results are consistent with rats using temporal and motivational characteristics to guide responding during string-pulling behavior.

Losses Disguised as Wins Affect Game Selection on Multiline Slots.

Multiline slots are exciting games that contain features which make them alluring. One such feature is a loss disguised as a win (LDW); wherein, players win less than they wager (e.g., bet 2 dollars, win back 50 cents), but this net loss is disguised by flashing graphics and winning sounds. Research to date concludes that LDWs are both rewarding and reinforcing. Here, we investigated whether LDWs affect players' game selection. Thirty-two undergraduate students with experience playing slot machines played 100 spins on four games-two had positive payback percentages (115%) and two had negative payback percentages (85%) after 100 spins. For each payback percentage condition, there was a game with no LDWs and a game with a moderate number of LDWs. For the 100 spins, players could choose to play whichever game they wished. They then rated their preference for each game following the 100-spins and chose a game to continue playing. The majority of players preferred playing the positive payback percentage game with LDWs and chose to continue playing this game over the three other games. We conclude that in addition to LDWs being reinforcing and rewarding, LDWs do in fact influence game selection. We conclude that responsible gambling initiatives should educate players about LDWs.

Effects of inter-food interval on the variety effect in an instrumental food-seeking task. Clarifying the role of habituation.

Food variety increases consumption and the rate of instrumental behavior that is reinforced by food in humans and animals. The present experiment investigated the relationship between the variety effect and habituation to food by testing the role of the interval between successive food presentations on responding in an operant food-seeking task. Habituation to food was expected at short, but not long, interfood intervals. The effects of variety on food's long-term reinforcing value were also tested. Four groups of rats were trained to lever-press on different random-interval (RI) schedules of reinforcement to earn 45-mg food pellets. Half the rats in each group received an unpredictable mix of grain and sucrose pellets, while the other half consistently received sucrose pellets. Response rate began at a high rate and then decreased within each 30-min session for groups that received short inter-pellet intervals (i.e., RI-3 s and RI-6 s reinforcement schedules) but not in groups that received longer inter-pellet intervals (i.e., RI-12 s and RI-24 s). A variety effect in the form of higher responding in the mix group than the sucrose-only group was also only evident at the shorter intervals. Habituation and variety effects were also most evident with the short intervals when we controlled for the number of reinforcers earned, suggesting that they were not merely due to rapid satiation. The variety effect also appeared quickly when groups trained with longer inter-pellet intervals (RI-12 s and RI-24 s) were transitioned to shorter intervals (RI-3 s and RI-6 s). There was no effect of variety on resistance to extinction or on resistance to the response-suppressing effects of pre-session feeding. The results more clearly link this version of the variety effect to the short-term effect of variety on food habituation.

Response-dependent point loss and response force as disrupting operations on behavioral resistance to change in humans.

Behavioral momentum theory (BMT) provides a theoretical and methodological framework for understanding how differentially maintained operant responding resists disruption. A common way to test operant resistance involves contingencies with suppressive effects, such as extinction or prefeeding. Other contingencies with known suppressive effects, such as response-cost procedures arranged as point-loss or increases in response force, remain untested as disruptive events within the BMT framework. In the present set of three experiments, responding of humans was maintained by point accumulation programmed according to a multiple variable-interval (VI) VI schedule with different reinforcement rates in either of two components. Subsequently, subtracting a point following each response (Experiment 1) or increasing the force required for the response to be registered (Experiments 2 and 3 decreased response rates, but responding was less disrupted in the component associated with the higher reinforcement rate. The point-loss contingency and increased response force similarly affected response rates by suppressing responding and human persistence, replicating previous findings with humans and nonhuman animals when other types of disruptive events (e.g., extinction and prefeeding) were investigated. The present findings moreover extend the generality of the effects of reinforcement rate on persistence, and thus BMT, extending the analysis of resistance to two well-known manipulations used to reduce responding in the experimental analysis of behavior.

Resurgence of destructive behavior following differential rates of alternative reinforcement.

Behavioral momentum theory (BMT) suggests that resurgence of destructive behavior may be at least partly determined by the rate of alternative reinforcement, with lean schedules of reinforcement producing less resurgence than dense schedules. Findings from basic and translational studies have been mixed, and the effects of alternative reinforcement rate on resurgence remain unclear. In the current study, we conducted a within-subject evaluation of resurgence during extinction with four children following functional communication training using dense and lean (BMT-informed) schedules of alternative reinforcement. We observed no reliable differences in resurgence across the dense and lean conditions. We discuss implications of these findings in relation to future research using quantitative analyses to evaluate the relative effects of alternative reinforcement rate and other BMT-based strategies for mitigating resurgence in applied settings.

Relationship between reinforcement rate and response rate in Pavlovian and operant conditioning with compound stimuli in rats.

Harris, Andrew, and Livesey (2012) proposed that response rate (R) to a stimulus (e.g., A) of a compound (e.g., AB) would be directly related to the difference between reinforcement rate (r) during AB and the other stimulus (e.g., B). To explore the extent of this proposal to operant conditioning, three experiments were carried out. Experiment 1 replicated Harris et al. (2012 - Experiment 1) using sucrose as US in Pavlovian conditioning. In Experiment 2 rats pressed a lever under a variable interval schedule in two phases. In Training, responses during a compound stimulus AB100% were reinforced 100%, responses during B50% and C50% 50% and responses during D25%, 25% of the trials. Then, nonreinforced A trials were introduced (Probe). Results from Experiment 2 showed that r controlled the R to AB100%, B50%, C50% and D25% during training and probe, however, were inconclusive about r and R relationship during A. In Experiment 3, responses during A100% were reinforced 100% and during B50% and C50% 50% of the trials (Training). Next, nonreinforced BC trials were presented (Probe). Results from Experiment 3 showed that response rate during BC were similar to A100%, but higher than B50% and C50%, suggesting summation of B50% and C50% reinforcement rates. Overall, the results showed that Harris et al. (2012) proposal extends to discriminative operant relations.

A procedural analogue of prey detection and applied signal detection.

Researchers have employed a variety of laboratory analogues of cryptic prey detection and applied signal detection to study factors influencing learning and performance in these ethological and applied scenarios. However, these procedural analogues do not appear to map closely onto their "real-world" counterparts, particularly with respect to the role of the "yes" (i.e., "attack") response and the payoff for this response (or its absence) on signal-present and signal-absent trials. Using domestic hens, we developed a procedural analogue in which a "yes" response requires some time to emit; such responses were reinforced only in the presence of a signal. In Experiment 1, we evaluated the influence of the "yes" response requirement by manipulating the number of responses required to qualify as a "yes" response. As the "yes" response requirement was increased, bias toward responding "no" increased, revealing that this is a critical factor controlling accuracy in this procedure. In Experiment 2, we evaluated the influence of signal probability and reinforcement rate on signal detection accuracy and found that neither of these factors significantly influenced accuracy or bias. These findings suggest that this procedural analogue may represent a valuable alternative for studying behaviour in relevant signal detection scenarios.

Negative reinforcement rate and persistent avoidance following response-prevention extinction.

Persistent avoidance may be influenced by prior negative reinforcement rate (i.e., how effective the response is at controlling threat). In clinical settings, the effectiveness of extinction-based methods for treating anxiety-related avoidance may be impacted by prior reinforcement rate. Here, we conducted a laboratory-based treatment study to investigate the persistence of avoidance following response-prevention extinction (RPE) when prior avoidance had been differentially effective at cancelling shock. Participants in three negative reinforcement rate groups (100%, 50%, and 0%) completed a validated avoidance conditioning paradigm involving Pavlovian fear extinction, RPE, and re-extinction phases. It was hypothesised that partially reinforced avoidance would lead to diminished resistance to fear extinction following response prevention, compared to continuously- or never-reinforced avoidance. Persistent avoidance was related to prior negative reinforcement rate, with higher rates more resistant to extinction. These findings illustrate the role of reinforcement rate in the persistence of avoidance and may aid understanding of treatment relapse.

The effect of reinforcement probability on time discrimination in the midsession reversal task.

We examined how biasing time perception affects choice in a midsession reversal task. Given a simultaneous discrimination between stimuli S1 and S2, with choices of S1 reinforced during the first, but not the second half of the trials, and choices of S2 reinforced during the second, but not the first half of the trials, pigeons show anticipation errors (premature choices of S2) and perseveration errors (belated choices of S1). This suggests that choice depends on timing processes, on predicting when the contingency reverses based on session duration. We exposed 7 pigeons to a midsession reversal task and manipulated the reinforcement rate on each half of the session. Compared to equal reinforcement rates on both halves of the session, when the reinforcement rate on the first half was lower than on the second half, performance showed more anticipation and less perseveration errors, and when the reinforcement rate on the first half was higher than on the second half, performance showed a remarkable reduction of both types of errors. These results suggest that choice depends on both time into the session and the outcome of previous trials. They also challenge current models of timing to integrate local effects.

Repeated, within-session resurgence.

Resurgence is a reliable, transient effect that only occasionally is replicated more than once within a single experiment or subject. In the present experiments, within-session resurgence was generated repeatedly by dividing individual sessions into three phases (Training, Alternative-Reinforcement, and Resurgence-Test). In Experiments 1 and 2, resurgence reliably occurred in most of the 22-30 daily sessions when responding was reinforced on, respectively, fixed- and variable-interval schedules. Resurgence magnitude and duration did decrease across replications for some subjects, but not for others. To examine the utility of the procedure in studying the effects of an independent variable on resurgence, in Experiment 3 the effects of rich and lean baseline and alternative reinforcement rates on resurgence were compared. The target response was eliminated more rapidly, resurgence occurred more often, and usually was greater following rich alternative reinforcement rates. Resurgence was of greater magnitude when the baseline reinforcement rate was relatively lean compared to the alternative reinforcement rate. These experiments provide a reliable method for generating resurgence within individual sessions, instead of across multiple-session conditions, that can be repeated over many successive sessions.

Baseline reinforcement rate and resurgence of destructive behavior.

Concepts from behavioral momentum theory, along with some empirical findings, suggest that the rate of baseline reinforcement may contribute to the relapse of severe destructive behavior. With seven children who engaged in destructive behavior, we tested this hypothesis in the context of functional communication training by comparing the effects of different baseline reinforcement rates on resurgence during a treatment challenge (i.e., extinction). We observed convincing resurgence of destructive behavior in four of seven participants, and we observed more resurgence in the condition associated with high-rate baseline reinforcement (i.e., variable-interval 2 s in Experiment 1 or fixed-ratio 1 in Experiment 2) compared to a low-rate baseline reinforcement condition. We discuss the implications of these results relative to schedules of reinforcement in the treatment of destructive behavior and strategies to mitigate resurgence in clinical settings.

Greater reinforcement rate during training increases spontaneous recovery.

Spontaneous recovery occurs when a previously reinforced and recently extinguished response reemerges over the course of time, often at the beginning of a new session of extinction. Spontaneous recovery could underlie instances of treatment relapse that threaten otherwise effective behavioral interventions for problem behavior. In two experiments, we arranged multiple schedules with pigeons and a human child to assess the effects of different training reinforcer rates on spontaneous recovery. In both experiments, responding was both more resistant to extinction and more likely to relapse following training with greater reinforcement rates upon returning to extinction after time off from extinction testing. A quantitative model based on behavioral momentum theory accounted well for the data, which suggests reexposure to the extinction context following time off during extinction resulted in (1) the failure of extinction learning to generalize, and (2) greater generalization of original learning during training. The present model attempts to quantify theories attributing spontaneous recovery to changes in temporal context.

Behavioral momentum theory fails to account for the effects of reinforcement rate on resurgence.

The behavioral-momentum model of resurgence predicts reinforcer rates within a resurgence preparation should have three effects on target behavior. First, higher reinforcer rates in baseline (Phase 1) produce more persistent target behavior during extinction plus alternative reinforcement. Second, higher rate alternative reinforcement during Phase 2 generates greater disruption of target responding during extinction. Finally, higher rates of either reinforcement source should produce greater responding when alternative reinforcement is suspended in Phase 3. Recent empirical reports have produced mixed results in terms of these predictions. Thus, the present experiment further examined reinforcer-rate effects on persistence and resurgence. Rats pressed target levers for high-rate or low-rate variable-interval food during Phase 1. In Phase 2, target-lever pressing was extinguished, an alternative nose-poke became available, and nose-poking produced either high-rate variable-interval, low-rate variable-interval, or no (an extinction control) alternative reinforcement. Alternative reinforcement was suspended in Phase 3. For groups that received no alternative reinforcement, target-lever pressing was less persistent following high-rate than low-rate Phase-1 reinforcement. Target behavior was more persistent with low-rate alternative reinforcement than with high-rate alternative reinforcement or extinction alone. Finally, no differences in Phase-3 responding were observed for groups that received either high-rate or low-rate alternative reinforcement, and resurgence occurred only following high-rate alternative reinforcement. These findings are inconsistent with the momentum-based model of resurgence. We conclude this model mischaracterizes the effects of reinforcer rates on persistence and resurgence of operant behavior.

Higher rate alternative non-drug reinforcement produces faster suppression of cocaine seeking but more resurgence when removed.

Relapse following removal of an alternative source of reinforcement introduced during extinction of a target behavior is called resurgence. This form of relapse may be related to relapse of drug taking following loss of alternative non-drug reinforcement in human populations. Laboratory investigations of factors mediating resurgence with food-maintained behavior suggest higher rates of alternative reinforcement produce faster suppression of target behavior but paradoxically generate more relapse when alternative reinforcement is discontinued. At present, it is unknown if a similar effect occurs when target behavior is maintained by drug reinforcement and the alternative is a non-drug reinforcer. In the present experiment three groups of rats were trained to lever press for infusions of cocaine during baseline. Next, during treatment, cocaine reinforcement was suspended and an alternative response was reinforced with either high-rate, low-rate, or no alternative food reinforcement. Finally, all reinforcement was suspended to test for relapse of cocaine seeking. Higher rate alternative reinforcement produced faster elimination of cocaine seeking than lower rates or extinction alone, but when treatment was suspended resurgence of cocaine seeking occurred following only high-rate alternative reinforcement. Thus, although higher rate alternative reinforcement appears to more effectively suppress drug seeking, should it become unavailable, it can have the unfortunate effect of increasing relapse.

Resistance to change and resurgence in humans engaging in a computer task.

The relation between persistence, as measured by resistance to change, and resurgence has been examined with nonhuman animals but not systematically with humans. The present study examined persistence and resurgence with undergraduate students engaging in a computer task for points exchangeable for money. In Phase 1, a target response was maintained on a multiple variable-interval (VI) 15-s (Rich) VI 60-s (Lean) schedule of reinforcement. In Phase 2, the target response was extinguished while an alternative response was reinforced at equal rates in both schedule components. In Phase 3, the target and the alternative responses were extinguished. In an additional test of persistence (Phase 4), target responding was reestablished as in Phase 1 and then disrupted by access to videos in both schedule components. In Phases 2 and 4, target responding was more persistent in the Rich than in the Lean component. Also, resurgence generally was greater in the Rich than in the Lean component in Phase 3. The present findings with humans extend the generality of those obtained with nonhuman animals showing that higher reinforcement rates produce both greater persistence and resurgence, and suggest that common processes underlie response persistence and relapse.

Relatively high motivation for context-evoked reward produces the magnitude effect in rats.

Using a concurrent-chain schedule, we demonstrated the effect of absolute reinforcement (i.e., the magnitude effect) on choice behavior in rats. In general, animals' simultaneous choices conform to a relative reinforcement ratio between alternatives. However, studies in pigeons and rats have found that on a concurrent-chain schedule, the overall reinforcement ratio, or absolute amount, also influences choice. The effect of reinforcement amount has also been studied in inter-temporal choice situations, and this effect has been referred to as the magnitude effect. The magnitude effect has been observed in humans under various conditions, but little research has assessed it in animals (e.g., pigeons and rats). The present study confirmed the effect of reinforcement amount in rats during simultaneous and inter-temporal choice situations. We used a concurrent-chain procedure to examine the cause of the magnitude effect during inter-temporal choice. Our results suggest that rats can use differences in reinforcement amount as a contextual cue during choice, and the direction of the magnitude effect in rats might be similar to humans when using the present procedure. Furthermore, our results indicate that the magnitude effect was caused by the initial-link effect when the reinforcement amount was relatively small, while a loss aversion tendency was observed when the reinforcement amount changed within a session. The emergence of the initial-link effect and loss aversion suggests that rats make choices through cognitive processes predicted by prospect theory.

Response elimination, reinforcement rate and resurgence of operant behavior.

The effects of reinforcement rate of alternative responding on resurgence were studied in six experiments with pigeons. In Experiment 1A, key pecking was maintained on a multiple variable-interval (VI) VI schedule in the Training phase. In the Response-Elimination phase, a variable differential-reinforcement-of-other-behavior (DRO) schedule was in effect in each component. Reinforcement rates were equal and then, higher in one (rich) component, and lower in the other (lean), than in the Training phase. More resurgence occurred in the lean component, but this could have resulted from response-rate differences between components in the Training-phase. Experiment 1B was a replication of Experiment 1A, but with experimentally-naïve pigeons. Response-Elimination phase reinforcement rates were manipulated systematically in subsequent experiments: In Experiment 2, reinforcement rate was equal, in one component, and lower or higher in the other, than in the Training phase. In Experiment 3, reinforcers were discontinued before differential reinforcement rates were effected. In Experiment 4, reinforcement rates first were differential and, then, equal to those in the Training phase. In Experiments 5 and 6, differential reinforcement rates were arranged by using fixed-DROs and VIs for pecking a different key, respectively. Even though resurgence was not obtained with every pigeon, at least some small-magnitude resurgence occurred in each experiment and was not related systematically to reinforcement rates of alternative responding. Schedule differences, response topography, order of conditions and the length of each phase were not sufficient to account for these results.

Delayed reinforcement and fixed-ratio performance.

Effects of delayed reinforcement on fixed-ratio (FR) maintained responding of pigeons were investigated. In Experiments 1-3, the delay of reinforcement was increased across successive sessions until pigeons paused for 300 s. Both signaled and unsignaled delays were studied across different conditions. Overall response rates and run rates (timed from the first to last response of a ratio) decreased and postreinforcement pauses increased as delays increased in each experiment. As delays increased, the likelihood of pausing during a ratio run also increased. When these measures were plotted as a function of obtained delays, signaled delays had less of an effect on the above measures than did unsignaled ones. In Experiment 2, delays had a greater effect on the above measures than did a control condition arranging equivalent interreinforcer intervals to those accompanying the delays. Experiments 3 and 4 examined the generality of the effects obtained in the first two experiments. In Experiment 3, delays imposed on FR or yoked-interval schedules had similar behavioral effects. In Experiment 4, effects similar to those found in Experiments 1-3 for 1, 10, and 20-s delays imposed on FR 50 schedules were found when the FR requirement increased across sessions. Despite the different contingencies relating response rate and reinforcement rates on interval and ratio schedules, delays of reinforcement generally affect performance on these schedules similarly.

Rate, probability and matching: comments on "The identities hidden in the matching laws, and their uses" by David Thorne.

David Thorne's (2010) article, "The identities hidden in the matching laws, and their uses" performs a valuable service in pointing out alternative expressions of matching. However, some identities tend to obscure rather than illuminate empirical relationships. Three such problematic instances are discussed: interresponse time as a function of interval and ratio schedule parameters; probability equality as implying rate matching; the apparent simplicity of probabilistic functions, as opposed to response rate functions, of reinforcement rate.