Longer operant lever-press duration requirements induce fewer but longer response bouts in rats.

Operant behavior is organized in bouts that are particularly visible under variable-interval (VI) schedules of reinforcement. Previous research showed that increasing the work required to produce a response decreases the rate at which bouts are emitted and increases the minimum interresponse time (IRT). In the current study, the minimum effective IRT was directly manipulated by changing the minimum duration of effective lever presses reinforced on a VI 40-s schedule. Contrary to assumptions of previous models, response durations were variable. Response durations were typically 0.5 s greater than the minimum duration threshold; durations that exceeded this threshold were approximately log-normally distributed. As the required duration threshold increased, rats emitted fewer but longer bouts. This effect may reflect an effort-induced reduction in motivation and a duration-induced facilitation of a response-outcome association.

Development and characterization of an automated behavioral assessment platform for the Göttingen minipig.

The Göttingen minipig is fast becoming the standard for assessing dermal chemical hazards because, like most swine, its skin is predictive of human skin response and because this strain's smaller size makes laboratory manipulations and husbandry easier. Unfortunately, standard behavioral tests and apparatus have not been developed for behavioral assessments of this swine strain. Indeed, computer-controlled automated behavioral testing procedures are much needed. The present research advanced this goal by producing a home-cage behavioral testing system that could accommodate minipigs of various sizes (ages). An aluminum frame housed three levers for recording operant responses, and LEDs above and below each lever served as discriminative stimuli. A commercially available food pellet dispenser was attached to a specialized pellet receptacle capable of measuring pellet retrieval. Two behavioral tests were selected and adapted from our commonly used non-human primate behavioral assessments: delayed match-to-sample (a memory test) and temporal response differentiation (a time-estimation test). Minipigs were capable of learning both tests and attaining stable performance. Next, scopolamine was used to validate the sensitivity of the behavioral tests for gauging behavioral perturbations in this swine strain. Scopolamine dose-effect functions were comparable to those observed in other species, including non-human primates, wherein 37.5 µg/kg of scopolamine (administered intramuscularly) reduced responding approximately 50%. Thus, we were successful in developing the apparatus and automated operant behavioral tests necessary to characterize drug safety in this swine strain. This capability will be valuable for characterizing chemical agent toxicity as well as the safety and efficacy of medical countermeasures.

Reinforced behavioral variability in the valproate rat model of autism spectrum disorder.

Individuals diagnosed with autism spectrum disorder (ASD) tend to display restricted, repetitive behaviors and deficits in social interaction. Rats exposed to valproate (VPA) in utero have been shown to model symptoms of ASD. In previous research, VPA rats engaged in less social interaction and more repetitive responding than controls. The purpose of the present study was to further investigate behavioral variability in the VPA rat model of ASD by testing VPA and control rats in a reinforced-behavioral-variability operant task. In this procedure, rats emitted sequences of lever presses, some of which produced food. During baseline, food was delivered probabilistically, and variability was not required. Next, rats were exposed either to a variability contingency, in which food was only delivered following sequences that differed sufficiently from previous sequences (i.e., variability required), or to a yoked contingency, in which variability was not required. We hypothesized that VPA rats would behave less variably than controls in this task. However, VPA and control rats responded similarly variably when variability was required. Furthermore, VPA rats behaved slightly more variably than controls during baseline and yoked conditions, when variability was not required. These findings contribute to the complex literature surrounding the VPA rat model of ASD.

Effects of environmental enrichment on exploratory behavior, win-stay and lose-shift performance, motor sequence learning, and reversal learning during the three-lever operant task in mice.

Beneficial effects of environmental enrichment (EE) on the central nervous system have been demonstrated. Although the effects of EE on spatial learning have been extensively studied, studies on reward-based motor learning are limited. In this study we examined the effects of EE on the performance of operant tasks using three levers (A-C). Mice were divided into two groups and housed either in the control condition or in the physical EE condition. The mice were trained in three types of operant tasks in sequence. First, mice were trained to press one of the active levers for a food reward (one-lever task). Second, mice were trained to press the three levers in the order of A, B, and C (three-lever task). Third, the lever order was reversed to C, B, and A (reverse three-lever task). We found some behavioral differences between control and EE mice. When all three levers were active in the one-lever task, mice tended to press the three levers equally at first, then shifted to press one lever preferentially. This behavioral shift from exploration to exploitation was delayed in EE mice. When only one lever was active, EE mice showed a higher lose-shift performance. In the three-lever and reverse three-lever tasks, EE mice pressed three levers more often and acquired more food rewards, compared to control mice, although the success rate in both tasks was not different between the two groups. These behavioral features observed in EE mice (higher lose-shift performance and higher trial and error activity) might be advantageous when circumstances are not stable.

Social preference in rats.

Rats were given repeated choices between social and nonsocial outcomes, and between familiar and unfamiliar social outcomes. Lever presses on either of 2 levers in the middle chamber of a 3-chamber apparatus opened a door adjacent to the lever, permitting 45-s access to social interaction with the rat in the chosen side chamber. In Experiment 1, rats preferred (a) social over nonsocial options, choosing their cagemate rat over an empty chamber, and (b) an unfamiliar over a familiar rat, choosing a non-cagemate over their cagemate. These findings were replicated in Experiment 2 with 2 different non-cagemate rats. Rats preferred both non-cagemate rats to a similar degree when pitted against their cagemate, but were indifferent when the 2 non-cagemates were pitted against each other. Similar preference for social over nonsocial and non-cagemate over cagemate was seen in Experiment 3, with new non-cagemate rats introduced after every third session. Response rates (for both cagemate and non-cagemate rats) were elevated under conditions of nonsocial (isolated) housing compared to conditions of social (paired) housing, demonstrating a social deprivation effect. Together, the experiments contribute to an experimental analysis of social preference within a social reinforcement framework, drawing on methods with proven efficacy in the analysis of reinforcement more generally.

Failure to Find Altruistic Food Sharing in Rats.

Prior research has found that one rat will release a second rat from a restraint in the presence of food, thereby allowing that second rat access to food. Such behavior, clearly beneficial to the second rat and costly to the first, has been interpreted as altruistic. Because clear demonstrations of altruism in rats are rare, such findings deserve a careful look. The present study aimed to replicate this finding, but with more systematic methods to examine whether, and under what conditions, a rat might share food with its cagemate partner. Rats were given repeated choices between high-valued food (sucrose pellets) and 30-s social access to a familiar rat, with the (a) food size (number of food pellets per response), and (b) food motivation (extra-session access to food) varied across conditions. Rats responded consistently for both food and social interaction, but at different levels and with different sensitivity to the food-access manipulations. Food production and consumption was high when food motivation was also high (food restriction) but substantially lower when food motivation was low (unlimited food access). Social release occurred at moderate levels, unaffected by the food-based manipulations. When food was abundant and food motivation low, the rats chose food and social options about equally often, but sharing (food left unconsumed prior to social release) occurred at low levels across sessions and conditions. Even under conditions of low food motivation, sharing occurred on only 1% of the sharing opportunities. The results are therefore inconsistent with claims in the literature that rats are altruistically motivated to share food with other rats.

Blunted satiety in fatty Zucker rats.

Levels of weight gain have hit an epidemic level with rates of overweight and obesity diagnoses topping all-time highs. Elevated body weight has been linked to increased rates of cardiac problems, blood pressure issues, and risk of developing type 2 diabetes. Leptin, a hormone produced by the body that is involved in energy balance by inhibiting hunger has been implicated as an underlying mechanism that differentially contributes to food-seeking motivation. Using a scientifically validated animal model of obesity, the fatty Zucker rat, which has mutated leptin receptor genes, leptin's role in behavioral motivation can be assessed. Animals were on a 2 -h food access restriction with one-hour access to rewards in session and one hour of free-feeding access. Pre-session and post-session food access differences were evaluated in looking at motivation for food rewards during satiation while responding on differing levels of fixed-ratio schedules. The results showed robust differential behavior from satiation, demonstrating a basis for a biological mechanism involving leptin sensitivity that could underlie obesity. Although further experimentation is needed, understanding leptin could help bridge the gap in our understanding of satiation and non-satiation.

Performance on a progressive-ratio schedule of food reinforcement during concurrent access to a sucrose solution or tap water.

Seven food-deprived rats were observed on a progressive-ratio 5 schedule of food reinforcement under baseline conditions and during concurrent access to tap water, a 1 % sucrose solution, or a 5 % sucrose solution. Subjects responded for the alternative reinforcer primarily during the preratio pause. Responding for the alternative reinforcer caused a local increase in pause durations in all conditions, however, the mean pause duration for the session increased only when the 5 % sucrose solution was available. Otherwise, the increase in pause durations that accompanied responding for the alternative reinforcer was offset by reducing pausing at other times. Access to the 5 % sucrose solution changed the relation of ratio size to pause duration in several subjects in that the longest pause durations occurred before the end of the session rather than when the highest ratios were in effect. Breakpoints were largely unaffected by alternative reinforcement.

Effects of negative incentive shifts in food reward on rats' consumption of concurrent ethanol solutions.

Frustration stress, typically operationalized as the unexpected loss of reinforcement, has been shown to engender substance use. Abrupt reductions in reinforcer magnitude likely also function as frustration stressors. These negative incentive shifts were previously shown to produce tap- and sweetened-water drinking in rats. The purpose of this study was to investigate whether these shifts in food reward would occasion oral ethanol self-administration. Nine male Long-Evans rats operated on a two-component multiple fixed-ratio schedule with signaled components producing either a large (4 pellets) or small (1pellet) reinforcer. Components were pseudorandomly arranged to present 4 transitions between past and upcoming reinforcer magnitudes: small-to-large, small-to-small, large-to-large, and large-to-small (negative incentive shift). Experiment 1 investigated the effects of negative incentive shifts on consumption of concurrent, freely available 10% sucrose, 10% sucrose plus 10% ethanol, and following sucrose fading, 10% ethanol. Experiment 2 entailed continuation of schedule contingencies with a dose manipulation of 4 ethanol concentrations (0, 5, 10, and 20%) to assess dose-dependent differences in transition-type control and consumption. A lever-press extinction condition was then conducted with 10% ethanol availability. In this novel model of frustration stress, negative incentive shifts prompted ethanol self-administration at each dose investigated, whereas the other transitions did not.

Endocannabinoid Signaling from 2-Arachidonoylglycerol to CB1 Cannabinoid Receptor Facilitates Reward-based Learning of Motor Sequence.

The endocannabinoid system modulates synaptic transmission, controls neuronal excitability, and is involved in various brain functions including learning and memory. 2-arachidonoylglycerol, a major endocannabinoid produced by diacylglycerol lipase-α (DGLα), is released from postsynaptic neurons, retrogradely activates presynaptic CB1 cannabinoid receptors, and induces short-term or long-term synaptic plasticity. To examine whether and how the endocannabinoid system contributes to reward-based learning of a motor sequence, we subjected male CB1-knockout (KO) and DGLα-KO mice to three types of operant lever-press tasks. First, we trained mice to press one of three levers labeled A, B, and C for a food reward (one-lever task). Second, we trained mice to press the three levers in the order of A, B, and C (three-lever task). Third, the order of the levers was reversed to C, B, and A (reverse three-lever task). We found that CB1-KO mice and DGLα-KO mice exhibited essentially the same deficits in the operant lever-press tasks. In the one-lever task, both strains of knockout mice showed a slower rate of learning to press a lever for food. In the three-lever task, both strains of knockout mice showed a slower rate of learning of the motor sequence. In the reverse three-lever task, both strains of knockout mice needed more lever presses for reversal learning. These results suggest that the endocannabinoid system facilitates reward-based learning of a motor sequence by conferring the flexibility with which animals can switch between strategies.

Response duration is sensitive to both immediate and delayed reinforcement.

We investigated the duration of lever pressing by rats when the delivery of appetitive reinforcers was contingent upon response duration. In the first experiment, response durations increased when duration requirements were imposed, and they decreased when duration requirements were removed. This effect occurred whether reinforcers were immediate or delayed by 8 s. In order to maintain the integrity of the delay intervals, reinforcer delivery was dependent upon both lever depression and release. In a second experiment, lever depression only and a response duration of at least 4 s were required for reinforcer delivery. Compared to immediate reinforcement conditions, delayed reinforcers increased both variability and the length of the maximum response durations. In a third experiment, immediate reinforcers were delivered contingent upon lever depression and release under a variety of duration requirements. Median lever-press durations tracked the contingencies rapidly. Across all three experiments, rats emitted numerous response durations that were too short to satisfy the reinforcer requirements, and bimodal distributions similar to those produced by differential reinforcement of low rate schedules were evident for most rats. In many aspects, response duration responds to reinforcement parameters in a fashion similar to rate of discrete responding, but an examination of this continuous dimension of behavior may provide additional information about environment-behavior relationships.

Punishment and the potential for negative reinforcement with histamine injection.

The present study examined punishment of responding with histamine injection, and its potential to generate avoidance of punishment. Sprague-Dawley rats were trained under concurrent schedules in which responses on one lever (the punishment lever) produced food under a variable-interval schedule, and under some conditions intermittent injections of histamine, which suppressed behavior. Responses on a second (avoidance) lever prevented histamine injections scheduled on the punishment lever. After stabilization of punished responding, a variable-interval 15-s schedule of cancellation of histamine (avoidance) was added for responding on the second/avoidance lever, without subsequent acquisition of responding on that lever. Progressive decreases in the length of the punishment variable-interval schedule increased suppression on the punishment lever without increases in response rates on the avoidance lever. Exchanging contingencies on the levers ensured that response rates on the avoidance lever were sufficiently high to decrease the histamine injection frequency; nonetheless response rates on the avoidance lever decreased over subsequent sessions. Under no condition was responding maintained on the avoidance lever despite continued punishing effectiveness of histamine throughout. The present results suggest that avoidance conditioning is not a necessary condition for effective punishment, and confirm the importance of empirical rather than presumed categorization of behavioral effects of stimulus events.

Impulsive choice and pre-exposure to delays: iv. effects of delay- and immediacy-exposure training relative to maturational changes in impulsivity.

Impulsive choice describes preference for smaller, sooner rewards over larger, later rewards. Excessive delay discounting (i.e., rapid devaluation of delayed rewards) underlies some impulsive choices, and is observed in many maladaptive behaviors (e.g., substance abuse, gambling). Interventions designed to reduce delay discounting may provide therapeutic gains. One such intervention provides rats with extended training with delayed reinforcers. When compared to a group given extended training with immediate reinforcers, delay-exposed rats make significantly fewer impulsive choices. To what extent is this difference due to delay-exposure training shifting preference toward self-control or immediacy-exposure training (the putative control group) shifting preference toward impulsivity? The current study compared the effects of delay- and immediacy-exposure training to a no-training control group and evaluated within-subject changes in impulsive choice across 51 male Wistar rats. Delay-exposed rats made significantly fewer impulsive choices than immediacy-exposed and control rats. Between-group differences in impulsive choice were not observed in the latter two groups. While delay-exposed rats showed large, significant pre- to posttraining reductions in impulsive choice, immediacy-exposed and control rats showed small reductions in impulsive choice. These results suggest that extended training with delayed reinforcers reduces impulsive choice, and that extended training with immediate reinforcers does not increase impulsive choice.

Response-reinforcer dependency and resistance to change.

The effects of the response-reinforcer dependency on resistance to change were studied in three experiments with rats. In Experiment 1, lever pressing produced reinforcers at similar rates after variable interreinforcer intervals in each component of a two-component multiple schedule. Across conditions, in the fixed component, all reinforcers were response-dependent; in the alternative component, the percentage of response-dependent reinforcers was 100, 50 (i.e., 50% response-dependent and 50% response-independent) or 10% (i.e., 10% response-dependent and 90% response-independent). Resistance to extinction was greater in the alternative than in the fixed component when the dependency in the former was 10%, but was similar between components when this dependency was 100 or 50%. In Experiment 2, a three-component multiple schedule was used. The dependency was 100% in one component and 10% in the other two. The 10% components differed on how reinforcers were programmed. In one component, as in Experiment 1, a reinforcer had to be collected before the scheduling of other response-dependent or independent reinforcers. In the other component, response-dependent and -independent reinforcers were programmed by superimposing a variable-time schedule on an independent variable-interval schedule. Regardless of the procedure used to program the dependency, resistance to extinction was greater in the 10% components than in the 100% component. These results were replicated in Experiment 3 in which, instead of extinction, VT schedules replaced the baseline schedules in each multiple-schedule component during the test. We argue that the relative change in dependency from Baseline to Test, which is greater when baseline dependencies are high rather than low, could account for the differential resistance to change in the present experiments. The inconsistencies in results across the present and previous experiments suggest that the effects of dependency on resistance to change are not well understood. Additional systematic analyses are important to further understand the effects of the response-reinforcer relation on resistance to change and to the development of a more comprehensive theory of behavioral persistence.

Reinforcement value and fixed-interval performance.

The concept of reinforcement value summarizes the effect of different variables, such as reinforcement delay, reinforcement magnitude, and deprivation level, on behavior. In the present set of experiments, we evaluated the effect of reinforcement devaluation on performance under FI schedules. The literature on timing and reinforcement value suggests that devaluation generates longer expected times to reinforcement than the same intervals trained under control conditions. We devalued reinforcement with delay in Experiments 1A, 1B, and 2, and diminished deprivation in Experiments 3A and 3B. Devaluation reduced response rates, increased the number of one-response intervals, and lengthened postreinforcement pauses, but had inconsistent effects on other timing measures such as quarter life and breakpoint. The results of delayed reinforcement and diminished deprivation manipulations are well summarized as reinforcement devaluation effects. These results suggest that devaluation may reduce stimulus control. In addition, we argue that the process by which delayed reinforcement affects behavior might also explain some effects observed in other devaluation procedures through the concept of reinforcement value.

Behavioral effects of delayed timeouts from reinforcement.

Timeouts are sometimes used in applied settings to reduce target responses, and in some circumstances delays are unavoidably imposed between the onset of a timeout and the offset of the response that produces it. The present study examined the effects of signaled and unsignaled timeouts in rats exposed to concurrent fixed-ratio 1 fixed-ratio 1 schedules of food delivery, where each response on one lever, the location of which changed across conditions, produced both food and a delayed 10-s timeout. Delays of 0 to 38 s were examined. Delayed timeouts often, but not always, substantially reduced the number of responses emitted on the lever that produced timeouts relative to the number emitted on the lever that did not produce timeouts. In general, greater sensitivity was observed to delayed timeouts when they were signaled. These results demonstrate that delayed timeouts, like other delayed consequences, can affect behavior, albeit less strongly than immediate consequences.

The role of temporal intervals on reinforcer accumulation.

Animals accumulate reinforcers when they forgo the opportunity to consume available food in favor of acquiring additional food for later consumption. Laboratory research has shown that reinforcer accumulation is facilitated when an interval (either spatial or temporal) separates earning from consuming reinforcers. However, there has been no systematic investigation on the interval separating consuming reinforcers from earning additional reinforcers. This oversight is problematic because this second interval is an integral part of much of the previous research on reinforcer accumulation. The purpose of the current study was to determine the independent contributions of these two temporal intervals on reinforcer accumulation in rats. Each left lever press earned a single food pellet; delivery of the accumulated pellet(s) occurred upon a right lever press. Conditions varied based on the presence of either an intertrial interval (ITI) that separated pellet delivery from the further opportunity to accumulate more pellets, or a delay-to-reinforcement that separated the right lever press from the delivery of the accumulated pellet(s). Delay and ITI values of 0, 5, 10 and 20 s were investigated. The delay-to-reinforcement conditions produced greater accumulation relative to the ITI conditions, despite accumulation increasing the density of reinforcement more substantially in the ITI conditions. This finding suggests that the temporal separation between reinforcer accumulation and subsequent delivery and consumption was a more critical variable in controlling reinforcer accumulation.

Behavioral variability as avoidance behavior.

This study aimed to investigate whether variable patterns of responses can be acquired and maintained by negative reinforcement under an avoidance contingency. Six male Wistar rats were exposed to sessions in which behavioral variability was reinforced according to a Lag contingency: Sequences of three responses on two levers had to differ from one, two or three previous sequences for shocks to be avoided (Lag 1, Lag 2 and Lag 3, respectively). Performance under the Lag conditions was compared with performance on a Yoke condition in which the animals received the same reinforcement frequency and distribution as in the Lag condition but behavioral variability was not required. The results showed that most of the subjects varied their sequences under the Lag contingencies, avoiding shocks with relatively high probability (≥ 0.7). Under the Yoke procedure, responding continued to occur with high probability, but the behavioral variability decreased. These results suggest that behavioral variability can be negatively reinforced.

A comparison of resetting and nonresetting contingencies in progressive-duration schedules.

We compared two progressive schedules of reinforcement in which rats received access to sweetened condensed milk for depressing and holding down a response lever. Duration requirements increased after each reinforcer delivery in a manner similar to progressive-ratio schedules. Under one schedule, any response duration less than that required for reinforcement had no programmed consequences. Under the second schedule, the cumulative duration of all responses could meet the reinforcement criteria. Breaking points were consistently higher when all lever presses, regardless of duration, contributed to meeting the reinforcer requirements. Breaking points under both schedules increased when food deprivation was long enough to result in body-weight reductions, but the sensitivity of the schedules to brief periods of food deprivation was inconsistent. Under both schedules, food deprivation produced an increase in shorter durations, thus reducing the efficiency of responding.

Effects of shifts in food reinforcement context on rats' consumption of concurrently available water or sucrose solution.

The purpose of this study was to investigate the effects of signaled transitions from relatively rich to lean conditions of food reinforcement on drinking concurrently available water or sucrose-sweetened water in rats. Past research demonstrated that these negative incentive shifts produce behavioral disruption in the form of extended pausing on fixed-ratio schedules. Four male Long-Evans rats operated on a two-component multiple fixed-ratio fixed-ratio schedule. In one manipulation, the ratio was held constant and the components arranged either a large six-pellet reinforcer (rich) or small one-pellet reinforcer (lean). In a second manipulation, the components both produced a one-pellet reinforcer but differed in terms of the ratio requirement, with the rich and lean conditions corresponding to relatively small and large ratios. In both manipulations, components were pseudorandomly presented to arrange four transitions signaled by retractable levers: lean-to-lean, lean-to-rich, rich-to-rich, and rich-to-lean (the negative incentive shift). During experimental conditions, a bottle with lickometer was inserted in the chamber, providing concurrent access either to tap water or a 10% sucrose solution. The negative incentive shift produced considerably more drinking than the other transitions in all rats during both manipulations. The level of drinking was not polydipsic; rather, it appears that the negative incentive shift enhanced the value of concurrently available reinforcers relative to food reinforcement.