Social familiarity and reinforcement value: a behavioral-economic analysis of demand for social interaction with cagemate and non-cagemate female rats.

Rats were studied in social reinforcement procedures in which lever presses opened a door separating two adjacent spaces, permitting access to social interaction with a partner rat. The number of lever presses required for social interaction was systematically increased across blocks of sessions according to fixed-ratio schedules, generating demand functions at three different social reinforcement durations: 10 s, 30 s, and 60 s. The social partner rats were cagemates in one phase, and non-cagemates in a second phase. The rate at which social interactions were produced declined with the fixed-ratio price, and was well described by an exponential model that has been successfully employed with a range of social and non-social reinforcers. None of the main parameters of the model varied systematically with social interaction duration or with the social familiarity of the partner rat. On the whole, the results provide further evidence of the reinforcing value of social interaction, and its functional parallels with non-social reinforcers.

Failure to Find Altruistic Food Sharing in Rats.

Prior research has found that one rat will release a second rat from a restraint in the presence of food, thereby allowing that second rat access to food. Such behavior, clearly beneficial to the second rat and costly to the first, has been interpreted as altruistic. Because clear demonstrations of altruism in rats are rare, such findings deserve a careful look. The present study aimed to replicate this finding, but with more systematic methods to examine whether, and under what conditions, a rat might share food with its cagemate partner. Rats were given repeated choices between high-valued food (sucrose pellets) and 30-s social access to a familiar rat, with the (a) food size (number of food pellets per response), and (b) food motivation (extra-session access to food) varied across conditions. Rats responded consistently for both food and social interaction, but at different levels and with different sensitivity to the food-access manipulations. Food production and consumption was high when food motivation was also high (food restriction) but substantially lower when food motivation was low (unlimited food access). Social release occurred at moderate levels, unaffected by the food-based manipulations. When food was abundant and food motivation low, the rats chose food and social options about equally often, but sharing (food left unconsumed prior to social release) occurred at low levels across sessions and conditions. Even under conditions of low food motivation, sharing occurred on only 1% of the sharing opportunities. The results are therefore inconsistent with claims in the literature that rats are altruistically motivated to share food with other rats.

Social preference in rats.

Rats were given repeated choices between social and nonsocial outcomes, and between familiar and unfamiliar social outcomes. Lever presses on either of 2 levers in the middle chamber of a 3-chamber apparatus opened a door adjacent to the lever, permitting 45-s access to social interaction with the rat in the chosen side chamber. In Experiment 1, rats preferred (a) social over nonsocial options, choosing their cagemate rat over an empty chamber, and (b) an unfamiliar over a familiar rat, choosing a non-cagemate over their cagemate. These findings were replicated in Experiment 2 with 2 different non-cagemate rats. Rats preferred both non-cagemate rats to a similar degree when pitted against their cagemate, but were indifferent when the 2 non-cagemates were pitted against each other. Similar preference for social over nonsocial and non-cagemate over cagemate was seen in Experiment 3, with new non-cagemate rats introduced after every third session. Response rates (for both cagemate and non-cagemate rats) were elevated under conditions of nonsocial (isolated) housing compared to conditions of social (paired) housing, demonstrating a social deprivation effect. Together, the experiments contribute to an experimental analysis of social preference within a social reinforcement framework, drawing on methods with proven efficacy in the analysis of reinforcement more generally.

Human risky choice: delay sensitivity depends on reinforcer type.

The present study was designed to help bridge the methodological gap between human and nonhuman animal research in delay-based risky choice. In Part 1, 4 adult human subjects made repeated choices between variable-time and fixed-time schedules of 30-s video clips. Both alternatives had equal mean delays of 15 s, 30 s, or 60 s. Three of 4 subjects strongly preferred the variable-delay alternative across all conditions. In Part 2, these 3 subjects were then provided pairwise choices between 2 variable-time schedules with different delay distributions. Subjects generally preferred the variable-delay distributions with a higher probability of short-reinforcer delays, consistent with accounts based on nonlinear discounting of delayed reinforcement. There was only weak correspondence between experimental results and verbal reports. The overall pattern of results is inconsistent with prior risky choice research with human subjects but is consistent with prior results with nonhuman subjects, suggesting that procedural differences may be a critical factor determining risk-sensitivity across species.

Combinations of response-dependent and response-independent schedule-correlated stimulus presentation in an observing procedure.

Pigeons pecked a response key on a variable-interval (VI) schedule, in which responses produced food every 40 s, on average. These VI periods, or components, alternated in irregular fashion with extinction components in which food was unavailable. Pecks on a second (observing) key briefly produced exteroceptive stimuli (houselight flashes) correlated with the component schedule currently in effect. Across conditions within a phase, the dependency between observing and presentation of the stimuli was decreased systematically while the density of stimulus presentation was held constant. Across phases, the proportion of session time spent in the VI component was adjusted from 0.5 to 0.25, and then to 0.75. Results indicate that rate of observing decreased as the dependency between responses and stimulus presentations was decreased. Further, discriminative control by the schedule-correlated stimuli was systematically weakened as dependency was decreased. Increasing the proportion of session time spent in VI decreased the rate of observing. This effect was additive with the manipulation of the dependency between observing and presentation of the stimuli. Overall, these results show that conditioned reinforcers function similarly to unconditioned reinforcers with respect to response-consequence dependencies, and that stimulus control is enhanced under conditions in which the relevant stimuli are produced by an organism's behavior.

Reinforcer accumulation in a token-reinforcement context with pigeons.

Four pigeons were exposed to a token-reinforcement procedure with stimulus lights serving as tokens. Responses on one key (the token-production key) produced tokens that could be exchanged for food during an exchange period. Exchange periods could be produced by satisfying a ratio requirement on a second key (the exchange-production key). The exchange-production key was available any time after one token had been produced, permitting up to 12 tokens to accumulate prior to exchange. Token accumulation, measured in terms of both frequency (percent cycles with accumulation) and magnitude (mean number of tokens accumulated), decreased as the token-production ratio increased from 1 to 10 across conditions (with exchange-production ratio held constant), and increased as the exchange-production ratio increased from 1 to 250 across conditions (with token-production ratio held constant). When tokens were removed, accumulation decreased markedly compared to conditions with tokens and the same schedules. These data show that token accumulation is an orderly function of token-production and exchange-production schedules, and they are broadly consistent with a unit-price model based on local and global responses per reinforcer.

Token reinforcement: Translational research and application.

The present paper provides an integrative review of research on token reinforcement systems, organized in relation to basic behavioral functions and economic variables. This type of functional taxonomy provides a useful way to organize the literature, bringing order to a wide range of findings across species and settings, and revealing gaps in the research and areas especially ripe for analysis and application. Unlike standard translational research, based on a unidirectional model in which the analysis moves from laboratory to the applied realm, work in the area of token systems is best served by a bidirectional interplay between laboratory and applied research, where applied questions inspire research on basic mechanisms. When based on and contributing to an analysis, applied research on token economies can be on the leading edge of theoretical advances, helping set the scientific research agenda.

To free, or not to free: Social reinforcement effects in the social release paradigm with rats.

The present research measured social reinforcement in rats, using a social-release procedure in which lever presses permitted 10-s access to a familiar social partner. The work requirements for reinforcement increased systematically according to progressive-ratio (PR) schedules. Social and food reinforcement value were compared across blocks of sessions (Experiment 1) and concurrently within the same sessions (Experiment 2). To assess motivational effects, response and reinforcer rates for both reinforcer types were studied under food restriction, social restriction, and combined food and social restriction. Responding was maintained by both reinforcers, albeit at substantially higher levels for food than for social access. Responding for social access decreased to low levels under extinction conditions, demonstrating functional control by the social-reinforcement contingency. Sensitivity to social restriction was seen in some conditions in Experiment 2, in which social reinforcers were earned earlier in the session (at lower food prices) under social restriction than under the other deprivation conditions. Altogether, results are consistent with a social reinforcement conceptualization, and demonstrate an important role for social contact in social release behavior. The study demonstrates a promising set of methods for analyzing and quantifying social reinforcement.

Discriminated timeout avoidance in pigeons: the roles of added stimuli.

Two experiments examined pigeons' postponement of a signaled extinction period, or timeout (TO), from an ongoing schedule of response-dependent food delivery. A concurrent-operant procedure was used in which responses on one (food) key produced food according to a variable-interval schedule and responses on a second (postponement) key delayed the next scheduled TO according to a response-TO (R-TO) interval. A series of response-independent stimulus changes on the food key temporally partitioned the R-TO into three equal segments (S1, S2, and S3). Postponement responses, in addition to postponing TO, also reinstated S1, the stimulus correlated with the greatest temporal distance from TO. In Experiment 1, the R-TO interval was manipulated systematically across blocks of sessions (conditions) at a given ratio of R-TO:TO duration. This R-TO:TO ratio was manipulated across blocks of conditions (phases). Postponement response rates varied inversely with R-TO interval in each phase. Changes in the R-TO:TO ratio did not produce consistent differences except at the 1:10 ratio for some pigeons, where it disrupted postponement responding in some conditions. Most of the postponement responses occurred in the presence of S2 and S3, the stimuli most proximal to TO, whereas most of the food-key responses occurred in S1. In Experiment 2, the R-TO contingencies were systematically manipulated in the presence of the time-correlated stimuli. In one set of conditions, the R-TO contingencies were made either ineffective or less effective in the presence of one or more stimuli. Postponement responses typically shifted to stimuli in the presence of which responses were relatively more effective. Postponement responses decreased markedly when the added stimuli were removed, and then recovered when the stimuli were reinstated. Results from both experiments indicate that the added stimuli in a discriminated TO-avoidance procedure serve predominately discriminative functions, delineating periods during which behavior is maximally effective. The results parallel those obtained in shock-avoidance procedures, providing further evidence that TO functions as an aversive stimulus.

IRT-stimulus contingencies in chained schedules: implications for the concept of conditioned reinforcement.

Two experiments with pigeons investigated the effects of contingencies between interresponse times (IRTs) and the transitions between the components of 2- and 4-component chained schedules (Experiments 1 and 2, respectively). The probability of component transitions varied directly with the most recent (Lag 0) IRT in some experimental conditions and with the 4th (Lag 4) IRT preceding the most recent one in others. Mean component durations were constant across conditions, so the reinforcing effect of stimulus change was dissociated from that of delay to food. IRTs were longer in the Lag-0 than in the Lag-4 conditions of both experiments, thus demonstrating that stimulus change functioned as a reinforcer. In the Lag-0 conditions of Experiment 2, the Component-1 IRTs increased more than the Component-2 IRTs, which in turn increased more than the Component-3 IRTs. This finding runs counter to the conditioned-positive-reinforcement account of chained-schedule responding, which holds that the reinforcing effect of stimulus change should vary in strength as an inverse function of the delay to the unconditioned reinforcer at the end of the chain because conditioned reinforcement is due to first- or higher-order classical conditioning. Therefore, we present other possible explanations for this effect.

Substitution effects in a generalized token economy with pigeons.

Pigeons made repeated choices between earning and exchanging reinforcer-specific tokens (green tokens exchangeable for food, red tokens exchangeable for water) and reinforcer-general tokens (white tokens exchangeable for food or water) in a closed token economy. Food and green food tokens could be earned on one panel; water and red water tokens could be earned on a second panel; white generalized tokens could be earned on either panel. Responses on one key produced tokens according to a fixed-ratio schedule, whereas responses on a second key produced exchange periods, during which all previously earned tokens could be exchanged for the appropriate commodity. Most conditions were conducted in a closed economy, and pigeons distributed their token allocation in ways that permitted food and water consumption. When the price of all tokens was equal and low, most pigeons preferred the generalized tokens. When token-production prices were manipulated, pigeons reduced production of the tokens that increased in price while increasing production of the generalized tokens that remained at a fixed price. The latter is consistent with a substitution effect: Generalized tokens increased and were exchanged for the more expensive reinforcer. When food and water were made freely available outside the session, token production and exchange was sharply reduced but was not eliminated, even in conditions when it no longer produced tokens. The results join with other recent data in showing sustained generalized functions of token reinforcers, and demonstrate the utility of token-economic methods for assessing demand for and substitution among multiple commodities in a laboratory context.

Opioidergic and dopaminergic modulation of cost/benefit decision-making in Long Evans Rats.

Eating disorders are associated with impaired decision-making and dysfunctional reward-related neurochemistry. The present study examined the potential contributions of dopamine and opioid signaling to these processes using two different decision-making tasks. In one task, Long Evans Rats chose between working for a preferred food (high-carbohydrate banana-flavored sucrose pellets) by lever pressing on a progressive-ratio schedule of reinforcement vs. obtaining less preferred laboratory chow that was concurrently available. In a second (effort-free) task, rats chose between the same two reinforcers when they were both available freely. Rats were trained in these tasks before receiving haloperidol (0.00, 0.05, 0.10mg/kg, intraperitoneally (i.p.)) or naloxone (0.0, 1.5, 3.0mg/kg, i.p.). In the first task, haloperidol decreased breakpoint, lever presses, number of reinforcers earned, and increased chow intake, whereas naloxone decreased breakpoint and number of reinforcers earned but had no effect on chow consumption. In the effort-free task, haloperidol reduced intakes of both foods without affecting preference, whereas naloxone selectively reduced the consumption of banana-pellets. The present findings support converging evidence suggesting that DA signaling affects processes more closely related to appetitive motivation, leaving other components of motivation unchanged. By contrast, opioid signaling appears to mediate aspects of hedonic feeding by selectively altering intakes of highly palatable foods. For preferred foods, both appetitive and consummatory aspects of food intake were altered by opioid receptor antagonism. Our findings argue against a general suppression of appetite by either compound, as appetite manipulations have been shown to unselectively alter intakes of both types of food regardless of the task employed.

Second-order schedules of token reinforcement with pigeons: implications for unit price.

Four pigeons were exposed to second-order schedules of token reinforcement, with stimulus lights serving as token reinforcers. Tokens were earned according to a fixed-ratio (token-production) schedule, with the opportunity to exchange tokens for food (exchange period) occurring after a fixed number had been produced (exchange-production ratio). The token-production and exchange-production ratios were manipulated systematically across conditions. Response rates varied inversely with the token-production ratio at each exchange-production ratio. Response rates also varied inversely with the exchange-production ratio at each token-production ratio, particularly at the higher token-production ratios. At higher token-production and exchange-production ratios, response rates increased in token-production segments closer to exchange periods and food. Some conditions were conducted in a closed economy, in which the pigeons earned all their daily ration of food within the session. Relative to comparable open-economy conditions, response rates in the closed economy were less affected by changes in token-production ratio, resulting in higher levels of food intake and body weight. Some of the results are consistent with the economic concept of unit price, a cost-benefit ratio comprised of responses per unit of food delivery, but most are well accounted for by a consideration of the number of responses required to produce exchange periods, without regard to the amount of reinforcement available during those exchange periods.

Functional analysis of mutual behavior in laboratory rats (Rattus norvegicus).

Three pairs of rats were trained to synchronize their lever pressing according to a mutual reinforcement contingency, in which alternating lever presses that fell within a 500-ms window were reinforced with food. In Experiment 1, rats worked in adjacent chambers separated by a transparent barrier, and the effects of the mutual reinforcement contingency were compared with those under yoked-control conditions that provided the same rate of food reinforcement but without the temporal coordination response requirement. In Experiment 2, coordinated behavior was compared with and without a barrier, and across different barrier types: transparent, opaque, wire mesh. In Experiment 3, the effects of social familiarity were assessed by switching partners, enabling a comparison of coordinated behavior with familiar and unfamiliar partners. The overall pattern of results shows that the coordinated behavior of 2 rats was (a) maintained by mutual reinforcement contingencies, (b) unrelated to the type or presence of a barrier separating the rats, and (c) sufficiently flexible to adjust to the presence and behavior of an unfamiliar partner. Taken as a whole, the study illustrates a promising approach to conceptualizing and analyzing behavioral mechanisms of mutual behavior, an important component of an integrated study of social behavior.

Response-cost punishment via token loss with pigeons.

Two experiments were conducted to investigate punishment via response-contingent removal of conditioned token reinforcers (response cost) with pigeons. In Experiment 1, key pecking was maintained on a two-component multiple second-order schedule of token delivery, with light emitting diodes (LEDs) serving as token reinforcers. In both components, responding produced tokens according to a random-interval 20-s schedule and exchange periods according to a variable-ratio schedule. During exchange periods, each token was exchangeable for 2.5-s access to grain. In one component, responses were conjointly punished according to fixed-ratio schedules of token removal. Response rates in this punishment component decreased to low levels while response rates in the alternate (no-punishment) component were unaffected. Responding was eliminated when it produced neither tokens nor exchange periods (Extinction), but was maintained at moderate levels when it produced tokens in the signaled absence of food reinforcement, suggesting that tokens served as effective conditioned reinforcers. In Experiment 2, the effect of the response-cost punishment contingency was separated from changes in the density of food reinforcement. This was accomplished by yoking either the number of food deliveries per component (Yoked Food) or the temporal placement of all stimulus events (tokens, exchanges, food deliveries) (Yoked Complete), from the punishment to the no-punishment component. Response rates decreased in both components, but decreased more rapidly and were generally maintained at lower levels in the punishment component than in the yoked component. In showing that the response-cost contingency had a suppressive effect on responding in addition to that produced by reductions in reinforcement density, the present results suggest that response-cost punishment shares important features with other forms of punishment.

Pigeons' demand and preference for specific and generalized conditioned reinforcers in a token economy.

Pigeons' demand and preference for specific and generalized tokens was examined in a token economy. Pigeons could produce and exchange different colored tokens for food, for water, or for food or water. Token production was measured across three phases, which examined: (1) across-session price increases (typical demand curve method); (2) within-session price increases (progressive-ratio, PR, schedule); and (3) concurrent pairwise choices between the token types. Exponential demand curves were fitted to the response data and accounted for over 90% total variance. Demand curve parameter values, Pmax , Omax and α showed that demand was ordered in the following way: food tokens, generalized tokens, water tokens, both in Phase 1 and in Phase 3. This suggests that the preferences were predictable on the basis of elasticity and response output from the demand analysis. Pmax and Omax values failed to consistently predict breakpoints and peak response rates in the PR schedules in Phase 2, however, suggesting limits on a unitary conception of reinforcer efficacy. The patterns of generalized token production and exchange in Phase 3 suggest that the generalized tokens served as substitutes for the specific food and water tokens. Taken together, the present findings demonstrate the utility of behavioral economic concepts in the analysis of generalized reinforcement.

The several roles of stimuli in token reinforcement.

Three experiments were conducted with pigeons to identify the stimulus functions of tokens in second-order token-reinforcement schedules. All experiments employed two-component multiple schedules with a token-reinforcement schedule in one component and a schedule with equivalent response requirements and/or reinforcer density in the other. In Experiment 1, response rates were lower under a token-reinforcement schedule than under a tandem schedule with the same response requirements, suggesting a discriminative role for the tokens. In Experiment 2, response rates varied systematically with signaling functions of the tokens in a series of conditions designed to explore other aspects of the temporal-correlative relations between tokens and food. In Experiment 3, response rates were reduced but not eliminated by presenting tokens independent of responding, yoked to their temporal occurrence in a preceding token component, suggesting both a reinforcing function and eliciting/evocative functions based on stimulus-food relations. Only when tokens were removed entirely was responding eliminated. On the whole, the results suggest that tokens, as stimuli temporally correlated with food, may serve multiple stimulus functions in token-reinforcement procedures--reinforcing, discriminative, or eliciting--depending on the precise arrangement of the contingencies in which they are embedded.

Unit price and choice in a token-reinforcement context.

Pigeons were exposed to multiple and concurrent second-order schedules of token reinforcement, with stimulus lights serving as token reinforcers. Tokens were produced and exchanged for food according to various fixed-ratio schedules, yielding equal and unequal unit prices (responses per unit food delivery). On one schedule (termed the standard schedule), the unit price was held constant across conditions. On a second schedule (the alternative schedule), the unit price was either the same or different from the standard. Under conditions with unequal unit prices, near-exclusive preference for the lower unit price was obtained. Under conditions with equal unit prices, the direction and degree of preference depended on ratio size (number of responses per exchange period). When this ratio differed, strong preferences for the smaller ratio were observed. When this ratio was equal, preferences were nearer indifference. Response rates on the multiple schedule were generally consistent with the preference data in showing sensitivity to ratio size. Results are discussed in terms of a unit-price model that includes handling and reinforcer immediacy as additional costs. On the whole, results show that preferences were determined primarily by delay to the exchange period.

Determinants of pigeons' choices in token-based self-control procedures.

Four pigeons were exposed to a token-based self-control procedure with stimulus lights serving as token reinforcers. Smaller-reinforcer choices produced one token immediately; larger-reinforcer choices produced three tokens following a delay. Each token could be exchanged for 2-s access to food during a signaled exchange period each trial. The main variables of interest were the exchange delays (delays from the choice to the exchange stimulus) and the food delays (also timed from the choice), which were varied separately and together across blocks of sessions. When exchange delays and food delays were shorter following smaller-reinforcer choices, strong preference for the smaller reinforcer was observed. When exchange delays and food delays were equal for both options, strong preference for the larger reinforcer was observed. When food delays were equal for both options but exchange delays were shorter for smaller-reinforcer choices, preference for the larger reinforcer generally was less extreme than under conditions in which both exchange and food delays were equal. When exchange delays were equal for both options but food delays were shorter for smaller-reinforcer choices, preference for the smaller reinforcer generally was less extreme than under conditions in which both exchange and food delays favored smaller-reinforcer choices. On the whole, the results were consistent with prior research on token-based self-control procedures in showing that choices are governed by reinforcer immediacy when exchange and food delays are unequal and by reinforcer amount when exchange and food delays are equal. Further, by decoupling the exchange delays from food delays, the results tentatively support a role for the exchange stimulus as a conditioned reinforcer.