Assessing stimulus preference using response force in a conjugate preparation: A replication and extension.

The current study examined 98 participants' preferences for five pictorial stimuli. The researchers used a verbal multiple-stimulus-without-replacement (VMSWO) preference assessment with each participant to identify high-preference and low-preference pictorial stimuli. Next, participants viewed each pictorial stimulus in a randomized order on a computer while using a hand dynamometer that measured the amount of force they exerted to increase or maintain the visual clarity of each image. The results indicate that over 75% of participants' force response ranks corresponded with participants' VMSWO high-preference stimuli, VMSWO low-preference stimuli, or both. The results of the current study provide further evidence for the use of conjugate schedules in the assessment of stimulus preference with potential for use as a reinforcer assessment. Implications along with directions for future research and limitations of the findings are discussed.

Comparing stimulus preference and response force in a conjugate preparation: A replication with auditory stimulation.

This study examined a conjugate approach for evaluating auditory stimulus preference for 81 participants using force as a continuous response dimension. First, the researchers used a verbal preference assessment to evaluate each participant's preference for listening to five genres of music. This process identified high-preference and low-preference music for each participant. Thereafter, the researchers exposed each participant to the five music genres in a randomized order while using a hand dynamometer to measure their response force to increase the auditory clarity of the music. The results indicate (a) 63% of the participants' high-preference music genres corresponded to the genre for which they exerted the highest mean force and (b) most participants' low-preference music genres corresponded to the genre for which they exerted the lowest mean force. These findings are consistent with those from Davis et al. (2021) and further support using conjugate preparations for measuring the relative value of some stimulus events.

Operant responding: Beyond rate and interresponse times.

Steve Fowler is best known for his contributions to neuroscience and pharmacology, especially in the behavioral characterization of antipsychotic drugs, which he pursued with great skill and ingenuity. The present review highlights some of his important contributions in understanding the interactions between pharmacology and learning systems, in particular operant learning. Much of Steve's work resulted in novel measurement systems that offered important insights about behavior that were not accessible with traditional operant approaches, which tend to emphasize response rate and interresponse times. A brief review of the emergence of response rate and temporal features of behavior as the dominant measures in operant learning is presented. Then, Steve's approach to behavioral measurement, grounded in his work under Joe Notterman, is developed. I will review selected aspects of his research program as they touch upon and illuminate the dopamine and the anhedonia hypothesis, behavioral characterizations of typical and atypical antipsychotics, the functional roles of response force in learning, and finally the nature of the operant itself.

Comparing stimulus preference and response force in a conjugate preparation.

Researchers used a conjugate preparation to evaluate how response force changed based on participants' preferences for visual stimuli. First, researchers used a verbal preference assessment to evaluate each participant's preference for viewing for five object categories of visual stimuli; this process identified high preference (HP) stimuli and low preference stimuli for most participants. Thereafter, researchers exposed each participant to the five stimulus categories in a randomized order while using a force dynamometer to measure their response force to increase visual clarity of each stimulus. Results indicate the majority of participants' HP stimuli corresponded to the stimulus category for which they exerted the highest mean force. These preliminary findings suggest conjugate preparations involving response force may provide another option for measuring the relative value of some stimulus events.

A review of the behavioral effects of response force: Perspectives on measurement and functions.

Response force is a fundamental dimension of behavior. Yet, little is known about its functional significance for learning. The present review examines the behavioral effects of force across several domains. Along the way, advantages of different measurement strategies that have been used to study force are evaluated. The behavioral functions of force are also considered in light of two commonly expressed notions about behavior. First, the Law of Least Effort predicts that animals and humans will act in ways that minimize the costs of behaving. Second, it is widely held that work requirements and effortful responding are aversive. A review of the literature, especially regarding behavioral adaptations to force, is consistent with the Law of Least Effort on many points. Empirical data are less clear on the aversive properties of force requirements. For much of the literature, there is little in coherent findings. Many disagreements and inconsistencies pertain to measurement strategies that fail to record the full range of response variants. Consistent support for aversive functions of force requirements have been obtained only when studied as a negative reinforcer.

The effects of force requirements on microstructure: Subcriterion responses and the mismeasurement of bouts.

The effects of response force on microstructure were evaluated. A strain-gauge operandum permitted the manipulation of the force required to produce reinforcers (criterion responses) independently from the force defining response threshold. Thus, we could detect subcriterion forces that fell short of the force criterion. Eight rats earned food according to variable-interval (VI) 30- and 120-s schedules. The force requirements were set to 5.6 or 32.0 g; the response threshold was fixed at 5.6 g. Interresponse times were computed when subcriterion responses were both included and omitted from the analysis. Log-survivor functions of interresponse times showed that increasing force requirements elevated the mean between-bout interval of the VI 120-s schedule, but only if subcriterion behavior was excluded. Omitting subcriterion responses thus leads to overestimation of intervals separating response bouts. Increasing force requirements also increased the skewness of the between-bout distribution. A subsequent analysis found that subcriterion responses are most plentiful following reinforcer delivery, which helps to explain why their omission might inflate between-bout intervals, as this period is an important transition from reinforcer consumption to engagement in operant activity. The data suggest caution interpreting the effects of force on microstructure when subcriterion behavior is not or cannot be measured.

The role of response force on the persistence and structure of behavior during extinction.

Behavior Momentum Theory has emerged as a prominent account of resistance to change in both basic and applied research. Although laboratory studies often define precise, repeatable responses, application research often deals with response classes that may vary widely along a number of dimensions. In general, Behavior Momentum Theory has not addressed how response dimensions impact resistance to change, providing an opportunity to expand the model in new directions. Four rats pressed a force transducer under a multiple variable interval (VI) 60-s VI 60-s schedule of reinforcement. In one component, responses satisfied the schedule only if the response force fell within a "low" force band requirement; responses in the other schedule were required to satisfy a "high" force band. Once responding stabilized, extinction was programmed for three sessions. Then, the procedures were replicated. The results showed that response force came under discriminative control, but force requirements had no impact on resistance to extinction. In a follow-up condition, the schedule was changed to a multiple VI 30-s VI 120-s schedule and the low-force band operated in both components. The results showed that behavior maintained by the VI 30-s schedule was generally more resistant to extinction. A secondary analysis showed that force distributions created under baseline maintained during extinction. Overall, the results suggest that differential response force requirements prevailing in steady state do not affect the course of extinction.

Aversive functions of response effort: Fact or artifact?

Historically, effort has been viewed as aversive. Most supporting evidence comes from studies demonstrating increased force/effort requirements reduce operant responding. Changes in force/effort requirements, however, are often accompanied by changes in response definition when mechanical devices are used to define the response. As a consequence, responses measured at one point in a study may go unmeasured at other points. In an alternative approach, we used a continuous measurement strategy that provided a means to fix the threshold force defining the response class and simultaneously allowed independent manipulation of the force criteria required to produce reinforcement. Rats pressed a force transducer according to a fixed-ratio 5 schedule of food delivery. The criterion force was systematically increased and decreased; the threshold for response detection was constant. When response rates included only criterion responses, overall rate decreased when force requirements increased. By contrast, when all responses, both those meeting force criteria and those that did not (above the threshold but below the criteria for reinforcement) were included in the rate calculation, increases in force increased response rate. Increases in force criteria also increased the maximum force (g) and time-integral of force (g-s) of operant behavior. Control conditions showed increases in responding could be explained by the emergence of subcriterion responses, irrespective of force. We conclude that prior results showing effort decreases response rates are due to an artifact arising from inadvertent changes in response definitions. Increases in effort may better be understood as changes in the response:reinforcer payoff owing to the emergence of a subcriterion response class.

Determinants of choice, and vulnerability and recovery in addiction.

Addiction may be viewed as choice governed by competing contingencies. One factor impacting choice, particularly as it relates to addiction, is sensitivity to delayed rewards. Discounting of delayed rewards influences addiction vulnerability because of competition between relatively immediate gains of drug use, e.g. intoxication, versus relatively remote gains of abstinence, e.g. family stability. Factors modifying delay sensitivity can be modeled in the laboratory. For instance, increased delay sensitivity can be similarly observed in adolescent humans and non-human animals. Similarly, genetic factors influence delay sensitivity in humans and animals. Recovery from addiction may also be viewed as choice behavior. Thus, reinforcing alternative behavior facilitates recovery because reinforcing alternative behavior decreases the frequency of using drugs. How reinforcing alternative behavior influences recovery can also be modeled in the laboratory. For instance, relapse risk decreases as abstinence duration increases, and this decreasing risk can be modeled in animals using choice procedures. In summary, addiction in many respects can be conceptualized as a problem of choice. Animal models of choice disorders stand to increase our understanding of the core processes that establish and maintain addiction and serve as a proving ground for development of novel treatments.

Conditioned stimuli's role in relapse: preclinical research on Pavlovian-Instrumental-Transfer.

RATIONALE AND OBJECTIVE: Pavlovian learning is central to many theories of addiction. In these theories, stimuli paired with drug ingestion become conditioned stimuli (CS) and subsequently elicit drug-seeking and drug-taking. However, in most relevant studies, Pavlovian and instrumental learning are confounded. This confound may be avoided in Pavlovian-Instrumental-Transfer (PIT) procedures. In PIT, Pavlovian and instrumental learning are established separately and then combined. In order to better understand the role of CSs in addiction, we review the relevant studies using PIT. FINDINGS: We identified seven articles examining PIT effects of ethanol- or cocaine-paired CSs. Under at least one condition, six of these articles reported CS-elicited increases in responding previously maintained by drug. However, the only study using the optimal control condition failed to find a CS-elicited increase. Two studies examining CS specificity found the CS also increased responding maintained by a different reinforcer. Two studies examined if CSs elicit increases in actual drug-taking. Both failed to find CS-elicited increases, i.e., no study shows CS-elicited increases in actual drug-taking. Further, CS-elicited increases in extinguished responding are short-lived. CONCLUSIONS: These findings are not entirely consistent with Pavlovian learning playing a central role in addiction. However, design issues can explain most of these inconsistencies. Studies without these design issues are needed. Additionally, existing theories hypothesize drug-paired CSs increase drug-taking by increasing motivation, by eliciting conditioned responses that make drug-seeking more probable, or by a combination of these. Work distinguishing between these mechanisms would also be useful.

Ethanol self-administration in mice under a second-order schedule.

Long Fixed-Interval (FI) schedules, particularly second-order schedules, can engender substantial responding before drug or ethanol delivery that is uninfluenced by the direct effects of the drug or ethanol. Thus, these schedules can be used to study the effects of medications upon drug- or ethanol-seeking, uninfluenced by the direct effects of the self-administered drug or ethanol. Long FI second-order schedules are frequently used in primates and occasionally in rats. Under second-order schedules, completion of one response requirement, e.g., a Fixed Ratio 10 (FR10:S), produces a brief stimulus presentation, e.g., a 1-s 80-dB 4-kHZ tone, and this FR10:S serves as the response unit under another schedule, e.g., an FI 1800-s. Thus, the first FR10 completed after 1800 s would result in delivery both of the tone and of reinforcement, e.g., 10 × 0.01 mL 16% (w/v) ethanol. To examine if such schedules could be effectively used in mice, which have advantages in neurobiological and genetic studies, we trained eight C57BL/6J mice to respond under the schedule just described. This schedule maintained substantial responding. The temporal pattern of behavior was typical of an FI schedule with responding accelerating across the interval. We also examined the effects of acute and chronic administration of fluvoxamine on this responding, and these were modest. Finally, we examined responding when alcohol and/or tone deliveries were withheld, and found that extinction occurred most rapidly when both were withheld. This work demonstrates that long FI schedules of ethanol delivery may be useful in studying ethanol seeking in mice.

Reinforcer magnitude and rate dependency: evaluation of resistance-to-change mechanisms.

Under many circumstances, reinforcer magnitude appears to modulate the rate-dependent effects of drugs such that when schedules arrange for relatively larger reinforcer magnitudes rate dependency is attenuated compared with behavior maintained by smaller magnitudes. The current literature on resistance to change suggests that increased reinforcer density strengthens operant behavior, and such strengthening effects appear to extend to the temporal control of behavior. As rate dependency may be understood as a loss of temporal control, the effects of reinforcer magnitude on rate dependency may be due to increased resistance to disruption of temporally controlled behavior. In the present experiments, pigeons earned different magnitudes of grain during signaled components of a multiple FI schedule. Three drugs, clonidine, haloperidol, and morphine, were examined. All three decreased overall rates of key pecking; however, only the effects of clonidine were attenuated as reinforcer magnitude increased. An analysis of within-interval performance found rate-dependent effects for clonidine and morphine; however, these effects were not modulated by reinforcer magnitude. In addition, we included prefeeding and extinction conditions, standard tests used to measure resistance to change. In general, rate-decreasing effects of prefeeding and extinction were attenuated by increasing reinforcer magnitudes. Rate-dependent analyses of prefeeding showed rate-dependency following those tests, but in no case were these effects modulated by reinforcer magnitude. The results suggest that a resistance-to-change interpretation of the effects of reinforcer magnitude on rate dependency is not viable.

Force dynamics in fixed-ratio schedules.

Fixed-ratio schedules are widely used in behavioral research. Although fixed-ratio schedules often conjure up relationships to work and effort, little is known about effort-related measures in these schedules. Early research had shown that force and effort of operant behavior vary systematically during the execution of ratio schedules, and the goal of the present study was to revisit early research on force dynamics in fixed-ratio schedules. Four rats earned sucrose by pressing an isometric force transducer. Presses produced sucrose after ten or twenty responses. In general, the force of responses increased then decreased systematically across the ratio. The possibility that decreases in force during ratio execution was due to a trade-off with the differential reinforcement of short inter-response times (IRT) was investigated in an additional condition where sucrose was made available according to a tandem fixed-ratio 19 inter-response (IRT)> t schedule. The tandem IRT requirement did not eliminate decreasing trends in force across the ratio; unexpectedly, the tandem requirement did eliminate increases in force early in the ratio, which may reflect sequence-level organization operating in the control of force dynamics.

Ethanol effects on multiple fixed-interval, fixed-ratio responding in mice with deletions of the serotonin transporter gene.

Serotonin transporter knockout (KO) mice self-administer less ethanol than either heterozygous or wild-type mice; however, the mechanistic basis for this difference remains unclear. Here we examine the possibility that ethanol more readily decreases responding in KO mice, thereby limiting ethanol self-administration. To examine whether KO mice were more sensitive to the response-decreasing effects of ethanol, we administered ethanol (0.2-3.2 g/kg) to mice responding under a multiple fixed-ratio 30-response, fixed-interval 300-s schedule of milk presentation. Ethanol decreased responding similarly in all three genotypes. Fixed-ratio responding tended to be decreased at lower doses than fixed-interval responding. The decreased level of ethanol self-administration in serotonin transporter KO mice is not explained by an increased sensitivity to the response-decreasing effects of ethanol in KO mice, as sensitivity to the response-decreasing effects of ethanol was similar in the KO, heterozygous, and wild-type mice.

Reinforcer magnitude attenuates: apomorphine's effects on operant pecking.

When given to pigeons, the direct-acting dopamine agonist apomorphine elicits pecking. The response has been likened to foraging pecking because it bears remarkable similarity to foraging behavior, and it is enhanced by food deprivation. On the other hand, other data suggest the response is not related to foraging behavior and may even interfere with food ingestion. Although elicited pecking interferes with food capture, it may selectively alter procurement phases of feeding, which can be isolated in operant preparations. To explore the relation between operant and elicited pecking, we provided pigeons the opportunity to earn different reinforcer magnitudes during experimental sessions. During signaled components, each of 4 pigeons could earn 2-, 4-, or 8-s access to grain for a single peck made at the end of a 5-min interval. In general, responding increased as a function of reinforcer magnitude. Apomorphine increased pecking for 2 pigeons and decreased pecking for the other 2. In both cases, apomorphine was more potent under the component providing the smallest reinforcer magnitude. Analysis of the pattern of pecking across the interval indicated that behavior lost its temporal organization as dose increased. Because apomorphine-induced pecking varied inversely with reinforcer magnitude, we conclude that elicited pecks are not functionally related to food procurement. The data are consistent with the literature on behavioral resistance to change and suggest that the effects of apomorphine may be modulated by prevailing stimulus-reinforcer relationships.

Delay discounting in Lewis and Fischer 344 rats: steady-state and rapid-determination adjusting-amount procedures.

Lewis rats have been shown to make more impulsive choices than Fischer 344 rats in discrete-trial choice procedures that arrange fixed (i.e., nontitrating) reinforcement parameters. However, nontitrating procedures yield only gross estimates of preference, as choice measures in animal subjects are rarely graded at the level of the individual subject. The present study was designed to examine potential strain differences in delay discounting using an adjusting-amount procedure, in which distributed (rather than exclusive) choice is observed due to dynamic titration of reinforcer magnitude across trials. Using a steady-state version of the adjusting-amount procedure in which delay was manipulated between experimental conditions, steeper delay discounting was observed in Lewis rats compared to Fischer 344 rats; further, delay discounting in both strains was well described by the traditional hyperbolic discounting model. However, upon partial completion of the present study, a study published elsewhere (Wilhelm & Mitchell, 2009) demonstrated no difference in delay discounting between these strains with the use of a more rapid version of the adjusting-amount procedure (i.e., in which delay is manipulated daily). Thus, following completion of the steady-state assessment in the present study, all surviving Lewis and Fischer 344 rats completed an approximation of this rapid-determination procedure in which no strain difference in delay discounting was observed.

The potency of fluvoxamine to reduce ethanol self-administration decreases with concurrent availability of food.

The selective serotonin reuptake inhibitor fluvoxamine reduces responding for ethanol at lower doses than responding for food when each is available in separate components or separate groups of rats. However, when both are available concurrently and deliveries earned per session are equal, this apparent selectivity inverts and food-maintained behavior is more sensitive than ethanol-maintained behavior to rate-decreasing effects of fluvoxamine. Here, we investigated further the impact that concurrent access to both food and ethanol has on the potency of fluvoxamine. Fluvoxamine (5.6-17.8 mg/kg) potency was assessed under conditions in which food and ethanol were available concurrently and response rates were equal [average variable intervals (VIs) 405 and 14 s for food and ethanol, respectively], as well as when density of food delivery was increased (average VI 60 s for food and VI 14 s for ethanol). The potency of fluvoxamine was also determined when only ethanol was available (food extinction and average VI 14 s for ethanol) and under multiple VIs (VI 30 s for food and ethanol) wherein either food or ethanol was the only programmed reinforcement available during each component. Fluvoxamine was less potent at decreasing ethanol self-administration when food was available concurrently {ED50 [95% confidence limit (CL): 8.2 (6.5-10.3) and 10.7 (7.9-14.4)]} versus when ethanol was available in isolation [ED50: 4.0 (2.7-5.9) and 5.1 (4.3-6.0)]. Effects on food were similar under each condition in which food was available. The results demonstrate that the potency of fluvoxamine in reducing ethanol-maintained behavior depends on whether ethanol is available in isolation or in the context of concurrently scheduled food reinforcement.

Reinforcement magnitude modulation of rate dependent effects in pigeons and rats.

Response rate can influence the behavioral effects of many drugs. Reinforcement magnitude may also influence drug effects. Further, reinforcement magnitude can influence rate-dependent effects. For example, in an earlier report, we showed that rate-dependent effects of two antidepressants depended on reinforcement magnitude. The ability of reinforcement magnitude to interact with rate-dependency has not been well characterized. It is not known whether our previous results are specific to antidepressants or generalize to other drug classes. Here, we further examine rate-magnitude interactions by studying effects of two stimulants (d-amphetamine [0.32-5.6 mg/kg] and cocaine [0.32-10 mg/kg]) and two sedatives (chlordiazepoxide [1.78-32 mg/kg] and pentobarbital [1.0-17.8 mg/kg]) in pigeons responding under a 3-component multiple fixed-interval (FI) 300-s schedule maintained by 2-, 4-, or 8-s of food access. We also examine the effects of d-amphetamine [0.32-3.2 mg/kg] and pentobarbital [1.8-10 mg/kg] in rats responding under a similar multiple FI300-s schedule maintained by 2- or 10- food pellet (45 mg) delivery. In pigeons, cocaine and, to a lesser extent, chlordiazepoxide exerted rate-dependent effects that were diminished by increasing durations of food access. The relationship was less apparent for pentobarbital, and not present for d-amphetamine. In rats, rate-dependent effects of pentobarbital and d-amphetamine were not modulated by reinforcement magnitude. In conclusion, some drugs appear to exert rate-dependent effect which are diminished when reinforcement magnitude is relatively high. Subsequent analysis of the rate-dependency data suggest the effects of reinforcement magnitude may be due to a diminution of drug-induced increases in low-rate behavior that occurs early in the fixed-interval.

Delay discounting in C57BL/6J and DBA/2J mice: adolescent-limited and life-persistent patterns of impulsivity.

Impulsivity is a defining characteristic of adolescence. Compared to adults, for example, adolescents engage in higher rates of drug and alcohol experimentation, risky sexual practices, and criminal activity. Such behavior may reflect reduced sensitivity to long-term consequences of behavior during adolescence. Recently, our lab has attempted to refine mouse procedures to study developmental trends in decision making in the laboratory. In the present experiment, we examined sensitivity to delayed rewards in C57BL/6J (B6) and DBA/2J (D2) mice during adolescence and adulthood using an adaptation of a 2-week delay discounting procedure developed by Adriani and Laviola (2003). During training, mice could choose between a 20- or 100-µl drop of milk delivered after a 1-s delay. During testing, the delay to the large drop of milk was increased from 1 to 100 seconds. As the delay to the larger volume increased, preference shifted to the smaller, more immediate option. In adolescence, both strains showed similar shifts in preference. In contrast, adult B6 mice were less sensitive to increasing delays than were adult D2 mice, who continued to perform much as their adolescent counterparts. A subsequent resistance-to-extinction test ruled out the possibility that the slower change in the adult B6 mice was due to perseverative responding. The present findings suggest that B6 and D2 strains may be differentially suited to uncovering the biological mechanism of short-term and long-term patterns of impulsive behavior.