Divided stimulus control depends on differential and nondifferential reinforcement: Testing a quantitative model.

We investigated the effects of differential and nondifferential reinforcers on divided control by compound-stimulus dimensions. Six pigeons responded in a delayed matching-to-sample procedure in which a blue or yellow sample stimulus flashed on/off at a fast or slow rate, and subjects reported its color or alternation frequency. The dimension to report was unsignaled (Phase 1) or signaled (Phase 2). Correct responses were reinforced with a probability of .70, and the probability of reinforcers for errors varied across conditions. Comparison choice depended on reinforcer ratios for correct and incorrect responding; as the frequency of error reinforcers according to a dimension increased, control (measured by log d) by that dimension decreased and control by the other dimension increased. Davison and Nevin's (1999) model described data when the dimension to report was unsignaled, whereas model fits were poorer when it was signaled, perhaps due to carryover between conditions. We are the first to test this quantitative model of divided control with reinforcers for errors and when the dimension to report is signaled; hence, further research is needed to establish the model's generality. We question whether divided stimulus control is dimensional and suggest it may instead reflect joint control by compound stimuli and reinforcer ratios.

Is superstitious responding a matter of detectability? A replication of Killeen (1978).

Organisms may sometimes behave as if a contingency exists between behavior and consequences, even if this is not actually the case. Killeen (1978) suggested that such superstition occurs because of factors that bias subjects to behave "superstitiously" rather than because of failures of discrimination. We systematically replicated Killeen's experiment and compared contingency discrimination between different consequences. Six pigeons responded in a matching-to-sample procedure in which a response-independent or response-dependent stimulus change, food delivery, or blackout occurred. The pigeons reported whether the consequence was response dependent or response independent by choosing between two side keys. Discrimination was strongest after stimulus changes, weaker after blackouts, and weakest after food deliveries. These differences persisted even after additional training, suggesting asymmetries that may reflect differences in the disruptive effects of different consequences on remembering and/or behavioral mnemonics. Importantly, the pigeons were not biased to report response-dependent consequences unless that response was consistent with locational biases; that is, they behaved "superstitiously" when there was a reason to be biased to do so. These findings corroborate Killeen's and demonstrate that behavior may deviate from contingencies not necessarily because subjects cannot discriminate those contingencies but because they are biased to behave otherwise.

Revaluation of overselected stimuli: Emergence of control by underselected stimuli depends on degree of overselectivity.

Stimulus overselectivity describes strong control by one stimulus element at the expense of other equally relevant elements. Research suggests that control by underselected stimuli emerges following extinction of the overselected stimulus ("revaluation") and the emergence is larger when overselectivity is greater. We compared such revaluation effects with a control compound or condition in two experiments. Human participants chose between compound S+ and S- stimuli. Then, to assess control by compound-stimulus elements, participants chose between individual elements in a testing phase without feedback. The S+ element chosen most often (the overselected element) underwent revaluation, during which choice of that element was extinguished and choice of a novel element reinforced. Thereafter, participants completed a retesting phase. Revaluation reduced choice of the overselected element. Choice of the underselected element decreased for participants with low overselectivity but increased for participants with high overselectivity. This was not the case for a control compound that did not undergo revaluation (Experiments 1 and 2) or in a control condition in which the overselected element continued to be reinforced during revaluation (Experiment 2). These findings suggest that overselectivity levels may modulate revaluation effects, and they also highlight the importance of the contingency change in postrevaluation changes in stimulus control.

Effects of brief post-sample cues signaling presence or absence of reinforcers in delayed matching-to-sample.

When short-term memory is assessed in the delayed matching-to-sample (DMTS) procedure, performance is better when cues signal larger reinforcer magnitudes or higher reinforcer probabilities for correct responding. Previous studies demonstrating signaled-magnitude or signaled-probability effects presented cues for a prolonged period during the sample stimulus and/or retention interval. The present study asked whether a signaled-probability effect would occur with brief post-sample cues that signaled the presence or absence of reinforcement. Five pigeons responded in a DMTS task in which sample stimuli were sometimes followed by a 0.5-s cue signaling that reinforcers would either be available or not available in the current trial, and the retention interval varied from 0.5 s to 20 s. A reliable signaled-probability effect was found when reinforcers were arranged independently and for all correct responses, whereas a smaller, less systematic effect was found when reinforcers were arranged dependently and probabilistically. These findings highlight the importance of reinforcement contingencies and contingency discriminability in remembering, and add to the evidence showing that cues signaling differential reinforcement in DMTS may affect processes during the retention interval and comparison phase, rather than attention to the sample stimulus.

Learned control of urinary reflexes in cattle to help reduce greenhouse gas emissions.

Indiscriminate voiding of excreta by cattle contributes to greenhouse gas (GHG) emissions and soil and water contamination1,2. Emissions are higher in animal-friendly husbandry offering cattle more space2 - a trade-off we call the 'climate killer conundrum'. Voiding in a specific location (latrine) would help resolve this dilemma by allowing ready capture and treatment of excreta under more spacious farming conditions. For urination, toileting requires self-control and coordination of a complex chain of behaviors including awareness of bladder fullness, overriding of excretory reflexes, selection of a latrine and intentional relaxation of the external urethral sphincter3. Attempts to train toileting in cattle have so far been only partly successful4-6, even though their excretion and associated neurophysiological control are similar to those in species capable of toileting3. Similarly, very young infants have been considered incapable of self-initiated voiding, but they can be taught with extensive training7. Using a backward chaining, reward-based training procedure, we here show that cattle can control their micturition reflex and use a latrine for urination. Such self-control provides evidence that animals can learn to respond to and reveal internal experiences via appropriately trained operant behaviors, thereby providing another way to explore their subjective states.

How can Cattle be Toilet Trained? Incorporating Reflexive Behaviours into a Behavioural Chain.

Untrained cattle do not defecate or urinate in defined locations. The toilet training of cattle would allow urine and faeces to be separated and stored, reducing climate-damaging emissions and improving animal health. In a proof-of-concept study, we evaluated a novel protocol for toilet training in cattle. Five heifer calves (and yoked controls) were trained in the voluntary (operant) behaviours of a toileting chain. Then, reflexive urinating responses were incorporated into the chain, with toileting signalled by a tactile (vibratory) stimulus. On 95% of occasions, the calves inhibited/interrupted urination when receiving the stimulus, and on 65% of these occasions, reinitiated urination in the latrine. Furthermore, during 63% of urinations in the latrine, the calves oriented to the reward location before any food was delivered, providing additional evidence that calves can be successfully toilet trained with food rewards. Yoked controls failed to learn most of the operant elements and all the reflexive responses of toilet training. The results show that reflexive behaviours can be incorporated into voluntary toileting sequences with cattle and extend the range of species that can be toilet trained. Future refinement of the protocol to allow training under practical farm conditions offers the potential to mitigate climate damage and improve animal health.

Conditionability of 'voluntary' and 'reflexive-like' behaviors, with special reference to elimination behavior in cattle.

Typically, cattle urinate and defecate with little or no control over time and place. The resulting excreta contributes to a range of adverse effects on the environment and the animals themselves. These adverse effects could be substantially ameliorated if livestock could be toilet trained. Toilet training requires an animal to suppress impending voiding (a reflexive-like behavior), move to a latrine (voluntary behavior) and reinitiate voiding. Here, we review the neurophysiological processes and learning mechanisms regulating toileting. The suppression and initiation of voiding occur primarily via the coordinated activity of smooth and striated anal and urinary sphincter muscles. The autonomic and somatic nervous systems, along with central processes, regulate these muscles. In several mammalian species, voluntary control of the sphincters has been demonstrated using classical and/or operant conditioning. In this review, we demonstrate that the neurophysiological and behavioral regulation of voiding in cattle is likely to be similarly conditionable. The management of excreta deposition in cattle could have major benefits for reducing livestock greenhouse gas emissions and improving animal health/welfare.

Signaled reinforcement: Effects of signal reliability on choice between signaled and unsignaled alternatives.

When reinforcer availability on one alternative of a concurrent schedule is signaled by a discriminative stimulus, responding on that alternative decreases. We investigated how the correlation between signal presentation and reinforcement (signal reliability) affects choice between signaled and unsignaled alternatives. Six pigeons responded in a concurrent schedule, in which reinforcers on one alternative were signaled by a key-color change. Across conditions, the probability of reinforcement following signal presentation varied (the probability in its absence was the complement). As signal reliability increased, response rates and latencies following signal onset on the signaled alternative decreased, whereas responding on the unsignaled alternative remained unchanged. Because the signal did not alter overall reinforcer rates, these findings are consistent with matching theories and research suggesting that responding on one alternative of a concurrent schedule depends on reinforcer, but not response, rates on other alternatives. However, these findings are inconsistent with others demonstrating concomitant changes in responding on signaled and unsignaled alternatives. We consider whether a response-competition account of concurrent performance can explain these discrepancies, and suggest avenues for future studies to investigate the mechanisms underlying effects of signaled reinforcement in concurrent schedules.

Assessing potential reinforcement-like effects of brief stimuli unrelated to food reinforcers.

It is widely assumed that reinforcers are biologically relevant stimuli, or stimuli that have been associated with biologically relevant stimuli. However, brief, arbitrary stimuli have also been reported to have reinforcement-like effects, despite being unrelated to biologically relevant stimuli like food. The present study explored the potential reinforcement-like effects of brief stimuli across 5 experiments. In Experiments 1 through 4, pigeon subjects responded for food reinforcement and brief stimulus presentations in a 2-component multiple schedule. Neither baseline response rates nor resistance to change during disruption tests were systematically greater in a component with versus without brief stimulus presentations. Increasing the rate and duration of brief stimulus presentations in Experiment 4 did not reveal reinforcement-like effects when compared directly with food. In Experiment 5, pigeons chose between independent terminal links in a concurrent-chains procedure. Across conditions, varying the location, duration, and rate of brief stimulus presentations in the terminal links had no systematic effects on preference. In contrast, varying rates of food reinforcers resulted in large and reliable shifts in preference. Therefore, the present study found no systematic evidence that brief stimuli unrelated to food reliably increase response rates, resistance to change, or preference. These data demonstrate the value of systematic replication, and a behavioral momentum approach to assessing potential reinforcement-like effects.

Timing compound stimuli: Relative reinforcer probabilities divide stimulus control in the multiple peak procedure.

Previous research suggests that when multiple stimuli signal the location of future reinforcers, the extent of control by each stimulus depends on the relative reinforcer probability associated with that stimulus. In this experiment, we asked whether relative reinforcer probabilities also divide control between stimuli that signal the time to future reinforcers. Six pigeons responded in a multiple peak procedure in which 2 dimensions of a compound stimulus-a color (red or green) and a frequency of on/off alternation (fast or slow)-signaled a fixed-interval 2- or 8-s schedule. Across conditions, we varied the probability of reinforcer deliveries associated with each dimension from .1 to .9. Relative reinforcer probabilities determined the degree of control by each stimulus dimension; as the probability of reinforcer deliveries associated with one dimension increased, a peak in response rates at the time signaled by that dimension became apparent while a peak at the time signaled by the other dimension diminished. However, these effects of relative reinforcers were smaller than in previous research, probably because elapsed time also competed for control over behavior. These findings extend the relation between relative reinforcers and divided stimulus control from spatial to temporal discriminations. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Quantifying the effects of the differential outcomes procedure in humans: A systematic review and a meta-analysis.

We present a systematic review and a meta-analysis comparing the differential outcomes procedure to a nondifferential outcomes procedure among clinical and nonclinical populations. Sixty distinct experiments were included in the systematic review, 43 of which were included in the meta-analysis. We calculated pooled effect sizes for accuracy (overall accuracy, test accuracy, transfer accuracy) and acquisition outcomes (latency, errors, and trials to mastery). The meta-analysis revealed significant medium-to-large effect sizes for all three accuracy measures (pooled effect size range, 0.57 to 1.30). We found relatively greater effect sizes among clinical populations (effect size = 1.04). The single-subject experimental literature included in the systematic review was consistent with the findings from the group studies, demonstrating improvements in accuracy and speed of learning for the majority of participants. Moderator and subgroup analyses suggest that discrimination difficulty may induce relatively larger differential outcomes effects. The results indicate that the differential outcomes procedure can be a valuable addition to reinforcement-based interventions.

Relative reinforcer rates determine pigeons' attention allocation when separately trained stimuli are presented together.

Previous research suggests that organisms allocate more attention to stimuli associated with higher reinforcer rates. This finding has been replicated several times when stimuli are trained together as compounds, but not in other procedures. Thus, the generality of the relation between relative reinforcer rates and divided attention is not well established. Therefore, we investigated whether relative reinforcer rates determine attention allocation when stimuli are trained separately and then encountered together. Pigeons learned to associate two colors and two frequencies of key light on/off alternation with a left or right comparison key in a symbolic 0-s delayed matching-to-sample task. Across conditions, we varied the probability of reinforcement associated with each stimulus dimension during training. After training, we introduced test trials in which a color and flash-frequency stimulus were presented simultaneously. During sample-stimulus presentation in test trials, all pigeons preferred the stimulus associated with the higher reinforcer rate, suggesting that more attention was allocated to that stimulus. Interestingly, such attention allocation did not result in preference for the comparison that matched that stimulus. Instead, all pigeons preferred the comparison that was physically closer to the stimulus associated with the higher reinforcer rate, suggesting that comparison choice was controlled by the location of that stimulus. Nevertheless, overall, our results provide the first evidence that relative reinforcer rates determine divided attention between separately trained stimuli and thus demonstrate the generality of the relation between relative reinforcement and attention allocation. We suggest several avenues for future research to establish further the generality of this relation.

The nanoeconomics of concurrent choice behavior.

We introduce to behavior analysis a way of analyzing choice behavior that exploits recent developments in nanoeconomics, financial economics, and econometrics. A response return, modeled on an economic return, is the log differenced count of responses allocated to each of two alternatives during a short time window, compared with that in the immediately preceding window. The response return is a new dependent variable which offers a novel and useful way of looking at operant behavior, especially at the molecular level of analysis. The response-return series is a near-instantaneous measure of an organism's dynamic preferences for each of two alternatives. Analyzing such a series requires time-analytic techniques, including Auto-Regressive Conditional Heteroskedastic (ARCH) models. We illustrate these techniques by analyzing choices between combinations of arithmetic and exponential variable-interval schedules with pigeon subjects. All response-return series were well-fitted by one of three ARCH-family models. The fitted models were differentially sensitive to transition versus steady-state data samples. The particular insights that the ARCH analyses offer are improved understanding of the effects of the instantaneous effects of reinforcers and their absence, of how the distribution of reinforcers in time affects choice, and of the differences between choice in transition and at steady state.

Rank-permutation tests for behavior analysis, and a test for trend allowing unequal data numbers for each subject.

We advocate for rank-permutation tests as the best choice for null-hypothesis significance testing of behavioral data, because these tests require neither distributional assumptions about the populations from which our data were drawn nor the measurement assumption that our data are measured on an interval scale. We provide an algorithm that enables exact-probability versions of such tests without recourse to either large-sample approximation or resampling approaches. We particularly consider a rank-permutation test for monotonic trend, and provide an extension of this test that allows unequal number of data points, or observations, for each subject. We provide an extended table of critical values of the test statistic for this test, and both a spreadsheet implementation and an Oracle® Java Web Start application to generate other critical values at https://sites.google.com/a/eastbayspecialists.co.nz/rank-permutation/.

Environment tracking and signal following in a reinforcer-ratio reversal procedure.

Several studies suggest that the degree of control by reinforcer ratios (environment tracking) and by exteroceptive stimuli that signal future reinforcer availability (signal following) depends on environmental certainty: As reinforcers become more likely at one location, environmental contingencies exert stronger control and exteroceptive stimuli exert weaker control. This research has not yet been extended to environments in which reinforcer availability changes across time, even though such changes are present in most natural environments. Thus, in the present experiment, we examined environment tracking and signal following in a concurrent schedule in which the reinforcer ratio reversed to its reciprocal 30 s after a reinforcer delivery and keylight-color stimuli signaled the likely or definite time or location of the next reinforcer. Across conditions, we manipulated environmental certainty by varying the probability of reinforcer deliveries on the locally richer key. This made the location of future reinforcers at a particular time more or less certain, but did not change the overall reinforcer ratio. Changes in local environmental certainty had little to no effect on environment tracking and signal following; in all conditions, keylight-color stimuli strongly controlled choice and reinforcer ratios exerted weak control. The present findings suggest that the extent of environment tracking and signal following is primarily determined by global, not local, environmental certainty.

A multivariate assessment of the rapidly changing procedure with McDowell's Evolutionary Theory of Behavior Dynamics.

A multivariate analysis is concerned with more than one dependent variable simultaneously. Models that generate event records have a privileged status in a multivariate analysis. From a model that generates event records, we may compute predictions for any dependent variable associated with those event records. However, because of the generality that is afforded to us by these kinds of models, we must carefully consider the selection of dependent variables. Thus, we present a conditional compromise heuristic for the selection of dependent variables from a large group of variables. The heuristic is applied to McDowell's Evolutionary Theory of Behavior Dynamics (ETBD) for fitting to a concurrent variable-interval schedule in-transition dataset. From the parameters obtained from fitting ETBD, we generated predictions for a wide range of dependent variables. Overall, we found that our ETBD implementation accounted well for various flavors of the log response ratio, but had difficulty accounting for the overall response rates and cumulative reinforcer effects. Based on these results, we argue that the predictions of our ETBD implementation could be improved by decreasing the base response probabilities, either by increasing the response latencies or by decreasing the sizes of the operant classes.

Performance on the rapidly changing procedure, according to an associative learner.

From a model that generates event records, we may generate predictions for any dependent variable computable from those event records. In the present paper, we examined an implementation of Shimp's (1978) Associative Learner for two-alternative concurrent variable-interval variable-interval schedules in transition. In general, our Associative Learner implementation had difficulty accounting for the inter-response times as a function of the reinforcer ratio, the general frequency of inter-response times, the acquisition of consecutive reinforcers from the same alternative, and the response rates to the not-just-reinforced alternative after a reinforcer. An extended Associative Learner implementation was designed with provisions for increasing responding to the just-reinforced alternative after a reinforcer and organising bout-like responses. The extended implementation rectified many of the weaknesses of the base implementation.

Generalization of response patterns in a multiple peak procedure.

Stimulus generalization is typically assessed by analyzing overall response rates. Studies of generalization of response-rate patterns across time are less common, despite the ubiquitous nature of time and the strong temporal control over behavior in the natural world. Thus, we investigated generalization of response-rate patterns across time using a multiple peak procedure in pigeons. The frequency (fast or slow) at which the color of a keylight changed signaled a fixed-interval (FI) 5-s or 20-s schedule, counterbalanced across subjects. In peak trials, the frequency of keylight-color changes was varied. For the fast and slow training stimuli, response rates in peak trials were controlled by the arranged FI schedule value; they increased as the arranged reinforcer time approached, and decreased thereafter. Response-rate patterns to all test stimuli were similar to response-rate patterns to the slow training stimulus for all subjects. Thus, overall, strong generalization from the slow training stimulus to all test stimuli was evident, whereas there was little to no generalization from the fast training stimulus. These findings extend past research examining generalization of temporally controlled response-rate patterns, and provide a useful starting point for future investigations of generalization of fixed-interval responding. A thorough understanding of generalization processes requires analysis of dependent variables other than overall response rates, especially when responding is likely to be temporally controlled.

Does a negative discriminative stimulus function as a punishing consequence?

The study and use of punishment in behavioral treatments has been constrained by ethical concerns. However, there remains a need to reduce harmful behavior unable to be reduced by differential-reinforcement procedures. We investigated whether response-contingent presentation of a negative discriminative stimulus previously correlated with an absence of reinforcers would punish behavior maintained by positive reinforcers. Across four conditions, pigeons were trained to discriminate between a positive discriminative stimulus (S+) signaling the presence of food, and a negative discriminative stimulus (S-) signaling the absence of food. Once learned, every five responses on average to the S+ produced S- for a duration of 1.5 s. S+ response rate decreased for a majority of pigeons when responses produced S-, compared to when they did not, or when a neutral control stimulus was presented. In Condition 5, choice between two concurrently presented S+ alternatives shifted away from the alternative producing S-, despite a 1:1 reinforcer ratio. Therefore, presenting contingent S- stimuli punishes operant behavior maintained on simple schedules and in choice situations. Development of negative discriminative stimuli as punishers of operant behavior could provide an effective approach to behavioral treatments for problem behavior and subverting suboptimal choices involved in addictions.