Controlling and Predicting Unpredictable Behavior.

Behaving predictably can be advantageous in some situations, but unpredictability can also be advantageous in some competitive situations like sports, games, and war. Can, however, unpredictable behavior be conditioned? If a contingency of reinforcement based upon the predictability of behavior generates unpredictable responding, is it possible to conclude that predictability is itself a reinforceable dimension of behavior? In this paper, I address these questions by examining the concept and measures of predictability and the procedures generally used to increase unpredictable responding. I discuss the hypothesis that contingencies based on response frequency shape the generalized operant "to vary" and an alternative hypothesis that such contingencies generate unpredictable responding by balancing the strength of each alternative response over time. I discuss the findings that support the balance hypothesis as well as its limitations. I conclude that the two alternative hypotheses may be complementary in explaining unpredictable responding.

Negative reinforcement of behavioral variability without previous training with positive reinforcement.

In a previous study, Fonseca Júnior and Hunziker (2017) provided evidence of the control of behavioral variability by avoidance contingencies. However, their study presented three methodological weaknesses: (1) only one contingency reversal was conducted in the experimental design adopted, (2) the lever press was shaped using a positive reinforcement procedure, and (3) the intermittency of reinforcement between Lag n and Yoke conditions was not successfully equaled. The present study aimed to overcome these three methodological limitations. Three experimentally naive male rats with no prior history with positive reinforcement had sequences of three lever presses between two levers negatively reinforced under the following discrete trial avoidance contingencies that provided similar intermittency of reinforcement: Lag 1, Yoke, Lag 1, and Yoke. All subjects showed high levels of behavioral variability under the Lag 1 contingency and decreased levels of behavioral variability under the Yoke contingency. These results replicate the findings of Fonseca Júnior and Hunziker, suggesting that behavioral variability can be controlled by avoidance contingency without previous exposure to positive reinforcement. Moreover, these findings suggest that the high levels of variation obtained under the Lag n condition are an effect of operant contingency and not a mere byproduct of the intermittency of reinforcement.

The voluntary operant and the operant nature of volition: Three views.

Many philosophers, psychologists, and lay folk associate volition with autonomy (actions are independent of an individual's environment) and free will (individuals originate their actions). Most behaviorists hold these views to be incompatible with behavior analyses. The present paper describes volition as interpreted by B. F. Skinner, Howard Rachlin, and Allen Neuringer. Skinner relates volition to positively reinforced operant behavior. That works because, like operants, voluntary actions are free, in the sense of not physically constrained; they affect their environments, often resulting in positive outcomes, and are sometimes unpredictable. Rachlin, while incorporating Skinnerian methods, interprets volition within his own Teleological Behaviorism framework. For Rachlin, reinforcement of an individual response is often incompatible with voluntary control, thereby disagreeing with Skinner. Responses are voluntary only when they are members of extended response patterns. Neuringer also begins with Skinner's operants, but argues that, under the control of reinforcing consequences, both voluntary actions and operant responses are sometimes predictable and other times "truly" unpredictable. Neuringer does not assume that environments determine voluntary actions, thereby disagreeing with Skinner and Rachlin. Taken together, the agreements and disagreements among these three behaviorists may help to shed light on the relationship between operants and volition.

Effect of signaled reinforcement on response variability.

Three experiments examined the effect of signaling reinforcement on rats' lever pressing on contingencies that reinforced variable responding to extend the exploration of signaled reinforcement to a schedule that has previously not been examined in this respect. In Experiment 1, rats responding on a lag-8 variability schedule with signaled reinforcement displayed greater levels of variability (U values) than rats on the same schedule lacking a reinforcement signal. In Experiment 2, rats responding on a differential reinforcement of least frequent responses schedule also displayed greater operant variability with a signal for reinforcement compared with rats without a reinforcement signal. In Experiment 3, a reinforcement signal decreased the variability of a response sequence when there was no variability requirement. These results offer empirical corroboration that operant variability responds to manipulations in the same manner as do other forms of operant response and that a reinforcement signal facilitates the emission of the required operant.

Operant variability: a conceptual analysis.

Some researchers claim that variability is an operant dimension of behavior. The present paper reviews the concept of operant behavior and emphasizes that differentiation is the behavioral process that demonstrates an operant relation. Differentiation is conceived as change in the overlap between two probability distributions: the distribution of reinforcement probability as a function of some response property (S distribution) and the probability distribution of the response property itself (R distribution). This concept implies that the differentiation process can be measured only if S distribution and R distribution are both established on the same response property. To determine whether the differentially reinforced behavioral variability fits the proposed concept of operant behavior, I examine the main procedures (lag n and threshold procedures) and the main dependent variable (U value) employed in the studies of operant variability. Because lag n and threshold procedures establish their S distributions on properties distinct from U value, differentiation cannot be measured over the change in U value. I conclude that studies of operant variability have failed to provide a direct demonstration that variability is an operant dimension of behavior. Hence, studies in which measures of variability provide a basis to measure differentiation can better support the claim that variability is an operant dimension of behavior.

An investigation of resurgence of reinforced behavioral variability in humans.

The present study examined resurgence of reinforced variability in college students, who completed a 3-phase computer-based variability task. In the first phase, baseline, points were delivered for drawing rectangles that sufficiently differed from previous rectangles in terms of a target dimension (size or location, counterbalanced) but were sufficiently similar in terms of the alternative dimension. In the second phase, alternative, points were only delivered for rectangles that were sufficiently different in terms of the alternative dimension, but repetitive in terms of the target dimension. In the third phase, extinction, no points were delivered. In baseline, participants made rectangles that were highly varied in terms of the target dimension and less varied in terms of the alternative dimension, and vice versa in the alternative phase. During extinction, levels of variability increased for the target dimension, providing evidence for resurgence of reinforced variability of a specific dimension of behavior. However, levels of variability also remained high for the alternative dimension, indicating that extinction-induced response variability may also have impacted the results. Although future research is needed to explore other explanations, the results of this study replicate prior research with pigeons and provide some support for the notion of variability as an operant.

Effects of a Lag Schedule with Progressive Time Delay on Sign Mand Variability in a Boy with Autism.

For some children with autism, mand training can produce highly repetitive manding unless the environment is arranged in a manner that promotes mand variability. Prior research demonstrated that mand training using a lag schedule and progressive time delay increased variability in vocal manding in children with autism. Whether lag schedules have similar effects on sign mand topographies is unknown. The current study evaluated the effects of mand training with a Lag 1 schedule of reinforcement and progressive time delay (TD) on topographical variability and the development of a sign mand response class hierarchy in a boy with autism. The results suggest independent use of all sign mand topographies occurred, a mand response class hierarchy was developed, and topographically variant sign manding increased under the Lag 1 + TD schedule compared to a Lag 0 schedule of reinforcement. Implications for practitioners, limitations, and directions for future research are discussed.

Persistence and relapse of reinforced behavioral variability.

The present study examined persistence and relapse of reinforced behavioral variability in pigeons. Pigeons emitted four-response sequences across two keys. Sequences produced food according to a lag schedule, in which a response sequence was followed by food if it differed from a certain number of previous sequences. In Experiment 1, food was delivered for sequences that satisfied a lag schedule in both components of a multiple schedule. When reinforcement was removed for one component (i.e., extinction), levels of behavioral variability decreased for only that component. In Experiment 2, food was delivered for sequences satisfying a lag schedule in one component of a multiple schedule. In the other component, food was delivered at the same rate, but without the lag variability requirement (i.e., yoked). Following extinction, levels of behavioral variability returned to baseline for both components after response-independent food delivery (i.e., reinstatement). In Experiment 3, one group of pigeons responded on a lag variability schedule, and the other group responded on a lag repetition schedule. For both groups, levels of behavioral variability increased when alternative reinforcement was suspended (i.e., resurgence). In each experiment, we observed some evidence for extinction-induced response variability and for variability as an operant dimension of behavior.

Reinforcement of variability facilitates learning in humans.

Studies with rats and pigeons showed that reinforcement of response variability improved learning of difficult response sequences. The results suggested that concurrent reinforcement of variability might be a helpful tool when educators or therapists attempt to teach individuals with learning difficulties. However, similar experiments with humans failed to confirm the results. In fact, in the human case, concurrent reinforcement of variability interfered with learning. The present experiment studied the same phenomenon with human participants in the context of a computer-based game. Our results were consistent with the nonhuman animal findings. When students in our experiment were concurrently reinforced for sequence variability, they were more likely than control participants to learn a difficult response sequence. We conclude that reinforcement of variability can facilitate learning-in humans as well as animals -and discuss possible reasons for the difference between our results and the previous human findings.

Behavioral variability as avoidance behavior.

This study aimed to investigate whether variable patterns of responses can be acquired and maintained by negative reinforcement under an avoidance contingency. Six male Wistar rats were exposed to sessions in which behavioral variability was reinforced according to a Lag contingency: Sequences of three responses on two levers had to differ from one, two or three previous sequences for shocks to be avoided (Lag 1, Lag 2 and Lag 3, respectively). Performance under the Lag conditions was compared with performance on a Yoke condition in which the animals received the same reinforcement frequency and distribution as in the Lag condition but behavioral variability was not required. The results showed that most of the subjects varied their sequences under the Lag contingencies, avoiding shocks with relatively high probability (≥ 0.7). Under the Yoke procedure, responding continued to occur with high probability, but the behavioral variability decreased. These results suggest that behavioral variability can be negatively reinforced.

Reinforced behavioral variability: Working towards an understanding of its behavioral mechanisms.

There is disagreement about how to characterize the environment-behavior relations involved in the reinforcement of behavioral variability. The present research examined some of these issues using food-maintained, 4-peck sequences in pigeons. Experiment 1 evaluated the claim that behavioral variability is not reinforced directly but, rather, is the byproduct of changing over within sequences. Considerably higher levels of behavioral variation occurred under a relative-frequency threshold contingency than under a contingency that required a changeover but not variability per se. These results are consistent with the argument that behavioral variability is reinforced directly. Experiment 2 assessed the effects on variation levels of manipulating inter-trial and inter-response intervals. Variability increased with longer inter-response intervals but not with longer inter-trial intervals. These results are consistent with multiple explanations, including the notion that remembering past behavior interferes with the emission of reinforced variation. Consequently, Experiment 3 examined more directly the relation between remembering and reinforced variation. Variation levels were not affected by a concurrent contingency that encouraged pigeons to remember their past behavior. The implications of this research are presented in the context of working towards an understanding of the environment-behavior relations involved in the reinforcement of behavioral variability.

Avoidance and behavioural flexibility in obsessive compulsive disorder.

Three experiments measured differences in responding between participants scoring either higher or lower on obsessive-compulsive trait measures. A Sidman avoidance procedure was employed in Experiment 1, in which participants were required to identify an avoidance response that postponed an aversive event, and noted that higher scorers maintained this response more successfully. Experiments 2 and 3 involved an operant variability procedure to differentiate between variable and rigid responding among participants demonstrating high versus low obsessive-compulsive traits, and revealed no differential sensitivity to rigid responding between the groups. The results provide insight into the nature of obsessive-compulsive behavioural traits, suggesting that avoidance but not stereotypy is primary in OCD.