Selective effects of conspecific movement on social preference in zebrafish (Danio rerio) using real-time 3D tracking and 3D animation.

Zebrafish show social behavior such as shoaling and schooling, which is a result of complex and interdependent interactions among conspecifics. Zebrafish social behavior is interdependent in the sense that one fish's behavior affects both conspecific behavior and, as a result, their own behavior. Previous research examined effects of the interdependent interactions on the preference for social stimulus but lacked clear evidence that specific conspecific movements were reinforcing. The present research examined whether dependency between individual experimental fish's motion and a social-stimulus fish's motions contributes to preference for the social stimulus. In Experiment 1, a 3D animated stimulus fish either chased individual experimental fish or was motionless, serving as dependent and independent motions, respectively. In Experiment 2, the stimulus fish either chased experimental fish, moved away, or moved independently of the experimental fish. In both experiments, experimental fish spent more time near the stimulus fish showing dependent and interactive movements, indicating preference for dependent motion over independent motion, and chasing over other motions. Implications of these results are discussed including a potential role of operant conditioning in the preference for social stimuli.

A quantitative analysis of resurgence following downshifts in alternative-reinforcer magnitude.

Resurgence is the increase in a previously reinforced and then extinguished target response due to changes in reinforcement conditions for an alternative response, including reductions in the rate or magnitude of reinforcement for the alternative response. Research with nonhumans suggests that reductions in both alternative-reinforcer rate and magnitude produce resurgence, but the present study was the first to examine effects of downshifts in alternative-reinforcer magnitude on humans' resurgence. Moreover, it was the first to evaluate whether the quantitative framework, resurgence as choice in context (RaC2 ), could account for those effects. Consistent with predictions of RaC2 , resurgence of a target button press occurred with reductions in point gain for an alternative response, with greater reductions producing higher levels of resurgence. However, the model consistently underpredicted and then overpredicted resurgence during tests with low-magnitude reinforcement and extinction. Systematic deviations in model predictions of alternative responding were also evident and consistent with previous fits of RaC2 to nonhuman data. Overall, our findings suggest that RaC2 could be a useful quantitative theoretical framework for understanding processes contributing to resurgence in humans, but further theoretical development is needed to account for the apparent divergent effects of extinction versus downshifts in reinforcer magnitude.

Effects of signaled and unsignaled delays of reinforcement on response maintenance in zebrafish (Danio rerio).

Effects of delays of reinforcement on zebrafish behavior were examined following training with immediate reinforcement. The delay was either signaled by an exteroceptive stimulus present for the entire delay period (fully signaled), signaled briefly only at delay onset (partial signal), or without a stimulus (unsignaled). Unsignaled delays consistently resulted in low response rates. Fully signaled delays resulted in higher response rates than unsignaled delays when the delay was 3 s, but this difference in response rates disappeared at 6-s delays. Partially signaled delays were less effective in maintaining responding than fully signaled delays, but more effective than unsignaled delays, although these results were only suggestive. These results indicate that stimulus changes that occur during delays to reinforcement have similar effects with zebrafish as with other species, but also that responding of zebrafish has a relatively low tolerance to the delay duration. A vast majority of experiments examining zebrafish behavior suggests that the fish have potential to serve as an interface between biological and behavioral science, but this may not be the case in some research areas involving delays, such as delay discounting.

A quantitative analysis of the effects of alternative reinforcement rate and magnitude on resurgence.

Resurgence occurs when a previously reinforced and then extinguished target response increases due to a worsening of reinforcement conditions for an alternative response. We conducted four crowdsourcing experiments to evaluate effects of alternative-reinforcer rate and magnitude on resurgence with humans. Contingent on an alternative response, we manipulated across groups either the rate of point delivery (Experiment 1) or number of points delivered per reinforcer (Experiments 2-3). Experiment 4 arranged combinations of high- or low-rate and high- or low-magnitude alternative reinforcement across four groups. When extinguishing alternative responding across experiments, we observed resurgence of target responding with relatively high rates of alternative reinforcement but differences in reinforcer magnitude did not influence resurgence. A quantitative model based on the concatenated matching law, Resurgence as Choice in Context (RaC2), provided a poor fit to the data, generally underpredicted target responding, and could not account for data from control groups experiencing extinction of target responding in the absence of alternative reinforcement. We then fit a modified version of RaC2 borrowing an assumption from theories of choice that suggest some proportion of reinforcers are misallocated between responses - this modified version of RaC2 provided a better account of these findings.

Assessing human performance during contingency changes and extinction tests in reversal-learning tasks.

Serial reversal-learning procedures are simple preparations that allow for a better understanding of how animals learn about environmental changes, including flexibly shifting responding to adapt to changing reinforcement contingencies. The present study examined serial reversal learning with humans by arranging both midsession and variable contingency reversals across two experiments. We also examined the effects of extinction by adding nonreinforced trials at the end of later sessions and provided the first evaluation of effects of win-stay/lose-shift versus counting strategies on accuracy and response latency of humans' reversal-learning performance. In each experiment, responding tracked contingency reversals, primarily with participants using either win-stay/lose-shift or counting strategies. Introducing variable reversal points in the second experiment resulted in near-exclusive win-stay/lose-shift responding among participants and eliminated counting of trials. Each experiment also revealed an immediate shift from S2 to S1 after experiencing extinction during the initial test trial, indicating resurgence of the initial response through a win-stay/lose-shift response pattern. Therefore, the present study replicates and extends prior findings of a win-stay/lose shift response pattern in situations of greater uncertainty. These findings suggest that differences in environmental certainty induce qualitatively different decision-making strategies.

Blackouts can serve as a contextual feature and enhance resurgence.

Resurgence occurs when a worsening of conditions for an alternative response (e.g., extinction) increases a previously reinforced and subsequently extinguished target response. In contrast, renewal is an increase in a response previously eliminated by extinction following a contextual change. Moreover, arranging contextual changes during resurgence tests has enhanced relapse compared with the absence of contextual changes. Several laboratory studies evaluating resurgence in humans have included interruptions of operant tasks by presenting instructions or "blackouts" immediately prior to phase changes in which reinforcement contingencies change. Such interruptions could enhance relapse during testing similarly to changes in contextual stimuli. We tested this possibility in the present study with human participants recruited via Amazon Mechanical Turk by interrupting the task across groups with 1-s blackouts every 12 s, 60 s, between phases, or by arranging no blackouts in a control group. We found that blackouts prior to phase changes increased resurgence effects relative to no blackouts, suggesting that blackouts between phases are a type of contextual manipulation serving to increase the salience of contingency changes.

Evaluating effects of context changes on resurgence in humans.

Laboratory models of relapse provide methods for evaluating challenges to behavioral treatments with differential reinforcement of an alternative response (DRA). Resurgence occurs with the worsening of conditions of reinforcement for appropriate behavior and renewal occurs when transitioning out of a treatment context. Across five experiments, participants recruited via online crowdsourcing pressed onscreen buttons to earn points exchangeable for money and contexts sometimes changed through changes in the background image. Returning to the training context (ABA, Experiment 1) and transitioning to a novel context (ABC, Experiment 2) produced greater resurgence when removing alternative reinforcement in comparison with remaining in the treatment context (ABB). In contrast, we observed little difference in resurgence among AAA, ABB, and AAC context manipulations (Experiment 3) and ABA, ABC, and AAC context manipulations (Experiment 4). In Experiment 5, we evaluated relative contributions of the presence versus absence of context changes (ABA vs. ABB) in combination with or without the removal of alternative reinforcement. Both changing context and removing alternative reinforcement increased responding in isolation and the combination produced greater-than-additive effects. Overall, the present findings demonstrate a consistent effect of removing alternative reinforcement on relapse that, under certain conditions, can be enhanced by context change.

Examining effects of training duration on humans' resurgence and variability using a novel touchscreen procedure.

Resurgence occurs when a previously reinforced and then extinguished target response increases due to reducing/eliminating an alternative source of reinforcement or punishing an alternative response. We evaluated whether duration of reinforcement history for a target response (1) affects the degree to which resurgence is observed in humans and (2) produces different gradients of response generalization around target responding during extinction testing. We arranged a novel touchscreen interface in which university students could swipe a 3D soccer ball to spin any direction. In Phase 1, the first direction swiped became the target and produced points exchangeable for money for 3 or 1 min across 2 groups. The first swipe was recorded but had no programmed consequence in a third group. In Phase 2, swipes 180-degrees from the target resulted in points for 3 min in all groups. Point deliveries ceased for 2 min to test for resurgence in Phase 3. Target responses resurged during testing to a relatively greater extent with longer Phase-1 training but gradients of response generalization did not differ among groups. These findings extend prior research on the role of training duration on resurgence. We discuss methodological and conceptual issues surrounding the assessment of response generalization in resurgence.

Zebrafish choice behavior is sensitive to reinforcer rate, immediacy, and magnitude ratios.

Behavioral flexibility has, in part, been defined by choice behavior changing as a function of changes in reinforcer payoffs. We examined whether the generalized matching law quantitatively described changes in choice behavior in zebrafish when relative reinforcer rates, delays/immediacy, and magnitudes changed between two alternatives across conditions. Choice was sensitive to each of the three reinforcer properties. Sensitivity estimates to changes in relative reinforcer rates were greater when 2 variable-interval schedules were arranged independently between alternatives (Experiment 1a) than when a single schedule pseudorandomly arranged reinforcers between alternatives (Experiment 1b). Sensitivity estimates for changes in relative reinforcer immediacy (Experiment 2) and magnitude (Experiment 3) were similar but lower than estimates for reinforcer rates. These differences in sensitivity estimates are consistent with studies examining other species, suggesting flexibility in zebrafish choice behavior in the face of changes in payoff as described by the generalized matching law.

Evaluating Extinction, Renewal, and Resurgence of Operant Behavior in Humans with Amazon Mechanical Turk.

Amazon Mechanical Turk (MTurk) is a crowdsourcing marketplace providing researchers with the opportunity to collect behavioral data from remote participants at a low cost. Recent research demonstrated reliable extinction effects, as well as renewal and resurgence of button pressing with MTurk participants. To further examine the generality of these findings, we replicated and extended these methods across six experiments arranging reinforcement and extinction of a target button press. In contrast to previous findings, we did not observe as reliable of decreases in button pressing during extinction (1) after training with VR or VI schedules of reinforcement, (2) in the presence or absence of context changes, or (3) with an added response cost for button pressing. However, we found that that a 1-point response cost for all button presses facilitated extinction to a greater extent than the absence of response cost. Nevertheless, we observed ABA renewal of button pressing when changing background contexts across phases and resurgence when extinguishing presses on an alternative button. Our findings suggest that MTurk could be a viable platform from which to ask and address questions about extinction and relapse processes, but further procedural refinements will be necessary to improve the replicability of control by experimental contingencies.

Early extinction effects following intermittent reinforcement: Little evidence of extinction bursts.

The occurrence of extinction bursts-transient increases in response rate in excess of those observed in baseline during the period immediately following discontinuation of reinforcement of a response-was examined. In Experiment 1, key pecking of pigeons was reinforced according to a multiple schedule in which a variable-ratio schedule alternated with an interval schedule in which the reinforcers were yoked to the preceding variable-ratio component. In Experiments 2 and 3, rats were screened such that the lever-press response rates of different rats maintained by variable-interval schedules were either relatively high or relatively low. Following these baseline conditions, in Experiments 1 and 2 responding was extinguished by eliminating the food reinforcer and in Experiment 3 by removing the response-reinforcer dependency. Responses immediately following extinction implementation were examined. Response increases relative to baseline during the first 20 min of a 324.75-min extinction session (Experiment 1) or during the first 30-min extinction session (Experiments 2 and 3) were rare and unsystematic. The results (a) reinforce earlier meta-analyses concluding that extinction bursts may be a less ubiquitous early effect of extinction than has been suggested and (b) invite further experimentation to establish their generality as a function of preceding reinforcement conditions.

Effects of punishing target response during extinction on resurgence and renewal in zebrafish (Danio rerio).

We examined the effects of the presence and absence of punishment on the resurgence and renewal of extinguished operant behavior with zebrafish. With resurgence, food deliveries reinforced target responding in Phase 1 was exposed to shock punishment plus extinction (PUN + EXT) versus extinction alone (EXT) while introducing alternative reinforcement in Phase 2. All contingencies were discontinued in Phase 3. With renewal, target reinforcement during Phase 1 occurred in Context A and then during Phase 2 was exposed to either PUN + EXT or EXT in Context B. All contingencies were discontinued in Context A during Phase 3. During Phase 2 for both resurgence and renewal, decreases in target responding were more rapid with PUN + EXT than EXT. During testing in Phase 3, resurgence was less following PUN + EXT than EXT. In contrast, renewal was greater following PUN + EXT than EXT but differences in response rates at the end of Phase 2 complicated the interpretation. We discuss these differences between resurgence and renewal as due to differences in learning about alternative reinforcement (resurgence) versus more general contextual changes (renewal).

An experimental analysis of jackpot reinforcers.

Animal trainers and others often recommend the use of jackpot reinforcers, which are disproportionally large and come as a "surprise" to the animals. Because the actual behavioral effects of these jackpots remain uninvestigated, many basic questions about their use and even definition are unanswered. This series of experiments explored the definitions of jackpots using several different behavioral tests with both rats and pigeons. Because the original description of a jackpot resembled the reinstatement of previously reinforced and extinguished responding with response-independent deliveries of reinforcer, reinstatement effects of a jackpot, defined by its quality, were examined with rats in Experiment 1. In Experiment 2, response-potentiating effects of response-independent and -dependent deliveries of a jackpot, defined by its quantity, were assessed with pigeons when responding had nearly ceased. The response-potentiating effects of the frequency of jackpot-reinforcer delivery were investigated when responding of pigeons was maintained in single (Experiment 3) or concurrent (Experiment 4) schedules of reinforcement. Effects of jackpots on resistance to change were assessed with rats in Experiment 5. The effects of jackpots in each experiment were either absent or unsystematic across the subjects, casting doubt on their utility in animal training. Possible factors contributing to the negative results are discussed.

Repeated resurgence with and without a context change.

Resurgence is the reoccurrence of a target response when reinforcement for a more recently reinforced alternative response is eliminated or reduced. The present study arranged two successive three-phase procedures to assess whether resurgence decreases with repeated assessments. Moreover, we arranged a contextual change from the first to second assessment for some groups. Phase 1 reinforced a target response on a touchscreen computer with typically developing adults as participants according to either variable-ratio or variable-interval schedules of reinforcement. Phase 2 extinguished target responding and reinforced alternative responding. Phase 3 tested for resurgence by extinguishing alternative responding. Resurgence reliably occurred in all tests and decreased from the first to second exposure to the procedures but there were no effects of context change. Therefore, repeated exposures to resurgence tests reduced those effects but contextual changes had no effect.

Adventitious reinforcement during long-duration DRO exposure.

Differential reinforcement of other behavior (DRO) is a procedure often used to decrease problem behavior, but the processes responsible for behavior reduction are not well understood. This study assessed whether adventitious reinforcement of other behavior contributes to DRO effectiveness when, relative to previous research, DRO exposure is prolonged. Two response options were presented on a computer and target responding was reinforced on a variable-ratio schedule. Response rates were then compared during DRO versus yoked variable-time or extinction probes. Across 2 experiments, DRO decreased target responding and increased other responding more than control conditions. However, increases in other responding did not usually maintain despite target responding remaining at low levels. DRO might adventitiously reinforce other responses transiently but the decreases in target behavior could not be entirely explained by adventitious reinforcement of the other response. Instead, reductions in target responding likely depend on the discriminability of the DRO contingency.

The role of adventitious reinforcement during differential reinforcement of other behavior: A systematic replication.

Differential reinforcement of other behavior (DRO) is commonly used to decrease problem behavior by presenting reinforcers contingent upon the absence of a target response. Although it is well demonstrated that DROs decrease response rates, the processes producing these decreases are not well understood. The present study systematically replicated previous research assessing whether adventitious reinforcement of alternative behavior contributes to the effectiveness of DRO. We presented university students with two options on a computer and reinforced target responding on a variable-ratio schedule. Next, we compared decreases in target-response rates and any increases in alternative responding during DRO schedules versus yoked variable-time schedules or extinction probes. DRO schedules resulted in the lowest target-response rate and highest alternative-response rate. These findings generally provide some support for the adventitious reinforcement of "other" behavior.

Resurgence is greater following a return to the training context than remaining in the extinction context.

The present study examined whether resurgence of a previously reinforced target response upon removing alternative reinforcement would be greater when (1) returning to the original training context (ABA context changes) versus (2) remaining in the analogue treatment context in which the alternative response was differentially reinforced (ABB context changes). Experiment 1 arranged reinforcement of button pressing with points exchangeable for money in university students. Experiment 2 arranged reinforcement of lever pressing with food for rats. Experiment 3 arranged reinforcement of responses to a touchscreen with small bites of food with children diagnosed with ASD. Overall, resurgence of target responding tended to be greater when returning to the original training context (A) than when remaining in the analogue treatment context (B). These findings suggest context changes with differential reinforcement treatments could exacerbate the recurrence of problem behavior resulting from reductions in treatment integrity through failure to reinforce appropriate behavior.

Predator videos and electric shock function as punishers for zebrafish (Danio rerio).

Zebrafish (Danio rerio) are a promising animal model for studying the effects of gene-environment interactions on behavior. Two experiments were conducted to assess punishment effects of presenting predator videos (Indian leaf fish; Nandus nandus) and electric shock on operant approach responses in zebrafish. In Experiment 1, the predator video and shock stimuli were presented upon a response maintained by a single variable-interval schedule of food reinforcement in different groups of fish. In Experiment 2, the predator video and shock stimuli were presented upon one of two response alternatives maintain by concurrently available variable-interval schedules of food reinforcement in different groups of fish. Responding decreased when the predator video and shock stimuli were presented relative to their absence in both experiments. Moreover, responding on an unpunished alternative did not reliably decrease in Experiment 2. These results indicate that the decrease in responding resulted from the punishment contingency rather than from elicited species-specific defense responses or conditioned avoidance. Thus, the predator video and electric shock functioned as punishers of operant behavior for zebrafish. Identifying punishers for this species could lead to research on how gene-environment interactions influence individual differences in sensitivity to punishment.

A system for the real-time tracking of operant behavior as an application of 3D camera.

The capacity of 3D cameras to measure many different aspects of behavior (e.g., velocity, pattern, and posture) could contribute to the understanding of behavior. The present article describes a system for the real-time tracking of operant behavior, which is applicable to other domains of behavioral science as well. Methods for real-time 3D tracking of animal behavior are described, along with sample C++ programs. A demonstration using one zebrafish as a subject indicated that the present system successfully tracked the 3D motion of the fish. Moreover, the acquisition of a target response (i.e., approach to a corner of the aquarium) was demonstrated with the arrangement of a reinforcement contingency at the corner in the absence of a traditional, salient operandum. The system offers the capacity to characterize more completely ongoing behavior in learning tasks across a range of species than simply performance of discrete operant responses. The system also is capable of tracking multiple individuals simultaneously so it is possible both to study social interactions and arrange contingencies for engaging in social behavior. Other possible applications of 3D cameras are discussed.

Behavioral control by the response-reinforcer correlation.

Using a discrete-trials procedure, two experiments examined the effects of response-reinforcer correlations on responding while controlling molecular variables that operated at the moment of reinforcer delivery (e.g., response-reinforcer temporal contiguity, interresponse times preceding reinforcement). Each trial consisted of three successive components: Response, Timeout, and Reinforcement, with the duration of each component held constant. The correlation between the number of responses in the Response component and reinforcer deliveries in the Reinforcement component was varied. In the Positive-correlation condition, a larger number of responses in the Response component programmed a higher reinforcement rate (Experiment 1) or a shorter time to reinforcement (Experiment 2) in the Reinforcement component. Although programmed in this way, the actual reinforcer delivery was dependent on, and occurred immediately after, a response in the Reinforcement component. In the Zero-correlation condition, the programmed rates of reinforcement (Experiment 1) or the times to reinforcement (Experiment 2) in the Reinforcement component of each trial were yoked to those in the preceding Positive-correlation condition. Responding in the Response component was higher in the Positive- than in the Zero-correlation condition, without systematic changes in molecular variables. The results suggest that the response-reinforcer correlation can be a controlling variable of behavior.