Midsession shifts in reward probability and the control of behavioral variability.

Extensive research has shown that the variability of organismal behavior is great when contingent reinforcement is delayed, small, or improbable. This research has generally employed stable response-reinforcer relationships, and therefore is limited in its explanatory scope with respect to a dynamic environment. We conducted two experiments to investigate whether pigeons' conditioned pecking behavior shows anticipatory or perseverative patterns of behavioral variability when the reinforcement probability reliably changes within experimental sessions. In Experiment 1, three pigeons received alternating sessions in which the reinforcement probability (35% or 4.2%) was shifted at the midpoint of each session in the presence of the same discrete cue. Experiment 2 featured a similar design, but with the inclusion of a discrete visual cue that changed at the session midpoint, and thus unambiguously indicated reinforcement probability. Local reinforcement rates only reliably controlled response variability when a discrete visual cue was available. Without this, pigeons did not discriminate between trial types in the first halves of sessions, and showed evidence of perseveration of response variability following a within-session shift. Critically, this is the first experimental demonstration that the relationship between reinforcement probability and behavioral variability is moderated by another factor (i.e., trial position within a session). This study thus expands our understanding of the control of behavioral variability as a function of experiential factors.

Reward expectation modulates variability in path choice in rats.

The relationship between behavioral variability and reward expectation has been examined in recent years. This relationship is predictive: when an animal has a low expectation of reinforcement for a particular behavioral set, they engage in high levels of variability in their actions. We conducted two experiments to further investigate this relationship using a novel measure of behavioral variability. In Experiment 1, two groups of rats were trained to travel through a column maze, with many possible reinforced pathways, to receive either their maintenance diet (i.e., chow) or a highly palatable sugary reward (cereal). We hypothesized that animals trained with a maintenance-diet food source (chow) would demonstrate more variation in the pathways taken to the goal location than those animals trained with the highly palatable alternative. In Experiment 2, all rats were trained to travel through the maze to receive alternating outcomes of chow or cereal in a within-subjects design. Results from both experiments indicated that rats emitted greater variability in paths taken to the goal when the reinforcer was the maintenance chow. These results corroborate the relationship between reward and behavioral variability in a new behavioral measure.

The behavioral origins of phylogenic responses and ontogenic habits.

An examination of innate behavior and its possible origins suggests parallels with the formation of habitual behavior. Inflexible but adaptive responses-innate reflexive behavior, Pavlovian conditioned responses, and operant habits-may have evolved from variable behavior in phylogeny and ontogeny. This form of "plasticity-first" scientific narrative was unpopular post-Darwin but has recently gained credibility in evolutionary biology. The present article seeks to identify originating events and contingencies contributing to such inflexible but adaptive behavior at both phylogenic and ontogenic levels of selection. In ontogeny, the development of inflexible performance (i.e., habit) from variable operant behavior is reminiscent of the genetic accommodation of initially variable phylogenic traits. The effects characteristic of habit (e.g., unresponsiveness to reinforcer devaluation) are explicable as the result of a conflict between behaviors at distinct levels of selection. The present interpretation validates the practice of seeking hard analogies between evolutionary biology and operant behavior. Finding such parallels implies the validity of a claim that organismal behavior, both innate and learned, is a product of selection by consequences. A complete and coherent account of organismal behavior may ultimately focus on functional selective histories in much the same way evolutionary biology does with its subject matter.

Faustian bargains: Short-term and long-term contingencies in phylogeny, ontogeny, and sociogeny.

Rachlin's interpretations of self-control depend on the short-term versus the long-term consequences of behavior. Sometimes these effects support each other (typing an abstract produces a written product now and is later read by others). Sometimes they conflict (procrastination now is incompatible with finishing the abstract by deadline). We usually reserve the language of self-control for human cases where long-term consequences are chosen over short-term ones. Rachlin made this distinction salient in ontogeny, but it also applies to selection in phylogeny (Darwinian evolution) and sociogeny (behavior passed from one organism to another). Our account examines relations between short-term and long-term consequences at each level of selection. For example, sexual selection has adaptive, short-term mating consequences but may drive species to extreme specializations that jeopardize long-term survival. In sociogeny, as in the Tragedy of the Commons, group members may get immediate economic benefits from exploiting resources but exhaust those resources over the long term. Whatever the level, when short-term and long-term consequences have opposing effects, adaptive behavior may depend on whether temporally extended contingencies exert more control than more immediate benefits.

Devaluation of a conditioned reinforcer requires its reexposure.

A change in motivational state does not guarantee a change in operant behaviour. Only after an organism has had contact with an outcome while in a relevant motivational state does behaviour change, a phenomenon called incentive learning. While ample evidence indicates that this is true for primary reinforcers, it has not been established for conditioned reinforcers. We performed an experiment with rats where lever-presses were reinforced by presentations of an audiovisual stimulus that had previously preceded food delivery; in the critical experimental groups, the audiovisual stimulus was then paired a single time with a strong electric shock. Some animals were reexposed to the audiovisual stimulus. Lever-presses yielding no outcomes were recorded in a subsequent test. Animals that had been reexposed to the audiovisual stimulus after the aversive training responded less than did those that had not received reexposure. Indeed, those animals that were not reexposed did not differ from a control group that received no aversive conditioning of the audiovisual stimulus. Moreover, these results were not mediated by a change in the food's reinforcement value, but instead reflect a change in behaviour with respect to the conditioned reinforcer itself. These are the first data to indicate that the affective value of conditioned stimuli, like that of unconditioned ones, is established when the organism comes into contact with them.

A non-selective serotonin antagonist promotes rapid habituation in the terrestrial hermit crab.

Research has indicated that serotonin (5-HT) modulates non-associative learning in a variety of invertebrate species. Recent work has demonstrated that the terrestrial hermit crab is a suitable animal model for non-associative learning phenomena, including habituation, sensitization, and dishabituation. We examined the potential role of a non-selective 5-HT antagonist, methysergide, in non-associative learning in the hermit crab. We administered methysergide prior to delivering repeated stimulus presentations of a looming visual predator. We found evidence for more rapid habituation relative to a control condition in which crabs did not receive the drug. These results indicate a role for 5-HT in the defensive behavior of the hermit crab and importantly, suggest a conserved role for 5-HT in modulating basic learning processes in invertebrates.

The coelacanth still lives: Bringing selection back to the fore in a science of behavior.

There is little scientific debate regarding the validity of Darwin's theory of evolution by natural selection, which effectively describes how relevant ancestral histories produce both an organism's genetic characteristics and innate behavioral repertoires. The combination of variation and selection in the production of novel forms can be extended beyond Darwinian theory to encompass facts of ontogeny. The present article sheds light on an underappreciated and critical insight, namely, that the consequences of behavior have a selective effect analogous to that observed in biological evolution. Three levels of environmental selection (phylogenic, ontogenic, and cultural) constitute a full account of the causes for action. This perspective identifies the relevant functional contingencies of which behavior is a product, it accurately and parsimoniously predicts a wide variety of disparate behavioral findings, it resolves old debates on nativism and empiricism, it unites psychological science under a central organizing principle, and it specifies psychology's position in relation to biology. Wholesale adoption of this perspective should be considered a positive advance for the field of psychology. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Overshadowing between visual and tactile stimulus elements in an object recognition task.

In recent decades there has been great progress in discovering the conditions under which cue competition occurs during animal learning. In humans, however, the evidence remains equivocal regarding the degree to which stimuli compete with one another for behavioral control. We report here the results of a single experiment wherein thirty-nine college students completed a novel cue competition task with visual and tactile stimuli. Participants visually and/or haptically examined a series of novel objects. They were then asked to select the objects with which they had interacted from a larger pool of both novel and familiar objects. Potentiation (or facilitation) by simultaneous visual and haptic inspection was possible. Alternatively, stimulus elements may have competed with one another (i.e., overshadowing), which would present as poorer recognition at test for objects to which participants had simultaneous, dual-modality training exposure. We report the latter effect. We situate these findings in the broader context of associative learning and suggest that our data is relevant to applied settings.

Stimulus concordance and risk-assessment in hermit crabs (Coenobita clypeatus): implications for attention.

Recent research has demonstrated that the topography of defensive reactions depends on factors that are extraneous to the stimulus that elicits the defensive response. For example, hermit crabs will withdraw more slowly to the approach of a simulated visual predator (i.e., the eliciting stimulus) when in the presence of a coincident acoustic stimulus. Multiple properties related to the magnitude (e.g., duration, amplitude) of the acoustic stimulus have been found to modulate the crabs' withdrawal response (Chan et al., 2010b). We demonstrate that the proximity in spatial location between a threatening visual stimulus and a potentially distracting extraneous auditory stimulus is an important determinant of anti-predator behavior in hermit crabs. We suggest that a distal relationship between the eliciting stimulus and an unrelated signal may produce greater distraction. This marks the first reported experimental evidence of this relationship in an invertebrate species.

Quantifying personality in the terrestrial hermit crab: different measures, different inferences.

There is much interest in studying animal personalities but considerable debate as to how to define and evaluate them. We assessed the utility of one proposed framework while studying personality in terrestrial hermit crabs (Coenobita clypeatus). We recorded the latency of individuals to emerge from their shells over multiple trials in four unique manipulations. We used the specific testing situations within these manipulations to define two temperament categories (shyness-boldness and exploration-avoidance). Our results identified individual behavioral consistency (i.e., personality) across repeated trials of the same situations, within both categories. Additionally, we found correlations between behaviors across contexts (traits) that suggested that the crabs had behavioral syndromes. While we found some correlations between behaviors that are supposed to measure the same temperament trait, these correlations were not inevitable. Furthermore, a principal component analysis (PCA) of our data revealed new relationships between behaviors and provided the foundation for an alternate interpretation: measured behaviors may be situation-specific, and may not reflect general personality traits at all. These results suggest that more attention must be placed on how we infer personalities from standardized methods, and that we must be careful to not force our data to fit our frameworks.

The Modulation of Operant Variation by the Probability, Magnitude, and Delay of Reinforcement.

Recent studies have demonstrated that the expectation of reward delivery has an inverse relationship with operant behavioral variation (e.g., Stahlman, Roberts, & Blaisdell, 2010). Research thus far has largely focused on one aspect of reinforcement - the likelihood of food delivery. In two experiments with pigeons, we examined the effect of two other aspects of reinforcement: the magnitude of the reward and the temporal delay between the operant response and outcome delivery. In the first experiment, we found that a large reward magnitude resulted in reduced spatiotemporal variation in pigeons' pecking behavior. In the second experiment, we found that a 4-s delay between response-dependent trial termination and reward delivery increased variation in behavior. These results indicate that multiple dimensions of the reinforcer modulate operant response variation.

Auditory stimulation dishabituates anti-predator escape behavior in hermit crabs (Coenobita clypeatus).

Responses to innocuous stimuli often habituate with repeated stimulation, but the mechanisms involved in dishabituation are less well studied. Chan et al. (2010b) found that hermit crabs were quicker to perform an anti-predator withdrawal response in the presence of a short-duration white noise relative to a longer noise stimulus. In two experiments, we examined whether this effect could be explicable in terms of a non-associative learning process. We delivered repeated presentations of a simulated visual predator to hermit crabs, which initially caused the crabs to withdraw into their shells. After a number of trials, the visual stimulus lost the ability to elicit the withdrawal response. We then presented the crabs with an auditory stimulus prior to an additional presentation of the visual predator. In Experiment 1, the presentation of a 10-s, 89-dB SPL noise produced no significant dishabituation of the response. In Experiment 2 we increased the duration (50s) and intensity (95dB) of the noise, and found that the crabs recovered their withdrawal response to the visual predator. This finding illustrates dishabituation of an antipredator response and suggests two distinct processes-distraction and sensitization-are influenced by the same stimulus parameters, and interact to modulate the strength of the anti-predator response.

Response variability in pigeons in a Pavlovian task.

Instrumental response variation is inversely related to reward probability. Gharib, Derby, and Roberts (2001) theorized that individuals behave more variably when their expectation of reward is low. They postulate that this behavioral rule assists the discovery of alternative actions when a target response is unlikely to be reinforced. This suggests that response variability may be unaffected in a situation in which an animal's behavior is inconsequential to outcome delivery. We trained 6 pigeons in a within-subjects Pavlovian autoshaping procedure. On any given trial, the pigeons were presented with one of six colored discs on a touchscreen; each stimulus was associated with a particular probability of food, ranging from 100% to 0.6%. Pecking was more variable with low probabilities of food delivery, thus extending the rule relating variability and expectation to a Pavlovian situation.

Effect of reward probability on spatial and temporal variation.

Gharib, Derby, and Roberts (2001) proposed that reducing reward expectation increases variation of response form. We tested this rule in a new situation and asked if it also applied to variation of response location and timing. In 2 discrete-trial experiments, pigeons pecked colored circles for food. The circles were of 6 possible colors, each associated with a different probability of reward. Reducing reward expectation did not affect peck duration (a measure of form) but did increase horizontal variation of peck location and interpeck-interval variation. The effect of reward probability on the standard deviation of interpeck intervals was clearer (larger t value) than its effect on mean interpeck interval. Two datasets from rats had similar interresponse-interval effects.

Blocking of spatial control by landmarks in rats.

We investigated spatial blocking among landmarks in an open-field foraging task in rats. In Phase 1, rats were presented with A+ trials during which landmark (LM) A signaled the location of hidden food. In Phase 2, rats were given AX+ trials in which LM X served as a redundant spatial cue to the location of food. Additionally, BY+ trials were given as a within-subjects overshadowing-control procedure. At test, rats received nonreinforced presentations of LM X and LM Y on separate trials. Rats took longer to find the training goal location in the presence of LM X than of LM Y, thereby demonstrating that spatial control by LM X was blocked by prior learning with LM A. This constitutes the first evidence in rats for spatial blocking of one proximal landmark by another-approximating a conventional blocking design.