On the value of advanced information about delayed rewards.

In a variety of laboratory preparations, several animal species prefer signaled over unsignaled outcomes. Here we examine whether pigeons prefer options that signal the delay to reward over options that do not and how this preference changes with the ratio of the delays. We offered pigeons repeated choices between two alternatives leading to a short or a long delay to reward. For one alternative (informative), the short and long delays were reliably signaled by different stimuli (e.g., SS for short delays, SL for long delays). For the other (non-informative), the delays were not reliably signaled by the stimuli presented (S1 and S2). Across conditions, we varied the durations of the short and long delays, hence their ratio, while keeping the average delay to reward constant. Pigeons preferred the informative over the non-informative option and this preference became stronger as the ratio of the long to the short delay increased. A modified version of the Δ-Σ hypothesis (González et al., J Exp Anal Behav 113(3):591-608. https://doi.org/10.1002/jeab.595 , 2020a) incorporating a contrast-like process between the immediacies to reward signaled by each stimulus accounted well for our findings. Functionally, we argue that a preference for signaled delays hinges on the potential instrumental advantage typically conveyed by information.

Time, uncertainty, and suboptimal choice.

Under certain conditions, pigeons prefer information about whether food will be forthcoming at the end of an interval to a higher chance of obtaining the food. In the typical protocol, choosing one option (Informative) is followed by one of two 10-s long terminal-link stimuli: SG always ending in food or SR never ending in food, with SG occurring only 20% of the trials. The other option (Non-informative) is also followed by one of two 10-s long terminal-link stimuli: SB or SY, both ending in food 50% of the trials. Although the Informative option yields food with a lower probability than the Non-informative (0.2 vs. 0.5), pigeons prefer it. To determine whether such preference occurs because SG and SR disambiguate the trial outcome immediately upon choice, we delayed the moment the disambiguation took place in two experiments. In Experiment 1, when the Informative option was chosen, SG always ensued for t seconds of the terminal-link, and then the standard contingencies followed. Experiment 2 was similar, except that SR always ensued for t seconds. Across conditions, t varied from 0 to 10 s. In both experiments, preference for the Informative option decreased with t, but the effect was stronger in Experiment 1. We discuss the implication of these findings for functional and mechanistic models of suboptimal choice.

Giving time a chance in the midsession reversal task.

The midsession reversal task involves a simultaneous discrimination between stimuli S1 and S2. Choice of S1 but not S2 is reinforced during the first 40 trials, and choice of S2 but not S1 is reinforced during the last 40 trials. Trials are separated by a constant intertrial interval (ITI). Pigeons learn the task seemingly by timing the moment of the reversal trial. Hence, most of their errors occur around trial 40 (S2 choices before trial 41 and S1 choices after trial 40). It has been found that when the ITI is doubled on a test session, the reversal trial is halved, a result consistent with timing. However, inconsistent with timing, halving the ITI on a test session did not double the reversal trial. The asymmetry of ITI effects could be due to the intrusion of novel cues during testing, cues that preempt the timing cue. To test this hypothesis, we ran two types of tests after the regular training in the midsession reversal task, one with S1 and S2 choices always reinforced, and another with S1 always reinforced but S2 reinforced only after 20 trials when the ITI doubled or 40 trials when the ITI halved. For most pigeons, performance was consistent with timing both when the ITI doubled and when it was halved, but some pigeons appeared to follow strategies based on counting or on reinforcement contingencies.

Midsession reversal task with starlings: A quantitative test of the timing hypothesis.

In the Mid-Session Reversal task (MSR), an animal chooses between two options, S1 and S2. Rewards follow S1 but not S2 from trials 1-40, and S2 but not S1 from trials 41-80. With pigeons, the psychometric function relating S1 choice proportion to trial number starts close to 1 and ends close to 0, with indifference (PSE) close to trial 40. Surprisingly, pigeons make anticipatory errors, choosing S2 before trial 41, and perseverative errors, choosing S1 after trial 40. These errors suggest that they use time into the session as the preference reversal cue. We tested this timing hypothesis with 10 Spotless starlings. After learning the MSR task with a T-s Inter-Trial Interval (ITI), they were exposed to either 2 T or T/2 ITIs during testing. Doubling the ITI should shift the psychometric function to the left and halve its PSE, whereas halving the ITI should shift the function to the right and double its PSE. When the starlings received one pellet per reward, the ITI manipulation was effective: The psychometric functions shifted in the direction and by the amount predicted by the timing hypothesis. However, non-temporal cues also influenced choice.

Time to contrast models of timing: The structure of temporal memory.

In the study of animal timing over the last 100 years, we identify three different periods, each characterized by a distinct activity. In the first period, researchers brought timing into the laboratory and explored its multiple expressions empirically. In the second period, the growing body of empirical findings inspired researchers to develop a plethora of timing models that vary in theoretical orientation, scope, depth, and quantitative explicitness. We argue that it is now the time to advance towards a third period, wherein researchers select models by comparing them with one another and with data. We make our case by contrasting how the scalar expectancy theory and the learning-to-time model conceive of temporal memory and learning both in concurrent timing tasks and in retrospective timing tasks. We identify four problems related to the structure of temporal memory and to the rules of temporal learning that challenge these models and that should drive the next steps in modeling the timing abilities of animals. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Constantly timing, but not always controlled by time: Evidence from the midsession reversal task.

We used a midsession reversal task to investigate how temporal and situational cues may combine to determine choice in frequently changing environments. Pigeons learned a simultaneous discrimination with 2 stimuli: S1 and S2. Choices of S1 were reinforced only during the first trials, and choices of S2 were reinforced only during the last trials of the session, that is, the reinforcement contingencies reversed once during the session. To weaken the temporal cue (time into the session) that signaled the reversal trial, we varied the location of reversal trial randomly across sessions; to weaken the situational cue (the outcome of the previous trials that might support a win-stay/loose shift strategy), we varied the payoff probabilities associated with S1 and S2. Performance was consistent neither with the exclusive use of a timing strategy, nor with the exclusive use of a situational, win-stay/lose-shift strategy. Instead, choice seemed to be under joint control of both cues. The relative influence of these cues was dynamic: When payoff was higher for S1 than S2, behavior was less time-controlled than when the payoff was higher for S2 that S1, or when they were equal. We advance a descriptive mixture model of joint control for the midsession reversal task. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Base rates bias performance in a temporal bisection task.

We investigated how base rates affect temporal discrimination. In a temporal bisection task, pigeons learned to choose one key after a short sample and another key after a long sample. When presented with a range of intermediate samples they produced a psychometric function characterized by a bias and a scale parameter. When one of the trained samples was more frequent than the other, only the location parameter changed, with the pigeons biasing their choices toward the key associated with the most frequent sample. We then reproduced the bisection task in a long operant chamber, with choice keys far apart, and tracked the pigeons' motion patterns during the sample. Pigeons learned to approach the short key following sample onset, wait on the "short side" for a few seconds, and then, when the sample continued, depart toward the long key. This time-place curve was affected by sample base rate: The probability of pigeons going directly to the long side after sample onset increased when long samples were more frequent than short samples, indicating a decrease of temporal control. We found no evidence of changes in temporal sensitivity. The results are most consistent with models of timing that take into account biasing effects and competition for stimulus control. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Correction to: Rules of Conduct for Behavior Analysts in the Presence of Hypothetical Constructs: A Commentary on Eckard and Lattal (2020).

[This corrects the article DOI: 10.1007/s40614-020-00272-w.].

The functional equivalence of two variants of the suboptimal choice task: choice proportion and response latency as measures of value.

In the suboptimal-choice task, birds systematically choose the leaner but informative option (suboptimal) over the richer but non-informative option (optimal). The task has two variations. In the standard task, the optimal option includes two terminal link stimuli. In the original task, it includes a single terminal link stimulus. Two models, the temporal information account (Cunningham and Shahan, J Exp Psychol Anim Learn Cogn 44:1-22, 2018) and the ∆-∑ hypothesis (González et al., J Exp Anal Behav 113:591-608, 2020), presuppose that these procedures are equivalent, but no formal comparison is available. Here we test whether or not these procedures are functionally equivalent. One group of pigeons was trained with the standard procedure, another group with the original procedure, and a third group was trained with a hybrid of the other two (i.e., the two options were the optimal links of the standard and original procedures). Our findings indicate that the number of terminal link stimuli in the optimal option is inconsequential vis-à-vis choice. Moreover, our findings also indicate that latencies to respond are a sensitive metric of value and choice. As predicted by the Sequential Choice Model, we were able to predict simultaneous choices from the latencies of sequential choices and observed a substantial shortening of latencies during simultaneous choices.

Rules of Conduct for Behavior Analysts in the Presence of Hypothetical Constructs: A Commentary on Eckard and Lattal (2020).

Eckard and Lattal (2020) summarized the behavioristic view of hypothetical constructs and theories, and then, in a novel and timely manner, applied this view to a critique of internal clock models of temporal control. In our three-part commentary, we aim to contribute to the authors' discussion by first expanding upon their view of the positive contributions afforded by constructs and theories. We then refine and question their view of the perils of reifying constructs and assigning them causal properties. Finally, we suggest to behavior analysts four rules of conduct for dealing with mediational theories: tolerate constructs proposed with sufficient reason; consider them seriously, both empirically and conceptually; develop alternative, behavior-analytic models with overlapping empirical domains; and contrast the various models. Through variation and selection, behavioral science will evolve.

Step changes in the intertrial interval in the midsession reversal task: Predicting pigeons' performance with the learning-to-time model.

Our goal was to assess the role of timing in pigeons' performance in the midsession reversal task. In discrete-trial sessions, pigeons learned to discriminate between 2 stimuli, S1 and S2. Choices of S1 were reinforced only in the first half of the session and choices of S2 were reinforced only in the second half. Typically, pigeons choose S2 before the contingency reverses (anticipatory errors) and S1 after (perseverative errors), suggesting that they time the interval from the beginning of the session to the contingency reversal. To test this hypothesis, we exposed pigeons to a midsession reversal task and, depending on the group, either increased or decreased the ITI duration. We then contrasted the pigeons' performance with the predictions of the Learning-to-Time (LeT) model: In both conditions, preference was expected to reverse at the same time as in the previous sessions. When the ITI was doubled, pigeons' preference reversal occurred at half the trial number but at the same time as in the previous sessions. When the ITI was halved, pigeons' preference reversal occurred at a later trial but at an earlier time than in the previous sessions. Hence, pigeons' performance was only partially consistent with the predictions of LeT, suggesting that besides timing, other sources of control, such as the outcome of previous trials, seem to influence choice.

Effects of differential probabilities of reinforcement on human timing.

We investigated how differential payoffs affect the temporal discrimination of humans. In a temporal bisection task, participants learned to make one response after a short sample and another after a long sample. When presented with a range of intermediate samples, the proportion of responses fitted well a Gaussian-like distribution function characterized by a location (bias), a scale (sensitivity) parameter, and two asymptote (discrimination) parameters. In Experiment 1, when one response yielded more reinforcers than the other, parameters were unaltered, but overall responses increased for the response producing higher payoffs. In Experiment 2, we used a video game to track motion during the sample and participants learned to approach the "short" response location at sample onset and remain there before departing to the "long" location on long trials. Departure times were shorter when "long" choices produced higher payoffs than "short" and matched well the shifted psychometric functions. However, on some trials, subjects were biased for short, returning to the short side after having departed towards long. Evidence was found for effects of differential payoffs on response bias, but discrimination and sensitivity did not change consistently. These results favor a behavioral account of timing processes.

The Δ-∑ hypothesis: How contrast and reinforcement rate combine to generate suboptimal choice.

When given a choice between two alternatives, each offering food after the same delay with different but signaled probabilities, pigeons often prefer the low probability alternative. This preference is surprising because pigeons fail to maximize the rate of food intake; they exhibit a suboptimal preference. We advance a new explanation, the Δ-∑ hypothesis, in which the difference in probability of reinforcement within terminal links (Δ) and the overall reinforcement probability rate of each alternative (∑) are the key variables responsible for such suboptimal preference. We tested the Δ-∑ hypothesis in two experiments. In Experiment 1, we manipulated the Δs while maintaining constant all other parameters of the task, in particular the ∑s. We predicted a preference for the alternative with the larger Δ. In Experiment 2, we examined the effect of the overall reinforcement probabilities, the ∑s, while maintaining constant all other parameters of the task, in particular the Δs. We predicted a preference for the larger ∑. The results of both experiments support the Δ-∑ hypothesis.

Testing the Δ-∑ hypothesis in the suboptimal choice task: Same delta with different probabilities of reinforcement.

In a concurrent-chain procedure, pigeons choose between 2 initial-link stimuli; one is followed by terminal link stimuli that signal reliably whether food will be delivered after a delay; the other is followed by terminal link stimuli that do not signal whether food will be delivered after the delay. Pigeons prefer the former alternative even when it yields a lower overall probability of food. Recently, we proposed the Delta-Sigma (∆-∑) hypothesis to explain the effect: Preference depends on the difference (∆) between the reinforcement probabilities associated with the terminal link stimuli, and the overall probability of reinforcement (∑) associated with the alternative. The hypothesis predicts that, for constant ∑, animals should prefer alternatives with greater ∆ values regardless of the specific probabilities of reinforcement that determine ∆. In 2 experiments, we tested this prediction by comparing a ∆ = .5 against a ∆ = 0 alternative, with the former obtained with different pairs of reinforcement probabilities across conditions. The results supported the hypothesis when the 2 probabilities defining ∆ were significantly greater than 0, but not when one of them was close to 0. The results challenge our theoretical accounts of suboptimal choice and the variables considered to determine pigeons' preference.

Simple discrimination in stingless bees (Melipona quadrifasciata): Probing for select- and reject-stimulus control.

Simple and conditional discrimination training may produce various types of controlling relations. Responses may be controlled primarily by the positive stimulus (select-control relation) or by the negative stimulus (reject-control relation; the subject excludes the negative stimulus and chooses the positive). Bees learn to respond in simple and conditional discriminations. However, no study has searched for reject-control responding in Melipona bees. We trained Melipona quadrifasciata on a simple discrimination task (S+ vs. S-; e.g., blue vs. yellow) and then probed for stimulus control with two types of probe trials, S+ versus a new stimulus (Select-control probes) and S- versus a new stimulus (Reject-control probes). For Group Different, a new-stimulus color (e.g., white) was used in one type of probe and another color (e.g., black) was used in the other type. For Group Same, a single new-stimulus color was used in both types of probes. On Select probes, the bees always preferred S+ to the new stimulus. On Reject probes, results were mixed. Depending on the colors used in training and probing, bees responded to both stimuli, and even preferred the S-. The data suggest no control by the negative function of the S- and support the select-stimulus control hypothesis of responding.

Corrigendum: Effects of Nodal Distance on Conditioned Stimulus Valences Across Time.

[This corrects the article DOI: 10.3389/fpsyg.2019.00742.].

Effects of Nodal Distance on Conditioned Stimulus Valences Across Time.

A meaningless symbol that repeatedly co-occurs with emotionally salient faces (US) can transform into a valenced symbol (CS). US-to-CS valence transformations have been observed for CS that have been directly (US→CS0) and indirectly (US→CS0→CS1→CS2) linked with face US. The structure of a US→CS0→CS1→CS2 series may be conceptualized in terms of "nodal distance," where CS0, CS1, and CS2 are 0, 1, and 2 nodes from the US respectively. Increasing nodal distance between an evaluated CS and its linked US can reduce magnitude of observed CS valence transformations. We explored currently whether nodal distance can influence CS valence extinction, which describes reductions in CS valence following repeated exposures to CS without any accompanying US. In our study, faces with happy/neutral/sad expressions (US) were directly linked with nonsense words (US→CS0). The directly linked CS0 was concurrently linked with other words (CS0→CS1, CS1→CS2). Subjects evaluated all stimuli before and after conditioning, then continued to provide CS evaluations twice a week for 6 weeks. Bayesian factors provided credible evidence for the transformation and extinction of CS valences that were 0 and 1 nodes from US (all BF10's > 100). The variability across post-conditioning CS evaluations provides indirect evidence for context-sensitive/propositional and structural/associative operations during CS evaluations.

The effect of reinforcement probability on time discrimination in the midsession reversal task.

We examined how biasing time perception affects choice in a midsession reversal task. Given a simultaneous discrimination between stimuli S1 and S2, with choices of S1 reinforced during the first, but not the second half of the trials, and choices of S2 reinforced during the second, but not the first half of the trials, pigeons show anticipation errors (premature choices of S2) and perseveration errors (belated choices of S1). This suggests that choice depends on timing processes, on predicting when the contingency reverses based on session duration. We exposed 7 pigeons to a midsession reversal task and manipulated the reinforcement rate on each half of the session. Compared to equal reinforcement rates on both halves of the session, when the reinforcement rate on the first half was lower than on the second half, performance showed more anticipation and less perseveration errors, and when the reinforcement rate on the first half was higher than on the second half, performance showed a remarkable reduction of both types of errors. These results suggest that choice depends on both time into the session and the outcome of previous trials. They also challenge current models of timing to integrate local effects.

Biasing performance through differential payoff in a temporal bisection task.

We investigated how differential payoffs affect temporal discrimination. In a temporal bisection task, pigeons learned to choose one key after a short sample and another key after a long sample. When presented with a range of intermediate samples they produced a Gaussian psychometric function characterized by a location (bias) parameter and a scale (sensitivity) parameter. When one key yielded more reinforcers than the other, the location parameter changed, with the pigeons biasing their choices toward the richer key. We then reproduced the bisection task in a long operant chamber, with choice keys far apart, and tracked the pigeons' motion patterns during the sample. These patterns were highly stereotypical-on the long sample trials, the pigeons approached the short key at sample onset, stayed there for a while, and then departed to the long key. The distribution of departure times also was biased when the payoff probabilities differed. Moreover, it is likely that temporal control decreased while control by location increased. No evidence was found of changes in temporal sensitivity. The results are consistent with models of timing that take into account bias effects and competition of stimulus control. (PsycINFO Database Record (c) 2019 APA, all rights reserved).