Dorsolateral Striatal Task-initiation Bursts Represent Past Experiences More than Future Action Plans.

The dorsolateral striatum (DLS) is involved in learning and executing procedural actions. Cell ensembles in the DLS, but not the dorsomedial striatum (DMS), exhibit a burst of firing at the start of a well-learned action sequence ("task-bracketing"). However, it is currently unclear what information is contained in these bursts. Some theories suggest that these bursts should represent the procedural action sequence itself (that they should be about future action chains), whereas others suggest that they should contain representations of the current state of the world, taking into account primarily past information. In addition, the DLS local field potential shows transient bursts of power in the 50 Hz range (γ50) around the time a learned action sequence is initiated. However, it is currently unknown how bursts of activity in DLS cell ensembles and bursts of γ50 power in the DLS local field potential are related to each other. We found that DLS bursts at lap initiation in rats represented recently experienced reward locations more than future procedural actions, indicating that task-initiation DLS bursts contain primarily retrospective, rather than prospective, information to guide procedural actions. Furthermore, representations of past reward locations increased during periods of increased γ50 power in the DLS. There was no evidence of task-initiation bursts, increased γ50 power, or retrospective reward location information in the neighboring dorsomedial striatum. These data support a role for the DLS in model-free theories of procedural decision-making over planned action-chain theories, suggesting that procedural actions derive from representations of the current and recent past.SIGNIFICANCE STATEMENT While it is well-established that the dorsolateral striatum (DLS) plays a critical role in procedural decision-making, open questions remain about the kinds of representations contained in DLS ensemble activity that guide procedural actions. We found that DLS, but not DMS, cell ensembles contained nonlocal representations of past reward locations that appear moments before task-initiation DLS bursts. These retrospective representations were temporally linked to a rise in γ50 power that also preceded the characteristic DLS burst at task-initiation. These results support models of procedural decision-making based on associations between available actions and the current state of the world over models based on planning over action-chains.

Sunk cost sensitivity during change-of-mind decisions is informed by both the spent and remaining costs.

Sunk cost sensitivity describes escalating decision commitment with increased spent resources. On neuroeconomic foraging tasks, mice, rats, and humans show similar escalations from sunk costs while quitting an ongoing countdown to reward. In a new analysis taken across computationally parallel foraging tasks across species and laboratories, we find that these behaviors primarily occur on choices that are economically inconsistent with the subject's other choices, and that they reflect not only the time spent, but also the time remaining, suggesting that these are change-of-mind re-evaluation processes. Using a recently proposed change-of-mind drift-diffusion model, we find that the sunk cost sensitivity in this model arises from decision-processes that directly take into account the time spent (costs sunk). Applying these new insights to experimental data, we find that sensitivity to sunk costs during re-evaluation decisions depends on the information provided to the subject about the time spent and the time remaining.

Delays to food-predictive stimuli do not affect suboptimal choice in rats.

A variety of animals sometimes engage in a form of maladaptive decision-making characterized by repeatedly choosing an option providing food-predictive stimuli even though they earn less food for doing so. The temporal information-theoretic model suggests that such suboptimal choice depends on competition between the bits of temporal information conveyed by food-predictive stimuli (which encourages suboptimal choice) and the rate of food delivery (which encourages optimal choice). The model assumes that competition between these two sources of control is based on the ratio of the delay to food (Df) and the delay to food-predictive stimuli (Ds) at the choice point (i.e., Df/Ds). Research with both rats and pigeons suggests that temporal information outcompetes the rate of food delivery, thereby generating suboptimal choice, when the delay to food (Df) is sufficiently long. Limited data with pigeons, and none with rats, suggests that the rate of food delivery outcompetes temporal information, thereby generating optimal choice, when the delay to food-predictive stimuli (Ds) is sufficiently long. The present experiment sought to clarify whether longer delays to food-predictive stimuli decrease suboptimal choice in rats. We found that while longer delays to food (Df) increased suboptimal choice in rats, longer delays to food-predictive stimuli (Ds) did not decrease suboptimal choice. These results suggest a potential difference between rats and pigeons in the manner in which food-predictive stimuli and food itself compete to control choice. In terms of the temporal information-theoretic model, competition between temporal information and the rate of food delivery in rats appears to be influenced only by the delay to food at the choice point. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Rats engage in suboptimal choice when the delay to food is sufficiently long.

Numerous examples in the decision-making literature demonstrate that animals sometimes make choices that are not in their long-term best interest. One particular example finds pigeons preferring a low-probability alternative in lieu of a high-probability alternative, referred to as suboptimal choice. Although there is ample evidence that pigeons engage in such suboptimal choice, there is currently weak evidence (at best) that rats also do so. Cunningham and Shahan's (2018) temporal information-theoretic model suggests that suboptimal choice in pigeons arises when (1) the low-probability alternative provides stimuli that convey more temporal information than stimuli associated with the high-probability alternative and (2) when the delay to food is much longer relative to the delay to temporally informative signals at the choice point. The latter condition plays the important role of biasing decision making to be governed by the relative temporal information conveyed by stimuli rather than the relative rate of food delivery. The present experiment explored the possibility that rats will engage in suboptimal choice if the delay to food at the choice point is sufficiently long, as the temporal information-theoretic model suggests. Rats were given a choice between a suboptimal alternative providing food 20% of the time and an optimal alternative providing food 50% of the time. The suboptimal alternative provided stimuli that differentially signaled choice outcomes whereas the optimal alternative did not. The postchoice delay was manipulated across conditions and ranged from 10 s to 50 s. As with previous research, rats did not engage in suboptimal choice when the postchoice delay was 10 s. However, once the delay was at least 30 s, rats engaged in suboptimal choice. These results are consistent with the temporal information-theoretic model of suboptimal choice and suggest that rats and pigeons likely do not differ in the decision-making processes involved in the suboptimal choice procedure. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Suboptimal choice, reward-predictive signals, and temporal information.

Suboptimal choice refers to preference for an alternative offering a low probability of food (suboptimal alternative) over an alternative offering a higher probability of food (optimal alternative). Numerous studies have found that stimuli signaling probabilistic food play a critical role in the development and maintenance of suboptimal choice. However, there is still much debate about how to characterize how these stimuli influence suboptimal choice. There is substantial evidence that the temporal information conveyed by a food-predictive signal governs its function as both a Pavlovian conditioned stimulus and as an instrumental conditioned reinforcer. Thus, we explore the possibility that food-predictive signals influence suboptimal choice via the temporal information they convey. Application of this temporal information-theoretic approach to suboptimal choice provides a formal, quantitative framework that describes how food-predictive signals influence suboptimal choice in a manner consistent with related phenomena in Pavlovian conditioning and conditioned reinforcement. Our reanalysis of previous data on suboptimal choice suggests that, generally speaking, preference in the suboptimal choice procedure tracks relative temporal information conveyed by food-predictive signals for the suboptimal and optimal alternatives. The model suggests that suboptimal choice develops when the food-predictive signal for the suboptimal alternative conveys more temporal information than that for the optimal alternative. Finally, incorporating a role for competition between temporal information provided by food-predictive signals and relative primary reinforcement rate provides a reasonable account of existing data on suboptimal choice. (PsycINFO Database Record

Delivering alternative reinforcement in a distinct context reduces its counter-therapeutic effects on relapse.

Delivery of alternative reinforcers in the presence of stimuli previously associated with reinforcement for target behavior increases the susceptibility of target behavior to relapse. To explore contingencies that might mitigate this counter-therapeutic effect, we trained pigeons on a procedure that entailed extinction of previously reinforced target-key pecking, access to a distinct stimulus context contingently on refraining from target behavior (differential-reinforcement-of-other-behavior; DRO), and reinforcement of alternative-key pecks (differential-reinforcement of alternative behavior; DRA) in that context. This DRO-DRA treatment was compared with standard DRA in successive conditions, counterbalanced across pigeons. Target behavior extinguished more rapidly in the Standard-DRA condition. When alternative reinforcement was discontinued, however, there was less resurgence after DRO-DRA than after Standard DRA. In a third condition, the DRO contingency was suspended so that the former DRA stimuli were not presented (DRO-NAC), and resurgence was greater than in the Standard-DRA and DRO-DRA conditions. Reinstatement produced by response-independent reinforcers was small and similar across conditions. Subsequent reacquisition of target-key pecking under baseline reinforcement conditions was faster following DRO-NAC than Standard-DRA or DRO-DRA. These findings suggest that DRO-DRA might serve as a useful method in clinical settings for reducing problem behavior while minimizing the threat of posttreatment relapse.

Quantitative models of persistence and relapse from the perspective of behavioral momentum theory: Fits and misfits.

We review quantitative accounts of behavioral momentum theory (BMT), its application to clinical treatment, and its extension to post-intervention relapse of target behavior. We suggest that its extension can account for relapse using reinstatement and renewal models, but that its application to resurgence is flawed both conceptually and in its failure to account for recent data. We propose that the enhanced persistence of target behavior engendered by alternative reinforcers is limited to their concurrent availability within a distinctive stimulus context. However, a failure to find effects of stimulus-correlated reinforcer rates in a Pavlovian-to-Instrumental Transfer (PIT) paradigm challenges even a straightforward Pavlovian account of alternative reinforcer effects. BMT has been valuable in understanding basic research findings and in guiding clinical applications and accounting for their data, but alternatives are needed that can account more effectively for resurgence while encompassing basic data on resistance to change as well as other forms of relapse.

A within-subject between-apparatus comparison of impulsive choice: T-maze and two-lever chamber.

Whereas intertemporal choice procedures are a common method for examining impulsive choice in nonhuman subjects, the apparatus used to implement this procedure varies across studies. The purpose of the present study was to compare impulsive choice between a two-lever chamber and a T-maze. In Experiment 1, rats chose between a smaller, immediate reinforcer and a larger, delayed reinforcer, first in a two-lever chamber and then in a T-maze. Delay to the larger reinforcer changed in an ascending and descending order (0-32 s) across sessions. Experiment 2 examined the same between-apparatus comparison but under steady-state conditions with the delay fixed at 32 s. In Experiment 1, choice for the larger, delayed reinforcer was generally higher in the T-maze than in the two-lever chamber. Similarly in Experiment 2, steady-state choice for the larger, delayed reinforcer was higher in the T-maze. Choice for the 32-s delayed reinforcer was also greater in Experiment 2 than in Experiment 1, suggesting that extended exposure to the delay is required for the T-maze to yield reliable impulsive choice data. While the reasons for the between-apparatus discrepancies are at present unknown, results from both experiments clearly demonstrate that the apparatus matters when assessing overall level and reliability of impulsive choice data.

Behavioral momentum and accumulation of mass in multiple schedules.

Behavioral momentum theory suggests that the relation between a discriminative-stimulus situation and reinforcers obtained in that context (i.e., the Pavlovian stimulus-reinforcer relation) governs persistence of operant behavior. Within the theory, a mass-like aspect of behavior has been shown to be a power function of predisruption reinforcement rates. Previous investigations of resistance to change in multiple schedules, however, have been restricted to examining response persistence following protracted periods of stability in reinforcer rates within a discriminative situation. Thus, it is unclear how long a stimulus-reinforcer relation must be in effect prior to disruption in order to affect resistance to change. The present experiment examined resistance to change of pigeon's key pecking following baseline conditions where reinforcer rates that were correlated with discriminative-stimulus situations changed. Across conditions, one multiple-schedule component arranged either relatively higher rates or lower rates of variable-interval food delivery, while the other component arranged the opposite rate. These schedules alternated between multiple-schedule components across blocks of sessions such that reinforcer rates in the components were held constant for 20, 5, 3, 2, or 1 session(s) between alternations. Resistance to extinction was higher in the component that most recently was associated with higher rates of food delivery in all conditions except when schedules alternated daily or every other day. These data suggest that resistance to change in multiple schedules is related to recently experienced reinforcer rates but only when multiple-schedule components are associated with specific reinforcer rates for several sessions.