Maintained goal-directed control with overtraining on ratio schedules.

It is thought that goal-directed control of actions weakens or becomes masked by habits over time. We tested the opposing hypothesis that goal-directed control becomes stronger over time, and that this growth is modulated by the overall action-outcome contiguity. Despite group differences in action-outcome contiguity early in training, rats trained under random and fixed ratio schedules showed equivalent goal-directed control of lever pressing that appeared to grow over time. We confirmed that goal-directed control was maintained after extended training under another type of ratio schedule-continuous reinforcement-using specific satiety and taste aversion devaluation methods. These results add to the growing literature showing that extensive training does not reliably weaken goal-directed control and that it may strengthen it, or at least maintain it.

End-to-end collaboration to transform biopharmaceutical development and manufacturing.

An ambitious 10-year collaborative program is described to invent, design, demonstrate, and support commercialization of integrated biopharmaceutical manufacturing technology intended to transform the industry. Our goal is to enable improved control, robustness, and security of supply, dramatically reduced capital and operating cost, flexibility to supply an extremely diverse and changing portfolio of products in the face of uncertainty and changing demand, and faster product development and supply chain velocity, with sustainable raw materials, components, and energy use. The program is organized into workstreams focused on end-to-end control strategy, equipment flexibility, next generation technology, sustainability, and a physical test bed to evaluate and demonstrate the technologies that are developed. The elements of the program are synergistic. For example, process intensification results in cost reduction as well as increased sustainability. Improved robustness leads to less inventory, which improves costs and supply chain velocity. Flexibility allows more products to be consolidated into fewer factories, reduces the need for new facilities, simplifies the acquisition of additional capacity if needed, and reduces changeover time, which improves cost and velocity. The program incorporates both drug substance and drug product manufacturing, but this paper will focus on the drug substance elements of the program.

Chemogenetic inhibition in the dorsal striatum reveals regional specificity of direct and indirect pathway control of action sequencing.

Animals engage in intricate action sequences that are constructed during instrumental learning. There is broad consensus that the basal ganglia play a crucial role in the formation and fluid performance of action sequences. To investigate the role of the basal ganglia direct and indirect pathways in action sequencing, we virally expressed Cre-dependent Gi-DREADDs in either the dorsomedial (DMS) or dorsolateral (DLS) striatum during and/or after action sequence learning in D1 and D2 Cre rats. Action sequence performance in D1 Cre rats was slowed down early in training when DREADDs were activated in the DMS, but sped up when activated in the DLS. Acquisition of the reinforced sequence was hindered when DREADDs were activated in the DLS of D2 Cre rats. Outcome devaluation tests conducted after training revealed that the goal-directed control of action sequence rates was immune to chemogenetic inhibition-rats suppressed the rate of sequence performance when rewards were devalued. Sequence initiation latencies were generally sensitive to outcome devaluation, except in the case where DREADD activation was removed in D2 Cre rats that previously experienced DREADD activation in the DMS during training. Sequence completion latencies were generally not sensitive to outcome devaluation, except in the case where D1 Cre rats experienced DREADD activation in the DMS during training and test. Collectively, these results suggest that the indirect pathway originating from the DLS is part of a circuit involved in the effective reinforcement of action sequences, while the direct and indirect pathways originating from the DMS contribute to the goal-directed control of sequence completion and initiation, respectively.

Exploring the relationship between actions, habits, and automaticity in an action sequence task.

It is tempting to equate the automatization of an action sequence with the formation of a habit. However, the term "habit" specifically implies a failure to evaluate future consequences to guide behavior. To test if automatized sequences become habitual, we trained rats on an action sequence task for either 20 or 60 d and then conducted reward devaluation tests. While both groups showed equivalent goal-directed performance of the trained action sequence on a global measure of behavior, sequence initiation and completion times were differentially sensitive to outcome devaluation in moderately and extensively trained rats.

Mesostriatal dopamine is sensitive to changes in specific cue-reward contingencies.

Learning causal relationships relies on understanding how often one event precedes another. To investigate how dopamine neuron activity and neurotransmitter release change when a retrospective relationship is degraded for a specific pair of events, we used outcome-selective Pavlovian contingency degradation in rats. Conditioned responding was attenuated for the cue-reward contingency that was degraded, as was dopamine neuron activity in the midbrain and dopamine release in the ventral striatum in response to the cue and subsequent reward. Contingency degradation also abolished the trial-by-trial history dependence of the dopamine responses at the time of trial outcome. This profile of changes in cue- and reward-evoked responding is not easily explained by a standard reinforcement learning model. An alternative model based on learning causal relationships was better able to capture dopamine responses during contingency degradation, as well as conditioned behavior following optogenetic manipulations of dopamine during noncontingent rewards. Our results suggest that mesostriatal dopamine encodes the contingencies between meaningful events during learning.

Optogenetic induction of orbitostriatal long-term potentiation in the dorsomedial striatum elicits a persistent reduction of alcohol-seeking behavior in rats.

Uncontrolled drug-seeking and -taking behaviors are generally driven by maladaptive corticostriatal synaptic plasticity. The orbital frontal cortex (OFC) and its projections to the dorsomedial striatum (DMS) have been extensively implicated in drug-seeking and relapse behaviors. The influence of the synaptic plasticity of OFC projections to the DMS (OFC→DMS) on drug-seeking and -taking behaviors has not been fully characterized. To investigate this, we trained rats to self-administer 20% alcohol and then delivered an in vivo optogenetic protocol designed to induce long-term potentiation (LTP) selectively at OFC→DMS synapses. We selected LTP induction because we found that voluntary alcohol self-administration suppressed OFC→DMS transmission and LTP may normalize this transmission, consequently reducing alcohol-seeking behavior. Importantly, ex vivo slice electrophysiology studies confirmed that this in vivo optical stimulation protocol resulted in a significant increase in excitatory OFC→DMS transmission strength on day two after stimulation, suggesting that LTP was induced in vivo. Rat alcohol-seeking and -taking behaviors were significantly reduced on days 1-3, but not on days 7-11, after LTP induction. Striatal synaptic plasticity is modulated by several critical neurotransmitter receptors, including dopamine D1 receptors (D1Rs) and adenosine A2A receptors (A2ARs). We found that delivery of in vivo optical stimulation in the presence of a D1R antagonist abolished the LTP-associated decrease in alcohol-seeking behavior, whereas delivery in the presence of an A2AR antagonist may facilitate this LTP-induced behavioral change. These results demonstrate that alcohol-seeking behavior was negatively regulated by the potentiation of excitatory OFC→DMS neurotransmission. Our findings provide direct evidence that the OFC exerts "top-down" control of alcohol-seeking behavior via the DMS.

Goal-directed control on interval schedules does not depend on the action-outcome correlation.

When an organism's action is based on an anticipation of its consequences, that action is said to be goal-directed. It has long been thought that goal-directed control is made possible by experiencing a strong correlation between response rates and reward rates (Dickinson, 1985). To test this idea, we designed a set of experiments to determine whether the response rate-reward rate correlation is a reliable predictor of goal-directed control on interval schedules. In Experiment 1, rats were trained on random interval (RI) schedules in which the response rate-reward rate correlation was manipulated across groups. In tests of reward devaluation, rats behaved in a goal-directed manner regardless of the experienced correlation. In Experiment 2, rats once again experienced either a strong or weak correlation, but on RI schedules with lower overall reward densities. This time, behavior appeared habitual regardless of the experienced correlation. Experiment 3 confirmed that the density of the RI schedule influences goal-directed control, and also revealed that extensive training on these schedules resulted in goal-directed action. Finally, in Experiment 4 goal-directed responding was greater and emerged sooner on fixed than random interval schedules, but, again, was manifest after extensive training on the RI schedule. Taken together, our data suggest that goal-directed and habitual control are not determined by the correlation between response rates and reward rates. We discuss the importance of temporal uncertainty, action-outcome contiguity, and reinforcement probability in goal-directed control. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Contributions of the basal ganglia to action sequence learning and performance.

Animals engage in intricately woven and choreographed action sequences that are constructed from trial-and-error learning. The mechanisms by which the brain links together individual actions which are later recalled as fluid chains of behavior are not fully understood, but there is broad consensus that the basal ganglia play a crucial role in this process. This paper presents a comprehensive review of the role of the basal ganglia in action sequencing, with a focus on whether the computational framework of reinforcement learning can capture key behavioral features of sequencing and the neural mechanisms that underlie them. While a simple neurocomputational model of reinforcement learning can capture key features of action sequence learning, this model is not sufficient to capture goal-directed control of sequences or their hierarchical representation. The hierarchical structure of action sequences, in particular, poses a challenge for building better models of action sequencing, and it is in this regard that further investigations into basal ganglia information processing may be informative.

Elemental, configural, and occasion setting mechanisms in biconditional and patterning discriminations.

Three experiments explored the utility of considering mechanisms of occasion setting for understanding patterning and biconditional discriminations - two more complex conditional discriminations in which the stimulus-outcome relations of occasion setting are embedded. In Experiment 1, rats were trained in an appetitive conditioning task with either a biconditional or a patterning discrimination using relatively brief CSs (10s) and differential outcomes as USs. In this study, rats learned the positive patterning task before they had learned negative patterning, and the biconditional task was the most difficult. However, a detailed examination of the results suggested that rats trained in the biconditional task responded to the stimulus compounds mainly on the basis of individual stimulus-outcome associations. Different conditioned response (CR) topographies as a function of reinforcer type complicated interpretation of these results. Experiment 2 confirmed that the biconditional task, with the parameters used here, was not learned, regardless of whether training involved differential or non-differential outcomes. In Experiment 3 the CS duration was increased to 30s and two different USs were used that each supported similar CR topographies. Under these conditions, we observed that whereas the positive patterning task was learned most rapidly, the biconditional discrimination was learned faster than the negative patterning task. Considered in relation to other findings on patterning and biconditional discriminations, the results suggest that elemental, configural, and/or modulatory occasion setting mechanisms may play different roles in these complex conditional discrimination tasks especially as a function of stimulus duration and differential outcome training.