Fractionating the all-or-nothing definition of goal-directed and habitual decision-making.

Goal-directed and habitual decision-making are fundamental processes that support the ongoing adaptive behavior. There is a growing interest in examining their disruption in psychiatric disease, often with a focus on a disease shifting control from one process to the other, usually a shift from goal-directed to habitual control. However, several different experimental procedures can be used to probe whether decision-making is under goal-directed or habitual control, including outcome devaluation and contingency degradation. These different experimental procedures may recruit diverse behavioral and neural processes. Thus, there are potentially many opportunities for these disease phenotypes to manifest as alterations to both goal-directed and habitual controls. In this review, we highlight the examples of behavioral and neural circuit divergence and similarity, and suggest that interpretation based on behavioral processes recruited during testing may leave more room for goal-directed and habitual decision-making to coexist. Furthermore, this may improve our understanding of precisely what the involved neural mechanisms underlying aspects of goal-directed and habitual behavior are, as well as how disease affects behavior and these circuits.

Chronic alcohol exposure alters action control via hyperactive premotor corticostriatal activity.

Alcohol use disorder (AUD) alters decision-making control over actions, but disruptions to the responsible neural circuit mechanisms are unclear. Premotor corticostriatal circuits are implicated in balancing goal-directed and habitual control over actions and show disruption in disorders with compulsive, inflexible behaviors, including AUD. However, whether there is a causal link between disrupted premotor activity and altered action control is unknown. Here, we find that mice chronically exposed to alcohol (chronic intermittent ethanol [CIE]) showed impaired ability to use recent action information to guide subsequent actions. Prior CIE exposure resulted in aberrant increases in the calcium activity of premotor cortex (M2) neurons that project to the dorsal medial striatum (M2-DMS) during action control. Chemogenetic reduction of this CIE-induced hyperactivity in M2-DMS neurons rescued goal-directed action control. This suggests a direct, causal relationship between chronic alcohol disruption to premotor circuits and decision-making strategy and provides mechanistic support for targeting activity of human premotor regions as a potential treatment in AUD.

Information normally considered task-irrelevant drives decision-making and affects premotor circuit recruitment.

Decision-making is a continuous and dynamic process with prior experience reflected in and used by the brain to guide adaptive behavior. However, most neurobiological studies constrain behavior and/or analyses to task-related variables, not accounting for the continuous internal and temporal space in which they occur. We show mice rely on information learned through recent and longer-term experience beyond just prior actions and reward - including checking behavior and the passage of time - to guide self-initiated, self-paced, and self-generated actions. These experiences are represented in secondary motor cortex (M2) activity and its projections into dorsal medial striatum (DMS). M2 integrates this information to bias strategy-level decision-making, and DMS projections reflect specific aspects of this recent experience to guide actions. This suggests diverse aspects of experience drive decision-making and its neural representation, and shows premotor corticostriatal circuits are crucial for using selective aspects of experiential information to guide adaptive behavior.

Orbitofrontal cortex populations are differentially recruited to support actions.

The ability to use information from one's prior actions is necessary for decision-making. While orbitofrontal cortex (OFC) has been hypothesized as key for inferences made using cue and value-related information, whether OFC populations contribute to the use of information from volitional actions to guide behavior is not clear. Here, we used a self-paced lever-press hold-down task in which mice infer prior lever-press durations to guide subsequent action performance. We show that the activity of genetically identified lateral OFC (lOFC) subpopulations differentially instantiate current and prior action information during ongoing action execution. Transient state-dependent lOFC circuit disruptions of specified subpopulations reduced the encoding of ongoing press durations but did not disrupt the use of prior action information to guide future action performance. In contrast, a chronic functional loss of lOFC circuit activity resulted in increased reliance on recently executed lever-press durations and impaired contingency reversal, suggesting the recruitment of compensatory mechanisms that resulted in repetitive action control. Our results identify a novel role for lOFC in the integration of action information to guide adaptive behavior.

Different Effects of Alcohol Exposure on Action and Outcome-Related Orbitofrontal Cortex Activity.

Alcohol dependence can result in long-lasting deficits to decision-making and action control. Neurobiological investigations have identified orbitofrontal cortex (OFC) as important for outcome-related contributions to goal-directed actions during decision-making. Prior work has shown that alcohol dependence induces long-lasting changes to OFC function that persist into protracted withdrawal and disrupts goal-directed control over actions. However, it is unclear whether these changes in function alter representation of action and outcome-related neural activity in OFC. Here, we used the well-validated chronic intermittent ethanol (CIE) exposure and withdrawal procedure to model alcohol dependence in mice and performed in vivo extracellular recordings during an instrumental task in which lever-press actions made for a food outcome. We found alcohol dependence disrupted goal-directed action control and increased OFC activity associated with lever-pressing but decreased OFC activity during outcome-related epochs. The ability to decode outcome-related information, but not action information, from OFC activity following CIE exposure was reduced. Hence, chronic alcohol exposure induced a long-lasting disruption to OFC function such that activity associated with actions was enhanced, but OFC activity contributions to outcome-related information was diminished. This has important implications for hypotheses regarding compulsive and habitual phenotypes observed in addiction.

A Push For Examining Subjective Experience in Value-Based Decision-Making.

Subjective experience is a powerful contributor to value-based decision-making. Not every decision is the same, nor made in isolation. Rather, decision-making relies on historical information and internal states for adaptive control. Hence, it is inherently continuous with respect to time - one decision or action evolves into the next. However, forays into the neurobiological underpinnings of decision-making have too frequently ignored the contribution of such continuous subjective experience, instead tying circuit activity and brain area involvement to discrete averaged behaviors and task parameters. While much information has been gained through these investigations, recent works have demonstrated the potential for a greater understanding of neural mechanisms when the continuous, experiential nature of behavior is integrated into the investigation. Such integration has important implications for disease states with disordered decision-making such as addiction, where subjective experience is a large contributor to the disorder.

Mechanism for differential recruitment of orbitostriatal transmission during actions and outcomes following chronic alcohol exposure.

Psychiatric disease often produces symptoms that have divergent effects on neural activity. For example, in drug dependence, dysfunctional value-based decision-making and compulsive-like actions have been linked to hypo- and hyperactivity of orbital frontal cortex (OFC)-basal ganglia circuits, respectively; however, the underlying mechanisms are unknown. Here we show that alcohol-exposed mice have enhanced activity in OFC terminals in dorsal striatum (OFC-DS) associated with actions, but reduced activity of the same terminals during periods of outcome retrieval, corresponding with a loss of outcome control over decision-making. Disrupted OFC-DS terminal activity was due to a dysfunction of dopamine-type 1 receptors on spiny projection neurons (D1R SPNs) that resulted in increased retrograde endocannabinoid signaling at OFC-D1R SPN synapses reducing OFC-DS transmission. Blocking CB1 receptors restored OFC-DS activity in vivo and rescued outcome-based control over decision-making. These findings demonstrate a circuit-, synapse-, and computation-specific mechanism gating OFC activity in alcohol-exposed mice.

Orbital Frontal Cortex Projections to Secondary Motor Cortex Mediate Exploitation of Learned Rules.

Animals face the dilemma between exploiting known opportunities and exploring new ones, a decision-making process supported by cortical circuits. While different types of learning may bias exploration, the circumstances and the degree to which bias occurs is unclear. We used an instrumental lever press task in mice to examine whether learned rules generalize to exploratory situations and the cortical circuits involved. We first trained mice to press one lever for food and subsequently assessed how that learning influenced pressing of a second novel lever. Using outcome devaluation procedures we found that novel lever exploration was not dependent on the food value associated with the trained lever. Further, changes in the temporal uncertainty of when a lever press would produce food did not affect exploration. Instead, accrued experience with the instrumental contingency was strongly predictive of test lever pressing with a positive correlation between experience and trained lever exploitation, but not novel lever exploration. Chemogenetic attenuation of orbital frontal cortex (OFC) projection into secondary motor cortex (M2) biased novel lever exploration, suggesting that experience increases OFC-M2 dependent exploitation of learned associations but leaves exploration constant. Our data suggests exploitation and exploration are parallel decision-making systems that do not necessarily compete.

Role of adenosine receptor subtypes in methamphetamine reward and reinforcement.

The neurobiology of methamphetamine (MA) remains largely unknown despite its high abuse liability. The present series of studies explored the role of adenosine receptors on MA reward and reinforcement and identified alterations in the expression of adenosine receptors in dopamine terminal areas following MA administration in rats. We tested whether stimulating adenosine A1 or A2A receptor subtypes would influence MA-induced place preference or MA self-administration on fixed and progressive ratio schedules in male Sprague-Dawley rats. Stimulation of either adenosine A1 or A2A receptors significantly reduced the development of MA-induced place preference. Stimulating adenosine A1, but not A2A, receptors reduced MA self-administration responding. We next tested whether repeated experimenter-delivered MA administration would alter the expression of adenosine receptors in the striatal areas using immunoblotting. We observed no change in the expression of adenosine receptors. Lastly, rats were trained to self-administer MA or saline for 14 days and we detected changes in adenosine A1 and A2A receptor expression using immunoblotting. MA self-administration significantly increased adenosine A1 in the nucleus accumbens shell, caudate-putamen and prefrontal cortex. MA self-administration significantly decreased adenosine A2A receptor expression in the nucleus accumbens shell, but increased A2A receptor expression in the amygdala. These findings demonstrate that MA self-administration produces selective alterations in adenosine receptor expression in the nucleus accumbens shell and that stimulation of adenosine receptors reduces several behavioral indices of MA addiction. Together, these studies shed light onto the neurobiological alterations incurred through chronic MA use that may aid in the development of treatments for MA addiction.

Effects of adolescent caffeine consumption on cocaine sensitivity.

Caffeine is the most commonly used psychoactive substance, and consumption by adolescents has risen markedly in recent years. We identified the effects of adolescent caffeine consumption on cocaine sensitivity and determined neurobiological changes within the nucleus accumbens (NAc) that may underlie caffeine-induced hypersensitivity to cocaine. Male Sprague-Dawley rats consumed caffeine (0.3 g/l) or water for 28 days during adolescence (postnatal day 28-55; P28-P55) or adulthood (P67-P94). Testing occurred in the absence of caffeine during adulthood (P62-82 or P101-121). Cocaine-induced and quinpirole (D2 receptor agonist)-induced locomotion was enhanced in rats that consumed caffeine during adolescence. Adolescent consumption of caffeine also enhanced the development of a conditioned place preference at a sub-threshold dose of cocaine (7.5 mg/kg, i.p.). These behavioral changes were not observed in adults consuming caffeine for an equivalent period of time. Sucrose preferences were not altered in rats that consumed caffeine during adolescence, suggesting there are no differences in natural reward. Caffeine consumption during adolescence reduced basal dopamine levels and augmented dopamine release in the NAc in response to cocaine (5 mg/kg, i.p.). Caffeine consumption during adolescence also increased the expression of the dopamine D2 receptor, dopamine transporter, and adenosine A1 receptor and decreased adenosine A2A receptor expression in the NAc. Consumption of caffeine during adulthood increased adenosine A1 receptor expression in the NAc, but no other protein expression changes were observed. Together these findings suggest that caffeine consumption during adolescence produced changes in the NAc that are evident in adulthood and may contribute to increases in cocaine-mediated behaviors.