Learning to Choose: Behavioral Dynamics Underlying the Initial Acquisition of Decision-Making.

Current theories of decision-making propose that decisions arise through competition between choice options. Computational models of the decision process estimate how quickly information about choice options is integrated and how much information is needed to trigger a choice. Experiments using this approach typically report data from well-trained participants. As such, we do not know how the decision process evolves as a decision-making task is learned for the first time. To address this gap, we used a behavioral design separating learning the value of choice options from learning to make choices. We trained male rats to respond to single visual stimuli with different reward values. Then, we trained them to make choices between pairs of stimuli. Initially, the rats responded more slowly when presented with choices. However, as they gained experience in making choices, this slowing reduced. Response slowing on choice trials persisted throughout the testing period. We found that it was specifically associated with increased exponential variability when the rats chose the higher value stimulus. Additionally, our analysis using drift diffusion modeling revealed that the rats required less information to make choices over time. These reductions in the decision threshold occurred after just a single session of choice learning. These findings provide new insights into the learning process of decision-making tasks. They suggest that the value of choice options and the ability to make choices are learned separately and that experience plays a crucial role in improving decision-making performance.

Learning to Choose: Behavioral Dynamics Underlying the Initial Acquisition of Decision Making.

Current theories of decision making propose that decisions arise through competition between choice options. Computational models of the decision process estimate how quickly information about choice options is integrated and how much information is needed to trigger a choice. Experiments using this approach typically report data from well-trained participants. As such, we do not know how the decision process evolves as a decision-making task is learned for the first time. To address this gap, we used a behavioral design separating learning the value of choice options from learning to make choices. We trained male rats to respond to single visual stimuli with different reward values. Then, we trained them to make choices between pairs of stimuli. Initially, the rats responded more slowly when presented with choices. However, as they gained experience in making choices, this slowing reduced. Response slowing on choice trials persisted throughout the testing period. We found that it was specifically associated with increased exponential variability when the rats chose the higher value stimulus. Additionally, our analysis using drift diffusion modeling revealed that the rats required less information to make choices over time. Surprisingly, we observed reductions in the decision threshold after just a single session of choice learning. These findings provide new insights into the learning process of decision-making tasks. They suggest that the value of choice options and the ability to make choices are learned separately, and that experience plays a crucial role in improving decision-making performance.

Noradrenergic regulation of two-armed bandit performance.

Reversal learning depends on cognitive flexibility. Many reversal learning studies assess cognitive flexibility based on the number of reversals that occur over a test session. Reversals occur when an option is repeatedly chosen, e.g., eight times in a row. This design feature encourages win-stay behavior and thus makes it difficult to understand how win-stay decisions influence reversal performance. We used an alternative design, reversals over blocks of trials independent of performance, to study how perturbations of the medial orbital cortex and the noradrenergic system influence reversal learning. We found that choice accuracy varies independently of win-stay behavior and the noradrenergic system controls sensitivity to positive feedback during reversal learning. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

An Open Source Platform for Presenting Dynamic Visual Stimuli.

Operant behavior procedures often rely on visual stimuli to cue the initiation or secession of a response, and to provide a means for discriminating between two or more simultaneously available responses. While primate and human studies typically use Liquid-Crystal Display (LCD) or Organic Light-Emitting Diode (OLED) monitors and touch screens, rodent studies use a variety of methods to present visual cues ranging from traditional incandescent light bulbs, single LEDs, and, more recently, touch screen monitors. Commercially available systems for visual stimulus presentation are costly, challenging to customize, and are typically closed source. We developed an open-source, highly-modifiable visual stimulus presentation platform that can be combined with a 3D-printed operant response device. The device uses an 8 × 8 matrix of LEDs, and can be expanded to control much larger LED matrices. Implementing the platform is low-cost (<$70 USD per device in the year 2020). Using the platform, we trained rats to make nosepoke responses and discriminate between two distinct visual cues in a location-independent manner. This visual stimulus presentation platform is a cost-effective way to implement complex visually-guided operant behavior, including the use of moving or dynamically changing visual stimuli.

The rat medial frontal cortex controls pace, but not breakpoint, in a progressive ratio licking task.

The medial frontal cortex (MFC) is crucial for selecting actions and evaluating their outcomes. Outcome monitoring may be triggered by rostral parts of the MFC, which contain neurons that are modulated by reward consumption and are necessary for the expression of relative reward value. Here, we examined if the MFC further has a role in the control of instrumental licking. We used a progressive ratio licking task in which rats had to make increasing numbers of licks to receive liquid sucrose rewards. We determined what measures of progressive ratio performance are sensitive to value by testing rats with rewards containing 0%-16% sucrose. We found some measures (breakpoint, number of licking bouts) were sensitive to sucrose concentration and others (response rate, duration of licking bouts) were not. Then, we examined the effects of reversibly inactivating rostral (medial orbital) and caudal (prelimbic) parts of the MFC. We were surprised to find that inactivation had no effects on measures associated with value (e.g., breakpoint). Instead, inactivation altered behavioral measures associated with the pace of task performance (response rate and time to break). These effects depended on where inactivations were made. Response rates increased and time to break decreased when the caudal prelimbic area was inactivated. By contrast, response rates decreased and the time to break increased when the rostral medial orbital cortex was inactivated. Our findings suggest that the medial frontal cortex has a role in maintaining task engagement, but not in the motivational control of action, in the progressive ratio licking task. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

What, If Anything, Is Rodent Prefrontal Cortex?

Prefrontal cortex (PFC) means different things to different people. In recent years, there has been a major increase in publications on the PFC, especially using mice. However, inconsistencies in the nomenclature and anatomical boundaries of PFC areas has made it difficult for researchers to compare data and interpret findings across species. We conducted a meta-analysis of publications on the PFC of humans and rodents and found dramatic differences in the focus of research on these species. In addition, we compared anatomical terms and criteria across several common rodent brain atlases and found inconsistencies among, and even within, leading atlases. To assess the impact of these issues on the research community, we conducted a survey of established PFC researchers on their use of anatomical terms and found little consensus. We report on the results of the survey and propose an alternative scheme for interpreting data from rodent studies, based on structural analysis of the corpus callosum and nomenclature used in research on the anterior cingulate cortex (ACC) of primates.

Cholinergic and ghrelinergic receptors and KCNQ channels in the medial PFC regulate the expression of palatability.

The medial prefrontal cortex (mPFC) is a key brain region for the control of consummatory behavior. Neuronal activity in this area is modulated when rats initiate consummatory licking and reversible inactivations eliminate reward contrast effects and reduce a measure of palatability, the duration of licking bouts. Together, these data suggest the hypothesis that rhythmic neuronal activity in the mPFC is crucial for the control of consummatory behavior. The muscarinic cholinergic system is known to regulate membrane excitability and control low-frequency rhythmic activity in the mPFC. Muscarinic receptors (mAChRs) act through KCNQ (Kv7) potassium channels, which have recently been linked to the orexigenic peptide ghrelin. To understand if drugs that act on KCNQ channels within the mPFC have effects on consummatory behavior, we made infusions of several muscarinic drugs (scopolamine, oxotremorine, physostigmine), the KCNQ channel blocker XE-991, and ghrelin into the mPFC and evaluated their effects on consummatory behavior. A consistent finding across all drugs was an effect on the duration of licking bouts when animals consume solutions with a relatively high concentration of sucrose. The muscarinic antagonist scopolamine reduced bout durations, both systemically and intra-cortically. By contrast, the muscarinic agonist oxotremorine, the cholinesterase inhibitor physostigmine, the KCNQ channel blocker XE-991, and ghrelin all increased the durations of licking bouts when infused into the mPFC. Our findings suggest that cholinergic and ghrelinergic signaling in the mPFC, acting through KCNQ channels, regulates the expression of palatability.