Serotonin Transporter Genetic Variation is Differentially Associated with Reflexive- and Reflective-Optimal Learning.

Learning to respond optimally under a broad array of environmental conditions is a critical brain function that requires engaging the cognitive systems that are optimal for solving the task at hand. Serotonin is implicated in learning and decision-making, but the specific functions of serotonin in system-level cognitive control remain unclear. Across 3 studies, we examined the influence of a polymorphism within the promoter region of the serotonin transporter gene (5-HTTLPR polymorphism in SLC6A4) on participants' ability to engage the task appropriate cognitive system when the reflexive (Experiments 1 and 2) or the reflective (Experiment 3) system was optimal. Critically, we utilized a learning task for which all aspects remain fixed with only the nature of the optimal cognitive processing system varying across experiments. Using large community samples, Experiments 1 and 2 (screened for psychiatric diagnosis) found that 5-HTTLPR S/LG allele homozygotes, with putatively lower serotonin transport functionality, outperformed LA allele homozygotes in a reflexive-optimal learning task. Experiment 3 used a large community sample, also screened for psychiatric diagnosis, and found that 5-HTTLPR LA homozygotes, with putatively higher serotonin transport functionality, outperformed S/LG allele homozygotes in a reflective-optimal learning task.

Negative Affect-Associated USV Acoustic Characteristics Predict Future Excessive Alcohol Drinking and Alcohol Avoidance in Male P and NP Rats.

BACKGROUND: Negative emotional status and adverse emotional events increase vulnerability to alcohol abuse. Ultrasonic vocalizations (USVs) emitted by rats are a well-established model of emotional status that can reflect positive or negative affective responses in real time. Most USV studies assess counts, yet each USV is a multidimensional data point characterized by several acoustic characteristics that may provide insight into the neurocircuitry underlying emotional response. METHODS: USVs emitted from selectively bred alcohol-naïve and alcohol-experienced alcohol-preferring and nonpreferring rats (P and NP rats) were recorded during 4-hour sessions on alternating days over 4 weeks. Linear mixed modeling (LMM) and linear discriminant analysis (LDA) were applied to USV acoustic characteristics (e.g., frequency, duration, power, and bandwidth) of negative affect (22 to 28 kilohertz [kHz])- and positive (50 to 55 kHz) affect-related USVs. RESULTS: Hundred percent separation between alcohol-naïve P and NP rats was achieved through a linear combination (produced by LDA) of USV acoustic characteristics of 22- to 28-kHz USVs, whereas poor separation (36.5%) was observed for 50- to 55-kHz USVs. 22- to 28-kHz LDA separation was high (87%) between alcohol-experienced P and NP rats, but was poor for 50- to 55-kHz USVs (57.3%). USV mean frequency and duration were the highest weighted characteristics in both the naïve and experienced 22- to 28-kHz LDA representations suggesting that alcohol experience does not alter the representations. LMM analyses of 22- to 28-kHz USV acoustic characteristics matched the LDA results. Poor LDA separation was observed between alcohol-naïve and alcohol-experienced P rats for both 22- to 28-kHz and 50- to 55-kHz USVs. CONCLUSIONS: Advanced statistical analysis of negative affect-associated USV data predicts future behaviors of excessive alcohol drinking and alcohol avoidance in selectively bred rats. USV characteristics across rat lines reveal affect-related motivation to consume alcohol and may predict neural pathways mediating emotional response. Further characterization of these differences could delineate particular neurocircuitry and methods to ameliorate dysregulated emotional states often observed in human alcohol abusers.

The Role of Corticostriatal Systems in Speech Category Learning.

One of the most difficult category learning problems for humans is learning nonnative speech categories. While feedback-based category training can enhance speech learning, the mechanisms underlying these benefits are unclear. In this functional magnetic resonance imaging study, we investigated neural and computational mechanisms underlying feedback-dependent speech category learning in adults. Positive feedback activated a large corticostriatal network including the dorsolateral prefrontal cortex, inferior parietal lobule, middle temporal gyrus, caudate, putamen, and the ventral striatum. Successful learning was contingent upon the activity of domain-general category learning systems: the fast-learning reflective system, involving the dorsolateral prefrontal cortex that develops and tests explicit rules based on the feedback content, and the slow-learning reflexive system, involving the putamen in which the stimuli are implicitly associated with category responses based on the reward value in feedback. Computational modeling of response strategies revealed significant use of reflective strategies early in training and greater use of reflexive strategies later in training. Reflexive strategy use was associated with increased activation in the putamen. Our results demonstrate a critical role for the reflexive corticostriatal learning system as a function of response strategy and proficiency during speech category learning.

Comparing the effects of positive and negative feedback in information-integration category learning.

Categorical learning is dependent on feedback. Here, we compare how positive and negative feedback affect information-integration (II) category learning. Ashby and O'Brien (2007) demonstrated that both positive and negative feedback are required to solve II category problems when feedback was not guaranteed on each trial, and reported no differences between positive-only and negative-only feedback in terms of their effectiveness. We followed up on these findings and conducted 3 experiments in which participants completed 2,400 II categorization trials across three days under 1 of 3 conditions: positive feedback only (PFB), negative feedback only (NFB), or both types of feedback (CP; control partial). An adaptive algorithm controlled the amount of feedback given to each group so that feedback was nearly equated. Using different feedback control procedures, Experiments 1 and 2 demonstrated that participants in the NFB and CP group were able to engage II learning strategies, whereas the PFB group was not. Additionally, the NFB group was able to achieve significantly higher accuracy than the PFB group by Day 3. Experiment 3 revealed that these differences remained even when we equated the information received on feedback trials. Thus, negative feedback appears significantly more effective for learning II category structures. This suggests that the human implicit learning system may be capable of learning in the absence of positive feedback.

Enhanced procedural learning of speech sound categories in a genetic variant of FOXP2.

A mutation of the forkhead box protein P2 (FOXP2) gene is associated with severe deficits in human speech and language acquisition. In rodents, the humanized form of FOXP2 promotes faster switching from declarative to procedural learning strategies when the two learning systems compete. Here, we examined a polymorphism of FOXP2 (rs6980093) in humans (214 adults; 111 females) for associations with non-native speech category learning success. Neurocomputational modeling results showed that individuals with the GG genotype shifted faster to procedural learning strategies, which are optimal for the task. These findings support an adaptive role for the FOXP2 gene in modulating the function of neural learning systems that have a direct bearing on human speech category learning.

Dopamine dependence in aggregate feedback learning: A computational cognitive neuroscience approach.

Procedural learning of skills depends on dopamine-mediated striatal plasticity. Most prior work investigated single stimulus-response procedural learning followed by feedback. However, many skills include several actions that must be performed before feedback is available. A new procedural-learning task is developed in which three independent and successive unsupervised categorization responses receive aggregate feedback indicating either that all three responses were correct, or at least one response was incorrect. Experiment 1 showed superior learning of stimuli in position 3, and that learning in the first two positions was initially compromised, and then recovered. An extensive theoretical analysis that used parameter space partitioning found that a large class of procedural-learning models, which predict propagation of dopamine release from feedback to stimuli, and/or an eligibility trace, fail to fully account for these data. The analysis also suggested that any dopamine released to the second or third stimulus impaired categorization learning in the first and second positions. A second experiment tested and confirmed a novel prediction of this large class of procedural-learning models that if the to-be-learned actions are introduced one-by-one in succession then learning is much better if training begins with the first action (and works forwards) than if it begins with the last action (and works backwards).

The role of age and executive function in auditory category learning.

Auditory categorization is a natural and adaptive process that allows for the organization of high-dimensional, continuous acoustic information into discrete representations. Studies in the visual domain have identified a rule-based learning system that learns and reasons via a hypothesis-testing process that requires working memory and executive attention. The rule-based learning system in vision shows a protracted development, reflecting the influence of maturing prefrontal function on visual categorization. The aim of the current study was twofold: (a) to examine the developmental trajectory of rule-based auditory category learning from childhood through adolescence and into early adulthood and (b) to examine the extent to which individual differences in rule-based category learning relate to individual differences in executive function. A sample of 60 participants with normal hearing-20 children (age range=7-12years), 21 adolescents (age range=13-19years), and 19 young adults (age range=20-23years)-learned to categorize novel dynamic "ripple" sounds using trial-by-trial feedback. The spectrotemporally modulated ripple sounds are considered the auditory equivalent of the well-studied "Gabor" patches in the visual domain. Results reveal that auditory categorization accuracy improved with age, with young adults outperforming children and adolescents. Computational modeling analyses indicated that the use of the task-optimal strategy (i.e., a conjunctive rule-based learning strategy) improved with age. Notably, individual differences in executive flexibility significantly predicted auditory category learning success. The current findings demonstrate a protracted development of rule-based auditory categorization. The results further suggest that executive flexibility coupled with perceptual processes play important roles in successful rule-based auditory category learning.

DRD4 long allele carriers show heightened attention to high-priority items relative to low-priority items.

Humans with seven or more repeats in exon III of the DRD4 gene (long DRD4 carriers) sometimes demonstrate impaired attention, as seen in attention-deficit hyperactivity disorder, and at other times demonstrate heightened attention, as seen in addictive behavior. Although the clinical effects of DRD4 are the focus of much work, this gene may not necessarily serve as a "risk" gene for attentional deficits, but as a plasticity gene where attention is heightened for priority items in the environment and impaired for minor items. Here we examine the role of DRD4 in two tasks that benefit from selective attention to high-priority information. We examine a category learning task where performance is supported by focusing on features and updating verbal rules. Here, selective attention to the most salient features is associated with good performance. In addition, we examine the Operation Span (OSPAN) task, a working memory capacity task that relies on selective attention to update and maintain items in memory while also performing a secondary task. Long DRD4 carriers show superior performance relative to short DRD4 homozygotes (six or less tandem repeats) in both the category learning and OSPAN tasks. These results suggest that DRD4 may serve as a "plasticity" gene where individuals with the long allele show heightened selective attention to high-priority items in the environment, which can be beneficial in the appropriate context.

The C957T polymorphism in the dopamine receptor D2 gene modulates domain-general category learning.

Adaptive learning from reward and punishment is vital for human survival. Striatal and frontal dopaminergic activities are associated with adaptive learning. For example, the C957T single nucleotide polymorphism of the dopamine receptor D2 (DRD2) gene alters striatal D2 receptor availability and affects individuals' adaptive learning ability. Specifically, individuals with the T/T genotype, which is associated with higher striatal D2 availability, show enhanced learning from negative outcomes. Prior work examining DRD2 genetic variability has focused primarily on frontally mediated reflective learning that is under effortful, conscious control. However, less is known about a more automatic, striatally mediated reflexive learning. Here we examined the extent to which this polymorphism differentially influences reflective and reflexive learning across visual and auditory modalities. We employed rule-based (RB) and information-integration (II) category learning paradigms that target reflective and reflexive learning, respectively. Results revealed an advantage in II category learning but poorer RB category learning in T/T homozygotes. The pattern of results was consistent across sensory modalities. These findings suggest that this DRD2 polymorphism exerts opposite influences on domain-general frontally mediated reflective learning and striatally mediated reflexive learning.

A frontal dopamine system for reflective exploratory behavior.

The COMT gene modulates dopamine levels in prefrontal cortex with Met allele carriers having lower COMT enzyme activity and, therefore, higher dopamine levels compared to Val/Val homozygotes. Concordantly, Val/Val homozygotes tend to perform worse and display increased (interpreted as inefficient) frontal activation in certain cognitive tasks. In a sample of 209 participants, we test the hypothesis that Met carriers will be advantaged in a decision-making task that demands sequencing exploratory and exploitive choices to minimize uncertainty about the reward structure in the environment. Previous work suggests that optimal performance depends on limited cognitive resources supported by prefrontal systems. If so, Met carriers should outperform Val/Val homozygotes, particularly under dual-task conditions that tax limited cognitive resources. In accord with these a priori predictions, Met carriers were more resilient in the face of cognitive load, continuing to explore in a sophisticated manner. We fit computational models that embody sophisticated reflective and simple reflexive strategies to further evaluate participants' exploration behavior. The Ideal Actor model reflectively updates beliefs and plans ahead, taking into account the information gained by each choice and making choices that maximize long-term payoffs. In contrast, the Naïve Reinforcement Learning (RL) model instantiates the reflexive account of choice, in which the values of actions are based only on the rewards experienced so far. Its beliefs are updated reflexively in response to observed changes in rewards. Converging with standard analyses, Met carriers were best characterized by the Ideal Actor model, whereas Val/Val homozygotes were best characterized by the Naive RL model, particularly under dual-task conditions.

Alcohol-preferring P rats emit spontaneous 22-28 kHz ultrasonic vocalizations that are altered by acute and chronic alcohol experience.

BACKGROUND: Emotional states are often thought to drive excessive alcohol intake and influence the development of alcohol use disorders. To gain insight into affective properties associated with excessive alcohol intake, we utilized ultrasonic vocalization (USV) detection and analyses to characterize the emotional phenotype of selectively bred alcohol-preferring (P) rats; an established animal model of excessive alcohol intake. USVs emitted by rodents have been convincingly associated with positive (50-55 kHz frequency-modulated [FM]) and negative (22-28 kHz) affective states. Therefore, we hypothesized that 50-55 and 22-28 kHz USV emission patterns in P rats would reveal a unique emotional phenotype sensitive to alcohol experience. METHODS: 50-55 kHz FM and 22-28 kHz USVs elicited from male P rats were assessed during access to water, 15 and 30% EtOH (v/v). Ethanol (EtOH; n = 12) or water only (Control; n = 4) across 8 weeks of daily drinking-in-the-dark (DID) sessions. RESULTS: Spontaneous 22-28 kHz USVs are emitted by alcohol-naïve P rats and are enhanced by alcohol experience. During DID sessions when alcohol was not available (e.g., "EtOH OFF" intervals), significantly more 22-28 kHz than 50-55 kHz USVs were elicited, while significantly more 50-55 kHz FM than 22-28 kHz USVs were emitted when alcohol was available (e.g., "EtOH ON" intervals). In addition, USV acoustic property analyses revealed chronic effects of alcohol experience on 22-28 kHz USV mean frequency, indicative of lasting alcohol-mediated alterations to neural substrates underlying emotional response. CONCLUSIONS: Our findings demonstrate that acute and chronic effects of alcohol exposure are reflected in changes in 22-28 and 50-55 kHz FM USV counts and acoustic patterns. These data support the notion that initiation and maintenance of alcohol intake in P rats may be due to a unique, alcohol-responsive emotional phenotype and further suggest that spontaneous 22-28 kHz USVs serve as behavioral markers for excessive drinking vulnerability.

Chronic motivational state interacts with task reward structure in dynamic decision-making.

Research distinguishes between a habitual, model-free system motivated toward immediately rewarding actions, and a goal-directed, model-based system motivated toward actions that improve future state. We examined the balance of processing in these two systems during state-based decision-making. We tested a regulatory fit hypothesis (Maddox & Markman, 2010) that predicts that global trait motivation affects the balance of habitual- vs. goal-directed processing but only through its interaction with the task framing as gain-maximization or loss-minimization. We found support for the hypothesis that a match between an individual's chronic motivational state and the task framing enhances goal-directed processing, and thus state-based decision-making. Specifically, chronic promotion-focused individuals under gain-maximization and chronic prevention-focused individuals under loss-minimization both showed enhanced state-based decision-making. Computational modeling indicates that individuals in a match between global chronic motivational state and local task reward structure engaged more goal-directed processing, whereas those in a mismatch engaged more habitual processing.

Criterion learning in rule-based categorization: simulation of neural mechanism and new data.

In perceptual categorization, rule selection consists of selecting one or several stimulus-dimensions to be used to categorize the stimuli (e.g., categorize lines according to their length). Once a rule has been selected, criterion learning consists of defining how stimuli will be grouped using the selected dimension(s) (e.g., if the selected rule is line length, define 'long' and 'short'). Very little is known about the neuroscience of criterion learning, and most existing computational models do not provide a biological mechanism for this process. In this article, we introduce a new model of rule learning called Heterosynaptic Inhibitory Criterion Learning (HICL). HICL includes a biologically-based explanation of criterion learning, and we use new category-learning data to test key aspects of the model. In HICL, rule selective cells in prefrontal cortex modulate stimulus-response associations using pre-synaptic inhibition. Criterion learning is implemented by a new type of heterosynaptic error-driven Hebbian learning at inhibitory synapses that uses feedback to drive cell activation above/below thresholds representing ionic gating mechanisms. The model is used to account for new human categorization data from two experiments showing that: (1) changing rule criterion on a given dimension is easier if irrelevant dimensions are also changing (Experiment 1), and (2) showing that changing the relevant rule dimension and learning a new criterion is more difficult, but also facilitated by a change in the irrelevant dimension (Experiment 2). We conclude with a discussion of some of HICL's implications for future research on rule learning.

Influence of depressive symptoms on speech perception in adverse listening conditions.

It is widely acknowledged that individuals with elevated depressive symptoms exhibit deficits in inter-personal communication. Research has primarily focused on speech production in individuals with elevated depressive symptoms. Little is known about speech perception in individuals with elevated depressive symptoms, especially in challenging listening conditions. Here, we examined speech perception in young adults with low- or high-depressive (HD) symptoms in the presence of a range of maskers. Maskers were selected to reflect various levels of informational masking (IM), which refers to cognitive interference due to signal and masker similarity, and energetic masking (EM), which refers to peripheral interference due to signal degradation by the masker. Speech intelligibility data revealed that individuals with HD symptoms did not differ from those with low-depressive symptoms during EM, but they exhibited a selective deficit during IM. Since IM is a common occurrence in real-world social settings, this listening deficit may exacerbate communicative difficulties.

Training attention improves decision making in individuals with elevated self-reported depressive symptoms.

Depression is often characterized by attentional biases toward negative items and away from positive items, which likely affects reward and punishment processing. Recent work has reported that training attention away from negative stimuli reduced this bias and reduced depressive symptoms. However, the effect of attention training on subsequent learning has yet to be explored. In the present study, participants were required to learn to maximize reward during decision making. Undergraduates with elevated self-reported depressive symptoms received attention training toward positive stimuli prior to performing the decision-making task (n = 20; active training). The active-training group was compared to two other groups: undergraduates with elevated self-reported depressive symptoms who received placebo training (n = 22; placebo training) and a control group with low levels of depressive symptoms (n = 33; nondepressive control). The placebo-training depressive group performed worse and switched between options more than did the nondepressive controls on the reward maximization task. However, depressives that received active training performed as well as the nondepressive controls. Computational modeling indicated that the placebo-trained group learned more from negative than from positive prediction errors, leading to more frequent switching. The nondepressive control and active-training depressive groups showed similar learning from positive and negative prediction errors, leading to less-frequent switching and better performance. Our results indicate that individuals with elevated depressive symptoms are impaired at reward maximization, but that the deficit can be improved with attention training toward positive stimuli.

State-based versus reward-based motivation in younger and older adults.

Recent decision-making work has focused on a distinction between a habitual, model-free neural system that is motivated toward actions that lead directly to reward and a more computationally demanding goal-directed, model-based system that is motivated toward actions that improve one's future state. In this article, we examine how aging affects motivation toward reward-based versus state-based decision making. Participants performed tasks in which one type of option provided larger immediate rewards but the alternative type of option led to larger rewards on future trials, or improvements in state. We predicted that older adults would show a reduced preference for choices that led to improvements in state and a greater preference for choices that maximized immediate reward. We also predicted that fits from a hybrid reinforcement-learning model would indicate greater model-based strategy use in younger than in older adults. In line with these predictions, older adults selected the options that maximized reward more often than did younger adults in three of the four tasks, and modeling results suggested reduced model-based strategy use. In the task where older adults showed similar behavior to younger adults, our model-fitting results suggested that this was due to the utilization of a win-stay-lose-shift heuristic rather than a more complex model-based strategy. Additionally, within older adults, we found that model-based strategy use was positively correlated with memory measures from our neuropsychological test battery. We suggest that this shift from state-based to reward-based motivation may be due to age related declines in the neural structures needed for more computationally demanding model-based decision making.

Context-dependent savings in procedural category learning.

Environmental context can have a profound influence on the efficacy of intervention protocols designed to eliminate undesirable behaviors. This is clearly seen in drug rehabilitation clinics where patients often relapse soon after leaving the context of the treatment facility. A similar pattern is commonly observed in controlled laboratory studies of context-dependent savings in instrumental conditioning, where simply placing an animal back into the original conditioning chamber can renew an extinguished instrumental response. Surprisingly, context-dependent savings in human procedural learning has not been carefully examined in the laboratory. Here, we provide the first known empirical demonstration of context-dependent savings in a perceptual categorization task known to recruit procedural learning. We also present a computational account of these savings using a biologically detailed model in which a key role is played by cholinergic interneurons in the striatum.

Age-related declines in the fidelity of newly acquired category representations.

We present a theory suggesting that the ability to build category representations that reflect the nuances of category structures in the environment depends upon clustering mechanisms instantiated in an MTL-PFC-based circuit. Because function in this circuit declines with age, we predict that the ability to build category representations will be impaired in older adults. Consistent with this prediction, we find that older adults are impaired relative to younger adults at learning nuanced category structures that contain exceptions to the rule. Model-based analysis reveals that this deficit arises from older adults' failure to engage clustering mechanisms to separate exception and rule-following items in memory.

With age comes wisdom: decision making in younger and older adults.

In two experiments, younger and older adults performed decision-making tasks in which reward values available were either independent of or dependent on the previous sequence of choices made. The choice-independent task involved learning and exploiting the options that gave the highest rewards on each trial. In this task, the stability of the expected reward for each option was not influenced by the previous choices participants made. The choice-dependent task involved learning how each choice influenced future rewards for two options and making the best decisions based on that knowledge. Younger adults performed better when rewards were independent of choice, whereas older adults performed better when rewards were dependent on choice. These findings suggest a fundamental difference in the way in which younger adults and older adults approach decision-making situations. We discuss the results in the context of prominent decision-making theories and offer possible explanations based on neurobiological and behavioral changes associated with aging.

Development of implicit and explicit category learning.

We present two studies that examined developmental differences in the implicit and explicit acquisition of category knowledge. College-attending adults consistently outperformed school-age children on two separate information-integration paradigms due to children's more frequent use of an explicit rule-based strategy. Accuracy rates were also higher for adults on a unidimensional rule-based task due to children's more frequent use of the irrelevant dimension to guide their behavior. Results across these two studies suggest that the ability to learn categorization structures may be dependent on a child's ability to inhibit output from the explicit system.