Reinforcement learning processes as forecasters of depression remission.

BACKGROUND: Aspects of reinforcement learning have been associated with specific depression symptoms and may inform the course of depressive illness. METHODS: We applied support vector machines to investigate whether blood­oxygen-level dependent (BOLD) responses linked with neural prediction error (nPE) and neural expected value (nEV) from a probabilistic learning task could forecast depression remission. We investigated whether predictions were moderated by treatment use or symptoms. Participants included 55 individuals (n = 39 female) with a depression diagnosis at baseline; 36 of these individuals completed standard cognitive behavioral therapy and 19 were followed during naturalistic course of illness. All participants were assessed for depression diagnosis at a follow-up visit. RESULTS: Both nPE and nEV classifiers forecasted remission significantly better than null classifiers. The nEV classifier performed significantly better than the nPE classifier. We found no main or interaction effects of treatment status on nPE or nEV accuracy. We found a significant interaction between nPE-forecasted remission status and anhedonia, but not for negative affect or anxious arousal, when controlling for nEV-forecasted remission status. LIMITATIONS: Our sample size, while comparable to that of other studies, limits options for maximizing and evaluating model performance. We addressed this with two standard methods for optimizing model performance (90:10 train and test scheme and bootstrapped sampling). CONCLUSIONS: Results support nEV and nPE as relevant biobehavioral signals for understanding depression outcome independent of treatment status, with nEV being stronger than nPE as a predictor of remission. Reinforcement learning variables may be useful components of an individualized medicine framework for depression healthcare.

Reinforcement Learning Disruptions in Individuals With Depression and Sensitivity to Symptom Change Following Cognitive Behavioral Therapy.

Importance: Major depressive disorder is prevalent and impairing. Parsing neurocomputational substrates of reinforcement learning in individuals with depression may facilitate a mechanistic understanding of the disorder and suggest new cognitive therapeutic targets. Objective: To determine associations among computational model-derived reinforcement learning parameters, depression symptoms, and symptom changes after treatment. Design, Setting, and Participants: In this mixed cross-sectional-cohort study, individuals performed reward and loss variants of a probabilistic learning task during functional magnetic resonance imaging at baseline and follow-up. A volunteer sample with and without a depression diagnosis was recruited from the community. Participants were assessed from July 2011 to February 2017, and data were analyzed from May 2017 to May 2021. Main Outcomes and Measures: Computational model-based analyses of participants' choices assessed a priori hypotheses about associations between components of reward-based and loss-based learning with depression symptoms. Changes in both learning parameters and symptoms were then assessed in a subset of participants who received cognitive behavioral therapy (CBT). Results: Of 101 included adults, 69 (68.3%) were female, and the mean (SD) age was 34.4 (11.2) years. A total of 69 participants with a depression diagnosis and 32 participants without a depression diagnosis were included at baseline; 48 participants (28 with depression who received CBT and 20 without depression) were included at follow-up (mean [SD] of 115.1 [15.6] days). Computational model-based analyses of behavioral choices and neural data identified associations of learning with symptoms during reward learning and loss learning, respectively. During reward learning only, anhedonia (and not negative affect or arousal) was associated with model-derived learning parameters (learning rate: posterior mean regression ß = -0.14; 95% credible interval [CrI], -0.12 to -0.03; outcome sensitivity: posterior mean regression ß = 0.18; 95% CrI, 0.02 to 0.37) and neural learning signals (moderation of association between striatal prediction error and expected value signals: t97 = -2.10; P = .04). During loss learning only, negative affect (and not anhedonia or arousal) was associated with learning parameters (outcome shift: posterior mean regression ß = -0.11; 95% CrI, -0.20 to -0.01) and disrupted neural encoding of learning signals (association with subgenual anterior cingulate prediction error signals: r = -0.28; P = .005). Symptom improvement following CBT was associated with normalization of learning parameters that were disrupted at baseline (reward learning rate: posterior mean regression ß = 0.15; 90% CrI, 0.001 to 0.41; loss outcome shift: posterior mean regression ß = 0.42; 90% CrI, 0.09 to 0.77). Conclusions and Relevance: In this study, the mapping of reinforcement learning components to symptoms of major depression revealed mechanistic features associated with these symptoms and points to possible learning-based therapeutic processes and targets.

In Cocaine Dependence, Neural Prediction Errors During Loss Avoidance Are Increased With Cocaine Deprivation and Predict Drug Use.

BACKGROUND: In substance-dependent individuals, drug deprivation and drug use trigger divergent behavioral responses to environmental cues. These divergent responses are consonant with data showing that short- and long-term adaptations in dopamine signaling are similarly sensitive to state of drug use. The literature suggests a drug state-dependent role of learning in maintaining substance use; evidence linking dopamine to both reinforcement learning and addiction provides a framework to test this possibility. METHODS: In a randomized crossover design, 22 participants with current cocaine use disorder completed a probabilistic loss-learning task during functional magnetic resonance imaging while on and off cocaine (44 sessions). Another 54 participants without Axis I psychopathology served as a secondary reference group. Within-drug state and paired-subjects' learning effects were assessed with computational model-derived individual learning parameters. Model-based neuroimaging analyses evaluated effects of drug use state on neural learning signals. Relationships among model-derived behavioral learning rates (α+, α-), neural prediction error signals (δ+, δ-), cocaine use, and desire to use were assessed. RESULTS: During cocaine deprivation, cocaine-dependent individuals exhibited heightened positive learning rates (α+), heightened neural positive prediction error (δ+) responses, and heightened association of α+ with neural δ+ responses. The deprivation-enhanced neural learning signals were specific to successful loss avoidance, comparable to participants without psychiatric conditions, and mediated a relationship between chronicity of drug use and desire to use cocaine. CONCLUSIONS: Neurocomputational learning signals are sensitive to drug use status and suggest that heightened reinforcement by successful avoidance of negative outcomes may contribute to drug seeking during deprivation. More generally, attention to drug use state is important for delineating substrates of addiction.

Associability-modulated loss learning is increased in posttraumatic stress disorder.

Disproportionate reactions to unexpected stimuli in the environment are a cardinal symptom of posttraumatic stress disorder (PTSD). Here, we test whether these heightened responses are associated with disruptions in distinct components of reinforcement learning. Specifically, using functional neuroimaging, a loss-learning task, and a computational model-based approach, we assessed the mechanistic hypothesis that overreactions to stimuli in PTSD arise from anomalous gating of attention during learning (i.e., associability). Behavioral choices of combat-deployed veterans with and without PTSD were fit to a reinforcement learning model, generating trial-by-trial prediction errors (signaling unexpected outcomes) and associability values (signaling attention allocation to the unexpected outcomes). Neural substrates of associability value and behavioral parameter estimates of associability updating, but not prediction error, increased with PTSD during loss learning. Moreover, the interaction of PTSD severity with neural markers of associability value predicted behavioral choices. These results indicate that increased attention-based learning may underlie aspects of PTSD and suggest potential neuromechanistic treatment targets.

The neural bases of acquisitiveness: decisions to acquire and discard everyday goods differ across frames, items, and individuals.

The human tendency to acquire and keep large quantities of goods has become a serious concern, but has yet to be examined from a neuroscientific perspective. The mesolimbocortical system, particularly the orbitofrontal cortex (OFC) and nucleus accumbens (NAcc), is implicated when humans and animals acquire rewards. However, this may not extend to acquisitiveness per se, which involves fairly mundane items and is interconnected with a failure to discard. Moreover, the NAcc has not been implicated in neuroimaging studies of the extreme acquisitiveness of compulsive hoarders. In a study of the neural bases of normal acquisitiveness, subjects made decisions during functional neuroimaging to acquire or remove everyday items from a hypothetical collection, while maximizing personal preference or monetary profit. All decisions engaged the OFC, but the OFC and all regions of interest shifted in their relative involvement across the four decision contexts. The NAcc was only engaged during personal acquisition to the extent of problematic hoarding, suggesting that even common items can acquire an incentive salience that makes them hard to resist for acquisitive individuals. The types of items preferred also shifted with condition, with subjects only being biased toward expensive items when instructed to maximize profit. Item preferences even differed depending on whether participants were acquiring versus removing items, even though the task only differed superficially in the two conditions. Acquisitiveness reflects a complex mix of affective, cognitive, and personality factors that extend well beyond the drive to acquire valuable resources, with important implications for basic decision science, sustainability, and pathologies associated with compulsive acquisition.

Male carriers of the FMR1 premutation show altered hippocampal-prefrontal function during memory encoding.

Previous functional MRI (fMRI) studies have shown that fragile X mental retardation 1 (FMR1) fragile X premutation allele carriers (FXPCs) exhibit decreased hippocampal activation during a recall task and lower inferior frontal activation during a working memory task compared to matched controls. The molecular characteristics of FXPCs includes 55-200 CGG trinucleotide expansions, increased FMR1 mRNA levels, and decreased FMRP levels especially at higher repeat sizes. In the current study, we utilized MRI to examine differences in hippocampal volume and function during an encoding task in young male FXPCs. While no decreases in either hippocampal volume or hippocampal activity were observed during the encoding task in FXPCs, FMRP level (measured in blood) correlated with decreases in parahippocampal activation. In addition, activity in the right dorsolateral prefrontal cortex during correctly encoded trials correlated negatively with mRNA levels. These results, as well as the established biological effects associated with elevated mRNA levels and decreased FMRP levels on dendritic maturation and axonal growth, prompted us to explore functional connectivity between the hippocampus, prefrontal cortex, and parahippocampal gyrus using a psychophysiological interaction analysis. In FXPCs, the right hippocampus evinced significantly lower connectivity with right ventrolateral prefrontal cortex (VLPFC) and right parahippocampal gyrus. Furthermore, the weaker connectivity between the right hippocampus and VLPFC was associated with reduced FMRP in the FXPC group. These results suggest that while FXPCs show relatively typical brain response during encoding, faulty connectivity between frontal and hippocampal regions may have subsequent effects on recall and working memory.

Investigation of amygdala volume in men with the fragile X premutation.

Premutation fragile X carriers have a CGG repeat expansion (55 to 200 repeats) in the promoter region of the fragile X mental retardation 1 (FMR1) gene. Amygdala dysfunction has been observed in premutation symptomatology, and recent research has suggested the amygdala as an area susceptible to the molecular effects of the premutation. The current study utilizes structural magnetic resonance imaging (MRI) to examine the relationship between amygdala volume, CGG expansion size, FMR1 mRNA, and psychological symptoms in male premutation carriers without FXTAS compared with age and IQ matched controls. No significant between group differences in amygdala volume were found. However, a significant negative correlation between amygdala volume and CGG was found in the lower range of CGG repeat expansions, but not in the higher range of CGG repeat expansions.

Decreased fragile X mental retardation protein expression underlies amygdala dysfunction in carriers of the fragile X premutation.

BACKGROUND: The fragile X premutation provides a unique opportunity for the study of genetic and brain mechanisms of behavior and cognition in the context of neurodevelopment and neurodegeneration. Although the neurodegenerative phenotype, fragile X-associated tremor/ataxia syndrome, is well described, evidence of a causal link between the premutation and psychiatric disorder earlier in life, clear delineation of a behavioral/cognitive phenotype, and characterization of the physiological basis of observed symptoms have been elusive. METHODS: We completed functional magnetic resonance imaging targeting the amygdala with an emotion-matching task and concurrent infrared eye tracking, FMR1 molecular genetic testing, and neuropsychological assessment in 23 men with the premutation (mean age = 32.9 years) and 25 male control subjects (mean age = 30.1 years). RESULTS: Premutation carriers had significantly smaller left and right amygdala volume and reduced right amygdala activation during the task relative to control subjects. Although both elevated FMR1 messenger RNA and reduced fragile X mental retardation protein (FMRP) were associated with the reduced activation, multiple regression analysis suggested that reduced FMRP is the primary factor. Premutation carriers also had higher ratings of autism spectrum symptoms than control subjects, which were associated with the reduced amygdala response. CONCLUSIONS: Although prior studies have emphasized a toxic gain-of-function effect of elevated messenger RNA associated with the premutation, the current results point to the role of reduced FMRP in alterations of brain activity and behavior.