Using Computational Phenotyping to Identify Divergent Strategies for Effort Allocation Across the Psychosis Spectrum.

BACKGROUND AND HYPOTHESIS: Disturbances in effort-cost decision-making have been highlighted as a potential transdiagnostic process underpinning negative symptoms in individuals with schizophrenia. However, recent studies using computational phenotyping show that individuals employ a range of strategies to allocate effort, and use of different strategies is associated with unique clinical and cognitive characteristics. Building on prior work in schizophrenia, this study evaluated whether effort allocation strategies differed in individuals with distinct psychotic disorders. STUDY DESIGN: We applied computational modeling to effort-cost decision-making data obtained from individuals with psychotic disorders (n = 190) who performed the Effort Expenditure for Rewards Task. The sample included 91 individuals with schizophrenia/schizoaffective disorder, 90 individuals with psychotic bipolar disorder, and 52 controls. STUDY RESULTS: Different effort allocation strategies were observed both across and within different disorders. Relative to individuals with psychotic bipolar disorder, a greater proportion of individuals with schizophrenia/schizoaffective disorder did not use reward value or probability information to guide effort allocation. Furthermore, across disorders, different effort allocation strategies were associated with specific clinical and cognitive features. Those who did not use reward value or probability information to guide effort allocation had more severe positive and negative symptoms, and poorer cognitive and community functioning. In contrast, those who only used reward value information showed a trend toward more severe positive symptoms. CONCLUSIONS: These findings indicate that similar deficits in effort-cost decision-making may arise from different computational mechanisms across the psychosis spectrum.

Hormone sensitivity predicts the beneficial effects of transdermal estradiol on reward-seeking behaviors in perimenopausal women: A randomized controlled trial.

Depression is highly prevalent during the menopause transition (perimenopause), and often presents with anxious and anhedonic features. This increased vulnerability for mood symptoms is likely driven in part by the dramatic hormonal changes that are characteristic of the menopause transition, as prior research has linked fluctuations in estradiol (E2) to emergence of depressed mood in at risk perimenopausal women. Transdermal estradiol (TE2) has been shown to reduce the severity of depression in clinically symptomatic women, particularly in those with recent stressful life events. This research extends prior work by examining the relation between E2 and reward seeking behaviors, a precise behavioral indicator of depression. Specifically, the current study utilizes a randomized, double blind, placebo-controlled design to investigate whether mood sensitivity to E2 flux ("hormone sensitivity") predicts the beneficial effects of TE2 interventions on reward seeking behaviors in perimenopausal women, and whether recent stressful life events moderate any observed associations. METHOD: Participants were 66 women who met standardized criteria for being early or late perimenopausal based on bleeding patterns. Participants were recruited from a community sample; therefore, mood symptoms varied across the continuum and the majority of participants did not meet diagnostic criteria for a depressive or anxiety disorder at the time of enrollment. Hormone sensitivity was quantified over an 8-week baseline period, using within-subjects correlations between repeated weekly measures of E2 serum concentrations and weekly anxiety (State Trait Anxiety Inventory) and anhedonia ratings (Snaith-Hamilton Pleasure Scale). Women were then randomized to receive 8 weeks of TE2 (0.1 mg) or transdermal placebo, and reward-seeking behaviors were assessed using the Effort-Expenditure for Rewards Task (EEfRT). RESULTS: Participants who were randomized to receive transdermal estradiol and who demonstrated greater anxiety sensitivity to E2 fluctuations at baseline, demonstrated more reward seeking behaviors on the EEfRT task. Notably, the strength of the association between E2-anxiety sensitivity and post-randomization EEfRT for TE2 participants increased when women experienced more recent stressful life events and rated those events as more stressful. E2-anhedonia sensitivity was not associated with reward-seeking behaviors. CONCLUSION: Perimenopausal women who are more sensitive to E2 fluctuations and experienced more recent life stress may experience a greater benefit of TE2 as evidenced by an increase in reward seeking behaviors.

Effects of acute stress and depression on functional connectivity between prefrontal cortex and the amygdala.

Stress is known to be a significant risk factor for the development of Major Depressive Disorder (MDD), yet the neural mechanisms that underlie this risk are poorly understood. Prior work has heavily implicated the corticolimbic system in the pathophysiology of MDD. In particular, the prefrontal cortex (PFC) and amygdala play a central role in regulating the response to stress, with dorsal PFC and ventral PFC exhibiting reciprocal excitatory and inhibitory influences on amygdala subregions. However, it remains unclear how best to disentangle the impact of stress from the impact of current MDD symptoms on this system. Here, we examined stress-induced changes in resting state functional connectivity (rsFC) within an a priori corticolimbic network in MDD patients and healthy controls (total n = 80) before and after an acute stressor or a "no stress" control condition. Using graph theoretic analysis, we found that connectivity between basolateral amygdala and dorsal prefrontal nodes of the corticolimbic network had a negative association with individual differences in chronic perceived stress at baseline. Following the acute stressor, healthy individuals showed a reduction of the amygdala node strength, while MDD patients exhibited little change. Finally, dorsal PFC-particularly dorsomedial PFC- connectivity to the basolateral amygdala was associated with the strength of the basolateral amygdala responses to loss feedback during a reinforcement learning task. These findings highlight attenuated connectivity between basolateral amygdala and prefrontal cortex in patients with MDD. In healthy individuals, acute stress exposure was found to push the corticolimbic network to a "stress-phenotype" that may be chronically present in patients with current depression and high levels of perceived stress. In sum, these results help to identify circuit mechanisms underlying the effects of acute stress and their role in mood disorders.

Age-related differences in the social associative learning of trust information.

Trust is a key component of social interaction. Older adults, however, often exhibit excessive trust relative to younger adults. One explanation is that older adults may learn to trust differently than younger adults. Here, we examine how younger (N = 33) and older adults (N = 30) learn to trust over time. Participants completed a classic iterative trust game with 3 partners. Younger and older adults shared similar amounts but differed in how they shared money. Compared to younger adults, older adults invested more with untrustworthy partners and less with trustworthy partners. As a group, older adults displayed less learning than younger adults. However, computational modeling suggests that this is not because older adults learn differently from positive and negative feedback than younger adults. Model-based fMRI analyses revealed several age- and learning-related differences in neural processing. Specifically, we found that older learners (N = 19), relative to older non-learners (N = 11), had greater reputation-related activity in metalizing/memory areas while making their decisions. Collectively, these findings suggest that older adult learners use social cues differently from non-learners.

Acute drug effects differentially predict desire to take dextroamphetamine again for work and recreation.

RATIONALE: Misuse of dextroamphetamine occurs in work and recreational contexts. While acute drug effects broadly predict abuse liability, few studies have considered the relationship between acute effects and context. OBJECTIVES: This study examined how individual differences in acute effects of dextroamphetamine relate to desire to take dextroamphetamine again in different contexts. METHODS: This secondary analysis used data from healthy adults with no history of moderate-to-severe substance use disorder, who received oral doses of placebo and dextroamphetamine (10 and 20 mg) over 3 sessions under double-blind, randomized conditions. Subjects rated subjective effects and completed reward-related behavioral tasks. Subjects rated their desire to take dextroamphetamine again in hypothetical work and recreational contexts. Multilevel models examined within-subjects change scores (10 mg-placebo; 20 mg-placebo) to determine how subjective effects and behavioral outcomes predicted desire to take dextroamphetamine again for work versus recreation. RESULTS: Subjects reported more desire to take 20 mg dextroamphetamine again for work than for recreation. At 20 mg, there was an interaction between context and liking/wanting, such that liking/wanting predicted desire to use dextroamphetamine for work only. There was also an interaction at 20 mg between context and psychomotor speed, such that psychomotor speed predicted interest in using dextroamphetamine for recreation only. CONCLUSIONS: We found that positive subjective effects predicted desire to use dextroamphetamine again for work, while increased motor effects predicted desire to use dextroamphetamine recreationally. Hedonic effects may be perceived as advantageous when working, while increased physical energy may be preferred during recreation, suggesting that context of intended use is important when examining abuse liability.

Reduced adaptation of glutamatergic stress response is associated with pessimistic expectations in depression.

Stress is a significant risk factor for the development of major depressive disorder (MDD), yet the underlying mechanisms remain unclear. Preclinically, adaptive and maladaptive stress-induced changes in glutamatergic function have been observed in the medial prefrontal cortex (mPFC). Here, we examine stress-induced changes in human mPFC glutamate using magnetic resonance spectroscopy (MRS) in two healthy control samples and a third sample of unmedicated participants with MDD who completed the Maastricht acute stress task, and one sample of healthy control participants who completed a no-stress control manipulation. In healthy controls, we find that the magnitude of mPFC glutamate response to the acute stressor decreases as individual levels of perceived stress increase. This adaptative glutamate response is absent in individuals with MDD and is associated with pessimistic expectations during a 1-month follow-up period. Together, this work shows evidence for glutamatergic adaptation to stress that is significantly disrupted in MDD.

Dose-response effects of d-amphetamine on effort-based decision-making and reinforcement learning.

Effort-related decision-making and reward learning are both dopamine-dependent, but preclinical research suggests they depend on different dopamine signaling dynamics. Therefore, the same dose of a dopaminergic medication could have differential effects on effort for reward vs. reward learning. However, no study has tested how effort and reward learning respond to the same dopaminergic medication within subjects. The current study aimed to test the effect of therapeutic doses of d-amphetamine on effort for reward and reward learning in the same healthy volunteers. Participants (n = 30) completed the Effort Expenditure for Reward Task (EEfRT) measure of effort-related decision-making, and the Probabilistic Reward Task (PRT) measure of reward learning, under placebo and two doses of d-amphetamine (10 mg, and 20 mg). Secondarily, we examined whether the individual characteristics of baseline working memory and willingness to exert effort for reward moderated the effects of d-amphetamine. d-Amphetamine increased willingness to exert effort, particularly at low to intermediate expected values of reward. Computational modeling analyses suggested this was due to decreased effort discounting rather than probability discounting or decision consistency. Both baseline effort and working memory emerged as moderators of this effect, such that d-amphetamine increased effort more in individuals with lower working memory and lower baseline effort, also primarily at low to intermediate expected values of reward. In contrast, d-amphetamine had no significant effect on reward learning. These results have implications for treatment of neuropsychiatric disorders, which may be characterized by multiple underlying reward dysfunctions.

Distinct regions of the striatum underlying effort, movement initiation and effort discounting.

The ventral striatum is believed to encode the subjective value of cost-benefit options; however, this effect has notably been absent during choices that involve physical effort. Previous work in freely moving animals has revealed opposing striatal signals, with greater response to increasing effort demands and reduced responses to rewards requiring effort. Yet, the relationship between these conflicting signals remains unknown. Using functional magnetic resonance imaging with a naturalistic maze-navigation paradigm, we identified functionally segregated regions within the ventral striatum that separately encoded effort activation, movement initiation and effort discounting of rewards. In addition, activity in regions associated with effort activation and discounting oppositely predicted striatal encoding of effort during effort-based decision-making. Our results suggest that the dorsomedial region hitherto associated with action may instead represent the cost of effort and raise fundamental questions regarding the interpretation of striatal 'reward' signals in the context of effort demands. This has implications for uncovering the neural architecture underlying motivated behaviour.

Effortful goal-directed behavior in schizophrenia: Computational subtypes and associations with cognition.

Schizophrenia is associated with amotivation and reduced goal-directed behavior, which have been linked to poor functional outcomes. Motivational deficits in schizophrenia are often measured using effort-based decision-making (EBDM) paradigms, revealing consistent alterations in effort expenditure relative to controls. Although these results have generally been interpreted in terms of decreased motivation, the ability to use trial-by-trial changes in reward magnitude or probability of receipt to guide effort allocation may also be affected by cognitive deficits. To date, it remains unclear whether altered performance in EBDM primarily reflects deficits in motivation, cognitive functioning, or both. We applied a newly developed computational modeling approach to the analysis of EBDM data from two previously collected samples comprising 153 patients and 105 controls to determine the extent to which individuals did or did not use available information about reward and probability to guide effort allocation. Half of the participants with schizophrenia failed to incorporate information about reward and probability when making effort-expenditure decisions. The subset of patients who exhibited difficulties using reward and probability information were characterized by greater impairments across measures of cognitive functioning. Interestingly, even within the subset of patients who successfully used reward and probability information to guide effort expenditure, higher levels of negative symptoms related to motivation and avolition were associated with greater effort aversion during the task. Taken together, these data suggest that prior reports of aberrant EBDM in schizophrenia patients are related to both cognitive function and individual differences in negative symptoms. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Can't or Won't? Immunometabolic Constraints on Dopaminergic Drive.

Inflammatory cytokines have been shown to have a direct effect on mesolimbic dopamine (DA) that is associated with a reduced willingness to expend effort for reward. To date, however, the broader implications of this communication between inflammation and mesolimbic DA have yet to be explored. Here, we suggest that the metabolic demands of chronic low-grade inflammation induce a reduction of striatal DA that in turn leads to a steeper effort-discounting curve because of reduced perceived ability (can't) versus preference (won't) for reward. This theoretical framework can inform how the mesolimbic DA system responds to increased immunometabolic demands during chronic inflammation, ultimately contributing to motivational impairments in psychiatric and other medical disorders.

Corticoinsular circuits encode subjective value expectation and violation for effortful goal-directed behavior.

We are presented with choices each day about how to invest our effort to achieve our goals. Critically, these decisions must frequently be made under conditions of incomplete information, where either the effort required or possible reward to be gained is uncertain. Such choices therefore require the development of potential value estimates to guide effortful goal-directed behavior. To date, however, the neural mechanisms for this expectation process are unknown. Here, we used computational fMRI during an effort-based decision-making task where trial-wise information about effort costs and reward magnitudes was presented separately over time, thereby allowing us to model distinct effort/reward computations as choice-relevant information unfolded. We found that ventromedial prefrontal cortex (vmPFC) encoded expected subjective value. Further, activity in dorsal anterior cingulate (dACC) and anterior insula (aI) reflected both effort discounting as well as a subjective value prediction error signal derived from trial history. While prior studies have identified these regions as being involved in effort-based decision making, these data demonstrate their specific role in the formation and maintenance of subjective value estimates as relevant information becomes available.

Anhedonia in depression: biological mechanisms and computational models.

Anhedonia is a severe condition that describes a near-complete absence of enjoyment, motivation, and interest. A core feature of depression, clinical manifestations of anhedonia can include deficits in experiencing pleasure, approach-related motivated behavior, and learning how to match expectations to the environment. To date, the precise neurobiological mechanisms of anhedonia in major depression are still poorly understood. We have previously argued that contradictory findings and the inability to identify specific neurobiological substrates for anhedonic symptoms may result from sample heterogeneity, suboptimal methods of assessment, and the challenge of dissociating between different components of anhedonia. Recently, however, computational advances to the operationalization of psychiatric symptoms have enhanced the ability to evaluate the neurobiology of constituent elements of this symptom domain. In this paper, we review (1) advances in behavioral and computational methods of assessing reward processing and motivation and (2) the development of new self-report, neurological, and biological methods of subtyping that may be useful in future pursuits to expand our understanding of the neurobiology of anhedonia in depression.

Association Between Interleukin-6 and Striatal Prediction-Error Signals Following Acute Stress in Healthy Female Participants.

BACKGROUND: Stress is widely known to alter behavioral responses to rewards and punishments. It is believed that stress may precipitate these changes through modulation of corticostriatal circuitry involved in reinforcement learning and motivation, although the intervening mechanisms remain unclear. One candidate is inflammation, which can rapidly increase following stress and can disrupt dopamine-dependent reward pathways. METHODS: Here, in a sample of 88 healthy female participants, we first assessed the effect of an acute laboratory stress paradigm on levels of plasma interleukin-6 (IL-6), a cytokine known to be both responsive to stress and elevated in depression. In a second laboratory session, we examined the effects of a second laboratory stress paradigm on reward prediction error (RPE) signaling in the ventral striatum. RESULTS: We show that individual differences in stress-induced increases in IL-6 (session 1) were associated with decreased ventral striatal RPE signaling during reinforcement learning (session 2), though there was no main effect of stress on RPE. Furthermore, changes in IL-6 following stress predicted intraindividual variability in perceived stress during a 4-month follow-up period. CONCLUSIONS: Taken together, these data identify a novel link between IL-6 and striatal RPEs during reinforcement learning in the context of acute psychological stress, as well as future appraisal of stressful life events.

Framing matters: Effects of framing on older adults' exploratory decision-making.

We examined framing effects on exploratory decision-making. In Experiment 1 we tested older and younger adults in two decision-making tasks separated by one week, finding that older adults' decision-making performance was preserved when maximizing gains, but it declined when minimizing losses. Computational modeling indicates that younger adults in both conditions, and older adults in gains maximization, utilized a decreasing threshold strategy (which is optimal), but older adults in losses were better fit by a fixed-probability model of exploration. In Experiment 2 we examined within-subject behavior in older and younger adults in the same exploratory decision-making task, but without a time separation between tasks. We replicated the older adult disadvantage in loss minimization from Experiment 1 and found that the older adult deficit was significantly reduced when the loss-minimization task immediately followed the gains-maximization task. We conclude that older adults' performance in exploratory decision-making is hindered when framed as loss minimization, but that this deficit is attenuated when older adults can first develop a strategy in a gains-framed task. (PsycINFO Database Record

Neural correlates of state-based decision-making in younger and older adults.

Older and younger adults performed a state-based decision-making task while undergoing functional MRI (fMRI). We proposed that younger adults would be more prone to base their decisions on expected value comparisons, but that older adults would be more reactive decision-makers who would act in response to recent changes in rewards or states, rather than on a comparison of expected values. To test this we regressed BOLD activation on two measures from a sophisticated reinforcement learning (RL) model. A value-based regressor was computed by subtracting the immediate value of the selected alternative from its long-term value. The other regressor was a state-change uncertainty signal that served as a proxy for whether the participant's state improved or declined, relative to the previous trial. Younger adults' activation was modulated by the value-based regressor in ventral striatal and medial PFC regions implicated in reinforcement learning. Older adults' activation was modulated by state-change uncertainty signals in right dorsolateral PFC, and activation in this region was associated with improved performance in the task. This suggests that older adults may depart from standard expected-value based strategies and recruit lateral PFC regions to engage in reactive decision-making strategies.

Information about foregone rewards impedes dynamic decision-making in older adults.

"Making an informed decision" implies that more information leads to better decisions, yet it may be the case that additional information biases decisions in a systematic and sometimes detrimental manner. In the present study, we examined the effect of additional information on older adults' decision-making using a task for which available rewards were dependent on the participant's recent pattern of choices. The optimal strategy was to forego the immediately rewarding option in favor of the option that yielded larger delayed reward. We found that providing information about true foregone rewards - the reward that would have been received had the participant chosen the other option - significantly reduced older adults' decision-making performance. However, false foregone rewards - foregone rewards manufactured to make the long-term option appear more immediately rewarding - led older adults to perform at a level equal to younger adults. We conclude that providing information about foregone rewards biases older adults toward immediate rewards at a greater rate than younger adults, leading to poorer older adult performance when immediate rewards and long-term rewards conflict, but intact performance when immediate rewards and long-term rewards appear to align.

Chronic motivational state interacts with task reward structure in dynamic decision-making.

Research distinguishes between a habitual, model-free system motivated toward immediately rewarding actions, and a goal-directed, model-based system motivated toward actions that improve future state. We examined the balance of processing in these two systems during state-based decision-making. We tested a regulatory fit hypothesis (Maddox & Markman, 2010) that predicts that global trait motivation affects the balance of habitual- vs. goal-directed processing but only through its interaction with the task framing as gain-maximization or loss-minimization. We found support for the hypothesis that a match between an individual's chronic motivational state and the task framing enhances goal-directed processing, and thus state-based decision-making. Specifically, chronic promotion-focused individuals under gain-maximization and chronic prevention-focused individuals under loss-minimization both showed enhanced state-based decision-making. Computational modeling indicates that individuals in a match between global chronic motivational state and local task reward structure engaged more goal-directed processing, whereas those in a mismatch engaged more habitual processing.

The influence of expertise on essence beliefs for mental and medical disorder categories.

Research suggests that expertise in a specific category domain influences categorization. Work related to beliefs about mental disorders finds that laypeople treat mental disorders as if they do have causal essences, while clinicians do not-differences that may be attributable to expertise (Ahn, Flanagan, Marsh, & Sanislow, 2006). To test whether reduced beliefs in essences are indicative of an overall influence of expertise or a demonstration of a phenomenon specific to expertise in the mental health domain we compared beliefs about mental and medical disorders held by practicing physicians (n = 43; 19 primary care and 24 non-psychiatry specialists) and laypeople (n = 40). We found differences between these groups in beliefs held concerning the necessity of removing shared category features to effectively cure disorders. While laypeople endorsed the idea that the cause needed to be removed to cure both mental and medical disorders, this endorsement decreased with expertise. Primary care providers were less willing to endorse this for mental disorders than for medical disorders. Our results support the notion that the reduction of beliefs concerning the existence of essences is a unique effect of expertise in the mental health domain, and does not extend to other areas of expertise. In physicians, this reduction of essentialist beliefs was most evident in questions regarding treatment. Similarities and differences to the results from Ahn et al. (2006) are discussed.

A frontal dopamine system for reflective exploratory behavior.

The COMT gene modulates dopamine levels in prefrontal cortex with Met allele carriers having lower COMT enzyme activity and, therefore, higher dopamine levels compared to Val/Val homozygotes. Concordantly, Val/Val homozygotes tend to perform worse and display increased (interpreted as inefficient) frontal activation in certain cognitive tasks. In a sample of 209 participants, we test the hypothesis that Met carriers will be advantaged in a decision-making task that demands sequencing exploratory and exploitive choices to minimize uncertainty about the reward structure in the environment. Previous work suggests that optimal performance depends on limited cognitive resources supported by prefrontal systems. If so, Met carriers should outperform Val/Val homozygotes, particularly under dual-task conditions that tax limited cognitive resources. In accord with these a priori predictions, Met carriers were more resilient in the face of cognitive load, continuing to explore in a sophisticated manner. We fit computational models that embody sophisticated reflective and simple reflexive strategies to further evaluate participants' exploration behavior. The Ideal Actor model reflectively updates beliefs and plans ahead, taking into account the information gained by each choice and making choices that maximize long-term payoffs. In contrast, the Naïve Reinforcement Learning (RL) model instantiates the reflexive account of choice, in which the values of actions are based only on the rewards experienced so far. Its beliefs are updated reflexively in response to observed changes in rewards. Converging with standard analyses, Met carriers were best characterized by the Ideal Actor model, whereas Val/Val homozygotes were best characterized by the Naive RL model, particularly under dual-task conditions.

State-based versus reward-based motivation in younger and older adults.

Recent decision-making work has focused on a distinction between a habitual, model-free neural system that is motivated toward actions that lead directly to reward and a more computationally demanding goal-directed, model-based system that is motivated toward actions that improve one's future state. In this article, we examine how aging affects motivation toward reward-based versus state-based decision making. Participants performed tasks in which one type of option provided larger immediate rewards but the alternative type of option led to larger rewards on future trials, or improvements in state. We predicted that older adults would show a reduced preference for choices that led to improvements in state and a greater preference for choices that maximized immediate reward. We also predicted that fits from a hybrid reinforcement-learning model would indicate greater model-based strategy use in younger than in older adults. In line with these predictions, older adults selected the options that maximized reward more often than did younger adults in three of the four tasks, and modeling results suggested reduced model-based strategy use. In the task where older adults showed similar behavior to younger adults, our model-fitting results suggested that this was due to the utilization of a win-stay-lose-shift heuristic rather than a more complex model-based strategy. Additionally, within older adults, we found that model-based strategy use was positively correlated with memory measures from our neuropsychological test battery. We suggest that this shift from state-based to reward-based motivation may be due to age related declines in the neural structures needed for more computationally demanding model-based decision making.