Neurocomputational mechanisms involved in adaptation to fluctuating intentions of others.

Humans frequently interact with agents whose intentions can fluctuate between competition and cooperation over time. It is unclear how the brain adapts to fluctuating intentions of others when the nature of the interactions (to cooperate or compete) is not explicitly and truthfully signaled. Here, we use model-based fMRI and a task in which participants thought they were playing with another player. In fact, they played with an algorithm that alternated without signaling between cooperative and competitive strategies. We show that a neurocomputational mechanism with arbitration between competitive and cooperative experts outperforms other learning models in predicting choice behavior. At the brain level, the fMRI results show that the ventral striatum and ventromedial prefrontal cortex track the difference of reliability between these experts. When attributing competitive intentions, we find increased coupling between these regions and a network that distinguishes prediction errors related to competition and cooperation. These findings provide a neurocomputational account of how the brain arbitrates dynamically between cooperative and competitive intentions when making adaptive social decisions.

Chronic Ethanol Exposure Produces Persistent Impairment in Cognitive Flexibility and Decision Signals in the Striatum.

Lack of cognitive flexibility is a hallmark of substance use disorders and has been associated with drug-induced synaptic plasticity in the dorsomedial striatum (DMS). Yet the possible impact of altered plasticity on real-time striatal neural dynamics during decision-making is unclear. Here, we identified persistent impairments induced by chronic ethanol (EtOH) exposure on cognitive flexibility and striatal decision signals. After a substantial withdrawal period from prior EtOH vapor exposure, male, but not female, rats exhibited reduced adaptability and exploratory behavior during a dynamic decision-making task. Reinforcement learning models showed that prior EtOH exposure enhanced learning from rewards over omissions. Notably, neural signals in the DMS related to the decision outcome were enhanced, while those related to choice and choice-outcome conjunction were reduced, in EtOH-treated rats compared to the controls. These findings highlight the profound impact of chronic EtOH exposure on adaptive decision-making, pinpointing specific changes in striatal representations of actions and outcomes as underlying mechanisms for cognitive deficits.

Foraging Under Uncertainty Follows the Marginal Value Theorem with Bayesian Updating of Environment Representations.

Foraging theory has been a remarkably successful approach to understanding the behavior of animals in many contexts. In patch-based foraging contexts, the marginal value theorem (MVT) shows that the optimal strategy is to leave a patch when the marginal rate of return declines to the average for the environment. However, the MVT is only valid in deterministic environments whose statistics are known to the forager; naturalistic environments seldom meet these strict requirements. As a result, the strategies used by foragers in naturalistic environments must be empirically investigated. We developed a novel behavioral task and a corresponding computational framework for studying patch-leaving decisions in head-fixed and freely moving mice. We varied between-patch travel time, as well as within-patch reward depletion rate, both deterministically and stochastically. We found that mice adopt patch residence times in a manner consistent with the MVT and not explainable by simple ethologically motivated heuristic strategies. Critically, behavior was best accounted for by a modified form of the MVT wherein environment representations were updated based on local variations in reward timing, captured by a Bayesian estimator and dynamic prior. Thus, we show that mice can strategically attend to, learn from, and exploit task structure on multiple timescales simultaneously, thereby efficiently foraging in volatile environments. The results provide a foundation for applying the systems neuroscience toolkit in freely moving and head-fixed mice to understand the neural basis of foraging under uncertainty.

Efficient production of (S)-limonene and geraniol in Saccharomyces cerevisiae through the utilization of an Erg20 mutant with enhanced GPP accumulation capability.

Monoterpenes and monoterpenoids such as (S)-limonene and geraniol are valuable chemicals with a wide range of applications, including cosmetics, pharmaceuticals, and biofuels. Saccharomyces cerevisiae has proven to be an effective host to produce various terpenes and terpenoids. (S)-limonene and geraniol are produced from geranyl pyrophosphate (GPP) through the enzymatic actions of limonene synthase (LS) and geraniol synthase (GES), respectively. However, a major hurdle in their production arises from the dual functionality of the Erg20, a farnesyl pyrophosphate (FPP) synthase, responsible for generating GPP. Erg20 not only synthesizes GPP by condensing isopentenyl pyrophosphate (IPP) with dimethylallyl pyrophosphate but also catalyzes further condensation of IPP with GPP to produce FPP. In this study, we have tackled this issue by harnessing previously developed Erg20 mutants, Erg20K197G (Erg20G) and Erg20F96W, N127W (Erg20WW), which enhance GPP accumulation. Through a combination of these mutants, we generated a novel Erg20WWG mutant with over four times higher GPP accumulating capability than Erg20WW, as observed through geraniol production levels. The Erg20WWG mutant was fused to the LS from Mentha spicata or the GES from Catharanthus roseus for efficient conversion of GPP to (S)-limonene and geraniol, respectively. Further improvements were achieved by localizing the entire mevalonate pathway and the Erg20WWG-fused enzymes in peroxisomes, while simultaneously downregulating the essential ERG20 gene using the glucose-sensing HXT1 promoter. In the case of (S)-limonene production, additional Erg20WWG-LS was expressed in the cytosol. As a result, the final strains produced 1063 mg/L of (S)-limonene and 1234 mg/L of geraniol by fed-batch biphasic fermentations with ethanol feeding. The newly identified Erg20WWG mutant opens doors for the efficient production of various other GPP-derived chemicals including monoterpene derivatives and cannabinoids.

Efficient production of mycosporine-like amino acids, natural sunscreens, in Yarrowia lipolytica.

BACKGROUND: Mycosporine-like amino acids (MAAs), including shinorine and porphyra-334, are gaining attention as safe natural sunscreens. The production of MAAs has been achieved in diverse microbial hosts, including Saccharomyces cerevisiae. While S. cerevisiae is the most extensively studied model yeast, the oleaginous yeast Yarrowia lipolytica has emerged as a promising candidate for the synthesis of valuable products. In this study, we explored the potential of Y. lipolytica as a host for producing MAAs, utilizing its advantages such as a robust pentose phosphate pathway flux and versatile carbon source utilization. RESULTS: We produced MAAs in Y. lipolytica by introducing the MAA biosynthetic genes from cyanobacteria Nostoc punctiforme and Anabaena variabilis. These genes include mysA, mysB, and mysC responsible for producing mycosporine-glycine (MG) from sedoheptulose 7-phosphate (S7P). The two strains utilize different enzymes, D-Ala-D-Ala ligase homologue (MysD) in N. punctiforme and NRPS-like enzyme (MysE) in A. variabilis, for amino acid conjugation to MG. MysE specifically generated shinorine, a serine conjugate of MG, while MysD exhibited substrate promiscuity, yielding both shinorine and a small amount of porphyra-334, a threonine conjugate of MG. We enhanced MAAs production by selecting mysA, mysB, and mysC from A. variabilis and mysD from N. punctiforme based on their activities. We further improved production by strengthening promoters, increasing gene copies, and introducing the xylose utilization pathway. Co-utilization of xylose with glucose or glycerol increased MAAs production by boosting the S7P pool through the pentose phosphate pathway. Overexpressing GND1 and ZWF1, key genes in the pentose phosphate pathway, further enhanced MAAs production. The highest achieved MAAs level was 249.0 mg/L (207.4 mg/L shinorine and 41.6 mg/L of porphyra-334) in YP medium containing 10 g/L glucose and 10 g/L xylose. CONCLUSIONS: Y. lipolytica was successfully engineered to produce MAAs, primarily shinorine. This achievement involved the introduction of MAA biosynthetic genes from cyanobacteria, establishing xylose utilizing pathway, and overexpressing the pentose phosphate pathway genes. These results highlight the potential of Y. lipolytica as a promising yeast chassis strain for MAAs production, notably attributed to its proficient expression of MysE enzyme, which remains non-functional in S. cerevisiae, and versatile utilization of carbon sources like glycerol.

Efficient production of natural sunscreens shinorine, porphyra-334, and mycosporine-2-glycine in Saccharomyces cerevisiae.

Mycosporine-like amino acids (MAAs) are promising natural sunscreens mainly produced in marine organisms. Until now, metabolic engineering efforts to produce MAAs in heterologous hosts have mainly focused on shinorine production, and the low production levels are still not suitable for industrial applications. In this study, we successfully developed Saccharomyces cerevisiae strains that can efficiently produce various disubstituted MAAs, including shinorine, porphyra-334, and mycosporine-2-glycine (M2G), which are formed by conjugating serine, threonine, and glycine to mycosporine-glycine (MG), respectively. We first generated an MG-producing strain by multiple integration of the biosynthetic genes from cyanobacteria and applying metabolic engineering strategies to increase sedoheptulose-7-phosphate pool, a substrate for MG production. Next, five mysD genes from cyanobacteria, which encode D-Ala-D-Ala ligase homologues that conjugate an amino acid to MG, were introduced into the MG-producing strain to determine the substrate preference of each MysD enzyme. MysDs from Lyngbya sp., Nostoclinckia, and Euhalothece sp. showed high specificity toward serine, threonine, and glycine, resulting in efficient production of shinorine, porphyra-334, and M2G, respectively. This is the first report on the production of porphyra-334 and M2G in S. cerevisiae. Furthermore, we identified that the substrate specificity of MysD was determined by the omega loop region of 43-45 amino acids predicted based on its structural homology to a D-Ala-D-Ala ligase from Thermus thermophilus involved in peptidoglycan biosynthesis. The substrate specificities of two MysD enzymes were interchangeable by swapping the omega loop region. Using the engineered strain expressing mysD from Lyngbya sp. or N. linckia, up to 1.53 g/L shinorine or 1.21 g/L porphyra-334 was produced by fed-batch fermentation in a 5-L bioreactor, the highest titer reported so far. These results suggest that S. cerevisiae is a promising host for industrial production of different types of MAAs, providing a sustainable and eco-friendly alternative for the development of natural sunscreens.

Functional brain networks reflect spatial and temporal autocorrelation.

High-throughput experimental methods in neuroscience have led to an explosion of techniques for measuring complex interactions and multi-dimensional patterns. However, whether sophisticated measures of emergent phenomena can be traced back to simpler, low-dimensional statistics is largely unknown. To explore this question, we examined resting-state functional magnetic resonance imaging (rs-fMRI) data using complex topology measures from network neuroscience. Here we show that spatial and temporal autocorrelation are reliable statistics that explain numerous measures of network topology. Surrogate time series with subject-matched spatial and temporal autocorrelation capture nearly all reliable individual and regional variation in these topology measures. Network topology changes during aging are driven by spatial autocorrelation, and multiple serotonergic drugs causally induce the same topographic change in temporal autocorrelation. This reductionistic interpretation of widely used complexity measures may help link them to neurobiology.

Coordinated anatomical and functional variability in the human brain during adolescence.

Adolescence represents a time of unparalleled brain development. In particular, developmental changes in morphometric and cytoarchitectural features are accompanied by maturation in the functional connectivity (FC). Here, we examined how three facets of the brain, including myelination, cortical thickness (CT), and resting-state FC, interact in children between the ages of 10 and 15. We investigated the pattern of coordination in these measures by computing correlation matrices for each measure as well as meta-correlations among them both at the regional and network levels. The results revealed consistently higher meta-correlations among myelin, CT, and FC in the sensory-motor cortical areas than in the association cortical areas. We also found that these meta-correlations were stable and little affected by age-related changes in each measure. In addition, regional variations in the meta-correlations were consistent with the previously identified gradient in the FC and therefore reflected the hierarchy of cortical information processing, and this relationship persists in the adult brain. These results demonstrate that heterogeneity in FC among multiple cortical areas are closely coordinated with the development of cortical myelination and thickness during adolescence.

Transient neuronal suppression for exploitation of new sensory evidence.

In noisy but stationary environments, decisions should be based on the temporal integration of sequentially sampled evidence. This strategy has been supported by many behavioral studies and is qualitatively consistent with neural activity in multiple brain areas. By contrast, decision-making in the face of non-stationary sensory evidence remains poorly understood. Here, we trained monkeys to identify and respond via saccade to the dominant color of a dynamically refreshed bicolor patch that becomes informative after a variable delay. Animals' behavioral responses were briefly suppressed after evidence changes, and many neurons in the frontal eye field displayed a corresponding dip in activity at this time, similar to that frequently observed after stimulus onset but sensitive to stimulus strength. Generalized drift-diffusion models revealed consistency of behavior and neural activity with brief suppression of motor output, but not with pausing or resetting of evidence accumulation. These results suggest that momentary arrest of motor preparation is important for dynamic perceptual decision making.

Reinforcement learning detuned in addiction: integrative and translational approaches.

Suboptimal decision-making strategies have been proposed to contribute to the pathophysiology of addiction. Decision-making, however, arises from a collection of computational components that can independently influence behavior. Disruptions in these different components can lead to decision-making deficits that appear similar behaviorally, but differ at the computational, and likely the neurobiological, level. Here, we discuss recent studies that have used computational approaches to investigate the decision-making processes underlying addiction. Studies in animal models have found that value updating following positive, but not negative, outcomes is predictive of drug use, whereas value updating following negative, but not positive, outcomes is disrupted following drug self-administration. We contextualize these findings with studies on the circuit and biological mechanisms of decision-making to develop a framework for revealing the biobehavioral mechanisms of addiction.

Relationship of Serum Total Insulin-Like Growth Factor Binding Protein-3 with Insulin-Like Growth Factor-I and Glucose Tolerance in Korean Children and Adolescents.

Insulin is important in glucose metabolism. However, insulin-like growth factor binding protein (IGFBP) also plays an important role in glucose homeostasis, although the IGF-independent role of IGFBP-3 in the glucose intolerance state is poorly understood. We investigated the relationship of serum IGF-I with total IGFBP-3 levels and glucose tolerance in Korean children and adolescents who underwent the oral glucose tolerance test (OGTT). A total of 187 children without known diabetes underwent OGTT, and data related to their clinical and laboratory parameters were collected. Serum IGF-I and total IGFBP-3 levels, fasting plasma glucose levels, lipid profiles, insulin levels, C-peptide levels, homeostasis model assessment of insulin resistance (HOMA-IR) index, and glycated hemoglobin (HbA1c) levels were measured. Serum IGF-I and total IGFBP-3 levels were significantly higher in individuals with impaired glucose tolerance and type 2 diabetes (DM) than in those with normal glucose tolerance (NGT) (P < 0.05). Serum IGF-I and IGFBP-3 levels were correlated with age, HbA1c, C-peptide, insulin, and HOMA-IR in the NGT group. However, these relationships were altered in patients with glucose intolerance, especially in those with DM. In the DM group, serum IGF-I and total IGFBP-3 levels were positively correlated with fasting plasma glucose and HbA1c levels. In addition, total IGFBP-3 levels were positively correlated with total cholesterol and low-density lipoprotein cholesterol and IGF-I levels but not with age or body mass index. The IGF-I-IGFBP-3 axis, especially IGFBP-3, may be involved in the pathogenesis and metabolic control of glucose intolerance, specifically in diabetes patients. Moreover, IGFBP-3 might be a therapeutic marker.

Unlocking the reinforcement-learning circuits of the orbitofrontal cortex.

Neuroimaging studies have consistently identified the orbitofrontal cortex (OFC) as being affected in individuals with neuropsychiatric disorders. OFC dysfunction has been proposed to be a key mechanism by which decision-making impairments emerge in diverse clinical populations, and recent studies employing computational approaches have revealed that distinct reinforcement-learning mechanisms of decision-making differ among diagnoses. In this perspective, we propose that these computational differences may be linked to select OFC circuits and present our recent work that has used a neurocomputational approach to understand the biobehavioral mechanisms of addiction pathology in rodent models. We describe how combining translationally analogous behavioral paradigms with reinforcement-learning algorithms and sophisticated neuroscience techniques in animals can provide critical insights into OFC pathology in biobehavioral disorders. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Timescales of Cognition in the Brain.

We live in a world that changes on many timescales. To learn and make decisions appropriately, the human brain has evolved to integrate various types of information, such as sensory evidence and reward feedback, on multiple timescales. This is reflected in cortical hierarchies of timescales consisting of heterogeneous neuronal activities and expression of genes related to neurotransmitters critical for learning. We review the recent findings on how timescales of sensory and reward integration are affected by the temporal properties of sensory and reward signals in the environment. Despite existing evidence linking behavioral and neuronal timescales, future studies must examine how neural computations at multiple timescales are adjusted and combined to influence behavior flexibly.

Robust and distributed neural representation of action values.

Studies in rats, monkeys, and humans have found action-value signals in multiple regions of the brain. These findings suggest that action-value signals encoded in these brain structures bias choices toward higher expected rewards. However, previous estimates of action-value signals might have been inflated by serial correlations in neural activity and also by activity related to other decision variables. Here, we applied several statistical tests based on permutation and surrogate data to analyze neural activity recorded from the striatum, frontal cortex, and hippocampus. The results show that previously identified action-value signals in these brain areas cannot be entirely accounted for by concurrent serial correlations in neural activity and action value. We also found that neural activity related to action value is intermixed with signals related to other decision variables. Our findings provide strong evidence for broadly distributed neural signals related to action value throughout the brain.

Neurocognitive Mechanisms of Social Inferences in Typical and Autistic Adolescents.

BACKGROUND: Many of our efforts in social interactions are dedicated to learning about others. Adolescents with autism have core deficits in social learning, but a mechanistic understanding of these deficits and how they relate to neural development is lacking. The present study aimed to specify how adolescents with and without autism represent and acquire social knowledge and how these processes are implemented in neural activity. METHODS: Typically developing adolescents (n = 26) and adolescents with autism spectrum disorder (ASD) (n = 20) rated in the magnetic resonance scanner how much 3 peers liked a variety of items and received trial-by-trial feedback about the peers' actual preference ratings. In a separate study, we established the preferences of a new sample of adolescents (N = 99), used to examine population preference structures. Using computational models, we tested whether participants in the magnetic resonance study relied on preference structures during learning and how model predictions were implemented in brain activity. RESULTS: Typically developing adolescents relied on average population preferences and prediction error updating. Importantly, prediction error updating was scaled by the similarity between items. In contrast, preferences of adolescents with ASD were best described by a No-Learning model that relied only on the participant's own preferences for each item. Model predictions were encoded in neural activity. Typically developing adolescents encoded prediction errors in the putamen, and adolescents with ASD showed greater encoding of own preferences in the angular gyrus. CONCLUSIONS: We specified how adolescents represent and update social knowledge during learning. Our findings indicate that adolescents with ASD rely only on their own preferences when making social inferences.

Confluence of Timing and Reward Biases in Perceptual Decision-Making Dynamics.

Although the decisions of our daily lives often occur in the context of temporal and reward structures, the impact of such regularities on decision-making strategy is poorly understood. Here, to explore how temporal and reward context modulate strategy, we trained 2 male rhesus monkeys to perform a novel perceptual decision-making task with asymmetric rewards and time-varying evidence reliability. To model the choice and response time patterns, we developed a computational framework for fitting generalized drift-diffusion models, which flexibly accommodate diverse evidence accumulation strategies. We found that a dynamic urgency signal and leaky integration, in combination with two independent forms of reward biases, best capture behavior. We also tested how temporal structure influences urgency by systematically manipulating the temporal structure of sensory evidence, and found that the time course of urgency was affected by temporal context. Overall, our approach identified key components of cognitive mechanisms for incorporating temporal and reward structure into decisions.SIGNIFICANCE STATEMENT In everyday life, decisions are influenced by many factors, including reward structures and stimulus timing. While reward and timing have been characterized in isolation, ecologically valid decision-making involves a multiplicity of factors acting simultaneously. This raises questions about whether the same decision-making strategy is used when these two factors are concurrently manipulated. To address these questions, we trained rhesus monkeys to perform a novel decision-making task with both reward asymmetry and temporal uncertainty. In order to understand their strategy and hint at its neural mechanisms, we used the new generalized drift diffusion modeling framework to model both reward and timing mechanisms. We found two of each reward and timing mechanisms are necessary to explain our data.

Multiple timescales of neural dynamics and integration of task-relevant signals across cortex.

A long-lasting challenge in neuroscience has been to find a set of principles that could be used to organize the brain into distinct areas with specific functions. Recent studies have proposed the orderly progression in the time constants of neural dynamics as an organizational principle of cortical computations. However, relationships between these timescales and their dependence on response properties of individual neurons are unknown, making it impossible to determine how mechanisms underlying such a computational principle are related to other aspects of neural processing. Here, we developed a comprehensive method to simultaneously estimate multiple timescales in neuronal dynamics and integration of task-relevant signals along with selectivity to those signals. By applying our method to neural and behavioral data during a dynamic decision-making task, we found that most neurons exhibited multiple timescales in their response, which consistently increased from parietal to prefrontal and cingulate cortex. While predicting rates of behavioral adjustments, these timescales were not correlated across individual neurons in any cortical area, resulting in independent parallel hierarchies of timescales. Additionally, none of these timescales depended on selectivity to task-relevant signals. Our results not only suggest the existence of multiple canonical mechanisms for increasing timescales of neural dynamics across cortex but also point to additional mechanisms that allow decorrelation of these timescales to enable more flexibility.

Dysregulation of Decision Making Related to Metabotropic Glutamate 5, but Not Midbrain D3, Receptor Availability Following Cocaine Self-administration in Rats.

BACKGROUND: Compulsive patterns of drug use are thought to be the consequence of drug-induced adaptations in the neural mechanisms that enable behavior to be flexible. Neuroimaging studies have found evidence of robust alterations in glutamate and dopamine receptors within brain regions that are known to be critical for decision-making processes in cocaine-dependent individuals, and these changes have been argued to be the consequence of persistent drug use. The causal relationships among drug-induced alterations, cocaine taking, and maladaptive decision-making processes, however, are difficult to establish in humans. METHODS: We assessed decision making in adult male rats using a probabilistic reversal learning task and used positron emission tomography with the [11C]-(+)-PHNO and [18F]FPEB radioligands to quantify regional dopamine D2/3 and metabotropic glutamate 5 (mGlu5) receptor availability, respectively, before and after 21 days of cocaine or saline self-administration. Tests of motivation and relapse-like behaviors were also conducted. RESULTS: We found that self-administration of cocaine, but not of saline, disrupted behavior in the probabilistic reversal learning task measured by selective impairments in negative-outcome updating and also increased cortical mGlu5 receptor availability following 2 weeks of forced abstinence. D2/3 and, importantly, midbrain D3 receptor availability was not altered following 2 weeks of abstinence from cocaine. Notably, the degree of the cocaine-induced increase in cortical mGlu5 receptor availability was related to the degree of disruption in negative-outcome updating. CONCLUSIONS: These findings suggest that cocaine-induced changes in mGlu5 signaling may be a mechanism by which disruptions in negative-outcome updating emerge in cocaine-dependent individuals.

Reinforcement Learning during Adolescence in Rats.

The most dynamic period of postnatal brain development occurs during adolescence, the period between childhood and adulthood. Neuroimaging studies have observed morphologic and functional changes during adolescence, and it is believed that these changes serve to improve the functions of circuits that underlie decision-making. Direct evidence in support of this hypothesis, however, has been limited because most preclinical decision-making paradigms are not readily translated to humans. Here, we developed a reversal-learning protocol for the rapid assessment of adaptive choice behavior in dynamic environments in rats as young as postnatal day 30. A computational framework was used to elucidate the reinforcement-learning mechanisms that change in adolescence and into adulthood. Using a cross-sectional and longitudinal design, we provide the first evidence that value-based choice behavior in a reversal-learning task improves during adolescence in male and female Long-Evans rats and demonstrate that the increase in reversal performance is due to alterations in value updating for positive outcomes. Furthermore, we report that reversal-learning trajectories in adolescence reliably predicted reversal performance in adulthood. This novel behavioral protocol provides a unique platform for conducting biological and systems-level analyses of the neurodevelopmental mechanisms of decision-making.SIGNIFICANCE STATEMENT The neurodevelopmental adaptations that occur during adolescence are hypothesized to underlie age-related improvements in decision-making, but evidence to support this hypothesis has been limited. Here, we describe a novel behavioral protocol for rapidly assessing adaptive choice behavior in adolescent rats with a reversal-learning paradigm. Using a computational approach, we demonstrate that age-related changes in reversal-learning performance in male and female Long-Evans rats are linked to specific reinforcement-learning mechanisms and are predictive of reversal-learning performance in adulthood. Our behavioral protocol provides a unique platform for elucidating key components of adolescent brain function.