Multiple timescales of neural dynamics and integration of task-relevant signals across cortex.

A long-lasting challenge in neuroscience has been to find a set of principles that could be used to organize the brain into distinct areas with specific functions. Recent studies have proposed the orderly progression in the time constants of neural dynamics as an organizational principle of cortical computations. However, relationships between these timescales and their dependence on response properties of individual neurons are unknown, making it impossible to determine how mechanisms underlying such a computational principle are related to other aspects of neural processing. Here, we developed a comprehensive method to simultaneously estimate multiple timescales in neuronal dynamics and integration of task-relevant signals along with selectivity to those signals. By applying our method to neural and behavioral data during a dynamic decision-making task, we found that most neurons exhibited multiple timescales in their response, which consistently increased from parietal to prefrontal and cingulate cortex. While predicting rates of behavioral adjustments, these timescales were not correlated across individual neurons in any cortical area, resulting in independent parallel hierarchies of timescales. Additionally, none of these timescales depended on selectivity to task-relevant signals. Our results not only suggest the existence of multiple canonical mechanisms for increasing timescales of neural dynamics across cortex but also point to additional mechanisms that allow decorrelation of these timescales to enable more flexibility.

Confluence of Timing and Reward Biases in Perceptual Decision-Making Dynamics.

Although the decisions of our daily lives often occur in the context of temporal and reward structures, the impact of such regularities on decision-making strategy is poorly understood. Here, to explore how temporal and reward context modulate strategy, we trained 2 male rhesus monkeys to perform a novel perceptual decision-making task with asymmetric rewards and time-varying evidence reliability. To model the choice and response time patterns, we developed a computational framework for fitting generalized drift-diffusion models, which flexibly accommodate diverse evidence accumulation strategies. We found that a dynamic urgency signal and leaky integration, in combination with two independent forms of reward biases, best capture behavior. We also tested how temporal structure influences urgency by systematically manipulating the temporal structure of sensory evidence, and found that the time course of urgency was affected by temporal context. Overall, our approach identified key components of cognitive mechanisms for incorporating temporal and reward structure into decisions.SIGNIFICANCE STATEMENT In everyday life, decisions are influenced by many factors, including reward structures and stimulus timing. While reward and timing have been characterized in isolation, ecologically valid decision-making involves a multiplicity of factors acting simultaneously. This raises questions about whether the same decision-making strategy is used when these two factors are concurrently manipulated. To address these questions, we trained rhesus monkeys to perform a novel decision-making task with both reward asymmetry and temporal uncertainty. In order to understand their strategy and hint at its neural mechanisms, we used the new generalized drift diffusion modeling framework to model both reward and timing mechanisms. We found two of each reward and timing mechanisms are necessary to explain our data.

Neural basis of reinforcement learning and decision making.

Reinforcement learning is an adaptive process in which an animal utilizes its previous experience to improve the outcomes of future choices. Computational theories of reinforcement learning play a central role in the newly emerging areas of neuroeconomics and decision neuroscience. In this framework, actions are chosen according to their value functions, which describe how much future reward is expected from each action. Value functions can be adjusted not only through reward and penalty, but also by the animal's knowledge of its current environment. Studies have revealed that a large proportion of the brain is involved in representing and updating value functions and using them to choose an action. However, how the nature of a behavioral task affects the neural mechanisms of reinforcement learning remains incompletely understood. Future studies should uncover the principles by which different computational elements of reinforcement learning are dynamically coordinated across the entire brain.

Therapeutic doses of ketamine acutely attenuate the aversive effect of losses during decision-making.

The discovery of rapid-acting antidepressant, ketamine has opened a pathway to a new generation of treatments for depression, and inspired neuroscientific investigation based on a new perspective that non-adaptive changes in the intrinsic excitatory and inhibitory circuitry might underlie the pathophysiology of depression. Nevertheless, it still remains largely unknown how the hypothesized molecular and synaptic levels of changes in the circuitry might mediate behavioral and neuropsychological changes underlying depression, and how ketamine might restore adaptive behavior. Here, we used computational models to analyze behavioral changes induced by therapeutic doses of ketamine, while rhesus macaques were iteratively making decisions based on gains and losses of tokens. When administered intramuscularly or intranasally, ketamine reduced the aversiveness of undesirable outcomes such as losses of tokens without significantly affecting the evaluation of gains, behavioral perseveration, motivation, and other cognitive aspects of learning such as temporal credit assignment and time scales of choice and outcome memory. Ketamine's potentially antidepressant effect was separable from other side effects such as fixation errors, which unlike outcome evaluation, was readily countered with strong motivation to avoid errors. We discuss how the acute effect of ketamine to reduce the initial impact of negative events could potentially mediate longer-term antidepressant effects through mitigating the cumulative effect of those events produced by slowly decaying memory, and how the disruption-resistant affective memory might pose challenges in treating depression. Our study also invites future investigations on ketamine's antidepressant action over diverse mood states and with affective events exerting their impacts at diverse time scales.

Single-nucleotide resolution of RNAs up to 59 nucleotides by high-performance liquid chromatography.

Ion-pair, reverse-phase high-performance liquid chromatography (HPLC) is a standard analytical platform for separating, purifying, and analyzing RNAs. However, a single-nucleotide resolution by using HPLC is currently limited to RNAs shorter than 25 nucleotides (nt). Here we describe a method of separating three RNA aptamers with 57, 58, and 59nt on an XBridge ion-pair, reverse-phase HPLC column by a single-nucleotide resolution. Under a similar condition, we also show the capability of our method to resolve two structurally different, yet sequence or mass identical, 59-nt aptamers. We establish that the optimal condition to achieve a single-nucleotide resolution correlates to 50°C and zero magnesium concentration in mobile phases. The ion-pairing agent, the buffer, and the solvent we use are also compatible for post-HPLC analysis such as mass spectrometry. Therefore, our method provides a new way of detecting, analyzing, and separating RNAs by conformation or structure and extends the ability to separate RNAs that are longer than 25nt by single-nucleotide resolution.

The first high-quality genome assembly and annotation of Patiria pectinifera.

The blue bat star, a highly adaptive species in the East Sea of Korea, has displayed remarkable success in adapting to recent climate change. The genetic mechanisms behind this success were not well-understood, prompting our report on the first chromosome-level assembly of the Patiria genus. We assembled the genome using Nanopore and Illumina sequences, yielding a total length of 615 Mb and a scaffold N50 of 24,204,423 bp. Hi-C analysis allowed us to anchor the scaffold sequences onto 22 pseudochromosomes. K-mer based analysis revealed 5.16% heterozygosity rate of the genome, higher than any previously reported echinoderm species. Our transposable element analysis exposed a substantial number of genome-wide retrotransposons and DNA transposons. These results offer valuable resources for understanding the evolutionary mechanisms behind P. pectinifera's successful adaptation in fluctuating environments.

Transient neuronal suppression for exploitation of new sensory evidence.

In noisy but stationary environments, decisions should be based on the temporal integration of sequentially sampled evidence. This strategy has been supported by many behavioral studies and is qualitatively consistent with neural activity in multiple brain areas. By contrast, decision-making in the face of non-stationary sensory evidence remains poorly understood. Here, we trained monkeys to identify and respond via saccade to the dominant color of a dynamically refreshed bicolor patch that becomes informative after a variable delay. Animals' behavioral responses were briefly suppressed after evidence changes, and many neurons in the frontal eye field displayed a corresponding dip in activity at this time, similar to that frequently observed after stimulus onset but sensitive to stimulus strength. Generalized drift-diffusion models revealed consistency of behavior and neural activity with brief suppression of motor output, but not with pausing or resetting of evidence accumulation. These results suggest that momentary arrest of motor preparation is important for dynamic perceptual decision making.

NMDAR Neurotransmission Needed for Persistent Neuronal Firing: Potential Roles in Mental Disorders.

The dorsolateral prefrontal cortex (dlPFC) generates the mental representations that are the foundation of abstract thought, and provides top-down regulation of emotion through projections to the medial PFC and cingulate cortices. Physiological recordings from dlPFC Delay cells have shown that the generation of mental representations during working memory relies on NMDAR neurotransmission, with surprisingly little contribution from AMPAR. Systemic administration of low "antidepressant" doses of the NMDAR antagonist, ketamine, erodes these representations and reduces dlPFC Delay cell firing. In contrast to the dlPFC, V1 neuronal firing to visual stimuli depends on AMPAR, with much less contribution from NMDAR. Similarly, neurons in the dlPFC that respond to sensory events (cue cells, response feedback cells) rely on AMPAR, and systemic ketamine increases their firing. Insults to NMDAR transmission, and the impaired ability for dlPFC to generate mental representations, may contribute to cognitive deficits in schizophrenia, e.g., from genetic insults that weaken NMDAR transmission, or from blockade of NMDAR by kynurenic acid. Elevated levels of kynurenic acid in dlPFC may also contribute to cognitive deficits in other disorders with pronounced neuroinflammation (e.g., Alzheimer's disease), or peripheral infections where kynurenine can enter brain (e.g., delirium from sepsis, "brain fog" in COVID19). Much less is known about NMDAR actions in the primate cingulate cortices. However, NMDAR neurotransmission appears to process the affective and visceral responses to pain and other aversive experiences mediated by the cingulate cortices, which may contribute to sustained alterations in mood state. We hypothesize that the very rapid, antidepressant effects of intranasal ketamine may involve the disruption of NMDAR-generated aversive mood states by the anterior and subgenual cingulate cortices, providing a "foot in the door" to allow the subsequent return of top-down regulation by higher PFC areas. Thus, the detrimental vs. therapeutic effects of NMDAR blockade may be circuit dependent.

Timescales of Cognition in the Brain.

We live in a world that changes on many timescales. To learn and make decisions appropriately, the human brain has evolved to integrate various types of information, such as sensory evidence and reward feedback, on multiple timescales. This is reflected in cortical hierarchies of timescales consisting of heterogeneous neuronal activities and expression of genes related to neurotransmitters critical for learning. We review the recent findings on how timescales of sensory and reward integration are affected by the temporal properties of sensory and reward signals in the environment. Despite existing evidence linking behavioral and neuronal timescales, future studies must examine how neural computations at multiple timescales are adjusted and combined to influence behavior flexibly.

Behavioral and neural changes after gains and losses of conditioned reinforcers.

Human behaviors can be more powerfully influenced by conditioned reinforcers, such as money, than by primary reinforcers. Moreover, people often change their behaviors to avoid monetary losses. However, the effect of removing conditioned reinforcers on choices has not been explored in animals, and the neural mechanisms mediating the behavioral effects of gains and losses are not well understood. To investigate the behavioral and neural effects of gaining and losing a conditioned reinforcer, we trained rhesus monkeys for a matching pennies task in which the positive and negative values of its payoff matrix were realized by the delivery and removal of a conditioned reinforcer. Consistent with the findings previously obtained with non-negative payoffs and primary rewards, the animal's choice behavior during this task was nearly optimal. Nevertheless, the gain and loss of a conditioned reinforcer significantly increased and decreased, respectively, the tendency for the animal to choose the same target in subsequent trials. We also found that the neurons in the dorsomedial frontal cortex, dorsal anterior cingulate cortex, and dorsolateral prefrontal cortex often changed their activity according to whether the animal earned or lost a conditioned reinforcer in the current or previous trial. Moreover, many neurons in the dorsomedial frontal cortex also signaled the gain or loss occurring as a result of choosing a particular action as well as changes in the animal's behaviors resulting from such gains or losses. Thus, primate medial frontal cortex might mediate the behavioral effects of conditioned reinforcers and their losses.

Lateral intraparietal cortex and reinforcement learning during a mixed-strategy game.

Activity of the neurons in the lateral intraparietal cortex (LIP) displays a mixture of sensory, motor, and memory signals. Moreover, they often encode signals reflecting the accumulation of sensory evidence that certain eye movements might lead to a desirable outcome. However, when the environment changes dynamically, animals are also required to combine the information about its previously chosen actions and their outcomes appropriately to update continually the desirabilities of alternative actions. Here, we investigated whether LIP neurons encoded signals necessary to update an animal's decision-making strategies adaptively during a computer-simulated matching-pennies game. Using a reinforcement learning algorithm, we estimated the value functions that best predicted the animal's choices on a trial-by-trial basis. We found that, immediately before the animal revealed its choice, approximately 18% of LIP neurons changed their activity according to the difference in the value functions for the two targets. In addition, a somewhat higher fraction of LIP neurons displayed signals related to the sum of the value functions, which might correspond to the state value function or an average rate of reward used as a reference point. Similar to the neurons in the prefrontal cortex, many LIP neurons also encoded the signals related to the animal's previous choices. Thus, the posterior parietal cortex might be a part of the network that provides the substrate for forming appropriate associations between actions and outcomes.

All about pain pharmacology: what pain physicians should know.

From the perspective of the definition of pain, pain can be divided into emotional and sensory components, which originate from potential and actual tissue damage, respectively. The pharmacologic treatment of the emotional pain component includes antianxiety drugs, antidepressants, and antipsychotics. The anti-anxiety drugs have anti-anxious, sedative, and somnolent effects. The antipsychotics are effective in patients with positive symptoms of psychosis. On the other hand, the sensory pain component can be divided into nociceptive and neuropathic pain. Non-steroidal anti-inflammatory drugs (NSAIDs) and opioids are usually applied for somatic and visceral nociceptive pain, respectively; anticonvulsants and antidepressants are administered for the treatment of neuropathic pain with positive and negative symptoms, respectively. The NSAIDs, which inhibit the cyclo-oxygenase pathway, exhibit anti-inflammatory, antipyretic, and analgesic effects; however, they have a therapeutic ceiling. The adverse reactions (ADRs) of the NSAIDs include gastrointestinal problems, generalized edema, and increased bleeding tendency. The opioids, which bind to the opioid receptors, present an analgesic effect only, without anti-inflammatory, antipyretic, or ceiling effects. The ADRs of the opioids start from itching and nausea/vomiting to cardiovascular and respiratory depression, as well as constipation. The anticonvulsants include carbamazepine, related to sodium channel blockade, and gabapentin and pregabalin, related to calcium blockade. The antidepressants show their analgesic actions mainly through inhibiting the reuptake of serotonin or norepinephrine. Most drugs, except NSAIDs, need an updose titration period. The principle of polypharmacy for analgesia in case of mixed components of pain is increasing therapeutic effects while reducing ADRs, based on the origin of the pain.

CD30 supports lung inflammation.

The physiological functions of CD30 have not been fully elucidated. Here we show that in CD30-deficient mice (CD30(-/-)), lung inflammation is significantly diminished in the ovalbumin (OVA) model of airway hyperreactivity. In CD30(-/-) mice, the recruitment of eosinophils into the airways after OVA-aerosol challenge of OVA-primed mice was significantly diminished when compared with wild-type (w.t.) mice. IL-13 levels were also significantly reduced in CD30(-/-) mice while levels of IFN-gamma, IL-4, IL-5 and IgE in bronchoalveolar lavage fluid, lung tissue and serum were comparable to w.t. mice. Peribronchial lymph node cells from CD30(-/-) mice, re-stimulated in vitro with OVA, secreted significantly lower levels of IL-13 than those from w.t. mice, but showed normal proliferative response and other cytokine production. Exogenous IL-13 reconstituted airway recruitment of leukocytes in OVA-challenged CD3O(-/-) mice. Adoptive transfer to naive w.t. mice of in vitro OVA-re-stimulated spleen cells from CD30(-/-) mice failed to induce eosinophilic pulmonary inflammation in contrast to transfer of primed cells from w.t. mice. These results indicate that CD30 is a regulator of T(h)2 responses in the effector-memory phase and a regulator of IL-13 production in memory cells in the lung.

Therapeutic target validation of protein kinase C(PKC)-zeta for asthma using a mouse model.

Protein kinase C (PKC) is a complex family consisting of many types of isoenzymes, of which PKC-zeta, an atypical isoform, has been reportedly implicated in the regulation of apoptosis and NF-kappaB, as well as control of T-dependent responses. Based on the recent report that PKC-zeta controls TH2 response, the current study was aimed to evaluate PKC-zeta as a potential therapeutic target for asthma using a mouse model. Mouse allergic asthma was induced by repeated sensitization followed by intranasal challenge with OVA and PKC-zeta pseudosubstrate inhibitor (PPI) was intratracheally instilled before each OVA challenge. Airway hyperreactivity (AHR) was measured by beta-methacoline-induced airflow obstruction. Cellular and cytokine profile in bronchoalveolar lavage fluid (BALF) and level of serum IgE as well as cytokine production by draining lymph node cells were compared. AHR and numbers of eosinophils in BALF were significantly lowered by PPI, indicating that blocking of PKC-zeta activation alleviates asthmatic manifestations. Additionally, PPI instillation decreased IL-5 and IL-13 levels in BALF to approximately 20% of controls, but not IFN-gamma level. Instillation of PPI also caused a marked fall in the level of TNF-alpha, another NF-kappaB-dependent, proinflammatory cytokine. Serum OVA-specific IgE level and ex vivo IL-4, IL-5 and IL-13, but not IFN-gamma, production by peribronchial lymph node cells was also considerably lower in PPI-treated mice. In conclusion, blockade of PKC-zeta signals by intratracheal instillation of PPI alleviates allergen-specific TH2 response as well as asthmatic manifestations and hence PKC-zeta is a promising target for treatment of asthma.

Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game.

The process of decision making in humans and other animals is adaptive and can be tuned through experience so as to optimize the outcomes of their choices in a dynamic environment. Previous studies have demonstrated that the anterior cingulate cortex plays an important role in updating the animal's behavioral strategies when the action outcome contingencies change. Moreover, neurons in the anterior cingulate cortex often encode the signals related to expected or actual reward. We investigated whether reward-related activity in the anterior cingulate cortex is affected by the animal's previous reward history. This was tested in rhesus monkeys trained to make binary choices in a computer-simulated competitive zero-sum game. The animal's choice behavior was relatively close to the optimal strategy but also revealed small systematic biases that are consistent with the use of a reinforcement learning algorithm. In addition, the activity of neurons in the dorsal anterior cingulate cortex that was related to the reward received by the animal in a given trial often was modulated by the rewards in the previous trials. Some of these neurons encoded the rate of rewards in previous trials, whereas others displayed activity modulations more closely related to the reward prediction errors. In contrast, signals related to the animal's choices were represented only weakly in this cortical area. These results suggest that neurons in the dorsal anterior cingulate cortex might be involved in the subjective evaluation of choice outcomes based on the animal's reward history.

Essential oil of niaouli preferentially potentiates antigen-specific cellular immunity and cytokine production by macrophages.

In vivo immunomodulatory effect of essential oil of niaouli (EON) was investigated using a mouse model, in which mice were immunized with keyhole limpet hemocyanin (KLH) and intraperitoneally given EON (less than 500 microl kg(-1) body weight). In vivo efficacy of EON for immune potentiation was convinced by significantly higher expression of an activation marker, CD25, on freshly isolated draining lymph node (LN) T cells, but not B cells. However, immunofluoresence analysis failed to show any proportional change in T/B and CD4(+)/CD8(+) T cell ratios. Data of KLH-specific immunoglobulin serum levels showed that EON does not affect humoral immune response. Instead, proliferative response and IFNgamma production of LN T cells ex vivo stimulated with KLH were significantly higher in EON-treated group, but not IL-2 and IL-4 production. These results clearly show that EON preferentially upregulates T-cell mediated cellular immunity. We further clarified the accessory cells' contribution to the EON-mediated potentiation of cellular immunity and found considerably higher production of and TNF-alpha and IL-12 by splenic macrophages from EON-treated mice when stimulated with lipopolysaccharide (LPS) and IFNgamma. Collectively, in vivo EON treatment potentiates T cell-mediated cellular immunity and macrophage activity, but not humoral immunity. The current study provides a rationale for clinical application of EON to control infectious diseases, in particular, those caused by intracellular pathogens.

Mechanisms of reinforcement learning and decision making in the primate dorsolateral prefrontal cortex.

To a first approximation, decision making is a process of optimization in which the decision maker tries to maximize the desirability of the outcomes resulting from chosen actions. Estimates of desirability are referred to as utilities or value functions, and they must be continually revised through experience according to the discrepancies between the predicted and obtained rewards. Reinforcement learning theory prescribes various algorithms for updating value functions and can parsimoniously account for the results of numerous behavioral, neurophysiological, and imaging studies in humans and other primates. In this article, we first discuss relative merits of various decision-making tasks used in neurophysiological studies of decision making in nonhuman primates. We then focus on how reinforcement learning theory can shed new light on the function of the primate dorsolateral prefrontal cortex. Similar to the findings from other brain areas, such as cingulate cortex and basal ganglia, activity in the dorsolateral prefrontal cortex often signals the value of expected reward and actual outcome. Thus, the dorsolateral prefrontal cortex is likely to be a part of the broader network involved in adaptive decision making. In addition, reward-related activity in the dorsolateral prefrontal cortex is influenced by the animal's choices and other contextual information, and therefore may provide a neural substrate by which the animals can flexibly modify their decision-making strategies according to the demands of specific tasks.

Effective microorganism fermentation extract (EM-X) attenuates airway hyperreactivity and inflammation through selective inhibition of the TH2 response independently of antioxidant activity.

The effective microorganism fermentation extract (EM-X) is an antioxidant cocktail derived from the fermentation of plant material with effective microorganisms, and its clinical application is being increasingly scrutinized. In the current study, the antiasthmatic effect of EM-X was investigated using a mouse model. Inhalation of EM-X during OVA challenge resulted in a significant reduction in airway hyperreactivity (AHR) and airway recruitment of leukocytes including eosinophils. However, the level of 8-isoprostane in bronchoalveolar lavage fluid (BALF), a marker of oxidative stress in asthmatic patients, was unaltered by EM-X inhalation. Instead, ELISA data showed that levels of IL-4, IL-5 and IL-13 in BALF or lung tissues were significantly lower in EM-X-inhaling mice than in the control mice, but not the IFN-gamma level. A considerably lower amount of Ag-specific IgE and IgG1 was detected in the serum of EM-X-inhaling mice than in the serum of the controls, whereas their IgG2a secretion was similar. In addition, Ag-specific ex vivo IL-4, IL-5 and IL-13 production of draining lymph node cells was markedly diminished by EM-X inhalation, but not IFN-gamma. These data clearly show that inhaled EM-X suppresses type 2 helper T (TH2), but not type 1 helper T (TH1), response. In conclusion, inhalation of EM-X attenuates AHR and airway inflammation which results from selective inhibition of the TH2 response to allergen, but independently of antioxidant activity. Our data also suggest that EM-X may be effectively applied for control of allergic asthma.

Metaplasticity as a Neural Substrate for Adaptive Learning and Choice under Uncertainty.

Value-based decision making often involves integration of reward outcomes over time, but this becomes considerably more challenging if reward assignments on alternative options are probabilistic and non-stationary. Despite the existence of various models for optimally integrating reward under uncertainty, the underlying neural mechanisms are still unknown. Here we propose that reward-dependent metaplasticity (RDMP) can provide a plausible mechanism for both integration of reward under uncertainty and estimation of uncertainty itself. We show that a model based on RDMP can robustly perform the probabilistic reversal learning task via dynamic adjustment of learning based on reward feedback, while changes in its activity signal unexpected uncertainty. The model predicts time-dependent and choice-specific learning rates that strongly depend on reward history. Key predictions from this model were confirmed with behavioral data from non-human primates. Overall, our results suggest that metaplasticity can provide a neural substrate for adaptive learning and choice under uncertainty.