A method for evaluating hunger and thirst in monkeys by measuring blood ghrelin and osmolality levels.

Hunger and thirst drive animals' consumption behavior and regulate their decision-making concerning rewards. We previously assessed the thirst states of monkeys by measuring blood osmolality under controlled water access and examined how these thirst states influenced their risk-taking behaviour in decisions involving fluid rewards. However, hunger assessment in monkeys remains poorly performed. Moreover, the lack of precise measures for hunger states leads to another issue regarding how hunger and thirst states interact with each other in each individual. Thus, when controlling food access to motivate performance, it remains unclear how these two physiological needs are satisfied in captive monkeys. Here, we measured blood ghrelin and osmolality levels to respectively assess hunger and thirst in four captive macaques. Using an enzyme-linked immunosorbent assay, we identified that the levels of blood ghrelin, a widely measured hunger-related peptide hormone in humans, were high after 20 h of no food access (with ad libitum water). This reflects a typical controlled food access condition. One hour after consuming a regular dry meal, the blood ghrelin levels in three out of four monkeys decreased to within their baseline range. Additionally, blood osmolality measured from the same blood sample, the standard hematological index of hydration status, increased after consuming the regular dry meal with no water access. Thus, ghrelin and osmolality may reflect the physiological states of individual monkeys regarding hunger and thirst, suggesting that these indices can be used as tools for monitoring hunger and thirst levels that mediate an animal's decision to consume rewards.Significance statement Standard methods for behavioral and neurophysiological experiments in non-human primates rely on controlled access to food or fluid rewards to motivate their performance. We previously assessed the thirst state of monkeys by measuring blood osmolality, the most widely used hematological index of hydration status. Here, we assessed the hunger state of monkeys by measuring blood ghrelin levels, a widely measured hunger-related peptide hormone in humans, using an enzyme-linked immunosorbent assay. We measured these indices and found that they reflected the hunger and thirst states of the monkeys before and after consuming dry meals, with no relation to each other. Thus, these two physical indices can be utilized to monitor hunger and thirst in primates.

Hydrogen sulfide production in the medullary respiratory center modulates the neural circuit for respiratory pattern and rhythm generations.

Hydrogen sulfide (H2S), which is synthesized in the brain, modulates the neural network. Recently, the importance of H2S in respiratory central pattern generation has been recognized, yet the function of H2S in the medullary respiratory network remains poorly understood. Here, to evaluate the functional roles of H2S in the medullary respiratory network, the Bötzinger complex (BötC), the pre-Bötzinger complex (preBötC), and the rostral ventral respiratory group (rVRG), we observed the effects of inhibition of H2S synthesis at each region on the respiratory pattern by using an in situ arterially perfused preparation of decerebrated male rats. After microinjection of an H2S synthase inhibitor, cystathionine ß-synthase, into the BötC or preBötC, the amplitude of the inspiratory burst decreased and the respiratory frequency increased according to shorter expiration and inspiration, respectively. These alterations were abolished or attenuated in the presence of a blocker of excitatory synaptic transmission. On the other hand, after microinjection of the H2S synthase inhibitor into the rVRG, the amplitude of the inspiratory burst was attenuated, and the respiratory frequency decreased, which was the opposite effect to those obtained by blockade of inhibitory synaptic transmission at the rVRG. These results suggest that H2S synthesized in the BötC and preBötC functions to limit respiratory frequency by sustaining the respiratory phase and to maintain the power of inspiration. In contrast, H2S synthesized in the rVRG functions to promote respiratory frequency by modulating the interval of inspiration and to maintain the power of inspiration. The underlying mechanism might facilitate excitatory synaptic transmission and/or attenuate inhibitory synaptic transmission.

Distinct roles of the orbitofrontal cortex, ventral striatum, and dopamine neurons in counterfactual thinking of decision outcomes.

Individuals often assess past decisions by comparing what was gained with what would have been gained had they acted differently. Thoughts of past alternatives that counter what actually happened are called "counterfactuals." Recent theories emphasize the role of the prefrontal cortex in processing counterfactual outcomes in decision-making, although how subcortical regions contribute to this process remains to be elucidated. Here we report a clear distinction among the roles of the orbitofrontal cortex, ventral striatum and midbrain dopamine neurons in processing counterfactual outcomes in monkeys. Our findings suggest that actually gained and counterfactual outcome signals are both processed in the cortico-subcortical network constituted by these regions but in distinct manners and integrated only in the orbitofrontal cortex in a way to compare these outcomes. This study extends the prefrontal theory of counterfactual thinking and provides key insights regarding how the prefrontal cortex cooperates with subcortical regions to make decisions using counterfactual information.

Stable Neural Population Dynamics in the Regression Subspace for Continuous and Categorical Task Parameters in Monkeys.

Neural population dynamics provide a key computational framework for understanding information processing in the sensory, cognitive, and motor functions of the brain. They systematically depict complex neural population activity, dominated by strong temporal dynamics as trajectory geometry in a low-dimensional neural space. However, neural population dynamics are poorly related to the conventional analytical framework of single-neuron activity, the rate-coding regime that analyzes firing rate modulations using task parameters. To link the rate-coding and dynamic models, we developed a variant of state-space analysis in the regression subspace, which describes the temporal structures of neural modulations using continuous and categorical task parameters. In macaque monkeys, using two neural population datasets containing either of two standard task parameters, continuous and categorical, we revealed that neural modulation structures are reliably captured by these task parameters in the regression subspace as trajectory geometry in a lower dimension. Furthermore, we combined the classical optimal-stimulus response analysis (usually used in rate-coding analysis) with the dynamic model and found that the most prominent modulation dynamics in the lower dimension were derived from these optimal responses. Using those analyses, we successfully extracted geometries for both task parameters that formed a straight geometry, suggesting that their functional relevance is characterized as a unidimensional feature in their neural modulation dynamics. Collectively, our approach bridges neural modulation in the rate-coding model and the dynamic system, and provides researchers with a significant advantage in exploring the temporal structure of neural modulations for pre-existing datasets.

Dynamic prospect theory: Two core decision theories coexist in the gambling behavior of monkeys and humans.

Research in the multidisciplinary field of neuroeconomics has mainly been driven by two influential theories regarding human economic choice: prospect theory, which describes decision-making under risk, and reinforcement learning theory, which describes learning for decision-making. We hypothesized that these two distinct theories guide decision-making in a comprehensive manner. Here, we propose and test a decision-making theory under uncertainty that combines these highly influential theories. Collecting many gambling decisions from laboratory monkeys allowed for reliable testing of our model and revealed a systematic violation of prospect theory's assumption that probability weighting is static. Using the same experimental paradigm in humans, substantial similarities between these species were uncovered by various econometric analyses of our dynamic prospect theory model, which incorporates decision-by-decision learning dynamics of prediction errors into static prospect theory. Our model provides a unified theoretical framework for exploring a neurobiological model of economic choice in human and nonhuman primates.

A Noninvasive Method for Monitoring Breathing Patterns in Nonhuman Primates Using a Nasal Thermosensor.

Respiration is strongly linked to internal states such as arousal, emotion, and even cognitive processes and provides objective biological information to estimate these states in humans and animals. However, the measurement of respiration has not been established in macaque monkeys, which have been widely used as model animals for understanding various higher brain functions. In the present study, we developed a method to monitor the respiration of behaving monkeys. We first measured the temperature of their nasal breathing, which changes between inspiration and expiration phases, in an anesthetized condition and estimated the respiration pattern. We compared the estimated pattern with that obtained by a conventional chest band method that has been used in humans and applied to anesthetized, but not behaving, monkeys. These respiration patterns matched well, suggesting that the measurement of nasal air temperature can be used to monitor the respiration of monkeys. Furthermore, we confirmed that the respiration frequency in behaving monkeys monitored by the measurement of nasal air temperature was not affected by the orofacial movement of licking to obtain the liquid reward. We next examined the frequency of respiration when they listened to music or white noise. The respiratory frequency was higher when the monkeys listened to music than the noise. This result is consistent with a phenomenon in humans and indicates the accuracy of our monitoring method. These data suggest that the measurement of nasal air temperature enables us to monitor the respiration of behaving monkeys and thereby estimate their internal states.

A neuronal prospect theory model in the brain reward circuitry.

Prospect theory, arguably the most prominent theory of choice, is an obvious candidate for neural valuation models. How the activity of individual neurons, a possible computational unit, obeys prospect theory remains unknown. Here, we show, with theoretical accuracy equivalent to that of human neuroimaging studies, that single-neuron activity in four core reward-related cortical and subcortical regions represents the subjective valuation of risky gambles in monkeys. The activity of individual neurons in monkeys passively viewing a lottery reflects the desirability of probabilistic rewards parameterized as a multiplicative combination of utility and probability weighting functions, as in the prospect theory framework. The diverse patterns of valuation signals were not localized but distributed throughout most parts of the reward circuitry. A network model aggregating these signals reconstructed the risk preferences and subjective probability weighting revealed by the animals' choices. Thus, distributed neural coding explains the computation of subjective valuations under risk.

Meta-learning, social cognition and consciousness in brains and machines.

The intersection between neuroscience and artificial intelligence (AI) research has created synergistic effects in both fields. While neuroscientific discoveries have inspired the development of AI architectures, new ideas and algorithms from AI research have produced new ways to study brain mechanisms. A well-known example is the case of reinforcement learning (RL), which has stimulated neuroscience research on how animals learn to adjust their behavior to maximize reward. In this review article, we cover recent collaborative work between the two fields in the context of meta-learning and its extension to social cognition and consciousness. Meta-learning refers to the ability to learn how to learn, such as learning to adjust hyperparameters of existing learning algorithms and how to use existing models and knowledge to efficiently solve new tasks. This meta-learning capability is important for making existing AI systems more adaptive and flexible to efficiently solve new tasks. Since this is one of the areas where there is a gap between human performance and current AI systems, successful collaboration should produce new ideas and progress. Starting from the role of RL algorithms in driving neuroscience, we discuss recent developments in deep RL applied to modeling prefrontal cortex functions. Even from a broader perspective, we discuss the similarities and differences between social cognition and meta-learning, and finally conclude with speculations on the potential links between intelligence as endowed by model-based RL and consciousness. For future work we highlight data efficiency, autonomy and intrinsic motivation as key research areas for advancing both fields.

Complex field reconstruction of optical OFDM signals based on temporal transport-of-intensity equation.

Field reconstruction of optical orthogonal frequency division multiplexing (OFDM) signals in a direct-detection (DD) receiver by using temporal transport-of-intensity equation (TIE) is studied. It is shown that in this DD field reconstruction scheme, better BER performance is obtained by using the OFDM modulation than by using single carrier signals especially when low-frequency subcarriers of OFDM signals are not used. How phase errors are generated in solving the TIE is analyzed and it is shown that the process of integration with respect to time gives rise to low-frequency errors that degrade the performance. The DD phase retrieval scheme has favorable features that the solution is non-iterative and it allows using double side-band signals. Nevertheless, it is shown that the scheme has high sensitivity to electrical noise in detection and requires relatively high carrier to signal power ratio, to which further studies are expected to be devoted.

Parallel and hierarchical neural mechanisms for adaptive and predictive behavioral control.

Our brain can be recognized as a network of largely hierarchically organized neural circuits that operate to control specific functions, but when acting in parallel, enable the performance of complex and simultaneous behaviors. Indeed, many of our daily actions require concurrent information processing in sensorimotor, associative, and limbic circuits that are dynamically and hierarchically modulated by sensory information and previous learning. This organization of information processing in biological organisms has served as a major inspiration for artificial intelligence and has helped to create in silico systems capable of matching or even outperforming humans in several specific tasks, including visual recognition and strategy-based games. However, the development of human-like robots that are able to move as quickly as humans and respond flexibly in various situations remains a major challenge and indicates an area where further use of parallel and hierarchical architectures may hold promise. In this article we review several important neural and behavioral mechanisms organizing hierarchical and predictive processing for the acquisition and realization of flexible behavioral control. Then, inspired by the organizational features of brain circuits, we introduce a multi-timescale parallel and hierarchical learning framework for the realization of versatile and agile movement in humanoid robots.

Single-unit Recording in Awake Behaving Non-human Primates.

Non-human primates (NHPs) have been widely used as a species model in studies to understand higher brain functions in health and disease. These studies employ specifically designed behavioral tasks in which animal behavior is well-controlled, and record neuronal activity at high spatial and temporal resolutions while animals are performing the tasks. Here, we present a detailed procedure to conduct single-unit recording, which fulfils high spatial and temporal resolutions while macaque monkeys (i.e., widely used NHPs) perform behavioral tasks in a well-controlled manner. This procedure was used in our previous study to investigate the dynamics of neuronal activity during economic decision-making by the monkeys. Monkeys' behavior was quantitated by eye position tracking and button press/release detection. By inserting a microelectrode into the brain, with a grid system in reference to magnetic resonance imaging, we precisely recorded the brain regions. Our experimental system permits rigorous investigation of the link between neuronal activity and behavior.

Lateral Habenula Regulates Cardiovascular Autonomic Responses via the Serotonergic System in Rats.

The lateral habenula (LHb) plays essential roles in behavioral responses to stressful events. Stress is tightly linked to autonomic responses such as cardiovascular responses, yet how the LHb regulates these responses is not well understood. To address this issue, we electrically stimulated the LHb in rats, measured its effects on heart rate (HR) and mean arterial pressure (MAP), and investigated the neural circuits that mediate these LHb-induced cardiovascular responses via the autonomic nervous system. We observed that stimulation of the LHb induced bradycardia and pressor responses, whereas stimulation of the adjacent areas changed neither the HR nor the MAP. Bilateral vagotomy and administration of a muscarinic receptor antagonist suppressed the LHb stimulation effect on the HR but not on the MAP, whereas administration of a ß-adrenoceptor antagonist partly attenuated the effect on the MAP but not on the HR. Thus, the LHb-induced cardiovascular responses of the HR and the MAP were likely caused by activations of the cardiac parasympathetic nerves and the cardiovascular sympathetic nerves, respectively. Furthermore, administration of a non-selective 5-HT receptor antagonist significantly attenuated the LHb stimulation effects on both the MAP and the HR. A 5-HT2 receptor antagonist also attenuated the LHb stimulation effects. A low dose of a 5-HT1A receptor antagonist enhanced the LHb stimulation effects, but a high dose of the drug attenuated them. 5-HT1B and 5-HT1D receptor antagonists as well as a 5-HT7 receptor antagonist did not affect the LHb stimulation effects. Taken together, our findings suggest that the LHb regulates autonomic cardiovascular responses at least partly through the serotonergic system, particularly via the 5-HT1A and 5-HT2 receptors.

Tonic firing mode of midbrain dopamine neurons continuously tracks reward values changing moment-by-moment.

Animal behavior is regulated based on the values of future rewards. The phasic activity of midbrain dopamine neurons signals these values. Because reward values often change over time, even on a subsecond-by-subsecond basis, appropriate behavioral regulation requires continuous value monitoring. However, the phasic dopamine activity, which is sporadic and has a short duration, likely fails continuous monitoring. Here, we demonstrate a tonic firing mode of dopamine neurons that effectively tracks changing reward values. We recorded dopamine neuron activity in monkeys during a Pavlovian procedure in which the value of a cued reward gradually increased or decreased. Dopamine neurons tonically increased and decreased their activity as the reward value changed. This tonic activity was evoked more strongly by non-burst spikes than burst spikes producing a conventional phasic activity. Our findings suggest that dopamine neurons change their firing mode to effectively signal reward values in a given situation.

Neural Population Dynamics Underlying Expected Value Computation.

Computation of expected values (i.e., probability × magnitude) seems to be a dynamic integrative process performed by the brain for efficient economic behavior. However, neural dynamics underlying this computation is largely unknown. Using lottery tasks in monkeys (Macaca mulatta, male; Macaca fuscata, female), we examined (1) whether four core reward-related brain regions detect and integrate probability and magnitude cued by numerical symbols and (2) whether these brain regions have distinct dynamics in the integrative process. Extraction of the mechanistic structure of neural population signals demonstrated that expected value signals simultaneously arose in the central orbitofrontal cortex (cOFC; medial part of area 13) and ventral striatum (VS). Moreover, these signals were incredibly stable compared with weak and/or fluctuating signals in the dorsal striatum and medial OFC. Temporal dynamics of these stable expected value signals were unambiguously distinct: sharp and gradual signal evolutions in the cOFC and VS, respectively. These intimate dynamics suggest that the cOFC and VS compute the expected values with unique time constants, as distinct, partially overlapping processes.SIGNIFICANCE STATEMENT Our results differ from those of earlier studies suggesting that many reward-related regions in the brain signal probability and/or magnitude and provide a mechanistic structure for expected value computation employed in multiple neural populations. A central part of the orbitofrontal cortex (cOFC) and ventral striatum (VS) can simultaneously detect and integrate probability and magnitude into an expected value. Our empirical study on these neural population dynamics raises a possibility that the cOFC and VS cooperate on this computation with unique time constants as distinct, partially overlapping processes.

Nonhuman Primate Optogenetics: Current Status and Future Prospects.

Nonhuman primates (NHPs) have widely and crucially been utilized as model animals for understanding various higher brain functions and neurological disorders since their behavioral actions mimic both normal and disease states in humans. To know about how such behaviors emerge from the functions and dysfunctions of complex neural networks, it is essential to define the role of a particular pathway or neuron-type constituting these networks. Optogenetics is a potential technique that enables analyses of network functions. However, because of the large size of the NHP brain and the difficulty in creating genetically modified animal models, this technique is currently still hard to apply effectively and efficiently to NHP neuroscience. In this article, we focus on the issues that should be overcome for the development of NHP optogenetics, with special reference to the gene introduction strategy. We review the recent breakthroughs that have been made in NHP optogenetics to address these issues and discuss future prospects regarding more effective and efficient approaches to successful optogenetic manipulation in NHPs.

Endogenous hydrogen sulfide maintains eupnea in an in situ arterially perfused preparation of rats.

Hydrogen sulfide (H2S) is constitutively generated in the human body and works as a gasotransmitter in synaptic transmission. In this study, we aimed to evaluate the roles of endogenous H2S in generating eupnea at the respiratory center. We employed an in situ arterially perfused preparation of decerebrated rats and recorded the central respiratory outputs. When the H2S-producing enzyme cystathionine ß-synthase (CBS) was inhibited, respiration switched from the 3-phase eupneic pattern, which consists of inspiration, postinspiration, and expiration, to gasping-like respiration, which consists of inspiration only. On the other hand, when H2S synthesis was inhibited via cystathionine γ-lyase (CSE) or when H2S synthesis was activated via CBS, eupnea remained unchanged. These results suggest that H2S produced by CBS has crucial roles in maintaining the neuronal network to generate eupnea. The mechanism of respiratory pattern generation might be switched from a network-based system to a pacemaker cell-based system in low H2S conditions.

Signal dynamics of midbrain dopamine neurons during economic decision-making in monkeys.

When we make economic choices, the brain first evaluates available options and then decides whether to choose them. Midbrain dopamine neurons are known to reinforce economic choices through their signal evoked by outcomes after decisions are made. However, although critical internal processing is executed while decisions are being made, little is known about the role of dopamine neurons during this period. We found that dopamine neurons exhibited dynamically changing signals related to the internal processing while rhesus monkeys were making decisions. These neurons encoded the value of an option immediately after it was offered and then gradually changed their activity to represent the animal's upcoming choice. Similar dynamics were observed in the orbitofrontal cortex, a center for economic decision-making, but the value-to-choice signal transition was completed earlier in dopamine neurons. Our findings suggest that dopamine neurons are a key component of the neural network that makes choices from values during ongoing decision-making processes.

Preferential Representation of Past Outcome Information and Future Choice Behavior by Putative Inhibitory Interneurons Rather Than Putative Pyramidal Neurons in the Primate Dorsal Anterior Cingulate Cortex.

The dorsal anterior cingulate cortex (dACC) plays crucial roles in monitoring the outcome of a choice and adjusting a subsequent choice behavior based on the outcome information. In the present study, we investigated how different types of dACC neurons, that is, putative pyramidal neurons and putative inhibitory interneurons, contribute to these processes. We analyzed single-unit database obtained from the dACC in monkeys performing a reversal learning task. The monkey was required to adjust choice behavior from past outcome experiences. Depending on their action potential waveforms, the recorded neurons were classified into putative pyramidal neurons and putative inhibitory interneurons. We found that these neurons do not equally contribute to outcome monitoring and behavioral adjustment. Although both neuron types evenly responded to the current outcome, a larger proportion of putative inhibitory interneurons than putative pyramidal neurons stored the information about the past outcome. The putative inhibitory interneurons further represented choice-related signals more frequently, such as whether the monkey would shift the last choice to an alternative at the next choice opportunity. Our findings suggest that putative inhibitory interneurons, which are thought not to project to brain areas outside the dACC, preferentially transmit signals that would adjust choice behavior based on past outcome experiences.

Conversion of methane to C2 and C3 hydrocarbons over TiO2/ZSM-5 core-shell particles in an electric field.

Catalytic conversion of methane (CH4) to light olefins is motivated by increasing recoverable reserves of methane resources, abundantly available in natural gas, shale gas, and gas hydrates. The development of effective processes for conversion of CH4 to light olefins is still a great challenge. The interface of ZSM-5 zeolite and TiO2 nanoparticles is successfully constructed in their core-shell particles via mechanochemical treatment with high shear stress. The oxidative coupling of methane at a low temperature under application of an electric field may be induced by the O2 activation via electrons running through the surface of TiO2 located at the interface of TiO2 and zeolite particles. Moreover, C3H6 was also produced by the ethylene to propylene (ETP) reaction catalyzed by Brønsted acid sites in the ZSM-5 zeolite within core-shell particles.

Primate Nigrostriatal Dopamine System Regulates Saccadic Response Inhibition.

Animals need to inhibit inappropriate actions that would lead to unwanted outcomes. Although this ability, called response inhibition, is impaired in neurological/psychiatric disorders with dopaminergic dysfunctions, how dopamine regulates response inhibition remains unclear. Here we investigated neuronal signals of the nigrostriatal dopamine system in monkeys performing a saccadic countermanding task. Subsets of dopamine neurons in the substantia nigra and striatal neurons receiving the dopaminergic input were activated when the monkey was required to cancel a planned saccadic eye movement. These activations were stronger when canceling the eye movements was successful compared with failed and were enhanced in demanding trials. The activated dopamine neurons were distributed mainly in the dorsolateral, but not in the ventromedial, part of the nigra. Furthermore, pharmacological blockade of dopaminergic neurotransmission in the striatum dampened the performance of canceling saccadic eye movements. The present findings indicate that disruption of nigrostriatal dopamine signaling causes impairments in response inhibition.