Neural Population Dynamics Underlying Expected Value Computation.

Computation of expected values (i.e., probability × magnitude) seems to be a dynamic integrative process performed by the brain for efficient economic behavior. However, neural dynamics underlying this computation is largely unknown. Using lottery tasks in monkeys (Macaca mulatta, male; Macaca fuscata, female), we examined (1) whether four core reward-related brain regions detect and integrate probability and magnitude cued by numerical symbols and (2) whether these brain regions have distinct dynamics in the integrative process. Extraction of the mechanistic structure of neural population signals demonstrated that expected value signals simultaneously arose in the central orbitofrontal cortex (cOFC; medial part of area 13) and ventral striatum (VS). Moreover, these signals were incredibly stable compared with weak and/or fluctuating signals in the dorsal striatum and medial OFC. Temporal dynamics of these stable expected value signals were unambiguously distinct: sharp and gradual signal evolutions in the cOFC and VS, respectively. These intimate dynamics suggest that the cOFC and VS compute the expected values with unique time constants, as distinct, partially overlapping processes.SIGNIFICANCE STATEMENT Our results differ from those of earlier studies suggesting that many reward-related regions in the brain signal probability and/or magnitude and provide a mechanistic structure for expected value computation employed in multiple neural populations. A central part of the orbitofrontal cortex (cOFC) and ventral striatum (VS) can simultaneously detect and integrate probability and magnitude into an expected value. Our empirical study on these neural population dynamics raises a possibility that the cOFC and VS cooperate on this computation with unique time constants as distinct, partially overlapping processes.

Complex field reconstruction of optical OFDM signals based on temporal transport-of-intensity equation.

Field reconstruction of optical orthogonal frequency division multiplexing (OFDM) signals in a direct-detection (DD) receiver by using temporal transport-of-intensity equation (TIE) is studied. It is shown that in this DD field reconstruction scheme, better BER performance is obtained by using the OFDM modulation than by using single carrier signals especially when low-frequency subcarriers of OFDM signals are not used. How phase errors are generated in solving the TIE is analyzed and it is shown that the process of integration with respect to time gives rise to low-frequency errors that degrade the performance. The DD phase retrieval scheme has favorable features that the solution is non-iterative and it allows using double side-band signals. Nevertheless, it is shown that the scheme has high sensitivity to electrical noise in detection and requires relatively high carrier to signal power ratio, to which further studies are expected to be devoted.

Nonhuman Primate Optogenetics: Current Status and Future Prospects.

Nonhuman primates (NHPs) have widely and crucially been utilized as model animals for understanding various higher brain functions and neurological disorders since their behavioral actions mimic both normal and disease states in humans. To know about how such behaviors emerge from the functions and dysfunctions of complex neural networks, it is essential to define the role of a particular pathway or neuron-type constituting these networks. Optogenetics is a potential technique that enables analyses of network functions. However, because of the large size of the NHP brain and the difficulty in creating genetically modified animal models, this technique is currently still hard to apply effectively and efficiently to NHP neuroscience. In this article, we focus on the issues that should be overcome for the development of NHP optogenetics, with special reference to the gene introduction strategy. We review the recent breakthroughs that have been made in NHP optogenetics to address these issues and discuss future prospects regarding more effective and efficient approaches to successful optogenetic manipulation in NHPs.

Preferential Representation of Past Outcome Information and Future Choice Behavior by Putative Inhibitory Interneurons Rather Than Putative Pyramidal Neurons in the Primate Dorsal Anterior Cingulate Cortex.

The dorsal anterior cingulate cortex (dACC) plays crucial roles in monitoring the outcome of a choice and adjusting a subsequent choice behavior based on the outcome information. In the present study, we investigated how different types of dACC neurons, that is, putative pyramidal neurons and putative inhibitory interneurons, contribute to these processes. We analyzed single-unit database obtained from the dACC in monkeys performing a reversal learning task. The monkey was required to adjust choice behavior from past outcome experiences. Depending on their action potential waveforms, the recorded neurons were classified into putative pyramidal neurons and putative inhibitory interneurons. We found that these neurons do not equally contribute to outcome monitoring and behavioral adjustment. Although both neuron types evenly responded to the current outcome, a larger proportion of putative inhibitory interneurons than putative pyramidal neurons stored the information about the past outcome. The putative inhibitory interneurons further represented choice-related signals more frequently, such as whether the monkey would shift the last choice to an alternative at the next choice opportunity. Our findings suggest that putative inhibitory interneurons, which are thought not to project to brain areas outside the dACC, preferentially transmit signals that would adjust choice behavior based on past outcome experiences.

The Lateral Habenula Circuitry: Reward Processing and Cognitive Control.

There has been a growing interest in understanding the role of the lateral habenula (LHb) in reward processing, affect regulation, and goal-directed behaviors. The LHb gets major inputs from the habenula-projecting globus pallidus and the mPFC, sending its efferents to the dopaminergic VTA and SNc, serotonergic dorsal raphe nuclei, and the GABAergic rostromedial tegmental nucleus. Recent studies have made advances in our understanding of the LHb circuit organization, yet the precise mechanisms of its involvement in complex behaviors are largely unknown. To begin to address this unresolved question, we present here emerging cross-species perspectives with a goal to provide a more refined understanding of the role of the LHb circuits in reward and cognition. We begin by highlighting recent findings from rodent experiments using optogenetics, electrophysiology, molecular, pharmacology, and tracing techniques that reveal diverse neural phenotypes in the LHb circuits that may underlie previously undescribed behavioral functions. We then discuss results from electrophysiological studies in macaques that suggest that the LHb cooperates with the anterior cingulate cortex to monitor action outcomes and signal behavioral adjustment. Finally, we provide an integrated summary of cross-species findings and discuss how further research on the connectivity, neural signaling, and physiology of the LHb circuits can deepen our understanding of the role of the LHb in normal and maladaptive behaviors associated with mental illnesses and drug abuse.

Dopamine signals and physiological origin of cognitive dysfunction in Parkinson's disease.

The pathological hallmark of Parkinson's disease (PD) is the degeneration of midbrain dopamine neurons. Cognitive dysfunction is a feature of PD patients even at the early stages of the disease. Electrophysiological studies on dopamine neurons in awake animals provide contradictory accounts of the role of dopamine. These studies have established that dopamine neurons convey a unique signal associated with rewards rather than cognitive functions. Emphasizing their role in reward processing leads to difficulty in developing hypothesis as to how cognitive impairments in PD are associated with the degeneration of dopamine circuitry. A hint to resolve this contradiction came from recent electrophysiological studies reporting that dopamine neurons transmit more diverse signals than previously thought. These studies suggest that dopamine neurons are divided into at least two functional subgroups, one signaling "motivational value" and the other signaling "salience." The former subgroup fits well with the conventional reward theory, whereas the latter subgroup has been shown to transmit signals related to salient but non-rewarding experiences such as aversive stimulations and cognitively demanding situations. This article reviews recent advances in understanding the non-reward functions of dopamine, and then discusses the possibility that cognitive dysfunction in PD is at least partially caused by the degeneration of the dopamine neuron subgroup signaling the salience of events in the environment.

Two types of dopamine neuron distinctly convey positive and negative motivational signals.

Midbrain dopamine neurons are activated by reward or sensory stimuli predicting reward. These excitatory responses increase as the reward value increases. This response property has led to a hypothesis that dopamine neurons encode value-related signals and are inhibited by aversive events. Here we show that this is true only for a subset of dopamine neurons. We recorded the activity of dopamine neurons in monkeys (Macaca mulatta) during a Pavlovian procedure with appetitive and aversive outcomes (liquid rewards and airpuffs directed at the face, respectively). We found that some dopamine neurons were excited by reward-predicting stimuli and inhibited by airpuff-predicting stimuli, as the value hypothesis predicts. However, a greater number of dopamine neurons were excited by both of these stimuli, inconsistent with the hypothesis. Some dopamine neurons were also excited by both rewards and airpuffs themselves, especially when they were unpredictable. Neurons excited by the airpuff-predicting stimuli were located more dorsolaterally in the substantia nigra pars compacta, whereas neurons inhibited by the stimuli were located more ventromedially, some in the ventral tegmental area. A similar anatomical difference was observed for their responses to actual airpuffs. These findings suggest that different groups of dopamine neurons convey motivational signals in distinct manners.

Information-rate analysis of a fiber-optic transmission system including 2R signal regenerators.

Performance of a single-channel fiber-optic transmission system in which signal regenerators are periodically inserted is analyzed in terms of information rate (IR) considering channel memory. Limitations in using regenerators in a system having non-zero residual dispersion between the regenerators are discussed. It is shown that a type of signal impairment caused by the interaction between the transmission-fiber dispersion and the regenerator nonlinearity is pattern-dependent and will be mitigated by the use of sequence estimation after detection at the receiver.

Lateral habenula as a source of negative reward signals in dopamine neurons.

Midbrain dopamine neurons are key components of the brain's reward system, which is thought to guide reward-seeking behaviours. Although recent studies have shown how dopamine neurons respond to rewards and sensory stimuli predicting reward, it is unclear which parts of the brain provide dopamine neurons with signals necessary for these actions. Here we show that the primate lateral habenula, part of the structure called the epithalamus, is a major candidate for a source of negative reward-related signals in dopamine neurons. We recorded the activity of habenula neurons and dopamine neurons while rhesus monkeys were performing a visually guided saccade task with positionally biased reward outcomes. Many habenula neurons were excited by a no-reward-predicting target and inhibited by a reward-predicting target. In contrast, dopamine neurons were excited and inhibited by reward-predicting and no-reward-predicting targets, respectively. Each time the rewarded and unrewarded positions were reversed, both habenula and dopamine neurons reversed their responses as the bias in saccade latency reversed. In unrewarded trials, the excitation of habenula neurons started earlier than the inhibition of dopamine neurons. Furthermore, weak electrical stimulation of the lateral habenula elicited strong inhibitions in dopamine neurons. These results suggest that the inhibitory input from the lateral habenula plays an important role in determining the reward-related activity of dopamine neurons.

Hydrogen sulfide production in the medullary respiratory center modulates the neural circuit for respiratory pattern and rhythm generations.

Hydrogen sulfide (H2S), which is synthesized in the brain, modulates the neural network. Recently, the importance of H2S in respiratory central pattern generation has been recognized, yet the function of H2S in the medullary respiratory network remains poorly understood. Here, to evaluate the functional roles of H2S in the medullary respiratory network, the Bötzinger complex (BötC), the pre-Bötzinger complex (preBötC), and the rostral ventral respiratory group (rVRG), we observed the effects of inhibition of H2S synthesis at each region on the respiratory pattern by using an in situ arterially perfused preparation of decerebrated male rats. After microinjection of an H2S synthase inhibitor, cystathionine ß-synthase, into the BötC or preBötC, the amplitude of the inspiratory burst decreased and the respiratory frequency increased according to shorter expiration and inspiration, respectively. These alterations were abolished or attenuated in the presence of a blocker of excitatory synaptic transmission. On the other hand, after microinjection of the H2S synthase inhibitor into the rVRG, the amplitude of the inspiratory burst was attenuated, and the respiratory frequency decreased, which was the opposite effect to those obtained by blockade of inhibitory synaptic transmission at the rVRG. These results suggest that H2S synthesized in the BötC and preBötC functions to limit respiratory frequency by sustaining the respiratory phase and to maintain the power of inspiration. In contrast, H2S synthesized in the rVRG functions to promote respiratory frequency by modulating the interval of inspiration and to maintain the power of inspiration. The underlying mechanism might facilitate excitatory synaptic transmission and/or attenuate inhibitory synaptic transmission.

Distinct roles of the orbitofrontal cortex, ventral striatum, and dopamine neurons in counterfactual thinking of decision outcomes.

Individuals often assess past decisions by comparing what was gained with what would have been gained had they acted differently. Thoughts of past alternatives that counter what actually happened are called "counterfactuals." Recent theories emphasize the role of the prefrontal cortex in processing counterfactual outcomes in decision-making, although how subcortical regions contribute to this process remains to be elucidated. Here we report a clear distinction among the roles of the orbitofrontal cortex, ventral striatum and midbrain dopamine neurons in processing counterfactual outcomes in monkeys. Our findings suggest that actually gained and counterfactual outcome signals are both processed in the cortico-subcortical network constituted by these regions but in distinct manners and integrated only in the orbitofrontal cortex in a way to compare these outcomes. This study extends the prefrontal theory of counterfactual thinking and provides key insights regarding how the prefrontal cortex cooperates with subcortical regions to make decisions using counterfactual information.

Dynamic prospect theory: Two core decision theories coexist in the gambling behavior of monkeys and humans.

Research in the multidisciplinary field of neuroeconomics has mainly been driven by two influential theories regarding human economic choice: prospect theory, which describes decision-making under risk, and reinforcement learning theory, which describes learning for decision-making. We hypothesized that these two distinct theories guide decision-making in a comprehensive manner. Here, we propose and test a decision-making theory under uncertainty that combines these highly influential theories. Collecting many gambling decisions from laboratory monkeys allowed for reliable testing of our model and revealed a systematic violation of prospect theory's assumption that probability weighting is static. Using the same experimental paradigm in humans, substantial similarities between these species were uncovered by various econometric analyses of our dynamic prospect theory model, which incorporates decision-by-decision learning dynamics of prediction errors into static prospect theory. Our model provides a unified theoretical framework for exploring a neurobiological model of economic choice in human and nonhuman primates.

Stable Neural Population Dynamics in the Regression Subspace for Continuous and Categorical Task Parameters in Monkeys.

Neural population dynamics provide a key computational framework for understanding information processing in the sensory, cognitive, and motor functions of the brain. They systematically depict complex neural population activity, dominated by strong temporal dynamics as trajectory geometry in a low-dimensional neural space. However, neural population dynamics are poorly related to the conventional analytical framework of single-neuron activity, the rate-coding regime that analyzes firing rate modulations using task parameters. To link the rate-coding and dynamic models, we developed a variant of state-space analysis in the regression subspace, which describes the temporal structures of neural modulations using continuous and categorical task parameters. In macaque monkeys, using two neural population datasets containing either of two standard task parameters, continuous and categorical, we revealed that neural modulation structures are reliably captured by these task parameters in the regression subspace as trajectory geometry in a lower dimension. Furthermore, we combined the classical optimal-stimulus response analysis (usually used in rate-coding analysis) with the dynamic model and found that the most prominent modulation dynamics in the lower dimension were derived from these optimal responses. Using those analyses, we successfully extracted geometries for both task parameters that formed a straight geometry, suggesting that their functional relevance is characterized as a unidimensional feature in their neural modulation dynamics. Collectively, our approach bridges neural modulation in the rate-coding model and the dynamic system, and provides researchers with a significant advantage in exploring the temporal structure of neural modulations for pre-existing datasets.

A Noninvasive Method for Monitoring Breathing Patterns in Nonhuman Primates Using a Nasal Thermosensor.

Respiration is strongly linked to internal states such as arousal, emotion, and even cognitive processes and provides objective biological information to estimate these states in humans and animals. However, the measurement of respiration has not been established in macaque monkeys, which have been widely used as model animals for understanding various higher brain functions. In the present study, we developed a method to monitor the respiration of behaving monkeys. We first measured the temperature of their nasal breathing, which changes between inspiration and expiration phases, in an anesthetized condition and estimated the respiration pattern. We compared the estimated pattern with that obtained by a conventional chest band method that has been used in humans and applied to anesthetized, but not behaving, monkeys. These respiration patterns matched well, suggesting that the measurement of nasal air temperature can be used to monitor the respiration of monkeys. Furthermore, we confirmed that the respiration frequency in behaving monkeys monitored by the measurement of nasal air temperature was not affected by the orofacial movement of licking to obtain the liquid reward. We next examined the frequency of respiration when they listened to music or white noise. The respiratory frequency was higher when the monkeys listened to music than the noise. This result is consistent with a phenomenon in humans and indicates the accuracy of our monitoring method. These data suggest that the measurement of nasal air temperature enables us to monitor the respiration of behaving monkeys and thereby estimate their internal states.

A neuronal prospect theory model in the brain reward circuitry.

Prospect theory, arguably the most prominent theory of choice, is an obvious candidate for neural valuation models. How the activity of individual neurons, a possible computational unit, obeys prospect theory remains unknown. Here, we show, with theoretical accuracy equivalent to that of human neuroimaging studies, that single-neuron activity in four core reward-related cortical and subcortical regions represents the subjective valuation of risky gambles in monkeys. The activity of individual neurons in monkeys passively viewing a lottery reflects the desirability of probabilistic rewards parameterized as a multiplicative combination of utility and probability weighting functions, as in the prospect theory framework. The diverse patterns of valuation signals were not localized but distributed throughout most parts of the reward circuitry. A network model aggregating these signals reconstructed the risk preferences and subjective probability weighting revealed by the animals' choices. Thus, distributed neural coding explains the computation of subjective valuations under risk.

Meta-learning, social cognition and consciousness in brains and machines.

The intersection between neuroscience and artificial intelligence (AI) research has created synergistic effects in both fields. While neuroscientific discoveries have inspired the development of AI architectures, new ideas and algorithms from AI research have produced new ways to study brain mechanisms. A well-known example is the case of reinforcement learning (RL), which has stimulated neuroscience research on how animals learn to adjust their behavior to maximize reward. In this review article, we cover recent collaborative work between the two fields in the context of meta-learning and its extension to social cognition and consciousness. Meta-learning refers to the ability to learn how to learn, such as learning to adjust hyperparameters of existing learning algorithms and how to use existing models and knowledge to efficiently solve new tasks. This meta-learning capability is important for making existing AI systems more adaptive and flexible to efficiently solve new tasks. Since this is one of the areas where there is a gap between human performance and current AI systems, successful collaboration should produce new ideas and progress. Starting from the role of RL algorithms in driving neuroscience, we discuss recent developments in deep RL applied to modeling prefrontal cortex functions. Even from a broader perspective, we discuss the similarities and differences between social cognition and meta-learning, and finally conclude with speculations on the potential links between intelligence as endowed by model-based RL and consciousness. For future work we highlight data efficiency, autonomy and intrinsic motivation as key research areas for advancing both fields.

Adaptive delay control for time-interleaved multi-channel amplitude limiter based on saturation of four-wave mixing in a fiber.

An adaptive delay control to maintain time-interleaving condition of multi-channel input signals of all-optical amplitude limiter based on saturation of four-wave mixing (FWM) in a nonlinear fiber is demonstrated. The delay control utilizes as a monitor signal the optical power after the nonlinear fiber at a wavelength that is affected by interchannel FWM in the fiber. When the scheme is applied to 2 x 10 Gbit/s return-to-zero differential phase-shift keying signals where the time separation between the input channels is intentionally changed randomly, the delay control works well and error free detection after transmission is obtained.

Optical filter for highlighting spectral features part I: design and development of the filter for discrimination of human skin with and without an application of cosmetic foundation.

Light reflected from an object's surface contains much information about its physical and chemical properties. Changes in the physical properties of an object are barely detectable in spectra. Conventional trichromatic systems, on the other hand, cannot detect most spectral features because spectral information is compressively represented as trichromatic signals forming a three-dimensional subspace. We propose a method for designing a filter that optically modulates a camera's spectral sensitivity to find an alternative subspace highlighting an object's spectral features more effectively than the original trichromatic space. We designed and developed a filter that detects cosmetic foundations on human face. Results confirmed that the filter can visualize and nondestructively inspect the foundation distribution.

Optical filter highlighting spectral features part II: quantitative measurements of cosmetic foundation and assessment of their spatial distributions under realistic facial conditions.

We previously proposed a filter that could detect cosmetic foundations with high discrimination accuracy [Opt. Express 19, 6020 (2011)]. This study extends the filter's functionality to the quantification of the amount of foundation and applies the filter for the assessment of spatial distributions of foundation under realistic facial conditions. Human faces that are applied with quantitatively controlled amounts of cosmetic foundations were measured using the filter. A calibration curve between pixel values of the image and the amount of foundation was created. The optical filter was applied to visualize spatial foundation distributions under realistic facial conditions, which clearly indicated areas on the face where foundation remained even after cleansing. Results confirm that the proposed filter could visualize and nondestructively inspect the foundation distributions.

Single-unit Recording in Awake Behaving Non-human Primates.

Non-human primates (NHPs) have been widely used as a species model in studies to understand higher brain functions in health and disease. These studies employ specifically designed behavioral tasks in which animal behavior is well-controlled, and record neuronal activity at high spatial and temporal resolutions while animals are performing the tasks. Here, we present a detailed procedure to conduct single-unit recording, which fulfils high spatial and temporal resolutions while macaque monkeys (i.e., widely used NHPs) perform behavioral tasks in a well-controlled manner. This procedure was used in our previous study to investigate the dynamics of neuronal activity during economic decision-making by the monkeys. Monkeys' behavior was quantitated by eye position tracking and button press/release detection. By inserting a microelectrode into the brain, with a grid system in reference to magnetic resonance imaging, we precisely recorded the brain regions. Our experimental system permits rigorous investigation of the link between neuronal activity and behavior.