The Brain/MINDS Marmoset Connectivity Resource: An open-access platform for cellular-level tracing and tractography in the primate brain.

The primate brain has unique anatomical characteristics, which translate into advanced cognitive, sensory, and motor abilities. Thus, it is important that we gain insight on its structure to provide a solid basis for models that will clarify function. Here, we report on the implementation and features of the Brain/MINDS Marmoset Connectivity Resource (BMCR), a new open-access platform that provides access to high-resolution anterograde neuronal tracer data in the marmoset brain, integrated to retrograde tracer and tractography data. Unlike other existing image explorers, the BMCR allows visualization of data from different individuals and modalities in a common reference space. This feature, allied to an unprecedented high resolution, enables analyses of features such as reciprocity, directionality, and spatial segregation of connections. The present release of the BMCR focuses on the prefrontal cortex (PFC), a uniquely developed region of the primate brain that is linked to advanced cognition, including the results of 52 anterograde and 164 retrograde tracer injections in the cortex of the marmoset. Moreover, the inclusion of tractography data from diffusion MRI allows systematic analyses of this noninvasive modality against gold-standard cellular connectivity data, enabling detection of false positives and negatives, which provide a basis for future development of tractography. This paper introduces the BMCR image preprocessing pipeline and resources, which include new tools for exploring and reviewing the data.

A whole-brain analysis of functional connectivity and immediate early gene expression reveals functional network shifts after operant learning.

Previous studies of operant learning have addressed neuronal activities and network changes in specific brain areas, such as the striatum, sensorimotor cortex, prefrontal/orbitofrontal cortices, and hippocampus. However, how changes in the whole-brain network are caused by cellular-level changes remains unclear. We, therefore, combined resting-state functional magnetic resonance imaging (rsfMRI) and whole-brain immunohistochemical analysis of early growth response 1 (EGR1), a marker of neural plasticity, to elucidate the temporal and spatial changes in functional networks and underlying cellular processes during operant learning. We used an 11.7-Tesla MRI scanner and whole-brain immunohistochemical analysis of EGR1 in mice during the early and late stages of operant learning. In the operant training, mice received a reward when they pressed left and right buttons alternately, and were punished with a bright light when they made a mistake. A group of mice (n = 22) underwent the first rsfMRI acquisition before behavioral sessions, the second acquisition after 3 training-session-days (early stage), and the third after 21 training-session-days (late stage). Another group of mice (n = 40) was subjected to histological analysis 15 min after the early or late stages of behavioral sessions. Functional connectivity increased between the limbic areas and thalamus or auditory cortex after the early stage of training, and between the motor cortex, sensory cortex, and striatum after the late stage of training. The density of EGR1-immunopositive cells in the motor and sensory cortices increased in both the early and late stages of training, whereas the density in the amygdala increased only in the early stage of training. The subcortical networks centered around the limbic areas that emerged in the early stage have been implicated in rewards, pleasures, and fears. The connectivities between the motor cortex, somatosensory cortex, and striatum that consolidated in the late stage have been implicated in motor learning. Our multimodal longitudinal study successfully revealed temporal shifts in brain regions involved in behavioral learning together with the underlying cellular-level plasticity between these regions. Our study represents a first step towards establishing a new experimental paradigm that combines rsfMRI and immunohistochemistry to link macroscopic and microscopic mechanisms involved in learning.

Intrinsic Rewards for Exploration Without Harm From Observational Noise: A Simulation Study Based on the Free Energy Principle.

In reinforcement learning (RL), artificial agents are trained to maximize numerical rewards by performing tasks. Exploration is essential in RL because agents must discover information before exploiting it. Two rewards encouraging efficient exploration are the entropy of action policy and curiosity for information gain. Entropy is well established in the literature, promoting randomized action selection. Curiosity is defined in a broad variety of ways in literature, promoting discovery of novel experiences. One example, prediction error curiosity, rewards agents for discovering observations they cannot accurately predict. However, such agents may be distracted by unpredictable observational noises known as curiosity traps. Based on the free energy principle (FEP), this letter proposes hidden state curiosity, which rewards agents by the KL divergence between the predictive prior and posterior probabilities of latent variables. We trained six types of agents to navigate mazes: baseline agents without rewards for entropy or curiosity and agents rewarded for entropy and/or either prediction error curiosity or hidden state curiosity. We find that entropy and curiosity result in efficient exploration, especially both employed together. Notably, agents with hidden state curiosity demonstrate resilience against curiosity traps, which hinder agents with prediction error curiosity. This suggests implementing the FEP that may enhance the robustness and generalization of RL models, potentially aligning the learning processes of artificial and biological agents.

Enhancing reinforcement learning models by including direct and indirect pathways improves performance on striatal dependent tasks.

A major advance in understanding learning behavior stems from experiments showing that reward learning requires dopamine inputs to striatal neurons and arises from synaptic plasticity of cortico-striatal synapses. Numerous reinforcement learning models mimic this dopamine-dependent synaptic plasticity by using the reward prediction error, which resembles dopamine neuron firing, to learn the best action in response to a set of cues. Though these models can explain many facets of behavior, reproducing some types of goal-directed behavior, such as renewal and reversal, require additional model components. Here we present a reinforcement learning model, TD2Q, which better corresponds to the basal ganglia with two Q matrices, one representing direct pathway neurons (G) and another representing indirect pathway neurons (N). Unlike previous two-Q architectures, a novel and critical aspect of TD2Q is to update the G and N matrices utilizing the temporal difference reward prediction error. A best action is selected for N and G using a softmax with a reward-dependent adaptive exploration parameter, and then differences are resolved using a second selection step applied to the two action probabilities. The model is tested on a range of multi-step tasks including extinction, renewal, discrimination; switching reward probability learning; and sequence learning. Simulations show that TD2Q produces behaviors similar to rodents in choice and sequence learning tasks, and that use of the temporal difference reward prediction error is required to learn multi-step tasks. Blocking the update rule on the N matrix blocks discrimination learning, as observed experimentally. Performance in the sequence learning task is dramatically improved with two matrices. These results suggest that including additional aspects of basal ganglia physiology can improve the performance of reinforcement learning models, better reproduce animal behaviors, and provide insight as to the role of direct- and indirect-pathway striatal neurons.

The status of MRI databases across the world focused on psychiatric and neurological disorders.

Neuroimaging databases for neuro-psychiatric disorders enable researchers to implement data-driven research approaches by providing access to rich data that can be used to study disease, build and validate machine learning models, and even redefine disease spectra. The importance of sharing large, multi-center, multi-disorder databases has gradually been recognized in order to truly translate brain imaging knowledge into real-world clinical practice. Here, we review MRI databases that share data globally to serve multiple psychiatric or neurological disorders. We found 42 datasets consisting of 23,293 samples from patients with psychiatry and neurological disorders and healthy controls; 1245 samples from mood disorders (major depressive disorder and bipolar disorder), 2015 samples from developmental disorders (autism spectrum disorder, attention-deficit hyperactivity disorder), 675 samples from schizophrenia, 1194 samples from Parkinson's disease, 5865 samples from dementia (including Alzheimer's disease), We recognize that large, multi-center databases should include governance processes that allow data to be shared across national boundaries. Addressing technical and regulatory issues of existing databases can lead to better design and implementation and improve data access for the research community. The current trend toward the development of shareable MRI databases will contribute to a better understanding of the pathophysiology, diagnosis and assessment, and development of early interventions for neuropsychiatric disorders.

A biologically constrained spiking neural network model of the primate basal ganglia with overlapping pathways exhibits action selection.

Action selection has been hypothesized to be a key function of the basal ganglia, yet the nuclei involved, their interactions and the importance of the direct/indirect pathway segregation in such process remain debated. Here, we design a spiking computational model of the monkey basal ganglia derived from a previously published population model, initially parameterized to reproduce electrophysiological activity at rest and to embody as much quantitative anatomical data as possible. As a particular feature, both models exhibit the strong overlap between the direct and indirect pathways that has been documented in non-human primates. Here, we first show how the translation from a population to an individual neuron model was achieved, with the addition of a minimal number of parameters. We then show that our model performs action selection, even though it was built without any assumption on the activity carried out during behaviour. We investigate the mechanisms of this selection through circuit disruptions and found an instrumental role of the off-centre/on-surround structure of the MSN-STN-GPi circuit, as well as of the MSN-MSN and FSI-MSN projections. This validates their potency in enabling selection. We finally study the pervasive centromedian and parafascicular thalamic inputs that reach all basal ganglia nuclei and whose influence is therefore difficult to anticipate. Our model predicts that these inputs modulate the responsiveness of action selection, making them a candidate for the regulation of the speed-accuracy trade-off during decision-making.

Diffusion functional MRI reveals global brain network functional abnormalities driven by targeted local activity in a neuropsychiatric disease mouse model.

Diffusion functional magnetic resonance imaging (DfMRI) has been proposed as an alternative functional imaging method to detect brain activity without confounding hemodynamic effects. Here, taking advantage of this DfMRI feature, we investigated abnormalities of dynamic brain function in a neuropsychiatric disease mouse model (glial glutamate transporter-knockdown mice with obsessive-compulsive disorder [OCD]-related behavior). Our DfMRI approaches consisted of three analyses: resting state brain activity, functional connectivity, and propagation of neural information. We detected hyperactivation and biased connectivity across the cortico-striatal-thalamic circuitry, which is consistent with known blood oxygen-level dependent (BOLD)-fMRI patterns in OCD patients. In addition, we performed ignition-driven mean integration (IDMI) analysis, which combined activity and connectivity analyses, to evaluate neural propagation initiated from brain activation. This analysis revealed an unbalanced distribution of neural propagation initiated from intrinsic local activation to the global network, while these were not detected by the conventional method with BOLD-fMRI. This abnormal function detected by DfMRI was associated with OCD-related behavior. Together, our comprehensive DfMRI approaches can successfully provide information on dynamic brain function in normal and diseased brains.

Sparse kernel canonical correlation analysis for discovery of nonlinear interactions in high-dimensional data.

BACKGROUND: Advance in high-throughput technologies in genomics, transcriptomics, and metabolomics has created demand for bioinformatics tools to integrate high-dimensional data from different sources. Canonical correlation analysis (CCA) is a statistical tool for finding linear associations between different types of information. Previous extensions of CCA used to capture nonlinear associations, such as kernel CCA, did not allow feature selection or capturing of multiple canonical components. Here we propose a novel method, two-stage kernel CCA (TSKCCA) to select appropriate kernels in the framework of multiple kernel learning. RESULTS: TSKCCA first selects relevant kernels based on the HSIC criterion in the multiple kernel learning framework. Weights are then derived by non-negative matrix decomposition with L1 regularization. Using artificial datasets and nutrigenomic datasets, we show that TSKCCA can extract multiple, nonlinear associations among high-dimensional data and multiplicative interactions among variables. CONCLUSIONS: TSKCCA can identify nonlinear associations among high-dimensional data more reliably than previous nonlinear CCA methods.

Consensus Paper: Towards a Systems-Level View of Cerebellar Function: the Interplay Between Cerebellum, Basal Ganglia, and Cortex.

Despite increasing evidence suggesting the cerebellum works in concert with the cortex and basal ganglia, the nature of the reciprocal interactions between these three brain regions remains unclear. This consensus paper gathers diverse recent views on a variety of important roles played by the cerebellum within the cerebello-basal ganglia-thalamo-cortical system across a range of motor and cognitive functions. The paper includes theoretical and empirical contributions, which cover the following topics: recent evidence supporting the dynamical interplay between cerebellum, basal ganglia, and cortical areas in humans and other animals; theoretical neuroscience perspectives and empirical evidence on the reciprocal influences between cerebellum, basal ganglia, and cortex in learning and control processes; and data suggesting possible roles of the cerebellum in basal ganglia movement disorders. Although starting from different backgrounds and dealing with different topics, all the contributors agree that viewing the cerebellum, basal ganglia, and cortex as an integrated system enables us to understand the function of these areas in radically different ways. In addition, there is unanimous consensus between the authors that future experimental and computational work is needed to understand the function of cerebellar-basal ganglia circuitry in both motor and non-motor functions. The paper reports the most advanced perspectives on the role of the cerebellum within the cerebello-basal ganglia-thalamo-cortical system and illustrates other elements of consensus as well as disagreements and open questions in the field.

Distinct neural representation in the dorsolateral, dorsomedial, and ventral parts of the striatum during fixed- and free-choice tasks.

The striatum is a major input site of the basal ganglia, which play an essential role in decision making. Previous studies have suggested that subareas of the striatum have distinct roles: the dorsolateral striatum (DLS) functions in habitual action, the dorsomedial striatum (DMS) in goal-directed actions, and the ventral striatum (VS) in motivation. To elucidate distinctive functions of subregions of the striatum in decision making, we systematically investigated information represented by phasically active neurons in DLS, DMS, and VS. Rats performed two types of choice tasks: fixed- and free-choice tasks. In both tasks, rats were required to perform nose poking to either the left or right hole after cue-tone presentation. A food pellet was delivered probabilistically depending on the presented cue and the selected action. The reward probability was fixed in fixed-choice task and varied in a block-wise manner in free-choice task. We found the following: (1) when rats began the tasks, a majority of VS neurons increased their firing rates and information regarding task type and state value was most strongly represented in VS; (2) during action selection, information of action and action values was most strongly represented in DMS; (3) action-command information (action representation before action selection) was stronger in the fixed-choice task than in the free-choice task in both DLS and DMS; and (4) action-command information was strongest in DLS, particularly when the same choice was repeated. We propose a hypothesis of hierarchical reinforcement learning in the basal ganglia to coherently explain these results.

Parallel Representation of Value-Based and Finite State-Based Strategies in the Ventral and Dorsal Striatum.

Previous theoretical studies of animal and human behavioral learning have focused on the dichotomy of the value-based strategy using action value functions to predict rewards and the model-based strategy using internal models to predict environmental states. However, animals and humans often take simple procedural behaviors, such as the "win-stay, lose-switch" strategy without explicit prediction of rewards or states. Here we consider another strategy, the finite state-based strategy, in which a subject selects an action depending on its discrete internal state and updates the state depending on the action chosen and the reward outcome. By analyzing choice behavior of rats in a free-choice task, we found that the finite state-based strategy fitted their behavioral choices more accurately than value-based and model-based strategies did. When fitted models were run autonomously with the same task, only the finite state-based strategy could reproduce the key feature of choice sequences. Analyses of neural activity recorded from the dorsolateral striatum (DLS), the dorsomedial striatum (DMS), and the ventral striatum (VS) identified significant fractions of neurons in all three subareas for which activities were correlated with individual states of the finite state-based strategy. The signal of internal states at the time of choice was found in DMS, and for clusters of states was found in VS. In addition, action values and state values of the value-based strategy were encoded in DMS and VS, respectively. These results suggest that both the value-based strategy and the finite state-based strategy are implemented in the striatum.

[Brain mechanisms of depression--preliminary evidence from fMRI studies].

Although brain monoamines serotonin, noradrenaline, and dopamine have been repeatedly shown to be linked to depression, it remains unclear how monoamine dysfunction is mechanistically related to symptoms of depression. We hypothesized that imbalances in the networks of regions innervated by monoamines disrupt patients' learning and decision-making abilities, and this disruption could, in turn, lead to symptoms of depression. We have conducted functional magnetic resonance imaging (fMRI) studies on learning and decision-making, mainly focusing on the role of serotonin. Our results suggest that parallel organization for reward prediction at different time scales in the striatum is under differential modulation by serotonin, and that depression is associated with a diminished recruitment of the dorsal striatum, involved in long-term reward prediction. Based on these findings, the brain mechanisms of depression are discussed.

Synergizing habits and goals with variational Bayes.

Behaving efficiently and flexibly is crucial for biological and artificial embodied agents. Behavior is generally classified into two types: habitual (fast but inflexible), and goal-directed (flexible but slow). While these two types of behaviors are typically considered to be managed by two distinct systems in the brain, recent studies have revealed a more sophisticated interplay between them. We introduce a theoretical framework using variational Bayesian theory, incorporating a Bayesian intention variable. Habitual behavior depends on the prior distribution of intention, computed from sensory context without goal-specification. In contrast, goal-directed behavior relies on the goal-conditioned posterior distribution of intention, inferred through variational free energy minimization. Assuming that an agent behaves using a synergized intention, our simulations in vision-based sensorimotor tasks explain the key properties of their interaction as observed in experiments. Our work suggests a fresh perspective on the neural mechanisms of habits and goals, shedding light on future research in decision making.

Respiratory modulation of the heart rate: A potential biomarker of cardiorespiratory function in human.

In recent decade, wearable digital devices have shown potentials for the discovery of novel biomarkers of humans' physiology and behavior. Heart rate (HR) and respiration rate (RR) are most crucial bio-signals in humans' digital phenotyping research. HR is a continuous and non-invasive proxy to autonomic nervous system and ample evidence pinpoints the critical role of respiratory modulation of cardiac function. In the present study, we recorded longitudinal (7 days, 4.63 ± 1.52) HR and RR of 89 freely behaving human subjects (Female: 39, age 57.28 ± 5.67, Male: 50, age 58.48 ± 6.32) and analyzed their dynamics using linear models and information theoretic measures. While HR's linear and nonlinear characteristics were expressed within the plane of the HR-RR directed flow of information (HR→RR - RR→HR), their dynamics were determined by its RR→HR axis. More importantly, RR→HR quantified the effect of alcohol consumption on individuals' cardiorespiratory function independent of their consumed amount of alcohol, thereby signifying the presence of this habit in their daily life activities. The present findings provided evidence for the critical role of the respiratory modulation of HR, which was previously only studied in non-human animals. These results can contribute to humans' phenotyping research by presenting RR→HR as a digital diagnosis/prognosis marker of humans' cardiorespiratory pathology.

Optogenetic activation of dorsal raphe serotonin neurons induces brain-wide activation.

Serotonin is a neuromodulator that affects multiple behavioral and cognitive functions. Nonetheless, how serotonin causes such a variety of effects via brain-wide projections and various receptors remains unclear. Here we measured brain-wide responses to optogenetic stimulation of serotonin neurons in the dorsal raphe nucleus (DRN) of the male mouse brain using functional MRI with an 11.7 T scanner and a cryoprobe. Transient activation of DRN serotonin neurons caused brain-wide activation, including the medial prefrontal cortex, the striatum, and the ventral tegmental area. The same stimulation under anesthesia with isoflurane decreased brain-wide activation, including the hippocampal complex. These brain-wide response patterns can be explained by DRN serotonergic projection topography and serotonin receptor expression profiles, with enhanced weights on 5-HT1 receptors. Together, these results provide insight into the DR serotonergic system, which is consistent with recent discoveries of its functions in adaptive behaviors.

Activation of dorsal raphe serotonin neurons is necessary for waiting for delayed rewards.

The forebrain serotonergic system is a crucial component in the control of impulsive behaviors. We previously reported that the activity of serotonin neurons in the midbrain dorsal raphe nucleus increased when rats performed a task that required them to wait for delayed rewards. However, the causal relationship between serotonin neural activity and the tolerance for the delayed reward remained unclear. Here, we test whether the inhibition of serotonin neural activity by the local application of the 5-HT(1A) receptor agonist 8-hydroxy-2-(di-n-propylamino) tetralin in the dorsal raphe nucleus impairs rats' tolerance for delayed rewards. Rats performed a sequential food-water navigation task that required them to visit food and water sites alternately via a tone site to get rewards at both sites after delays. During the short (2 s) delayed reward condition, the inhibition of serotonin neural activity did not significantly influence the numbers of reward choice errors (nosepoke at an incorrect reward site following a conditioned reinforcer tone), reward wait errors (failure to wait for the delayed rewards), or total trials (sum of reward choice errors, reward wait errors, and acquired rewards). By contrast, during the long (7-11 s) delayed reward condition, the number of wait errors significantly increased while the numbers of total trials and choice errors did not significantly change. These results indicate that the activation of dorsal raphe serotonin neurons is necessary for waiting for long delayed rewards and suggest that elevated serotonin activity facilitates waiting behavior when there is the prospect of forthcoming rewards.

Body and visual instabilities functionally modulate implicit reaching corrections.

Hierarchical brain-information-processing schemes have frequently assumed that the flexible but slow voluntary action modulates a direct sensorimotor process that can quickly generate a reaction in dynamical interaction. Here we show that the quick visuomotor process for manual movement is modulated by postural and visual instability contexts that are related but remote and prior states to manual movements. A preceding unstable postural context significantly enhanced the reflexive manual response induced by a large-field visual motion during hand reaching while the response was evidently weakened by imposing a preceding random-visual-motion context. These modulations are successfully explained by the Bayesian optimal formulation in which the manual response elicited by visual motion is ascribed to the compensatory response to the estimated self-motion affected by the preceding contextual situations. Our findings suggest an implicit and functional mechanism that links the variability and uncertainty of remote states to the quick sensorimotor transformation.

Neuronal Representation of a Working Memory-Based Decision Strategy in the Motor and Prefrontal Cortico-Basal Ganglia Loops.

While animal and human decision strategies are typically explained by model-free and model-based reinforcement learning (RL), their choice sequences often follow simple procedures based on working memory (WM) of past actions and rewards. Here, we address how working memory-based choice strategies, such as win-stay-lose-switch (WSLS), are represented in the prefrontal and motor cortico-basal ganglia loops by simultaneous recording of neuronal activities in the dorsomedial striatum (DMS), the dorsolateral striatum (DLS), the medial prefrontal cortex (mPFC), and the primary motor cortex (M1). In order to compare neuronal representations when rats employ working memory-based strategies, we developed a new task paradigm, a continuous/intermittent choice task, consisting of choice and no-choice trials. While the continuous condition (CC) consisted of only choice trials, in the intermittent condition (IC), a no-choice trial was inserted after each choice trial to disrupt working memory of the previous choice and reward. Behaviors in CC showed high proportions of win-stay and lose-switch choices, which could be regarded as "a noisy WSLS strategy." Poisson regression of neural spikes revealed encoding specifically in CC of the previous action and reward before action choice and prospective coding of WSLS action during action execution. A striking finding was that the DLS and M1 in the motor cortico-basal ganglia loop carry substantial WM information about previous choices, rewards, and their interactions, in addition to current action coding.

Neuronal Population Activity in Macaque Visual Cortices Dynamically Changes through Repeated Fixations in Active Free Viewing.

During free viewing, we move our eyes and fixate on objects to recognize the visual scene of our surroundings. To investigate the neural representation of objects in this process, we studied individual and population neuronal activity in three different visual regions of the brains of macaque monkeys (Macaca fuscata): the primary and secondary visual cortices (V1, V2) and the inferotemporal cortex (IT). We designed a task where the animal freely selected objects in a stimulus image to fixate on while we examined the relationship between spiking activity, the order of fixations, and the fixated objects. We found that activity changed across repeated fixations on the same object in all three recorded areas, with observed reductions in firing rates. Furthermore, the responses of individual neurons became sparser and more selective with individual objects. The population activity for individual objects also became distinct. These results suggest that visual neurons respond dynamically to repeated input stimuli through a smaller number of spikes, thereby allowing for discrimination between individual objects with smaller energy.

Embodied bidirectional simulation of a spiking cortico-basal ganglia-cerebellar-thalamic brain model and a mouse musculoskeletal body model distributed across computers including the supercomputer Fugaku.

Embodied simulation with a digital brain model and a realistic musculoskeletal body model provides a means to understand animal behavior and behavioral change. Such simulation can be too large and complex to conduct on a single computer, and so distributed simulation across multiple computers over the Internet is necessary. In this study, we report our joint effort on developing a spiking brain model and a mouse body model, connecting over the Internet, and conducting bidirectional simulation while synchronizing them. Specifically, the brain model consisted of multiple regions including secondary motor cortex, primary motor and somatosensory cortices, basal ganglia, cerebellum and thalamus, whereas the mouse body model, provided by the Neurorobotics Platform of the Human Brain Project, had a movable forelimb with three joints and six antagonistic muscles to act in a virtual environment. Those were simulated in a distributed manner across multiple computers including the supercomputer Fugaku, which is the flagship supercomputer in Japan, while communicating via Robot Operating System (ROS). To incorporate models written in C/C++ in the distributed simulation, we developed a C++ version of the rosbridge library from scratch, which has been released under an open source license. These results provide necessary tools for distributed embodied simulation, and demonstrate its possibility and usefulness toward understanding animal behavior and behavioral change.