Optogenetic activation of dorsal raphe serotonin neurons induces brain-wide activation.

Serotonin is a neuromodulator that affects multiple behavioral and cognitive functions. Nonetheless, how serotonin causes such a variety of effects via brain-wide projections and various receptors remains unclear. Here we measured brain-wide responses to optogenetic stimulation of serotonin neurons in the dorsal raphe nucleus (DRN) of the male mouse brain using functional MRI with an 11.7 T scanner and a cryoprobe. Transient activation of DRN serotonin neurons caused brain-wide activation, including the medial prefrontal cortex, the striatum, and the ventral tegmental area. The same stimulation under anesthesia with isoflurane decreased brain-wide activation, including the hippocampal complex. These brain-wide response patterns can be explained by DRN serotonergic projection topography and serotonin receptor expression profiles, with enhanced weights on 5-HT1 receptors. Together, these results provide insight into the DR serotonergic system, which is consistent with recent discoveries of its functions in adaptive behaviors.

Synergizing habits and goals with variational Bayes.

Behaving efficiently and flexibly is crucial for biological and artificial embodied agents. Behavior is generally classified into two types: habitual (fast but inflexible), and goal-directed (flexible but slow). While these two types of behaviors are typically considered to be managed by two distinct systems in the brain, recent studies have revealed a more sophisticated interplay between them. We introduce a theoretical framework using variational Bayesian theory, incorporating a Bayesian intention variable. Habitual behavior depends on the prior distribution of intention, computed from sensory context without goal-specification. In contrast, goal-directed behavior relies on the goal-conditioned posterior distribution of intention, inferred through variational free energy minimization. Assuming that an agent behaves using a synergized intention, our simulations in vision-based sensorimotor tasks explain the key properties of their interaction as observed in experiments. Our work suggests a fresh perspective on the neural mechanisms of habits and goals, shedding light on future research in decision making.

Respiratory modulation of the heart rate: A potential biomarker of cardiorespiratory function in human.

In recent decade, wearable digital devices have shown potentials for the discovery of novel biomarkers of humans' physiology and behavior. Heart rate (HR) and respiration rate (RR) are most crucial bio-signals in humans' digital phenotyping research. HR is a continuous and non-invasive proxy to autonomic nervous system and ample evidence pinpoints the critical role of respiratory modulation of cardiac function. In the present study, we recorded longitudinal (7 days, 4.63 ± 1.52) HR and RR of 89 freely behaving human subjects (Female: 39, age 57.28 ± 5.67, Male: 50, age 58.48 ± 6.32) and analyzed their dynamics using linear models and information theoretic measures. While HR's linear and nonlinear characteristics were expressed within the plane of the HR-RR directed flow of information (HR→RR - RR→HR), their dynamics were determined by its RR→HR axis. More importantly, RR→HR quantified the effect of alcohol consumption on individuals' cardiorespiratory function independent of their consumed amount of alcohol, thereby signifying the presence of this habit in their daily life activities. The present findings provided evidence for the critical role of the respiratory modulation of HR, which was previously only studied in non-human animals. These results can contribute to humans' phenotyping research by presenting RR→HR as a digital diagnosis/prognosis marker of humans' cardiorespiratory pathology.

Embodied bidirectional simulation of a spiking cortico-basal ganglia-cerebellar-thalamic brain model and a mouse musculoskeletal body model distributed across computers including the supercomputer Fugaku.

Embodied simulation with a digital brain model and a realistic musculoskeletal body model provides a means to understand animal behavior and behavioral change. Such simulation can be too large and complex to conduct on a single computer, and so distributed simulation across multiple computers over the Internet is necessary. In this study, we report our joint effort on developing a spiking brain model and a mouse body model, connecting over the Internet, and conducting bidirectional simulation while synchronizing them. Specifically, the brain model consisted of multiple regions including secondary motor cortex, primary motor and somatosensory cortices, basal ganglia, cerebellum and thalamus, whereas the mouse body model, provided by the Neurorobotics Platform of the Human Brain Project, had a movable forelimb with three joints and six antagonistic muscles to act in a virtual environment. Those were simulated in a distributed manner across multiple computers including the supercomputer Fugaku, which is the flagship supercomputer in Japan, while communicating via Robot Operating System (ROS). To incorporate models written in C/C++ in the distributed simulation, we developed a C++ version of the rosbridge library from scratch, which has been released under an open source license. These results provide necessary tools for distributed embodied simulation, and demonstrate its possibility and usefulness toward understanding animal behavior and behavioral change.

Neuronal Population Activity in Macaque Visual Cortices Dynamically Changes through Repeated Fixations in Active Free Viewing.

During free viewing, we move our eyes and fixate on objects to recognize the visual scene of our surroundings. To investigate the neural representation of objects in this process, we studied individual and population neuronal activity in three different visual regions of the brains of macaque monkeys (Macaca fuscata): the primary and secondary visual cortices (V1, V2) and the inferotemporal cortex (IT). We designed a task where the animal freely selected objects in a stimulus image to fixate on while we examined the relationship between spiking activity, the order of fixations, and the fixated objects. We found that activity changed across repeated fixations on the same object in all three recorded areas, with observed reductions in firing rates. Furthermore, the responses of individual neurons became sparser and more selective with individual objects. The population activity for individual objects also became distinct. These results suggest that visual neurons respond dynamically to repeated input stimuli through a smaller number of spikes, thereby allowing for discrimination between individual objects with smaller energy.

Enhancing reinforcement learning models by including direct and indirect pathways improves performance on striatal dependent tasks.

A major advance in understanding learning behavior stems from experiments showing that reward learning requires dopamine inputs to striatal neurons and arises from synaptic plasticity of cortico-striatal synapses. Numerous reinforcement learning models mimic this dopamine-dependent synaptic plasticity by using the reward prediction error, which resembles dopamine neuron firing, to learn the best action in response to a set of cues. Though these models can explain many facets of behavior, reproducing some types of goal-directed behavior, such as renewal and reversal, require additional model components. Here we present a reinforcement learning model, TD2Q, which better corresponds to the basal ganglia with two Q matrices, one representing direct pathway neurons (G) and another representing indirect pathway neurons (N). Unlike previous two-Q architectures, a novel and critical aspect of TD2Q is to update the G and N matrices utilizing the temporal difference reward prediction error. A best action is selected for N and G using a softmax with a reward-dependent adaptive exploration parameter, and then differences are resolved using a second selection step applied to the two action probabilities. The model is tested on a range of multi-step tasks including extinction, renewal, discrimination; switching reward probability learning; and sequence learning. Simulations show that TD2Q produces behaviors similar to rodents in choice and sequence learning tasks, and that use of the temporal difference reward prediction error is required to learn multi-step tasks. Blocking the update rule on the N matrix blocks discrimination learning, as observed experimentally. Performance in the sequence learning task is dramatically improved with two matrices. These results suggest that including additional aspects of basal ganglia physiology can improve the performance of reinforcement learning models, better reproduce animal behaviors, and provide insight as to the role of direct- and indirect-pathway striatal neurons.

The Brain/MINDS Marmoset Connectivity Resource: An open-access platform for cellular-level tracing and tractography in the primate brain.

The primate brain has unique anatomical characteristics, which translate into advanced cognitive, sensory, and motor abilities. Thus, it is important that we gain insight on its structure to provide a solid basis for models that will clarify function. Here, we report on the implementation and features of the Brain/MINDS Marmoset Connectivity Resource (BMCR), a new open-access platform that provides access to high-resolution anterograde neuronal tracer data in the marmoset brain, integrated to retrograde tracer and tractography data. Unlike other existing image explorers, the BMCR allows visualization of data from different individuals and modalities in a common reference space. This feature, allied to an unprecedented high resolution, enables analyses of features such as reciprocity, directionality, and spatial segregation of connections. The present release of the BMCR focuses on the prefrontal cortex (PFC), a uniquely developed region of the primate brain that is linked to advanced cognition, including the results of 52 anterograde and 164 retrograde tracer injections in the cortex of the marmoset. Moreover, the inclusion of tractography data from diffusion MRI allows systematic analyses of this noninvasive modality against gold-standard cellular connectivity data, enabling detection of false positives and negatives, which provide a basis for future development of tractography. This paper introduces the BMCR image preprocessing pipeline and resources, which include new tools for exploring and reviewing the data.

Neuronal Representation of a Working Memory-Based Decision Strategy in the Motor and Prefrontal Cortico-Basal Ganglia Loops.

While animal and human decision strategies are typically explained by model-free and model-based reinforcement learning (RL), their choice sequences often follow simple procedures based on working memory (WM) of past actions and rewards. Here, we address how working memory-based choice strategies, such as win-stay-lose-switch (WSLS), are represented in the prefrontal and motor cortico-basal ganglia loops by simultaneous recording of neuronal activities in the dorsomedial striatum (DMS), the dorsolateral striatum (DLS), the medial prefrontal cortex (mPFC), and the primary motor cortex (M1). In order to compare neuronal representations when rats employ working memory-based strategies, we developed a new task paradigm, a continuous/intermittent choice task, consisting of choice and no-choice trials. While the continuous condition (CC) consisted of only choice trials, in the intermittent condition (IC), a no-choice trial was inserted after each choice trial to disrupt working memory of the previous choice and reward. Behaviors in CC showed high proportions of win-stay and lose-switch choices, which could be regarded as "a noisy WSLS strategy." Poisson regression of neural spikes revealed encoding specifically in CC of the previous action and reward before action choice and prospective coding of WSLS action during action execution. A striking finding was that the DLS and M1 in the motor cortico-basal ganglia loop carry substantial WM information about previous choices, rewards, and their interactions, in addition to current action coding.

Multi-modal brain magnetic resonance imaging database covering marmosets with a wide age range.

Magnetic resonance imaging (MRI) is a non-invasive neuroimaging technique that is useful for identifying normal developmental and aging processes and for data sharing. Marmosets have a relatively shorter life expectancy than other primates, including humans, because they grow and age faster. Therefore, the common marmoset model is effective in aging research. The current study investigated the aging process of the marmoset brain and provided an open MRI database of marmosets across a wide age range. The Brain/MINDS Marmoset Brain MRI Dataset contains brain MRI information from 216 marmosets ranging in age from 1 and 10 years. At the time of its release, it is the largest public dataset in the world. It also includes multi-contrast MRI images. In addition, 91 of 216 animals have corresponding high-resolution ex vivo MRI datasets. Our MRI database, available at the Brain/MINDS Data Portal, might help to understand the effects of various factors, such as age, sex, body size, and fixation, on the brain. It can also contribute to and accelerate brain science studies worldwide.

Body and visual instabilities functionally modulate implicit reaching corrections.

Hierarchical brain-information-processing schemes have frequently assumed that the flexible but slow voluntary action modulates a direct sensorimotor process that can quickly generate a reaction in dynamical interaction. Here we show that the quick visuomotor process for manual movement is modulated by postural and visual instability contexts that are related but remote and prior states to manual movements. A preceding unstable postural context significantly enhanced the reflexive manual response induced by a large-field visual motion during hand reaching while the response was evidently weakened by imposing a preceding random-visual-motion context. These modulations are successfully explained by the Bayesian optimal formulation in which the manual response elicited by visual motion is ascribed to the compensatory response to the estimated self-motion affected by the preceding contextual situations. Our findings suggest an implicit and functional mechanism that links the variability and uncertainty of remote states to the quick sensorimotor transformation.

A Spiking Neural Network Builder for Systematic Data-to-Model Workflow.

In building biological neural network models, it is crucial to efficiently convert diverse anatomical and physiological data into parameters of neurons and synapses and to systematically estimate unknown parameters in reference to experimental observations. Web-based tools for systematic model building can improve the transparency and reproducibility of computational models and can facilitate collaborative model building, validation, and evolution. Here, we present a framework to support collaborative data-driven development of spiking neural network (SNN) models based on the Entity-Relationship (ER) data description commonly used in large-scale business software development. We organize all data attributes, including species, brain regions, neuron types, projections, neuron models, and references as tables and relations within a database management system (DBMS) and provide GUI interfaces for data registration and visualization. This allows a robust "business-oriented" data representation that supports collaborative model building and traceability of source information for every detail of a model. We tested this data-to-model framework in cortical and striatal network models by successfully combining data from papers with existing neuron and synapse models and by generating NEST simulation codes for various network sizes. Our framework also helps to check data integrity and consistency and data comparisons across species. The framework enables the modeling of any region of the brain and is being deployed to support the integration of anatomical and physiological datasets from the brain/MINDS project for systematic SNN modeling of the marmoset brain.

Deploying and Optimizing Embodied Simulations of Large-Scale Spiking Neural Networks on HPC Infrastructure.

Simulating the brain-body-environment trinity in closed loop is an attractive proposal to investigate how perception, motor activity and interactions with the environment shape brain activity, and vice versa. The relevance of this embodied approach, however, hinges entirely on the modeled complexity of the various simulated phenomena. In this article, we introduce a software framework that is capable of simulating large-scale, biologically realistic networks of spiking neurons embodied in a biomechanically accurate musculoskeletal system that interacts with a physically realistic virtual environment. We deploy this framework on the high performance computing resources of the EBRAINS research infrastructure and we investigate the scaling performance by distributing computation across an increasing number of interconnected compute nodes. Our architecture is based on requested compute nodes as well as persistent virtual machines; this provides a high-performance simulation environment that is accessible to multi-domain users without expert knowledge, with a view to enable users to instantiate and control simulations at custom scale via a web-based graphical user interface. Our simulation environment, entirely open source, is based on the Neurorobotics Platform developed in the context of the Human Brain Project, and the NEST simulator. We characterize the capabilities of our parallelized architecture for large-scale embodied brain simulations through two benchmark experiments, by investigating the effects of scaling compute resources on performance defined in terms of experiment runtime, brain instantiation and simulation time. The first benchmark is based on a large-scale balanced network, while the second one is a multi-region embodied brain simulation consisting of more than a million neurons and a billion synapses. Both benchmarks clearly show how scaling compute resources improves the aforementioned performance metrics in a near-linear fashion. The second benchmark in particular is indicative of both the potential and limitations of a highly distributed simulation in terms of a trade-off between computation speed and resource cost. Our simulation architecture is being prepared to be accessible for everyone as an EBRAINS service, thereby offering a community-wide tool with a unique workflow that should provide momentum to the investigation of closed-loop embodiment within the computational neuroscience community.

Social impact and governance of AI and neurotechnologies.

Advances in artificial intelligence (AI) and brain science are going to have a huge impact on society. While technologies based on those advances can provide enormous social benefits, adoption of new technologies poses various risks. This article first reviews the co-evolution of AI and brain science and the benefits of brain-inspired AI in sustainability, healthcare, and scientific discoveries. We then consider possible risks from those technologies, including intentional abuse, autonomous weapons, cognitive enhancement by brain-computer interfaces, insidious effects of social media, inequity, and enfeeblement. We also discuss practical ways to bring ethical principles into practice. One proposal is to stop giving explicit goals to AI agents and to enable them to keep learning human preferences. Another is to learn from democratic mechanisms that evolved in human society to avoid over-consolidation of power. Finally, we emphasize the importance of open discussions not only by experts, but also including a diverse array of lay opinions.

A whole brain probabilistic generative model: Toward realizing cognitive architectures for developmental robots.

Building a human-like integrative artificial cognitive system, that is, an artificial general intelligence (AGI), is the holy grail of the artificial intelligence (AI) field. Furthermore, a computational model that enables an artificial system to achieve cognitive development will be an excellent reference for brain and cognitive science. This paper describes an approach to develop a cognitive architecture by integrating elemental cognitive modules to enable the training of the modules as a whole. This approach is based on two ideas: (1) brain-inspired AI, learning human brain architecture to build human-level intelligence, and (2) a probabilistic generative model (PGM)-based cognitive architecture to develop a cognitive system for developmental robots by integrating PGMs. The proposed development framework is called a whole brain PGM (WB-PGM), which differs fundamentally from existing cognitive architectures in that it can learn continuously through a system based on sensory-motor information. In this paper, we describe the rationale for WB-PGM, the current status of PGM-based elemental cognitive modules, their relationship with the human brain, the approach to the integration of the cognitive modules, and future challenges. Our findings can serve as a reference for brain studies. As PGMs describe explicit informational relationships between variables, WB-PGM provides interpretable guidance from computational sciences to brain science. By providing such information, researchers in neuroscience can provide feedback to researchers in AI and robotics on what the current models lack with reference to the brain. Further, it can facilitate collaboration among researchers in neuro-cognitive sciences as well as AI and robotics.

Forward and inverse reinforcement learning sharing network weights and hyperparameters.

This paper proposes model-free imitation learning named Entropy-Regularized Imitation Learning (ERIL) that minimizes the reverse Kullback-Leibler (KL) divergence. ERIL combines forward and inverse reinforcement learning (RL) under the framework of an entropy-regularized Markov decision process. An inverse RL step computes the log-ratio between two distributions by evaluating two binary discriminators. The first discriminator distinguishes the state generated by the forward RL step from the expert's state. The second discriminator, which is structured by the theory of entropy regularization, distinguishes the state-action-next-state tuples generated by the learner from the expert ones. One notable feature is that the second discriminator shares hyperparameters with the forward RL, which can be used to control the discriminator's ability. A forward RL step minimizes the reverse KL estimated by the inverse RL step. We show that minimizing the reverse KL divergence is equivalent to finding an optimal policy. Our experimental results on MuJoCo-simulated environments and vision-based reaching tasks with a robotic arm show that ERIL is more sample-efficient than the baseline methods. We apply the method to human behaviors that perform a pole-balancing task and describe how the estimated reward functions show how every subject achieves her goal.

A biologically constrained spiking neural network model of the primate basal ganglia with overlapping pathways exhibits action selection.

Action selection has been hypothesized to be a key function of the basal ganglia, yet the nuclei involved, their interactions and the importance of the direct/indirect pathway segregation in such process remain debated. Here, we design a spiking computational model of the monkey basal ganglia derived from a previously published population model, initially parameterized to reproduce electrophysiological activity at rest and to embody as much quantitative anatomical data as possible. As a particular feature, both models exhibit the strong overlap between the direct and indirect pathways that has been documented in non-human primates. Here, we first show how the translation from a population to an individual neuron model was achieved, with the addition of a minimal number of parameters. We then show that our model performs action selection, even though it was built without any assumption on the activity carried out during behaviour. We investigate the mechanisms of this selection through circuit disruptions and found an instrumental role of the off-centre/on-surround structure of the MSN-STN-GPi circuit, as well as of the MSN-MSN and FSI-MSN projections. This validates their potency in enabling selection. We finally study the pervasive centromedian and parafascicular thalamic inputs that reach all basal ganglia nuclei and whose influence is therefore difficult to anticipate. Our model predicts that these inputs modulate the responsiveness of action selection, making them a candidate for the regulation of the speed-accuracy trade-off during decision-making.

Optimization and validation of diffusion MRI-based fiber tracking with neural tracer data as a reference.

Diffusion-weighted magnetic resonance imaging (dMRI) allows non-invasive investigation of whole-brain connectivity, which can reveal the brain's global network architecture and also abnormalities involved in neurological and mental disorders. However, the reliability of connection inferences from dMRI-based fiber tracking is still debated, due to low sensitivity, dominance of false positives, and inaccurate and incomplete reconstruction of long-range connections. Furthermore, parameters of tracking algorithms are typically tuned in a heuristic way, which leaves room for manipulation of an intended result. Here we propose a general data-driven framework to optimize and validate parameters of dMRI-based fiber tracking algorithms using neural tracer data as a reference. Japan's Brain/MINDS Project provides invaluable datasets containing both dMRI and neural tracer data from the same primates. A fundamental difference when comparing dMRI-based tractography and neural tracer data is that the former cannot specify the direction of connectivity; therefore, evaluating the fitting of dMRI-based tractography becomes challenging. The framework implements multi-objective optimization based on the non-dominated sorting genetic algorithm II. Its performance is examined in two experiments using data from ten subjects for optimization and six for testing generalization. The first uses a seed-based tracking algorithm, iFOD2, and objectives for sensitivity and specificity of region-level connectivity. The second uses a global tracking algorithm and a more refined set of objectives: distance-weighted coverage, true/false positive ratio, projection coincidence, and commissural passage. In both experiments, with optimized parameters compared to default parameters, fiber tracking performance was significantly improved in coverage and fiber length. Improvements were more prominent using global tracking with refined objectives, achieving an average fiber length from 10 to 17 mm, voxel-wise coverage of axonal tracts from 0.9 to 15%, and the correlation of target areas from 40 to 68%, while minimizing false positives and impossible cross-hemisphere connections. Optimized parameters showed good generalization capability for test brain samples in both experiments, demonstrating the flexible applicability of our framework to different tracking algorithms and objectives. These results indicate the importance of data-driven adjustment of fiber tracking algorithms and support the validity of dMRI-based tractography, if appropriate adjustments are employed.

Serotonergic projections to the orbitofrontal and medial prefrontal cortices differentially modulate waiting for future rewards.

Optogenetic activation of serotonergic neurons in the dorsal raphe nucleus (DRN) enhances patience when waiting for future rewards, and this effect is maximized by both high probability and high timing uncertainty of reward. Here, we explored which serotonin projection areas contribute to these effects using optogenetic axon terminal stimulation. We found that serotonin stimulation in the orbitofrontal cortex (OFC) is nearly as effective as that in the DRN for promoting waiting, while in the nucleus accumbens, it does not promote waiting. We also found that serotonin stimulation in the medial prefrontal cortex (mPFC) promotes waiting only when the timing of future rewards is uncertain. Our Bayesian decision model of waiting assumed that the OFC and mPFC calculate the posterior probability of reward delivery separately. These results suggest that serotonin in the mPFC affects evaluation of time committed, while serotonin in the OFC is responsible for overall valuation of delayed rewards.

Diffusion functional MRI reveals global brain network functional abnormalities driven by targeted local activity in a neuropsychiatric disease mouse model.

Diffusion functional magnetic resonance imaging (DfMRI) has been proposed as an alternative functional imaging method to detect brain activity without confounding hemodynamic effects. Here, taking advantage of this DfMRI feature, we investigated abnormalities of dynamic brain function in a neuropsychiatric disease mouse model (glial glutamate transporter-knockdown mice with obsessive-compulsive disorder [OCD]-related behavior). Our DfMRI approaches consisted of three analyses: resting state brain activity, functional connectivity, and propagation of neural information. We detected hyperactivation and biased connectivity across the cortico-striatal-thalamic circuitry, which is consistent with known blood oxygen-level dependent (BOLD)-fMRI patterns in OCD patients. In addition, we performed ignition-driven mean integration (IDMI) analysis, which combined activity and connectivity analyses, to evaluate neural propagation initiated from brain activation. This analysis revealed an unbalanced distribution of neural propagation initiated from intrinsic local activation to the global network, while these were not detected by the conventional method with BOLD-fMRI. This abnormal function detected by DfMRI was associated with OCD-related behavior. Together, our comprehensive DfMRI approaches can successfully provide information on dynamic brain function in normal and diseased brains.