Deploying and Optimizing Embodied Simulations of Large-Scale Spiking Neural Networks on HPC Infrastructure.

Simulating the brain-body-environment trinity in closed loop is an attractive proposal to investigate how perception, motor activity and interactions with the environment shape brain activity, and vice versa. The relevance of this embodied approach, however, hinges entirely on the modeled complexity of the various simulated phenomena. In this article, we introduce a software framework that is capable of simulating large-scale, biologically realistic networks of spiking neurons embodied in a biomechanically accurate musculoskeletal system that interacts with a physically realistic virtual environment. We deploy this framework on the high performance computing resources of the EBRAINS research infrastructure and we investigate the scaling performance by distributing computation across an increasing number of interconnected compute nodes. Our architecture is based on requested compute nodes as well as persistent virtual machines; this provides a high-performance simulation environment that is accessible to multi-domain users without expert knowledge, with a view to enable users to instantiate and control simulations at custom scale via a web-based graphical user interface. Our simulation environment, entirely open source, is based on the Neurorobotics Platform developed in the context of the Human Brain Project, and the NEST simulator. We characterize the capabilities of our parallelized architecture for large-scale embodied brain simulations through two benchmark experiments, by investigating the effects of scaling compute resources on performance defined in terms of experiment runtime, brain instantiation and simulation time. The first benchmark is based on a large-scale balanced network, while the second one is a multi-region embodied brain simulation consisting of more than a million neurons and a billion synapses. Both benchmarks clearly show how scaling compute resources improves the aforementioned performance metrics in a near-linear fashion. The second benchmark in particular is indicative of both the potential and limitations of a highly distributed simulation in terms of a trade-off between computation speed and resource cost. Our simulation architecture is being prepared to be accessible for everyone as an EBRAINS service, thereby offering a community-wide tool with a unique workflow that should provide momentum to the investigation of closed-loop embodiment within the computational neuroscience community.

Human-scale Brain Simulation via Supercomputer: A Case Study on the Cerebellum.

Performance of supercomputers has been steadily and exponentially increasing for the past 20 years, and is expected to increase further. This unprecedented computational power enables us to build and simulate large-scale neural network models composed of tens of billions of neurons and tens of trillions of synapses with detailed anatomical connections and realistic physiological parameters. Such "human-scale" brain simulation could be considered a milestone in computational neuroscience and even in general neuroscience. Towards this milestone, it is mandatory to introduce modern high-performance computing technology into neuroscience research. In this article, we provide an introductory landscape about large-scale brain simulation on supercomputers from the viewpoints of computational neuroscience and modern high-performance computing technology for specialists in experimental as well as computational neurosciences. This introduction to modeling and simulation methods is followed by a review of various representative large-scale simulation studies conducted to date. Then, we direct our attention to the cerebellum, with a review of more simulation studies specific to that region. Furthermore, we present recent simulation results of a human-scale cerebellar network model composed of 86 billion neurons on the Japanese flagship supercomputer K (now retired). Finally, we discuss the necessity and importance of human-scale brain simulation, and suggest future directions of such large-scale brain simulation research.

Simulation of a Human-Scale Cerebellar Network Model on the K Computer.

Computer simulation of the human brain at an individual neuron resolution is an ultimate goal of computational neuroscience. The Japanese flagship supercomputer, K, provides unprecedented computational capability toward this goal. The cerebellum contains 80% of the neurons in the whole brain. Therefore, computer simulation of the human-scale cerebellum will be a challenge for modern supercomputers. In this study, we built a human-scale spiking network model of the cerebellum, composed of 68 billion spiking neurons, on the K computer. As a benchmark, we performed a computer simulation of a cerebellum-dependent eye movement task known as the optokinetic response. We succeeded in reproducing plausible neuronal activity patterns that are observed experimentally in animals. The model was built on dedicated neural network simulation software called MONET (Millefeuille-like Organization NEural neTwork), which calculates layered sheet types of neural networks with parallelization by tile partitioning. To examine the scalability of the MONET simulator, we repeatedly performed simulations while changing the number of compute nodes from 1,024 to 82,944 and measured the computational time. We observed a good weak-scaling property for our cerebellar network model. Using all 82,944 nodes, we succeeded in simulating a human-scale cerebellum for the first time, although the simulation was 578 times slower than the wall clock time. These results suggest that the K computer is already capable of creating a simulation of a human-scale cerebellar model with the aid of the MONET simulator.

Large-Scale Simulation of a Layered Cortical Sheet of Spiking Network Model Using a Tile Partitioning Method.

One of the grand challenges for computational neuroscience and high-performance computing is computer simulation of a human-scale whole brain model with spiking neurons and synaptic plasticity using supercomputers. To achieve such a simulation, the target network model must be partitioned onto a number of computational nodes, and the sub-network models are executed in parallel while communicating spike information across different nodes. However, it remains unclear how the target network model should be partitioned for efficient computing on next generation of supercomputers. Specifically, reducing the communication of spike information across compute nodes is essential, because of the relatively slower network performance than processor and memory. From the viewpoint of biological features, the cerebral cortex and cerebellum contain 99% of neurons and synapses and form layered sheet structures. Therefore, an efficient method to split the network should exploit the layered sheet structures. In this study, we indicate that a tile partitioning method leads to efficient communication. To demonstrate it, a simulation software called MONET (Millefeuille-like Organization NEural neTwork simulator) that partitions a network model as described above was developed. The MONET simulator was implemented on the Japanese flagship supercomputer K, which is composed of 82,944 computational nodes. We examined a performance of calculation, communication and memory consumption in the tile partitioning method for a cortical model with realistic anatomical and physiological parameters. The result showed that the tile partitioning method drastically reduced communication data amount by replacing network communication with DRAM access and sharing the communication data with neighboring neurons. We confirmed the scalability and efficiency of the tile partitioning method on up to 63,504 compute nodes of the K computer for the cortical model. In the companion paper by Yamaura et al., the performance for a cerebellar model was examined. These results suggest that the tile partitioning method will have advantage for a human-scale whole-brain simulation on exascale computers.

Involvement of cholinergic system in hyperactivity in dopamine-deficient mice.

Dopaminergic systems have been known to be involved in the regulation of locomotor activity and development of psychosis. However, the observations that some Parkinson's disease patients can move effectively under appropriate conditions despite low dopamine levels (eg, kinesia paradoxia) and that several psychotic symptoms are typical antipsychotic resistant and atypical antipsychotic sensitive indicate that other systems beyond the dopaminergic system may also affect locomotor activity and psychosis. The present study showed that dopamine-deficient (DD) mice, which had received daily L-DOPA injections, could move effectively and even be hyperactive 72 h after the last L-DOPA injection when dopamine was almost completely depleted. Such hyperactivity was ameliorated by clozapine but not haloperidol or ziprasidone. Among multiple actions of clozapine, muscarinic acetylcholine (ACh) activation markedly reduced locomotor activity in DD mice. Furthermore, the expression of choline acetyltransferase, an ACh synthase, was reduced and extracellular ACh levels were significantly reduced in DD mice. These results suggest that the cholinergic system, in addition to the dopaminergic system, may be involved in motor control, including hyperactivity and psychosis. The present findings provide additional evidence that the cholinergic system may be targeted for the treatment of Parkinson's disease and psychosis.

Deficits in memory-guided limb movements impair obstacle avoidance locomotion in Alzheimer's disease mouse model.

Memory function deficits induced by Alzheimer's disease (AD) are believed to be one of the causes of an increased risk of tripping in patients. Working memory contributes to accurate stepping over obstacles during locomotion, and AD-induced deficits of this memory function may lead to an increased risk of contact with obstacles. We used the triple transgenic (3xTg) mice to examine the effects of memory deficits in terms of tripping and contact with obstacles. We found that the frequency of contact of the hindlimbs during an obstacle avoidance task increased significantly in 10-13 month-old 3xTg (Old-3xTg) mice compared with control mice. However, no changes in limb kinematics during unobstructed locomotion or successful obstacle avoidance locomotion were observed in the Old-3xTg mice. Furthermore, we found that memory-based movements in stepping over an obstacle were impaired in these mice. Our findings suggest that working memory deficits as a result of AD are associated with an increased risk of tripping during locomotion.

Ultrasound-enhanced delivery of morpholino with Bubble liposomes ameliorates the myotonia of myotonic dystrophy model mice.

Phosphorodiamidate morpholino oligonucleotide (PMO)-mediated control of the alternative splicing of the chloride channel 1 (CLCN1) gene is a promising treatment for myotonic dystrophy type 1 (DM1) because the abnormal splicing of this gene causes myotonia in patients with DM1. In this study, we optimised a PMO sequence to correct Clcn1 alternative splicing and successfully remedied the myotonic phenotype of a DM1 mouse model, the HSALR mouse. To enhance the efficiency of delivery of PMO into HSALR mouse muscles, Bubble liposomes, which have been used as a gene delivery tool, were applied with ultrasound exposure. Effective delivery of PMO led to increased expression of Clcn1 protein in skeletal muscle and the amelioration of myotonia. Thus, PMO-mediated control of the alternative splicing of the Clcn1 gene must be important target of antisense therapy of DM1.

Contributions of phase resetting and interlimb coordination to the adaptive control of hindlimb obstacle avoidance during locomotion in rats: a simulation study.

Obstacle avoidance during locomotion is essential for safe, smooth locomotion. Physiological studies regarding muscle synergy have shown that the combination of a small number of basic patterns produces the large part of muscle activities during locomotion and the addition of another pattern explains muscle activities for obstacle avoidance. Furthermore, central pattern generators in the spinal cord are thought to manage the timing to produce such basic patterns. In the present study, we investigated sensory-motor coordination for obstacle avoidance by the hindlimbs of the rat using a neuromusculoskeletal model. We constructed the musculoskeletal part of the model based on empirical anatomical data of the rat and the nervous system model based on the aforementioned physiological findings of central pattern generators and muscle synergy. To verify the dynamic simulation by the constructed model, we compared the simulation results with kinematic and electromyographic data measured during actual locomotion in rats. In addition, we incorporated sensory regulation models based on physiological evidence of phase resetting and interlimb coordination and examined their functional roles in stepping over an obstacle during locomotion. Our results show that the phase regulation based on interlimb coordination contributes to stepping over a higher obstacle and that based on phase resetting contributes to quick recovery after stepping over the obstacle. These results suggest the importance of sensory regulation in generating successful obstacle avoidance during locomotion.

Characteristics of gait ataxia in δ2 glutamate receptor mutant mice, ho15J.

The cerebellum plays a fundamental, but as yet poorly understood, role in the control of locomotion. Recently, mice with gene mutations or knockouts have been used to investigate various aspects of cerebellar function with regard to locomotion. Although many of the mutant mice exhibit severe gait ataxia, kinematic analyses of limb movements have been performed in only a few cases. Here, we investigated locomotion in ho15J mice that have a mutation of the δ2 glutamate receptor. The cerebellum of ho15J mice shows a severe reduction in the number of parallel fiber-Purkinje synapses compared with wild-type mice. Analysis of hindlimb kinematics during treadmill locomotion showed abnormal hindlimb movements characterized by excessive toe elevation during the swing phase, and by severe hyperflexion of the ankles in ho15J mice. The great trochanter heights in ho15J mice were lower than in wild-type mice throughout the step cycle. However, there were no significant differences in various temporal parameters between ho15J and wild-type mice. We suggest that dysfunction of the cerebellar neuronal circuits underlies the observed characteristic kinematic abnormality of hindlimb movements during locomotion of ho15J mice.

Development of hand rehabilitation system for paralysis patient - universal design using wire-driven mechanism.

We have developed a hand rehabilitation system for patients suffering from paralysis or contracture. It consists of two components: a hand rehabilitation machine, which moves human finger joints with motors, and a data glove, which provides control of the movement of finger joints attached to the rehabilitation machine. The machine is based on the arm structure type of hand rehabilitation machine; a motor indirectly moves a finger joint via a closed four-link mechanism. We employ a wire-driven mechanism and develop a compact design that can control all three joints (i.e., PIP, DIP and MP ) of a finger and that offers a wider range of joint motion than conventional systems. Furthermore, we demonstrate the hand rehabilitation process, finger joints of the left hand attached to the machine are controlled by the finger joints of the right hand wearing the data glove.

FraG is necessary for filament integrity and heterocyst maturation in the cyanobacterium Anabaena sp. strain PCC 7120.

Anabaena sp. strain PCC 7120 is a filamentous cyanobacterium that differentiates nitrogen-fixing heterocysts when fixed nitrogen becomes growth limiting in the medium. The gene alr2338 (designated fraG herein), located immediately upstream of the master regulator of differentiation hetR, was identified in a genetic screen for mutants unable to grow diazotrophically. Filaments with a mutation in fraG were unable to fix nitrogen or synthesize heterocyst-specific glycolipids, and they fragmented initially to approximately nine cells in length at 24 h after induction of heterocyst development and eventually became unicellular. The fragmentation phenotype could be duplicated in the presence of fixed nitrogen when differentiation of heterocysts was elicited by overexpression of hetR, suggesting that a defect in differentiation, and not a lack of fixed nitrogen in the medium, was the more direct cause of fragmentation. An intact fraG gene was necessary for differentiation of mature heterocysts, but was not required for proper pattern formation, as indicated by a normal pattern of expression of hetR in a fraG mutant. A transcriptional GFP reporter fusion indicated that the level of expression of fraG was low in vegetative cells in both nitrogen-replete and nitrogen-free media, and was induced in heterocysts. fraG appears to play a role in filament integrity and differentiation of proheterocysts into mature heterocysts.