Interaction mechanisms quantified from dynamical features of frog choruses.

We employ a mathematical model (a phase oscillator model) to describe the deterministic and stochastic features of frog choruses in which male frogs attempt to avoid call overlaps. The mathematical model with a general interaction term is identified using a Bayesian approach, and it qualitatively reproduces the stationary and dynamical features of the empirical data. In addition, we quantify the magnitude of attention paid among the male frogs from the identified model, and then analyse the relationship between attention and behavioural parameters using a statistical approach. Our analysis demonstrates a negative correlation between attention and inter-frog distance, and also suggests a behavioural strategy in which male frogs selectively attend to a less attractive male frog (i.e. a male producing calls at longer intervals) in order to more effectively advertise their superior relative attractiveness to females.

Effect of recurrent infomax on the information processing capability of input-driven recurrent neural networks.

Reservoir computing is a framework for exploiting the inherent transient dynamics of recurrent neural networks (RNNs) as a computational resource. On the basis of this framework, much research has been conducted to evaluate the relationship between the dynamics of RNNs and the RNNs' information processing capability. In this study, we present a detailed analysis of the information processing capability of an RNN optimized by recurrent infomax (RI), an unsupervised learning method that maximizes the mutual information of RNNs by adjusting the connection weights of the network. The results indicate that RI leads to the emergence of a delay-line structure and that the network optimized by the RI possesses a superior short-term memory, which is the ability to store the temporal information of the input stream in its transient dynamics.

A dynamical systems approach for estimating phase interactions between rhythms of different frequencies from experimental data.

Synchronization of neural oscillations as a mechanism of brain function is attracting increasing attention. Neural oscillation is a rhythmic neural activity that can be easily observed by noninvasive electroencephalography (EEG). Neural oscillations show the same frequency and cross-frequency synchronization for various cognitive and perceptual functions. However, it is unclear how this neural synchronization is achieved by a dynamical system. If neural oscillations are weakly coupled oscillators, the dynamics of neural synchronization can be described theoretically using a phase oscillator model. We propose an estimation method to identify the phase oscillator model from real data of cross-frequency synchronized activities. The proposed method can estimate the coupling function governing the properties of synchronization. Furthermore, we examine the reliability of the proposed method using time-series data obtained from numerical simulation and an electronic circuit experiment, and show that our method can estimate the coupling function correctly. Finally, we estimate the coupling function between EEG oscillation and the speech sound envelope, and discuss the validity of these results.

Bayesian Estimation of Phase Dynamics Based on Partially Sampled Spikes Generated by Realistic Model Neurons.

A dynamic system showing stable rhythmic activity can be represented by the dynamics of phase oscillators. This would provide a useful mathematical framework through which one can understand the system's dynamic properties. A recent study proposed a Bayesian approach capable of extracting the underlying phase dynamics directly from time-series data of a system showing rhythmic activity. Here we extended this method to spike data that otherwise provide only limited phase information. To determine how this method performs with spike data, we applied it to simulated spike data generated by a realistic neuronal network model. We then compared the estimated dynamics obtained based on the spike data with the dynamics theoretically derived from the model. The method successfully extracted the modeled phase dynamics, particularly the interaction function, when the amount of available data was sufficiently large. Furthermore, the method was able to infer synaptic connections based on the estimated interaction function. Thus, the method was found to be applicable to spike data and practical for understanding the dynamic properties of rhythmic neural systems.

Dynamics of two populations of phase oscillators with different frequency distributions.

A large variety of rhythms are observed in nature. Rhythms such as electroencephalogram signals in the brain can often be regarded as interacting. In this study, we investigate the dynamical properties of rhythmic systems in two populations of phase oscillators with different frequency distributions. We assume that the average frequency ratio between two populations closely approximates some small integer. Most importantly, we adopt a specific coupling function derived from phase reduction theory. Under some additional assumptions, the system of two populations of coupled phase oscillators reduces to a low-dimensional system in the continuum limit. Consequently, we find chimera states in which clustering and incoherent states coexist. Finally, we confirm consistent behaviors of the derived low-dimensional model and the original model.

Evaluation of the Phase-Dependent Rhythm Control of Human Walking Using Phase Response Curves.

Humans and animals control their walking rhythms to maintain motion in a variable environment. The neural mechanism for controlling rhythm has been investigated in many studies using mechanical and electrical stimulation. However, quantitative evaluation of rhythm variation in response to perturbation at various timings has rarely been investigated. Such a characteristic of rhythm is described by the phase response curve (PRC). Dynamical simulations of human skeletal models with changing walking rhythms (phase reset) described a relation between the effective phase reset on stability and PRC, and phase reset around touch-down was shown to improve stability. A PRC of human walking was estimated by pulling the swing leg, but such perturbations hardly influenced the stance leg, so the relation between the PRC and walking events was difficult to discuss. This research thus examines human response to variations in floor velocity. Such perturbation yields another problem, in that the swing leg is indirectly (and weakly) perturbed, so the precision of PRC decreases. To solve this problem, this research adopts the weighted spike-triggered average (WSTA) method. In the WSTA method, a sequential pulsed perturbation is used for stimulation. This is in contrast with the conventional impulse method, which applies an intermittent impulsive perturbation. The WSTA method can be used to analyze responses to a large number of perturbations for each sequence. In the experiment, perturbations are applied to walking subjects by rapidly accelerating and decelerating a treadmill belt, and measured data are analyzed by the WSTA and impulse methods. The PRC obtained by the WSTA method had clear and stable waveforms with a higher temporal resolution than those obtained by the impulse method. By investigation of the rhythm transition for each phase of walking using the obtained PRC, a rhythm change that extends the touch-down and mid-single support phases is found to occur.

Learning in neural networks based on a generalized fluctuation theorem.

Information maximization has been investigated as a possible mechanism of learning governing the self-organization that occurs within the neural systems of animals. Within the general context of models of neural systems bidirectionally interacting with environments, however, the role of information maximization remains to be elucidated. For bidirectionally interacting physical systems, universal laws describing the fluctuation they exhibit and the information they possess have recently been discovered. These laws are termed fluctuation theorems. In the present study, we formulate a theory of learning in neural networks bidirectionally interacting with environments based on the principle of information maximization. Our formulation begins with the introduction of a generalized fluctuation theorem, employing an interpretation appropriate for the present application, which differs from the original thermodynamic interpretation. We analytically and numerically demonstrate that the learning mechanism presented in our theory allows neural networks to efficiently explore their environments and optimally encode information about them.

Self-organization of complex networks as a dynamical system.

To understand the dynamics of real-world networks, we investigate a mathematical model of the interplay between the dynamics of random walkers on a weighted network and the link weights driven by a resource carried by the walkers. Our numerical studies reveal that, under suitable conditions, the co-evolving dynamics lead to the emergence of stationary power-law distributions of the resource and link weights, while the resource quantity at each node ceaselessly changes with time. We analyze the network organization as a deterministic dynamical system and find that the system exhibits multistability, with numerous fixed points, limit cycles, and chaotic states. The chaotic behavior of the system leads to the continual changes in the microscopic network dynamics in the absence of any external random noises. We conclude that the intrinsic interplay between the states of the nodes and network reformation constitutes a major factor in the vicissitudes of real-world networks.

A biologically plausible learning rule for the Infomax on recurrent neural networks.

A fundamental issue in neuroscience is to understand how neuronal circuits in the cerebral cortex play their functional roles through their characteristic firing activity. Several characteristics of spontaneous and sensory-evoked cortical activity have been reproduced by Infomax learning of neural networks in computational studies. There are, however, still few models of the underlying learning mechanisms that allow cortical circuits to maximize information and produce the characteristics of spontaneous and sensory-evoked cortical activity. In the present article, we derive a biologically plausible learning rule for the maximization of information retained through time in dynamics of simple recurrent neural networks. Applying the derived learning rule in a numerical simulation, we reproduce the characteristics of spontaneous and sensory-evoked cortical activity: cell-assembly-like repeats of precise firing sequences, neuronal avalanches, spontaneous replays of learned firing sequences and orientation selectivity observed in the primary visual cortex. We further discuss the similarity between the derived learning rule and the spike timing-dependent plasticity of cortical neurons.

A mathematical model of negative covariability of inter-columnar excitatory synaptic actions caused by presynaptic inhibition.

We previously showed that a positive covariability between intracortical excitatory synaptic actions onto the two layer three pyramidal cells (PCs) located in mutually adjacent columns is changed into a negative covariability by column-wise presynaptic inhibition of intracortical inputs, implicated as a basis for the desynchronization of inter-columnar synaptic actions. Here we investigated how the inter-columnar desynchronization is modulated by the strength of presynaptic inhibition or other factors, by using a mathematical model. Based on our previous findings on the paired-pulse depression (PPD) of intracortical excitatory postsynaptic currents (EPSCs) evoked in PCs located in the stimulated home column (HC) but no PPD in PCs located in the adjacent column (AC), a mathematical model of synaptic connections between PCs and inhibitory interneurons was constructed. When the paired-pulse ratio (PPR) was decreased beyond 0.80, the correlation coefficient between the two second EPSC amplitudes in the paired PCs located in the HC and AC and that in the paired PCs located in the same HC exhibited opposite changes, and reached a global negative maximum and local positive maximum, respectively, at almost the same PPR (0.40). At this PPR, the desynchronization between the two cell assemblies in mutually adjacent columns would be maximized. These positive and negative covariabilities were not produced without background oscillatory synchronization across columns and were enhanced by increasing the synchronization magnitude, indicating that the synchronization leads to the desynchronization. We propose that a slow oscillatory synchronization across columns may emerge following the liberation from the column-wise presynaptic inhibition of inter-columnar synaptic inputs.

Scale-free structures emerging from co-evolution of a network and the distribution of a diffusive resource on it.

Co-evolution exhibited by a network system, involving the intricate interplay between the dynamics of the network itself and the subsystems connected by it, is a key concept for understanding the self-organized, flexible nature of real-world network systems. We propose a simple model of such coevolving network dynamics, in which the diffusion of a resource over a weighted network and the resource-driven evolution of the link weights occur simultaneously. We demonstrate that, under feasible conditions, the network robustly acquires scale-free characteristics in the asymptotic state. Interestingly, in the case that the system includes dissipation, it asymptotically realizes a dynamical phase characterized by an organized scale-free network, in which the ranking of each node with respect to the quantity of the resource possessed thereby changes ceaselessly. Our model offers a unified framework for understanding some real-world diffusion-driven network systems of diverse types.

Replicating receptive fields of simple and complex cells in primary visual cortex in a neuronal network model with temporal and population sparseness and reliability.

We propose a new principle for replicating receptive field properties of neurons in the primary visual cortex. We derive a learning rule for a feedforward network, which maintains a low firing rate for the output neurons (resulting in temporal sparseness) and allows only a small subset of the neurons in the network to fire at any given time (resulting in population sparseness). Our learning rule also sets the firing rates of the output neurons at each time step to near-maximum or near-minimum levels, resulting in neuronal reliability. The learning rule is simple enough to be written in spatially and temporally local forms. After the learning stage is performed using input image patches of natural scenes, output neurons in the model network are found to exhibit simple-cell-like receptive field properties. When the output of these simple-cell-like neurons are input to another model layer using the same learning rule, the second-layer output neurons after learning become less sensitive to the phase of gratings than the simple-cell-like input neurons. In particular, some of the second-layer output neurons become completely phase invariant, owing to the convergence of the connections from first-layer neurons with similar orientation selectivity to second-layer neurons in the model network. We examine the parameter dependencies of the receptive field properties of the model neurons after learning and discuss their biological implications. We also show that the localized learning rule is consistent with experimental results concerning neuronal plasticity and can replicate the receptive fields of simple and complex cells.

Multistable attractors in a network of phase oscillators with three-body interactions.

Three-body interactions have been found in physics, biology, and sociology. To investigate their effect on dynamical systems, as a first step, we study numerically and theoretically a system of phase oscillators with a three-body interaction. As a result, an infinite number of multistable synchronized states appear above a critical coupling strength, while a stable incoherent state always exists for any coupling strength. Owing to the infinite multistability, the degree of synchrony in an asymptotic state can vary continuously within some range depending on the initial phase pattern.

Asymmetric neighborhood functions accelerate ordering process of self-organizing maps.

A self-organizing map (SOM) algorithm can generate a topographic map from a high-dimensional stimulus space to a low-dimensional array of units. Because a topographic map preserves neighborhood relationships between the stimuli, the SOM can be applied to certain types of information processing such as data visualization. During the learning process, however, topological defects frequently emerge in the map. The presence of defects tends to drastically slow down the formation of a globally ordered topographic map. To remove such topological defects, it has been reported that an asymmetric neighborhood function is effective, but only in the simple case of mapping one-dimensional stimuli to a chain of units. In this paper, we demonstrate that even when high-dimensional stimuli are used, the asymmetric neighborhood function is effective for both artificial and real-world data. Our results suggest that applying the asymmetric neighborhood function to the SOM algorithm improves the reliability of the algorithm. In addition, it enables processing of complicated, high-dimensional data by using this algorithm.

Self-organized network of phase oscillators coupled by activity-dependent interactions.

We investigate a network of coupled phase oscillators whose interactions evolve dynamically depending on the relative phases between the oscillators. We found that this coevolving dynamical system robustly yields three basic states of collective behavior with their self-organized interactions. The first is the two-cluster state, in which the oscillators are organized into two synchronized groups. The second is the coherent state, in which the oscillators are arranged sequentially in time. The third is the chaotic state, in which the relative phases between oscillators and their coupling weights are chaotically shuffled. Furthermore, we demonstrate that self-assembled multiclusters can be designed by controlling the weight dynamics. Note that the phase patterns of the oscillators and the weighted network of interactions between them are simultaneously organized through this coevolving dynamics. We expect that these results will provide new insight into self-assembly mechanisms by which the collective behavior of a rhythmic system emerges as a result of the dynamics of adaptive interactions.

Bayesian estimation of phase response curves.

Phase response curve (PRC) of an oscillatory neuron describes the response of the neuron to external perturbation. The PRC is useful to predict synchronized dynamics of neurons; hence, its measurement from experimental data attracts increasing interest in neural science. This paper introduces a Bayesian method for estimating PRCs from data, which allows for the correlation of errors in explanatory and response variables of the PRC. The method is implemented with a replica exchange Monte Carlo technique; this avoids local minima and enables efficient calculation of posterior averages. A test with artificial data generated by the noisy Morris-Lecar equation shows that the proposed method outperforms conventional regression that ignores errors in the explanatory variable. Experimental data from the pyramidal cells in the rat motor cortex is also analyzed with the method; a case is found where the result with the proposed method is considerably different from that obtained by conventional regression.

Weighted spike-triggered average of a fluctuating stimulus yielding the phase response curve.

We demonstrate that the phase response curve (PRC) can be reconstructed using a weighted spike-triggered average of an injected fluctuating input. The key idea is to choose the weight to be proportional to the magnitude of the fluctuation of the oscillatory period. Particularly, when a neuron exhibits random switching behavior between two bursting modes, two corresponding PRCs can be simultaneously reconstructed, even from the data of a single trial. This method offers an efficient alternative to the experimental investigation of oscillatory systems, without the need for detailed modeling.

Co-evolution of phases and connection strengths in a network of phase oscillators.

We investigate co-evolving dynamics in a weighted network of phase oscillators in which the phases of the oscillators at the nodes and the weights of the links interact with each other. We find that depending on the type of the dynamics of the weights, the system exhibits three kinds of asymptotic behavior: a two-cluster state, a coherent state with a fixed phase relation, and a chaotic state with frustration. Because of its structural stability, it is believed that our model captures the essential characteristics of a class of co-evolving and adaptive networks.

Ordering process of self-organizing maps improved by asymmetric neighborhood function.

The Self-organizing map (SOM) is an unsupervised learning method based on the neural computation, which has found wide applications. However, the learning process sometime takes multi-stable states, within which the map is trapped to an undesirable disordered state including topological defects on the map. These topological defects critically aggravate the performance of the SOM. In order to overcome this problem, we propose to introduce an asymmetric neighborhood function for the SOM algorithm. Compared with the conventional symmetric one, the asymmetric neighborhood function accelerates the ordering process even in the presence of the defect. However, this asymmetry tends to generate a distorted map. This can be suppressed by an improved method of the asymmetric neighborhood function. In the case of one-dimensional SOM, it is found that the required steps for perfect ordering is numerically shown to be reduced from O(N (3)) to O(N (2)). We also discuss the ordering process of a twisted state in two-dimensional SOM, which can not be rectified by the ordinary symmetric neighborhood function.

Recurrent infomax generates cell assemblies, neuronal avalanches, and simple cell-like selectivity.

Recently multineuronal recording has allowed us to observe patterned firings, synchronization, oscillation, and global state transitions in the recurrent networks of central nervous systems. We propose a learning algorithm based on the process of information maximization in a recurrent network, which we call recurrent infomax (RI). RI maximizes information retention and thereby minimizes information loss through time in a network. We find that feeding in external inputs consisting of information obtained from photographs of natural scenes into an RI-based model of a recurrent network results in the appearance of Gabor-like selectivity quite similar to that existing in simple cells of the primary visual cortex. We find that without external input, this network exhibits cell assembly-like and synfire chain-like spontaneous activity as well as a critical neuronal avalanche. In addition, we find that RI embeds externally input temporal firing patterns to the network so that it spontaneously reproduces these patterns after learning. RI provides a simple framework to explain a wide range of phenomena observed in in vivo and in vitro neuronal networks, and it will provide a novel understanding of experimental results for multineuronal activity and plasticity from an information-theoretic point of view.