Computation harvesting from nature dynamics for predicting wind speed and direction.

Natural phenomena generate complex dynamics because of nonlinear interactions among their components. The dynamics can be exploited as a kind of computational resource. For example, in the framework of natural computation, various natural phenomena such as quantum mechanics and cellular dynamics are used to realize general purpose calculations or logical operations. In recent years, simple collection of such nature dynamics has become possible in a sensor-rich society. For example, images of plant movement that have been captured indirectly by a surveillance camera can be regarded as sensor outputs reflecting the state of the wind striking the plant. Herein, based on ideas of physical reservoir computing, we present a methodology for wind speed and direction estimation from naturally occurring sensors in movies. Then we demonstrate its effectiveness through experimentation. Specifically using the proposed methodology, we investigate the computational capability of the nature dynamics, revealing its high robustness and generalization performance for computation.

Biological neurons act as generalization filters in reservoir computing.

Reservoir computing is a machine learning paradigm that transforms the transient dynamics of high-dimensional nonlinear systems for processing time-series data. Although the paradigm was initially proposed to model information processing in the mammalian cortex, it remains unclear how the nonrandom network architecture, such as the modular architecture, in the cortex integrates with the biophysics of living neurons to characterize the function of biological neuronal networks (BNNs). Here, we used optogenetics and calcium imaging to record the multicellular responses of cultured BNNs and employed the reservoir computing framework to decode their computational capabilities. Micropatterned substrates were used to embed the modular architecture in the BNNs. We first show that the dynamics of modular BNNs in response to static inputs can be classified with a linear decoder and that the modularity of the BNNs positively correlates with the classification accuracy. We then used a timer task to verify that BNNs possess a short-term memory of several 100 ms and finally show that this property can be exploited for spoken digit classification. Interestingly, BNN-based reservoirs allow categorical learning, wherein a network trained on one dataset can be used to classify separate datasets of the same category. Such classification was not possible when the inputs were directly decoded by a linear decoder, suggesting that BNNs act as a generalization filter to improve reservoir computing performance. Our findings pave the way toward a mechanistic understanding of information representation within BNNs and build future expectations toward the realization of physical reservoir computing systems based on BNNs.

Monkey Prefrontal Single-Unit Activity Reflecting Category-Based Logical Thinking Process and Its Neural Network Model.

Category-based thinking is a fundamental form of logical thinking. Here, we aimed to investigate its neural process at the local circuit level in the prefrontal cortex (PFC). We recorded single-unit PFC activity while male monkeys (Macaca fuscata) performed a task in which the category and rule were prerequisites of logical thinking and the outcome contingency was its consequence. Different groups of neurons coded a single type of information discretely or multiple types in a transitional form. Results of time-by-time analysis of neuronal activity suggest an information flow from category-coding and rule-coding neurons to transitional intermediate neurons, and then to contingency-coding neurons. Category-coding, rule-coding, and contingency-coding neurons showed stable coding of information, whereas intermediate neurons showed dynamic coding, as if it integrated category and rule to derive contingency. A similar process was confirmed by using a spiking neural network model that consisted of subnetworks coding category and rule on the input layer and those coding contingency on the output layer, with a subnetwork for integration in the intermediate layer. These results suggest that category-based logical thinking is realized in the PFC by separated neural populations organized for working in a feedforward manner.SIGNIFICANCE STATEMENT To elucidate the neural process for logical thinking, we combined an in-depth analysis of single-unit activity data with a biologically plausible computational model. Results of time-by-time analysis of prefrontal neuronal activity suggest an information flow from category-coding and rule-coding neurons to transitional intermediate neurons, and then to contingency-coding neurons. Category-coding, rule-coding, and contingency-coding neurons showed stable coding, whereas intermediate neurons showed dynamic coding, as if they integrated category and rule to derive contingency. A spiking neural network model reproduced similar temporal changes of information as the recorded neuronal data. Our results suggest that the prefrontal cortex (PFC) is critically involved in category-based thought process, and this process may be produced by separated neural populations organized for working in a feedforward manner.

Chaos may enhance expressivity in cerebellar granular layer.

Recent evidence suggests that Golgi cells in the cerebellar granular layer are densely connected to each other with massive gap junctions. Here, we propose that the massive gap junctions between the Golgi cells contribute to the representational complexity of the granular layer of the cerebellum by inducing chaotic dynamics. We construct a model of cerebellar granular layer with diffusion coupling through gap junctions between the Golgi cells, and evaluate the representational capability of the network with the reservoir computing framework. First, we show that the chaotic dynamics induced by diffusion coupling results in complex output patterns containing a wide range of frequency components. Second, the long non-recursive time series of the reservoir represents the passage of time from an external input. These properties of the reservoir enable mapping different spatial inputs into different temporal patterns.

Chaotic dynamics as a mechanism of rapid transition of hippocampal local field activity between theta and non-theta states.

We propose a dynamical model of the local hippocampal circuit realizing the transition between the theta and non-theta states. We model the interaction between hippocampal local rhythm generators and the external periodic input from the medial septum and diagonal band of Broca (MS-DBB). With our model, bifurcation of the nonlinear dynamics serves as a mechanism that realizes two distinctive oscillations in the hippocampus, where the amplitude of the oscillatory input from the MS-DBB works as a bifurcation parameter. We model the network of the hippocampal interneurons with a network of simple class 1 neuron models connected mutually with gap junctions. The model neurons exhibit highly synchronous periodic oscillations under the existence of an external force from the MS-DBB, just as the real hippocampus shows theta oscillation under the rhythmic input from the MS-DBB. The model shows diffusion-induced chaotic dynamics under an aperiodic MS-DBB activity, just as the large amplitude irregular activity appears following the disappearance of the rhythmicity of the MS-DBB neurons in the real brain. The model is consistent with both previous experimental findings reporting the existence of local rhythm generators in the hippocampus and the executive role of the MS-DBB in synchronizing theta oscillation in vivo. Our model also replicates the traveling waves of theta oscillations in two-dimensionally coupled networks.

Bifurcation Analysis on Phase-Amplitude Cross-Frequency Coupling in Neural Networks with Dynamic Synapses.

We investigate a discrete-time network model composed of excitatory and inhibitory neurons and dynamic synapses with the aim at revealing dynamical properties behind oscillatory phenomena possibly related to brain functions. We use a stochastic neural network model to derive the corresponding macroscopic mean field dynamics, and subsequently analyze the dynamical properties of the network. In addition to slow and fast oscillations arising from excitatory and inhibitory networks, respectively, we show that the interaction between these two networks generates phase-amplitude cross-frequency coupling (CFC), in which multiple different frequency components coexist and the amplitude of the fast oscillation is modulated by the phase of the slow oscillation. Furthermore, we clarify the detailed properties of the oscillatory phenomena by applying the bifurcation analysis to the mean field model, and accordingly show that the intermittent and the continuous CFCs can be characterized by an aperiodic orbit on a closed curve and one on a torus, respectively. These two CFC modes switch depending on the coupling strength from the excitatory to inhibitory networks, via the saddle-node cycle bifurcation of a one-dimensional torus in map (MT1SNC), and may be associated with the function of multi-item representation. We believe that the present model might have potential for studying possible functional roles of phase-amplitude CFC in the cerebral cortex.

Controlled generation of switching dynamics among metastable states in pulse-coupled oscillator networks.

Switching dynamics among saddles in a network of nonlinear oscillators can be exploited for information encoding and processing (hence computing), but stable attractors in the system can terminate the switching behavior. An effective control strategy is presented to sustain switching dynamics in networks of pulse-coupled oscillators. The support for the switching behavior is a set of saddles, or unstable invariant sets in the phase space. We thus identify saddles with a common property, localize the system in the vicinity of them, and then guide the system from one metastable state to another to generate desired switching dynamics. We demonstrate that the control method successfully generates persistent switching trajectories and prevents the system from entering stable attractors. In addition, there exists correspondence between the network structure and the switching dynamics, providing fundamental insights on the development of a computing paradigm based on the switching dynamics.

Increased firing irregularity as an emergent property of neural-state transition in monkey prefrontal cortex.

Flexible behaviors are organized by complex neural networks in the prefrontal cortex. Recent studies have suggested that such networks exhibit multiple dynamical states, and can switch rapidly from one state to another. In many complex systems such as the brain, the early-warning signals that may predict whether a critical threshold for state transitions is approaching are extremely difficult to detect. We hypothesized that increases in firing irregularity are a crucial measure for predicting state transitions in the underlying neuronal circuits of the prefrontal cortex. We used both experimental and theoretical approaches to test this hypothesis. Experimentally, we analyzed activities of neurons in the prefrontal cortex while monkeys performed a maze task that required them to perform actions to reach a goal. We observed increased firing irregularity before the activity changed to encode goal-to-action information. Theoretically, we constructed theoretical generic neural networks and demonstrated that changes in neuronal gain on functional connectivity resulted in a loss of stability and an altered state of the networks, accompanied by increased firing irregularity. These results suggest that assessing the temporal pattern of neuronal fluctuations provides important clues regarding the state stability of the prefrontal network. We also introduce a novel scheme that the prefrontal cortex functions in a metastable state near the critical point of bifurcation. According to this scheme, firing irregularity in the prefrontal cortex indicates that the system is about to change its state and the flow of information in a flexible manner, which is essential for executive functions. This metastable and/or critical dynamical state of the prefrontal cortex may account for distractibility and loss of flexibility in the prefrontal cortex in major mental illnesses such as schizophrenia.

Stability analysis of associative memory network composed of stochastic neurons and dynamic synapses.

We investigate the dynamical properties of an associative memory network consisting of stochastic neurons and dynamic synapses that show short-term depression and facilitation. In the stochastic neuron model used in this study, the efficacy of the synaptic transmission changes according to the short-term depression or facilitation mechanism. We derive a macroscopic mean field model that captures the overall dynamical properties of the stochastic model. We analyze the stability and bifurcation structure of the mean field model, and show the dependence of the memory retrieval performance on the noise intensity and parameters that determine the properties of the dynamic synapses, i.e., time constants for depressing and facilitating processes. The associative memory network exhibits a variety of dynamical states, including the memory and pseudo-memory states, as well as oscillatory states among memory patterns. This study provides comprehensive insight into the dynamical properties of the associative memory network with dynamic synapses.

Solution to the inverse problem of estimating gap-junctional and inhibitory conductance in inferior olive neurons from spike trains by network model simulation.

The inferior olive (IO) possesses synaptic glomeruli, which contain dendritic spines from neighboring neurons and presynaptic terminals, many of which are inhibitory and GABAergic. Gap junctions between the spines electrically couple neighboring neurons whereas the GABAergic synaptic terminals are thought to act to decrease the effectiveness of this coupling. Thus, the glomeruli are thought to be important for determining the oscillatory and synchronized activity displayed by IO neurons. Indeed, the tendency to display such activity patterns is enhanced or reduced by the local administration of the GABA-A receptor blocker picrotoxin (PIX) or the gap junction blocker carbenoxolone (CBX), respectively. We studied the functional roles of the glomeruli by solving the inverse problem of estimating the inhibitory (gi) and gap-junctional conductance (gc) using an IO network model. This model was built upon a prior IO network model, in which the individual neurons consisted of soma and dendritic compartments, by adding a glomerular compartment comprising electrically coupled spines that received inhibitory synapses. The model was used in the forward mode to simulate spike data under PIX and CBX conditions for comparison with experimental data consisting of multi-electrode recordings of complex spikes from arrays of Purkinje cells (complex spikes are generated in a one-to-one manner by IO spikes and thus can substitute for directly measuring IO spike activity). The spatiotemporal firing dynamics of the experimental and simulation spike data were evaluated as feature vectors, including firing rates, local variation, auto-correlogram, cross-correlogram, and minimal distance, and were contracted onto two-dimensional principal component analysis (PCA) space. gc and gi were determined as the solution to the inverse problem such that the simulation and experimental spike data were closely matched in the PCA space. The goodness of the match was confirmed by an analysis of variance (ANOVA) of the PCA scores between the experimental and simulation spike data. In the PIX condition, gi was found to decrease to approximately half its control value. CBX caused an approximately 30% decrease in gc from control levels. These results support the hypothesis that the glomeruli are control points for determining the spatiotemporal characteristics of olivocerebellar activity and thus may shape its ability to convey signals to the cerebellum that may be used for motor learning or motor control purposes.

An FPGA-Based Silicon Neuronal Network with Selectable Excitability Silicon Neurons.

This paper presents a digital silicon neuronal network which simulates the nerve system in creatures and has the ability to execute intelligent tasks, such as associative memory. Two essential elements, the mathematical-structure-based digital spiking silicon neuron (DSSN) and the transmitter release based silicon synapse, allow us to tune the excitability of silicon neurons and are computationally efficient for hardware implementation. We adopt mixed pipeline and parallel structure and shift operations to design a sufficient large and complex network without excessive hardware resource cost. The network with 256 full-connected neurons is built on a Digilent Atlys board equipped with a Xilinx Spartan-6 LX45 FPGA. Besides, a memory control block and USB control block are designed to accomplish the task of data communication between the network and the host PC. This paper also describes the mechanism of associative memory performed in the silicon neuronal network. The network is capable of retrieving stored patterns if the inputs contain enough information of them. The retrieving probability increases with the similarity between the input and the stored pattern increasing. Synchronization of neurons is observed when the successful stored pattern retrieval occurs.

Representational switching by dynamical reorganization of attractor structure in a network model of the prefrontal cortex.

The prefrontal cortex (PFC) plays a crucial role in flexible cognitive behavior by representing task relevant information with its working memory. The working memory with sustained neural activity is described as a neural dynamical system composed of multiple attractors, each attractor of which corresponds to an active state of a cell assembly, representing a fragment of information. Recent studies have revealed that the PFC not only represents multiple sets of information but also switches multiple representations and transforms a set of information to another set depending on a given task context. This representational switching between different sets of information is possibly generated endogenously by flexible network dynamics but details of underlying mechanisms are unclear. Here we propose a dynamically reorganizable attractor network model based on certain internal changes in synaptic connectivity, or short-term plasticity. We construct a network model based on a spiking neuron model with dynamical synapses, which can qualitatively reproduce experimentally demonstrated representational switching in the PFC when a monkey was performing a goal-oriented action-planning task. The model holds multiple sets of information that are required for action planning before and after representational switching by reconfiguration of functional cell assemblies. Furthermore, we analyzed population dynamics of this model with a mean field model and show that the changes in cell assemblies' configuration correspond to those in attractor structure that can be viewed as a bifurcation process of the dynamical system. This dynamical reorganization of a neural network could be a key to uncovering the mechanism of flexible information processing in the PFC.

QUANTITATIVE MODELING OF SPATIO-TEMPORAL DYNAMICS OF INFERIOR OLIVE NEURONS WITH A SIMPLE CONDUCTANCE-BASED MODEL.

Inferior olive (IO) neurons project to the cerebellum and contribute to motor control. They can show intriguing spatio-temporal dynamics with rhythmic and synchronized spiking. IO neurons are connected to their neighbors via gap junctions to form an electrically coupled network, and so it is considered that this coupling contributes to the characteristic dynamics of this nucleus. Here, we demonstrate that a gap junction-coupled network composed of simple conductance-based model neurons (a simplified version of a Hodgkin-Huxley type neuron) reproduce important aspects of IO activity. The simplified phenomenological model neuron facilitated the analysis of the single cell and network properties of the IO while still quantitatively reproducing the spiking patterns of complex spike activity observed by simultaneous recording in anesthetized rats. The results imply that both intrinsic bistability of each neuron and gap junction coupling among neurons play key roles in the generation of the spatio-temporal dynamics of IO neurons.

Testing a neural coding hypothesis using surrogate data.

Determining how a particular neuron, or population of neurons, encodes information in their spike trains is not a trivial problem, because multiple coding schemes exist and are not necessarily mutually exclusive. Coding schemes generally fall into one of two broad categories, which we refer to as rate and temporal coding. In rate coding schemes, information is encoded in the variations of the average firing rate of the spike train. In contrast, in temporal coding schemes, information is encoded in the specific timing of the individual spikes that comprise the train. Here, we describe a method for testing the presence of temporal encoding of information. Suppose that a set of original spike trains is given. First, surrogate spike trains are generated by randomizing each of the original spike trains subject to the following constraints: the local average firing rate is approximately preserved, while the overall average firing rate and the distribution of primary interspike intervals are perfectly preserved. These constraints ensure that any rate coding of information present in the original spike trains is preserved in the members of the surrogate population. The null-hypothesis is rejected when additional information is found to be present in the original spike trains, implying that temporal coding is present. The method is validated using artificial data, and then demonstrated using real neuronal data.

Dynamic switching of neural codes in networks with gap junctions.

Population rate coding and temporal coding are common neural codes. Recent studies suggest that these two codes may be alternatively used in one neural system. Based on the fact that there are massive gap junctions in the brain, we explore how this switching behavior may be related to neural codes in networks of neurons connected by gap junctions. First, we show that under time-varying inputs, such neural networks show switching between synchronous and asynchronous states. Then, we quantify network dynamics by three mutual information measures to show that population rate coding carries more information in asynchronous states and temporal coding does so in synchronous states.