Deep Learning for Face Detection and Pain Assessment in Japanese macaques (Macaca fuscata).

Facial expressions have increasingly been used to assess emotional states in mammals. The recognition of pain in research animals is essential for their well-being and leads to more reliable research outcomes. Automating this process could contribute to early pain diagnosis and treatment. Artificial neural networks have become a popular option for image classification tasks in recent years due to the development of deep learning. In this study, we investigated the ability of a deep learning model to detect pain in Japanese macaques based on their facial expression. Thirty to 60 min of video footage from Japanese macaques undergoing laparotomy was used in the study. Macaques were recorded undisturbed in their cages before surgery (No Pain) and one day after the surgery before scheduled analgesia (Pain). Videos were processed for facial detection and image extraction with the algorithms RetinaFace (adding a bounding box around the face for image extraction) or Mask R-CNN (contouring the face for extraction). ResNet50 used 75% of the images to train systems; the other 25% were used for testing. Test accuracy varied from 48 to 54% after box extraction. The low accuracy of classification after box extraction was likely due to the incorporation of features that were not relevant for pain (for example, background, illumination, skin color, or objects in the enclosure). However, using contour extraction, preprocessing the images, and fine-tuning, the network resulted in 64% appropriate generalization. These results suggest that Mask R-CNN can be used for facial feature extractions and that the performance of the classifying model is relatively accurate for nonannotated single-frame images.

The lower bound of the network connectivity guaranteeing in-phase synchronization.

In-phase synchronization is a stable state of identical Kuramoto oscillators coupled on a network with identical positive connections, regardless of network topology. However, this fact does not mean that the networks always synchronize in-phase because other attractors besides the stable state may exist. The critical connectivity µc is defined as the network connectivity above which only the in-phase state is stable for all the networks. In other words, below µc, one can find at least one network that has a stable state besides the in-phase sync. The best known evaluation of the value so far is 0.6828…≤µc≤0.7889. In this paper, focusing on the twisted states of the circulant networks, we provide a method to systematically analyze the linear stability of all possible twisted states on all possible circulant networks. This method using integer programming enables us to find the densest circulant network having a stable twisted state besides the in-phase sync, which breaks a record of the lower bound of the µc from 0.6828… to 0.6838…. We confirm the validity of the theory by numerical simulations of the networks not converging to the in-phase state.

Highly Heterogeneous Excitatory Connections Require Less Amount of Noise to Sustain Firing Activities in Cortical Networks.

Cortical networks both in vivo and in vitro sustain asynchronous irregular firings with extremely low frequency. To realize such self-sustained activity in neural network models, balance between excitatory and inhibitory activities is known to be one of the keys. In addition, recent theoretical studies have revealed that another feature commonly observed in cortical networks, i.e., sparse but strong connections and dense weak connections, plays an essential role. The previous studies, however, have not thoroughly considered the cooperative dynamics between a network of such heterogeneous synaptic connections and intrinsic noise. The noise stimuli, representing inherent nature of the neuronal activities, e.g., variability of presynaptic discharges, should be also of significant importance for sustaining the irregular firings in cortical networks. Here, we numerically demonstrate that highly heterogeneous distribution, typically a lognormal type, of excitatory-to-excitatory connections, reduces the amount of noise required to sustain the network firing activities. In the sense that noise consumes an energy resource, the heterogeneous network receiving less amount of noise stimuli is considered to realize an efficient dynamics in cortex. A noise-driven network of bi-modally distributed synapses further shows that many weak and a few very strong synapses are the key feature of the synaptic heterogeneity, supporting the network firing activity.

Computational Neuroscience: Mathematical and Statistical Perspectives.

Mathematical and statistical models have played important roles in neuroscience, especially by describing the electrical activity of neurons recorded individually, or collectively across large networks. As the field moves forward rapidly, new challenges are emerging. For maximal effectiveness, those working to advance computational neuroscience will need to appreciate and exploit the complementary strengths of mechanistic theory and the statistical paradigm.

Macroscopic neural mass model constructed from a current-based network model of spiking neurons.

Neural mass models (NMMs) are efficient frameworks for describing macroscopic cortical dynamics including electroencephalogram and magnetoencephalogram signals. Originally, these models were formulated on an empirical basis of synaptic dynamics with relatively long time constants. By clarifying the relations between NMMs and the dynamics of microscopic structures such as neurons and synapses, we can better understand cortical and neural mechanisms from a multi-scale perspective. In a previous study, the NMMs were analytically derived by averaging the equations of synaptic dynamics over the neurons in the population and further averaging the equations of the membrane-potential dynamics. However, the averaging of synaptic current assumes that the neuron membrane potentials are nearly time invariant and that they remain at sub-threshold levels to retain the conductance-based model. This approximation limits the NMM to the non-firing state. In the present study, we newly propose a derivation of a NMM by alternatively approximating the synaptic current which is assumed to be independent of the membrane potential, thus adopting a current-based model. Our proposed model releases the constraint of the nearly constant membrane potential. We confirm that the obtained model is reducible to the previous model in the non-firing situation and that it reproduces the temporal mean values and relative power spectrum densities of the average membrane potentials for the spiking neurons. It is further ensured that the existing NMM properly models the averaged dynamics over individual neurons even if they are spiking in the populations.

Effective Suppression of Pathological Synchronization in Cortical Networks by Highly Heterogeneous Distribution of Inhibitory Connections.

Even without external random input, cortical networks in vivo sustain asynchronous irregular firing with low firing rate. In addition to detailed balance between excitatory and inhibitory activities, recent theoretical studies have revealed that another feature commonly observed in cortical networks, i.e., long-tailed distribution of excitatory synapses implying coexistence of many weak and a few extremely strong excitatory synapses, plays an essential role in realizing the self-sustained activity in recurrent networks of biologically plausible spiking neurons. The previous studies, however, have not considered highly non-random features of the synaptic connectivity, namely, bidirectional connections between cortical neurons are more common than expected by chance and strengths of synapses are positively correlated between pre- and postsynaptic neurons. The positive correlation of synaptic connections may destabilize asynchronous activity of networks with the long-tailed synaptic distribution and induce pathological synchronized firing among neurons. It remains unclear how the cortical network avoids such pathological synchronization. Here, we demonstrate that introduction of the correlated connections indeed gives rise to synchronized firings in a cortical network model with the long-tailed distribution. By using a simplified feed-forward network model of spiking neurons, we clarify the underlying mechanism of the synchronization. We then show that the synchronization can be efficiently suppressed by highly heterogeneous distribution, typically a lognormal distribution, of inhibitory-to-excitatory connection strengths in a recurrent network model of cortical neurons.

Dense neuron clustering explains connectivity statistics in cortical microcircuits.

Local cortical circuits appear highly non-random, but the underlying connectivity rule remains elusive. Here, we analyze experimental data observed in layer 5 of rat neocortex and suggest a model for connectivity from which emerge essential observed non-random features of both wiring and weighting. These features include lognormal distributions of synaptic connection strength, anatomical clustering, and strong correlations between clustering and connection strength. Our model predicts that cortical microcircuits contain large groups of densely connected neurons which we call clusters. We show that such a cluster contains about one fifth of all excitatory neurons of a circuit which are very densely connected with stronger than average synapses. We demonstrate that such clustering plays an important role in the network dynamics, namely, it creates bistable neural spiking in small cortical circuits. Furthermore, introducing local clustering in large-scale networks leads to the emergence of various patterns of persistent local activity in an ongoing network activity. Thus, our results may bridge a gap between anatomical structure and persistent activity observed during working memory and other cognitive processes.

Optimal spike-based communication in excitable networks with strong-sparse and weak-dense links.

The connectivity of complex networks and functional implications has been attracting much interest in many physical, biological and social systems. However, the significance of the weight distributions of network links remains largely unknown except for uniformly- or Gaussian-weighted links. Here, we show analytically and numerically, that recurrent neural networks can robustly generate internal noise optimal for spike transmission between neurons with the help of a long-tailed distribution in the weights of recurrent connections. The structure of spontaneous activity in such networks involves weak-dense connections that redistribute excitatory activity over the network as noise sources to optimally enhance the responses of individual neurons to input at sparse-strong connections, thus opening multiple signal transmission pathways. Electrophysiological experiments confirm the importance of a highly broad connectivity spectrum supported by the model. Our results identify a simple network mechanism for internal noise generation by highly inhomogeneous connection strengths supporting both stability and optimal communication.

Associative memory model with long-tail-distributed Hebbian synaptic connections.

The postsynaptic potentials of pyramidal neurons have a non-Gaussian amplitude distribution with a heavy tail in both hippocampus and neocortex. Such distributions of synaptic weights were recently shown to generate spontaneous internal noise optimal for spike propagation in recurrent cortical circuits. However, whether this internal noise generation by heavy-tailed weight distributions is possible for and beneficial to other computational functions remains unknown. To clarify this point, we construct an associative memory (AM) network model of spiking neurons that stores multiple memory patterns in a connection matrix with a lognormal weight distribution. In AM networks, non-retrieved memory patterns generate a cross-talk noise that severely disturbs memory recall. We demonstrate that neurons encoding a retrieved memory pattern and those encoding non-retrieved memory patterns have different subthreshold membrane-potential distributions in our model. Consequently, the probability of responding to inputs at strong synapses increases for the encoding neurons, whereas it decreases for the non-encoding neurons. Our results imply that heavy-tailed distributions of connection weights can generate noise useful for AM recall.

Effective long-time phase dynamics of limit-cycle oscillators driven by weak colored noise.

An effective white-noise Langevin equation is derived that describes long-time phase dynamics of a limit-cycle oscillator driven by weak stationary colored noise. Effective drift and diffusion coefficients are given in terms of the phase sensitivity of the oscillator and the correlation function of the noise, and are explicitly calculated for oscillators with sinusoidal phase sensitivity functions driven by two typical colored Gaussian processes. The results are verified by numerical simulations using several types of stochastic or chaotic noise. The drift and diffusion coefficients of oscillators driven by chaotic noise exhibit anomalous dependence on the oscillator frequency, reflecting the peculiar power spectrum of the chaotic noise.

Dynamics of limit-cycle oscillators subject to general noise.

The phase description is a powerful tool for analyzing noisy limit-cycle oscillators. The method, however, has found only limited applications so far, because the present theory is applicable only to Gaussian noise while noise in the real world often has non-Gaussian statistics. Here, we provide the phase reduction method for limit-cycle oscillators subject to general, colored and non-Gaussian, noise including a heavy-tailed one. We derive quantifiers like mean frequency, diffusion constant, and the Lyapunov exponent to confirm consistency of the results. Applying our results, we additionally study a resonance between the phase and noise.

Stochastic phase reduction for a general class of noisy limit cycle oscillators.

We formulate a phase-reduction method for a general class of noisy limit cycle oscillators and find that the phase equation is parametrized by the ratio between time scales of the noise correlation and amplitude relaxation of the limit cycle. The equation naturally includes previously proposed and mutually exclusive phase equations as special cases. The validity of the theory is numerically confirmed. Using the method, we reveal how noise and its correlation time affect limit cycle oscillations.

Temporal precision of spike response to fluctuating input in pulse-coupled networks of oscillating neurons.

A single neuron is known to generate almost identical spike trains when the same fluctuating input is repeatedly applied. Here, we study the reliability of spike firing in a pulse-coupled network of oscillator neurons receiving fluctuating inputs. We can study the precise responses of the network as synchronization between uncoupled copies of the network by a common noisy input. To study the noise-induced synchronization between networks, we derive a self-consistent equation for the distribution of spike-time differences between the networks. Solving this equation, we elucidate how the spike precision changes as a function of the coupling strength.

Complex evolution of spike patterns during burst propagation through feed-forward networks.

Stable signal transmission is crucial for information processing by the brain. Synfire-chains, defined as feed-forward networks of spiking neurons, are a well-studied class of circuit structure that can propagate a packet of single spikes while maintaining a fixed packet profile. Here, we studied the stable propagation of spike bursts, rather than single spike activities, in a feed-forward network of a general class of excitable bursting neurons. In contrast to single spikes, bursts can propagate stably without converging to any fixed profiles. Spike timings of bursts continue to change cyclically or irregularly during propagation depending on intrinsic properties of the neurons and the coupling strength of the network. To find the conditions under which bursts lose fixed profiles, we propose an analysis based on timing shifts of burst spikes similar to the phase response analysis of limit-cycle oscillators.

[Formation of cortical cell assemblies--synaptic plasticity and beyond].

Information processing by the brain relies on the functions of neuronal networks. Therefore, understanding the structure and computational mechanisms of the brain circuitry is crucial for clarifying how cognitive functions emerge and how they can be best modeled for engineering applications. The manner in which neocortical and hippocampal circuits represent and process information has not been understood in detail. However, if sloppy processors like neurons can process enormous amounts of information efficiently, a large assembly of neurons is likely to operate in a parallel manner. In this report, we discuss the structure of cortical circuits that undergo self-organization through spike-timing-dependent plasticity, under the influence of two-state membrane potential fluctuations. Furthermore, we propose a stochastic rule for the generation of synapses, i.e., neuronal wiring, in a large population of cortical neurons.

Sequential associative memory with nonuniformity of the layer sizes.

Sequence retrieval has a fundamental importance in information processing by the brain, and has extensively been studied in neural network models. Most of the previous sequential associative memory embedded sequences of memory patterns have nearly equal sizes. It was recently shown that local cortical networks display many diverse yet repeatable precise temporal sequences of neuronal activities, termed "neuronal avalanches." Interestingly, these avalanches displayed size and lifetime distributions that obey power laws. Inspired by these experimental findings, here we consider an associative memory model of binary neurons that stores sequences of memory patterns with highly variable sizes. Our analysis includes the case where the statistics of these size variations obey the above-mentioned power laws. We study the retrieval dynamics of such memory systems by analytically deriving the equations that govern the time evolution of macroscopic order parameters. We calculate the critical sequence length beyond which the network cannot retrieve memory sequences correctly. As an application of the analysis, we show how the present variability in sequential memory patterns degrades the power-law lifetime distribution of retrieved neural activities.

Local cortical circuit model inferred from power-law distributed neuronal avalanches.

How cortical neurons process information crucially depends on how their local circuits are organized. Spontaneous synchronous neuronal activity propagating through neocortical slices displays highly diverse, yet repeatable, activity patterns called "neuronal avalanches". They obey power-law distributions of the event sizes and lifetimes, presumably reflecting the structure of local circuits developed in slice cultures. However, the explicit network structure underlying the power-law statistics remains unclear. Here, we present a neuronal network model of pyramidal and inhibitory neurons that enables stable propagation of avalanche-like spiking activity. We demonstrate a neuronal wiring rule that governs the formation of mutually overlapping cell assemblies during the development of this network. The resultant network comprises a mixture of feedforward chains and recurrent circuits, in which neuronal avalanches are stable if the former structure is predominant. Interestingly, the recurrent synaptic connections formed by this wiring rule limit the number of cell assemblies embeddable in a neuron pool of given size. We investigate how the resultant power laws depend on the details of the cell-assembly formation as well as on the inhibitory feedback. Our model suggests that local cortical circuits may have a more complex topological design than has previously been thought.

Synchronization of excitatory neurons with strongly heterogeneous phase responses.

In many real-world oscillator systems, the phase response curves are highly heterogeneous. However, the dynamics of heterogeneous oscillator networks has not been seriously addressed. We propose a theoretical framework to analyze such a system by dealing explicitly with the heterogeneous phase response curves. We develop a method to solve the self-consistent equations for order parameters by using formal complex-valued phase variables, and apply our theory to networks of in vitro cortical neurons. We find a novel state transition that is not observed in previous oscillator network models.

A cellular mechanism for graded persistent activity in a model neuron and its implications in working memory.

Working memory represents the ability of the brain to hold information for relatively short periods of time. Working memory is believed mediated by persistent neuronal firing. A broadly accepted hypothesis is that persistent activity is generated by reverberating synaptic input. However, single cortical neurons capable of showing persistent firing were recently reported. In this modeling study, we propose a cellular mechanism to generate persistent firing of multiple firing rates in single neurons. In the proposed mechanism, bistable concentrations of inositol 1,4,5-trisphosphate (IP3) and Ca2+ is achieved by IP3 formation and IP3-induced Ca2+ release from stores in multiple subcellular domains. A postsynaptic firing rate-dependent switching of these bistable elements can demonstrate graded persistent firing of the rat entorhinal neurons. Such a firing rate-dependent switching may be extended to a variety of intracellular Ca2+ signaling cascades.

Robustness of the noise-induced phase synchronization in a general class of limit cycle oscillators.

We show that a wide class of uncoupled limit-cycle oscillators can be in-phase synchronized by common weak additive noise. An expression of the Lyapunov exponent is analytically derived to study the stability of the noise-driven synchronizing state. The result shows that such a synchronization can be achieved in a broad class of oscillators with little constraint on their intrinsic property. On the other hand, the leaky integrate-and-fire neuron oscillators do not belong to this class, generating intermittent phase slips according to a power law distribution of their intervals.