Bimodular continuous attractor neural networks with static and moving stimuli.

We investigated the dynamical behaviors of bimodular continuous attractor neural networks, each processing a modality of sensory input and interacting with each other. We found that when bumps coexist in both modules, the position of each bump is shifted towards the other input when the intermodular couplings are excitatory and is shifted away when inhibitory. When one intermodular coupling is excitatory while another is moderately inhibitory, temporally modulated population spikes can be generated. On further increase of the inhibitory coupling, momentary spikes will emerge. In the regime of bump coexistence, bump heights are primarily strengthened by excitatory intermodular couplings, but there is a lesser weakening effect due to a bump being displaced from the direct input. When bimodular networks serve as decoders of multisensory integration, we extend the Bayesian framework to show that excitatory and inhibitory couplings encode attractive and repulsive priors, respectively. At low disparity, the bump positions decode the posterior means in the Bayesian framework, whereas at high disparity, multiple steady states exist. In the regime of multiple steady states, the less stable state can be accessed if the input causing the more stable state arrives after a sufficiently long delay. When one input is moving, the bump in the corresponding module is pinned when the moving stimulus is weak, unpinned at intermediate stimulus strength, and tracks the input at strong stimulus strength, and the stimulus strengths for these transitions increase with the velocity of the moving stimulus. These results are important to understanding multisensory integration of static and dynamic stimuli.

Bifurcation behaviors shape how continuous physical dynamics solves discrete Ising optimization.

Simulating physical dynamics to solve hard combinatorial optimization has proven effective for medium- to large-scale problems. The dynamics of such systems is continuous, with no guarantee of finding optimal solutions of the original discrete problem. We investigate the open question of when simulated physical solvers solve discrete optimizations correctly, with a focus on coherent Ising machines (CIMs). Having established the existence of an exact mapping between CIM dynamics and discrete Ising optimization, we report two fundamentally distinct bifurcation behaviors of the Ising dynamics at the first bifurcation point: either all nodal states simultaneously deviate from zero (synchronized bifurcation) or undergo a cascade of such deviations (retarded bifurcation). For synchronized bifurcation, we prove that when the nodal states are uniformly bounded away from the origin, they contain sufficient information for exactly solving the Ising problem. When the exact mapping conditions are violated, subsequent bifurcations become necessary and often cause slow convergence. Inspired by those findings, we devise a trapping-and-correction (TAC) technique to accelerate dynamics-based Ising solvers, including CIMs and simulated bifurcation. TAC takes advantage of early bifurcated "trapped nodes" which maintain their sign throughout the Ising dynamics to reduce computation time effectively. Using problem instances from open benchmark and random Ising models, we validate the superior convergence and accuracy of TAC.

Neural feedback facilitates rough-to-fine information retrieval.

Categorical relationships between objects are encoded as overlapped neural representations in the brain, where the more similar the objects are, the larger the correlations between their evoked neuronal responses. These representation correlations, however, inevitably incur interference when memories are retrieved. Here, we propose that neural feedback, which is widely observed in the brain but whose function remains largely unknown, contributes to disentangle neural correlations to improve information retrieval. We study a hierarchical neural network storing the hierarchical categorical information of objects, and information retrieval goes from rough-to-fine, aided by the push-pull neural feedback. We elucidate that the push and the pull components of the feedback suppress the interferences due to the representation correlations between objects from different and the same categories, respectively. Our model reproduces the push-pull phenomenon observed in neural data and sheds light on our understanding of the role of feedback in neural information processing.

Associative memory model with arbitrary Hebbian length.

Conversion of temporal to spatial correlations in the cortex is one of the most intriguing functions in the brain. The learning at synapses triggering the correlation conversion can take place in a wide integration window, whose influence on the correlation conversion remains elusive. Here we propose a generalized associative memory model of pattern sequences, in which pattern separations within an arbitrary Hebbian length are learned. The model can be analytically solved, and predicts that a small Hebbian length can already significantly enhance the correlation conversion, i.e., the stimulus-induced attractor can be highly correlated with a significant number of patterns in the stored sequence, thereby facilitating state transitions in the neural representation space. Moreover, an anti-Hebbian component is able to reshape the energy landscape of memories, akin to the memory regulation function during sleep. Our work thus establishes the fundamental connection between associative memory, Hebbian length, and correlation conversion in the brain.

Eigenvalue spectrum of neural networks with arbitrary Hebbian length.

Associative memory is a fundamental function in the brain. Here, we generalize the standard associative memory model to include long-range Hebbian interactions at the learning stage, corresponding to a large synaptic integration window. In our model, the Hebbian length can be arbitrarily large. The spectral density of the coupling matrix is derived using the replica method, which is also shown to be consistent with the results obtained by applying the free probability method. The maximal eigenvalue is then obtained by an iterative equation, related to the paramagnetic to spin glass transition in the model. Altogether, this work establishes the connection between the associative memory with arbitrary Hebbian length and the asymptotic eigen-spectrum of the neural-coupling matrix.

Optimizing synchronization stability of the Kuramoto model in complex networks and power grids.

Maintaining the stability of synchronization state is crucial for the functioning of many natural and artificial systems. In this study, we develop methods to optimize the synchronization stability of the Kuramoto model by minimizing the dominant Lyapunov exponent. Using the recently proposed cut-set space approximation of the steady states, we greatly simplify the objective function, and further derive its gradient and Hessian with respect to natural frequencies, which leads to an efficient algorithm with the quasi-Newton's method. The optimized systems are demonstrated to achieve better synchronization stability for the Kuramoto model with or without inertia in certain regimes. Hence our method is applicable in improving the stability of power grids. It is also viable to adjust the coupling strength of each link to improve the stability of the system. Various operational constraints can also be easily integrated into our scope by employing the interior point method in convex optimization. The properties of the optimized networks are also discussed.

Evolving power grids with self-organized intermittent strain releases: An analogy with sandpile models and earthquakes.

The stability of powergrid is crucial since its disruption affects systems ranging from street lightings to hospital life-support systems. While short-term dynamics of single-event cascading failures have been extensively studied, less is understood on the long-term evolution and self-organization of powergrids. In this paper, we introduce a simple model of evolving powergrid and establish its connection with the sandpile model and earthquakes, i.e., self-organized systems with intermittent strain releases. Various aspects during its self-organization are examined, including blackout magnitudes, their interevent waiting time, the predictability of large blackouts, as well as the spatiotemporal rescaling of blackout data. We examined the self-organized strain releases on simulated networks as well as the IEEE 118-bus system, and we show that both simulated and empirical blackout waiting times can be rescaled in space and time similarly to those observed between earthquakes. Finally, we suggested proactive maintenance strategies to drive the powergrids away from self-organization to suppress large blackouts.

Continuous Attractor Neural Networks: Candidate of a Canonical Model for Neural Information Representation.

Owing to its many computationally desirable properties, the model of continuous attractor neural networks (CANNs) has been successfully applied to describe the encoding of simple continuous features in neural systems, such as orientation, moving direction, head direction, and spatial location of objects. Recent experimental and computational studies revealed that complex features of external inputs may also be encoded by low-dimensional CANNs embedded in the high-dimensional space of neural population activity. The new experimental data also confirmed the existence of the M-shaped correlation between neuronal responses, which is a correlation structure associated with the unique dynamics of CANNs. This body of evidence, which is reviewed in this report, suggests that CANNs may serve as a canonical model for neural information representation.

Stochastic Oscillation in Self-Organized Critical States of Small Systems: Sensitive Resting State in Neural Systems.

Self-organized critical states (SOCs) and stochastic oscillations (SOs) are simultaneously observed in neural systems, which appears to be theoretically contradictory since SOCs are characterized by scale-free avalanche sizes but oscillations indicate typical scales. Here, we show that SOs can emerge in SOCs of small size systems due to temporal correlation between large avalanches at the finite-size cutoff, resulting from the accumulation-release process in SOCs. In contrast, the critical branching process without accumulation-release dynamics cannot exhibit oscillations. The reconciliation of SOCs and SOs is demonstrated both in the sandpile model and robustly in biologically plausible neuronal networks. The oscillations can be suppressed if external inputs eliminate the prominent slow accumulation process, providing a potential explanation of the widely studied Berger effect or event-related desynchronization in neural response. The features of neural oscillations and suppression are confirmed during task processing in monkey eye-movement experiments. Our results suggest that finite-size, columnar neural circuits may play an important role in generating neural oscillations around the critical states, potentially enabling functional advantages of both SOCs and oscillations for sensitive response to transient stimuli.

Rich spectrum of neural field dynamics in the presence of short-term synaptic depression.

In continuous attractor neural networks (CANNs), spatially continuous information such as orientation, head direction, and spatial location is represented by Gaussian-like tuning curves that can be displaced continuously in the space of the preferred stimuli of the neurons. We investigate how short-term synaptic depression (STD) can reshape the intrinsic dynamics of the CANN model and its responses to a single static input. In particular, CANNs with STD can support various complex firing patterns and chaotic behaviors. These chaotic behaviors have the potential to encode various stimuli in the neuronal system.

Fluctuation-response relation unifies dynamical behaviors in neural fields.

Anticipation is a strategy used by neural fields to compensate for transmission and processing delays during the tracking of dynamical information and can be achieved by slow, localized, inhibitory feedback mechanisms such as short-term synaptic depression, spike-frequency adaptation, or inhibitory feedback from other layers. Based on the translational symmetry of the mobile network states, we derive generic fluctuation-response relations, providing unified predictions that link their tracking behaviors in the presence of external stimuli to the intrinsic dynamics of the neural fields in their absence.

Coverage versus supply cost in facility location: physics of frustrated spin systems.

A comprehensive coverage is crucial for communication, supply, and transportation networks, yet it is limited by the requirement of extensive infrastructure and heavy energy consumption. Here, we draw an analogy between spins in antiferromagnet and outlets in supply networks, and apply techniques from the studies of disordered systems to elucidate the effects of balancing the coverage and supply costs on the network behavior. A readily applicable, coverage optimization algorithm is derived. Simulation results show that magnetized and antiferromagnetic domains emerge and coexist to balance the need for coverage and energy saving. The scaling of parameters with system size agrees with the continuum approximation in two dimensions and the tree approximation in random graphs. Due to frustration caused by the competition between coverage and supply cost, a transition between easy and hard computation regimes is observed. We further suggest a local expansion approach to greatly simplify the message updates which shed light on simplifications in other problems.

From the physics of interacting polymers to optimizing routes on the London Underground.

Optimizing paths on networks is crucial for many applications, ranging from subway traffic to Internet communication. Because global path optimization that takes account of all path choices simultaneously is computationally hard, most existing routing algorithms optimize paths individually, thus providing suboptimal solutions. We use the physics of interacting polymers and disordered systems to analyze macroscopic properties of generic path optimization problems and derive a simple, principled, generic, and distributed routing algorithm capable of considering all individual path choices simultaneously. We demonstrate the efficacy of the algorithm by applying it to: (i) random graphs resembling Internet overlay networks, (ii) travel on the London Underground network based on Oyster card data, and (iii) the global airport network. Analytically derived macroscopic properties give rise to insightful new routing phenomena, including phase transitions and scaling laws, that facilitate better understanding of the appropriate operational regimes and their limitations, which are difficult to obtain otherwise.

Resolution enhancement in neural networks with dynamical synapses.

Conventionally, information is represented by spike rates in the neural system. Here, we consider the ability of temporally modulated activities in neuronal networks to carry information extra to spike rates. These temporal modulations, commonly known as population spikes, are due to the presence of synaptic depression in a neuronal network model. We discuss its relevance to an experiment on transparent motions in macaque monkeys by Treue et al. in 2000. They found that if the moving directions of objects are too close, the firing rate profile will be very similar to that with one direction. As the difference in the moving directions of objects is large enough, the neuronal system would respond in such a way that the network enhances the resolution in the moving directions of the objects. In this paper, we propose that this behavior can be reproduced by neural networks with dynamical synapses when there are multiple external inputs. We will demonstrate how resolution enhancement can be achieved, and discuss the conditions under which temporally modulated activities are able to enhance information processing performances in general.

Dynamical synapses enhance neural information processing: gracefulness, accuracy, and mobility.

Experimental data have revealed that neuronal connection efficacy exhibits two forms of short-term plasticity: short-term depression (STD) and short-term facilitation (STF). They have time constants residing between fast neural signaling and rapid learning and may serve as substrates for neural systems manipulating temporal information on relevant timescales. This study investigates the impact of STD and STF on the dynamics of continuous attractor neural networks and their potential roles in neural information processing. We find that STD endows the network with slow-decaying plateau behaviors: the network that is initially being stimulated to an active state decays to a silent state very slowly on the timescale of STD rather than on that of neuralsignaling. This provides a mechanism for neural systems to hold sensory memory easily and shut off persistent activities gracefully. With STF, we find that the network can hold a memory trace of external inputs in the facilitated neuronal interactions, which provides a way to stabilize the network response to noisy inputs, leading to improved accuracy in population decoding. Furthermore, we find that STD increases the mobility of the network states. The increased mobility enhances the tracking performance of the network in response to time-varying stimuli, leading to anticipative neural responses. In general, we find that STD and STP tend to have opposite effects on network dynamics and complementary computational advantages, suggesting that the brain may employ a strategy of weighting them differentially depending on the computational purpose.

A moving bump in a continuous manifold: a comprehensive study of the tracking dynamics of continuous attractor neural networks.

Understanding how the dynamics of a neural network is shaped by the network structure and, consequently, how the network structure facilitates the functions implemented by the neural system is at the core of using mathematical models to elucidate brain functions. This study investigates the tracking dynamics of continuous attractor neural networks (CANNs). Due to the translational invariance of neuronal recurrent interactions, CANNs can hold a continuous family of stationary states. They form a continuous manifold in which the neural system is neutrally stable. We systematically explore how this property facilitates the tracking performance of a CANN, which is believed to have clear correspondence with brain functions. By using the wave functions of the quantum harmonic oscillator as the basis, we demonstrate how the dynamics of a CANN is decomposed into different motion modes, corresponding to distortions in the amplitude, position, width, or skewness of the network state. We then develop a perturbation approach that utilizes the dominating movement of the network's stationary states in the state space. This method allows us to approximate the network dynamics up to an arbitrary accuracy depending on the order of perturbation used. We quantify the distortions of a gaussian bump during tracking and study their effects on tracking performance. Results are obtained on the maximum speed for a moving stimulus to be trackable and the reaction time for the network to catch up with an abrupt change in the stimulus.

Phase transitions in transportation networks with nonlinearities.

We investigate a model of transportation networks with nonlinear elements which may represent local shortage of resources. Frustrations arise from competition for resources. When the initial resources are uniform, different regimes with discrete fractions of satisfied nodes are observed, resembling the Devil's staircase. We demonstrate how functional recursions are converted to simple recursions of probabilities. Behaviors similar to those in the vertex cover or close packing problems are found. When the initial resources are bimodally distributed, increases in the fraction of rich nodes induce a glassy transition, entering an algorithmically hard regime.

Effects of payoff functions and preference distributions in an adaptive population.

Adaptive populations such as those in financial markets and distributed control can be modeled by the Minority Game. We consider how their dynamics depends on the agents' initial preferences of strategies, when the agents use linear or quadratic payoff functions to evaluate their strategies. We find that the fluctuations of the population making certain decisions (the volatility) depends on the diversity of the distribution of the initial preferences of strategies. When the diversity decreases, more agents tend to adapt their strategies together. In systems with linear payoffs, this results in dynamical transitions from vanishing volatility to a nonvanishing one. For low signal dimensions, the dynamical transitions for the different signals do not take place at the same critical diversity. Rather, a cascade of dynamical transitions takes place when the diversity is reduced. In contrast, no phase transitions are found in systems with the quadratic payoffs. Instead, a basin boundary of attraction separates two groups of samples in the space of the agents' decisions. Initial states inside this boundary converge to small volatility, while those outside diverge to a large one. Furthermore, when the preference distribution becomes more polarized, the dynamics becomes more erratic. All the above results are supported by good agreement between simulations and theory.

Models of financial markets with extensive participation incentives.

We consider models of financial markets in which all parties involved find incentives to participate. Strategies are evaluated directly by their virtual wealth. By tuning the price sensitivity and market impact, a phase diagram with several attractor behaviors resembling those of real markets emerge, reflecting the roles played by the arbitrageurs and trendsetters, and including a phase with irregular price trends and positive sums. The positive sumness of the players' wealth provides participation incentives for them. Evolution and the bid-ask spread provide mechanisms for the gain in wealth of both the players and market makers. New players survive in the market if the evolutionary rate is sufficiently slow. We test the applicability of the model on real Hang Seng Index data over 20 years. Comparisons with other models show that our model has a superior average performance when applied to real financial data.