Recurrent chaotic clustering and slow chaos in adaptive networks.

Adaptive dynamical networks are network systems in which the structure co-evolves and interacts with the dynamical state of the nodes. We study an adaptive dynamical network in which the structure changes on a slower time scale relative to the fast dynamics of the nodes. We identify a phenomenon we refer to as recurrent adaptive chaotic clustering (RACC), in which chaos is observed on a slow time scale, while the fast time scale exhibits regular dynamics. Such slow chaos is further characterized by long (relative to the fast time scale) regimes of frequency clusters or frequency-synchronized dynamics, interrupted by fast jumps between these regimes. We also determine parameter values where the time intervals between jumps are chaotic and show that such a state is robust to changes in parameters and initial conditions.

Modeling brain network flexibility in networks of coupled oscillators: a feasibility study.

Modeling the functionality of the human brain is a major goal in neuroscience for which many powerful methodologies have been developed over the last decade. The impact of working memory and the associated brain regions on the brain dynamics is of particular interest due to their connection with many functions and malfunctions in the brain. In this context, the concept of brain flexibility has been developed for the characterization of brain functionality. We discuss emergence of brain flexibility that is commonly measured by the identification of changes in the cluster structure of co-active brain regions. We provide evidence that brain flexibility can be modeled by a system of coupled FitzHugh-Nagumo oscillators where the network structure is obtained from human brain Diffusion Tensor Imaging (DTI). Additionally, we propose a straightforward and computationally efficient alternative macroscopic measure, which is derived from the Pearson distance of functional brain matrices. This metric exhibits similarities to the established patterns of brain template flexibility that have been observed in prior investigations. Furthermore, we explore the significance of the brain's network structure and the strength of connections between network nodes or brain regions associated with working memory in the observation of patterns in networks flexibility. This work enriches our understanding of the interplay between the structure and function of dynamic brain networks and proposes a modeling strategy to study brain flexibility.

Bursting multistability induced by double-Hopf bifurcation.

We study the slow-fast dynamics of a system with a double-Hopf bifurcation and a slowly varying parameter. The model consists of coupled Bonhöffer-van der Pol oscillators excited by a periodic slow-varying AC source. We consider two cases where the slowly varying parameter passes by or crosses the double-Hopf bifurcation, respectively. Due to the system's multistability, two bursting solutions are observed in each case: single-mode bursting and two-mode bursting. Further investigation reveals that the double-Hopf bifurcation causes a stable coexistence of these two bursting solutions. The mechanism of such coexistence is explained using the slowly changing phase portraits of the fast subsystem. We also show the robustness of the observed effect in the vicinity of the double-Hopf bifurcation.

Extreme multistability in symmetrically coupled clocks.

Extreme multistability (EM) is characterized by the emergence of infinitely many coexisting attractors or continuous families of stable states in dynamical systems. EM implies complex and hardly predictable asymptotic dynamical behavior. We analyze a model for pendulum clocks coupled by springs and suspended on an oscillating base and show how EM can be induced in this system by specifically designed coupling. First, we uncover that symmetric coupling can increase the dynamical complexity. In particular, the coexistence of multiple isolated attractors and continuous families of stable periodic states is generated in a symmetric cross-coupling scheme of four pendulums. These coexisting infinitely many states are characterized by different levels of phase synchronization between the pendulums, including anti-phase and in-phase states. Some of the states are characterized by splitting of the pendulums into groups with silent sub-threshold and oscillating behavior, respectively. The analysis of the basins of attraction further reveals the complex dependence of EM on initial conditions.

Rate and memory effects in bifurcation-induced tipping.

A variation in the environment of a system, such as the temperature, the concentration of a chemical solution, or the appearance of a magnetic field, may lead to a drift in one of the parameters. If the parameter crosses a bifurcation point, the system can tip from one attractor to another (bifurcation-induced tipping). Typically, this stability exchange occurs at a parameter value beyond the bifurcation value. This is what we call, here, the shifted stability exchange. We perform a systematic study on how the shift is affected by the initial parameter value and its change rate. To that end, we present numerical simulations and partly analytical results for different types of bifurcations and different paradigmatic systems. We show that the nonautonomous dynamics can be split into two regimes. Depending on whether we exceed the numerical or experimental precision or not, the system may enter the nondeterministic or the deterministic regime. This is determined solely by the conditions of the drift. Finally, we deduce the scaling laws governing this phenomenon and we observe very similar behavior for different systems and different bifurcations in both regimes.

Perspectives on adaptive dynamical systems.

Adaptivity is a dynamical feature that is omnipresent in nature, socio-economics, and technology. For example, adaptive couplings appear in various real-world systems, such as the power grid, social, and neural networks, and they form the backbone of closed-loop control strategies and machine learning algorithms. In this article, we provide an interdisciplinary perspective on adaptive systems. We reflect on the notion and terminology of adaptivity in different disciplines and discuss which role adaptivity plays for various fields. We highlight common open challenges and give perspectives on future research directions, looking to inspire interdisciplinary approaches.

Asymmetric adaptivity induces recurrent synchronization in complex networks.

Rhythmic activities that alternate between coherent and incoherent phases are ubiquitous in chemical, ecological, climate, or neural systems. Despite their importance, general mechanisms for their emergence are little understood. In order to fill this gap, we present a framework for describing the emergence of recurrent synchronization in complex networks with adaptive interactions. This phenomenon is manifested at the macroscopic level by temporal episodes of coherent and incoherent dynamics that alternate recurrently. At the same time, the dynamics of the individual nodes do not change qualitatively. We identify asymmetric adaptation rules and temporal separation between the adaptation and the dynamics of individual nodes as key features for the emergence of recurrent synchronization. Our results suggest that asymmetric adaptation might be a fundamental ingredient for recurrent synchronization phenomena as seen in pattern generators, e.g., in neuronal systems.

Heterogeneous Nucleation in Finite-Size Adaptive Dynamical Networks.

Phase transitions in equilibrium and nonequilibrium systems play a major role in the natural sciences. In dynamical networks, phase transitions organize qualitative changes in the collective behavior of coupled dynamical units. Adaptive dynamical networks feature a connectivity structure that changes over time, coevolving with the nodes' dynamical state. In this Letter, we show the emergence of two distinct first-order nonequilibrium phase transitions in a finite-size adaptive network of heterogeneous phase oscillators. Depending on the nature of defects in the internal frequency distribution, we observe either an abrupt single-step transition to full synchronization or a more gradual multistep transition. This observation has a striking resemblance to heterogeneous nucleation. We develop a mean-field approach to study the interplay between adaptivity and nodal heterogeneity and describe the dynamics of multicluster states and their role in determining the character of the phase transition. Our work provides a theoretical framework for studying the interplay between adaptivity and nodal heterogeneity.

Master Memory Function for Delay-Based Reservoir Computers With Single-Variable Dynamics.

We show that many delay-based reservoir computers considered in the literature can be characterized by a universal master memory function (MMF). Once computed for two independent parameters, this function provides linear memory capacity for any delay-based single-variable reservoir with small inputs. Moreover, we propose an analytical description of the MMF that enables its efficient and fast computation. Our approach can be applied not only to single-variable delay-based reservoirs governed by known dynamical rules, such as the Mackey-Glass or Stuart-Landau-like systems, but also to reservoirs whose dynamical model is not available.

Collective Activity Bursting in a Population of Excitable Units Adaptively Coupled to a Pool of Resources.

We study the collective dynamics in a population of excitable units (neurons) adaptively interacting with a pool of resources. The resource pool is influenced by the average activity of the population, whereas the feedback from the resources to the population is comprised of components acting homogeneously or inhomogeneously on individual units of the population. Moreover, the resource pool dynamics is assumed to be slow and has an oscillatory degree of freedom. We show that the feedback loop between the population and the resources can give rise to collective activity bursting in the population. To explain the mechanisms behind this emergent phenomenon, we combine the Ott-Antonsen reduction for the collective dynamics of the population and singular perturbation theory to obtain a reduced system describing the interaction between the population mean field and the resources.

Generalized splay states in phase oscillator networks.

Networks of coupled phase oscillators play an important role in the analysis of emergent collective phenomena. In this article, we introduce generalized m-splay states constituting a special subclass of phase-locked states with vanishing mth order parameter. Such states typically manifest incoherent dynamics, and they often create high-dimensional families of solutions (splay manifolds). For a general class of phase oscillator networks, we provide explicit linear stability conditions for splay states and exemplify our results with the well-known Kuramoto-Sakaguchi model. Importantly, our stability conditions are expressed in terms of just a few observables such as the order parameter or the trace of the Jacobian. As a result, these conditions are simple and applicable to networks of arbitrary size. We generalize our findings to phase oscillators with inertia and adaptively coupled phase oscillator models.

Deep neural networks using a single neuron: folded-in-time architecture using feedback-modulated delay loops.

Deep neural networks are among the most widely applied machine learning tools showing outstanding performance in a broad range of tasks. We present a method for folding a deep neural network of arbitrary size into a single neuron with multiple time-delayed feedback loops. This single-neuron deep neural network comprises only a single nonlinearity and appropriately adjusted modulations of the feedback signals. The network states emerge in time as a temporal unfolding of the neuron's dynamics. By adjusting the feedback-modulation within the loops, we adapt the network's connection weights. These connection weights are determined via a back-propagation algorithm, where both the delay-induced and local network connections must be taken into account. Our approach can fully represent standard Deep Neural Networks (DNN), encompasses sparse DNNs, and extends the DNN concept toward dynamical systems implementations. The new method, which we call Folded-in-time DNN (Fit-DNN), exhibits promising performance in a set of benchmark tasks.

What adaptive neuronal networks teach us about power grids.

Power grid networks, as well as neuronal networks with synaptic plasticity, describe real-world systems of tremendous importance for our daily life. The investigation of these seemingly unrelated types of dynamical networks has attracted increasing attention over the past decade. In this paper, we provide insight into the fundamental relation between these two types of networks. For this, we consider well-established models based on phase oscillators and show their intimate relation. In particular, we prove that phase oscillator models with inertia can be viewed as a particular class of adaptive networks. This relation holds even for more general classes of power grid models that include voltage dynamics. As an immediate consequence of this relation, we discover a plethora of multicluster states for phase oscillators with inertia. Moreover, the phenomenon of cascading line failure in power grids is translated into an adaptive neuronal network.

Desynchronization Transitions in Adaptive Networks.

Adaptive networks change their connectivity with time, depending on their dynamical state. While synchronization in structurally static networks has been studied extensively, this problem is much more challenging for adaptive networks. In this Letter, we develop the master stability approach for a large class of adaptive networks. This approach allows for reducing the synchronization problem for adaptive networks to a low-dimensional system, by decoupling topological and dynamical properties. We show how the interplay between adaptivity and network structure gives rise to the formation of stability islands. Moreover, we report a desynchronization transition and the emergence of complex partial synchronization patterns induced by an increasing overall coupling strength. We illustrate our findings using adaptive networks of coupled phase oscillators and FitzHugh-Nagumo neurons with synaptic plasticity.

Deep time-delay reservoir computing: Dynamics and memory capacity.

The deep time-delay reservoir computing concept utilizes unidirectionally connected systems with time-delays for supervised learning. We present how the dynamical properties of a deep Ikeda-based reservoir are related to its memory capacity (MC) and how that can be used for optimization. In particular, we analyze bifurcations of the corresponding autonomous system and compute conditional Lyapunov exponents, which measure generalized synchronization between the input and the layer dynamics. We show how the MC is related to the systems' distance to bifurcations or magnitude of the conditional Lyapunov exponent. The interplay of different dynamical regimes leads to an adjustable distribution between the linear and nonlinear MC. Furthermore, numerical simulations show resonances between the clock cycle and delays of the layers in all degrees of MC. Contrary to MC losses in single-layer reservoirs, these resonances can boost separate degrees of MC and can be used, e.g., to design a system with maximum linear MC. Accordingly, we present two configurations that empower either high nonlinear MC or long time linear MC.

Dynamics of a stochastic excitable system with slowly adapting feedback.

We study an excitable active rotator with slowly adapting nonlinear feedback and noise. Depending on the adaptation and the noise level, this system may display noise-induced spiking, noise-perturbed oscillations, or stochastic bursting. We show how the system exhibits transitions between these dynamical regimes, as well as how one can enhance or suppress the coherence resonance or effectively control the features of the stochastic bursting. The setup can be considered a paradigmatic model for a neuron with a slow recovery variable or, more generally, as an excitable system under the influence of a nonlinear control mechanism. We employ a multiple timescale approach that combines the classical adiabatic elimination with averaging of rapid oscillations and stochastic averaging of noise-induced fluctuations by a corresponding stationary Fokker-Planck equation. This allows us to perform a numerical bifurcation analysis of a reduced slow system and to determine the parameter regions associated with different types of dynamics. In particular, we demonstrate the existence of a region of bistability, where the noise-induced switching between a stationary and an oscillatory regime gives rise to stochastic bursting.

Performance boost of time-delay reservoir computing by non-resonant clock cycle.

The time-delay-based reservoir computing setup has seen tremendous success in both experiment and simulation. It allows for the construction of large neuromorphic computing systems with only few components. However, until now the interplay of the different timescales has not been investigated thoroughly. In this manuscript, we investigate the effects of a mismatch between the time-delay and the clock cycle for a general model. Typically, these two time scales are considered to be equal. Here we show that the case of equal or resonant time-delay and clock cycle could be actively detrimental and leads to an increase of the approximation error of the reservoir. In particular, we can show that non-resonant ratios of these time scales have maximal memory capacities. We achieve this by translating the periodically driven delay-dynamical system into an equivalent network. Networks that originate from a system with resonant delay-times and clock cycles fail to utilize all of their degrees of freedom, which causes the degradation of their performance.

Hierarchical frequency clusters in adaptive networks of phase oscillators.

Adaptive dynamical networks appear in various real-word systems. One of the simplest phenomenological models for investigating basic properties of adaptive networks is the system of coupled phase oscillators with adaptive couplings. In this paper, we investigate the dynamics of this system. We extend recent results on the appearance of hierarchical frequency multiclusters by investigating the effect of the time scale separation. We show that the slow adaptation in comparison with the fast phase dynamics is necessary for the emergence of the multiclusters and their stability. Additionally, we study the role of double antipodal clusters, which appear to be unstable for all considered parameter values. We show that such states can be observed for a relatively long time, i.e., they are metastable. A geometrical explanation for such an effect is based on the emergence of a heteroclinic orbit.

Frequency cluster formation and slow oscillations in neural populations with plasticity.

We report the phenomenon of frequency clustering in a network of Hodgkin-Huxley neurons with spike timing-dependent plasticity. The clustering leads to a splitting of a neural population into a few groups synchronized at different frequencies. In this regime, the amplitude of the mean field undergoes low-frequency modulations, which may contribute to the mechanism of the emergence of slow oscillations of neural activity observed in spectral power of local field potentials or electroencephalographic signals at high frequencies. In addition to numerical simulations of such multi-clusters, we investigate the mechanisms of the observed phenomena using the simplest case of two clusters. In particular, we propose a phenomenological model which describes the dynamics of two clusters taking into account the adaptation of coupling weights. We also determine the set of plasticity functions (update rules), which lead to multi-clustering.