Attractor reconstruction with reservoir computers: The effect of the reservoir's conditional Lyapunov exponents on faithful attractor reconstruction.

Reservoir computing is a machine learning framework that has been shown to be able to replicate the chaotic attractor, including the fractal dimension and the entire Lyapunov spectrum, of the dynamical system on which it is trained. We quantitatively relate the generalized synchronization dynamics of a driven reservoir during the training stage to the performance of the trained reservoir computer at the attractor reconstruction task. We show that, in order to obtain successful attractor reconstruction and Lyapunov spectrum estimation, the maximal conditional Lyapunov exponent of the driven reservoir must be significantly more negative than the most negative Lyapunov exponent of the target system. We also find that the maximal conditional Lyapunov exponent of the reservoir depends strongly on the spectral radius of the reservoir adjacency matrix; therefore, for attractor reconstruction and Lyapunov spectrum estimation, small spectral radius reservoir computers perform better in general. Our arguments are supported by numerical examples on well-known chaotic systems.

Estimating the master stability function from the time series of one oscillator via reservoir computing.

The master stability function (MSF) yields the stability of the globally synchronized state of a network of identical oscillators in terms of the eigenvalues of the adjacency matrix. In order to compute the MSF, one must have an accurate model of an uncoupled oscillator, but often such a model does not exist. We present a reservoir computing technique for estimating the MSF given only the time series of a single, uncoupled oscillator. We demonstrate the generality of our technique by considering a variety of coupling configurations of networks consisting of Lorenz oscillators or Hénon maps.

Synchronizing chaos using reservoir computing.

We attempt to achieve complete synchronization between a drive system unidirectionally coupled with a response system, under the assumption that limited knowledge on the states of the drive is available at the response. Machine-learning techniques have been previously implemented to estimate the states of a dynamical system from limited measurements. We consider situations in which knowledge of the non-measurable states of the drive system is needed in order for the response system to synchronize with the drive. We use a reservoir computer to estimate the non-measurable states of the drive system from its measured states and then employ these measured states to achieve complete synchronization of the response system with the drive.

Time-shift selection for reservoir computing using a rank-revealing QR algorithm.

Reservoir computing, a recurrent neural network paradigm in which only the output layer is trained, has demonstrated remarkable performance on tasks such as prediction and control of nonlinear systems. Recently, it was demonstrated that adding time-shifts to the signals generated by a reservoir can provide large improvements in performance accuracy. In this work, we present a technique to choose the time-shifts by maximizing the rank of the reservoir matrix using a rank-revealing QR algorithm. This technique, which is not task dependent, does not require a model of the system and, therefore, is directly applicable to analog hardware reservoir computers. We demonstrate our time-shift selection technique on two types of reservoir computer: an optoelectronic reservoir computer and the traditional recurrent network with a t a n h activation function. We find that our technique provides improved accuracy over random time-shift selection in essentially all cases.

Dark current and single photon detection by 1550 nm avalanche photodiodes: dead time corrected probability distributions and entropy rates.

Single photon detectors have dark count rates that depend strongly on the bias level for detector operation. In the case of weak light sources such as novel lasers or single-photon emitters, the rate of counts due to the light source can be comparable to that of the detector dark counts. In such cases, a characterization of the statistical properties of the dark counts is necessary. The dark counts are often assumed to follow a Poisson process that is statistically independent of the incident photon counts. This assumption must be validated for specific types of photodetectors. In this work, we focus on single-photon avalanche photodiodes (SPADs) made for 1550 nm. For the InGaAs detectors used, we find the measured distributions often differ significantly from Poisson due to the presence of dead time and afterpulsing with the difference increasing with the bias level used for obtaining higher quantum efficiencies. We find that when the dead time is increased to remove the effects of afterpulsing, it is necessary to correct the measured distributions for the effects of the dead time. To this end, we apply an iterative algorithm to remove dead time effects from the probability distribution for dark counts as well as for the case where light from an external weak laser source (known to be Poisson) is detected together with the dark counts. We believe this to be the first instance of the comprehensive application of this algorithm to real data and find that the dead time corrected probability distributions are Poisson distributions in both cases. We additionally use the Grassberger-Procaccia algorithm to estimate the entropy production rates of the dark count processes, which provides a single metric that characterizes the temporal correlations between dark counts as well as the shape of the distribution. We have thus developed a systematic procedure for taking data with 1550 nm SPADs and obtaining accurate photocount statistics to examine novel light sources.

Time shifts to reduce the size of reservoir computers.

A reservoir computer is a type of dynamical system arranged to do computation. Typically, a reservoir computer is constructed by connecting a large number of nonlinear nodes in a network that includes recurrent connections. In order to achieve accurate results, the reservoir usually contains hundreds to thousands of nodes. This high dimensionality makes it difficult to analyze the reservoir computer using tools from the dynamical systems theory. Additionally, the need to create and connect large numbers of nonlinear nodes makes it difficult to design and build analog reservoir computers that can be faster and consume less power than digital reservoir computers. We demonstrate here that a reservoir computer may be divided into two parts: a small set of nonlinear nodes (the reservoir) and a separate set of time-shifted reservoir output signals. The time-shifted output signals serve to increase the rank and memory of the reservoir computer, and the set of nonlinear nodes may create an embedding of the input dynamical system. We use this time-shifting technique to obtain excellent performance from an opto-electronic delay-based reservoir computer with only a small number of virtual nodes. Because only a few nonlinear nodes are required, construction of a reservoir computer becomes much easier, and delay-based reservoir computers can operate at much higher speeds.

Laminar chaos in experiments and nonlinear delayed Langevin equations: A time series analysis toolbox for the detection of laminar chaos.

Recently, it was shown that certain systems with large time-varying delay exhibit different types of chaos, which are related to two types of time-varying delay: conservative and dissipative delays. The known high-dimensional turbulent chaos is characterized by strong fluctuations. In contrast, the recently discovered low-dimensional laminar chaos is characterized by nearly constant laminar phases with periodic durations and a chaotic variation of the intensity from phase to phase. In this paper we extend our results from our preceding publication [Hart, Roy, Müller-Bender, Otto, and Radons, Phys. Rev. Lett. 123, 154101 (2019)PRLTAO0031-900710.1103/PhysRevLett.123.154101], where it is demonstrated that laminar chaos is a robust phenomenon, which can be observed in experimental systems. We provide a time series analysis toolbox for the detection of robust features of laminar chaos. We benchmark our toolbox by experimental time series and time series of a model system which is described by a nonlinear Langevin equation with time-varying delay. The benchmark is done for different noise strengths for both the experimental system and the model system, where laminar chaos can be detected, even if it is hard to distinguish from turbulent chaos by a visual analysis of the trajectory.

Laminar Chaos in Experiments: Nonlinear Systems with Time-Varying Delays and Noise.

A new type of dynamics called laminar chaos was recently discovered through a theoretical analysis of a scalar delay differential equation with time-varying delay. Laminar chaos is a low-dimensional dynamics characterized by laminar phases of nearly constant intensity with periodic durations and a chaotic variation of the intensity from one laminar phase to the next laminar phase. This is in stark contrast to the typically observed higher-dimensional turbulent chaos, which is characterized by strong fluctuations. In this Letter we provide the first experimental observation of laminar chaos by studying an optoelectronic feedback loop with time-varying delay. The noise inherent in the experiment requires the development of a nonlinear Langevin equation with variable delay. The results show that laminar chaos can be observed in higher-order systems, and that the phenomenon is robust to noise and a digital implementation of the variable time delay.

Delayed dynamical systems: networks, chimeras and reservoir computing.

We present a systematic approach to reveal the correspondence between time delay dynamics and networks of coupled oscillators. After early demonstrations of the usefulness of spatio-temporal representations of time-delay system dynamics, extensive research on optoelectronic feedback loops has revealed their immense potential for realizing complex system dynamics such as chimeras in rings of coupled oscillators and applications to reservoir computing. Delayed dynamical systems have been enriched in recent years through the application of digital signal processing techniques. Very recently, we have showed that one can significantly extend the capabilities and implement networks with arbitrary topologies through the use of field programmable gate arrays. This architecture allows the design of appropriate filters and multiple time delays, and greatly extends the possibilities for exploring synchronization patterns in arbitrary network topologies. This has enabled us to explore complex dynamics on networks with nodes that can be perfectly identical, introduce parameter heterogeneities and multiple time delays, as well as change network topologies to control the formation and evolution of patterns of synchrony. This article is part of the theme issue 'Nonlinear dynamics of delay systems'.

Topological Control of Synchronization Patterns: Trading Symmetry for Stability.

Symmetries are ubiquitous in network systems and have profound impacts on the observable dynamics. At the most fundamental level, many synchronization patterns are induced by underlying network symmetry, and a high degree of symmetry is believed to enhance the stability of identical synchronization. Yet, here we show that the synchronizability of almost any symmetry cluster in a network of identical nodes can be enhanced precisely by breaking its structural symmetry. This counterintuitive effect holds for generic node dynamics and arbitrary network structure and is, moreover, robust against noise and imperfections typical of real systems, which we demonstrate by implementing a state-of-the-art optoelectronic experiment. These results lead to new possibilities for the topological control of synchronization patterns, which we substantiate by presenting an algorithm that optimizes the structure of individual clusters under various constraints.

Symmetry- and input-cluster synchronization in networks.

We study cluster synchronization in networks and show that the stability of all possible cluster synchronization patterns depends on a small set of Lyapunov exponents. Our approach can be applied to clusters corresponding to both orbital partitions of the network nodes (symmetry-cluster synchronization) and equitable partitions of the network nodes (input-cluster synchronization). Our results are verified experimentally in networks of coupled optoelectronic oscillators.

Experiments with arbitrary networks in time-multiplexed delay systems.

We report a new experimental approach using an optoelectronic feedback loop to investigate the dynamics of oscillators coupled on large complex networks with arbitrary topology. Our implementation is based on a single optoelectronic feedback loop with time delays. We use the space-time interpretation of systems with time delay to create large networks of coupled maps. Others have performed similar experiments using high-pass filters to implement the coupling; this restricts the network topology to the coupling of only a few nearest neighbors. In our experiment, the time delays and coupling are implemented on a field-programmable gate array, allowing the creation of networks with arbitrary coupling topology. This system has many advantages: the network nodes are truly identical, the network is easily reconfigurable, and the network dynamics occur at high speeds. We use this system to study cluster synchronization and chimera states in both small and large networks of different topologies.

Experimental observation of chimera and cluster states in a minimal globally coupled network.

A "chimera state" is a dynamical pattern that occurs in a network of coupled identical oscillators when the symmetry of the oscillator population is broken into synchronous and asynchronous parts. We report the experimental observation of chimera and cluster states in a network of four globally coupled chaotic opto-electronic oscillators. This is the minimal network that can support chimera states, and our study provides new insight into the fundamental mechanisms underlying their formation. We use a unified approach to determine the stability of all the observed partially synchronous patterns, highlighting the close relationship between chimera and cluster states as belonging to the broader phenomenon of partial synchronization. Our approach is general in terms of network size and connectivity. We also find that chimera states often appear in regions of multistability between global, cluster, and desynchronized states.

Adding connections can hinder network synchronization of time-delayed oscillators.

We provide experimental evidence that adding links to a network's structure can hinder synchronization. Our experiments and theoretical analysis of networks of time-delayed optoelectronic oscillators uncover the scenario of loss of identical synchronization upon connectivity modifications. This counterintuitive loss of synchronization can occur even when the network structure is improved from a connectivity perspective. Utilizing a master stability function approach, we show that a time delay in the coupling of nodes plays a crucial role in determining a network's synchronization properties and that this effect is more prominent in directed networks than in undirected networks, especially for large networks. Our results provide insight into the impact of structural modifications in networks with equal coupling delays and open the path to design changes to the network connectivity to sustain and control the performance of real-world networks.