Statistical finite elements for misspecified models.

We present a statistical finite element method for nonlinear, time-dependent phenomena, illustrated in the context of nonlinear internal waves (solitons). We take a Bayesian approach and leverage the finite element method to cast the statistical problem as a nonlinear Gaussian state-space model, updating the solution, in receipt of data, in a filtering framework. The method is applicable to problems across science and engineering for which finite element methods are appropriate. The Korteweg-de Vries equation for solitons is presented because it reflects the necessary complexity while being suitably familiar and succinct for pedagogical purposes. We present two algorithms to implement this method, based on the extended and ensemble Kalman filters, and demonstrate effectiveness with a simulation study and a case study with experimental data. The generality of our approach is demonstrated in SI Appendix, where we present examples from additional nonlinear, time-dependent partial differential equations (Burgers equation, Kuramoto-Sivashinsky equation).

Selecting embedding delays: An overview of embedding techniques and a new method using persistent homology.

Delay embedding methods are a staple tool in the field of time series analysis and prediction. However, the selection of embedding parameters can have a big impact on the resulting analysis. This has led to the creation of a large number of methods to optimize the selection of parameters such as embedding lag. This paper aims to provide a comprehensive overview of the fundamentals of embedding theory for readers who are new to the subject. We outline a collection of existing methods for selecting embedding lag in both uniform and non-uniform delay embedding cases. Highlighting the poor dynamical explainability of existing methods of selecting non-uniform lags, we provide an alternative method of selecting embedding lags that includes a mixture of both dynamical and topological arguments. The proposed method, Significant Times on Persistent Strands (SToPS), uses persistent homology to construct a characteristic time spectrum that quantifies the relative dynamical significance of each time lag. We test our method on periodic, chaotic, and fast-slow time series and find that our method performs similar to existing automated non-uniform embedding methods. Additionally, n-step predictors trained on embeddings constructed with SToPS were found to outperform other embedding methods when predicting fast-slow time series.

Multiple Sensors Data Integration for Traffic Incident Detection Using the Quadrant Scan.

Non-recurrent congestion disrupts normal traffic operations and lowers travel time (TT) reliability, which leads to many negative consequences such as difficulties in trip planning, missed appointments, loss in productivity, and driver frustration. Traffic incidents are one of the six causes of non-recurrent congestion. Early and accurate detection helps reduce incident duration, but it remains a challenge due to the limitation of current sensor technologies. In this paper, we employ a recurrence-based technique, the Quadrant Scan, to analyse time series traffic volume data for incident detection. The data is recorded by multiple sensors along a section of urban highway. The results show that the proposed method can detect incidents better by integrating data from the multiple sensors in each direction, compared to using them individually. It can also distinguish non-recurrent traffic congestion caused by incidents from recurrent congestion. The results show that the Quadrant Scan is a promising algorithm for real-time traffic incident detection with a short delay. It could also be extended to other non-recurrent congestion types.

Grading your models: Assessing dynamics learning of models using persistent homology.

Assessing model accuracy for complex and chaotic systems is a non-trivial task that often relies on the calculation of dynamical invariants, such as Lyapunov exponents and correlation dimensions. Well-performing models are able to replicate the long-term dynamics and ergodic properties of the desired system. We term this phenomenon "dynamics learning." However, existing estimates based on dynamical invariants, such as Lyapunov exponents and correlation dimensions, are not unique to each system, not necessarily robust to noise, and struggle with detecting pathological errors, such as errors in the manifold density distribution. This can make meaningful and accurate model assessment difficult. We explore the use of a topological data analysis technique, persistent homology, applied to uniformly sampled trajectories from constructed reservoir models of the Lorenz system to assess the learning quality of a model. A proposed persistent homology point summary, conformance, was able to identify models with successful dynamics learning and detect discrepancies in the manifold density distribution.

The active selfish herd.

The selfish herd hypothesis provides an explanation for group aggregation via the selfish avoidance of predators. Conceptually, and as was first proposed, this movement should aim to minimise the danger domain of each individual. Whilst many reasonable proxies have been proposed, none have directly sought to reduce the danger domain. In this work we present a two dimensional stochastic model that actively optimises these domains. The individuals' dynamics are determined by sampling the space surrounding them and moving to achieve the largest possible domain reduction. Two variants of this idea are investigated with sampling occurring either locally or globally. We simulate our models and two of the previously proposed benchmark selfish herd models: k-nearest neighbours (kNN); and local crowded horizon (LCH). The resulting positions are analysed to determine the benefit to the individual and the group's ability to form a compact group. To do this, the group level metric of packing fraction and individual level metric of domain size are observed over time for a range of noise levels. With these measures we show a clear stratification of the four models when noise is not included. kNN never resulted in centrally compacted herd, while the local active selfish model and LCH did so with varying levels of success. The most centralised groups were achieved with our global active selfish herd model. The inclusion of noise improved aggregation in all models. This was particularly so with the local active selfish model with a change to ordering of performance so that it marginally outperformed LCH in aggregation. By more closely following Hamilton's original conception and aligning the individual's goal of a reduced danger domain with the movement it makes increased cohesion is observed, thus confirming his hypothesis, however, these findings are dependent on noise. Moreover, many features originally conjectured by Hamilton are also observed in our simulations.

Learned emergence in selfish collective motion.

To understand the collective motion of many individuals, we often rely on agent-based models with rules that may be computationally complex and involved. For biologically inspired systems in particular, this raises questions about whether the imposed rules are necessarily an accurate reflection of what is being followed. The basic premise of updating one's state according to some underlying motivation is well suited to the realm of reservoir computing; however, entire swarms of individuals are yet to be tasked with learning movement in this framework. This work focuses on the specific case of many selfish individuals simultaneously optimizing their domains in a manner conducive to reducing their personal risk of predation. Using an echo state network and data generated from the agent-based model, we show that, with an appropriate representation of input and output states, this selfish movement can be learned. This suggests that a more sophisticated neural network, such as a brain, could also learn this behavior and provides an avenue to further the search for realistic movement rules in systems of autonomous individuals.

Consistency in echo-state networks.

Consistency is an extension to generalized synchronization which quantifies the degree of functional dependency of a driven nonlinear system to its input. We apply this concept to echo-state networks, which are an artificial-neural network version of reservoir computing. Through a replica test, we measure the consistency levels of the high-dimensional response, yielding a comprehensive portrait of the echo-state property.

Optimal Shadowing Filter for a Positioning and Tracking Methodology with Limited Information.

Positioning and tracking a moving target from limited positional information is a frequently-encountered problem. For given noisy observations of the target's position, one wants to estimate the true trajectory and reconstruct the full phase space including velocity and acceleration. The shadowing filter offers a robust methodology to achieve such an estimation and reconstruction. Here, we highlight and validate important merits of this methodology for real-life applications. In particular, we explore the filter's performance when dealing with correlated or uncorrelated noise, irregular sampling in time and how it can be optimised even when the true dynamics of the system are not known.

Prediction of flow dynamics using point processes.

Describing a time series parsimoniously is the first step to study the underlying dynamics. For a time-discrete system, a generating partition provides a compact description such that a time series and a symbolic sequence are one-to-one. But, for a time-continuous system, such a compact description does not have a solid basis. Here, we propose to describe a time-continuous time series using a local cross section and the times when the orbit crosses the local cross section. We show that if such a series of crossing times and some past observations are given, we can predict the system's dynamics with fine accuracy. This reconstructability neither depends strongly on the size nor the placement of the local cross section if we have a sufficiently long database. We demonstrate the proposed method using the Lorenz model as well as the actual measurement of wind speed.

Regenerating time series from ordinal networks.

Recently proposed ordinal networks not only afford novel methods of nonlinear time series analysis but also constitute stochastic approximations of the deterministic flow time series from which the network models are constructed. In this paper, we construct ordinal networks from discrete sampled continuous chaotic time series and then regenerate new time series by taking random walks on the ordinal network. We then investigate the extent to which the dynamics of the original time series are encoded in the ordinal networks and retained through the process of regenerating new time series by using several distinct quantitative approaches. First, we use recurrence quantification analysis on traditional recurrence plots and order recurrence plots to compare the temporal structure of the original time series with random walk surrogate time series. Second, we estimate the largest Lyapunov exponent from the original time series and investigate the extent to which this invariant measure can be estimated from the surrogate time series. Finally, estimates of correlation dimension are computed to compare the topological properties of the original and surrogate time series dynamics. Our findings show that ordinal networks constructed from univariate time series data constitute stochastic models which approximate important dynamical properties of the original systems.

Counting forbidden patterns in irregularly sampled time series. I. The effects of under-sampling, random depletion, and timing jitter.

It has been established that the count of ordinal patterns, which do not occur in a time series, called forbidden patterns, is an effective measure for the detection of determinism in noisy data. A very recent study has shown that this measure is also partially robust against the effects of irregular sampling. In this paper, we extend said research with an emphasis on exploring the parameter space for the method's sole parameter-the length of the ordinal patterns-and find that the measure is more robust to under-sampling and irregular sampling than previously reported. Using numerically generated data from the Lorenz system and the hyper-chaotic Rössler system, we investigate the reliability of the relative proportion of ordinal patterns in periodic and chaotic time series for various degrees of under-sampling, random depletion of data, and timing jitter. Discussion and interpretation of results focus on determining the limitations of the measure with respect to optimal parameter selection, the quantity of data available, the sampling period, and the Lyapunov and de-correlation times of the system.

Counting forbidden patterns in irregularly sampled time series. II. Reliability in the presence of highly irregular sampling.

We are motivated by real-world data that exhibit severe sampling irregularities such as geological or paleoclimate measurements. Counting forbidden patterns has been shown to be a powerful tool towards the detection of determinism in noisy time series. They constitute a set of ordinal symbolic patterns that cannot be realised in time series generated by deterministic systems. The reliability of the estimator of the relative count of forbidden patterns from irregularly sampled data has been explored in two recent studies. In this paper, we explore highly irregular sampling frequency schemes. Using numerically generated data, we examine the reliability of the estimator when the sampling period has been drawn from exponential, Pareto and Gamma distributions of varying skewness. Our investigations demonstrate that some statistical properties of the sampling distribution are useful heuristics for assessing the estimator's reliability. We find that sampling in the presence of large chronological gaps can still yield relatively accurate estimates as long as the time series contains sufficiently many densely sampled areas. Furthermore, we show that the reliability of the estimator of forbidden patterns is poor when there is a high number of sampling intervals, which are larger than a typical correlation time of the underlying system.

Time lagged ordinal partition networks for capturing dynamics of continuous dynamical systems.

We investigate a generalised version of the recently proposed ordinal partition time series to network transformation algorithm. First, we introduce a fixed time lag for the elements of each partition that is selected using techniques from traditional time delay embedding. The resulting partitions define regions in the embedding phase space that are mapped to nodes in the network space. Edges are allocated between nodes based on temporal succession thus creating a Markov chain representation of the time series. We then apply this new transformation algorithm to time series generated by the Rössler system and find that periodic dynamics translate to ring structures whereas chaotic time series translate to band or tube-like structures-thereby indicating that our algorithm generates networks whose structure is sensitive to system dynamics. Furthermore, we demonstrate that simple network measures including the mean out degree and variance of out degrees can track changes in the dynamical behaviour in a manner comparable to the largest Lyapunov exponent. We also apply the same analysis to experimental time series generated by a diode resonator circuit and show that the network size, mean shortest path length, and network diameter are highly sensitive to the interior crisis captured in this particular data set.

Estimating topological entropy using ordinal partition networks.

We propose a computationally simple and efficient network-based method for approximating topological entropy of low-dimensional chaotic systems. This approach relies on the notion of an ordinal partition. The proposed methodology is compared to the three existing techniques based on counting ordinal patterns-all of which derive from collecting statistics about the symbolic itinerary-namely (i) the gradient of the logarithm of the number of observed patterns as a function of the pattern length, (ii) direct application of the definition of topological permutation entropy, and (iii) the outgrowth ratio of patterns of increasing length. In contrast to these alternatives, our method involves the construction of a sequence of complex networks that constitute stochastic approximations of the underlying dynamics on an increasingly finer partition. An ordinal partition network can be computed using any scalar observable generated by multidimensional ergodic systems, provided the measurement function comprises a monotonic transformation if nonlinear. Numerical experiments on an ensemble of systems demonstrate that the logarithm of the spectral radius of the connectivity matrix produces significantly more accurate approximations than existing alternatives-despite practical constraints dictating the selection of low finite values for the pattern length.

Laminar chaos in nonlinear electronic circuits with delay clock modulation.

We study laminar chaos in an electronic experiment. A two-diode nonlinear circuit with delayed feedback shows chaotic dynamics similar to the Mackey-Glass or Ikeda delay systems. Clock modulation of a single delay line leads to a conservative variable delay, which with a second delay line is augmented to dissipative delays, leading to laminar chaotic regimes. We discuss the properties of this particular delay modulation and demonstrate experimental aspects of laminar chaos in terms of power spectra and return maps.

Failures of sequential Bayesian filters and the successes of shadowing filters in tracking of nonlinear deterministic and stochastic systems.

Sequential Bayesian filters, such as particle filters, are often presented as an ideal means of tracking the state of nonlinear systems. Here shadowing filters are demonstrated to perform better than sequential filters at tracking under specific circumstances. The success of shadowing filters is attributed to avoiding both well-known deficiencies of particle filters, and some newly identified problems.

Markov modeling via ordinal partitions: An alternative paradigm for network-based time-series analysis.

Mapping time series to complex networks to analyze observables has recently become popular, both at the theoretical and the practitioner's level. The intent is to use network metrics to characterize the dynamics of the underlying system. Applications cover a wide range of problems, from geoscientific measurements to biomedical data and financial time series. It has been observed that different dynamics can produce networks with distinct topological characteristics under a variety of time-series-to-network transforms that have been proposed in the literature. The direct connection, however, remains unclear. Here, we investigate a network transform based on computing statistics of ordinal permutations in short subsequences of the time series, the so-called ordinal partition network. We propose a Markovian framework that allows the interpretation of the network using ergodic-theoretic ideas and demonstrate, via numerical experiments on an ensemble of time series, that this viewpoint renders this technique especially well-suited to nonlinear chaotic signals. The aim is to test the mapping's faithfulness as a representation of the dynamics and the extent to which it retains information from the input data. First, we show that generating networks by counting patterns of increasing length is essentially a mechanism for approximating the analog of the Perron-Frobenius operator in a topologically equivalent higher-dimensional space to the original state space. Then, we illustrate a connection between the connectivity patterns of the networks generated by this mapping and indicators of dynamics such as the hierarchy of unstable periodic orbits embedded within a chaotic attractor. The input is a scalar observable and any projection of a multidimensional flow suffices for reconstruction of the essential dynamics. Additionally, we create a detailed guide for parameter tuning. We argue that there is no optimal value of the pattern length m, rather it admits a scaling region akin to traditional embedding practice. In contrast, the embedding lag and overlap between successive patterns can be chosen exactly in an optimal way. Our analysis illustrates the potential of this transform as a complementary toolkit to traditional time-series methods.

Noise-free stochastic resonance at an interior crisis.

We report on the observation of noise-free stochastic resonance in an externally driven diode resonator close to an interior crisis. At sufficiently high excitation amplitudes the diode resonator shows a strange attractor which after the collision with an unstable period-three orbit exhibits crisis-induced intermittency. In the intermittency regime the system jumps between the previously stable chaotic attractor and the phase space region which has been made accessible by the crisis. This random process can be used to amplify a weak periodic signal through the mechanism of stochastic resonance. In contrast to conventional stochastic resonance no external noise is needed. The chaotic intrinsic dynamics plays the role of the stochastic forcing. Our data obtained from the diode resonator are compared with numerical simulations of the logistic map where a similar crisis-induced intermittency is observed.

Tracking a single pigeon using a shadowing filter algorithm.

Miniature GPS devices now allow for measurement of the movement of animals in real time and provide high- quality and high-resolution data. While these new data sets are a great improvement, one still encounters some measurement errors as well as device failures. Moreover, these devices only measure position and require further reconstruction techniques to extract the full dynamical state space with the velocity and acceleration. Direct differentiation of position is generally not adequate. We report on the successful implementation of a shadowing filter algorithm that (1) minimizes measurement errors and (2) reconstructs at the same time the full phase-space from a position recording of a flying pigeon. This filter is based on a very simple assumption that the pigeon's dynamics are Newtonian. We explore not only how to choose the filter's parameters but also demonstrate its improvements over other techniques and give minimum data requirements. In contrast to competing filters, the shadowing filter's approach has not been widely implemented for practical problems. This article addresses these practicalities and provides a prototype for such application.

Tweaking synchronization by connectivity modifications.

Natural and man-made networks often possess locally treelike substructures. Taking such tree networks as our starting point, we show how the addition of links changes the synchronization properties of the network. We focus on two different methods of link addition. The first method adds single links that create cycles of a well-defined length. Following a topological approach, we introduce cycles of varying length and analyze how this feature, as well as the position in the network, alters the synchronous behavior. We show that in particular short cycles can lead to a maximum change of the Laplacian's eigenvalue spectrum, dictating the synchronization properties of such networks. The second method connects a certain proportion of the initially unconnected nodes. We simulate dynamical systems on these network topologies, with the nodes' local dynamics being either discrete or continuous. Here our main result is that a certain number of additional links, with the relative position in the network being crucial, can be beneficial to ensure stable synchronization.