Higher-order Granger reservoir computing: simultaneously achieving scalable complex structures inference and accurate dynamics prediction.

Recently, machine learning methods, including reservoir computing (RC), have been tremendously successful in predicting complex dynamics in many fields. However, a present challenge lies in pushing for the limit of prediction accuracy while maintaining the low complexity of the model. Here, we design a data-driven, model-free framework named higher-order Granger reservoir computing (HoGRC), which owns two major missions: The first is to infer the higher-order structures incorporating the idea of Granger causality with the RC, and, simultaneously, the second is to realize multi-step prediction by feeding the time series and the inferred higher-order information into HoGRC. We demonstrate the efficacy and robustness of the HoGRC using several representative systems, including the classical chaotic systems, the network dynamical systems, and the UK power grid system. In the era of machine learning and complex systems, we anticipate a broad application of the HoGRC framework in structure inference and dynamics prediction.

Reservoir computing decoupling memory-nonlinearity trade-off.

Reservoir computing (RC), a variant recurrent neural network, has very compact architecture and ability to efficiently reconstruct nonlinear dynamics by combining both memory capacity and nonlinear transformations. However, in the standard RC framework, there is a trade-off between memory capacity and nonlinear mapping, which limits its ability to handle complex tasks with long-term dependencies. To overcome this limitation, this paper proposes a new RC framework called neural delayed reservoir computing (ND-RC) with a chain structure reservoir that can decouple the memory capacity and nonlinearity, allowing for independent tuning of them, respectively. The proposed ND-RC model offers a promising solution to the memory-nonlinearity trade-off problem in RC and provides a more flexible and effective approach for modeling complex nonlinear systems with long-term dependencies. The proposed ND-RC framework is validated with typical benchmark nonlinear systems and is particularly successful in reconstructing and predicting the Mackey-Glass system with high time delays. The memory-nonlinearity decoupling ability is further confirmed by several standard tests.

Harvesting random embedding for high-frequency change-point detection in temporal complex systems.

Recent investigations have revealed that dynamics of complex networks and systems are crucially dependent on the temporal structures. Accurate detection of the time instant at which a system changes its internal structures has become a tremendously significant mission, beneficial to fully understanding the underlying mechanisms of evolving systems, and adequately modeling and predicting the dynamics of the systems as well. In real-world applications, due to a lack of prior knowledge on the explicit equations of evolving systems, an open challenge is how to develop a practical and model-free method to achieve the mission based merely on the time-series data recorded from real-world systems. Here, we develop such a model-free approach, named temporal change-point detection (TCD), and integrate both dynamical and statistical methods to address this important challenge in a novel way. The proposed TCD approach, basing on exploitation of spatial information of the observed time series of high dimensions, is able not only to detect the separate change points of the concerned systems without knowing, a priori, any information of the equations of the systems, but also to harvest all the change points emergent in a relatively high-frequency manner, which cannot be directly achieved by using the existing methods and techniques. Practical effectiveness is comprehensively demonstrated using the data from the representative complex dynamics and real-world systems from biology to geology and even to social science.

Continuity Scaling: A Rigorous Framework for Detecting and Quantifying Causality Accurately.

Data-based detection and quantification of causation in complex, nonlinear dynamical systems is of paramount importance to science, engineering, and beyond. Inspired by the widely used methodology in recent years, the cross-map-based techniques, we develop a general framework to advance towards a comprehensive understanding of dynamical causal mechanisms, which is consistent with the natural interpretation of causality. In particular, instead of measuring the smoothness of the cross-map as conventionally implemented, we define causation through measuring the scaling law for the continuity of the investigated dynamical system directly. The uncovered scaling law enables accurate, reliable, and efficient detection of causation and assessment of its strength in general complex dynamical systems, outperforming those existing representative methods. The continuity scaling-based framework is rigorously established and demonstrated using datasets from model complex systems and the real world.

Responses of AM fungal abundance to the drivers of global climate change: A meta-analysis.

Arbuscular mycorrhizal fungi (AMF), playing critical roles in carbon cycling, are vulnerable to climate change. However, the responses of AM fungal abundance to climate change are unclear. A global-scale meta-analysis was conducted to investigate the response patterns of AM fungal abundance to warming, elevated CO2 concentration (eCO2), and N addition. Both warming and eCO2 significantly stimulated AM fungal abundance by 18.6% (95%CI: 5.9%-32.8%) and 21.4% (15.1%-28.1%) on a global scale, respectively. However, the response ratios (RR) of AM fungal abundance decreased with the degree of warming while increased with the degree of eCO2. Furthermore, in warming experiments, as long as the warming exceeded 4 °C, its effects on AM fungal abundance changed from positive to negative regardless of the experimental durations, methods, periods, and ecosystem types. The effects of N addition on AM fungal abundance are -5.4% (-10.6%-0.2%), and related to the nitrogen fertilizer input rate and ecosystem type. The RR of AM fungal abundance is negative in grasslands and farmlands when the degree of N addition exceeds 33.85 and 67.64 kg N ha-1 yr-1, respectively; however, N addition decreases AM fungal abundance in forests only when the degree of N addition exceeds 871.31 kg N ha-1 yr-1. The above results provide an insight into predicting ecological functions of AM fungal abundance under global changes.

[Stability of Soil Aggregates at Different Altitudes in Qinling Mountains and Its Coupling Relationship with Soil Enzyme Activities].

To explore changes in soil aggregate stability along an elevation gradient, and its regulating factors, soil samples were taken from the 0-10 cm surface layer at 3 different elevations on Taibai Mountain. We measured and analyzed the distribution of soil aggregates, physical and chemical properties, microbial biomass, and extracellular enzymes. The results showed that: ① the soil aggregates from the 3 elevations had mean weight diameters (MWD) of 2.17 mm, 1.83 mm, and 1.82 mm (increasing elevation), and geometric mean diameters (GMD) of 1.66 mm, 1.39 mm, and 1.32 mm, respectively. ② The change in soil aggregate stability along an elevation gradient was regulated by extracellular enzymes in the soil, in particular, the LAP in soil meso-aggregate and the BG in soil micro-aggregate. ③ Microorganisms can alleviate the N limitation at high elevations by adjusting the relative production of extracellular enzymes and altering nutrient utilization efficiency, which also changes soil aggregate stability along an elevation gradient. The results of this study have important scientific significance for soil quality evaluation and ecological environment protection in Taibai Mountain.

Partial cross mapping eliminates indirect causal influences.

Causality detection likely misidentifies indirect causations as direct ones, due to the effect of causation transitivity. Although several methods in traditional frameworks have been proposed to avoid such misinterpretations, there still is a lack of feasible methods for identifying direct causations from indirect ones in the challenging situation where the variables of the underlying dynamical system are non-separable and weakly or moderately interacting. Here, we solve this problem by developing a data-based, model-independent method of partial cross mapping based on an articulated integration of three tools from nonlinear dynamics and statistics: phase-space reconstruction, mutual cross mapping, and partial correlation. We demonstrate our method by using data from different representative models and real-world systems. As direct causations are keys to the fundamental underpinnings of a variety of complex dynamics, we anticipate our method to be indispensable in unlocking and deciphering the inner mechanisms of real systems in diverse disciplines from data.

The Effects of Graded Levels of Calorie Restriction XV: Phase Space Attractors Reveal Distinct Behavioral Phenotypes.

Calorie restriction (CR) has a positive impact on health and life span. Previous work, however, does not reveal the whole underlying mechanism of behavioral phenotypes under CR. We propose a new approach based on phase space reconstruction (PSR) to analyze the behavioral responses of mice to graded CR. This involved reconstructing high-dimensional attractors which topologically represent the intrinsic dynamics of mice based on low-dimensional time series of movement counts observed during the 90-day time course of restriction. PSR together with correlation dimensions (CD), Kolmogorov entropy (KE), and multifractal spectra builds a map from internal attractors to the phenotype of mice and reveals the mice with increasing CR levels undergo significant changes from a normal to a new state. Features of the attractors (CD and KE) were significantly associated with gene expression profiles in the hypothalamus of the same individuals.

Predicting future dynamics from short-term time series using an Anticipated Learning Machine.

Predicting time series has significant practical applications over different disciplines. Here, we propose an Anticipated Learning Machine (ALM) to achieve precise future-state predictions based on short-term but high-dimensional data. From non-linear dynamical systems theory, we show that ALM can transform recent correlation/spatial information of high-dimensional variables into future dynamical/temporal information of any target variable, thereby overcoming the small-sample problem and achieving multistep-ahead predictions. Since the training samples generated from high-dimensional data also include information of the unknown future values of the target variable, it is called anticipated learning. Extensive experiments on real-world data demonstrate significantly superior performances of ALM over all of the existing 12 methods. In contrast to traditional statistics-based machine learning, ALM is based on non-linear dynamics, thus opening a new way for dynamics-based machine learning.

Reconstructing directional causal networks with random forest: Causality meeting machine learning.

Inspired by the decision tree algorithm in machine learning, a novel causal network reconstruction framework is proposed with the name Importance Causal Analysis (ICA). The ICA framework is designed in a network level and fills the gap between traditional mutual causality detection methods and the reconstruction of causal networks. The potential of the method to identify the true causal relations in complex networks is validated by both benchmark systems and real-world data sets.

Detecting unstable periodic orbits based only on time series: When adaptive delayed feedback control meets reservoir computing.

In this article, we focus on a topic of detecting unstable periodic orbits (UPOs) only based on the time series observed from the nonlinear dynamical system whose explicit model is completely unknown a priori. We articulate a data-driven and model-free method which connects a well-known machine learning technique, the reservoir computing, with a widely-used control strategy of nonlinear dynamical systems, the adaptive delayed feedback control. We demonstrate the advantages and effectiveness of the articulated method through detecting and controlling UPOs in representative examples and also show how those configurations of the reservoir computing in our method influence the accuracy of UPOs detection. Additionally and more interestingly, from the viewpoint of synchronization, we analytically and numerically illustrate the effectiveness of the reservoir computing in dynamical systems learning and prediction.

Randomly distributed embedding making short-term high-dimensional data predictable.

Future state prediction for nonlinear dynamical systems is a challenging task, particularly when only a few time series samples for high-dimensional variables are available from real-world systems. In this work, we propose a model-free framework, named randomly distributed embedding (RDE), to achieve accurate future state prediction based on short-term high-dimensional data. Specifically, from the observed data of high-dimensional variables, the RDE framework randomly generates a sufficient number of low-dimensional "nondelay embeddings" and maps each of them to a "delay embedding," which is constructed from the data of a to be predicted target variable. Any of these mappings can perform as a low-dimensional weak predictor for future state prediction, and all of such mappings generate a distribution of predicted future states. This distribution actually patches all pieces of association information from various embeddings unbiasedly or biasedly into the whole dynamics of the target variable, which after operated by appropriate estimation strategies, creates a stronger predictor for achieving prediction in a more reliable and robust form. Through applying the RDE framework to data from both representative models and real-world systems, we reveal that a high-dimension feature is no longer an obstacle but a source of information crucial to accurate prediction for short-term data, even under noise deterioration.

Detection of time delays and directional interactions based on time series from complex dynamical systems.

Data-based and model-free accurate identification of intrinsic time delays and directional interactions is an extremely challenging problem in complex dynamical systems and their networks reconstruction. A model-free method with new scores is proposed to be generally capable of detecting single, multiple, and distributed time delays. The method is applicable not only to mutually interacting dynamical variables but also to self-interacting variables in a time-delayed feedback loop. Validation of the method is carried out using physical, biological, and ecological models and real data sets. Especially, applying the method to air pollution data and hospital admission records of cardiovascular diseases in Hong Kong reveals the major air pollutants as a cause of the diseases and, more importantly, it uncovers a hidden time delay (about 30-40 days) in the causal influence that previous studies failed to detect. The proposed method is expected to be universally applicable to ascertaining and quantifying subtle interactions (e.g., causation) in complex systems arising from a broad range of disciplines.

Detection meeting control: Unstable steady states in high-dimensional nonlinear dynamical systems.

We articulate an adaptive and reference-free framework based on the principle of random switching to detect and control unstable steady states in high-dimensional nonlinear dynamical systems, without requiring any a priori information about the system or about the target steady state. Starting from an arbitrary initial condition, a proper control signal finds the nearest unstable steady state adaptively and drives the system to it in finite time, regardless of the type of the steady state. We develop a mathematical analysis based on fast-slow manifold separation and Markov chain theory to validate the framework. Numerical demonstration of the control and detection principle using both classic chaotic systems and models of biological and physical significance is provided.

Detecting causality from nonlinear dynamics with short-term time series.

Quantifying causality between variables from observed time series data is of great importance in various disciplines but also a challenging task, especially when the observed data are short. Unlike the conventional methods, we find it possible to detect causality only with very short time series data, based on embedding theory of an attractor for nonlinear dynamics. Specifically, we first show that measuring the smoothness of a cross map between two observed variables can be used to detect a causal relation. Then, we provide a very effective algorithm to computationally evaluate the smoothness of the cross map, or "Cross Map Smoothness" (CMS), and thus to infer the causality, which can achieve high accuracy even with very short time series data. Analysis of both mathematical models from various benchmarks and real data from biological systems validates our method.

Detecting unstable periodic orbits in high-dimensional chaotic systems from time series: reconstruction meeting with adaptation.

Detecting unstable periodic orbits (UPOs) in chaotic systems based solely on time series is a fundamental but extremely challenging problem in nonlinear dynamics. Previous approaches were applicable but mostly for low-dimensional chaotic systems. We develop a framework, integrating approximation theory of neural networks and adaptive synchronization, to address the problem of time-series-based detection of UPOs in high-dimensional chaotic systems. An example of finding UPOs from the classic Mackey-Glass equation is presented.

Locating unstable periodic orbits: when adaptation integrates into delayed feedback control.

Finding unstable periodic orbits (UPOs) is always a challenging demand in biophysics and computational biology, which needs efficient algorithms. To meet this need, an approach to locating unstable periodic orbits in chaotic dynamical system is presented. The uniqueness of the approach lies in the introduction of adaptive rules for both feedback gain and time delay in the system without requiring any information of the targeted UPO periods a priori. This approach is theoretically validated under some mild conditions and successfully tested with some practical strategies in several typical chaotic systems with or without significant time delays.

Adaptive identification of time delays in nonlinear dynamical models.

This paper develops an adaptive synchronization strategy to identify both discrete and distributed time delays in nonlinear dynamical models. In contrast with adaptive techniques for parameter estimation in the literature, the adaptive strategy developed here for time-delay identification invites more precise results that have physical and dynamical importance. It is analytically and numerically found that distributed time delays in a model with an asymptotically stable steady state can be adaptively identified, and which is different from the case of discrete time-delays identification. Other aspects of the strategy developed here, for time-delay identification, are illustrated by several representative dynamical models. Aside from illustrations for toy models and their generated data, the strategy developed is used with experimental data, to identify a time delay, called transcriptional delay, in a model describing the transcription of messenger RNAs (mRNAs) for Notch signaling molecules.

Failure of parameter identification based on adaptive synchronization techniques.

In this paper, several examples as well as their numerical simulations are provided to show some possible failures of parameter identification based on the so-called adaptive synchronization techniques. These failures might arise not only when the synchronized orbit produced by the driving system is designed to be either some kind of equilibrium or to be some kind of periodic orbit, but also when this orbit is deliberately designed to be chaotic. The reason for emergence of these failures is theoretically analyzed in the paper and the boundedness of all trajectories generated by the coupled systems is rigorously proved. Moreover, synchronization techniques are proposed to realize complete synchronization and unknown parameter identification in a class of systems where nonlinear terms are not globally Lipschitz. In addition, unknown parameter identification is studied in coupled systems with time delays.