A review of symbolic dynamics and symbolic reconstruction of dynamical systems.

Discretizing a nonlinear time series enables us to calculate its statistics fast and rigorously. Before the turn of the century, the approach using partitions was dominant. In the last two decades, discretization via permutations has been developed to a powerful methodology, while recurrence plots have recently begun to be recognized as a method of discretization. In the meantime, horizontal visibility graphs have also been proposed to discretize time series. In this review, we summarize these methods and compare them from the viewpoint of symbolic dynamics, which is the right framework to study the symbolic representation of nonlinear time series and the inverse process: the symbolic reconstruction of dynamical systems. As we will show, symbolic dynamics is currently a very active research field with interesting applications.

Differentiating patients with obstructive sleep apnea from healthy controls based on heart rate-blood pressure coupling quantified by entropy-based indices.

We introduce an entropy-based classification method for pairs of sequences (ECPS) for quantifying mutual dependencies in heart rate and beat-to-beat blood pressure recordings. The purpose of the method is to build a classifier for data in which each item consists of two intertwined data series taken for each subject. The method is based on ordinal patterns and uses entropy-like indices. Machine learning is used to select a subset of indices most suitable for our classification problem in order to build an optimal yet simple model for distinguishing between patients suffering from obstructive sleep apnea and a control group.

SETApp: A machine learning and image analysis based application to automate the sea urchin embryo test.

Since countless xenobiotic compounds are being found in the environment, ecotoxicology faces an astounding challenge in identifying toxicants. The combination of high-throughput in vivo/in vitro bioassays with high-resolution chemical analysis is an effective way to elucidate the cause-effect relationship. However, these combined strategies imply an enormous workload that can hinder their implementation in routine analysis. The purpose of this study was to develop a new high throughput screening method that could be used as a predictive expert system that automatically quantifies the size increase and malformation of the larvae and, thus, eases the application of the sea urchin embryo test in complex toxicant identification pipelines such as effect-directed analysis. For this task, a training set of 242 images was used to calibrate the size-increase and malformation level of the larvae. Two classification models based on partial least squares discriminant analysis (PLS-DA) were built and compared. Moreover, Hierarchical PLS-DA shows a high proficiency in classifying the larvae, achieving a prediction accuracy of 84 % in validation. The scripts built along the work were compiled in a user-friendly standalone app (SETApp) freely accessible at https://github.com/UPV-EHU-IBeA/SETApp. The SETApp was tested in a real case scenario to fulfill the tedious requirements of a WWTP effect-directed analysis.

Permutation group entropy: A new route to complexity for real-valued processes.

This is a review of group entropy and its application to permutation complexity. Specifically, we revisit a new approach to the notion of complexity in the time series analysis based on both permutation entropy and group entropy. As a result, the permutation entropy rate can be extended from deterministic dynamics to random processes. More generally, our approach provides a unified framework to discuss chaotic and random behaviors.

Information Geometry, Complexity Measures and Data Analysis.

In the last several years, a new approach to information theory, called information geometry, has emerged [...].

A generalized permutation entropy for noisy dynamics and random processes.

Permutation entropy measures the complexity of a deterministic time series via a data symbolic quantization consisting of rank vectors called ordinal patterns or simply permutations. Reasons for the increasing popularity of this entropy in time series analysis include that (i) it converges to the Kolmogorov-Sinai entropy of the underlying dynamics in the limit of ever longer permutations and (ii) its computation dispenses with generating and ad hoc partitions. However, permutation entropy diverges when the number of allowed permutations grows super-exponentially with their length, as happens when time series are output by dynamical systems with observational or dynamical noise or purely random processes. In this paper, we propose a generalized permutation entropy, belonging to the class of group entropies, that is finite in that situation, which is actually the one found in practice. The theoretical results are illustrated numerically by random processes with short- and long-term dependencies, as well as by noisy deterministic signals.

Forecasting wind power ramps with prediction coordinates.

To the best of our knowledge, the method of prediction coordinates is the only forecasting method in nonlinear time series analysis that explicitly uses the stochastic characteristics of a system with dynamical noise. Specifically, it generates multiple predictions to jointly infer the current states and dynamical noises. Recent findings based on hypothesis testing show that weather is nonlinear and stochastic and, therefore, so are renewable energy power outputs. This being the case, in this paper, we apply the method of prediction coordinates to forecast wind power ramps, which are rapid transitions in the wind power output that can deteriorate the quality of the electricity supply. First, the method of prediction coordinates is tested using numerical simulations. Then, we present an example of wind power ramp forecasting with empirical data. The results show that the method of prediction coordinates compares favorably with other methods, validating it as a reliable tool for forecasting transitions in nonlinear stochastic dynamics, particularly in the field of renewable energies.

Attention Mechanisms and Their Applications to Complex Systems.

Deep learning models and graphics processing units have completely transformed the field of machine learning. Recurrent neural networks and long short-term memories have been successfully used to model and predict complex systems. However, these classic models do not perform sequential reasoning, a process that guides a task based on perception and memory. In recent years, attention mechanisms have emerged as a promising solution to these problems. In this review, we describe the key aspects of attention mechanisms and some relevant attention techniques and point out why they are a remarkable advance in machine learning. Then, we illustrate some important applications of these techniques in the modeling of complex systems.

Classification and Quantification of Microplastics (<100 µm) Using a Focal Plane Array-Fourier Transform Infrared Imaging System and Machine Learning.

Microplastics are defined as microscopic plastic particles in the range from few micrometers and up to 5 mm. These small particles are classified as primary microplastics when they are manufactured in this size range, whereas secondary microplastics arise from the fragmentation of larger objects. Microplastics are widespread emerging pollutants, and investigations are underway to determine potential harmfulness to biota and human health. However, progress is hindered by the lack of suitable analytical methods for rapid, routine, and unbiased measurements. This work aims to develop an automated analytical method for the characterization of small microplastics (<100 µm) using micro-Fourier transform infrared (µ-FTIR) hyperspectral imaging and machine learning tools. Partial least squares discriminant analysis (PLS-DA) and soft independent modeling of class analogy (SIMCA) models were evaluated, applying different data preprocessing strategies for classification of nine of the most common polymers produced worldwide. The hyperspectral images were also analyzed to quantify particle abundance and size automatically. PLS-DA presented a better analytical performance in comparison with SIMCA models with higher sensitivity, sensibility, and lower misclassification error. PLS-DA was less sensitive to edge effects on spectra and poorly focused regions of particles. The approach was tested on a seabed sediment sample (Roskilde Fjord, Denmark) to demonstrate the method efficiency. The proposed method offers an efficient automated approach for microplastic polymer characterization, abundance numeration, and size distribution with substantial benefits for method standardization.

Entropy Monotonicity and Superstable Cycles for the Quadratic Family Revisited.

The main result of this paper is a proof using real analysis of the monotonicity of the topological entropy for the family of quadratic maps, sometimes called Milnor's Monotonicity Conjecture. In contrast, the existing proofs rely in one way or another on complex analysis. Our proof is based on tools and algorithms previously developed by the authors and collaborators to compute the topological entropy of multimodal maps. Specifically, we use the number of transverse intersections of the map iterations with the so-called critical line. The approach is technically simple and geometrical. The same approach is also used to briefly revisit the superstable cycles of the quadratic maps, since both topics are closely related.

Surrogate Data Preserving All the Properties of Ordinal Patterns up to a Certain Length.

We propose a method for generating surrogate data that preserves all the properties of ordinal patterns up to a certain length, such as the numbers of allowed/forbidden ordinal patterns and transition likelihoods from ordinal patterns into others. The null hypothesis is that the details of the underlying dynamics do not matter beyond the refinements of ordinal patterns finer than a predefined length. The proposed surrogate data help construct a test of determinism that is free from the common linearity assumption for a null-hypothesis.

Detecting directional couplings from multivariate flows by the joint distance distribution.

The identification of directional couplings (or drive-response relationships) in the analysis of interacting nonlinear systems is an important piece of information to understand their dynamics. This task is especially challenging when the analyst's knowledge of the systems reduces virtually to time series of observations. Spurred by the success of Granger causality in econometrics, the study of cause-effect relationships (not to be confounded with statistical correlations) was extended to other fields, thus favoring the introduction of further tools such as transfer entropy. Currently, the research on old and new causality tools along with their pitfalls and applications in ever more general situations is going through a time of much activity. In this paper, we re-examine the method of the joint distance distribution to detect directional couplings between two multivariate flows. This method is based on the forced Takens theorem, and, more specifically, it exploits the existence of a continuous mapping from the reconstructed attractor of the response system to the reconstructed attractor of the driving system, an approach that is increasingly drawing the attention of the data analysts. The numerical results with Lorenz and Rössler oscillators in three different interaction networks (including hidden common drivers) are quite satisfactory, except when phase synchronization sets in. They also show that the method of the joint distance distribution outperforms the lowest dimensional transfer entropy in the cases considered. The robustness of the results to the sampling interval, time series length, observational noise, and metric is analyzed too.

A Brief Review of Generalized Entropies.

Entropy appears in many contexts (thermodynamics, statistical mechanics, information theory, measure-preserving dynamical systems, topological dynamics, etc.) as a measure of different properties (energy that cannot produce work, disorder, uncertainty, randomness, complexity, etc.). In this review, we focus on the so-called generalized entropies, which from a mathematical point of view are nonnegative functions defined on probability distributions that satisfy the first three Shannon-Khinchin axioms: continuity, maximality and expansibility. While these three axioms are expected to be satisfied by all macroscopic physical systems, the fourth axiom (separability or strong additivity) is in general violated by non-ergodic systems with long range forces, this having been the main reason for exploring weaker axiomatic settings. Currently, non-additive generalized entropies are being used also to study new phenomena in complex dynamics (multifractality), quantum systems (entanglement), soft sciences, and more. Besides going through the axiomatic framework, we review the characterization of generalized entropies via two scaling exponents introduced by Hanel and Thurner. In turn, the first of these exponents is related to the diffusion scaling exponent of diffusion processes, as we also discuss. Applications are addressed as the description of the main generalized entropies advances.

On the limits of probabilistic forecasting in nonlinear time series analysis II: Differential entropy.

In a previous paper, the authors studied the limits of probabilistic prediction in nonlinear time series analysis in a perfect model scenario, i.e., in the ideal case that the uncertainty of an otherwise deterministic model is due to only the finite precision of the observations. The model consisted of the symbolic dynamics of a measure-preserving transformation with respect to a finite partition of the state space, and the quality of the predictions was measured by the so-called ignorance score, which is a conditional entropy. In practice, though, partitions are dispensed with by considering numerical and experimental data to be continuous, which prompts us to trade off in this paper the Shannon entropy for the differential entropy. Despite technical differences, we show that the core of the previous results also hold in this extended scenario for sufficiently high precision. The corresponding imperfect model scenario will be revisited too because it is relevant for the applications. The theoretical part and its application to probabilistic forecasting are illustrated with numerical simulations and a new prediction algorithm.

Influence of the hole geometry on the flow distribution in ventricular catheters for hydrocephalus.

BACKGROUND: Hydrocephalus is a medical condition consisting of an abnormal accumulation of cerebrospinal fluid within the brain. A catheter is inserted in one of the brain ventricles and then connected to an external valve to drain the excess of cerebrospinal fluid. The main drawback of this technique is that, over time, the ventricular catheter ends up getting blocked by the cells and macromolecules present in the cerebrospinal fluid. A crucial factor influencing this obstruction is a non-uniform flow pattern through the catheter, since it facilitates adhesion of suspended particles to the walls. In this paper we focus on the effects that tilted holes as well as conical holes have on the flow distribution and shear stress. METHODS: We have carried out 3D computational simulations to study the effect of the hole geometry on the cerebrospinal fluid flow through ventricular catheters. All the simulations were done with the OpenFOAM® toolbox. In particular, three different groups of models were investigated by varying (i) the tilt angles of the holes, (ii) the inner and outer diameters of the holes, and (iii) the distances between the so-called hole segments. RESULTS: The replacement of cylindrical holes by conical holes was found to have a strong influence on the flow distribution and to lower slightly the shear stress. Tilted holes did not involve flow distribution changes when the hole segments are sufficiently separated, but the mean shear stress was certainly reduced. CONCLUSIONS: The authors present new results about the behavior of the fluid flow through ventricular catheters. These results complete earlier work on this topic by adding the influence of the hole geometry. The overall objective pursued by this research is to provide guidelines to improve existing commercially available ventricular catheters.

Parametric study of ventricular catheters for hydrocephalus.

BACKGROUND: To drain the excess of cerebrospinal fluid in a hydrocephalus patient, a catheter is inserted into one of the brain ventricles and then connected to a valve. This so-called ventricular catheter is a standard-size, flexible tubing with a number of holes placed symmetrically around several transversal sections or "drainage segments". Three-dimensional computational dynamics shows that most of the fluid volume flows through the drainage segment closest to the valve. This fact raises the likelihood that those holes and then the lumen get clogged by the cells and macromolecules present in the cerebrospinal fluid, provoking malfunction of the whole system. In order to better understand the flow pattern, we have carried out a parametric study via numerical models of ventricular catheters. METHODS: The parameters chosen are the number of drainage segments, the distances between them, the number and diameter of the holes on each segment, as well as their relative angular position. RESULTS: These parameters were found to have a direct consequence on the flow distribution and shear stress of the catheter. As a consequence, we formulate general principles for ventricular catheter design. CONCLUSIONS: These principles can help develop new catheters with homogeneous flow patterns, thus possibly extending their lifetime.

On the limits of probabilistic forecasting in nonlinear times series analysis.

The ignorance score measures the quality of probabilistic forecasting. In this paper, we study its basic properties in the perfect model scenario, i.e., under the assumption that the system producing the data is perfectly known. Two further qualifications are added to this general setting. First, the system is a discrete-time, measure-preserving dynamical system. Moreover, randomness results from the quantization of the state space (i.e., from the finite precision of the observations), rather than being introduced via observational noise. In this "non-linear" perfect model scenario we derive, in particular, the admissible domain of the ignorance score and relate it with the ignorance score in imperfect models.

Computing algebraic transfer entropy and coupling directions via transcripts.

Most random processes studied in nonlinear time series analysis take values on sets endowed with a group structure, e.g., the real and rational numbers, and the integers. This fact allows to associate with each pair of group elements a third element, called their transcript, which is defined as the product of the second element in the pair times the first one. The transfer entropy of two such processes is called algebraic transfer entropy. It measures the information transferred between two coupled processes whose values belong to a group. In this paper, we show that, subject to one constraint, the algebraic transfer entropy matches the (in general, conditional) mutual information of certain transcripts with one variable less. This property has interesting practical applications, especially to the analysis of short time series. We also derive weak conditions for the 3-dimensional algebraic transfer entropy to yield the same coupling direction as the corresponding mutual information of transcripts. A related issue concerns the use of mutual information of transcripts to determine coupling directions in cases where the conditions just mentioned are not fulfilled. We checked the latter possibility in the lowest dimensional case with numerical simulations and cardiovascular data, and obtained positive results.

New designs of ventricular catheters for hydrocephalus by 3-D computational fluid dynamics.

INTRODUCTION: Based on a landmark study by Lin et al. of the two-dimensional flow in ventricular catheters (VCs) via computational fluid dynamics (CFD), we studied in a previous paper the three-dimensional flow patterns of five commercially available VC. We found that the drainage of the cerebrospinal fluid (CSF) mostly occurs through the catheter's most proximal holes. In this paper, we design five VC prototypes with equalized flow characteristics. METHODS: We study five prototypes of VC by means of CFD in three-dimensional (3-D) automated models and compare the fluid-mechanical results with our previous study of currently in use VC. The general procedure for the development of a CFD model calls for transforming the physical dimensions of the system to be studied into a virtual wire-frame model, which provides the coordinates for the virtual space of a CFD mesh. The incompressible Navier-Stokes equations, a system of strongly coupled, nonlinear, partial differential equations governing the motion of the flow field, are then solved numerically. RESULTS: By varying the number of drainage holes and the ratio hole/segment, we improved flow characteristics in five prototypes of VC. Models 1, 2, and 3 have a distal to proximal decreasing flow. Model 4 has an inverse flow to the previous ones, that is, a distal to proximal increasing flow, while model 5 has a constant flow over the segments. CONCLUSIONS: New catheter designs with variable hole diameter, number of holes, and ratio hole/segment along the catheter allow the fluid to enter the catheter more uniformly along its length, thus reducing the chance that the catheter becomes occluded.