Mobile Sensor Path Planning for Kalman Filter Spatiotemporal Estimation.

The estimation of spatiotemporal data from limited sensor measurements is a required task across many scientific disciplines. In this paper, we consider the use of mobile sensors for estimating spatiotemporal data via Kalman filtering. The sensor selection problem, which aims to optimize the placement of sensors, leverages innovations in greedy algorithms and low-rank subspace projection to provide model-free, data-driven estimates. Alternatively, Kalman filter estimation balances model-based information and sparsely observed measurements to collectively make better estimation with limited sensors. It is especially important with mobile sensors to utilize historical measurements. We show that mobile sensing along dynamic trajectories can achieve the equivalent performance of a larger number of stationary sensors, with performance gains related to three distinct timescales: (i) the timescale of the spatiotemporal dynamics, (ii) the velocity of the sensors, and (iii) the rate of sampling. Taken together, these timescales strongly influence how well-conditioned the estimation task is. We draw connections between the Kalman filter performance and the observability of the state space model and propose a greedy path planning algorithm based on minimizing the condition number of the observability matrix. This approach has better scalability and computational efficiency compared to previous works. Through a series of examples of increasing complexity, we show that mobile sensing along our paths improves Kalman filter performance in terms of better limiting estimation and faster convergence. Moreover, it is particularly effective for spatiotemporal data that contain spatially localized structures, whose features are captured along dynamic trajectories.

Hierarchical deep learning of multiscale differential equation time-steppers.

Nonlinear differential equations rarely admit closed-form solutions, thus requiring numerical time-stepping algorithms to approximate solutions. Further, many systems characterized by multiscale physics exhibit dynamics over a vast range of timescales, making numerical integration expensive. In this work, we develop a hierarchy of deep neural network time-steppers to approximate the dynamical system flow map over a range of time-scales. The model is purely data-driven, enabling accurate and efficient numerical integration and forecasting. Similar ideas can be used to couple neural network-based models with classical numerical time-steppers. Our hierarchical time-stepping scheme provides advantages over current time-stepping algorithms, including (i) capturing a range of timescales, (ii) improved accuracy in comparison with leading neural network architectures, (iii) efficiency in long-time forecasting due to explicit training of slow time-scale dynamics, and (iv) a flexible framework that is parallelizable and may be integrated with standard numerical time-stepping algorithms. The method is demonstrated on numerous nonlinear dynamical systems, including the Van der Pol oscillator, the Lorenz system, the Kuramoto-Sivashinsky equation, and fluid flow pass a cylinder; audio and video signals are also explored. On the sequence generation examples, we benchmark our algorithm against state-of-the-art methods, such as LSTM, reservoir computing and clockwork RNN. This article is part of the theme issue 'Data-driven prediction in dynamical systems'.

Data-driven discovery of coordinates and governing equations.

The discovery of governing equations from scientific data has the potential to transform data-rich fields that lack well-characterized quantitative descriptions. Advances in sparse regression are currently enabling the tractable identification of both the structure and parameters of a nonlinear dynamical system from data. The resulting models have the fewest terms necessary to describe the dynamics, balancing model complexity with descriptive ability, and thus promoting interpretability and generalizability. This provides an algorithmic approach to Occam's razor for model discovery. However, this approach fundamentally relies on an effective coordinate system in which the dynamics have a simple representation. In this work, we design a custom deep autoencoder network to discover a coordinate transformation into a reduced space where the dynamics may be sparsely represented. Thus, we simultaneously learn the governing equations and the associated coordinate system. We demonstrate this approach on several example high-dimensional systems with low-dimensional behavior. The resulting modeling framework combines the strengths of deep neural networks for flexible representation and sparse identification of nonlinear dynamics (SINDy) for parsimonious models. This method places the discovery of coordinates and models on an equal footing.

Neural-inspired sensors enable sparse, efficient classification of spatiotemporal data.

Sparse sensor placement is a central challenge in the efficient characterization of complex systems when the cost of acquiring and processing data is high. Leading sparse sensing methods typically exploit either spatial or temporal correlations, but rarely both. This work introduces a sparse sensor optimization that is designed to leverage the rich spatiotemporal coherence exhibited by many systems. Our approach is inspired by the remarkable performance of flying insects, which use a few embedded strain-sensitive neurons to achieve rapid and robust flight control despite large gust disturbances. Specifically, we identify neural-inspired sensors at a few key locations on a flapping wing that are able to detect body rotation. This task is particularly challenging as the rotational twisting mode is three orders of magnitude smaller than the flapping modes. We show that nonlinear filtering in time, built to mimic strain-sensitive neurons, is essential to detect rotation, whereas instantaneous measurements fail. Optimized sparse sensor placement results in efficient classification with approximately 10 sensors, achieving the same accuracy and noise robustness as full measurements consisting of hundreds of sensors. Sparse sensing with neural-inspired encoding establishes an alternative paradigm in hyperefficient, embodied sensing of spatiotemporal data and sheds light on principles of biological sensing for agile flight control.

Discovering governing equations from data by sparse identification of nonlinear dynamical systems.

Extracting governing equations from data is a central challenge in many diverse areas of science and engineering. Data are abundant whereas models often remain elusive, as in climate science, neuroscience, ecology, finance, and epidemiology, to name only a few examples. In this work, we combine sparsity-promoting techniques and machine learning with nonlinear dynamical systems to discover governing equations from noisy measurement data. The only assumption about the structure of the model is that there are only a few important terms that govern the dynamics, so that the equations are sparse in the space of possible functions; this assumption holds for many physical systems in an appropriate basis. In particular, we use sparse regression to determine the fewest terms in the dynamic governing equations required to accurately represent the data. This results in parsimonious models that balance accuracy with model complexity to avoid overfitting. We demonstrate the algorithm on a wide range of problems, from simple canonical systems, including linear and nonlinear oscillators and the chaotic Lorenz system, to the fluid vortex shedding behind an obstacle. The fluid example illustrates the ability of this method to discover the underlying dynamics of a system that took experts in the community nearly 30 years to resolve. We also show that this method generalizes to parameterized systems and systems that are time-varying or have external forcing.

Spatiotemporal Feedback and Network Structure Drive and Encode Caenorhabditis elegans Locomotion.

Using a computational model of the Caenorhabditis elegans connectome dynamics, we show that proprioceptive feedback is necessary for sustained dynamic responses to external input. This is consistent with the lack of biophysical evidence for a central pattern generator, and recent experimental evidence that proprioception drives locomotion. The low-dimensional functional response of the Caenorhabditis elegans network of neurons to proprioception-like feedback is optimized by input of specific spatial wavelengths which correspond to the spatial scale of real body shape dynamics. Furthermore, we find that the motor subcircuit of the network is responsible for regulating this response, in agreement with experimental expectations. To explore how the connectomic dynamics produces the observed two-mode, oscillatory limit cycle behavior from a static fixed point, we probe the fixed point's low-dimensional structure using Dynamic Mode Decomposition. This reveals that the nonlinear network dynamics encode six clusters of dynamic modes, with timescales spanning three orders of magnitude. Two of these six dynamic mode clusters correspond to previously-discovered behavioral modes related to locomotion. These dynamic modes and their timescales are encoded by the network's degree distribution and specific connectivity. This suggests that behavioral dynamics are partially encoded within the connectome itself, the connectivity of which facilitates proprioceptive control.

Sparse identification of nonlinear dynamics for rapid model recovery.

Big data have become a critically enabling component of emerging mathematical methods aimed at the automated discovery of dynamical systems, where first principles modeling may be intractable. However, in many engineering systems, abrupt changes must be rapidly characterized based on limited, incomplete, and noisy data. Many leading automated learning techniques rely on unrealistically large data sets, and it is unclear how to leverage prior knowledge effectively to re-identify a model after an abrupt change. In this work, we propose a conceptual framework to recover parsimonious models of a system in response to abrupt changes in the low-data limit. First, the abrupt change is detected by comparing the estimated Lyapunov time of the data with the model prediction. Next, we apply the sparse identification of nonlinear dynamics (SINDy) regression to update a previously identified model with the fewest changes, either by addition, deletion, or modification of existing model terms. We demonstrate this sparse model recovery on several examples for abrupt system change detection in periodic and chaotic dynamical systems. Our examples show that sparse updates to a previously identified model perform better with less data, have lower runtime complexity, and are less sensitive to noise than identifying an entirely new model. The proposed abrupt-SINDy architecture provides a new paradigm for the rapid and efficient recovery of a system model after abrupt changes.

Extremum-seeking control of the beam pattern of a reconfigurable holographic metamaterial antenna.

Robust, continuous, and software-defined beam pattern control of holographic metamaterial antennas is necessary to realize the potential of these low-power-consumption, thin, lightweight, inexpensive antennas for consumer usage of satellite communication. We present a complete feedback control approach that enables adaptive control of the radiation pattern for the electronically scanned metamaterial antenna that is robust to measurement noise and is able to continuously optimize performance throughout changing environmental conditions and antenna characteristics. The physical size, weight, and cost advantages of the metamaterial antenna make it an attractive technology when paired with robust and adaptive on-board software strategies to optimize antenna performance and self-tune for various environmental conditions.

Classification of birefringence in mode-locked fiber lasers using machine learning and sparse representation.

It has been observed that changes in the birefringence, which are difficult or impossible to directly measure, can significantly affect mode-locking in a fiber laser. In this work we develop techniques to estimate the effective birefringence by comparing a test measurement of a given objective function against a learned library. In particular, a toroidal search algorithm is applied to the laser cavity for various birefringence values by varying the waveplate and polarizer angles at incommensurate angular frequencies, thus producing a time-series of the objective function. The resulting time series, which is converted to a spectrogram and then dimensionally reduced with a singular value decomposition, is then labelled with the corresponding effective birefringence and concatenated into a library of modes. A sparse search algorithm (L(1)-norm optimization) is then applied to a test measurement in order to classify the birefringence of the fiber laser. Simulations show that the sparse search algorithm performs very well in recognizing cavity birefringence even in the presence of noise and/or noisy measurements. Once classified, the wave plates and polarizers can be adjusted using servo-control motors to the optimal positions obtained from the toroidal search. The result is an efficient, self-tuning laser.

Promising directions of machine learning for partial differential equations.

Partial differential equations (PDEs) are among the most universal and parsimonious descriptions of natural physical laws, capturing a rich variety of phenomenology and multiscale physics in a compact and symbolic representation. Here, we examine several promising avenues of PDE research that are being advanced by machine learning, including (1) discovering new governing PDEs and coarse-grained approximations for complex natural and engineered systems, (2) learning effective coordinate systems and reduced-order models to make PDEs more amenable to analysis, and (3) representing solution operators and improving traditional numerical algorithms. In each of these fields, we summarize key advances, ongoing challenges, and opportunities for further development.

Towards extending the aircraft flight envelope by mitigating transonic airfoil buffet.

In the age of globalization, commercial aviation plays a central role in maintaining our international connectivity by providing fast air transport services for passengers and freight. However, the upper limit of the aircraft flight envelope, i.e., its operational limit in the high-speed (transonic) regime, is usually fixed by the occurrence of transonic aeroelastic effects. These harmful structural vibrations are associated with an aerodynamic instability called transonic buffet. It refers to shock wave oscillations occurring on the aircraft wings, which induce unsteady aerodynamic loads acting on the wing structure. Since the structural response can cause severe structural damage endangering flight safety, the aviation industry is highly interested in suppressing transonic buffet to extend the flight envelope to higher aircraft speeds. In this contribution, we demonstrate experimentally that the application of porous trailing edges substantially attenuates the buffet phenomenon. Since porous trailing edges have the additional benefit of reducing acoustic aircraft emissions, they could prospectively provide faster air transport with reduced noise emissions.

Sensorimotor delays constrain robust locomotion in a 3D kinematic model of fly walking.

Walking animals must maintain stability in the presence of external perturbations, despite significant temporal delays in neural signaling and muscle actuation. Here, we develop a 3D kinematic model with a layered control architecture to investigate how sensorimotor delays constrain robustness of walking behavior in the fruit fly, Drosophila. Motivated by the anatomical architecture of insect locomotor control circuits, our model consists of three component layers: a neural network that generates realistic 3D joint kinematics for each leg, an optimal controller that executes the joint kinematics while accounting for delays, and an inter-leg coordinator. The model generates realistic simulated walking that matches real fly walking kinematics and sustains walking even when subjected to unexpected perturbations, generalizing beyond its training data. However, we found that the model's robustness to perturbations deteriorates when sensorimotor delay parameters exceed the physiological range. These results suggest that fly sensorimotor control circuits operate close to the temporal limit at which they can detect and respond to external perturbations. More broadly, we show how a modular, layered model architecture can be used to investigate physiological constraints on animal behavior.

The experimental multi-arm pendulum on a cart: A benchmark system for chaos, learning, and control.

The single, double, and triple pendulum has served as an illustrative experimental benchmark system for scientists to study dynamical behavior for more than four centuries. The pendulum system exhibits a wide range of interesting behaviors, from simple harmonic motion in the single pendulum to chaotic dynamics in multi-arm pendulums. Under forcing, even the single pendulum may exhibit chaos, providing a simple example of a damped-driven system. All multi-armed pendulums are characterized by the existence of index-one saddle points, which mediate the transport of trajectories in the system, providing a simple mechanical analog of various complex transport phenomena, from biolocomotion to transport within the solar system. Further, pendulum systems have long been used to design and test both linear and nonlinear control strategies, with the addition of more arms making the problem more challenging. In this work, we provide extensive designs for the construction and operation of a high-performance, multi-link pendulum on a cart system. Although many experimental setups have been built to study the behavior of pendulum systems, such an extensive documentation on the design, construction, and operation is missing from the literature. The resulting experimental system is highly flexible, enabling a wide range of benchmark problems in dynamical systems modeling, system identification and learning, and control. To promote reproducible research, we have made our entire system open-source, including 3D CAD drawings, basic tutorial code, and data. Moreover, we discuss the possibility of extending our system capability to be operated remotely, enabling researchers all around the world to use it, thus increasing access.

Arousal as a universal embedding for spatiotemporal brain dynamics.

Neural activity in awake organisms shows widespread and spatiotemporally diverse correlations with behavioral and physiological measurements. We propose that this covariation reflects in part the dynamics of a unified, arousal-related process that regulates brain-wide physiology on the timescale of seconds. Taken together with theoretical foundations in dynamical systems, this interpretation leads us to a surprising prediction: that a single, scalar measurement of arousal (e.g., pupil diameter) should suffice to reconstruct the continuous evolution of multimodal, spatiotemporal measurements of large-scale brain physiology. To test this hypothesis, we perform multimodal, cortex-wide optical imaging and behavioral monitoring in awake mice. We demonstrate that spatiotemporal measurements of neuronal calcium, metabolism, and blood-oxygen can be accurately and parsimoniously modeled from a low-dimensional state-space reconstructed from the time history of pupil diameter. Extending this framework to behavioral and electrophysiological measurements from the Allen Brain Observatory, we demonstrate the ability to integrate diverse experimental data into a unified generative model via mappings from an intrinsic arousal manifold. Our results support the hypothesis that spontaneous, spatially structured fluctuations in brain-wide physiology-widely interpreted to reflect regionally-specific neural communication-are in large part reflections of an arousal-related process. This enriched view of arousal dynamics has broad implications for interpreting observations of brain, body, and behavior as measured across modalities, contexts, and scales.

Enhancing computational fluid dynamics with machine learning.

Machine learning is rapidly becoming a core technology for scientific computing, with numerous opportunities to advance the field of computational fluid dynamics. Here we highlight some of the areas of highest potential impact, including to accelerate direct numerical simulations, to improve turbulence closure modeling and to develop enhanced reduced-order models. We also discuss emerging areas of machine learning that are promising for computational fluid dynamics, as well as some potential limitations that should be taken into account.

Finite-horizon, energy-efficient trajectories in unsteady flows.

Intelligent mobile sensors, such as uninhabited aerial or underwater vehicles, are becoming prevalent in environmental sensing and monitoring applications. These active sensing platforms operate in unsteady fluid flows, including windy urban environments, hurricanes and ocean currents. Often constrained in their actuation capabilities, the dynamics of these mobile sensors depend strongly on the background flow, making their deployment and control particularly challenging. Therefore, efficient trajectory planning with partial knowledge about the background flow is essential for teams of mobile sensors to adaptively sense and monitor their environments. In this work, we investigate the use of finite-horizon model predictive control (MPC) for the energy-efficient trajectory planning of an active mobile sensor in an unsteady fluid flow field. We uncover connections between trajectories optimized over a finite-time horizon and finite-time Lyapunov exponents of the background flow, confirming that energy-efficient trajectories exploit invariant coherent structures in the flow. We demonstrate our findings on the unsteady double gyre vector field, which is a canonical model for chaotic mixing in the ocean. We present an exhaustive search through critical MPC parameters including the prediction horizon, maximum sensor actuation, and relative penalty on the accumulated state error and actuation effort. We find that even relatively short prediction horizons can often yield energy-efficient trajectories. We also explore these connections on a three-dimensional flow and ocean flow data from the Gulf of Mexico. These results are promising for the adaptive planning of energy-efficient trajectories for swarms of mobile sensors in distributed sensing and monitoring.

An empirical mean-field model of symmetry-breaking in a turbulent wake.

Improved turbulence modeling remains a major open problem in mathematical physics. Turbulence is notoriously challenging, in part due to its multiscale nature and the fact that large-scale coherent structures cannot be disentangled from small-scale fluctuations. This closure problem is emblematic of a greater challenge in complex systems, where coarse-graining and statistical mechanics descriptions break down. This work demonstrates an alternative data-driven modeling approach to learn nonlinear models of the coherent structures, approximating turbulent fluctuations as state-dependent stochastic forcing. We demonstrate this approach on a high-Reynolds number turbulent wake experiment, showing that our model reproduces empirical power spectra and probability distributions. The model is interpretable, providing insights into the physical mechanisms underlying the symmetry-breaking behavior in the wake. This work suggests a path toward low-dimensional models of globally unstable turbulent flows from experimental measurements, with broad implications for other multiscale systems.

Kernel learning for robust dynamic mode decomposition: linear and nonlinear disambiguation optimization.

Research in modern data-driven dynamical systems is typically focused on the three key challenges of high dimensionality, unknown dynamics and nonlinearity. The dynamic mode decomposition (DMD) has emerged as a cornerstone for modelling high-dimensional systems from data. However, the quality of the linear DMD model is known to be fragile with respect to strong nonlinearity, which contaminates the model estimate. By contrast, sparse identification of nonlinear dynamics learns fully nonlinear models, disambiguating the linear and nonlinear effects, but is restricted to low-dimensional systems. In this work, we present a kernel method that learns interpretable data-driven models for high-dimensional, nonlinear systems. Our method performs kernel regression on a sparse dictionary of samples that appreciably contribute to the dynamics. We show that this kernel method efficiently handles high-dimensional data and is flexible enough to incorporate partial knowledge of system physics. It is possible to recover the linear model contribution with this approach, thus separating the effects of the implicitly defined nonlinear terms. We demonstrate our approach on data from a range of nonlinear ordinary and partial differential equations. This framework can be used for many practical engineering tasks such as model order reduction, diagnostics, prediction, control and discovery of governing laws.

Principal component trajectories for modeling spectrally continuous dynamics as forced linear systems.

Delay embeddings of time series data have emerged as a promising coordinate basis for data-driven estimation of the Koopman operator, which seeks a linear representation for observed nonlinear dynamics. Recent work has demonstrated the efficacy of dynamic mode decomposition (DMD) for obtaining finite-dimensional Koopman approximations in delay coordinates. In this paper we demonstrate how nonlinear dynamics with sparse Fourier spectra can be (i) represented by a superposition of principal component trajectories and (ii) modeled by DMD in this coordinate space. For continuous or mixed (discrete and continuous) spectra, DMD can be augmented with an external forcing term. We present a method for learning linear control models in delay coordinates while simultaneously discovering the corresponding exogenous forcing signal in a fully unsupervised manner. This extends the existing DMD with control (DMDc) algorithm to cases where a control signal is not known a priori. We provide examples to validate the learned forcing against a known ground truth and illustrate their statistical similarity. Finally, we offer a demonstration of this method applied to real-world power grid load data to show its utility for diagnostics and interpretation on systems in which somewhat periodic behavior is strongly forced by unknown and unmeasurable environmental variables.

Dimensionally consistent learning with Buckingham Pi.

In the absence of governing equations, dimensional analysis is a robust technique for extracting insights and finding symmetries in physical systems. Given measurement variables and parameters, the Buckingham Pi theorem provides a procedure for finding a set of dimensionless groups that spans the solution space, although this set is not unique. We propose an automated approach using the symmetric and self-similar structure of available measurement data to discover the dimensionless groups that best collapse these data to a lower dimensional space according to an optimal fit. We develop three data-driven techniques that use the Buckingham Pi theorem as a constraint: (1) a constrained optimization problem with a non-parametric input-output fitting function, (2) a deep learning algorithm (BuckiNet) that projects the input parameter space to a lower dimension in the first layer and (3) a technique based on sparse identification of nonlinear dynamics to discover dimensionless equations whose coefficients parameterize the dynamics. We explore the accuracy, robustness and computational complexity of these methods and show that they successfully identify dimensionless groups in three example problems: a bead on a rotating hoop, a laminar boundary layer and Rayleigh-Bénard convection.