Fast, accurate, and interpretable decoding of electrocorticographic signals using dynamic mode decomposition.

Dynamic mode (DM) decomposition decomposes spatiotemporal signals into basic oscillatory components (DMs). DMs can improve the accuracy of neural decoding when used with the nonlinear Grassmann kernel, compared to conventional power features. However, such kernel-based machine learning algorithms have three limitations: large computational time preventing real-time application, incompatibility with non-kernel algorithms, and low interpretability. Here, we propose a mapping function corresponding to the Grassmann kernel that explicitly transforms DMs into spatial DM (sDM) features, which can be used in any machine learning algorithm. Using electrocorticographic signals recorded during various movement and visual perception tasks, the sDM features were shown to improve the decoding accuracy and computational time compared to conventional methods. Furthermore, the components of the sDM features informative for decoding showed similar characteristics to the high-γ power of the signals, but with higher trial-to-trial reproducibility. The proposed sDM features enable fast, accurate, and interpretable neural decoding.

Estimating Counterfactual Treatment Outcomes Over Time in Complex Multiagent Scenarios.

Evaluation of intervention in a multiagent system, for example, when humans should intervene in autonomous driving systems and when a player should pass to teammates for a good shot, is challenging in various engineering and scientific fields. Estimating the individual treatment effect (ITE) using counterfactual long-term prediction is practical to evaluate such interventions. However, most of the conventional frameworks did not consider the time-varying complex structure of multiagent relationships and covariate counterfactual prediction. This may lead to erroneous assessments of ITE and difficulty in interpretation. Here, we propose an interpretable, counterfactual recurrent network in multiagent systems to estimate the effect of the intervention. Our model leverages graph variational recurrent neural networks (GVRNNs) and theory-based computation with domain knowledge for the ITE estimation framework based on long-term prediction of multiagent covariates and outcomes, which can confirm the circumstances under which the intervention is effective. On simulated models of an automated vehicle and biological agents with time-varying confounders, we show that our methods achieved lower estimation errors in counterfactual covariates and the most effective treatment timing than the baselines. Furthermore, using real basketball data, our methods performed realistic counterfactual predictions and evaluated the counterfactual passes in shot scenarios.

Manifold alteration between major depressive disorder and healthy control subjects using dynamic mode decomposition in resting-state fMRI data.

Background: The World Health Organization has reported that approximately 300 million individuals suffer from the mood disorder known as MDD. Non-invasive measurement techniques have been utilized to reveal the mechanism of MDD, with rsfMRI being the predominant method. The previous functional connectivity and energy landscape studies have shown the difference in the coactivation patterns between MDD and HCs. However, these studies did not consider oscillatory temporal dynamics. Methods: In this study, the dynamic mode decomposition, a method to compute a set of coherent spatial patterns associated with the oscillation frequency and temporal decay rate, was employed to investigate the alteration of the occurrence of dynamic modes between MDD and HCs. Specifically, The BOLD signals of each subject were transformed into dynamic modes representing coherent spatial patterns and discrete-time eigenvalues to capture temporal variations using dynamic mode decomposition. All the dynamic modes were disentangled into a two-dimensional manifold using t-SNE. Density estimation and density ratio estimation were applied to the two-dimensional manifolds after the two-dimensional manifold was split based on HCs and MDD. Results: The dynamic modes that uniquely emerged in the MDD were not observed. Instead, we have found some dynamic modes that have shown increased or reduced occurrence in MDD compared with HCs. The reduced dynamic modes were associated with the visual and saliency networks while the increased dynamic modes were associated with the default mode and sensory-motor networks. Conclusion: To the best of our knowledge, this study showed initial evidence of the alteration of occurrence of the dynamic modes between MDD and HCs. To deepen understanding of how the alteration of the dynamic modes emerges from the structure, it is vital to investigate the relationship between the dynamic modes, cortical thickness, and surface areas.

Dynamic mode decomposition for Koopman spectral analysis of elementary cellular automata.

We apply dynamic mode decomposition (DMD) to elementary cellular automata (ECA). Three types of DMD methods are considered, and the reproducibility of the system dynamics and Koopman eigenvalues from observed time series is investigated. While standard DMD fails to reproduce the system dynamics and Koopman eigenvalues associated with a given periodic orbit in some cases, Hankel DMD with delay-embedded time series improves reproducibility. However, Hankel DMD can still fail to reproduce all the Koopman eigenvalues in specific cases. We propose an extended DMD method for ECA that uses nonlinearly transformed time series with discretized Walsh functions and show that it can completely reproduce the dynamics and Koopman eigenvalues. Linear-algebraic backgrounds for the reproducibility of the system dynamics and Koopman eigenvalues are also discussed.

Decentralized policy learning with partial observation and mechanical constraints for multiperson modeling.

Extracting the rules of real-world multi-agent behaviors is a current challenge in various scientific and engineering fields. Biological agents independently have limited observation and mechanical constraints; however, most of the conventional data-driven models ignore such assumptions, resulting in lack of biological plausibility and model interpretability for behavioral analyses. Here we propose sequential generative models with partial observation and mechanical constraints in a decentralized manner, which can model agents' cognition and body dynamics, and predict biologically plausible behaviors. We formulate this as a decentralized multi-agent imitation-learning problem, leveraging binary partial observation and decentralized policy models based on hierarchical variational recurrent neural networks with physical and biomechanical penalties. Using real-world basketball and soccer datasets, we show the effectiveness of our method in terms of the constraint violations, long-term trajectory prediction, and partial observation. Our approach can be used as a multi-agent simulator to generate realistic trajectories using real-world data.

Stable invariant models via Koopman spectra.

Weight-tied models have attracted attention in the modern development of neural networks. The deep equilibrium model (DEQ) represents infinitely deep neural networks with weight-tying, and recent studies have shown the potential of this type of approach. DEQs are needed to iteratively solve root-finding problems in training and are built on the assumption that the underlying dynamics determined by the models converge to a fixed point. In this paper, we present the stable invariant model (SIM), a new class of deep models that in principle approximates DEQs under stability and extends the dynamics to more general ones converging to an invariant set (not restricted in a fixed point). The key ingredient in deriving SIMs is a representation of the dynamics with the spectra of the Koopman and Perron-Frobenius operators. This perspective approximately reveals stable dynamics with DEQs and then derives two variants of SIMs. We also propose an implementation of SIMs that can be learned in the same way as feedforward models. We illustrate the empirical performance of SIMs with experiments and demonstrate that SIMs achieve comparative or superior performance against DEQs in several learning tasks.

Predicting behavior through dynamic modes in resting-state fMRI data.

Dynamic properties of resting-state functional connectivity (FC) provide rich information on brain-behavior relationships. Dynamic mode decomposition (DMD) has been used as a method to characterize FC dynamics. However, it remains unclear whether dynamic modes (DMs), spatial-temporal coherent patterns computed by DMD, provide information about individual behavioral differences. This study established a methodological approach to predict individual differences in behavior using DMs. Furthermore, we investigated the contribution of DMs within each of seven specific frequency bands (0-0.1,...,0.6-0.7 Hz) for prediction. To validate our approach, we tested whether each of 59 behavioral measures could be predicted by performing multivariate pattern analysis on a Gram matrix, which was created using subject-specific DMs computed from resting-state functional magnetic resonance imaging (rs-fMRI) data of individuals. DMD successfully predicted behavior and outperformed temporal and spatial independent component analysis, which is the conventional data decomposition method for extracting spatial activity patterns. Most of the behavioral measures that were predicted with significant accuracy in a permutation test were related to cognition. We found that DMs within frequency bands <0.2 Hz primarily contributed to prediction and had spatial structures similar to several common resting-state networks. Our results indicate that DMD is efficient in extracting spatiotemporal features from rs-fMRI data.

Koopman spectral analysis of elementary cellular automata.

We perform a Koopman spectral analysis of elementary cellular automata (ECA). By lifting the system dynamics using a one-hot representation of the system state, we derive a matrix representation of the Koopman operator as the transpose of the adjacency matrix of the state-transition network. The Koopman eigenvalues are either zero or on the unit circle in the complex plane, and the associated Koopman eigenfunctions can be explicitly constructed. From the Koopman eigenvalues, we can judge the reversibility, determine the number of connected components in the state-transition network, evaluate the period of asymptotic orbits, and derive the conserved quantities for each system. We numerically calculate the Koopman eigenvalues of all rules of ECA on a one-dimensional lattice of 13 cells with periodic boundary conditions. It is shown that the spectral properties of the Koopman operator reflect Wolfram's classification of ECA.

Prediction of Compound Bioactivities Using Heat-Diffusion Equation.

Machine learning is expected to improve low throughput and high assay cost in cell-based phenotypic screening. However, it is still a challenge to apply machine learning to achieving sufficiently complex phenotypic screening due to imbalanced datasets, non-linear prediction, and unpredictability of new chemotypes. Here, we developed a prediction model based on the heat-diffusion equation (PM-HDE) to address this issue. The algorithm was verified as feasible for virtual compound screening using biotest data of 946 assay systems registered with PubChem. PM-HDE was then applied to actual screening. Based on supervised learning of the data of about 50,000 compounds from biological phenotypic screening with motor neurons derived from ALS-patient-induced pluripotent stem cells, virtual screening of >1.6 million compounds was implemented. We confirmed that PM-HDE enriched the hit compounds and identified new chemotypes. This prediction model could overcome the inflexibility in machine learning, and our approach could provide a novel platform for drug discovery.

Cognition and interpersonal coordination of patients with schizophrenia who have sports habits.

Team sports activities are effective for improving the negative symptoms and cognitive functions in patients with schizophrenia. However, the interpersonal coordination during the sports and visual cognition of patients with schizophrenia who have team sports habits are unknown. The main objectives of this study were to test two hypotheses: first, patients with schizophrenia perform the skill requiring ball passing and receiving worse than healthy controls; and second, the patients will be impaired in these functionings in accordance with the previous studies regarding schizophrenia in general. Twelve patients with schizophrenia and 15 healthy controls, who had habits in football, participated in this study. The participants performed three conventional cognitive tests and a 3-vs-1 ball possession task to evaluate their interpersonal coordination. The results showed that in the 3-vs-1 possession task, the displacement in the pass angle for the patients was significantly smaller than that for the control. The recall in the complex figure test, the performance in the trail making test, and that in the five-choice reaction task for the patients were worse than those for the control. Moreover, we found the significant partial correlations in the patients between the extradimensional shift error and the pass angle as well as between the time in the trail making test and the displacement in the pass angle, whereas there was no significant correlation in the control group. This study clarified the impaired interpersonal coordination during team sports and the visual cognition of patients with schizophrenia who have team sports habits.

Neural decoding of electrocorticographic signals using dynamic mode decomposition.

OBJECTIVE: Brain-computer interfaces (BCIs) using electrocorticographic (ECoG) signals have been developed to restore the communication function of severely paralyzed patients. However, the limited amount of information derived from ECoG signals hinders their clinical applications. We aimed to develop a method to decode ECoG signals using spatiotemporal patterns characterizing movement types to increase the amount of information gained from these signals. APPROACH: Previous studies have demonstrated that motor information could be decoded using powers of specific frequency bands of the ECoG signals estimated by fast Fourier transform (FFT) or wavelet analysis. However, because FFT is evaluated for each channel, the temporal and spatial patterns among channels are difficult to evaluate. Here, we used dynamic mode decomposition (DMD) to evaluate the spatiotemporal pattern of ECoG signals and evaluated the accuracy of motor decoding with the DMD modes. We used ECoG signals during three types of hand movements, which were recorded from 11 patients implanted with subdural electrodes. From the signals at the time of the movements, the modes and powers were evaluated by DMD and FFT and were decoded using support vector machine. We used the Grassmann kernel to evaluate the distance between modes estimated by DMD (DMD mode). In addition, we decoded the DMD modes, in which the phase components were shuffled, to compare the classification accuracy. MAIN RESULTS: The decoding accuracy using DMD modes was significantly better than that using FFT powers. The accuracy significantly decreased when the phases of the DMD mode were shuffled. Among the frequency bands, the DMD mode at approximately 100 Hz demonstrated the highest classification accuracy. SIGNIFICANCE: DMD successfully captured the spatiotemporal patterns characterizing the movement types and contributed to improving the decoding accuracy. This method can be applied to improve BCIs to help severely paralyzed patients communicate.

Physically-interpretable classification of biological network dynamics for complex collective motions.

Understanding biological network dynamics is a fundamental issue in various scientific and engineering fields. Network theory is capable of revealing the relationship between elements and their propagation; however, for complex collective motions, the network properties often transiently and complexly change. A fundamental question addressed here pertains to the classification of collective motion network based on physically-interpretable dynamical properties. Here we apply a data-driven spectral analysis called graph dynamic mode decomposition, which obtains the dynamical properties for collective motion classification. Using a ballgame as an example, we classified the strategic collective motions in different global behaviours and discovered that, in addition to the physical properties, the contextual node information was critical for classification. Furthermore, we discovered the label-specific stronger spectra in the relationship among the nearest agents, providing physical and semantic interpretations. Our approach contributes to the understanding of principles of biological complex network dynamics from the perspective of nonlinear dynamical systems.

Principal points analysis via p-median problem for binary data.

Analysis with principal points is a useful statistical tool for summarizing large data. In this paper, we propose a subgradient-based algorithm to calculate a set of principal points for multivariate binary data by the formulating it as a p-median problem. This enables us to find a globally optimal set of principal points or an ε-optimal solution in the middle of the calculation by combining an upper bound found using the greedy method. This algorithm is an iterative procedure where each iteration can be calculated in an efficient manner. We investigate the applicability of the proposed framework with questionnaire data and arXiv co-authors data.

Data-driven spectral analysis for coordinative structures in periodic human locomotion.

Living organisms dynamically and flexibly operate a great number of components. As one of such redundant control mechanisms, low-dimensional coordinative structures among multiple components have been investigated. However, structures extracted from the conventional statistical dimensionality reduction methods do not reflect dynamical properties in principle. Here we regard coordinative structures in biological periodic systems with unknown and redundant dynamics as a nonlinear limit-cycle oscillation, and apply a data-driven operator-theoretic spectral analysis, which obtains dynamical properties of coordinative structures such as frequency and phase from the estimated eigenvalues and eigenfunctions of a composition operator. Using segmental angle series during human walking as an example, we first extracted the coordinative structures based on dynamics; e.g. the speed-independent coordinative structures in the harmonics of gait frequency. Second, we discovered the speed-dependent time-evolving behaviours of the phase by estimating the eigenfunctions via our approach on the conventional low-dimensional structures. We also verified our approach using the double pendulum and walking model simulation data. Our results of locomotion analysis suggest that our approach can be useful to analyse biological periodic phenomena from the perspective of nonlinear dynamical systems.

Dynamic mode decomposition in vector-valued reproducing kernel Hilbert spaces for extracting dynamical structure among observables.

Understanding nonlinear dynamical systems (NLDSs) is challenging in a variety of engineering and scientific fields. Dynamic mode decomposition (DMD), which is a numerical algorithm for the spectral analysis of Koopman operators, has been attracting attention as a way of obtaining global modal descriptions of NLDSs without requiring explicit prior knowledge. However, since existing DMD algorithms are in principle formulated based on the concatenation of scalar observables, it is not directly applicable to data with dependent structures among observables, which take, for example, the form of a sequence of graphs. In this paper, we formulate Koopman spectral analysis for NLDSs with structures among observables and propose an estimation algorithm for this problem. This method can extract and visualize the underlying low-dimensional global dynamics of NLDSs with structures among observables from data, which can be useful in understanding the underlying dynamics of such NLDSs. To this end, we first formulate the problem of estimating spectra of the Koopman operator defined in vector-valued reproducing kernel Hilbert spaces, and then develop an estimation procedure for this problem by reformulating tensor-based DMD. As a special case of our method, we propose the method named as Graph DMD, which is a numerical algorithm for Koopman spectral analysis of graph dynamical systems, using a sequence of adjacency matrices. We investigate the empirical performance of our method by using synthetic and real-world data.

Automatically recognizing strategic cooperative behaviors in various situations of a team sport.

Understanding multi-agent cooperative behavior is challenging in various scientific and engineering domains. In some cases, such as team sports, many cooperative behaviors can be visually categorized and labeled manually by experts. However, these actions which are manually categorized with the same label based on its function have low spatiotemporal similarity. In other words, it is difficult to find similar and different structures of the motions with the same and different labels, respectively. Here, we propose an automatic recognition system for strategic cooperative plays, which are the minimal, basic, and diverse plays in a ball game. Using player's moving distance, geometric information, and distances among players, the proposed method accurately discriminated not only the cooperative plays in a primary area, i.e., near the ball, but also those distant from a primary area. We also propose a method to classify more detailed types of cooperative plays in various situations. The proposed framework, which sheds light on inconspicuous players to play important roles, could have a potential to detect well-defined and labeled cooperative behaviors.

Prediction and classification in equation-free collective motion dynamics.

Modeling the complex collective behavior is a challenging issue in several material and life sciences. The collective motion has been usually modeled by simple interaction rules and explained by global statistics. However, it remains difficult to bridge the gap between the dynamic properties of the complex interaction and the emerging group-level functions. Here we introduce decomposition methods to directly extract and classify the latent global dynamics of nonlinear dynamical systems in an equation-free manner, even including complex interaction in few data dimensions. We first verified that the basic decomposition method can extract and discriminate the dynamics of a well-known rule-based fish-schooling (or bird-flocking) model. The method extracted different temporal frequency modes with spatial interaction coherence among three distinct emergent motions, whereas these wave properties in multiple spatiotemporal scales showed similar dispersion relations. Second, we extended the basic method to map high-dimensional feature space for application to actual small-dimensional systems complexly changing the interaction rules. Using group sports human data, we classified the dynamics and predicted the group objective achievement. Our methods have a potential for classifying collective motions in various domains which obey in non-trivial dominance law known as active matters.

Highly biocompatible super-resolution fluorescence imaging using the fast photoswitching fluorescent protein Kohinoor and SPoD-ExPAN with Lp-regularized image reconstruction.

Far-field super-resolution fluorescence microscopy has enabled us to visualize live cells in great detail and with an unprecedented resolution. However, the techniques developed thus far have required high-power illumination (102-106 W/cm2), which leads to considerable phototoxicity to live cells and hampers time-lapse observation of the cells. In this study we show a highly biocompatible super-resolution microscopy technique that requires a very low-power illumination. The present technique combines a fast photoswitchable fluorescent protein, Kohinoor, with SPoD-ExPAN (super-resolution by polarization demodulation/excitation polarization angle narrowing). With this technique, we successfully observed Kohinoor-fusion proteins involving vimentin, paxillin, histone and clathrin expressed in HeLa cells at a spatial resolution of 70-80 nm with illumination power densities as low as ~1 W/cm2 for both excitation and photoswitching. Furthermore, although the previous SPoD-ExPAN technique used L1-regularized maximum-likelihood calculations to reconstruct super-resolved images, we devised an extension to the Lp-regularization to obtain super-resolved images that more accurately describe objects at the specimen plane. Thus, the present technique would significantly extend the applicability of super-resolution fluorescence microscopy for live-cell imaging.

Subspace dynamic mode decomposition for stochastic Koopman analysis.

The analysis of nonlinear dynamical systems based on the Koopman operator is attracting attention in various applications. Dynamic mode decomposition (DMD) is a data-driven algorithm for Koopman spectral analysis, and several variants with a wide range of applications have been proposed. However, popular implementations of DMD suffer from observation noise on random dynamical systems and generate inaccurate estimation of the spectra of the stochastic Koopman operator. In this paper, we propose subspace DMD as an algorithm for the Koopman analysis of random dynamical systems with observation noise. Subspace DMD first computes the orthogonal projection of future snapshots to the space of past snapshots and then estimates the spectra of a linear model, and its output converges to the spectra of the stochastic Koopman operator under standard assumptions. We investigate the empirical performance of subspace DMD with several dynamical systems and show its utility for the Koopman analysis of random dynamical systems.

Toxicity prediction from toxicogenomic data based on class association rule mining.

While the recent advent of new technologies in biology such as DNA microarray and next-generation sequencer has given researchers a large volume of data representing genome-wide biological responses, it is not necessarily easy to derive knowledge that is accurate and understandable at the same time. In this study, we applied the Classification Based on Association (CBA) algorithm, one of the class association rule mining techniques, to the TG-GATEs database, where both toxicogenomic and toxicological data of more than 150 compounds in rat and human are stored. We compared the generated classifiers between CBA and linear discriminant analysis (LDA) and showed that CBA is superior to LDA in terms of both predictive performances (accuracy: 83% for CBA vs. 75% for LDA, sensitivity: 82% for CBA vs. 72% for LDA, specificity: 85% for CBA vs. 75% for LDA) and interpretability.