Existence of reservoir with finite-dimensional output for universal reservoir computing.

In this paper, we prove the existence of a reservoir that has a finite-dimensional output and makes the reservoir computing model universal. Reservoir computing is a method for dynamical system approximation that trains the static part of a model but fixes the dynamical part called the reservoir. Hence, reservoir computing has the advantage of training models with a low computational cost. Moreover, fixed reservoirs can be implemented as physical systems. Such reservoirs have attracted attention in terms of computation speed and energy consumption. The universality of a reservoir computing model is its ability to approximate an arbitrary system with arbitrary accuracy. Two sufficient reservoir conditions to make the model universal have been proposed. The first is the combination of fading memory and the separation property. The second is the neighborhood separation property, which we proposed recently. To date, it has been unknown whether a reservoir with a finite-dimensional output can satisfy these conditions. In this study, we prove that no reservoir with a finite-dimensional output satisfies the former condition. By contrast, we propose a single output reservoir that satisfies the latter condition. This implies that, for any dimension, a reservoir making the model universal exists with the output of that specified dimension. These results clarify the practical importance of our proposed conditions.

Automatic Temperature Parameter Tuning for Reinforcement Learning Using Path Integral Policy Improvement.

In this article, we propose a novel variant of path integral policy improvement with covariance matrix adaptation ( [Formula: see text] - [Formula: see text] ), which is a reinforcement learning (RL) algorithm that aims to optimize a parameterized policy for the continuous behavior of robots. [Formula: see text] - [Formula: see text] has a hyperparameter called the temperature parameter, and its value is critical for performance; however, little research has been conducted on it and the existing method still contains a tunable parameter, which can be critical to performance. Therefore, tuning by trial and error is necessary in the existing method. Moreover, we show that there is a problem setting that cannot be learned by the existing method. The proposed method solves both problems by automatically adjusting the temperature parameter for each update. We confirmed the effectiveness of the proposed method using numerical tests.

Nonessentiality of Reservoir's Fading Memory for Universality of Reservoir Computing.

This article describes a novel sufficient condition concerning approximations with reservoir computing (RC). Recently, RC using a physical system as the reservoir has attracted attention. Because many physical systems are modeled as state-space systems, it is necessary to guarantee the approximations given by reservoirs represented as nonlinear state-space systems. There are two problems with existing approaches: a reservoir must have a property called fading memory and must be represented as a set of maps between input and output signals on the bi-infinite-time (BIT) interval. These two conditions are too strict for reservoirs represented as nonlinear state-space systems as they require the reservoir to have a unique equilibrium state for the zero input. This article proposes an approach that employs operators from right-infinite-time (RIT) inputs to RIT outputs. Furthermore, we develop a novel extension of the Stone-Weierstrass theorem to handle discontinuous functions. To apply the extended theorem, we define functionals corresponding to operators and introduce a metric on the domain of the functionals. The resulting sufficient condition does not require the reservoir to have fading memory or continuity with respect to inputs and time. Therefore, our result guarantees the approximations with very common reservoirs and provides a rationale for physical RC. We present an example of a physical reservoir without fading memory. With the example reservoir, the RC model successfully approximates NARMA10, a benchmark task for time series predictions.

Structural Bistability Analysis of Flower-Shaped and Chain-Shaped Boolean Networks.

Bistability, i.e., the existence of just two stable equilibria, is known to play an important role in biological systems, e.g., cellular differentiation and apoptosis. In this paper, we consider the bistability but as a structural property of a class of network systems, that is, the bistability under the assumption that the information on the network structure is available but the information on the components is not available. First, we introduce Boolean networks as a model of biological network systems and give the notion of structural bistability. We next focus on the systems with a flower-shaped network structure and present a necessary and sufficient condition based on three characteristics of the network topology. Finally, the result is extended to the Boolean networks with a chain-shaped network structure.

Performance Analysis of Chemotaxis Controllers: Which has Better Chemotaxis Controller, Escherichia coli or Paramecium caudatum?

Chemotaxis is the biological phenomenon in which organisms move to a more favorable location in an environment with a chemical attractant or repellent. Since chemotaxis is a typical example of the environmental response of organisms, it is a fundamental topic in biology and related fields. We discuss the performance of the internal controllers that generate chemotaxis. We first propose performance indices to evaluate the controllers. Based on these indices, we evaluate the performance of two controller models of Escherichia coli and Paramecium caudatum. As a result, it is disclosed that the E. coli-type controller achieves chemotaxis quickly but roughly, whereas the P. caudatum-type controller achieves it slowly but precisely. This result will be a biological contribution from a control theoretic point of view.

Model predictive control for optimally scheduling intermittent androgen suppression of prostate cancer.

Mathematical modeling of prostate cancer under intermittent androgen suppression revealed that we may be able to delay relapse by optimally scheduling the hormone therapy for each patient. However, our previous study showed the difficulty of the scheduling by minimizing the maximal tumor growth rate because the transient dynamics is also important and can help to delay the relapse for a finite time. Here, we propose to use model predictive control for scheduling intermittent androgen suppression. We find that model predictive control tends to delay the relapse of prostate specific antigen more than the method with minimizing the maximal tumor growth rate. Therefore, model predictive control is a promising approach for practically applying the mathematical model to optimally schedule intermittent androgen suppression.