Comparing Robot Controller Optimization Methods on Evolvable Morphologies.

In this paper we compare Bayesian Optimization, Differential Evolution, and an Evolution Strategy, employed as a gait learning algorithm in modular robots. The motivational scenario is the joint evolution of morphologies and controllers, where 'newborn' robots also undergo a learning process to optimize their inherited controllers (without changing their bodies). This context raises the question: How do gait learning algorithms compare when applied to various morphologies that are not known in advance (thus need to be treated without priors)? To answer this question, we use a test suite of twenty different robot morphologies to evaluate our gait learners and compare their efficiency, efficacy, and sensitivity to morphological differences. The results indicate that Bayesian Optimization and Differential Evolution deliver the same solution quality (walking speed for the robot) with fewer evaluations than the Evolution Strategy. Furthermore, the Evolution Strategy is more sensitive for morphological differences (its efficacy varies more between different morphologies) and is more subject to luck (repeated runs on the same morphology show greater variance in the outcomes).

How the History of Changing Environments Affects Traits of Evolvable Robot Populations.

The environment is one of the key factors in the emergence of intelligent creatures, but it has received little attention within the Evolutionary Robotics literature. This article investigates the effects of changing environments on morphological and behavioral traits of evolvable robots. In particular, we extend a previous study by evolving robot populations under diverse changing-environment setups, varying the magnitude, frequency, duration, and dynamics of the changes. The results show that long-lasting effects of early generations occur not only when transitioning from easy to hard conditions, but also when going from hard to easy conditions. Furthermore, we demonstrate how the impact of environmental scaffolding is dependent on the nature of the environmental changes involved.

Real-World Robot Evolution: Why Would it (not) Work?

This paper takes a critical look at the concept of real-world robot evolution discussing specific challenges for making it practicable. After a brief review of the state of the art several enablers are discussed in detail. It is noted that sample efficient evolution is one of the key prerequisites and there are various promising directions towards this in different stages of maturity, including learning as part of the evolutionary system, genotype filtering, and hybridizing real-world evolution with simulations in a new way. Furthermore, it is emphasized that an evolutionary system that works in the real world needs robots that work in the real world. Obvious as it may seem, to achieve this significant complexification of the robots and their tasks is needed compared to the current practice. Finally, the importance of not only building but also understanding evolving robot systems is emphasised, stating that in order to have the technology work we also need the science behind it.

Environmental influences on evolvable robots.

The field of Evolutionary Robotics addresses the challenge of automatically designing robotic systems. Furthermore, the field can also support biological investigations related to evolution. In this paper, we evolve (simulated) modular robots under diverse environmental conditions and analyze the influences that these conditions have on the evolved morphologies, controllers, and behavior. To this end, we introduce a set of morphological, controller, and behavioral descriptors that together span a multi-dimensional trait space. Using these descriptors, we demonstrate how changes in environmental conditions induce different levels of differentiation in this trait space. Our main goal is to gain deeper insights into the effect of the environment on a robotic evolutionary process.

Environmental Regulation Using Plasticoding for the Evolution of Robots.

Evolutionary robot systems are usually affected by the properties of the environment indirectly through selection. In this paper, we present and investigate a system where the environment also has a direct effect-through regulation. We propose a novel robot encoding method where a genotype encodes multiple possible phenotypes, and the incarnation of a robot depends on the environmental conditions taking place in a determined moment of its life. This means that the morphology, controller, and behavior of a robot can change according to the environment. Importantly, this process of development can happen at any moment of a robot's lifetime, according to its experienced environmental stimuli. We provide an empirical proof-of-concept, and the analysis of the experimental results shows that environmental regulation improves adaptation (task performance) while leading to different evolved morphologies, controllers, and behavior.

Importance of Parameter Settings on the Benefits of Robot-to-Robot Learning in Evolutionary Robotics.

Robot-to-robot learning, a specific case of social learning in robotics, enables multiple robots to share learned skills while completing a task. The literature offers various statements of its benefits. Robots using this type of social learning can reach a higher performance, an increased learning speed, or both, compared to robots using individual learning only. No general explanation has been advanced for the difference in observations, which make the results highly dependent on the particular system and parameter setting. In this paper, we perform a detailed analysis into the effects of robot-to-robot learning. As a result, we show that this type of social learning can reduce the sensitivity of the learning process to the choice of parameters in two ways. First, robot-to-robot learning can reduce the number of bad performing individuals in the population. Second, robot-to-robot learning can increase the chance of having a successful run, where success is defined as the presence of a high performing individual. Additionally, we show that robot-to-robot learning results in an increased learning speed for almost all parameter settings. Our results indicate that robot-to-robot learning is a powerful mechanism which leads to benefits in both performance and learning speed.

Lamarckian Evolution of Simulated Modular Robots.

We study evolutionary robot systems where not only the robot brains but also the robot bodies are evolvable. Such systems need to include a learning period right after 'birth' to acquire a controller that fits the newly created body. In this paper we investigate the possibility of bootstrapping infant robot learning through employing Lamarckian inheritance of parental controllers. In our system controllers are encoded by a combination of a morphology dependent component, a Central Pattern Generator (CPG), and a morphology independent part, a Compositional Pattern Producing Network (CPPN). This makes it possible to transfer the CPPN part of controllers between different morphologies and to create a Lamarckian system. We conduct experiments with simulated modular robots whose fitness is determined by the speed of locomotion, establish the benefits of inheriting optimized parental controllers, shed light on the conditions that influence these benefits, and observe that changing the way controllers are evolved also impacts the evolved morphologies.

Assessment of temporal predictive models for health care using a formal method.

Recent developments in the field of sensor devices provide new possibilities to measure a variety of health related aspects in a precise and fine-grained manner. Subsequently, more empirical data will be generated than ever before. While this greatly improves the opportunities for creating accurate predictive models, other types of models besides the more traditional machine learning approaches can provide insights into temporal relationships in the data. Models that express temporal relationships between states in a mathematical manner are examples of such models. However, the evaluation methods traditionally used in the field of predictive modeling are not appropriate for those models, making it difficult to distinguish them in terms of validity. Appropriate assessment methodology is therefore necessary to drive the research of mathematical modeling forward. In this paper we investigate the applicability of such a formalized method. The method takes into account important model aspects, namely descriptive and predictive capability, parameter sensitivity and model complexity. As a case study the method is applied to a mathematical model in the domain of mental health, showing that the method generates useful insights into the behavior of the model.

Real-World Evolution of Robot Morphologies: A Proof of Concept.

Evolutionary robotics using real hardware has been almost exclusively restricted to evolving robot controllers, but the technology for evolvable morphologies is advancing quickly. We discuss a proof-of-concept study to demonstrate real robots that can reproduce. Following a general system plan, we implement a robotic habitat that contains all system components in the simplest possible form. We create an initial population of two robots and run a complete life cycle, resulting in a new robot, parented by the first two. Even though the individual steps are simplified to the maximum, the whole system validates the underlying concepts and provides a generic workflow for the creation of more complex incarnations. This hands-on experience provides insights and helps us elaborate on interesting research directions for future development.

Online Gait Learning for Modular Robots with Arbitrary Shapes and Sizes.

Evolutionary robotics using real hardware is currently restricted to evolving robot controllers, but the technology for evolvable morphologies is advancing quickly. Rapid prototyping (3D printing) and automated assembly are the main enablers of robotic systems where robot offspring can be produced based on a blueprint that specifies the morphologies and the controllers of the parents. This article addresses the problem of gait learning in newborn robots whose morphology is unknown in advance. We investigate a reinforcement learning method and conduct simulation experiments using robot morphologies with different size and complexity. We establish that reinforcement learning does the job well and that it outperforms two alternative algorithms. The experiments also give insights into the online dynamics of gait learning and into the influence of the size, shape, and morphological complexity of the modular robots. These insights can potentially be used to predict the viability of modular robotic organisms before they are constructed.

Combining environment-driven adaptation and task-driven optimisation in evolutionary robotics.

Embodied evolutionary robotics is a sub-field of evolutionary robotics that employs evolutionary algorithms on the robotic hardware itself, during the operational period, i.e., in an on-line fashion. This enables robotic systems that continuously adapt, and are therefore capable of (re-)adjusting themselves to previously unknown or dynamically changing conditions autonomously, without human oversight. This paper addresses one of the major challenges that such systems face, viz. that the robots must satisfy two sets of requirements. Firstly, they must continue to operate reliably in their environment (viability), and secondly they must competently perform user-specified tasks (usefulness). The solution we propose exploits the fact that evolutionary methods have two basic selection mechanisms-survivor selection and parent selection. This allows evolution to tackle the two sets of requirements separately: survivor selection is driven by the environment and parent selection is based on task-performance. This idea is elaborated in the Multi-Objective aNd open-Ended Evolution (monee) framework, which we experimentally validate. Experiments with robotic swarms of 100 simulated e-pucks show that monee does indeed promote task-driven behaviour without compromising environmental adaptation. We also investigate an extension of the parent selection process with a 'market mechanism' that can ensure equitable distribution of effort over multiple tasks, a particularly pressing issue if the environment promotes specialisation in single tasks.

Embodied artificial evolution: Artificial evolutionary systems in the 21st Century.

Evolution is one of the major omnipresent powers in the universe that has been studied for about two centuries. Recent scientific and technical developments make it possible to make the transition from passively understanding to actively using evolutionary processes. Today this is possible in Evolutionary Computing, where human experimenters can design and manipulate all components of evolutionary processes in digital spaces. We argue that in the near future it will be possible to implement artificial evolutionary processes outside such imaginary spaces and make them physically embodied. In other words, we envision the "Evolution of Things", rather than just the evolution of digital objects, leading to a new field of Embodied Artificial Evolution (EAE). The main objective of this paper is to present a unifying vision in order to aid the development of this high potential research area. To this end, we introduce the notion of EAE, discuss a few examples and applications, and elaborate on the expected benefits as well as the grand challenges this developing field will have to address.

Empirical investigation of multiparent recombination operators in evolution strategies.

An extension of evolution strategies to multiparent recombination involving a variable number [symbol: see text] of parents to create an offspring individual is proposed. The extension is experimentally evaluated on a test suite of functions differing in their modality and separability and the regular/irregular arrangement of their local optima. Multiparent diagonal crossover and uniform scanning crossover and a multiparent version of intermediary recombination are considered in the experiments. The performance of the algorithm is observed to depend on the particular combination of recombination operator and objective function. In most of the cases a significant increase in performance is observed as the number of parents increases. However, there might also be no significant impact of recombination at all, and for one of the unimodal objective functions, the performance is observed to deteriorate over the course of evolution for certain choices of the recombination operator and the number of parents. Additional experiments with a skewed initialization of the population clarify that intermediary recombination does not cause a search bias toward the origin of the coordinate system in the case of domains of variables that are symmetric around zero.