neuroWalknet, a controller for hexapod walking allowing for context dependent behavior.

Decentralized control has been established as a key control principle in insect walking and has been successfully leveraged to account for a wide range of walking behaviors in the proposed neuroWalknet architecture. This controller allows for walking patterns at different velocities in both, forward and backward direction-quite similar to the behavior shown in stick insects-, for negotiation of curves, and for robustly dealing with various disturbances. While these simulations focus on the cooperation of different, decentrally controlled legs, here we consider a set of biological experiments not yet been tested by neuroWalknet, that focus on the function of the individual leg and are context dependent. These intraleg studies deal with four groups of interjoint reflexes. The reflexes are elicited by stimulation of the femoral chordotonal organ (fCO) or groups of campaniform sensilla (CS). Motor output signals are recorded from the alpha-joint, the beta-joint or the gamma-joint of the leg. Furthermore, the influence of these sensory inputs to artificially induced oscillations by application of pilocarpine has been studied. Although these biological data represent results obtained from different local reflexes in different contexts, they fit with and are embedded into the behavior shown by the global structure of neuroWalknet. In particular, a specific and intensively studied behavior, active reaction, has since long been assumed to represent a separate behavioral element, from which it is not clear why it occurs in some situations, but not in others. This question could now be explained as an emergent property of the holistic structure of neuroWalknet which has shown to be able to produce artificially elicited pilocarpine-driven oscillation that can be controlled by sensory input without the need of explicit innate CPG structures. As the simulation data result from a holistic system, further results were obtained that could be used as predictions to be tested in further biological experiments.

On the 'cognitive map debate' in insect navigation.

In a historical account recently published in this journal Dhein argues that the current debate whether insects like bees and ants use cognitive maps (centralized map hypothesis) or other means of navigation (decentralized network hypothesis) largely reflects the classical debate between American experimental psychologists à la Tolman and German ethologists à la Lorenz, respectively. In this dichotomy we, i.e., the proponents of the network hypothesis, are inappropriately placed on the Lorenzian line. In particular, we argue that in contrast to Dhein's claim our concepts are not based on merely instinctive or peripheral modes of information processing. In general, on the one side our approaches have largely been motivated by the early biocybernetics way of thinking. On the other side they are deeply rooted in studies on the insect's behavioral ecology, i.e., in the ecological setting within which the navigational strategies have evolved and within which the animal now operates. Following such a bottom-up approach we are not "anti-cognitive map researchers" but argue that the results we have obtained in ants, and also the results of some decisive experiments in bees, can be explained and simulated without the need of invoking metric maps.

Decentralized control of insect walking: A simple neural network explains a wide range of behavioral and neurophysiological results.

Controlling the six legs of an insect walking in an unpredictable environment is a challenging task, as many degrees of freedom have to be coordinated. Solutions proposed to deal with this task are usually based on the highly influential concept that (sensory-modulated) central pattern generators (CPG) are required to control the rhythmic movements of walking legs. Here, we investigate a different view. To this end, we introduce a sensor based controller operating on artificial neurons, being applied to a (simulated) insectoid robot required to exploit the "loop through the world" allowing for simplification of neural computation. We show that such a decentralized solution leads to adaptive behavior when facing uncertain environments which we demonstrate for a broad range of behaviors never dealt with in a single system by earlier approaches. This includes the ability to produce footfall patterns such as velocity dependent "tripod", "tetrapod", "pentapod" as well as various stable intermediate patterns as observed in stick insects and in Drosophila. These patterns are found to be stable against disturbances and when starting from various leg configurations. Our neuronal architecture easily allows for starting or interrupting a walk, all being difficult for CPG controlled solutions. Furthermore, negotiation of curves and walking on a treadmill with various treatments of individual legs is possible as well as backward walking and performing short steps. This approach can as well account for the neurophysiological results usually interpreted to support the idea that CPGs form the basis of walking, although our approach is not relying on explicit CPG-like structures. Application of CPGs may however be required for very fast walking. Our neuronal structure allows to pinpoint specific neurons known from various insect studies. Interestingly, specific common properties observed in both insects and crustaceans suggest a significance of our controller beyond the realm of insects.

Steering intermediate courses: desert ants combine information from various navigational routines.

A number of systems of navigation have been studied in some detail in insects. These include path integration, a system that keeps track of the straight-line distance and direction travelled on the current trip, the use of panoramic landmarks and scenery for orientation, and systematic searching. A traditional view is that only one navigational system is in operation at any one time, with different systems running in sequence depending on the context and conditions. We review selected data suggesting that often, different navigational cues (e.g., compass cues) and different systems of navigation are in operation simultaneously in desert ant navigation. The evidence suggests that all systems operate in parallel forming a heterarchical network. External and internal conditions determine the weights to be accorded to each cue and system. We also show that a model of independent modules feeding into a central summating device, the Navinet model, can in principle account for such data. No central executive processor is necessary aside from a weighted summation of the different cues and systems. Such a heterarchy of parallel systems all in operation represents a new view of insect navigation that has already been expressed informally by some authors.

Walknet, a bio-inspired controller for hexapod walking.

Walknet comprises an artificial neural network that allows for the simulation of a considerable amount of behavioral data obtained from walking and standing stick insects. It has been tested by kinematic and dynamic simulations as well as on a number of six-legged robots. Over the years, various different expansions of this network have been provided leading to different versions of Walknet. This review summarizes the most important biological findings described by Walknet and how they can be simulated. Walknet shows how a number of properties observed in insects may emerge from a decentralized architecture. Examples are the continuum of so-called "gaits," coordination of up to 18 leg joints during stance when walking forward or backward over uneven surfaces and negotiation of curves, dealing with leg loss, as well as being able following motion trajectories without explicit precalculation. The different Walknet versions are compared to other approaches describing insect-inspired hexapod walking. Finally, we briefly address the ability of this decentralized reactive controller to form the basis for the simulation of higher-level cognitive faculties exceeding the capabilities of insects.

No need for a cognitive map: decentralized memory for insect navigation.

In many animals the ability to navigate over long distances is an important prerequisite for foraging. For example, it is widely accepted that desert ants and honey bees, but also mammals, use path integration for finding the way back to their home site. It is however a matter of a long standing debate whether animals in addition are able to acquire and use so called cognitive maps. Such a 'map', a global spatial representation of the foraging area, is generally assumed to allow the animal to find shortcuts between two sites although the direct connection has never been travelled before. Using the artificial neural network approach, here we develop an artificial memory system which is based on path integration and various landmark guidance mechanisms (a bank of individual and independent landmark-defined memory elements). Activation of the individual memory elements depends on a separate motivation network and an, in part, asymmetrical lateral inhibition network. The information concerning the absolute position of the agent is present, but resides in a separate memory that can only be used by the path integration subsystem to control the behaviour, but cannot be used for computational purposes with other memory elements of the system. Thus, in this simulation there is no neural basis of a cognitive map. Nevertheless, an agent controlled by this network is able to accomplish various navigational tasks known from ants and bees and often discussed as being dependent on a cognitive map. For example, map-like behaviour as observed in honey bees arises as an emergent property from a decentralized system. This behaviour thus can be explained without referring to the assumption that a cognitive map, a coherent representation of foraging space, must exist. We hypothesize that the proposed network essentially resides in the mushroom bodies of the insect brain.

Integrative Biomimetics of Autonomous Hexapedal Locomotion.

Despite substantial advances in many different fields of neurorobotics in general, and biomimetic robots in particular, a key challenge is the integration of concepts: to collate and combine research on disparate and conceptually disjunct research areas in the neurosciences and engineering sciences. We claim that the development of suitable robotic integration platforms is of particular relevance to make such integration of concepts work in practice. Here, we provide an example for a hexapod robotic integration platform for autonomous locomotion. In a sequence of six focus sections dealing with aspects of intelligent, embodied motor control in insects and multipedal robots-ranging from compliant actuation, distributed proprioception and control of multiple legs, the formation of internal representations to the use of an internal body model-we introduce the walking robot HECTOR as a research platform for integrative biomimetics of hexapedal locomotion. Owing to its 18 highly sensorized, compliant actuators, light-weight exoskeleton, distributed and expandable hardware architecture, and an appropriate dynamic simulation framework, HECTOR offers many opportunities to integrate research effort across biomimetics research on actuation, sensory-motor feedback, inter-leg coordination, and cognitive abilities such as motion planning and learning of its own body size.

Walking in Aretaon asperrimus.

This article describes basic parameters characterizing walking of the stick insect Aretaon asperrimus to allow a comparative approach with other insects studied. As in many other animals, geometrical parameters such as step amplitude and leg extreme positions do not vary with walking velocity. However, the relation between swing duration and stance duration is quite constant, in contrast to most insects studied. Therefore, velocity profiles during swing vary with walking velocity whereas time course of leg trajectories and leg angle trajectories are independent of walking velocity. Nevertheless, A. asperrimus does not show a classical tripod gait, but performs a metachronal, or tetrapod, gait, showing phase values differing from 0.5 between ipsilateral neighbouring legs. As in Carausius morosus, the detailed shape of the swing trajectory may depend on the form of the substrate. Effects describing coordinating influences between legs have been found that prevent the start of a swing as long as the posterior leg performs a swing. Further, the treading on tarsus reflex can be observed in Aretaon. No hint to the existence of a targeting influence has been found. Control of rearward walking is easiest interpreted by maintaining the basic rules but an anterior-posterior reversal of the information flow.

ReaCog, a Minimal Cognitive Controller Based on Recruitment of Reactive Systems.

It has often been stated that for a neuronal system to become a cognitive one, it has to be large enough. In contrast, we argue that a basic property of a cognitive system, namely the ability to plan ahead, can already be fulfilled by small neuronal systems. As a proof of concept, we propose an artificial neural network, termed reaCog, that, first, is able to deal with a specific domain of behavior (six-legged-walking). Second, we show how a minor expansion of this system enables the system to plan ahead and deploy existing behavioral elements in novel contexts in order to solve current problems. To this end, the system invents new solutions that are not possible for the reactive network. Rather these solutions result from new combinations of given memory elements. This faculty does not rely on a dedicated system being more or less independent of the reactive basis, but results from exploitation of the reactive basis by recruiting the lower-level control structures in a way that motor planning becomes possible as an internal simulation relying on internal representation being grounded in embodied experiences.

Leg coordination during turning on an extremely narrow substrate in a bug, Mesocerus marginatus (Heteroptera, Coreidae).

The turning movement of a bug, Mesocerus marginatus, is observed when it walks upside-down below a horizontal beam and, at the end of the beam, performs a sharp turn by 180 degrees . The turn at the end of the beam is accomplished in three to five steps, without strong temporal coordination among legs. During the stance, leg endpoints (tarsi) run through rounded trajectories, rotating to the same side in all legs. During certain phases of the turn, a leg is strongly depressed and the tarsus crosses the midline. Swing movements rotate to the same side as do leg endpoints in stance, in strong contrast to the typical swing movements found in turns or straight walk on a flat surface. Terminal location is found after the search through a trajectory that first moves away from the body and then loops back to find substrate. When a leg during stance has crossed the midline, in the following swing movement the leg may move even stronger on the contralateral side, i.e. is stronger depressed, in contrast to swing movements in normal walking, where the leg is elevated. These results suggest that the animals apply a different control strategy compared to walking and turning on a flat surface.

A biologically inspired active compliant joint using local positive velocity feedback (LPVF).

Starting from studies which revealed that positive feedback is found in the control system for walking in arthropods, we have constructed a new positive feedback driven joint that can be used for solving compliant motion tasks. We propose two different joint constructions each of which shows passive compliance. Based on these joints we introduce three different local positive velocity feedback (LPVF) controllers and discuss their properties in the context of motion generation in closed kinematic chains. The third circuit named undelayed dLPVF is used for the control of a compliant planar manipulator which turns a crank. Our concept is of highly decentralized nature and follows the idea of embodiment. In our case this means that a process which is controlled by LPVF controllers reveals its nature when the controllers interact with this process.

How and to what end may consciousness contribute to action? Attributing properties of consciousness to an embodied, minimally cognitive artificial neural network.

An artificial neural network called reaCog is described which is based on a decentralized, reactive and embodied architecture developed to control non-trivial hexapod walking in an unpredictable environment (Walknet) while using insect-like navigation (Navinet). In reaCog, these basic networks are extended in such a way that the complete system, reaCog, adopts the capability of inventing new behaviors and - via internal simulation - of planning ahead. This cognitive expansion enables the reactive system to be enriched with additional procedures. Here, we focus on the question to what extent properties of phenomena to be characterized on a different level of description as for example consciousness can be found in this minimally cognitive system. Adopting a monist view, we argue that the phenomenal aspect of mental phenomena can be neglected when discussing the function of such a system. Under this condition, reaCog is discussed to be equipped with properties as are bottom-up and top-down attention, intentions, volition, and some aspects of Access Consciousness. These properties have not been explicitly implemented but emerge from the cooperation between the elements of the network. The aspects of Access Consciousness found in reaCog concern the above mentioned ability to plan ahead and to invent and guide (new) actions. Furthermore, global accessibility of memory elements, another aspect characterizing Access Consciousness is realized by this network. reaCog allows for both reactive/automatic control and (access-) conscious control of behavior. We discuss examples for interactions between both the reactive domain and the conscious domain. Metacognition or Reflexive Consciousness is not a property of reaCog. Possible expansions are discussed to allow for further properties of Access Consciousness, verbal report on internal states, and for Metacognition. In summary, we argue that already simple networks allow for properties of consciousness if leaving the phenomenal aspect aside.

A hexapod walker using a heterarchical architecture for action selection.

Moving in a cluttered environment with a six-legged walking machine that has additional body actuators, therefore controlling 22 DoFs, is not a trivial task. Already simple forward walking on a flat plane requires the system to select between different internal states. The orchestration of these states depends on walking velocity and on external disturbances. Such disturbances occur continuously, for example due to irregular up-and-down movements of the body or slipping of the legs, even on flat surfaces, in particular when negotiating tight curves. The number of possible states is further increased when the system is allowed to walk backward or when front legs are used as grippers and cannot contribute to walking. Further states are necessary for expansion that allow for navigation. Here we demonstrate a solution for the selection and sequencing of different (attractor) states required to control different behaviors as are forward walking at different speeds, backward walking, as well as negotiation of tight curves. This selection is made by a recurrent neural network (RNN) of motivation units, controlling a bank of decentralized memory elements in combination with the feedback through the environment. The underlying heterarchical architecture of the network allows to select various combinations of these elements. This modular approach representing an example of neural reuse of a limited number of procedures allows for adaptation to different internal and external conditions. A way is sketched as to how this approach may be expanded to form a cognitive system being able to plan ahead. This architecture is characterized by different types of modules being arranged in layers and columns, but the complete network can also be considered as a holistic system showing emergent properties which cannot be attributed to a specific module.

What's Next: Recruitment of a Grounded Predictive Body Model for Planning a Robot's Actions.

Even comparatively simple, reactive systems are able to control complex motor tasks, such as hexapod walking on unpredictable substrate. The capability of such a controller can be improved by introducing internal models of the body and of parts of the environment. Such internal models can be applied as inverse models, as forward models or to solve the problem of sensor fusion. Usually, separate models are used for these functions. Furthermore, separate models are used to solve different tasks. Here we concentrate on internal models of the body as the brain considers its own body the most important part of the world. The model proposed is formed by a recurrent neural network with the property of pattern completion. The model shows a hierarchical structure but nonetheless comprises a holistic system. One and the same model can be used as a forward model, as an inverse model, for sensor fusion, and, with a simple expansion, as a model to internally simulate (new) behaviors to be used for prediction. The model embraces the geometrical constraints of a complex body with many redundant degrees of freedom, and allows finding geometrically possible solutions. To control behavior such as walking, climbing, or reaching, this body model is complemented by a number of simple reactive procedures together forming a procedural memory. In this article, we illustrate the functioning of this network. To this end we present examples for solutions of the forward function and the inverse function, and explain how the complete network might be used for predictive purposes. The model is assumed to be "innate," so learning the parameters of the model is not (yet) considered.

Selforganizing memory: active learning of landmarks used for navigation.

We propose a memory architecture that is suited to solve a specific task, namely homing, that is finding a not directly visible home place by using visually accessible landmarks. We show that an agent equipped with such a memory structure can autonomously learn the situation and can later use its memory to accomplish homing behaviour. The architecture is based on neuronal structures and grows in a self-organized way depending on experience. The basic architecture consists of three parts, (i) a pre-processor, (ii) a simple, one-layered feed-forward network, called distributor net, and (iii) a full recurrently connected net for representing the situation models to be stored. Apart from Hebbian learning and a local version of the delta-rule, explorative learning is applied that is not based on passive detection of correlations, but is actively searching for interesting hypotheses. Hypotheses are spontaneously introduced and are verified or falsified depending on how well the network representing the hypothesis approaches an internal error of zero. The stability of this approach is successfully tested by removal of one landmark or shifting the position of one or several landmarks showing results comparable to those found in biological experiments. Furthermore, we applied noise in two ways. The trained network was either due to sensory noise or to noise applied to the bias weights describing the memory content. Finally, we tested to what extent learning of the weights is affected by noisy input given to the sensor data. The architecture proposed is discussed to have some at least superficial similarity to the mushroom bodies of insects.

Controlling a system with redundant degrees of freedom: II. Solution of the force distribution problem without a body model.

What strategies may insects use when controlling redundant degrees of freedom? We investigate this question in standing stick insects. Specifically, the question is addressed how the changes of the torques are coordinated that are produced by the 18 leg joints in a still standing animal. Using a generalization of the principal component analysis, three coordination rules have been identified. These rules are sufficient to describe more than half of the variation observed in the data. To move from a descriptive approach to hypotheses on how the neuronal system may be structured, two simulation approaches are proposed. In both cases, torques are decreased by randomly selected values. In the first simulation, the coordination rules derived from the principal components are used to produce changes in torques. In the second simulation, the individual joint torques are modified using a simple local approach. In both approaches, the resulting torques are re-adjusted by Integral controllers applied in each joint. The results show that the torque distribution problem can be solved by a local approach without requiring a body model.

Controlling a system with redundant degrees of freedom. I. Torque distribution in still standing stick insects.

The question is investigated as to how a stick insect solves the task of distributing its body weight onto its six legs, i.e., how are the torques coordinated that are produced by the 18 joints (3 per leg). Three-dimensional force measurements of ground reaction forces have been used to calculate the torques developed by each of the 18 joints. Torques were found to change considerably although the body and the legs of the animal did not move. This result implies a tight cooperation between the 18 joint controllers. Indeed, in each individual experiment, strong correlations could be observed between specific pairs of joints. However, in spite of thorough analysis, no general correlation rules between torques could be detected. The only common attribute found for all experiments was that high absolute torques observed at the beginning of the experiment tend to converge to some minimum over time. Thus, the insects tend to decrease the torques while standing still, but do not use fixed rules. Rather they appear to exploit their extra degrees of freedom and produce time courses that can strongly vary between experiments. Possible mechanisms underlying this behaviour are discussed in a companion paper [Lévy and Cruse (2008) Controlling a system with redundant degrees of freedom: ii. solution of the force distribution problem without a body model, submitted].

Winching up heavy loads with a compliant arm: a new local joint controller.

A closed kinematic chain, like an arm that operates a crank, has a constrained movement space. A meaningful movement of the chain's endpoint is only possible along the free movement directions which are given implicitly by the contour of the object that confines the movement of the chain. Many technical solutions for such a movement task, in particular those used in robotics, use central controllers and force-torque sensors in the arm's wrist or a leg's ankle to construct a coordinate system (task frame formalism) at the local point of contact the axes of which coincide with the free and constrained movement directions. Motivated by examples from biology, we introduce a new control system that solves a constrained movement task. The control system is inspired by the control architecture that is found in stick insects like Carausius morosus. It consists of decentral joint controllers that work on elastic joints (compliant manipulator). The decentral controllers are based on local positive velocity feedback (LPVF). It has been shown earlier that LPVF enables contour following of a limb in a compliant motion task without a central controller. In this paper we extend LPVF in such a way that it is even able to follow a contour if a considerable counter force drags the limb away along the contour in a direction opposite to the desired. The controller extension is based on the measurement of the local mechanical power generated in the elastic joint and is called power-controlled relaxation LPVF. The new control approach has the following advantages. First, it still uses local joint controllers without knowledge of the kinematics. Second, it does not need a force or torque measurement at the end of the limb. In this paper we test power-controlled relaxation LPVF on a crank turning task in which a weight has to be winched up by a two-joint compliant manipulator.

Sensorimotor supremacy: Investigating conscious and unconscious vision by masked priming.

According to the sensorimotor supremacy hypothesis, conscious perception draws on motor action. In the present report, we will sketch two lines of potential development in the field of masking research based on the sensorimotor supremacy hypothesis. In the first part of the report, evidence is reviewed that masked, invisible stimuli can affect motor responses, attention shifts, and semantic processes. After the review of the corresponding evidence - so-called masked priming effects - an approach based on the sensorimotor supremacy hypothesis is detailed as to how the question of a unitary mechanism of unconscious vision can be pursued by masked priming studies. In the second part of the report, different models and theories of backward masking and masked priming are reviewed. Types of models based on the sensorimotor hypothesis are discussed that can take into account ways in which sensorimotor processes (reflected in masked priming effects) can affect conscious vision under backward masking conditions.