A multizone cerebellar chip for bioinspired adaptive robot control and sensorimotor processing.

The cerebellum is a neural structure essential for learning, which is connected via multiple zones to many different regions of the brain, and is thought to improve human performance in a large range of sensory, motor and even cognitive processing tasks. An intriguing possibility for the control of complex robotic systems would be to develop an artificial cerebellar chip with multiple zones that could be similarly connected to a variety of subsystems to optimize performance. The novel aim of this paper, therefore, is to propose and investigate a multizone cerebellar chip applied to a range of tasks in robot adaptive control and sensorimotor processing. The multizone cerebellar chip was evaluated using a custom robotic platform consisting of an array of tactile sensors driven by dielectric electroactive polymers mounted upon a standard industrial robot arm. The results demonstrate that the performance in each task was improved by the concurrent, stable learning in each cerebellar zone. This paper, therefore, provides the first empirical demonstration that a synthetic, multizone, cerebellar chip could be embodied within existing robotic systems to improve performance in a diverse range of tasks, much like the cerebellum in a biological system.

World Statistics Drive Learning of Cerebellar Internal Models in Adaptive Feedback Control: A Case Study Using the Optokinetic Reflex.

The cerebellum is widely implicated in having an important role in adaptive motor control. Many of the computational studies on cerebellar motor control to date have focused on the associated architecture and learning algorithms in an effort to further understand cerebellar function. In this paper we switch focus to the signals driving cerebellar adaptation that arise through different motor behavior. To do this, we investigate computationally the contribution of the cerebellum to the optokinetic reflex (OKR), a visual feedback control scheme for image stabilization. We develop a computational model of the adaptation of the cerebellar response to the world velocity signals that excite the OKR (where world velocity signals are used to emulate head velocity signals when studying the OKR in head-fixed experimental laboratory conditions). The results show that the filter learnt by the cerebellar model is highly dependent on the power spectrum of the colored noise world velocity excitation signal. Thus, the key finding here is that the cerebellar filter is determined by the statistics of the OKR excitation signal.

Sensorimotor maps can be dynamically calibrated using an adaptive-filter model of the cerebellum.

Substantial experimental evidence suggests the cerebellum is involved in calibrating sensorimotor maps. Consistent with this involvement is the well-known, but little understood, massive cerebellar projection to maps in the superior colliculus. Map calibration would be a significant new role for the cerebellum given the ubiquity of map representations in the brain, but how it could perform such a task is unclear. Here we investigated a dynamic method for map calibration, based on electrophysiological recordings from the superior colliculus, that used a standard adaptive-filter cerebellar model. The method proved effective for complex distortions of both unimodal and bimodal maps, and also for predictive map-based tracking of moving targets. These results provide the first computational evidence for a novel role for the cerebellum in dynamic sensorimotor map calibration, of potential importance for coordinate alignment during ongoing motor control, and for map calibration in future biomimetic systems. This computational evidence also provides testable experimental predictions concerning the role of the connections between cerebellum and superior colliculus in previously observed dynamic coordinate transformations.

Cerebellar-inspired algorithm for adaptive control of nonlinear dielectric elastomer-based artificial muscle.

Electroactive polymer actuators are important for soft robotics, but can be difficult to control because of compliance, creep and nonlinearities. Because biological control mechanisms have evolved to deal with such problems, we investigated whether a control scheme based on the cerebellum would be useful for controlling a nonlinear dielectric elastomer actuator, a class of artificial muscle. The cerebellum was represented by the adaptive filter model, and acted in parallel with a brainstem, an approximate inverse plant model. The recurrent connections between the two allowed for direct use of sensory error to adjust motor commands. Accurate tracking of a displacement command in the actuator's nonlinear range was achieved by either semi-linear basis functions in the cerebellar model or semi-linear functions in the brainstem corresponding to recruitment in biological muscle. In addition, allowing transfer of training between cerebellum and brainstem as has been observed in the vestibulo-ocular reflex prevented the steady increase in cerebellar output otherwise required to deal with creep. The extensibility and relative simplicity of the cerebellar-based adaptive-inverse control scheme suggests that it is a plausible candidate for controlling this type of actuator. Moreover, its performance highlights important features of biological control, particularly nonlinear basis functions, recruitment and transfer of training.

At the Edge of Chaos: How Cerebellar Granular Layer Network Dynamics Can Provide the Basis for Temporal Filters.

Models of the cerebellar microcircuit often assume that input signals from the mossy-fibers are expanded and recoded to provide a foundation from which the Purkinje cells can synthesize output filters to implement specific input-signal transformations. Details of this process are however unclear. While previous work has shown that recurrent granule cell inhibition could in principle generate a wide variety of random outputs suitable for coding signal onsets, the more general application for temporally varying signals has yet to be demonstrated. Here we show for the first time that using a mechanism very similar to reservoir computing enables random neuronal networks in the granule cell layer to provide the necessary signal separation and extension from which Purkinje cells could construct basis filters of various time-constants. The main requirement for this is that the network operates in a state of criticality close to the edge of random chaotic behavior. We further show that the lack of recurrent excitation in the granular layer as commonly required in traditional reservoir networks can be circumvented by considering other inherent granular layer features such as inverted input signals or mGluR2 inhibition of Golgi cells. Other properties that facilitate filter construction are direct mossy fiber excitation of Golgi cells, variability of synaptic weights or input signals and output-feedback via the nucleocortical pathway. Our findings are well supported by previous experimental and theoretical work and will help to bridge the gap between system-level models and detailed models of the granular layer network.

Biohybrid Control of General Linear Systems Using the Adaptive Filter Model of Cerebellum.

The adaptive filter model of the cerebellar microcircuit has been successfully applied to biological motor control problems, such as the vestibulo-ocular reflex (VOR), and to sensory processing problems, such as the adaptive cancelation of reafferent noise. It has also been successfully applied to problems in robotics, such as adaptive camera stabilization and sensor noise cancelation. In previous applications to inverse control problems, the algorithm was applied to the velocity control of a plant dominated by viscous and elastic elements. Naive application of the adaptive filter model to the displacement (as opposed to velocity) control of this plant results in unstable learning and control. To be more generally useful in engineering problems, it is essential to remove this restriction to enable the stable control of plants of any order. We address this problem here by developing a biohybrid model reference adaptive control (MRAC) scheme, which stabilizes the control algorithm for strictly proper plants. We evaluate the performance of this novel cerebellar-inspired algorithm with MRAC scheme in the experimental control of a dielectric electroactive polymer, a class of artificial muscle. The results show that the augmented cerebellar algorithm is able to accurately control the displacement response of the artificial muscle. The proposed solution not only greatly extends the practical applicability of the cerebellar-inspired algorithm, but may also shed light on cerebellar involvement in a wider range of biological control tasks.

Model cerebellar granule cells can faithfully transmit modulated firing rate signals.

A crucial assumption of many high-level system models of the cerebellum is that information in the granular layer is encoded in a linear manner. However, granule cells are known for their non-linear and resonant synaptic and intrinsic properties that could potentially impede linear signal transmission. In this modeling study we analyse how electrophysiological granule cell properties and spike sampling influence information coded by firing rate modulation, assuming no signal-related, i.e., uncorrelated inhibitory feedback (open-loop mode). A detailed one-compartment granule cell model was excited in simulation by either direct current or mossy-fiber synaptic inputs. Vestibular signals were represented as tonic inputs to the flocculus modulated at frequencies up to 20 Hz (approximate upper frequency limit of vestibular-ocular reflex, VOR). Model outputs were assessed using estimates of both the transfer function, and the fidelity of input-signal reconstruction measured as variance-accounted-for. The detailed granule cell model with realistic mossy-fiber synaptic inputs could transmit information faithfully and linearly in the frequency range of the vestibular-ocular reflex. This was achieved most simply if the model neurons had a firing rate at least twice the highest required frequency of modulation, but lower rates were also adequate provided a population of neurons was utilized, especially in combination with push-pull coding. The exact number of neurons required for faithful transmission depended on the precise values of firing rate and noise. The model neurons were also able to combine excitatory and inhibitory signals linearly, and could be replaced by a simpler (modified) integrate-and-fire neuron in the case of high tonic firing rates. These findings suggest that granule cells can in principle code modulated firing-rate inputs in a linear manner, and are thus consistent with the high-level adaptive-filter model of the cerebellar microcircuit.

Decorrelation learning in the cerebellum: computational analysis and experimental questions.

Many cerebellar models use a form of synaptic plasticity that implements decorrelation learning. Parallel fibers carrying signals positively correlated with climbing-fiber input have their synapses weakened (long-term depression), whereas those carrying signals negatively correlated with climbing input have their synapses strengthened (long-term potentiation). Learning therefore ceases when all parallel-fiber signals have been decorrelated from climbing-fiber input. This is a computationally powerful rule for supervised learning and can be cast in a spike-timing dependent plasticity form for comparison with experimental evidence. Decorrelation learning is particularly well suited to sensory prediction, for example, in the reafference problem where external sensory signals are interfered with by reafferent signals from the organism's own movements, and the required circuit appears similar to the one found to mediate classical eye blink conditioning. However, for certain stimuli, avoidance is a much better option than simple prediction, and decorrelation learning can also be used to acquire appropriate avoidance movements. One example of a stimulus to be avoided is retinal slip that degrades visual processing, and decorrelation learning appears to play a role in the vestibulo-ocular reflex that stabilizes gaze in the face of unpredicted head movements. Decorrelation learning is thus suitable for both sensory prediction and motor control. It may also be well suited for generic spatial and temporal coordination, because of its ability to remove the unwanted side effects of movement. Finally, because it can be used with any kind of time-varying signal, the cerebellum could play a role in cognitive processing.

An adaptive filter model of cerebellar zone C3 as a basis for safe limb control?

The review asks how the adaptive filter model of the cerebellum might be relevant to experimental work on zone C3, one of the most extensively studied regions of cerebellar cortex. As far as features of the cerebellar microcircuit are concerned, the model appears to fit very well with electrophysiological discoveries concerning the importance of molecular layer interneurons and their plasticity, the significance of long-term potentiation and the striking number of silent parallel fibre synapses. Regarding external connectivity and functionality, a key feature of the adaptive filter model is its use of the decorrelation algorithm, which renders it uniquely suited to problems of sensory noise cancellation. However, this capacity can be extended to the avoidance of sensory interference, by appropriate movements of, for example, the eyes in the vestibulo-ocular reflex. Avoidance becomes particularly important when painful signals are involved, and as the climbing fibre input to zone C3 is extremely responsive to nociceptive stimuli, it is proposed that one function of this zone is the avoidance of pain by, for example, adjusting movements of the body to avoid self-harm. This hypothesis appears consistent with evidence from humans and animals concerning the role of the intermediate cerebellum in classically conditioned withdrawal reflexes, but further experiments focusing on conditioned avoidance are required to test the hypothesis more stringently. The proposed architecture may also be useful for automatic self-adjusting damage avoidance in robots, an important consideration for next generation 'soft' robots designed to interact with people.

Adaptive filters and internal models: multilevel description of cerebellar function.

Cerebellar function is increasingly discussed in terms of engineering schemes for motor control and signal processing that involve internal models. To address the relation between the cerebellum and internal models, we adopt the chip metaphor that has been used to represent the combination of a homogeneous cerebellar cortical microcircuit with individual microzones having unique external connections. This metaphor indicates that identifying the function of a particular cerebellar chip requires knowledge of both the general microcircuit algorithm and the chip's individual connections. Here we use a popular candidate algorithm as embodied in the adaptive filter, which learns to decorrelate its inputs from a reference ('teaching', 'error') signal. This algorithm is computationally powerful enough to be used in a very wide variety of engineering applications. However, the crucial issue is whether the external connectivity required by such applications can be implemented biologically. We argue that some applications appear to be in principle biologically implausible: these include the Smith predictor and Kalman filter (for state estimation), and the feedback-error-learning scheme for adaptive inverse control. However, even for plausible schemes, such as forward models for noise cancellation and novelty-detection, and the recurrent architecture for adaptive inverse control, there is unlikely to be a simple mapping between microzone function and internal model structure. This initial analysis suggests that cerebellar involvement in particular behaviours is therefore unlikely to have a neat classification into categories such as 'forward model'. It is more likely that cerebellar microzones learn a task-specific adaptive-filter operation which combines a number of signal-processing roles.

An internal model architecture for novelty detection: implications for cerebellar and collicular roles in sensory processing.

The cerebellum is thought to implement internal models for sensory prediction, but details of the underlying circuitry are currently obscure. We therefore investigated a specific example of internal-model based sensory prediction, namely detection of whisker contacts during whisking. Inputs from the vibrissae in rats can be affected by signals generated by whisker movement, a phenomenon also observable in whisking robots. Robot novelty-detection can be improved by adaptive noise-cancellation, in which an adaptive filter learns a forward model of the whisker plant that allows the sensory effects of whisking to be predicted and thus subtracted from the noisy sensory input. However, the forward model only uses information from an efference copy of the whisking commands. Here we show that the addition of sensory information from the whiskers allows the adaptive filter to learn a more complex internal model that performs more robustly than the forward model, particularly when the whisking-induced interference has a periodic structure. We then propose a neural equivalent of the circuitry required for adaptive novelty-detection in the robot, in which the role of the adaptive filter is carried out by the cerebellum, with the comparison of its output (an estimate of the self-induced interference) and the original vibrissal signal occurring in the superior colliculus, a structure noted for its central role in novelty detection. This proposal makes a specific prediction concerning the whisker-related functions of a region in cerebellar cortical zone A(2) that in rats receives climbing fibre input from the superior colliculus (via the inferior olive). This region has not been observed in non-whisking animals such as cats and primates, and its functional role in vibrissal processing has hitherto remained mysterious. Further investigation of this system may throw light on how cerebellar-based internal models could be used in broader sensory, motor and cognitive contexts.

Evaluating the adaptive-filter model of the cerebellum.

The adaptive-filter model of the cerebellar microcircuit is in widespread use, combining as it does an explanation of key microcircuit features with well-specified computational power. Here we consider two methods for its evaluation. One is to test its predictions concerning relations between cerebellar inputs and outputs. Where the relevant experimental data are available, e.g. for the floccular role in image stabilization, the predictions appear to be upheld. However, for the majority of cerebellar microzones these data have yet to be obtained. The second method is to test model predictions about details of the microcircuit. We focus on features apparently incompatible with the model, in particular non-linear patterns in Purkinje cell simple-spike firing. Analysis of these patterns suggests the following three conclusions. (i) It is important to establish whether they can be observed during task-related behaviour. (ii) Highly non-linear models based on these patterns are unlikely to be universal, because they would be incompatible with the (approximately) linear nature of floccular function. (iii) The control tasks for which these models are computationally suited need to be identified. At present, therefore, the adaptive filter remains a candidate model of at least some cerebellar microzones, and its evaluation suggests promising lines for future enquiry.

Synaptic plasticity in medial vestibular nucleus neurons: comparison with computational requirements of VOR adaptation.

BACKGROUND: Vestibulo-ocular reflex (VOR) gain adaptation, a longstanding experimental model of cerebellar learning, utilizes sites of plasticity in both cerebellar cortex and brainstem. However, the mechanisms by which the activity of cortical Purkinje cells may guide synaptic plasticity in brainstem vestibular neurons are unclear. Theoretical analyses indicate that vestibular plasticity should depend upon the correlation between Purkinje cell and vestibular afferent inputs, so that, in gain-down learning for example, increased cortical activity should induce long-term depression (LTD) at vestibular synapses. METHODOLOGY/PRINCIPAL FINDINGS: Here we expressed this correlational learning rule in its simplest form, as an anti-Hebbian, heterosynaptic spike-timing dependent plasticity interaction between excitatory (vestibular) and inhibitory (floccular) inputs converging on medial vestibular nucleus (MVN) neurons (input-spike-timing dependent plasticity, iSTDP). To test this rule, we stimulated vestibular afferents to evoke EPSCs in rat MVN neurons in vitro. Control EPSC recordings were followed by an induction protocol where membrane hyperpolarizing pulses, mimicking IPSPs evoked by flocculus inputs, were paired with single vestibular nerve stimuli. A robust LTD developed at vestibular synapses when the afferent EPSPs coincided with membrane hyperpolarization, while EPSPs occurring before or after the simulated IPSPs induced no lasting change. Furthermore, the iSTDP rule also successfully predicted the effects of a complex protocol using EPSP trains designed to mimic classical conditioning. CONCLUSIONS: These results, in strong support of theoretical predictions, suggest that the cerebellum alters the strength of vestibular synapses on MVN neurons through hetero-synaptic, anti-Hebbian iSTDP. Since the iSTDP rule does not depend on post-synaptic firing, it suggests a possible mechanism for VOR adaptation without compromising gaze-holding and VOR performance in vivo.

Sensory prediction or motor control? Application of marr-albus type models of cerebellar function to classical conditioning.

Marr-Albus adaptive filter models of the cerebellum have been applied successfully to a range of sensory and motor control problems. Here we analyze their properties when applied to classical conditioning of the nictitating membrane response in rabbits. We consider a system-level model of eyeblink conditioning based on the anatomy of the eyeblink circuitry, comprising an adaptive filter model of the cerebellum, a comparator model of the inferior olive and a linear dynamic model of the nictitating membrane plant. To our knowledge, this is the first model that explicitly includes all these principal components, in particular the motor plant that is vital for shaping and timing the behavioral response. Model assumptions and parameters were systematically investigated to disambiguate basic computational capacities of the model from features requiring tuning of properties and parameter values. Without such tuning, the model robustly reproduced a range of behaviors related to sensory prediction, by displaying appropriate trial-level associative learning effects for both single and multiple stimuli, including blocking and conditioned inhibition. In contrast, successful reproduction of the real-time motor behavior depended on appropriate specification of the plant, cerebellum and comparator models. Although some of these properties appear consistent with the system biology, fundamental questions remain about how the biological parameters are chosen if the cerebellar microcircuit applies a common computation to many distinct behavioral tasks. It is possible that the response profiles in classical conditioning of the eyeblink depend upon operant contingencies that have previously prevailed, for example in naturally occurring avoidance movements.

Nonlinear dynamic modeling of isometric force production in primate eye muscle.

Although the oculomotor plant is usually modeled as a linear system, recent studies of ocular motoneuron behavior have drawn attention to the presence of significant nonlinearities. One source of these is the development of muscle force in response to changes in motoneuron firing rate. Here, we attempt to simulate the production of isometric force by the primate lateral rectus muscle in response to electrical stimulation [A. Fuchs and E. Luschei, "Development of isometric tension in simian extraocular muscle," J. Physiol., vol. 219, no. 1, pp. 155-166, 1971] by comparing four different modeling approaches. The data could be well fitted either by parameter estimation for physically based models of force production [J. Bobet, E. R. Gossen, and R. B. Stein, "A comparison of models of force production during stimulated isometric ankle dorsiflexion in humans," IEEE Trans. Neural Syst. Rehabil. Eng., vol. 13, no. 4, pp. 444-451, Dec. 2005; E. Mavritsaki, N. Lepora, J. Porrill, C. H. Yeo, and P. Dean, "Response linearity determined by recruitment strategy in detailed model of nictitating membrane control," Biol. Cybern., vol. 96, no. 1, pp. 39-57, 2007], or by the application of a generic method for nonlinear system identification (the nonlinear autoregressive with exogenous input (NARX) model). These results suggest that nonlinear system identification may be a useful method for modeling more general aspects of muscle function, and provide a basis for distributed models of motor units in extraocular muscle for understanding dynamic oculomotor control. The success of previous linear models points to the potential importance of motor unit recruitment in overcoming nonlinearities in the oculomotor plant.

The accuracy of metric judgements: perception of surface normal.

Observers adjusted a probe (a short rod) to appear normal to a planar surface slanted in depth. In Experiment 1, observers (N=12) performed this metric task in two conditions: with reduced cues to calibration of binocular viewing parameters and with full cues. The results provided evidence for the use of an internal working metric in metric tasks because they confirm predictions that (i) errors should be largely systematic and accounted for by assuming an inaccurate working metric and (ii) this metric should be consistent with miscalibration of relevant viewing parameters. The data support the prediction that performance errors decrease in a manner consistent with improved binocular calibration, when better cues to relevant viewing parameters are provided. We performed two additional control experiments as further tests of the binocular miscalibration account, to determine whether performance in Experiment 1 could be explained instead by the use of monocular cues. We found that monocular performance was significantly poorer than binocular performance in reduced-cue conditions (Experiment 2) and full-cue conditions (Experiment 3). These control experiments provide confirmation that binocular cues contribute to performance in the full-cue conditions of Experiment 1, and that disparity was the only effective cue to slant in reduced-cue conditions.

The cerebellum as an adaptive filter: a general model?

Many functional models of the cerebellar microcircuit are based on the adaptive-filter model first proposed by Fujita. The adaptive filter has powerful signal processing capacities that are suitable for both sensory and motor tasks, and uses a simple and intuitively plausible decorrelation learning rule that offers and account of the evolution of the inferior olive. Moreover, in those cases where the input-output transformations of cerebellar microzones have been sufficiently characterised, they appear to conform to those predicted by the adaptive-filter model. However, these cases are few in number, and comparing the model with the internal operations of the microcircuit itself has not proved straightforward. Whereas some microcircuit features appear compatible with adaptive-filter function, others such as simple granular-layer processing or Purkinje cell bistability, do not. How far these seeming incompatibilities indicate additional computational roles for the cerebellar microcircuit remains to be determined.

The cerebellar microcircuit as an adaptive filter: experimental and computational evidence.

Initial investigations of the cerebellar microcircuit inspired the Marr-Albus theoretical framework of cerebellar function. We review recent developments in the experimental understanding of cerebellar microcircuit characteristics and in the computational analysis of Marr-Albus models. We conclude that many Marr-Albus models are in effect adaptive filters, and that evidence for symmetrical long-term potentiation and long-term depression, interneuron plasticity, silent parallel fibre synapses and recurrent mossy fibre connectivity is strikingly congruent with predictions from adaptive-filter models of cerebellar function. This congruence suggests that insights from adaptive-filter theory might help to address outstanding issues of cerebellar function, including both microcircuit processing and extra-cerebellar connectivity.

Cerebellar-inspired adaptive control of a robot eye actuated by pneumatic artificial muscles.

In this paper, a model of cerebellar function is implemented and evaluated in the control of a robot eye actuated by pneumatic artificial muscles. The investigated control problem is stabilization of the visual image in response to disturbances. This is analogous to the vestibuloocular reflex (VOR) in humans. The cerebellar model is structurally based on the adaptive filter, and the learning rule is computationally analogous to least-mean squares, where parameter adaptation at the parallel fiber/Purkinje cell synapse is driven by the correlation of the sensory error signal (carried by the climbing fiber) and the motor command signal. Convergence of the algorithm is first analyzed in simulation on a model of the robot and then tested online in both one and two degrees of freedom. The results show that this model of neural function successfully works on a real-world problem, providing empirical evidence for validating: 1) the generic cerebellar learning algorithm; 2) the function of the cerebellum in the VOR; and 3) the signal transmission between functional neural components of the VOR.

Dynamics of primate oculomotor plant revealed by effects of abducens microstimulation.

Despite their importance for deciphering oculomotor commands, the mechanics of the extraocular muscles and orbital tissues (oculomotor plant) are poorly understood. In particular, the significance of plant nonlinearities is uncertain. Here primate plant dynamics were investigated by measuring the eye movements produced by stimulating the abducens nucleus with brief pulse trains of varying frequency. Statistical analysis of these movements indicated that the effects of stimulation lasted about 40 ms after the final pulse, after which the eye returned passively toward its position before stimulation. Behavior during the passive phase could be approximated by a linear plant model, corresponding to Voigt elements in series, with properties independent of initial eye position. In contrast, behavior during the stimulation phase revealed a sigmoidal relation between stimulation frequency and estimated steady-state tetanic tension, together with a frequency-dependent rate of tension increase, that appeared very similar to the nonlinearities previously found for isometric-force production in primate lateral rectus muscle. These results suggest that the dynamics of the oculomotor plant have an approximately linear component related to steady-state viscoelasticity and a nonlinear component related to changes in muscle activation. The latter may in part account for the nonlinear relations observed between eye-movement parameters and single-unit firing patterns in the abducens nucleus. These findings point to the importance of recruitment as a simplifying factor for motor control with nonlinear plants.