Sensorimotor maps can be dynamically calibrated using an adaptive-filter model of the cerebellum.

Substantial experimental evidence suggests the cerebellum is involved in calibrating sensorimotor maps. Consistent with this involvement is the well-known, but little understood, massive cerebellar projection to maps in the superior colliculus. Map calibration would be a significant new role for the cerebellum given the ubiquity of map representations in the brain, but how it could perform such a task is unclear. Here we investigated a dynamic method for map calibration, based on electrophysiological recordings from the superior colliculus, that used a standard adaptive-filter cerebellar model. The method proved effective for complex distortions of both unimodal and bimodal maps, and also for predictive map-based tracking of moving targets. These results provide the first computational evidence for a novel role for the cerebellum in dynamic sensorimotor map calibration, of potential importance for coordinate alignment during ongoing motor control, and for map calibration in future biomimetic systems. This computational evidence also provides testable experimental predictions concerning the role of the connections between cerebellum and superior colliculus in previously observed dynamic coordinate transformations.

At the Edge of Chaos: How Cerebellar Granular Layer Network Dynamics Can Provide the Basis for Temporal Filters.

Models of the cerebellar microcircuit often assume that input signals from the mossy-fibers are expanded and recoded to provide a foundation from which the Purkinje cells can synthesize output filters to implement specific input-signal transformations. Details of this process are however unclear. While previous work has shown that recurrent granule cell inhibition could in principle generate a wide variety of random outputs suitable for coding signal onsets, the more general application for temporally varying signals has yet to be demonstrated. Here we show for the first time that using a mechanism very similar to reservoir computing enables random neuronal networks in the granule cell layer to provide the necessary signal separation and extension from which Purkinje cells could construct basis filters of various time-constants. The main requirement for this is that the network operates in a state of criticality close to the edge of random chaotic behavior. We further show that the lack of recurrent excitation in the granular layer as commonly required in traditional reservoir networks can be circumvented by considering other inherent granular layer features such as inverted input signals or mGluR2 inhibition of Golgi cells. Other properties that facilitate filter construction are direct mossy fiber excitation of Golgi cells, variability of synaptic weights or input signals and output-feedback via the nucleocortical pathway. Our findings are well supported by previous experimental and theoretical work and will help to bridge the gap between system-level models and detailed models of the granular layer network.

The cerebellar microcircuit as an adaptive filter: experimental and computational evidence.

Initial investigations of the cerebellar microcircuit inspired the Marr-Albus theoretical framework of cerebellar function. We review recent developments in the experimental understanding of cerebellar microcircuit characteristics and in the computational analysis of Marr-Albus models. We conclude that many Marr-Albus models are in effect adaptive filters, and that evidence for symmetrical long-term potentiation and long-term depression, interneuron plasticity, silent parallel fibre synapses and recurrent mossy fibre connectivity is strikingly congruent with predictions from adaptive-filter models of cerebellar function. This congruence suggests that insights from adaptive-filter theory might help to address outstanding issues of cerebellar function, including both microcircuit processing and extra-cerebellar connectivity.

An adaptive filter model of cerebellar zone C3 as a basis for safe limb control?

The review asks how the adaptive filter model of the cerebellum might be relevant to experimental work on zone C3, one of the most extensively studied regions of cerebellar cortex. As far as features of the cerebellar microcircuit are concerned, the model appears to fit very well with electrophysiological discoveries concerning the importance of molecular layer interneurons and their plasticity, the significance of long-term potentiation and the striking number of silent parallel fibre synapses. Regarding external connectivity and functionality, a key feature of the adaptive filter model is its use of the decorrelation algorithm, which renders it uniquely suited to problems of sensory noise cancellation. However, this capacity can be extended to the avoidance of sensory interference, by appropriate movements of, for example, the eyes in the vestibulo-ocular reflex. Avoidance becomes particularly important when painful signals are involved, and as the climbing fibre input to zone C3 is extremely responsive to nociceptive stimuli, it is proposed that one function of this zone is the avoidance of pain by, for example, adjusting movements of the body to avoid self-harm. This hypothesis appears consistent with evidence from humans and animals concerning the role of the intermediate cerebellum in classically conditioned withdrawal reflexes, but further experiments focusing on conditioned avoidance are required to test the hypothesis more stringently. The proposed architecture may also be useful for automatic self-adjusting damage avoidance in robots, an important consideration for next generation 'soft' robots designed to interact with people.

Evaluating the adaptive-filter model of the cerebellum.

The adaptive-filter model of the cerebellar microcircuit is in widespread use, combining as it does an explanation of key microcircuit features with well-specified computational power. Here we consider two methods for its evaluation. One is to test its predictions concerning relations between cerebellar inputs and outputs. Where the relevant experimental data are available, e.g. for the floccular role in image stabilization, the predictions appear to be upheld. However, for the majority of cerebellar microzones these data have yet to be obtained. The second method is to test model predictions about details of the microcircuit. We focus on features apparently incompatible with the model, in particular non-linear patterns in Purkinje cell simple-spike firing. Analysis of these patterns suggests the following three conclusions. (i) It is important to establish whether they can be observed during task-related behaviour. (ii) Highly non-linear models based on these patterns are unlikely to be universal, because they would be incompatible with the (approximately) linear nature of floccular function. (iii) The control tasks for which these models are computationally suited need to be identified. At present, therefore, the adaptive filter remains a candidate model of at least some cerebellar microzones, and its evaluation suggests promising lines for future enquiry.

A multizone cerebellar chip for bioinspired adaptive robot control and sensorimotor processing.

The cerebellum is a neural structure essential for learning, which is connected via multiple zones to many different regions of the brain, and is thought to improve human performance in a large range of sensory, motor and even cognitive processing tasks. An intriguing possibility for the control of complex robotic systems would be to develop an artificial cerebellar chip with multiple zones that could be similarly connected to a variety of subsystems to optimize performance. The novel aim of this paper, therefore, is to propose and investigate a multizone cerebellar chip applied to a range of tasks in robot adaptive control and sensorimotor processing. The multizone cerebellar chip was evaluated using a custom robotic platform consisting of an array of tactile sensors driven by dielectric electroactive polymers mounted upon a standard industrial robot arm. The results demonstrate that the performance in each task was improved by the concurrent, stable learning in each cerebellar zone. This paper, therefore, provides the first empirical demonstration that a synthetic, multizone, cerebellar chip could be embodied within existing robotic systems to improve performance in a diverse range of tasks, much like the cerebellum in a biological system.

The cerebellum as an adaptive filter: a general model?

Many functional models of the cerebellar microcircuit are based on the adaptive-filter model first proposed by Fujita. The adaptive filter has powerful signal processing capacities that are suitable for both sensory and motor tasks, and uses a simple and intuitively plausible decorrelation learning rule that offers and account of the evolution of the inferior olive. Moreover, in those cases where the input-output transformations of cerebellar microzones have been sufficiently characterised, they appear to conform to those predicted by the adaptive-filter model. However, these cases are few in number, and comparing the model with the internal operations of the microcircuit itself has not proved straightforward. Whereas some microcircuit features appear compatible with adaptive-filter function, others such as simple granular-layer processing or Purkinje cell bistability, do not. How far these seeming incompatibilities indicate additional computational roles for the cerebellar microcircuit remains to be determined.

World Statistics Drive Learning of Cerebellar Internal Models in Adaptive Feedback Control: A Case Study Using the Optokinetic Reflex.

The cerebellum is widely implicated in having an important role in adaptive motor control. Many of the computational studies on cerebellar motor control to date have focused on the associated architecture and learning algorithms in an effort to further understand cerebellar function. In this paper we switch focus to the signals driving cerebellar adaptation that arise through different motor behavior. To do this, we investigate computationally the contribution of the cerebellum to the optokinetic reflex (OKR), a visual feedback control scheme for image stabilization. We develop a computational model of the adaptation of the cerebellar response to the world velocity signals that excite the OKR (where world velocity signals are used to emulate head velocity signals when studying the OKR in head-fixed experimental laboratory conditions). The results show that the filter learnt by the cerebellar model is highly dependent on the power spectrum of the colored noise world velocity excitation signal. Thus, the key finding here is that the cerebellar filter is determined by the statistics of the OKR excitation signal.

Silent synapses, LTP, and the indirect parallel-fibre pathway: computational consequences of optimal cerebellar noise-processing.

Computational analysis of neural systems is at its most useful when it uncovers principles that provide a unified account of phenomena across multiple scales and levels of description. Here we analyse a widely used model of the cerebellar contribution to sensori-motor learning to demonstrate both that its response to intrinsic and sensor noise is optimal, and that the unexpected synaptic and behavioural consequences of this optimality can explain a wide range of experimental data. The response of the Marr-Albus adaptive-filter model of the cerebellar microcircuit to noise was examined in the context of vestibulo-ocular reflex calibration. We found that, when appropriately connected, an adaptive-filter model using the covariance learning rule to adjust the weights of synapses between parallel fibres and Purkinje cells learns weight values that are optimal given the relative amount of signal and noise carried by each parallel fibre. This optimality principle is consistent with data on the cerebellar role in smooth pursuit eye movements, and predicts that many synaptic weights must be very small, providing an explanation for the experimentally observed preponderance of silent synapses. Such a preponderance has in its turn two further consequences. First, an additional inhibitory pathway from parallel fibre to Purkinje cell is required if Purkinje cell activity is to be altered in either direction from a starting point of silent synapses. Second, cerebellar learning tasks must often proceed via LTP, rather than LTD as is widely assumed. Taken together, these considerations have profound behavioural consequences, including the optimal combination of sensori-motor information, and asymmetry and hysteresis of sensori-motor learning rates.

Adaptive-filter models of the cerebellum: computational analysis.

Many current models of the cerebellar cortical microcircuit are equivalent to an adaptive filter using the covariance learning rule. The adaptive filter is a development of the original Marr-Albus framework that deals naturally with continuous time-varying signals, thus addressing the issue of 'timing' in cerebellar function, and it can be connected in a variety of ways to other parts of the system, consistent with the microzonal organization of cerebellar cortex. However, its computational capacities are not well understood. Here we summarise the results of recent work that has focused on two of its intrinsic properties. First, an adaptive filter seeks to decorrelate its (mossy fibre) inputs from a (climbing fibre) teaching signal. This procedure can be used both for sensory processing, e.g. removal of interference from sensory signals, and for learning accurate motor commands, by decorrelating an efference copy of those commands from a sensory signal of inaccuracy. As a model of the cerebellum the adaptive filter thus forms a natural link between events at the cellular level, such as forms of synaptic plasticity and the learning rules they embody, and intelligent behaviour at the system level. Secondly, it has been shown that the covariance learning rule enables the filter to handle input and intrinsic noise optimally. Such optimality may underlie the recently described role of the cerebellum in producing accurate smooth pursuit eye movements in the face of sensory noise. Moreover, it has the consequence of driving most input weights to very small values, consistent with experimental data that many parallel-fibre synapses are normally silent. The effectiveness of silent synapses can only be altered by LTP, so learning tasks depending on a reduction of Purkinje cell firing require the synapses to be embedded in a second, inhibitory pathway from parallel fibre to Purkinje cell. This pathway and the appropriate climbing-fibre related plasticity have been described experimentally, and its presence has implications for asymmetries and hysteresis in behavioural learning rates that are also consistent with experimental observations. These computational properties of the adaptive filter suggest that it is both powerful and realistic enough to be a suitable candidate model of the cerebellar cortical microcircuit.

Cerebellar motor learning: when is cortical plasticity not enough?

Classical Marr-Albus theories of cerebellar learning employ only cortical sites of plasticity. However, tests of these theories using adaptive calibration of the vestibulo-ocular reflex (VOR) have indicated plasticity in both cerebellar cortex and the brainstem. To resolve this long-standing conflict, we attempted to identify the computational role of the brainstem site, by using an adaptive filter version of the cerebellar microcircuit to model VOR calibration for changes in the oculomotor plant. With only cortical plasticity, introducing a realistic delay in the retinal-slip error signal of 100 ms prevented learning at frequencies higher than 2.5 Hz, although the VOR itself is accurate up to at least 25 Hz. However, the introduction of an additional brainstem site of plasticity, driven by the correlation between cerebellar and vestibular inputs, overcame the 2.5 Hz limitation and allowed learning of accurate high-frequency gains. This "cortex-first" learning mechanism is consistent with a wide variety of evidence concerning the role of the flocculus in VOR calibration, and complements rather than replaces the previously proposed "brainstem-first" mechanism that operates when ocular tracking mechanisms are effective. These results (i) describe a process whereby information originally learnt in one area of the brain (cerebellar cortex) can be transferred and expressed in another (brainstem), and (ii) indicate for the first time why a brainstem site of plasticity is actually required by Marr-Albus type models when high-frequency gains must be learned in the presence of error delay.

System identification from multiple short-time-duration signals.

System identification problems often arise where the only modeling records available consist of multiple short-time-duration signals. This motivates the development of a modeling approach that is tailored for this situation. An identification algorithm is presented here for parameter estimation based on minimizing the simulated prediction error, across multiple signals. The additional complexity of estimating the initial states corresponding to each signal is removed from the estimation algorithm. A numerical simulation demonstrates that the proposed algorithm performs well in comparison to the often-used least squares method (which leads to biased estimates when identifying systems from measurement noise corrupted signals). The approach is applied to the identification of the passive oculomotor plant; parameters are estimated that describe the dynamics of the plant, which represent the time constants of the visco-elastic elements that characterize the plant connective tissue.

Cerebellar-inspired algorithm for adaptive control of nonlinear dielectric elastomer-based artificial muscle.

Electroactive polymer actuators are important for soft robotics, but can be difficult to control because of compliance, creep and nonlinearities. Because biological control mechanisms have evolved to deal with such problems, we investigated whether a control scheme based on the cerebellum would be useful for controlling a nonlinear dielectric elastomer actuator, a class of artificial muscle. The cerebellum was represented by the adaptive filter model, and acted in parallel with a brainstem, an approximate inverse plant model. The recurrent connections between the two allowed for direct use of sensory error to adjust motor commands. Accurate tracking of a displacement command in the actuator's nonlinear range was achieved by either semi-linear basis functions in the cerebellar model or semi-linear functions in the brainstem corresponding to recruitment in biological muscle. In addition, allowing transfer of training between cerebellum and brainstem as has been observed in the vestibulo-ocular reflex prevented the steady increase in cerebellar output otherwise required to deal with creep. The extensibility and relative simplicity of the cerebellar-based adaptive-inverse control scheme suggests that it is a plausible candidate for controlling this type of actuator. Moreover, its performance highlights important features of biological control, particularly nonlinear basis functions, recruitment and transfer of training.

Evidence for wide range of time scales in oculomotor plant dynamics: implications for models of eye-movement control.

Oculomotor-plant dynamics are not well characterised, despite their importance for modelling eye-movement control. We analysed the time course of the globe's return after horizontal displacements in three rhesus monkeys lightly anaesthetised with ketamine. The eye-position traces were well fitted by a sum of four exponentials (time constants 0.012, 0.099, 0.46, 7.8 s). The two long time-constant terms accounted for 25% of plant compliance, and led to a model that accounted for hitherto unexplained features of ocular motoneuron firing such as (i) hysteresis, and (ii) the inability of a 2 time-constant model to fit data for both fast and slow eye-movements.

Biohybrid Control of General Linear Systems Using the Adaptive Filter Model of Cerebellum.

The adaptive filter model of the cerebellar microcircuit has been successfully applied to biological motor control problems, such as the vestibulo-ocular reflex (VOR), and to sensory processing problems, such as the adaptive cancelation of reafferent noise. It has also been successfully applied to problems in robotics, such as adaptive camera stabilization and sensor noise cancelation. In previous applications to inverse control problems, the algorithm was applied to the velocity control of a plant dominated by viscous and elastic elements. Naive application of the adaptive filter model to the displacement (as opposed to velocity) control of this plant results in unstable learning and control. To be more generally useful in engineering problems, it is essential to remove this restriction to enable the stable control of plants of any order. We address this problem here by developing a biohybrid model reference adaptive control (MRAC) scheme, which stabilizes the control algorithm for strictly proper plants. We evaluate the performance of this novel cerebellar-inspired algorithm with MRAC scheme in the experimental control of a dielectric electroactive polymer, a class of artificial muscle. The results show that the augmented cerebellar algorithm is able to accurately control the displacement response of the artificial muscle. The proposed solution not only greatly extends the practical applicability of the cerebellar-inspired algorithm, but may also shed light on cerebellar involvement in a wider range of biological control tasks.

Decorrelation control by the cerebellum achieves oculomotor plant compensation in simulated vestibulo-ocular reflex.

We introduce decorrelation control as a candidate algorithm for the cerebellar microcircuit and demonstrate its utility for oculomotor plant compensation in a linear model of the vestibulo-ocular reflex (VOR). Using an adaptive-filter representation of cerebellar cortex and an anti-Hebbian learning rule, the algorithm learnt to compensate for the oculomotor plant by minimizing correlations between a predictor variable (eye-movement command) and a target variable (retinal slip), without requiring a motor-error signal. Because it also provides an estimate of the unpredicted component of the target variable, decorrelation control can simplify both motor coordination and sensory acquisition. It thus unifies motor and sensory cerebellar functions.

Recurrent cerebellar architecture solves the motor-error problem.

Current views of cerebellar function have been heavily influenced by the models of Marr and Albus, who suggested that the climbing fibre input to the cerebellum acts as a teaching signal for motor learning. It is commonly assumed that this teaching signal must be motor error (the difference between actual and correct motor command), but this approach requires complex neural structures to estimate unobservable motor error from its observed sensory consequences. We have proposed elsewhere a recurrent decorrelation control architecture in which Marr-Albus models learn without requiring motor error. Here, we prove convergence for this architecture and demonstrate important advantages for the modular control of systems with multiple degrees of freedom. These results are illustrated by modelling adaptive plant compensation for the three-dimensional vestibular ocular reflex. This provides a functional role for recurrent cerebellar connectivity, which may be a generic anatomical feature of projections between regions of cerebral and cerebellar cortex.

Visual awareness and the cerebellum: possible role of decorrelation control.

The two roles in awareness most often suggested for the cerebellum are (i) keeping the details of motor skills away from forebrain computation, and (ii) signaling to the forebrain when a sensory event is not predictable from prior motor commands. However, it is unclear how current models of the cerebellum could carry out these roles. Their architecture, based on the seminal ideas of Marr and Albus, appears to need 'motor error' to learn correct motor commands. However, since motor error is the difference between the actual motor command and what the command should have been, it is a signal unavailable to the organism in principle. We propose a possible solution to this problem, termed decorrelation control, in which the cerebellum learns to decorrelate the motor command sent to the muscles from the sensory consequences of motor error. This method was tested in a linear model of oculomotor plant compensation in the vestibulo-ocular reflex. A copy of the eye-movement command was sent as mossy-fiber input to the flocculus, represented as a simple adaptive filter version of the Marr-Albus architecture. The sensory consequences of motor error were retinal slip, delivered as climbing fiber input to the flocculus. A standard anti-Hebbian learning rule was used to decorrelate the two. Simulations of the linearized problem showed the method to be effective and robust for plant compensation. Decorrelation control is thus a candidate algorithm for the basic cerebellar microcircuit, indicating how it could achieve motor learning using only signals available to the system. Such learning might then enable the cerebellum to free up visual awareness, and also, by providing a sensory signal decorrelated from motor command, supply awareness with crucial information about the external world.

Decorrelation learning in the cerebellum: computational analysis and experimental questions.

Many cerebellar models use a form of synaptic plasticity that implements decorrelation learning. Parallel fibers carrying signals positively correlated with climbing-fiber input have their synapses weakened (long-term depression), whereas those carrying signals negatively correlated with climbing input have their synapses strengthened (long-term potentiation). Learning therefore ceases when all parallel-fiber signals have been decorrelated from climbing-fiber input. This is a computationally powerful rule for supervised learning and can be cast in a spike-timing dependent plasticity form for comparison with experimental evidence. Decorrelation learning is particularly well suited to sensory prediction, for example, in the reafference problem where external sensory signals are interfered with by reafferent signals from the organism's own movements, and the required circuit appears similar to the one found to mediate classical eye blink conditioning. However, for certain stimuli, avoidance is a much better option than simple prediction, and decorrelation learning can also be used to acquire appropriate avoidance movements. One example of a stimulus to be avoided is retinal slip that degrades visual processing, and decorrelation learning appears to play a role in the vestibulo-ocular reflex that stabilizes gaze in the face of unpredicted head movements. Decorrelation learning is thus suitable for both sensory prediction and motor control. It may also be well suited for generic spatial and temporal coordination, because of its ability to remove the unwanted side effects of movement. Finally, because it can be used with any kind of time-varying signal, the cerebellum could play a role in cognitive processing.

Model cerebellar granule cells can faithfully transmit modulated firing rate signals.

A crucial assumption of many high-level system models of the cerebellum is that information in the granular layer is encoded in a linear manner. However, granule cells are known for their non-linear and resonant synaptic and intrinsic properties that could potentially impede linear signal transmission. In this modeling study we analyse how electrophysiological granule cell properties and spike sampling influence information coded by firing rate modulation, assuming no signal-related, i.e., uncorrelated inhibitory feedback (open-loop mode). A detailed one-compartment granule cell model was excited in simulation by either direct current or mossy-fiber synaptic inputs. Vestibular signals were represented as tonic inputs to the flocculus modulated at frequencies up to 20 Hz (approximate upper frequency limit of vestibular-ocular reflex, VOR). Model outputs were assessed using estimates of both the transfer function, and the fidelity of input-signal reconstruction measured as variance-accounted-for. The detailed granule cell model with realistic mossy-fiber synaptic inputs could transmit information faithfully and linearly in the frequency range of the vestibular-ocular reflex. This was achieved most simply if the model neurons had a firing rate at least twice the highest required frequency of modulation, but lower rates were also adequate provided a population of neurons was utilized, especially in combination with push-pull coding. The exact number of neurons required for faithful transmission depended on the precise values of firing rate and noise. The model neurons were also able to combine excitatory and inhibitory signals linearly, and could be replaced by a simpler (modified) integrate-and-fire neuron in the case of high tonic firing rates. These findings suggest that granule cells can in principle code modulated firing-rate inputs in a linear manner, and are thus consistent with the high-level adaptive-filter model of the cerebellar microcircuit.