Language within our grasp.

In monkeys, the rostral part of ventral premotor cortex (area F5) contains neurons that discharge, both when the monkey grasps or manipulates objects and when it observes the experimenter making similar actions. These neurons (mirror neurons) appear to represent a system that matches observed events to similar, internally generated actions, and in this way forms a link between the observer and the actor. Transcranial magnetic stimulation and positron emission tomography (PET) experiments suggest that a mirror system for gesture recognition also exists in humans and includes Broca's area. We propose here that such an observation/execution matching system provides a necessary bridge from'doing' to'communicating',as the link between actor and observer becomes a link between the sender and the receiver of each message.

Grasping objects: the cortical mechanisms of visuomotor transformation.

Grasping requires coding of the object's intrinsic properties (size and shape), and the transformation of these properties into a pattern of distal (finger and wrist) movements. Computational models address this behavior through the interaction of perceptual and motor schemas. In monkeys, the transformation of an object's intrinsic properties into specific grips takes place in a circuit that is formed by the inferior parietal lobule and the inferior premotor area (area F5). Neurons in both these areas code size, shape and orientation of objects, and specific types of grip that are necessary to grasp them. Grasping movements are coded more globally in the inferior parietal lobule, whereas they are more segmented in area F5. In humans, neuropsychological studies of patients with lesions to the parietal lobule confirm that primitive shape characteristics of an object for grasping are analyzed in the parietal lobe, and also demonstrate that this 'pragmatic' analysis of objects is separated from the 'semantic' analysis performed in the temporal lobe.

Self and society: between God and brain.

Cognitive Neuroscience and Divine Action, 21-27 June 1998, Paserbiec, Poland.

Modeling functions of striatal dopamine modulation in learning and planning.

The activity of midbrain dopamine neurons is strikingly similar to the reward prediction error of temporal difference reinforcement learning models. Experimental evidence and simulation studies suggest that dopamine neuron activity serves as an effective reinforcement signal for learning of sensorimotor associations in striatal matrisomes. In the current study, we simulate dopamine neuron activity with the extended temporal difference model of Pavlovian learning and examine the influences of this signal on medium spiny neurons in striatal matrisomes. The modeled influences include transient membrane effects of dopamine D(1) receptor activation, dopamine-dependent long-term adaptations of corticostriatal transmission, and effects of dopamine on rhythmic fluctuations of the membrane potential between an elevated "up-state" and a hyperpolarized "down-state". The most dominant activity in the striatal matrisomes is assumed to elicit behaviors via projections from the basal ganglia to the thalamus and the cortex. This "standard model" performs successfully when tested for sensorimotor learning and goal-directed behavior (planning). To investigate the contributions of our model assumptions to learning and planning, we test the performance of several model variants that lack one of these mechanisms. These simulations show that the adaptation of the dopamine-like signal is necessary for sensorimotor learning and planning. Sensorimotor learning requires dopamine-dependent long-term adaptation of corticostriatal transmission. Lack of dopamine-like novelty responses decreases the number of exploratory acts, which impairs planning capabilities. The model loses its planning capabilities if the dopamine-like signal is simulated with the original temporal difference model, because the original temporal difference model does not form novel associative chains. Transient membrane effects of the dopamine-like signal on striatal firing substantially shorten the reaction time in the planning task. The capability for planning is improved by influences of dopamine on the durations of membrane potential fluctuations and by manipulations that prolong the reaction time of the model. These results suggest that responses of dopamine neurons to conditioned stimuli contribute to sensorimotor reward learning, novelty responses of dopamine neurons stimulate exploration, and transient dopamine membrane effects are important for planning.

Co-evolution of human consciousness and language.

This article recalls Cajal's brief mention of consciousness in the Textura as a function of the human brain quite distinct from reflex action, and discusses the view that human consciousness may share aspects of "animal awareness" with other species, but has its unique form because humans possess language. Three ingredients of a theory of the evolution of human consciousness are offered: the view that a précis of intended activity is necessarily formed in the brain of a human that communicates in a human way; the notion that such a précis constitutes consciousness; and a new theory of the evolution of human language based on the mirror system of monkeys and the role of communication by means of hand gestures as a stepping-stone to speech.

Quantitative modeling of responses of anuran retina: stimulus shape and size dependency.

Teeters and Arbib presented a model of the anuran retina which qualitatively accounts for the characteristic response properties used to distinguish ganglion cell type in anurans. In this paper we test the model's ability to reproduce quantitatively tabulated data on the dependency on stimulus shape and size, with a new implementation of the model in the neural simulation language NSL. Data of Ewert and Hock relating toad R2, R3, and R4 ganglion cell responses to moving worm, antiworm, and square-shaped stimuli of various edge lengths are used to test stimulus shape and size dependency. A close match to the data can be achieved by tuning some of the model parameters while still retaining the characteristic responses to the typical stimulus types. We stress here the importance of a populational approach to the models. We place more emphasis on the variation of response properties in a population of neurons of the same class, rather than questing for the neuron of a given type. As an example of the populational approach we offer a model for the respiratory R3 response following researchers who argue that a subclass of R3 neurons are activated by stationary boundaries owing to the anuran's self induced respiratory eye movement.

Modeling the physiological responses of anuran R3 ganglion cells.

Teeters and Arbib (Bio Cybernet 1991;64:197-207) presented a model of the anuran retina which qualitatively accounts for some of the characteristic response properties used to distinguish ganglion cell type in anurans. Teeters et al. (Vis Res 1993;33:2361-2379) tested the model's ability to reproduce data of Ewert and Hock (Exp Brain Res 1972;16:41-59) relating toad R2, R3 and R4 ganglion cell responses to moving worm, antiworm and square-shaped stimuli of various edge lengths for stimulus shape and size dependency. In this paper we provide an exhaustive analysis of the performance of the modeled R3 cells with respect to most of the known qualitative and quantitative physiological properties of natural R3 ganglion cells. We also introduce several relevant predictions of the model relating different responses of R3 cells under the effect of changes in different model components. In some cases the predictions have been tested in neurophysiological experiments.

Synthetic brain imaging: grasping, mirror neurons and imitation.

The article contributes to the quest to relate global data on brain and behavior (e.g. from PET, Positron Emission Tomography, and fMRI. functional Magnetic Resonance Imaging) to the underpinning neural networks. Models tied to human brain imaging data often focus on a few "boxes" based on brain regions associated with exceptionally high blood flow, rather than analyzing the cooperative computation of multiple brain regions. For analysis directly at the level of such data, a schema-based model may be most appropriate. To further address neurophysiological data, the Synthetic PET imaging method uses computational models of biological neural circuitry based on animal data to predict and analyze the results of human PET studies. This technique makes use of the hypothesis that rCBF (regional cerebral blood flow) is correlated with the integrated synaptic activity in a localized brain region. We also describe the possible extension of the Synthetic PET method to fMRI. The second half of the paper then exemplifies this general research program with two case studies, one on visuo-motor processing for control of grasping (Section 3 in which the focus is on Synthetic PET) and the imitation of motor skills (Sections 4 and 5, with a focus on Synthetic fMRI). Our discussion of imitation pays particular attention to data on the mirror system in monkey (neural circuitry which allows the brain to recognize actions as well as execute them). Finally, Section 6 outlines the immense challenges in integrating models of different portions of the nervous system which address detailed neurophysiological data from studies of primates and other species; summarizes key issues for developing the methodology of Synthetic Brain Imaging; and shows how comparative neuroscience and evolutionary arguments will allow us to extend Synthetic Brain Imaging even to language and other cognitive functions for which few or no animal data are available.

From visual affordances in monkey parietal cortex to hippocampo-parietal interactions underlying rat navigation.

This paper explores the hypothesis that various subregions (but by no means all) of the posterior parietal cortex are specialized to process visual information to extract a variety of affordances for behaviour. Two biologically based models of regions of the posterior parietal cortex of the monkey are introduced. The model of the lateral intraparietal area (LIP) emphasizes its roles in dynamic remapping of the representation of targets during a double saccade task, and in combining stored, updated input with current visual input. The model of the anterior intraparietal area (AIP) addresses parietal-premotor interactions involved in grasping, and analyses the interaction between the AIP and premotor area F5. The model represents the role of other intraparietal areas working in concert with the inferotemporal cortex as well as with corollary discharge from F5 to provide and augment the affordance information in the AIP, and suggests how various constraints may resolve the action opportunities provided by multiple affordances. Finally, a systems-level model of hippocampo parietal interactions underlying rat navigation is developed, motivated by the monkey data used in developing the above two models as well as by data on neurones in the posterior parietal cortex of the monkey that are sensitive to visual motion. The formal similarity between dynamic remapping (primate saccades) and path integration (rat navigation) is noted, and certain available data on rat posterior parietal cortex in terms of affordances for locomotion are explained. The utility of further modelling, linking the World Graph model of cognitive maps for motivated behaviour with hippocampal-parietal interactions involved in navigation, is also suggested. These models demonstrate that posterior parietal cortex is not only itself a network of interacting subsystems, but functions through cooperative computation with many other brain regions.

Schemas for the temporal organization of behaviour.

Those aspects of the timing of behaviour are emphasized which derive from the need for the organism to coordinate its actions with objects in the environment. Such coordination may require the serial performance of certain actions, yet permit elements of concurrency as well. Perceptual and motor schemas are introduced as units for the functional description of behaviour intermediate between a purely phenomenological description and an account of the detailed neural mechanisms of behaviour. The language of coordinated control programs is outlined to suggest how such schemas are orchestrated in visually and tactilely guided behaviour. Finally, a crucial property of the timing of many movements is discussed: their division into a fast (feedforward, ballistic) phase followed by a slow (feedback) phase. This division is analyzed in the light of the effect of brain damage on reaching movements.

Brain theory and cooperative computation.

"Top-down" brain theory, based upon functional analysis of cognitive processes in terms of interacting schemas, is distinguished from "bottom-up" brain theory based on analysis of the dynamics of neural nets. "Cooperative computation" is proposed as the style of interaction of neural subsystems at various levels. Perceptual schemas are introduced as the building blocks for the representation of the perceived environment, and motor schemas serve as control systems to be coordinated into programs for the control of movement. A cooperative computation view of the design of machine vision systems is exemplified both by an algorithm for computing optic flow which offers interesting insights into the evolution of hierarchical neural structures, and by an analysis of knowledge representation for machine interpretation of visual scenes. The interaction between top-down analysis and detailed neural modelling is illustrated by the study of visuomotor coordination in frogs and toads.

Rana computatrix: progress report, 1984.

This paper describes an approach to analyzing mechanisms of visuomotor coordination that is twofold: top-down, to offer a coordinated control program of interacting schemata (functional units, underlying behavior, which can be activated in different combinations) to achieve behavior noted by neuroethologists; and bottom-up, to provide detailed models of neural networks that are consistent with known anatomy and physiology, but that involve additional assumptions, amenable to experimental testing, to yield a network capable of exhibiting appropriate behavior. Rana computatrix, an evolving series of models of frog and toad visuomotor coordination, is described.

The cue interaction model of depth perception: a stability analysis.

In this paper, we offer a stability analysis of "the cue interaction model" of depth perception (House (1984]. Depth estimation using stereopsis suffers from the "matching problem", the problem of correctly matching the retinal image of a feature in one eye, to its retinal image in the other eye. The Cue Interaction Model overcomes this by using monocular cues to disambiguate between the "correct matches" and the "incorrect matches". Its decision making is based on the concept of cooperation and competition in a neural network. A general class of cooperative and competitive models has been mathematically analysed by Amari and Arbib (1977), with special attention given to equilibrium states and stability. In this paper we adapt their methods to study the above model. In particular, we prove that if the parameters are correctly tuned, the model successfully achieves its goals by suppressing the cues which represent the "incorrect matches".

Précis of Neural organization: structure, function, and dynamics.

NEURAL ORGANIZATION: Structure, function, and dynamics shows how theory and experiment can supplement each other in an integrated, evolving account of the brain's structure, function, and dynamics. (1) STRUCTURE: Studies of brain function and dynamics build on and contribute to an understanding of many brain regions, the neural circuits that constitute them, and their spatial relations. We emphasize Szentágothai's modular architectonics principle, but also stress the importance of the microcomplexes of cerebellar circuitry and the lamellae of hippocampus. (2) FUNCTION: Control of eye movements, reaching and grasping, cognitive maps, and the roles of vision receive a functional decomposition in terms of schemas. Hypotheses as to how each schema is implemented through the interaction of specific brain regions provide the basis for modeling the overall function by neural networks constrained by neural data. Synthetic PET integrates modeling of primate circuitry with data from human brain imaging. (3) DYNAMICS: Dynamic system theory analyzes spatiotemporal neural phenomena, such as oscillatory and chaotic activity in both single neurons and (often synchronized) neural networks, the self-organizing development and plasticity of ordered neural structures, and learning and memory phenomena associated with synaptic modification. Rhythm generation involves multiple levels of analysis, from intrinsic cellular processes to loops involving multiple brain regions. A variety of rhythms are related to memory functions. The Précis presents a multifaceted case study of the hippocampus. We conclude with the claim that language and other cognitive processes can be fruitfully studied within the framework of neural organization that the authors have charted with John Szentágothai.

A mathematical analysis of the force-stiffness characteristics of muscles in control of a single joint system.

Feldman (1966) has proposed that a muscle endowed with its spinal reflex system behaves as a non-linear spring with an adjustable resting length. In contrast, because of the length-tension properties of muscles, many researchers have modeled them as non-linear springs with adjustable stiffness. Here we test the merits of each approach: Initially, it is proven that the adjustable stiffness model predicts that isometric muscle force and stiffness are linearly related. We show that this prediction is not supported by data on the static stiffness-force characteristics of reflexive muscles, where stiffness grows non-linearly with force. Therefore, an intact muscle-reflex system does not behave as a non-linear spring with an adjustable stiffness. However, when the same muscle is devoid of its reflexes, the data shows that stiffness grows linearly with force. We aim to understand the functional advantage of the non-linear stiffness-force relationship present in the reflexive muscle. Control of an inverted pendulum with a pair of antagonist muscles is considered. Using an active-state muscle model we describe force development in an areflexive muscle. From the data on the relationship of stiffness and force in the intact muscle we derive the length-tension properties of a reflexive muscle. It is shown that a muscle under the control of its spinal reflexes resembles a non-linear spring with an adjustable resting length. This provides independent evidence in support of the Feldman hypothesis of an adjustable resting length as the control parameter of a reflexive muscle, but it disagrees with his particular formulation. In order to maintain stability of the single joint system, we prove that a necessary condition is that muscle stiffness must grow at least linearly with force at isometric conditions. This shows that co-contraction of antagonist muscles may actually destabilize the limb if the slope of this stiffness-force relationship is less than an amount specified by the change in the moment arm of the muscle as a function of joint configuration. In a reflexive muscle where stiffness grows faster than linearly with force, co-contraction will always lead to an increase in stiffness. Furthermore, with the reflexive muscles, the same level of joint stiffness can be produced by much smaller muscle forces because of the non-linear stiffness-force relationship. This allows the joint to remain stable at a fraction of the metabolic energy cost associated with maintaining stability with areflexive muscles.

A cortico-subcortical model for generation of spatially accurate sequential saccades.

This article provides a systems framework for the analysis of cortical and subcortical interactions in the control of saccadic eye movements, A major thesis of this model is that a topography of saccade direction and amplitude is preserved through multiple projections between brain regions until they are finally transformed into a temporal pattern of activity that drives the eyes to the target. The control of voluntary saccades to visual and remembered targets is modeled in terms of interactions between posterior parietal cortex, frontal eye fields, the basal ganglia (caudate and substantia nigra), superior colliculus, mediodorsal thalamus, and the saccade generator of the brainstem. Interactions include the modulation of eye movement motor error maps by topographic inhibitory projections, dynamic remapping of spatial target representations in saccade motor error maps, and sustained neural activity that embodies spatial memory. Models of these mechanisms implemented in our Neural Simulation Language simulate behavior and neural activity described in the literature, and suggest new experiments.

The prey localisation model: a stability analysis.

This paper analyzes the "Prey localisation Model" (House 1984), for animals that are unable to verge their eyes. The Prey localisation Model selects a single point or a portion of the scene in its visual space. In particular it imitates the behaviour of frogs and toads of selecting the closer target when two equally attractive prey are presented to it. The model achieves its goal by tightly coupling two prey selection processes, one for each eye, with lens accommodation. In this paper we offer a stability analysis of the model, and show how lens accommodation, i.e. adjustment in the focal length of the lens, biases the selection of the proximal target. We examine the properties of the model that are responsible for its behaviour and derive a set of conditions which guarantees the localisation of the correct target.

A neural model of the interaction of tectal columns in prey-catching behavior.

Building on a simple model of a tectal column as the unit of processing in the amphibian tectum, we conduct a computer analysis of the interaction of a linear array of such columns. The model suggests that the inhibitory and excitatory activity in the tectum may have three functions: 1) spatio-temporal facilitation of column activity to a moving stimulus; 2) preference for the head of the stimulus, probably to avoid possible defensive reactions of the prey; and 3) modulating the state of excitation of the column once it has produced a response. The model also shows that the spatio-temporal effects of excitation and inhibition increases the acuity of the animal to the direction of the prey, through processes similar to lateral inhibition.

The extended branch-arrow model of the formation of retino-tectal connections.

This paper presents XBAM (the Extended Branch-Arrow Model), a new model of the development of the retino-tectal topographic mapping as observed in frog, toad, and goldfish visual systems. The updating process employed by XBAM is distributed in nature and depends upon interactions between branches of retinal fibers, the branches and the boundaries of the tectum and grafts, and the branches and the tectal surface. Results of computer simulation of the model are related to experimental data obtained from tectal and retinal graft and lesion studies, and comparisons are also made with other models.