Reading population codes: a neural implementation of ideal observers.

Many sensory and motor variables are encoded in the nervous system by the activities of large populations of neurons with bell-shaped tuning curves. Extracting information from these population codes is difficult because of the noise inherent in neuronal responses. In most cases of interest, maximum likelihood (ML) is the best read-out method and would be used by an ideal observer. Using simulations and analysis, we show that a close approximation to ML can be implemented in a biologically plausible model of cortical circuitry. Our results apply to a wide range of nonlinear activation functions, suggesting that cortical areas may, in general, function as ideal observers of activity in preceding areas.

Efficient computation and cue integration with noisy population codes.

The brain represents sensory and motor variables through the activity of large populations of neurons. It is not understood how the nervous system computes with these population codes, given that individual neurons are noisy and thus unreliable. We focus here on two general types of computation, function approximation and cue integration, as these are powerful enough to handle a range of tasks, including sensorimotor transformations, feature extraction in sensory systems and multisensory integration. We demonstrate that a particular class of neural networks, basis function networks with multidimensional attractors, can perform both types of computation optimally with noisy neurons. Moreover, neurons in the intermediate layers of our model show response properties similar to those observed in several multimodal cortical areas. Thus, basis function networks with multidimensional attractors may be used by the brain to compute efficiently with population codes.

Computational approaches to sensorimotor transformations.

Behaviors such as sensing an object and then moving your eyes or your hand toward it require that sensory information be used to help generate a motor command, a process known as a sensorimotor transformation. Here we review models of sensorimotor transformations that use a flexible intermediate representation that relies on basis functions. The use of basis functions as an intermediate is borrowed from the theory of nonlinear function approximation. We show that this approach provides a unifying insight into the neural basis of three crucial aspects of sensorimotor transformations, namely, computation, learning and short-term memory. This mathematical formalism is consistent with the responses of cortical neurons and provides a fresh perspective on the issue of frames of reference in spatial representations.

Probabilistic population codes and the exponential family of distributions.

Many experiments have shown that human behavior is nearly Bayes optimal in a variety of tasks. This implies that neural activity is capable of representing both the value and uncertainty of a stimulus, if not an entire probability distribution, and can also combine such representations in an optimal manner. Moreover, this computation can be performed optimally despite the fact that observed neural activity is highly variable (noisy) on a trial-by-trial basis. Here, we argue that this observed variability is actually expected in a neural system which represents uncertainty. Specifically, we note that Bayes' rule implies that a variable pattern of activity provides a natural representation of a probability distribution, and that the specific form of neural variability can be structured so that optimal inference can be executed using simple operations available to neural circuits.

Relating unilateral neglect to the neural coding of space.

Neuropsychological findings on the human neglect syndrome after parietal damage may relate to the physiological properties of single cells that have been studied in monkey parietal cortex and in related brain areas. Human neglect may reflect partial loss or dysfunction of similar cell populations, producing a pathological gradient in the numbers of cells representing particular lateral positions in space, for particular functions. This can explain the graded deficits seen in patients. We relate the patient deficits to cellular properties for several current issues: spatial frames-of-reference; multimodal integration; effective treatments for neglect; motor components to parietal function; and residual unconscious processing. A neural perspective may resolve traditional debates in the neglect literature and outline directions for future research.

Simulating a lesion in a basis function model of spatial representations: comparison with hemineglect.

The basis function theory of spatial representations explains how neurons in the parietal cortex can perform nonlinear transformations from sensory to motor coordinates. The authors present computer simulations showing that unilateral parietal lesions leading to a neuronal gradient in basis function maps can account for the behavior of patients with hemineglect, including (a) neglect in line cancellation and line bisection experiments; (b) neglect in multiple frames of reference simultaneously; (c) relative neglect, a form of what is sometime called object-centered neglect; and (d) neglect without optic ataxia. Contralateral neglect arises in the model because the lesion produces an imbalance in the salience of stimuli that is modulated by the orientation of the body in space. These results strongly support the basis function theory for spatial representations in humans and provide a computational model of hemineglect at the single-cell level.

Temporal limits of the susceptibility of depth perception to proprioceptive deafferentations of extraocular muscles.

In a previous study, extraocular muscle proprioception (E.O.M.P.) was shown to play an important role in the postnatal development of depth perception: following unilateral or bilateral sections of the ophthalmic branch of the trigeminal nerve (V1th nerve) performed at 6-8 weeks of age, the binocular thresholds were 2 to 3 times higher than in control animals. Since the V1-sections produced no deficits when performed in adults, the temporal limits of a period of susceptibility remained to be determined. In order to assess the lower and upper limits of the period during which these perceptual deficits could be induced, unilateral or bilateral V1-sections were performed in kittens at different ages. Depth perception thresholds were measured by using the jumping stand technique. Sections of the V1 nerve only produced significant impairments of the binocular depth thresholds when performed after 3 weeks of age. They could be observed when unilateral sections were performed at up to 13 weeks of age and with bilateral sections at up to 10 weeks of age. These functional impairments appeared to remain permanently through adult life.

Population coding with motion energy filters: the impact of correlations.

The codes obtained from the responses of large populations of neurons are known as population codes. Several studies have shown that the amount of information conveyed by such codes, and the format of this information, is highly dependent on the pattern of correlations. However, very little is known about the impact of response correlations (as found in actual cortical circuits) on neural coding. To address this problem, we investigated the properties of population codes obtained from motion energy filters, which provide one of the best models for motion selectivity in early visual areas. It is therefore likely that the correlations that arise among energy filters also arise among motion-selective neurons. We adopted an ideal observer approach to analyze filter responses to three sets of images: noisy sine gratings, random dots kinematograms, and images of natural scenes. We report that in our model, the structure of the population code varies with the type of image. We also show that for all sets of images, correlations convey a large fraction of the information: 40% to 90% of the total information. Moreover, ignoring those correlations when decoding leads to considerable information loss-from 50% to 93%, depending on the image type. Finally we show that it is important to consider a large population of motion energy filters in order to see the impact of correlations. Study of pairs of neurons, as is often done experimentally, can underestimate the effect of correlations.

Decoding M1 neurons during multiple finger movements.

We tested several techniques for decoding the activity of primary motor cortex (M1) neurons during movements of single fingers or pairs of fingers. We report that single finger movements can be decoded with >99% accuracy using as few as 30 neurons randomly selected from populations of task-related neurons recorded from the M1 hand representation. This number was reduced to 20 neurons or less when the neurons were not picked randomly but selected on the basis of their information content. We extended techniques for decoding single finger movements to the problem of decoding the simultaneous movement of two fingers. Movements of pairs of fingers were decoded with 90.9% accuracy from 100 neurons. The techniques we used to obtain these results can be applied, not only to movements of single fingers and pairs of fingers as reported here, but also to movements of arbitrary combinations of fingers. The remarkably small number of neurons needed to decode a relatively large repertoire of movements involving either one or two effectors is encouraging for the development of neural prosthetics that will control hand movements.

A neural model of the cortical representation of egocentric distance.

Neurons in the visual cortex of monkeys respond selectively to the disparity between the images in the two eyes. Recent recordings have shown that some of the disparity-selective neurons in the primary visual cortex and the posterior parietal cortex are modulated by the distance of fixation. A population of such gain-modulated, disparity-selective neurons forms a set of basis functions of horizontal disparity and distance of fixation that can be used as an intermediate representation for computing egocentric distance. This distributed representation is consistent with psychophysical studies of human depth perception; in contrast, neurons explicitly tuned to distance are not consistent with how we perceive distance. In a population model that includes noise in the firing rates of neurons, the perceived distance is shown to be the estimate of geometrical distance that minimizes the variance of the estimation.

Information processing with population codes.

Information is encoded in the brain by populations or clusters of cells, rather than by single cells. This encoding strategy is known as population coding. Here we review the standard use of population codes for encoding and decoding information, and consider how population codes can be used to support neural computations such as noise removal and nonlinear mapping. More radical ideas about how population codes may directly represent information about stimulus uncertainty are also discussed.

Spatial transformations in the parietal cortex using basis functions.

Sensorimotor transformations are nonlinear mappings of sensory inputs to motor responses. We explore here the possibility that the responses of single neurons in the parietal cortex serve as basis functions for these transformations. Basis function decomposition is a general method for approximating nonlinear functions that is computationally efficient and well suited for adaptive modification. In particular, the responses of single parietal neurons can be approximated by the product of a Gaussian function of retinal location and a sigmoid function of eye position, called a gain field. A large set of such functions forms a basis set that can be used to perform an arbitrary motor response through a direct projection. We compare this hypothesis with other approaches that are commonly used to model population codes, such as computational maps and vectorial representations. Neither of these alternatives can fully account for the responses of parietal neurons, and they are computationally less efficient for nonlinear transformations. Basis functions also have the advantage of not depending on any coordinate system or reference frame. As a consequence, the position of an object can be represented in multiple reference frames simultaneously, a property consistent with the behavior of hemineglect patients with lesions in the parietal cortex.

A new view of hemineglect based on the response properties of parietal neurones.

Lesion studies of the parietal cortex have led to a wide range of conclusions regarding the coordinate reference frame in which hemineglect is expressed. A model of spatial representation in the parietal cortex has recently been developed in which the position of an object is not encoded in a particular frame of reference, but instead involves neurones computing basis functions of sensory inputs. In this type of representation, a nonlinear sensorimotor transformation of an object is represented in a population of units having the response properties of neurones that are observed in the parietal cortex. A simulated lesion in a basis-function representation was found to replicate three of the most important aspects of hemineglect: (i) the model behaved like parietal patients in line-cancellation and line-bisection experiments; (ii) the deficit affected multiple frames of reference; and (iii) the deficit could be object-centred. These results support the basis-function hypothesis for spatial representations and provide a testable computational theory of hemineglect at the level of single cells.

Egocentric spaw representation in early vision.

Abstract Recent physiological experiments have shown that the responses of many neurons in V1 and V3a are modulated by the direction of gaze. We have developed a neural network model of the hierarchy of maps in visual cortex to explore the hypothesis that visual features are encoded in egocentric (spatio-topic) coordinates at early stages of visual processing. Most psychophysical studies that have attempted to examine this question have concluded that features are represented in retinal coordinates, but the interpretation of these experiments does not preclude the type of retinospatiotopic representation that is embodied in our model. The model also explains why electrical stimulation experiments in visual cortex cannot distinguish between retinal and retinospatiotopic coordinates in the early stages of visual processing. Psychophysical predictions are made for testing the existence of retinospatiotopic representations.

MSTd neuronal basis functions for the population encoding of heading direction.

Basis functions have been extensively used in models of neural computation because they can be combined linearly to approximate any nonlinear functions of the encoded variables. We investigated whether dorsal medial superior temporal (MSTd) area neurons use basis functions to simultaneously encode heading direction, eye position, and the velocity of ocular pursuit. Using optimal linear estimators, we first show that the head-centered and eye-centered position of a focus of expansion (FOE) in optic flow, pursuit direction, and eye position can all be estimated from the single-trial responses of 144 MSTd neurons with an average accuracy of 2-3 degrees, a value consistent with the discrimination thresholds measured in humans and monkeys. We then examined the format of the neural code for the head-centered position of the FOE, eye position, and pursuit direction. The basis function hypothesis predicts that a large majority of cells in MSTd should encode two or more signals simultaneously and combine these signals nonlinearly. Our analysis shows that 95% of the neurons encode two or more signals, whereas 76% code all three signals. Of the 95% of cells encoding two or more signals, 90% show nonlinear interactions between the encoded variables. These findings support the notion that MSTd may use basis functions to represent the FOE in optic flow, eye position, and pursuit.

Statistically efficient estimation using population coding.

Coarse codes are widely used throughout the brain to encode sensory and motor variables. Methods designed to interpret these codes, such as population vector analysis, are either inefficient (the variance of the estimate is much larger than the smallest possible variance) or biologically implausible, like maximum likelihood. Moreover, these methods attempt to compute a scalar or vector estimate of the encoded variable. Neurons are faced with a similar estimation problem. They must read out the responses of the presynaptic neurons, but, by contrast, they typically encode the variable with a further population code rather than as a scalar. We show how a nonlinear recurrent network can be used to perform estimation in a near-optimal way while keeping the estimate in a coarse code format. This work suggests that lateral connections in the cortex may be involved in cleaning up uncorrelated noise among neurons representing similar variables.

Probabilistic interpretation of population codes.

We present a general encoding-decoding framework for interpreting the activity of a population of units. A standard population code interpretation method, the Poisson model, starts from a description as to how a single value of an underlying quantity can generate the activities of each unit in the population. In casting it in the encoding-decoding framework, we find that this model is too restrictive to describe fully the activities of units in population codes in higher processing areas, such as the medial temporal area. Under a more powerful model, the population activity can convey information not only about a single value of some quantity but also about its whole distribution, including its variance, and perhaps even the certainty the system has in the actual presence in the world of the entity generating this quantity. We propose a novel method for forming such probabilistic interpretations of population codes and compare it to the existing method.

Interactions between natural and electrically evoked saccades. III. Is the nonstationarity the result of an integrator not instantaneously reset?

In the monkey, fixed-vector saccades evoked by superior colliculus (SC) stimulation when the animal fixates can be dramatically modified if the stimulation is applied during or immediately after an initial natural saccade. The vector is then deviated in the direction opposite to the displacement just accomplished as if it were compensating for part of the preceding trajectory. Recently, it was suggested that the amplitude of the compensatory deviation is related to the amplitude of the initial saccade linearly, and that the ratio between the two decreases exponentially as stimulation is applied later. These two findings (spatial linearity and temporal nonstationarity) were invoked as evidence for the noninstantaneous resetting of a feedback integrator. Such an integrator is included in most models of saccade generation for the specific purpose of terminating a saccade when it has reached its intended goal. However, the hypothesis of a feedback integrator in the process of being reset implies that the exponential decay of the compensatory deviation is temporally linked to the end of the initial saccade. We analyzed the time course of this decay in stimulation experiments performed at 24 SC sites in two monkeys. The results show that if the start of the exponential decay of compensation is assumed to be linked to the end of the initial saccade, then the relation between the amount of compensatory deviation and the amplitude of the initial saccade is not linear. On the other hand, it is possible to show a linear relation if the measurements of compensatory deviation are made in terms of delay of stimulation from the saccade beginning. We conclude that stimulating the SC just after a visually guided saccade does not seem to test the properties of a feedback integrator. Whether such an integrator is or is not resettable is not likely to be decided by this approach. Conversely, as the nonstationarity of compensation is linked to the beginning of the saccade, the nonstationarity seems to represent a property of an event occurring at saccade onset. We suggest that this event, close to the input of the oculomotor apparatus, is the summation of the visual signal with a damped signal of eye position or displacement.