Learning multisensory integration and coordinate transformation via density estimation.

Sensory processing in the brain includes three key operations: multisensory integration-the task of combining cues into a single estimate of a common underlying stimulus; coordinate transformations-the change of reference frame for a stimulus (e.g., retinotopic to body-centered) effected through knowledge about an intervening variable (e.g., gaze position); and the incorporation of prior information. Statistically optimal sensory processing requires that each of these operations maintains the correct posterior distribution over the stimulus. Elements of this optimality have been demonstrated in many behavioral contexts in humans and other animals, suggesting that the neural computations are indeed optimal. That the relationships between sensory modalities are complex and plastic further suggests that these computations are learned-but how? We provide a principled answer, by treating the acquisition of these mappings as a case of density estimation, a well-studied problem in machine learning and statistics, in which the distribution of observed data is modeled in terms of a set of fixed parameters and a set of latent variables. In our case, the observed data are unisensory-population activities, the fixed parameters are synaptic connections, and the latent variables are multisensory-population activities. In particular, we train a restricted Boltzmann machine with the biologically plausible contrastive-divergence rule to learn a range of neural computations not previously demonstrated under a single approach: optimal integration; encoding of priors; hierarchical integration of cues; learning when not to integrate; and coordinate transformation. The model makes testable predictions about the nature of multisensory representations.

Superlinear population encoding of dynamic hand trajectory in primary motor cortex.

Neural activity in primary motor cortex (MI) is known to correlate with hand position and velocity. Previous descriptions of this tuning have (1) been linear in position or velocity, (2) depended only instantaneously on these signals, and/or (3) not incorporated the effects of interneuronal dependencies on firing rate. We show here that many MI cells encode a superlinear function of the full time-varying hand trajectory. Approximately 20% of MI cells carry information in the hand trajectory beyond just the position, velocity, and acceleration at a single time lag. Moreover, approximately one-third of MI cells encode the trajectory in a significantly superlinear manner; as one consequence, even small position changes can dramatically modulate the gain of the velocity tuning of MI cells, in agreement with recent psychophysical evidence. We introduce a compact nonlinear "preferred trajectory" model that predicts the complex structure of the spatiotemporal tuning functions described in previous work. Finally, observing the activity of neighboring cells in the MI network significantly increases the predictability of the firing rate of a single MI cell; however, we find interneuronal dependencies in MI to be much more locked to external kinematic parameters than those described recently in the hippocampus. Nevertheless, this neighbor activity is approximately as informative as the hand velocity, supporting the view that neural encoding in MI is best understood at a population level.

Reliability of signals from a chronically implanted, silicon-based electrode array in non-human primate primary motor cortex.

Multiple-electrode arrays are valuable both as a research tool and as a sensor for neuromotor prosthetic devices, which could potentially restore voluntary motion and functional independence to paralyzed humans. Long-term array reliability is an important requirement for these applications. Here, we demonstrate the reliability of a regular array of 100 microelectrodes to obtain neural recordings from primary motor cortex (MI) of monkeys for at least three months and up to 1.5 years. We implanted Bionic (Cyberkinetics, Inc., Foxboro, MA) silicon probe arrays in MI of three Macaque monkeys. Neural signals were recorded during performance of an eight-direction, push-button task. Recording reliability was evaluated for 18, 35, or 51 sessions distributed over 83, 179, and 569 days after implantation, respectively, using qualitative and quantitative measures. A four-point signal quality scale was defined based on the waveform amplitude relative to noise. A single observer applied this scale to score signal quality for each electrode. A mean of 120 (+/- 17.6 SD), 146 (+/- 7.3), and 119 (+/- 16.9) neural-like waveforms were observed from 65-85 electrodes across subjects for all recording sessions of which over 80% were of high quality. Quantitative measures demonstrated that waveforms had signal-to-noise ratio (SNR) up to 20 with maximum peak-to-peak amplitude of over 1200 microv with a mean SNR of 4.8 for signals ranked as high quality. Mean signal quality did not change over the duration of the evaluation period (slope 0.001, 0.0068 and 0.03; NS). By contrast, neural waveform shape varied between, but not within days in all animals, suggesting a shifting population of recorded neurons over time. Arm-movement related modulation was common and 66% of all recorded neurons were tuned to reach direction. The ability for the array to record neural signals from parietal cortex was also established. These results demonstrate that neural recordings that can provide movement related signals for neural prostheses, as well as for fundamental research applications, can be reliably obtained for long time periods using a monolithic microelectrode array in primate MI and potentially from other cortical areas as well.

Development of a chipscale integrated microelectrode/microelectronic device for brain implantable neuroengineering applications.

An ultralow power analog CMOS chip and a silicon based microelectrode array have been fully integrated to a microminiaturized "neuroport" for brain implantable neuroengineering applications. The CMOS integrated circuit (IC) includes preamplifier and multiplexing circuitry, and a hybrid flip-chip bonding technique was developed to fabricate a functional, encapsulated microminiaturized neuroprobe device. Our neuroport has been evaluated using various methods, including pseudospike detection and local excitation measurement, and showed suitable characteristics for recording neural activities. As a proof-of-concept demonstration, we have measured local field potentials from thalamocortical brain slices of rats, suggesting that the new neuroport can form a prime platform for the development of a microminiaturized neural interface to the brain in a single implantable unit. An alternative power delivery scheme using photovoltaic power converter, and an encapsulation strategy for chronic implantation are also discussed.

Statistical encoding model for a primary motor cortical brain-machine interface.

A number of studies of the motor system suggest that the majority of primary motor cortical neurons represent simple movement-related kinematic and dynamic quantities in their time-varying activity patterns. An example of such an encoding relationship is the cosine tuning of firing rate with respect to the direction of hand motion. We present a systematic development of statistical encoding models for movement-related motor neurons using multielectrode array recordings during a two-dimensional (2-D) continuous pursuit-tracking task. Our approach avoids massive averaging of responses by utilizing 2-D normalized occupancy plots, cascaded linear-nonlinear (LN) system models and a method for describing variability in discrete random systems. We found that the expected firing rate of most movement-related motor neurons is related to the kinematic values by a linear transformation, with a significant nonlinear distortion in about 1/3 of the neurons. The measured variability of the neural responses is markedly non-Poisson in many neurons and is well captured by a "normalized-Gaussian" statistical model that is defined and introduced here. The statistical model is seamlessly integrated into a nearly-optimal recursive method for decoding movement from neural responses based on a Sequential Monte Carlo filter.

Robust, automatic spike sorting using mixtures of multivariate t-distributions.

A number of recent methods developed for automatic classification of multiunit neural activity rely on a Gaussian model of the variability of individual waveforms and the statistical methods of Gaussian mixture decomposition. Recent evidence has shown that the Gaussian model does not accurately capture the multivariate statistics of the waveform samples' distribution. We present further data demonstrating non-Gaussian statistics, and show that the multivariate t-distribution, a wide-tailed family of distributions, provides a significantly better fit to the true statistics. We introduce an adaptation of a new expectation-maximization based competitive mixture decomposition algorithm and show that it efficiently and reliably performs mixture decomposition of t-distributions. Our algorithm determines the number of units in multiunit neural recordings, even in the presence of significant noise contamination resulting from random threshold crossings and overlapping spikes.

A point process framework for relating neural spiking activity to spiking history, neural ensemble, and extrinsic covariate effects.

Multiple factors simultaneously affect the spiking activity of individual neurons. Determining the effects and relative importance of these factors is a challenging problem in neurophysiology. We propose a statistical framework based on the point process likelihood function to relate a neuron's spiking probability to three typical covariates: the neuron's own spiking history, concurrent ensemble activity, and extrinsic covariates such as stimuli or behavior. The framework uses parametric models of the conditional intensity function to define a neuron's spiking probability in terms of the covariates. The discrete time likelihood function for point processes is used to carry out model fitting and model analysis. We show that, by modeling the logarithm of the conditional intensity function as a linear combination of functions of the covariates, the discrete time point process likelihood function is readily analyzed in the generalized linear model (GLM) framework. We illustrate our approach for both GLM and non-GLM likelihood functions using simulated data and multivariate single-unit activity data simultaneously recorded from the motor cortex of a monkey performing a visuomotor pursuit-tracking task. The point process framework provides a flexible, computationally efficient approach for maximum likelihood estimation, goodness-of-fit assessment, residual analysis, model selection, and neural decoding. The framework thus allows for the formulation and analysis of point process models of neural spiking activity that readily capture the simultaneous effects of multiple covariates and enables the assessment of their relative importance.

Instant neural control of a movement signal.

The activity of motor cortex (MI) neurons conveys movement intent sufficiently well to be used as a control signal to operate artificial devices, but until now this has called for extensive training or has been confined to a limited movement repertoire. Here we show how activity from a few (7-30) MI neurons can be decoded into a signal that a monkey is able to use immediately to move a computer cursor to any new position in its workspace (14 degrees x 14 degrees visual angle). Our results, which are based on recordings made by an electrode array that is suitable for human use, indicate that neurally based control of movement may eventually be feasible in paralysed humans.

Spatiotemporal tuning of motor cortical neurons for hand position and velocity.

A pursuit-tracking task (PTT) and multielectrode recordings were used to investigate the spatiotemporal encoding of hand position and velocity in primate primary motor cortex (MI). Continuous tracking of a randomly moving visual stimulus provided a broad sample of velocity and position space, reduced statistical dependencies between kinematic variables, and minimized the nonstationarities that are found in typical "step-tracking" tasks. These statistical features permitted the application of signal-processing and information-theoretic tools for the analysis of neural encoding. The multielectrode method allowed for the comparison of tuning functions among simultaneously recorded cells. During tracking, MI neurons showed heterogeneity of position and velocity coding, with markedly different temporal dynamics for each. Velocity-tuned neurons were approximately sinusoidally tuned for direction, with linear speed scaling; other cells showed sinusoidal tuning for position, with linear scaling by distance. Velocity encoding led behavior by about 100 ms for most cells, whereas position tuning was more broadly distributed, with leads and lags suggestive of both feedforward and feedback coding. Individual cells encoded velocity and position weakly, with comparable amounts of information about each. Linear regression methods confirmed that random, 2-D hand trajectories can be reconstructed from the firing of small ensembles of randomly selected neurons (3-19 cells) within the MI arm area. These findings demonstrate that MI carries information about evolving hand trajectory during visually guided pursuit tracking, including information about arm position both during and after its specification. However, the reconstruction methods used here capture only the low-frequency components of movement during the PTT. Hand motion signals appear to be represented as a distributed code in which diverse information about position and velocity is available within small regions of MI.