A high-performance brain-computer interface.

Recent studies have demonstrated that monkeys and humans can use signals from the brain to guide computer cursors. Brain-computer interfaces (BCIs) may one day assist patients suffering from neurological injury or disease, but relatively low system performance remains a major obstacle. In fact, the speed and accuracy with which keys can be selected using BCIs is still far lower than for systems relying on eye movements. This is true whether BCIs use recordings from populations of individual neurons using invasive electrode techniques or electroencephalogram recordings using less- or non-invasive techniques. Here we present the design and demonstration, using electrode arrays implanted in monkey dorsal premotor cortex, of a manyfold higher performance BCI than previously reported. These results indicate that a fast and accurate key selection system, capable of operating with a range of keyboard sizes, is possible (up to 6.5 bits per second, or approximately 15 words per minute, with 96 electrodes). The highest information throughput is achieved with unprecedentedly brief neural recordings, even as recording quality degrades over time. These performance results and their implications for system design should substantially increase the clinical viability of BCIs in humans.

Roles of monkey premotor neuron classes in movement preparation and execution.

Dorsal premotor cortex (PMd) is known to be involved in the planning and execution of reaching movements. However, it is not understood how PMd plan activity-often present in the very same neurons that respond during movement-is prevented from itself producing movement. We investigated whether inhibitory interneurons might "gate" output from PMd, by maintaining high levels of inhibition during planning and reducing inhibition during execution. Recently developed methods permit distinguishing interneurons from pyramidal neurons using extracellular recordings. We extend these methods here for use with chronically implanted multi-electrode arrays. We then applied these methods to single- and multi-electrode recordings in PMd of two monkeys performing delayed-reach tasks. Responses of putative interneurons were not generally in agreement with the hypothesis that they act to gate output from the area: in particular it was not the case that interneurons tended to reduce their firing rates around the time of movement. In fact, interneurons increased their rates more than putative pyramidal neurons during both the planning and movement epochs. The two classes of neurons also differed in a number of other ways, including greater modulation across conditions for interneurons, and interneurons more frequently exhibiting increases in firing rate during movement planning and execution. These findings provide novel information about the greater responsiveness of putative PMd interneurons in motor planning and execution and suggest that we may need to consider new possibilities for how planning activity is structured such that it does not itself produce movement.

Single-neuron stability during repeated reaching in macaque premotor cortex.

Some movements that animals and humans make are highly stereotyped, repeated with little variation. The patterns of neural activity associated with repeats of a movement may be highly similar, or the same movement may arise from different patterns of neural activity, if the brain exploits redundancies in the neural projections to muscles. We examined the stability of the relationship between neural activity and behavior. We asked whether the variability in neural activity that we observed during repeated reaching was consistent with a noisy but stable relationship, or with a changing relationship, between neural activity and behavior. Monkeys performed highly similar reaches under tight behavioral control, while many neurons in the dorsal aspect of premotor cortex and the primary motor cortex were simultaneously monitored for several hours. Neural activity was predominantly stable over time in all measured properties: firing rate, directional tuning, and contribution to a decoding model that predicted kinematics from neural activity. The small changes in neural activity that we did observe could be accounted for primarily by subtle changes in behavior. We conclude that the relationship between neural activity and practiced behavior is reasonably stable, at least on timescales of minutes up to 48 h. This finding has significant implications for the design of neural prosthetic systems because it suggests that device recalibration need not be overly frequent, It also has implications for studies of neural plasticity because a stable baseline permits identification of nonstationary shifts.

Cortical neural prosthesis performance improves when eye position is monitored.

Neural prostheses that extract signals directly from cortical neurons have recently become feasible as assistive technologies for tetraplegic individuals. Significant effort toward improving the performance of these systems is now warranted. A simple technique that can improve prosthesis performance is to account for the direction of gaze in the operation of the prosthesis. This proposal stems from recent discoveries that the direction of gaze influences neural activity in several areas that are commonly targeted for electrode implantation in neural prosthetics. Here, we first demonstrate that neural prosthesis performance does improve when eye position is taken into account. We then show that eye position can be estimated directly from neural activity, and thus performance gains can be realized even without a device that tracks eye position.

Neural variability in premotor cortex provides a signature of motor preparation.

We present experiments and analyses designed to test the idea that firing rates in premotor cortex become optimized during motor preparation, approaching their ideal values over time. We measured the across-trial variability of neural responses in dorsal premotor cortex of three monkeys performing a delayed-reach task. Such variability was initially high, but declined after target onset, and was maintained at a rough plateau during the delay. An additional decline was observed after the go cue. Between target onset and movement onset, variability declined by an average of 34%. This decline in variability was observed even when mean firing rate changed little. We hypothesize that this effect is related to the progress of motor preparation. In this interpretation, firing rates are initially variable across trials but are brought, over time, to their "appropriate" values, becoming consistent in the process. Consistent with this hypothesis, reaction times were longer if the go cue was presented shortly after target onset, when variability was still high, and were shorter if the go cue was presented well after target onset, when variability had fallen to its plateau. A similar effect was observed for the natural variability in reaction time: longer (shorter) reaction times tended to occur on trials in which firing rates were more (less) variable. These results reveal a remarkable degree of temporal structure in the variability of cortical neurons. The relationship with reaction time argues that the changes in variability approximately track the progress of motor preparation.

Free-paced high-performance brain-computer interfaces.

Neural prostheses aim to improve the quality of life of severely disabled patients by translating neural activity into control signals for guiding prosthetic devices or computer cursors. We recently demonstrated that plan activity from premotor cortex, which specifies the endpoint of the upcoming arm movement, can be used to swiftly and accurately guide computer cursors to the desired target locations. However, these systems currently require additional, non-neural information to specify when plan activity is present. We report here the design and performance of state estimator algorithms for automatically detecting the presence of plan activity using neural activity alone. Prosthesis performance was nearly as good when state estimation was used as when perfect plan timing information was provided separately ( approximately 5 percentage points lower, when using 200 ms of plan activity). These results strongly suggest that a completely neurally-driven high-performance brain-computer interface is possible.

HermesB: a continuous neural recording system for freely behaving primates.

Chronically implanted electrode arrays have enabled a broad range of advances in basic electrophysiology and neural prosthetics. Those successes motivate new experiments, particularly, the development of prototype implantable prosthetic processors for continuous use in freely behaving subjects, both monkeys and humans. However, traditional experimental techniques require the subject to be restrained, limiting both the types and duration of experiments. In this paper, we present a dual-channel, battery-powered neural recording system with an integrated three-axis accelerometer for use with chronically implanted electrode arrays in freely behaving primates. The recording system called HermesB, is self-contained, autonomous, programmable, and capable of recording broadband neural (sampled at 30 kS/s) and acceleration data to a removable compact flash card for up to 48 h. We have collected long-duration data sets with HermesB from an adult macaque monkey which provide insight into time scales and free behaviors inaccessible under traditional experiments. Variations in action potential shape and root-mean square (RMS) noise are observed across a range of time scales. The peak-to-peak voltage of action potentials varied by up to 30% over a 24-h period including step changes in waveform amplitude (up to 25%) coincident with high acceleration movements of the head. These initial results suggest that spike-sorting algorithms can no longer assume stable neural signals and will need to transition to adaptive signal processing methodologies to maximize performance. During physically active periods (defined by head-mounted accelerometer), significantly reduced 5-25-Hz local field potential (LFP) power and increased firing rate variability were observed. Using a threshold fit to LFP power, 93% of 403 5-min recording blocks were correctly classified as active or inactive, potentially providing an efficient tool for identifying different behavioral contexts in prosthetic applications. These results demonstrate the utility of the HermesB system and motivate using this type of system to advance neural prosthetics and electrophysiological experiments.

Power feasibility of implantable digital spike sorting circuits for neural prosthetic systems.

A new class of neural prosthetic systems aims to assist disabled patients by translating cortical neural activity into control signals for prosthetic devices. Based on the success of proof-of-concept systems in the laboratory, there is now considerable interest in increasing system performance and creating implantable electronics for use in clinical systems. A critical question that impacts system performance and the overall architecture of these systems is whether it is possible to identify the neural source of each action potential (spike sorting) in real-time and with low power. Low power is essential both for power supply considerations and heat dissipation in the brain. In this paper we report that state-of-the-art spike sorting algorithms are not only feasible using modern complementary metal oxide semiconductor very large scale integration processes, but may represent the best option for extracting large amounts of data in implantable neural prosthetic interfaces.

Single-trial neural correlates of arm movement preparation.

The process by which neural circuitry in the brain plans and executes movements is not well understood. Until recently, most available data were limited either to single-neuron electrophysiological recordings or to measures of aggregate field or metabolism. Neither approach reveals how individual neurons' activities are coordinated within the population, and thus inferences about how the neural circuit forms a motor plan for an upcoming movement have been indirect. Here we build on recent advances in the measurement and description of population activity to frame and test an "initial condition hypothesis" of arm movement preparation and initiation. This hypothesis leads to a model in which the timing of movements may be predicted on each trial using neurons' moment-by-moment firing rates and rates of change of those rates. Using simultaneous microelectrode array recordings from premotor cortex of monkeys performing delayed-reach movements, we compare such single-trial predictions to those of other theories. We show that our model can explain approximately 4-fold more arm-movement reaction-time variance than the best alternative method. Thus, the initial condition hypothesis elucidates a view of the relationship between single-trial preparatory neural population dynamics and single-trial behavior.

Stimulus onset quenches neural variability: a widespread cortical phenomenon.

Neural responses are typically characterized by computing the mean firing rate, but response variability can exist across trials. Many studies have examined the effect of a stimulus on the mean response, but few have examined the effect on response variability. We measured neural variability in 13 extracellularly recorded datasets and one intracellularly recorded dataset from seven areas spanning the four cortical lobes in monkeys and cats. In every case, stimulus onset caused a decline in neural variability. This occurred even when the stimulus produced little change in mean firing rate. The variability decline was observed in membrane potential recordings, in the spiking of individual neurons and in correlated spiking variability measured with implanted 96-electrode arrays. The variability decline was observed for all stimuli tested, regardless of whether the animal was awake, behaving or anaesthetized. This widespread variability decline suggests a rather general property of cortex, that its state is stabilized by an input.

Factor-analysis methods for higher-performance neural prostheses.

Neural prostheses aim to provide treatment options for individuals with nervous-system disease or injury. It is necessary, however, to increase the performance of such systems before they can be clinically viable for patients with motor dysfunction. One performance limitation is the presence of correlated trial-to-trial variability that can cause neural responses to wax and wane in concert as the subject is, for example, more attentive or more fatigued. If a system does not properly account for this variability, it may mistakenly interpret such variability as an entirely different intention by the subject. We report here the design and characterization of factor-analysis (FA)-based decoding algorithms that can contend with this confound. We characterize the decoders (classifiers) on experimental data where monkeys performed both a real reach task and a prosthetic cursor task while we recorded from 96 electrodes implanted in dorsal premotor cortex. The decoder attempts to infer the underlying factors that comodulate the neurons' responses and can use this information to substantially lower error rates (one of eight reach endpoint predictions) by 150 ms, although still advantageous at shorter timescales, that Gaussian-based algorithms performed better than the analogous Poisson-based algorithms and that the FA algorithm is robust even with a limited amount of training data. We propose that FA-based methods are effective in modeling correlated trial-to-trial neural variability and can be used to substantially increase overall prosthetic system performance.

Gaussian-process factor analysis for low-dimensional single-trial analysis of neural population activity.

We consider the problem of extracting smooth, low-dimensional neural trajectories that summarize the activity recorded simultaneously from many neurons on individual experimental trials. Beyond the benefit of visualizing the high-dimensional, noisy spiking activity in a compact form, such trajectories can offer insight into the dynamics of the neural circuitry underlying the recorded activity. Current methods for extracting neural trajectories involve a two-stage process: the spike trains are first smoothed over time, then a static dimensionality-reduction technique is applied. We first describe extensions of the two-stage methods that allow the degree of smoothing to be chosen in a principled way and that account for spiking variability, which may vary both across neurons and across time. We then present a novel method for extracting neural trajectories-Gaussian-process factor analysis (GPFA)-which unifies the smoothing and dimensionality-reduction operations in a common probabilistic framework. We applied these methods to the activity of 61 neurons recorded simultaneously in macaque premotor and motor cortices during reach planning and execution. By adopting a goodness-of-fit metric that measures how well the activity of each neuron can be predicted by all other recorded neurons, we found that the proposed extensions improved the predictive ability of the two-stage methods. The predictive ability was further improved by going to GPFA. From the extracted trajectories, we directly observed a convergence in neural state during motor planning, an effect that was shown indirectly by previous studies. We then show how such methods can be a powerful tool for relating the spiking activity across a neural population to the subject's behavior on a single-trial basis. Finally, to assess how well the proposed methods characterize neural population activity when the underlying time course is known, we performed simulations that revealed that GPFA performed tens of percent better than the best two-stage method.

Detecting neural-state transitions using hidden Markov models for motor cortical prostheses.

Neural prosthetic interfaces use neural activity related to the planning and perimovement epochs of arm reaching to afford brain-directed control of external devices. Previous research has primarily centered on accurately decoding movement intention from either plan or perimovement activity, but has assumed that temporal boundaries between these epochs are known to the decoding system. In this work, we develop a technique to automatically differentiate between baseline, plan, and perimovement epochs of neural activity. Specifically, we use a generative model of neural activity to capture how neural activity varies between these three epochs. Our approach is based on a hidden Markov model (HMM), in which the latent variable (state) corresponds to the epoch of neural activity, coupled with a state-dependent Poisson firing model. Using an HMM, we demonstrate that the time of transition from baseline to plan epochs, a transition in neural activity that is not accompanied by any external behavior changes, can be detected using a threshold on the a posteriori HMM state probabilities. Following detection of the plan epoch, we show that the intended target of a center-out movement can be detected about as accurately as that by a maximum-likelihood estimator using a window of known plan activity. In addition, we demonstrate that our HMM can detect transitions in neural activity corresponding to targets not found in training data. Thus the HMM technique for automatically detecting transitions between epochs of neural activity enables prosthetic interfaces that can operate autonomously.

The use of a virtual integration environment for the real-time implementation of neural decode algorithms.

We have developed a virtual integration environment (VIE) for the development of neural prosthetic systems. This paper, the second of two companion articles, describes the use of the VIE as a common platform for the implementation of neural decode algorithms. In this paper, a linear filter decode and a recursive Bayesian algorithm are implemented as separate signal analysis modules of the VIE for the real-time decode of end effector trajectory. The process of implementing each algorithm is described and the real-time behavior as well as computational cost for each algorithm is examined. This is the first report of the real-time implementation of the Mixture of Trajectory Models decode [10]. These real-time algorithms can be easily interfaced with pre-existing modules of the VIE to control simulated and real devices.

Reference frames for reach planning in macaque dorsal premotor cortex.

When a human or animal reaches out to grasp an object, the brain rapidly computes a pattern of muscular contractions that can acquire the target. This computation involves a reference frame transformation because the target's position is initially available only in a visual reference frame, yet the required control signal is a set of commands to the musculature. One of the core brain areas involved in visually guided reaching is the dorsal aspect of the premotor cortex (PMd). Using chronically implanted electrode arrays in two Rhesus monkeys, we studied the contributions of PMd to the reference frame transformation for reaching. PMd neurons are influenced by the locations of reach targets relative to both the arm and the eyes. Some neurons encode reach goals using limb-centered reference frames, whereas others employ eye-centered reference fames. Some cells encode reach goals in a reference frame best described by the combined position of the eyes and hand. In addition to neurons like these where a reference frame could be identified, PMd also contains cells that are influenced by both the eye- and limb-centered locations of reach goals but for which a distinct reference frame could not be determined. We propose two interpretations for these neurons. First, they may encode reach goals using a reference frame we did not investigate, such as intrinsic reference frames. Second, they may not be adequately characterized by any reference frame.

Mixture of trajectory models for neural decoding of goal-directed movements.

Probabilistic decoding techniques have been used successfully to infer time-evolving physical state, such as arm trajectory or the path of a foraging rat, from neural data. A vital element of such decoders is the trajectory model, expressing knowledge about the statistical regularities of the movements. Unfortunately, trajectory models that both 1) accurately describe the movement statistics and 2) admit decoders with relatively low computational demands can be hard to construct. Simple models are computationally inexpensive, but often inaccurate. More complex models may gain accuracy, but at the expense of higher computational cost, hindering their use for real-time decoding. Here, we present a new general approach to defining trajectory models that simultaneously meets both requirements. The core idea is to combine simple trajectory models, each accurate within a limited regime of movement, in a probabilistic mixture of trajectory models (MTM). We demonstrate the utility of the approach by using an MTM decoder to infer goal-directed reaching movements to multiple discrete goals from multi-electrode neural data recorded in monkey motor and premotor cortex. Compared with decoders using simpler trajectory models, the MTM decoder reduced the decoding error by 38 (48) percent in two monkeys using 98 (99) units, without a necessary increase in running time. When available, prior information about the identity of the upcoming reach goal can be incorporated in a principled way, further reducing the decoding error by 20 (11) percent. Taken together, these advances should allow prosthetic cursors or limbs to be moved more accurately toward intended reach goals.

Preparatory activity in premotor and motor cortex reflects the speed of the upcoming reach.

Neurons in premotor and motor cortex show preparatory activity during an instructed-delay task. It has been suggested that such activity primarily reflects visuospatial aspects of the movement, such as target location or reach direction and extent. We asked whether a more dynamic feature, movement speed, is also reflected. Two monkeys were trained to reach at different speeds ("slow" or "fast," peak speed being approximately 50-100% higher for the latter) depending on target color. Targets were presented in seven directions and at two distances. Of 95 neurons with tuned delay-period activity, 95, 78, and 94% showed a significant influence of direction, distance, and instructed speed, respectively. Average peak modulations with respect to direction, distance and speed were 18, 10, and 11 spikes/s. Although robust, modulations of firing rate with target direction were not necessarily invariant: for 45% of neurons, the preferred direction depended significantly on target distance and/or instructed speed. We collected an additional dataset, examining in more detail the effect of target distance (5 distances from 3 to 12 cm in 2 directions). Of 41 neurons with tuned delay-period activity, 85, 83, and 98% showed a significant impact of direction, distance, and instructed speed. Statistical interactions between the effects of distance and instructed speed were common, but it was nevertheless clear that distance "tuning" was not in general a simple consequence of speed tuning. We conclude that delay-period preparatory activity robustly reflects a nonspatial aspect of the upcoming reach. However, it is unclear whether the recorded neural responses conform to any simple reference frame, intrinsic or extrinsic.

Neural recording stability of chronic electrode arrays in freely behaving primates.

Chronically implanted electrode arrays have enabled a broad range of advances, particularly in the field of neural prosthetics. Those successes motivate development of prototype implantable prosthetic processors for long duration, continuous use in freely behaving subjects. However, traditional experimental protocols have provided limited information regarding the stability of the electrode arrays and their neural recordings. In this paper we present preliminary results derived from long duration neural recordings in a freely behaving primate which show variations in action potential shape and RMS noise across a range of time scales. These preliminary results suggest that spike sorting algorithms can no longer assume stable neural signals and will need to transition to adaptive signal processing methodologies to maximize performance.

Multiday electrophysiological recordings from freely behaving primates.

Continuous multiday broadband neural data provide a means for observing effects at fine timescales over long periods. In this paper we present analyses on such data sets to demonstrate neural correlates for physically active and inactive time periods, as defined by the response of a head-mounted accelerometer. During active periods, we found that 5-25 Hz local field potential (LFP) power was significantly reduced, firing rate variability increased, and firing rates have greater temporal correlation. Using a single threshold fit to LFP power, 93% of the 403 5 minute blocks tested were correctly classified as active or inactive (as labeled by thresholding each block's maximal accelerometer magnitude). These initial results motivate the use of such data sets for testing neural prosthetics systems and for finding the neural correlates of natural behaviors.

An autonomous, broadband, multi-channel neural recording system for freely behaving primates.

Successful laboratory proof-of-concept experiments with neural prosthetic systems motivate continued algorithm and hardware development. For these efforts to move beyond traditional fixed laboratory setups, new tools are needed to enable broadband, multi-channel, long duration neural recording from freely behaving primates. In this paper we present a dual-channel, battery powered, neural recording system with integrated 3-axis accelerometer for use with chronically implanted electrode arrays. The recording system, called HermesB, is self-contained, autonomous, programmable and capable of recording broadband neural and head acceleration data to a removable compact flash card for up to 48 hours.