A high-performance brain-computer interface.

Recent studies have demonstrated that monkeys and humans can use signals from the brain to guide computer cursors. Brain-computer interfaces (BCIs) may one day assist patients suffering from neurological injury or disease, but relatively low system performance remains a major obstacle. In fact, the speed and accuracy with which keys can be selected using BCIs is still far lower than for systems relying on eye movements. This is true whether BCIs use recordings from populations of individual neurons using invasive electrode techniques or electroencephalogram recordings using less- or non-invasive techniques. Here we present the design and demonstration, using electrode arrays implanted in monkey dorsal premotor cortex, of a manyfold higher performance BCI than previously reported. These results indicate that a fast and accurate key selection system, capable of operating with a range of keyboard sizes, is possible (up to 6.5 bits per second, or approximately 15 words per minute, with 96 electrodes). The highest information throughput is achieved with unprecedentedly brief neural recordings, even as recording quality degrades over time. These performance results and their implications for system design should substantially increase the clinical viability of BCIs in humans.

A central source of movement variability.

Movements are universally, sometimes frustratingly, variable. When such variability causes error, we typically assume that something went wrong during the movement. The same assumption is made by recent and influential models of motor control. These posit that the principal limit on repeatable performance is neuromuscular noise that corrupts movement as it occurs. An alternative hypothesis is that movement variability arises before movements begin, during motor preparation. We examined this possibility directly by recording the preparatory activity of single cortical neurons during a highly practiced reach task. Small variations in preparatory neural activity were predictive of small variations in the upcoming reach. Effect magnitudes were such that at least half of the observed movement variability likely had its source during motor preparation. Thus, even for a highly practiced task, the ability to repeatedly plan the same movement limits our ability to repeatedly execute the same movement.

Roles of monkey premotor neuron classes in movement preparation and execution.

Dorsal premotor cortex (PMd) is known to be involved in the planning and execution of reaching movements. However, it is not understood how PMd plan activity-often present in the very same neurons that respond during movement-is prevented from itself producing movement. We investigated whether inhibitory interneurons might "gate" output from PMd, by maintaining high levels of inhibition during planning and reducing inhibition during execution. Recently developed methods permit distinguishing interneurons from pyramidal neurons using extracellular recordings. We extend these methods here for use with chronically implanted multi-electrode arrays. We then applied these methods to single- and multi-electrode recordings in PMd of two monkeys performing delayed-reach tasks. Responses of putative interneurons were not generally in agreement with the hypothesis that they act to gate output from the area: in particular it was not the case that interneurons tended to reduce their firing rates around the time of movement. In fact, interneurons increased their rates more than putative pyramidal neurons during both the planning and movement epochs. The two classes of neurons also differed in a number of other ways, including greater modulation across conditions for interneurons, and interneurons more frequently exhibiting increases in firing rate during movement planning and execution. These findings provide novel information about the greater responsiveness of putative PMd interneurons in motor planning and execution and suggest that we may need to consider new possibilities for how planning activity is structured such that it does not itself produce movement.

Single-neuron stability during repeated reaching in macaque premotor cortex.

Some movements that animals and humans make are highly stereotyped, repeated with little variation. The patterns of neural activity associated with repeats of a movement may be highly similar, or the same movement may arise from different patterns of neural activity, if the brain exploits redundancies in the neural projections to muscles. We examined the stability of the relationship between neural activity and behavior. We asked whether the variability in neural activity that we observed during repeated reaching was consistent with a noisy but stable relationship, or with a changing relationship, between neural activity and behavior. Monkeys performed highly similar reaches under tight behavioral control, while many neurons in the dorsal aspect of premotor cortex and the primary motor cortex were simultaneously monitored for several hours. Neural activity was predominantly stable over time in all measured properties: firing rate, directional tuning, and contribution to a decoding model that predicted kinematics from neural activity. The small changes in neural activity that we did observe could be accounted for primarily by subtle changes in behavior. We conclude that the relationship between neural activity and practiced behavior is reasonably stable, at least on timescales of minutes up to 48 h. This finding has significant implications for the design of neural prosthetic systems because it suggests that device recalibration need not be overly frequent, It also has implications for studies of neural plasticity because a stable baseline permits identification of nonstationary shifts.

Cortical neural prosthesis performance improves when eye position is monitored.

Neural prostheses that extract signals directly from cortical neurons have recently become feasible as assistive technologies for tetraplegic individuals. Significant effort toward improving the performance of these systems is now warranted. A simple technique that can improve prosthesis performance is to account for the direction of gaze in the operation of the prosthesis. This proposal stems from recent discoveries that the direction of gaze influences neural activity in several areas that are commonly targeted for electrode implantation in neural prosthetics. Here, we first demonstrate that neural prosthesis performance does improve when eye position is taken into account. We then show that eye position can be estimated directly from neural activity, and thus performance gains can be realized even without a device that tracks eye position.

Free-paced high-performance brain-computer interfaces.

Neural prostheses aim to improve the quality of life of severely disabled patients by translating neural activity into control signals for guiding prosthetic devices or computer cursors. We recently demonstrated that plan activity from premotor cortex, which specifies the endpoint of the upcoming arm movement, can be used to swiftly and accurately guide computer cursors to the desired target locations. However, these systems currently require additional, non-neural information to specify when plan activity is present. We report here the design and performance of state estimator algorithms for automatically detecting the presence of plan activity using neural activity alone. Prosthesis performance was nearly as good when state estimation was used as when perfect plan timing information was provided separately ( approximately 5 percentage points lower, when using 200 ms of plan activity). These results strongly suggest that a completely neurally-driven high-performance brain-computer interface is possible.

HermesB: a continuous neural recording system for freely behaving primates.

Chronically implanted electrode arrays have enabled a broad range of advances in basic electrophysiology and neural prosthetics. Those successes motivate new experiments, particularly, the development of prototype implantable prosthetic processors for continuous use in freely behaving subjects, both monkeys and humans. However, traditional experimental techniques require the subject to be restrained, limiting both the types and duration of experiments. In this paper, we present a dual-channel, battery-powered neural recording system with an integrated three-axis accelerometer for use with chronically implanted electrode arrays in freely behaving primates. The recording system called HermesB, is self-contained, autonomous, programmable, and capable of recording broadband neural (sampled at 30 kS/s) and acceleration data to a removable compact flash card for up to 48 h. We have collected long-duration data sets with HermesB from an adult macaque monkey which provide insight into time scales and free behaviors inaccessible under traditional experiments. Variations in action potential shape and root-mean square (RMS) noise are observed across a range of time scales. The peak-to-peak voltage of action potentials varied by up to 30% over a 24-h period including step changes in waveform amplitude (up to 25%) coincident with high acceleration movements of the head. These initial results suggest that spike-sorting algorithms can no longer assume stable neural signals and will need to transition to adaptive signal processing methodologies to maximize performance. During physically active periods (defined by head-mounted accelerometer), significantly reduced 5-25-Hz local field potential (LFP) power and increased firing rate variability were observed. Using a threshold fit to LFP power, 93% of 403 5-min recording blocks were correctly classified as active or inactive, potentially providing an efficient tool for identifying different behavioral contexts in prosthetic applications. These results demonstrate the utility of the HermesB system and motivate using this type of system to advance neural prosthetics and electrophysiological experiments.

Single-trial neural correlates of arm movement preparation.

The process by which neural circuitry in the brain plans and executes movements is not well understood. Until recently, most available data were limited either to single-neuron electrophysiological recordings or to measures of aggregate field or metabolism. Neither approach reveals how individual neurons' activities are coordinated within the population, and thus inferences about how the neural circuit forms a motor plan for an upcoming movement have been indirect. Here we build on recent advances in the measurement and description of population activity to frame and test an "initial condition hypothesis" of arm movement preparation and initiation. This hypothesis leads to a model in which the timing of movements may be predicted on each trial using neurons' moment-by-moment firing rates and rates of change of those rates. Using simultaneous microelectrode array recordings from premotor cortex of monkeys performing delayed-reach movements, we compare such single-trial predictions to those of other theories. We show that our model can explain approximately 4-fold more arm-movement reaction-time variance than the best alternative method. Thus, the initial condition hypothesis elucidates a view of the relationship between single-trial preparatory neural population dynamics and single-trial behavior.

Factor-analysis methods for higher-performance neural prostheses.

Neural prostheses aim to provide treatment options for individuals with nervous-system disease or injury. It is necessary, however, to increase the performance of such systems before they can be clinically viable for patients with motor dysfunction. One performance limitation is the presence of correlated trial-to-trial variability that can cause neural responses to wax and wane in concert as the subject is, for example, more attentive or more fatigued. If a system does not properly account for this variability, it may mistakenly interpret such variability as an entirely different intention by the subject. We report here the design and characterization of factor-analysis (FA)-based decoding algorithms that can contend with this confound. We characterize the decoders (classifiers) on experimental data where monkeys performed both a real reach task and a prosthetic cursor task while we recorded from 96 electrodes implanted in dorsal premotor cortex. The decoder attempts to infer the underlying factors that comodulate the neurons' responses and can use this information to substantially lower error rates (one of eight reach endpoint predictions) by 150 ms, although still advantageous at shorter timescales, that Gaussian-based algorithms performed better than the analogous Poisson-based algorithms and that the FA algorithm is robust even with a limited amount of training data. We propose that FA-based methods are effective in modeling correlated trial-to-trial neural variability and can be used to substantially increase overall prosthetic system performance.

Detecting neural-state transitions using hidden Markov models for motor cortical prostheses.

Neural prosthetic interfaces use neural activity related to the planning and perimovement epochs of arm reaching to afford brain-directed control of external devices. Previous research has primarily centered on accurately decoding movement intention from either plan or perimovement activity, but has assumed that temporal boundaries between these epochs are known to the decoding system. In this work, we develop a technique to automatically differentiate between baseline, plan, and perimovement epochs of neural activity. Specifically, we use a generative model of neural activity to capture how neural activity varies between these three epochs. Our approach is based on a hidden Markov model (HMM), in which the latent variable (state) corresponds to the epoch of neural activity, coupled with a state-dependent Poisson firing model. Using an HMM, we demonstrate that the time of transition from baseline to plan epochs, a transition in neural activity that is not accompanied by any external behavior changes, can be detected using a threshold on the a posteriori HMM state probabilities. Following detection of the plan epoch, we show that the intended target of a center-out movement can be detected about as accurately as that by a maximum-likelihood estimator using a window of known plan activity. In addition, we demonstrate that our HMM can detect transitions in neural activity corresponding to targets not found in training data. Thus the HMM technique for automatically detecting transitions between epochs of neural activity enables prosthetic interfaces that can operate autonomously.

The use of a virtual integration environment for the real-time implementation of neural decode algorithms.

We have developed a virtual integration environment (VIE) for the development of neural prosthetic systems. This paper, the second of two companion articles, describes the use of the VIE as a common platform for the implementation of neural decode algorithms. In this paper, a linear filter decode and a recursive Bayesian algorithm are implemented as separate signal analysis modules of the VIE for the real-time decode of end effector trajectory. The process of implementing each algorithm is described and the real-time behavior as well as computational cost for each algorithm is examined. This is the first report of the real-time implementation of the Mixture of Trajectory Models decode [10]. These real-time algorithms can be easily interfaced with pre-existing modules of the VIE to control simulated and real devices.

Reference frames for reach planning in macaque dorsal premotor cortex.

When a human or animal reaches out to grasp an object, the brain rapidly computes a pattern of muscular contractions that can acquire the target. This computation involves a reference frame transformation because the target's position is initially available only in a visual reference frame, yet the required control signal is a set of commands to the musculature. One of the core brain areas involved in visually guided reaching is the dorsal aspect of the premotor cortex (PMd). Using chronically implanted electrode arrays in two Rhesus monkeys, we studied the contributions of PMd to the reference frame transformation for reaching. PMd neurons are influenced by the locations of reach targets relative to both the arm and the eyes. Some neurons encode reach goals using limb-centered reference frames, whereas others employ eye-centered reference fames. Some cells encode reach goals in a reference frame best described by the combined position of the eyes and hand. In addition to neurons like these where a reference frame could be identified, PMd also contains cells that are influenced by both the eye- and limb-centered locations of reach goals but for which a distinct reference frame could not be determined. We propose two interpretations for these neurons. First, they may encode reach goals using a reference frame we did not investigate, such as intrinsic reference frames. Second, they may not be adequately characterized by any reference frame.

Mixture of trajectory models for neural decoding of goal-directed movements.

Probabilistic decoding techniques have been used successfully to infer time-evolving physical state, such as arm trajectory or the path of a foraging rat, from neural data. A vital element of such decoders is the trajectory model, expressing knowledge about the statistical regularities of the movements. Unfortunately, trajectory models that both 1) accurately describe the movement statistics and 2) admit decoders with relatively low computational demands can be hard to construct. Simple models are computationally inexpensive, but often inaccurate. More complex models may gain accuracy, but at the expense of higher computational cost, hindering their use for real-time decoding. Here, we present a new general approach to defining trajectory models that simultaneously meets both requirements. The core idea is to combine simple trajectory models, each accurate within a limited regime of movement, in a probabilistic mixture of trajectory models (MTM). We demonstrate the utility of the approach by using an MTM decoder to infer goal-directed reaching movements to multiple discrete goals from multi-electrode neural data recorded in monkey motor and premotor cortex. Compared with decoders using simpler trajectory models, the MTM decoder reduced the decoding error by 38 (48) percent in two monkeys using 98 (99) units, without a necessary increase in running time. When available, prior information about the identity of the upcoming reach goal can be incorporated in a principled way, further reducing the decoding error by 20 (11) percent. Taken together, these advances should allow prosthetic cursors or limbs to be moved more accurately toward intended reach goals.

Multiday electrophysiological recordings from freely behaving primates.

Continuous multiday broadband neural data provide a means for observing effects at fine timescales over long periods. In this paper we present analyses on such data sets to demonstrate neural correlates for physically active and inactive time periods, as defined by the response of a head-mounted accelerometer. During active periods, we found that 5-25 Hz local field potential (LFP) power was significantly reduced, firing rate variability increased, and firing rates have greater temporal correlation. Using a single threshold fit to LFP power, 93% of the 403 5 minute blocks tested were correctly classified as active or inactive (as labeled by thresholding each block's maximal accelerometer magnitude). These initial results motivate the use of such data sets for testing neural prosthetics systems and for finding the neural correlates of natural behaviors.

An autonomous, broadband, multi-channel neural recording system for freely behaving primates.

Successful laboratory proof-of-concept experiments with neural prosthetic systems motivate continued algorithm and hardware development. For these efforts to move beyond traditional fixed laboratory setups, new tools are needed to enable broadband, multi-channel, long duration neural recording from freely behaving primates. In this paper we present a dual-channel, battery powered, neural recording system with integrated 3-axis accelerometer for use with chronically implanted electrode arrays. The recording system, called HermesB, is self-contained, autonomous, programmable and capable of recording broadband neural and head acceleration data to a removable compact flash card for up to 48 hours.

Neural recording stability of chronic electrode arrays in freely behaving primates.

Chronically implanted electrode arrays have enabled a broad range of advances, particularly in the field of neural prosthetics. Those successes motivate development of prototype implantable prosthetic processors for long duration, continuous use in freely behaving subjects. However, traditional experimental protocols have provided limited information regarding the stability of the electrode arrays and their neural recordings. In this paper we present preliminary results derived from long duration neural recordings in a freely behaving primate which show variations in action potential shape and RMS noise across a range of time scales. These preliminary results suggest that spike sorting algorithms can no longer assume stable neural signals and will need to transition to adaptive signal processing methodologies to maximize performance.

Increasing the performance of cortically-controlled prostheses.

Neural prostheses have received considerable attention due to their potential to dramatically improve the quality of life of severely disabled patients. Cortically-controlled prostheses are able to translate neural activity from cerebral cortex into control signals for guiding computer cursors or prosthetic limbs. Non-invasive and invasive electrode techniques can be used to measure neural activity, with the latter promising considerably higher levels of performance and therefore functionality to patients. We review here some of our recent experimental and computational work aimed at establishing a principled design methodology to increase electrode-based cortical prosthesis performance to near theoretical limits. Studies discussed include translating unprecedentedly brief periods of "plan" activity into high information rate (6.5 bits/s)control signals, improving decode algorithms and optimizing visual target locations for further performance increases, and recording from chronically implanted arrays in freely behaving monkeys to characterize neuron stability. Taken together, these results should substantially increase the clinical viability of cortical prostheses.