Decoding and perturbing decision states in real time.

In dynamic environments, subjects often integrate multiple samples of a signal and combine them to reach a categorical judgment1. The process of deliberation can be described by a time-varying decision variable (DV), decoded from neural population activity, that predicts a subject's upcoming decision2. Within single trials, however, there are large moment-to-moment fluctuations in the DV, the behavioural significance of which is unclear. Here, using real-time, neural feedback control of stimulus duration, we show that within-trial DV fluctuations, decoded from motor cortex, are tightly linked to decision state in macaques, predicting behavioural choices substantially better than the condition-averaged DV or the visual stimulus alone. Furthermore, robust changes in DV sign have the statistical regularities expected from behavioural studies of changes of mind3. Probing the decision process on single trials with weak stimulus pulses, we find evidence for time-varying absorbing decision bounds, enabling us to distinguish between specific models of decision making.

Deep posteromedial cortical rhythm in dissociation.

Advanced imaging methods now allow cell-type-specific recording of neural activity across the mammalian brain, potentially enabling the exploration of how brain-wide dynamical patterns give rise to complex behavioural states1-12. Dissociation is an altered behavioural state in which the integrity of experience is disrupted, resulting in reproducible cognitive phenomena including the dissociation of stimulus detection from stimulus-related affective responses. Dissociation can occur as a result of trauma, epilepsy or dissociative drug use13,14, but despite its substantial basic and clinical importance, the underlying neurophysiology of this state is unknown. Here we establish such a dissociation-like state in mice, induced by precisely-dosed administration of ketamine or phencyclidine. Large-scale imaging of neural activity revealed that these dissociative agents elicited a 1-3-Hz rhythm in layer 5 neurons of the retrosplenial cortex. Electrophysiological recording with four simultaneously deployed high-density probes revealed rhythmic coupling of the retrosplenial cortex with anatomically connected components of thalamus circuitry, but uncoupling from most other brain regions was observed-including a notable inverse correlation with frontally projecting thalamic nuclei. In testing for causal significance, we found that rhythmic optogenetic activation of retrosplenial cortex layer 5 neurons recapitulated dissociation-like behavioural effects. Local retrosplenial hyperpolarization-activated cyclic-nucleotide-gated potassium channel 1 (HCN1) pacemakers were required for systemic ketamine to induce this rhythm and to elicit dissociation-like behavioural effects. In a patient with focal epilepsy, simultaneous intracranial stereoencephalography recordings from across the brain revealed a similarly localized rhythm in the homologous deep posteromedial cortex that was temporally correlated with pre-seizure self-reported dissociation, and local brief electrical stimulation of this region elicited dissociative experiences. These results identify the molecular, cellular and physiological properties of a conserved deep posteromedial cortical rhythm that underlies states of dissociation.

Neural population dynamics during reaching.

Most theories of motor cortex have assumed that neural activity represents movement parameters. This view derives from what is known about primary visual cortex, where neural activity represents patterns of light. Yet it is unclear how well the analogy between motor and visual cortex holds. Single-neuron responses in motor cortex are complex, and there is marked disagreement regarding which movement parameters are represented. A better analogy might be with other motor systems, where a common principle is rhythmic neural activity. Here we find that motor cortex responses during reaching contain a brief but strong oscillatory component, something quite unexpected for a non-periodic behaviour. Oscillation amplitude and phase followed naturally from the preparatory state, suggesting a mechanistic role for preparatory neural activity. These results demonstrate an unexpected yet surprisingly simple structure in the population response. This underlying structure explains many of the confusing features of individual neural responses.

A Non-Human Primate Brain-Computer Typing Interface.

Brain-computer interfaces (BCIs) record brain activity and translate the information into useful control signals. They can be used to restore function to people with paralysis by controlling end effectors such as computer cursors and robotic limbs. Communication neural prostheses are BCIs that control user interfaces on computers or mobile devices. Here we demonstrate a communication prosthesis by simulating a typing task with two rhesus macaques implanted with electrode arrays. The monkeys used two of the highest known performing BCI decoders to type out words and sentences when prompted one symbol/letter at a time. On average, Monkeys J and L achieved typing rates of 10.0 and 7.2 words per minute (wpm), respectively, copying text from a newspaper article using a velocity-only two dimensional BCI decoder with dwell-based symbol selection. With a BCI decoder that also featured a discrete click for key selection, typing rates increased to 12.0 and 7.8 wpm. These represent the highest known achieved communication rates using a BCI. We then quantified the relationship between bitrate and typing rate and found it approximately linear: typing rate in wpm is nearly three times bitrate in bits per second. We also compared the metrics of achieved bitrate and information transfer rate and discuss their applicability to real-world typing scenarios. Although this study cannot model the impact of cognitive load of word and sentence planning, the findings here demonstrate the feasibility of BCIs to serve as communication interfaces and represent an upper bound on the expected achieved typing rate for a given BCI throughput.

A scalable platform for acquisition of high-fidelity human intracranial EEG with minimal clinical burden.

Human neuroscience research has been significantly advanced by neuroelectrophysiological studies from people with refractory epilepsy-the only routine clinical intervention that acquires multi-day, multi-electrode human intracranial electroencephalography (iEEG). While a sampling rate below 2 kHz is sufficient for manual iEEG review by epileptologists, computational methods and research studies may benefit from higher resolution, which requires significant technical development. At adult and pediatric Stanford hospitals, research ports of commercial clinical acquisition systems were configured to collect 10 kHz iEEG of up to 256 electrodes simultaneously with the clinical data. The research digital stream was designed to be acquired post-digitization, resulting in no loss in clinical signal quality. This novel framework implements a near-invisible research platform to facilitate the secure, routine collection of high-resolution iEEG that minimizes research hardware footprint and clinical workflow interference. The addition of a pocket-sized router in the patient room enabled an encrypted tunnel to securely transmit research-quality iEEG across hospital networks to a research computer within the hospital server room, where data was coded, de-identified, and uploaded to cloud storage. Every eligible patient undergoing iEEG clinical evaluation at both hospitals since September 2017 has been recruited; participant recruitment is ongoing. Over 350+ terabytes (representing 1000+ days) of neuroelectrophysiology were recorded across 200+ participants of diverse demographics. To our knowledge, this is the first report of such a research integration within a hospital setting. It is a promising approach to promoting equitable participant enrollment and building comprehensive data repositories with consistent, high-fidelity specifications towards new discoveries in human neuroscience.

A closed-loop human simulator for investigating the role of feedback control in brain-machine interfaces.

Neural prosthetic systems seek to improve the lives of severely disabled people by decoding neural activity into useful behavioral commands. These systems and their decoding algorithms are typically developed "offline," using neural activity previously gathered from a healthy animal, and the decoded movement is then compared with the true movement that accompanied the recorded neural activity. However, this offline design and testing may neglect important features of a real prosthesis, most notably the critical role of feedback control, which enables the user to adjust neural activity while using the prosthesis. We hypothesize that understanding and optimally designing high-performance decoders require an experimental platform where humans are in closed-loop with the various candidate decode systems and algorithms. It remains unexplored the extent to which the subject can, for a particular decode system, algorithm, or parameter, engage feedback and other strategies to improve decode performance. Closed-loop testing may suggest different choices than offline analyses. Here we ask if a healthy human subject, using a closed-loop neural prosthesis driven by synthetic neural activity, can inform system design. We use this online prosthesis simulator (OPS) to optimize "online" decode performance based on a key parameter of a current state-of-the-art decode algorithm, the bin width of a Kalman filter. First, we show that offline and online analyses indeed suggest different parameter choices. Previous literature and our offline analyses agree that neural activity should be analyzed in bins of 100- to 300-ms width. OPS analysis, which incorporates feedback control, suggests that much shorter bin widths (25-50 ms) yield higher decode performance. Second, we confirm this surprising finding using a closed-loop rhesus monkey prosthetic system. These findings illustrate the type of discovery made possible by the OPS, and so we hypothesize that this novel testing approach will help in the design of prosthetic systems that will translate well to human patients.

A markerless platform for ambulatory systems neuroscience.

Motor systems neuroscience seeks to understand how the brain controls movement. To minimize confounding variables, large-animal studies typically constrain body movement from areas not under observation, ensuring consistent, repeatable behaviors. Such studies have fueled decades of research, but they may be artificially limiting the richness of neural data observed, preventing generalization to more natural movements and settings. Neuroscience studies of unconstrained movement would capture a greater range of behavior and a more complete view of neuronal activity, but instrumenting an experimental rig suitable for large animals presents substantial engineering challenges. Here, we present a markerless, full-body motion tracking and synchronized wireless neural electrophysiology platform for large, ambulatory animals. Composed of four depth (RGB-D) cameras that provide a 360° view of a 4.5-square-meters enclosed area, this system is designed to record a diverse range of neuroethologically relevant behaviors. This platform also allows for the simultaneous acquisition of hundreds of wireless neural recording channels in multiple brain regions. As behavioral and neuronal data are generated at rates below 200 megabytes per second, a single desktop can facilitate hours of continuous recording. This setup is designed for systems neuroscience and neuroengineering research, where synchronized kinematic behavior and neural data are the foundation for investigation. By enabling the study of previously unexplored movement tasks, this system can generate insights into the functioning of the mammalian motor system and provide a platform to develop brain-machine interfaces for unconstrained applications.

Publisher Correction: Brain-machine interface cursor position only weakly affects monkey and human motor cortical activity in the absence of arm movements.

A correction to this article has been published and is linked from the HTML and PDF versions of this paper. The error has been fixed in the paper.

Neural Population Dynamics Underlying Motor Learning Transfer.

Covert motor learning can sometimes transfer to overt behavior. We investigated the neural mechanism underlying transfer by constructing a two-context paradigm. Subjects performed cursor movements either overtly using arm movements, or covertly via a brain-machine interface that moves the cursor based on motor cortical activity (in lieu of arm movement). These tasks helped evaluate whether and how cortical changes resulting from "covert rehearsal" affect overt performance. We found that covert learning indeed transfers to overt performance and is accompanied by systematic population-level changes in motor preparatory activity. Current models of motor cortical function ascribe motor preparation to achieving initial conditions favorable for subsequent movement-period neural dynamics. We found that covert and overt contexts share these initial conditions, and covert rehearsal manipulates them in a manner that persists across context changes, thus facilitating overt motor learning. This transfer learning mechanism might provide new insights into other covert processes like mental rehearsal.

Feasibility of Automatic Error Detect-and-Undo System in Human Intracortical Brain-Computer Interfaces.

OBJECTIVE: Brain-computer interfaces (BCIs) aim to help people with impaired movement ability by directly translating their movement intentions into command signals for assistive technologies. Despite large performance improvements over the last two decades, BCI systems still make errors that need to be corrected manually by the user. This decreases system performance and is also frustrating for the user. The deleterious effects of errors could be mitigated if the system automatically detected when the user perceives that an error was made and automatically intervened with a corrective action; thus, sparing users from having to make the correction themselves. Our previous preclinical work with monkeys demonstrated that task-outcome correlates exist in motor cortical spiking activity and can be utilized to improve BCI performance. Here, we asked if these signals also exist in the human hand area of motor cortex, and whether they can be decoded with high accuracy. METHODS: We analyzed posthoc the intracortical neural activity of two BrainGate2 clinical trial participants who were neurally controlling a computer cursor to perform a grid target selection task and a keyboard-typing task. RESULTS: Our key findings are that: 1) there exists a putative outcome error signal reflected in both the action potentials and local field potentials of the human hand area of motor cortex, and 2) target selection outcomes can be classified with high accuracy (70-85%) of errors successfully detected with minimal (0-3%) misclassifications of success trials, based on neural activity alone. SIGNIFICANCE: These offline results suggest that it will be possible to improve the performance of clinical intracortical BCIs by incorporating a real-time error detect-and-undo system alongside the decoding of movement intention.

Brain-machine interface cursor position only weakly affects monkey and human motor cortical activity in the absence of arm movements.

Brain-machine interfaces (BMIs) that decode movement intentions should ignore neural modulation sources distinct from the intended command. However, neurophysiology and control theory suggest that motor cortex reflects the motor effector's position, which could be a nuisance variable. We investigated motor cortical correlates of BMI cursor position with or without concurrent arm movement. We show in two monkeys that subtracting away estimated neural correlates of position improves online BMI performance only if the animals were allowed to move their arm. To understand why, we compared the neural variance attributable to cursor position when the same task was performed using arm reaching, versus arms-restrained BMI use. Firing rates correlated with both BMI cursor and hand positions, but hand positional effects were greater. To examine whether BMI position influences decoding in people with paralysis, we analyzed data from two intracortical BMI clinical trial participants and performed an online decoder comparison in one participant. We found only small motor cortical correlates, which did not affect performance. These results suggest that arm movement and proprioception are the major contributors to position-related motor cortical correlates. Cursor position visual feedback is therefore unlikely to affect the performance of BMI-driven prosthetic systems being developed for people with paralysis.

Cortical control of a tablet computer by people with paralysis.

General-purpose computers have become ubiquitous and important for everyday life, but they are difficult for people with paralysis to use. Specialized software and personalized input devices can improve access, but often provide only limited functionality. In this study, three research participants with tetraplegia who had multielectrode arrays implanted in motor cortex as part of the BrainGate2 clinical trial used an intracortical brain-computer interface (iBCI) to control an unmodified commercial tablet computer. Neural activity was decoded in real time as a point-and-click wireless Bluetooth mouse, allowing participants to use common and recreational applications (web browsing, email, chatting, playing music on a piano application, sending text messages, etc.). Two of the participants also used the iBCI to "chat" with each other in real time. This study demonstrates, for the first time, high-performance iBCI control of an unmodified, commercially available, general-purpose mobile computing device by people with tetraplegia.

Bi-Fi: an embedded sensor/system architecture for REMOTE biological monitoring.

Wireless-enabled processor modules intended for communicating low-frequency phenomena (i.e., temperature, humidity, and ambient light) have been enabled to acquire and transmit multiple biological signals in real time, which has been achieved by using computationally efficient data acquisition, filtering, and compression algorithms, and interfacing the modules with biological interface hardware. The sensor modules can acquire and transmit raw biological signals at a rate of 32 kb/s, which is near the hardware limit of the modules. Furthermore, onboard signal processing enables one channel, sampled at a rate of 4000 samples/s at 12-bit resolution, to be compressed via adaptive differential-pulse-code modulation (ADPCM) and transmitted in real time. In addition, the sensors can be configured to filter and transmit individual time-referenced "spike" waveforms, or to transmit the spike height and width for alleviating network traffic and increasing battery life. The system is capable of acquiring eight channels of analog signals as well as data via an asynchronous serial connection. A back-end server archives the biological data received via networked gateway sensors, and hosts them to a client application that enables users to browse recorded data. The system also acquires, filters, and transmits oxygen saturation and pulse rate via a commercial-off-the-shelf interface board. The system architecture can be configured for performing real-time nonobtrusive biological monitoring of humans or rodents. This paper demonstrates that low-power, computational, and bandwidth-constrained wireless-enabled platforms can indeed be leveraged for wireless biosignal monitoring.

A High-Performance Neural Prosthesis Incorporating Discrete State Selection With Hidden Markov Models.

Communication neural prostheses aim to restore efficient communication to people with motor neurological injury or disease by decoding neural activity into control signals. These control signals are both analog (e.g., the velocity of a computer mouse) and discrete (e.g., clicking an icon with a computer mouse) in nature. Effective, high-performing, and intuitive-to-use communication prostheses should be capable of decoding both analog and discrete state variables seamlessly. However, to date, the highest-performing autonomous communication prostheses rely on precise analog decoding and typically do not incorporate high-performance discrete decoding. In this report, we incorporated a hidden Markov model (HMM) into an intracortical communication prosthesis to enable accurate and fast discrete state decoding in parallel with analog decoding. In closed-loop experiments with nonhuman primates implanted with multielectrode arrays, we demonstrate that incorporating an HMM into a neural prosthesis can increase state-of-the-art achieved bitrate by 13.9% and 4.2% in two monkeys ( ). We found that the transition model of the HMM is critical to achieving this performance increase. Further, we found that using an HMM resulted in the highest achieved peak performance we have ever observed for these monkeys, achieving peak bitrates of 6.5, 5.7, and 4.7 bps in Monkeys J, R, and L, respectively. Finally, we found that this neural prosthesis was robustly controllable for the duration of entire experimental sessions. These results demonstrate that high-performance discrete decoding can be beneficially combined with analog decoding to achieve new state-of-the-art levels of performance.

High performance communication by people with paralysis using an intracortical brain-computer interface.

Brain-computer interfaces (BCIs) have the potential to restore communication for people with tetraplegia and anarthria by translating neural activity into control signals for assistive communication devices. While previous pre-clinical and clinical studies have demonstrated promising proofs-of-concept (Serruya et al., 2002; Simeral et al., 2011; Bacher et al., 2015; Nuyujukian et al., 2015; Aflalo et al., 2015; Gilja et al., 2015; Jarosiewicz et al., 2015; Wolpaw et al., 1998; Hwang et al., 2012; Spüler et al., 2012; Leuthardt et al., 2004; Taylor et al., 2002; Schalk et al., 2008; Moran, 2010; Brunner et al., 2011; Wang et al., 2013; Townsend and Platsko, 2016; Vansteensel et al., 2016; Nuyujukian et al., 2016; Carmena et al., 2003; Musallam et al., 2004; Santhanam et al., 2006; Hochberg et al., 2006; Ganguly et al., 2011; O'Doherty et al., 2011; Gilja et al., 2012), the performance of human clinical BCI systems is not yet high enough to support widespread adoption by people with physical limitations of speech. Here we report a high-performance intracortical BCI (iBCI) for communication, which was tested by three clinical trial participants with paralysis. The system leveraged advances in decoder design developed in prior pre-clinical and clinical studies (Gilja et al., 2015; Kao et al., 2016; Gilja et al., 2012). For all three participants, performance exceeded previous iBCIs (Bacher et al., 2015; Jarosiewicz et al., 2015) as measured by typing rate (by a factor of 1.4-4.2) and information throughput (by a factor of 2.2-4.0). This high level of performance demonstrates the potential utility of iBCIs as powerful assistive communication devices for people with limited motor function.Clinical Trial No: NCT00912041.

A TinyOS-enabled MICA2-based wireless neural interface.

Existing approaches used to develop compact low-power multichannel wireless neural recording systems range from creating custom-integrated circuits to assembling commercial-off-the-shelf (COTS) PC-based components. Custom-integrated-circuit designs yield extremely compact and low-power devices at the expense of high development and upgrade costs and turn-around times, while assembling COTS-PC-technology yields high performance at the expense of large system size and increased power consumption. To achieve a balance between implementing an ultra-compact custom-fabricated neural transceiver and assembling COTS-PC-technology, an overlay of a neural interface upon the TinyOS-based MICA2 platform is described. The system amplifies, digitally encodes, and transmits neural signals real-time at a rate of 9.6 kbps, while consuming less than 66 mW of power. The neural signals are received and forwarded to a client PC over a serial connection. This data rate can be divided for recording on up to 6 channels, with a resolution of 8 bits/sample. This work demonstrates the strengths and limitations of the TinyOS-based sensor technology as a foundation for chronic remote biological monitoring applications and, thus, provides an opportunity to create a system that can leverage from the frequent networking and communications advancements being made by the global TinyOS-development community.

A high-performance keyboard neural prosthesis enabled by task optimization.

Communication neural prostheses are an emerging class of medical devices that aim to restore efficient communication to people suffering from paralysis. These systems rely on an interface with the user, either via the use of a continuously moving cursor (e.g., mouse) or the discrete selection of symbols (e.g., keyboard). In developing these interfaces, many design choices have a significant impact on the performance of the system. The objective of this study was to explore the design choices of a continuously moving cursor neural prosthesis and optimize the interface to maximize information theoretic performance. We swept interface parameters of two keyboard-like tasks to find task and subject-specific optimal parameters as measured by achieved bitrate using two rhesus macaques implanted with multielectrode arrays. In this paper, we present the highest performing free-paced neural prosthesis under any recording modality with sustainable communication rates of up to 3.5 bits/s. These findings demonstrate that meaningful high performance can be achieved using an intracortical neural prosthesis, and that, when optimized, these systems may be appropriate for use as communication devices for those with physical disabilities.

A high performing brain-machine interface driven by low-frequency local field potentials alone and together with spikes.

OBJECTIVE: Brain-machine interfaces (BMIs) seek to enable people with movement disabilities to directly control prosthetic systems with their neural activity. Current high performance BMIs are driven by action potentials (spikes), but access to this signal often diminishes as sensors degrade over time. Decoding local field potentials (LFPs) as an alternative or complementary BMI control signal may improve performance when there is a paucity of spike signals. To date only a small handful of LFP decoding methods have been tested online; there remains a need to test different LFP decoding approaches and improve LFP-driven performance. There has also not been a reported demonstration of a hybrid BMI that decodes kinematics from both LFP and spikes. Here we first evaluate a BMI driven by the local motor potential (LMP), a low-pass filtered time-domain LFP amplitude feature. We then combine decoding of both LMP and spikes to implement a hybrid BMI. APPROACH: Spikes and LFP were recorded from two macaques implanted with multielectrode arrays in primary and premotor cortex while they performed a reaching task. We then evaluated closed-loop BMI control using biomimetic decoders driven by LMP, spikes, or both signals together. MAIN RESULTS: LMP decoding enabled quick and accurate cursor control which surpassed previously reported LFP BMI performance. Hybrid decoding of both spikes and LMP improved performance when spikes signal quality was mediocre to poor. SIGNIFICANCE: These findings show that LMP is an effective BMI control signal which requires minimal power to extract and can substitute for or augment impoverished spikes signals. Use of this signal may lengthen the useful lifespan of BMIs and is therefore an important step towards clinically viable BMIs.

Single-trial dynamics of motor cortex and their applications to brain-machine interfaces.

Increasing evidence suggests that neural population responses have their own internal drive, or dynamics, that describe how the neural population evolves through time. An important prediction of neural dynamical models is that previously observed neural activity is informative of noisy yet-to-be-observed activity on single-trials, and may thus have a denoising effect. To investigate this prediction, we built and characterized dynamical models of single-trial motor cortical activity. We find these models capture salient dynamical features of the neural population and are informative of future neural activity on single trials. To assess how neural dynamics may beneficially denoise single-trial neural activity, we incorporate neural dynamics into a brain-machine interface (BMI). In online experiments, we find that a neural dynamical BMI achieves substantially higher performance than its non-dynamical counterpart. These results provide evidence that neural dynamics beneficially inform the temporal evolution of neural activity on single trials and may directly impact the performance of BMIs.