Assistive sensory-motor perturbations influence learned neural representations.

Task errors are used to learn and refine motor skills. We investigated how task assistance influences learned neural representations using Brain-Computer Interfaces (BCIs), which map neural activity into movement via a decoder. We analyzed motor cortex activity as monkeys practiced BCI with a decoder that adapted to improve or maintain performance over days. Population dimensionality remained constant or increased with learning, counter to trends with non-adaptive BCIs. Yet, over time, task information was contained in a smaller subset of neurons or population modes. Moreover, task information was ultimately stored in neural modes that occupied a small fraction of the population variance. An artificial neural network model suggests the adaptive decoders contribute to forming these compact neural representations. Our findings show that assistive decoders manipulate error information used for long-term learning computations, like credit assignment, which informs our understanding of motor learning and has implications for designing real-world BCIs.

Inserting a Neuropixels probe into awake monkey cortex: two probes, two methods.

BACKGROUND: Neuropixels probes have revolutionized neurophysiological studies in the rodent, but inserting these probes through the much thicker primate dura remains a challenge. NEW METHODS: Here we describe two methods we have developed for the insertion of two types of Neuropixels probes acutely into the awake macaque monkey cortex. For the fine rodent probe (Neuropixels 1.0, IMEC), which is unable to pierce native primate dura, we developed a dural-eyelet method to insert the probe repeatedly without breakage. For the thicker short NHP probe (Neuropixels NP1010), we developed an artificial dura system to insert the probe. RESULTS AND COMPARISON WITH EXISTING METHODS: We have now conducted successful experiments in 3 animals across 7 recording chambers with the procedures described here and have achieved recordings with similar yields over several months in each case. CONCLUSION: We hope that our hardware, surgical preparation, methods for insertion and methods for removal of broken probe parts are of value to primate physiologists everywhere.

Invariant neural dynamics drive commands to control different movements.

It has been proposed that the nervous system has the capacity to generate a wide variety of movements because it reuses some invariant code. Previous work has identified that dynamics of neural population activity are similar during different movements, where dynamics refer to how the instantaneous spatial pattern of population activity changes in time. Here, we test whether invariant dynamics of neural populations are actually used to issue the commands that direct movement. Using a brain-machine interface (BMI) that transforms rhesus macaques' motor-cortex activity into commands for a neuroprosthetic cursor, we discovered that the same command is issued with different neural-activity patterns in different movements. However, these different patterns were predictable, as we found that the transitions between activity patterns are governed by the same dynamics across movements. These invariant dynamics are low dimensional, and critically, they align with the BMI, so that they predict the specific component of neural activity that actually issues the next command. We introduce a model of optimal feedback control (OFC) that shows that invariant dynamics can help transform movement feedback into commands, reducing the input that the neural population needs to control movement. Altogether our results demonstrate that invariant dynamics drive commands to control a variety of movements and show how feedback can be integrated with invariant dynamics to issue generalizable commands.

Neural Plasticity in Sensorimotor Brain-Machine Interfaces.

Brain-machine interfaces (BMIs) aim to treat sensorimotor neurological disorders by creating artificial motor and/or sensory pathways. Introducing artificial pathways creates new relationships between sensory input and motor output, which the brain must learn to gain dexterous control. This review highlights the role of learning in BMIs to restore movement and sensation, and discusses how BMI design may influence neural plasticity and performance. The close integration of plasticity in sensory and motor function influences the design of both artificial pathways and will be an essential consideration for bidirectional devices that restore both sensory and motor function.

Quantifying the influence of stimulation protocols on neural network connectivity inference to optimize rapid network measurements.

Connectivity is key to understanding neural circuit computations. However, estimating in vivo connectivity using recording of activity alone is challenging. Issues include common input and bias errors in inference, and limited temporal resolution due to large data requirements. Perturbations (e.g. stimulation) can improve inference accuracy and accelerate estimation. However, optimal stimulation protocols for rapid network estimation are not yet established. Here, we use neural network simulations to identify stimulation protocols that minimize connectivity inference errors when using generalized linear model inference. We find that stimulation parameters that balance excitatory and inhibitory activity minimize inference error. We also show that pairing optimized stimulation with adaptive protocols that choose neurons to stimulate via Bayesian inference may ultimately enable rapid network inference.

Multi-scale neural decoding and analysis.

Objective. Complex spatiotemporal neural activity encodes rich information related to behavior and cognition. Conventional research has focused on neural activity acquired using one of many different measurement modalities, each of which provides useful but incomplete assessment of the neural code. Multi-modal techniques can overcome tradeoffs in the spatial and temporal resolution of a single modality to reveal deeper and more comprehensive understanding of system-level neural mechanisms. Uncovering multi-scale dynamics is essential for a mechanistic understanding of brain function and for harnessing neuroscientific insights to develop more effective clinical treatment.Approach. We discuss conventional methodologies used for characterizing neural activity at different scales and review contemporary examples of how these approaches have been combined. Then we present our case for integrating activity across multiple scales to benefit from the combined strengths of each approach and elucidate a more holistic understanding of neural processes.Main results. We examine various combinations of neural activity at different scales and analytical techniques that can be used to integrate or illuminate information across scales, as well the technologies that enable such exciting studies. We conclude with challenges facing future multi-scale studies, and a discussion of the power and potential of these approaches.Significance. This roadmap will lead the readers toward a broad range of multi-scale neural decoding techniques and their benefits over single-modality analyses. This Review article highlights the importance of multi-scale analyses for systematically interrogating complex spatiotemporal mechanisms underlying cognition and behavior.

Windows and periscopes into primate behavior.

Calcium imaging of neurons in monkeys making reaches is complicated by brain movements and limited by shallow imaging depth. In a pair of recent studies, Trautmann et al., 2021 and Bollimunta et al. (2021) present complementary solutions to these problems.

Sufficient sampling for kriging prediction of cortical potential in rat, monkey, and human µECoG.

Objective. Large channel count surface-based electrophysiology arrays (e.g. µECoG) are high-throughput neural interfaces with good chronic stability. Electrode spacing remains ad hoc due to redundancy and nonstationarity of field dynamics. Here, we establish a criterion for electrode spacing based on the expected accuracy of predicting unsampled field potential from sampled sites.Approach. We applied spatial covariance modeling and field prediction techniques based on geospatial kriging to quantify sufficient sampling for thousands of 500 ms µECoG snapshots in human, monkey, and rat. We calculated a probably approximately correct (PAC) spacing based on kriging that would be required to predict µECoG fields at≤10% error for most cases (95% of observations).Main results. Kriging theory accurately explained the competing effects of electrode density and noise on predicting field potential. Across five frequency bands from 4-7 to 75-300 Hz, PAC spacing was sub-millimeter for auditory cortex in anesthetized and awake rats, and posterior superior temporal gyrus in anesthetized human. At 75-300 Hz, sub-millimeter PAC spacing was required in all species and cortical areas.Significance. PAC spacing accounted for the effect of signal-to-noise on prediction quality and was sensitive to the full distribution of non-stationary covariance states. Our results show that µECoG arrays should sample at sub-millimeter resolution for applications in diverse cortical areas and for noise resilience.

Development of a neural interface for high-definition, long-term recording in rodents and nonhuman primates.

Long-lasting, high-resolution neural interfaces that are ultrathin and flexible are essential for precise brain mapping and high-performance neuroprosthetic systems. Scaling to sample thousands of sites across large brain regions requires integrating powered electronics to multiplex many electrodes to a few external wires. However, existing multiplexed electrode arrays rely on encapsulation strategies that have limited implant lifetimes. Here, we developed a flexible, multiplexed electrode array, called "Neural Matrix," that provides stable in vivo neural recordings in rodents and nonhuman primates. Neural Matrix lasts over a year and samples a centimeter-scale brain region using over a thousand channels. The long-lasting encapsulation (projected to last at least 6 years), scalable device design, and iterative in vivo optimization described here are essential components to overcoming current hurdles facing next-generation neural technologies.

A Modular Implant System for Multimodal Recording and Manipulation of the Primate Brain.

Neural circuitry can be investigated and manipulated using a variety of techniques, including electrical and optical recording and stimulation. At present, most neural interfaces are designed to accommodate a single mode of neural recording and/or manipulation, which limits the amount of data that can be extracted from a single population of neurons. To overcome these technical limitations, we developed a chronic, multi-scale, multi-modal chamber-based neural implant for use in non-human primates that accommodates electrophysiological recording and stimulation, optical manipulation, and wide-field imaging. We present key design features of the system and mechanical validation. We also present sample data from two non-human primate subjects to validate the efficacy of the design in vivo.

Parsing learning in networks using brain-machine interfaces.

Brain-machine interfaces (BMIs) define new ways to interact with our environment and hold great promise for clinical therapies. Motor BMIs, for instance, re-route neural activity to control movements of a new effector and could restore movement to people with paralysis. Increasing experience shows that interfacing with the brain inevitably changes the brain. BMIs engage and depend on a wide array of innate learning mechanisms to produce meaningful behavior. BMIs precisely define the information streams into and out of the brain, but engage wide-spread learning. We take a network perspective and review existing observations of learning in motor BMIs to show that BMIs engage multiple learning mechanisms distributed across neural networks. Recent studies demonstrate the advantages of BMI for parsing this learning and its underlying neural mechanisms. BMIs therefore provide a powerful tool for studying the neural mechanisms of learning that highlights the critical role of learning in engineered neural therapies.

Rapid control and feedback rates enhance neuroprosthetic control.

Brain-machine interfaces (BMI) create novel sensorimotor pathways for action. Much as the sensorimotor apparatus shapes natural motor control, the BMI pathway characteristics may also influence neuroprosthetic control. Here, we explore the influence of control and feedback rates, where control rate indicates how often motor commands are sent from the brain to the prosthetic, and feedback rate indicates how often visual feedback of the prosthetic is provided to the subject. We developed a new BMI that allows arbitrarily fast control and feedback rates, and used it to dissociate the effects of each rate in two monkeys. Increasing the control rate significantly improved control even when feedback rate was unchanged. Increasing the feedback rate further facilitated control. We also show that our high-rate BMI significantly outperformed state-of-the-art methods due to higher control and feedback rates, combined with a different point process mathematical encoding model. Our BMI paradigm can dissect the contribution of different elements in the sensorimotor pathway, providing a unique tool for studying neuroprosthetic control mechanisms.

Robust Brain-Machine Interface Design Using Optimal Feedback Control Modeling and Adaptive Point Process Filtering.

Much progress has been made in brain-machine interfaces (BMI) using decoders such as Kalman filters and finding their parameters with closed-loop decoder adaptation (CLDA). However, current decoders do not model the spikes directly, and hence may limit the processing time-scale of BMI control and adaptation. Moreover, while specialized CLDA techniques for intention estimation and assisted training exist, a unified and systematic CLDA framework that generalizes across different setups is lacking. Here we develop a novel closed-loop BMI training architecture that allows for processing, control, and adaptation using spike events, enables robust control and extends to various tasks. Moreover, we develop a unified control-theoretic CLDA framework within which intention estimation, assisted training, and adaptation are performed. The architecture incorporates an infinite-horizon optimal feedback-control (OFC) model of the brain's behavior in closed-loop BMI control, and a point process model of spikes. The OFC model infers the user's motor intention during CLDA-a process termed intention estimation. OFC is also used to design an autonomous and dynamic assisted training technique. The point process model allows for neural processing, control and decoder adaptation with every spike event and at a faster time-scale than current decoders; it also enables dynamic spike-event-based parameter adaptation unlike current CLDA methods that use batch-based adaptation on much slower adaptation time-scales. We conducted closed-loop experiments in a non-human primate over tens of days to dissociate the effects of these novel CLDA components. The OFC intention estimation improved BMI performance compared with current intention estimation techniques. OFC assisted training allowed the subject to consistently achieve proficient control. Spike-event-based adaptation resulted in faster and more consistent performance convergence compared with batch-based methods, and was robust to parameter initialization. Finally, the architecture extended control to tasks beyond those used for CLDA training. These results have significant implications towards the development of clinically-viable neuroprosthetics.

Development of semi-chronic microdrive system for large-scale circuit mapping in macaque mesolimbic and basal ganglia systems.

The development of novel neurotechnologies for treating refractory neuropsychiatry disorders depends on understanding and manipulating the dynamics of neural circuits across large-scale brain networks. The mesolimbic pathway plays an essential role in reward processing and mood regulation and disorders of this pathway underlie many neuropsychiatric disorders. Here, we present the design of a customized semi-chronic microdrive array that precisely targets the anatomical structures of non-human primate (NHP) mesolimbic and basal ganglia systems. We present an integrated experimental paradigm that uses this device to map and manipulate large-scale neural circuits. The system combines electrophysiology, spatiotemporal multisite patterned intracortical microstimulation (ICMS), and diffusion tractography. We propose that this system provides a flexible platform for exploring and identifying neural signatures which can serve as novel targets for closed-loop stimulation in the clinical treatment of neuropsychiatric disorders.

Continuous closed-loop decoder adaptation with a recursive maximum likelihood algorithm allows for rapid performance acquisition in brain-machine interfaces.

Closed-loop decoder adaptation (CLDA) is an emerging paradigm for both improving and maintaining online performance in brain-machine interfaces (BMIs). The time required for initial decoder training and any subsequent decoder recalibrations could be potentially reduced by performing continuous adaptation, in which decoder parameters are updated at every time step during these procedures, rather than waiting to update the decoder at periodic intervals in a more batch-based process. Here, we present recursive maximum likelihood (RML), a CLDA algorithm that performs continuous adaptation of a Kalman filter decoder's parameters. We demonstrate that RML possesses a variety of useful properties and practical algorithmic advantages. First, we show how RML leverages the accuracy of updates based on a batch of data while still adapting parameters on every time step. Second, we illustrate how the RML algorithm is parameterized by a single, intuitive half-life parameter that can be used to adjust the rate of adaptation in real time. Third, we show how even when the number of neural features is very large, RML's memory-efficient recursive update rules can be reformulated to also be computationally fast so that continuous adaptation is still feasible. To test the algorithm in closed-loop experiments, we trained three macaque monkeys to perform a center-out reaching task by using either spiking activity or local field potentials to control a 2D computer cursor. RML achieved higher levels of performance more rapidly in comparison to a previous CLDA algorithm that adapts parameters on a more intermediate timescale. Overall, our results indicate that RML is an effective CLDA algorithm for achieving rapid performance acquisition using continuous adaptation.

Closed-loop decoder adaptation shapes neural plasticity for skillful neuroprosthetic control.

Neuroplasticity may play a critical role in developing robust, naturally controlled neuroprostheses. This learning, however, is sensitive to system changes such as the neural activity used for control. The ultimate utility of neuroplasticity in real-world neuroprostheses is thus unclear. Adaptive decoding methods hold promise for improving neuroprosthetic performance in nonstationary systems. Here, we explore the use of decoder adaptation to shape neuroplasticity in two scenarios relevant for real-world neuroprostheses: nonstationary recordings of neural activity and changes in control context. Nonhuman primates learned to control a cursor to perform a reaching task using semistationary neural activity in two contexts: with and without simultaneous arm movements. Decoder adaptation was used to improve initial performance and compensate for changes in neural recordings. We show that beneficial neuroplasticity can occur alongside decoder adaptation, yielding performance improvements, skill retention, and resistance to interference from native motor networks. These results highlight the utility of neuroplasticity for real-world neuroprostheses.

Designing dynamical properties of brain-machine interfaces to optimize task-specific performance.

Brain-machine interfaces (BMIs) are dynamical systems whose properties ultimately influence performance. For instance, a 2-D BMI in which cursor position is controlled using a Kalman filter will, by default, create an attractor point that "pulls" the cursor to particular points in the workspace. If created unintentionally, such effects may not be beneficial for BMI performance. However, there have been few empirical studies exploring how various dynamical effects of closed-loop BMIs ultimately influence performance. In this work, we utilize experimental data from two macaque monkeys operating a closed-loop BMI to reach to 2-D targets and show that certain dynamical properties correlate with performance loss. We also show that other dynamical properties represent tradeoffs between naturally competing objectives, such as speed versus accuracy. These findings highlight the importance of fine-tuning the dynamical properties of closed-loop BMIs to optimize task-specific performance.

Subject-specific modulation of local field potential spectral power during brain-machine interface control in primates.

OBJECTIVE: Intracortical brain-machine interfaces (BMIs) have predominantly utilized spike activity as the control signal. However, an increasing number of studies have shown the utility of local field potentials (LFPs) for decoding motor related signals. Currently, it is unclear how well different LFP frequencies can serve as features for continuous, closed-loop BMI control. APPROACH: We demonstrate 2D continuous LFP-based BMI control using closed-loop decoder adaptation, which adapts decoder parameters to subject-specific LFP feature modulations during BMI control. We trained two macaque monkeys to control a 2D cursor in a center-out task by modulating LFP power in the 0-150 Hz range. MAIN RESULTS: While both monkeys attained control, they used different strategies involving different frequency bands. One monkey primarily utilized the low-frequency spectrum (0-80 Hz), which was highly correlated between channels, and obtained proficient performance even with a single channel. In contrast, the other monkey relied more on higher frequencies (80-150 Hz), which were less correlated between channels, and had greater difficulty with control as the number of channels decreased. We then restricted the monkeys to use only various sub-ranges (0-40, 40-80, and 80-150 Hz) of the 0-150 Hz band. Interestingly, although both monkeys performed better with some sub-ranges than others, they were able to achieve BMI control with all sub-ranges after decoder adaptation, demonstrating broad flexibility in the frequencies that could potentially be used for LFP-based BMI control. SIGNIFICANCE: Overall, our results demonstrate proficient, continuous BMI control using LFPs and provide insight into the subject-specific spectral patterns of LFP activity modulated during control.