Subdural CMOS optical probe (SCOPe) for bidirectional neural interfacing.

Optical neurotechnologies use light to interface with neurons and can monitor and manipulate neural activity with high spatial-temporal precision over large cortical extents. While there has been significant progress in miniaturizing microscope for head-mounted configurations, these existing devices are still very bulky and could never be fully implanted. Any viable translation of these technologies to human use will require a much more noninvasive, fully implantable form factor. Here, we leverage advances in microelectronics and heterogeneous optoelectronic packaging to develop a transformative, ultrathin, miniaturized device for bidirectional optical stimulation and recording: the subdural CMOS Optical Probe (SCOPe). By being thin enough to lie entirely within the subdural space of the primate brain, SCOPe defines a path for the eventual human translation of a new generation of brain-machine interfaces based on light.

A Modular Implant System for Multimodal Recording and Manipulation of the Primate Brain.

Neural circuitry can be investigated and manipulated using a variety of techniques, including electrical and optical recording and stimulation. At present, most neural interfaces are designed to accommodate a single mode of neural recording and/or manipulation, which limits the amount of data that can be extracted from a single population of neurons. To overcome these technical limitations, we developed a chronic, multi-scale, multi-modal chamber-based neural implant for use in non-human primates that accommodates electrophysiological recording and stimulation, optical manipulation, and wide-field imaging. We present key design features of the system and mechanical validation. We also present sample data from two non-human primate subjects to validate the efficacy of the design in vivo.

Eliciting naturalistic cortical responses with a sensory prosthesis via optimized microstimulation.

OBJECTIVE: Lost sensations, such as touch, could one day be restored by electrical stimulation along the sensory neural pathways. Such stimulation, when informed by electronic sensors, could provide naturalistic cutaneous and proprioceptive feedback to the user. Perceptually, microstimulation of somatosensory brain regions produces localized, modality-specific sensations, and several spatiotemporal parameters have been studied for their discernibility. However, systematic methods for encoding a wide array of naturally occurring stimuli into biomimetic percepts via multi-channel microstimulation are lacking. More specifically, generating spatiotemporal patterns for explicitly evoking naturalistic neural activation has not yet been explored. APPROACH: We address this problem by first modeling the dynamical input-output relationship between multichannel microstimulation and downstream neural responses, and then optimizing the input pattern to reproduce naturally occurring touch responses as closely as possible. MAIN RESULTS: Here we show that such optimization produces responses in the S1 cortex of the anesthetized rat that are highly similar to natural, tactile-stimulus-evoked counterparts. Furthermore, information on both pressure and location of the touch stimulus was found to be highly preserved. SIGNIFICANCE: Our results suggest that the currently presented stimulus optimization approach holds great promise for restoring naturalistic levels of sensation.

Reward value is encoded in primary somatosensory cortex and can be decoded from neural activity during performance of a psychophysical task.

Encoding of reward valence has been shown in various brain regions, including deep structures such as the substantia nigra as well as cortical structures such as the orbitofrontal cortex. While the correlation between these signals and reward valence have been shown in aggregated data comprised of many trials, little work has been done investigating the feasibility of decoding reward valence on a single trial basis. Towards this goal, one non-human primate (macaca radiata) was trained to grip and hold a target level of force in order to earn zero, one, two, or three juice rewards. The animal was informed of the impending result before reward delivery by means of a visual cue. Neural data was recorded from primary somatosensory cortex (S1) during these experiments and firing rate histograms were created following the appearance of the visual cue and used as input to a variety of classifiers. Reward valence was decoded with high levels of accuracy from S1 both in the post-cue and post-reward periods. Additionally, the proportion of units showing significant changes in their firing rates was influenced in a predictable way based on reward valence. The existence of a signal within S1 cortex that encodes reward valence could have utility for implementing reinforcement learning algorithms for brain machine interfaces. The ability to decode this reward signal in real time with limited data is paramount to the usability of such a signal in practical applications.

A tensor-product-kernel framework for multiscale neural activity decoding and control.

Brain machine interfaces (BMIs) have attracted intense attention as a promising technology for directly interfacing computers or prostheses with the brain's motor and sensory areas, thereby bypassing the body. The availability of multiscale neural recordings including spike trains and local field potentials (LFPs) brings potential opportunities to enhance computational modeling by enriching the characterization of the neural system state. However, heterogeneity on data type (spike timing versus continuous amplitude signals) and spatiotemporal scale complicates the model integration of multiscale neural activity. In this paper, we propose a tensor-product-kernel-based framework to integrate the multiscale activity and exploit the complementary information available in multiscale neural activity. This provides a common mathematical framework for incorporating signals from different domains. The approach is applied to the problem of neural decoding and control. For neural decoding, the framework is able to identify the nonlinear functional relationship between the multiscale neural responses and the stimuli using general purpose kernel adaptive filtering. In a sensory stimulation experiment, the tensor-product-kernel decoder outperforms decoders that use only a single neural data type. In addition, an adaptive inverse controller for delivering electrical microstimulation patterns that utilizes the tensor-product kernel achieves promising results in emulating the responses to natural stimulation.

Neural decoding with kernel-based metric learning.

In studies of the nervous system, the choice of metric for the neural responses is a pivotal assumption. For instance, a well-suited distance metric enables us to gauge the similarity of neural responses to various stimuli and assess the variability of responses to a repeated stimulus-exploratory steps in understanding how the stimuli are encoded neurally. Here we introduce an approach where the metric is tuned for a particular neural decoding task. Neural spike train metrics have been used to quantify the information content carried by the timing of action potentials. While a number of metrics for individual neurons exist, a method to optimally combine single-neuron metrics into multineuron, or population-based, metrics is lacking. We pose the problem of optimizing multineuron metrics and other metrics using centered alignment, a kernel-based dependence measure. The approach is demonstrated on invasively recorded neural data consisting of both spike trains and local field potentials. The experimental paradigm consists of decoding the location of tactile stimulation on the forepaws of anesthetized rats. We show that the optimized metrics highlight the distinguishing dimensions of the neural response, significantly increase the decoding accuracy, and improve nonlinear dimensionality reduction methods for exploratory neural analysis.

Information-theoretic metric learning: 2-D linear projections of neural data for visualization.

Intracortical neural recordings are typically high-dimensional due to many electrodes, channels, or units and high sampling rates, making it very difficult to visually inspect differences among responses to various conditions. By representing the neural response in a low-dimensional space, a researcher can visually evaluate the amount of information the response carries about the conditions. We consider a linear projection to 2-D space that also parametrizes a metric between neural responses. The projection, and corresponding metric, should preserve class-relevant information pertaining to different behavior or stimuli. We find the projection as a solution to the information-theoretic optimization problem of maximizing the information between the projected data and the class labels. The method is applied to two datasets using different types of neural responses: motor cortex neuronal firing rates of a macaque during a center-out reaching task, and local field potentials in the somatosensory cortex of a rat during tactile stimulation of the forepaw. In both cases, projected data points preserve the natural topology of targets or peripheral touch sites. Using the learned metric on the neural responses increases the nearest-neighbor classification rate versus the original data; thus, the metric is tuned to distinguish among the conditions.

Decoding stimuli from multi-source neural responses.

Spike trains and local field potentials (LFPs) are two different manifestations of neural activity recorded simultaneously from the same electrode array and contain complementary information of stimuli or behaviors. This paper proposes a tensor product kernel based decoder, which allows modeling the sample from different sources individually and mapping them onto the same reproducing kernel Hilbert space (RKHS) defined by the tensor product of the individual kernels for each source, where linear regression is conducted to identify the nonlinear mapping from the multi-type neural responses to the stimuli. The decoding results of the rat sensory stimulation experiment show that the tensor-product-kernel-based decoder outperforms the decoders with either single-type neural activities.

Subspace matching thalamic microstimulation to tactile evoked potentials in rat somatosensory cortex.

We show experimental results that the evoked local field potentials of the rat somatosensory cortex from natural tactile touch of forepaw digits and matched thalamic microstimulation can be qualitatively and quantitively similar. In ongoing efforts to optimize the microstimulation settings (e.g., location, amplitude, etc.) to match the natural response, we investigate whether subspace projection methods, specifically the eigenface approach proposed in the computer vision community (Turk and Pentland 1991 [1]), can be used to choose the parameters of microstimulation such that the response matches a single tactile touch realization. Since the evoked potentials from multiple electrodes are high dimensional spatio-temporal data, the subspace projections improve computational efficiency and can reduce the effect of noisy realizations. In computing the PCA projections we use the peristimulus averages instead of the realizations. The dataset is pruned of unreliable stimulation types. A new subspace is computed for the pruned stimulation type, and is used to estimate a sequence of microstimulations to best match the natural responses. This microstimulation sequence is applied in vivo and quantitative analysis shows that per realization matching does statistically better than choosing randomly from the pruned subset.

Optimizing microstimulation using a reinforcement learning framework.

The ability to provide sensory feedback is desired to enhance the functionality of neuroprosthetics. Somatosensory feedback provides closed-loop control to the motor system, which is lacking in feedforward neuroprosthetics. In the case of existing somatosensory function, a template of the natural response can be used as a template of desired response elicited by electrical microstimulation. In the case of no initial training data, microstimulation parameters that produce responses close to the template must be selected in an online manner. We propose using reinforcement learning as a framework to balance the exploration of the parameter space and the continued selection of promising parameters for further stimulation. This approach avoids an explicit model of the neural response from stimulation. We explore a preliminary architecture--treating the task as a k-armed bandit--using offline data recorded for natural touch and thalamic microstimulation, and we examine the methods efficiency in exploring the parameter space while concentrating on promising parameter forms. The best matching stimulation parameters, from k = 68 different forms, are selected by the reinforcement learning algorithm consistently after 334 realizations.