Improved Grip Force Prediction Using a Loss Function that Penalizes Reward Related Neural Information.

Neural activity in the sensorimotor cortices has been previously shown to correlate with kinematics, kinetics, and non-sensorimotor variables, such as reward. In this work, we compare the grip force offline Brain Machine Interface (BMI) prediction performance, of a simple artificial neural network (ANN), under two loss functions: the standard mean squared error (MSE) and a modified reward penalized mean squared error (RP_MSE), which penalizes for correlation between reward and grip force. Our results show that the ANN performs significantly better under the RP_MSE loss function in three brain regions: dorsal premotor cortex (PMd), primary motor cortex (M1) and the primary somatosensory cortex (S1) by approximately 6%.

Reward-dependent Graded Suppression of Sensorimotor Beta-band Local Field Potentials During an Arm Reaching Task in NHP.

A better understanding of reward signaling in the sensorimotor cortices can aid in developing Reinforcement Learning-based Brain-Computer Interfaces (RLBCI) for restoration of movement functions with fewer implants. Brain-computer interfaces (BCIs) using local field potentials (LFPs) have recently achieved performance comparable to spike-BCIs [1]. With superior stability over time, LFPs may be the preferred signal for BCIs. We show that sensorimotor LFPs can provide reward level information (R1 - R3) like spikes[2]. We used a cued reward-level reaching task in which reward information was temporally dissociated from movement information. This allowed the study of reward- and movement-related modulations in LFPs. We recorded simultaneously from contralateral primary -somatosensory (S1), -motor (M1), and the dorsal premotor (PMd) cortices in a female Macaca Mulatta. We found that all three cortices' average beta band (14-30 Hz) amplitude showed robust modulation with reward levels during the cue presentation period. Such modulation was consistently observed after controlling for cue color, differences in behavioral variables, and electromyogram (EMG) activity. Statistical amplitude analysis showed that reward level could be extracted from the simple LFP feature of beta band amplitude, even before a reaching target appeared, and no specific reach plan could be developed. Clinical Relevance - The availability of reward-related signals in the sensorimotor cortical (S1, M1,and PMd) LFPs' prior to movement planning opens new avenues to build RLBCIs with fewer implants recording fewer sites among different cortices Reward and motivational representations derived from LFPs compared to spikes allow the development of long-term clinical applications given LFP's stability and ease of recording over long periods.

Similarities Between Somatosensory Cortical Responses Induced via Natural Touch and Microstimulation in the Ventral Posterior Lateral Thalamus in Macaques.

Lost sensations, such as touch, could be restored by microstimulation (MiSt) along the sensory neural substrate. Such neuroprosthetic sensory information can be used as feedback from an invasive brain-machine interface (BMI) to control a robotic arm/hand, such that tactile and proprioceptive feedback from the sensorized robotic arm/hand is directly given to the BMI user. Microstimulation in the human somatosensory thalamus (Vc) has been shown to produce somatosensory perceptions. However, until recently, systematic methods for using thalamic stimulation to evoke naturalistic touch perceptions were lacking. We have recently presented rigorous methods for determining a mapping between ventral posterior lateral thalamus (VPL) MiSt, and neural responses in the somatosensory cortex (S1), in a rodent model (Choi et al., 2016; Choi and Francis, 2018). Our technique minimizes the difference between S1 neural responses induced by natural sensory stimuli and those generated via VPL MiSt. Our goal is to develop systems that know what neural response a given MiSt will produce and possibly allow the development of natural "sensation." To date, our optimization has been conducted in the rodent model and simulations. Here, we present data from simple non-optimized thalamic MiSt during peri-operative experiments, where we used MiSt in the VPL of macaques, which have a somatosensory system more like humans, as compared to our previous rat work (Li et al., 2014; Choi et al., 2016). We implanted arrays of microelectrodes across the hand area of the macaque S1 cortex as well as in the VPL. Multi and single-unit recordings were used to compare cortical responses to natural touch and thalamic MiSt in the anesthetized state. Post-stimulus time histograms were highly correlated between the VPL MiSt and natural touch modalities, adding support to the use of VPL MiSt toward producing a somatosensory neuroprosthesis in humans.

Normalization by valence and motivational intensity in the sensorimotor cortices (PMd, M1, and S1).

Our brain's ability to represent vast amounts of information, such as continuous ranges of reward spanning orders of magnitude, with limited dynamic range neurons, may be possible due to normalization. Recently our group and others have shown that the sensorimotor cortices are sensitive to reward value. Here we ask if psychological affect causes normalization of the sensorimotor cortices by modulating valence and motivational intensity. We had two non-human primates (NHP) subjects (one male bonnet macaque and one female rhesus macaque) make visually cued grip-force movements while simultaneously cueing the level of possible reward if successful, or timeout punishment, if unsuccessful. We recorded simultaneously from 96 electrodes in each the following: caudal somatosensory, rostral motor, and dorsal premotor cortices (cS1, rM1, PMd). We utilized several normalization models for valence and motivational intensity in all three regions. We found three types of divisive normalized relationships between neural activity and the representation of valence and motivation, linear, sigmodal, and hyperbolic. The hyperbolic relationships resemble receptive fields in psychological affect space, where a unit is susceptible to a small range of the valence/motivational space. We found that these cortical regions have both strong valence and motivational intensity representations.

Mirror neurons are modulated by grip force and reward expectation in the sensorimotor cortices (S1, M1, PMd, PMv).

Mirror Neurons (MNs) respond similarly when primates make or observe grasping movements. Recent work indicates that reward expectation influences rostral M1 (rM1) during manual, observational, and Brain Machine Interface (BMI) reaching movements. Previous work showed MNs are modulated by subjective value. Here we expand on the above work utilizing two non-human primates (NHPs), one male Macaca Radiata (NHP S) and one female Macaca Mulatta (NHP P), that were trained to perform a cued reward level isometric grip-force task, where the NHPs had to apply visually cued grip-force to move and transport a virtual object. We found a population of (S1 area 1-2, rM1, PMd, PMv) units that significantly represented grip-force during manual and observational trials. We found the neural representation of visually cued force was similar during observational trials and manual trials for the same units; however, the representation was weaker during observational trials. Comparing changes in neural time lags between manual and observational tasks indicated that a subpopulation fit the standard MN definition of observational neural activity lagging the visual information. Neural activity in (S1 areas 1-2, rM1, PMd, PMv) significantly represented force and reward expectation. In summary, we present results indicating that sensorimotor cortices have MNs for visually cued force and value.

Noise-Correlation Is Modulated by Reward Expectation in the Primary Motor Cortex Bilaterally During Manual and Observational Tasks in Primates.

Reward modulation is represented in the motor cortex (M1) and could be used to implement more accurate decoding models to improve brain-computer interfaces (BCIs; Zhao et al., 2018). Analyzing trial-to-trial noise-correlations between neural units in the presence of rewarding (R) and non-rewarding (NR) stimuli adds to our understanding of cortical network dynamics. We utilized Pearson's correlation coefficient to measure shared variability between simultaneously recorded units (32-112) and found significantly higher noise-correlation and positive correlation between the populations' signal- and noise-correlation during NR trials as compared to R trials. This pattern is evident in data from two non-human primates (NHPs) during single-target center out reaching tasks, both manual and action observation versions. We conducted a mean matched noise-correlation analysis to decouple known interactions between event-triggered firing rate changes and neural correlations. Isolated reward discriminatory units demonstrated stronger correlational changes than units unresponsive to reward firing rate modulation, however, the qualitative response was similar, indicating correlational changes within the network as a whole can serve as another information channel to be exploited by BCIs that track the underlying cortical state, such as reward expectation, or attentional modulation. Reward expectation and attention in return can be utilized with reinforcement learning (RL) towards autonomous BCI updating.

Near Perfect Neural Critic from Motor Cortical Activity Toward an Autonomously Updating Brain Machine Interface.

We are developing an autonomously updating brain machine interface (BMI) utilizing reinforcement learning principles. One aspect of this system is a neural critic that determines reward expectations from neural activity. This critic is then used to update a BMI decoder toward an improved performance from the user's perspective. Here we demonstrate the ability of a neural critic to classify trial reward value given activity from the primary motor cortex (M1), using neural features from single/multi units (SU/MU), and local field potentials (LFPs) with prediction accuracies up to 97% correct. A nonhuman primate subject conducted a cued center out reaching task, either manually, or observationally. The cue indicated the reward value of a trial. Features such as power spectral density (PSD) of the LFPs and spike-field coherence (SFC) between SU/MU and corresponding LFPs were calculated and used as inputs to several classifiers. We conclude that hybrid features of PSD and SFC show higher classification performance than PSD or SFC alone (accuracy was 92% for manual tasks, and 97% for observational). In the future, we will employ these hybrid features toward our autonomously updating BMI.

Persistent Increases of PKMζ in Sensorimotor Cortex Maintain Procedural Long-Term Memory Storage.

Procedural motor learning and memory are accompanied by changes in synaptic plasticity, neural dynamics, and synaptogenesis. Missing is information on the spatiotemporal dynamics of the molecular machinery maintaining these changes. Here we examine whether persistent increases in PKMζ, an atypical protein kinase C (PKC) isoform, store long-term memory for a reaching task in rat sensorimotor cortex that could reveal the sites of procedural memory storage. Specifically, perturbing PKMζ synthesis (via antisense oligodeoxynucleotides) and blocking atypical PKC activity (via zeta inhibitory peptide [ZIP]) in S1/M1 disrupts and erases long-term motor memory maintenance, indicating atypical PKCs and specifically PKMζ store consolidated long-term procedural memories. Immunostaining reveals that PKMζ increases in S1/M1 layers II/III and V as performance improved to an asymptote. After storage for 1 month without reinforcement, the increase in M1 layer V persists without decrement. Thus, the persistent increases in PKMζ that store long-term procedural memory are localized to the descending output layer of the primary motor cortex.

Tactile information processing in primate hand somatosensory cortex (S1) during passive arm movement.

Motor output mostly depends on sensory input, which also can be affected by action. To further our understanding of how tactile information is processed in the primary somatosensory cortex (S1) in dynamic environments, we recorded neural responses to tactile stimulation of the hand in three awake monkeys under arm/hand passive movement and rest. We found that neurons generally responded to tactile stimulation under both conditions and were modulated by movement: with a higher baseline firing rate, a suppressed peak rate, and a smaller dynamic range during passive movement than during rest, while the area under the response curve was stable across these two states. By using an information theory-based method, the mutual information between tactile stimulation and neural responses was quantified with rate and spatial coding models under the two conditions. The two potential encoding models showed different contributions depending on behavioral contexts. Tactile information encoded with rate coding from individual units was lower than spatial coding of unit pairs, especially during movement; however, spatial coding had redundant information between unit pairs. Passive movement regulated the mutual information, and such regulation might play different roles depending on the encoding strategies used. The underlying mechanisms of our observation most likely come from a bottom-up strategy, where neurons in S1 were regulated through the activation of the peripheral tactile/proprioceptive receptors and the interactions between these different types of information.

An electric field model for prediction of somatosensory (S1) cortical field potentials induced by ventral posterior lateral (VPL) thalamic microstimulation.

Microstimulation (MiSt) is used experimentally and clinically to activate localized populations of neural elements. However, it is difficult to predict-and subsequently control-neural responses to simultaneous current injection through multiple electrodes in an array. This is due to the unknown locations of neuronal elements in the extracellular medium that are excited by the superposition of multiple parallel current sources. We, therefore, propose a model that maps the computed electric field in the 3-D space surrounding the stimulating electrodes in one brain region to the local field potential (LFP) fluctuations evoked in a downstream region. Our model is trained with the recorded LFP waveforms in the primary somatosensory cortex (S1) resulting from MiSt applied in multiple electrode configurations in the ventral posterolateral nucleus (VPL) of the quiet awake rat. We then predict the cortical responses to MiSt in "novel" electrode configurations, a result that suggests that this technique could aid in the design of spatially optimized MiSt patterns through a multielectrode array.

Neuroplasticity of the sensorimotor cortex during learning.

We will discuss some of the current issues in understanding plasticity in the sensorimotor (SM) cortices on the behavioral, neurophysiological, and synaptic levels. We will focus our paper on reaching and grasping movements in the rat. In addition, we will discuss our preliminary work utilizing inhibition of protein kinase Mζ (PKMζ), which has recently been shown necessary and sufficient for the maintenance of long-term potentiation (LTP) (Ling et al., 2002). With this new knowledge and inhibitors to this system, as well as the ability to overexpress this system, we can start to directly modulate LTP and determine its influence on behavior as well as network level processing dependent at least in part due to this form of LTP. We will also briefly introduce the use of brain machine interface (BMI) paradigms to ask questions about sensorimotor plasticity and discuss current analysis techniques that may help in our understanding of neuroplasticity.

Erasing sensorimotor memories via PKMzeta inhibition.

Sensorimotor cortex has a role in procedural learning. Previous studies suggested that this learning is subserved by long-term potentiation (LTP), which is in turn maintained by the persistently active kinase, protein kinase Mzeta (PKMzeta). Whereas the role of PKMzeta in animal models of declarative knowledge is established, its effect on procedural knowledge is not well understood. Here we show that PKMzeta inhibition, via injection of zeta inhibitory peptide (ZIP) into the rat sensorimotor cortex, disrupts sensorimotor memories for a skilled reaching task even after several weeks of training. The rate of relearning the task after the memory disruption by ZIP was indistinguishable from the rate of initial learning, suggesting no significant savings after the memory loss. These results indicate a shared molecular mechanism of storage for declarative and procedural forms of memory.