Evaluation of abstract rule-based associations in the human premotor cortex during passive observation.

Decision-making often manifests in behavior, typically yielding overt motor actions. This complex process requires the registration of sensory information with one's internal representation of the current context, before a categorical judgment of the most appropriate motor behavior can be issued. The construct concept of embodied decision-making encapsulates this sequence of complex processes, whereby behaviorally salient information from the environment is represented in an abstracted space of potential motor actions rather than only in an abstract cognitive "decision" space. Theoretical foundations and some empirical evidence account for support the involvement of premotor cortical circuits in embodied cognitive functions. Animal models show that premotor circuits participate in the registration and evaluation of actions performed by peers in social situations, that is, prior to controlling one's voluntary movements guided by arbitrary stimulus-response rules. However, such evidence from human data is currently limited. Here we used time-resolved magnetoencephalography imaging to characterize activations of the premotor cortex as human participants observed arbitrary, non-biological visual stimuli that either respected or violated a simple stimulus-response association rule. The participants had learned this rule previously, either actively, by performing a motor task (active learning), or passively, by observing a computer perform the same task (passive learning). We discovered that the human premotor cortex is activated during the passive observation of the correct execution of a sequence of events according to a rule learned previously. Premotor activation also differs when the subjects observe incorrect stimulus sequences. These premotor effects are present even when the observed events are of a non-motor, abstract nature, and even when the stimulus-response association rule was learned via passive observations of a computer agent performing the task, without requiring overt motor actions from the human participant. We found evidence of these phenomena by tracking cortical beta-band signaling in temporal alignment with the observation of task events and behavior. We conclude that premotor cortical circuits that are typically engaged during voluntary motor behavior are also involved in the interpretation of events of a non-ecological, unfamiliar nature but related to a learned abstract rule. As such, the present study provides the first evidence of neurophysiological processes of embodied decision-making in human premotor circuits when the observed events do not involve motor actions of a third party.

Emerging ideas and tools to study the emergent properties of the cortical neural circuits for voluntary motor control in non-human primates.

For years, neurophysiological studies of the cerebral cortical mechanisms of voluntary motor control were limited to single-electrode recordings of the activity of one or a few neurons at a time. This approach was supported by the widely accepted belief that single neurons were the fundamental computational units of the brain (the "neuron doctrine"). Experiments were guided by motor-control models that proposed that the motor system attempted to plan and control specific parameters of a desired action, such as the direction, speed or causal forces of a reaching movement in specific coordinate frameworks, and that assumed that the controlled parameters would be expressed in the task-related activity of single neurons. The advent of chronically implanted multi-electrode arrays about 20 years ago permitted the simultaneous recording of the activity of many neurons. This greatly enhanced the ability to study neural control mechanisms at the population level. It has also shifted the focus of the analysis of neural activity from quantifying single-neuron correlates with different movement parameters to probing the structure of multi-neuron activity patterns to identify the emergent computational properties of cortical neural circuits. In particular, recent advances in "dimension reduction" algorithms have attempted to identify specific covariance patterns in multi-neuron activity which are presumed to reflect the underlying computational processes by which neural circuits convert the intention to perform a particular movement into the required causal descending motor commands. These analyses have led to many new perspectives and insights on how cortical motor circuits covertly plan and prepare to initiate a movement without causing muscle contractions, transition from preparation to overt execution of the desired movement, generate muscle-centered motor output commands, and learn new motor skills. Progress is also being made to import optical-imaging and optogenetic toolboxes from rodents to non-human primates to overcome some technical limitations of multi-electrode recording technology.

Macaque dorsal premotor cortex exhibits decision-related activity only when specific stimulus-response associations are known.

How deliberation on sensory cues and action selection interact in decision-related brain areas is still not well understood. Here, monkeys reached to one of two targets, whose colors alternated randomly between trials, by discriminating the dominant color of a checkerboard cue composed of different numbers of squares of the two target colors in different trials. In a Targets First task the colored targets appeared first, followed by the checkerboard; in a Checkerboard First task, this order was reversed. After both cues appeared in both tasks, responses of dorsal premotor cortex (PMd) units covaried with action choices, strength of evidence for action choices, and RTs- hallmarks of decision-related activity. However, very few units were modulated by checkerboard color composition or the color of the chosen target, even during the checkerboard deliberation epoch of the Checkerboard First task. These findings implicate PMd in the action-selection but not the perceptual components of the decision-making process in these tasks.

Correlation-based model of artificially induced plasticity in motor cortex by a bidirectional brain-computer interface.

Experiments show that spike-triggered stimulation performed with Bidirectional Brain-Computer-Interfaces (BBCI) can artificially strengthen connections between separate neural sites in motor cortex (MC). When spikes from a neuron recorded at one MC site trigger stimuli at a second target site after a fixed delay, the connections between sites eventually strengthen. It was also found that effective spike-stimulus delays are consistent with experimentally derived spike-timing-dependent plasticity (STDP) rules, suggesting that STDP is key to drive these changes. However, the impact of STDP at the level of circuits, and the mechanisms governing its modification with neural implants remain poorly understood. The present work describes a recurrent neural network model with probabilistic spiking mechanisms and plastic synapses capable of capturing both neural and synaptic activity statistics relevant to BBCI conditioning protocols. Our model successfully reproduces key experimental results, both established and new, and offers mechanistic insights into spike-triggered conditioning. Using analytical calculations and numerical simulations, we derive optimal operational regimes for BBCIs, and formulate predictions concerning the efficacy of spike-triggered conditioning in different regimes of cortical activity.

Dorsal premotor cortex: neural correlates of reach target decisions based on a color-location matching rule and conflicting sensory evidence.

We recorded single-neuron activity in dorsal premotor (PMd) and primary motor cortex (M1) of two monkeys in a reach-target selection task. The monkeys chose between two color-coded potential targets by determining which target's color matched the predominant color of a multicolored checkerboard-like Decision Cue (DC). Different DCs contained differing numbers of colored squares matching each target. The DCs provided evidence about the correct target ranging from unambiguous (one color only) to very ambiguous and conflicting (nearly equal number of squares of each color). Differences in choice behavior (reach response times and success rates as a function of DC ambiguity) of the monkeys suggested that each applied a different strategy for using the target-choice evidence in the DCs. Nevertheless, the appearance of the DCs evoked a transient coactivation of PMd neurons preferring both potential targets in both monkeys. Reach response time depended both on how long it took activity to increase in neurons that preferred the chosen target and on how long it took to suppress the activity of neurons that preferred the rejected target, in both correct-choice and error-choice trials. These results indicate that PMd neurons in this task are not activated exclusively by a signal proportional to the net color bias of the DCs. They are instead initially modulated by the conflicting evidence supporting both response choices; final target selection may result from a competition between representations of the alternative choices. The results also indicate a temporal overlap between action selection and action initiation processes in PMd and M1.

Motor cortex single-neuron and population contributions to compensation for multiple dynamic force fields.

To elucidate how primary motor cortex (M1) neurons contribute to the performance of a broad range of different and even incompatible motor skills, we trained two monkeys to perform single-degree-of-freedom elbow flexion/extension movements that could be perturbed by a variety of externally generated force fields. Fields were presented in a pseudorandom sequence of trial blocks. Different computer monitor background colors signaled the nature of the force field throughout each block. There were five different force fields: null field without perturbing torque, assistive and resistive viscous fields proportional to velocity, a resistive elastic force field proportional to position and a resistive viscoelastic field that was the linear combination of the resistive viscous and elastic force fields. After the monkeys were extensively trained in the five field conditions, neural recordings were subsequently made in M1 contralateral to the trained arm. Many caudal M1 neurons altered their activity systematically across most or all of the force fields in a manner that was appropriate to contribute to the compensation for each of the fields. The net activity of the entire sample population likewise provided a predictive signal about the differences in the time course of the external forces encountered during the movements across all force conditions. The neurons showed a broad range of sensitivities to the different fields, and there was little evidence of a modular structure by which subsets of M1 neurons were preferentially activated during movements in specific fields or combinations of fields.

Reach target selection in humans using ambiguous decision cues containing variable amounts of conflicting sensory evidence supporting each target choice.

Human subjects chose between two color-coded reach targets using multicolored checkerboard-like decision cues (DCs) that presented variable amounts of conflicting sensory evidence supporting both target choices. Different DCs contained different numbers of small squares of the two target colors. The most ambiguous DCs contained nearly equal numbers of squares of both target colors. The subjects reached as soon as they selected a target after the appearance of the DC ("choose-and-go" task). The choice behavior of the subjects showed many similarities to prior studies using other stimulus properties (e.g., visual motion coherence, brightness), including progressively longer response times and higher target-choice error rates for more ambiguous DCs. However, certain trends in their choice behavior could not be fully captured by simple drift-diffusion models. Allowing the subjects to view the DCs for a period of time before presenting the targets ("match-to-sample" task) resulted in much shorter response times overall, but also revealed a reluctance of subjects to commit to a decision about the predominant color of the more ambiguous DCs during the initial extended observation period. Model processing and simulation analyses suggest that the subjects might adjust the dynamics of their decision-making process on a trial-to-trial basis in response to the variable level of ambiguous and conflicting evidence in different DCs between trials.

Conditioning and time representation in long short-term memory networks.

Dopaminergic models based on the temporal-difference learning algorithm usually do not differentiate trace from delay conditioning. Instead, they use a fixed temporal representation of elapsed time since conditioned stimulus onset. Recently, a new model was proposed in which timing is learned within a long short-term memory (LSTM) artificial neural network representing the cerebral cortex (Rivest et al. in J Comput Neurosci 28(1):107-130, 2010). In this paper, that model's ability to reproduce and explain relevant data, as well as its ability to make interesting new predictions, are evaluated. The model reveals a strikingly different temporal representation between trace and delay conditioning since trace conditioning requires working memory to remember the past conditioned stimulus while delay conditioning does not. On the other hand, the model predicts no important difference in DA responses between those two conditions when trained on one conditioning paradigm and tested on the other. The model predicts that in trace conditioning, animal timing starts with the conditioned stimulus offset as opposed to its onset. In classical conditioning, it predicts that if the conditioned stimulus does not disappear after the reward, the animal may expect a second reward. Finally, the last simulation reveals that the buildup of activity of some units in the networks can adapt to new delays by adjusting their rate of integration. Most importantly, the paper shows that it is possible, with the proposed architecture, to acquire discharge patterns similar to those observed in dopaminergic neurons and in the cerebral cortex on those tasks simply by minimizing a predictive cost function.

Descending corticospinal control of intersegmental dynamics.

To make an accurate movement, the CNS has to overcome the inherent complexities of the multijoint limb. For example, interaction torques arise when motion of individual arm segments propagates to adjacent segments causing their movement without any muscle contractions. Since these passive joint torques significantly add to the overall torques generated by active muscular contractions, they must be taken into account during planning or execution of goal-directed movements. We investigated the role of the corticospinal tract in compensating for the interaction torques during arm movements in humans. Twelve subjects reached to visual targets with their arm supported by a robotic exoskeleton. Reaching to one target was accompanied by interaction torques that assisted the movement, while reaching to the other target was accompanied by interaction torques that resisted the movement. Corticospinal excitability was assessed at different times during movement using single-pulse transcranial magnetic stimulation (TMS) over the upper-arm region of M1 (primary motor cortex). We found that TMS responses in shoulder monoarticular and elbow-shoulder biarticular muscles changed together with the interaction torques during movements in which the interaction torques were resistive. In contrast, TMS responses did not correlate with assistive interaction torques or with co-contraction. This suggests that the descending motor command includes compensation for passive limb dynamics. Furthermore, our results suggest that compensation for interaction torques involves the biarticular muscles, which span both shoulder and elbow joints and are in a biomechanically advantageous position to provide such compensation.

Colored context cues can facilitate the ability to learn and to switch between multiple dynamical force fields.

We tested the efficacy of color context cues during adaptation to dynamic force fields. Four groups of human subjects performed elbow flexion/extension movements to move a cursor between targets on a monitor while encountering a resistive (Vr) or assistive (Va) viscous force field. They performed two training sets of 256 trials daily, for 10 days. The monitor background color changed (red, green) every four successful trials but provided different degrees of force field context information to each group. For the irrelevant-cue groups, the color changed every four trials, but one group encountered only the Va field and the other only the Vr field. For the reliable-cue group, the force field alternated between Va and Vr each time the monitor changed color (Vr, red; Va, green). For the unreliable-cue group, the force field changed between Va and Vr pseudorandomly at each color change. All subjects made increasingly stereotyped movements over 10 training days. Reliable-cue subjects typically learned the association between color cues and fields and began to make predictive changes in motor output at each color change during the first day. Their performance continued to improve over the remaining days. Unreliable-cue subjects also improved their performance across training days but developed a strategy of probing the nature of the field at each color change by emitting a default motor response and then adjusting their motor output in subsequent trials. These findings show that subjects can extract explicit and implicit information from color context cues during force field adaptation.

Learning to move machines with the mind.

Brain-computer interfaces (BCIs) extract signals from neural activity to control remote devices ranging from computer cursors to limb-like robots. They show great potential to help patients with severe motor deficits perform everyday tasks without the constant assistance of caregivers. Understanding the neural mechanisms by which subjects use BCI systems could lead to improved designs and provide unique insights into normal motor control and skill acquisition. However, reports vary considerably about how much training is required to use a BCI system, the degree to which performance improves with practice and the underlying neural mechanisms. This review examines these diverse findings, their potential relationship with motor learning during overt arm movements, and other outstanding questions concerning the volitional control of BCI systems.

Neural mechanisms for interacting with a world full of action choices.

The neural bases of behavior are often discussed in terms of perceptual, cognitive, and motor stages, defined within an information processing framework that was originally inspired by models of human abstract problem solving. Here, we review a growing body of neurophysiological data that is difficult to reconcile with this influential theoretical perspective. As an alternative foundation for interpreting neural data, we consider frameworks borrowed from ethology, which emphasize the kinds of real-time interactive behaviors that animals have engaged in for millions of years. In particular, we discuss an ethologically-inspired view of interactive behavior as simultaneous processes that specify potential motor actions and select between them. We review how recent neurophysiological data from diverse cortical and subcortical regions appear more compatible with this parallel view than with the classical view of serial information processing stages.

Alternative time representation in dopamine models.

Dopaminergic neuron activity has been modeled during learning and appetitive behavior, most commonly using the temporal-difference (TD) algorithm. However, a proper representation of elapsed time and of the exact task is usually required for the model to work. Most models use timing elements such as delay-line representations of time that are not biologically realistic for intervals in the range of seconds. The interval-timing literature provides several alternatives. One of them is that timing could emerge from general network dynamics, instead of coming from a dedicated circuit. Here, we present a general rate-based learning model based on long short-term memory (LSTM) networks that learns a time representation when needed. Using a naïve network learning its environment in conjunction with TD, we reproduce dopamine activity in appetitive trace conditioning with a constant CS-US interval, including probe trials with unexpected delays. The proposed model learns a representation of the environment dynamics in an adaptive biologically plausible framework, without recourse to delay lines or other special-purpose circuits. Instead, the model predicts that the task-dependent representation of time is learned by experience, is encoded in ramp-like changes in single-neuron activity distributed across small neural networks, and reflects a temporal integration mechanism resulting from the inherent dynamics of recurrent loops within the network. The model also reproduces the known finding that trace conditioning is more difficult than delay conditioning and that the learned representation of the task can be highly dependent on the types of trials experienced during training. Finally, it suggests that the phasic dopaminergic signal could facilitate learning in the cortex.

Evidence for a proprioception-based rapid on-line error correction mechanism for hand orientation during reaching movements in blind subjects.

The contribution of visual experience to the perception and sensorimotor control of spatial orientation of the hand was investigated in blind subjects. In "orientation-matching" tasks, subjects aligned a match handle held in their right hand to a target handle held in their left hand and fixed in different orientations, with both arms outstretched. In "letter-posting" task 1, the same subjects reached out and simultaneously oriented their right hand to insert the match handle into a target slot fixed in the same range of orientations. Orientations were signaled proprioceptively by a reference handle held in the left hand. Final hand orientation errors were smaller when blind subjects simultaneously reached out and rotated their hand to insert the match handle into the target slot in letter-posting task 1 than when they held their arm extended and aligned the handles in the orientation-matching task. In letter-posting task 2, blind subjects first aligned their hand to the orientation of the target and then subsequently reached to the target with the instruction to not change hand orientation during reaching. Despite the instruction, subjects showed a reduction in absolute hand orientation error from the beginning to the end of the reach. In all tasks, performance of blind subjects was very similar to that of blindfolded normally sighted subjects. These findings provide the first evidence of an automatic on-line error-correction mechanism for hand orientation guided only by proprioceptive inputs during reaching in blind subjects, and reveal that the on-line mechanism does not depend on prior visual experience.

From intention to action: motor cortex and the control of reaching movements.

The motor cortex was experimentally identified more than a century ago using surface electrical stimulation and lesions. Those first studies initiated a debate about the role of the motor cortex in the control of voluntary movement that continues to this day. The main issue concerns the degree to which the descending motor command emanating from the motor cortex specifies the spatiotemporal form of a movement or its causal forces, torques and muscle activity. The neurophysiological evidence supports both perspectives. This chapter surveys some of that evidence, with particular focus on the latter, more 'traditional', role of motor cortex.

A functional approach to modeling M1 single-unit activity recorded in three primate motor control studies.

When monkeys make movements with or without external force perturbations, or generate isometric forces in different directions from different workspace positions, primary motor cortex (M1) cell activity shows systematic changes in directional tuning and in force-generation gains as a function of arm posture. However, it may be simplistic to assume most control intelligence is in the cortex while the brainstem and especially the spinal cord do little more than passively implement pontifical descending commands. More recent studies like [1-4] do suggest a different perspective. Furthermore, systematic changes in directionality of M1 cell and limb muscle EMG activity may stem partly from the feedback (aka reflex) loops, physical properties of limb biomechanics, muscle anisotropy and force production nonlinearities, and their interplay with task conditions, and not only due to predictive feedforward central commands.