From motor planning to execution: a sensorimotor loop perspective.

How is an evanescent wish to move translated into a concrete action? This simple question and puzzling miracle remains a focal point of motor systems neuroscience. Where does the difficulty lie? A great deal has been known about biomechanics for quite some time. More recently, there have been significant advances in our understanding of how the spinal system is organized into modules corresponding to spinal synergies, which are fixed patterns of multimuscle recruitment. But much less is known about how the supraspinal system recruits these synergies in the correct spatiotemporal pattern to effectively control movement. We argue that what makes the problem of supraspinal control so difficult is that it emerges as a result of multiple convergent and redundant sensorimotor loops. Because these loops are convergent, multiple modes of information are mixed before being sent to the spinal system; because they are redundant, information is overlapping such that a mechanism must exist to eliminate the redundancy before the signal is sent to the spinal system. Given these complex interactions, simple correlation analyses between movement variables and neural activity are likely to render a confusing and inconsistent picture. Here, we suggest that the perspective of sensorimotor loops might help in achieving a better systems-level understanding. Furthermore, state-of-the-art techniques in neurotechnology, such as optogenetics, appear to be well suited for investigating the problem of motor control at the level of loops.

Representation of Muscle Synergies in the Primate Brain.

Evidence suggests that the CNS uses motor primitives to simplify movement control, but whether it actually stores primitives instead of computing solutions on the fly to satisfy task demands is a controversial and still-unanswered possibility. Also in contention is whether these primitives take the form of time-invariant muscle coactivations ("spatial" synergies) or time-varying muscle commands ("spatiotemporal" synergies). Here, we examined forelimb muscle patterns and motor cortical spiking data in rhesus macaques (Macaca mulatta) handling objects of variable shape and size. From these data, we extracted both spatiotemporal and spatial synergies using non-negative decomposition. Each spatiotemporal synergy represents a sequence of muscular or neural activations that appeared to recur frequently during the animals' behavior. Key features of the spatiotemporal synergies (including their dimensionality, timing, and amplitude modulation) were independently observed in the muscular and neural data. In addition, both at the muscular and neural levels, these spatiotemporal synergies could be readily reconstructed as sequential activations of spatial synergies (a subset of those extracted independently from the task data), suggestive of a hierarchical relationship between the two levels of synergies. The possibility that motor cortex may execute even complex skill using spatiotemporal synergies has novel implications for the design of neuroprosthetic devices, which could gain computational efficiency by adopting the discrete and low-dimensional control that these primitives imply. SIGNIFICANCE STATEMENT: We studied the motor cortical and forearm muscular activity of rhesus macaques (Macaca mulatta) as they reached, grasped, and carried objects of varied shape and size. We applied non-negative matrix factorization separately to the cortical and muscular data to reduce their dimensionality to a smaller set of time-varying "spatiotemporal" synergies. Each synergy represents a sequence of cortical or muscular activity that recurred frequently during the animals' behavior. Salient features of the synergies (including their dimensionality, timing, and amplitude modulation) were observed at both the cortical and muscular levels. The possibility that the brain may execute even complex behaviors using spatiotemporal synergies has implications for neuroprosthetic algorithm design, which could become more computationally efficient by adopting the discrete and low-dimensional control that they afford.

A theory for how sensorimotor skills are learned and retained in noisy and nonstationary neural circuits.

During the process of skill learning, synaptic connections in our brains are modified to form motor memories of learned sensorimotor acts. The more plastic the adult brain is, the easier it is to learn new skills or adapt to neurological injury. However, if the brain is too plastic and the pattern of synaptic connectivity is constantly changing, new memories will overwrite old memories, and learning becomes unstable. This trade-off is known as the stability-plasticity dilemma. Here a theory of sensorimotor learning and memory is developed whereby synaptic strengths are perpetually fluctuating without causing instability in motor memory recall, as long as the underlying neural networks are sufficiently noisy and massively redundant. The theory implies two distinct stages of learning--preasymptotic and postasymptotic--because once the error drops to a level comparable to that of the noise-induced error, further error reduction requires altered network dynamics. A key behavioral prediction derived from this analysis is tested in a visuomotor adaptation experiment, and the resultant learning curves are modeled with a nonstationary neural network. Next, the theory is used to model two-photon microscopy data that show, in animals, high rates of dendritic spine turnover, even in the absence of overt behavioral learning. Finally, the theory predicts enhanced task selectivity in the responses of individual motor cortical neurons as the level of task expertise increases. From these considerations, a unique interpretation of sensorimotor memory is proposed--memories are defined not by fixed patterns of synaptic weights but, rather, by nonstationary synaptic patterns that fluctuate coherently.

Muscle synergy patterns as physiological markers of motor cortical damage.

The experimental findings herein reported are aimed at gaining a perspective on the complex neural events that follow lesions of the motor cortical areas. Cortical damage, whether by trauma or stroke, interferes with the flow of descending signals to the modular interneuronal structures of the spinal cord. These spinal modules subserve normal motor behaviors by activating groups of muscles as individual units (muscle synergies). Damage to the motor cortical areas disrupts the orchestration of the modules, resulting in abnormal movements. To gain insights into this complex process, we recorded myoelectric signals from multiple upper-limb muscles in subjects with cortical lesions. We used a factorization algorithm to identify the muscle synergies. Our factorization analysis revealed, in a quantitative way, three distinct patterns of muscle coordination-including preservation, merging, and fractionation of muscle synergies-that reflect the multiple neural responses that occur after cortical damage. These patterns varied as a function of both the severity of functional impairment and the temporal distance from stroke onset. We think these muscle-synergy patterns can be used as physiological markers of the status of any patient with stroke or trauma, thereby guiding the development of different rehabilitation approaches, as well as future physiological experiments for a further understanding of postinjury mechanisms of motor control and recovery.

Activity of the same motor cortex neurons during repeated experience with perturbed movement dynamics.

Neurons in the primary motor cortex (M1) have been shown to have persistent, memory-like activity following adaptation to altered movement dynamics. However, the techniques used to study these memory traces limited recordings to only single sessions lasting no more than a few hours. Here, chronically implanted microelectrode arrays were used to study the long-term neuronal responses to repeated experience with perturbing, velocity-dependent force fields. Force-field-related neuronal activity within each session was similar to that found previously. That is, the directional tuning curves of the M1 neurons shifted in a manner appropriate to compensate for the forces. Next, the across-session behavior was examined. Long-term learning was evident in the performance improvements across multiple force-field sessions. Correlated with this change, the neuronal population had smaller within-session spike rate changes as experience with the force field increased. The smaller within-session changes were a result of persistent across-session shifts in directional tuning. The results extend the observation of memory traces of newly learned dynamics and provide further evidence for the role of M1 in early motor memory formation.

Simplified and effective motor control based on muscle synergies to exploit musculoskeletal dynamics.

The basic hypothesis of producing a range of behaviors using a small set of motor commands has been proposed in various forms to explain motor behaviors ranging from basic reflexes to complex voluntary movements. Yet many fundamental questions regarding this long-standing hypothesis remain unanswered. Indeed, given the prominent nonlinearities and high dimensionality inherent in the control of biological limbs, the basic feasibility of a low-dimensional controller and an underlying principle for its creation has remained elusive. We propose a principle for the design of such a controller, that it endeavors to control the natural dynamics of the limb, taking into account the nature of the task being performed. Using this principle, we obtained a low-dimensional model of the hindlimb and a set of muscle synergies to command it. We demonstrate that this set of synergies was capable of producing effective control, establishing the viability of this muscle synergy hypothesis. Finally, by combining the low-dimensional model and the muscle synergies we were able to build a relatively simple controller whose overall performance was close to that of the system's full-dimensional nonlinear controller. Taken together, the results of this study establish that a low-dimensional controller is capable of simplifying control without degrading performance.

Stability of muscle synergies for voluntary actions after cortical stroke in humans.

Production of voluntary movements relies critically on the functional integration of several motor cortical areas, such as the primary motor cortex, and the spinal circuitries. Surprisingly, after almost 40 years of research, how the motor cortices specify descending neural signals destined for the downstream interneurons and motoneurons has remained elusive. In light of the many recent experimental demonstrations that the motor system may coordinate muscle activations through a linear combination of muscle synergies, we hypothesize that the motor cortices may function to select and activate fixed muscle synergies specified by the spinal or brainstem networks. To test this hypothesis, we recorded electromyograms (EMGs) from 12-16 upper arm and shoulder muscles from both the unaffected and the stroke-affected arms of stroke patients having moderate-to-severe unilateral ischemic lesions in the frontal motor cortical areas. Analyses of EMGs using a nonnegative matrix factorization algorithm revealed that in seven of eight patients the muscular compositions of the synergies for both the unaffected and the affected arms were strikingly similar to each other despite differences in motor performance between the arms, and differences in cerebral lesion sizes and locations between patients. This robustness of muscle synergies that we observed supports the notion that descending cortical signals represent neuronal drives that select, activate, and flexibly combine muscle synergies specified by networks in the spinal cord and/or brainstem. Our conclusion also suggests an approach to stroke rehabilitation by focusing on those synergies with altered activations after stroke.

Motor learning with unstable neural representations.

It is often assumed that learning takes place by changing an otherwise stable neural representation. To test this assumption, we studied changes in the directional tuning of primate motor cortical neurons during reaching movements performed in familiar and novel environments. During the familiar task, tuning curves exhibited slow random drift. During learning of the novel task, random drift was accompanied by systematic shifts of tuning curves. Our analysis suggests that motor learning is based on a surprisingly unstable neural representation. To explain these results, we propose that motor cortex is a redundant neural network, i.e., any single behavior can be realized by multiple configurations of synaptic strengths. We further hypothesize that synaptic modifications underlying learning contain a random component, which causes wandering among synaptic configurations with equivalent behaviors but different neural representations. We use a simple model to explore the implications of these assumptions.

Modules in the brain stem and spinal cord underlying motor behaviors.

Previous studies using intact and spinalized animals have suggested that coordinated movements can be generated by appropriate combinations of muscle synergies controlled by the central nervous system (CNS). However, which CNS regions are responsible for expressing muscle synergies remains an open question. We address whether the brain stem and spinal cord are involved in expressing muscle synergies used for executing a range of natural movements. We analyzed the electromyographic (EMG) data recorded from frog leg muscles before and after transection at different levels of the neuraxis-rostral midbrain (brain stem preparations), rostral medulla (medullary preparations), and the spinal-medullary junction (spinal preparations). Brain stem frogs could jump, swim, kick, and step, while medullary frogs could perform only a partial repertoire of movements. In spinal frogs, cutaneous reflexes could be elicited. Systematic EMG analysis found two different synergy types: 1) synergies shared between pre- and posttransection states and 2) synergies specific to individual states. Almost all synergies found in natural movements persisted after transection at rostral midbrain or medulla but not at the spinal-medullary junction for swim and step. Some pretransection- and posttransection-specific synergies for a certain behavior appeared as shared synergies for other motor behaviors of the same animal. These results suggest that the medulla and spinal cord are sufficient for the expression of most muscle synergies in frog behaviors. Overall, this study provides further evidence supporting the idea that motor behaviors may be constructed by muscle synergies organized within the brain stem and spinal cord and activated by descending commands from supraspinal areas.

Generalization in vision and motor control.

Learning is more than memory. It is not simply the building of a look-up table of labelled images, or a phone-directory-like list of motor acts and the corresponding sequences of muscle activation. Central to learning and intelligence is the ability to predict, that is, to generalize to new situations, beyond the memory of specific examples. The key to generalization, in turn, is the architecture of the system, more than the rules of synaptic plasticity. We propose a specific architecture for generalization for both the motor and the visual systems, and argue for a canonical microcircuit underlying visual and motor learning.

Modulation of muscle synergy recruitment in primate grasping.

In grasping, the CNS controls a particularly large number of degrees of freedom. We tested the idea that this control is facilitated by the presence of muscle synergies. According to the strong version of this concept, these synergies are invariant, hard-wired patterns of activation across muscles. Synergies may serve as modules that linearly sum, each with specific amplitude and timing coefficients, to generate a large array of muscle patterns. We tested two predictions of the synergy model. A small number of synergies should (1) account for a large fraction of variation in muscle activity, and (2) be modulated in their recruitment by task variables, even in novel behavioral contexts. We also examined whether the synergies would (3) have broadly similar structures across animals. We recorded from 15 to 19 electrodes implanted in forelimb muscles of two rhesus macaques as they grasped and transported 25 objects of variable shape and size. We show that three synergies accounted for 81% of the electromyographic data variation in each monkey. Each synergy was modulated in its recruitment strength and/or timing by object shape and/or size. Even when synergies were extracted from a small subset of object shape and size conditions and then used to reconstruct the entire dataset, we observed highly similar synergies and patterns of modulation. The synergies were well conserved between monkeys, with two of the synergies exceeding chance structural similarity, and the third being recruited, in both animals, in proportion to the size of the object handled.

Cortical processing during dynamic motor adaptation.

In this chapter we investigate the role of motor cortex in adapting movements to novel dynamic environments. We present results from two experiments in which monkey or human subjects learned to make two-dimensional reaching movements while holding a robotic manipulandum that applied a predictable pattern of forces (a curl field) to their hand. In the first study, we analyzed electrophysiological data recorded in motor cortex while monkeys adapted or readapted to the novel forces on each day of the experiment. In the second study, we perturbed the excitability of motor cortex using repetitive transcranial magnetic stimulation (rTMS) as human participants adapted to the forces. From the first experiment, we present qualitative evidence that a network of cortical areas including the supplementary motor area, premotor cortex, and primary motor cortex (M1) not only encodes kinematic and dynamic parameters of motor execution, but also registers changes in encoding that could provide a substrate for motor memory. Based on the second experiment, we qualify the role of M1 in motor memory, by showing that its disruption by rTMS does not interfere with the process of initial motor adaptation, but rather with offline improvement as measured at retest on the following day.

Neuronal correlates of kinematics-to-dynamics transformation in the supplementary motor area.

It is widely acknowledged that movements are planned at the level of the kinematics. However, the central nervous system must ultimately transform kinematic plans into dynamics-related commands. How, when, and where the kinematics-to-dynamics (KD) transformation is processed represent fundamental and unanswered questions. We recorded from the supplementary motor area (SMA) of two monkeys as they executed visually instructed reaching movements. We specifically analyzed a delay period following the instruction but prior to the go signal (motor planning). During the delay, a group of neurons in the SMA progressively came to reflect the dynamics rather than the desired kinematics of the upcoming movement. This finding suggests that some neurons in the SMA participate in the KD transformation.

Simultaneous sensorimotor adaptation and sequence learning.

Sensorimotor adaptation and sequence learning have often been treated as distinct forms of motor learning. But frequently the motor system must acquire both types of experience simultaneously. Here, we investigated the interaction of these two forms of motor learning by having subjects adapt to predictable forces imposed by a robotic manipulandum while simultaneously reaching to an implicit sequence of targets. We show that adaptation to novel dynamics and learning of a sequence of movements can occur simultaneously and without significant interference or facilitation. When both conditions were presented simultaneously to subjects, their trajectory error and reaction time decreased to the same extent as those of subjects who experienced the force field or sequence independently.

Combinations of muscle synergies in the construction of a natural motor behavior.

A central issue in motor control is how the central nervous system generates the muscle activity patterns necessary to achieve a variety of behavioral goals. The many degrees of freedom of the musculoskeletal apparatus provide great flexibility but make the control problem extremely complex. Muscle synergies--coherent activations, in space or time, of a group of muscles--have been proposed as building blocks that could simplify the construction of motor behaviors. To evaluate this hypothesis, we developed a new method to extract invariant spatiotemporal components from the simultaneous recordings of the activity of many muscles. We used this technique to analyze the muscle patterns of intact and unrestrained frogs during kicking, a natural defensive behavior. Here we show that combinations of three time-varying muscle synergies underlie the variety of muscle patterns required to kick in different directions, that the recruitment of these synergies is related to movement kinematics, and that there are similarities among the synergies extracted from different behaviors.

Intermittent practice facilitates stable motor memories.

Humans adaptively control reaching movements to maintain good performance in the presence of novel forces acting on the arm. A recent study suggested that motor memories of different force conditions are not transformed from fragile to stable states, but rather are always vulnerable to interference from newly learned conditions (Caithness et al., 2004). This is contrary to the results of previous studies (Brashers-Krug et al., 1996; Shadmehr and Brashers-Krug, 1997), although all of these studies followed similar methods. Here, we show that a seemingly insignificant and inconsistently applied methodological detail may reconcile this discrepancy. Catch trials, in which the novel forces are removed, may be randomly interspersed among the more frequent force trials to assess how a subject is learning to predict the pattern of forces. In the absence of an interfering condition, subjects retained their learning until retest a day later regardless of whether they experienced catch trials. But in the presence of an interfering condition, only the subjects who had experienced forces intermittently retained their learning and thereby showed resistance to the interference. Thus, intermittent rather than constant practice conditions appear to be critical for dynamic motor memory stabilization.

Disruption of primary motor cortex before learning impairs memory of movement dynamics.

Although multiple lines of evidence implicate the primary motor cortex (M1) in motor learning, the precise role of M1 in the adaptation to novel movement dynamics and in the subsequent consolidation of a memory of those dynamics remains unclear. Here we used repetitive transcranial magnetic stimulation (rTMS) to dissociate the contribution of M1 to these distinct aspects of motor learning. Subjects performed reaching movements in velocity-dependent force fields over three epochs: a null-field baseline epoch, a clockwise-field learning epoch (15 min after the baseline epoch), and a clockwise-field retest epoch (24 h after the learning epoch). Half of the subjects received 15 min of 1 Hz rTMS to M1 between the baseline and learning epochs. Subjects given rTMS performed identically to control subjects during the learning epoch. However, control subjects performed with significantly less error than rTMS subjects in the retest epoch on the following day. These results suggest that M1 is not critical to the network supporting motor adaptation per se but that, within this network, M1 may be important for initiating the development of long-term motor memories.

Critical Points and Traveling Wave in Locomotion: Experimental Evidence and Some Theoretical Considerations.

The central pattern generator (CPG) architecture for rhythm generation remains partly elusive. We compare cat and frog locomotion results, where the component unrelated to pattern formation appears as a temporal grid, and traveling wave respectively. Frog spinal cord microstimulation with N-methyl-D-Aspartate (NMDA), a CPG activator, produced a limited set of force directions, sometimes tonic, but more often alternating between directions similar to the tonic forces. The tonic forces were topographically organized, and sites evoking rhythms with different force subsets were located close to the constituent tonic force regions. Thus CPGs consist of topographically organized modules. Modularity was also identified as a limited set of muscle synergies whose combinations reconstructed the EMGs. The cat CPG was investigated using proprioceptive inputs during fictive locomotion. Critical points identified both as abrupt transitions in the effect of phasic perturbations, and burst shape transitions, had biomechanical correlates in intact locomotion. During tonic proprioceptive perturbations, discrete shifts between these critical points explained the burst durations changes, and amplitude changes occurred at one of these points. Besides confirming CPG modularity, these results suggest a fixed temporal grid of anchoring points, to shift modules onsets and offsets. Frog locomotion, reconstructed with the NMDA synergies, showed a partially overlapping synergy activation sequence. Using the early synergy output evoked by NMDA at different spinal sites, revealed a rostrocaudal topographic organization, where each synergy is preferentially evoked from a few, albeit overlapping, cord regions. Comparing the locomotor synergy sequence with this topography suggests that a rostrocaudal traveling wave would activate the synergies in the proper sequence for locomotion. This output was reproduced in a two-layer model using this topography and a traveling wave. Together our results suggest two CPG components: modules, i.e., synergies; and temporal patterning, seen as a temporal grid in the cat, and a traveling wave in the frog. Animal and limb navigation have similarities. Research relating grid cells to the theta rhythm and on segmentation during navigation may relate to our temporal grid and traveling wave results. Winfree's mathematical work, combining critical phases and a traveling wave, also appears important. We conclude suggesting tracing, and imaging experiments to investigate our CPG model.

Central and sensory contributions to the activation and organization of muscle synergies during natural motor behaviors.

Previous studies have suggested that the motor system may simplify control by combining a small number of muscle synergies represented as activation profiles across a set of muscles. The role of sensory feedback in the activation and organization of synergies has remained an open question. Here, we assess to what extent the motor system relies on centrally organized synergies activated by spinal and/or supraspinal commands to generate motor outputs by analyzing electromyographic (EMG) signals collected from 13 hindlimb muscles of the bullfrog during swimming and jumping, before and after deafferentation. We first established that, for both behaviors, the intact and deafferented data sets possess low and similar dimensionalities. Subsequently, we used a novel reformulation of the non-negative matrix factorization algorithm to simultaneously search for synergies shared by, and synergies specific to, the intact and deafferented data sets. Most muscle synergies were identified as shared synergies, suggesting that EMGs of locomotor behaviors are generated primarily by centrally organized synergies. Both the amplitude and temporal patterns of the activation coefficients of most shared synergies, however, were altered by deafferentation, suggesting that sensory inflow modulates activation of those centrally organized synergies. For most synergies, effects of deafferentation on the activation coefficients were not consistent across frogs, indicating substantial interanimal variability of feedback actions. We speculate that sensory feedback might adapt recruitment of muscle synergies to behavioral constraints, and the few synergies specific to the intact or deafferented states might represent afferent-specific modules or feedback reorganization of spinal neuronal networks.

Intrinsic musculoskeletal properties stabilize wiping movements in the spinalized frog.

The mechanical stability properties of hindlimb-hindlimb wiping movements of the spinalized frog were examined. One hindlimb, the wiping limb, was implanted with 12 electromyographic (EMG) electrodes and attached to a robot that both recorded its trajectory and applied brief force perturbations. Cutaneous electrical stimulation was applied to the other hindlimb, the target limb, to evoke the hindlimb-hindlimb wiping reflex. Kinematic and EMG data from both unperturbed trials and trials in which a phasic perturbation was applied were collected from each spinalized frog. In the perturbed behaviors, we found that the initially large displacement attributable to the perturbation was compensated such that the final position was statistically indistinguishable from the unperturbed final position in all of the frogs, thus indicating the dynamic stability of these movements. This stability was robust to the range of perturbation amplitudes and nominal kinematic variation observed in this study. In addition, we investigated the extent to which intrinsic viscoelastic properties of the limb and proprioceptive feedback play a role in stabilizing the movements. No significant changes were seen in the EMGs after the perturbation. Furthermore, deafferentation of the wiping limb did not significantly affect the stability of the wiping reflex. Thus, we found that the intrinsic viscoelastic properties of the hindlimb conferred robust stability properties to the hindlimb-hindlimb wiping behavior. This stability mechanism may simplify the control required by the frog spinal motor systems to produce successful movements in an unpredictable and varying environment.