Evolution, biomechanics, and neurobiology converge to explain selective finger motor control.

Humans use their fingers to perform a variety of tasks, from simple grasping to manipulating objects, to typing and playing musical instruments, a variety wider than any other species. The more sophisticated the task, the more it involves individuated finger movements, those in which one or more selected fingers perform an intended action while the motion of other digits is constrained. Here we review the neurobiology of such individuated finger movements. We consider their evolutionary origins, the extent to which finger movements are in fact individuated, and the evolved features of neuromuscular control that both enable and limit individuation. We go on to discuss other features of motor control that combine with individuation to create dexterity, the impairment of individuation by disease, and the broad extent of capabilities that individuation confers on humans. We comment on the challenges facing the development of a truly dexterous bionic hand. We conclude by identifying topics for future investigation that will advance our understanding of how neural networks interact across multiple regions of the central nervous system to create individuated movements for the skills humans use to express their cognitive activity.

Optimal reaching subject to computational and physical constraints reveals structure of the sensorimotor control system.

Optimal feedback control provides an abstract framework describing the architecture of the sensorimotor system without prescribing implementation details such as what coordinate system to use, how feedback is incorporated, or how to accommodate changing task complexity. We investigate how such details are determined by computational and physical constraints by creating a model of the upper limb sensorimotor system in which all connection weights between neurons, feedback, and muscles are unknown. By optimizing these parameters with respect to an objective function, we find that the model exhibits a preference for an intrinsic (joint angle) coordinate representation of inputs and feedback and learns to calculate a weighted feedforward and feedback error. We further show that complex reaches around obstacles can be achieved by augmenting our model with a path-planner based on via points. The path-planner revealed "avoidance" neurons that encode directions to reach around obstacles and "placement" neurons that make fine-tuned adjustments to via point placement. Our results demonstrate the surprising capability of computationally constrained systems and highlight interesting characteristics of the sensorimotor system.

Structure of Population Activity in Primary Motor Cortex for Single Finger Flexion and Extension.

How is the primary motor cortex (M1) organized to control fine finger movements? We investigated the population activity in M1 for single finger flexion and extension, using 7T functional magnetic resonance imaging (fMRI) in female and male human participants and compared these results to the neural spiking patterns recorded in two male monkeys performing the identical task. fMRI activity patterns were distinct for movements of different fingers, but were quite similar for flexion and extension of the same finger. In contrast, spiking patterns in monkeys were quite distinct for both fingers and directions, which is similar to what was found for muscular activity patterns. The discrepancy between fMRI and electrophysiological measurements can be explained by two (non-mutually exclusive) characteristics of the organization of finger flexion and extension movements. Given that fMRI reflects predominantly input and recurrent activity, the results can be explained by an architecture in which neural populations that control flexion or extension of the same finger produce distinct outputs, but interact tightly with each other and receive similar inputs. Additionally, neurons tuned to different movement directions for the same finger (or combination of fingers) may cluster closely together, while neurons that control different finger combinations may be more spatially separated. When measuring this organization with fMRI at a coarse spatial scale, the activity patterns for flexion and extension of the same finger would appear very similar. Overall, we suggest that the discrepancy between fMRI and electrophysiological measurements provides new insights into the general organization of fine finger movements in M1.SIGNIFICANCE STATEMENT The primary motor cortex (M1) is important for producing individuated finger movements. Recent evidence shows that movements that commonly co-occur are associated with more similar activity patterns in M1. Flexion and extension of the same finger, which never co-occur, should therefore be associated with distinct representations. However, using carefully controlled experiments and multivariate analyses, we demonstrate that human fMRI activity patterns for flexion or extension of the same finger are highly similar. In contrast, spiking patterns measured in monkey M1 are clearly distinct. This suggests that populations controlling opposite movements of the same finger, while producing distinct outputs, may cluster together and share inputs and local processing. These results provide testable hypotheses about the organization of hand control in M1.

Mirror Neuron Populations Represent Sequences of Behavioral Epochs During Both Execution and Observation.

Mirror neurons (MNs) have the distinguishing characteristic of modulating during both execution and observation of an action. Although most studies of MNs have focused on various features of the observed movement, MNs also may monitor the behavioral circumstances in which the movement is embedded, including time periods preceding and following the observed movement. Here, we recorded multiple MNs simultaneously from implanted electrode arrays as two male monkeys executed and observed a reach, grasp, and manipulate task involving different target objects. MNs were recorded from premotor cortex (PM-MNs) and primary motor cortex (M1-MNs). During execution trials, hidden Markov models (HMMs) applied to the activity of either PM-MN or M1-MN populations most often detected sequences of four hidden states, which we named according to the behavioral epoch during which each state began: initial, reaction, movement, and final. The hidden states of MN populations thus reflected not only the movement, but also three behavioral epochs during which no movement occurred. HMMs trained on execution trials could decode similar sequences of hidden states in observation trials, with complete hidden state sequences decoded more frequently from PM-MN populations than from M1-MN populations. Moreover, population trajectories projected in a 2D plane defined by execution trials were preserved in observation trials more for PM-MN than for M1-MN populations. These results suggest that MN populations represent entire behavioral sequences, including both movement and non-movement. PM-MN populations showed greater similarity than M1-MN populations in their representation of behavioral sequences during execution versus observation.SIGNIFICANCE STATEMENT Mirror neurons (MNs) are thought to provide a neural mechanism for understanding the actions of others. However, for an action to be understood, both the movement per se and the non-movement context before and after the movement need to be represented. We found that simultaneously recorded MN populations encoded sequential hidden neural states corresponding approximately to sequential behavioral epochs of a reach, grasp, and manipulate task. During observation trials, hidden state sequences were similar to those identified in execution trials. Hidden state similarity was stronger for MN populations in premotor cortex than for those in primary motor cortex. Execution/observation similarity of hidden state sequences may contribute to understanding the actions of others without actually performing the action oneself.

Mirror neurons precede non-mirror neurons during action execution.

Mirror neurons are thought to represent an individual's ability to understand the actions of others by discharging as one individual performs or observes another individual performing an action. Studies typically have focused on mirror neuron activity during action observation, examining activity during action execution primarily to validate mirror neuron involvement in the motor act. As a result, little is known about the precise role of mirror neurons during action execution. In this study, during execution of reach-grasp-manipulate movements, we found activity of mirror neurons generally preceded that of non-mirror neurons. Not only did the onset of task-related modulation occur earlier in mirror neurons, but state transitions detected by hidden Markov models also occurred earlier in mirror neuron populations. Our findings suggest that mirror neurons may be at the forefront of action execution.NEW & NOTEWORTHY Mirror neurons commonly are thought to provide a neural substrate for understanding the actions of others, but mirror neurons also are active during action execution, when additional, non-mirror neurons are active as well. Examining the timing of activity during execution of a naturalistic reach-grasp-manipulate task, we found that mirror neuron activity precedes that of non-mirror neurons at both the unit and the population level. Thus mirror neurons may be at the leading edge of action execution.

How is electrical stimulation of the brain experienced, and how can we tell? Selected considerations on sensorimotor function and speech.

Electrical stimulation of the nervous system is a powerful tool for localizing and examining the function of numerous brain regions. Delivered to certain regions of the cerebral cortex, electrical stimulation can evoke a variety of first-order effects, including observable movements or an urge to move, or somatosensory, visual, or auditory percepts. In still other regions the subject may be oblivious to the stimulation. Often overlooked, however, is whether the subject is aware of the stimulation, and if so, how the stimulation is experienced by the subject. In this review of how electrical stimulation has been used to study selected aspects of sensorimotor and language function, we raise questions that future studies might address concerning the subjects' second-order experiences of intention and agency regarding evoked movements, of the naturalness of evoked sensory percepts, and of other qualia that might be evoked in the absence of an overt first-order experience.

Spatiotemporal Distribution of Location and Object Effects in Primary Motor Cortex Neurons during Reach-to-Grasp.

Reaching and grasping typically are considered to be spatially separate processes that proceed concurrently in the arm and the hand, respectively. The proximal representation in the primary motor cortex (M1) controls the arm for reaching, while the distal representation controls the hand for grasping. Many studies of M1 activity therefore have focused either on reaching to various locations without grasping different objects, or else on grasping different objects all at the same location. Here, we recorded M1 neurons in the anterior bank and lip of the central sulcus as monkeys performed more naturalistic movements, reaching toward, grasping, and manipulating four different objects in up to eight different locations. We quantified the extent to which variation in firing rates depended on location, on object, and on their interaction-all as a function of time. Activity proceeded largely in two sequential phases: the first related predominantly to the location to which the upper extremity reached, and the second related to the object about to be grasped. Both phases involved activity distributed widely throughout the sampled territory, spanning both the proximal and the distal upper extremity representation in caudal M1. Our findings indicate that naturalistic reaching and grasping, rather than being spatially segregated processes that proceed concurrently, each are spatially distributed processes controlled by caudal M1 in large part sequentially. Rather than neuromuscular processes separated in space but not time, reaching and grasping are separated more in time than in space. SIGNIFICANCE STATEMENT: Reaching and grasping typically are viewed as processes that proceed concurrently in the arm and hand, respectively. The arm region in the primary motor cortex (M1) is assumed to control reaching, while the hand region controls grasping. During naturalistic reach-grasp-manipulate movements, we found, however, that neuron activity proceeds largely in two sequential phases, each spanning both arm and hand representations in M1. The first phase is related predominantly to the reach location, and the second is related to the object about to be grasped. Our findings indicate that reaching and grasping are successive aspects of a single movement. Initially the arm and the hand both are projected toward the object's location, and later both are shaped to grasp and manipulate.

Spatiotemporal distribution of location and object effects in the electromyographic activity of upper extremity muscles during reach-to-grasp.

In reaching to grasp an object, proximal muscles that act on the shoulder and elbow classically have been viewed as transporting the hand to the intended location, while distal muscles that act on the fingers simultaneously shape the hand to grasp the object. Prior studies of electromyographic (EMG) activity in upper extremity muscles therefore have focused, by and large, either on proximal muscle activity during reaching to different locations or on distal muscle activity as the subject grasps various objects. Here, we examined the EMG activity of muscles from the shoulder to the hand, as monkeys reached and grasped in a task that dissociated location and object. We quantified the extent to which variation in the EMG activity of each muscle depended on location, on object, and on their interaction-all as a function of time. Although EMG variation depended on both location and object beginning early in the movement, an early phase of substantial location effects in muscles from proximal to distal was followed by a later phase in which object effects predominated throughout the extremity. Interaction effects remained relatively small. Our findings indicate that neural control of reach-to-grasp may occur largely in two sequential phases: the first, serving to project the entire upper extremity toward the intended location, and the second, acting predominantly to shape the entire extremity for grasping the object.

Handedness and index finger movements performed on a small touchscreen.

Many studies of right/left differences in motor performance related to handedness have employed tasks that use arm movements or combined arm and hand movements rather than movements of the fingers per se, the well-known exception being rhythmic finger tapping. We therefore explored four simple tasks performed on a small touchscreen with relatively isolated movements of the index finger. Each task revealed a different right/left performance asymmetry. In a step-tracking Target Task, left-handed subjects showed greater accuracy with the index finger of the dominant left hand than with the nondominant right hand. In a Center-Out Task, right-handed subjects produced trajectories with the nondominant left hand that had greater curvature than those produced with the dominant right hand. In a continuous Circle Tracking Task, slips of the nondominant left index finger showed higher jerk than slips of the dominant right index finger. And in a continuous Complex Tracking Task, the nondominant left index finger showed shorter time lags in tracking the relatively unpredictable target than the dominant right index finger. Our findings are broadly consistent with previous studies indicating left hemisphere specialization for dynamic control and predictable situations vs. right hemisphere specialization for impedance control and unpredictable situations, the specialized contributions of the two hemispheres being combined to different degrees in the right vs. left hands of right-handed vs. left-handed individuals.

Temporal and kinematic consistency predict sequence awareness.

Many human motor skills can be represented as a hierarchical series of movement patterns. Awareness of underlying patterns can improve performance and decrease cognitive load. Subjects (n = 30) tapped a finger sequence with changing stimulus-to-response mapping and a common movement sequence. Thirteen subjects (43 %) became aware that they were tapping a familiar movement sequence during the experiment. Subjects who became aware of the underlying motor pattern tapped with greater kinematic and temporal consistency from task onset, but consistency was not sufficient for awareness. We found no effect of age, musical experience, tapping evenness, or inter-key-interval on awareness of the pattern in the motor response. We propose that temporal or kinematic consistency reinforces a pattern representation, but cognitive engagement with the contents of the sequence is necessary to bring the pattern to conscious awareness. These findings predict benefit for movement strategies that limit temporal and kinematic variability during motor learning.

Spatiotemporal distribution of location and object effects in reach-to-grasp kinematics.

In reaching to grasp an object, the arm transports the hand to the intended location as the hand shapes to grasp the object. Prior studies that tracked arm endpoint and grip aperture have shown that reaching and grasping, while proceeding in parallel, are interdependent to some degree. Other studies of reaching and grasping that have examined the joint angles of all five digits as the hand shapes to grasp various objects have not tracked the joint angles of the arm as well. We, therefore, examined 22 joint angles from the shoulder to the five digits as monkeys reached, grasped, and manipulated in a task that dissociated location and object. We quantified the extent to which each angle varied depending on location, on object, and on their interaction, all as a function of time. Although joint angles varied depending on both location and object beginning early in the movement, an early phase of location effects in joint angles from the shoulder to the digits was followed by a later phase in which object effects predominated at all joint angles distal to the shoulder. Interaction effects were relatively small throughout the reach-to-grasp. Whereas reach trajectory was influenced substantially by the object, grasp shape was comparatively invariant to location. Our observations suggest that neural control of reach-to-grasp may occur largely in two sequential phases: the first determining the location to which the arm transports the hand, and the second shaping the entire upper extremity to grasp and manipulate the object.

Principal components of hand kinematics and neurophysiological signals in motor cortex during reach to grasp movements.

A few kinematic synergies identified by principal component analysis (PCA) account for most of the variance in the coordinated joint rotations of the fingers and wrist used for a wide variety of hand movements. To examine the possibility that motor cortex might control the hand through such synergies, we collected simultaneous kinematic and neurophysiological data from monkeys performing a reach-to-grasp task. We used PCA, jPCA and isomap to extract kinematic synergies from 18 joint angles in the fingers and wrist and analyzed the relationships of both single-unit and multiunit spike recordings, as well as local field potentials (LFPs), to these synergies. For most spike recordings, the maximal absolute cross-correlations of firing rates were somewhat stronger with an individual joint angle than with any principal component (PC), any jPC or any isomap dimension. In decoding analyses, where spikes and LFP power in the 100- to 170-Hz band each provided better decoding than other LFP-based signals, the first PC was decoded as well as the best decoded joint angle. But the remaining PCs and jPCs were predicted with lower accuracy than individual joint angles. Although PCs, jPCs or isomap dimensions might provide a more parsimonious description of kinematics, our findings indicate that the kinematic synergies identified with these techniques are not represented in motor cortex more strongly than the original joint angles. We suggest that the motor cortex might act to sculpt the synergies generated by subcortical centers, superimposing an ability to individuate finger movements and adapt the hand to grasp a wide variety of objects.

Rapid acquisition of novel interface control by small ensembles of arbitrarily selected primary motor cortex neurons.

Pioneering studies demonstrated that novel degrees of freedom could be controlled individually by directly encoding the firing rate of single motor cortex neurons, without regard to each neuron's role in controlling movement of the native limb. In contrast, recent brain-computer interface work has emphasized decoding outputs from large ensembles that include substantially more neurons than the number of degrees of freedom being controlled. To bridge the gap between direct encoding by single neurons and decoding output from large ensembles, we studied monkeys controlling one degree of freedom by comodulating up to four arbitrarily selected motor cortex neurons. Performance typically exceeded random quite early in single sessions and then continued to improve to different degrees in different sessions. We therefore examined factors that might affect performance. Performance improved with larger ensembles. In contrast, other factors that might have reflected preexisting synaptic architecture-such as the similarity of preferred directions-had little if any effect on performance. Patterns of comodulation among ensemble neurons became more consistent across trials as performance improved over single sessions. Compared with the ensemble neurons, other simultaneously recorded neurons showed less modulation. Patterns of voluntarily comodulated firing among small numbers of arbitrarily selected primary motor cortex (M1) neurons thus can be found and improved rapidly, with little constraint based on the normal relationships of the individual neurons to native limb movement. This rapid flexibility in relationships among M1 neurons may in part underlie our ability to learn new movements and improve motor skill.

Progressively shifting patterns of co-modulation among premotor cortex neurons carry dynamically similar signals during action execution and observation.

Many neurons in the premotor cortex show firing rate modulation whether the subject performs an action or observes another individual performing a similar action. Although such mirror neurons have been thought to have highly congruent discharge during execution and observation, many if not most show non-congruent activity. Studies of such neuronal populations have shown that the most prevalent patterns of co-modulation, captured as neural trajectories, pass through subspaces which are shared in part, but in part are visited exclusively during either execution or observation. These studies focused on reaching movements for which low-dimensional neural trajectories exhibit comparatively simple dynamical motifs. But the neural dynamics of hand movements are more complex. We developed a novel approach to examine prevalent patterns of co-modulation during execution and observation of a task that involved reaching, grasping, and manipulation. Rather than following neural trajectories in subspaces that contain their entire time course, we identified time series of instantaneous subspaces, calculated principal angles among them, sampled trajectory segments at the times of selected behavioral events, and projected those segments into the series of instantaneous subspaces. We found that instantaneous neural subspaces generally remained distinct during execution versus observation. Nevertheless, execution and observation could be partially aligned with canonical correlation, indicating some dynamical similarity of the neural representations of different movements relative to one another during execution and observation which may enable the nervous system to recognize corresponding actions performed by the subject or by another individual and/or may reflect social interaction between the two. During action execution, mirror neurons showed consistent patterns of co-modulation both within and between sessions, but other neurons that were modulated only during action execution and not during observation showed considerable variability of co-modulation. We speculate that during execution, mirror neurons carry a consistent forward model of the intended movement, while action-execution only neurons process more variable feedback.

State-based decoding of hand and finger kinematics using neuronal ensemble and LFP activity during dexterous reach-to-grasp movements.

The performance of brain-machine interfaces (BMIs) that continuously control upper limb neuroprostheses may benefit from distinguishing periods of posture and movement so as to prevent inappropriate movement of the prosthesis. Few studies, however, have investigated how decoding behavioral states and detecting the transitions between posture and movement could be used autonomously to trigger a kinematic decoder. We recorded simultaneous neuronal ensemble and local field potential (LFP) activity from microelectrode arrays in primary motor cortex (M1) and dorsal (PMd) and ventral (PMv) premotor areas of two male rhesus monkeys performing a center-out reach-and-grasp task, while upper limb kinematics were tracked with a motion capture system with markers on the dorsal aspect of the forearm, hand, and fingers. A state decoder was trained to distinguish four behavioral states (baseline, reaction, movement, hold), while a kinematic decoder was trained to continuously decode hand end point position and 18 joint angles of the wrist and fingers. LFP amplitude most accurately predicted transition into the reaction (62%) and movement (73%) states, while spikes most accurately decoded arm, hand, and finger kinematics during movement. Using an LFP-based state decoder to trigger a spike-based kinematic decoder [r = 0.72, root mean squared error (RMSE) = 0.15] significantly improved decoding of reach-to-grasp movements from baseline to final hold, compared with either a spike-based state decoder combined with a spike-based kinematic decoder (r = 0.70, RMSE = 0.17) or a spike-based kinematic decoder alone (r = 0.67, RMSE = 0.17). Combining LFP-based state decoding with spike-based kinematic decoding may be a valuable step toward the realization of BMI control of a multifingered neuroprosthesis performing dexterous manipulation.

Somatosensory cortex microstimulation modulates primary motor and ventral premotor cortex neurons with extensive spatial convergence and divergence.

Intracortical microstimulation (ICMS) is known to affect distant neurons transynaptically, yet the extent to which ICMS pulses delivered in one cortical area modulate neurons in other cortical areas remains largely unknown. Here we assessed how the individual pulses of multi-channel ICMS trains delivered in the upper extremity representation of the macaque primary somatosensory area (S1) modulate neuron firing in the primary motor cortex (M1) and in the ventral premotor cortex (PMv). S1-ICMS pulses modulated the majority of units recorded both in the M1 upper extremity representation and in PMv, producing more inhibition than excitation. Effects converged on individual neurons in both M1 and PMv from extensive S1 territories. Conversely, effects of ICMS delivered in a small region of S1 diverged to wide territories in both M1 and PMv. The effects of this direct modulation of M1 and PMv neurons produced by multi-electrode S1-ICMS like that used here may need to be taken into account by bidirectional brain-computer interfaces that decode intended movements from neural activity in these cortical motor areas. Significance Statement: Although ICMS is known to produce effects transynaptically, relatively little is known about how ICMS in one cortical area affects neurons in other cortical areas. We show that the effects of multi-channel ICMS in a small patch of S1 diverge to affect neurons distributed widely in both M1 and PMv, and conversely, individual neurons in each of these areas can be affected by ICMS converging from much of the S1 upper extremity representation. Such direct effects of ICMS may complicate the decoding of motor intent from M1 or PMv when artificial sensation is delivered via S1-ICMS in bidirectional brain-computer interfaces.

Spatiotemporal variation of multiple neurophysiological signals in the primary motor cortex during dexterous reach-to-grasp movements.

To examine the spatiotemporal distribution of discriminable information about reach-to-grasp movements in the primary motor cortex upper extremity representation, we implanted four microelectrode arrays in the anterior bank and lip of the central sulcus in each of two monkeys. We used linear discriminant analysis to compare information, quantified as decoding accuracy, contained in various neurophysiological signals. For all signal types, decoding accuracy increased immediately after the movement cue, peaked around movement onset, and declined during the static hold. Spike recordings and local field potential (LFP) time domain amplitude provided more discriminable information than LFP frequency domain power. Discriminable information on movement type was distributed evenly across recording sites by LFP amplitude and 1-4 Hz power but unevenly by 100-170 Hz power and spike recordings. These latter two signal types provided higher decoding accuracies closer to the hemispheric surface than deep in the anterior bank and also provided accuracies that varied along the central sulcus. This variation in the distribution of movement-type information may be related to differences in the rostral versus caudal regions of the primary motor cortex and to its underlying somatotopic organization. The even distribution of information by LFP amplitude and 1-4 Hz power compared with the more localized distribution by 100-170 Hz power and spikes suggest that these different neurophysiological signals reflect different underlying processes that distribute information through the motor cortex during reach-to-grasp movements.

Cyclic, Condition-Independent Activity in Primary Motor Cortex Predicts Corrective Movement Behavior.

Reaching movements are known to have large condition-independent (CI) neural activity and cyclic neural dynamics. A new precision center-out task was performed by rhesus macaques to test the hypothesis that cyclic, CI neural activity in the primary motor cortex (M1) occurs not only during initial reaching movements but also during subsequent corrective movements. Corrective movements were observed to be discrete with time courses and bell-shaped speed profiles similar to the initial movements. CI cyclic neural trajectories were similar and repeated for initial and each additional corrective submovement. The phase of the cyclic CI neural activity predicted the time of peak movement speed more accurately than regression of instantaneous firing rate, even when the subject made multiple corrective movements. Rather than being controlled as continuations of the initial reach, a discrete cycle of motor cortex activity encodes each corrective submovement.