Restoring the Sense of Touch Using a Sensorimotor Demultiplexing Neural Interface.

Paralyzed muscles can be reanimated following spinal cord injury (SCI) using a brain-computer interface (BCI) to enhance motor function alone. Importantly, the sense of touch is a key component of motor function. Here, we demonstrate that a human participant with a clinically complete SCI can use a BCI to simultaneously reanimate both motor function and the sense of touch, leveraging residual touch signaling from his own hand. In the primary motor cortex (M1), residual subperceptual hand touch signals are simultaneously demultiplexed from ongoing efferent motor intention, enabling intracortically controlled closed-loop sensory feedback. Using the closed-loop demultiplexing BCI almost fully restored the ability to detect object touch and significantly improved several sensorimotor functions. Afferent grip-intensity levels are also decoded from M1, enabling grip reanimation regulated by touch signaling. These results demonstrate that subperceptual neural signals can be decoded from the cortex and transformed into conscious perception, significantly augmenting function.

Glia Accumulate Evidence that Actions Are Futile and Suppress Unsuccessful Behavior.

When a behavior repeatedly fails to achieve its goal, animals often give up and become passive, which can be strategic for preserving energy or regrouping between attempts. It is unknown how the brain identifies behavioral failures and mediates this behavioral-state switch. In larval zebrafish swimming in virtual reality, visual feedback can be withheld so that swim attempts fail to trigger expected visual flow. After tens of seconds of such motor futility, animals became passive for similar durations. Whole-brain calcium imaging revealed noradrenergic neurons that responded specifically to failed swim attempts and radial astrocytes whose calcium levels accumulated with increasing numbers of failed attempts. Using cell ablation and optogenetic or chemogenetic activation, we found that noradrenergic neurons progressively activated brainstem radial astrocytes, which then suppressed swimming. Thus, radial astrocytes perform a computation critical for behavior: they accumulate evidence that current actions are ineffective and consequently drive changes in behavioral states. VIDEO ABSTRACT.

Competitive Disinhibition Mediates Behavioral Choice and Sequences in Drosophila.

Even a simple sensory stimulus can elicit distinct innate behaviors and sequences. During sensorimotor decisions, competitive interactions among neurons that promote distinct behaviors must ensure the selection and maintenance of one behavior, while suppressing others. The circuit implementation of these competitive interactions is still an open question. By combining comprehensive electron microscopy reconstruction of inhibitory interneuron networks, modeling, electrophysiology, and behavioral studies, we determined the circuit mechanisms that contribute to the Drosophila larval sensorimotor decision to startle, explore, or perform a sequence of the two in response to a mechanosensory stimulus. Together, these studies reveal that, early in sensory processing, (1) reciprocally connected feedforward inhibitory interneurons implement behavioral choice, (2) local feedback disinhibition provides positive feedback that consolidates and maintains the chosen behavior, and (3) lateral disinhibition promotes sequence transitions. The combination of these interconnected circuit motifs can implement both behavior selection and the serial organization of behaviors into a sequence.

The science and engineering behind sensitized brain-controlled bionic hands.

Advances in our understanding of brain function, along with the development of neural interfaces that allow for the monitoring and activation of neurons, have paved the way for brain-machine interfaces (BMIs), which harness neural signals to reanimate the limbs via electrical activation of the muscles or to control extracorporeal devices, thereby bypassing the muscles and senses altogether. BMIs consist of reading out motor intent from the neuronal responses monitored in motor regions of the brain and executing intended movements with bionic limbs, reanimated limbs, or exoskeletons. BMIs also allow for the restoration of the sense of touch by electrically activating neurons in somatosensory regions of the brain, thereby evoking vivid tactile sensations and conveying feedback about object interactions. In this review, we discuss the neural mechanisms of motor control and somatosensation in able-bodied individuals and describe approaches to use neuronal responses as control signals for movement restoration and to activate residual sensory pathways to restore touch. Although the focus of the review is on intracortical approaches, we also describe alternative signal sources for control and noninvasive strategies for sensory restoration.

Optimal reaching subject to computational and physical constraints reveals structure of the sensorimotor control system.

Optimal feedback control provides an abstract framework describing the architecture of the sensorimotor system without prescribing implementation details such as what coordinate system to use, how feedback is incorporated, or how to accommodate changing task complexity. We investigate how such details are determined by computational and physical constraints by creating a model of the upper limb sensorimotor system in which all connection weights between neurons, feedback, and muscles are unknown. By optimizing these parameters with respect to an objective function, we find that the model exhibits a preference for an intrinsic (joint angle) coordinate representation of inputs and feedback and learns to calculate a weighted feedforward and feedback error. We further show that complex reaches around obstacles can be achieved by augmenting our model with a path-planner based on via points. The path-planner revealed "avoidance" neurons that encode directions to reach around obstacles and "placement" neurons that make fine-tuned adjustments to via point placement. Our results demonstrate the surprising capability of computationally constrained systems and highlight interesting characteristics of the sensorimotor system.

Visual guidance fine-tunes probing movements of an insect appendage.

Visually guided reaching, a regular feature of human life, comprises an intricate neural control task. It includes identifying the target's position in 3D space, passing the representation to the motor system that controls the respective appendages, and adjusting ongoing movements using visual and proprioceptive feedback. Given the complexity of the neural control task, invertebrates, with their numerically constrained central nervous systems, are often considered incapable of this level of visuomotor guidance. Here, we provide mechanistic insights into visual appendage guidance in insects by studying the probing movements of the hummingbird hawkmoth's proboscis as they search for a flower's nectary. We show that visually guided proboscis movements fine-tune the coarse control provided by body movements in flight. By impairing the animals' view of their proboscis, we demonstrate that continuous visual feedback is required and actively sought out to guide this appendage. In doing so, we establish an insect model for the study of neural strategies underlying eye-appendage control in a simple nervous system.

Proprioception: A New Era Set in Motion by Emerging Genetic and Bionic Strategies?

The generation of an internal body model and its continuous update is essential in sensorimotor control. Although known to rely on proprioceptive sensory feedback, the underlying mechanism that transforms this sensory feedback into a dynamic body percept remains poorly understood. However, advances in the development of genetic tools for proprioceptive circuit elements, including the sensory receptors, are beginning to offer new and unprecedented leverage to dissect the central pathways responsible for proprioceptive encoding. Simultaneously, new data derived through emerging bionic neural machine-interface technologies reveal clues regarding the relative importance of kinesthetic sensory feedback and insights into the functional proprioceptive substrates that underlie natural motor behaviors.

A cortical network processes auditory error signals during human speech production to maintain fluency.

Hearing one's own voice is critical for fluent speech production as it allows for the detection and correction of vocalization errors in real time. This behavior known as the auditory feedback control of speech is impaired in various neurological disorders ranging from stuttering to aphasia; however, the underlying neural mechanisms are still poorly understood. Computational models of speech motor control suggest that, during speech production, the brain uses an efference copy of the motor command to generate an internal estimate of the speech output. When actual feedback differs from this internal estimate, an error signal is generated to correct the internal estimate and update necessary motor commands to produce intended speech. We were able to localize the auditory error signal using electrocorticographic recordings from neurosurgical participants during a delayed auditory feedback (DAF) paradigm. In this task, participants hear their voice with a time delay as they produced words and sentences (similar to an echo on a conference call), which is well known to disrupt fluency by causing slow and stutter-like speech in humans. We observed a significant response enhancement in auditory cortex that scaled with the duration of feedback delay, indicating an auditory speech error signal. Immediately following auditory cortex, dorsal precentral gyrus (dPreCG), a region that has not been implicated in auditory feedback processing before, exhibited a markedly similar response enhancement, suggesting a tight coupling between the 2 regions. Critically, response enhancement in dPreCG occurred only during articulation of long utterances due to a continuous mismatch between produced speech and reafferent feedback. These results suggest that dPreCG plays an essential role in processing auditory error signals during speech production to maintain fluency.

A high-throughput method to deliver targeted optogenetic stimulation to moving C. elegans populations.

We present a high-throughput optogenetic illumination system capable of simultaneous closed-loop light delivery to specified targets in populations of moving Caenorhabditis elegans. The instrument addresses three technical challenges: It delivers targeted illumination to specified regions of the animal's body such as its head or tail; it automatically delivers stimuli triggered upon the animal's behavior; and it achieves high throughput by targeting many animals simultaneously. The instrument was used to optogenetically probe the animal's behavioral response to competing mechanosensory stimuli in the the anterior and posterior gentle touch receptor neurons. Responses to more than 43,418 stimulus events from a range of anterior-posterior intensity combinations were measured. The animal's probability of sprinting forward in response to a mechanosensory stimulus depended on both the anterior and posterior stimulation intensity, while the probability of reversing depended primarily on the anterior stimulation intensity. We also probed the animal's response to mechanosensory stimulation during the onset of turning, a relatively rare behavioral event, by delivering stimuli automatically when the animal began to turn. Using this closed-loop approach, over 9,700 stimulus events were delivered during turning onset at a rate of 9.2 events per worm hour, a greater than 25-fold increase in throughput compared to previous investigations. These measurements validate with greater statistical power previous findings that turning acts to gate mechanosensory evoked reversals. Compared to previous approaches, the current system offers targeted optogenetic stimulation to specific body regions or behaviors with many fold increases in throughput to better constrain quantitative models of sensorimotor processing.

Precise cortical contributions to sensorimotor feedback control during reactive balance.

The role of the cortex in shaping automatic whole-body motor behaviors such as walking and balance is poorly understood. Gait and balance are typically mediated through subcortical circuits, with the cortex becoming engaged as needed on an individual basis by task difficulty and complexity. However, we lack a mechanistic understanding of how increased cortical contribution to whole-body movements shapes motor output. Here we use reactive balance recovery as a paradigm to identify relationships between hierarchical control mechanisms and their engagement across balance tasks of increasing difficulty in young adults. We hypothesize that parallel sensorimotor feedback loops engaging subcortical and cortical circuits contribute to balance-correcting muscle activity, and that the involvement of cortical circuits increases with balance challenge. We decomposed balance-correcting muscle activity based on hypothesized subcortically- and cortically-mediated feedback components driven by similar sensory information, but with different loop delays. The initial balance-correcting muscle activity was engaged at all levels of balance difficulty. Its onset latency was consistent with subcortical sensorimotor loops observed in the lower limb. An even later, presumed, cortically-mediated burst of muscle activity became additionally engaged as balance task difficulty increased, at latencies consistent with longer transcortical sensorimotor loops. We further demonstrate that evoked cortical activity in central midline areas measured using electroencephalography (EEG) can be explained by a similar sensory transformation as muscle activity but at a delay consistent with its role in a transcortical loop driving later cortical contributions to balance-correcting muscle activity. These results demonstrate that a neuromechanical model of muscle activity can be used to infer cortical contributions to muscle activity without recording brain activity. Our model may provide a useful framework for evaluating changes in cortical contributions to balance that are associated with falls in older adults and in neurological disorders such as Parkinson's disease.

Dorsomedial frontal cortex damage impairs error-based, but not reinforcement-based motor learning in humans.

We adapt our movements to new and changing environments through multiple processes. Sensory error-based learning counteracts environmental perturbations that affect the sensory consequences of movements. Sensory errors also cause the upregulation of reflexes and muscle co-contraction. Reinforcement-based learning enhances the selection of movements that produce rewarding outcomes. Although some findings have identified dissociable neural substrates of sensory error- and reinforcement-based learning, correlative methods have implicated dorsomedial frontal cortex in both. Here, we tested the causal contributions of dorsomedial frontal to adaptive motor control, studying people with chronic damage to this region. Seven human participants with focal brain lesions affecting the dorsomedial frontal and 20 controls performed a battery of arm movement tasks. Three experiments tested: (i) the upregulation of visuomotor reflexes and muscle co-contraction in response to unpredictable mechanical perturbations, (ii) sensory error-based learning in which participants learned to compensate predictively for mechanical force-field perturbations, and (iii) reinforcement-based motor learning based on binary feedback in the absence of sensory error feedback. Participants with dorsomedial frontal damage were impaired in the early stages of force field adaptation, but performed similarly to controls in all other measures. These results provide evidence for a specific and selective causal role for the dorsomedial frontal in sensory error-based learning.

Sensory error drives fine motor adjustment.

Fine audiovocal control is a hallmark of human speech production and depends on precisely coordinated muscle activity guided by sensory feedback. Little is known about shared audiovocal mechanisms between humans and other mammals. We hypothesized that real-time audiovocal control in bat echolocation uses the same computational principles as human speech. To test the prediction of this hypothesis, we applied state feedback control (SFC) theory to the analysis of call frequency adjustments in the echolocating bat, Hipposideros armiger. This model organism exhibits well-developed audiovocal control to sense its surroundings via echolocation. Our experimental paradigm was analogous to one implemented in human subjects. We measured the bats' vocal responses to spectrally altered echolocation calls. Individual bats exhibited highly distinct patterns of vocal compensation to these altered calls. Our findings mirror typical observations of speech control in humans listening to spectrally altered speech. Using mathematical modeling, we determined that the same computational principles of SFC apply to bat echolocation and human speech, confirming the prediction of our hypothesis.

Proprioceptive and Visual Feedback Responses in Macaques Exploit Goal Redundancy.

A common problem in motor control concerns how to generate patterns of muscle activity when there are redundant solutions to attain a behavioral goal. Optimal feedback control is a theory that has guided many behavioral studies exploring how the motor system incorporates task redundancy. This theory predicts that kinematic errors that deviate the limb should not be corrected if one can still attain the behavioral goal. Studies in humans demonstrate that the motor system can flexibly integrate visual and proprioceptive feedback of the limb with goal redundancy within 90 ms and 70 ms, respectively. Here, we show monkeys (Macaca mulatta) demonstrate similar abilities to exploit goal redundancy. We trained four male monkeys to reach for a goal that was either a narrow square or a wide, spatially redundant rectangle. Monkeys exhibited greater trial-by-trial variability when reaching to the wide goal consistent with exploiting goal redundancy. On random trials we jumped the visual feedback of the hand and found monkeys corrected for the jump when reaching to the narrow goal and largely ignored the jump when reaching for the wide goal. In a separate set of experiments, we applied mechanical loads to the arm of the monkey and found similar corrective responses based on goal shape. Muscle activity reflecting these different corrective responses were detected for the visual and mechanical perturbations starting at ∼90 and ∼70 ms, respectively. Thus, rapid motor responses in macaques can exploit goal redundancy similar to humans, creating a paradigm to study the neural basis of goal-directed motor action and motor redundancy.SIGNIFICANCE STATEMENT Moving in the world requires selecting from an infinite set of possible motor commands. Theories predict that motor commands are selected that exploit redundancies. Corrective responses in humans to either visual or proprioceptive disturbances of the limb can rapidly exploit redundant trajectories to a goal in <100 ms after a disturbance. However, uncovering the neural correlates generating these rapid motor corrections has been hampered by the absence of an animal model. We developed a behavioral paradigm in monkeys that incorporates redundancy in the form of the shape of the goal. Critically, monkeys exhibit corrective responses and timings similar to humans performing the same task. Our paradigm provides a model for investigating the neural correlates of sophisticated rapid motor corrections.

Brain-Machine Interfaces: From Basic Science to Neuroprostheses and Neurorehabilitation.

Brain-machine interfaces (BMIs) combine methods, approaches, and concepts derived from neurophysiology, computer science, and engineering in an effort to establish real-time bidirectional links between living brains and artificial actuators. Although theoretical propositions and some proof of concept experiments on directly linking the brains with machines date back to the early 1960s, BMI research only took off in earnest at the end of the 1990s, when this approach became intimately linked to new neurophysiological methods for sampling large-scale brain activity. The classic goals of BMIs are 1) to unveil and utilize principles of operation and plastic properties of the distributed and dynamic circuits of the brain and 2) to create new therapies to restore mobility and sensations to severely disabled patients. Over the past decade, a wide range of BMI applications have emerged, which considerably expanded these original goals. BMI studies have shown neural control over the movements of robotic and virtual actuators that enact both upper and lower limb functions. Furthermore, BMIs have also incorporated ways to deliver sensory feedback, generated from external actuators, back to the brain. BMI research has been at the forefront of many neurophysiological discoveries, including the demonstration that, through continuous use, artificial tools can be assimilated by the primate brain's body schema. Work on BMIs has also led to the introduction of novel neurorehabilitation strategies. As a result of these efforts, long-term continuous BMI use has been recently implicated with the induction of partial neurological recovery in spinal cord injury patients.

Implicit Adaptation Is Modulated by the Relevance of Feedback.

Given that informative and relevant feedback in the real world is often intertwined with distracting and irrelevant feedback, we asked how the relevancy of visual feedback impacts implicit sensorimotor adaptation. To tackle this question, we presented multiple cursors as visual feedback in a center-out reaching task and varied the task relevance of these cursors. In other words, participants were instructed to hit a target with a specific task-relevant cursor, while ignoring the other cursors. In Experiment 1, we found that reach aftereffects were attenuated by the mere presence of distracting cursors, compared with reach aftereffects in response to a single task-relevant cursor. The degree of attenuation did not depend on the position of the distracting cursors. In Experiment 2, we examined the interaction between task relevance and attention. Participants were asked to adapt to a task-relevant cursor/target pair, while ignoring the task-irrelevant cursor/target pair. Critically, we jittered the location of the relevant and irrelevant target in an uncorrelated manner, allowing us to index attention via how well participants tracked the position of target. We found that participants who were better at tracking the task-relevant target/cursor pair showed greater aftereffects, and interestingly, the same correlation applied to the task-irrelevant target/cursor pair. Together, these results highlight a novel role of task relevancy on modulating implicit adaptation, perhaps by giving greater attention to informative sources of feedback, increasing the saliency of the sensory prediction error.

Improving motor imagery through a mirror box for BCI users.

The aim of this study was to evaluate mirror visual feedback (MVF) as a training tool for brain-computer interface (BCI) users. This is because approximately 20-30% of subjects require more training to operate a BCI system using motor imagery. Electroencephalograms (EEGs) were recorded from 18 healthy subjects, using event-related desynchronization (ERD) to observe the responses during the movement or movement intention of the hand for the conditions of control, imagination, and the MVF with the mirror box. We constituted two groups: group 1: control, imagination, and MVF; group 2: control, MVF, and imagination. There were significant differences in imagination conditions between groups using MVF before or after imagination (right-hand, P = 0.0403; left-hand, P = 0.00939). The illusion of movement through MVF is not possible in all subjects, but even in those cases, we found an increase in imagination when the subject used the MVF previously. The increase in the r2s of imagination in the right and left hands suggests cross-learning. The increase in motor imagery recorded with EEG after MVF suggests that the mirror box made it easier to imagine movements. Our results provide evidence that the MVF could be used as a training tool to improve motor imagery.NEW & NOTEWORTHY The increase in motor imagery recorded with EEG after MVF (mirror visual feedback) suggests that the mirror box made it easier to imagine movements. Our results demonstrate that MVF could be used as a training tool to improve motor imagery.

Imprecise perception of hand position during early motor adaptation.

Localizing one's body parts is important for movement control and motor learning. Recent studies have shown that the precision with which people localize their hand places constraints on motor adaptation. Although these studies have assumed that hand localization remains equally precise across learning, we show that precision decreases rapidly during early motor learning. In three experiments, healthy young participants (n = 92) repeatedly adapted to a 45° visuomotor rotation for a cycle of two to four reaches, followed by a cycle of two to four reaches with veridical feedback. Participants either used an aiming strategy that fully compensated for the rotation (experiment 1), or always aimed directly at the target, so that adaptation was implicit (experiment 2). We omitted visual feedback for the last reach of each cycle, after which participants localized their unseen hand. We observed an increase in the variability of angular localization errors when subjects used a strategy to counter the visuomotor rotation (experiment 1). This decrease in precision was less pronounced in the absence of reaiming (experiment 2), and when subjects knew that they would have to localize their hand on the upcoming trial, and could thus focus on hand position (experiment 3). We propose that strategic reaiming decreases the precision of perceived hand position, possibly due to attention to vision rather than proprioception. We discuss how these dynamics in precision during early motor learning could impact on motor control and shape the interplay between implicit and strategy-based motor adaptation.NEW & NOTEWORTHY Recent studies indicate that the precision with which people localize their hand limits implicit visuomotor learning. We found that localization precision is not static, but decreases early during learning. This decrease is pronounced when people apply a reaiming strategy to compensate for a visuomotor perturbation and is partly resistant to allocation of attention to the hand. We propose that these dynamics in position sense during learning may influence how implicit and strategy-based motor adaption interact.

Corollary discharge enables proprioception from lateral line sensory feedback.

Animals modulate sensory processing in concert with motor actions. Parallel copies of motor signals, called corollary discharge (CD), prepare the nervous system to process the mixture of externally and self-generated (reafferent) feedback that arises during locomotion. Commonly, CD in the peripheral nervous system cancels reafference to protect sensors and the central nervous system from being fatigued and overwhelmed by self-generated feedback. However, cancellation also limits the feedback that contributes to an animal's awareness of its body position and motion within the environment, the sense of proprioception. We propose that, rather than cancellation, CD to the fish lateral line organ restructures reafference to maximize proprioceptive information content. Fishes' undulatory body motions induce reafferent feedback that can encode the body's instantaneous configuration with respect to fluid flows. We combined experimental and computational analyses of swimming biomechanics and hair cell physiology to develop a neuromechanical model of how fish can track peak body curvature, a key signature of axial undulatory locomotion. Without CD, this computation would be challenged by sensory adaptation, typified by decaying sensitivity and phase distortions with respect to an input stimulus. We find that CD interacts synergistically with sensor polarization to sharpen sensitivity along sensors' preferred axes. The sharpening of sensitivity regulates spiking to a narrow interval coinciding with peak reafferent stimulation, which prevents adaptation and homogenizes the otherwise variable sensor output. Our integrative model reveals a vital role of CD for ensuring precise proprioceptive feedback during undulatory locomotion, which we term external proprioception.

New evidence for the sensorimotor mismatch theory of weight perception and the size-weight illusion.

The size-weight illusion is a phenomenon where a smaller object is perceived heavier than an equally weighted larger object. The sensorimotor mismatch theory proposed that this illusion occurs because of a mismatch between efferent motor commands and afferent sensory feedback received when lifting large and small objects (i.e., the application of too little and too much lifting force, respectively). This explanation has been undermined by studies demonstrating a separation between the perceived weight of objects and the lifting forces that are applied on them. However, this research suffers from inconsistencies in the choice of lifting force measures reported. Therefore, we examined the contribution of sensorimotor mismatch in the perception of weight in the size-weight illusion and in non-size-weight illusion stimuli and evaluated the use of a lifting force aggregate measure comprising the four most common lifting force measures used in previous research. In doing so, the sensorimotor mismatch theory was mostly supported. In a size-weight illusion experiment, the lifting forces correlated with weight perception and, contrary to some earlier research, did not adapt over time. In a non-size-weight illusion experiment, switches between lifting light and heavy objects resulted in perceiving the weight of these objects differently compared to no switch trials, which mirrored differences in the manner participants applied forces on the objects. Additionally, we reveal that our force aggregate measure can allow for a more sensitive and objective examination of the effects of lifting forces on objects.

Impairment of the internal forward model and feedback mechanisms for vocal sensorimotor control in post-stroke aphasia: evidence from directional responses to altered auditory feedback.

The present study examined opposing and following vocal responses to altered auditory feedback (AAF) to determine how damage to left-hemisphere brain networks impairs the internal forward model and feedback mechanisms in post-stroke aphasia. Forty-nine subjects with aphasia and sixty age-matched controls performed speech vowel production tasks while their auditory feedback was altered using randomized ± 100 cents upward and downward pitch-shift stimuli. Data analysis revealed that when vocal responses were averaged across all trials (i.e., opposing and following), the overall magnitude of vocal compensation was significantly reduced in the aphasia group compared with controls. In addition, when vocal responses were analyzed separately for opposing and following trials, subjects in the aphasia group showed a significantly lower percentage of opposing and higher percentage of following vocal response trials compared with controls, particularly for the upward pitch-shift stimuli. However, there was no significant difference in the magnitude of opposing and following vocal responses between the two groups. These findings further support previous evidence on the impairment of vocal sensorimotor control in aphasia and provide new insights into the distinctive impact of left-hemisphere stroke on the internal forward model and feedback mechanisms. In this context, we propose that the lower percentage of opposing responses in aphasia may be accounted for by deficits in feedback-dependent mechanisms of audio-vocal integration and motor control. In addition, the higher percentage of following responses may reflect aberrantly increased reliance of the speech system on the internal forward model for generating sensory predictions during vocal error detection and motor control.