Incorporating feedback from multiple sensory modalities enhances brain-machine interface control.

The brain typically uses a rich supply of feedback from multiple sensory modalities to control movement in healthy individuals. In many individuals, these afferent pathways, as well as their efferent counterparts, are compromised by disease or injury resulting in significant impairments and reduced quality of life. Brain-machine interfaces (BMIs) offer the promise of recovered functionality to these individuals by allowing them to control a device using their thoughts. Most current BMI implementations use visual feedback for closed-loop control; however, it has been suggested that the inclusion of additional feedback modalities may lead to improvements in control. We demonstrate for the first time that kinesthetic feedback can be used together with vision to significantly improve control of a cursor driven by neural activity of the primary motor cortex (MI). Using an exoskeletal robot, the monkey's arm was moved to passively follow a cortically controlled visual cursor, thereby providing the monkey with kinesthetic information about the motion of the cursor. When visual and proprioceptive feedback were congruent, both the time to successfully reach a target decreased and the cursor paths became straighter, compared with incongruent feedback conditions. This enhanced performance was accompanied by a significant increase in the amount of movement-related information contained in the spiking activity of neurons in MI. These findings suggest that BMI control can be significantly improved in paralyzed patients with residual kinesthetic sense and provide the groundwork for augmenting cortically controlled BMIs with multiple forms of natural or surrogate sensory feedback.

Study of stability of time-domain features for electromyographic pattern recognition.

BACKGROUND: Significant progress has been made towards the clinical application of human-machine interfaces (HMIs) based on electromyographic (EMG) pattern recognition for various rehabilitation purposes. Making this technology practical and available to patients with motor deficits requires overcoming real-world challenges, such as physical and physiological changes, that result in variations in EMG signals and systems that are unreliable for long-term use. In this study, we aimed to address these challenges by (1) investigating the stability of time-domain EMG features during changes in the EMG signals and (2) identifying the feature sets that would provide the most robust EMG pattern recognition. METHODS: Variations in EMG signals were introduced during physical experiments. We identified three disturbances that commonly affect EMG signals: EMG electrode location shift, variation in muscle contraction effort, and muscle fatigue. The impact of these disturbances on individual features and combined feature sets was quantified by changes in classification performance. The robustness of feature sets was evaluated by a stability index developed in this study. RESULTS: Muscle fatigue had the smallest effect on the studied EMG features, while electrode location shift and varying effort level significantly reduced the classification accuracy for most of the features. Under these disturbances, the most stable EMG feature set with combination of four features produced at least 16.0% higher classification accuracy than the least stable set. EMG autoregression coefficients and cepstrum coefficients showed the most robust classification performance of all studied time-domain features. CONCLUSIONS: Selecting appropriate EMG feature combinations can overcome the impact of the studied disturbances on EMG pattern classification to a certain extent; however, this simple solution is still inadequate. Stabilizing electrode contact locations and developing effective classifier training strategies are suggested to further improve the robustness of HMIs based on EMG pattern recognition.

Congruent activity during action and action observation in motor cortex.

A variety of studies have shown that motor cortical areas can be activated by observation of familiar actions. Here, we describe single-neuron responses in monkey primary motor (MI) and dorsal premotor (PMd) cortices during passive observation and execution of a familiar task. We show that the spiking modulation, preferred directions, and encoded information of cells in MI and PMd remain consistent during both observation and movement. Furthermore, we find that the presence of a visual target is necessary to elicit this congruent neural activity during observation. These findings along with results from our analysis of the oscillatory power in the beta frequency of the local field potential are consistent with previous imaging and EEG studies that have suggested that congruence between observation and action is a general feature of the motor system, even outside of canonical "mirror" areas. Such congruent activity has proposed relevance to motor learning, mimicry, and communication and has practical applications for the development of motor-cortical neuroprostheses in paralyzed patients.

Biomimetic brain machine interfaces for the control of movement.

Quite recently, it has become possible to use signals recorded simultaneously from large numbers of cortical neurons for real-time control. Such brain machine interfaces (BMIs) have allowed animal subjects and human patients to control the position of a computer cursor or robotic limb under the guidance of visual feedback. Although impressive, such approaches essentially ignore the dynamics of the musculoskeletal system, and they lack potentially critical somatosensory feedback. In this mini-symposium, we will initiate a discussion of systems that more nearly mimic the control of natural limb movement. The work that we will describe is based on fundamental observations of sensorimotor physiology that have inspired novel BMI approaches. We will focus on what we consider to be three of the most important new directions for BMI development related to the control of movement. (1) We will present alternative methods for building decoders, including structured, nonlinear models, the explicit incorporation of limb state information, and novel approaches to the development of decoders for paralyzed subjects unable to generate an output signal. (2) We will describe the real-time prediction of dynamical signals, including joint torque, force, and EMG, and the real-time control of physical plants with dynamics like that of the real limb. (3) We will discuss critical factors that must be considered to incorporate somatosensory feedback to the BMI user, including its potential benefits, the differing representations of sensation and perception across cortical areas, and the changes in the cortical representation of tactile events after spinal injury.

Real-time and offline performance of pattern recognition myoelectric control using a generic electrode grid with targeted muscle reinnervation patients.

Targeted muscle reinnervation (TMR) is a surgical technique that creates myoelectric prosthesis control sites for high-level amputees. The electromyographic (EMG) signal patterns provided by the reinnervated muscles are well-suited for pattern recognition control. Pattern recognition allows for control of a greater number of degrees of freedom (DOF) than the conventional, EMG amplitude-based approach. Previous pattern recognition studies have shown benefit in placing electrodes directly over the reinnervated muscles. Localizing the optimal TMR locations is inconvenient and time consuming. In this contribution, we demonstrate that a clinically practical grid arrangement of electrodes yields real-time control performance that is equivalent to, or better than, the site-specific electrode placement for simultaneous control of multiple DOFs using pattern recognition. Additional findings indicate that grid-like electrode arrangement yields significantly lower classification errors for classifiers with a large number of movement classes ( > 9). These findings suggest that a grid electrode arrangement can be effectively used to control a multi-DOF upper limb prosthesis while reducing the time and effort associated with fitting the prosthesis due to clinical localization of control sites on amputee patients.

Exploiting multiple sensory modalities in brain-machine interfaces.

Recent improvements in cortically-controlled brain-machine interfaces (BMIs) have raised hopes that such technologies may improve the quality of life of severely motor-disabled patients. However, current generation BMIs do not perform up to their potential due to the neglect of the full range of sensory feedback in their strategies for training and control. Here we confirm that neurons in the primary motor cortex (MI) encode sensory information and demonstrate a significant heterogeneity in their responses with respect to the type of sensory modality available to the subject about a reaching task. We further show using mutual information and directional tuning analyses that the presence of multi-sensory feedback (i.e. vision and proprioception) during replay of movements evokes neural responses in MI that are almost indistinguishable from those responses measured during overt movement. Finally, we suggest how these playback-evoked responses may be used to improve BMI performance.

Observation-based learning for brain-machine interfaces.

Canonically, 'mirror neurons' are cells in area F5 of the ventral premotor cortex that are active during both observation and execution of goal-directed movements. Recently, cells with similar properties have been observed in a number of other areas in the motor system, including the primary motor cortex. Mirror neurons are a part of a system whose function is thought to involve the prediction and interpretation of the sensory consequences of our own actions as well as the actions of others. Mirror-like responses are relevant to the development of brain-machine interfaces (BMIs) because they provide a robust way to map neural activity to behavior, and because they represent high-level information about goals and intentions that may have utility in future BMI applications.

Data exploration tools for the Gene Ontology database.

MOTIVATION: To improve the ability of biologists (both researchers and students) to ask biologically interesting questions of the Gene Ontology (GO) database and to explore the ontologies by seeing large portions of the ontology graphs in context, along with details of individual terms in the ontologies. RESULTS: GoGet and GoView are two new tools built as part of an extensible web application system based on Java 2 Enterprise Edition technology. GoGet has a user interface that enables users to ask biologically interesting questions, such as (1) What are the DNA binding proteins involved in DNA repair, but not in DNA replication? and (2) Of the terms containing the word triphosphatase, which have associated gene products from mouse, but not fruit fly? The results of such queries can be viewed in a collapsed tabular format that eases the burden of getting through large tables of data. GoView enables users to explore the large directed acyclic graph structure of the ontologies in the GO database. The two tools are coordinated, so that results from queries in GoGet can be visualized in GoView in the ontology in which they appear, and explorations started from GoView can request details of gene product associations to appear in a result table in GoGet. AVAILABILITY: Free access to the GoGet query tool and free download of the GoView ontology viewer are provided to all users at http://db.math.macalester.edu/goproject. In addition, source code for the GoView tool is also available from this site, along with a user manual for both tools.