Competing at the Cybathlon championship for people with disabilities: long-term motor imagery brain-computer interface training of a cybathlete who has tetraplegia.

BACKGROUND: The brain-computer interface (BCI) race at the Cybathlon championship, for people with disabilities, challenges teams (BCI researchers, developers and pilots with spinal cord injury) to control an avatar on a virtual racetrack without movement. Here we describe the training regime and results of the Ulster University BCI Team pilot who has tetraplegia and was trained to use an electroencephalography (EEG)-based BCI intermittently over 10 years, to compete in three Cybathlon events. METHODS: A multi-class, multiple binary classifier framework was used to decode three kinesthetically imagined movements (motor imagery of left arm, right arm, and feet), and relaxed state. Three game paradigms were used for training i.e., NeuroSensi, Triad, and Cybathlon Race: BrainDriver. An evaluation of the pilot's performance is presented for two Cybathlon competition training periods-spanning 20 sessions over 5 weeks prior to the 2019 competition, and 25 sessions over 5 weeks in the run up to the 2020 competition. RESULTS: Having participated in BCI training in 2009 and competed in Cybathlon 2016, the experienced pilot achieved high two-class accuracy on all class pairs when training began in 2019 (decoding accuracy > 90%, resulting in efficient NeuroSensi and Triad game control). The BrainDriver performance (i.e., Cybathlon race completion time) improved significantly during the training period, leading up to the competition day, ranging from 274-156 s (255 ± 24 s to 191 ± 14 s mean ± std), over 17 days (10 sessions) in 2019, and from 230-168 s (214 ± 14 s to 181 ± 4 s), over 18 days (13 sessions) in 2020. However, on both competition occasions, towards the race date, the performance deteriorated significantly. CONCLUSIONS: The training regime and framework applied were highly effective in achieving competitive race completion times. The BCI framework did not cope with significant deviation in electroencephalography (EEG) observed in the sessions occurring shortly before and during the race day. Changes in cognitive state as a result of stress, arousal level, and fatigue, associated with the competition challenge and performance pressure, were likely contributing factors to the non-stationary effects that resulted in the BCI and pilot achieving suboptimal performance on race day. Trial registration not registered.

Deep Learning Based Inter-subject Continuous Decoding of Motor Imagery for Practical Brain-Computer Interfaces.

Inter-subject transfer learning is a long-standing problem in brain-computer interfaces (BCIs) and has not yet been fully realized due to high inter-subject variability in the brain signals related to motor imagery (MI). The recent success of deep learning-based algorithms in classifying different brain signals warrants further exploration to determine whether it is feasible for the inter-subject continuous decoding of MI signals to provide contingent neurofeedback which is important for neurorehabilitative BCI designs. In this paper, we have shown how a convolutional neural network (CNN) based deep learning framework can be used for inter-subject continuous decoding of MI related electroencephalographic (EEG) signals using the novel concept of Mega Blocks for adapting the network against inter-subject variabilities. These Mega Blocks have the capacity to repeat a specific architectural block several times such as one or more convolutional layers in a single Mega Block. The parameters of such Mega Blocks can be optimized using Bayesian hyperparameter optimization. The results, obtained on the publicly available BCI competition IV-2b dataset, yields an average inter-subject continuous decoding accuracy of 71.49% (κ = 0.42) and 70.84% (κ = 0.42) for two different training methods such as adaptive moment estimation (Adam) and stochastic gradient descent (SGDM), respectively, in 7 out of 9 subjects. Our results show for the first time that it is feasible to use CNN based architectures for inter-subject continuous decoding with a sufficient level of accuracy for developing calibration-free MI-BCIs for practical purposes.

Assessing impact of channel selection on decoding of motor and cognitive imagery from MEG data.

OBJECTIVE: Magnetoencephalography (MEG) based brain-computer interface (BCI) involves a large number of sensors allowing better spatiotemporal resolution for assessing brain activity patterns. There have been many efforts to develop BCI using MEG with high accuracy, though an increase in the number of channels (NoC) means an increase in computational complexity. However, not all sensors necessarily contribute significantly to an increase in classification accuracy (CA) and specifically in the case of MEG-based BCI no channel selection methodology has been performed. Therefore, this study investigates the effect of channel selection on the performance of MEG-based BCI. APPROACH: MEG data were recorded for two sessions from 15 healthy participants performing motor imagery, cognitive imagery and a mixed imagery task pair using a unique paradigm. Performance of four state-of-the-art channel selection methods (i.e. Class-Correlation, ReliefF, Random Forest, and Infinite Latent Feature Selection were applied across six binary tasks in three different frequency bands) were evaluated in this study on two state-of-the-art features, i.e. bandpower and common spatial pattern (CSP). MAIN RESULTS: All four methods provided a statistically significant increase in CA compared to a baseline method using all gradiometer sensors, i.e. 204 channels with band-power features from alpha (8-12 Hz), beta (13-30 Hz), or broadband (α + ß) (8-30 Hz). It is also observed that the alpha frequency band performed better than the beta and broadband frequency bands. The performance of the beta band gave the lowest CA compared with the other two bands. Channel selection improved accuracy irrespective of feature types. Moreover, all the methods reduced the NoC significantly, from 204 to a range of 1-25, using bandpower as a feature and from 15 to 105 for CSP. The optimal channel number also varied not only in each session but also for each participant. Reducing the NoC will help to decrease the computational cost and maintain numerical stability in cases of low trial numbers. SIGNIFICANCE: The study showed significant improvement in performance of MEG-BCI with channel selection irrespective of feature type and hence can be successfully applied for BCI applications.

Tangent Space Features-Based Transfer Learning Classification Model for Two-Class Motor Imagery Brain-Computer Interface.

The performance of a brain-computer interface (BCI) will generally improve by increasing the volume of training data on which it is trained. However, a classifier's generalization ability is often negatively affected when highly non-stationary data are collected across both sessions and subjects. The aim of this work is to reduce the long calibration time in BCI systems by proposing a transfer learning model which can be used for evaluating unseen single trials for a subject without the need for training session data. A method is proposed which combines a generalization of the previously proposed subject-specific "multivariate empirical-mode decomposition" preprocessing technique by taking a fixed band of 8-30Hz for all four motor imagery tasks and a novel classification model which exploits the structure of tangent space features drawn from the Riemannian geometry framework, that is shared among the training data of multiple sessions and subjects. Results demonstrate comparable performance improvement across multiple subjects without subject-specific calibration, when compared with other state-of-the-art techniques.

An electronic approach to minimising moisture-associated skin damage in ostomy patients.

Marked developments in the design of ostomy appliances in recent years have revolutionised stoma care and management but the prevalence of peristomal skin complications continues to be problematic with incidence rates ranging from 10% to 70%. Despite requisite pre and post-operative education for new patients, complications continue to arise - even under the close supervision of specialist nurses. Prolonged exposure of the skin to high pH stoma effluent is widely accepted as a key contributor to the onset of moisture-associated skin disease and it is our hypothesis that a "smart wafer", employing electrochemical manipulation of local pH, could mitigate some of the issues currently plaguing ostomy management. Current electrochemical research strategies translatable to stoma care are presented and their possible implementations critically appraised.

Sensorimotor modulation assessment and brain-computer interface training in disorders of consciousness.

OBJECTIVES: To assess awareness in subjects who are in a minimally conscious state by using an electroencephalogram-based brain-computer interface (BCI), and to determine whether these patients may learn to modulate sensorimotor rhythms with visual feedback, stereo auditory feedback, or both. DESIGN: Initial assessment included imagined hand movement or toe wiggling to activate sensorimotor areas and modulate brain rhythms in 90 trials (4 subjects). Within-subject and within-group analyses were performed to evaluate significant activations. A within-subject analysis was performed involving multiple BCI technology training sessions to improve the capacity of the user to modulate sensorimotor rhythms through visual and auditory feedback. SETTING: Hospital, homes of subjects, and a primary care facility. PARTICIPANTS: Subjects (N=4; 3 men, 1 woman) who were in a minimally conscious state (age range, 27-53 y; 1-12 y after brain injury). INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: Awareness detection was determined from sensorimotor patterns that differed for each motor imagery task. BCI performance was determined from the mean classification accuracy of brain patterns by using a BCI signal processing framework and assessment of performance in multiple sessions. RESULTS: All subjects demonstrated significant and appropriate brain activation during the initial assessment, and real-time feedback was provided to improve arousal. Consistent activation was observed in multiple sessions. CONCLUSIONS: The electroencephalogram-based assessment showed that patients in a minimally conscious state may have the capacity to operate a simple BCI-based communication system, even without any detectable volitional control of movement.

Is sensorimotor BCI performance influenced differently by mono, stereo, or 3-D auditory feedback?

Imagination of movement can be used as a control method for a brain-computer interface (BCI) allowing communication for the physically impaired. Visual feedback within such a closed loop system excludes those with visual problems and hence there is a need for alternative sensory feedback pathways. In the context of substituting the visual channel for the auditory channel, this study aims to add to the limited evidence that it is possible to substitute visual feedback for its auditory equivalent and assess the impact this has on BCI performance. Secondly, the study aims to determine for the first time if the type of auditory feedback method influences motor imagery performance significantly. Auditory feedback is presented using a stepped approach of single (mono), double (stereo), and multiple (vector base amplitude panning as an audio game) loudspeaker arrangements. Visual feedback involves a ball-basket paradigm and a spaceship game. Each session consists of either auditory or visual feedback only with runs of each type of feedback presentation method applied in each session. Results from seven subjects across five sessions of each feedback type (visual, auditory) (10 sessions in total) show that auditory feedback is a suitable substitute for the visual equivalent and that there are no statistical differences in the type of auditory feedback presented across five sessions.

Sensorimotor learning with stereo auditory feedback for a brain-computer interface.

Motor imagery can be used to modulate sensorimotor rhythms (SMR) enabling detection of voltage fluctuations on the surface of the scalp using electroencephalographic electrodes. Feedback is essential in learning to modulate SMR for non-muscular communication using a brain-computer interface (BCI). A BCI not reliant upon the visual modality not only releases the visual channel for other uses but also offers an attractive means of communication for the physically impaired who are also blind or vision impaired. This study demonstrates the feasibility of replacing the traditional visual feedback modality with stereo auditory feedback. Results from a pilot study were used to select the most appropriate sounds for auditory feedback based on three options: broadband noise and two anechoic instrument samples. Subsequently, an SMR BCI was used to examine the effect on sensorimotor learning with broadband noise utilising a modified stereophonic presentation method. Twenty participants split into equal groups took part in ten sessions. The visual group performed best initially but did not improve over time whilst the auditory group improved as the study progressed. The results demonstrate the feasibility of using stereophonic auditory feedback with broadband noise as opposed to other auditory feedback presentation methods and sounds which are less intuitive.

Learning to modulate sensorimotor rhythms with stereo auditory feedback for a brain-computer interface.

Motor imagery can be used to modulate sensorimotor rhythms (SMR) enabling detection of voltage fluctuations on the surface of the scalp using electroencephalographic (EEG) electrodes. Feedback is essential in learning how to intentionally modulate SMR in non-muscular communication using a brain-computer interface (BCI). A BCI that is not reliant upon the visual modality for feedback is an attractive means of communication for the blind and the vision impaired and to release the visual channel for other purposes during BCI usage. The aim of this study is to demonstrate the feasibility of replacing the traditional visual feedback modality with stereo auditory feedback. Twenty participants split into equal groups took part in ten BCI sessions involving motor imagery. The visual feedback group performed best using two performance measures but did not show improvement over time whilst the auditory group improved as the study progressed. Multiple loudspeaker presentation of audio allows the listener to intuitively assign each of two classes to the corresponding lateral position in a free-field listening environment.