Effect of Lower Limb Exoskeleton on the Modulation of Neural Activity and Gait Classification.

Neurorehabilitation with robotic devices requires a paradigm shift to enhance human-robot interaction. The coupling of robot assisted gait training (RAGT) with a brain-machine interface (BMI) represents an important step in this direction but requires better elucidation of the effect of RAGT on the user's neural modulation. Here, we investigated how different exoskeleton walking modes modify brain and muscular activity during exoskeleton assisted gait. We recorded electroencephalographic (EEG) and electromyographic (EMG) activity from ten healthy volunteers walking with an exoskeleton with three modes of user assistance (i.e., transparent, adaptive and full assistance) and during free overground gait. Results identified that exoskeleton walking (irrespective of the exoskeleton mode) induces a stronger modulation of central mid-line mu (8-13 Hz) and low-beta (14-20 Hz) rhythms compared to free overground walking. These modifications are accompanied by a significant re-organization of the EMG patterns in exoskeleton walking. On the other hand, we observed no significant differences in neural activity during exoskeleton walking with the different assistance levels. We subsequently implemented four gait classifiers based on deep neural networks trained on the EEG data during the different walking conditions. Our hypothesis was that exoskeleton modes could impact the creation of a BMI-driven RAGT. We demonstrated that all classifiers achieved an average accuracy of 84.13±3.49% in classifying swing and stance phases on their respective datasets. In addition, we demonstrated that the classifier trained on the transparent mode exoskeleton data can classify gait phases during adaptive and full modes with an accuracy of 78.3±4.8% , while the classifier trained on free overground walking data fails to classify the gait during exoskeleton walking (accuracy of 59.4±11.8% ). These findings provide important insights into the effect of robotic training on neural activity and contribute to the advancement of BMI technology for improving robotic gait rehabilitation therapy.

Real-Time Free Space Semantic Segmentation for Detection of Traversable Space for an Intelligent Wheelchair.

The last decades saw a great innovation in computer vision. Recently, the field has been fundamental in the development of autonomous navigation systems. Modern assistive technologies, like smart wheelchairs, could employ autonomous navigation to assist users during operation. A prerequisite for such systems is to recognise the navigable space in real-time. The current research features an off-the-shelf powered wheelchair customised into an intelligent robot, which perceives the environment using Point Cloud Semantic Segmentation (PCSS). The implemented algorithm is used to distinguish between two conditions, traversable and non-traversable space, in real-time, using the aforementioned conditions as the two labelled classes. The accuracy of traversable space detection resulted as 99.64% while the accuracy of non-traversable space detection was 91.79%. The performance of the suggested method was invariant to changes in wheelchair velocity indicating that the latency of the suggested algorithm is within the tolerable limits for real-time operation.

Neural correlates of user learning during long-term BCI training for the Cybathlon competition.

BACKGROUND: Brain-computer interfaces (BCIs) are systems capable of translating human brain patterns, measured through electroencephalography (EEG), into commands for an external device. Despite the great advances in machine learning solutions to enhance the performance of BCI decoders, the translational impact of this technology remains elusive. The reliability of BCIs is often unsatisfactory for end-users, limiting their application outside a laboratory environment. METHODS: We present the analysis on the data acquired from an end-user during the preparation for two Cybathlon competitions, where our pilot won the gold medal twice in a row. These data are of particular interest given the mutual learning approach adopted during the longitudinal training phase (8 months), the long training break in between the two events (1 year) and the demanding evaluation scenario. A multifaceted perspective on long-term user learning is proposed: we enriched the information gathered through conventional metrics (e.g., accuracy, application performances) by investigating novel neural correlates of learning in different neural domains. RESULTS: First, we showed that by focusing the training on user learning, the pilot was capable of significantly improving his performance over time even with infrequent decoder re-calibrations. Second, we revealed that the analysis of the within-class modifications of the pilot's neural patterns in the Riemannian domain is more effective in tracking the acquisition and the stabilization of BCI skills, especially after the 1-year break. These results further confirmed the key role of mutual learning in the acquisition of BCI skills, and particularly highlighted the importance of user learning as a key to enhance BCI reliability. CONCLUSION: We firmly believe that our work may open new perspectives and fuel discussions in the BCI field to shift the focus of future research: not only to the machine learning of the decoder, but also in investigating novel training procedures to boost the user learning and the stability of the BCI skills in the long-term. To this end, the analyses and the metrics proposed could be used to monitor the user learning during training and provide a marker guiding the decoder re-calibration to maximize the mutual adaptation of the user to the BCI system.

ROS-Neuro: An Open-Source Platform for Neurorobotics.

The growing interest in neurorobotics has led to a proliferation of heterogeneous neurophysiological-based applications controlling a variety of robotic devices. Although recent years have seen great advances in this technology, the integration between human neural interfaces and robotics is still limited, making evident the necessity of creating a standardized research framework bridging the gap between neuroscience and robotics. This perspective paper presents Robot Operating System (ROS)-Neuro, an open-source framework for neurorobotic applications based on ROS. ROS-Neuro aims to facilitate the software distribution, the repeatability of the experimental results, and support the birth of a new community focused on neuro-driven robotics. In addition, the exploitation of Robot Operating System (ROS) infrastructure guarantees stability, reliability, and robustness, which represent fundamental aspects to enhance the translational impact of this technology. We suggest that ROS-Neuro might be the future development platform for the flourishing of a new generation of neurorobots to promote the rehabilitation, the inclusion, and the independence of people with disabilities in their everyday life.

Learning to control a BMI-driven wheelchair for people with severe tetraplegia.

Mind-controlled wheelchairs are an intriguing assistive mobility solution applicable in complete paralysis. Despite progress in brain-machine interface (BMI) technology, its translation remains elusive. The primary objective of this study is to probe the hypothesis that BMI skill acquisition by end-users is fundamental to control a non-invasive brain-actuated intelligent wheelchair in real-world settings. We demonstrate that three tetraplegic spinal-cord injury users could be trained to operate a non-invasive, self-paced thought-controlled wheelchair and execute complex navigation tasks. However, only the two users exhibiting increasing decoding performance and feature discriminancy, significant neuroplasticity changes and improved BMI command latency, achieved high navigation performance. In addition, we show that dexterous, continuous control of robots is possible through low-degree of freedom, discrete and uncertain control channels like a motor imagery BMI, by blending human and artificial intelligence through shared-control methodologies. We posit that subject learning and shared-control are the key components paving the way for translational non-invasive BMI.

Age-related differences in visual P300 ERP during dual-task postural balance.

Standing and concurrently performing a cognitive task is a very common situation in everyday life. It is associated with a higher risk of falling in the elderly. Here, we aim at evaluating the differences of the P300 evoked potential elicited by a visual oddball paradigm between healthy younger (< 35 y) and older (> 64 y) adults during a simultaneous postural task. We found that P300 latency increases significantly (p < 0.001) when the elderly are engaged in more challenging postural tasks; younger adults show no effect of balance condition. Our results demonstrate that, even if the elderly have the same accuracy in odd stimuli detection as younger adults do, they require a longer processing time for stimulus discrimination. This finding suggests an increased attentional load which engages additional cerebral reserves.

Experimental Protocol to Assess Neuromuscular Plasticity Induced by an Exoskeleton Training Session.

Exoskeleton gait rehabilitation is an emerging area of research, with potential applications in the elderly and in people with central nervous system lesions, e.g., stroke, traumatic brain/spinal cord injury. However, adaptability of such technologies to the user is still an unmet goal. Despite important technological advances, these robotic systems still lack the fine tuning necessary to adapt to the physiological modification of the user and are not yet capable of a proper human-machine interaction. Interfaces based on physiological signals, e.g., recorded by electroencephalography (EEG) and/or electromyography (EMG), could contribute to solving this technological challenge. This protocol aims to: (1) quantify neuro-muscular plasticity induced by a single training session with a robotic exoskeleton on post-stroke people and on a group of age and sex-matched controls; (2) test the feasibility of predicting lower limb motor trajectory from physiological signals for future use as control signal for the robot. An active exoskeleton that can be set in full mode (i.e., the robot fully replaces and drives the user motion), adaptive mode (i.e., assistance to the user can be tuned according to his/her needs), and free mode (i.e., the robot completely follows the user movements) will be used. Participants will undergo a preparation session, i.e., EMG sensors and EEG cap placement and inertial sensors attachment to measure, respectively, muscular and cortical activity, and motion. They will then be asked to walk in a 15 m corridor: (i) self-paced without the exoskeleton (pre-training session); (ii) wearing the exoskeleton and walking with the three modes of use; (iii) self-paced without the exoskeleton (post-training session). From this dataset, we will: (1) quantitatively estimate short-term neuroplasticity of brain connectivity in chronic stroke survivors after a single session of gait training; (2) compare muscle activation patterns during exoskeleton-gait between stroke survivors and age and sex-matched controls; and (3) perform a feasibility analysis on the use of physiological signals to decode gait intentions.

Hybrid Human-Machine Interface for Gait Decoding Through Bayesian Fusion of EEG and EMG Classifiers.

Despite the advances in the field of brain computer interfaces (BCI), the use of the sole electroencephalography (EEG) signal to control walking rehabilitation devices is currently not viable in clinical settings, due to its unreliability. Hybrid interfaces (hHMIs) represent a very recent solution to enhance the performance of single-signal approaches. These are classification approaches that combine multiple human-machine interfaces, normally including at least one BCI with other biosignals, such as the electromyography (EMG). However, their use for the decoding of gait activity is still limited. In this work, we propose and evaluate a hybrid human-machine interface (hHMI) to decode walking phases of both legs from the Bayesian fusion of EEG and EMG signals. The proposed hHMI significantly outperforms its single-signal counterparts, by providing high and stable performance even when the reliability of the muscular activity is compromised temporarily (e.g., fatigue) or permanently (e.g., weakness). Indeed, the hybrid approach shows a smooth degradation of classification performance after temporary EMG alteration, with more than 75% of accuracy at 30% of EMG amplitude, with respect to the EMG classifier whose performance decreases below 60% of accuracy. Moreover, the fusion of EEG and EMG information helps keeping a stable recognition rate of each gait phase of more than 80% independently on the permanent level of EMG degradation. From our study and findings from the literature, we suggest that the use of hybrid interfaces may be the key to enhance the usability of technologies restoring or assisting the locomotion on a wider population of patients in clinical applications and outside the laboratory environment.

Uncovering EEG Correlates of Covert Attention in Soccer Goalkeepers: Towards Innovative Sport Training Procedures.

Advances in sports sciences and neurosciences offer new opportunities to design efficient and motivating sport training tools. For instance, using NeuroFeedback (NF), athletes can learn to self-regulate specific brain rhythms and consequently improve their performances. Here, we focused on soccer goalkeepers' Covert Visual Spatial Attention (CVSA) abilities, which are essential for these athletes to reach high performances. We looked for Electroencephalography (EEG) markers of CVSA usable for virtual reality-based NF training procedures, i.e., markers that comply with the following criteria: (1) specific to CVSA, (2) detectable in real-time and (3) related to goalkeepers' performance/expertise. Our results revealed that the best-known EEG marker of CVSA-increased α-power ipsilateral to the attended hemi-field- was not usable since it did not comply with criteria 2 and 3. Nonetheless, we highlighted a significant positive correlation between athletes' improvement in CVSA abilities and the increase of their α-power at rest. While the specificity of this marker remains to be demonstrated, it complied with both criteria 2 and 3. This result suggests that it may be possible to design innovative ecological training procedures for goalkeepers, for instance using a combination of NF and cognitive tasks performed in virtual reality.

Real-time EEG Feedback on Alpha Power Lateralization Leads to Behavioral Improvements in a Covert Attention Task.

Visual attention can be spatially oriented, even in the absence of saccadic eye-movements, to facilitate the processing of incoming visual information. One behavioral proxy for this so-called covert visuospatial attention (CVSA) is the validity effect (VE): the reduction in reaction time (RT) to visual stimuli at attended locations and the increase in RT to stimuli at unattended locations. At the electrophysiological level, one correlate of CVSA is the lateralization in the occipital [Formula: see text]-band oscillations, resulting from [Formula: see text]-power increases ipsilateral and decreases contralateral to the attended hemifield. While this [Formula: see text]-band lateralization has been considerably studied using electroencephalography (EEG) or magnetoencephalography (MEG), little is known about whether it can be trained to improve CVSA behaviorally. In this cross-over sham-controlled study we used continuous real-time feedback of the occipital [Formula: see text]-lateralization to modulate behavioral and electrophysiological markers of covert attention. Fourteen subjects performed a cued CVSA task, involving fast responses to covertly attended stimuli. During real-time feedback runs, trials extended in time if subjects reached states of high [Formula: see text]-lateralization. Crucially, the ongoing [Formula: see text]-lateralization was fed back to the subject by changing the color of the attended stimulus. We hypothesized that this ability to self-monitor lapses in CVSA and thus being able to refocus attention accordingly would lead to improved CVSA performance during subsequent testing. We probed the effect of the intervention by evaluating the pre-post changes in the VE and the [Formula: see text]-lateralization. Behaviorally, results showed a significant interaction between feedback (experimental-sham) and time (pre-post) for the validity effect, with an increase in performance only for the experimental condition. We did not find corresponding pre-post changes in the [Formula: see text]-lateralization. Our findings suggest that EEG-based real-time feedback is a promising tool to enhance the level of covert visuospatial attention, especially with respect to behavioral changes. This opens up the exploration of applications of the proposed training method for the cognitive rehabilitation of attentional disorders.

The Cybathlon BCI race: Successful longitudinal mutual learning with two tetraplegic users.

This work aims at corroborating the importance and efficacy of mutual learning in motor imagery (MI) brain-computer interface (BCI) by leveraging the insights obtained through our participation in the BCI race of the Cybathlon event. We hypothesized that, contrary to the popular trend of focusing mostly on the machine learning aspects of MI BCI training, a comprehensive mutual learning methodology that reinstates the three learning pillars (at the machine, subject, and application level) as equally significant could lead to a BCI-user symbiotic system able to succeed in real-world scenarios such as the Cybathlon event. Two severely impaired participants with chronic spinal cord injury (SCI), were trained following our mutual learning approach to control their avatar in a virtual BCI race game. The competition outcomes substantiate the effectiveness of this type of training. Most importantly, the present study is one among very few to provide multifaceted evidence on the efficacy of subject learning during BCI training. Learning correlates could be derived at all levels of the interface-application, BCI output, and electroencephalography (EEG) neuroimaging-with two end-users, sufficiently longitudinal evaluation, and, importantly, under real-world and even adverse conditions.

Behavioral and Cortical Effects during Attention Driven Brain-Computer Interface Operations in Spatial Neglect: A Feasibility Case Study.

During the last years, several studies have suggested that Brain-Computer Interface (BCI) can play a critical role in the field of motor rehabilitation. In this case report, we aim to investigate the feasibility of a covert visuospatial attention (CVSA) driven BCI in three patients with left spatial neglect (SN). We hypothesize that such a BCI is able to detect attention task-specific brain patterns in SN patients and can induce significant changes in their abnormal cortical activity (α-power modulation, feature recruitment, and connectivity). The three patients were asked to control online a CVSA BCI by focusing their attention at different spatial locations, including their neglected (left) space. As primary outcome, results show a significant improvement of the reaction time in the neglected space between calibration and online modalities (p < 0.01) for the two out of three patients that had the slowest initial behavioral response. Such an evolution of reaction time negatively correlates (p < 0.05) with an increment of the Individual α-Power computed in the pre-cue interval. Furthermore, all patients exhibited a significant reduction of the inter-hemispheric imbalance (p < 0.05) over time in the parieto-occipital regions. Finally, analysis on the inter-hemispheric functional connectivity suggests an increment across modalities for regions in the affected (right) hemisphere and decrement for those in the healthy. Although preliminary, this feasibility study suggests a possible role of BCI in the therapeutic treatment of lateralized, attention-based visuospatial deficits.

Transferring brain-computer interfaces beyond the laboratory: successful application control for motor-disabled users.

OBJECTIVES: Brain-computer interfaces (BCIs) are no longer only used by healthy participants under controlled conditions in laboratory environments, but also by patients and end-users, controlling applications in their homes or clinics, without the BCI experts around. But are the technology and the field mature enough for this? Especially the successful operation of applications - like text entry systems or assistive mobility devices such as tele-presence robots - requires a good level of BCI control. How much training is needed to achieve such a level? Is it possible to train naïve end-users in 10 days to successfully control such applications? MATERIALS AND METHODS: In this work, we report our experiences of training 24 motor-disabled participants at rehabilitation clinics or at the end-users' homes, without BCI experts present. We also share the lessons that we have learned through transferring BCI technologies from the lab to the user's home or clinics. RESULTS: The most important outcome is that 50% of the participants achieved good BCI performance and could successfully control the applications (tele-presence robot and text-entry system). In the case of the tele-presence robot the participants achieved an average performance ratio of 0.87 (max. 0.97) and for the text entry application a mean of 0.93 (max. 1.0). The lessons learned and the gathered user feedback range from pure BCI problems (technical and handling), to common communication issues among the different people involved, and issues encountered while controlling the applications. CONCLUSION: The points raised in this paper are very widely applicable and we anticipate that they might be faced similarly by other groups, if they move on to bringing the BCI technology to the end-user, to home environments and towards application prototype control.

A hybrid BCI for enhanced control of a telepresence robot.

Motor-disabled end users have successfully driven a telepresence robot in a complex environment using a Brain-Computer Interface (BCI). However, to facilitate the interaction aspect that underpins the notion of telepresence, users must be able to voluntarily and reliably stop the robot at any moment, not just drive from point to point. In this work, we propose to exploit the user's residual muscular activity to provide a fast and reliable control channel, which can start/stop the telepresence robot at any moment. Our preliminary results show that not only does this hybrid approach increase the accuracy, but it also helps to reduce the workload and was the preferred control paradigm of all the participants.

Tools for Brain-Computer Interaction: A General Concept for a Hybrid BCI.

The aim of this work is to present the development of a hybrid Brain-Computer Interface (hBCI) which combines existing input devices with a BCI. Thereby, the BCI should be available if the user wishes to extend the types of inputs available to an assistive technology system, but the user can also choose not to use the BCI at all; the BCI is active in the background. The hBCI might decide on the one hand which input channel(s) offer the most reliable signal(s) and switch between input channels to improve information transfer rate, usability, or other factors, or on the other hand fuse various input channels. One major goal therefore is to bring the BCI technology to a level where it can be used in a maximum number of scenarios in a simple way. To achieve this, it is of great importance that the hBCI is able to operate reliably for long periods, recognizing and adapting to changes as it does so. This goal is only possible if many different subsystems in the hBCI can work together. Since one research institute alone cannot provide such different functionality, collaboration between institutes is necessary. To allow for such a collaboration, a new concept and common software framework is introduced. It consists of four interfaces connecting the classical BCI modules: signal acquisition, preprocessing, feature extraction, classification, and the application. But it provides also the concept of fusion and shared control. In a proof of concept, the functionality of the proposed system was demonstrated.

Brain-controlled telepresence robot by motor-disabled people.

In this paper we present the first results of users with disabilities in mentally controlling a telepresence robot, a rather complex task as the robot is continuously moving and the user must control it for a long period of time (over 6 minutes) to go along the whole path. These two users drove the telepresence robot from their clinic more than 100 km away. Remarkably, although the patients had never visited the location where the telepresence robot was operating, they achieve similar performances to a group of four healthy users who were familiar with the environment. In particular, the experimental results reported in this paper demonstrate the benefits of shared control for brain-controlled telepresence robots. It allows all subjects (including novel BMI subjects as our users with disabilities) to complete a complex task in similar time and with similar number of commands to those required by manual control.