Bioinspired Stretchable Fiber-Based Sensor toward Intelligent Human-Machine Interactions.

Wearable integrated sensing devices with flexible electronic elements exhibit enormous potential in human-machine interfaces (HMI), but they have limitations such as complex structures, poor waterproofness, and electromagnetic interference. Herein, inspired by the profile of Lindernia nummularifolia (LN), a bionic stretchable optical strain (BSOS) sensor composed of an LN-shaped optical fiber incorporated with a stretchable substrate is developed for intelligent HMI. Such a sensor enables large strain and bending angle measurements with temperature self-compensation by the intensity difference of two fiber Bragg gratings' (FBGs') center wavelength. Such configurations enable an excellent tensile strain range of up to 80%, moreover, leading to ultrasensitivity, durability (≥20,000 cycles), and waterproofness. The sensor is also capable of measuring different human activities and achieving HMI control, including immersive virtual reality, robot remote interactive control, and personal hands-free communication. Combined with the machine learning technique, gesture classification can be achieved using muscle activity signals captured from the BSOS sensor, which can be employed to obtain the motion intention of the prosthetic. These merits effectively indicate its potential as a solution for medical care HMI and show promise in smart medical and rehabilitation medicine.

Usability, occupational performance and satisfaction evaluation of a smart environment controlled by infrared oculography by people with severe motor disabilities.

A smart environment is an assistive technology space that can enable people with motor disabilities to control their equipment (TV, radio, fan, etc.) through a human-machine interface activated by different inputs. However, assistive technology resources are not always considered useful, reaching quite high abandonment rate. This study aims to evaluate the effectiveness of a smart environment controlled through infrared oculography by people with severe motor disabilities. The study sample was composed of six individuals with motor disabilities. Initially, sociodemographic data forms, the Functional Independence Measure (FIMTM), and the Canadian Occupational Performance Measure (COPM) were applied. The participants used the system in their domestic environment for a week. Afterwards, they were reevaluated with regards to occupational performance (COPM), satisfaction with the use of the assistive technology resource (QUEST 2.0), psychosocial impact (PIADS) and usability of the system (SUS), as well as through semi-structured interviews for suggestions or complaints. The most common demand from the participants of this research was 'control of the TV'. Two participants did not use the system. All participants who used the system (four) presented positive results in all assessment protocols, evidencing greater independence in the control of the smart environment equipment. In addition, they evaluated the system as useful and with good usability. Non-acceptance of disability and lack of social support may have influenced the results.

Effects of brain-computer interface training on upper limb function recovery in stroke patients: A protocol for systematic review and meta-analysis.

BACKGROUND: In recent years, with the development of medical technology and the increase of inter-disciplinary cooperation technology, new methods in the field of artificial intelligence medicine emerge in an endless stream. Brain-computer interface (BCI), as a frontier technology of multidisciplinary integration, has been widely used in various fields. Studies have shown that BCI-assisted training can improve upper limb function in stroke patients, but its effect is still controversial and lacks evidence-based evidence, which requires further exploration and confirmation. Therefore, the main purpose of this paper is to systematically evaluate the efficacy of different BCI-assisted training on upper limb function recovery in stroke patients, to provide a reference for the application of BCI-assisted technology in stroke rehabilitation. METHODS: We will search PubMed, Web of Science, The Cochrane Library, Chinese National Knowledge Infrastructure Database, Wanfang Data, Weipu Electronics, and other databases (from the establishment to February 2021) for full text in Chinese and English. Randomized controlled trials were collected to examine the effect of BCI-assisted training on upper limb functional recovery in stroke patients. We will consider inclusion, select high-quality articles for data extraction and analysis, and summarize the intervention effect of BCI-assisted training on the upper limb function of stroke patients. Two reviewers will screen titles, abstracts, and full texts independently according to inclusion criteria; Data extraction and risk of bias assessment were performed in the included studies. We will use a hierarchy of recommended assessment, development, and assessment methods to assess the overall certainty of the evidence and report findings accordingly. Endnote X8 will be applied in selecting the study, Review Manager 5.3 will be applied in analyzing and synthesizing. RESULTS: The results will provide evidence for judging whether BCI is effective and safe in improving upper limb function in patients with stroke. CONCLUSION: Our study will provide reliable evidence for the effect of BCI technology on the improvement of upper limb function in stroke patients. PROSPERO REGISTRATION NUMBER: CRD42021250378.

P300 latency jitter and its correlates in people with amyotrophic lateral sclerosis.

OBJECTIVE: People with amyotrophic lateral sclerosis (ALS) can benefit from brain-computer interfaces (BCIs). However, users with ALS may experience significant variations in BCI performance and event-related potential (ERP) characteristics. This study investigated latency jitter and its correlates in ALS. METHODS: Electroencephalographic (EEG) responses were recorded from six people with ALS and nine neurotypical controls. ERP amplitudes and latencies were extracted. Classifier-based latency estimation was used to calculate latency jitter. ERP components and latency jitter were compared between groups using Wilcoxon rank-sum tests. Correlations between latency jitter and each of the clinical measures, ERP features, and performance measures were investigated using Spearman and repeated measures correlations. RESULTS: Latency jitter was significantly increased in participants with ALS and significantly negatively correlated with BCI performance in both ALS and control participants. ERP amplitudes were significantly attenuated in ALS, and significant correlations between ERP features and latency jitter were observed. There was no significant correlation between latency jitter and clinical measures. CONCLUSIONS: Latency jitter is increased in ALS and correlates with both BCI performance and ERP features. SIGNIFICANCE: These results highlight the associations of latency jitter with BCI performance and ERP characteristics and could inform future BCI designs for people with ALS.

Efficient human-machine control with asymmetric marginal reliability input devices.

Input devices such as motor-imagery brain-computer interfaces (BCIs) are often unreliable. In theory, channel coding can be used in the human-machine loop to robustly encapsulate intention through noisy input devices but standard feedforward error correction codes cannot be practically applied. We present a practical and general probabilistic user interface for binary input devices with very high noise levels. Our approach allows any level of robustness to be achieved, regardless of noise level, where reliable feedback such as a visual display is available. In particular, we show efficient zooming interfaces based on feedback channel codes for two-class binary problems with noise levels characteristic of modalities such as motor-imagery based BCI, with accuracy <75%. We outline general principles based on separating channel, line and source coding in human-machine loop design. We develop a novel selection mechanism which can achieve arbitrarily reliable selection with a noisy two-state button. We show automatic online adaptation to changing channel statistics, and operation without precise calibration of error rates. A range of visualisations are used to construct user interfaces which implicitly code for these channels in a way that it is transparent to users. We validate our approach with a set of Monte Carlo simulations, and empirical results from a human-in-the-loop experiment showing the approach operates effectively at 50-70% of the theoretical optimum across a range of channel conditions.

A TrAdaBoost Method for Detecting Multiple Subjects' N200 and P300 Potentials Based on Cross-Validation and an Adaptive Threshold.

Traditional training methods need to collect a large amount of data for every subject to train a subject-specific classifier, which causes subjects fatigue and training burden. This study proposes a novel training method, TrAdaBoost based on cross-validation and an adaptive threshold (CV-T-TAB), to reduce the amount of data required for training by selecting and combining multiple subjects' classifiers that perform well on a new subject to train a classifier. This method adopts cross-validation to extend the amount of the new subject's training data and sets an adaptive threshold to select the optimal combination of the classifiers. Twenty-five subjects participated in the N200- and P300-based brain-computer interface. The study compares CV-T-TAB to five traditional training methods by testing them on the training of a support vector machine. The accuracy, information transfer rate, area under the curve, recall and precision are used to evaluate the performances under nine conditions with different amounts of data. CV-T-TAB outperforms the other methods and retains a high accuracy even when the amount of data is reduced to one-third of the original amount. The results imply that CV-T-TAB is effective in improving the performance of a subject-specific classifier with a small amount of data by adopting multiple subjects' classifiers, which reduces the training cost.

Utility and lower limits of frequency detection in surface electrode stimulation for somatosensory brain-computer interface in humans.

OBJECTIVE: Stimulation of the primary somatosensory cortex (S1) has been successful in evoking artificial somatosensation in both humans and animals, but much is unknown about the optimal stimulation parameters needed to generate robust percepts of somatosensation. In this study, the authors investigated frequency as an adjustable stimulation parameter for artificial somatosensation in a closed-loop brain-computer interface (BCI) system. METHODS: Three epilepsy patients with subdural mini-electrocorticography grids over the hand area of S1 were asked to compare the percepts elicited with different stimulation frequencies. Amplitude, pulse width, and duration were held constant across all trials. In each trial, subjects experienced 2 stimuli and reported which they thought was given at a higher stimulation frequency. Two paradigms were used: first, 50 versus 100 Hz to establish the utility of comparing frequencies, and then 2, 5, 10, 20, 50, or 100 Hz were pseudorandomly compared. RESULTS: As the magnitude of the stimulation frequency was increased, subjects described percepts that were "more intense" or "faster." Cumulatively, the participants achieved 98.0% accuracy when comparing stimulation at 50 and 100 Hz. In the second paradigm, the corresponding overall accuracy was 73.3%. If both tested frequencies were less than or equal to 10 Hz, accuracy was 41.7% and increased to 79.4% when one frequency was greater than 10 Hz (p = 0.01). When both stimulation frequencies were 20 Hz or less, accuracy was 40.7% compared with 91.7% when one frequency was greater than 20 Hz (p < 0.001). Accuracy was 85% in trials in which 50 Hz was the higher stimulation frequency. Therefore, the lower limit of detection occurred at 20 Hz, and accuracy decreased significantly when lower frequencies were tested. In trials testing 10 Hz versus 20 Hz, accuracy was 16.7% compared with 85.7% in trials testing 20 Hz versus 50 Hz (p < 0.05). Accuracy was greater than chance at frequency differences greater than or equal to 30 Hz. CONCLUSIONS: Frequencies greater than 20 Hz may be used as an adjustable parameter to elicit distinguishable percepts. These findings may be useful in informing the settings and the degrees of freedom achievable in future BCI systems.

Interhemispheric Asymmetry and Personality Traits of Brain-Computer Interface Users in Hand Movement Imagination.

Personality traits of users can affect the success in controlling brain-computer interfaces (BCIs), and the activity of right and left brain structures may differ depending on personality traits. Earlier, it was not known, how the success of BCI control with different personality traits is associated with interhemispheric asymmetry. In this work, the dependence of the success of imagination of movements, estimated by the success of recognition of EEG signals during imagination of hand movements compared to rest state, on the user's personal characteristics was studied. It is shown that in single control of BCI by naive subjects, recognition success in imagining right-hand (RH) movements was higher in expressive sensitive extroverts, and in imagining left-hand movements (LH) it was higher in practical, reserved, skeptical, and not very sociable persons. It is suggested that this phenomenon may be based on interhemispheric differences in dopamine level and in the way of encoding movement information.

Brain-computer interface (BCI) researcher perspectives on neural data ownership and privacy.

OBJECTIVE: Brain-computer interface (BCI) research and commercially available neural devices generate large amounts of neural data. These data have significant potential value to researchers and industry. Individuals from whose brains neural data derive may want to exert control over what happens to their neural data at study conclusion or as a result of using a consumer device. It is unclear how BCI researchers understand the relationship between neural data and BCI users and what control individuals should have over their neural data. APPROACH: An online survey of BCI researchers (n = 122) gathered perspectives on control of neural data generated in research and non-research contexts. The survey outcomes are discussed and other relevant concerns are highlighted. MAIN RESULTS: The study found that 58% of BCI researchers endorsed giving research participants access to their raw neural data at the conclusion of a study. However, researchers felt that individuals should be limited in their freedom to either donate or sell these data. A majority of researchers viewed raw neural data as a kind of medical data. Survey respondents felt that current laws and regulations were inadequate to protect consumer neural data privacy, though many respondents were also unfamiliar with the details of existing guidelines. SIGNIFICANCE: The majority of BCI researchers believe that individuals should have some but not unlimited control over neural data produced in research and non-research contexts.

EEG artifact correction strategies for online trial-by-trial analysis.

Brain-machine interfaces (BMIs) use brain signals to control closed-loop systems in real-time. This comes with substantial challenges, such as having to remove artifacts in order to extract reliable features, especially when using electroencephalography (EEG). Some approaches have been described in the literature to address online artifact correction. However, none are being used as a 'gold-standard' method, and no research has been conducted to analyze and compare their respective effects on statistical data analysis (inference-based decision). OBJECTIVE: In this paper, we evaluate methods for artifact correction and describe the necessary adjustments to implement them for online EEG data analysis. APPROACH: We investigate the following methods: artifact subspace reconstruction (ASR), fully online and automated artifact removal for brain-computer interfacing (FORCe), online empirical model decomposition (EMD), and online independent component analysis. For assessment, we simulated online data processing using real data from an auditory oddball task. We compared the above methods with classical offline data processing, in their ability (i) to reveal a significant mismatch negativity (MMN) response to auditory stimuli; (ii) to reveal the more subtle modulation of the MMN by contextual changes (namely, the predictability of the sound sequence), and (iii) to identify the most likely learning process that explains the MMN response. MAIN RESULTS: Our results show that ASR and EMD are both able to reveal a significant MMN and its modulation by predictability, and even appear more sensitive than the offline analysis when comparing alternative models of perception underlying auditory evoked responses. SIGNIFICANCE: ASR and EMD show many advantages when compared to other online artifact correction methods. Besides, subtle modulation analysis of the MMN, embedded in perception computational models is a novel method for assessing the quality of artifact correction methods.

Invasive Brain Machine Interface System.

Because of high spatial-temporal resolution of neural signals obtained by invasive recording, the invasive brain-machine interfaces (BMI) have achieved great progress in the past two decades. With success in animal research, BMI technology is transferring to clinical trials for helping paralyzed people to restore their lost motor functions. This chapter gives a brief review of BMI development from animal experiments to human clinical studies in the following aspects: (1) BMIs based on rodent animals; (2) BMI based on non-human primates; and (3) pilot BMIs studies in clinical trials. In the end, the chapter concludes with a summary of potential opportunities and future challenges in BMI technology.

An Integrated Brain-Machine Interface Platform With Thousands of Channels.

Brain-machine interfaces hold promise for the restoration of sensory and motor function and the treatment of neurological disorders, but clinical brain-machine interfaces have not yet been widely adopted, in part, because modest channel counts have limited their potential. In this white paper, we describe Neuralink's first steps toward a scalable high-bandwidth brain-machine interface system. We have built arrays of small and flexible electrode "threads," with as many as 3072 electrodes per array distributed across 96 threads. We have also built a neurosurgical robot capable of inserting six threads (192 electrodes) per minute. Each thread can be individually inserted into the brain with micron precision for avoidance of surface vasculature and targeting specific brain regions. The electrode array is packaged into a small implantable device that contains custom chips for low-power on-board amplification and digitization: The package for 3072 channels occupies less than 23×18.5×2 mm3. A single USB-C cable provides full-bandwidth data streaming from the device, recording from all channels simultaneously. This system has achieved a spiking yield of up to 70% in chronically implanted electrodes. Neuralink's approach to brain-machine interface has unprecedented packaging density and scalability in a clinically relevant package.

A novel dual and triple shifted RSVP paradigm for P300 speller.

BACKGROUND: A speller system enables disabled people, specifically those with spinal cord injuries, to visually select and spell characters. A problem of primary speller systems is that they are gaze shift dependent. To overcome this problem, a single Rapid Serial Visual Presentation (RSVP) paradigm was initially introduced in which characters are displayed one-by-one at the center of a screen. NEW METHOD: Two new protocols, Dual and Triple shifted RSVP paradigms, are introduced and compared against the single paradigm. In the Dual and Triple paradigms, two and three characters are displayed at the center of the screen simultaneously, holding the advantage of displaying the target character twice and three times respectively, compared to the one-time appearance in the single paradigm. To compare the named paradigms, three subjects participated in experiments using all three paradigms. RESULTS: Offline results demonstrate an average character detection accuracy of 97% for the single and double protocols, and 80% for the Triple paradigm. In addition, average ITR is calculated to be 5.45, 7.62 and 7.90 bit/min for the single, Dual and Triple paradigms respectively. Results identify the Dual RSVP paradigm as the most suitable approach that provides the best balance between ITR and character detection accuracy. COMPARISON WITH EXISTING METHODS: The novel speller system (the Dual paradigm) suggested in this paper demonstrates improved performance compared to existing methods, and overcomes the gaze dependency issue. CONCLUSIONS: Overall, our novel method is a reliable alternative that both removes limitations for users suffering from impaired oculomotor control and improves performance.

Neuroprosthetic Speech: The Ethical Significance of Accuracy, Control and Pragmatics.

Neuroprosthetic speech devices are an emerging technology that can offer the possibility of communication to those who are unable to speak. Patients with 'locked in syndrome,' aphasia, or other such pathologies can use covert speech-vividly imagining saying something without actual vocalization-to trigger neural controlled systems capable of synthesizing the speech they would have spoken, but for their impairment.We provide an analysis of the mechanisms and outputs involved in speech mediated by neuroprosthetic devices. This analysis provides a framework for accounting for the ethical significance of accuracy, control, and pragmatic dimensions of prosthesis-mediated speech. We first examine what it means for the output of the device to be accurate, drawing a distinction between technical accuracy on the one hand and semantic accuracy on the other. These are conceptual notions of accuracy.Both technical and semantic accuracy of the device will be necessary (but not yet sufficient) for the user to have sufficient control over the device. Sufficient control is an ethical consideration: we place high value on being able to express ourselves when we want and how we want. Sufficient control of a neural speech prosthesis requires that a speaker can reliably use their speech apparatus as they want to, and can expect their speech to authentically represent them. We draw a distinction between two relevant features which bear on the question of whether the user has sufficient control: voluntariness of the speech and the authenticity of the speech. These can come apart: the user might involuntarily produce an authentic output (perhaps revealing private thoughts) or might voluntarily produce an inauthentic output (e.g., when the output is not semantically accurate). Finally, we consider the role of the interlocutor in interpreting the content and purpose of the communication.These three ethical dimensions raise philosophical questions about the nature of speech, the level of control required for communicative accuracy, and the nature of 'accuracy' with respect to both natural and prosthesis-mediated speech.

An Experiential Learning-Based Approach to Neurofeedback Visualisation in Serious Games.

This study explores brain-computer interfacing, its possible use in serious or educational games and frameworks. Providing real-time feedback regarding cognitive states and behaviours can be a powerful tool for mental health education and games can offer unique and engaging environments for these neurofeedback experiences. We explore how EEG neurofeedback systems can be affordably created for further research and experimentation and suggest design choices that may assist in developing effective experiences of this nature.

Evaluation of flashing stimuli shape and colour heterogeneity using a P300 brain-computer interface speller.

Previous works using a visual P300-based speller have reported an improvement modifying the shape or colour of the presented stimulus. However, the effects of both blended factors have not been yet studied. Thus, the aim of the present work was to study both factors and assess the interaction between them. Fifteen naïve participants tested four different spellers in a calibration and online task. All spellers were similar except the employed illumination of the target stimulus: white letters, white blocks, coloured letters, and coloured blocks. Regarding the results, the block-shaped conditions offered an improvement versus the letter-shaped conditions in the calibration (accuracy) and online (accuracy and correct commands per minute) tasks. The analysis of the event-related potential waveforms showed a larger difference between target and no target stimuli waveforms for the block-shaped conditions versus the letter-shaped. The hypothesis regarding the colour heterogeneity of the stimuli was not found at any level of the analysis. Therefore, this first study combining block-shaped and colour factors, and offering an exhaustive evaluation of both, demonstrated the superiority of block-shaped illumination versus the standard letter-shaped flashing stimuli in classification performance.

Reliability of motor and sensory neural decoding by threshold crossings for intracortical brain-machine interface.

OBJECTIVE: For intracortical neurophysiological studies, spike sorting is an important procedure to isolate single units for analyzing specific functions. However, whether spike sorting is necessary or not for neural decoding applications is controversial. Several studies showed that using threshold crossings (TC) instead of spike sorting could also achieve a similar satisfactory performance. However, such studies were limited in similar behavioral tasks, and the neural signal source mainly focused on the motor-related cortical regions. It is not certain if this conclusion is applicable to other situations. Therefore, we compared the performance of TC and spike sorting in neural decoding with more comprehensive paradigms and parameters. APPROACH: Two rhesus macaques implanted with Utah or floating microelectrode arrays (FMAs) in motor or sensory-related cortical regions were trained to perform a motor or a sensory task. Data from each monkey were preprocessed with three different schemes: TC, automatic sorting (AS), and manual sorting (MS). A support vector machine was used as the decoder, and the decoding accuracy was used for evaluating the performance of three preprocessing methods. Different neural signal sources, different decoders, and related parameters and decoding stability were further tested to systematically compare three preprocessing methods. MAIN RESULTS: TC could achieve a similar (-4.5 RMS threshold) or better (-3.0 RMS threshold) decoding performance compared to the other two sorting methods in the motor or sensory tasks even if the neural signal sources or decoder-related parameters were changed. Moreover, TC was much more stable in neural decoding across sessions and robust to changes of threshold. SIGNIFICANCE: Our results indicated that spike-firing patterns could be stably extracted through TC from multiple cortices in both motor and sensory neural decoding applications. Considering the stability of TC, it might be more suitable for neural decoding compared to sorting methods.

Effect of a combination of flip and zooming stimuli on the performance of a visual brain-computer interface for spelling.

Brain-computer interface (BCI) systems can allow their users to communicate with the external world by recognizing intention directly from their brain activity without the assistance of the peripheral motor nervous system. The P300-speller is one of the most widely used visual BCI applications. In previous studies, a flip stimulus (rotating the background area of the character) that was based on apparent motion, suffered from less refractory effects. However, its performance was not improved significantly. In addition, a presentation paradigm that used a "zooming" action (changing the size of the symbol) has been shown to evoke relatively higher P300 amplitudes and obtain a better BCI performance. To extend this method of stimuli presentation within a BCI and, consequently, to improve BCI performance, we present a new paradigm combining both the flip stimulus with a zooming action. This new presentation modality allowed BCI users to focus their attention more easily. We investigated whether such an action could combine the advantages of both types of stimuli presentation to bring a significant improvement in performance compared to the conventional flip stimulus. The experimental results showed that the proposed paradigm could obtain significantly higher classification accuracies and bit rates than the conventional flip paradigm (p<0.01).

Integrating EEG and MEG Signals to Improve Motor Imagery Classification in Brain-Computer Interface.

We adopted a fusion approach that combines features from simultaneously recorded electroencephalogram (EEG) and magnetoencephalogram (MEG) signals to improve classification performances in motor imagery-based brain-computer interfaces (BCIs). We applied our approach to a group of 15 healthy subjects and found a significant classification performance enhancement as compared to standard single-modality approaches in the alpha and beta bands. Taken together, our findings demonstrate the advantage of considering multimodal approaches as complementary tools for improving the impact of noninvasive BCIs.

Artifact-free recordings in human bidirectional brain-computer interfaces.

OBJECTIVE: Intracortical microstimulation has shown promise as a means of evoking somatosensory percepts as part of a bidirectional brain-computer interface (BCI). However, microstimulation generates large electrical artifacts that dominate the recordings necessary for BCI control. These artifacts must be eliminated from the signal in real-time to allow for uninterrupted BCI decoding. APPROACH: We present a simple, robust modification to an existing clinical BCI system to allow for simultaneous recording and stimulation using a combination of signal blanking and digital filtering, without needing to explicitly account for varying parameters such as electrode locations or amplitudes. We validated our artifact rejection scheme by recording from microelectrodes in primary motor cortex (M1) while stimulating in somatosensory cortex of a person with a spinal cord injury. MAIN RESULTS: M1 recordings were digitally blanked using a sample-and-hold circuit triggered just prior to stimulus onset and a first-order 750 Hz high-pass Butterworth filter was used to reduce distortion of the remaining artifact. This scheme enabled spike detection in M1 to resume as soon as 740 µs after each stimulus pulse. We demonstrated the effectiveness of the complete bidirectional BCI system by comparing functional performance during a 5 degree of freedom robotic arm control task, with and without stimulation. When stimulation was delivered without this artifact rejection scheme, the number of objects the subject was able to move across a table in 2 min under BCI control declined significantly compared to trials without stimulation (p < 0.01). When artifact rejection was implemented, performance was no different than in trials that did not include stimulation (p = 0.621). SIGNIFICANCE: The proposed technique uses simple changes in filtering and digital signal blanking with FDA-cleared hardware and enables artifact-free recordings during bidirectional BCI control.