In-Car Environment Control Using an SSVEP-Based Brain-Computer Interface with Visual Stimuli Presented on Head-Up Display: Performance Comparison with a Button-Press Interface.

Controlling the in-car environment, including temperature and ventilation, is necessary for a comfortable driving experience. However, it often distracts the driver's attention, potentially causing critical car accidents. In the present study, we implemented an in-car environment control system utilizing a brain-computer interface (BCI) based on steady-state visual evoked potential (SSVEP). In the experiment, four visual stimuli were displayed on a laboratory-made head-up display (HUD). This allowed the participants to control the in-car environment by simply staring at a target visual stimulus, i.e., without pressing a button or averting their eyes from the front. The driving performances in two realistic driving tests-obstacle avoidance and car-following tests-were then compared between the manual control condition and SSVEP-BCI control condition using a driving simulator. In the obstacle avoidance driving test, where participants needed to stop the car when obstacles suddenly appeared, the participants showed significantly shorter response time (1.42 ± 0.26 s) in the SSVEP-BCI control condition than in the manual control condition (1.79 ± 0.27 s). No-response rate, defined as the ratio of obstacles that the participants did not react to, was also significantly lower in the SSVEP-BCI control condition (4.6 ± 14.7%) than in the manual control condition (20.5 ± 25.2%). In the car-following driving test, where the participants were instructed to follow a preceding car that runs at a sinusoidally changing speed, the participants showed significantly lower speed difference with the preceding car in the SSVEP-BCI control condition (15.65 ± 7.04 km/h) than in the manual control condition (19.54 ± 11.51 km/h). The in-car environment control system using SSVEP-based BCI showed a possibility that might contribute to safer driving by keeping the driver's focus on the front and thereby enhancing the overall driving performance.

New Method for Pure-Tone Audiometry Using Electrooculogram: A Proof-of-Concept Study.

Precise and timely evaluation of an individual's hearing loss plays an important role in determining appropriate treatment strategies, including medication and aural rehabilitation. However, currently available hearing assessment systems do not satisfy the need for an objective assessment tool with a simple and non-invasive procedure. In this paper, we propose a new method for pure-tone audiometry, which may potentially be used to assess an individual's hearing ability objectively and quantitatively, without need for the user's active response. The proposed method is based on the auditory oculogyric reflex, where the eyes involuntary rotate towards the source of a sound, in response to spatially moving pure-tone audio stimuli modulated at specific frequencies and intensities. We quantitatively analyzed horizontal electrooculograms (EOG) recorded with a pair of electrodes under two conditions-when pure-tone stimuli were (1) "inaudible" or (2) "audible" to a participant. Preliminary experimental results showed significantly increased EOG amplitude in the audible condition compared to the inaudible condition for all ten healthy participants. This demonstrates potential use of the proposed method as a new non-invasive hearing assessment tool.

Speech synthesis from three-axis accelerometer signals using conformer-based deep neural network.

Silent speech interfaces (SSIs) have emerged as innovative non-acoustic communication methods, and our previous study demonstrated the significant potential of three-axis accelerometer-based SSIs to identify silently spoken words with high classification accuracy. The developed accelerometer-based SSI with only four accelerometers and a small training dataset outperformed a conventional surface electromyography (sEMG)-based SSI. In this study, motivated by the promising initial results, we investigated the feasibility of synthesizing spoken speech from three-axis accelerometer signals. This exploration aimed to assess the potential of accelerometer-based SSIs for practical silent communication applications. Nineteen healthy individuals participated in our experiments. Five accelerometers were attached to the face to acquire speech-related facial movements while the participants read 270 Korean sentences aloud. For the speech synthesis, we used a convolution-augmented Transformer (Conformer)-based deep neural network model to convert the accelerometer signals into a Mel spectrogram, from which an audio waveform was synthesized using HiFi-GAN. To evaluate the quality of the generated Mel spectrograms, ten-fold cross-validation was performed, and the Mel cepstral distortion (MCD) was chosen as the evaluation metric. As a result, an average MCD of 5.03 ± 0.65 was achieved using four optimized accelerometers based on our previous study. Furthermore, the quality of generated Mel spectrograms was significantly enhanced by adding one more accelerometer attached under the chin, achieving an average MCD of 4.86 ± 0.65 (p < 0.001, Wilcoxon signed-rank test). Although an objective comparison is difficult, these results surpass those obtained using conventional SSIs based on sEMG, electromagnetic articulography, and electropalatography with the fewest sensors and a similar or smaller number of sentences to train the model. Our proposed approach will contribute to the widespread adoption of accelerometer-based SSIs, leveraging the advantages of accelerometers like low power consumption, invulnerability to physiological artifacts, and high portability.

Novel hybrid visual stimuli incorporating periodic motions into conventional flickering or pattern-reversal visual stimuli for steady-state visual evoked potential-based brain-computer interfaces.

In this study, we proposed a new type of hybrid visual stimuli for steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs), which incorporate various periodic motions into conventional flickering stimuli (FS) or pattern reversal stimuli (PRS). Furthermore, we investigated optimal periodic motions for each FS and PRS to enhance the performance of SSVEP-based BCIs. Periodic motions were implemented by changing the size of the stimulus according to four different temporal functions denoted by none, square, triangular, and sine, yielding a total of eight hybrid visual stimuli. Additionally, we developed the extended version of filter bank canonical correlation analysis (FBCCA), which is a state-of-the-art training-free classification algorithm for SSVEP-based BCIs, to enhance the classification accuracy for PRS-based hybrid visual stimuli. Twenty healthy individuals participated in the SSVEP-based BCI experiment to discriminate four visual stimuli with different frequencies. An average classification accuracy and information transfer rate (ITR) were evaluated to compare the performances of SSVEP-based BCIs for different hybrid visual stimuli. Additionally, the user's visual fatigue for each of the hybrid visual stimuli was also evaluated. As the result, for FS, the highest performances were reported when the periodic motion of the sine waveform was incorporated for all window sizes except for 3 s. For PRS, the periodic motion of the square waveform showed the highest classification accuracies for all tested window sizes. A significant statistical difference in the performance between the two best stimuli was not observed. The averaged fatigue scores were reported to be 5.3 ± 2.05 and 4.05 ± 1.28 for FS with sine-wave periodic motion and PRS with square-wave periodic motion, respectively. Consequently, our results demonstrated that FS with sine-wave periodic motion and PRS with square-wave periodic motion could effectively improve the BCI performances compared to conventional FS and PRS. In addition, thanks to its low visual fatigue, PRS with square-wave periodic motion can be regarded as the most appropriate visual stimulus for the long-term use of SSVEP-based BCIs, particularly for window sizes equal to or larger than 2 s.

Subject-Independent Functional Near-Infrared Spectroscopy-Based Brain-Computer Interfaces Based on Convolutional Neural Networks.

Functional near-infrared spectroscopy (fNIRS) has attracted increasing attention in the field of brain-computer interfaces (BCIs) owing to their advantages such as non-invasiveness, user safety, affordability, and portability. However, fNIRS signals are highly subject-specific and have low test-retest reliability. Therefore, individual calibration sessions need to be employed before each use of fNIRS-based BCI to achieve a sufficiently high performance for practical BCI applications. In this study, we propose a novel deep convolutional neural network (CNN)-based approach for implementing a subject-independent fNIRS-based BCI. A total of 18 participants performed the fNIRS-based BCI experiments, where the main goal of the experiments was to distinguish a mental arithmetic task from an idle state task. Leave-one-subject-out cross-validation was employed to evaluate the average classification accuracy of the proposed subject-independent fNIRS-based BCI. As a result, the average classification accuracy of the proposed method was reported to be 71.20 ± 8.74%, which was higher than the threshold accuracy for effective BCI communication (70%) as well as that obtained using conventional shrinkage linear discriminant analysis (65.74 ± 7.68%). To achieve a classification accuracy comparable to that of the proposed subject-independent fNIRS-based BCI, 24 training trials (of approximately 12 min) were necessary for the traditional subject-dependent fNIRS-based BCI. It is expected that our CNN-based approach would reduce the necessity of long-term individual calibration sessions, thereby enhancing the practicality of fNIRS-based BCIs significantly.

Multi-channel transorbital electrical stimulation for effective stimulation of posterior retina.

Transorbital electrical stimulation (tES) has been studied as a new noninvasive method for treating intractable eye diseases by delivering weak electrical current to the eye through a pair of electrodes attached to the skin around the eye. Studies have reported that the therapeutic effect of tES is determined by the effective stimulation of retinal cells that are densely distributed in the posterior part of the retina. However, in conventional tES with a pair of electrodes, a greater portion of the electric field is delivered to the anterior part of the retina. In this study, to address this issue, a new electrode montage with multiple electrodes was proposed for the effective delivery of electric fields to the posterior retina. Electric field analysis based on the finite element method was performed with a realistic human head model, and optimal injection currents were determined using constrained convex optimization. The resultant electric field distributions showed that the proposed multi-channel tES enables a more effective stimulation of the posterior retina than the conventional tES with a pair of electrodes.

Performance Improvement of Near-Infrared Spectroscopy-Based Brain-Computer Interfaces Using Transcranial Near-Infrared Photobiomodulation With the Same Device.

Transcranial near-infrared photobiomodulation (tNIR-PBM) can modulate physiological characteristics of the human brain, such as the cerebral blood flow and oxidative metabolism. Here, we investigated whether the performance of near-infrared spectroscopy (NIRS)-based brain-computer interfaces (BCIs) can be improved by tNIR-PBM applied to the prefrontal cortex with the same NIRS device. A total of 14 healthy individuals participated in the NIRS-based BCI study where the aim was to distinguish the mental arithmetic task from the idle state (IS) task either after tNIR-PBM or after sham stimulation, with the two experiments being conducted at least two days apart. The tNIR-PBM was applied by simply turning on the NIRS recording equipment for 20 min. To evaluate the degree of performance improvement obtained after tNIR-PBM, the average BCI classification accuracy obtained under the tNIR-PBM condition was compared with that obtained under the sham stimulation condition. The classification accuracy of NIRS-based BCI was significantly improved upon conduction of tNIR-PBM (82.74%) as compared to that in the sham stimulation condition (76.07%, p < 0.005). Thus, our results suggest that simply turning on the NIRS recording equipment before the BCI experiment can improve the performance of the NIRS-based BCI system.

Toward a compact hybrid brain-computer interface (BCI): Performance evaluation of multi-class hybrid EEG-fNIRS BCIs with limited number of channels.

It has been demonstrated that the performance of typical unimodal brain-computer interfaces (BCIs) can be noticeably improved by combining two different BCI modalities. This so-called "hybrid BCI" technology has been studied for decades; however, hybrid BCIs that particularly combine electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) (hereafter referred to as hBCIs) have not been widely used in practical settings. One of the main reasons why hBCI systems are so unpopular is that their hardware is generally too bulky and complex. Therefore, to make hBCIs more appealing, it is necessary to implement a lightweight and compact hBCI system with minimal performance degradation. In this study, we investigated the feasibility of implementing a compact hBCI system with significantly less EEG channels and fNIRS source-detector (SD) pairs, but that can achieve a classification accuracy high enough to be used in practical BCI applications. EEG and fNIRS data were acquired while participants performed three different mental tasks consisting of mental arithmetic, right-hand motor imagery, and an idle state. Our analysis results showed that the three mental states could be classified with a fairly high classification accuracy of 77.6 ± 12.1% using an hBCI system with only two EEG channels and two fNIRS SD pairs.

A Ternary Hybrid EEG-NIRS Brain-Computer Interface for the Classification of Brain Activation Patterns during Mental Arithmetic, Motor Imagery, and Idle State.

The performance of a brain-computer interface (BCI) can be enhanced by simultaneously using two or more modalities to record brain activity, which is generally referred to as a hybrid BCI. To date, many BCI researchers have tried to implement a hybrid BCI system by combining electroencephalography (EEG) and functional near-infrared spectroscopy (NIRS) to improve the overall accuracy of binary classification. However, since hybrid EEG-NIRS BCI, which will be denoted by hBCI in this paper, has not been applied to ternary classification problems, paradigms and classification strategies appropriate for ternary classification using hBCI are not well investigated. Here we propose the use of an hBCI for the classification of three brain activation patterns elicited by mental arithmetic, motor imagery, and idle state, with the aim to elevate the information transfer rate (ITR) of hBCI by increasing the number of classes while minimizing the loss of accuracy. EEG electrodes were placed over the prefrontal cortex and the central cortex, and NIRS optodes were placed only on the forehead. The ternary classification problem was decomposed into three binary classification problems using the "one-versus-one" (OVO) classification strategy to apply the filter-bank common spatial patterns filter to EEG data. A 10 × 10-fold cross validation was performed using shrinkage linear discriminant analysis (sLDA) to evaluate the average classification accuracies for EEG-BCI, NIRS-BCI, and hBCI when the meta-classification method was adopted to enhance classification accuracy. The ternary classification accuracies for EEG-BCI, NIRS-BCI, and hBCI were 76.1 ± 12.8, 64.1 ± 9.7, and 82.2 ± 10.2%, respectively. The classification accuracy of the proposed hBCI was thus significantly higher than those of the other BCIs (p < 0.005). The average ITR for the proposed hBCI was calculated to be 4.70 ± 1.92 bits/minute, which was 34.3% higher than that reported for a previous binary hBCI study.

Performance enhancement of a brain-computer interface using high-density multi-distance NIRS.

This study investigated the effectiveness of using a high-density multi-distance source-detector (SD) separations in near-infrared spectroscopy (NIRS), for enhancing the performance of a functional NIRS (fNIRS)-based brain-computer interface (BCI). The NIRS system that was used for the experiment was capable of measuring signals from four SD separations: 15, 21.2, 30, and 33.5 mm, and this allowed the measurement of hemodynamic response alterations at various depths. Fifteen participants were asked to perform mental arithmetic and word chain tasks, to induce task-related hemodynamic response variations, or they were asked to stay relaxed to acquire a baseline signal. To evaluate the degree of BCI performance enhancement by high-density channel configuration, the classification accuracy obtained using a typical low-density lattice SD arrangement, was compared to that obtained using the high-density SD arrangement, while maintaining the SD separation at 30 mm. The analysis results demonstrated that the use of a high-density channel configuration did not result in a noticeable enhancement of classification accuracy. However, the combination of hemodynamic variations, measured by two multi-distance SD separations, resulted in the significant enhancement of overall classification accuracy. The results of this study indicated that the use of high-density multi-distance SD separations can likely provide a new method for enhancing the performance of an fNIRS-BCI.