Electroencephalogram-Based Approaches for Driver Drowsiness Detection and Management: A Review.

Drowsiness is not only a core challenge to safe driving in traditional driving conditions but also a serious obstacle for the wide acceptance of added services of self-driving cars (because drowsiness is, in fact, one of the most representative early-stage symptoms of self-driving carsickness). In view of the importance of detecting drivers' drowsiness, this paper reviews the algorithms of electroencephalogram (EEG)-based drivers' drowsiness detection (DDD). To facilitate the review, the EEG-based DDD approaches are organized into a tree structure taxonomy, having two main categories, namely "detection only (open-loop)" and "management (closed-loop)", both aimed at designing better DDD systems that ensure early detection, reliability and practical utility. To achieve this goal, we addressed seven questions, the answers of which helped in developing an EEG-based DDD system that is superior to the existing ones. A basic assumption in this review article is that although driver drowsiness and carsickness-induced drowsiness are caused by different factors, the brain network that regulates drowsiness is the same.

Affective Computing on Machine Learning-Based Emotion Recognition Using a Self-Made EEG Device.

In this research, we develop an affective computing method based on machine learning for emotion recognition using a wireless protocol and a wearable electroencephalography (EEG) custom-designed device. The system collects EEG signals using an eight-electrode placement on the scalp; two of these electrodes were placed in the frontal lobe, and the other six electrodes were placed in the temporal lobe. We performed experiments on eight subjects while they watched emotive videos. Six entropy measures were employed for extracting suitable features from the EEG signals. Next, we evaluated our proposed models using three popular classifiers: a support vector machine (SVM), multi-layer perceptron (MLP), and one-dimensional convolutional neural network (1D-CNN) for emotion classification; both subject-dependent and subject-independent strategies were used. Our experiment results showed that the highest average accuracies achieved in the subject-dependent and subject-independent cases were 85.81% and 78.52%, respectively; these accuracies were achieved using a combination of the sample entropy measure and 1D-CNN. Moreover, our study investigates the T8 position (above the right ear) in the temporal lobe as the most critical channel among the proposed measurement positions for emotion classification through electrode selection. Our results prove the feasibility and efficiency of our proposed EEG-based affective computing method for emotion recognition in real-world applications.

Smart Wearables with Sensor Fusion for Fall Detection in Firefighting.

During the past decade, falling has been one of the top three causes of death amongst firefighters in China. Even though there are many studies on fall-detection systems (FDSs), the majority use a single motion sensor. Furthermore, few existing studies have considered the impact sensor placement and positioning have on fall-detection performance; most are targeted toward fall detection of the elderly. Unfortunately, floor cracks and unstable building structures in the fireground increase the difficulty of detecting the fall of a firefighter. In particular, the movement activities of firefighters are more varied; hence, distinguishing fall-like activities from actual falls is a significant challenge. This study proposed a smart wearable FDS for firefighter fall detection by integrating motion sensors into the firefighter's personal protective clothing on the chest, elbows, wrists, thighs, and ankles. The firefighter's fall activities are detected by the proposed multisensory recurrent neural network, and the performances of different combinations of inertial measurement units (IMUs) on different body parts were also investigated. The results indicated that the sensor fusion of IMUs from all five proposed body parts achieved performances of 94.10%, 92.25%, and 94.59% in accuracy, sensitivity, and specificity, respectively.

Lightweight multi-hop VLC using compression and data-dependent multiple pulse modulation.

Particulate matter (PM) has a diameter of few micrometers, which causes different illnesses. We used visible light communication (VLC) to transfer PM data to a user monitoring terminal in real-time. To reduce the time and power required for communication, we compressed the PM data. Subsequently, these compressed data were transmitted using a modulation technique called data-dependent multiple pulse position modulation (DDMPPM). We evaluate the performance of DDMPPM for multi-hop communication in VLC through practical experiments. For the same data set, DDMPPM utilizes a lesser frame to transfer PM data. Using DDMPPM, we achieved a total communication distance of 48 m.

Monitoring Transepidermal Water Loss and Skin Wettedness Factor with Battery-Free NFC Sensor.

The transepidermal water loss (TEWL) and the skin wettedness factor (SWF) are considered parts of a key perspective related to skincare. The former is used to determine the loss of water content from the stratum corneum (SC), while the latter is used to determine the human skin comfort level. Herein, we developed two novel approaches: (1) determination of the TEWL and the SWF based on a battery-free humidity sensor, and (2) the design of a battery-free smart skincare sensor device tag that can harvest energy from a near field communication (NFC)-enabled smartphone, making it a battery-free design approach. The designed skincare device tag has a diameter of 2.6 cm and could harvest energy (~3 V) from the NFC-enabled smartphone. A series of experimental tests involving the participation of eight and six subjects were conducted in vivo for the indoor and outdoor environments, respectively. During the experimental analysis, the skin moisture content level was measured at different times of the day using an android smartphone. The TEWL and SWF values were calculated based on these sensor readings. For the TEWL case: if the skin moisture is high, the TEWL is high, and if the skin moisture is low, the TEWL is low, ensuring that the skin moisture and the TEWL follow the same trend. Our smart skincare device is enclosed in a 3D flexible design print, and it is battery-free with an android application interface that is more convenient to carry outside than other commercially available battery-based devices.

Artificial Intelligence-Based Optimal Grasping Control.

A new tactile sensing module was proposed to sense the contact force and location of an object on a robot hand, which was attached on the robot finger. Three air pressure sensors are installed at the tip of the finger to detect the contacting force at the points. To obtain a nominal contact force at the finger from data from the three air pressure sensors, a force estimation was developed based upon the learning of a deep neural network. The data from the three air pressure sensors were utilized as inputs to estimate the contact force at the finger. In the tactile module, the arrival time of the air pressure sensor data has been utilized to recognize the contact point of the robot finger against an object. Using the three air pressure sensors and arrival time, the finger location can be divided into 3 × 3 block locations. The resolution of the contact point recognition was improved to 6 × 4 block locations on the finger using an artificial neural network. The accuracy and effectiveness of the tactile module were verified using real grasping experiments. With this stable grasping, an optimal grasping force was estimated empirically with fuzzy rules for a given object.

Sensor Fusion of Motion-Based Sign Language Interpretation with Deep Learning.

Sign language was designed to allow hearing-impaired people to interact with others. Nonetheless, knowledge of sign language is uncommon in society, which leads to a communication barrier with the hearing-impaired community. Many studies of sign language recognition utilizing computer vision (CV) have been conducted worldwide to reduce such barriers. However, this approach is restricted by the visual angle and highly affected by environmental factors. In addition, CV usually involves the use of machine learning, which requires collaboration of a team of experts and utilization of high-cost hardware utilities; this increases the application cost in real-world situations. Thus, this study aims to design and implement a smart wearable American Sign Language (ASL) interpretation system using deep learning, which applies sensor fusion that "fuses" six inertial measurement units (IMUs). The IMUs are attached to all fingertips and the back of the hand to recognize sign language gestures; thus, the proposed method is not restricted by the field of view. The study reveals that this model achieves an average recognition rate of 99.81% for dynamic ASL gestures. Moreover, the proposed ASL recognition system can be further integrated with ICT and IoT technology to provide a feasible solution to assist hearing-impaired people in communicating with others and improve their quality of life.

A Bipolar-Channel Hybrid Brain-Computer Interface System for Home Automation Control Utilizing Steady-State Visually Evoked Potential and Eye-Blink Signals.

The goal of this study was to develop and validate a hybrid brain-computer interface (BCI) system for home automation control. Over the past decade, BCIs represent a promising possibility in the field of medical (e.g., neuronal rehabilitation), educational, mind reading, and remote communication. However, BCI is still difficult to use in daily life because of the challenges of the unfriendly head device, lower classification accuracy, high cost, and complex operation. In this study, we propose a hybrid BCI system for home automation control with two brain signals acquiring electrodes and simple tasks, which only requires the subject to focus on the stimulus and eye blink. The stimulus is utilized to select commands by generating steady-state visually evoked potential (SSVEP). The single eye blinks (i.e., confirm the selection) and double eye blinks (i.e., deny and re-selection) are employed to calibrate the SSVEP command. Besides that, the short-time Fourier transform and convolution neural network algorithms are utilized for feature extraction and classification, respectively. The results show that the proposed system could provide 38 control commands with a 2 s time window and a good accuracy (i.e., 96.92%) using one bipolar electroencephalogram (EEG) channel. This work presents a novel BCI approach for the home automation application based on SSVEP and eye blink signals, which could be useful for the disabled. In addition, the provided strategy of this study-a friendly channel configuration (i.e., one bipolar EEG channel), high accuracy, multiple commands, and short response time-might also offer a reference for the other BCI controlled applications.

Battery-Free and Noninvasive Estimation of Food pH and CO2 Concentration for Food Monitoring Based on Pressure Measurement.

In this paper, we developed a battery-free system that can be used to estimate food pH level and carbon dioxide (CO2) concentration in a food package from headspace pressure measurement. While being stored, food quality degrades gradually as a function of time and storage conditions. A food monitoring system is, therefore, essential to prevent the detrimental problems of food waste and eating spoilt food. Since conventional works that invasively measure food pH level and CO2 concentration in food packages have shown several disadvantages in terms of power consumption, system size, cost, and reliability, our study proposes a system utilizing package headspace pressure to accurately and noninvasively extract food pH level and CO2 concentration, which reflection food quality. To read pressure data in the food container, a 2.5 cm × 2.5 cm smart sensor tag was designed and integrated with near-field communication (NFC)-based energy harvesting technology for battery-free operation. To validate the reliability of the proposed extraction method, various experiments were conducted with different foods, such as pork, chicken, and fish, in two storage environments. The experimental results show that the designed system can operate in a fully passive mode to communicate with an NFC-enabled smartphone. High correlation coefficients of the headspace pressure with the food pH level and the headspace CO2 concentration were observed in all experiments, demonstrating the ability of the proposed system to estimate food pH level and CO2 concentration with high accuracy. A linear regression model was then trained to linearly fit the sensor data. To display the estimated results, we also developed an Android mobile application with an easy-to-use interface.

High-accuracy VLC-based indoor positioning system using multi-level modulation.

In this study, a novel high-precision positioning algorithm was proposed by using Visible Light Communication (VLC) with only a simple and single receiver. The received voltage-level difference with multi-level modulation was adopted as the input variable, in order to minimize the negative consequences of noise. Then, the relationship between the received voltage-level difference, noise, and position was developed based on the optical propagation model. The minimum mean squares error algorithm and extended-Kalman filter was employed in order to improve the accuracy of the optical model and achieve high performance. Using the developed algorithm, high-accuracy results with a 0.9 cm average position error in the simulation and 2.56 cm average position error in the practical experiments were obtained.

Fine particulate matter monitoring via a visible light communication in DCT-based optical OFDM.

An electromagnetic interference (EMI)-free wide-range indoor dust monitoring system that employs the optical orthogonal frequency-division multiplexing (OFDM)-based visible-light communication (VLC) is proposed. For the long-term transmission of dust information, VLC can be utilized even in EMI-restricted areas, such as medical centers, emergency rooms, and nursing homes. Discrete cosine transform-based optical OFDM is adopted to transmit a large amount of dust information. For robust light detection from eliminate ambient light and low-frequency noise, an average voltage-tracking technique is utilized and as a result LED illumination is detected over 18 m distance with reliable error rate. Wide-range dust information from multiple dust sensors are clearly displayed through the designed user interface. Users can then monitor the air quality in real-time, improving the environmental awareness of individuals.

Detection of Driver Braking Intention Using EEG Signals During Simulated Driving.

In this work, we developed a novel system to detect the braking intention of drivers in emergency situations using electroencephalogram (EEG) signals. The system acquired eight-channel EEG and motion-sensing data from a custom-designed EEG headset during simulated driving. A novel method for accurately labeling the training data during an extremely short period after the onset of an emergency stimulus was introduced. Two types of features, including EEG band power-based and autoregressive (AR)-based, were investigated. It turned out that the AR-based feature in combination with artificial neural network classifier provided better detection accuracy of the system. Experimental results for ten subjects indicated that the proposed system could detect the emergency braking intention approximately 600 ms before the onset of the executed braking event, with high accuracy of 91%. Thus, the proposed system demonstrated the feasibility of developing a brain-controlled vehicle for real-world applications.

An Enhanced Multiplication of RF Energy Harvesting Efficiency Using Relay Resonator for Food Monitoring.

Recently, radio frequency (RF) energy harvesting (RFEH) has become a promising technology for a battery-less sensor module. The ambient RF radiation from the available sources is captured by receiver antennas and converted to electrical energy, which is used to supply smart sensor modules. In this paper, an enhanced method to improve the efficiency of the RFEH system using strongly coupled electromagnetic resonance technology was proposed. A relay resonator was added between the reader and tag antennas to improve the wireless power transmission efficiency to the sensor module. The design of the relay resonator was based on the resonant technique and near-field magnetic coupling concept to improve the communication distance and the power supply for a sensor module. It was designed such that the self-resonant frequencies of the reader antenna, tag antenna, and the relay resonator are synchronous at the HF frequency (13.56MHz). The proposed method was analyzed using Thevenin equivalent circuit, simulated and experimental validated to evaluate its performance. The experimental results showed that the proposed harvesting method is able to generate a great higher power up to 10 times than that provided by conventional harvesting methods without a relay resonator. Moreover, as an empirical feasibility test of the proposed RF energy harvesting device, a smart sensor module which is placed inside a meat box was developed. It was utilized to collect vital data, including temperature, relative humidity and gas concentration, to monitor the freshness of meat. Overall, by exploiting relay resonator, the proposed smart sensor tag could continuously monitor meat freshness without any batteries at the innovative maximum distance of approximately 50 cm.

A Robust Bioimpedance Structure for Smartwatch-Based Blood Pressure Monitoring.

One potential method to estimate noninvasive cuffless blood pressure (BP) is pulse wave velocity (PWV), which can be calculated by using the distance and the transit time of the blood between two arterial sites. To obtain the pulse waveform, bioimpedance (BI) measurement is a promising approach because it continuously reflects the change in BP through the change in the arterial cross-sectional area. Many studies have investigated BI channels in a vertical direction with electrodes located along the wrist and the finger to calculate PWV and convert to BP; however, the measurement systems were relatively large in size. In order to reduce the total device size for use in a PWV-based BP smartwatch, this study proposed and examined a robust horizontal BI structure. The BI device was also designed to apply in a very small body area. The proposed structure was based on two sets of four electrodes attached around the wrist. Our model was evaluated on 15 human subjects; the PWV values were obtained with various distances between two BI channels to assess the efficacy. The results showed that the designed BI system can monitor pulse rate efficiently in only a 0.5 × 1.75 cm² area of the body. The correlation of pulse rate from the proposed design against the reference was 0.98 ± 0.07 (p < 0.001). Our structure yielded higher detection ratios for PWV measurements of 99.0 ± 2.2%, 99.0 ± 2.1%, and 94.8 ± 3.7% at 1, 2, and 3 cm between two BI channels, respectively. The measured PWVs correlated well with the BP standard device at 0.81 ± 0.08 and 0.84 ± 0.07 with low root-mean-squared-errors at 7.47 ± 2.15 mmHg and 5.17 ± 1.81 mmHg for SBP and DBP, respectively. The result demonstrates the potential of a new wearable BP smartwatch structure.

Single cell three-channel wavelength division multiplexing in visible light communication.

This paper examines the design of a prototype of a single cell three-channel visible light communication (VLC) based on wavelength division multiplexing for a radio frequency (RF)-free indoor healthcare. For a low complexity application, a single red green blue (RGB)-type white light-emitting diode (LED) and a single color sensor are adopted. An active low pass filter is utilized for robust light detection to eliminate ambient light and low frequency noise. The incoming tri-color lights are separated by an adopted color sensor and simultaneously demodulated by a receiver processor. Then, the collected data are monitored in real-time and analyzed to provide the necessary medical attention to the concerned patient.

Demonstration of long-distance hazard-free wearable EEG monitoring system using mobile phone visible light communication.

A wearable electroencephalogram (EEG) is a small mobile device used for long-term brain monitoring systems. Applications of these systems include fatigue monitoring, mental/emotional monitoring, and brain-computer interfaces. However, the usage of wireless wearable EEG systems is limited due to the risks posed by the wireless RF communication radiation in a long-term exposure to the human brain. A novel microwave radiation-free system was developed by integrating visible light communication technology into a wearable EEG device. In this work, we investigated the system's performance in transmitting EEG data at different illuminance level and proposed an algorithm that functions at low illuminance levels for increased transmission distance. Using a 30 Hz smartphone camera, the proposed system was able to transmit 2.4 kbps of error-free EEG data up to 4 meter, which is equal to ~300 lux using an aspheric focus lens.

A Hygroscopic Sensor Electrode for Fast Stabilized Non-Contact ECG Signal Acquisition.

A capacitive electrocardiography (cECG) technique using a non-invasive ECG measuring technology that does not require direct contact between the sensor and the skin has attracted much interest. The system encounters several challenges when the sensor electrode and subject's skin are weakly coupled. Because there is no direct physical contact between the subject and any grounding point, there is no discharge path for the built-up electrostatic charge. Subsequently, the electrostatic charge build-up can temporarily contaminate the ECG signal from being clearly visible; a stabilization period (3-15 min) is required for the measurement of a clean, stable ECG signal at low humidity levels (below 55% relative humidity). Therefore, to obtain a clear ECG signal without noise and to reduce the ECG signal stabilization time to within 2 min in a dry ambient environment, we have developed a fabric electrode with embedded polymer (FEEP). The designed hygroscopic FEEP has an embedded superabsorbent polymer layer. The principle of FEEP as a conductive electrode is to provide humidity to the capacitive coupling to ensure strong coupling and to allow for the measurement of a stable, clear biomedical signal. The evaluation results show that hygroscopic FEEP is capable of rapidly measuring high-accuracy ECG signals with a higher SNR ratio.

A Context-Aware EEG Headset System for Early Detection of Driver Drowsiness.

Driver drowsiness is a major cause of mortality in traffic accidents worldwide. Electroencephalographic (EEG) signal, which reflects the brain activities, is more directly related to drowsiness. Thus, many Brain-Machine-Interface (BMI) systems have been proposed to detect driver drowsiness. However, detecting driver drowsiness at its early stage poses a major practical hurdle when using existing BMI systems. This study proposes a context-aware BMI system aimed to detect driver drowsiness at its early stage by enriching the EEG data with the intensity of head-movements. The proposed system is carefully designed for low-power consumption with on-chip feature extraction and low energy Bluetooth connection. Also, the proposed system is implemented using JAVA programming language as a mobile application for on-line analysis. In total, 266 datasets obtained from six subjects who participated in a one-hour monotonous driving simulation experiment were used to evaluate this system. According to a video-based reference, the proposed system obtained an overall detection accuracy of 82.71% for classifying alert and slightly drowsy events by using EEG data alone and 96.24% by using the hybrid data of head-movement and EEG. These results indicate that the combination of EEG data and head-movement contextual information constitutes a robust solution for the early detection of driver drowsiness.

Estimation of eye closure degree using EEG sensors and its application in driver drowsiness detection.

Currently, driver drowsiness detectors using video based technology is being widely studied. Eyelid closure degree (ECD) is the main measure of the video-based methods, however, drawbacks such as brightness limitations and practical hurdles such as distraction of the drivers limits its success. This study presents a way to compute the ECD using EEG sensors instead of video-based methods. The premise is that the ECD exhibits a linear relationship with changes of the occipital EEG. A total of 30 subjects are included in this study: ten of them participated in a simple proof-of-concept experiment to verify the linear relationship between ECD and EEG, and then twenty participated in a monotonous highway driving experiment in a driving simulator environment to test the robustness of the linear relationship in real-life applications. Taking the video-based method as a reference, the Alpha power percentage from the O2 channel is found to be the best input feature for linear regression estimation of the ECD. The best overall squared correlation coefficient (SCC, denoted by r2) and mean squared error (MSE) validated by linear support vector regression model and leave one subject out method is r2 = 0.930 and MSE = 0.013. The proposed linear EEG-ECD model can achieve 87.5% and 70.0% accuracy for male and female subjects, respectively, for a driver drowsiness application, percentage eyelid closure over the pupil over time (PERCLOS). This new ECD estimation method not only addresses the video-based method drawbacks, but also makes ECD estimation more computationally efficient and easier to implement in EEG sensors in a real time way.

Mobile healthcare for automatic driving sleep-onset detection using wavelet-based EEG and respiration signals.

Driving drowsiness is a major cause of traffic accidents worldwide and has drawn the attention of researchers in recent decades. This paper presents an application for in-vehicle non-intrusive mobile-device-based automatic detection of driver sleep-onset in real time. The proposed application classifies the driving mental fatigue condition by analyzing the electroencephalogram (EEG) and respiration signals of a driver in the time and frequency domains. Our concept is heavily reliant on mobile technology, particularly remote physiological monitoring using Bluetooth. Respiratory events are gathered, and eight-channel EEG readings are captured from the frontal, central, and parietal (Fpz-Cz, Pz-Oz) regions. EEGs are preprocessed with a Butterworth bandpass filter, and features are subsequently extracted from the filtered EEG signals by employing the wavelet-packet-transform (WPT) method to categorize the signals into four frequency bands: α, ß, θ, and δ. A mutual information (MI) technique selects the most descriptive features for further classification. The reduction in the number of prominent features improves the sleep-onset classification speed in the support vector machine (SVM) and results in a high sleep-onset recognition rate. Test results reveal that the combined use of the EEG and respiration signals results in 98.6% recognition accuracy. Our proposed application explores the possibility of processing long-term multi-channel signals.