Association Between Sleep Quality and Deep Learning-Based Sleep Onset Latency Distribution Using an Electroencephalogram.

To evaluate sleep quality, it is necessary to monitor overnight sleep duration. However, sleep monitoring typically requires more than 7 hours, which can be inefficient in termxs of data size and analysis. Therefore, we proposed to develop a deep learning-based model using a 30 sec sleep electroencephalogram (EEG) early in the sleep cycle to predict sleep onset latency (SOL) distribution and explore associations with sleep quality (SQ). We propose a deep learning model composed of a structure that decomposes and restores the signal in epoch units and a structure that predicts the SOL distribution. We used the Sleep Heart Health Study public dataset, which includes a large number of study subjects, to estimate and evaluate the proposed model. The proposed model estimated the SOL distribution and divided it into four clusters. The advantage of the proposed model is that it shows the process of falling asleep for individual participants as a probability graph over time. Furthermore, we compared the baseline of good SQ and SOL and showed that less than 10 minutes SOL correlated better with good SQ. Moreover, it was the most suitable sleep feature that could be predicted using early EEG, compared with the total sleep time, sleep efficiency, and actual sleep time. Our study showed the feasibility of estimating SOL distribution using deep learning with an early EEG and showed that SOL distribution within 10 minutes was associated with good SQ.

Ensuring spatial scalability with temporal-wise spatial attentive pooling for temporal action detection.

Recent temporal action detection models have focused on end-to-end trainable approaches to utilize the representational power of backbone networks. Despite the advantages of end-to-end trainable methods, these models still employ a small spatial resolution (e.g., 96 × 96) due to the inefficient trade-off between computational cost and spatial resolution. In this study, we argue that a simple pooling method (e.g., adaptive average pooling) acts as a bottleneck at the spatial aggregation part, restricting representational power. To address this issue, we propose a temporal-wise spatial attentive pooling (TSAP), which alleviates the bottleneck between the backbone and the detection head using a temporal-wise attention mechanism. Our approach mitigates the inefficient trade-off between spatial resolution and computational cost, thereby enhancing spatial scalability in temporal action detection. Moreover, TSAP is adaptable to previous end-to-end approaches by simply replacing the spatial pooling part. Our experiments demonstrated the essential role of spatial aggregation, and consistent improvements are observed by incorporating TSAP into previous end-to-end methods.

MEDi-SOL: Multi Ensemble Distribution Model for Estimating Sleep Onset Latency.

Sleep onset latency (SOL) is an important factor relating to the sleep quality of a subject. Therefore, accurate prediction of SOL is useful to identify individuals at risk of sleep disorders and to improve sleep quality. In this study, we estimate SOL distribution and falling asleep function using an electroencephalogram (EEG), which can measure the electric field of brain activity. We proposed a Multi Ensemble Distribution model for estimating Sleep Onset Latency (MEDi-SOL), consisting of a temporal encoder and a time distribution decoder. We evaluated the performance of the proposed model using a public dataset from the Sleep Heart Health Study. We considered four distributions, Normal, log-Normal, Weibull, and log-Logistic, and compared them with a survival model and a regression model. The temporal encoder with the ensemble log-Logistic and log-Normal distribution showed the best and second-best scores in the concordance index (C-index) and mean absolute error (MAE). Our MEDi-SOL, multi ensemble distribution with combining log-Logistic and log-Normal distribution, shows the best score in C-index and MAE, with a fast training time. Furthermore, our model can visualize the process of falling asleep for individual subjects. As a result, a distribution-based ensemble approach with appropriate distribution is more useful than point estimation.

Geometry-driven self-supervision for 3D human pose estimation.

Although 3D human pose estimation has recently made strides, it is still difficult to precisely recreate a 3D human posture from a single image without the aid of 3D annotation for the following reasons. Firstly, the process of reconstruction inherently suffers from ambiguity, as multiple 3D poses can be projected onto the same 2D pose. Secondly, accurately measuring camera rotation without laborious camera calibration is a difficult task. While some approaches attempt to address these issues using traditional computer vision algorithms, they are not differentiable and cannot be optimized through training. This paper introduces two modules that explicitly leverage geometry to overcome these challenges, without requiring any 3D ground-truth or camera parameters. The first module, known as the relative depth estimation module, effectively mitigates depth ambiguity by narrowing down the possible depths for each joint to only two candidates. The second module, referred to as the differentiable pose alignment module, calculates camera rotation by aligning poses from different views. The use of these geometrically interpretable modules reduces the complexity of training and yields superior performance. By adopting our proposed method, we achieve state-of-the-art results on standard benchmark datasets, surpassing other self-supervised methods and even outperforming several fully-supervised approaches that heavily rely on 3D annotations.

DeepHealthNet: Adolescent Obesity Prediction System Based on a Deep Learning Framework.

The global prevalence of childhood and adolescent obesity is a major concern due to its association with chronic diseases and long-term health risks. Artificial intelligence technology has been identified as a potential solution to accurately predict obesity rates and provide personalized feedback to adolescents. This study highlights the importance of early identification and prevention of obesity-related health issues. To develop effective algorithms for the prediction of obesity rates and provide personalized feedback, factors such as height, weight, waist circumference, calorie intake, physical activity levels, and other relevant health information must be taken into account. Therefore, by collecting health datasets from 321 adolescents who participated in Would You Do It! application, we proposed an adolescent obesity prediction system that provides personalized predictions and assists individuals in making informed health decisions. Our proposed deep learning framework, DeepHealthNet, effectively trains the model using data augmentation techniques, even when daily health data are limited, resulting in improved prediction accuracy (acc: 0.8842). Additionally, the study revealed variations in the prediction of the obesity rate between boys (acc: 0.9320) and girls (acc: 0.9163), allowing the identification of disparities and the determination of the optimal time to provide feedback. Statistical analysis revealed that the performance of the proposed deep learning framework was more statistically significant (p 0.001) compared to the other general models. The proposed system has the potential to effectively address childhood and adolescent obesity.

Identification of an effective and safe bolus dose and lockout time for patient-controlled sedation (PCS) using dexmedetomidine in dental treatments: a randomized clinical trial.

Background: This study investigated a safe and effective bolus dose and lockout time for patient-controlled sedation (PCS) with dexmedetomidine for dental treatments. The depth of sedation, vital signs, and patient satisfaction were investigated to demonstrate safety. Methods: Thirty patients requiring dental scaling were enrolled and randomly divided into three groups based on bolus doses and lockout times: group 1 (low dose group, bolus dose 0.05 µg/kg, 1-minute lockout time), group 2 (middle dose group, 0.1 µg/kg, 1-minute), and group 3 (high dose group, 0.2 µg/kg, 3-minute) (n = 10 each). ECG, pulse, oxygen saturation, blood pressure, end-tidal CO2, respiratory rate, and bispectral index scores (BIS) were measured and recorded. The study was conducted in two stages: the first involved sedation without dental treatment and the second included sedation with dental scaling. Patients were instructed to press the drug demand button every 10 s, and the process of falling asleep and waking up was repeated 1-5 times. In the second stage, during dental scaling, patients were instructed to press the drug demand button. Loss of responsiveness (LOR) was defined as failure to respond to auditory stimuli six times, determining sleep onset. Patient and dentist satisfaction were assessed before and after experimentation. Results: Thirty patients (22 males) participated in the study. Scaling was performed in 29 patients after excluding one who experienced dizziness during the first stage. The average number of drug administrations until first LOR was significantly lower in group 3 (2.8 times) than groups 1 and 2 (8.0 and 6.5 times, respectively). The time taken to reach the LOR showed no difference between groups. During the second stage, the average time required to reach the LOR during scaling was 583.4 seconds. The effect site concentrations (Ce) was significantly lower in group 1 than groups 2 and 3. In the participant survey on PCS, 8/10 in group 3 reported partial memory loss, whereas 17/20 in groups 1 and 2 recalled the procedure fully or partially. Conclusion: PCS with dexmedetomidine can provide a rapid onset of sedation, safe vital sign management, and minimal side effects, thus facilitating smooth dental sedation.

Toward Domain-Free Transformer for Generalized EEG Pre-Training.

Electroencephalography (EEG) signals are the brain signals acquired using the non-invasive approach. Owing to the high portability and practicality, EEG signals have found extensive application in monitoring human physiological states across various domains. In recent years, deep learning methodologies have been explored to decode the intricate information embedded in EEG signals. However, since EEG signals are acquired from humans, it has issues with acquiring enormous amounts of data for training the deep learning models. Therefore, previous research has attempted to develop pre-trained models that could show significant performance improvement through fine-tuning when data are scarce. Nonetheless, existing pre-trained models often struggle with constraints, such as the necessity to operate within datasets of identical configurations or the need to distort the original data to apply the pre-trained model. In this paper, we proposed the domain-free transformer, called DFformer, for generalizing the EEG pre-trained model. In addition, we presented the pre-trained model based on DFformer, which is capable of seamless integration across diverse datasets without necessitating architectural modification or data distortion. The proposed model achieved competitive performance across motor imagery and sleep stage classification datasets. Notably, even when fine-tuned on datasets distinct from the pre-training phase, DFformer demonstrated marked performance enhancements. Hence, we demonstrate the potential of DFformer to overcome the conventional limitations in pre-trained model development, offering robust applicability across a spectrum of domains.

Masked Kinematic Continuity-aware Hierarchical Attention Network for pose estimation in videos.

Existing methods for estimating human poses from video content exploit the temporal features of the video sequences and have shown impressive results. However, most methods address spatiotemporal issues separately. They compromise on accuracy to reduce jitter, or require high-resolution images to deal with occlusion, preventing full consideration of temporal features. Unfortunately, these two issues are interrelated. For example, occlusion causes uncertainty between successive frames, leading to unsmoothed results. To address these issues, we propose the Masked Kinematic Continuity-aware Hierarchical Attention Network (M-HANet) as a novel framework that exploits masked kinematic keypoint features by extending our framework HANet framework. First, we randomly select and mask a keypoint to treat the masked keypoint as it is occluded, which allows us to make the network resilient to occlusion. We also use the velocity and acceleration of each individual keypoint to effectively capture temporal features. Second, the proposed hierarchical transformer encoder refines a 2D or 3D input pose derived from existing estimators by aggregating the masked continuity of the spatiotemporal dependencies of human motion. Finally, to facilitate collaborative optimization, we perform an online cross-supervision between the final pose from our decoder and the refined input pose produced by our encoder. We validate the effectiveness of our model demonstrating that our proposed approach improves PCK@0.05 by 14.1% and MPJPE by 8.7 mm compared to the existing method on a variety of tasks, including 2D and 3D pose estimation, body mesh recovery, and sparsely annotated multi-human pose estimation.

Neurophysiological Response Based on Auditory Sense for Brain Modulation Using Monaural Beat.

Brain modulation is a modification process of brain activity through external stimulations. However, which condition can induce the activation is still unclear. Therefore, we aimed to identify brain activation conditions using 40 Hz monaural beat (MB). Under this stimulation, auditory sense status which is determined by frequency and power range is the condition to consider. Hence, we designed five sessions to compare; no stimulation, audible (AB), inaudible in frequency, inaudible in power, and inaudible in frequency and power. Ten healthy participants underwent each stimulation session for ten minutes with electroencephalogram (EEG) recording. For analysis, we calculated the power spectral density (PSD) of EEG for each session and compared them in frequency, time, and five brain regions. As a result, we observed the prominent power peak at 40 Hz in only AB. The induced EEG amplitude increase started at one minute and increased until the end of the session. These results of AB had significant differences in frontal, central, temporal, parietal, and occipital regions compared to other stimulations. From the statistical analysis, the PSD of the right temporal region was significantly higher than the left. We figure out the role that the auditory sense is important to lead brain activation. These findings help to understand the neurophysiological principle and effects of auditory stimulation.

SeriesSleepNet: an EEG time series model with partial data augmentation for automatic sleep stage scoring.

Introduction: We propose an automatic sleep stage scoring model, referred to as SeriesSleepNet, based on convolutional neural network (CNN) and bidirectional long short-term memory (bi-LSTM) with partial data augmentation. We used single-channel raw electroencephalography signals for automatic sleep stage scoring. Methods: Our framework was focused on time series information, so we applied partial data augmentation to learn the connected time information in small series. In specific, the CNN module learns the time information of one epoch (intra-epoch) whereas the bi-LSTM trains the sequential information between the adjacent epochs (inter-epoch). Note that the input of the bi-LSTM is the augmented CNN output. Moreover, the proposed loss function was used to fine-tune the model by providing additional weights. To validate the proposed framework, we conducted two experiments using the Sleep-EDF and SHHS datasets. Results and Discussion: The results achieved an overall accuracy of 0.87 and 0.84 and overall F1-score of 0.80 and 0.78 and kappa value of 0.81 and 0.78 for five-class classification, respectively. We showed that the SeriesSleepNet was superior to the baselines based on each component in the proposed framework. Our architecture also outperformed the state-of-the-art methods with overall F1-score, accuracy, and kappa value. Our framework could provide information on sleep disorders or quality of sleep to automatically classify sleep stages with high performance.

Learning Spatiotemporal Graph Representations for Visual Perception Using EEG Signals.

Perceiving and recognizing objects enable interaction with the external environment. Recently, decoding brain signals based on brain-computer interface (BCI) that recognize the user's intentions by just looking at objects has attracted attention as a next-generation intuitive interface. However, classifying signals from different objects is very challenging, and in practice, decoding performance for visual perception is not yet high enough to be used in real environments. In this study, we aimed to classify single-trial electroencephalography signals evoked by visual stimuli into their corresponding semantic category. We proposed a two-stream convolutional neural network to increase classification performance. The model consists of a spatial stream and a temporal stream that use graph convolutional neural network and channel-wise convolutional neural network respectively. Two public datasets were used to evaluate the proposed model; (i) SU DB (a set of 72 photographs of objects belonging to 6 semantic categories) and MPI DB (8 exemplars belonging to two categories). Our results outperform state-of-the-art methods, with accuracies of 54.28 ± 7.89% for SU DB (6-class) and 84.40 ± 8.03% for MPI DB (2-class). These results could facilitate the application of intuitive BCI systems based on visual perception.

Real-Time Deep Neurolinguistic Learning Enhances Noninvasive Neural Language Decoding for Brain-Machine Interaction.

Electroencephalogram (EEG)-based brain-machine interface (BMI) has been utilized to help patients regain motor function and has recently been validated for its use in healthy people because of its ability to directly decipher human intentions. In particular, neurolinguistic research using EEGs has been investigated as an intuitive and naturalistic communication tool between humans and machines. In this study, the human mind directly decoded the neural languages based on speech imagery using the proposed deep neurolinguistic learning. Through real-time experiments, we evaluated whether BMI-based cooperative tasks between multiple users could be accomplished using a variety of neural languages. We successfully demonstrated a BMI system that allows a variety of scenarios, such as essential activity, collaborative play, and emotional interaction. This outcome presents a novel BMI frontier that can interact at the level of human-like intelligence in real time and extends the boundaries of the communication paradigm.

Decoding declarative memory process for predicting memory retrieval based on source localization.

Many studies have focused on understanding memory processes due to their importance in daily life. Differences in timing and power spectra of brain signals during encoding task have been linked to later remembered items and were recently used to predict memory retrieval performance. However, accuracies remain low when using non-invasive methods for acquiring brain signals, mainly due to the low spatial resolution. This study investigates the prediction of successful retrieval using estimated source activity corresponding either to cortical or subcortical structures through source localization. Electroencephalogram (EEG) signals were recorded while participants performed a declarative memory task. Frequency-time analysis was performed using signals from encoding and retrieval tasks to confirm the importance of neural oscillations and their relationship with later remembered and forgotten items. Significant differences in the power spectra between later remembered and forgotten items were found before and during the presentation of the stimulus in the encoding task. Source activity estimation revealed differences in the beta band power over the medial parietal and medial prefrontal areas prior to the presentation of the stimulus, and over the cuneus and lingual areas during the presentation of the stimulus. Additionally, there were significant differences during the stimuli presentation during the retrieval task. Prediction of later remembered items was performed using surface potentials and estimated source activity. The results showed that source localization increases classification performance compared to the one using surface potentials. These findings support the importance of incorporating spatial features of neural activity to improve the prediction of memory retrieval.

Attentional feature pyramid network for small object detection.

Recent state-of-the-art detectors generally exploit the Feature Pyramid Networks (FPN) due to its advantage of detecting objects at different scales. Despite significant advances in object detection owing to the design of feature pyramids, it is still challenging to detect small objects with low resolution and dense distribution in complex scenes. To address these problems, we propose Attentional Feature Pyramid Network, a new feature pyramid architecture named AFPN which consists of three components to enhance the small object detection ability, specifically: Dynamic Texture Attention, Foreground-Aware Co-Attention, and Detail Context Attention. First, Dynamic Texture Attention augments the texture features dynamically by filtering out redundant semantics to highlight small objects in lower layers and amplifying credible details to emphasize large objects in higher layers. Then, Foreground-Aware Co-Attention is explored to detect densely arranged small objects by enhancing the objects feature via foreground-correlated contexts and suppressing the background noise. Finally, to better capture the features of small objects, Detail Context Attention adaptively aggregates detail cues of RoI features with different scales for a more accurate feature representation. By substituting FPN with AFPN in Faster R-CNN, our method performs on par with the state-of-the-art performance on Tsinghua-Tencent 100K. Furthermore, we achieve highly competitive results on small category of both PASCAL VOC and MS COCO.

Prototype-based Domain Generalization Framework for Subject-Independent Brain-Computer Interfaces.

Brain-computer interface (BCI) is challenging to use in practice due to the inter/intra-subject variability of electroencephalography (EEG). The BCI system, in general, necessitates a calibration technique to obtain subject/session-specific data in order to tune the model each time the system is utilized. This issue is acknowledged as a key hindrance to BCI, and a new strategy based on domain generalization has recently evolved to address it. In light of this, we've concentrated on developing an EEG classification framework that can be applied directly to data from unknown domains (i.e. subjects), using only data acquired from separate subjects previously. For this purpose, in this paper, we proposed a framework that employs the open-set recognition technique as an auxiliary task to learn subject-specific style features from the source dataset while helping the shared feature extractor with mapping the features of the unseen target dataset as a new unseen domain. Our aim is to impose cross-instance style in-variance in the same domain and reduce the open space risk on the potential unseen subject in order to improve the generalization ability of the shared feature extractor. Our experiments showed that using the domain information as an auxiliary network increases the generalization performance. Clinical relevance-This study suggests a strategy to improve the performance of the subject-independent BCI systems. Our framework can help to reduce the need for further calibration and can be utilized for a range of mental state monitoring tasks (e.g. neurofeedback, identification of epileptic seizures, and sleep disorders).

Decoding Neural Correlation of Language-Specific Imagined Speech using EEG Signals.

Speech impairments due to cerebral lesions and degenerative disorders can be devastating. For humans with severe speech deficits, imagined speech in the brain-computer interface has been a promising hope for reconstructing the neural signals of speech production. However, studies in the EEG-based imagined speech domain still have some limitations due to high variability in spatial and temporal information and low signal-to-noise ratio. In this paper, we investigated the neural signals for two groups of native speakers with two tasks with different languages, English and Chinese. Our assumption was that English, a non-tonal and phonogram-based language, would have spectral differences in neural computation compared to Chinese, a tonal and ideogram-based language. The results showed the significant difference in the relative power spectral density between English and Chinese in specific frequency band groups. Also, the spatial evaluation of Chinese native speakers in the theta band was distinctive during the imagination task. Hence, this paper would suggest the key spectral and spatial information of word imagination with specialized language while decoding the neural signals of speech. Clinical Relevance- Imagined speech-related studies lead to the development of assistive communication technology especially for patients with speech disorders such as aphasia due to brain damage. This study suggests significant spectral features by analyzing cross-language differences of EEG-based imagined speech using two widely used languages.

2020 International brain-computer interface competition: A review.

The brain-computer interface (BCI) has been investigated as a form of communication tool between the brain and external devices. BCIs have been extended beyond communication and control over the years. The 2020 international BCI competition aimed to provide high-quality neuroscientific data for open access that could be used to evaluate the current degree of technical advances in BCI. Although there are a variety of remaining challenges for future BCI advances, we discuss some of more recent application directions: (i) few-shot EEG learning, (ii) micro-sleep detection (iii) imagined speech decoding, (iv) cross-session classification, and (v) EEG(+ear-EEG) detection in an ambulatory environment. Not only did scientists from the BCI field compete, but scholars with a broad variety of backgrounds and nationalities participated in the competition to address these challenges. Each dataset was prepared and separated into three data that were released to the competitors in the form of training and validation sets followed by a test set. Remarkable BCI advances were identified through the 2020 competition and indicated some trends of interest to BCI researchers.

SASDL and RBATQ: Sparse Autoencoder With Swarm Based Deep Learning and Reinforcement Based Q-Learning for EEG Classification.

The most vital information about the electrical activities of the brain can be obtained with the help of Electroencephalography (EEG) signals. It is quite a powerful tool to analyze the neural activities of the brain and various neurological disorders like epilepsy, schizophrenia, sleep related disorders, parkinson disease etc. can be investigated well with the help of EEG signals. Goal: In this paper, two versatile deep learning methods are proposed for the efficient classification of epilepsy and schizophrenia from EEG datasets. Methods: The main advantage of using deep learning when compared to other machine learning algorithms is that it has the capability to accomplish feature engineering on its own. Swarm intelligence is also a highly useful technique to solve a wide range of real-world, complex, and non-linear problems. Therefore, taking advantage of these factors, the first method proposed is a Sparse Autoencoder (SAE) with swarm based deep learning method and it is named as (SASDL) using Particle Swarm Optimization (PSO) technique, Cuckoo Search Optimization (CSO) technique and Bat Algorithm (BA) technique; and the second technique proposed is the Reinforcement Learning based on Bidirectional Long-Short Term Memory (BiLSTM), Attention Mechanism, Tree LSTM and Q learning, and it is named as (RBATQ) technique. Results and Conclusions: Both these two novel deep learning techniques are tested on epilepsy and schizophrenia EEG datasets and the results are analyzed comprehensively, and a good classification accuracy of more than 93% is obtained for all the datasets.

Uncertainty-Aware Portfolio Management With Risk-Sensitive Multiagent Network.

As deep neural networks (DNNs) have gained considerable attention in recent years, there have been several cases applying DNNs to portfolio management (PM). Although some researchers have experimentally demonstrated its ability to make a profit, it is still insufficient to use in real situations because existing studies have failed to answer how risky investment decisions are. Furthermore, even though the objective of PM is to maximize returns within a risk tolerance, they overlook the predictive uncertainty of DNNs in the process of risk management. To overcome these limitations, we propose a novel framework called risk-sensitive multiagent network (RSMAN), which includes risk-sensitive agents (RSAs) and a risk adaptive portfolio generator (RAPG). Standard DNNs do not understand the risks of their decision, whereas RSA can take risk-sensitive decisions by estimating market uncertainty and parameter uncertainty. Acting as a trader, this agent is trained via reinforcement learning from dynamic trading simulations to estimate the distribution of reward and via unsupervised learning to assess parameter uncertainty without labeled data. We also present an RAPG that can generate a portfolio fitting the user's risk appetite without retraining by exploiting the estimated information from the RSAs. We tested our framework on the U.S. and Korean real financial markets to demonstrate the practicality of the RSMAN.

Motor Impairment in Stroke Patients Is Associated With Network Properties During Consecutive Motor Imagery.

OBJECTIVE: Our study aimed to predict the Fugl-Meyer assessment (FMA) upper limb using network properties during motor imagery using electroencephalography (EEG) signals. METHODS: The subjects performed a finger tapping imagery task according to consecutive cues. We measured the weighted phase lag index (wPLI) as functional connectivity and directed transfer function (DTF) as causal connectivity in healthy controls and stroke patients. The network properties based on the wPLI and DTF were calculated. We predicted the FMA upper limb using partial least squares regression. RESULTS: A higher DTF in the mu band was observed in stroke patients than in healthy controls. Notably, the difference in local properties at node F3 was negatively correlated with motor impairment in stroke patients. Finally, using significant network properties based on the wPLI and DTF, we predicted motor impairments using the FMA upper limb with a root-mean-square error of 1.68 ( R2 = 0.97). This outperformed the state-of-the-art predictors. CONCLUSION: These findings demonstrate that network properties based on functional and causal connectivity were highly associated with motor function in stroke patients. SIGNIFICANCE: Our network properties can help calculate the predictor of motor impairments in stroke rehabilitation and provide insight into the neural correlates related to motor function based on EEG after reorganization induced by stroke.