Pilot study of using transcranial temporal interfering theta-burst stimulation for modulating motor excitability in rat.

Transcranial temporal interference stimulation (tTIS) is a promising brain stimulation method that can target deep brain regions by delivering an interfering current from surface electrodes. Most instances of tTIS stimulate the brain with a single-frequency sinusoidal waveform generated by wave interference. Theta burst stimulation is an effective stimulation scheme that can modulate neuroplasticity by generating long-term potentiation- or depression-like effects. To broaden tTIS application, we developed a theta burst protocol using tTIS technique to modulate neuroplasticity in rats. Two cannula electrodes were unilaterally implanted into the intact skull over the primary motor cortex. Electrical field of temporal interference envelopes generated by tTIS through cannula electrodes were recorded from primary motor cortex. Theta burst schemes were characterized, and motor activation induced by the stimulation was also evaluated simultaneously by observing electromyographic signals from the corresponding brachioradialis muscle. After validating the stimulation scheme, we further tested the modulatory effects of theta burst stimulation delivered by tTIS and by conventional transcranial electrical stimulation on primary motor cortex excitability. Changes in the amplitude of motor evoked potentials, elicited when the primary motor cortex was activated by electrical pulses, were measured before and after theta burst stimulation by both techniques. Significant potentiation and suppression were found at 15 to 30 min after the intermittent and continuous theta burst stimulation delivered using tTIS, respectively. However, comparing to theta burst stimulations delivered using conventional form of transcranial electrical stimulation, using tTIS expressed no significant difference in modulating motor evoked potential amplitudes. Sham treatment from both methods had no effect on changing the motor evoked potential amplitude. The present study demonstrated the feasibility of using tTIS to achieve a theta burst stimulation scheme for motor cortical neuromodulation. These findings also indicated the future potential of using tTIS to carry out theta burst stimulation protocols in deep-brain networks for modulating neuroplasticity.

A time-course study of urodynamic analyses in rat models with dopaminergic depletion induced through unilateral and bilateral 6-hydroxydopamine injections.

BACKGROUND: Bladder dysfunction is a common non-motor disorder in Parkinson's disease (PD). This study attempted to determine the bladder dysfunction with disease progression in the PD rat model produced from unilateral/bilateral injections of 6-hydroxydopamine (6-OHDA). METHODS: Cystometrographic (CMG) and external urethral sphincter electromyographic (EUS-EMG) measurements were scheduled in a time-course manner to determine the disease timing, onset, and severity. Animals were allotted into normal control, unilateral, bilateral 6-OHDA injected groups and subjected to scheduled CMG, EUS-EMG analyses at weeks 1, 2, and 4. RESULTS: The urodynamic results concluded that voiding efficiency (VE) was reduced in both unilateral and bilateral PD rats at all-time points. VE had decreased from 57 ± 11% to 31 ± 7% in unilateral PD rats and in bilateral PD rats, a decreased VE of 20 ± 6% was observed compared to control and unilateral PD rats. The EMG results in unilateral PD rats indicated declines in bursting period (BP) (3.78-2.94 s), active period (AP) (93.38-88.75 ms), and silent period (SP) (161.62-114.30 ms). A sudden reduction was noticed in BP (3.62-2.82 s), AP (92.21-86.01 ms), and SP (128.61-60.16 ms) of bilateral PD rats than in control and unilateral PD rats. Histological evidence exhibited a progressive dopaminergic neurons (DA) depletion in the substantia nigra (SN) region in 6-OHDA lesioned rats. CONCLUSION: The experimental outcomes strongly implied that significant variations in bladder function and VE decline were due to the depletion of DA neurons in the SN region of the brain.

Effects of paired stimulation with specific waveforms on cortical and spinal plasticity in subjects with a chronic spinal cord injury.

BACKGROUND/PURPOSE: Paired stimulation can cause neuroplasticity in corticospinal and spinal pathways in subjects with a chronic spinal cord injury (SCI). We aimed to know the effects of different waveforms using paired stimulations with bicycling in subjects with a chronic SCI. METHODS: Recruited subjects with an SCI underwent three treatment interventions in random order for 4-20 min followed by 30 min of bicycling (control, repetitive transcranial magnetic stimulation (TMS; rTMS) at 20 Hz with transspinal direct current stimulation (tsDCS), and intermittent theta burst stimulation (iTBS) with tsDCS with a 1-week gap period. A TMS method was employed to record the resting motor threshold (RMT), the 90% values of which was used as the stimulation intensity, and the Hoffman (H)-reflex was measured by stimulating the tibial nerve in the popliteal fossa. The RMT, motor evoked potential (MEP) latency, MEP peak-to-peak amplitude, and H-reflex latency as primary variables and lower extremity motor scale (LEMS) and modified Ashworth spasticity scale (MAS) as secondary variables were analyzed before and after the interventions. RESULTS: The MEP latency, MEP amplitude, and LEMS significantly improved with the rTMS-iTBS/tsDCS or the rTMS-20 Hz/tsDCS (p < 0.050) protocols compared to the control intervention. All other outcome measures, including RMT, H-reflex latency, and MAS score showed some changes but did not fully attain a level of significance. CONCLUSION: The paired stimulation with rTMS-iTBS/tsDCS was equally effective to produce neuroplastic effect in subjects with chronic SCI compared to the conventional TMS-20 Hz/tsDCS intervention.

Assessment System for Predicting Maximal Safe Range for Heel Height by Using Force-Sensing Resistor Sensors and Regression Models.

Women often wear high-heeled shoes for professional or esthetic reasons. However, high-heeled shoes can cause discomfort and injury and can change the body's center of gravity when maintaining balance. This study developed an assessment system for predicting the maximal safe range for heel height by recording the plantar pressure of participants' feet by using force-sensing resistor (FSR) sensors and conducting analyses using regression models. Specifically, 100 young healthy women stood on an adjustable platform while physicians estimated the maximal safe height of high-heeled shoes. The collected FSR data combined with and without personal features were analyzed using regression models. The experimental results showed that the regression model based on the pressure data for the right foot had better predictive power than that based on data for the left foot, regardless of the module. The model with two heights had higher predictive power than that with a single height. Furthermore, adding personal features under the condition of two heights afforded the best predictive effect. These results can help wearers choose maximal safe high-heeled shoes to reduce injuries to the bones and lower limbs.

Design of a Multi-Sensor System for Exploring the Relation between Finger Spasticity and Voluntary Movement in Patients with Stroke.

A novel wearable multi-sensor data glove system is developed to explore the relation between finger spasticity and voluntary movement in patients with stroke. Many stroke patients suffer from finger spasticity, which is detrimental to their manual dexterity. Diagnosing and assessing the degrees of spasticity require neurological testing performed by trained professionals to estimate finger spasticity scores via the modified Ashworth scale (MAS). The proposed system offers an objective, quantitative solution to assess the finger spasticity of patients with stroke and complements the manual neurological test. In this work, the hardware and software components of this system are described. By requiring patients to perform five designated tasks, biomechanical measurements including linear and angular speed, acceleration, and pressure at every finger joint and upper limb are recorded, making up more than 1000 features for each task. We conducted a preliminary clinical test with 14 subjects using this system. Statistical analysis is performed on the acquired measurements to identify a small subset of features that are most likely to discriminate a healthy patient from patients suffering from finger spasticity. This encouraging result validates the feasibility of this proposed system to quantitatively and objectively assess finger spasticity.

Safety of Special Waveform of Transcranial Electrical Stimulation (TES): In Vivo Assessment.

Intermittent theta burst (iTBS) powered by direct current stimulation (DCS) can safely be applied transcranially to induce neuroplasticity in the human and animal brain cortex. tDCS-iTBS is a special waveform that is used by very few studies, and its safety needs to be confirmed. Therefore, we aimed to evaluate the safety of tDCS-iTBS in an animal model after brain stimulations for 1 h and 4 weeks. Thirty-one Sprague Dawley rats were divided into two groups: (1) short-term stimulation for 1 h/session (sham, low, and high) and (2) long-term for 30 min, 3 sessions/week for 4 weeks (sham and high). The anodal stimulation applied over the primary motor cortex ranged from 2.5 to 4.5 mA/cm2. The brain biomarkers and scalp tissues were assessed using ELISA and histological analysis (H&E staining) after stimulations. The caspase-3 activity, cortical myelin basic protein (MBP) expression, and cortical interleukin (IL-6) levels increased slightly in both groups compared to sham. The serum MBP, cortical neuron-specific enolase (NSE), and serum IL-6 slightly changed from sham after stimulations. There was no obvious edema or cell necrosis seen in cortical histology after the intervention. The short- and long-term stimulations did not induce significant adverse effects on brain and scalp tissues upon assessing biomarkers and conducting histological analysis.

Motor Neuroplastic Effects of a Novel Paired Stimulation Technology in an Incomplete Spinal Cord Injury Animal Model.

Paired stimulation of the brain and spinal cord can remodel the central nervous tissue circuitry in an animal model to induce motor neuroplasticity. The effects of simultaneous stimulation vary according to the extent and severity of spinal cord injury. Therefore, our study aimed to determine the significant effects on an incomplete SCI rat brain and spinal cord through 3 min and 20 min stimulations after 4 weeks of intervention. Thirty-three Sprague Dawley rats were classified into six groups: (1) normal, (2) sham, (3) iTBS/tsDCS, (4) iTBS/ts-iTBS, (5) rTMS/tsDCS, and (6) rTMS/ts-iTBS. Paired stimulation of the brain cortex and spinal cord thoracic (T10) level was applied simultaneously for 3−20 min. The motor evoked potential (MEP) and Basso, Beattie, and Bresnahan (BBB) scores were recorded after every week of intervention for four weeks along with wheel training for 20 min. Three-minute stimulation with the iTBS/tsDCS intervention induced a significant (p < 0.050 *) increase in MEP after week 2 and week 4 treatments, while 3 min iTBS/ts-iTBS significantly improved MEP (p < 0.050 *) only after the week 3 intervention. The 20 min rTMS/ts-iTBS intervention showed a significant change only in post_5 min after week 4. The BBB score also changed significantly in all groups except for the 20 min rTMS/tsDCS intervention. iTBS/tsDCS and rTMS/ts-iTBS interventions induce neuroplasticity in an incomplete SCI animal model by significantly changing electrophysiological (MEP) and locomotion (BBB) outcomes.

Hemodynamics and Tissue Optical Properties in Bimodal Infarctions Induced by Middle Cerebral Artery Occlusion.

Various infarct sizes induced by middle cerebral artery occlusion (MCAO) generate inconsistent outcomes for stroke preclinical study. Monitoring cerebral hemodynamics may help to verify the outcome of MCAO. The aim of this study was to investigate the changes in brain tissue optical properties by frequency-domain near-infrared spectroscopy (FD-NIRS), and establish the relationship between cerebral hemodynamics and infarct variation in MCAO model. The rats were undergone transient MCAO using intraluminal filament. The optical properties and hemodynamics were measured by placing the FD-NIRS probes on the scalp of the head before, during, and at various time-courses after MCAO. Bimodal infarction severities were observed after the same 90-min MCAO condition. Significant decreases in concentrations of oxygenated hemoglobin ([HbO]) and total hemoglobin ([HbT]), tissue oxygenation saturation (StO2), absorption coefficient (µa) at 830 nm, and reduced scattering coefficient (µs') at both 690 and 830 nm were detected during the occlusion in the severe infarction but not the mild one. Of note, the significant increases in [HbO], [HbT], StO2, and µa at both 690 and 830 nm were found on day 3; and increases in µs' at both 690 and 830 nm were found on day 2 and day 3 after MCAO, respectively. The interhemispheric correlation coefficient (IHCC) was computed from low-frequency hemodynamic oscillation of both hemispheres. Lower IHCCs standing for interhemispheric desynchronizations were found in both mild and severe infarction during occlusion, and only in severe infarction after reperfusion. Our finding supports that sequential FD-NIRS parameters may associated with the severity of the infarction in MCAO model, and the consequent pathologies such as vascular dysfunction and brain edema. Further study is required to validate the potential use of FD-NIRS as a monitor for MCAO verification.

The impact of aging and reaching movements on grip stability control during manual precision tasks.

BACKGROUND: Operating an object by generating stable hand-grip force during static or dynamic posture control of the upper extremities simultaneously is an important daily activity. Older adults require different attentional resources during grip strength control and arm movements. However, the impact of aging and reaching movements on precise grip strength and stability control among older adults is not well understood. This study investigated the impact of aging and reaching movements on grip strength and stability control in both hands of the upper extremities. METHODS: Fifty healthy young adults (age: 28.8 ± 14.0 years) and 54 healthy older adults (73.6 ± 6.3 years) were recruited to perform isometric grip strength test at 20% maximal voluntary contraction as the target force during three manual precision tasks simultaneously: stationary task (without arm movements), forward-reach task, and backward-reach task. The average grip force (in kg) and coefficient of variation values (expressed as a percentage) during manual precision tasks were calculated to determine the quality of participants' grip strength. The deviation error, absolute error, and force-stability index values were calculated to determine the strength control relative to the target force. RESULTS: For both the young and older groups, the force-stability index values in both hands were significantly higher during forward- and backward-reaching movements than in the stationary condition (p < 0.05). The older group exhibited a significantly lower hand-grip strength and stability of strength control in both hands than the young group (p < 0.05). CONCLUSIONS: Aging and reaching task performance reduced the grip strength of participants and increased the variations in strength control of both hands relative to the target force, indicating that older adults exhibit poor grip strength and stability control when performing arm-reaching movements. These findings may help clinical therapists in establishing objective indexes for poor grip-stability control screening and developing appropriate rehabilitation programs or health-promotion exercises that can improve grip strength and stability control in older people.

Depth-Camera Based Energy Expenditure Estimation System for Physical Activity Using Posture Classification Algorithm.

Insufficient physical activity is common in modern society. By estimating the energy expenditure (EE) of different physical activities, people can develop suitable exercise plans to improve their lifestyle quality. However, several limitations still exist in the related works. Therefore, the aim of this study is to propose an accurate EE estimation model based on depth camera data with physical activity classification to solve the limitations in the previous research. To decide the best location and amount of cameras of the EE estimation, three depth cameras were set at three locations, namely the side, rear side, and rear views, to obtain the kinematic data and EE estimation. Support vector machine was used for physical activity classification. Three EE estimation models, namely linear regression, multilayer perceptron (MLP), and convolutional neural network (CNN) models, were compared and determined the model with optimal performance in different experimental settings. The results have shown that if only one depth camera is available, optimal EE estimation can be obtained using the side view and MLP model. The mean absolute error (MAE), mean square error (MSE), and root MSE (RMSE) of the classification results under the aforementioned settings were 0.55, 0.66, and 0.81, respectively. If higher accuracy is required, two depth cameras can be set at the side and rear views, the CNN model can be used for light-to-moderate activities, and the MLP model can be used for vigorous activities. The RMSEs for estimating the EEs of standing, walking, and running were 0.19, 0.57, and 0.96, respectively. By applying the different models on different amounts of cameras, the optimal performance can be obtained, and this is also the first study to discuss the issue.

Wearable Cardiopulmonary Function Evaluation System for Six-Minute Walking Test.

As a submaximal exercise test, a 6-min walking test (6MWT) can be considered a suitable index for the exercise capacity of patients with a respiratory problem. Traditionally, medical staff manually collect cardiopulmonary information using different devices. However, no integrated monitoring system is currently available to simultaneously record the real-time breathing sound, heart rhythm, and precise walking information (i.e., walking distance, speed, and acceleration) during the 6MWT. In this study, a wearable and wireless multiparameter monitoring system is proposed to simultaneously monitor the breathing sound, oxygen saturation (SpO2), electrocardiograph (ECG) signals, and precise walking information during the 6MWT. Here, a wearable mechanical design was successfully used to reduce the effect of motion artifacts on the breathing sound and ECG signal. A multiparameter detection algorithm was designed to effectively estimate heart and breathing rates. Finally, the cardiopulmonary function of smokers was evaluated using the proposed system. The evaluation indicated that this system could reveal dynamic changes and differences in the breathing rate, heart rate, SpO2, walking speed, and acceleration during the 6MWT. The proposed system can serve as a more integrated approach to monitor cardiopulmonary parameters and obtain precise walking information simultaneously during the 6MWT.

Design of a Wearable 12-Lead Noncontact Electrocardiogram Monitoring System.

A standard 12-lead electrocardiogram (ECG) is an important tool in the diagnosis of heart diseases. Here, Ag/AgCl electrodes with conductive gels are usually used in a 12-lead ECG system to access biopotentials. However, using Ag/AgCl electrodes with conductive gels might be inconvenient in a prehospital setting. In previous studies, several dry electrodes have been developed to improve this issue. However, these dry electrodes have contact with the skin directly, and they might be still unsuitable for patients with wounds. In this study, a wearable 12-lead electrocardiogram monitoring system was proposed to improve the above issue. Here, novel noncontact electrodes were also designed to access biopotentials without contact with the skin directly. Moreover, by using the mechanical design, this system allows the user to easily wear and take off the device and to adjust the locations of the noncontact electrodes. The experimental results showed that the proposed system could exactly provide a good ECG signal quality even while walking and could detect the ECG features of the patients with myocardial ischemia, installation pacemaker, and ventricular premature contraction.

Design of an Inertial-Sensor-Based Data Glove for Hand Function Evaluation.

Capturing hand motions for hand function evaluations is essential in the medical field. Various data gloves have been developed for rehabilitation and manual dexterity assessments. This study proposed a modular data glove with 9-axis inertial measurement units (IMUs) to obtain static and dynamic parameters during hand function evaluation. A sensor fusion algorithm is used to calculate the range of motion of joints. The data glove is designed to have low cost, easy wearability, and high reliability. Owing to the modular design, the IMU board is independent and extensible and can be used with various microcontrollers to realize more medical applications. This design greatly enhances the stability and maintainability of the glove.

Simple Smartphone-Based Guiding System for Visually Impaired People.

Visually impaired people are often unaware of dangers in front of them, even in familiar environments. Furthermore, in unfamiliar environments, such people require guidance to reduce the risk of colliding with obstacles. This study proposes a simple smartphone-based guiding system for solving the navigation problems for visually impaired people and achieving obstacle avoidance to enable visually impaired people to travel smoothly from a beginning point to a destination with greater awareness of their surroundings. In this study, a computer image recognition system and smartphone application were integrated to form a simple assisted guiding system. Two operating modes, online mode and offline mode, can be chosen depending on network availability. When the system begins to operate, the smartphone captures the scene in front of the user and sends the captured images to the backend server to be processed. The backend server uses the faster region convolutional neural network algorithm or the you only look once algorithm to recognize multiple obstacles in every image, and it subsequently sends the results back to the smartphone. The results of obstacle recognition in this study reached 60%, which is sufficient for assisting visually impaired people in realizing the types and locations of obstacles around them.

Design of Wearable Breathing Sound Monitoring System for Real-Time Wheeze Detection.

In the clinic, the wheezing sound is usually considered as an indicator symptom to reflect the degree of airway obstruction. The auscultation approach is the most common way to diagnose wheezing sounds, but it subjectively depends on the experience of the physician. Several previous studies attempted to extract the features of breathing sounds to detect wheezing sounds automatically. However, there is still a lack of suitable monitoring systems for real-time wheeze detection in daily life. In this study, a wearable and wireless breathing sound monitoring system for real-time wheeze detection was proposed. Moreover, a breathing sounds analysis algorithm was designed to continuously extract and analyze the features of breathing sounds to provide the objectively quantitative information of breathing sounds to professional physicians. Here, normalized spectral integration (NSI) was also designed and applied in wheeze detection. The proposed algorithm required only short-term data of breathing sounds and lower computational complexity to perform real-time wheeze detection, and is suitable to be implemented in a commercial portable device, which contains relatively low computing power and memory. From the experimental results, the proposed system could provide good performance on wheeze detection exactly and might be a useful assisting tool for analysis of breathing sounds in clinical diagnosis.

Development of a Wearable Motor-Imagery-Based Brain-Computer Interface.

A motor-imagery-based brain-computer interface (BCI) is a translator that converts the motor intention of the brain into a control command to control external machines without muscles. Numerous motor-imagery-based BCIs have been successfully proposed in previous studies. However, several electroencephalogram (EEG) channels are typically required for providing sufficient information to maintain a specific accuracy and bit rate, and the bulk volume of these EEG machines is also inconvenient. A wearable motor imagery-based BCI system was proposed and implemented in this study. A wearable mechanical design with novel active comb-shaped dry electrodes was developed to measure EEG signals without conductive gels at hair sites, which is easy and convenient for users wearing the EEG machine. In addition, a wireless EEG acquisition module was also designed to measure EEG signals, which provides a user with more freedom of motion. The proposed wearable motor-imagery-based BCI system was validated using an electrical specifications test and a hand motor imagery experiment. Experimental results showed that the proposed wearable motor-imagery-based BCI system provides favorable signal quality for measuring EEG signals and detecting motor imagery.

Design of a mobile brain computer interface-based smart multimedia controller.

Music is a way of expressing our feelings and emotions. Suitable music can positively affect people. However, current multimedia control methods, such as manual selection or automatic random mechanisms, which are now applied broadly in MP3 and CD players, cannot adaptively select suitable music according to the user's physiological state. In this study, a brain computer interface-based smart multimedia controller was proposed to select music in different situations according to the user's physiological state. Here, a commercial mobile tablet was used as the multimedia platform, and a wireless multi-channel electroencephalograph (EEG) acquisition module was designed for real-time EEG monitoring. A smart multimedia control program built in the multimedia platform was developed to analyze the user's EEG feature and select music according his/her state. The relationship between the user's state and music sorted by listener's preference was also examined in this study. The experimental results show that real-time music biofeedback according a user's EEG feature may positively improve the user's attention state.

Temporal and Spatial Denoising of Depth Maps.

This work presents a procedure for refining depth maps acquired using RGB-D (depth) cameras. With numerous new structured-light RGB-D cameras, acquiring high-resolution depth maps has become easy. However, there are problems such as undesired occlusion, inaccurate depth values, and temporal variation of pixel values when using these cameras. In this paper, a proposed method based on an exemplar-based inpainting method is proposed to remove artefacts in depth maps obtained using RGB-D cameras. Exemplar-based inpainting has been used to repair an object-removed image. The concept underlying this inpainting method is similar to that underlying the procedure for padding the occlusions in the depth data obtained using RGB-D cameras. Therefore, our proposed method enhances and modifies the inpainting method for application in and the refinement of RGB-D depth data image quality. For evaluating the experimental results of the proposed method, our proposed method was tested on the Tsukuba Stereo Dataset, which contains a 3D video with the ground truths of depth maps, occlusion maps, RGB images, the peak signal-to-noise ratio, and the computational time as the evaluation metrics. Moreover, a set of self-recorded RGB-D depth maps and their refined versions are presented to show the effectiveness of the proposed method.

Design and Implementation of a Set-Top Box-Based Homecare System Using Hybrid Cloud.

INTRODUCTION: Telemedicine has become a prevalent topic in recent years, and several telemedicine systems have been proposed; however, such systems are an unsuitable fit for the daily requirements of users. MATERIALS AND METHODS: The system proposed in this study was developed as a set-top box integrated with the Android™ (Google, Mountain View, CA) operating system to provide a convenient and user-friendly interface. The proposed system can assist with family healthcare management, telemedicine service delivery, and information exchange among hospitals. To manage the system, a novel type of hybrid cloud architecture was also developed. RESULTS: Updated information is stored on a public cloud, enabling medical staff members to rapidly access information when diagnosing patients. In the long term, the stored data can be reduced to improve the efficiency of the database. CONCLUSIONS: The proposed design offers a robust architecture for storing data in a homecare system and can thus resolve network overload and congestion resulting from accumulating data, which are inherent problems in centralized architectures, thereby improving system efficiency.

Multi-Modal Sleep Stage Classification With Two-Stream Encoder-Decoder.

Sleep staging serves as a fundamental assessment for sleep quality measurement and sleep disorder diagnosis. Although current deep learning approaches have successfully integrated multimodal sleep signals, enhancing the accuracy of automatic sleep staging, certain challenges remain, as follows: 1) optimizing the utilization of multi-modal information complementarity, 2) effectively extracting both long- and short-range temporal features of sleep information, and 3) addressing the class imbalance problem in sleep data. To address these challenges, this paper proposes a two-stream encode-decoder network, named TSEDSleepNet, which is inspired by the depth sensitive attention and automatic multi-modal fusion (DSA2F) framework. In TSEDSleepNet, a two-stream encoder is used to extract the multiscale features of electrooculogram (EOG) and electroencephalogram (EEG) signals. And a self-attention mechanism is utilized to fuse the multiscale features, generating multi-modal saliency features. Subsequently, the coarser-scale construction module (CSCM) is adopted to extract and construct multi-resolution features from the multiscale features and the salient features. Thereafter, a Transformer module is applied to capture both long- and short-range temporal features from the multi-resolution features. Finally, the long- and short-range temporal features are restored with low-layer details and mapped to the predicted classification results. Additionally, the Lovász loss function is applied to alleviate the class imbalance problem in sleep datasets. Our proposed method was tested on the Sleep-EDF-39 and Sleep-EDF-153 datasets, and it achieved classification accuracies of 88.9% and 85.2% and Macro-F1 scores of 84.8% and 79.7%, respectively, thus outperforming conventional traditional baseline models. These results highlight the efficacy of the proposed method in fusing multi-modal information. This method has potential for application as an adjunct tool for diagnosing sleep disorders.