Computational imaging for rapid detection of grade-I cerebral small vessel disease (cSVD).

An early identification and subsequent management of cerebral small vessel disease (cSVD) grade 1 can delay progression into grades II and III. Machine learning algorithms have shown considerable promise in medical image interpretation automation. An experimental cross-sectional study aimed to develop an automated computer-aided diagnostic system based on AI (artificial intelligence) tools to detect grade 1-cSVD with improved accuracy. Patients with Fazekas grade 1 cSVD on Non-Contrast Magnetic Resonance Imaging (MRI) Brain of age >40 years of both genders were included. The dataset was pre-processed to be fed into a 3D convolutional neural network (CNN) model. A 3D stack with the shape (120, 128, 128, 1) containing axial slices from the brain magnetic resonance image was created. The model was created from scratch and contained four convolutional and three fully connected (FC) layers. The dataset was preprocessed by making a 3D stack, and normalizing, resizing, and completing the stack was performed. A 3D-CNN model architecture was designed to train and test preprocessed images. We achieved an accuracy of 93.12 % when 2D axial slices were used. When the 2D slices of a patient were stacked to form a 3D image, an accuracy of 85.71 % was achieved on the test set. Overall, the 3D-CNN model performed very well on the test set. The earliest and the most accurate diagnosis from computational imaging methods can help reduce the huge burden of cSVD and its associated morbidity in the form of vascular dementia.

Denoising of volumetric magnetic resonance imaging using multi-channel three-dimensional convolutional neural network with applications on fast spin echo acquisitions.

Background: Three-dimensional (3D) magnetic resonance imaging (MRI) can be acquired with a high spatial resolution with flexibility being reformatted into arbitrary planes, but at the cost of reduced signal-to-noise ratio. Deep-learning methods are promising for denoising in MRI. However, the existing 3D denoising convolutional neural networks (CNNs) rely on either a multi-channel two-dimensional (2D) network or a single-channel 3D network with limited ability to extract high dimensional features. We aim to develop a deep learning approach based on multi-channel 3D convolution to utilize inherent noise information embedded in multiple number of excitation (NEX) acquisition for denoising 3D fast spin echo (FSE) MRI. Methods: A multi-channel 3D CNN is developed for denoising multi-NEX 3D FSE magnetic resonance (MR) images based on the feature extraction of 3D noise distributions embedded in 2-NEX 3D MRI. The performance of the proposed approach was compared to several state-of-the-art MRI denoising methods on both synthetic and real knee data using 2D and 3D metrics of peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM). Results: The proposed method achieved improved denoising performance compared to the current state-of-the-art denoising methods in both slice-by-slice 2D and volumetric 3D metrics of PSNR and SSIM. Conclusions: A multi-channel 3D CNN is developed for denoising of multi-NEX 3D FSE MR images. The superior performance of the proposed multi-channel 3D CNN in denoising multi-NEX 3D MRI demonstrates its potential in tasks that require the extraction of high-dimensional features.

The Use of fMRI Regional Analysis to Automatically Detect ADHD Through a 3D CNN-Based Approach.

Attention deficit hyperactivity disorder (ADHD) is a neurodevelopmental disorder characterized by a reduced attention span, hyperactivity, and impulsive behaviors, which typically manifest during childhood. This study employs functional magnetic resonance imaging (fMRI) to use spontaneous brain activity for classifying individuals with ADHD, focusing on a 3D convolutional neural network (CNN) architecture to facilitate the design of decision support systems. We developed a novel deep learning model based on 3D CNNs using the ADHD-200 database, which comprises datasets from NeuroImage (NI), New York University (NYU), and Peking University (PU). We used fractional amplitude of low-frequency fluctuations (fALFF) and regional homogeneity (ReHo) data in three dimensions and performed a fivefold cross-validation to address the dataset imbalance. We aimed to verify the efficacy of our proposed 3D CNN by contrasting it with a fully connected neural network (FCNN) architecture. The 3D CNN achieved accuracy rates of 76.19% (NI), 69.92% (NYU), and 70.77% (PU) for fALFF data. The FCNN model yielded lower accuracy rates across all datasets. For generalizability, we trained on NI and NYU datasets and tested on PU. The 3D CNN achieved 69.48% accuracy on fALFF outperforming the FCNN. Our results demonstrate that using 3D CNNs for classifying fALFF data is an effective approach for diagnosing ADHD. Also, FCNN confirmed the efficiency of the designed model.

Deep learning based diagnosis of PTSD using 3D-CNN and resting-state fMRI data.

BACKGROUND: The incidence rate of Posttraumatic stress disorder (PTSD) is currently increasing due to wars, terrorism, and pandemic disease situations. Therefore, accurate detection of PTSD is crucial for the treatment of the patients, for this purpose, the present study aims to classify individuals with PTSD versus healthy control. METHODS: The resting-state functional MRI (rs-fMRI) scans of 19 PTSD and 24 healthy control male subjects have been used to identify the activation pattern in most affected brain regions using group-level independent component analysis (ICA) and t-test. To classify PTSD-affected subjects from healthy control six machine learning techniques including random forest, Naive Bayes, support vector machine, decision tree, K-nearest neighbor, linear discriminant analysis, and deep learning three-dimensional 3D-CNN have been performed on the data and compared. RESULTS: The rs-fMRI scans of the most commonly investigated 11 regions of trauma-exposed and healthy brains are analyzed to observe their level of activation. Amygdala and insula regions are determined as the most activated regions from the regions-of-interest in the brain of PTSD subjects. In addition, machine learning techniques have been applied to the components extracted from ICA but the models provided low classification accuracy. The ICA components are also fed into the 3D-CNN model, which is trained with a 5-fold cross-validation method. The 3D-CNN model demonstrated high accuracies, such as 98.12%, 98.25 %, and 98.00 % on average with training, validation, and testing datasets, respectively. CONCLUSION: The findings indicate that 3D-CNN is a surpassing method than the other six considered techniques and it helps to recognize PTSD patients accurately.

Enhancing Lung Cancer Survival Prediction: 3D CNN Analysis of CT Images Using Novel GTV1-SliceNum Feature and PEN-BCE Loss Function.

Lung cancer is a prevalent malignancy associated with a high mortality rate, with a 5-year relative survival rate of 23%. Traditional survival analysis methods, reliant on clinician judgment, may lack accuracy due to their subjective nature. Consequently, there is growing interest in leveraging AI-based systems for survival analysis using clinical data and medical imaging. The purpose of this study is to improve survival classification for lung cancer patients by utilizing a 3D-CNN architecture (ResNet-34) applied to CT images from the NSCLC-Radiomics dataset. Through comprehensive ablation studies, we evaluate the effectiveness of different features and methodologies in classification performance. Key contributions include the introduction of a novel feature (GTV1-SliceNum), the proposal of a novel loss function (PEN-BCE) accounting for false negatives and false positives, and the showcasing of their efficacy in classification. Experimental work demonstrates results surpassing those of the existing literature, achieving a classification accuracy of 0.7434 and an ROC-AUC of 0.7768. The conclusions of this research indicate that the AI-driven approach significantly improves survival prediction for lung cancer patients, highlighting its potential for enhancing personalized treatment strategies and prognostic modeling.

An end-to-end multi-scale airway segmentation framework based on pulmonary CT image.

Objective. Automatic and accurate airway segmentation is necessary for lung disease diagnosis. The complex tree-like structures leads to gaps in the different generations of the airway tree, and thus airway segmentation is also considered to be a multi-scale problem. In recent years, convolutional neural networks have facilitated the development of medical image segmentation. In particular, 2D CNNs and 3D CNNs can extract different scale features. Hence, we propose a two-stage and 2D + 3D framework for multi-scale airway tree segmentation.Approach. In stage 1, we use a 2D full airway SegNet(2D FA-SegNet) to segment the complete airway tree. Multi-scale atros spatial pyramid and Atros Residual Skip connection modules are inserted to extract different scales feature. We designed a hard sample selection strategy to increase the proportion of intrapulmonary airway samples in stage 2. 3D airway RefineNet (3D ARNet) as stage 2 takes the results of stage 1 asa prioriinformation. Spatial information extracted by 3D convolutional kernel compensates for the loss of in 2D FA-SegNet. Furthermore, we added false positive losses and false negative losses to improve the segmentation performance of airway branches within the lungs.Main results. We performed data enhancement on the publicly available dataset of ISICDM 2020 Challenge 3, and on which evaluated our method. Comprehensive experiments show that the proposed method has the highest dice similarity coefficient (DSC) of 0.931, and IoU of 0.871 for the whole airway tree and DSC of 0.699, and IoU of 0.543 for the intrapulmonary bronchi tree. In addition, 3D ARNet proposed in this paper cascaded with other state-of-the-art methods to increase detected tree length rate by up to 46.33% and detected tree branch rate by up to 42.97%.Significance. The quantitative and qualitative evaluation results show that our proposed method performs well in segmenting the airway at different scales.

Multi-scale 3D-CRU for EEG emotion recognition.

In this paper, we propose a novel multi-scale 3D-CRU model, with the goal of extracting more discriminative emotion feature from EEG signals. By concurrently exploiting the relative electrode locations and different frequency subbands of EEG signals, a three-dimensional feature representation is reconstructed wherein the Delta (δ) frequency pattern is included. We employ a multi-scale approach, termed 3D-CRU, to concurrently extract frequency and spatial features at varying levels of granularity within each time segment. In the proposed 3D-CRU, we introduce a multi-scale 3D Convolutional Neural Network (3D-CNN) to effectively capture discriminative information embedded within the 3D feature representation. To model the temporal dynamics across consecutive time segments, we incorporate a Gated Recurrent Unit (GRU) module to extract temporal representations from the time series of combined frequency-spatial features. Ultimately, the 3D-CRU model yields a global feature representation, encompassing comprehensive information across time, frequency, and spatial domains. Numerous experimental assessments conducted on publicly available DEAP and SEED databases provide empirical evidence supporting the enhanced performance of our proposed model in the domain of emotion recognition. These findings underscore the efficacy of the features extracted by the proposed multi-scale 3D-GRU model, particularly with the incorporation of the Delta (δ) frequency pattern. Specifically, on the DEAP dataset, the accuracy of Valence and Arousal are 93.12% and 94.31%, respectively, while on the SEED dataset, the accuracy is 92.25%.

Epileptic seizure prediction based on EEG using pseudo-three-dimensional CNN.

Epileptic seizures are characterized by their sudden and unpredictable nature, posing significant risks to a patient's daily life. Accurate and reliable seizure prediction systems can provide alerts before a seizure occurs, as well as give the patient and caregivers provider enough time to take appropriate measure. This study presents an effective seizure prediction method based on deep learning that combine with handcrafted features. The handcrafted features were selected by Max-Relevance and Min-Redundancy (mRMR) to obtain the optimal set of features. To extract the epileptic features from the fused multidimensional structure, we designed a P3D-BiConvLstm3D model, which is a combination of pseudo-3D convolutional neural network (P3DCNN) and bidirectional convolutional long short-term memory 3D (BiConvLstm3D). We also converted EEG signals into a multidimensional structure that fused spatial, manual features, and temporal information. The multidimensional structure is then fed into a P3DCNN to extract spatial and manual features and feature-to-feature dependencies, followed by a BiConvLstm3D input to explore temporal dependencies while preserving the spatial features, and finally, a channel attention mechanism is implemented to emphasize the more representative information in the multichannel output. The proposed has an average accuracy of 98.13%, an average sensitivity of 98.03%, an average precision of 98.30% and an average specificity of 98.23% for the CHB-MIT scalp EEG database. A comparison of the proposed model with other baseline methods was done to confirm the better performance of features through time-space nonlinear feature fusion. The results show that the proposed P3DCNN-BiConvLstm3D-Attention3D method for epilepsy prediction by time-space nonlinear feature fusion is effective.

A semi-supervised multiview-MRI network for the detection of Knee Osteoarthritis.

Knee OsteoArthritis (OA) is a prevalent chronic condition, affecting a significant proportion of the global population. Detecting knee OA is crucial as the degeneration of the knee joint is irreversible. In this paper, we introduce a semi-supervised multi-view framework and a 3D CNN model for detecting knee OA using 3D Magnetic Resonance Imaging (MRI) scans. We introduce a semi-supervised learning approach combining labeled and unlabeled data to improve the performance and generalizability of the proposed model. Experimental results show the efficacy of our proposed approach in detecting knee OA from 3D MRI scans using a large cohort of 4297 subjects. An ablation study was conducted to investigate the contributions of various components of the proposed model, providing insights into the optimal design of the model. Our results indicate the potential of the proposed approach to improve the accuracy and efficiency of OA diagnosis. The proposed framework reported an AUC of 93.20% for the detection of knee OA.

Multimodal audio-visual robot fusing 3D CNN and CRNN for player behavior recognition and prediction in basketball matches.

Introduction: Intelligent robots play a crucial role in enhancing efficiency, reducing costs, and improving safety in the logistics industry. However, traditional path planning methods often struggle to adapt to dynamic environments, leading to issues such as collisions and conflicts. This study aims to address the challenges of path planning and control for logistics robots in complex environments. Methods: The proposed method integrates information from different perception modalities to achieve more accurate path planning and obstacle avoidance control, thereby enhancing the autonomy and reliability of logistics robots. Firstly, a 3D convolutional neural network (CNN) is employed to learn the feature representation of objects in the environment for object recognition. Next, long short-term memory (LSTM) is used to model spatio-temporal features and predict the behavior and trajectory of dynamic obstacles. This enables the robot to accurately predict the future position of obstacles in complex environments, reducing collision risks. Finally, the Dijkstra algorithm is applied for path planning and control decisions to ensure the robot selects the optimal path in various scenarios. Results: Experimental results demonstrate the effectiveness of the proposed method in terms of path planning accuracy and obstacle avoidance performance. The method outperforms traditional approaches, showing significant improvements in both aspects. Discussion: The intelligent path planning and control scheme presented in this paper enhances the practicality of logistics robots in complex environments, thereby promoting efficiency and safety in the logistics industry.

Systematic comparison of 3D Deep learning and classical machine learning explanations for Alzheimer's Disease detection.

Black-box deep learning (DL) models trained for the early detection of Alzheimer's Disease (AD) often lack systematic model interpretation. This work computes the activated brain regions during DL and compares those with classical Machine Learning (ML) explanations. The architectures used for DL were 3D DenseNets, EfficientNets, and Squeeze-and-Excitation (SE) networks. The classical models include Random Forests (RFs), Support Vector Machines (SVMs), eXtreme Gradient Boosting (XGBoost), Light Gradient Boosting (LightGBM), Decision Trees (DTs), and Logistic Regression (LR). For explanations, SHapley Additive exPlanations (SHAP) values, Local Interpretable Model-agnostic Explanations (LIME), Gradient-weighted Class Activation Mapping (GradCAM), GradCAM++ and permutation-based feature importance were implemented. During interpretation, correlated features were consolidated into aspects. All models were trained on the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset. The validation includes internal and external validation on the Australian Imaging and Lifestyle flagship study of Ageing (AIBL) and the Open Access Series of Imaging Studies (OASIS). DL and ML models reached similar classification performances. Regarding the brain regions, both types focus on different regions. The ML models focus on the inferior and middle temporal gyri, and the hippocampus, and amygdala regions previously associated with AD. The DL models focus on a wider range of regions including the optical chiasm, the entorhinal cortices, the left and right vessels, and the 4th ventricle which were partially associated with AD. One explanation for the differences is the input features (textures vs. volumes). Both types show reasonable similarity to a ground truth Voxel-Based Morphometry (VBM) analysis. Slightly higher similarities were measured for ML models.

Improving brain age prediction with anatomical feature attention-enhanced 3D-CNN.

Currently, significant progress has been made in predicting brain age from structural Magnetic Resonance Imaging (sMRI) data using deep learning techniques. However, despite the valuable structural information they contain, the traditional engineering features known as anatomical features have been largely overlooked in this context. To address this issue, we propose an attention-based network design that integrates anatomical and deep convolutional features, leveraging an anatomical feature attention (AFA) module to effectively capture salient anatomical features. In addition, we introduce a fully convolutional network, which simplifies the extraction of deep convolutional features and overcomes the high computational memory requirements associated with deep learning. Our approach outperforms several widely-used models on eight publicly available datasets (n = 2501), with a mean absolute error (MAE) of 2.20 years in predicting brain age. Comparisons with deep learning models lacking the AFA module demonstrate that our fusion model effectively improves overall performance. These findings provide a promising approach for combining anatomical and deep convolutional features from sMRI data to predict brain age, with potential applications in clinical diagnosis and treatment, particularly for populations with age-related cognitive decline or neurological disorders.

Study on the detection of CT image based on multi-task network method of global attention for fine-fracture / 中国医学装备

Objective:To improve the perception of computed tomography(CT)images in detecting fine fracture through multi-task network of global attention,and to realize the detection of the target of fine fracture at case level through multi-task,and to quickly and accurately identify and locate fracture from a large number of CT images,so as to assist doctors to timely conduct treatment.Methods:A grouped Non-local network method was introduced to calculate the remote dependency relationship between each position of CT image continuous sections and channel.A single-stage detector of multi-objective detection model three dimension(3D)RetinaNet was integrated with the medical image semantic segmentation architecture(3D U-Net).A end-to-end multi-task 3D convolutional network was realized,which realized the detection of case level for fine fracture through multi-task collaboration.Select 600 CT scan images from the Rib Frac Dataset of rib fractures provided by the MICCAI 2020 Challenge,and they were divided into training set(500 cases)and test set(100 cases)as the ratio of 5:1 to test the precise performance of multi-task 3D convolutional network.Results:The precise performance of multi-task 3D convolutional network method was better than that of single-task FracNet,3D RetinaNet and 3D Retina U-Net in detection,which average precision was respectively higher 7.8%and 11.4%than 3D RetinaNet and 3D Retina U-Net.It was better than two kinds of single-task network detection method included 3D Faster R-CNN and 3D Mask R-CNN,and the average precision of that was respectively higher 6.7%and 3.1%than them.Conclusion:The integrated different modules of global attention multi-task network can improve the detection performance of fine fracture.The introduction of grouped Non-local network method can further improve the precise performance for the targets of fine fractures in detection.

Two-stage video-based convolutional neural networks for adult spinal deformity classification.

Introduction: Assessment of human gait posture can be clinically effective in diagnosing human gait deformities early in life. Currently, two methods-static and dynamic-are used to diagnose adult spinal deformity (ASD) and other spinal disorders. Full-spine lateral standing radiographs are used in the standard static method. However, this is a static assessment of joints in the standing position and does not include information on joint changes when the patient walks. Careful observation of long-distance walking can provide a dynamic assessment that reveals an uncompensated posture; however, this increases the workload of medical practitioners. A three-dimensional (3D) motion system is proposed for the dynamic method. Although the motion system successfully detected dynamic posture changes, access to the facilities was limited. Therefore, a diagnostic approach that is facility-independent, has low practice flow, and does not involve patient contact is required. Methods: We focused on a video-based method to classify patients with spinal disorders either as ASD, or other forms of ASD. To achieve this goal, we present a video-based two-stage machine-learning method. In the first stage, deep learning methods are used to locate the patient and extract the area where the patient is located. In the second stage, a 3D CNN (convolutional neural network) device is used to capture spatial and temporal information (dynamic motion) from the extracted frames. Disease classification is performed by discerning posture and gait from the extracted frames. Model performance was assessed using the mean accuracy, F1 score, and area under the receiver operating characteristic curve (AUROC), with five-fold cross-validation. We also compared the final results with professional observations. Results: Our experiments were conducted using a gait video dataset comprising 81 patients. The experimental results indicated that our method is effective for classifying ASD and other spinal disorders. The proposed method achieved a mean accuracy of 0.7553, an F1 score of 0.7063, and an AUROC score of 0.7864. Additionally, ablation experiments indicated the importance of the first stage (detection stage) and transfer learning of our proposed method. Discussion: The observations from the two doctors were compared using the proposed method. The mean accuracies observed by the two doctors were 0.4815 and 0.5247, with AUROC scores of 0.5185 and 0.5463, respectively. We proved that the proposed method can achieve accurate and reliable medical testing results compared with doctors' observations using videos of 1 s duration. All our code, models, and results are available at https://github.com/ChenKaiXuSan/Walk_Video_PyTorch. The proposed framework provides a potential video-based method for improving the clinical diagnosis for ASD and non-ASD. This framework might, in turn, benefit both patients and clinicians to treat the disease quickly and directly and further reduce facility dependency and data-driven systems.

Brain Age Prediction Using 2D Projections Based on Higher-Order Statistical Moments and Eigenslices from 3D Magnetic Resonance Imaging Volumes.

Brain age prediction from 3D MRI volumes using deep learning has recently become a popular research topic, as brain age has been shown to be an important biomarker. Training deep networks can be very computationally demanding for large datasets like the U.K. Biobank (currently 29,035 subjects). In our previous work, it was demonstrated that using a few 2D projections (mean and standard deviation along three axes) instead of each full 3D volume leads to much faster training at the cost of a reduction in prediction accuracy. Here, we investigated if another set of 2D projections, based on higher-order statistical central moments and eigenslices, leads to a higher accuracy. Our results show that higher-order moments do not lead to a higher accuracy, but that eigenslices provide a small improvement. We also show that an ensemble of such models provides further improvement.

Multimodal intelligent logistics robot combining 3D CNN, LSTM, and visual SLAM for path planning and control.

Introduction: In today's dynamic logistics landscape, the role of intelligent robots is paramount for enhancing efficiency, reducing costs, and ensuring safety. Traditional path planning methods often struggle to adapt to changing environments, resulting in issues like collisions and conflicts. This research addresses the challenge of path planning and control for logistics robots operating in complex environments. The proposed method aims to integrate information from various perception sources to enhance path planning and obstacle avoidance, thereby increasing the autonomy and reliability of logistics robots. Methods: The method presented in this paper begins by employing a 3D Convolutional Neural Network (CNN) to learn feature representations of objects within the environment, enabling object recognition. Subsequently, Long Short-Term Memory (LSTM) models are utilized to capture spatio-temporal features and predict the behavior and trajectories of dynamic obstacles. This predictive capability empowers robots to more accurately anticipate the future positions of obstacles in intricate settings, thereby mitigating potential collision risks. Finally, the Dijkstra algorithm is employed for path planning and control decisions to ensure the selection of optimal paths across diverse scenarios. Results: In a series of rigorous experiments, the proposed method outperforms traditional approaches in terms of both path planning accuracy and obstacle avoidance performance. These substantial improvements underscore the efficacy of the intelligent path planning and control scheme. Discussion: This research contributes to enhancing the practicality of logistics robots in complex environments, thereby fostering increased efficiency and safety within the logistics industry. By combining object recognition, spatio-temporal modeling, and optimized path planning, the proposed method enables logistics robots to navigate intricate scenarios with higher precision and reliability, ultimately advancing the capabilities of autonomous logistics operations.

Multi-Slice Generation sMRI and fMRI for Autism Spectrum Disorder Diagnosis Using 3D-CNN and Vision Transformers.

Researchers have explored various potential indicators of ASD, including changes in brain structure and activity, genetics, and immune system abnormalities, but no definitive indicator has been found yet. Therefore, this study aims to investigate ASD indicators using two types of magnetic resonance images (MRI), structural (sMRI) and functional (fMRI), and to address the issue of limited data availability. Transfer learning is a valuable technique when working with limited data, as it utilizes knowledge gained from a pre-trained model in a domain with abundant data. This study proposed the use of four vision transformers namely ConvNeXT, MobileNet, Swin, and ViT using sMRI modalities. The study also investigated the use of a 3D-CNN model with sMRI and fMRI modalities. Our experiments involved different methods of generating data and extracting slices from raw 3D sMRI and 4D fMRI scans along the axial, coronal, and sagittal brain planes. To evaluate our methods, we utilized a standard neuroimaging dataset called NYU from the ABIDE repository to classify ASD subjects from typical control subjects. The performance of our models was evaluated against several baselines including studies that implemented VGG and ResNet transfer learning models. Our experimental results validate the effectiveness of the proposed multi-slice generation with the 3D-CNN and transfer learning methods as they achieved state-of-the-art results. In particular, results from 50-middle slices from the fMRI and 3D-CNN showed a profound promise in ASD classifiability as it obtained a maximum accuracy of 0.8710 and F1-score of 0.8261 when using the mean of 4D images across the axial, coronal, and sagittal. Additionally, the use of the whole slices in fMRI except the beginnings and the ends of brain views helped to reduce irrelevant information and showed good performance of 0.8387 accuracy and 0.7727 F1-score. Lastly, the transfer learning with the ConvNeXt model achieved results higher than other transformers when using 50-middle slices sMRI along the axial, coronal, and sagittal planes.

3D Convolution neural network with multiscale spatial and temporal cues for motor imagery EEG classification.

Recently, deep learning-based methods have achieved meaningful results in the Motor imagery electroencephalogram (MI EEG) classification. However, because of the low signal-to-noise ratio and the various characteristics of brain activities among subjects, these methods lack a subject adaptive feature extraction mechanism. Another issue is that they neglect important spatial topological information and the global temporal variation trend of MI EEG signals. These issues limit the classification accuracy. Here, we propose an end-to-end 3D CNN to extract multiscale spatial and temporal dependent features for improving the accuracy performance of 4-class MI EEG classification. The proposed method adaptively assigns higher weights to motor-related spatial channels and temporal sampling cues than the motor-unrelated ones across all brain regions, which can prevent influences caused by biological and environmental artifacts. Experimental evaluation reveals that the proposed method achieved an average classification accuracy of 93.06% and 97.05% on two commonly used datasets, demonstrating excellent performance and robustness for different subjects compared to other state-of-the-art methods.In order to verify the real-time performance in actual applications, the proposed method is applied to control the robot based on MI EEG signals. The proposed approach effectively addresses the issues of existing methods, improves the classification accuracy and the performance of BCI system, and has great application prospects.

S3DCN-OLSR: A shallow 3D CNN method for online learning state recognition.

The repeated recurrence of COVID-19 has significantly disrupted learning for students in face-to-face instructional settings. While moving from offline to online instruction has proven to be one of the best solutions, classroom management and capturing students' learning states have emerged as important challenges with the increasing popularity of online instruction. To address these challenges, in this paper we propose an online learning status recognition method based on shallow 3D convolution (S3DC-OLSR) for online students, to identify students' online learning states by analysing their micro-expressions. Specifically, we first use the data augmentation method proposed in this paper to decompose the students' online video file into three features: horizontal component of optical flow, vertical component of optical flow and optical amplitude. Next, the students' online learning status is recognised by feeding the processed data into a shallow 3D convolution neural network. To test the performance of our method, we conduct extensive experiments on the CASME II and SMIC datasets, and the results indicate that our method outperforms the other state-of-the-art methods considered in terms of recognition accuracy, UF1 and UAR, which demonstrates the superiority of our method in identifying students' online learning states.

Ensemble of local and global information for Protein-Ligand Binding Affinity Prediction.

Accurately predicting protein-ligand binding affinities is crucial for determining molecular properties and understanding their physical effects. Neural networks and transformers are the predominant methods for sequence modeling, and both have been successfully applied independently for protein-ligand binding affinity prediction. As local and global information of molecules are vital for protein-ligand binding affinity prediction, we aim to combine bi-directional gated recurrent unit (BiGRU) and convolutional neural network (CNN) to effectively capture both local and global molecular information. Additionally, attention mechanisms can be incorporated to automatically learn and adjust the level of attention given to local and global information, thereby enhancing the performance of the model. To achieve this, we propose the PLAsformer approach, which encodes local and global information of molecules using 3DCNN and BiGRU with attention mechanism, respectively. This approach enhances the model's ability to encode comprehensive local and global molecular information. PLAsformer achieved a Pearson's correlation coefficient of 0.812 and a Root Mean Square Error (RMSE) of 1.284 when comparing experimental and predicted affinity on the PDBBind-2016 dataset. These results surpass the current state-of-the-art methods for binding affinity prediction. The high accuracy of PLAsformer's predictive performance, along with its excellent generalization ability, is clearly demonstrated by these findings.