Proto-neural networks from thermal proteins.

Proteinoids are synthetic polymers that have structural similarities to natural proteins, and their formation is achieved through the application of heat to amino acid combinations in a dehydrated environment. The thermal proteins, initially synthesised by Sidney Fox during the 1960s, has the ability to undergo self-assembly, resulting in the formation of microspheres that resemble cells. These microspheres have fascinating biomimetic characteristics. In recent studies, substantial advancements have been made in elucidating the electrical signalling phenomena shown by proteinoids, hence showcasing their promising prospects in the field of neuro-inspired computing. This study demonstrates the advancement of experimental prototypes that employ proteinoids in the construction of fundamental neural network structures. The article provides an overview of significant achievements in proteinoid systems, such as the demonstration of electrical excitability, emulation of synaptic functions, capabilities in pattern recognition, and adaptability of network structures. This study examines the similarities and differences between proteinoid networks and spontaneous neural computation. We examine the persistent challenges associated with deciphering the underlying mechanisms of emergent proteinoid-based intelligence. Additionally, we explore the potential for developing bio-inspired computing systems using synthetic thermal proteins in forthcoming times. The results of this study offer a theoretical foundation for the advancement of adaptive, self-assembling electronic systems that operate using artificial bio-neural principles.

Self-supervised contrastive learning for EEG-based cross-subject motor imagery recognition.

Objective. The extensive application of electroencephalography (EEG) in brain-computer interfaces (BCIs) can be attributed to its non-invasive nature and capability to offer high-resolution data. The acquisition of EEG signals is a straightforward process, but the datasets associated with these signals frequently exhibit data scarcity and require substantial resources for proper labeling. Furthermore, there is a significant limitation in the generalization performance of EEG models due to the substantial inter-individual variability observed in EEG signals.Approach. To address these issues, we propose a novel self-supervised contrastive learning framework for decoding motor imagery (MI) signals in cross-subject scenarios. Specifically, we design an encoder combining convolutional neural network and attention mechanism. In the contrastive learning training stage, the network undergoes training with the pretext task of data augmentation to minimize the distance between pairs of homologous transformations while simultaneously maximizing the distance between pairs of heterologous transformations. It enhances the amount of data utilized for training and improves the network's ability to extract deep features from original signals without relying on the true labels of the data.Main results. To evaluate our framework's efficacy, we conduct extensive experiments on three public MI datasets: BCI IV IIa, BCI IV IIb, and HGD datasets. The proposed method achieves cross-subject classification accuracies of 67.32%, 82.34%, and 81.13%on the three datasets, demonstrating superior performance compared to existing methods.Significance. Therefore, this method has great promise for improving the performance of cross-subject transfer learning in MI-based BCI systems.

Develop prediction model to help forecast advanced prostate cancer patients' prognosis after surgery using neural network.

Background: The effect of surgery on advanced prostate cancer (PC) is unclear and predictive model for postoperative survival is lacking yet. Methods: We investigate the National Cancer Institute's Surveillance, Epidemiology, and End Results (SEER) database, to collect clinical features of advanced PC patients. According to clinical experience, age, race, grade, pathology, T, N, M, stage, size, regional nodes positive, regional nodes examined, surgery, radiotherapy, chemotherapy, history of malignancy, clinical Gleason score (composed of needle core biopsy or transurethral resection of the prostate specimens), pathological Gleason score (composed of prostatectomy specimens) and prostate-specific antigen (PSA) are the potential predictive variables. All samples are divided into train cohort (70% of total, for model training) and test cohort (30% of total, for model validation) by random sampling. We then develop neural network to predict advanced PC patients' overall. Area under receiver operating characteristic curve (AUC) is used to evaluate model's performance. Results: 6380 patients, diagnosed with advanced (stage III-IV) prostate cancer and receiving surgery, have been included. The model using all collected clinical features as predictors and based on neural network algorithm performs best, which scores 0.7058 AUC (95% CIs, 0.7021-0.7068) in train cohort and 0.6925 AUC (95% CIs, 0.6906-0.6956) in test cohort. We then package it into a Windows 64-bit software. Conclusion: Patients with advanced prostate cancer may benefit from surgery. In order to forecast their overall survival, we first build a clinical features-based prognostic model. This model is accuracy and may offer some reference on clinical decision making.

Identification of Panax notoginseng origin using terahertz precision spectroscopy and neural network algorithm.

Panax notoginseng (P. notoginseng), a Chinese herb containing various saponins, benefits immune system in medicines development, which from Wenshan (authentic cultivation) is often counterfeited by others for large demand and limited supply. Here, we proposed a method for identifying P. notoginseng origin combining terahertz (THz) precision spectroscopy and neural network. Based on the comparative analysis of four qualitative identification methods, we chose high-performance liquid chromatography (HPLC) and THz spectroscopy to detect 252 samples from five origins. After classifications using Convolutional Neural Networks (CNNs) model, we found that the performance of THz spectra was superior to that of HPLC. The underlying mechanism is that there are clear nonlinear relations among the THz spectra and the origins due to the wide spectra and multi-parameter characteristics, which makes the accuracy of five-classification origin identification up to 97.62%. This study realizes the rapid, non-destructive and accurate identification of P. notoginseng origin, providing a practical reference for herbal medicine.

Small object detection algorithm incorporating swin transformer for tea buds.

Accurate identification of small tea buds is a key technology for tea harvesting robots, which directly affects tea quality and yield. However, due to the complexity of the tea plantation environment and the diversity of tea buds, accurate identification remains an enormous challenge. Current methods based on traditional image processing and machine learning fail to effectively extract subtle features and morphology of small tea buds, resulting in low accuracy and robustness. To achieve accurate identification, this paper proposes a small object detection algorithm called STF-YOLO (Small Target Detection with Swin Transformer and Focused YOLO), which integrates the Swin Transformer module and the YOLOv8 network to improve the detection ability of small objects. The Swin Transformer module extracts visual features based on a self-attention mechanism, which captures global and local context information of small objects to enhance feature representation. The YOLOv8 network is an object detector based on deep convolutional neural networks, offering high speed and precision. Based on the YOLOv8 network, modules including Focus and Depthwise Convolution are introduced to reduce computation and parameters, increase receptive field and feature channels, and improve feature fusion and transmission. Additionally, the Wise Intersection over Union loss is utilized to optimize the network. Experiments conducted on a self-created dataset of tea buds demonstrate that the STF-YOLO model achieves outstanding results, with an accuracy of 91.5% and a mean Average Precision of 89.4%. These results are significantly better than other detectors. Results show that, compared to mainstream algorithms (YOLOv8, YOLOv7, YOLOv5, and YOLOx), the model improves accuracy and F1 score by 5-20.22 percentage points and 0.03-0.13, respectively, proving its effectiveness in enhancing small object detection performance. This research provides technical means for the accurate identification of small tea buds in complex environments and offers insights into small object detection. Future research can further optimize model structures and parameters for more scenarios and tasks, as well as explore data augmentation and model fusion methods to improve generalization ability and robustness.

Evaluating the Robustness of a Deep Learning Bone Age Algorithm to Clinical Image Variation Using Computational Stress Testing.

Purpose To evaluate the robustness of an award-winning bone age deep learning (DL) model to extensive variations in image appearance. Materials and Methods In December 2021, the DL bone age model that won the 2017 RSNA Pediatric Bone Age Challenge was retrospectively evaluated using the RSNA validation set (1425 pediatric hand radiographs; internal test set in this study) and the Digital Hand Atlas (DHA) (1202 pediatric hand radiographs; external test set). Each test image underwent seven types of transformations (rotations, flips, brightness, contrast, inversion, laterality marker, and resolution) to represent a range of image appearances, many of which simulate real-world variations. Computational "stress tests" were performed by comparing the model's predictions on baseline and transformed images. Mean absolute differences (MADs) of predicted bone ages compared with radiologist-determined ground truth on baseline versus transformed images were compared using Wilcoxon signed rank tests. The proportion of clinically significant errors (CSEs) was compared using McNemar tests. Results There was no evidence of a difference in MAD of the model on the two baseline test sets (RSNA = 6.8 months, DHA = 6.9 months; P = .05), indicating good model generalization to external data. Except for the RSNA dataset images with an appended radiologic laterality marker (P = .86), there were significant differences in MAD for both the DHA and RSNA datasets among other transformation groups (rotations, flips, brightness, contrast, inversion, and resolution). There were significant differences in proportion of CSEs for 57% of the image transformations (19 of 33) performed on the DHA dataset. Conclusion Although an award-winning pediatric bone age DL model generalized well to curated external images, it had inconsistent predictions on images that had undergone simple transformations reflective of several real-world variations in image appearance. Keywords: Pediatrics, Hand, Convolutional Neural Network, Radiography Supplemental material is available for this article. © RSNA, 2024 See also commentary by Faghani and Erickson in this issue.

On interaction of proteinoids with simulated neural networks.

Proteinoid-neuron networks combine biological neurons with spiking proteinoid microspheres, which are generated by thermal condensation of amino acids. Complex and dynamic spiking patterns in response to varied stimuli make these networks suitable for unconventional computing. This research examines the interaction of proteinoid-neuron networks with function-generator-artificial neural networks (ANN) that may create distinct electrical waveforms. Function-generator- artificial neural network (ANN) stimulates and modulates proteinoid-neuron network spiking activity and synchronisation to encode and decode information. We employ function-generator-ANN to study proteinoid-neuron network nonlinear dynamics and chaos and optimise their performance and energy efficiency. Function-generator-ANN improves proteinoid-neuron networks' computational capacities and robustness and creates unique hybrid systems with electrical devices. We address the benefits as well as the drawbacks of employing proteinoid-neuron networks for unconventional computing with function-generator-ANN.

Fully connected-convolutional (FC-CNN) neural network based on hyperspectral images for rapid identification of P. ginseng growth years.

P. ginseng is a precious traditional Chinese functional food, which is used for both medicinal and food purposes, and has various effects such as immunomodulation, anti-tumor and anti-oxidation. The growth year of P. ginseng has an important impact on its medicinal and economic values. Fast and nondestructive identification of the growth year of P. ginseng is crucial for its quality evaluation. In this paper, we propose a FC-CNN network that incorporates spectral and spatial features of hyperspectral images to characterize P. ginseng from different growth years. The importance ranking of the spectra was obtained using the random forest method for optimal band selection. Based on the hyperspectral reflectance data of P. ginseng after radiometric calibration and the images of the best five VNIR bands and five SWIR bands selected, the year-by-year identification of P. ginseng age and its identification experiments for food and medicinal purposes were conducted, and the FC-CNN network and its FCNN and CNN branch networks were tested and compared in terms of their effectiveness in the identification of P. ginseng growth years. It has been experimentally verified that the best year-by-year recognition was achieved by utilizing images from five visible and near-infrared important bands and all spectral curves, and the recognition accuracy of food and medicinal use reached 100%. The FC-CNN network is significantly better than its branching model in the effect of edible and medicinal identification. The results show that for P. ginseng growth year identification, VNIR images have much more useful information than SWIR images. Meanwhile, the FC-CNN network utilizing the spectral and spatial features of hyperspectral images is an effective method for the identification of P. ginseng growth year.

GraphormerDTI: A graph transformer-based approach for drug-target interaction prediction.

The application of Artificial Intelligence (AI) to screen drug molecules with potential therapeutic effects has revolutionized the drug discovery process, with significantly lower economic cost and time consumption than the traditional drug discovery pipeline. With the great power of AI, it is possible to rapidly search the vast chemical space for potential drug-target interactions (DTIs) between candidate drug molecules and disease protein targets. However, only a small proportion of molecules have labelled DTIs, consequently limiting the performance of AI-based drug screening. To solve this problem, a machine learning-based approach with great ability to generalize DTI prediction across molecules is desirable. Many existing machine learning approaches for DTI identification failed to exploit the full information with respect to the topological structures of candidate molecules. To develop a better approach for DTI prediction, we propose GraphormerDTI, which employs the powerful Graph Transformer neural network to model molecular structures. GraphormerDTI embeds molecular graphs into vector-format representations through iterative Transformer-based message passing, which encodes molecules' structural characteristics by node centrality encoding, node spatial encoding and edge encoding. With a strong structural inductive bias, the proposed GraphormerDTI approach can effectively infer informative representations for out-of-sample molecules and as such, it is capable of predicting DTIs across molecules with an exceptional performance. GraphormerDTI integrates the Graph Transformer neural network with a 1-dimensional Convolutional Neural Network (1D-CNN) to extract the drugs' and target proteins' representations and leverages an attention mechanism to model the interactions between them. To examine GraphormerDTI's performance for DTI prediction, we conduct experiments on three benchmark datasets, where GraphormerDTI achieves a superior performance than five state-of-the-art baselines for out-of-molecule DTI prediction, including GNN-CPI, GNN-PT, DeepEmbedding-DTI, MolTrans and HyperAttentionDTI, and is on a par with the best baseline for transductive DTI prediction. The source codes and datasets are publicly accessible at https://github.com/mengmeng34/GraphormerDTI.

Tongue feature dataset construction and real-time detection.

BACKGROUND: Tongue diagnosis in traditional Chinese medicine (TCM) provides clinically important, objective evidence from direct observation of specific features that assist with diagnosis. However, the current interpretation of tongue features requires a significant amount of manpower and time. TCM physicians may have different interpretations of features displayed by the same tongue. An automated interpretation system that interprets tongue features would expedite the interpretation process and yield more consistent results. MATERIALS AND METHODS: This study applied deep learning visualization to tongue diagnosis. After collecting tongue images and corresponding interpretation reports by TCM physicians in a single teaching hospital, various tongue features such as fissures, tooth marks, and different types of coatings were annotated manually with rectangles. These annotated data and images were used to train a deep learning object detection model. Upon completion of training, the position of each tongue feature was dynamically marked. RESULTS: A large high-quality manually annotated tongue feature dataset was constructed and analyzed. A detection model was trained with average precision (AP) 47.67%, 58.94%, 71.25% and 59.78% for fissures, tooth marks, thick and yellow coatings, respectively. At over 40 frames per second on a NVIDIA GeForce GTX 1060, the model was capable of detecting tongue features from any viewpoint in real time. CONCLUSIONS/SIGNIFICANCE: This study constructed a tongue feature dataset and trained a deep learning object detection model to locate tongue features in real time. The model provided interpretability and intuitiveness that are often lacking in general neural network models and implies good feasibility for clinical application.

Classification of Motor Imagery EEG signals using high resolution time-frequency representations and convolutional neural network.

A Motor Imagery (MI) based Brain Computer Interface (BCI) system aims to provide neuro-rehabilitation for the motor disabled people and patients with brain injuries (e.g., stroke patients) etc. The aim of this work is to classify the left and right hand MI tasks by utilizing the occurrence of event related desynchronization and synchronization (ERD\ERS) in the Electroencephalogram (EEG) during these tasks. This study proposes to use a set of Complex Morlet Wavelets (CMW) having frequency dependent widths to generate high-resolution time-frequency representations (TFR) of the MI EEG signals present in the channels C3 and C4. A novel method for the selection of the value of number of cycles relative to the center frequency of the CMW is studied here for extracting the MI task features. The generated TFRs are given as input to a Convolutional neural network (CNN) for classifying them into left or right hand MI tasks. The proposed framework attains a classification accuracy of 82.2% on the BCI Competition IV dataset 2a, showing that the TFRs generated in this work give a higher classification accuracy than the baseline methods and other existing algorithms.

DNA as a universal chemical substrate for computing and data storage.

DNA computing and DNA data storage are emerging fields that are unlocking new possibilities in information technology and diagnostics. These approaches use DNA molecules as a computing substrate or a storage medium, offering nanoscale compactness and operation in unconventional media (including aqueous solutions, water-in-oil microemulsions and self-assembled membranized compartments) for applications beyond traditional silicon-based computing systems. To build a functional DNA computer that can process and store molecular information necessitates the continued development of strategies for computing and data storage, as well as bridging the gap between these fields. In this Review, we explore how DNA can be leveraged in the context of DNA computing with a focus on neural networks and compartmentalized DNA circuits. We also discuss emerging approaches to the storage of data in DNA and associated topics such as the writing, reading, retrieval and post-synthesis editing of DNA-encoded data. Finally, we provide insights into how DNA computing can be integrated with DNA data storage and explore the use of DNA for near-memory computing for future information technology and health analysis applications.

Graph neural networks for clinical risk prediction based on electronic health records: A survey.

OBJECTIVE: This study aims to comprehensively review the use of graph neural networks (GNNs) for clinical risk prediction based on electronic health records (EHRs). The primary goal is to provide an overview of the state-of-the-art of this subject, highlighting ongoing research efforts and identifying existing challenges in developing effective GNNs for improved prediction of clinical risks. METHODS: A search was conducted in the Scopus, PubMed, ACM Digital Library, and Embase databases to identify relevant English-language papers that used GNNs for clinical risk prediction based on EHR data. The study includes original research papers published between January 2009 and May 2023. RESULTS: Following the initial screening process, 50 articles were included in the data collection. A significant increase in publications from 2020 was observed, with most selected papers focusing on diagnosis prediction (n = 36). The study revealed that the graph attention network (GAT) (n = 19) was the most prevalent architecture, and MIMIC-III (n = 23) was the most common data resource. CONCLUSION: GNNs are relevant tools for predicting clinical risk by accounting for the relational aspects among medical events and entities and managing large volumes of EHR data. Future studies in this area may address challenges such as EHR data heterogeneity, multimodality, and model interpretability, aiming to develop more holistic GNN models that can produce more accurate predictions, be effectively implemented in clinical settings, and ultimately improve patient care.

Classification Algorithm for fNIRS-based Brain Signals Using Convolutional Neural Network with Spatiotemporal Feature Extraction Mechanism.

Brain Computer Interface (BCI) is a highly promising human-computer interaction method that can utilize brain signals to control external devices. BCI based on functional near-infrared spectroscopy (fNIRS) is considered a relatively new and promising paradigm. fNIRS is a technique of measuring functional changes in cerebral hemodynamics. It detects changes in the hemodynamic activity of the cerebral cortex by measuring oxyhemoglobin and deoxyhemoglobin (HbR) concentrations and inversely predicts the neural activity of the brain. At the present time, Deep learning (DL) methods have not been widely used in fNIRS decoding, and there are fewer studies considering both spatial and temporal dimensions for fNIRS classification. To solve these problems, we proposed an end-to-end hybrid neural network for feature extraction of fNIRS. The method utilizes a spatial-temporal convolutional layer for automatic extraction of temporally valid information and uses a spatial attention mechanism to extract spatially localized information. A temporal convolutional network (TCN) is used to further utilize the temporal information of fNIRS before the fully connected layer. We validated our approach on a publicly available dataset including 29 subjects, including left-hand and right-hand motor imagery (MI), mental arithmetic (MA), and a baseline task. The results show that the method has few training parameters and high accuracy, providing a meaningful reference for BCI development.

TIRPnet: Risk prediction of traditional Chinese medicine ingredients based on a deep neural network.

ETHNOPHARMACOLOGICAL RELEVANCE: Traditional Chinese medicine (TCM) has a history of over 3000 years of medical practice. Due to the complex ingredients and unclear pharmacological mechanism of TCM, it is very difficult to predict its risks. With the increase in the number and severity of spontaneous reports of adverse drug reactions (ADRs) of TCM, its safety has received widespread attention. AIM OF THE STUDY: In this study, we proposed a framework based on deep learning to predict the probability of adverse reactions caused by TCM ingredients and validated the model using real-world data. MATERIALS AND METHODS: The spontaneous reporting data from Jiangsu Province of China was selected as the research data, which included 72,561 ADR reports of TCMs. All the ingredients of these TCMs were collected from the medical website and correlated with the corresponding ADRs. Then, a risk prediction model was constructed based on a deep neural network (DNN), named TIRPnet. Based on one-hot encoded data, our model achieved the optimal performance by fine-tuning some hyperparameters. The ten most commonly used TCM ingredients and their ADRs were collected as the test set to evaluate their performance as objective criteria. RESULTS: TIRPnet was constructed as a 7-layer DNN. The experimental results showed that TIRPnet performs excellently in all indicators, with a sensitivity of 0.950, specificity of 0.995, accuracy of 0.994, precision of 0.708, and F1 of 0.811. CONCLUSIONS: The proposed TIRPnet owns the ability to predict the ADRs of a single TCM ingredient by learning a large number of TCM-related spontaneous reports, which can help doctors design safe prescriptions and provide technical support for the pharmacovigilance of TCM.

The topology of interpersonal neural network in weak social ties.

The strategies for social interaction between strangers differ from those between acquaintances, whereas the differences in neural basis of social interaction have not been fully elucidated. In this study, we examined the geometrical properties of interpersonal neural networks in pairs of strangers and acquaintances during antiphase joint tapping. Dual electroencephalogram (EEG) of 29 channels per participant was measured from 14 strangers and 13 acquaintance pairs.Intra-brain synchronizations were calculated using the weighted phase lag index (wPLI) for intra-brain electrode combinations, and inter-brain synchronizations were calculated using the phase locking value (PLV) for inter-brain electrode combinations in the theta, alpha, and beta frequency bands. For each participant pair, electrode combinations with larger wPLI/PLV than their surrogates were defined as the edges of the neural networks. We calculated global efficiency, local efficiency, and modularity derived from graph theory for the combined intra- and inter-brain networks of each pair. In the theta band networks, stranger pairs showed larger local efficiency than acquaintance pairs, indicating that the two brains of stranger pairs were more densely connected. Hence, weak social ties require extensive social interactions and result in high efficiency of information transfer between neighbors in neural network.

Improving the geographical origin classification of Radix glycyrrhizae (licorice) through hyperspectral imaging assisted by U-Net fine structure recognition.

Radix glycyrrhizae (licorice) is extensively employed in traditional Chinese medicine, and serves as a crucial raw material in industries such as food and cosmetics. The quality of licorice from different origins varies greatly, so classification of its geographical origin is particularly important. This study proposes a technique for fine structure recognition and segmentation of hyperspectral images of licorice using deep learning U-Net neural networks to segment the tissue structure patterns (phloem, xylem, and pith). Firstly, the three partitions were separately labeled using the Labelme tool, which was utilized to train the U-Net model. Secondly, the obtained optimal U-Net model was applied to predict three partitions of all samples. Lastly, various machine learning models (LDA, SVM, and PLS-DA) were trained based on segmented hyperspectral data. In addition, a threshold method and a circumcircle method were applied to segment licorice hyperspectral images for comparison. The results revealed that compared with the threshold segmentation method (which yielded SVM classifier accuracies of 99.17%, 91.15%, and 92.50% on the training set, validation set, and test set, respectively), the U-Net segmentation method significantly enhanced the accuracy of origin classification (99.06%, 94.72% and 96.07%). Conversely, the circumcircle segmentation method did not effectively improve the accuracy of origin classification (99.65%, 91.16% and 92.13%). By integrating Raman imaging of licorice, it can be inferred that the U-Net model, designed for region segmentation based on the inherent tissue structure of licorice, can effectively improve the accuracy origin classification, which has positive significance in the development of intelligence and information technology of Chinese medicine quality control.

How Integration of a Brain-Machine Interface and Obstacle Detection System Can Improve Wheelchair Control via Movement Imagery.

This study presents a human-computer interaction combined with a brain-machine interface (BMI) and obstacle detection system for remote control of a wheeled robot through movement imagery, providing a potential solution for individuals facing challenges with conventional vehicle operation. The primary focus of this work is the classification of surface EEG signals related to mental activity when envisioning movement and deep relaxation states. Additionally, this work presents a system for obstacle detection based on image processing. The implemented system constitutes a complementary part of the interface. The main contributions of this work include the proposal of a modified 10-20-electrode setup suitable for motor imagery classification, the design of two convolutional neural network (CNNs) models employed to classify signals acquired from sixteen EEG channels, and the implementation of an obstacle detection system based on computer vision integrated with a brain-machine interface. The models developed in this study achieved an accuracy of 83% in classifying EEG signals. The resulting classification outcomes were subsequently utilized to control the movement of a mobile robot. Experimental trials conducted on a designated test track demonstrated real-time control of the robot. The findings indicate the feasibility of integration of the obstacle detection system for collision avoidance with the classification of motor imagery for the purpose of brain-machine interface control of vehicles. The elaborated solution could help paralyzed patients to safely control a wheelchair through EEG and effectively prevent unintended vehicle movements.

A study on pharmaceutical text relationship extraction based on heterogeneous graph neural networks.

Effective information extraction of pharmaceutical texts is of great significance for clinical research. The ancient Chinese medicine text has streamlined sentences and complex semantic relationships, and the textual relationships may exist between heterogeneous entities. The current mainstream relationship extraction model does not take into account the associations between entities and relationships when extracting, resulting in insufficient semantic information to form an effective structured representation. In this paper, we propose a heterogeneous graph neural network relationship extraction model adapted to traditional Chinese medicine (TCM) text. First, the given sentence and predefined relationships are embedded by bidirectional encoder representation from transformers (BERT fine-tuned) word embedding as model input. Second, a heterogeneous graph network is constructed to associate words, phrases, and relationship nodes to obtain the hidden layer representation. Then, in the decoding stage, two-stage subject-object entity identification method is adopted, and the identifier adopts a binary classifier to locate the start and end positions of the TCM entities, identifying all the subject-object entities in the sentence, and finally forming the TCM entity relationship group. Through the experiments on the TCM relationship extraction dataset, the results show that the precision value of the heterogeneous graph neural network embedded with BERT is 86.99% and the F1 value reaches 87.40%, which is improved by 8.83% and 10.21% compared with the relationship extraction models CNN, Bert-CNN, and Graph LSTM.

Multiclass classification of motor imagery tasks based on multi-branch convolutional neural network and temporal convolutional network model.

Motor imagery (MI) is a cognitive process wherein an individual mentally rehearses a specific movement without physically executing it. Recently, MI-based brain-computer interface (BCI) has attracted widespread attention. However, accurate decoding of MI and understanding of neural mechanisms still face huge challenges. These seriously hinder the clinical application and development of BCI systems based on MI. Thus, it is very necessary to develop new methods to decode MI tasks. In this work, we propose a multi-branch convolutional neural network (MBCNN) with a temporal convolutional network (TCN), an end-to-end deep learning framework to decode multi-class MI tasks. We first used MBCNN to capture the MI electroencephalography signals information on temporal and spectral domains through different convolutional kernels. Then, we introduce TCN to extract more discriminative features. The within-subject cross-session strategy is used to validate the classification performance on the dataset of BCI Competition IV-2a. The results showed that we achieved 75.08% average accuracy for 4-class MI task classification, outperforming several state-of-the-art approaches. The proposed MBCNN-TCN-Net framework successfully captures discriminative features and decodes MI tasks effectively, improving the performance of MI-BCIs. Our findings could provide significant potential for improving the clinical application and development of MI-based BCI systems.