Research on performance variations of classifiers with the influence of pre-processing methods for Chinese short text classification.

Text pre-processing is an important component of a Chinese text classification. At present, however, most of the studies on this topic focus on exploring the influence of preprocessing methods on a few text classification algorithms using English text. In this paper we experimentally compared fifteen commonly used classifiers on two Chinese datasets using three widely used Chinese preprocessing methods that include word segmentation, Chinese specific stop word removal, and Chinese specific symbol removal. We then explored the influence of the preprocessing methods on the final classifications according to various conditions such as classification evaluation, combination style, and classifier selection. Finally, we conducted a battery of various additional experiments, and found that most of the classifiers improved in performance after proper preprocessing was applied. Our general conclusion is that the systematic use of preprocessing methods can have a positive impact on the classification of Chinese short text, using classification evaluation such as macro-F1, combination of preprocessing methods such as word segmentation, Chinese specific stop word and symbol removal, and classifier selection such as machine and deep learning models. We find that the best macro-f1s for categorizing text for the two datasets are 92.13% and 91.99%, which represent improvements of 0.3% and 2%, respectively over the compared baselines.

MD-GNN: A mechanism-data-driven graph neural network for molecular properties prediction and new material discovery.

Molecular properties prediction and new material discovery are significant for the pharmaceutical industry, food, chemistry, and other fields. The popular methods are theoretical mechanism calculation and machine learning. There is a deviation between the theoretical mechanism calculation results and the experimental data. Machine learning method provides a promising solution. However, the process is lack of interpretability, and the reliability and the generalization depend on the training data. In this paper, a mechanism correction model combined with graph neural network (GNN) model which is based on the fusion of graph embedding and descriptors vector is proposed as backbone network to proceed molecule properties prediction and new material discovery. The molecular structure is input to graph neural network and the abstracted features are fused with numerical features together for training. The experiment data and computing data are designed as label constructor, and then the theoretical computation (mechanism driven model) is fused with the output of GNN (data-driven model) to form a fused model to modulate the output for the molecular property prediction. Experiments for public data set are executed and the results show that Mechanism-Data-Driven Graph Neural Network (MD-GNN) can effectively make the predicted results more accurate. Nineteen molecules by different construction are designed for potential drug discovery, the prediction from the proposed MD-GNN model shows that there are 9 candidates are discovered.

Spatiotemporal and kinematic characteristics augmentation using Dual-GAN for ankle instability detection.

Obtaining massive amounts of training data is often crucial for computer-assisted diagnosis using deep learning. Unfortunately, patient data is often small due to varied constraints. We develop a new approach to extract significant features from a small clinical gait analysis dataset to improve computer-assisted diagnosis of Chronic Ankle Instability (CAI) patients. In this paper, we present an approach for augmenting spatiotemporal and kinematic characteristics using the Dual Generative Adversarial Networks (Dual-GAN) to train a series of modified Long Short-Term Memory (LSTM) detection models making the training process more data-efficient. Namely, we use LSTM-, LSTM-Fully Convolutional Networks (FCN)-, and Convolutional LSTM-based detection models to identify the patients with CAI. The Dual-GAN enables the synthesized data to approximate the real data distribution visualized by the t-distributed Stochastic Neighbor Embedding (t-SNE) algorithm. Then we trained the proposed detection models using real data collected from a controlled laboratory study and mixed data from real and synthesized gait features. The detection models were tested in real data to validate the positive role in data augmentation as well as to demonstrate the capability and effectiveness of the modified LSTM algorithm for CAI detection using spatiotemporal and kinematic characteristics in walking. Dual-GAN generated efficient spatiotemporal and kinematic characteristics to augment the training set promoting the performance of CAI detection and the modified LSTM algorithm yielded an enhanced classification outcome to identify those CAI patients from a group of control subjects based on gait analysis data than any previous reports.

Investigating Multi-Level Semantic Extraction with Squash Capsules for Short Text Classification.

At present, short text classification is a hot topic in the area of natural language processing. Due to the sparseness and irregularity of short text, the task of short text classification still faces great challenges. In this paper, we propose a new classification model from the aspects of short text representation, global feature extraction and local feature extraction. We use convolutional networks to extract shallow features from short text vectorization, and introduce a multi-level semantic extraction framework. It uses BiLSTM as the encoding layer while the attention mechanism and normalization are used as the interaction layer. Finally, we concatenate the convolution feature vector and semantic results of the semantic framework. After several rounds of feature integration, the framework improves the quality of the feature representation. Combined with the capsule network, we obtain high-level local information by dynamic routing and then squash them. In addition, we explore the optimal depth of semantic feature extraction for short text based on a multi-level semantic framework. We utilized four benchmark datasets to demonstrate that our model provides comparable results. The experimental results show that the accuracy of SUBJ, TREC, MR and ProcCons are 93.8%, 91.94%, 82.81% and 98.43%, respectively, which verifies that our model has greatly improves classification accuracy and model robustness.

An Improved Multitask Learning Model with Matching Network and Its Application in Traditional Chinese Medicine Syndrome Recommendation.

Multitask learning (MTL) is an open and challenging problem in various real-world applications, such as recommendation systems, natural language processing, and computer vision. The typical way of conducting multitask learning is establishing some global parameter sharing mechanism among all tasks or assigning each task an individual set of parameters with cross-connections between tasks. However, for most existing approaches, the raw features are abstracted step by step, semantic information is mined from input space, and matching relation features are not introduced into the model. To solve the above problems, we propose a novel MMOE-match network to model the matches between medical cases and syndrome elements and introduce the recommendation algorithm into traditional Chinese medicine (TCM) study. Accurate medical record recommendation is significant for intelligent medical treatment. Ranking algorithms can be introduced in multi-TCM scenarios, such as syndrome element recommendation, symptom recommendation, and drug prescription recommendation. The recommendation system includes two main stages: recalling and ranking. The core of recalling and ranking is a two-tower matching network and multitask learning. MMOE-match combines the advantages of recalling and ranking model to design a new network. Furtherly, we try to take the matching network output as the input of multitask learning and compare the matching features designed by the manual. The data show that our model can bring significant positive benefits.

Ankle-Injury Patients Perform More Microadjustments during Walking: Evidence from Velocity Profiles in Gait Analysis.

INTRODUCTION: We evaluated the velocity profiles of patients with lateral collateral ligament (LCL) injuries of the ankle with a goal of understanding the control mechanism involved in walking. METHODS: We tracked motions of patients' legs and feet in 30 gait cycles recorded from patients with LCL injuries of the ankle and compared them to 50 gait cycles taken from normal control subjects. Seventeen markers were placed on the foot following the Heidelberg foot measurement model. Velocity profiles and microadjustments of the knee, ankle, and foot were calculated during different gait phases and compared between the patient and control groups. RESULTS: Patients had a smaller first rocker percentage and larger second rocker percentage in the gait cycle compared to controls. Patients also displayed shorter stride length and slower strides and performed more microadjustments in the second rocker phase than in other rocker/swing phases. Patients' mean velocities of the knee, ankle, and foot in the second rocker phase were also significantly higher than that in control subjects. Discussion. Evidence from velocity profiles suggested that patients with ligament injury necessitated more musculoskeletal microadjustments to maintain body balance, but these may also be due to secondary injury. Precise descriptions of the spatiotemporal gait characteristics are therefore crucial for our understanding of movement control during locomotion.

A Multitask Learning Model with Multiperspective Attention and Its Application in Recommendation.

Training models to predict click and order targets at the same time. For better user satisfaction and business effectiveness, multitask learning is one of the most important methods in e-commerce. Some existing researches model user representation based on historical behaviour sequence to capture user interests. It is often the case that user interests may change from their past routines. However, multi-perspective attention has broad horizon, which covers different characteristics of human reasoning, emotions, perception, attention, and memory. In this paper, we attempt to introduce the multi-perspective attention and sequence behaviour into multitask learning. Our proposed method offers better understanding of user interest and decision. To achieve more flexible parameter sharing and maintaining the special feature advantage of each task, we improve the attention mechanism at the view of expert interactive. To the best of our knowledge, we firstly propose the implicit interaction mode, the explicit hard interaction mode, the explicit soft interaction mode, and the data fusion mode in multitask learning. We do experiments on public data and lab medical data. The results show that our model consistently achieves remarkable improvements to the state-of-the-art method.

Sentiment classification for employees reviews using regression vector- stochastic gradient descent classifier (RV-SGDC).

The satisfaction of employees is very important for any organization to make sufficient progress in production and to achieve its goals. Organizations try to keep their employees satisfied by making their policies according to employees' demands which help to create a good environment for the collective. For this reason, it is beneficial for organizations to perform staff satisfaction surveys to be analyzed, allowing them to gauge the levels of satisfaction among employees. Sentiment analysis is an approach that can assist in this regard as it categorizes sentiments of reviews into positive and negative results. In this study, we perform experiments for the world's big six companies and classify their employees' reviews based on their sentiments. For this, we proposed an approach using lexicon-based and machine learning based techniques. Firstly, we extracted the sentiments of employees from text reviews and labeled the dataset as positive and negative using TextBlob. Then we proposed a hybrid/voting model named Regression Vector-Stochastic Gradient Descent Classifier (RV-SGDC) for sentiment classification. RV-SGDC is a combination of logistic regression, support vector machines, and stochastic gradient descent. We combined these models under a majority voting criteria. We also used other machine learning models in the performance comparison of RV-SGDC. Further, three feature extraction techniques: term frequency-inverse document frequency (TF-IDF), bag of words, and global vectors are used to train learning models. We evaluated the performance of all models in terms of accuracy, precision, recall, and F1 score. The results revealed that RV-SGDC outperforms with a 0.97 accuracy score using the TF-IDF feature due to its hybrid architecture.

Short Text Paraphrase Identification Model Based on RDN-MESIM.

In the rapid development of various technologies at the present stage, representative artificial intelligence technology has developed more prominently. Therefore, it has been widely applied in various social service areas. The application of artificial intelligence technology in tax consultation can optimize the application scenarios and update the application mode, thus further improving the efficiency and quality of tax data inquiry. In this paper, we propose a novel model, named RDN-MESIM, for paraphrase identification tasks in the tax consulting area. The main contribution of this work is designing the RNN-Dense network and modifying the original ESIM to adapt to the RDN structure. The results demonstrate that RDN-MESIM obtained a better performance as compared to other existing relevant models and archived the highest accuracy, of up to 97.63%.

A Feature Fusion Method with Guided Training for Classification Tasks.

In this paper, a feature fusion method with guiding training (FGT-Net) is constructed to fuse image data and numerical data for some specific recognition tasks which cannot be classified accurately only according to images. The proposed structure is divided into the shared weight network part, the feature fused layer part, and the classification layer part. First, the guided training method is proposed to optimize the training process, the representative images and training images are input into the shared weight network to learn the ability that extracts the image features better, and then the image features and numerical features are fused together in the feature fused layer to input into the classification layer for the classification task. Experiments are carried out to verify the effectiveness of the proposed model. Loss is calculated by the output of both the shared weight network and classification layer. The results of experiments show that the proposed FGT-Net achieves the accuracy of 87.8%, which is 15% higher than the CNN model of ShuffleNetv2 (which can process image data only) and 9.8% higher than the DNN method (which processes structured data only).

Multiscale Road Extraction in Remote Sensing Images.

Recent advances in convolutional neural networks (CNNs) have shown impressive results in semantic segmentation. Among the successful CNN-based methods, U-Net has achieved exciting performance. In this paper, we proposed a novel network architecture based on U-Net and atrous spatial pyramid pooling (ASPP) to deal with the road extraction task in the remote sensing field. On the one hand, U-Net structure can effectively extract valuable features; on the other hand, ASPP is able to utilize multiscale context information in remote sensing images. Compared to the baseline, this proposed model has improved the pixelwise mean Intersection over Union (mIoU) of 3 points. Experimental results show that the proposed network architecture can deal with different types of road surface extraction tasks under various terrains in Yinchuan city, solve the road connectivity problem to some extent, and has certain tolerance to shadows and occlusion.

Dynamic scene stitching driven by visual cognition model.

Dynamic scene stitching still has a great challenge in maintaining the global key information without missing or deforming if multiple motion interferences exist in the image acquisition system. Object clips, motion blurs, or other synthetic defects easily occur in the final stitching image. In our research work, we proceed from human visual cognitive mechanism and construct a hybrid-saliency-based cognitive model to automatically guide the video volume stitching. The model consists of three elements of different visual stimuli, that is, intensity, edge contour, and scene depth saliencies. Combined with the manifold-based mosaicing framework, dynamic scene stitching is formulated as a cut path optimization problem in a constructed space-time graph. The cutting energy function for column width selections is defined according to the proposed visual cognition model. The optimum cut path can minimize the cognitive saliency difference throughout the whole video volume. The experimental results show that it can effectively avoid synthetic defects caused by different motion interferences and summarize the key contents of the scene without loss. The proposed method gives full play to the role of human visual cognitive mechanism for the stitching. It is of high practical value to environmental surveillance and other applications.