Enhancing deep neural network training efficiency and performance through linear prediction.

Deep neural networks have achieved remarkable success in various fields. However, training an effective deep neural network still poses challenges. This paper aims to propose a method to optimize the training effectiveness of deep neural networks, with the goal of improving their performance. Firstly, based on the observation that parameters (weights and bias) of deep neural network change in certain rules during training process, the potential of parameters prediction for improving training efficiency is discovered. Secondly, the potential of parameters prediction to improve the performance of deep neural network by noise injection introduced by prediction errors is revealed. And then, considering the limitations comprehensively, a deep neural network Parameters Linear Prediction method is exploit. Finally, performance and hyperparameter sensitivity validations are carried out on some representative backbones. Experimental results show that by employing proposed Parameters Linear Prediction method, as opposed to SGD, has led to an approximate 1% increase in accuracy for optimal model, along with a reduction of about 0.01 in top-1/top-5 error. Moreover, it also exhibits stable performance under various hyperparameter settings, shown the effectiveness of the proposed method and validated its capacity in enhancing network's training efficiency and performance.

Recovery time reduction to decrease experimental duration (R2D2): A simple and universal method to accelerate NMR experiments.

Acceleration techniques for one dimensional Nuclear Magnetic Resonance (1D NMR) are very useful, both for NMR enthusiasts and for chemists that use NMR for structural elucidation. To the latter, such techniques need to be straightforward. Recovery time Reduction to Decrease the experimental Duration (R2D2) relies on the incremental reduction of a pulse sequence's Recycle Time (TR). A pseudo-2D spectrum is acquired and after two Fourier transform, extraction and addition of the central rows, a 1D spectrum is obtained. Not only can it be applied to any pulse sequence that contains a TR, but it also requires only a list of recovery times and 2D processes to operate. With this method, we were able to easily reduce the experimental time by a factor of 2 and up to 4 using single-pulse, APT and DEPT 13C sequences. Moreover, R2D2 has the potential to be used on other low abundance nuclei (such as 15N or 2H) and numerous other pulse sequences.

A predictor-corrector phase unwrapping algorithm for temporally undersampled gradient-echo MRI.

PURPOSE: To develop a method for unwrapping temporally undersampled and nonlinear gradient recalled echo (GRE) phase. THEORY AND METHODS: Temporal unwrapping is performed as a sequential one step prediction of the echo phase, followed by a correction to the nearest integer wrap-count. A spatio-temporal extension of the 1D predictor corrector unwrapping (PCU) algorithm improves the prediction accuracy, and thereby maintains spatial continuity. The proposed method is evaluated using numerical phantom, physical phantom, and in vivo brain data at both 3 T and 9.4 T. The unwrapping performance is compared with the state-of-the-art temporal and spatial unwrapping algorithms, and the spatio-temporal iterative virtual-echo based Nyquist sampled (iVENyS) algorithm. RESULTS: Simulation results showed significant reduction in unwrapping errors at higher echoes compared with the state-of-the-art algorithms. Similar to the iVENyS algorithm, the PCU algorithm was able to generate spatially smooth phase images for in vivo data acquired at 3 T and 9.4 T, bypassing the use of additional spatial unwrapping step. A key advantage over iVENyS algorithm is the superior performance of PCU algorithm at higher echoes. CONCLUSION: PCU algorithm serves as a robust phase unwrapping method for temporally undersampled and nonlinear GRE phase, particularly in the presence of high field gradients.

An improved linear prediction evolution algorithm based on topological opposition-based learning for optimization.

Prediction-based evolutionary algorithm is one of the emerging category of meta-heuristic optimization techniques. The improved linear prediction evolution algorithm (ILPE) is a recently developed meta-heuristic optimization technique that draws inspiration from non-linear least-square fitting models. This article implements the concept of topological opposition-based learning, which was first applied in grey prediction evolutionary algorithms to the ILPE. In traditional evolutionary algorithms, after employing the mutation and crossover operator, it generates trial populations. The proposed algorithm constructs a new reproduction operator using the non-linear least square fitting model with topological opposition-based learning to generate trial individuals. This reproduction operator considers the population series as a time series and uses the topological opposition-based non-linear least square fitting model to predict the next generation of populations. The efficiency and accuracy of the algorithm are demonstrated through numerical experiments on CEC2014 and CEC2017 benchmark functions. The results of these experiments show that the proposed algorithm is highly effective in solving optimization problems.•An improved linear prediction evolution algorithm based on topological opposition based learning (TILPE) is proposed.•The proposed strategy treat the the population series as a time series.•To validate the efficacy of TILPE, CEC2014 and CEC2017 benchmark functions are used.

Parameter Estimation in Spectral Resolution Enhancement Based on Forward-Backward Linear Prediction Total Least Square Method.

In a Fourier transform infrared (IR) spectrometer, the Michelson interference signal extrapolation method based on linear prediction is often used to improve spectral resolution. In this method, an autoregressive (AR) model is established for the Michelson interference signal in the spectrometer. Once the AR model parameters are determined, the AR process is predictable. The interference signal can be used to figure out the AR model's parameters. Based on this, the AR model can be used to extrapolate the interference signal to improve the spectral resolution. In this paper, the forward-backward linear prediction total least squares (FB-TLS) method is proposed to estimate the parameters of the AR model. The parameters that are estimated are used to improve the IR spectral resolution. By simulating different order and signal-to-noise ratio situations, the effects of the Burg, the least square, and the FB-TLS parameter estimation methods on spectral resolution enhancement are studied. The simulation results demonstrate that the FB-TLS parameter estimation method can effectively suppress noise and avoid spurious peaks. The experimental results demonstrate that the FB-TLS parameter estimation method is effective for spectral resolution enhancement technology based on linear prediction. When the FB-TLS method is used to enhance NH3 IR spectral resolution from 2 cm-1 to 1 cm-1, the spectral prediction error in the NH3 characteristic band is only 0.21% compared with the measured NH3 spectrum, whose spectral resolution is 1 cm-1.

Signal Acquisition-Independent Lossless Electrocardiogram Compression Using Adaptive Linear Prediction.

In this paper, we propose a lossless electrocardiogram (ECG) compression method using a prediction error-based adaptive linear prediction technique. This method combines the adaptive linear prediction, which minimizes the prediction error in the ECG signal prediction, and the modified Golomb-Rice coding, which encodes the prediction error to the binary code as the compressed data. We used the PTB Diagnostic ECG database, the European ST-T database, and the MIT-BIH Arrhythmia database for the evaluation and achieved the average compression ratios for single-lead ECG signals of 3.16, 3.75, and 3.52, respectively, despite different signal acquisition setup in each database. As the prediction order is very crucial for this particular problem, we also investigate the validity of the popular linear prediction coefficients that are generally used in ECG compression by determining the prediction coefficients from the three databases using the autocorrelation method. The findings are in agreement with the previous works in that the second-order linear prediction is suitable for the ECG compression application.

Lossless Image Coding Using Non-MMSE Algorithms to Calculate Linear Prediction Coefficients.

This paper presents a lossless image compression method with a fast decoding time and flexible adjustment of coder parameters affecting its implementation complexity. A comparison of several approaches for computing non-MMSE prediction coefficients with different levels of complexity was made. The data modeling stage of the proposed codec was based on linear (calculated by the non-MMSE method) and non-linear (complemented by a context-dependent constant component removal block) predictions. Prediction error coding uses a two-stage compression: an adaptive Golomb code and a binary arithmetic code. The proposed solution results in 30% shorter decoding times and a lower bit average than competing solutions (by 7.9% relative to the popular JPEG-LS codec).

A Current Noise Cancellation Method Based on Fractional Linear Prediction for Bearing Fault Detection.

The stator current in an induction motor contains a large amount of information, which is unrelated to bearing faults. This information is considered as the noise component for the detection of bearing faults. When there is noise information in the current signal, it can affect the detection of motor bearing faults and lead to the possibility of false alarms. Therefore, to accomplish an effective bearing fault detection, all or some of these noise components must be properly eliminated. This paper proposes the use of fractional linear prediction (FLP) as a noise elimination method in bearing fault diagnosis, which makes these noise components the predictable components and this bearing fault information the unpredictable components. The basis of the FLP is to eliminate noise components in the current signal by predicting predictable components through linear prediction theory and optimal prediction order. Meanwhile, this paper adopts the FLP model with limited memory samples. After determining the optimal number of memories, only the fractional derivative order parameter needs to be optimized, which greatly reduces the computational complexity and difficulty in parameter optimization. In addition, this paper uses spectral analysis of the current signals through experimental simulation to compare the FLP method with the linear prediction (LP) method and the time-shifting (TS) method that have been successfully applied to bearing fault diagnosis. Based on the analysis results, the FLP method can better extract fault features and achieve better bearing fault diagnosis results, verifying the effectiveness and superiority of the FLP method in the field of bearing fault diagnosis. Additionally, the predictive performance of thevFLP and LP was compared based on experimental data, verifying the advantages of the FLP method in predictive performance, indicating that this method has a better noise cancellation effect.

Detection of Neurogenic Voice Disorders Using the Fisher Vector Representation of Cepstral Features.

Neurogenic voice disorders (NVDs) are caused by damage or malfunction of the central or peripheral nervous system that controls vocal fold movement. In this paper, we investigate the potential of the Fisher vector (FV) encoding in automatic detection of people with NVDs. FVs are used to convert features from frame level (local descriptors) to utterance level (global descriptors). At the frame level, we extract two popular cepstral representations, namely, Mel-frequency cepstral coefficients (MFCCs) and perceptual linear prediction cepstral coefficients (PLPCCs), from acoustic voice signals. In addition, the MFCC features are also extracted from every frame of the glottal source signal computed using a glottal inverse filtering (GIF) technique. The global descriptors derived from the local descriptors are used to train a support vector machine (SVM) classifier. Experiments are conducted using voice signals from 80 healthy speakers and 80 patients with NVDs (40 with spasmodic dysphonia (SD) and 40 with recurrent laryngeal nerve palsy (RLNP)) taken from the Saarbruecken voice disorder (SVD) database. The overall results indicate that the use of the FV encoding leads to better identification of people with NVDs, compared to the defacto temporal encoding. Furthermore, the SVM trained using the combination of FVs derived from the cepstral and glottal features provides the overall best detection performance.

Denoise ultra-low-field 3D magnetic resonance images using a joint signal-image domain filter.

Ultra-low-field magnetic resonance imaging (MRI) could suffer from heavy uncorrelated noise, and its removal could be a critical post-processing task. As a primary source of interference, Gaussian noise could corrupt the sampled MR signal (k-space data), especially at lower B0 field strength. For this reason, we consider both signal and image domains by proposing a new joint filter characterized by a Kalman filter with linear prediction and a nonlocal mean filter with higher-order singular value decomposition (HOSVD) for denoising 3D MR data. The Kalman filter first attenuates the noise in k-space, and then its reconstruction images are used to guide HOSVD denoising process with exploring self-similarity among 3D structures. The clearer prefiltered images could also generate improved HOSVD learned bases used to transform the noise corrupted patch groups in the original MR data. The flexibility of proposed method is also demonstrated by integrating other k-space filters into the algorithm scheme. Experimental data includes simulated MR images with the varying noise level and real MR images obtained from our 50 mT MRI scanner. The results reveal that our method has a better noise-removal ability and introduces lesser unexpected artifacts than other related MRI denoising approaches.

Dance emotion recognition based on linear predictive Meir frequency cepstrum coefficient and bidirectional long short-term memory from robot environment.

Dance emotion recognition is an important research direction of automatic speech recognition, especially in the robot environment. It is an important research content of dance emotion recognition to extract the features that best represent speech emotion and to construct an acoustic model with strong robustness and generalization. The dance emotion data set is small in size and high in dimension. The traditional recurrent neural network (RNN) has the problem of long-range dependence disappearance, and due to the focus on local information of convolutional neural network (CNN), the mining of potential relationships between frames in the input sequence is insufficient. To solve the above problems, this paper proposes a novel linear predictive Meir frequency cepstrum coefficient and bidirectional long short-term memory (LSTM) for dance emotion recognition. In this paper, the linear prediction coefficient (LPC) and Meier frequency cepstrum coefficient (MFCC) are combined to obtain a new feature, namely the linear prediction Meier frequency cepstrum coefficient (LPMFCC). Then, the combined feature obtained by combining LPMFCC with energy feature is used as the extracted dance feature. The extracted features are input into the bidirectional LSTM network for training. Finally, support vector machine (SVM) is used to classify the obtained features through the full connection layer. Finally, we conduct experiments on public data sets and obtain the better effectiveness compared with the state-of-art dance motion recognition methods.

Global and local feature fusion via long and short-term memory mechanism for dance emotion recognition in robot.

In recent years, there are more and more intelligent machines in people's life, such as intelligent wristbands, sweeping robots, intelligent learning machines and so on, which can simply complete a single execution task. We want robots to be as emotional as humans. In this way, human-computer interaction can be more natural, smooth and intelligent. Therefore, emotion research has become a hot topic that researchers pay close attention to. In this paper, we propose a new dance emotion recognition based on global and local feature fusion method. If the single feature of audio is extracted, the global information of dance cannot be reflected. And the dimension of data features is very high. In this paper, an improved long and short-term memory (LSTM) method is used to extract global dance information. Linear prediction coefficient is used to extract local information. Considering the complementarity of different features, a global and local feature fusion method based on discriminant multi-canonical correlation analysis is proposed in this paper. Experimental results on public data sets show that the proposed method can effectively identify dance emotion compared with other state-of-the-art emotion recognition methods.

Fusion-Based Versatile Video Coding Intra Prediction Algorithm with Template Matching and Linear Prediction.

The new generation video coding standard Versatile Video Coding (VVC) has adopted many novel technologies to improve compression performance, and consequently, remarkable results have been achieved. In practical applications, less data, in terms of bitrate, would reduce the burden of the sensors and improve their performance. Hence, to further enhance the intra compression performance of VVC, we propose a fusion-based intra prediction algorithm in this paper. Specifically, to better predict areas with similar texture information, we propose a fusion-based adaptive template matching method, which directly takes the error between reference and objective templates into account. Furthermore, to better utilize the correlation between reference pixels and the pixels to be predicted, we propose a fusion-based linear prediction method, which can compensate for the deficiency of single linear prediction. We implemented our algorithm on top of the VVC Test Model (VTM) 9.1. When compared with the VVC, our proposed fusion-based algorithm saves a bitrate of 0.89%, 0.84%, and 0.90% on average for the Y, Cb, and Cr components, respectively. In addition, when compared with some other existing works, our algorithm showed superior performance in bitrate savings.

Pairwise and higher-order measures of brain-heart interactions in children with temporal lobe epilepsy.

Objective.While it is well-known that epilepsy has a clear impact on the activity of both the central nervous system (CNS) and the autonomic nervous system (ANS), its role on the complex interplay between CNS and ANS has not been fully elucidated yet. In this work, pairwise and higher-order predictability measures based on the concepts of Granger Causality (GC) and partial information decomposition (PID) were applied on time series of electroencephalographic (EEG) brain wave amplitude and heart rate variability (HRV) in order to investigate directed brain-heart interactions associated with the occurrence of focal epilepsy.Approach.HRV and the envelopes ofδandαEEG activity recorded from ipsilateral (ipsi-EEG) and contralateral (contra-EEG) scalp regions were analyzed in 18 children suffering from temporal lobe epilepsy monitored during pre-ictal, ictal and post-ictal periods. After linear parametric model identification, we compared pairwise GC measures computed between HRV and a single EEG component with PID measures quantifying the unique, redundant and synergistic information transferred from ipsi-EEG and contra-EEG to HRV.Main results.The analysis of GC revealed a dominance of the information transfer from EEG to HRV and negligible transfer from HRV to EEG, suggesting that CNS activities drive the ANS modulation of the heart rhythm, but did not evidence clear differences betweenδandαrhythms, ipsi-EEG and contra-EEG, or pre- and post-ictal periods. On the contrary, PID revealed that epileptic seizures induce a reorganization of the interactions from brain to heart, as the unique predictability of HRV originated from the ipsi-EEG for theδwaves and from the contra-EEG for theαwaves in the pre-ictal phase, while these patterns were reversed after the seizure.Significance.These results highlight the importance of considering higher-order interactions elicited by PID for the study of the neuro-autonomic effects of focal epilepsy, and may have neurophysiological and clinical implications.

Blind Speech Watermarking Method with Frame Self-Synchronization Based on Spread-Spectrum Using Linear Prediction Residue.

State-of-the-art speech watermarking techniques enable speech signals to be authenticated and protected against any malicious attack to ensure secure speech communication. In general, reliable speech watermarking methods must satisfy four requirements: inaudibility, robustness, blind-detectability, and confidentiality. We previously proposed a method of non-blind speech watermarking based on direct spread spectrum (DSS) using a linear prediction (LP) scheme to solve the first two issues (inaudibility and robustness) due to distortion by spread spectrum. This method not only effectively embeds watermarks with small distortion but also has the same robustness as the DSS method. There are, however, two remaining issues with blind-detectability and confidentiality. In this work, we attempt to resolve these issues by developing an approach called the LP-DSS scheme, which takes two forms of data embedding for blind detection and frame synchronization. We incorporate blind detection with frame synchronization into the scheme to satisfy blind-detectability and incorporate two forms of data embedding process, front-side and back-side embedding for blind detection and frame synchronization, to satisfy confidentiality. We evaluated these improved processes by carrying out four objective tests (PESQ, LSD, Bit-error-rate, and accuracy of frame synchronization) to determine whether inaudibility and blind-detectability could be satisfied. We also evaluated all combinations with the two forms of data embedding for blind detection with frame synchronization by carrying out BER tests to determine whether confidentiality could be satisfied. Finally, we comparatively evaluated the proposed method by carrying out ten robustness tests against various processing and attacks. Our findings showed that an inaudible, robust, blindly detectable, and confidential speech watermarking method based on the proposed LP-DSS scheme could be achieved.

An Urdu speech corpus for emotion recognition.

Emotion recognition from acoustic signals plays a vital role in the field of audio and speech processing. Speech interfaces offer humans an informal and comfortable means to communicate with machines. Emotion recognition from speech signals has a variety of applications in the area of human computer interaction (HCI) and human behavior analysis. In this work, we develop the first emotional speech database of the Urdu language. We also develop the system to classify five different emotions: sadness, happiness, neutral, disgust, and anger using different machine learning algorithms. The Mel Frequency Cepstrum Coefficient (MFCC), Linear Prediction Coefficient (LPC), energy, spectral flux, spectral centroid, spectral roll-off, and zero-crossing were used as speech descriptors. The classification tests were performed on the emotional speech corpus collected from 20 different subjects. To evaluate the quality of speech emotions, subjective listing tests were conducted. The recognition of correctly classified emotions in the complete Urdu emotional speech corpus was 66.5% with K-nearest neighbors. It was found that the disgust emotion has a lower recognition rate as compared to the other emotions. Removing the disgust emotion significantly improves the performance of the classifier to 76.5%.

Singing Voice Detection: A Survey.

Singing voice detection or vocal detection is a classification task that determines whether there is a singing voice in a given audio segment. This process is a crucial preprocessing step that can be used to improve the performance of other tasks such as automatic lyrics alignment, singing melody transcription, singing voice separation, vocal melody extraction, and many more. This paper presents a survey on the techniques of singing voice detection with a deep focus on state-of-the-art algorithms such as convolutional LSTM and GRU-RNN. It illustrates a comparison between existing methods for singing voice detection, mainly based on the Jamendo and RWC datasets. Long-term recurrent convolutional networks have reached impressive results on public datasets. The main goal of the present paper is to investigate both classical and state-of-the-art approaches to singing voice detection.

Running Speed Estimation Using Shoe-Worn Inertial Sensors: Direct Integration, Linear, and Personalized Model.

The overground speed is a key component of running analysis. Today, most speed estimation wearable systems are based on GNSS technology. However, these devices can suffer from sparse communication with the satellites and have a high-power consumption. In this study, we propose three different approaches to estimate the overground speed in running based on foot-worn inertial sensors and compare the results against a reference GNSS system. First, a method is proposed by direct strapdown integration of the foot acceleration. Second, a feature-based linear model and finally a personalized online-model based on the recursive least squares' method were devised. We also evaluated the performance differences between two sets of features; one automatically selected set (i.e., optimized) and a set of features based on the existing literature. The data set of this study was recorded in a real-world setting, with 33 healthy individuals running at low, preferred, and high speed. The direct estimation of the running speed achieved an inter-subject mean ± STD accuracy of 0.08 ± 0.1 m/s and a precision of 0.16 ± 0.04 m/s. In comparison, the best feature-based linear model achieved 0.00 ± 0.11 m/s accuracy and 0.11 ± 0.05 m/s precision, while the personalized model obtained a 0.00 ± 0.01 m/s accuracy and 0.09 ± 0.06 m/s precision. The results of this study suggest that (1) the direct estimation of the velocity of the foot are biased, and the error is affected by the overground velocity and the slope; (2) the main limitation of a general linear model is the relatively high inter-subject variance of the bias, which reflects the intrinsic differences in gait patterns among individuals; (3) this inter-subject variance can be nulled using a personalized model.

Voice Disorder Identification by using Hilbert-Huang Transform (HHT) and K Nearest Neighbor (KNN).

OBJECTIVES: Clinical evaluation of dysphonic voices involves a multidimensional approach, including a variety of instrumental and noninstrumental measures. Acoustic analyses provide an objective, noninvasive and intelligent measures of voice quality. Based on sound recordings, this paper proposes a new classification method of voice disorders with HHT and KNN. METHODS: In this research, 12 features of each sample is calculated by HHT. Based on the algorithm of Linear Prediction Coefficient (LPCC), a sample can be characterized by 9 features. After each sample is expressed by 21 features, the classifier is constructed based on KNN. In addition, classifier based on KNN was further compared with random forest and extra trees classifiers in relation to their classification performance of voice disorder. RESULTS: The experiment results revel that classifier based on KNN showed better performance than other two classifiers with accuracy rate of 93.3%, precision of 93%, recall rate of 95%, F1-score of 94% and the area of receiver operating characteristic curve is 0.976. CONCLUSIONS: The method put forward in this paper can be effectively used to classify voice disorders.

Information Transfer in Linear Multivariate Processes Assessed through Penalized Regression Techniques: Validation and Application to Physiological Networks.

The framework of information dynamics allows the dissection of the information processed in a network of multiple interacting dynamical systems into meaningful elements of computation that quantify the information generated in a target system, stored in it, transferred to it from one or more source systems, and modified in a synergistic or redundant way. The concepts of information transfer and modification have been recently formulated in the context of linear parametric modeling of vector stochastic processes, linking them to the notion of Granger causality and providing efficient tools for their computation based on the state-space (SS) representation of vector autoregressive (VAR) models. Despite their high computational reliability these tools still suffer from estimation problems which emerge, in the case of low ratio between data points available and the number of time series, when VAR identification is performed via the standard ordinary least squares (OLS). In this work we propose to replace the OLS with penalized regression performed through the Least Absolute Shrinkage and Selection Operator (LASSO), prior to computation of the measures of information transfer and information modification. First, simulating networks of several coupled Gaussian systems with complex interactions, we show that the LASSO regression allows, also in conditions of data paucity, to accurately reconstruct both the underlying network topology and the expected patterns of information transfer. Then we apply the proposed VAR-SS-LASSO approach to a challenging application context, i.e., the study of the physiological network of brain and peripheral interactions probed in humans under different conditions of rest and mental stress. Our results, which document the possibility to extract physiologically plausible patterns of interaction between the cardiovascular, respiratory and brain wave amplitudes, open the way to the use of our new analysis tools to explore the emerging field of Network Physiology in several practical applications.