2Memristor-1Capacitor Integrated Temporal Kernel for High-Dimensional Data Mapping.

Compact but precise feature-extracting ability is core to processing complex computational tasks in neuromorphic hardware. Physical reservoir computing (RC) offers a robust framework to map temporal data into a high-dimensional space using the time dynamics of a material system, such as a volatile memristor. However, conventional physical RC systems have limited dynamics for the given material properties, restricting the methods to increase their dimensionality. This study proposes an integrated temporal kernel composed of a 2-memristor and 1-capacitor (2M1C) using a W/HfO2/TiN memristor and TiN/ZrO2/Al2O3/ZrO2/TiN capacitor to achieve higher dimensionality and tunable dynamics. The kernel elements are carefully designed and fabricated into an integrated array, of which performances are evaluated under diverse conditions. By optimizing the time dynamics of the 2M1C kernel, each memristor simultaneously extracts complementary information from input signals. The MNIST benchmark digit classification task achieves a high accuracy of 94.3% with a (196×10) single-layer network. Analog input mapping ability is tested with a Mackey-Glass time series prediction, and the system records a normalized root mean square error of 0.04 with a 20×1 readout network, the smallest readout network ever used for Mackey-Glass prediction in RC. These performances demonstrate its high potential for efficient temporal data analysis.

Early prediction of sudden cardiac death risk with Nested LSTM based on electrocardiogram sequential features.

Electrocardiogram (ECG) signals are very important for heart disease diagnosis. In this paper, a novel early prediction method based on Nested Long Short-Term Memory (Nested LSTM) is developed for sudden cardiac death risk detection. First, wavelet denoising and normalization techniques are utilized for reliable reconstruction of ECG signals from extreme noise conditions. Then, a nested LSTM structure is adopted, which can guide the memory forgetting and memory selection of ECG signals, so as to improve the data processing ability and prediction accuracy of ECG signals. To demonstrate the effectiveness of the proposed method, four different models with different signal prediction techniques are used for comparison. The extensive experimental results show that this method can realize an accurate prediction of the cardiac beat's starting point and track the trend of ECG signals effectively. This study holds significant value for timely intervention for patients at risk of sudden cardiac death.

Multiview Spatial-Temporal Meta-Learning for Multivariate Time Series Forecasting.

Multivariate time series modeling has been essential in sensor-based data mining tasks. However, capturing complex dynamics caused by intra-variable (temporal) and inter-variable (spatial) relationships while simultaneously taking into account evolving data distributions is a non-trivial task, which faces accumulated computational overhead and multiple temporal patterns or distribution modes. Most existing methods focus on the former direction without adaptive task-specific learning ability. To this end, we developed a holistic spatial-temporal meta-learning probabilistic inference framework, entitled ST-MeLaPI, for the efficient and versatile learning of complex dynamics. Specifically, first, a multivariate relationship recognition module is utilized to learn task-specific inter-variable dependencies. Then, a multiview meta-learning and probabilistic inference strategy was designed to learn shared parameters while enabling the fast and flexible learning of task-specific parameters for different batches. At the core are spatial dependency-oriented and temporal pattern-oriented meta-learning approximate probabilistic inference modules, which can quickly adapt to changing environments via stochastic neurons at each timestamp. Finally, a gated aggregation scheme is leveraged to realize appropriate information selection for the generative style prediction. We benchmarked our approach against state-of-the-art methods with real-world data. The experimental results demonstrate the superiority of our approach over the baselines.

Constrained Flooding Based on Time Series Prediction and Lightweight GBN in BLE Mesh.

Bluetooth Low Energy Mesh (BLE Mesh) enables Bluetooth flexibility and coverage by introducing Low-Power Nodes (LPNs) and enhanced networking protocol. It is also a commonly used communication method in sensor networks. In BLE Mesh, LPNs are periodically woken to exchange messages in a stop-and-wait way, where the tradeoff between energy and efficiency is a hard problem. Related works have reduced the energy consumption of LPNs mainly in the direction of changing the bearer layer, improving time synchronization and broadcast channel utilization. These algorithms improve communication efficiency; however, they cause energy loss, especially for the LPNs. In this paper, we propose a constrained flooding algorithm based on time series prediction and lightweight GBN (Go-Back-N). On the one hand, the wake-up cycle of the LPNs is determined by the time series prediction of the surrounding load. On the other, LPNs exchange messages through lightweight GBN, which improves the window and ACK mechanisms. Simulation results validate the effectiveness of the Time series Prediction and LlightWeight GBN (TP-LW) algorithm in energy consumption and throughput. Compared with the original algorithm of BLE Mesh, when fewer packets are transmitted, the throughput is increased by 214.71%, and the energy consumption is reduced by 65.14%.

BiLSTM-MLAM: A Multi-Scale Time Series Prediction Model for Sensor Data Based on Bi-LSTM and Local Attention Mechanisms.

This paper introduces BiLSTM-MLAM, a novel multi-scale time series prediction model. Initially, the approach utilizes bidirectional long short-term memory to capture information from both forward and backward directions in time series data. Subsequently, a multi-scale patch segmentation module generates various long sequences composed of equal-length segments, enabling the model to capture data patterns across multiple time scales by adjusting segment lengths. Finally, the local attention mechanism enhances feature extraction by accurately identifying and weighting important time segments, thereby strengthening the model's understanding of the local features of the time series, followed by feature fusion. The model demonstrates outstanding performance in time series prediction tasks by effectively capturing sequence information across various time scales. Experimental validation illustrates the superior performance of BiLSTM-MLAM compared to six baseline methods across multiple datasets. When predicting the remaining life of aircraft engines, BiLSTM-MLAM outperforms the best baseline model by 6.66% in RMSE and 11.50% in MAE. In the LTE dataset, it achieves RMSE improvements of 12.77% and MAE enhancements of 3.06%, while in the load dataset, it demonstrates RMSE enhancements of 17.96% and MAE improvements of 30.39%. Additionally, ablation experiments confirm the positive impact of each module on prediction accuracy. Through segment length parameter tuning experiments, combining different segment lengths has resulted in lower prediction errors, affirming the effectiveness of the multi-scale fusion strategy in enhancing prediction accuracy by integrating information from multiple time scales.

Coordinated Decision Control of Lane-Change and Car-Following for Intelligent Vehicle Based on Time Series Prediction and Deep Reinforcement Learning.

Adaptive cruise control and autonomous lane-change systems represent pivotal advancements in intelligent vehicle technology. To enhance the operational efficiency of intelligent vehicles in combined lane-change and car-following scenarios, we propose a coordinated decision control model based on hierarchical time series prediction and deep reinforcement learning under the influence of multiple surrounding vehicles. Firstly, we analyze the lane-change behavior and establish boundary conditions for safe lane-change, and divide the lane-change trajectory planning problem into longitudinal velocity planning and lateral trajectory planning. LSTM network is introduced to predict the driving states of surrounding vehicles in multi-step time series, combining D3QN algorithm to make decisions on lane-change behavior. Secondly, based on the following state between the ego vehicle and the leader vehicle in the initial lane, as well as the relationship between the initial distance and the expected distance with the leader vehicle in the target lane, with the primary objective of maximizing driving efficiency, longitudinal velocity is planned based on driving conditions recognition. The lateral trajectory and conditions recognition are then planned using the GA-LSTM-BP algorithm. In contrast to conventional adaptive cruise control systems, the DDPG algorithm serves as the lower-level control model for car-following, enabling continuous velocity control. The proposed model is subsequently simulated and validated using the NGSIM dataset and a lane-change scenarios dataset. The results demonstrate that the algorithm facilitates intelligent vehicle lane-change and car-following coordinated control while ensuring safety and stability during lane-changes. Comparative analysis with other decision control models reveals a notable 17.58% increase in driving velocity, underscoring the algorithm's effectiveness in improving driving efficiency.

A Multi-Modal Deep-Learning Air Quality Prediction Method Based on Multi-Station Time-Series Data and Remote-Sensing Images: Case Study of Beijing and Tianjin.

The profound impacts of severe air pollution on human health, ecological balance, and economic stability are undeniable. Precise air quality forecasting stands as a crucial necessity, enabling governmental bodies and vulnerable communities to proactively take essential measures to reduce exposure to detrimental pollutants. Previous research has primarily focused on predicting air quality using only time-series data. However, the importance of remote-sensing image data has received limited attention. This paper proposes a new multi-modal deep-learning model, Res-GCN, which integrates high spatial resolution remote-sensing images and time-series air quality data from multiple stations to forecast future air quality. Res-GCN employs two deep-learning networks, one utilizing the residual network to extract hidden visual information from remote-sensing images, and another using a dynamic spatio-temporal graph convolution network to capture spatio-temporal information from time-series data. By extracting features from two different modalities, improved predictive performance can be achieved. To demonstrate the effectiveness of the proposed model, experiments were conducted on two real-world datasets. The results show that the Res-GCN model effectively extracts multi-modal features, significantly enhancing the accuracy of multi-step predictions. Compared to the best-performing baseline model, the multi-step prediction's mean absolute error, root mean square error, and mean absolute percentage error increased by approximately 6%, 7%, and 7%, respectively.

Multiple feature fusion transformer for modeling penicillin fermentation process with unequal sampling intervals.

The quality prediction of batch processes is an important task in the field of biological fermentation. However, dynamic nonlinearity, unequal sampling intervals, uneven duration, and multiple features of a batch process make this task challenging. Thus, the multiple-feature fusion transformer (MFFT) model is proposed for the time series quality prediction of a batch process. First, the application of sequence-to-sequence architecture enables MFFT to perform a wide range of sequence prediction tasks. Second, the transformer parallel operation model imposes no rigid requirement for the order of sequence input, allowing the model to deal with problems of unequal interval sampling and utilize the sequence information. Third, MFFT integrates a pretrained ResNet50 as a mycelium status classifier for fusing image information into the features. Moreover, a multiple-feature encoding structure is proposed to integrate sampling time and mycelium status. Finally, multiple tasks in penicillin fermentation have shown that MFFT significantly outperforms existing methods for time series prediction.

TransFNN: A Novel Overtemperature Prediction Method for HVDC Converter Valves Based on an Improved Transformer and the F-NN Algorithm.

Appropriate cooling of the converter valve in a high-voltage direct current (HVDC) transmission system is highly significant for the safety, stability, and economical operation of a power grid. The proper adjustment of cooling measures is based on the accurate perception of the valve's future overtemperature state, which is characterized by the valve's cooling water temperature. However, very few previous studies have focused on this need, and the existing Transformer model, which excels in time-series predictions, cannot be directly applied to forecast the valve overtemperature state. In this study, we modified the Transformer and present a hybrid Transformer-FCM-NN (TransFNN) model to predict the future overtemperature state of the converter valve. The TransFNN model decouples the forecast process into two stages: (i) The modified Transformer is used to obtain the future values of the independent parameters; (ii) the relation between the valve cooling water temperature and the six independent operating parameters is fit, and the output of the Transformer is used to calculate the future values of the cooling water temperature. The results of the quantitative experiments showed that the proposed TransFNN model outperformed other models with which it was compared; with TransFNN being applied to predict the overtemperature state of the converter valves, the forecast accuracy was 91.81%, which was improved by 6.85% compared with that of the original Transformer model. Our work provides a novel approach to predicting the valve overtemperature state and acts as a data-driven tool for operation and maintenance personnel to use to adjust valve cooling measures punctually, effectively, and economically.

A Low-Cost, Low-Power, Multisensory Device and Multivariable Time Series Prediction for Beehive Health Monitoring.

We present a custom platform that integrates data from several sensors measuring synchronously different variables of the beehive and wirelessly transmits all measurements to a cloud server. There is a rich literature on beehive monitoring. The choice of our work is not to use ready platforms such as Arduino and Raspberry Pi and to present a low cost and power solution for long term monitoring. We integrate sensors that are not limited to the typical toolbox of beehive monitoring such as gas, vibrations and bee counters. The synchronous sampling of all sensors every 5 min allows us to form a multivariable time series that serves in two ways: (a) it provides immediate alerting in case a measurement exceeds predefined boundaries that are known to characterize a healthy beehive, and (b) based on historical data predict future levels that are correlated with hive's health. Finally, we demonstrate the benefit of using additional regressors in the prediction of the variables of interest. The database, the code and a video of the vibrational activity of two months are made open to the interested readers.

Wind Speed Prediction Based on Error Compensation.

Wind speed prediction is very important in the field of wind power generation technology. It is helpful for increasing the quantity and quality of generated wind power from wind farms. By using univariate wind speed time series, this paper proposes a hybrid wind speed prediction model based on Autoregressive Moving Average-Support Vector Regression (ARMA-SVR) and error compensation. First, to explore the balance between the computation cost and the sufficiency of the input features, the characteristics of ARMA are employed to determine the number of historical wind speeds for the prediction model. According to the selected number of input features, the original data are divided into multiple groups that can be used to train the SVR-based wind speed prediction model. Furthermore, in order to compensate for the time lag introduced by the frequent and sharp fluctuations in natural wind speed, a novel Extreme Learning Machine (ELM)-based error correction technique is developed to decrease the deviations between the predicted wind speed and its real values. By this means, more accurate wind speed prediction results can be obtained. Finally, verification studies are conducted by using real data collected from actual wind farms. Comparison results demonstrate that the proposed method can achieve better prediction results than traditional approaches.

An Interpretable Time Series Data Prediction Framework for Severe Accidents in Nuclear Power Plants.

Accurately predicting severe accident data in nuclear power plants is of utmost importance for ensuring their safety and reliability. However, existing methods often lack interpretability, thereby limiting their utility in decision making. In this paper, we present an interpretable framework, called GRUS, for forecasting severe accident data in nuclear power plants. Our approach combines the GRU model with SHAP analysis, enabling accurate predictions and offering valuable insights into the underlying mechanisms. To begin, we preprocess the data and extract temporal features. Subsequently, we employ the GRU model to generate preliminary predictions. To enhance the interpretability of our framework, we leverage SHAP analysis to assess the contributions of different features and develop a deeper understanding of their impact on the predictions. Finally, we retrain the GRU model using the selected dataset. Through extensive experimentation utilizing breach data from MSLB accidents and LOCAs, we demonstrate the superior performance of our GRUS framework compared to the mainstream GRU, LSTM, and ARIMAX models. Our framework effectively forecasts trends in core parameters during severe accidents, thereby bolstering decision-making capabilities and enabling more effective emergency response strategies in nuclear power plants.

COVID-19 Spatio-Temporal Evolution Using Deep Learning at a European Level.

COVID-19 evolution imposes significant challenges for the European healthcare system. The heterogeneous spread of the pandemic within EU regions elicited a wide range of policies, such as school closure, transport restrictions, etc. However, the implementation of these interventions is not accompanied by the implementation of quantitative methods, which would indicate their effectiveness. As a result, the efficacy of such policies on reducing the spread of the virus varies significantly. This paper investigates the effectiveness of using deep learning paradigms to accurately model the spread of COVID-19. The deep learning approaches proposed in this paper are able to effectively map the temporal evolution of a COVID-19 outbreak, while simultaneously taking into account policy interventions directly into the modelling process. Thus, our approach facilitates data-driven decision making by utilizing previous knowledge to train models that predict not only the spread of COVID-19, but also the effect of specific policy measures on minimizing this spread. Global models at the EU level are proposed, which can be successfully applied at the national level. These models use various inputs in order to successfully model the spatio-temporal variability of the phenomenon and obtain generalization abilities. The proposed models are compared against the traditional epidemiological and Autoregressive Integrated Moving Average (ARIMA) models.

Short-Term Drift Prediction of Multi-Functional Buoys in Inland Rivers Based on Deep Learning.

The multi-functional buoy is an important facility for assisting the navigation of inland waterway ships. Therefore, real-time tracking of its position is an essential process to ensure the safety of ship navigation. Aiming at the problem of the low accuracy of multi-functional buoy drift prediction, an integrated deep learning model incorporating the attention mechanism and ResNet-GRU (RGA) to predict short-term drift values of buoys is proposed. The model has the strong feature expression capability of ResNet and the temporal memory capability of GRU, and the attention mechanism can capture important information adaptively, which can solve the nonlinear time series drift prediction problem well. In this paper, the data collected from multi-functional buoy #4 at Nantong anchorage No. 2 in the Yangtze River waters in China were studied as an example, and first linear interpolation was used for filling in missing values; then, input variables were selected based on Pearson correlation analysis, and finally, the model structure was designed for training and testing. The experimental results show that the mean square error, mean absolute error, root mean square error and mean percentage error of the RGA model on the test set are 5.113036, 1.609969, 2.261202 and 15.575886, respectively, which are significantly better than other models. This study provides a new idea for predicting the short-term drift of multi-functional buoys, which is helpful for their tracking and management.

Power Line Communication and Sensing Using Time Series Forecasting.

Smart electrical grids rely on data communication to support their operation and on sensing for diagnostics and maintenance. Usually, the roles of communication and sensing equipment are different, i.e., communication equipment does not participate in sensing tasks and vice versa. Power line communication (PLC) offers a cost-effective solution for joint communication and sensing for smart grids. This is because the high-frequency PLC signals used for data communication also reveal detailed information regarding the health of the power lines that they travel through. Traditional PLC-based power line or cable diagnostic solutions are dependent on prior knowledge of the cable type, network topology, and/or characteristics of the anomalies. In this paper, we develop a power line sensing technique that can detect various types of cable anomalies without any prior domain knowledge. To this end, we design a solution that first uses time-series forecasting to predict the PLC channel state information at any given point in time based on its historical data. Under the approximation that the prediction error follows a Gaussian distribution, we then perform chi-squared statistical test to build an anomaly detector which identifies the occurrence of a cable fault. We demonstrate the effectiveness and universality of our sensing solution via evaluations conducted using both synthetic and real-world data extracted from low- and medium-voltage distribution networks.

Sensor Data Prediction in Missile Flight Tests.

Sensor data from missile flights are highly valuable, as a test requires considerable resources, but some sensors may be detached or fail to collect data. Remotely acquired missile sensor data are incomplete, and the correlations between the missile data are complex, which results in the prediction of sensor data being difficult. This article proposes a deep learning-based prediction network combined with the wavelet analysis method. The proposed network includes an imputer network and a prediction network. In the imputer network, the data are decomposed using wavelet transform, and the generative adversarial networks assist the decomposed data in reproducing the detailed information. The prediction network consists of long short-term memory with an attention and dilation network for accurate prediction. In the test, the actual sensor data from missile flights were used. For the performance evaluation, the test was conducted from the data with no missing values to the data with five different missing rates. The test results showed that the proposed system predicts the missile sensor most accurately in all cases. In the frequency analysis, the proposed system has similar frequency responses to the actual sensors and showed that the proposed system accurately predicted the sensors in both tendency and frequency aspects.

Visibility graph for time series prediction and image classification: a review.

The analysis of time series and images is significant across different fields due to their widespread applications. In the past few decades, many approaches have been developed, including data-driven artificial intelligence methods, mechanism-driven physical methods, and hybrid mechanism and data-driven models. Complex networks have been used to model numerous complex systems due to its characteristics, including time series prediction and image classification. In order to map time series and images into complex networks, many visibility graph algorithms have been developed, such as horizontal visibility graph, limited penetrable visibility graph, multiplex visibility graph, and image visibility graph. The family of visibility graph algorithms will construct different types of complex networks, including (un-) weighted, (un-) directed, and (single-) multi-layered networks, thereby focusing on different kinds of properties. Different types of visibility graph algorithms will be reviewed in this paper. Through exploring the topological structure and information in the network based on statistical physics, the property of time series and images can be discovered. In order to forecast (multivariate) time series, several variations of local random walk algorithms and different information fusion approaches are applied to measure the similarity between nodes in the network. Different forecasting frameworks are also proposed to consider the information in the time series based on the similarity. In order to classify the image, several machine learning models (such as support vector machine and linear discriminant) are used to classify images based on global features, local features, and multiplex features. Through various simulations on a variety of datasets, researchers have found that the visibility graph algorithm outperformed existing algorithms, both in time series prediction and image classification. Clearly, complex networks are closely connected with time series and images by visibility graph algorithms, rendering complex networks to be an important tool for understanding the characteristics of time series and images. Finally, we conclude in the last section with future outlooks for the visibility graph.

Analysis of the socioeconomic impact due to COVID-19 using a deep clustering approach.

One of the main problems that countries are currently having is being able to measure the impact of the pandemic in other areas of society (for example, economic or social). In that sense, being able to combine variables about the behavior of COVID-19 with other variables in the environment, to build models about its impact, which help the decision-making of national authorities, is a current challenge. In this sense, this work proposes an approach that allows monitoring the socioeconomic behavior of the regions/departments of a country (in this case, Colombia) due to the effect of COVID-19. To do this, an approach is proposed in which the behavior of the infected is initially predicted, and together with other context variables (climate, economics and socials) determines the current socioeconomic situation of a region. This classification of a region, with the pattern that characterizes it, is a fundamental input for those who make decisions. Thus, this work presents an approach based on machine learning techniques to identify regions with similar socioeconomic behaviors due to COVID-19, so they should eventually have similar public policies. The proposed hybrid model initially consists of a time series prediction model of infected, to which are added several context variables (climate, socioeconomic, incidence of COVID-19 at the level of deaths, suspects, etc.) in an unsupervised learning model, to determine the socioeconomic impact in the regions. Particularly, the unsupervised model groups similar regions together, and the pattern of each group describes the socioeconomic similarities between them, to help decision-makers in the process of defining policies to be implemented in the regions. The experiments showed the ability of the hybrid model to follow the evolution of the regions after 4 weeks. The quality metrics for the predictive model were around the values of 0.35 for MAPE and 0.68 for R 2 , and in the case of the clustering model were around the values of 0.3 for the Silhouette index and 0.6 for the Davies-Boulding index. The hybrid model allowed determining things like some regions that initially belonged to a group with a very low incidence of positive cases and very unfavorable socioeconomic conditions, became part of groups with moderately high incidences. Our preliminary results are very satisfactory since they allow studying the evolution of the socioeconomic impact in each region/department.

Interval type-3 fuzzy aggregators for ensembles of neural networks in COVID-19 time series prediction.

In this work we are presenting an approach for fuzzy aggregation in ensembles of neural networks for forecasting. The aggregator is used in an ensemble to combine the outputs of the networks forming the ensemble. This is done in such a way that the total output of the ensemble is better than the outputs of the individual modules. In our approach a fuzzy system is used to estimate the weights that will be assigned to the outputs in the process of combining them in a weighted average calculation. The uncertainty in the process of aggregation is modeled with interval type-3 fuzzy, which in theory can outperform type-2 and type-1. Publicly available data sets of COVID-19 cases for several countries in the world were utilized to test the proposed approach. Simulation results of the COVID-19 data show the potential of the approach to outperform other aggregators in the literature.

Prediction of Time Series Gene Expression and Structural Analysis of Gene Regulatory Networks Using Recurrent Neural Networks.

Methods for time series prediction and classification of gene regulatory networks (GRNs) from gene expression data have been treated separately so far. The recent emergence of attention-based recurrent neural network (RNN) models boosted the interpretability of RNN parameters, making them appealing for the understanding of gene interactions. In this work, we generated synthetic time series gene expression data from a range of archetypal GRNs and we relied on a dual attention RNN to predict the gene temporal dynamics. We show that the prediction is extremely accurate for GRNs with different architectures. Next, we focused on the attention mechanism of the RNN and, using tools from graph theory, we found that its graph properties allow one to hierarchically distinguish different architectures of the GRN. We show that the GRN responded differently to the addition of noise in the prediction by the RNN and we related the noise response to the analysis of the attention mechanism. In conclusion, this work provides a way to understand and exploit the attention mechanism of RNNs and it paves the way to RNN-based methods for time series prediction and inference of GRNs from gene expression data.