Attention-Based Models for Multivariate Time Series Forecasting: Multi-step Solar Irradiation Prediction.

Bangladesh's subtropical climate with an abundance of sunlight throughout the greater portion of the year results in increased effectiveness of solar panels. Solar irradiance forecasting is an essential aspect of grid-connected photovoltaic systems to efficiently manage solar power's variation and uncertainty and to assist in balancing power supply and demand. This is why it is essential to forecast solar irradiation accurately. Many meteorological factors influence solar irradiation, which has a high degree of fluctuation and uncertainty. Predicting solar irradiance multiple steps ahead makes it difficult for forecasting models to capture long-term sequential relationships. Attention-based models are widely used in the field of Natural Language Processing for their ability to learn long-term dependencies within sequential data. In this paper, our aim is to present an attention-based model framework for multivariate time series forecasting. Using data from two different locations in Bangladesh with a resolution of 30 min, the Attention-based encoder-decoder, Transformer, and Temporal Fusion Transformer (TFT) models are trained and tested to predict over 24 steps ahead and compared with other forecasting models. According to our findings, adding the attention mechanism significantly increased prediction accuracy and TFT has shown to be more precise than the rest of the algorithms in terms of accuracy and robustness. The obtained mean square error (MSE), the mean absolute error (MAE), and the coefficient of determination (R2) values for TFT are 0.151, 0.212, and 0.815, respectively. In comparison to the benchmark and sequential models (including the Naive, MLP, and Encoder-Decoder models), TFT has a reduction in the MSE and MAE of 8.4-47.9% and 6.1-22.3%, respectively, while R2 is raised by 2.13-26.16%. The ability to incorporate long-distance dependency increases the predictive power of attention models.

Forecasting environmental water availability of lakes using temporal fusion transformer: case studies of China's two largest freshwater lakes.

Preserving lacustrine ecosystems is vital for sustainable watershed development, and forecasting the environmental water availability of lakes would support policymakers in developing sound management strategies. This study proposed a methodology that merges the lake water level prediction and environmental water availability evaluation. The temporal fusion transformer (TFT) model forecasted the lake water levels for the next 7 days by inputting the streamflow and lake water level data for the past 30 days. The environmental water availability was assessed by comparing the forecasted lake water levels with the environmental water requirements, resulting in adequate, regular, scarce, and severely scarce environmental water availability. The methodology was tested in two case studies: Poyang Lake and Dongting Lake, the two largest freshwater lakes in the Yangtze River Basin, China. The TFT model performed well in forecasting the lake water levels, as shown by the high coefficient of determination and finite root mean square error. The coefficients of determination exceeded 0.98 during the model training, validation, and test for both Poyang Lake and Dongting Lake, and the root mean square errors ranged from 0.06 to 0.46 m. The accurate prediction of lake water level promoted the precise forecasting of the environmental water availability with the high Kappa coefficient exceeding 0.90. Results indicated the rationality and effectiveness of integrating the lake water level prediction and environmental water availability evaluation. Future research includes the applicability of the TFT model to other lakes worldwide to test the proposed approach and investigate strategies to cope with environmental water scarcity.

Machine Fault Detection Using a Hybrid CNN-LSTM Attention-Based Model.

The predictive maintenance of electrical machines is a critical issue for companies, as it can greatly reduce maintenance costs, increase efficiency, and minimize downtime. In this paper, the issue of predicting electrical machine failures by predicting possible anomalies in the data is addressed through time series analysis. The time series data are from a sensor attached to an electrical machine (motor) measuring vibration variations in three axes: X (axial), Y (radial), and Z (radial X). The dataset is used to train a hybrid convolutional neural network with long short-term memory (CNN-LSTM) architecture. By employing quantile regression at the network output, the proposed approach aims to manage the uncertainties present in the data. The application of the hybrid CNN-LSTM attention-based model, combined with the use of quantile regression to capture uncertainties, yielded superior results compared to traditional reference models. These results can benefit companies by optimizing their maintenance schedules and improving the overall performance of their electric machines.

Machine learning for quantile regression of biogas production rates in anaerobic digesters.

Anaerobic digestion is a well-established tool at wastewater treatment plants for processing raw sludge; it can also be used to generate renewable energy by harvesting biogas in anaerobic digesters. Operational parameters, such as temperature, are usually set by plant operators according to expert knowledge. To completely utilize the potential of operational management, in this study, we calibrated a novel Temporal Fusion Transformer based on six years of life-scale time series data together with categorical features such as public holidays. The model design allows for the interpretability of the output in contrast to traditional data-driven techniques, using multi-head attention. In addition to forecasting the median biogas production rates for the following seven days, our model also yields quantiles, making it less prone to strong fluctuations. We used three well-known statistical techniques as benchmarks. The mean absolute percentage error of our forecasting approach is below 8 %.