Affect and stress detection based on feature fusion of LSTM and 1DCNN.

The impact of emotions on health, especially stress, is receiving increasing attention. It is important to provide a non-invasive affect detection system that can be continuously monitored for a long period of time. Multi-sensor fusion strategies can better improve the performance of affect detection models, but there are also problems such as insufficient feature extraction and poor spatiotemporal feature fusion. Therefore, this study proposes a feature-level fusion method based on long short-term memory and one-dimensional convolutional neural network to extract temporal and spatial features of electrocardiogram, electromyogram, electrical activity, temperature, accelerator and response data, respectively, and then fuse them in a summation fashion for affect and stress detection. In particular, we added the tanh activation function before feature fusion, which can improve the model's performance. We used the wearable affect and stress detection dataset to train the model, which includes three different emotion states (neutral, stress, and amusement) for three-class emotion classification with accuracy and F1-scores of 87.82% and 86.68%, respectively. Due to the importance of stress, we also studied binary classification for stress detection, where neutral and amusement were combined as non-stress, with accuracy and F1-scores of 94.9% and 94.98%, respectively. The performance of the proposed model outperforms other control models and can effectively improve the performance of affect and stress detection, and promote medical care, health care and elderly care.

A pre-impact fall detection data segmentation method based on multi-channel convolutional neural network and class activation mapping.

Objective.A segmentation method for pre-impact fall detection data is investigated. Specifically, it studies how to partition data segments that are important for classification from continuous inertial sensor data for pre-impact fall detection.Approach.In this study, a trigger-based algorithm combining multi-channel convolutional neural network (CNN) and class activation mapping was proposed to solve the problem of data segmentation. First, a pre-impact fall detection training dataset was established and divided into two parts. For falls, the 1 s data was divided from the peak value of the acceleration signal magnitude vector to the starting direction. For activities of daily living, the cycle segmentation was performed for a 1 s window size. Second, a heat map of the class activation regions of the sensor data was formed using a multi-channel CNN and a class activation mapping algorithm. Finally, the data segmentation strategy was established based on the heat map, the basic law of falls and the real-time requirements.Main results.This method was verified by the SisFall dataset. The obtained segmentation strategy (i.e. to start segmenting a small data segment with a window duration of 325 ms when the acceleration signal magnitude vector is less than 9.217 m s-2) met the real-time requirements for pre-impact fall detection. Moreover, it was suitable for various machine learning algorithms, and the accuracy of the machine learning algorithms used exceeded 94.8%, with the machine learning algorithms verifying the data segmentation strategy.Significance.The proposed method can automatically identify the class activation area, save the computing resources of wearable devices, shorten the duration of segmentation window, and ensure the real-time performance of pre-impact fall detection.