A city-wide examination of fine-grained human emotions through social media analysis.

The proliferation of Social Media and Open Web data has provided researchers with a unique opportunity to better understand human behavior at different levels. In this paper, we show how data from Open Street Map and Twitter could be analyzed and used to portray detailed Human Emotions at a city wide level in two cities, San Francisco and London. Neural Network classifiers for fine-grained emotions were developed, tested and used to detect emotions from tweets in the two cites. The detected emotions were then matched to key locations extracted from Open Street Map. Through an analysis of the resulting data set, we highlight the effect different days, locations and POI neighborhoods have on the expression of human emotions in the cities.

Exploring the use of AI text-to-image generation to downregulate negative emotions in an expressive writing application.

Conventional writing therapies are versatile, accessible and easy to facilitate online, but often require participants to self-disclose traumatic experiences. To make expressive writing therapies safer for online, unsupervised environments, we explored the use of text-to-image generation as a means to downregulate negative emotions during a fictional writing exercise. We developed a writing tool, StoryWriter, that uses Generative Adversarial Network models to generate artwork from users' narratives in real time. These images were intended to positively distract users from their negative emotions throughout the writing task. In this paper, we report the outcomes of two user studies: Study 1 (N = 388), which experimentally examined the efficacy of this application via negative versus neutral emotion induction and image generation versus no image generation control groups; and Study 2 (N = 54), which qualitatively examined open-ended feedback. Our results are heterogeneous: both studies suggested that StoryWriter somewhat contributed to improved emotion outcomes for participants with pre-existing negative emotions, but users' open-ended responses indicated that these outcomes may be adversely modulated by the generated images, which could undermine the therapeutic benefits of the writing task itself.

Electroencephalogram signals emotion recognition based on convolutional neural network-recurrent neural network framework with channel-temporal attention mechanism for older adults.

Reminiscence and conversation between older adults and younger volunteers using past photographs are very effective in improving the emotional state of older adults and alleviating depression. However, we need to evaluate the emotional state of the older adult while conversing on the past photographs. While electroencephalogram (EEG) has a significantly stronger association with emotion than other physiological signals, the challenge is to eliminate muscle artifacts in the EEG during speech as well as to reduce the number of dry electrodes to improve user comfort while maintaining high emotion recognition accuracy. Therefore, we proposed the CTA-CNN-Bi-LSTM emotion recognition framework. EEG signals of eight channels (P3, P4, F3, F4, F7, F8, T7, and T8) were first implemented in the MEMD-CCA method on three brain regions separately (Frontal, Temporal, Parietal) to remove the muscle artifacts then were fed into the Channel-Temporal attention module to get the weights of channels and temporal points most relevant to the positive, negative and neutral emotions to recode the EEG data. A Convolutional Neural Networks (CNNs) module then extracted the spatial information in the new EEG data to obtain the spatial feature maps which were then sequentially inputted into a Bi-LSTM module to learn the bi-directional temporal information for emotion recognition. Finally, we designed four group experiments to demonstrate that the proposed CTA-CNN-Bi-LSTM framework outperforms the previous works. And the highest average recognition accuracy of the positive, negative, and neutral emotions achieved 98.75%.

An EMG-based Eating Behaviour Monitoring system with haptic feedback to promote mindful eating.

Mindless eating, or the lack of awareness of the food we are consuming, has been linked to health problems attributed to unhealthy eating behaviour, including obesity. Traditional approaches used to moderate eating behaviour often rely on inaccurate self-logging, manual observations or bulky equipment. Overall, there is a clear unmet clinical need to develop an intelligent and lightweight system which can automatically monitor eating behaviour and provide feedback. In this paper, we investigate: i) the development of an automated system for detecting eating behaviour using wearable Electromyography (EMG) sensors, and ii) the application of the proposed system combined with real-time wristband haptic feedback to facilitate mindful eating. For this, the collected data from 16 participants were used to develop an algorithm for detecting chewing and swallowing. We extracted 18 features from EMG which were presented to different classifiers, to develop a system enabling participants to self-moderate their chewing behaviour using haptic feedback. An additional experimental study was conducted with 20 further participants to evaluate the effectiveness of eating monitoring and haptic interface in promoting mindful eating. We used a standard validation scheme with a leave-one-participant-out to assess model performance using standard metrics (F1-score). The proposed algorithm automatically assessed eating behaviour accurately using the EMG-extracted features and a Support Vector Machine (SVM): F1-Score = 0.95 for chewing classification, and F1-Score = 0.87 for swallowing classification. The experimental study showed that participants exhibited a lower rate of chewing when haptic feedback was delivered in the form of wristband vibration, compared to a baseline and non-haptic condition (F (2,38) = 58.243, p < .001). These findings may have major implications for research in eating behaviour, providing key insights into the impact of automatic chewing detection and haptic feedback systems on moderating eating behaviour towards improving health outcomes.

Game Design in Mental Health Care: Case Study-Based Framework for Integrating Game Design Into Therapeutic Content.

While there has been increasing interest in the use of gamification in mental health care, there is a lack of design knowledge on how elements from games could be integrated into existing therapeutic treatment activities in a manner that is balanced and effective. To help address this issue, we propose a design process framework to support the development of mental health gamification. Based on the concept of experienced game versus therapy worlds, we highlight 4 different therapeutic components that could be gamified to increase user engagement. By means of a Dual-Loop model, designers can balance the therapeutic and game design components and design the core elements of a mental health care gamification. To support the proposed framework, 4 cases of game design in mental health care (eg, therapeutic protocols for addiction, anxiety, and low self-esteem) are presented.

A Library of Old Photos Supporting Conversation of Two Generations Serving Reminiscence Therapy.

In Japan, a shift in family patterns has led to a sense of social isolation among older people, which increases the risk of major neurocognitive disorder. Interventions for them using old photos to implement reminiscence therapy (RT) have been proved to be effective. A super-aged society has in turn led to a shortage of medical resources and older people prefer home care over institutional care. Therefore, there is an urgent need for volunteers to help in RT. However, the age of volunteers tends to be increasingly younger. The lack of knowledge and experience of the past for the young volunteers makes it difficult for them to select appropriate stimulated materials. To improve this situation, a library of old photos for RT was developed to support conversation between the two generations. A two-factor experiment and emotion assessment scales were designed to explore the effect of different old photo types on the fluency of conversation between the two generations and their emotion. It was found that the types of old photos have little effect on older people and that conversations were almost pleasant. However, the pleasantness of older people was enhanced when using photos that they wanted to talk about (P = 0.006). Meanwhile, pleasure in conversation of the older people increased with the attention of the young people to the topic (R = 0.304, p < 0.001). Conversely, photo type has a strong impact on young people. When photos are selected that older people do not want to talk about or photos that young people do not know the content and are not interested in, concern for the topic of young people drops dramatically. Therefore, when RT, it is important to avoid using the types of photos above that cause a drop in younger people's attention.

Wireless Soft Scalp Electronics and Virtual Reality System for Motor Imagery-Based Brain-Machine Interfaces.

Motor imagery offers an excellent opportunity as a stimulus-free paradigm for brain-machine interfaces. Conventional electroencephalography (EEG) for motor imagery requires a hair cap with multiple wired electrodes and messy gels, causing motion artifacts. Here, a wireless scalp electronic system with virtual reality for real-time, continuous classification of motor imagery brain signals is introduced. This low-profile, portable system integrates imperceptible microneedle electrodes and soft wireless circuits. Virtual reality addresses subject variance in detectable EEG response to motor imagery by providing clear, consistent visuals and instant biofeedback. The wearable soft system offers advantageous contact surface area and reduced electrode impedance density, resulting in significantly enhanced EEG signals and classification accuracy. The combination with convolutional neural network-machine learning provides a real-time, continuous motor imagery-based brain-machine interface. With four human subjects, the scalp electronic system offers a high classification accuracy (93.22 ± 1.33% for four classes), allowing wireless, real-time control of a virtual reality game.

Heart sound recognition technology based on convolutional neural network.

The mortality rate of heart disease continues to rise each year: developing mechanisms to reduce mortality from heart disease is a top concern in today's society. Heart sound auscultation is a crucial skill used to detect and diagnose heart disease. In this study, we propose a heart sound signal classification algorithm based on a convolutional neural network. The algorithm is based on heart sound data collected in the clinic and from medical books. The heart sound signals were first preprocessed into a grayscale image of 5 seconds. The training samples were then used to train and optimize the convolutional neural network; obtaining a training result with an accuracy of 95.17% and a loss value of 0.23. Finally, the convolutional neural network was used to test the test set samples. The results showed an accuracy of 94.80%, sensitivity of 94.29%, specificity of 95.54%, precision of 93.44%, F1_score of 93.84%, and an AUC of 0.943. Compared with other algorithms, the accuracy and sensitivity of the algorithms were improved. This shows that the method used in this study can effectively classify heart sound signals and could prove useful in assisting heart sound auscultation.

Emotion Recognition Using Electroencephalography Signals of Older People for Reminiscence Therapy.

Objective: Numerous communication support systems based on reminiscence therapy have been developed. However, when using communication support systems, the emotional assessment of older people is generally conducted using verbal feedback or questionnaires. The purpose of this study is to investigate the feasibility of using Electroencephalography (EEG) signals for automatic emotion recognition during RT for older people. Participants: Eleven older people (mean 71.25, SD 4.66) and seven young people (mean 22.4, SD 1.51) participated in the experiment. Methods: Old public photographs were used as material for reminiscence therapy. The EEG signals of the older people were collected while the older people and young people were talking about the contents of the photos. Since emotions change slowly and responses are characterized by delayed effects in EEG, the depth models LSTM and Bi-LSTM were selected to extract complex emotional features from EEG signals for automatic recognition of emotions. Results: The EEG data of 8 channels were inputted into the LSTM and Bi-LSTM models to classify positive and negative emotions. The recognition highest accuracy rate of the two models were 90.8% and 95.8% respectively. The four-channel EEG data based Bi-LSTM also reached 94.4%. Conclusion: Since the Bi-LSTM model could tap into the influence of "past" and "future" emotional states on the current emotional state in the EEG signal, we found that it can help improve the ability to recognize positive and negative emotions in older people. In particular, it is feasible to use EEG signals without the necessity of multimodal physiological signals for emotion recognition in the communication support systems for reminiscence therapy when using this model.

Believing Is Seeing: A Proof-of-Concept Semiexperimental Study on Using Mobile Virtual Reality to Boost the Effects of Interpretation Bias Modification for Anxiety.

BACKGROUND: Cognitive Bias Modification of Interpretations (CBM-I) is a computerized intervention designed to change negatively biased interpretations of ambiguous information, which underlie and reinforce anxiety. The repetitive and monotonous features of CBM-I can negatively impact training adherence and learning processes. OBJECTIVE: This proof-of-concept study aimed to examine whether performing a CBM-I training using mobile virtual reality technology (virtual reality Cognitive Bias Modification of Interpretations [VR-CBM-I]) improves training experience and effectiveness. METHODS: A total of 42 students high in trait anxiety completed 1 session of either VR-CBM-I or standard CBM-I training for performance anxiety. Participants' feelings of immersion and presence, emotional reactivity to a stressor, and changes in interpretation bias and state anxiety, were assessed. RESULTS: The VR-CBM-I resulted in greater feelings of presence (P<.001, d=1.47) and immersion (P<.001, ηp2=0.74) in the training scenarios and outperformed the standard training in effects on state anxiety (P<.001, ηp2=0.3) and emotional reactivity to a stressor (P=.03, ηp2=0.12). Both training varieties successfully increased the endorsement of positive interpretations (P<.001, drepeated measures [drm]=0.79) and decreased negative ones. (P<.001, drm=0.72). In addition, changes in the emotional outcomes were correlated with greater feelings of immersion and presence. CONCLUSIONS: This study provided first evidence that (1) the putative working principles underlying CBM-I trainings can be translated into a virtual environment and (2) virtual reality holds promise as a tool to boost the effects of CMB-I training for highly anxious individuals while increasing users' experience with the training application.

Investigating the use of sensor-based IoET to facilitate learning for children in rural Thailand.

A novel sensor-based Internet of Educational Things (IoET) platform named OBSY was iteratively designed, developed and evaluated to support education in rural regions in Thailand. To assess the effectiveness of this platform, a study was carried out at four primary schools located near the Thai northern border with 244 students and 8 teachers. Participants were asked to carry out three science-based learning activities and were measured for improvements in learning outcome and learning engagement. Overall, the results showed that students in the IoET group who had used OBSY to learn showed significantly higher learning outcome and had better learning engagement than those in the control condition. In addition, for those in the IoET group, there was no significant effect regarding gender, home location (Urban or Rural), age, prior experience with technology and ethnicity on learning outcome. For learning engagement, only age was found to influence interest/enjoyment. The study demonstrated the potential of IoET technologies in underprivileged area, through a co-design approach with teachers and students, taking into account the local contexts.