Evaluating Ensemble Learning Methods for Multi-Modal Emotion Recognition Using Sensor Data Fusion.

Automatic recognition of human emotions is not a trivial process. There are many factors affecting emotions internally and externally. Expressing emotions could also be performed in many ways such as text, speech, body gestures or even physiologically by physiological body responses. Emotion detection enables many applications such as adaptive user interfaces, interactive games, and human robot interaction and many more. The availability of advanced technologies such as mobiles, sensors, and data analytics tools led to the ability to collect data from various sources, which enabled researchers to predict human emotions accurately. Most current research uses them in the lab experiments for data collection. In this work, we use direct and real time sensor data to construct a subject-independent (generic) multi-modal emotion prediction model. This research integrates both on-body physiological markers, surrounding sensory data, and emotion measurements to achieve the following goals: (1) Collecting a multi-modal data set including environmental, body responses, and emotions. (2) Creating subject-independent Predictive models of emotional states based on fusing environmental and physiological variables. (3) Assessing ensemble learning methods and comparing their performance for creating a generic subject-independent model for emotion recognition with high accuracy and comparing the results with previous similar research. To achieve that, we conducted a real-world study "in the wild" with physiological and mobile sensors. Collecting the data-set is coming from participants walking around Minia university campus to create accurate predictive models. Various ensemble learning models (Bagging, Boosting, and Stacking) have been used, combining the following base algorithms (K Nearest Neighbor KNN, Decision Tree DT, Random Forest RF, and Support Vector Machine SVM) as base learners and DT as a meta-classifier. The results showed that, the ensemble stacking learner technique gave the best accuracy of 98.2% compared with other variants of ensemble learning methods. On the contrary, bagging and boosting methods gave (96.4%) and (96.6%) accuracy levels respectively.

A COVID-19-Based Modified Epidemiological Model and Technological Approaches to Help Vulnerable Individuals Emerge from the Lockdown in the UK.

COVID-19 has shown a relatively low case fatality rate in young healthy individuals, with the majority of this group being asymptomatic or having mild symptoms. However, the severity of the disease among the elderly as well as in individuals with underlying health conditions has caused significant mortality rates worldwide. Understanding this variance amongst different sectors of society and modelling this will enable the different levels of risk to be determined to enable strategies to be applied to different groups. Long-established compartmental epidemiological models like SIR and SEIR do not account for the variability encountered in the severity of the SARS-CoV-2 disease across different population groups. The objective of this study is to investigate how a reduction in the exposure of vulnerable individuals to COVID-19 can minimise the number of deaths caused by the disease, using the UK as a case study. To overcome the limitation of long-established compartmental epidemiological models, it is proposed that a modified model, namely SEIR-v, through which the population is separated into two groups regarding their vulnerability to SARS-CoV-2 is applied. This enables the analysis of the spread of the epidemic when different contention measures are applied to different groups in society regarding their vulnerability to the disease. A Monte Carlo simulation (100,000 runs) along the proposed SEIR-v model is used to study the number of deaths which could be avoided as a function of the decrease in the exposure of vulnerable individuals to the disease. The results indicate a large number of deaths could be avoided by a slight realistic decrease in the exposure of vulnerable groups to the disease. The mean values across the simulations indicate 3681 and 7460 lives could be saved when such exposure is reduced by 10% and 20% respectively. From the encouraging results of the modelling a number of mechanisms are proposed to limit the exposure of vulnerable individuals to the disease. One option could be the provision of a wristband to vulnerable people and those without a smartphone and contact-tracing app, filling the gap created by systems relying on smartphone apps only. By combining very dense contact tracing data from smartphone apps and wristband signals with information about infection status and symptoms, vulnerable people can be protected and kept safer.

DigitalExposome: quantifying impact of urban environment on wellbeing using sensor fusion and deep learning.

The increasing level of air pollutants (e.g. particulates, noise and gases) within the atmosphere are impacting mental wellbeing. In this paper, we define the term 'DigitalExposome' as a conceptual framework that takes us closer towards understanding the relationship between environment, personal characteristics, behaviour and wellbeing using multimodal mobile sensing technology. Specifically, we simultaneously collected (for the first time) multi-sensor data including urban environmental factors (e.g. air pollution including: Particulate Matter (PM1), (PM2.5), (PM10), Oxidised, Reduced, Ammonia (NH3) and Noise, People Count in the vicinity), body reaction (physiological reactions including: EDA, HR, HRV, Body Temperature, BVP and movement) and individuals' perceived responses (e.g. self-reported valence) in urban settings. Our users followed a pre-specified urban path and collected the data using a comprehensive sensing edge device. The data is instantly fused, time-stamped and geo-tagged at the point of collection. A range of multivariate statistical analysis techniques have been applied including Principle Component Analysis, Regression and Spatial Visualisations to unravel the relationship between the variables. Results showed that Electrodermal Activity (EDA) and Heart Rate Variability (HRV) are noticeably impacted by the level of Particulate Matter in the environment. Furthermore, we adopted Convolutional Neural Network (CNN) to classify self-reported wellbeing from the multimodal dataset which achieved an f1-score of 0.76.

An EMG-based Eating Behaviour Monitoring system with haptic feedback to promote mindful eating.

Mindless eating, or the lack of awareness of the food we are consuming, has been linked to health problems attributed to unhealthy eating behaviour, including obesity. Traditional approaches used to moderate eating behaviour often rely on inaccurate self-logging, manual observations or bulky equipment. Overall, there is a clear unmet clinical need to develop an intelligent and lightweight system which can automatically monitor eating behaviour and provide feedback. In this paper, we investigate: i) the development of an automated system for detecting eating behaviour using wearable Electromyography (EMG) sensors, and ii) the application of the proposed system combined with real-time wristband haptic feedback to facilitate mindful eating. For this, the collected data from 16 participants were used to develop an algorithm for detecting chewing and swallowing. We extracted 18 features from EMG which were presented to different classifiers, to develop a system enabling participants to self-moderate their chewing behaviour using haptic feedback. An additional experimental study was conducted with 20 further participants to evaluate the effectiveness of eating monitoring and haptic interface in promoting mindful eating. We used a standard validation scheme with a leave-one-participant-out to assess model performance using standard metrics (F1-score). The proposed algorithm automatically assessed eating behaviour accurately using the EMG-extracted features and a Support Vector Machine (SVM): F1-Score = 0.95 for chewing classification, and F1-Score = 0.87 for swallowing classification. The experimental study showed that participants exhibited a lower rate of chewing when haptic feedback was delivered in the form of wristband vibration, compared to a baseline and non-haptic condition (F (2,38) = 58.243, p < .001). These findings may have major implications for research in eating behaviour, providing key insights into the impact of automatic chewing detection and haptic feedback systems on moderating eating behaviour towards improving health outcomes.

Problematic Smartphone Use: Investigating Contemporary Experiences Using a Convergent Design.

Internet-enabled smartphones are increasingly ubiquitous in the Western world. Research suggests a number of problems can result from mobile phone overuse, including dependence, dangerous and prohibited use. For over a decade, this has been measured by the Problematic Mobile Phone Use Questionnaire (PMPU-Q). Given the rapid developments in mobile technologies, changes of use patterns and possible problematic and addictive use, the aim of the present study was to investigate and validate an updated contemporary version of the PMPU-Q (PMPU-Q-R). A mixed methods convergent design was employed, including a psychometric survey (N = 512) alongside qualitative focus groups (N = 21), to elicit experiences and perceptions of problematic smartphone use. The results suggest the PMPU-Q-R factor structure can be updated to include smartphone dependence, dangerous driving, and antisocial smartphone use factors. Theories of problematic mobile phone use require consideration of the ubiquity and indispensability of smartphones in the present day and age, particularly regarding use whilst driving and in social interactions.

Problematic Mobile Phone Use and Addiction Across Generations: the Roles of Psychopathological Symptoms and Smartphone Use.

Contemporary technological advances have led to a significant increase in using mobile technologies. Recent research has pointed to potential problems as a consequence of mobile overuse, including addiction, financial problems, dangerous use (i.e. whilst driving) and prohibited use (i.e. use in forbidden areas). The aim of this study is to extend previous findings regarding the predictive power of psychopathological symptoms (depression, anxiety and stress), mobile phone use (i.e. calls, SMS, time spent on the phone, as well as the engagement in specific smartphone activities) across Generations X and Y on problematic mobile phone use in a sample of 273 adults. Findings revealed prohibited use and dependence were predicted by calls/day, time on the phone and using social media. Only for dependent mobile phone use (rather than prohibited), stress appeared as significant. Using social media and anxiety significantly predicted belonging to Generation Y, with calls per day predicted belonging to Generation X. This finding suggests Generation Y are more likely to use asynchronous social media-based communication, whereas Generation X engage more in synchronous communication. The findings have implications for prevention and awareness-raising efforts of possibly problematic mobile phone use for educators, parents and individuals, particularly including dependence and prohibited use.

NeuroPlace: Categorizing urban places according to mental states.

Urban spaces have a great impact on how people's emotion and behaviour. There are number of factors that impact our brain responses to a space. This paper presents a novel urban place recommendation approach, that is based on modelling in-situ EEG data. The research investigations leverages on newly affordable Electroencephalogram (EEG) headsets, which has the capability to sense mental states such as meditation and attention levels. These emerging devices have been utilized in understanding how human brains are affected by the surrounding built environments and natural spaces. In this paper, mobile EEG headsets have been used to detect mental states at different types of urban places. By analysing and modelling brain activity data, we were able to classify three different places according to the mental state signature of the users, and create an association map to guide and recommend people to therapeutic places that lessen brain fatigue and increase mental rejuvenation. Our mental states classifier has achieved accuracy of (%90.8). NeuroPlace breaks new ground not only as a mobile ubiquitous brain monitoring system for urban computing, but also as a system that can advise urban planners on the impact of specific urban planning policies and structures. We present and discuss the challenges in making our initial prototype more practical, robust, and reliable as part of our on-going research. In addition, we present some enabling applications using the proposed architecture.