Human Activity Recognition, Monitoring, and Analysis Facilitated by Novel and Widespread Applications of Sensors.

The Special Issue Sensors for Human Activity Recognition has received a total of 30 submissions so far, and from these, this new edition will publish 10 academic articles [...].

Exploring the Real-Time Variability and Complexity of Sitting Patterns in Office Workers with Non-Specific Chronic Spinal Pain and Pain-Free Individuals.

Chronic spinal pain (CSP) is a prevalent condition, and prolonged sitting at work can contribute to it. Ergonomic factors like this can cause changes in motor variability. Variability analysis is a useful method to measure changes in motor performance over time. When performing the same task multiple times, different performance patterns can be observed. This variability is intrinsic to all biological systems and is noticeable in human movement. This study aims to examine whether changes in movement variability and complexity during real-time office work are influenced by CSP. The hypothesis is that individuals with and without pain will have different responses to office work tasks. Six office workers without pain and ten with CSP participated in this study. Participant's trunk movements were recorded during work for an entire week. Linear and nonlinear measures of trunk kinematic displacement were used to assess movement variability and complexity. A mixed ANOVA was utilized to compare changes in movement variability and complexity between the two groups. The effects indicate that pain-free participants showed more complex and less predictable trunk movements with a lower degree of structure and variability when compared to the participants suffering from CSP. The differences were particularly noticeable in fine movements.

Sensor-Based Human Activity and Behavior Research: Where Advanced Sensing and Recognition Technologies Meet.

Human activity recognition (HAR) and human behavior recognition (HBR) have been playing increasingly important roles in the digital age [...].

Automatic Cognitive Fatigue Detection Using Wearable fNIRS and Machine Learning.

Wearable sensors have increasingly been applied in healthcare to generate data and monitor patients unobtrusively. Their application for Brain-Computer Interfaces (BCI) allows for unobtrusively monitoring one's cognitive state over time. A particular state relevant in multiple domains is cognitive fatigue, which may impact performance and attention, among other capabilities. The monitoring of this state will be applied in real learning settings to detect and advise on effective break periods. In this study, two functional near-infrared spectroscopy (fNIRS) wearable devices were employed to build a BCI to automatically detect the state of cognitive fatigue using machine learning algorithms. An experimental procedure was developed to effectively induce cognitive fatigue that included a close-to-real digital lesson and two standard cognitive tasks: Corsi-Block task and a concentration task. Machine learning models were user-tuned to account for the individual dynamics of each participant, reaching classification accuracy scores of around 70.91 ± 13.67 %. We concluded that, although effective for some subjects, the methodology needs to be individually validated before being applied. Moreover, time on task was not a particularly determining factor for classification, i.e., to induce cognitive fatigue. Further research will include other physiological signals and human-computer interaction variables.

Comparing Handcrafted Features and Deep Neural Representations for Domain Generalization in Human Activity Recognition.

Human Activity Recognition (HAR) has been studied extensively, yet current approaches are not capable of generalizing across different domains (i.e., subjects, devices, or datasets) with acceptable performance. This lack of generalization hinders the applicability of these models in real-world environments. As deep neural networks are becoming increasingly popular in recent work, there is a need for an explicit comparison between handcrafted and deep representations in Out-of-Distribution (OOD) settings. This paper compares both approaches in multiple domains using homogenized public datasets. First, we compare several metrics to validate three different OOD settings. In our main experiments, we then verify that even though deep learning initially outperforms models with handcrafted features, the situation is reversed as the distance from the training distribution increases. These findings support the hypothesis that handcrafted features may generalize better across specific domains.

Respiratory Inductance Plethysmography to Assess Fatigability during Repetitive Work.

Cumulative fatigue during repetitive work is associated with occupational risk and productivity reduction. Usually, subjective measures or muscle activity are used for a cumulative evaluation; however, Industry 4.0 wearables allow overcoming the challenges observed in those methods. Thus, the aim of this study is to analyze alterations in respiratory inductance plethysmography (RIP) to measure the asynchrony between thorax and abdomen walls during repetitive work and its relationship with local fatigue. A total of 22 healthy participants (age: 27.0 ± 8.3 yrs; height: 1.72 ± 0.09 m; mass: 63.4 ± 12.9 kg) were recruited to perform a task that includes grabbing, moving, and placing a box in an upper and lower shelf. This task was repeated for 10 min in three trials with a fatigue protocol between them. Significant main effects were found from Baseline trial to the Fatigue trials (p < 0.001) for both RIP correlation and phase synchrony. Similar results were found for the activation amplitude of agonist muscle (p < 0.001), and to the muscle acting mainly as a joint stabilizer (p < 0.001). The latter showed a significant effect in predicting both RIP correlation and phase synchronization. Both RIP correlation and phase synchronization can be used for an overall fatigue assessment during repetitive work.

Human Activity Recognition for Indoor Localization Using Smartphone Inertial Sensors.

With the fast increase in the demand for location-based services and the proliferation of smartphones, the topic of indoor localization is attracting great interest. In indoor environments, users' performed activities carry useful semantic information. These activities can then be used by indoor localization systems to confirm users' current relative locations in a building. In this paper, we propose a deep-learning model based on a Convolutional Long Short-Term Memory (ConvLSTM) network to classify human activities within the indoor localization scenario using smartphone inertial sensor data. Results show that the proposed human activity recognition (HAR) model accurately identifies nine types of activities: not moving, walking, running, going up in an elevator, going down in an elevator, walking upstairs, walking downstairs, or going up and down a ramp. Moreover, predicted human activities were integrated within an existing indoor positioning system and evaluated in a multi-story building across several testing routes, with an average positioning error of 2.4 m. The results show that the inclusion of human activity information can reduce the overall localization error of the system and actively contribute to the better identification of floor transitions within a building. The conducted experiments demonstrated promising results and verified the effectiveness of using human activity-related information for indoor localization.

ECG Biometrics Using Deep Learning and Relative Score Threshold Classification.

The field of biometrics is a pattern recognition problem, where the individual traits are coded, registered, and compared with other database records. Due to the difficulties in reproducing Electrocardiograms (ECG), their usage has been emerging in the biometric field for more secure applications. Inspired by the high performance shown by Deep Neural Networks (DNN) and to mitigate the intra-variability challenges displayed by the ECG of each individual, this work proposes two architectures to improve current results in both identification (finding the registered person from a sample) and authentication (prove that the person is whom it claims) processes: Temporal Convolutional Neural Network (TCNN) and Recurrent Neural Network (RNN). Each architecture produces a similarity score, based on the prediction error of the former and the logits given by the last, and fed to the same classifier, the Relative Score Threshold Classifier (RSTC).The robustness and applicability of these architectures were trained and tested on public databases used by literature in this context: Fantasia, MIT-BIH, and CYBHi databases. Results show that overall the TCNN outperforms the RNN achieving almost 100%, 96%, and 90% accuracy, respectively, for identification and 0.0%, 0.1%, and 2.2% equal error rate (EER) for authentication processes. When comparing to previous work, both architectures reached results beyond the state-of-the-art. Nevertheless, the improvement of these techniques, such as enriching training with extra varied data and transfer learning, may provide more robust systems with a reduced time required for validation.

An Infrastructure-Free Magnetic-Based Indoor Positioning System with Deep Learning.

Infrastructure-free Indoor Positioning Systems (IPS) are becoming popular due to their scalability and a wide range of applications. Such systems often rely on deployed Wi-Fi networks. However, their usability may be compromised, either due to scanning restrictions from recent Android versions or the proliferation of 5G technology. This raises the need for new infrastructure-free IPS independent of Wi-Fi networks. In this paper, we propose the use of magnetic field data for IPS, through Deep Neural Networks (DNN). Firstly, a dataset of human indoor trajectories was collected with different smartphones. Afterwards, a magnetic fingerprint was constructed and relevant features were extracted to train a DNN that returns a probability map of a user's location. Finally, two postprocessing methods were applied to obtain the most probable location regions. We asserted the performance of our solution against a test dataset, which produced a Success Rate of around 80%. We believe that these results are competitive for an IPS based on a single sensing source. Moreover, the magnetic field can be used as an additional information layer to increase the robustness and redundancy of current multi-source IPS.

Biosensing and Actuation-Platforms Coupling Body Input-Output Modalities for Affective Technologies.

Research in the use of ubiquitous technologies, tracking systems and wearables within mental health domains is on the rise. In recent years, affective technologies have gained traction and garnered the interest of interdisciplinary fields as the research on such technologies matured. However, while the role of movement and bodily experience to affective experience is well-established, how to best address movement and engagement beyond measuring cues and signals in technology-driven interactions has been unclear. In a joint industry-academia effort, we aim to remodel how affective technologies can help address body and emotional self-awareness. We present an overview of biosignals that have become standard in low-cost physiological monitoring and show how these can be matched with methods and engagements used by interaction designers skilled in designing for bodily engagement and aesthetic experiences. Taking both strands of work together offers unprecedented design opportunities that inspire further research. Through first-person soma design, an approach that draws upon the designer's felt experience and puts the sentient body at the forefront, we outline a comprehensive work for the creation of novel interactions in the form of couplings that combine biosensing and body feedback modalities of relevance to affective health. These couplings lie within the creation of design toolkits that have the potential to render rich embodied interactions to the designer/user. As a result we introduce the concept of "orchestration". By orchestration, we refer to the design of the overall interaction: coupling sensors to actuation of relevance to the affective experience; initiating and closing the interaction; habituating; helping improve on the users' body awareness and engagement with emotional experiences; soothing, calming, or energising, depending on the affective health condition and the intentions of the designer. Through the creation of a range of prototypes and couplings we elicited requirements on broader orchestration mechanisms. First-person soma design lets researchers look afresh at biosignals that, when experienced through the body, are called to reshape affective technologies with novel ways to interpret biodata, feel it, understand it and reflect upon our bodies.

Autonomic nervous system response to remote ischemic conditioning: heart rate variability assessment.

BACKGROUND: Remote ischemic conditioning (RIC) is a procedure applied in a limb for triggering endogenous protective pathways in distant organs, namely brain or heart. The underlying mechanisms of RIC are still not fully understood, and it is hypothesized they are mediated either by humoral factors, immune cells and/or the autonomic nervous system. Herein, heart rate variability (HRV) was used to evaluate the electrophysiological processes occurring in the heart during RIC and, in turn to assess the role of autonomic nervous system. METHODS: Healthy subjects were submitted to RIC protocol and electrocardiography (ECG) was used to evaluate HRV, by assessing the variability of time intervals between two consecutive heart beats. This is a pilot study based on the analysis of 18 ECG from healthy subjects submitted to RIC. HRV was characterized in three domains (time, frequency and non-linear features) that can be correlated with the autonomic nervous system function. RESULTS: RIC procedure increased significantly the non-linear parameter SD2, which is associated with long term HRV. This effect was observed in all subjects and in the senior (> 60 years-old) subset analysis. SD2 increase suggests an activation of both parasympathetic and sympathetic nervous system, namely via fast vagal response (parasympathetic) and the slow sympathetic response to the baroreceptors stimulation. CONCLUSIONS: RIC procedure modulates both parasympathetic and sympathetic autonomic nervous system. Furthermore, this modulation is more pronounced in the senior subset of subjects. Therefore, the autonomic nervous system regulation could be one of the mechanisms for RIC therapeutic effectiveness.

A Semi-Automatic Annotation Approach for Human Activity Recognition.

Modern smartphones and wearables often contain multiple embedded sensors which generate significant amounts of data. This information can be used for body monitoring-based areas such as healthcare, indoor location, user-adaptive recommendations and transportation. The development of Human Activity Recognition (HAR) algorithms involves the collection of a large amount of labelled data which should be annotated by an expert. However, the data annotation process on large datasets is expensive, time consuming and difficult to obtain. The development of a HAR approach which requires low annotation effort and still maintains adequate performance is a relevant challenge. We introduce a Semi-Supervised Active Learning (SSAL) based on Self-Training (ST) approach for Human Activity Recognition to partially automate the annotation process, reducing the annotation effort and the required volume of annotated data to obtain a high performance classifier. Our approach uses a criterion to select the most relevant samples for annotation by the expert and propagate their label to the most confident samples. We present a comprehensive study comparing supervised and unsupervised methods with our approach on two datasets composed of daily living activities. The results showed that it is possible to reduce the required annotated data by more than 89% while still maintaining an accurate model performance.

Fingerprints and Floor Plans Construction for Indoor Localisation Based on Crowdsourcing.

The demand for easily deployable indoor localisation solutions has been growing. Although several systems have been proposed, their limitations regarding the high implementation costs hinder most of them to be widely used. Fingerprinting-based IPS (Indoor Positioning Systems) depend on characteristics pervasively available in buildings. However, such systems require indoor floor plans, which might not be available, as well as environmental fingerprints, that need to be collected through human resources intensive processes. To overcome these limitations, this paper proposes an algorithm for the automatic construction of indoor maps and fingerprints, solely depending on non-annotated crowdsourced data from smartphones. Our system relies on multiple gait-model based filtering techniques for accurate movement quantification in combination with opportunistic sensing observations. After the reconstruction of users' movement with PDR (Pedestrian Dead Reckoning) techniques, Wi-Fi measurements are clustered to partition the trajectories into segments. Similar segments, which belong to the same cluster, are identified using an adaptive approach based on a geomagnetic field distance. Finally, the floor plans are obtained through a data fusion process. Merging the acquired environmental data using the obtained floor plan, fingerprints are aligned to physical locations. Experimental results show that the proposed solution achieved comparable floor plans and fingerprints to those acquired manually, allowing the conclusion that is possible to automate the setup process of infrastructure-free IPS.

Tunable Gas Sensing Gels by Cooperative Assembly.

The cooperative assembly of biopolymers and small molecules can yield functional materials with precisely tunable properties. Here, the fabrication, characterization, and use of multicomponent hybrid gels as selective gas sensors are reported. The gels are composed of liquid crystal droplets self-assembled in the presence of ionic liquids, which further coassemble with biopolymers to form stable matrices. Each individual component can be varied and acts cooperatively to tune gels' structure and function. The unique molecular environment in hybrid gels is explored for supramolecular recognition of volatile compounds. Gels with distinct compositions are used as optical and electrical gas sensors, yielding a combinatorial response conceptually mimicking olfactory biological systems, and tested to distinguish volatile organic compounds and to quantify ethanol in automotive fuel. The gel response is rapid, reversible, and reproducible. These robust, versatile, modular, pliant electro-optical soft materials possess new possibilities in sensing triggered by chemical and physical stimuli.

Biosignals learning and synthesis using deep neural networks.

BACKGROUND: Modeling physiological signals is a complex task both for understanding and synthesize biomedical signals. We propose a deep neural network model that learns and synthesizes biosignals, validated by the morphological equivalence of the original ones. This research could lead the creation of novel algorithms for signal reconstruction in heavily noisy data and source detection in biomedical engineering field. METHOD: The present work explores the gated recurrent units (GRU) employed in the training of respiration (RESP), electromyograms (EMG) and electrocardiograms (ECG). Each signal is pre-processed, segmented and quantized in a specific number of classes, corresponding to the amplitude of each sample and fed to the model, which is composed by an embedded matrix, three GRU blocks and a softmax function. This network is trained by adjusting its internal parameters, acquiring the representation of the abstract notion of the next value based on the previous ones. The simulated signal was generated by forecasting a random value and re-feeding itself. RESULTS AND CONCLUSIONS: The resulting generated signals are similar with the morphological expression of the originals. During the learning process, after a set of iterations, the model starts to grasp the basic morphological characteristics of the signal and later their cyclic characteristics. After training, these models' prediction are closer to the signals that trained them, specially the RESP and ECG. This synthesis mechanism has shown relevant results that inspire the use to characterize signals from other physiological sources.

Ensemble Predictors: Possibilistic Combination of Conformal Predictors for Multivariate Time Series Classification.

In this article we propose a conceptual framework to study ensembles of conformal predictors (CP), that we call Ensemble Predictors (EP). Our approach is inspired by the application of imprecise probabilities in information fusion. Based on the proposed framework, we study, for the first time in the literature, the theoretical properties of CP ensembles in a general setting, by focusing on simple and commonly used possibilistic combination rules. We also illustrate the applicability of the proposed methods in the setting of multivariate time-series classification, showing that these methods provide better performance (in terms of both robustness, conservativeness, accuracy and running time) than both standard classification algorithms and other combination rules proposed in the literature, on a large set of benchmarks from the UCR time series archive.

Flexographic printed microwave-assisted grown zinc oxide nanostructures for sensing applications.

The development of flexible electronics has increased the demand for wearable pressure sensors that can be used to monitor various biomedical signals. In this context, pressure sensors based on zinc oxide (ZnO) have great potential since, besides the biocompatibility and biodegradability of this metal oxide, it also has piezoelectric properties. The common feature of these sensors is the alignment of the ZnO nanostructures in the strain direction. This alignment is achieved through a three-stage procedure: deposition of a ZnO nanoparticle layer (seed layer) followed by its patterning and the subsequent growth of nanostructures from the seed layer nanoparticles. Herein, a process compatible with industrial scale for depositing seed layers by flexographic printing is proposed, allowing seed layers to be deposited and patterned swiftly and efficiently in a single step on flexible indium tin oxide coated polyethylene terephthalate substrates, significantly decreasing the time and cost required to produce pressure sensors. The growth conditions of ZnO nanorods on these substrates were also studied to analyze their influence on the morphological and structural characteristics of the nanostructures. Nanorods with length of (0.27 ± 0.04) µm and density of (296 ± 6) nanorods per µm2 were obtained in microwave-assisted hydrothermal syntheses carried out at 100 °C for 30 min, with a 1 M zinc acetate seed layer and using an equimolar growth solution of zinc nitrate and hexamethylenetetramine. These conditions were used to produce ZnO-based pressure sensors with two patterns (one square and 16 individual squares). Although the single square sensors displayed a higher average output voltage ((12 ± 5) V for an impact pressure of 150 kPa), their response was considerably more variable than the patterned sensors (with 16 squares), which displayed an average output voltage of (8 ± 2) V under an applied pressure of 150 kPa and sensitivity values of (0.06 ± 0.01) V kPa-1, demonstrating their potential for wearables and portable electronics.

Predicting post-discharge complications in cardiothoracic surgery: A clinical decision support system to optimize remote patient monitoring resources.

Cardiac surgery patients are highly prone to severe complications post-discharge. Close follow-up through remote patient monitoring can help detect adverse outcomes earlier or prevent them, closing the gap between hospital and home care. However, equipment is limited due to economic and human resource constraints. This issue raises the need for efficient risk estimation to provide clinicians with insights into the potential benefit of remote monitoring for each patient. Standard models, such as the EuroSCORE, predict the mortality risk before the surgery. While these are used and validated in real settings, the models lack information collected during or following the surgery, determinant to predict adverse outcomes occurring further in the future. This paper proposes a Clinical Decision Support System based on Machine Learning to estimate the risk of severe complications within 90 days following cardiothoracic surgery discharge, an innovative objective underexplored in the literature. Health records from a cardiothoracic surgery department regarding 5 045 patients (60.8% male) collected throughout ten years were used to train predictive models. Clinicians' insights contributed to improving data preparation and extending traditional pipeline optimization techniques, addressing medical Artificial Intelligence requirements. Two separate test sets were used to evaluate the generalizability, one derived from a patient-grouped 70/30 split and another including all surgeries from the last available year. The achieved Area Under the Receiver Operating Characteristic curve on these test sets was 69.5% and 65.3%, respectively. Also, additional testing was implemented to simulate a real-world use case considering the weekly distribution of remote patient monitoring resources post-discharge. Compared to the random resource allocation, the selection of patients with respect to the outputs of the proposed model was proven beneficial, as it led to a higher number of high-risk patients receiving remote monitoring equipment.

Acoustic and Clinical Data Analysis of Vocal Recordings: Pandemic Insights and Lessons.

Background/Objectives: The interest in processing human speech and other human-generated audio signals as a diagnostic tool has increased due to the COVID-19 pandemic. The project OSCAR (vOice Screening of CoronA viRus) aimed to develop an algorithm to screen for COVID-19 using a dataset of Portuguese participants with voice recordings and clinical data. Methods: This cross-sectional study aimed to characterise the pattern of sounds produced by the vocal apparatus in patients with SARS-CoV-2 infection documented by a positive RT-PCR test, and to develop and validate a screening algorithm. In Phase II, the algorithm developed in Phase I was tested in a real-world setting. Results: In Phase I, after filtering, the training group consisted of 166 subjects who were effectively available to train the classification model (34.3% SARS-CoV-2 positive/65.7% SARS-CoV-2 negative). Phase II enrolled 58 participants (69.0% SARS-CoV-2 positive/31.0% SARS-CoV-2 negative). The final model achieved a sensitivity of 85%, a specificity of 88.9%, and an F1-score of 84.7%, suggesting voice screening algorithms as an attractive strategy for COVID-19 diagnosis. Conclusions: Our findings highlight the potential of a voice-based detection strategy as an alternative method for respiratory tract screening.

Occupational health knowledge discovery based on association rules applied to workers' body parts protection: a case study in the automotive industry.

Occupational Health Protection (OHP) is mandatory by law and can be accomplished by considering the participation of others besides occupational physicians. The data shared can originate knowledge that might influence other processes related to occupational risk prevention. In this study, we used Artificial Intelligence (AI) methods to extract patterns among records shared under these circumstances over two years in the automotive industry. Records featuring OHP data against physical working conditions were selected, and a database of 383 profiles was designed. As Occupational Health Protection profiles under study are associated with work functional ability reduction, the body part(s) (n = 14) where it occurred were identified. Association Rules (ARs) coupled with Natural Language Processing techniques were applied to find meaningful hidden relationships and to identify the occurrence of protection profiles being assigned to at least two body parts simultaneously. After filtering ARs using three metrics (support, confidence, and lift), 54 ARs were found. The distribution of simultaneous body parts is presented as being higher in Special projects (n = 5). The results can use in: (i) design a multi-site body parts functional work ability (loss) model; (ii) model the capacity of organizations to retain workers in their working settings and (iii) prevent work-related musculoskeletal symptoms.