Classifying Muscle States with One-Dimensional Radio-Frequency Signals from Single Element Ultrasound Transducers.

The reliable assessment of muscle states, such as contracted muscles vs. non-contracted muscles or relaxed muscles vs. fatigue muscles, is crucial in many sports and rehabilitation scenarios, such as the assessment of therapeutic measures. The goal of this work was to deploy machine learning (ML) models based on one-dimensional (1-D) sonomyography (SMG) signals to facilitate low-cost and wearable ultrasound devices. One-dimensional SMG is a non-invasive technique using 1-D ultrasound radio-frequency signals to measure muscle states and has the advantage of being able to acquire information from deep soft tissue layers. To mimic real-life scenarios, we did not emphasize the acquisition of particularly distinct signals. The ML models exploited muscle contraction signals of eight volunteers and muscle fatigue signals of 21 volunteers. We evaluated them with different schemes on a variety of data types, such as unprocessed or processed raw signals and found that comparatively simple ML models, such as Support Vector Machines or Logistic Regression, yielded the best performance w.r.t. accuracy and evaluation time. We conclude that our framework for muscle contraction and muscle fatigue classifications is very well-suited to facilitate low-cost and wearable devices based on ML models using 1-D SMG.

The State-of-the-Art Sensing Techniques in Human Activity Recognition: A Survey.

Human activity recognition (HAR) has become an intensive research topic in the past decade because of the pervasive user scenarios and the overwhelming development of advanced algorithms and novel sensing approaches. Previous HAR-related sensing surveys were primarily focused on either a specific branch such as wearable sensing and video-based sensing or a full-stack presentation of both sensing and data processing techniques, resulting in weak focus on HAR-related sensing techniques. This work tries to present a thorough, in-depth survey on the state-of-the-art sensing modalities in HAR tasks to supply a solid understanding of the variant sensing principles for younger researchers of the community. First, we categorized the HAR-related sensing modalities into five classes: mechanical kinematic sensing, field-based sensing, wave-based sensing, physiological sensing, and hybrid/others. Specific sensing modalities are then presented in each category, and a thorough description of the sensing tricks and the latest related works were given. We also discussed the strengths and weaknesses of each modality across the categorization so that newcomers could have a better overview of the characteristics of each sensing modality for HAR tasks and choose the proper approaches for their specific application. Finally, we summarized the presented sensing techniques with a comparison concerning selected performance metrics and proposed a few outlooks on the future sensing techniques used for HAR tasks.

Induced Magnetic Field-Based Indoor Positioning System for Underwater Environments.

Autonomous underwater vehicles (AUV) are seen as an emerging technology for maritime exploration but are still restricted by the availability of short range, accurate positioning methods necessary, e.g., when docking remote assets. Typical techniques used for high-accuracy positioning in indoor use case scenarios, such as systems using ultra-wide band radio signals (UWB), cannot be applied for underwater positioning because of the quick absorption of the positioning medium caused by the water. Acoustic and optic solutions for underwater positioning also face known problems, such as the multi-path effects, high propagation delay (acoustics), and environmental dependency. This paper presents an oscillating magnetic field-based indoor and underwater positioning system. Unlike those radio wave-based positioning modalities, the magnetic approach generates a bubble-formed magnetic field that will not be deformed by the environmental variation because of the very similar permeability of water and air. The proposed system achieves an underwater positioning mean accuracy of 13.3 cm in 2D and 19.0 cm in 3D with the multi-lateration positioning method and concludes the potential of the magnetic field-based positioning technique for underwater applications. A similar accuracy was also achieved for various indoor environments that were used to test the influence of cluttered environment and of cross environment. The low cost and power consumption system is scalable for extensive coverage area and could plug-and-play without pre-calibration.

Using Privacy Respecting Sound Analysis to Improve Bluetooth Based Proximity Detection for COVID-19 Exposure Tracing and Social Distancing.

We propose to use ambient sound as a privacy-aware source of information for COVID-19-related social distance monitoring and contact tracing. The aim is to complement currently dominant Bluetooth Low Energy Received Signal Strength Indicator (BLE RSSI) approaches. These often struggle with the complexity of Radio Frequency (RF) signal attenuation, which is strongly influenced by specific surrounding characteristics. This in turn renders the relationship between signal strength and the distance between transmitter and receiver highly non-deterministic. We analyze spatio-temporal variations in what we call "ambient sound fingerprints". We leverage the fact that ambient sound received by a mobile device is a superposition of sounds from sources at many different locations in the environment. Such a superposition is determined by the relative position of those sources with respect to the receiver. We present a method for using the above general idea to classify proximity between pairs of users based on Kullback-Leibler distance between sound intensity histograms. The method is based on intensity analysis only, and does not require the collection of any privacy sensitive signals. Further, we show how this information can be fused with BLE RSSI features using adaptive weighted voting. We also take into account that sound is not available in all windows. Our approach is evaluated in elaborate experiments in real-world settings. The results show that both Bluetooth and sound can be used to differentiate users within and out of critical distance (1.5 m) with high accuracies of 77% and 80% respectively. Their fusion, however, improves this to 86%, making evident the merit of augmenting BLE RSSI with sound. We conclude by discussing strengths and limitations of our approach and highlighting directions for future work.

Give more data, awareness and control to individual citizens, and they will help COVID-19 containment.

The rapid dynamics of COVID-19 calls for quick and effective tracking of virus transmission chains and early detection of outbreaks, especially in the "phase 2" of the pandemic, when lockdown and other restriction measures are progressively withdrawn, in order to avoid or minimize contagion resurgence. For this purpose, contact-tracing apps are being proposed for large scale adoption by many countries. A centralized approach, where data sensed by the app are all sent to a nation-wide server, raises concerns about citizens' privacy and needlessly strong digital surveillance, thus alerting us to the need to minimize personal data collection and avoiding location tracking. We advocate the conceptual advantage of a decentralized approach, where both contact and location data are collected exclusively in individual citizens' "personal data stores", to be shared separately and selectively (e.g., with a backend system, but possibly also with other citizens), voluntarily, only when the citizen has tested positive for COVID-19, and with a privacy preserving level of granularity. This approach better protects the personal sphere of citizens and affords multiple benefits: it allows for detailed information gathering for infected people in a privacy-preserving fashion; and, in turn this enables both contact tracing, and, the early detection of outbreak hotspots on more finely-granulated geographic scale. The decentralized approach is also scalable to large populations, in that only the data of positive patients need be handled at a central level. Our recommendation is two-fold. First to extend existing decentralized architectures with a light touch, in order to manage the collection of location data locally on the device, and allow the user to share spatio-temporal aggregates-if and when they want and for specific aims-with health authorities, for instance. Second, we favour a longer-term pursuit of realizing a Personal Data Store vision, giving users the opportunity to contribute to collective good in the measure they want, enhancing self-awareness, and cultivating collective efforts for rebuilding society.

Expressure: Detect Expressions Related to Emotional and Cognitive Activities Using Forehead Textile Pressure Mechanomyography.

We investigate how pressure-sensitive smart textiles, in the form of a headband, can detect changes in facial expressions that are indicative of emotions and cognitive activities. Specifically, we present the Expressure system that performs surface pressure mechanomyography on the forehead using an array of textile pressure sensors that is not dependent on specific placement or attachment to the skin. Our approach is evaluated in systematic psychological experiments. First, through a mimicking expression experiment with 20 participants, we demonstrate the system's ability to detect well-defined facial expressions. We achieved accuracies of 0.824 to classify among three eyebrow movements (0.333 chance-level) and 0.381 among seven full-face expressions (0.143 chance-level). A second experiment was conducted with 20 participants to induce cognitive loads with N-back tasks. Statistical analysis has shown significant correlations between the Expressure features on a fine time granularity and the cognitive activity. The results have also shown significant correlations between the Expressure features and the N-back score. From the 10 most facially expressive participants, our approach can predict whether the N-back score is above or below the average with 0.767 accuracy.

Social Distance Monitor with a Wearable Magnetic Field Proximity Sensor.

Social distancing and contact/exposure tracing are accepted to be critical strategies in the fight against the COVID-19 epidemic. They are both closely connected to the ability to reliably establish the degree of proximity between people in real-world environments. We proposed, implemented, and evaluated a wearable proximity sensing system based on an oscillating magnetic field that overcomes many of the weaknesses of the current state of the art Bluetooth based proximity detection. In this paper, we first described the underlying physical principle, proposed a protocol for the identification and coordination of the transmitter (which is compatible with the current smartphone-based exposure tracing protocols). Subsequently, the system architecture and implementation were described, finally an elaborate characterization and evaluation of the performance (both in systematic lab experiments and in real-world settings) were performed. Our work demonstrated that the proposed system is much more reliable than the widely-used Bluetooth-based approach, particularly when it comes to distinguishing between distances above and below the 2.0 m threshold due to the magnetic field's physical properties.

Accurate Spirometry with Integrated Barometric Sensors in Face-Worn Garments.

Cardiorespiratory (CR) signals are crucial vital signs for fitness condition tracking, medical diagnosis, and athlete performance evaluation. Monitoring such signals in real-life settings is among the most widespread applications of wearable computing. We investigate how miniaturized barometers can be used to perform accurate spirometry in a wearable system that is built on off-the-shelf training masks often used by athletes as a training aid. We perform an evaluation where differential barometric pressure sensors are compared concurrently with a digital spirometer, during an experimental setting of clinical forced vital capacity (FVC) test procedures with 20 participants. The relationship between the two instruments is derived by mathematical modeling first, then by various regression methods from experiment data. The results show that the error of FVC vital values between the two instruments can be as low as 2∼3%. Beyond clinical tests, the method can also measure continuous tidal breathing air volumes with a 1∼3% error margin. Overall, we conclude that barometers with millimeter footprints embedded in face mask apparel can perform similarly to a digital spirometer to monitor breathing airflow and volume in pulmonary function tests.

Facial Muscle Activity Recognition with Reconfigurable Differential Stethoscope-Microphones.

Many human activities and states are related to the facial muscles' actions: from the expression of emotions, stress, and non-verbal communication through health-related actions, such as coughing and sneezing to nutrition and drinking. In this work, we describe, in detail, the design and evaluation of a wearable system for facial muscle activity monitoring based on a re-configurable differential array of stethoscope-microphones. In our system, six stethoscopes are placed at locations that could easily be integrated into the frame of smart glasses. The paper describes the detailed hardware design and selection and adaptation of appropriate signal processing and machine learning methods. For the evaluation, we asked eight participants to imitate a set of facial actions, such as expressions of happiness, anger, surprise, sadness, upset, and disgust, and gestures, like kissing, winkling, sticking the tongue out, and taking a pill. An evaluation of a complete data set of 2640 events with 66% training and a 33% testing rate has been performed. Although we encountered high variability of the volunteers' expressions, our approach shows a recall = 55%, precision = 56%, and f1-score of 54% for the user-independent scenario(9% chance-level). On a user-dependent basis, our worst result has an f1-score = 60% and best result with f1-score = 89%. Having a recall ≥60% for expressions like happiness, anger, kissing, sticking the tongue out, and neutral(Null-class).

Participatory Disease Surveillance Systems: Ethical Framework.

Advances in information technology are changing public health at an unprecedented rate. Participatory surveillance systems are contributing to public health by actively engaging digital (eg, Web-based) communities of volunteer citizens to report symptoms and other pertinent information on public health threats and also by empowering individuals to promptly respond to them. However, this digital model raises ethical issues on top of those inherent in traditional forms of public health surveillance. Research ethics are undergoing significant changes in the digital era where not only participants' physical and psychological well-being but also the protection of their sensitive data have to be considered. In this paper, the digital platform of Influenzanet is used as a case study to illustrate those ethical challenges posed to participatory surveillance systems using digital platforms and mobile apps. These ethical challenges include the implementation of electronic consent, the protection of participants' privacy, the promotion of justice, and the need for interdisciplinary capacity building of research ethics committees. On the basis of our analysis, we propose a framework to regulate and strengthen ethical approaches in the field of digital public health surveillance.

Textile Pressure Mapping Sensor for Emotional Touch Detection in Human-Robot Interaction.

In this paper, we developed a fully textile sensing fabric for tactile touch sensing as the robot skin to detect human-robot interactions. The sensor covers a 20-by-20 cm 2 area with 400 sensitive points and samples at 50 Hz per point. We defined seven gestures which are inspired by the social and emotional interactions of typical people to people or pet scenarios. We conducted two groups of mutually blinded experiments, involving 29 participants in total. The data processing algorithm first reduces the spatial complexity to frame descriptors, and temporal features are calculated through basic statistical representations and wavelet analysis. Various classifiers are evaluated and the feature calculation algorithms are analyzed in details to determine each stage and segments' contribution. The best performing feature-classifier combination can recognize the gestures with a 93 . 3 % accuracy from a known group of participants, and 89 . 1 % from strangers.

iEat: automatic wearable dietary monitoring with bio-impedance sensing.

Diet is an inseparable part of good health, from maintaining a healthy lifestyle for the general population to supporting the treatment of patients suffering from specific diseases. Therefore it is of great significance to be able to monitor people's dietary activity in their daily life remotely. While the traditional practices of self-reporting and retrospective analysis are often unreliable and prone to errors; sensor-based remote diet monitoring is therefore an appealing approach. In this work, we explore an atypical use of bio-impedance by leveraging its unique temporal signal patterns, which are caused by the dynamic close-loop circuit variation between a pair of electrodes due to the body-food interactions during dining activities. Specifically, we introduce iEat, a wearable impedance-sensing device for automatic dietary activity monitoring without the need for external instrumented devices such as smart utensils. By deploying a single impedance sensing channel with one electrode on each wrist, iEat can recognize food intake activities (e.g., cutting, putting food in the mouth with or without utensils, drinking, etc.) and food types from a defined category. The principle is that, at idle, iEat measures only the normal body impedance between the wrist-worn electrodes; while the subject is doing the food-intake activities, new paralleled circuits will be formed through the hand, mouth, utensils, and food, leading to consequential impedance variation. To quantitatively evaluate iEat in real-life settings, a food intake experiment was conducted in an everyday table-dining environment, including 40 meals performed by ten volunteers. With a lightweight, user-independent neural network model, iEat could detect four food intake-related activities with a macro F1 score of 86.4% and classify seven types of foods with a macro F1 score of 64.2%.

Origami single-end capacitive sensing for continuous shape estimation of morphing structures.

In this work, we propose a novel single-end morphing capacitive sensing method for shape tracking, FxC, by combining Folding origami structures and Capacitive sensing to detect the morphing structural motions using state-of-the-art sensing circuits and deep learning. It was observed through embedding areas of origami structures with conductive materials as single-end capacitive sensing patches, that the sensor signals change coherently with the motion of the structure. Different from other origami capacitors where the origami structures are used in adjusting the thickness of the dielectric layer of double-plate capacitors, FxC uses only a single conductive plate per channel, and the origami structure directly changes the geometry of the conductive plate. We examined the operation principle of morphing single-end capacitors through 3D geometry simulation combined with physics theoretical deduction, which deduced similar behavior as observed in experimentation. Then a software pipeline was developed to use the sensor signals to reconstruct the dynamic structural geometry through data-driven deep neural network regression of geometric primitives extracted from vision tracking. We created multiple folding patterns to validate our approach, based on folding patterns including Accordion, Chevron, Sunray and V-Fold patterns with different layouts of capacitive sensors using paper-based and textile-based materials. Experimentation results show that the geometry primitives predicted from the capacitive signals have a strong correlation with the visual ground truth with R-squared value of up to 95% and tracking error of 6.5 mm for patches. The simulation and machine learning constitute two-way information exchange between the sensing signals and structural geometry. By embedding part of the origami surface with morphing single-end capacitive sensors, FxC presents a unique solution that leverages both the mechanical properties of origami and sensing properties of capacitive sensing.

Beyond the visible: preliminary evaluation of the first wearable augmented reality assistance system for pancreatic surgery.

PURPOSE: The retroperitoneal nature of the pancreas, marked by minimal intraoperative organ shifts and deformations, makes augmented reality (AR)-based systems highly promising for pancreatic surgery. This study presents preliminary data from a prospective study aiming to develop the first wearable AR assistance system, ARAS, for pancreatic surgery and evaluating its usability, accuracy, and effectiveness in enhancing the perioperative outcomes of patients. METHODS: We developed ARAS as a two-phase system for a wearable AR device to aid surgeons in planning and operation. This system was used to visualize and register patient-specific 3D anatomical models during the surgery. The location and precision of the registered 3D anatomy were evaluated by assessing the arterial pulse and employing Doppler and duplex ultrasonography. The usability, accuracy, and effectiveness of ARAS were assessed using a five-point Likert scale questionnaire. RESULTS: Perioperative outcomes of five patients underwent various pancreatic resections with ARAS are presented. Surgeons rated ARAS as excellent for preoperative planning. All structures were accurately identified without any noteworthy errors. Only tumor identification decreased after the preparation phase, especially in patients who underwent pancreaticoduodenectomy because of the extensive mobilization of peripancreatic structures. No perioperative complications related to ARAS were observed. CONCLUSIONS: ARAS shows promise in enhancing surgical precision during pancreatic procedures. Its efficacy in preoperative planning and intraoperative vascular identification positions it as a valuable tool for pancreatic surgery and a potential educational resource for future surgical residents.

Remaining useful life prediction of Lithium-ion batteries using spatio-temporal multimodal attention networks.

Lithium-ion batteries are widely used in various applications, including electric vehicles and renewable energy storage. The prediction of the remaining useful life (RUL) of batteries is crucial for ensuring reliable and efficient operation, as well as reducing maintenance costs. However, determining the life cycle of batteries in real-world scenarios is challenging, and existing methods have limitations in predicting the number of cycles iteratively. In addition, existing works often oversimplify the datasets, neglecting important features of the batteries such as temperature, internal resistance, and material type. To address these limitations, this paper proposes a two-stage RUL prediction scheme for Lithium-ion batteries using a spatio-temporal multimodal attention network (ST-MAN). The proposed ST-MAN is to capture the complex spatio-temporal dependencies in the battery data, including the features that are often neglected in existing works. Despite operating without prior knowledge of end-of-life (EOL) events, our method consistently achieves lower error rates, boasting mean absolute error (MAE) and mean square error (MSE) of 0.0275 and 0.0014, respectively, compared to existing convolutional neural networks (CNN) and long short-term memory (LSTM)-based methods. The proposed method has the potential to improve the reliability and efficiency of battery operations and is applicable in various industries.

Embedding textile capacitive sensing into smart wearables as a versatile solution for human motion capturing.

This work presents a novel and versatile approach to employ textile capacitive sensing as an effective solution for capturing human body movement through fashionable and everyday-life garments. Conductive textile patches are utilized for sensing the movement, working without the need for strain or direct body contact, wherefore the patches can sense only from their deformation within the garment. This principle allows the sensing area to be decoupled from the wearer's body for improved wearing comfort and more pleasant integration. We demonstrate our technology based on multiple prototypes which have been developed by an interdisciplinary team of electrical engineers, computer scientists, digital artists, and smart fashion designers through several iterations to seamlessly incorporate the technology of capacitive sensing with corresponding design considerations into textile materials. The resulting accumulation of textile capacitive sensing wearables showcases the versatile application possibilities of our technology from single-joint angle measurements towards multi-joint body part tracking.

CEGAN: Classification Enhancement Generative Adversarial Networks for unraveling data imbalance problems.

The data imbalance problem in classification is a frequent but challenging task. In real-world datasets, numerous class distributions are imbalanced and the classification result under such condition reveals extreme bias in the majority data class. Recently, the potential of GAN as a data augmentation method on minority data has been studied. In this paper, we propose a classification enhancement generative adversarial networks (CEGAN) to enhance the quality of generated synthetic minority data and more importantly, to improve the prediction accuracy in data imbalanced condition. In addition, we propose an ambiguity reduction method using the generated synthetic minority data for the case of multiple similar classes that are degenerating the classification accuracy. The proposed method is demonstrated with five benchmark datasets. The results indicate that approximating the real data distribution using CEGAN improves the classification performance significantly in data imbalanced conditions compared with various standard data augmentation methods.

Activity recognition of assembly tasks using body-worn microphones and accelerometers.

In order to provide relevant information to mobile users, such as workers engaging in the manual tasks of maintenance and assembly, a wearable computer requires information about the user's specific activities. This work focuses on the recognition of activities that are characterized by a hand motion and an accompanying sound. Suitable activities can be found in assembly and maintenance work. Here, we provide an initial exploration into the problem domain of continuous activity recognition using on-body sensing. We use a mock "wood workshop" assembly task to ground our investigation. We describe a method for the continuous recognition of activities (sawing, hammering, filing, drilling, grinding, sanding, opening a drawer, tightening a vise, and turning a screwdriver) using microphones and three-axis accelerometers mounted at two positions on the user's arms. Potentially "interesting" activities are segmented from continuous streams of data using an analysis of the sound intensity detected at the two different locations. Activity classification is then performed on these detected segments using linear discriminant analysis (LDA) on the sound channel and hidden Markov models (HMMs) on the acceleration data. Four different methods at classifier fusion are compared for improving these classifications. Using user-dependent training, we obtain continuous average recall and precision rates (for positive activities) of 78 percent and 74 percent, respectively. Using user-independent training (leave-one-out across five users), we obtain recall rates of 66 percent and precision rates of 63 percent. In isolation, these activities were recognized with accuracies of 98 percent, 87 percent, and 95 percent for the user-dependent, user-independent, and user-adapted cases, respectively.

Motion aware clothing for the personal health assistant.

This paper sketches the vision and first results of a 'Personal Health Assistant' PHA, opening up new vistas in patient centred healthcare. The PHA is comprised of a wearable sensing and communicating system, seamlessly embedded in daily clothing. Several on-body sensors monitor the biometric and contextual status of the wearer continuously. The embedded computer fuses the vital and physiological data with activity patterns of the wearer and with the social environment; based on these data the on-body computer generates the 'Life Balance Factor' LBF as an individual feedback to the user and to the surroundings afford-ing effective disease prevention, management and rehabilitation, the last also involving telemedicine. The state-of-the-art enabling technologies: smart textile technology and miniaturization of electronics combined with wireless communication, along with recent developments in wearable computing are presented and assessed in the context of multiparameter health monitoring.