IMU-Based Fitness Activity Recognition Using CNNs for Time Series Classification.

Mobile fitness applications provide the opportunity to show users real-time feedback on their current fitness activity. For such applications, it is essential to accurately track the user's current fitness activity using available mobile sensors, such as inertial measurement units (IMUs). Convolutional neural networks (CNNs) have been shown to produce strong results in different time series classification tasks, including the recognition of daily living activities. However, fitness activities can present unique challenges to the human activity recognition task (HAR), including greater similarity between individual activities and fewer available data for model training. In this paper, we evaluate the applicability of CNNs to the fitness activity recognition task (FAR) using IMU data and determine the impact of input data size and sensor count on performance. For this purpose, we adapted three existing CNN architectures to the FAR task and designed a fourth CNN variant, which we call the scaling fully convolutional network (Scaling-FCN). We designed a preprocessing pipeline and recorded a running exercise data set with 20 participants, in which we evaluated the respective recognition performances of the four networks, comparing them with three traditional machine learning (ML) methods commonly used in HAR. Although CNN architectures achieve at least 94% test accuracy in all scenarios, two traditional ML architectures surpass them in the default scenario, with support vector machines (SVMs) achieving 99.00 ± 0.34% test accuracy. The removal of all sensors except one foot sensor reduced the performance of traditional ML architectures but improved the performance of CNN architectures on our data set, with our Scaling-FCN reaching the highest accuracy of 99.86 ± 0.11% on the test set. Our results suggest that CNNs are generally well suited for fitness activity recognition, and noticeable performance improvements can be achieved if sensors are dropped selectively, although traditional ML architectures can still compete with or even surpass CNNs when favorable input data are utilized.

Give Me a Sign: Using Data Gloves for Static Hand-Shape Recognition.

Human-to-human communication via the computer is mainly carried out using a keyboard or microphone. In the field of virtual reality (VR), where the most immersive experience possible is desired, the use of a keyboard contradicts this goal, while the use of a microphone is not always desirable (e.g., silent commands during task-force training) or simply not possible (e.g., if the user has hearing loss). Data gloves help to increase immersion within VR, as they correspond to our natural interaction. At the same time, they offer the possibility of accurately capturing hand shapes, such as those used in non-verbal communication (e.g., thumbs up, okay gesture, …) and in sign language. In this paper, we present a hand-shape recognition system using Manus Prime X data gloves, including data acquisition, data preprocessing, and data classification to enable nonverbal communication within VR. We investigate the impact on accuracy and classification time of using an outlier detection and a feature selection approach in our data preprocessing. To obtain a more generalized approach, we also studied the impact of artificial data augmentation, i.e., we created new artificial data from the recorded and filtered data to augment the training data set. With our approach, 56 different hand shapes could be distinguished with an accuracy of up to 93.28%. With a reduced number of 27 hand shapes, an accuracy of up to 95.55% could be achieved. The voting meta-classifier (VL2) proved to be the most accurate, albeit slowest, classifier. A good alternative is random forest (RF), which was even able to achieve better accuracy values in a few cases and was generally somewhat faster. outlier detection was proven to be an effective approach, especially in improving the classification time. Overall, we have shown that our hand-shape recognition system using data gloves is suitable for communication within VR.

Genetic Variants Associated with the Age of Onset Identified by Whole-Exome Sequencing in Fatal Familial Insomnia.

Fatal familial insomnia (FFI) is a rare autosomal-dominant inherited prion disease with a wide variability in age of onset. Its causes are not known. In the present study, we aimed to analyze genetic risk factors other than the prion protein gene (PRNP), in FFI patients with varying ages of onset. Whole-exome sequencing (WES) analysis was performed for twenty-five individuals with FFI (D178N-129M). Gene ontology enrichment analysis was carried out by Reactome to generate hypotheses regarding the biological processes of the identified genes. In the present study, we used a statistical approach tailored to the specifics of the data and identified nineteen potential gene variants with a potential effect on the age of onset. Evidence for potential disease modulatory risk loci was observed in two pseudogenes (NR1H5P, GNA13P1) and three protein coding genes (EXOC1L, SRSF11 and MSANTD3). These genetic variants are absent in FFI patients with early disease onset (19-40 years). The biological function of these genes and PRNP is associated with programmed cell death, caspase-mediated cleavage of cytoskeletal proteins and apoptotic cleavage of cellular proteins. In conclusions, our study provided first evidence for the involvement of genetic risk factors additional to PRNP, which may influence the onset of clinical symptoms in FFI.

Correction: A case control study to investigate differences in motor control between individuals with and without non-specific low back pain during standing.

[This corrects the article DOI: 10.1371/journal.pone.0234858.].

Recognizing Full-Body Exercise Execution Errors Using the Teslasuit.

Regular physical exercise is essential for overall health; however, it is also crucial to mitigate the probability of injuries due to incorrect exercise executions. Existing health or fitness applications often neglect accurate full-body motion recognition and focus on a single body part. Furthermore, they often detect only specific errors or provide feedback first after the execution. This lack raises the necessity for the automated detection of full-body execution errors in real-time to assist users in correcting motor skills. To address this challenge, we propose a method for movement assessment using a full-body haptic motion capture suit. We train probabilistic movement models using the data of 10 inertial sensors to detect exercise execution errors. Additionally, we provide haptic feedback, employing transcutaneous electrical nerve stimulation immediately, as soon as an error occurs, to correct the movements. The results based on a dataset collected from 15 subjects show that our approach can detect severe movement execution errors directly during the workout and provide haptic feedback at respective body locations. These results suggest that a haptic full-body motion capture suit, such as the Teslasuit, is promising for movement assessment and can give appropriate haptic feedback to the users so that they can improve their movements.

Predicting functional performance via classification of lower extremity strength in older adults with exergame-collected data.

OBJECTIVE: The goal of this article is to present and to evaluate a sensor-based functional performance monitoring system. The system consists of an array of Wii Balance Boards (WBB) and an exergame that estimates whether the player can maintain physical independence, comparing the results with the 30 s Chair-Stand Test (30CST). METHODS: Sixteen participants recruited at a nursing home performed the 30CST and then played the exergame described here as often as desired during a period of 2 weeks. For each session, features related to walking and standing on the WBBs while playing the exergame were collected. Different classifier algorithms were used to predict the result of the 30CST on a binary basis as able or unable to maintain physical independence. RESULTS: By using a Logistic Model Tree, we achieved a maximum accuracy of 91% when estimating whether player's 30CST scores were over or under a threshold of 12 points, our findings suggest that predicting age- and sex-adjusted cutoff scores is feasible. CONCLUSION: An array of WBBs seems to be a viable solution to estimate lower extremity strength and thereby functional performance in a non-invasive and continuous manner. This study provides proof of concept supporting the use of exergames to identify and monitor elderly subjects at risk of losing physical independence.

Quality Criteria for Serious Games: Serious Part, Game Part, and Balance.

Serious games are digital games that have an additional goal beyond entertainment. Recently, many studies have explored different quality criteria for serious games, including effectiveness and attractiveness. Unfortunately, the double mission of serious games, that is, simultaneous achievement of intended effects (serious part) and entertainment (game part), is not adequately considered in these studies. This paper aims to identify essential quality criteria for serious games. The fundamental goal of our research is to identify important factors of serious games and to adapt the existing principles and requirements from game-related literature to effective and attractive serious games. In addition to the review of the relevant literature, we also include workshop results. Furthermore, we analyzed and summarized 22 state-of-the-art serious games for education and health. The selected best-practice serious games either prove their effectiveness through scientific studies or by winning game awards. For the analysis of these games, we refer to "DIN SPEC 91380 Serious Games Metadata Format." A summarized text states quality criteria for both the serious and the game part, and especially the balance between them. We provide guidelines for high-quality serious games drawn from literature analysis and in close cooperation with domain experts.

A case control study to investigate differences in motor control between individuals with and without non-specific low back pain during standing.

Recent literature has indicated altered motor control in individuals with non-specific low back pain (NSLBP). These individuals present variations in back muscular activity and center of mass (CoM) oscillations. The aim of this study is to explore the possibility of quantitatively measuring these differences using standard parameters with electronic devices. Twenty individuals with NSLBP and 20 healthy controls, matched by sex and age, performed a total of three trials under three different conditions for 90 seconds each. These conditions were standing on firm ground with eyes open, with eyes closed and standing on unstable foam with eyes open. Balance data was acquired via a Kistler force platform and muscular activity was measured by electromyography derived bilaterally from the erector spinae. Afterwards, participants were asked to complete a questionnaire on their current mood, pain rating, well-being, disability and physical activity. Descriptive data from the questionnaire showed an average acute pain score of 2.6 and an average pain score of 5.1 for the prior six weeks in the NSLBP group, while the control group reported an acute pain of 0.1 and an average pain of 0.5. For wellbeing, differences were found only for the physical scale. Average disability was low for the NSLBP group. No differences in physical activity were found among groups. A repeated measures ANOVA did not show significant differences between groups for any parameter. There was also no main effect for the standing conditions and no interaction between group and condition. Simultaneously measuring biomechanical and neuromuscular parameters, allowed for a fine grain approach to understanding motor control in individuals with NSLBP. This study did not find differences as described in the literature, and suggests further examination of factors involved in pain and control processes to better understand implications of NSLBP and possible avenues for support.

A Survey of Full-Body Motion Reconstruction in Immersive Virtual Reality Applications.

Due to recent advances in virtual reality (VR) technology, the development of immersive VR applications that track body motions and visualize a full-body avatar is attracting increasing research interest. This paper reviews related research to gather and to critically analyze recent improvements regarding the potential of full-body motion reconstruction in VR applications. We conducted a systematic literature search, matching VR and full-body tracking related keywords on IEEE Xplore, PubMed, ACM, and Scopus. Fifty-three publications were included and assigned in three groups: studies using markerless and marker-based motion tracking systems as well as systems using inertial measurement units. All analyzed research publications track the motions of the user wearing a head-mounted display and visualize a full-body avatar. The analysis confirmed that a full-body avatar can enhance the sense of embodiment and can improve the immersion within the VR. The results indicated that the Kinect device is still the most frequently used sensor (27 out of 53). Furthermore, there is a trend to track the movements of multiple users simultaneously. Many studies that enable multiplayer mode in VR use marker-based systems (7 out of 17) because they are much more robust and can accurately track full-body movements of multiple users in real-time.

Development of a Classifier to Determine Factors Causing Cybersickness in Virtual Reality Environments.

Objective: The goal of this contribution is to develop a classifier able to determine if cybersickness (CS) has occurred after immersion in a virtual reality (VR) scenario, based on a combination of biosignals and game parameters. Methods: We collected electrocardiographic, electrooculographic, respiratory, and skin conductivity data from a total of 66 participants. In addition, we also captured relevant game parameters such as avatar linear and angular speed as well as acceleration, head movements, and on-screen collisions. The data were collected while the participants were in a 10-minute VR experience, which was developed in Unity. The experience forced rotation and lateral movements upon the participants to provoke CS. A baseline was captured during a first simple scenario. The data were then split in per-level, per-60-second, and per-30-second windows. Furthermore, participants filled a pre- and postimmersion simulator sickness questionnaire. Simulator sickness scores were then used as a reference for binary (CS vs. no CS) and ternary (no CS-mild CS-severe CS) classification patterns. Several classification methods (support vector machines, K-nearest neighbors, and neural networks) were tested. Results: A maximum classification accuracy of 82% was achieved for binary classification and 56% for ternary classification. Conclusion: Given the sample size and the variety of movement patterns presented in the demonstration, we conclude that a combination of biosignals and game parameters suffice to determine the occurrence of CS. However, substantial further research is required to improve binary classification accuracy to adequate values for real-life scenarios and to determine better approaches to classify its severity.

The pupil response as an indicator of user experience in a digital exercise game.

To study whether psychophysiological indicators are suitable measures of user experience in a digital exercise game (exergame), a laboratory study employing both psychophysiological and self-report measures was conducted. Sixty-six participants cycled for 10 min on an ergometer while pupil diameter, skin conductance, and heart rate were measured; afterward, they completed a user experience questionnaire. The participants performed under three experimental conditions varying between subjects: active gaming (participants controlled the altitude of a digital bird by varying their pedal rate in order to catch letters flying across the screen), observing a game (they observed a replay of another participant's game), and no-game (blank screen). Only the gaming condition showed evidence for statistically significant pupil dilations-indicating emotional arousal-in response to game events (catching a letter) or corresponding points in time. The observational condition did not differ statistically from the no-game control condition. Self-reports also indicated that the gaming condition was rated most fun and least demanding. Other psychophysiological indicators (heart rate, skin conductance) showed no systematic effects in response to game events, rather they steadily increased during training. Thus, pupil responses were shown to be suitable indicators of positive emotional reactions to game events and user experience in a (training) game.

Recent advances in rehabilitation for Parkinson's Disease with Exergames: A Systematic Review.

OBJECTIVE: The goal of this contribution is to gather and to critically analyze recent evidence regarding the potential of exergaming for Parkinson's disease (PD) rehabilitation and to provide an up-to-date analysis of the current state of studies on exergame-based therapy in PD patients. METHODS: We performed our search based on the conclusions of a previous systematic review published in 2014. Inclusion criteria were articles published in the indexed databases Pubmed, Scopus, Sciencedirect, IEEE and Cochrane published since January 1, 2014. Exclusion criteria were papers with a target group other than PD patients exclusively, or contributions not based on exergames. Sixty-four publications out of 525 matches were selected. RESULTS: The analysis of the 64 selected publications confirmed the putative improvement in motor skills suggested by the results of the previous review. The reliability and safety of both Microsoft Kinect and Wii Balance Board in the proposed scenarios was further confirmed by several recent studies. Clinical trials present better (n = 5) or similar (n = 3) results than control groups (traditional rehabilitation or regular exercise) in motor (TUG, BBS) and cognitive (attention, alertness, working memory, executive function), thus emphasizing the potential of exergames in PD. Pilot studies (n = 11) stated the safety and feasibility of both Microsoft Kinect and Wii Balance Board, potentially in home scenarios as well. Technical papers (n = 30) stated the reliability of balance and gait data captured by both devices. Related meta-analyses and systematic reviews (n = 15) further support these statements, generally citing the need for adaptation to patient's skills and new input devices and sensors as identified gaps. CONCLUSION: Recent evidence indicates exergame-based therapy has been widely proven to be feasible, safe, and at least as effective as traditional PD rehabilitation. Further insight into new sensors, best practices and different cognitive stadiums of PD (such as PD with Mild Cognitive Impairment), as well as task specificity, are required. Also, studies linking game parameters and results with traditional assessment methods, such as UPDRS scores, are required. Outcomes for randomized controlled trials (RCTs) should be standardized, and follow-up studies are required, particularly for motor outcomes.

Serious Games for Health: The Potential of Metadata.

OBJECTIVE: Numerous serious games and health games exist, either as commercial products (typically with a focus on entertaining a broad user group) or smaller games and game prototypes, often resulting from research projects (typically tailored to a smaller user group with a specific health characteristic). A major drawback of existing health games is that they are not very well described and attributed with (machine-readable, quantitative, and qualitative) metadata such as the characterizing goal of the game, the target user group, or expected health effects well proven in scientific studies. This makes it difficult or even impossible for end users to find and select the most appropriate game for a specific situation (e.g., health needs). Therefore, the aim of this article was to motivate the need and potential/benefit of metadata for the description and retrieval of health games and to describe a descriptive model for the qualitative description of games for health. It was not the aim of the article to describe a stable, running system (portal) for health games. This will be addressed in future work. METHODS: Building on previous work toward a metadata format for serious games, a descriptive model for the formal description of games for health is introduced. For the conceptualization of this model, classification schemata of different existing health game repositories are considered. The classification schema consists of three levels: a core set of mandatory descriptive fields relevant for all games for health application areas, a detailed level with more comprehensive, optional information about the games, and so-called extension as level three with specific descriptive elements relevant for dedicated health games application areas, for example, cardio training. CONCLUSION: A metadata format provides a technical framework to describe, find, and select appropriate health games matching the needs of the end user. Future steps to improve, apply, and promote the metadata format in the health games market are discussed.

Personalized Adaptive Control of Training Load in Cardio-Exergames--A Feasibility Study.

OBJECTIVE: This article presents a feasibility study of using an algorithm for an individual and adaptive control of training load in an ergometer-controlled exergame for aerobic training. An additional goal was to investigate the effects of the adaptive game on the players' motivation. MATERIALS AND METHODS: A two-phase approach (calibration and exercise phase) was applied in a sample of 16 physically active adults. In the cardio-exergame "LetterBird," the flight of a pigeon was controlled by the pedaling rate of a bike ergometer as input device. During the calibration phase the individual heart rate (HR) responses of the players were measured. In the exercise phase, these data were used to adjust the resistance of the ergometer using the proposed algorithm. The purpose of this algorithm was to induce an individually defined target HR and to keep it in a steady state. In order to establish a reference for further studies, the game experience was measured using the kids-Game Experience Questionnaire. RESULTS: In 15 of 16 participants the actual HR reached the intended individual HR range within 10 minutes after onset of exercise. However, the induced HR initially exceeded the target HR in 13 participants, which made load adjustments necessary. The analysis of the kids-Game Experience Questionnaire confirmed the motivational effect of the exergame "LetterBird." CONCLUSIONS: The results confirm that the proposed algorithm for personalized HR control in the game "LetterBird" is feasible. Furthermore, the cardio-exergame "LetterBird" seems to have a substantial short-term motivating effect.