Fine-Grained Motion Recognition in At-Home Fitness Monitoring with Smartwatch: A Comparative Analysis of Explainable Deep Neural Networks.

The squat is a multi-joint exercise widely used for everyday at-home fitness. Focusing on the fine-grained classification of squat motions, we propose a smartwatch-based wearable system that can recognize subtle motion differences. For data collection, 52 participants were asked to perform one correct squat and five incorrect squats with three different arm postures (straight arm, crossed arm, and hands on waist). We utilized deep neural network-based models and adopted a conventional machine learning method (random forest) as a baseline. Experimental results revealed that the bidirectional GRU/LSTMs with an attention mechanism and the arm posture of hands on waist achieved the best test accuracy (F1-score) of 0.854 (0.856). High-dimensional embeddings in the latent space learned by attention-based models exhibit more clustered distributions than those by other DNN models, indicating that attention-based models learned features from the complex multivariate time-series motion signals more efficiently. To understand the underlying decision-making process of the machine-learning system, we analyzed the result of attention-based RNN models. The bidirectional GRU/LSTMs show a consistent pattern of attention for defined squat classes, but these models weigh the attention to the different kinematic events of the squat motion (e.g., descending and ascending). However, there was no significant difference found in classification performance.

Pre-Attentional Effects on Global Precedence Processing in Children with Autism Spectrum Disorder and Those with Typical Development on a Tablet-Based Modified Navon's Paradigm Task.

This study aimed to characterize the pre-attentional effects on global precedence processing in children with autism spectrum disorder (ASD) and those with typical development (TD). A sample of 17 participants, comprising eight children with ASD and nine TD children, were recruited for the study. A tablet-based assessment utilizing a global and local visual processing paradigm task was developed to investigate the participant's abilities. The task consisted of verbal instructions to locate and touch either a global or local figure, presented in five conditions: neutral, congruent, and incongruent. The percentage of correct answers and reaction time (RT) for each task were measured and analyzed statistically. Results revealed that children with ASD exhibited statistically significant differences in both the percentage of correct scores and RT among various conditions, while TD children displayed differences in RT but not in the percentage of correct answers. These findings suggest that conflicting processes affect both behavioral and cognitive processes in children with ASD, and that cognitive effort is still involved for children with TD, but does not affect behavioral processes. In children with ASD, the RT was the shortest in the congruent (report local figure) condition; in children with TD, the RT was the shortest in the congruent (report global figure) condition. This implies that children with TD exhibit a pre-attentive effect on global precedence processing, while children with ASD do not. These visual-processing-function characteristics may aid in screening for visual perception problems in children with ASD.

Persistence of metric biases in body representation during the body ownership illusion.

Our perception of the body's metric is influenced by bias according to the axis, called the systematic metric bias in body representation. Systematic metric bias was first reported as Weber's illusion and observed in several parts of the body in various patterns. However, the systematic metric bias was not observed with a fake hand under the influence of the body ownership illusion during the line length judgment task. The lack of metric bias observed during the line length judgment task with a fake hand implies that the tactile modality occupies a relatively less dominant position than perception occurring through the real body. The change in weight between visual and tactile modalities during the body ownership illusion has not been adequately investigated yet, despite being a factor that influences the perception through body ownership illusion. Therefore, this study aimed to investigate whether the dominance of vision over tactile modality is prominent, regardless of the task type. To investigate whether visual dominance persists during the process of inducing body ownership illusion regardless of task type, we introduced spatial visuotactile incongruence (2 cm, 3 cm) in the longitudinal and transverse axes during the visuotactile localization tasks and measured the intensity of the body ownership illusion using a questionnaire. The results indicated that participants perceived smaller visuotactile incongruence when the discrepancy occurred in the transverse axis rather than in the longitudinal axis. The anisotropy in the tolerance of visuotactile incongruence implies the persistence of metric biases in body representation. The results suggest the need for further research regarding the factors influencing the weight of visual and tactile modalities.

Estimation of Fine-Grained Foot Strike Patterns with Wearable Smartwatch Devices.

People who exercise may benefit or be injured depending on their foot striking (FS) style. In this study, we propose an intelligent system that can recognize subtle differences in FS patterns while walking and running using measurements from a wearable smartwatch device. Although such patterns could be directly measured utilizing pressure distribution of feet while striking on the ground, we instead focused on analyzing hand movements by assuming that striking patterns consequently affect temporal movements of the whole body. The advantage of the proposed approach is that FS patterns can be estimated in a portable and less invasive manner. To this end, first, we developed a wearable system for measuring inertial movements of hands and then conducted an experiment where participants were asked to walk and run while wearing a smartwatch. Second, we trained and tested the captured multivariate time series signals in supervised learning settings. The experimental results obtained demonstrated high and robust classification performances (weighted-average F1 score > 90%) when recent deep neural network models, such as 1D-CNN and GRUs, were employed. We conclude this study with a discussion of potential future work and applications that increase benefits while walking and running properly using the proposed approach.

A curtailed task for quantitative evaluation of visuomotor adaptation in the head-mounted display virtual reality environment.

To accurately perform a goal-directed movement in continuously changing environments, it is unavoidable for individuals to adapt accordingly. The cerebellum has been known to be responsible for such process, specifically adaptation using sensorimotor information. As shown in previous studies, using HMD-VR technology in an experimental setting has similar advantages as in the real-world environment: researchers can manipulate the experimental environment, precisely control the experiments, and quantitatively analyze errors in real time. Moreover, the HMD-VR environment provides high immersiveness and embodiment which even enhance motor learning and increase engagement and motivation of individuals more than real-world environments do. In our HMD-VR-based task, the subjects were trained to adapt to a condition in which the visual information was artificially 20°clockwise rotated from the actual cursor movement. The subjects used a virtual reality tracker to move the cursor from a starting point to a target that appeared randomly at five locations, 20 cm from the starting point with an interval of 15°. Although no significant side effects were expected from experiencing the HMD-VR environment, we considered the appropriate number of trials for patients with cerebellar disease for future use in clinical settings. To examine the feasibility of our task for analysis of visuomotor adaptation pattern as shown in a real-world-based task, we created and compared two paradigms with a difference in the number of trials. As we expected, the results showed that the heading angle error decreased as the participants of both paradigms continued the task and that there was no significant difference between the two paradigms. Next, we applied our short task paradigm to patients diagnosed with cerebellar ataxia and age-matched controls for further examination of applicability to diagnosis and rehabilitation of the patients. As a result, we observed the distinguishable adaptation pattern of the patient group by using our paradigm. Overall, the results suggest that our paradigm is feasible to analyze the visuomotor adaptation pattern of healthy individuals and patients with cerebellar ataxia so may contribute to the clinical field.

Recognition of Fine-Grained Walking Patterns Using a Smartwatch with Deep Attentive Neural Networks.

Generally, people do various things while walking. For example, people frequently walk while looking at their smartphones. Sometimes we walk differently than usual; for example, when walking on ice or snow, we tend to waddle. Understanding walking patterns could provide users with contextual information tailored to the current situation. To formulate this as a machine-learning problem, we defined 18 different everyday walking styles. Noting that walking strategies significantly affect the spatiotemporal features of hand motions, e.g., the speed and intensity of the swinging arm, we propose a smartwatch-based wearable system that can recognize these predefined walking styles. We developed a wearable system, suitable for use with a commercial smartwatch, that can capture hand motions in the form of multivariate timeseries (MTS) signals. Then, we employed a set of machine learning algorithms, including feature-based and recent deep learning algorithms, to learn the MTS data in a supervised fashion. Experimental results demonstrated that, with recent deep learning algorithms, the proposed approach successfully recognized a variety of walking patterns, using the smartwatch measurements. We analyzed the results with recent attention-based recurrent neural networks to understand the relative contributions of the MTS signals in the classification process.

Association of Extensive Video Gaming and Cognitive Function Changes in Brain-Imaging Studies of Pro Gamers and Individuals With Gaming Disorder: Systematic Literature Review.

BACKGROUND: The World Health Organization announced the inclusion of gaming disorder (GD) in the International Classification of Diseases, 11th Revision, despite some concerns. However, video gaming has been associated with the enhancement of cognitive function. Moreover, despite comparable extensive video gaming, pro gamers have not shown any of the negative symptoms that individuals with GD have reported. It is important to understand the association between extensive video gaming and alterations in brain regions more objectively. OBJECTIVE: This study aimed to systematically explore the association between extensive video gaming and changes in cognitive function by focusing on pro gamers and individuals with GD. METHODS: Studies about pro gamers and individuals with GD were searched for in the PubMed and Web of Science databases using relevant search terms, for example, "pro-gamers" and "(Internet) gaming disorder." While studies for pro gamers were searched for without date restrictions, only studies published since 2013 about individuals with GD were included in search results. Article selection was conducted by following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. RESULTS: By following the PRISMA guidelines, 1903 records with unique titles were identified. Through the screening process of titles and abstracts, 86 full-text articles were accessed to determine their eligibility. A total of 18 studies were included in this systematic review. Among the included 18 studies, six studies included pro gamers as participants, one study included both pro gamers and individuals with GD, and 11 studies included individuals with GD. Pro gamers showed structural and functional alterations in brain regions (eg, the left cingulate cortex, the insula subregions, and the prefrontal regions). Cognitive function (eg, attention and sensorimotor function) and cognitive control improved in pro gamers. Individuals with GD showed structural and functional alterations in brain regions (eg, the striatum, the orbitofrontal cortex, and the amygdala) that were associated with impaired cognitive control and higher levels of craving video game playing. They also showed increased cortical thickness in the middle temporal cortex, which indicated the acquisition of better skills. Moreover, it was suggested that various factors (eg, gaming expertise, duration or severity of GD, and level of self-control) seemed to modulate the association of extensive video game playing with changes in cognitive function. CONCLUSIONS: Although a limited number of studies were identified that included pro gamers and/or individuals who reported showing symptoms of GD for more than 1 year, this review contributed to the objective understanding of the association between extensive video game playing and changes in cognitive function. Conducting studies with a longitudinal design or with various comparison groups in the future would be helpful in deepening the understanding of this association.

Association of Depression With Functional Mobility in Schizophrenia.

BACKGROUND: Functional immobility can cause functional disability in patients with schizophrenia and has been linked to prognosis and mortality. Although depression might be a barrier for physical activity engagement, scarce data are present on the relationship between depression and functional mobility (FM) in schizophrenia. Thus, we aimed to investigate the associations among FM, depression, and other clinical correlates in individuals with schizophrenia. METHODS: FM was evaluated by the pedometer-assessed daily steps and Timed Up-and-Go (TUG) test in the daily-living and clinical settings, respectively. Psychiatric symptoms were assessed using the Beck Depression Inventory, Brief Psychiatric Rating Scale (BPRS), and State-Trait Anxiety Inventory. Cognitive function was evaluated using the Sternberg Working Memory (SWM) Task. Multiple regression analyses were performed to identify predictive factors associated with FM, with adjustment for relevant covariates. RESULTS: Sixty patients were enrolled in this study. Depression was the most consistent explanatory variable for both pedometer (ß = -0.34, p = 0.011) and TUG time (ß = 0.32, p = 0.018). Additionally, SWM accuracy (ß = -0.29, p = 0.018), BPRS-Withdrawal (ß = 0.19, p = 0.139), and fasting blood sugar (ß = 0.34, p = 0.008) were associated with TUG time. However, psychotic symptoms and anxiety were not associated with pedometer and TUG. CONCLUSIONS: We identified an association between depression and FM after adjusting for other disorder-related correlates in schizophrenia. Since the intervention goal is functional recovery, improving FM by treating depression may have considerable therapeutic value.

Commercial video games and cognitive functions: video game genres and modulating factors of cognitive enhancement.

BACKGROUND: Unlike the emphasis on negative results of video games such as the impulsive engagement in video games, cognitive training studies in individuals with cognitive deficits showed that characteristics of video game elements were helpful to train cognitive functions. Thus, this study aimed to have a more balanced view toward the video game playing by reviewing genres of commercial video games and the association of video games with cognitive functions and modulating factors. Literatures were searched with search terms (e.g. genres of video games, cognitive training) on database and Google scholar. RESULTS: video games, of which purpose is players' entertainment, were found to be positively associated with cognitive functions (e.g. attention, problem solving skills) despite some discrepancy between studies. However, the enhancement of cognitive functions through video gaming was limited to the task or performance requiring the same cognitive functions. Moreover, as several factors (e.g. age, gender) were identified to modulate cognitive enhancement, the individual difference in the association between video game playing and cognitive function was found. CONCLUSION: Commercial video games are suggested to have the potential for cognitive function enhancement. As understanding the association between video gaming and cognitive function in a more balanced view is essential to evaluate the potential outcomes of commercial video games that more people reported to engage, this review contributes to provide more objective evidence for commercial video gaming.

Physical performance is more strongly associated with cognition in schizophrenia than psychiatric symptoms.

BACKGROUND: Although neurocognitive dysfunction and physical performance are known to be impaired in patients with schizophrenia, evidence regarding the relationship between these two domains remains insufficient. Thus, we aimed to investigate the relationship between various physical performance domains and cognitive domains in individuals with schizophrenia, while considering other disorder-related clinical symptoms. METHODS: Sixty patients with schizophrenia participated in the study. Cardiorespiratory fitness and functional mobility were evaluated using the step test and supine-to-standing (STS) test, respectively. Executive function and working memory were assessed using the Stroop task and Sternberg working memory (SWM) task, respectively. Clinical symptoms were evaluated using the Brief Psychiatric Rating Scale, Beck Depression Inventory, and State-Trait Anxiety Inventory. Multivariate analyses were performed to adjust for relevant covariates and identify predictive factors associated with neurocognition. RESULTS: Multiple regression analysis revealed that the step test index was most strongly associated with reaction time in the Stroop task (ß = 0.434, p = 0.001) and SWM task (ß = 0.331, p = 0.026), while STS test time was most strongly associated with accuracy on the Stoop task (ß=-0.418, p = 0.001) and SWM task (ß=-0.383, p = 0.007). Total cholesterol levels were positively associated with Stroop task accuracy (ß=-0.307, p = 0.018) after controlling for other clinical correlates. However, clinical symptoms were not associated with any variables in Stroop or SWM task. CONCLUSIONS: The present findings demonstrate the relationship between physical performance and neurocognition in patients with schizophrenia. Considering that these factors are modifiable, exercise intervention may help to improve cognitive symptoms in patients with schizophrenia, thereby leading to improvements in function and prognosis.