A Skeleton-Based Rehabilitation Exercise Assessment System With Rotation Invariance.

Automated exercise assessment is of great importance for patients under rehabilitation exercise who require professional guidance. Among the existing approaches, the skeleton-based assessment model that classifies the correctness of the exercise has attracted much attention due to its relative ease of implementation and convenience in use. However, there are two problems with this approach. The first problem is its sensitivity to the orientation of the human skeleton. To solve this problem, we propose a novel rotation-invariant descriptor, the dot product matrix of the human skeleton, and prove mathematically that this descriptor discards only the orientation message that we do not expect while preserving all other useful information. Lack of feedback from the system is the second problem, because the exercisers do not know which parts of their exercises are incorrect. Therefore, we develop a visualization method for our system based on Gradient-Weighted Class Activation Mapping (Grad-CAM) and an quantitative metric called Overlap Ratio (OvR) to measure the quality of the visualization result. To demonstrate the effect of our method, we conduct experiments on two public datasets and a self-generated push-up dataset. The experimental results show that our rotation-invariant descriptor can achieve absolute robustness to orientation even under severe angle perturbations. In terms of accuracy and OvR, our method even outperforms previous works in most cases, indicating that the rotation-invariant descriptor helps the assessment model to extract more stable features. The visualization results are also informative to correct the movements; some examples are presented in this paper. The code of this paper and our push-up dataset are publicly available at https://github.com/Kelly510/RehabExerAssess.

Exploiting domain transformation and deep learning for hand gesture recognition using a low-cost dataglove.

Hand gesture recognition is one of the most widely explored areas under the human-computer interaction domain. Although various modalities of hand gesture recognition have been explored in the last three decades, in recent years, due to the availability of hardware and deep learning algorithms, hand gesture recognition research has attained renewed momentum. In this paper, we evaluate the effectiveness of a low-cost dataglove for classifying hand gestures in the light of deep learning. We have developed a cost-effective dataglove using five flex sensors, an inertial measurement unit, and a powerful microcontroller for onboard processing and wireless connectivity. We have collected data from 25 subjects for 24 static and 16 dynamic American sign language gestures for validating our system. Moreover, we proposed a novel Spatial Projection Image-based technique for dynamic hand gesture recognition. We also explored a parallel-path neural network architecture for handling multimodal data more effectively. Our method produced an F1-score of 82.19% for static gestures and 97.35% for dynamic gestures from a leave-one-out-cross-validation approach. Overall, this study demonstrates the promising performance of a generalized hand gesture recognition technique in hand gesture recognition. The dataset used in this work has been made publicly available.

Similarities and disparities between visual analysis and high-resolution electromyography of facial expressions.

Computer vision (CV) is widely used in the investigation of facial expressions. Applications range from psychological evaluation to neurology, to name just two examples. CV for identifying facial expressions may suffer from several shortcomings: CV provides indirect information about muscle activation, it is insensitive to activations that do not involve visible deformations, such as jaw clenching. Moreover, it relies on high-resolution and unobstructed visuals. High density surface electromyography (sEMG) recordings with soft electrode array is an alternative approach which provides direct information about muscle activation, even from freely behaving humans. In this investigation, we compare CV and sEMG analysis of facial muscle activation. We used independent component analysis (ICA) and multiple linear regression (MLR) to quantify the similarity and disparity between the two approaches for posed muscle activations. The comparison reveals similarity in event detection, but discrepancies and inconsistencies in source identification. Specifically, the correspondence between sEMG and action unit (AU)-based analyses, the most widely used basis of CV muscle activation prediction, appears to vary between participants and sessions. We also show a comparison between AU and sEMG data of spontaneous smiles, highlighting the differences between the two approaches. The data presented in this paper suggests that the use of AU-based analysis should consider its limited ability to reliably compare between different sessions and individuals and highlight the advantages of high-resolution sEMG for facial expression analysis.

Synchronization criteria of delayed inertial neural networks with generally Markovian jumping.

In this paper, the synchronization problem of inertial neural networks with time-varying delays and generally Markovian jumping is investigated. The second order differential equations are transformed into the first-order differential equations by utilizing the variable transformation method. The Markovian process in the systems is uncertain or partially known due to the delay of data transmission channel or the loss of data information, which is more general and practicable to consider generally Markovian jumping inertial neural networks. The synchronization criteria can be obtained by using the delay-dependent Lyapunov-Krasovskii functionals and higher order polynomial based relaxed inequality (HOPRII). In addition, the desired controllers are obtained by solving a set of linear matrix inequalities. Finally, the numerical examples are provided to demonstrate the effectiveness of the theoretical results.

Human Gait Recognition Based on Self-Adaptive Hidden Markov Model.

Human gait recognition has numerous challenges due to view angle changing, human dressing, bag carrying, and pedestrian walking speed, etc. In order to increase gait recognition accuracy under these circumstances, in this paper we propose a method for gait recognition based on a self-adaptive hidden Markov model (SAHMM). First, we present a feature extraction algorithm based on local gait energy image (LGEI) and construct an observation vector set. By using this set, we optimize parameters of the SAHMM-based method for gait recognition. Finally, the proposed method is evaluated extensively based on the CASIA Dataset B for gait recognition under various conditions such as cross view, human dressing, or bag carrying, etc. Furthermore, the generalization ability of this method is verified based on the OU-ISIR Large Population Dataset. Both experimental results show that the proposed method exhibits superior performance in comparison with those existing methods.

A brain-inspired network architecture for cost-efficient object recognition in shallow hierarchical neural networks.

The brain successfully performs visual object recognition with a limited number of hierarchical networks that are much shallower than artificial deep neural networks (DNNs) that perform similar tasks. Here, we show that long-range horizontal connections (LRCs), often observed in the visual cortex of mammalian species, enable such a cost-efficient visual object recognition in shallow neural networks. Using simulations of a model hierarchical network with convergent feedforward connections and LRCs, we found that the addition of LRCs to the shallow feedforward network significantly enhances the performance of networks for image classification, to a degree that is comparable to much deeper networks. We found that a combination of sparse LRCs and dense local connections dramatically increases performance per wiring cost. From network pruning with gradient-based optimization, we also confirmed that LRCs could emerge spontaneously by minimizing the total connection length while maintaining performance. Ablation of emerged LRCs led to a significant reduction of classification performance, which implies these LRCs are crucial for performing image classification. Taken together, our findings suggest a brain-inspired strategy for constructing a cost-efficient network architecture to implement parsimonious object recognition under physical constraints such as shallow hierarchical depth.

Texture Feature-Based Classification on Transrectal Ultrasound Image for Prostatic Cancer Detection.

Prostate cancer is one of the most common cancers in men. Early detection of prostate cancer is the key to successful treatment. Ultrasound imaging is one of the most suitable methods for the early detection of prostate cancer. Although ultrasound images can show cancer lesions, subjective interpretation is not accurate. Therefore, this paper proposes a transrectal ultrasound image analysis method, aiming at characterizing prostate tissue through image processing to evaluate the possibility of malignant tumours. Firstly, the input image is preprocessed by optical density conversion. Then, local binarization and Gaussian Markov random fields are used to extract texture features, and the linear combination is performed. Finally, the fused texture features are provided to SVM classifier for classification. The method has been applied to data set of 342 transrectal ultrasound images obtained from hospitals with an accuracy of 70.93%, sensitivity of 70.00%, and specificity of 71.74%. The experimental results show that it is possible to distinguish cancerous tissues from noncancerous tissues to some extent.

Recognition of Signed Expressions in an Experimental System Supporting Deaf Clients in the City Office.

The paper addresses the recognition of dynamic Polish Sign Language expressions in an experimental system supporting deaf people in an office when applying for an ID card. A method of processing a continuous stream of RGB-D data and a feature vector are proposed. The classification is carried out using the k-nearest neighbors algorithm with dynamic time warping, hidden Markov models, and bidirectional long short-term memory. The leave-one-subject-out protocol is used for the dataset containing 121 Polish Sign Language sentences performed five times by four deaf people. A data augmentation method is also proposed and tested. Preliminary observations and conclusions from the use of the system in a laboratory, as well as in real conditions with an experimental installation in the Office of Civil Affairs are given.

Multiple Object Tracking for Dense Pedestrians by Markov Random Field Model with Improvement on Potentials.

Pedestrian tracking in dense crowds is a challenging task, even when using a multi-camera system. In this paper, a new Markov random field (MRF) model is proposed for the association of tracklet couplings. Equipped with a new potential function improvement method, this model can associate the small tracklet coupling segments caused by dense pedestrian crowds. The tracklet couplings in this paper are obtained through a data fusion method based on image mutual information. This method calculates the spatial relationships of tracklet pairs by integrating position and motion information, and adopts the human key point detection method for correction of the position data of incomplete and deviated detections in dense crowds. The MRF potential function improvement method for dense pedestrian scenes includes assimilation and extension processing, as well as a message selective belief propagation algorithm. The former enhances the information of the fragmented tracklets by means of a soft link with longer tracklets and expands through sharing to improve the potentials of the adjacent nodes, whereas the latter uses a message selection rule to prevent unreliable messages of fragmented tracklet couplings from being spread throughout the MRF network. With the help of the iterative belief propagation algorithm, the potentials of the model are improved to achieve valid association of the tracklet coupling fragments, such that dense pedestrians can be tracked more robustly. Modular experiments and system-level experiments are conducted using the PETS2009 experimental data set, where the experimental results reveal that the proposed method has superior tracking performance.

Automated Brain Region Segmentation for Single Cell Resolution Histological Images Based on Markov Random Field.

The brain consists of massive regions with different functions and the precise delineation of brain region boundaries is important for brain region identification and atlas illustration. In this paper we propose a hierarchical Markov random field (MRF) model for brain region segmentation, where a MRF is applied to the downsampled low-resolution images and the result is used to initialize another MRF for the original high-resolution images. A fractional differential feature and a gray level co-occurrence matrix are extracted as the observed vector for the MRF and a new potential energy function, which can capture the spatial characteristic of brain regions, is proposed as well. A fuzzy entropy criterion is used to fine-tune the boundary from the hierarchical MRF model. We test the model both on synthetic images and real histological mouse brain images. The result suggests that the model can accurately identify target regions and even the whole mouse brain outline as a special case. An interesting observation is that the model cannot only segment regions with different cell density but also can segment regions with similar cell density and different cell morphology texture. Thus this model shows great potential for building the high-resolution 3D brain atlas.

Sequential Pattern Mining of Longitudinal Adverse Events After Left Ventricular Assist Device Implant.

Left ventricular assist devices (LVADs) are an increasingly common therapy for patients with advanced heart failure. However, implantation of the LVAD increases the risk of stroke, infection, bleeding, and other serious adverse events (AEs). Most post-LVAD AEs studies have focused on individual AEs in isolation, neglecting the possible interrelation, or causality between AEs. This study is the first to conduct an exploratory analysis to discover common sequential chains of AEs following LVAD implantation that are correlated with important clinical outcomes. This analysis was derived from 58,575 recorded AEs for 13,192 patients in International Registry for Mechanical Circulatory Support (INTERMACS) who received a continuous-flow LVAD between 2006 and 2015. The pattern mining procedure involved three main steps: (1) creating a bank of AE sequences by converting the AEs for each patient into a single, chronologically sequenced record, (2) grouping patients with similar AE sequences using hierarchical clustering, and (3) extracting temporal chains of AEs for each group of patients using Markov modeling. The mined results indicate the existence of seven groups of sequential chains of AEs, characterized by common types of AEs that occurred in a unique order. The groups were identified as: GRP1: Recurrent bleeding, GRP2: Trajectory of device malfunction & explant, GRP3: Infection, GRP4: Trajectories to transplant, GRP5: Cardiac arrhythmia, GRP6: Trajectory of neurological dysfunction & death, and GRP7: Trajectory of respiratory failure, renal dysfunction & death. These patterns of sequential post-LVAD AEs disclose potential interdependence between AEs and may aid prediction, and prevention, of subsequent AEs in future studies.

A High Precision Real-time Premature Ventricular Contraction Assessment Method based on the Complex Feature Set.

This paper presents a high precision and low computational complexity premature ventricular contraction (PVC) assessment method for the ECG human-machine interface device. The original signals are preprocessed by integrated filters. Then, R points and surrounding feature points are determined by corresponding detection algorithms. On this basis, a complex feature set and feature matrices are obtained according to the position feature points. Finally, an exponential Minkowski distance method is proposed for PVC recognition. Both public dataset and clinical experiments were utilized to verify the effectiveness and superiority of the proposed method. The results show that our R peak detection algorithm can substantially reduce the error rate, and obtained 98.97% accuracy for QRS complexes. Meanwhile, the accuracy of PVC recognition was 98.69% for the MIT-BIH database and 98.49% for clinical tests. Moreover, benefiting from the lightweight of our model, it can be easily applied to portable healthcare devices for human-computer interaction.

Accuracy of digital dental models using the low-cost DAVID laser scanner.

BACKGROUND: Accurate laser scanning of plaster casts using validated, low-cost hardware represents a key issue in 3D orthodontics. OBJECTIVES: The aim of this study was to compare the accuracy of measurements taken from plaster casts (gold standard) with digital models of those casts created with a low-cost structural light DAVID laser scanner. MATERIAL AND METHODS: Five different measurements were taken on each of 14 plaster casts by 2 independent observers with an electronic caliper. The measurements were repeated 10 times on all 14 plaster casts by each observer, with a 1-week interval between each set of measurements. All 14 plaster casts were digitized using a low-cost DAVID SLS 3 laser scanner. The same 5 measurements were performed on each of the 3D virtual surface models of the 14 plaster casts by 2 independent observers using Meshlab software in a manner similar to that used with the digital caliper. The measurements were repeated 10 times by the 2 observers with 1 week between each set of measurements. RESULTS: The laser-scanned models were more accurate than the plaster cast models in defining measurements based on simple tooth fissures. The accuracy of measurements based on complex tooth fissures were equivalent for the 2 types of model. For measurements based on interproximal dental contacts, the 2 methods of measurement were similar and both were notably poor in terms of accuracy. CONCLUSIONS: Three-dimensional virtual models obtained from the low-cost DAVID laser scanner can be used clinically, but only for certain types of measurements and indications.

Retinal image assessment using bi-level adaptive morphological component analysis.

The automated analysis of retinal images is a widely researched area which can help to diagnose several diseases like diabetic retinopathy in early stages of the disease. More specifically, separation of vessels and lesions is very critical as features of these structures are directly related to the diagnosis and treatment process of diabetic retinopathy. The complexity of the retinal image contents especially in images with severe diabetic retinopathy makes detection of vascular structure and lesions difficult. In this paper, a novel framework based on morphological component analysis (MCA) is presented which benefits from the adaptive representations obtained via dictionary learning. In the proposed Bi-level Adaptive MCA (BAMCA), MCA is extended to locally deal with sparse representation of the retinal images at patch level whereas the decomposition process occurs globally at the image level. BAMCA method with appropriately offline learnt dictionaries is adopted to work on retinal images with severe diabetic retinopathy in order to simultaneously separate vessels and exudate lesions as diagnostically useful morphological components. To obtain the appropriate dictionaries, K-SVD dictionary learning algorithm is modified to use a gated error which guides the process toward learning the main structures of the retinal images using vessel or lesion maps. Computational efficiency of the proposed framework is also increased significantly through some improvement leading to noticeable reduction in run time. We experimentally show how effective dictionaries can be learnt which help BAMCA to successfully separate exudate and vessel components from retinal images even in severe cases of diabetic retinopathy. In this paper, in addition to visual qualitative assessment, the performance of the proposed method is quantitatively measured in the framework of vessel and exudate segmentation. The reported experimental results on public datasets demonstrate that the obtained components can be used to achieve competitive results with regard to the state-of-the-art vessel and exudate segmentation methods.

Increase Trichomonas vaginalis detection based on urine routine analysis through a machine learning approach.

Trichomonas vaginalis (T. vaginalis) detection remains an unsolved problem in using of automated instruments for urinalysis. The study proposes a machine learning (ML)-based strategy to increase the detection rate of T. vaginalis in urine. On the basis of urinalysis data from a teaching hospital during 2009-2013, individuals underwent at least one urinalysis test were included. Logistic regression, support vector machine, and random forest, were used to select specimens with a high risk of T. vaginalis infection for confirmation through microscopic examinations. A total of 410,952 and 428,203 specimens from men and women were tested, of which 91 (0.02%) and 517 (0.12%) T. vaginalis-positive specimens were reported, respectively. The prediction models of T. vaginalis infection attained an area under the receiver operating characteristic curve of more than 0.87 for women and 0.83 for men. The Lift values of the top 5% risky specimens were above eight. While the most risky vigintile was picked out by the models and confirmed by microscopic examination, the incremental cost-effectiveness ratios for T. vaginalis detection in men and women were USD$170.1 and USD$29.7, respectively. On the basis of urinalysis, the proposed strategy can significantly increase the detection rate of T. vaginalis in a cost-effective manner.

Automatic ham classification method based on support vector machine model increases accuracy and benefits compared to manual classification.

The thickness of the subcutaneous fat (SFT) is a very important parameter in the ham, since determines the process the ham will be submitted. This study compares two methods to predict the SFT in slaughter line: an automatic system using an SVM model (Support Vector Machine) and a manual measurement of the fat carried out by an experienced operator, in terms of accuracy and economic benefit. These two methods were compared to the golden standard obtained by measuring SFT with a ruler in a sample of 400 hams equally distributed within each SFT class. The results show that the SFT prediction made by the SVM model achieves an accuracy of 75.3%, which represents an improvement of 5.5% compared to the manual measurement. Regarding economic benefits, SVM model can increase them between 12 and 17%. It can be concluded that the classification using SVM is more accurate than the one performed manually with an increase of the economic benefit for sorting.

Bayesian rank penalization.

Rank minimization is a key component of many computer vision and machine learning methods, including robust principal component analysis (RPCA) and low-rank representations (LRR). However, usual methods rely on optimization to produce a point estimate without characterizing uncertainty in this estimate, and also face difficulties in tuning parameter choice. Both of these limitations are potentially overcome with Bayesian methods, but there is currently a lack of general purpose Bayesian approaches for rank penalization. We address this gap using a positive generalized double Pareto prior, illustrating the approach in RPCA and LRR. Posterior computation relies on hybrid Gibbs sampling and geodesic Monte Carlo algorithms. We assess performance in simulation examples, and benchmark data sets.

MarmoDetector: A novel 3D automated system for the quantitative assessment of marmoset behavior.

BACKGROUND: Callithrix jacchus, generally known as the common marmoset, has recently garnered interest as an experimental primate model for better understanding the basis of human social behavior, architecture and function. Modelling human neurological and psychological diseases in marmosets can enhance the knowledge obtained from rodent research for future pre-clinical studies. Hence, comprehensive and quantitative assessments of marmoset behaviors are crucial. However, systems for monitoring and analyzing marmoset behaviors have yet to be established. NEW METHOD: In this paper, we present a novel multimodal system, MarmoDetector, for the automated 3D analysis of marmoset behavior under freely moving conditions. MarmoDetector allows the quantitative assessment of marmoset behaviors using computerised tracking analysis techniques that are based on a Kinect system equipped with video recordings, infrared images and depth analysis. RESULTS: Using MarmoDetector, we assessed behavioral circadian rhythms continuously over several days in home cages. In addition, MarmoDetector detected acute, transient complex behaviors of alcohol injected marmosets. COMPARISON TO EXISTING METHOD: Compared to 2D recording, MarmoDetector detects activities more precisely and is very sensitive as we could detect behavioral defects specifically induced by alcohol administration. CONCLUSION: MarmoDetector facilitates the rapid and accurate analysis of marmoset behavior and will enhance research on the neural basis of brain disorders.

Discovery and recognition of motion primitives in human activities.

We present a novel framework for the automatic discovery and recognition of motion primitives in videos of human activities. Given the 3D pose of a human in a video, human motion primitives are discovered by optimizing the 'motion flux', a quantity which captures the motion variation of a group of skeletal joints. A normalization of the primitives is proposed in order to make them invariant with respect to a subject anatomical variations and data sampling rate. The discovered primitives are unknown and unlabeled and are unsupervisedly collected into classes via a hierarchical non-parametric Bayes mixture model. Once classes are determined and labeled they are further analyzed for establishing models for recognizing discovered primitives. Each primitive model is defined by a set of learned parameters. Given new video data and given the estimated pose of the subject appearing on the video, the motion is segmented into primitives, which are recognized with a probability given according to the parameters of the learned models. Using our framework we build a publicly available dataset of human motion primitives, using sequences taken from well-known motion capture datasets. We expect that our framework, by providing an objective way for discovering and categorizing human motion, will be a useful tool in numerous research fields including video analysis, human inspired motion generation, learning by demonstration, intuitive human-robot interaction, and human behavior analysis.

Human Activity Recognition Using Gaussian Mixture Hidden Conditional Random Fields.

In healthcare, the analysis of patients' activities is one of the important factors that offer adequate information to provide better services for managing their illnesses well. Most of the human activity recognition (HAR) systems are completely reliant on recognition module/stage. The inspiration behind the recognition stage is the lack of enhancement in the learning method. In this study, we have proposed the usage of the hidden conditional random fields (HCRFs) for the human activity recognition problem. Moreover, we contend that the existing HCRF model is inadequate by independence assumptions, which may reduce classification accuracy. Therefore, we utilized a new algorithm to relax the assumption, allowing our model to use full-covariance distribution. Also, in this work, we proved that computation wise our method has very much lower complexity against the existing methods. For the experiments, we used four publicly available standard datasets to show the performance. We utilized a 10-fold cross-validation scheme to train, assess, and compare the proposed model with the conditional learning method, hidden Markov model (HMM), and existing HCRF model which can only use diagonal-covariance Gaussian distributions. From the experiments, it is obvious that the proposed model showed a substantial improvement with p value ≤0.2 regarding the classification accuracy.