Improving Small-Scale Human Action Recognition Performance Using a 3D Heatmap Volume.

In recent years, skeleton-based human action recognition has garnered significant research attention, with proposed recognition or segmentation methods typically validated on large-scale coarse-grained action datasets. However, there remains a lack of research on the recognition of small-scale fine-grained human actions using deep learning methods, which have greater practical significance. To address this gap, we propose a novel approach based on heatmap-based pseudo videos and a unified, general model applicable to all modality datasets. Leveraging anthropometric kinematics as prior information, we extract common human motion features among datasets through an ad hoc pre-trained model. To overcome joint mismatch issues, we partition the human skeleton into five parts, a simple yet effective technique for information sharing. Our approach is evaluated on two datasets, including the public Nursing Activities and our self-built Tai Chi Action dataset. Results from linear evaluation protocol and fine-tuned evaluation demonstrate that our pre-trained model effectively captures common motion features among human actions and achieves steady and precise accuracy across all training settings, while mitigating network overfitting. Notably, our model outperforms state-of-the-art models in recognition accuracy when fusing joint and limb modality features along the channel dimension.

Fingerprint matching using the onion peeling approach and turning function.

Fingerprint, as one of the most popular and robust biometric traits, can be used in automatic identification and verification systems to identify individuals. Fingerprint matching is a vital and challenging issue in fingerprint recognition systems. Most fingerprint matching algorithms are minutiae-based. The minutiae points are the ways that the fingerprint ridges can be discontinuous. Ridge ending and ridge bifurcation are two frequently used minutiae in most fingerprint matching algorithms. This article presents a new minutiae-based fingerprint matching using the onion peeling approach. In the proposed method, fingerprints are aligned to find the matched minutiae points. Then, the nested convex polygons of matched minutiae points are constructed and the comparison between peer-to-peer polygons is performed by the turning function distance. Simplicity, accuracy, and low time complexity of the onion peeling approach are three important factors that make it a standard method for fingerprint matching purposes. The performance of the proposed algorithm is evaluated on the database FVC2002. Since the fingerprints that the difference between the number of their layers is more than 2 and the a minutiae matching score lower than 0.15 are ignored, better results are obtained. KEYWORDS: Fingerprint Matching, Minutiae, Convex Layers, Turning Function, Computational Geometry.

Class discrepancy-guided sub-band filter-based common spatial pattern for motor imagery classification.

BACKGROUND: Motor imagery classification, an important branch of brain-computer interface (BCI), recognizes the intention of subjects to control external auxiliary equipment. Therefore, EEG-based motor imagery classification has received increasing attention in the fields of neuroscience. The common spatial pattern (CSP) algorithm has recently achieved great success in motor imagery classification. However, varying discriminative frequency bands and few-channel EEG limit the performance of CSP. NEW METHOD: A class discrepancy-guided sub-band filter-based CSP (CDFCSP) algorithm is proposed to automatically recognize and augment the discriminative frequency bands for CSP algorithms. Specifically, a priori knowledge and templates obtained from the training set were applied as the design guidelines of the class discrepancy-guided sub-band filter (CDF). Second, a filter bank CSP was used to extract features from EEG traces filtered by the CDF. Finally, the CSP features of multiple frequency bands were leveraged to train linear support vector machine classifier and generate prediction. RESULTS: BCI competition IV datasets 2a and 2b, which include EEGs from 18 subjects, were used to validate the performance improvement provided by the CDF. Student's t-tests of the CDFCSP versus the filter bank CSP without the CDF showed that the performance improvement was significant (i.e., p-values of 0.040 and 0.032 for the ratio and normalization mode CDFCSP, respectively). COMPARISON WITH EXISTING METHOD(S): The experiments show that the proposed CDFCSP improves the CSP algorithm and outperforms the other state-of-the-art algorithms evaluated in this paper. CONCLUSIONS: The increased performance of the proposed CDFCSP algorithm can promote the application of BCI systems.

Multi-modal imaging with specialized sequences improves accuracy of the automated subcortical grey matter segmentation.

The basal ganglia and limbic system, particularly the thalamus, putamen, internal and external globus pallidus, substantia nigra, and sub-thalamic nucleus, comprise a clinically relevant signal network for Parkinson's disease. In order to manually trace these structures, a combination of high-resolution and specialized sequences at 7 T are used, but it is not feasible to routinely scan clinical patients in those scanners. Targeted imaging sequences at 3 T have been presented to enhance contrast in a select group of these structures. In this work, we show that a series of atlases generated at 7 T can be used to accurately segment these structures at 3 T using a combination of standard and optimized imaging sequences, though no one approach provided the best result across all structures. In the thalamus and putamen, a median Dice Similarity Coefficient (DSC) over 0.88 and a mean surface distance <1.0 mm were achieved using a combination of T1 and an optimized inversion recovery imaging sequences. In the internal and external globus pallidus a DSC over 0.75 and a mean surface distance <1.2 mm were achieved using a combination of T1 and inversion recovery imaging sequences. In the substantia nigra and sub-thalamic nucleus a DSC of over 0.6 and a mean surface distance of <1.0 mm were achieved using the inversion recovery imaging sequence. On average, using T1 and optimized inversion recovery together significantly improved segmentation results than over individual modality (p < 0.05 Wilcoxon sign-rank test).

Parts-based representations of perceived face movements in the superior temporal sulcus.

Facial motion is a primary source of social information about other humans. Prior fMRI studies have identified regions of the superior temporal sulcus (STS) that respond specifically to perceived face movements (termed fSTS), but little is known about the nature of motion representations in these regions. Here we use fMRI and multivoxel pattern analysis to characterize the representational content of the fSTS. Participants viewed a set of specific eye and mouth movements, as well as combined eye and mouth movements. Our results demonstrate that fSTS response patterns contain information about face movements, including subtle distinctions between types of eye and mouth movements. These representations generalize across the actor performing the movement, and across small differences in visual position. Critically, patterns of response to combined movements could be well predicted by linear combinations of responses to individual eye and mouth movements, pointing to a parts-based representation of complex face movements. These results indicate that the fSTS plays an intermediate role in the process of inferring social content from visually perceived face movements, containing a representation that is sufficiently abstract to generalize across low-level visual details, but still tied to the kinematics of face part movements.

Development of an automated image analysis protocol for quantification of intracellular forms of Leishmania spp.

Leishmania parasites cause a set of neglected tropical diseases with considerable public health impact, the leishmaniases, which are often fatal if left untreated. Since current treatments for the leishmaniases exhibit high toxicity, low efficacy and prohibitive prices, many laboratories throughout the world are engaged in research for the discovery of novel chemotherapeutics. This entails the necessity of screening large numbers of compounds against the clinically relevant form of the parasite, the obligatory intracellular amastigote, a procedure that in many laboratories is still carried out by manual inspection. To overcome this well-known bottleneck in Leishmania drug development, several studies have recently attempted to automate this process. Here we implemented an image-based high content triage assay for Leishmania which has the added advantages of using primary macrophages instead of macrophage cell lines and of enabling identification of active compounds against parasite species developing both in small individual phagolysosomes (such as L. infantum) and in large communal vacuoles (such as L. amazonensis). The automated image analysis protocol is made available for IN Cell Analyzer systems, and, importantly, also for the open-source CellProfiler software, in this way extending its implementation to any laboratory involved in drug development as well as in other aspects of Leishmania research requiring analysis of in vitro infected macrophages.

Fully automatic detection and segmentation of abdominal aortic thrombus in post-operative CTA images using Deep Convolutional Neural Networks.

Computerized Tomography Angiography (CTA) based follow-up of Abdominal Aortic Aneurysms (AAA) treated with Endovascular Aneurysm Repair (EVAR) is essential to evaluate the progress of the patient and detect complications. In this context, accurate quantification of post-operative thrombus volume is required. However, a proper evaluation is hindered by the lack of automatic, robust and reproducible thrombus segmentation algorithms. We propose a new fully automatic approach based on Deep Convolutional Neural Networks (DCNN) for robust and reproducible thrombus region of interest detection and subsequent fine thrombus segmentation. The DetecNet detection network is adapted to perform region of interest extraction from a complete CTA and a new segmentation network architecture, based on Fully Convolutional Networks and a Holistically-Nested Edge Detection Network, is presented. These networks are trained, validated and tested in 13 post-operative CTA volumes of different patients using a 4-fold cross-validation approach to provide more robustness to the results. Our pipeline achieves a Dice score of more than 82% for post-operative thrombus segmentation and provides a mean relative volume difference between ground truth and automatic segmentation that lays within the experienced human observer variance without the need of human intervention in most common cases.

Estimation of Posturographic Trajectory Using k-Nearest Neighbors Classifier in Patients with Rheumatoid Arthritis and Osteoarthritis.

Rheumatoid arthritis (RA) and osteoarthritis (OA) are common rheumatic diseases and account for a significant percentage of disability. Posturography is a method that assesses postural stability and quantitatively evaluates postural sways. The objective of this study was to estimate posturographic trajectories applying pattern recognition algorithms. To this end, k-nearest neighbors (k-NN) classifier was used to differentiate between healthy subjects and patients with OA and RA. The following parameters of trajectories were computed: radius of sways, developed area, total length, and two directional components of sways: length of left-right and forward-backward motions. Posturographic tests were applied with eyes open and closed, and with biofeedback control. We found that in RA, the radius of sways, the trajectory area, and the biofeedback coordination were related to the patients' condition. The trajectory dynamics in OA patients were smaller compared to those in RA patients. The smallest misclassification errors were observed after feature selection in the biofeedback test compared with the eyes open and closed tests. We conclude that the estimation of posturographic trajectory with k-NN classifier could be helpful in monitoring the condition of RA patients.

Massively parallel C. elegans tracking provides multi-dimensional fingerprints for phenotypic discovery.

BACKGROUND: The nematode worm C. elegans is a model organism widely used for studies of genetics and of human disease. The health and fitness of the worms can be quantified in different ways, such as by measuring their bending frequency, speed or lifespan. Manual assays, however, are time consuming and limited in their scope providing a strong motivation for automation. NEW METHOD: We describe the development and application of an advanced machine vision system for characterising the behaviour of C. elegans, the Wide Field-of-View Nematode Tracking Platform (WF-NTP), which enables massively parallel data acquisition and automated multi-parameter behavioural profiling of thousands of worms simultaneously. RESULTS: We screened more than a million worms from several established models of neurodegenerative disorders and characterised the effects of potential therapeutic molecules for Alzheimer's and Parkinson's diseases. By using very large numbers of animals we show that the sensitivity and reproducibility of behavioural assays is very greatly increased. The results reveal the ability of this platform to detect even subtle phenotypes. COMPARISON WITH EXISTING METHODS: The WF-NTP method has substantially greater capacity compared to current automated platforms that typically either focus on characterising single worms at high resolution or tracking the properties of populations of less than 50 animals. CONCLUSIONS: The WF-NTP extends significantly the power of existing automated platforms by combining enhanced optical imaging techniques with an advanced software platform. We anticipate that this approach will further extend the scope and utility of C. elegans as a model organism.

Prediction of activation patterns preceding hallucinations in patients with schizophrenia using machine learning with structured sparsity.

Despite significant progress in the field, the detection of fMRI signal changes during hallucinatory events remains difficult and time-consuming. This article first proposes a machine-learning algorithm to automatically identify resting-state fMRI periods that precede hallucinations versus periods that do not. When applied to whole-brain fMRI data, state-of-the-art classification methods, such as support vector machines (SVM), yield dense solutions that are difficult to interpret. We proposed to extend the existing sparse classification methods by taking the spatial structure of brain images into account with structured sparsity using the total variation penalty. Based on this approach, we obtained reliable classifying performances associated with interpretable predictive patterns, composed of two clearly identifiable clusters in speech-related brain regions. The variation in transition-to-hallucination functional patterns not only from one patient to another but also from one occurrence to the next (e.g., also depending on the sensory modalities involved) appeared to be the major difficulty when developing effective classifiers. Consequently, second, this article aimed to characterize the variability within the prehallucination patterns using an extension of principal component analysis with spatial constraints. The principal components (PCs) and the associated basis patterns shed light on the intrinsic structures of the variability present in the dataset. Such results are promising in the scope of innovative fMRI-guided therapy for drug-resistant hallucinations, such as fMRI-based neurofeedback.

Exploiting Knowledge Composition to Improve Real-Life Hand Prosthetic Control.

In myoelectric prosthesis control, one of the hottest topics nowadays is enforcing simultaneous and proportional (s/p) control over several degrees of freedom. This problem is particularly hard and the scientific community has so far failed to provide a stable and reliable s/p control, effective in daily-life activities. In order to improve the reliability of this form of control, in this paper we propose on-the-fly knowledge composition, thereby reducing the burden of matching several patterns at the same time, and simplifying the task of the system. In particular, we show that using our method it is possible to dynamically compose a model by juxtaposing subsets of previously gathered (sample, target) pairs in real-time, rather than composing a single model in the beginning and then hoping it can reliably distinguish all patterns. Fourteen intact subjects participated in an experiment, where repetitive daily-life tasks (e.g. ironing a cloth) were performed using a commercially available dexterous prosthetic hand mounted on a splint and wirelessly controlled using a machine learning method. During the experiment, the subjects performed these tasks using myocontrol with and without knowledge composition and the results demonstrate that employing knowledge composition allowed better performance, i.e. reducing the overall task completion time by 30%.

Goal-recognition-based adaptive brain-computer interface for navigating immersive robotic systems.

OBJECTIVE: This work proposes principled strategies for self-adaptations in EEG-based Brain-computer interfaces (BCIs) as a way out of the bandwidth bottleneck resulting from the considerable mismatch between the low-bandwidth interface and the bandwidth-hungry application, and a way to enable fluent and intuitive interaction in embodiment systems. The main focus is laid upon inferring the hidden target goals of users while navigating in a remote environment as a basis for possible adaptations. APPROACH: To reason about possible user goals, a general user-agnostic Bayesian update rule is devised to be recursively applied upon the arrival of evidences, i.e. user input and user gaze. Experiments were conducted with healthy subjects within robotic embodiment settings to evaluate the proposed method. These experiments varied along three factors: the type of the robot/environment (simulated and physical), the type of the interface (keyboard or BCI), and the way goal recognition (GR) is used to guide a simple shared control (SC) driving scheme. MAIN RESULTS: Our results show that the proposed GR algorithm is able to track and infer the hidden user goals with relatively high precision and recall. Further, the realized SC driving scheme benefits from the output of the GR system and is able to reduce the user effort needed to accomplish the assigned tasks. Despite the fact that the BCI requires higher effort compared to the keyboard conditions, most subjects were able to complete the assigned tasks, and the proposed GR system is additionally shown able to handle the uncertainty in user input during SSVEP-based interaction. The SC application of the belief vector indicates that the benefits of the GR module are more pronounced for BCIs, compared to the keyboard interface. SIGNIFICANCE: Being based on intuitive heuristics that model the behavior of the general population during the execution of navigation tasks, the proposed GR method can be used without prior tuning for the individual users. The proposed methods can be easily integrated in devising more advanced SC schemes and/or strategies for automatic BCI self-adaptations.

Automated selective disruption of slow wave sleep.

BACKGROUND: Slow wave sleep (SWS) plays an important role in neurophysiologic restoration. Experimentally testing the effect of SWS disruption previously required highly time-intensive and subjective methods. Our goal was to develop an automated and objective protocol to reduce SWS without affecting sleep architecture. NEW METHOD: We developed a custom Matlab™ protocol to calculate electroencephalogram spectral power every 10s live during a polysomnogram, exclude artifact, and, if measurements met criteria for SWS, deliver increasingly louder tones through earphones. Middle-aged healthy volunteers (n=10) each underwent 2 polysomnograms, one with the SWS disruption protocol and one with sham condition. RESULTS: The SWS disruption protocol reduced SWS compared to sham condition, as measured by spectral power in the delta (0.5-4Hz) band, particularly in the 0.5-2Hz range (mean 20% decrease). A compensatory increase in the proportion of total spectral power in the theta (4-8Hz) and alpha (8-12Hz) bands was seen, but otherwise normal sleep features were preserved. N3 sleep decreased from 20±34 to 3±6min, otherwise there were no significant changes in total sleep time, sleep efficiency, or other macrostructural sleep characteristics. COMPARISON WITH EXISTING METHOD: This novel SWS disruption protocol produces specific reductions in delta band power similar to existing methods, but has the advantage of being automated, such that SWS disruption can be performed easily in a highly standardized and operator-independent manner. CONCLUSION: This automated SWS disruption protocol effectively reduces SWS without impacting overall sleep architecture.

A spatial-frequency-temporal optimized feature sparse representation-based classification method for motor imagery EEG pattern recognition.

Effective feature extraction and classification methods are of great importance for motor imagery (MI)-based brain-computer interface (BCI) systems. The common spatial pattern (CSP) algorithm is a widely used feature extraction method for MI-based BCIs. In this work, we propose a novel spatial-frequency-temporal optimized feature sparse representation-based classification method. Optimal channels are selected based on relative entropy criteria. Significant CSP features on frequency-temporal domains are selected automatically to generate a column vector for sparse representation-based classification (SRC). We analyzed the performance of the new method on two public EEG datasets, namely BCI competition III dataset IVa which has five subjects and BCI competition IV dataset IIb which has nine subjects. Compared to the performance offered by the existing SRC method, the proposed method achieves average classification accuracy improvements of 21.568 and 14.38% for BCI competition III dataset IVa and BCI competition IV dataset IIb, respectively. Furthermore, our approach also shows better classification performance when compared to other competing methods for both datasets.

EEG-Based Strategies to Detect Motor Imagery for Control and Rehabilitation.

Advances in brain-computer interface (BCI) technology have facilitated the detection of Motor Imagery (MI) from electroencephalography (EEG). First, we present three strategies of using BCI to detect MI from EEG: operant conditioning that employed a fixed model, machine learning that employed a subject-specific model computed from calibration, and adaptive strategy that continuously compute the subject-specific model. Second, we review prevailing works that employed the operant conditioning and machine learning strategies. Third, we present our past work on six stroke patients who underwent a BCI rehabilitation clinical trial with averaged accuracies of 79.8% during calibration and 69.5% across 18 online feedback sessions. Finally, we perform an offline study in this paper on our work employing the adaptive strategy. The results yielded significant improvements of 12% (p < 0.001) and 9% (p < 0.001) using all the data and using limited preceding data respectively in the feedback accuracies. The results showed an increase in the amount of training data yielded improvements. Nevertheless, results of using limited preceding data showed a larger part of the improvement was due to the adaptive strategy and changing subject-specific models did not deteriorate the accuracies. Hence the adaptive strategy is effective in addressing the non-stationarity between calibration and feedback sessions.

Ankle-Angle Estimation from Blind Source Separated Afferent Activity in the Sciatic Nerve for Closed-Loop Functional Neuromuscular Stimulation System.

Cuff electrode recording has been proposed as a solution to obtain robust feedback signals for closed-loop controlled functional neuromuscular stimulation (FNS) systems. However, single-channel cuff electrode recording requires several electrodes to obtain the feedback signal related to each muscle. In this study, we propose an ankle-angle estimation method in which recording is conducted from the proximal nerve trunk with a multichannel cuff electrode to minimize cuff electrode usage. In experiments, muscle afferent signals were recorded from a rabbit's proximal sciatic nerve trunk using a multichannel cuff electrode, and blind source separation and ankle-angle estimation were performed using fast independent component analysis (PP/FastICA) combined with dynamically driven recurrent neural network (DDRNN). The experimental results indicate that the proposed method has high ankle-angle estimation accuracy for both situations when the ankle motion is generated by position servo system or neuromuscular stimulation. Furthermore, the results confirm that the proposed method is applicable to closed-loop FNS systems to control limb motion.

Quantifying and Reducing Motion Artifacts in Wearable Seismocardiogram Measurements During Walking to Assess Left Ventricular Health.

GOAL: Our objective is to provide a framework for extracting signals of interest from the wearable seismocardiogram (SCG) measured during walking at normal (subject's preferred pace) and moderately fast (1.34-1.45 m/s) speeds. METHODS: We demonstrate, using empirical mode decomposition (EMD) and feature tracking algorithms, that the pre-ejection period (PEP) can be accurately estimated from a wearable patch that simultaneously measures electrocardiogram and sternal acceleration signals. We also provide a method to determine the minimum number of heartbeats required for an accurate estimate to be obtained for the PEP from the accelerometer signals during walking. RESULTS: The EMD-based denoising approach provides a statistically significant increase in the signal-to-noise ratio of wearable SCG signals and also improves estimation of PEP during walking. CONCLUSION: The algorithms described in this paper can be used to provide hemodynamic assessment from wearable SCG during walking. SIGNIFICANCE: A major limitation in the use of the SCG, a measure of local chest vibrations caused by cardiac ejection of blood in the vasculature, is that a user must remain completely still for high-quality measurements. The motion can create artifacts and practically render the signal unreadable. Addressing this limitation could allow, for the first time, SCG measurements to be obtained reliably during movement-aside from increasing the coverage throughout the day of cardiovascular monitoring, analyzing SCG signals during movement would quantify the cardiovascular system's response to stress (exercise), and thus provide a more holistic assessment of overall health.

Bipolar Intracardiac Electrogram Active Interval Extraction During Atrial Fibrillation.

OBJECTIVE: We introduce novel methods to identify the active intervals (AIs) of intracardiac electrograms (IEGMs) during complex arrhythmias, such as atrial fibrillation (AF). METHODS: We formulate the AI extraction problem, which consists of estimating the beginning and duration of the AIs, as a sequence of hypothesis tests. In each test, we compare the variance of a small portion of the bipolar IEGM with its adjacent segments. We propose modified general-likelihood ratio (MGLR) and separating-function-estimation tests; we derive five test statistics (TSs), and show that the AIs can be obtained by threshold crossing the TSs. We apply the proposed methods to the IEGM segments collected from the left atrium of 16 patients (62.4 ± 8.2-years old, four females, four paroxysmal, and twelve persistent AF) prior to catheter ablation. The accuracy of our methods is evaluated by comparing them with previously developed methods and manual annotation (MA). RESULTS: Our results show a high level of similarity between the AIs of the proposed methods and MA, e.g., the true and false positive rates of one of the MGLR-based methods were, respectively, 97.8% and 1.4%. The mean absolute error from estimation of the onset and end of AIs and also for the estimation of the mean cycle length for that approach was 8.7 ± 10.5, 13 ± 15.5, and 4.2 ± 9.4 ms, respectively. CONCLUSION: The proposed methods can accurately identify onset and duration of AI of the IEGM during AF. SIGNIFICANCE: The proposed methods can be used for real-time automated analysis of AF, the most challenging complex arrhythmia.

Data-Driven User Feedback: An Improved Neurofeedback Strategy considering the Interindividual Variability of EEG Features.

It has frequently been reported that some users of conventional neurofeedback systems can experience only a small portion of the total feedback range due to the large interindividual variability of EEG features. In this study, we proposed a data-driven neurofeedback strategy considering the individual variability of electroencephalography (EEG) features to permit users of the neurofeedback system to experience a wider range of auditory or visual feedback without a customization process. The main idea of the proposed strategy is to adjust the ranges of each feedback level using the density in the offline EEG database acquired from a group of individuals. Twenty-two healthy subjects participated in offline experiments to construct an EEG database, and five subjects participated in online experiments to validate the performance of the proposed data-driven user feedback strategy. Using the optimized bin sizes, the number of feedback levels that each individual experienced was significantly increased to 139% and 144% of the original results with uniform bin sizes in the offline and online experiments, respectively. Our results demonstrated that the use of our data-driven neurofeedback strategy could effectively increase the overall range of feedback levels that each individual experienced during neurofeedback training.

Identifying the Types of Ion Channel-Targeted Conotoxins by Incorporating New Properties of Residues into Pseudo Amino Acid Composition.

Conotoxins are a kind of neurotoxin which can specifically interact with potassium, sodium type, and calcium channels. They have become potential drug candidates to treat diseases such as chronic pain, epilepsy, and cardiovascular diseases. Thus, correctly identifying the types of ion channel-targeted conotoxins will provide important clue to understand their function and find potential drugs. Based on this consideration, we developed a new computational method to rapidly and accurately predict the types of ion-targeted conotoxins. Three kinds of new properties of residues were proposed to use in pseudo amino acid composition to formulate conotoxins samples. The support vector machine was utilized as classifier. A feature selection technique based on F-score was used to optimize features. Jackknife cross-validated results showed that the overall accuracy of 94.6% was achieved, which is higher than other published results, demonstrating that the proposed method is superior to published methods. Hence the current method may play a complementary role to other existing methods for recognizing the types of ion-target conotoxins.