Identifying activity levels and steps of people with stroke using a novel shoe-based sensor.

BACKGROUND/PURPOSE: Advances in sensor technologies provide a method to accurately assess activity levels of people with stroke in their community. This information could be used to determine the effectiveness of rehabilitation interventions as well as provide behavior-enhancing feedback. The purpose of this study was to assess the accuracy of a novel shoe-based sensor system (SmartShoe) to identify different functional postures and steps in people with stroke. The SmartShoe system consists of five force-sensitive resistors built into a flexible insole and an accelerometer on the back of the shoe. Pressure and acceleration data are sent via Bluetooth to a smart phone. METHODS: Participants with stroke wore the SmartShoe while they performed activities of daily living (ADLs) in sitting, standing, and walking positions. Data from four participants were used to develop a multilayer perceptron artificial neural network (ANN) to identify sitting, standing, and walking. A signal-processing algorithm used data from the pressure sensors to estimate the number of steps taken while walking. The accuracy, precision, and recall of the ANN for identifying the three functional postures were calculated with data from a different set of participants. Agreement between steps identified by SmartShoe and actual steps taken was analyzed by the Bland Altman method. RESULTS: The SmartShoe was able to accurately identify sitting, standing, and walking. Accuracy, precision, and recall were all greater than 95%. The mean difference between steps identified by SmartShoe and actual steps was less than one step. DISCUSSION: The SmartShoe was able to accurately identify different functional postures, using a unique combination of pressure and acceleration data, of people with stroke as they performed different ADLs. There was a strong level of agreement between actual steps taken and steps identified by the SmartShoe. Further study is needed to determine whether the SmartShoe could be used to provide valid information on activity levels of people with stroke while they go about their daily lives in their home and community.

Non-invasive monitoring of chewing and swallowing for objective quantification of ingestive behavior.

A methodology of studying of ingestive behavior by non-invasive monitoring of swallowing (deglutition) and chewing (mastication) has been developed. The target application for the developed methodology is to study the behavioral patterns of food consumption and producing volumetric and weight estimates of energy intake. Monitoring is non-invasive based on detecting swallowing by a sound sensor located over laryngopharynx or by a bone-conduction microphone and detecting chewing through a below-the-ear strain sensor. Proposed sensors may be implemented in a wearable monitoring device, thus enabling monitoring of ingestive behavior in free-living individuals. In this paper, the goals in the development of this methodology are two-fold. First, a system comprising sensors, related hardware and software for multi-modal data capture is designed for data collection in a controlled environment. Second, a protocol is developed for manual scoring of chewing and swallowing for use as a gold standard. The multi-modal data capture was tested by measuring chewing and swallowing in 21 volunteers during periods of food intake and quiet sitting (no food intake). Video footage and sensor signals were manually scored by trained raters. Inter-rater reliability study for three raters conducted on the sample set of five subjects resulted in high average intra-class correlation coefficients of 0.996 for bites, 0.988 for chews and 0.98 for swallows. The collected sensor signals and the resulting manual scores will be used in future research as a gold standard for further assessment of sensor design, development of automatic pattern recognition routines and study of the relationship between swallowing/chewing and ingestive behavior.

A wearable sensor system for monitoring cigarette smoking.

OBJECTIVE: Available methods of smoking assessment (e.g., self-report, portable puff-topography instruments) do not permit the collection of accurate measures of smoking behavior while minimizing reactivity to the assessment procedure. This article suggests a new method for monitoring cigarette smoking based on a wearable sensor system (Personal Automatic Cigarette Tracker [PACT]) that is completely transparent to the end user and does not require any conscious effort to achieve reliable monitoring of smoking in free-living individuals. METHOD: The proposed sensor system consists of a respiratory inductance plethysmograph for monitoring of breathing and a hand gesture sensor for detecting a cigarette at the mouth. The wearable sensor system was tested in a laboratory study of 20 individuals who performed 12 different activities including cigarette smoking. Signal processing was applied to evaluate the uniqueness of breathing patterns and their correlation with hand gestures. RESULTS: The results indicate that smoking manifests unique breathing patterns that are highly correlated with hand-to-mouth cigarette gestures and suggest that these signals can potentially be used to identify and characterize individual smoke inhalations. CONCLUSIONS: With the future development of signal processing and pattern-recognition methods, PACT can be used to automatically assess the frequency of smoking and inhalation patterns (such as depth of inhalation and smoke holding) throughout the day and provide an objective method of assessing the effectiveness of behavioral and pharmacological smoking interventions.

Monitoring of cigarette smoking using wearable sensors and support vector machines.

Cigarette smoking is a serious risk factor for cancer, cardiovascular, and pulmonary diseases. Current methods of monitoring of cigarette smoking habits rely on various forms of self-report that are prone to errors and under reporting. This paper presents a first step in the development of a methodology for accurate and objective assessment of smoking using noninvasive wearable sensors (Personal Automatic Cigarette Tracker-PACT) by demonstrating feasibility of automatic recognition of smoke inhalations from signals arising from continuous monitoring of breathing and hand-to-mouth gestures by support vector machine classifiers. The performance of subject-dependent (individually calibrated) models was compared to performance of subject-independent (group) classification models. The models were trained and validated on a dataset collected from 20 subjects performing 12 different activities representative of everyday living (total duration 19.5 h or 21,411 breath cycles). Precision and recall were used as the accuracy metrics. Group models obtained 87% and 80% of average precision and recall, respectively. Individual models resulted in 90% of average precision and recall, indicating a significant presence of individual traits in signal patterns. These results suggest the feasibility of monitoring cigarette smoking by means of a wearable and noninvasive sensor system in free living conditions.

Detection of cigarette smoke inhalations from respiratory signals using reduced feature set.

A combination of wearable Respiratory Inductive Plethysmograph and a hand-to-mouth Proximity Sensor (PS) can be used to monitor smoking habits and smoke exposure in cigarette smokers. In our previous work, detection of smoke inhalations was achieved by using a Support Vector Machine (SVM) classifier applied to raw sensor signals with 1503-element feature vectors. This study uses empirically-defined 27 features computed from the sensor signals to reduce the length of vectors. Further reduction in the length of the feature vectors was achieved by a forward feature selection algorithm, identifying from 2 to 16 features most critical for smoke inhalations detection. For individual detection models, the 1503-element feature vectors, 27-element feature vectors and reduced feature vectors resulted in F-scores of 90.1%, 68.7% and 94% respectively. For the group models, F-scores were 81.3%, 65% and 67% respectively. These results demonstrate feasibility of detecting smoke inhalations with a computed feature set, but suggest high individuality of breathing patterns associated with smoking.

Detection of Hand-to-Mouth Gestures Using a RF Operated Proximity Sensor for Monitoring Cigarette Smoking.

Common methods for monitoring of cigarette smoking, such as portable puff-topography instruments or self-report questionnaires, tend to be biased due to conscious or unconscious underreporting. Additionally, these methods may change the natural smoking behavior of individuals. Our long term objective is the development of a wearable non-invasive monitoring system (Personal Automatic Cigarette Tracker - PACT) to reliably monitor cigarette smoking behavior under free living conditions. PACT monitors smoking by observing characteristic breathing patterns of smoke inhalations that follow a cigarette-to-mouth hand gesture. As envisioned, PACT does not rely on self-report or require any conscious effort from the user. A major element of the PACT is a proximity sensor that detects typical cigarette-to-mouth gesture during cigarette smoking. This study describes the design and validation of a prototype RF proximity sensor that captures hand-to-mouth gestures with a high sensitivity (0.90), and a methodology that can reject up to 68% of artifacts gestures originating from activities other than cigarette smoking.

Identification of cigarette smoke inhalations from wearable sensor data using a Support Vector Machine classifier.

This study presents a subject-independent model for detection of smoke inhalations from wearable sensors capturing characteristic hand-to-mouth gestures and changes in breathing patterns during cigarette smoking. Wearable sensors were used to detect the proximity of the hand to the mouth and to acquire the respiratory patterns. The waveforms of sensor signals were used as features to build a Support Vector Machine classification model. Across a data set of 20 enrolled participants, precision of correct identification of smoke inhalations was found to be >87%, and a resulting recall >80%. These results suggest that it is possible to analyze smoking behavior by means of a wearable and non-invasive sensor system.

Automatic identification of the number of food items in a meal using clustering techniques based on the monitoring of swallowing and chewing.

The number of distinct foods consumed in a meal is of significant clinical concern in the study of obesity and other eating disorders. This paper proposes the use of information contained in chewing and swallowing sequences for meal segmentation by food types. Data collected from experiments of 17 volunteers were analyzed using two different clustering techniques. First, an unsupervised clustering technique, Affinity Propagation (AP), was used to automatically identify the number of segments within a meal. Second, performance of the unsupervised AP method was compared to a supervised learning approach based on Agglomerative Hierarchical Clustering (AHC). While the AP method was able to obtain 90% accuracy in predicting the number of food items, the AHC achieved an accuracy >95%. Experimental results suggest that the proposed models of automatic meal segmentation may be utilized as part of an integral application for objective Monitoring of Ingestive Behavior in free living conditions.

Automatic food intake detection based on swallowing sounds.

This paper presents a novel fully automatic food intake detection methodology, an important step toward objective monitoring of ingestive behavior. The aim of such monitoring is to improve our understanding of eating behaviors associated with obesity and eating disorders. The proposed methodology consists of two stages. First, acoustic detection of swallowing instances based on mel-scale Fourier spectrum features and classification using support vector machines is performed. Principal component analysis and a smoothing algorithm are used to improve swallowing detection accuracy. Second, the frequency of swallowing is used as a predictor for detection of food intake episodes. The proposed methodology was tested on data collected from 12 subjects with various degrees of adiposity. Average accuracies of >80% and >75% were obtained for intra-subject and inter-subject models correspondingly with a temporal resolution of 30s. Results obtained on 44.1 hours of data with a total of 7305 swallows show that detection accuracies are comparable for obese and lean subjects. They also suggest feasibility of food intake detection based on swallowing sounds and potential of the proposed methodology for automatic monitoring of ingestive behavior. Based on a wearable non-invasive acoustic sensor the proposed methodology may potentially be used in free-living conditions.

Automatic detection of temporal gait parameters in poststroke individuals.

Approximately one-third of people who recover from a stroke require some form of assistance to walk. Repetitive task-oriented rehabilitation interventions have been shown to improve motor control and function in people with stroke. Our long-term goal is to design and test an intensive task-oriented intervention that will utilize the two primary components of constrained-induced movement therapy: massed, task-oriented training and behavioral methods to increase use of the affected limb in the real world. The technological component of the intervention is based on a wearable footwear-based sensor system that monitors relative activity levels, functional utilization, and gait parameters of affected and unaffected lower extremities. The purpose of this study is to describe a methodology to automatically identify temporal gait parameters of poststroke individuals to be used in assessment of functional utilization of the affected lower extremity as a part of behavior enhancing feedback. An algorithm accounting for intersubject variability is capable of achieving estimation error in the range of 2.6-18.6% producing comparable results for healthy and poststroke subjects. The proposed methodology is based on inexpensive and user-friendly technology that will enable research and clinical applications for rehabilitation of people who have experienced a stroke.

Design of a instrumentation module for monitoring ingestive behavior in laboratory studies.

The development of accurate and objective tools for monitoring of ingestive behavior (MIB) is one of the most important needs facing studies of obesity and eating disorders. This paper presents the design of an instrumentation module for non-invasive monitoring of food ingestion in laboratory studies. The system can capture signals from a variety of sensors that characterize ingestion process (such as acoustical and other swallowing sensors, strain sensor for chewing detection and self-report buttons). In addition to the sensors, the data collection system integrates time-synchronous video footage that can be used for annotation of subject's activity. Both data and video are simultaneously and synchronously acquired and stored by a LabVIEW-based interface specifically developed for this application. This instrumentation module improves a previously developed system by eliminating the post-processing stage of data synchronization and by reducing the risks of operator's error.

Characterizing walking activity in people with stroke.

Stroke is the leading cause of disability in the U.S. Many people with stroke have limited walking ability and are inactive. In this paper we describe a novel shoe based sensor, SmartShoe, and a signal processing technique to identify walking activity. The technique was validated with 6 people with walking impairment due to stroke. The results suggest that the SmartShoe is able to accurately identify walking activity. This device could be used to monitor walking activity as well as provide behavioral enhancing feedback to increase activity levels and walking ability in people with stroke for extended periods of time in the real world.

Detection of periods of food intake using Support Vector Machines.

Studies of obesity and eating disorders need objective tools of Monitoring of Ingestive Behavior (MIB) that can detect and characterize food intake. In this paper we describe detection of food intake by a Support Vector Machine classifier trained on time history of chews and swallows. The training was performed on data collected from 18 subjects in 72 experiments involving eating and other activities (for example, talking). The highest accuracy of detecting food intake (94%) was achieved in configuration where both chews and swallows were used as predictors. Using only swallowing as a predictor resulted in 80% accuracy. Experimental results suggest that these two predictors may be used for differentiation between periods of resting and food intake with a resolution of 30 seconds. Proposed methods may be utilized for development of an accurate, inexpensive, and non-intrusive methodology to objectively monitor food intake in free living conditions.

Automatic detection of swallowing events by acoustical means for applications of monitoring of ingestive behavior.

Our understanding of etiology of obesity and overweight is incomplete due to lack of objective and accurate methods for monitoring of ingestive behavior (MIB) in the free-living population. Our research has shown that frequency of swallowing may serve as a predictor for detecting food intake, differentiating liquids and solids, and estimating ingested mass. This paper proposes and compares two methods of acoustical swallowing detection from sounds contaminated by motion artifacts, speech, and external noise. Methods based on mel-scale Fourier spectrum, wavelet packets, and support vector machines are studied considering the effects of epoch size, level of decomposition, and lagging on classification accuracy. The methodology was tested on a large dataset (64.5 h with a total of 9966 swallows) collected from 20 human subjects with various degrees of adiposity. Average weighted epoch-recognition accuracy for intravisit individual models was 96.8%, which resulted in 84.7% average weighted accuracy in detection of swallowing events. These results suggest high efficiency of the proposed methodology in separation of swallowing sounds from artifacts that originate from respiration, intrinsic speech, head movements, food ingestion, and ambient noise. The recognition accuracy was not related to body mass index, suggesting that the methodology is suitable for obese individuals.

Detection of food intake from swallowing sequences by supervised and unsupervised methods.

Studies of food intake and ingestive behavior in free-living conditions most often rely on self-reporting-based methods that can be highly inaccurate. Methods of Monitoring of Ingestive Behavior (MIB) rely on objective measures derived from chewing and swallowing sequences and thus can be used for unbiased study of food intake with free-living conditions. Our previous study demonstrated accurate detection of food intake in simple models relying on observation of both chewing and swallowing. This article investigates methods that achieve comparable accuracy of food intake detection using only the time series of swallows and thus eliminating the need for the chewing sensor. The classification is performed for each individual swallow rather than for previously used time slices and thus will lead to higher accuracy in mass prediction models relying on counts of swallows. Performance of a group model based on a supervised method (SVM) is compared to performance of individual models based on an unsupervised method (K-means) with results indicating better performance of the unsupervised, self-adapting method. Overall, the results demonstrate that highly accurate detection of intake of foods with substantially different physical properties is possible by an unsupervised system that relies on the information provided by the swallowing alone.

Toward objective monitoring of ingestive behavior in free-living population.

Understanding of eating behaviors associated with obesity requires objective and accurate monitoring of food intake patterns. Accurate methods are available for measuring total energy expenditure and its components in free-living populations, but methods for measuring food intake in free-living people are far less accurate and involve self-reporting or subjective monitoring. We suggest that chews and swallows can be used for objective monitoring of ingestive behavior. This hypothesis was verified in a human study involving 20 subjects. Chews and swallows were captured during periods of quiet resting, talking, and meals of varying size. The counts of chews and swallows along with other derived metrics were used to build prediction models for detection of food intake, differentiation between liquids and solids, and for estimation of the mass of ingested food. The proposed prediction models were able to detect periods of food intake with >95% accuracy and a fine time resolution of 30 s, differentiate solid foods from liquids with >91% accuracy, and predict mass of ingested food with >91% accuracy for solids and >83% accuracy for liquids. In earlier publications, we have shown that chews and swallows can be captured by noninvasive sensors that could be developed into a wearable device. Thus, the proposed methodology could lead to the development of an innovative new way of assessing human eating behavior in free-living conditions.

Limited receptive area neural classifier for recognition of swallowing sounds using continuous wavelet transform.

In this paper we propose a sound recognition technique based on the limited receptive area (LIRA) neural classifier and continuous wavelet transform (CWT). LIRA neural classifier was developed as a multipurpose image recognition system. Previous tests of LIRA demonstrated good results in different image recognition tasks including: handwritten digit recognition, face recognition, metal surface texture recognition, and micro work piece shape recognition. We propose a sound recognition technique where scalograms of sound instances serve as inputs of the LIRA neural classifier. The methodology was tested in recognition of swallowing sounds. Swallowing sound recognition may be employed in systems for automated swallowing assessment and diagnosis of swallowing disorders. The experimental results suggest high efficiency and reliability of the proposed approach.