Surface-Free Multi-Stroke Trajectory Reconstruction and Word Recognition Using an IMU-Enhanced Digital Pen.

Efficient handwriting trajectory reconstruction (TR) requires specific writing surfaces for detecting movements of digital pens. Although several motion-based solutions have been developed to remove the necessity of writing surfaces, most of them are based on classical sensor fusion methods limited, by sensor error accumulation over time, to tracing only single strokes. In this work, we present an approach to map the movements of an IMU-enhanced digital pen to relative displacement data. Training data is collected by means of a tablet. We propose several pre-processing and data-preparation methods to synchronize data between the pen and the tablet, which are of different sampling rates, and train a convolutional neural network (CNN) to reconstruct multiple strokes without the need of writing segmentation or post-processing correction of the predicted trajectory. The proposed system learns the relative displacement of the pen tip over time from the recorded raw sensor data, achieving a normalized error rate of 0.176 relative to unit-scaled tablet ground truth (GT) trajectory. To test the effectiveness of the approach, we train a neural network for character recognition from the reconstructed trajectories, which achieved a character error rate of 19.51%. Finally, a joint model is implemented that makes use of both the IMU data and the generated trajectories, which outperforms the sensor-only-based recognition approach by 0.75%.

Thigh-Derived Inertial Sensor Metrics to Assess the Sit-to-Stand and Stand-to-Sit Transitions in the Timed Up and Go (TUG) Task for Quantifying Mobility Impairment in Multiple Sclerosis.

Introduction: Inertial sensors generate objective and sensitive metrics of movement disability that may indicate fall risk in many clinical conditions including multiple sclerosis (MS). The Timed-Up-And-Go (TUG) task is used to assess patient mobility because it incorporates clinically-relevant submovements during standing. Most sensor-based TUG research has focused on the placement of sensors at the spine, hip or ankles; an examination of thigh activity in TUG in multiple sclerosis is wanting. Methods: We used validated sensors (x-IMU by x-io) to derive transparent metrics for the sit-to-stand (SI-ST) transition and the stand-to-sit (ST-SI) transition of TUG, and compared effect sizes for metrics from inertial sensors on the thighs to effect sizes for metrics from a sensor placed at the L3 level of the lumbar spine. Twenty-three healthy volunteers were compared to 17 ambulatory persons with MS (PwMS, HAI ≤ 2). Results: During the SI-ST transition, the metric with the largest effect size comparing healthy volunteers to PwMS was the Area Under the Curve of the thigh angular velocity in the pitch direction-representing both thigh and knee extension; the peak of the spine pitch angular velocity during SI-ST also had a large effect size, as did some temporal measures of duration of SI-ST, although less so. During the ST-SI transition the metric with the largest effect size in PwMS was the peak of the spine angular velocity curve in the roll direction. A regression was performed. Discussion: We propose for PwMS that the diminished peak angular velocity during SI-ST directly represents extensor weakness, while the increased roll during ST-SI represents diminished postural control. Conclusions: During the SI-ST transition of TUG, angular velocities can discriminate between healthy volunteers and ambulatory PwMS better than temporal features. Sensor placement on the thighs provides additional discrimination compared to sensor placement at the lumbar spine.

Sensor-based gait analysis of individualized improvement during apomorphine titration in Parkinson's disease.

Mobile, sensor-based gait analysis in Parkinson's disease (PD) facilitates the objective measurement of gait parameters in cross-sectional studies. Besides becoming outcome measures for clinical studies, the application of gait parameters in personalized clinical decision support is limited. Therefore, the aim of this study was to evaluate whether the individual response of PD patients to dopaminergic treatment may be measured by sensor-based gait analysis. 13 PD patients received apomorphine every 15 min to incrementally increase the bioavailable apomorphine dose. Motor performance (UPDRS III) was assessed 10 min after each apomorphine injection. Gait parameters were obtained after each UPDRS III rating from a 2 × 10 m gait sequence, providing 41.2 ± 9.2 strides per patient and injection. Gait parameters and UPDRS III ratings were compared cross-sectionally after apomorphine titration, and more importantly between consecutive injections for each patient individually. For the individual response, the effect size Cohen's d for gait parameter changes was calculated based on the stride variations of each gait sequence after each injection. Cross-sectionally, apomorphine improved stride speed, length, gait velocity, maximum toe clearance, and toe off angle. Between injections, the effect size for individual changes in stride speed, length, and maximum toe clearance correlated to the motor improvement in each patient. In addition, significant changes of stride length between injections were significantly associated with UPDRS III improvements. We therefore show, that sensor-based gait analysis provides objective gait parameters that support clinical assessment of individual PD patients during dopaminergic treatment. We propose clinically relevant instrumented gait parameters for treatment studies and especially clinical care.

Mobile Stride Length Estimation With Deep Convolutional Neural Networks.

OBJECTIVE: Accurate estimation of spatial gait characteristics is critical to assess motor impairments resulting from neurological or musculoskeletal disease. Currently, however, methodological constraints limit clinical applicability of state-of-the-art double integration approaches to gait patterns with a clear zero-velocity phase. METHODS: We describe a novel approach to stride length estimation that uses deep convolutional neural networks to map stride-specific inertial sensor data to the resulting stride length. The model is trained on a publicly available and clinically relevant benchmark dataset consisting of 1220 strides from 101 geriatric patients. Evaluation is done in a tenfold cross validation and for three different stride definitions. RESULTS: Even though best results are achieved with strides defined from midstance to midstance with average accuracy and precision of , performance does not strongly depend on stride definition. The achieved precision outperforms state-of-the-art methods evaluated on the same benchmark dataset by . CONCLUSION: Due to the independence of stride definition, the proposed method is not subject to the methodological constrains that limit applicability of state-of-the-art double integration methods. Furthermore, it was possible to improve precision on the benchmark dataset. SIGNIFICANCE: With more precise mobile stride length estimation, new insights to the progression of neurological disease or early indications might be gained. Due to the independence of stride definition, previously uncharted diseases in terms of mobile gait analysis can now be investigated by retraining and applying the proposed method.

Wearable sensors objectively measure gait parameters in Parkinson's disease.

Distinct gait characteristics like short steps and shuffling gait are prototypical signs commonly observed in Parkinson's disease. Routinely assessed by observation through clinicians, gait is rated as part of categorical clinical scores. There is an increasing need to provide quantitative measurements of gait, e.g. to provide detailed information about disease progression. Recently, we developed a wearable sensor-based gait analysis system as diagnostic tool that objectively assesses gait parameter in Parkinson's disease without the need of having a specialized gait laboratory. This system consists of inertial sensor units attached laterally to both shoes. The computed target of measures are spatiotemporal gait parameters including stride length and time, stance phase time, heel-strike and toe-off angle, toe clearance, and inter-stride variation from gait sequences. To translate this prototype into medical care, we conducted a cross-sectional study including 190 Parkinson's disease patients and 101 age-matched controls and measured gait characteristics during a 4x10 meter walk at the subjects' preferred speed. To determine intraindividual changes in gait, we monitored the gait characteristics of 63 patients longitudinally. Cross-sectional analysis revealed distinct spatiotemporal gait parameter differences reflecting typical Parkinson's disease gait characteristics including short steps, shuffling gait, and postural instability specific for different disease stages and levels of motor impairment. The longitudinal analysis revealed that gait parameters were sensitive to changes by mirroring the progressive nature of Parkinson's disease and corresponded to physician ratings. Taken together, we successfully show that wearable sensor-based gait analysis reaches clinical applicability providing a high biomechanical resolution for gait impairment in Parkinson's disease. These data demonstrate the feasibility and applicability of objective wearable sensor-based gait measurement in Parkinson's disease reaching high technological readiness levels for both, large scale clinical studies and individual patient care.

Instrumented gait analysis: a measure of gait improvement by a wheeled walker in hospitalized geriatric patients.

BACKGROUND: In an increasing aging society, reduced mobility is one of the most important factors limiting activities of daily living and overall quality of life. The ability to walk independently contributes to the mobility, but is increasingly restricted by numerous diseases that impair gait and balance. The aim of this cross-sectional observation study was to examine whether spatio-temporal gait parameters derived from mobile instrumented gait analysis can be used to measure the gait stabilizing effects of a wheeled walker (WW) and whether these gait parameters may serve as surrogate marker in hospitalized patients with multifactorial gait and balance impairment. METHODS: One hundred six patients (ages 68-95) wearing inertial sensor equipped shoes passed an instrumented walkway with and without gait support from a WW. The walkway assessed the risk of falling associated gait parameters velocity, swing time, stride length, stride time- and double support time variability. Inertial sensor-equipped shoes measured heel strike and toe off angles, and foot clearance. RESULTS: The use of a WW improved the risk of spatio-temporal parameters velocity, swing time, stride length and the sagittal plane associated parameters heel strike and toe off angles in all patients. First-time users (FTUs) showed similar gait parameter improvement patterns as frequent WW users (FUs). However, FUs with higher levels of gait impairment improved more in velocity, stride length and toe off angle compared to the FTUs. CONCLUSION: The impact of a WW can be quantified objectively by instrumented gait assessment. Thus, objective gait parameters may serve as surrogate markers for the use of walking aids in patients with gait and balance impairments.

Inertial sensor based gait analysis discriminates subjects with and without visual impairment caused by simulated macular degeneration.

Macular degeneration is the third leading cause of blindness worldwide and the leading cause of blindness in the developing world. The analysis of gait parameters can be used to assess the influence of macular degeneration on gait. This study examines the effect of macular degeneration on gait using inertial sensor based 3D spatio-temporal gait parameters. We acquired gait data from 21 young and healthy subjects during a 40 m obstacle walk. All subjects had to perform the gait trial with and without macular degeneration simulation glasses. The order of starting with or without glasses alternated between each subject in order to test for training effects. Multiple 3D spatio-temporal gait parameters were calculated for the normal vision as well as the impaired vision groups. The parameters trial time, stride time, stride time coefficient of variation (CV), stance time, stance time CV, stride length, cadence, gait velocity and angle at toe off showed statistically significant differences between the two groups. Training effects were visible for the trials which started without vision impairment. Inter-group differences in the gait pattern occurred due to an increased sense of insecurity related with the loss of visual acuity from the simulation glasses. In summary, we showed that 3D spatio-temporal gait parameters derived from inertial sensor data are viable to detect differences in the gait pattern of subjects with and without a macular degeneration simulation. We believe that this study provides the basis for an in-depth analysis regarding the impact of macular degeneration on gait.

Stride segmentation during free walk movements using multi-dimensional subsequence dynamic time warping on inertial sensor data.

Changes in gait patterns provide important information about individuals' health. To perform sensor based gait analysis, it is crucial to develop methodologies to automatically segment single strides from continuous movement sequences. In this study we developed an algorithm based on time-invariant template matching to isolate strides from inertial sensor signals. Shoe-mounted gyroscopes and accelerometers were used to record gait data from 40 elderly controls, 15 patients with Parkinson's disease and 15 geriatric patients. Each stride was manually labeled from a straight 40 m walk test and from a video monitored free walk sequence. A multi-dimensional subsequence Dynamic Time Warping (msDTW) approach was used to search for patterns matching a pre-defined stride template constructed from 25 elderly controls. F-measure of 98% (recall 98%, precision 98%) for 40 m walk tests and of 97% (recall 97%, precision 97%) for free walk tests were obtained for the three groups. Compared to conventional peak detection methods up to 15% F-measure improvement was shown. The msDTW proved to be robust for segmenting strides from both standardized gait tests and free walks. This approach may serve as a platform for individualized stride segmentation during activities of daily living.

Pull Test estimation in Parkinson's disease patients using wearable sensor technology.

Postural instability is one of the main motor impairment of Parkinson's disease (PD). The Pull Test is the most common clinical examination to assess postural instability in PD. However, the subjectivity and low discriminative power of this test presents as a major drawback. In this paper we propose a novel methodology to estimate the Pull Test scores from patients with PD. We capture the relationship between the Pull Test outcomes and patients' foot motion patterns, using wearable sensors mounted on their shoes. 139 idiopathic Parkinson's disease patients performed four motor function tests, including walking and repetitive foot motions, while acceleration and orientation data was recorded. A total of 684 features were extracted from the acceleration and orientation signals. Feature selection and classification algorithms were utilized to estimate the Pull Test score for each participant. Further, we estimate which motor function test would better predict the Pull Test score, depending on the patient's phenotype (i.e. bradykinetic, tremor-dominant or equivalent). When combining all phenotypes and all tests, the mean of the classification probability distribution achieved was 0.75 (CI: [0.69-0.82]). Foot circling was the best predictive test for the equivalent patients (mean = 0.79, CI: [0.69-0.87]) and the bradykinetic patients (mean: 0.75, CI: [0.64-0.85]), while 2×10 m. walk with stop-and-go proved superior for the tremor-dominant patients (mean: 0.75, CI: [0.64-0.85]). Overall, these results suggest that inertial data from patient's foot motion can be used to estimate postural instability in PD patients.

Timed Up-and-Go phase segmentation in Parkinson's disease patients using unobtrusive inertial sensors.

A widely accepted functional motor test for measuring basic mobility capabilities is the `Timed Up-and-Go' (TUG) test. Although several basic mobility tasks are included, only the total time is used as outcome parameter. It has been shown that timings of sub-phases can be used as relevant clinical parameters for the assessment of Parkinson's disease patients. A variety of systems and methods have been proposed for instrumenting the TUG test, but only limited information has been published regarding phase classification. In this paper an automated TUG phase classification methodology is proposed and validated in a study with 16 Parkinson's disease patients. Statistical, signal energy, chronological and gait features were extracted from acceleration and orientation signals of shoe mounted inertial measurement units. The phases `sit to walk', `walking', `first turn', `second turn' and `turn to sit' were segmented in a two stage classifier approach. Strides were used for a separation of the walking phase and classifiers like NaiveBayes, k-Nearest-Neighbor, Support Vector Machine (SVM) and Random Forest for the final phase segmentation. SVM performed best with a mean sensitivity of 81.80% over all phases. Additionally, the impact of UPDRS and Hoehn & Yahr ratings on the phase times was assessed. The proposed methodology could be used to analyze gait parameters of sub-phases like stride length, stride time, foot clearance, heel-strike or toe-off angle for an improved assessment of Parkinson's disease patients.

Inertial sensor based and shoe size independent gait analysis including heel and toe clearance estimation.

Falls are a major cause for morbidity and mortality in the ageing society. Inertial sensor based gait assessment including the analysis of the heel and toe clearance can be an indicator for the risk of falling. This paper presents a method for calculating the continuous heel and toe clearance without the knowledge of the shoe dimensions as well as the foot angle in the sagittal plane. These gait parameters were validated using an optical motion capture system. 20 healthy subjects from 3 different age groups (young, mid age, old) performed gait trials with different stride lengths and stride velocities. We obtained low mean absolute errors, low standard deviations and high Pearson correlations (0.91-0.99) for all gait parameters. In summary, we implemented a viable algorithm for the calculation of the heel and toe clearance without knowing the shoe dimensions as well as the foot angle in sagittal plane. We conclude that the given method is applicable for a mobile and unobtrusive gait assessment for healthy subjects from all age classes.

Inertial sensor-based stride parameter calculation from gait sequences in geriatric patients.

A detailed and quantitative gait analysis can provide evidence of various gait impairments in elderly people. To provide an objective decision-making basis for gait analysis, simple applicable tests analyzing a high number of strides are required. A mobile gait analysis system, which is mounted on shoes, can fulfill these requirements. This paper presents a method for computing clinically relevant temporal and spatial gait parameters. Therefore, an accelerometer and a gyroscope were positioned laterally below each ankle joint. Temporal gait events were detected by searching for characteristic features in the signals. To calculate stride length, the gravity compensated accelerometer signal was double integrated, and sensor drift was modeled using a piece-wise defined linear function. The presented method was validated using GAITRite-based gait parameters from 101 patients (average age 82.1 years). Subjects performed a normal walking test with and without a wheeled walker. The parameters stride length and stride time showed a correlation of 0.93 and 0.95 between both systems. The absolute error of stride length was 6.26 cm on normal walking test. The developed system as well as the GAITRite showed an increased stride length, when using a four-wheeled walker as walking aid. However, the walking aid interfered with the automated analysis of the GAITRite system, but not with the inertial sensor-based approach. In summary, an algorithm for the calculation of clinically relevant gait parameters derived from inertial sensors is applicable in the diagnostic workup and also during long-term monitoring approaches in the elderly population.

Wearable static posturography solution using a novel pressure sensor sole.

Static posturography is an important measurement in the diagnostic workup for patients with postural instability. New wearable sensor technologies enable researchers to use in-shoe pressure soles in the home environment and outdoor applications. In this study a newly developed in-shoe pressure sole was used for calculating the sway path and 95% confidence ellipse area as the standard parameters of typical static posturography. Insole posturography was validated on 24 subjects by a state of the art pressure plate assessment during three static posturography conditions (eyes open, eyes closed and barefoot). The adaptive low pass filtered data resulted in an overall correlation of 0.63 to 0.78 for the sway path and 0.66 to 0.79 for the 95% confidence ellipse area. Individual correlations of up to 0.97 for the sway path and 0.99 for the 95% confidence ellipse area could be obtained. Future applications could utilize the mobile advantage of in-shoe pressure soles and measure static and dynamic posturography in clinical and home environments.

Subsequence dynamic time warping as a method for robust step segmentation using gyroscope signals of daily life activities.

The segmentation of gait signals into single steps is an important basis for objective gait analysis. Only a precise detection of step beginning and end enables the computation of step parameters like step height, variability and duration. A special challenge for the application is the accurateness of such an algorithm when based on signals from daily live activities. In this study, gyroscopes were attached laterally to sport shoes to collect gait data. For the automated step segmentation, subsequence Dynamic Time Warping was used. 35 healthy controls and ten patients with Parkinson's disease performed a four times ten meter walk. Furthermore 4 subjects were recorded during different daily life activities. The algorithm enabled counting steps, detecting precisely step beginning and end and rejecting other movements. Results showed a recognition rate of steps during ten meter walk exercises of 97.7% and in daily life activities of 86.7%. The segmentation procedure can be used for gait analysis from daily life activities and can constitute the basis for computation of precise step parameters. The algorithm is applicable for long-term gait monitoring as well as for analyzing gait abnormalities.

Unbiased and mobile gait analysis detects motor impairment in Parkinson's disease.

Motor impairments are the prerequisite for the diagnosis in Parkinson's disease (PD). The cardinal symptoms (bradykinesia, rigor, tremor, and postural instability) are used for disease staging and assessment of progression. They serve as primary outcome measures for clinical studies aiming at symptomatic and disease modifying interventions. One major caveat of clinical scores such as the Unified Parkinson Disease Rating Scale (UPDRS) or Hoehn&Yahr (H&Y) staging is its rater and time-of-assessment dependency. Thus, we aimed to objectively and automatically classify specific stages and motor signs in PD using a mobile, biosensor based Embedded Gait Analysis using Intelligent Technology (eGaIT). eGaIT consist of accelerometers and gyroscopes attached to shoes that record motion signals during standardized gait and leg function. From sensor signals 694 features were calculated and pattern recognition algorithms were applied to classify PD, H&Y stages, and motor signs correlating to the UPDRS-III motor score in a training cohort of 50 PD patients and 42 age matched controls. Classification results were confirmed in a second independent validation cohort (42 patients, 39 controls). eGaIT was able to successfully distinguish PD patients from controls with an overall classification rate of 81%. Classification accuracy increased with higher levels of motor impairment (91% for more severely affected patients) or more advanced stages of PD (91% for H&Y III patients compared to controls), supporting the PD-specific type of analysis by eGaIT. In addition, eGaIT was able to classify different H&Y stages, or different levels of motor impairment (UPDRS-III). In conclusion, eGaIT as an unbiased, mobile, and automated assessment tool is able to identify PD patients and characterize their motor impairment. It may serve as a complementary mean for the daily clinical workup and support therapeutic decisions throughout the course of the disease.

Combined analysis of sensor data from hand and gait motor function improves automatic recognition of Parkinson's disease.

Objective and rater independent analysis of movement impairment is one of the most challenging tasks in medical engineering. Especially assessment of motor symptoms defines the clinical diagnosis in Parkinson's disease (PD). A sensor-based system to measure the movement of the upper and lower extremities would therefore complement the clinical evaluation of PD. In this study two different sensor-based systems were combined to assess movement of 18 PD patients and 17 healthy controls. First, hand motor function was evaluated using a sensor pen with integrated accelerometers and pressure sensors, and second, gait function was assessed using a sports shoe with attached inertial sensors (gyroscopes,accelerometers).Subjects performed standardized tests for both extremities.Features were calculated from sensor signals to differentiate between patients and controls. For the latter, pattern recognition methods were used and the performance of four classifiers was compared. In a first step classification was done for every single system and in a second step for combined features of both systems. Combination of both motor task assessments substantially improved classification rates to 97%using the AdaBoost classifier for the experiment patients vs.controls.The combination of two different analysis systems led to enhanced, more stable, objective, and rater independent recognition of motor impairment. The method can be used as a complementary diagnostic tool for movement disorders.

Biometric and mobile gait analysis for early diagnosis and therapy monitoring in Parkinson's disease.

Parkinson's disease (PD) is the most frequent neurodegenerative movement disorder. Early diagnosis and effective therapy monitoring is an important prerequisite to treat patients and reduce health care costs. Objective and non-invasive assessment strategies are an urgent need in order to achieve this goal. In this study we apply a mobile, lightweight and easy applicable sensor based gait analysis system to measure gait patterns in PD and to distinguish mild and severe impairment of gait. Examinations of 16 healthy controls, 14 PD patients in an early stage, and 13 PD patients in an intermediate stage were included. Subjects performed standardized gait tests while wearing sport shoes equipped with inertial sensors (gyroscopes and accelerometers). Signals were recorded wirelessly, features were extracted, and distinct subpopulations classified using different classification algorithms. The presented system is able to classify patients and controls (for early diagnosis) with a sensitivity of 88% and a specificity of 86%. In addition it is possible to distinguish mild from severe gait impairment (for therapy monitoring) with 100% sensitivity and 100% specificity. This system may be able to objectively classify PD gait patterns providing important and complementary information for patients, caregivers and therapists.