Detection and Assessment of Point-to-Point Movements During Functional Activities Using Deep Learning and Kinematic Analyses of the Stroke-Affected Wrist.

Stoke is a leading cause of long-term disability, including upper-limb hemiparesis. Frequent, unobtrusive assessment of naturalistic motor performance could enable clinicians to better assess rehabilitation effectiveness and monitor patients' recovery trajectories. We therefore propose and validate a two-phase data analytic pipeline to estimate upper-limb impairment based on the naturalistic performance of activities of daily living (ADLs). Eighteen stroke survivors were equipped with an inertial sensor on the stroke-affected wrist and performed up to four ADLs in a naturalistic manner. Continuous inertial time series were segmented into sliding windows, and a machine-learned model identified windows containing instances of point-to-point (P2P) movements. Using kinematic features extracted from the detected windows, a subsequent model was used to estimate upper-limb motor impairment, as measured by the Fugl-Meyer Assessment (FMA). Both models were evaluated using leave-one-subject-out cross-validation. The P2P movement detection model had an area under the precision-recall curve of 0.72. FMA estimates had a normalized root mean square error of 18.8% with R2=0.72. These promising results support the potential to develop seamless, ecologically valid measures of real-world motor performance.

Understanding voluntary human movement variability through data-driven segmentation and clustering.

Recently, we proposed a novel approach where movements are decomposed into sub-segments, termed movement elements. This approach, to date, provides a robust construct of how the brain may generate simple as well as complex movements. Here, we address the issue of motor variability during voluntary movements by applying an unsupervised clustering algorithm to group movement elements according to their morphological characteristics. We observed that most movement elements closely match the theoretical bell-shaped velocity profile expected from goal-directed movements. However, for those movement elements that deviate from this theoretical shape, a small number of defined patterns in their shape can be identified. Furthermore, we observed that the axis of the body from which the movement elements are extracted (i.e., medio-lateral, antero-posterior, and vertical) affect the proportion of the movement elements matching the theoretical model. These results provide novel insight into how the nervous system controls voluntary movements and may use variability in movement element properties to explore the environment.

Envisioning the use of in-situ arm movement data in stroke rehabilitation: Stroke survivors' and occupational therapists' perspectives.

BACKGROUND: The key for successful stroke upper-limb rehabilitation includes the personalization of therapeutic interventions based on patients' functional ability and performance level. However, therapists often encounter challenges in supporting personalized rehabilitation due to the lack of information about how stroke survivors use their stroke-affected arm outside the clinic. Wearable technologies have been considered as an effective, objective solution to monitor patients' arm use patterns in their naturalistic environments. However, these technologies have remained a proof of concept and have not been adopted as mainstream therapeutic products, and we lack understanding of how key stakeholders perceive the use of wearable technologies in their practice. OBJECTIVE: We aim to understand how stroke survivors and therapists perceive and envision the use of wearable sensors and arm activity data in practical settings and how we could design a wearable-based performance monitoring system to better support the needs of the stakeholders. METHODS: We conducted semi-structured interviews with four stroke survivors and 15 occupational therapists (OTs) based on real-world arm use data that we collected for contextualization. To situate our participants, we leveraged a pair of finger-worn accelerometers to collect stroke survivors' arm use data in real-world settings, which we used to create study probes for stroke survivors and OTs, respectively. The interview data was analyzed using the thematic approach. RESULTS: Our study unveiled a detailed account of (1) the receptiveness of stroke survivors and OTs for using wearable sensors in clinical practice, (2) OTs' envisioned strategies to utilize patient-generated sensor data in the light of providing patients with personalized therapy programs, and (3) practical challenges and design considerations to address for the accelerated integration of wearable systems into their practice. CONCLUSIONS: These findings offer promising directions for the design of a wearable solution that supports OTs to develop individually-tailored therapy programs for stroke survivors to improve their affected arm use.

LASense: Pushing the Limits of Fine-grained Activity Sensing Using Acoustic Signals.

Acoustic signals have been widely adopted in sensing fine-grained human activities, including respiration monitoring, finger tracking, eye blink detection, etc. One major challenge for acoustic sensing is the extremely limited sensing range, which becomes even more severe when sensing fine-grained activities. Different from the prior efforts that adopt multiple microphones and/or advanced deep learning techniques for long sensing range, we propose a system called LASense, which can significantly increase the sensing range for fine-grained human activities using a single pair of speaker and microphone. To achieve this, LASense introduces a virtual transceiver idea that purely leverages delicate signal processing techniques in software. To demonstrate the effectiveness of LASense, we apply the proposed approach to three fine-grained human activities, i.e., respiration, finger tapping and eye blink. For respiration monitoring, we significantly increase the sensing range from the state-of-the-art 2 m to 6 m. For finer-grained finger tapping and eye blink detection, we increase the state-of-the-art sensing range by 150% and 80%, respectively.

A Kinematic Data-Driven Approach to Differentiate Involuntary Choreic Movements in Individuals With Neurological Conditions.

OBJECTIVE: The ability to differentiate similar choreic involuntary movements could lay the groundwork for the development of a minimally-invasive screening tool for their etiology and provide in-depth understandings of pathophysiology. As a first step, we investigate kinematic differences between Huntington's disease (HD) chorea and Parkinson's disease (PD) choreic levodopa-induced dyskinesia (LID), which have distinct pathological causes yet share a great kinematic resemblance. METHODS: Twenty subjects with HD and ten subjects with PD stood with both upper limbs in front of them for approximately 60 seconds. The three-dimensional velocity time-series of involuntary movements of both hands were segmented into one-dimensional sub-movements abutted by velocity zero-crossings. A combination of unsupervised and supervised machine learning algorithms was employed to automatically select data features extracted from sub-movements and distinguish the two types of involuntary choreic movements. RESULTS: The trained model was able to accurately classify chorea vs. LID with an Area Under the Receiver Operating Characteristic Curve of 99.5%. A set of important features contributing to the construction of the classification model were identified and investigated. CONCLUSION: The trained model may serve as a tool for the automatic identification of different types of involuntary choreic movements, enabling continuous monitoring and personalized treatment for patients in various clinical settings. SIGNIFICANCE: The results provide insights into kinematic characteristics of HD chorea and PD LID, which is the first step towards an improved general understanding of involuntary choreic movements.

Analysis of Gait Sub-Movements to Estimate Ataxia Severity Using Ankle Inertial Data.

OBJECTIVE: Assessment of motor severity in cerebellar ataxia is critical for monitoring disease progression and evaluating the effectiveness of therapeutic interventions. Though wearable sensors have been used to monitor gait tasks in order to enable frequent assessment, existing solutions only estimate gait performance severity rather than comprehensive motor severity. In this study, we propose a new approach that analyzes sub-second movement profiles of the lower-limbs during gait to estimate overall motor severity in cerebellar ataxia. METHODS: A total of 37 ataxia subjects and 12 healthy subjects performed a 5 m walk-and-turn task with two ankle-worn inertial sensors. Lower-limb movements were decomposed into one-dimensional sub-movements, namely movement elements. Supervised regression models trained on data features of movement elements estimated the Brief Ataxia Rating Scale (BARS) and its sub-scores evaluated by clinicians. The proposed models were also compared to models trained on widely-accepted spatiotemporal gait features. RESULTS: Estimated total BARS showed strong agreement with clinician-evaluated scores with r2 = 0.72 and a root mean square error of 2.6 BARS points. Movement element-based models significantly outperformed conventional, spatiotemporal gait feature-based models. CONCLUSION: The proposed algorithm accurately assessed overall motor severity in cerebellar ataxia using inertial data collected from bilaterally-placed ankle sensors during a simple walk-and-turn task. SIGNIFICANCE: Our work could support fine-grained monitoring of disease progression and patients' responses to medical/clinical interventions.

Estimating Ground Reaction Force and Center of Pressure Using Low-Cost Wearable Devices.

OBJECTIVE: Ambulatory monitoring of ground reaction force (GRF) and center of pressure (CoP) could improve management of health conditions that impair mobility. Insoles instrumented with force-sensitive resistors (FSRs) are an unobtrusive, low-cost, and low-power technology for sampling GRF and CoP in real-world environments. However, FSRs have variable response characteristics that complicate estimation of GRF and CoP. This study introduces a unique data analytic pipeline that enables accurate estimation of GRF and CoP despite relatively inaccurate FSR responses. This paper also investigates whether inclusion of a complementary knee angle sensor improves estimation accuracy. METHODS: Seventeen healthy subjects were equipped with an insole instrumented with six FSRs and a string-based knee angle sensor. Subjects walked in a straight line at self-selected slow, preferred, and fast speeds over an in-ground force platform. Twenty repetitions were performed for each speed. Supervised machine learning models estimated weight-normalized GRF and shoe size-normalized CoP, which were re-scaled to obtain GRF and CoP. RESULTS: Anteroposterior GRF, Vertical GRF, and Anteroposterior CoP were estimated with a normalized root mean square error (NRMSE) of less than 5%. Mediolateral GRF and CoP were estimated with an NRMSE of 8.1% and 6.4%, respectively. Knee angle-related features slightly improved GRF estimates. CONCLUSION: Normalized models accurately estimated GRF and CoP despite deficiencies in FSR data. SIGNIFICANCE: Ambulatory use of the proposed system could enable objective, longitudinal monitoring of severity and progression for a variety of health conditions.

Decomposition of Reaching Movements Enables Detection and Measurement of Ataxia.

Technologies that enable frequent, objective, and precise measurement of ataxia severity would benefit clinical trials by lowering participation barriers and improving the ability to measure disease state and change. We hypothesized that analyzing characteristics of sub-second movement profiles obtained during a reaching task would be useful for objectively quantifying motor characteristics of ataxia. Participants with ataxia (N=88), participants with parkinsonism (N=44), and healthy controls (N=34) performed a computer tablet version of the finger-to-nose test while wearing inertial sensors on their wrists. Data features designed to capture signs of ataxia were extracted from participants' decomposed wrist velocity time-series. A machine learning regression model was trained to estimate overall ataxia severity, as measured by the Brief Ataxia Rating Scale (BARS). Classification models were trained to distinguish between ataxia participants and controls and between ataxia and parkinsonism phenotypes. Movement decomposition revealed expected and novel characteristics of the ataxia phenotype. The distance, speed, duration, morphology, and temporal relationships of decomposed movements exhibited strong relationships with disease severity. The regression model estimated BARS with a root mean square error of 3.6 points, r2 = 0.69, and moderate-to-excellent reliability. Classification models distinguished between ataxia participants and controls and ataxia and parkinsonism phenotypes with areas under the receiver-operating curve of 0.96 and 0.89, respectively. Movement decomposition captures core features of ataxia and may be useful for objective, precise, and frequent assessment of ataxia in home and clinic environments.

Estimating Quality of Reaching Movement Using a Wrist-Worn Inertial Sensor.

Stroke is a major cause of long-term disability. Because patients recovering from stroke often perform differently in clinical settings than in their naturalistic environments, remote monitoring of motor performance is needed to evaluate the true impact of prescribed therapies. Wearable sensors have been considered as a technical solution to this problem, but most existing systems focus on measuring the amount of movement without considering the quality of movement. We present a novel method to seamlessly and unobtrusively measure the quality of individual reaching movements by leveraging a motor control theory that describes how the central nervous system plans and executes movements. We trained and evaluated our system on 19 stroke survivors to estimate the Functional Ability Scale (FAS) of reaching movements. The analysis showed that we can estimate the FAS scores of reaching movements, with some confusion between adjacent scores. Furthermore, we estimated the average FAS scores of subjects with a normalized root mean square error (NRMSE) of 22.5%. Though our model's high error on two severe subjects influenced our overall estimation performance, we could accurately estimate scores in most of the mild-to-moderate subjects (NRMSE of 13.1% without the outliers). With further development and testing, we believe the proposed technique can be applied to monitor patient recovery in home and community settings.

A Simple Low-Cost Wearable Sensor for Long-Term Ambulatory Monitoring of Knee Joint Kinematics.

OBJECTIVE: Accurate monitoring of joint kinematics in individuals with neuromuscular and musculoskeletal disorders within ambulatory settings could provide important information about changes in disease status and the effectiveness of rehabilitation programs and/or pharmacological treatments. This paper introduces a reliable, power efficient, and low-cost wearable system designed for the long-term monitoring of joint kinematics in ambulatory settings. METHODS: Seventeen healthy subjects wore a retractable string sensor, fixed to two anchor points on the opposing segments of the knee joint, while walking at three different self-selected speeds. Joint angles were estimated from calibrated sensor values and their derivatives in a leave-one-subject-out cross validation manner using a random forest algorithm. RESULTS: The proposed system estimated knee flexion/extension angles with a root mean square error (RMSE) of 5.0°±1.0° across the study subjects upon removal of a single outlier subject. The outlier was likely a result of sensor miscalibration. CONCLUSION: The proposed wearable device can accurately estimate knee flexion/extension angles during locomotion at various walking speeds. SIGNIFICANCE: We believe that our novel wearable technology has great potential to enable joint kinematic monitoring in ambulatory settings and thus provide clinicians with an opportunity to closely monitor joint recovery, develop optimal, personalized rehabilitation programs, and ultimately maximize therapeutic outcomes.

Estimating Upper-Limb Impairment Level in Stroke Survivors Using Wearable Inertial Sensors and a Minimally-Burdensome Motor Task.

Upper-limb paresis is the most common motor impairment post stroke. Current solutions to automate the assessment of upper-limb impairment impose a number of critical burdens on patients and their caregivers that preclude frequent assessment. In this work, we propose an approach to estimate upper-limb impairment in stroke survivors using two wearable inertial sensors, on the wrist and the sternum, and a minimally-burdensome motor task. Twenty-three stroke survivors with no, mild, or moderate upper-limb impairment performed two repetitions of one-to-two minute-long continuous, random (i.e., patternless), voluntary upper-limb movements spanning the entire range of motion. The three-dimensional time-series of upper-limb movements were segmented into a series of one-dimensional submovements by employing a unique movement decomposition technique. An unsupervised clustering algorithm and a supervised regression model were used to estimate Fugl-Meyer Assessment (FMA) scores based on features extracted from these submovements. Our regression model estimated FMA scores with a normalized root mean square error of 18.2% ( r2=0.70 ) and needed as little as one minute of movement data to yield reasonable estimation performance. These results support the possibility of frequently monitoring stroke survivors' rehabilitation outcomes, ultimately enabling the development of individually-tailored rehabilitation programs.

A novel upper-limb function measure derived from finger-worn sensor data collected in a free-living setting.

The use of wrist-worn accelerometers has recently gained tremendous interest among researchers and clinicians as an objective tool to quantify real-world use of the upper limbs during the performance of activities of daily living (ADLs). However, wrist-worn accelerometers have shown a number of limitations that hinder their adoption in the clinic. Among others, the inability of wrist-worn accelerometers to capture hand and finger movements is particularly relevant to monitoring the performance of ADLs. This study investigates the use of finger-worn accelerometers to capture both gross arm and fine hand movements for the assessment of real-world upper-limb use. A system of finger-worn accelerometers was utilized to monitor eighteen neurologically intact young adults while performing nine motor tasks in a laboratory setting. The system was also used to monitor eighteen subjects during the day time of a day in a free-living setting. A novel measure of real-world upper-limb function-comparing the duration of activities of the two limbs-was derived to identify which upper limb subjects predominantly used to perform ADLs. Two validated handedness self-reports, namely the Waterloo Handedness Questionnaire and the Fazio Laterality Inventory, were collected to assess convergent validity. The analysis of the data recorded in the laboratory showed that the proposed measure of upper-limb function is suitable to accurately detect unilateral vs. bilateral use of the upper limbs, including both gross arm movements and fine hand movements. When applied to recordings collected in a free-living setting, the proposed measure showed high correlation with self-reported handedness indices (i.e., ρ = 0.78 with the Waterloo Handedness Questionnaire scores and ρ = 0.77 with the Fazio Laterality Inventory scores). The results herein presented establish face and convergent validity of the proposed measure of real-world upper-limb function derived using data collected by means of finger-worn accelerometers.

The Use of a Finger-Worn Accelerometer for Monitoring of Hand Use in Ambulatory Settings.

Objective assessment of stroke survivors' upper limb movements in ambulatory settings can provide clinicians with important information regarding the real impact of rehabilitation outside the clinic and help to establish individually-tailored therapeutic programs. This paper explores a novel approach to monitor the amount of hand use, which is relevant to the purposeful, goal-directed use of the limbs, based on a body networked sensor system composed of miniaturized finger- and wrist-worn accelerometers. The main contributions of this paper are twofold. First, this paper introduces and validates a new benchmark measurement of the amount of hand use based on data recorded by a motion capture system, the gold standard for human movement analysis. Second, this paper introduces a machine learning-based analytic pipeline that estimates the amount of hand use using data obtained from the wearable sensors and validates its estimation performance against the aforementioned benchmark measurement. Based on data collected from 18 neurologically intact individuals performing 11 motor tasks resembling various activities of daily living, the analytic results presented herein show that our new benchmark measure is reliable and responsive, and that the proposed wearable system can yield an accurate estimation of the amount of hand use (normalized root mean square error of 0.11 and average Pearson correlation of 0.78). This study has the potential to open up new research and clinical opportunities for monitoring hand function in ambulatory settings, ultimately enabling evidence-based, patient-centered rehabilitation and healthcare.

Finger-Worn Sensors for Accurate Functional Assessment of the Upper Limbs in Real-World Settings.

Remote monitoring of stroke survivors' upper limb performance (stroke-affected vs. unaffected limbs) can provide clinicians with information regarding the true impact of rehabilitation in the real-world settings, which allows opportunities to administer individually tailored therapeutic interventions. In this work, we examine the use of finger-worn accelerometers, which are capable of capturing gross-arm as well as fine-hand movements, in order to quantitatively compare the performance of the upper limbs during goal-directed activities of daily living (ADLs). In this proof-of-concept study, data were collected over an eight-hour duration from ten neurologically intact individuals who wore the sensors and continued with their daily living. The sensor-based measure was compared to two clinically validated measures of handedness, i.e., Waterloo Handedness Questionnaire and Fazio Laterality Inventory, that quantity the level of preference of the limbs in performing ADLs. The results yielded statistically significant correlations to the Waterloo and Fazio scores with Pearson correlation coefficients of 0.90 and 0.87 respectively, which was substantially superior compared to the previously studied measure based on wrist-worn accelerometers. We believe this study presents an opportunity to accurately monitor the goal-directed use of the upper limbs in the real-world settings.

Estimating Mini Mental State Examination Scores using Game-Specific Performance Values: A Preliminary Study.

Individuals with permanent cognitive impairment need to be evaluated and monitored. There exists a number of clinically validated cognitive assessment tools, but they often need to be administered by trained therapists in clinical settings. This serves as a major barrier for frequent, longitudinal monitoring of cognitive function. In this work, we introduce Neuro-World, a collection of innovative 3D mobile games, that allows one to self-administer the assessment of his/her cognitive function. The game performance is analyzed and converted into a clinically-accepted measure of cognitive function, specifically the Mini Mental State Examination (MMSE) score, improving the translational impact of the system in real-world clinical settings. To validate the feasibility of our approach, we collected game-specific performance data from 12 post-stroke patients, which was used to train a supervised machine learning model to estimate the corresponding MMSE score. Our experiment results showed a normalized root mean square error of 5.3% between the actual and estimated MMSE scores. This study enables new clinical and research opportunities for accurate longitudinal assessment of cognitive function via an interactive means of playing mobile games.

Effectiveness of the RAPAEL Smart Board for Upper Limb Therapy in Stroke Survivors: A Pilot Controlled Trial.

We aim to assess the effectiveness of using the RAPAEL Smart Board as an assistive tool for therapists in clinical rehabilitation therapy settings and to investigate if it can be used to improve the motor recovery rate of stroke survivors. The RAPAEL Smart Board is a therapy tool where therapists actively engage patients, giving necessary verbal and physical interventions as in traditional treatment sessions. We conducted a randomized controlled study with 17 stroke survivors. An experimental group received therapy using the RAPAEL Smart Board for 30 minutes a day, 5 days per week, for 4 weeks in addition to their traditional treatments (i.e., 30 minutes of functional arm movement therapy). A control group received two 30-minute sessions of traditional treatment 5 days per week, for 4 weeks. The upper-extremity function was measured using the Wolf Motor Function Test before and after the 4-week interventions. Our results demonstrate that using the RAPAEL Smart Board, in combination with traditional treatment, significantly improves motor recovery when compared to traditional treatments alone.

Towards the Ambulatory Assessment of Movement Quality in Stroke Survivors using a Wrist-worn Inertial Sensor.

Stroke is a leading cause of long-term disability that may lead to significant functional motor impairments in the upper limb (UL). Wrist-worn inertial sensors have emerged as an objective, minimally-obtrusive tool to monitor UL motor function in the real-world setting, such that rehabilitation interventions can be individually tailored to maximize functional performance. However, current wearable solutions focus on capturing the quantity of movement without considering the quality of movement. This paper introduces a novel approach to unobtrusively estimate the quality of UL movements in stroke survivors using a single wrist-worn inertial sensor during any type of voluntary UL movements. The proposed method exploits kinematic characteristics of voluntary limb movements that are optimized by the central nervous system during motor control. This work demonstrates that the proposed method could extract clinically important information during random UL movements in 16 stroke survivors, showing a statistically significant correlation to the Functional Ability Scale - a clinically validated score for movement quality.

Objectively quantifying walking ability in degenerative spinal disorder patients using sensor equipped smart shoes.

Lumbar spinal stenosis (LSS) is a condition associated with the degeneration of spinal disks in the lower back. A significant majority of the elderly population experiences LSS, and the number is expected to grow. The primary objective of medical treatment for LSS patients has focused on improving functional outcomes (e.g., walking ability) and thus, an accurate, objective, and inexpensive method to evaluate patients' functional levels is in great need. This paper aims to quantify the functional level of LSS patients by analyzing their clinical information and their walking ability from a 10 m self-paced walking test using a pair of sensorized shoes. Machine learning algorithms were used to estimate the Oswestry Disability Index, a clinically well-established functional outcome, from a total of 29 LSS patients. The estimated ODI scores showed a significant correlation to the reported ODI scores with a Pearson correlation coefficient (r) of 0.81 and p<3.5×10(-11). It was further shown that the data extracted from the sensorized shoes contribute most to the reported estimation results, and that the contribution of the clinical information was minimal. This study enables new research and clinical opportunities for monitoring the functional level of LSS patients in hospital and ambulatory settings.

A Prediction Model for Functional Outcomes in Spinal Cord Disorder Patients Using Gaussian Process Regression.

Predicting the functional outcomes of spinal cord disorder patients after medical treatments, such as a surgical operation, has always been of great interest. Accurate posttreatment prediction is especially beneficial for clinicians, patients, care givers, and therapists. This paper introduces a prediction method for postoperative functional outcomes by a novel use of Gaussian process regression. The proposed method specifically considers the restricted value range of the target variables by modeling the Gaussian process based on a truncated Normal distribution, which significantly improves the prediction results. The prediction has been made in assistance with target tracking examinations using a highly portable and inexpensive handgrip device, which greatly contributes to the prediction performance. The proposed method has been validated through a dataset collected from a clinical cohort pilot involving 15 patients with cervical spinal cord disorder. The results show that the proposed method can accurately predict postoperative functional outcomes, Oswestry disability index and target tracking scores, based on the patient's preoperative information with a mean absolute error of 0.079 and 0.014 (out of 1.0), respectively.