Employing deep reinforcement learning to maximize lower limb blood flow using intermittent pneumatic compression.

Intermittent pneumatic compression (IPC) systems apply external pressure to the lower limbs and enhance peripheral blood flow. We previously introduced a cardiac-gated compression system that enhanced arterial blood velocity (BV) in the lower limb compared to fixed compression timing (CT) for seated and standing sub7 jects. However, these pilot studies found that the CT that maximized BV was not constant across individuals and could change over time. Current CT modelling methods for IPC are limited to predictions for a single day and one heartbeat ahead. However, IPC therapy for may span weeks or longer, the BV response to compression can vary with physiological state, and the best CT for eliciting the desired physiological outcome may change, even for the same individual. We propose that a deep reinforcement learning (DRL) algorithm can learn and adaptively modify CT to achieve a selected outcome using IPC. Herein, we target maximizing lower limb arterial BV as the desired out19 come and build participant-specific simulated lower limb environments for 6 participants. We show that DRL can adaptively learn the CT for IPC that maximized arterial BV. Compared to previous work, the DRL agent achieves 98% ± 2 of the resultant blood flow and is faster at maximizing BV; the DRL agent can learn an "optimal" policy in 15 minutes ± 2 on average and can adapt on the fly. Given a desired objective, we posit that the proposed DRL agent can be implemented in IPC systems to rapidly learn the (potentially time-varying) "optimal" CT with a human-in-the-loop.

IMU-Based Real-Time Estimation of Gait Phase Using Multi-Resolution Neural Networks.

This work presents a real-time gait phase estimator using thigh- and shank-mounted inertial measurement units (IMUs). A multi-rate convolutional neural network (CNN) was trained to estimate gait phase for a dataset of 16 participants walking on an instrumented treadmill with speeds varying between 0.1 to 1.9 m/s, and conditions such as asymmetric walking, stop-start, and sudden speed changes. One-subject-out cross-validation was used to assess the robustness of the estimator to the gait patterns of new individuals. The proposed model had a spatial root mean square error of 5.00±1.65%, and a temporal mean absolute error of 2.78±0.97% evaluated at the heel strike. A second cross-validation was performed to show that leaving out any of the walking conditions from the training dataset did not result in significant performance degradation. A 2-sample Kolmogorov-Smirnov test showed that there was no significant increase in spatial or temporal error when testing on the abnormal walking conditions left out of the training set. The results of the two cross-validations demonstrate that the proposed model generalizes well across new participants, various walking speeds, and gait patterns, showcasing its potential for use in investigating patient populations with pathological gaits and facilitating robot-assisted walking.

Visual feedback decoding during bimanual circle drawing.

The between-hand interference during bimanual tasks is a consequence of the connection between the neural controllers of movement. Previous studies showed the existence of an asymmetric between-hand interference (caused by neural cross talk) when different kinematics plans were to be executed by each hand or when only one was visually guided and received perturbed visual feedback. Here, in continuous bimanual circle drawing tasks, we investigated if the central nervous system (CNS) can benefit from visual composite feedback, i.e., a weighted sum of hands' positions presented for the visually guided hand, to control the nonvisible hand. Our results demonstrated improvement in the nonvisible nondominant hand (NDH) performance in the presence of the composite feedback. When NDH was visually guided, the dominant hand's (DH) performance during asymmetric drawing deteriorated, whereas its performance during symmetric drawing improved. This indicates that the CNS's ability to leverage composite feedback, which can be the result of decoding the nonvisible hand positional information from the composite feedback, is task-dependent and can be asymmetric. Also, the nonvisible hand's performance degraded when DH or NDH was visually guided with amplified error feedback. The results of the amplified feedback condition do not strongly support the asymmetry of the interference during asymmetric circle drawing. Comparing muscle activations in the asymmetric experiment, we concluded that the observed kinematic differences were not due to alternation in muscle co-contractions.NEW & NOTEWORTHY Many daily activities involve bimanual coordination while simultaneous movement of the hands may result in interference with their movements. Here, we studied whether the central nervous system could use the relevant information in composite feedback, i.e., a weighted sum of positional information of nonvisible and visible hands, to improve the movement of the nonvisible hand. Our results suggest the ability to decode and associate task-relevant information from the composite feedback.

Accurate Real-Time Joint Torque Estimation for Dynamic Prediction of Human Locomotion.

Inverse dynamics is a common tool for determining human joint torques during walking. The traditional approaches rely on ground reaction force and kinematics measurements prior to analysis. A novel real-time hybrid method is proposed in this work by integrating a neural network and dynamic model that only requires kinematic data. An end-to-end neural network for direct joint torque estimation is also developed based on kinematic data. The neural networks are trained on a variety of walking conditions, including starting and stopping, sudden speed changes, and asymmetrical walking. The hybrid model is first tested in a detailed dynamic gait simulation (OpenSim) which results in root mean square errors less than 5 N.m and a correlation coefficient of greater than 0.95 for all the joints. Experiments demonstrate that the end-to-end model on average outperforms the hybrid model across the whole test when compared to the gold standard approach which requires both kinetic and kinematic information. The two torque estimators are also tested on one participant wearing a lower limb exoskeleton. In this case, the hybrid model (R 0.84) has significantly better performance than the end-to-end neural network (R 0.59). This indicates that the hybrid model is better applicable to scenarios which differ from the training data.

Accurate Real-time Phase Estimation for Normal and Asymmetric Gait.

An accurate real-time gait phase estimator for normal and asymmetric gait is developed by training and testing a time-delay neural network on gait data collected from six participants during treadmill walking. The trained model can generate smooth and highly accurate predictions of the gait phase with a root mean square error of less than 3.48% and 4.31% in normal and asymmetric gait, respectively. The coefficient of determination between the estimated and target phase is greater than 99% for all subjects with both normal and asymmetric gait. The proposed gait estimator also exhibits precise heel-strike event detection with an RMSE of 2.56% and 3.70% in normal and asymmetric gait, respectively. A spatial impedance controller is then employed and tested based on the estimated gait phase of a new participant. Obtained results confirm that the controller provided assistance in coordination with the user's motion both in normal and asymmetric gait conditions. The estimated gait phase is compared in the case of walking without and with the exoskeleton in passive and active modes, indicating persistent accuracy of the gait phase estimator regardless of the walking conditions.

Ultra-Robust Real-Time Estimation of Gait Phase.

An ultra-robust accurate gait phase estimator is developed by training a time-delay neural network (D67) on data collected from the hip and knee joint angles of 14 participants during treadmill and overground walking. Collected data include normal gait at speeds ranging from 0.1m/s to 1.9m/s and conditions such as long stride, short stride, asymmetric walking, stop-start, and abrupt speed changes. Spatial analysis of our method indicates an average RMSE of 1.74±0.23% and 2.35±0.52% in gait phase estimation of test participants in the treadmill and overground walking, respectively. The temporal analysis reveals that D67 detects heel-strike events with an average MAE of 1.70±0.54% and 2.74±0.92% of step duration on test participants in the treadmill and overground walking, respectively. Both spatial and temporal performances are uniform across participants and gait conditions. Further analyses indicate the robustness of the D67 to smooth and abrupt speed changes, limping, variation of stride length, and sudden start or stop of walking. The performance of the D67 is also compared to the state-of-the-art techniques confirming the superior and comparable performance of the D67 to techniques without and with a ground contact sensor, respectively. The estimator is finally tested on a participant walking with an active exoskeleton, demonstrating the robustness of D67 in interaction with an exoskeleton without being trained on any data from the test subject with or without an exoskeleton.

Cuffless Blood Pressure Estimation During Moderate- and Heavy-Intensity Exercise Using Wearable ECG and PPG.

OBJECTIVE: To develop and evaluate an accurate method for cuffless blood pressure (BP) estimation during moderate- and heavy-intensity exercise. METHODS: Twelve participants performed three cycling exercises: a ramp-incremental exercise to exhaustion, and moderate and heavy pseudorandom binary sequence exercises on an electronically braked cycle ergometer over the course of 21 minutes. Subject-specific and population-based nonlinear autoregressive models with exogenous inputs (NARX) were compared with feedforward artificial neural network (ANN) models and pulse arrival time (PAT) models. RESULTS: Population-based NARX models, (applying leave-one-subject-out cross-validation), performed better than the other models and showed good capability for estimating large changes in mean arterial pressure (MAP). The models were unable to track consistent decreases in BP during prolonged exercise caused by reduction in peripheral vascular resistance, since this information is apparently not encoded in the employed proxy physiological signals (electrocardiography and forehead PPG) used for BP estimation. Nevertheless, the population-based NARX model had an error standard deviation of 11.0 mmHg during the entire exercise window, which improved to 9.0 mmHg when the model was periodically calibrated every 7 minutes. CONCLUSION: Population-based NARX models can estimate BP during moderate- and heavy-intensity exercise but need periodic calibration to account for the change in vascular resistance during exertion. SIGNIFICANCE: MAP can be continuously tracked during exercise using only wearable sensors, making monitoring exercise physiology more convenient and accessible.

A Unified Gait Phase Estimation and Control of Exoskeleton using Virtual Energy Regulator (VER).

Virtual Energy Regulator (VER) is a time independent controller that can generate stable limit cycles in lower-limb exoskeleton devices. In this work, we apply VER to control a lower-limb exoskeleton for assistive walking. We design two different limit cycles for hip and knee joints to assist the user during overground walking with the Indego explorer lower-limb exoskeleton. We tested the designed VER on a single participant for overground walking at a self-selected speed. Interestingly, due to VER time-independent nature, it can properly coordinate with the user's motions and produce mechanically stable overground walking in which the user can walk overground without a walker or crutches. The resultant gait is also more similar to a normal gait with improved range of motion compared to cases without controller; range of motion improved from $42.9 \pm 4.8{deg}$ and $44.9 \pm 4.9 {deg}$ to $46.6 \pm 1.3 {deg}$ and $63.0 \pm 6.8 {deg}$ at hip and knee joints, respectively. Especially, for the knee joint, the user is able to fully extend her knee during stance phase only when the VER is in the loop. In VER, the radius of each desired limit cycle is a function of phase. Accordingly, during walking, the internal phase of the VER is a monotonically increasing parameter that can be considered as a candidate for real-time gait phase estimation and heel-strike event detection. Hence, for gait phase estimation, VER relies only on a single joint position provided by the exoskeleton.

Adaptive Reference Inverse Optimal Control for Natural Walking With Musculoskeletal Models.

An efficient inverse optimal control method named Adaptive Reference IOC is introduced to study natural walking with musculoskeletal models. Adaptive Reference IOC utilizes efficient inner-loop direct collocation for optimal trajectory prediction along with a gradient-based weight update inspired by structured classification in the outer-loop to achieve about 7 times faster convergence than existing derivative-free methods while maintaining similar outcomes in terms of gait trajectory matching. The proposed method adequately reconstructed the reference data when applied to experimental walking data from ten participants walking at various speeds and stride lengths. The proposed framework can facilitate efficient personalized cost function optimization for specific walking tasks, and provide guidance to personalized reference trajectory design for assistive robotic systems such as lower-limb exoskeletons.

A fusion approach to improve accuracy and estimate uncertainty in cuffless blood pressure monitoring.

A substantial barrier to the clinical adoption of cuffless blood pressure (BP) monitoring techniques is the lack of unified error standards and methods of estimating measurement uncertainty. This study proposes a fusion approach to improve accuracy and estimate prediction interval (PI) as a proxy for uncertainty for cuffless blood BP monitoring. BP was estimated during activities of daily living using three model architectures: nonlinear autoregressive models with exogenous inputs, feedforward neural network models, and pulse arrival time models. Multiple one-class support vector machine (OCSVM) models were trained to cluster data in terms of the percentage of outliers. New BP estimates were then assigned to a cluster using the OCSVMs hyperplanes, and the PIs were estimated using the BP error standard deviation associated with different clusters. The OCSVM was used to estimate the PI for the three BP models. The three BP estimations from the models were fused using the covariance intersection fusion algorithm, which improved BP and PI estimates in comparison with individual model precision by up to 24%. The employed model fusion shows promise in estimating BP and PI for potential clinical uses. The PI indicates that about 71%, 64%, and 29% of the data collected from sitting, standing, and walking can result in high-quality BP estimates. Our PI estimator offers an effective uncertainty metric to quantify the quality of BP estimates and can minimize the risk of false diagnosis.

Balance strategy in hoverboard control.

This study examines how people learn to perform lower limb control in a novel task with a hoverboard requiring to maintain dynamic balance. We designed an experiment to investigate the learning of hoverboard balance and two control strategies: A hip strategy, which mainly uses hip movements to change the angle of the foot, and an ankle strategy relying more on ankle motion to control the orientation of hoverboard plates controlling the motion. Motor learning was indicated by a significant [Formula: see text]% decrease in the trial completion time (p < 0.001) and a significant 24 ± 11% decrease in total muscle activation (p < 0.001). Furthermore, the participants, who had no prior experience riding a hoverboard, learned an ankle strategy to maintain their balance and control the hoverboard. This is supported by significantly stronger cross-correlation, phase synchrony, lower dynamic time warping distance between the hoverboard plate orientation controlling hoverboard motion, and the ankle angle when compared to the hip angle. The adopted ankle strategy was found to be robust to the foot orientation despite salient changes in muscle group activation patterns. Comparison with results of an experienced hoverboard rider confirmed that the first-time riders adopted an ankle strategy.

Accurate Blood Pressure Estimation During Activities of Daily Living: A Wearable Cuffless Solution.

The objective is to develop a cuffless method that accurately estimates blood pressure (BP) during activities of daily living. User-specific nonlinear autoregressive models with exogenous inputs (NARX) are implemented using artificial neural networks to estimate the BP waveforms from electrocardiography and photoplethysmography signals. To broaden the range of BP in the training data, subjects followed a short procedure consisting of sitting, standing, walking, Valsalva maneuvers, and static handgrip exercises. The procedure was performed before and after a six-hour testing phase wherein five participants went about their normal daily living activities. Data were further collected at a four-month time point for two participants and again at six months for one of the two. The performance of three different NARX models was compared with three pulse arrival time (PAT) models. The NARX models demonstrate superior accuracy and correlation with "ground truth" systolic and diastolic BP measures compared to the PAT models and a clear advantage in estimating the large range of BP. Preliminary results show that the NARX models can accurately estimate BP even months apart from the training. Preliminary testing suggests that it is robust against variabilities due to sensor placement. This establishes a method for cuffless BP estimation during activities of daily living that can be used for continuous monitoring and acute hypotension and hypertension detection.

Cable-Driven Robotic Interface for Lower Limb Neuromechanics Identification.

This paper presents a versatile cable-driven robotic interface to investigate the single-joint joint neuromechanics of the hip, knee and ankle in the sagittal plane. This endpoint-based interface offers highly dynamic interaction and accurate position control (as is typically required for neuromechanics identification), and provides measurements of position, interaction force and electromyography (EMG) of leg muscles. It can be used with the subject upright, corresponding to a natural posture during walking or standing, and does not impose kinematic constraints on a joint, in contrast to existing interfaces. Mechanical evaluations demonstrated that the interface yields a rigidity above 500 N/m with low viscosity. Tests with a rigid dummy leg and linear springs show that it can identify the mechanical impedance of a limb accurately. A smooth perturbation is developed and tested with a human subject, which can be used to estimate the hip neuromechanics.

Cuffless Blood Pressure Estimation for Activities of Daily Living.

This work presents a modelling approach to predict the blood pressure (BP) waveform time series during activities of daily living without the use of a traditional pressure cuff. A nonlinear autoregressive model with exogenous inputs (NARX) is implemented using artificial neural networks and trained to predict the BP waveform time series from electrocardiography (ECG) and forehead photoplethysmography (PPG) input signals. To broaden the range of blood pressures present in the training set, a protocol was implemented that included sitting, standing, walking, Valsalva manoeuvers, and static handgrip exercise. A five-minute interval of data in the sitting position at the end of the day was also used for training. The efficacy of the cuffless BP method for continuous BP estimation over 4.67 hours was evaluated on 3 participants for varying training data segments. A mean absolute error of 6.3 and 5.2 mmHg were achieved for systolic BP and diastolic BP estimates, respectively. Including static handgrips and Valsalva manoeuvers in the training dataset leads to better estimation of the higher ranges of BP observed throughout the day. The proposed method shows potential for estimating the range of BP experienced during activities of daily living.Clinical Relevance- Establishes a method for cuffless continuous blood pressure estimation during activities of daily living that can be used for continuous monitoring and acute hypertension detection.

Quantitative Modeling of Spasticity for Clinical Assessment, Treatment and Rehabilitation.

Spasticity, a common symptom in patients with upper motor neuron lesions, reduces the ability of a person to freely move their limbs by generating unwanted reflexes. Spasticity can interfere with rehabilitation programs and cause pain, muscle atrophy and musculoskeletal deformities. Despite its prevalence, it is not commonly understood. Widely used clinical scores are neither accurate nor reliable for spasticity assessment and follow up of treatments. Advancement of wearable sensors, signal processing and robotic platforms have enabled new developments and modeling approaches to better quantify spasticity. In this paper, we review quantitative modeling techniques that have been used for evaluating spasticity. These models generate objective measures to assess spasticity and use different approaches, such as purely mechanical modeling, musculoskeletal and neurological modeling, and threshold control-based modeling. We compare their advantages and limitations and discuss the recommendations for future studies. Finally, we discuss the focus on treatment and rehabilitation and the need for further investigation in those directions.

A Clustering-Based Approach to Identify Joint Impedance During Walking.

Mechanical impedance, which changes with posture and muscle activations, characterizes how the central nervous system regulates the interaction with the environment. Traditional approaches to impedance estimation, based on averaging of movement kinetics, requires a large number of trials and may introduce bias to the estimation due to the high variability in a repeated or periodic movement. Here, we introduce a data-driven modeling technique to estimate joint impedance considering the large gait variability. The proposed method can be used to estimate impedance in both the stance and swing phases of walking. A 2-pass clustering approach is used to extract groups of unperturbed gait data and estimate candidate baselines. Then patterns of perturbed data are matched with the most similar unperturbed baseline. The kinematic and torque deviations from the baselines are regressed locally to compute joint impedance at different gait phases. Simulations using the trajectory data of a subject's gait at different speeds demonstrate a more accurate estimation of ankle stiffness and damping with the proposed clustering-based method when compared with two methods: i) using average unperturbed baselines, and ii) matching shifted and scaled average unperturbed velocity baselines. Furthermore, the proposed method requires fewer trials than methods based on average unperturbed baselines. The experimental results on human hip impedance estimation show the feasibility of clustering-based technique and verifies that it reduces the estimation variability.

The Influence of Posture, Applied Force and Perturbation Direction on Hip Joint Viscoelasticity.

Limb viscoelasticity is a critical neuromechanical factor used to regulate the interaction with the environment. It plays a key role in modelling human sensorimotor control, and can be used to assess the condition of healthy and neurologically affected individuals. This paper reports the estimation of hip joint viscoelasticity during voluntary force control using a novel device that applies a leg displacement without constraining the hip joint. The influence of hip angle, applied limb force and perturbation direction on the stiffness and viscosity values was studied in ten subjects. No difference was detected in the hip joint stiffness between the dominant and non-dominant legs, but a small dependency was observed on the perturbation direction. Both hip stiffness and viscosity increased monotonically with the applied force magnitude, with posture being observed to have a slight influence. These results are in line with previous measurements carried out on upper limbs, and can be used as a baseline for lower limb movement simulation and further neuromechanical investigations.

A Robotic Glenohumeral Simulator for Investigating Prosthetic Implant Subluxation.

Total shoulder arthroplasty (TSA) is an effective treatment for glenohumeral (GH) osteoarthritis. However, it still suffers from a substantial rate of mechanical failure, which may be related to cyclic off-center loading of the humeral head on the glenoid. In this work, we present the design and evaluation of a GH joint robotic simulator developed to study GH translations. This five-degree-of-freedom robot was designed to replicate the rotations (±40 deg, accuracy 0.5 deg) and three-dimensional (3D) forces (up to 2 kN, with a 1% error settling time of 0.6 s) that the humeral implant exerts on the glenoid implant. We tested the performances of the simulator using force patterns measured in real patients. Moreover, we evaluated the effect of different orientations of the glenoid implant on joint stability. When simulating realistic dynamic forces and implant orientations, the simulator was able to reproduce stable behavior by measuring the translations of the humeral head of less than 24 mm with respect to the glenoid implant. Simulation with quasi-static forces showed dislocation in extreme ranges of implant orientation. The robotic GH simulator presented here was able to reproduce physiological GH forces and may therefore be used to further evaluate the effects of glenoid implant design and orientation on joint stability.

Prediction of Gait Freezing in Parkinsonian Patients: A Binary Classification Augmented With Time Series Prediction.

This paper presents a novel technique to predict freezing of gait in advanced stage Parkinsonian patients using movement data from wearable sensors. A two-class approach is presented which consists of autoregressive predictive models to project the feature time series, followed by machine learning based classifiers to discriminate freezing from nonfreezing based on the predicted features. To implement and validate our technique a set of time domain and frequency domain features were extracted from the 3D acceleration data, which was then analyzed using information theoretic and feature selection approaches to determine the most discriminative features. Predictive models were trained to predict the features from their past values, then fed into binary classifiers based on support vector machines and probabilistic neural networks which were rigorously cross validated. We compared the results of this approach with a three-class classification approach proposed in previous literature, in which a pre-freezing class was introduced and the problem of prediction of the gait freezing incident was reduced to solving a three-class classification problem. The two-class approach resulted in a sensitivity of 93±4%, specificity of 91±6%, with an expected prediction horizon of 1.72 s. Our subject-specific gait freezing prediction algorithm outperformed existing algorithms, yields consistent results across different subjects and is robust against the choice of classifier, with slight variations in the selected features. In addition, we analyzed the merits and limitations of different families of features to predict gait freezing.

Effect of Visual Information on Dominant and Non-dominant Hands During Bimanual Drawing with a Robotic Platform.

In a stable bimanual trajectory tracing task with interlimb spatial and temporal synchrony, blocking the visual information from one hand may alter the performance of either hand. In this paper, we investigate the effect of visual information on motor behaviour of dominant and non-dominant hands during a bimanual task, with a focus on motor lateralization theory's anticipation for a more pronounced distortion on one hand due to visual information withdrawal. To address this question, four bimanual circle tracing experiments were designed with two rehabilitation robotic arms with real time visual feedback. Two experiments were conducted under the free-visual condition whereas the visual feedback from one hand was blocked for the other two. The in-depth analysis of the metrics extracted from 685 circles, drawn by 6 participants, revealed that non-dominant hand, when visible, generally performs worse than the dominant hand, for instance it exhibits less circularity. In their invisible modes, the performance of the dominant and non-dominant hands displayed inconsistent difference across the participants. Moreover, both hands showed a higher pace when partial visual information was available. Our findings using this robotic framework as a systematic tool on developing new paradigms are discussed.