Modelling and identification of characteristic kinematic features preceding freezing of gait with convolutional neural networks and layer-wise relevance propagation.

BACKGROUND: Although deep neural networks (DNNs) are showing state of the art performance in clinical gait analysis, they are considered to be black-box algorithms. In other words, there is a lack of direct understanding of a DNN's ability to identify relevant features, hindering clinical acceptance. Interpretability methods have been developed to ameliorate this concern by providing a way to explain DNN predictions. METHODS: This paper proposes the use of an interpretability method to explain DNN decisions for classifying the movement that precedes freezing of gait (FOG), one of the most debilitating symptoms of Parkinson's disease (PD). The proposed two-stage pipeline consists of (1) a convolutional neural network (CNN) to model the reduction of movement present before a FOG episode, and (2) layer-wise relevance propagation (LRP) to visualize the underlying features that the CNN perceives as important to model the pathology. The CNN was trained with the sagittal plane kinematics from a motion capture dataset of fourteen PD patients with FOG. The robustness of the model predictions and learned features was further assessed on fourteen PD patients without FOG and fourteen age-matched healthy controls. RESULTS: The CNN proved highly accurate in modelling the movement that precedes FOG, with 86.8% of the strides being correctly identified. However, the CNN model was unable to model the movement for one of the seven patients that froze during the protocol. The LRP interpretability case study shows that (1) the kinematic features perceived as most relevant by the CNN are the reduced peak knee flexion and the fixed ankle dorsiflexion during the swing phase, (2) very little relevance for FOG is observed in the PD patients without FOG and the healthy control subjects, and (3) the poor predictive performance of one subject is attributed to the patient's unique and severely flexed gait signature. CONCLUSIONS: The proposed pipeline can aid clinicians in explaining DNN decisions in clinical gait analysis and aid machine learning practitioners in assessing the generalization of their models by ensuring that the predictions are based on meaningful kinematic features.

Comparative Study on Overground Gait of Stroke Survivors With a Conventional Cane and a Haptic Cane.

The conventional cane (single cane) is widely used to promote gait ability of stroke survivors as it provides postural stability by extending the base of support. However, its use can reduce muscle activity in the user's paretic side and cause upper limb neuropathies due to the intermittent and excessive loading of the upper limb. The provision of low magnitude support and speed regulation may result in collective improvement of gait parameters such as symmetry, balance and muscle activation. In this paper, we developed a robotic Haptic Cane (HC) that is composed of a tilted structure with motorized wheels and sensors to allow continuous haptic contact with the ground while moving at a regulated speed, and carried out gait experiments to compare the HC with an Instrumented conventional Cane (IC). The results show that use of the HC involved more continuous ground support force of a comparatively lesser magnitude than the IC, and resulted in greater improvements in the swing symmetry ratio and significant improvements in the step length symmetry ratio. Percentage of Non-Paretic Activity (%NPA) of paretic muscles (vastus medialis obliquus (VMO), semitendinosus (SMT), tibialis anterior (TBA) and gastrocnemius medialis (GCM)) in swing phase was significantly improved by the use of either device at fast speed. However, the use of HC improved %NPA of paretic VMO and SMT more than the use of IC at both preferred and fast speeds. It also significantly improved %NPA of paretic GCM in stance phase. Furthermore, comfortable speed with the HC was higher than with the IC and exhibited better RMS of anteroposterior (AP) tilt. Thus, the developed device with a simple and intuitive mechanism can provide efficient assistance for overground gait of stroke patients with a high possibility of widespread use.

Effects of Vibrotactile Biofeedback Coding Schemes on Gait Symmetry Training of Individuals With Stroke.

Variations in biofeedback coding schemes for postural control, in recent research, have shown significant differences in performance outcomes due to variations in coding schemes. However, the application of vibrotactile biofeedback coding schemes to gait symmetry training is not well explored. In this paper, we devised various vibrotactile biofeedback modes and identified their efficacy during gait symmetry training of individuals suffering from hemiparesis due to stroke. These modes are composed of variations in vibration type (on-time or intensity), and relation type (proportional or inversely-proportional) with the error in symmetry ratio. Eight individuals with stroke participated in walking trials. From dependent t-tests on the collected data, we found improved achievement of temporal gait symmetry while utilizing all the provided biofeedback modes compared to no biofeedback (P < 0.001). Furthermore, two-way repeated measures ANOVA revealed statistically significant difference in symmetry ratio for main effect of vibration type (P-value = 0.016, partial eta squared = 0.585). The participants performed better with modes of biofeedback with varying vibration on-times. Furthermore, participants showed better performance when the biofeedback varied proportionally with the error. These findings suggest that biofeedback coding schemes may have a significant effect on the performance of gait training.

Haptic-Based Manipulation Scheme of Magnetic Nanoparticles in a Multi-Branch Blood Vessel for Targeted Drug Delivery.

Magnetic drug targeting is a promising technique that can deliver drugs to the diseased region, while keeping the drug away from healthy parts of body. Introducing a human in the control loop of a targeted drug delivery system and using inherent bilateralism of a haptic device at the same time can considerably improve the performance of targeted drug delivery systems. In this paper, we suggest a novel intelligent haptic guidance scheme for steering a number of magnetic nanoparticles (MNPs) using forbidden region virtual fixtures and a haptic rendering scheme with multi particles. Forbidden region virtual fixtures are a general class of guidance modes implemented in software, which help a human-machine collaborative system accomplish a specific task by constraining a movement into limited regions. To examine the effectiveness of our proposed scheme, we implemented a magnetic guided drug delivery system in a virtual environment using a physics-based model of targeted drug delivery including a multi-branch blood vessel and realistic blood dynamics. We performed user studies with different guidance modes: unguided, semi virtual fixture and full virtual fixture modes. We found out that the efficiency of targeting was significantly improved using the forbidden region virtual fixture and the proposed haptic rendering of MNPs. We can expect that using intelligent haptic feedback in real targeted drug delivery systems can improve the targeting efficiency of MNPs in multi-branch vessels.

Evaluating the Effects of Kinesthetic Biofeedback Delivered Using Reaction Wheels on Standing Balance.

Aging, injury, or ailments can contribute to impaired balance control and increase the risk of falling. Provision of light touch augments the sense of balance and can thus reduce the amount of body sway. In this study, a wearable reaction wheel-based system is used to deliver light touch-based balance biofeedback on the subject's back. The system can sense torso tilt and, using reaction wheels, generates light touch. A group of 7 healthy young individuals performed balance tasks under 12 trial combinations based on two conditions each of standing stance and surface types and three of biofeedback device status. Torso tilt data, collected from a waist-mounted smartphone during all the trials, were analyzed to determine the efficacy of the system. Provision of biofeedback by the device significantly reduced RMS of mediolateral (ML) trunk tilt (p < 0.05) and ML trunk acceleration (p < 0.05). Repeated measures ANOVA revealed significant interaction between stance and surface on reduction in RMS of ML trunk tilt, AP trunk tilt, ML trunk acceleration, and AP trunk acceleration. The device shows promise for further applications such as virtual reality interaction and gait rehabilitation.

Evaluating the effects of delivering integrated kinesthetic and tactile cues to individuals with unilateral hemiparetic stroke during overground walking.

BACKGROUND: Integration of kinesthetic and tactile cues for application to post-stroke gait rehabilitation is a novel concept which needs to be explored. The combined provision of haptic cues may result in collective improvement of gait parameters such as symmetry, balance and muscle activation patterns. Our proposed integrated cue system can offer a cost-effective and voluntary gait training experience for rehabilitation of subjects with unilateral hemiparetic stroke. METHODS: Ten post-stroke ambulatory subjects participated in a 10 m walking trial while utilizing the haptic cues (either alone or integrated application), at their preferred and increased gait speeds. In the system a haptic cane device (HCD) provided kinesthetic perception and a vibrotactile feedback device (VFD) provided tactile cue on the paretic leg for gait modification. Balance, gait symmetry and muscle activity were analyzed to identify the benefits of utilizing the proposed system. RESULTS: When using kinesthetic cues, either alone or integrated with a tactile cue, an increase in the percentage of non-paretic peak activity in the paretic muscles was observed at the preferred gait speed (vastus medialis obliquus: p <  0.001, partial eta squared (η2) = 0.954; semitendinosus p <  0.001, partial η2 = 0.793) and increased gait speeds (vastus medialis obliquus: p <  0.001, partial η2 = 0.881; semitendinosus p = 0.028, partial η2 = 0.399). While using HCD and VFD (individual and integrated applications), subjects could walk at their preferred and increased gait speeds without disrupting trunk balance in the mediolateral direction. The temporal stance symmetry ratio was improved when using tactile cues, either alone or integrated with a kinesthetic cue, at their preferred gait speed (p <  0.001, partial η2 = 0.702). CONCLUSIONS: When combining haptic cues, the subjects walked at their preferred gait speed with increased temporal stance symmetry and paretic muscle activity affecting their balance. Similar improvements were observed at higher gait speeds. The efficacy of the proposed system is influenced by gait speed. Improvements were observed at a 20% increased gait speed, whereas, a plateau effect was observed at a 40% increased gait speed. These results imply that integration of haptic cues may benefit post-stroke gait rehabilitation by inducing simultaneous improvements in gait symmetry and muscle activity.

Identifying the effects of using integrated haptic feedback for gait rehabilitation of stroke patients.

This paper evaluates the prospects of using a novel Integrated Haptic Feedback (IHF) system. IHF can provide over-ground gait training regimens for post-stroke ambulatory subjects. IHF system combines the use of a portable cane for kinesthetic feedback and a wearable vibrotactor array for tactile feedback. Continuous somatosensory input is aiforded to the users at the handle of cane; it serves the purpose of balance assurance at higher gait speeds. Besides, restricted use of upper limb for weight-bearing inspires the users to involve the paretic lower limbs more actively. Furthermore, tactile feedback contributes in enhancing the gait symmetry through afferent signal of vibration. Six post-stroke ambulatory individuals participated in walking trials to identity the effects of IHF system. Results indicate that while walking faster patients' body sway was not disturbed. Statistically significant increase was observed in temporal stance symmetry (p-value=0.02) and in paretic muscle (vastus medialis obliquus and semitendinosus) activation during stance phase (p-value<0.01). The IHF system can be a valuable tool to assist physical therapist in gait rehabilitation of post-stroke individuals.

A novel balance training system using multimodal biofeedback.

BACKGROUND: A biofeedback-based balance training system can be used to provide the compromised sensory information to subjects in order to retrain their sensorimotor function. In this study, the design and evaluation of the low-cost, intuitive biofeedback system developed at Gyeongsang National University is extended to provide multimodal biofeedback for balance training by utilization of visual and haptic modalities. METHODS: The system consists of a smartphone attached to the waist of the subject to provide information about tilt of the torso, a personal computer running a purpose built software to process the smartphone data and provide visual biofeedback to the subject by means of a dedicated monitor and a dedicated Phantom Omni(®) device for haptic biofeedback. For experimental verification of the system, eleven healthy young participants performed balance tasks assuming two distinct postures for 30 s each while acquiring torso tilt. The postures used were the one foot stance and the tandem Romberg stance. For both the postures, the subjects stood on a foam platform which provided a certain amount of ground instability. RESULTS: Post-experiment data analysis was performed using MATLAB(®) to analyze reduction in body sway. Analysis parameters based on the projection of trunk tilt information were calculated in order to ascertain the reduction in body sway and improvements in postural control. Two-way analysis of variance (ANOVA) showed no statistically significant interactions between postures and biofeedback. Post-hoc analysis revealed statistically significant reduction in body sway on provision of biofeedback. Subjects exhibited maximum body sway during no biofeedback trial, followed by either haptic or visual biofeedback and in most of the trials the multimodal biofeedback of visual and haptic together resulted in minimization of body sway, thus indicating that the multimodal biofeedback system worked well to provide significant (p < 0.05) assistance in postural control. CONCLUSIONS: A multimodal biofeedback system can offer more customized training methods and hence provide therapists with a comprehensive solution for a diverse array of patients. It is necessary to identify the long-term effects of this novel biofeedback system. In the future, the balance training schemes for individuals with upright balance issues will be studied.

A Portable Gait Asymmetry Rehabilitation System for Individuals with Stroke Using a Vibrotactile Feedback.

Gait asymmetry caused by hemiparesis results in reduced gait efficiency and reduced activity levels. In this paper, a portable rehabilitation device is proposed that can serve as a tool in diagnosing gait abnormalities in individuals with stroke and has the capability of providing vibration feedback to help compensate for the asymmetric gait. Force-sensitive resistor (FSR) based insoles are used to detect ground contact and estimate stance time. A controller (Arduino) provides different vibration feedback based on the gait phase measurement. It also allows wireless interaction with a personal computer (PC) workstation using the XBee transceiver module, featuring data logging capabilities for subsequent analysis. Walking trials conducted with healthy young subjects allowed us to observe that the system can influence abnormality in the gait. The results of trials showed that a vibration cue based on temporal information was more effective than intensity information. With clinical experiments conducted for individuals with stroke, significant improvement in gait symmetry was observed with minimal disturbance caused to the balance and gait speed as an effect of the biofeedback. Future studies of the long-term rehabilitation effects of the proposed system and further improvements to the system will result in an inexpensive, easy-to-use, and effective rehabilitation device.

Effects of kinesthetic haptic feedback on standing stability of young healthy subjects and stroke patients.

BACKGROUND: Haptic control is a useful therapeutic option in rehabilitation featuring virtual reality interaction. As with visual and vibrotactile biofeedback, kinesthetic haptic feedback may assist in postural control, and can achieve balance control. Kinesthetic haptic feedback in terms of body sway can be delivered via a commercially available haptic device and can enhance the balance stability of both young healthy subjects and stroke patients. METHOD: Our system features a waist-attached smartphone, software running on a computer (PC), and a dedicated Phantom Omni® device. Young healthy participants performed balance tasks after assumption of each of four distinct postures for 30 s (one foot on the ground; the Tandem Romberg stance; one foot on foam; and the Tandem Romberg stance on foam) with eyes closed. Patient eyes were not closed and assumption of the Romberg stance (only) was tested during a balance task 25 s in duration. An Android application running continuously on the smartphone sent mediolateral (ML) and anteroposterior (AP) tilt angles to a PC, which generated kinesthetic haptic feedback via Phantom Omni®. A total of 16 subjects, 8 of whom were young healthy and 8 of whom had suffered stroke, participated in the study. RESULTS: Post-experiment data analysis was performed using MATLAB®. Mean Velocity Displacement (MVD), Planar Deviation (PD), Mediolateral Trajectory (MLT) and Anteroposterior Trajectory (APT) parameters were analyzed to measure reduction in body sway. Our kinesthetic haptic feedback system was effective to reduce postural sway in young healthy subjects regardless of posture and the condition of the substrate (the ground) and to improve MVD and PD in stroke patients who assumed the Romberg stance. Analysis of Variance (ANOVA) revealed that kinesthetic haptic feedback significantly reduced body sway in both categories of subjects. CONCLUSION: Kinesthetic haptic feedback can be implemented using a commercial haptic device and a smartphone. Intuitive balance cues were created using the handle of a haptic device, rendering the approach very simple yet efficient in practice. This novel form of biofeedback will be a useful rehabilitation tool improving the balance of stroke patients.

Low-cost design and fabrication of an anthropomorphic robotic hand.

Human hand signifies a magnificent and challenging example for scientists and engineers trying to replicate its complex structure and functionality. This paper proposes a bio-mechatronic approach for the design of an anthropomorphic artificial hand capable of performing basic human hand motions with fundamental gripping functionality. The dexterity of the artificial hand is exhibited by imitating the natural motion of the human fingers. Imitation is produced according to the data acquired from the flex sensors attached to the human fingers. In order to have proper gripping, closed-loop control is implemented using the tactile sensors. Feedback for the closed-loop control is provided by force sensing resistors (FSRs), attached on the fingertips of the robotic hand. These sensors also enable handling of fragile objects. The mathematical model is derived using forward kinematics and also simulated on MATLAB to ascertain the position of robotic fingers in 3D space.