Multi Degree of Freedom Hybrid FES and Robotic Control of the Upper Limb.

Individuals who have suffered a spinal cord injury often require assistance to complete daily activities, and for individuals with tetraplegia, recovery of upper-limb function is among their top priorities. Hybrid functional electrical stimulation (FES) and exoskeleton systems have emerged as a potential solution to provide upper limb movement assistance. These systems leverage the user's own muscles via FES and provide additional movement support via an assistive exoskeleton. To date, these systems have focused on single joint movements, limiting their utility for the complex movements necessary for independence. In this paper, we extend our prior work on model predictive control (MPC) of hybrid FES-exo systems and present a multi degree of freedom (DOF) hybrid controller that uses the controller's cost function to achieve desired behavior. In studies with neurologically intact individuals, the hybrid controller is compared to an exoskeleton acting alone for movement assistance scenarios incorporating multiple degrees-of-freedom of the limb to explore the potential for exoskeleton power consumption reduction and impacts on tracking accuracy. Additionally, each scenario is explored in simulation using the models required to generate the MPC formulation. The two DOF hybrid controller implementation saw reductions in power consumption and satisfactory trajectory tracking in both the physical and simulated systems. In the four DOF implementation, the experimental results showed minor improvements for some joints of the upper limb. In simulation, we observed comparable performance as in the two DOF implementation.

Design and Evaluation of a 3-DoF Haptic Device for Directional Shear Cues on the Forearm.

Wearable haptic devices on the forearm can relay information from virtual agents, robots, and other humans while leaving the hands free. We introduce and test a new wearable haptic device that uses soft actuators to provide normal and shear force to the skin of the forearm. A rigid housing and gear motor are used to control the direction of the shear force. A 6-axis force/torque sensor, distance sensor, and pressure sensors are integrated to quantify how the soft tactor interacts with the skin. When worn by participants, the device delivered consistent shear forces of up to 0.64 N and normal forces of up to 0.56 N over distances as large as 14.3 mm. To understand cue saliency, we conducted a user study asking participants to identify linear shear directional cues in a 4-direction task and an 8-direction task with different cue speeds, travel distances, and contact patterns. Participants identified cues with longer travel distances best, with an 85.1% accuracy in the 4-direction task, and a 43.5% accuracy in the 8-direction task. Participants had a directional bias, with a preferential response in the axis towards and away from the wrist bone.

Comparing the Perceived Intensity of Vibrotacitle Cues Scaled Based on Inherent Dynamic Range.

Wearable devices increasingly incorporate vibrotactile feedback notifications to users, which are limited by the frequency-dependent response characteristics of the low-cost actuators that they employ. To increase the range and type of information that can be conveyed to users via vibration feedback, it is crucial to understand user perception of vibration cue intensity across the narrow range of frequencies that these actuators operate. In this paper, we quantify user perception of vibration cues conveyed via a linear resonant actuator embedded in a bracelet interface using two psychophysical experiments. We also experimentally determine the frequency response characteristics of the wearable device. We then compare user perceived intensity of vibration cues delivered by the bracelet when the cues undergo frequency-specific amplitude modulation based on user perception compared to modulation based on the experimental or manufacturer-reported characterization of the actuator dynamic response. For applications in which designers rely on user perception of cue amplitudes across frequencies to be equivalent, it is recommended that a perceptual calibration experiment be conducted to determine appropriate modulation factors. For applications in which only relative perceived amplitudes are important, basing amplitude modulation factors on manufacturer data or experimentally determined dynamic response characteristics of the wearable device should be sufficient.

Representational Similarity Analysis for Tracking Neural Correlates of Haptic Learning on a Multimodal Device.

A goal of wearable haptic devices has been to enable haptic communication, where individuals learn to map information typically processed visually or aurally to haptic cues via a process of cross-modal associative learning. Neural correlates have been used to evaluate haptic perception and may provide a more objective approach to assess association performance than more commonly used behavioral measures of performance. In this article, we examine Representational Similarity Analysis (RSA) of electroencephalography (EEG) as a framework to evaluate how the neural representation of multifeatured haptic cues changes with association training. We focus on the first phase of cross-modal associative learning, perception of multimodal cues. A participant learned to map phonemes to multimodal haptic cues, and EEG data were acquired before and after training to create neural representational spaces that were compared to theoretical models. Our perceptual model showed better correlations to the neural representational space before training, while the feature-based model showed better correlations with the post-training data. These results suggest that training may lead to a sharpening of the sensory response to haptic cues. Our results show promise that an EEG-RSA approach can capture a shift in the representational space of cues, as a means to track haptic learning.

Defining Allowable Stimulus Ranges for Position and Force Controlled Cutaneous Cues.

Haptic cues delivered via wearable devices have great potential to enhance a user's experience by transmitting task information and touch sensations in domains such as virtual reality, teleoperation, and prosthetics. Much is still unknown on how haptic perception, and consequently optimal haptic cue design, varies between individuals. In this work we present three contributions. First, we propose a new metric, the Allowable Stimulus Range (ASR), as a way to capture subject-specific magnitudes for a given cue, using the method of adjustments and the staircase method. Second, we present a modular, grounded, 2-DOF, haptic testbed designed to conduct psychophysical experiments in multiple control schemes and with rapidly-interchangeable haptic interfaces. Third, we demonstrate an application of the testbed and our ASR metric, together with just noticeable differences (JND) measurements, to compare perception of haptic cues delivered via position or force control schemes. Our findings show that users demonstrate higher perceptual resolution in the position-control case, though survey results suggest that force-controlled haptic cues are more comfortable. The results of this work outline a framework to define perceptible and comfortable cue magnitudes for an individual, providing the groundwork to understand haptic variability, and compare the effectiveness of different types of haptic cues.

Hybrid FES-exoskeleton control: Using MPC to distribute actuation for elbow and wrist movements.

Introduction: Individuals who have suffered a cervical spinal cord injury prioritize the recovery of upper limb function for completing activities of daily living. Hybrid FES-exoskeleton systems have the potential to assist this population by providing a portable, powered, and wearable device; however, realization of this combination of technologies has been challenging. In particular, it has been difficult to show generalizability across motions, and to define optimal distribution of actuation, given the complex nature of the combined dynamic system. Methods: In this paper, we present a hybrid controller using a model predictive control (MPC) formulation that combines the actuation of both an exoskeleton and an FES system. The MPC cost function is designed to distribute actuation on a single degree of freedom to favor FES control effort, reducing exoskeleton power consumption, while ensuring smooth movements along different trajectories. Our controller was tested with nine able-bodied participants using FES surface stimulation paired with an upper limb powered exoskeleton. The hybrid controller was compared to an exoskeleton alone controller, and we measured trajectory error and torque while moving the participant through two elbow flexion/extension trajectories, and separately through two wrist flexion/extension trajectories. Results: The MPC-based hybrid controller showed a reduction in sum of squared torques by an average of 48.7 and 57.9% on the elbow flexion/extension and wrist flexion/extension joints respectively, with only small differences in tracking accuracy compared to the exoskeleton alone. Discussion: To realize practical implementation of hybrid FES-exoskeleton systems, the control strategy requires translation to multi-DOF movements, achieving more consistent improvement across participants, and balancing control to more fully leverage the muscles' capabilities.

Mechanofluidic Instability-Driven Wearable Textile Vibrotactor.

Vibration is a widely used mode of haptic communication, as vibrotactile cues provide salient haptic notifications to users and are easily integrated into wearable or handheld devices. Fluidic textile-based devices offer an appealing platform for the incorporation of vibrotactile haptic feedback, as they can be integrated into clothing and other conforming and compliant wearables. Fluidically driven vibrotactile feedback has primarily relied on valves to regulate actuating frequencies in wearable devices. The mechanical bandwidth of such valves limits the range of frequencies that can be achieved, particularly in attempting to reach the higher frequencies realized with electromechanical vibration actuators ( 100 Hz). In this paper, we introduce a soft vibrotactile wearable device constructed entirely of textiles and capable of rendering vibration frequencies between 183 and 233 Hz with amplitudes ranging from 23 to 114 g. We describe our methods of design and fabrication and the mechanism of vibration, which is realized by controlling inlet pressure and harnessing a mechanofluidic instability. Our design allows for controllable vibrotactile feedback that is comparable in frequency and greater in amplitude relative to state-of-the-art electromechanical actuators while offering the compliance and conformity of fully soft wearable devices.

Combinatorial Effects of Transcutaneous Spinal Stimulation and Task-Specific Training to Enhance Hand Motor Output after Paralysis.

Background: Despite the positive results in upper limb (UL) motor recovery after using electrical neuromodulation in individuals after cervical spinal cord injury (SCI) or stroke, there has been limited exploration of potential benefits of combining task-specific hand grip training with transcutaneous electrical spinal stimulation (TSS) for individuals with UL paralysis. Objectives: This study investigates the combinatorial effects of task-specific hand grip training and noninvasive TSS to enhance hand motor output after paralysis. Methods: Four participants with cervical SCI classified as AIS A and B and two participants with cerebral stroke were recruited in this study. The effects of cervical TSS without grip training and during training with sham stimulation were contrasted with hand grip training with TSS. TSS was applied at midline over cervical spinal cord. During hand grip training, 5 to 10 seconds of voluntary contraction were repeated at a submaximum strength for approximately 10 minutes, three days per week for 4 weeks. Signals from hand grip dynamometer along with the electromyography (EMG) activity from UL muscles were recorded and displayed as visual feedback. Results: Our case study series demonstrated that combined task-specific hand grip training and cervical TSS targeting the motor pools of distal muscles in the UL resulted in significant improvements in maximum hand grip strength. However, TSS alone or hand grip training alone showed limited effectiveness in improving grip strength. Conclusion: Task-specific hand grip training combined with TSS can result in restoration of hand motor function in paralyzed upper limbs in individuals with cervical SCI and stroke.

Measuring Torque Production with a Robotic Exoskeleton during Cervical Transcutaneous Spinal Stimulation.

Spinal cord injury (SCI) affects a large number of individuals in the United States. Unfortunately, traditional neurorehabilitation therapy leaves out clinical populations with limited motor function, such as severe stroke or spinal cord injury, as they are incapable of engaging in movement therapy. To increase the numbers of individuals who may be able to participate in robotic therapy, our long-term goal is to combine two validated interventions, transcutaneous spinal stimulation (TSS) and robotics, to elicit upper limb movements during rehabilitation following SCI. To achieve this goal, it is necessary to quantify the contributions of each intervention to realizing arm movements. Electromyography is typically used to assess the response to TSS, but the robot itself offers an additional source of data since the available sensors on the robot can be used to directly assess resultant actions of the upper limb after stimulation. We explore this approach in this paper. We showed that the effects of cutaneous TSS can be observed by measuring the holding torque required by the exoskeleton to keep a user's arm in a neutral position. Further, we can identify differences in resultant action based on the location of the stimulation electrodes with respect to the dorsal roots of the spinal cord. In the future, we can use measurements from the robot to guide the action of the robot and TSS intervention.

Shared Control of Elbow Movements with Functional Electrical Stimulation and Exoskeleton Assistance.

Individuals who suffer from paralysis as a result of a spinal cord injury list restoration of arm and hand function as a top priority. FES helps restore movement using the user's own muscles, but does not produce accurate and repeatable movements necessary for many functional tasks. Robots can assist users in achieving accurate and repeatable movements, but often require bulky hardware to generate the necessary torques. We propose sharing torque requirements between a robot and FES to reduce robot torque output compared to a robot acting alone, yet maintain high accuracy. Cooperative PD and model predictive control algorithms were designed to share the control between these two torque sources. Corresponding PD and MPC algorithms that do not use FES were also designed. The control algorithms were tested with 10 able-bodied subjects. Torque and position tracking accuracy were compared when the system was commanded to follow a functional elbow flexion/extension trajectory. The robot torque required to achieve these movements was reduced for the shared control cases compared to the algorithms acting without FES. We observed a reduction in position accuracy with the MPC shared controller compared to the PD shared controller, while the MPC shared controller resulted in greater reductions in torque requirements. Both of these shared algorithms showed improvements over existing options, and can be used on any given trajectory, allowing for better transferability to functional tasks.

Evaluation of Robotic-Assisted Carotid Artery Stenting in a Virtual Model Using Motion-Based Performance Metrics.

PURPOSE: Robotic-assisted carotid artery stenting (CAS) cases have been demonstrated with promising results. However, no quantitative measurements have been made to compare manual with robotic-assisted CAS. This study aims to quantify surgical performance using tool tip kinematic data and metrics of precision during CAS with manual and robotic control in an ex vivo model. MATERIALS AND METHODS: Transfemoral CAS cases were performed in a high-fidelity endovascular simulator. Participants completed cases with manual and robotic techniques in 2 different carotid anatomies in random order. C-arm angulations, table position, and endovascular devices were standardized. Endovascular tool tip kinematic data were extracted. We calculated the spectral arc length (SPARC), average velocity, and idle time during navigation in the common carotid artery and lesion crossing. Procedural time, fluoroscopy time, movements of the deployed filter wire, precision of stent, and balloon positioning were recorded. Data were analyzed and compared between the 2 modalities. RESULTS: Ten participants performed 40 CAS cases with a procedural success of 100% and 0% residual stenosis. The median procedural time was significantly higher during the robotic-assisted cases (seconds, median [interquartile range, IQR]: 128 [49.5] and 161.5 [62.5], p=0.02). Fluoroscopy time differed significantly between manual and robotic-assisted procedures (seconds, median [IQR]: 81.5 [32] and 98.5 [39.5], p=0.1). Movement of the deployed filter wire did not show significant difference between manual and robotic interventions (mm, median [IQR]: 13 [10.5] and 12.5 [11], p=0.5). The postdilation balloon exceeded the margin of the stent with a median of 2 [1] mm in both groups. Navigation with robotic assistance showed significantly lower SPARC values (-5.78±3.14 and -8.63±3.98, p=0.04) and higher idle time values (8.92±8.71 and 3.47±3.9, p=0.02) than those performed manually. CONCLUSIONS: Robotic-assisted and manual CAS cases are comparable in the precision of stent and balloon positioning. Navigation in the carotid artery is associated with smoother motion and higher idle time values. These findings highlight the accuracy and the motion stabilizing capability of the endovascular robotic system. CLINICAL IMPACT: Robotic assistance in the treatment of peripheral vascular disease is an emerging field and may be a tool for radiation protection and the geographic distribution of endovascular interventions in the future. This preclinical study compares the characteristics of manual and robotic-assisted carotid stenting (CAS). Our results highlight, that robotic-assisted CAS is associated with precise navigation and device positioning, and smoother navigation compared to manual CAS.

Haptic Feedback Based on Movement Smoothness Improves Performance in a Perceptual-Motor Task.

In many training scenarios, and in surgery in particular, feedback is provided to the trainee after the task has been performed, and the assessment is often qualitative in nature. In this paper, we demonstrate the effect of real-time objective performance feedback conveyed through a vibrotactile cue. Subjects performed a mirror-tracing task that requires coordination and dexterity similar in nature to that required in endovascular surgery. Movement smoothness, a characteristic associated with skilled and coordinated movement, was measured by spectral arc length, a frequency-domain measure of smoothness. The smoothness-based performance metric was encoded as a vibrotactile cue displayed on the user's arm. Performance on the mirror tracing task with smoothness-based feedback was compared to position-based feedback (where the subject was alerted when they moved outside the path boundary) and to a no vibrotactile feedback control condition. Subjects receiving smoothness-based feedback altered their task completion strategies, resulting in faster task completion times, but their accuracy was slightly worse overall than the other two groups. In procedures such as endovascular surgery, the reduction of procedure time that could be achieved with smoothness-based feedback training may be advantageous, despite the fact that accuracy was inferior to that observed with no feedback or position-based feedback.

Effect of Tactile Masking on Multi-Sensory Haptic Perception.

Multi-sensory wearable haptic devices are able to encode a variety of information using multiple haptic cues. However, simultaneous cues can be misperceived due to tactile masking effects. In this paper, we investigate the effect of masking on the perception of skin stretch and squeeze. We performed three experiments measuring the just-noticeable difference (JND) and the absolute threshold of skin stretch and squeeze alone and in the presence of simultaneous haptic cues. Additionally, we investigate the relative perceptual amplitudes of these haptic cues. Results indicate that the JND for a skin stretch cue increases with a masking squeeze cue, while the JND for a squeeze cue does not change with a masking stretch cue. Also, masking has a significant effect on the absolute threshold of both skin stretch and squeeze. These results suggest that the effect of masking diminishes as haptic cues become larger in amplitude. The results from the subjective equality experiment suggest a potential nonlinear relationship between perceptual magnitudes. Further testing should be carried out to investigate this relationship. Future multi-sensory devices can use these perceptual experiment findings to ensure the delivery of salient cues to users.

Comparing Manual and Robotic-Assisted Carotid Artery Stenting Using Motion-Based Performance Metrics.

Carotid artery stenting (CAS) is a minimally invasive endovascular procedure used to treat carotid artery disease and is an alternative treatment option for carotid artery stenosis. Robotic assistance is becoming increasingly widespread in these procedures and can provide potential benefits over manual intervention, including decreasing peri- and post-operative risks associated with CAS. However, the benefits of robotic assistance in CAS procedures have not been quantitatively verified at the level of surgical tool motions. In this work, we compare manual and robot-assisted navigation in CAS procedures using performance metrics that reliably indicate surgical navigation proficiency. After extracting guidewire tip motion profiles from recorded procedure videos, we computed spectral arc length (SPARC), a frequency-domain metric of movement smoothness, average guidewire velocity, and amount of idle tool motion (idle time) for a set of CAS procedures performed on a commercial endovascular surgical simulator. We analyzed the metrics for two procedural steps that influence post-operative outcomes. Our results indicate that during advancement of the sheath to the distal common carotid artery, there are significant differences in SPARC (F(1, 22.3) = 6.12, p = .021) and idle time (F(1, 22.6) = 6.26, p = .02) between manual and robot-assisted navigation, as well as a general trend of lower SPARC, lower average velocity, and higher idle time values associated with robot-assisted navigation for both procedural steps. Our findings indicate that significant differences exist between manual and robot-assisted CAS procedures. These are quantitatively detectable at the granular-level of physical tool motion, improving the ability to evaluate robotic assistance as it grows in clinical use.

A decade retrospective of medical robotics research from 2010 to 2020.

Robotics is a forward-looking discipline. Attention is focused on identifying the next grand challenges. In an applied field such as medical robotics, however, it is important to plan the future based on a clear understanding of what the research community has recently accomplished and where this work stands with respect to clinical needs and commercialization. This Review article identifies and analyzes the eight key research themes in medical robotics over the past decade. These thematic areas were identified using search criteria that identified the most highly cited papers of the decade. Our goal for this Review article is to provide an accessible way for readers to quickly appreciate some of the most exciting accomplishments in medical robotics over the past decade; for this reason, we have focused only on a small number of seminal papers in each thematic area. We hope that this article serves to foster an entrepreneurial spirit in researchers to reduce the widening gap between research and translation.

Effect of Robotic Exoskeleton Motion Constraints on Upper Limb Muscle Synergies: A Case Study.

Evidence exists that changes in composition, timing, and number of muscle synergies can be correlated to functional changes resulting from neurological injury. These changes can also serve as an indicator of level of motor impairment. As such, synergy analysis can be used as an assessment tool for robotic rehabilitation. However, it is unclear whether using a rehabilitation robot to isolate limb movements during training affects the subject's muscle synergies, which would affect synergy-based assessments. In this case study, electromyographic (EMG) data were collected to analyze muscle synergies generated during single degree-of-freedom (DoF) elbow and wrist movements performed by a single healthy subject in a four DoF robotic exoskeleton. For each trial, the subject was instructed to move a single DoF from a neutral position out to a target and back while the remaining DoFs were held in a neutral position by either the robot (constrained) or the subject (unconstrained). Four factorization methods were used to calculate muscle synergies for both types of trials: concatenation, averaging, single trials, and bootstrapping. The number of synergies was chosen to achieve 90% global variability accounted for. Our preliminary results indicate that muscle synergy composition and timing were highly similar for constrained and unconstrained trials, though some differences between the four factorization methods existed. These differences could be explained by higher trial-to-trial EMG variability for the unconstrained trials. These results suggest that using a robotic exoskeleton to constrain limb movements during robotic training may not alter a subject's muscle synergies, at least for healthy subjects.

Single limb cable driven wearable robotic device for upper extremity movement support after traumatic brain injury.

INTRODUCTION: Recently, soft exosuits have been proposed for upper limb movement assistance, most supporting single joint movements. We describe the design of a portable wearable robotic device (WRD), "Armstrong," able to support three degrees-of-freedom of arm movements, and report on its feasibility for movement support of individuals with hemiparesis after traumatic brain injury (TBI). METHODS: We introduce Armstrong and report on a pilot evaluation with two male individuals post-TBI (T1 and T2) and two healthy individuals. Testing involved elbow flexion/extension with and without robotic-assisted shoulder stabilization; shoulder abduction with and without robotic-assisted elbow stabilization; and assisted shoulder abduction and flexion. Outcome measures included range of motion and root mean square trajectory and velocity errors. RESULTS: TBI subjects performed active, passive, hybrid and active assistive movements with Armstrong. Subjects showed improvements in movement trajectory and velocity. T1 benefited from hybrid, active, and assistive modes due to upper extremity weakness and muscle tone. T2 benefited from hybrid and assistive modes due to impaired coordination. Healthy subjects performed isolated movements of shoulder and elbow with minimal trajectory and velocity errors. CONCLUSIONS: This study demonstrates the safety and feasibility of Armstrong for upper extremity movement assistance for individuals with TBI, with therapist supervision.

The SE-AssessWrist for robot-aided assessment of wrist stiffness and range of motion: Development and experimental validation.

INTRODUCTION: Physical human-robot interaction offers a compelling platform for assessing recovery from neurological injury; however, robots currently used for assessment have typically been designed for the requirements of rehabilitation, not assessment. In this work, we present the design, control, and experimental validation of the SE-AssessWrist, which extends the capabilities of prior robotic devices to include complete wrist range of motion assessment in addition to stiffness evaluation. METHODS: The SE-AssessWrist uses a Bowden cable-based transmission in conjunction with series elastic actuation to increase device range of motion while not sacrificing torque output. Experimental validation of robot-aided wrist range of motion and stiffness assessment was carried out with five able-bodied individuals. RESULTS: The SE-AssessWrist achieves the desired maximum wrist range of motion, while having sufficient position and zero force control performance for wrist biomechanical assessment. Measurements of two-degree-of-freedom wrist range of motion and stiffness envelopes revealed that the axis of greatest range of motion and least stiffness were oblique to the conventional anatomical axes, and approximately parallel to each other. CONCLUSIONS: Such an assessment could be beneficial in the clinic, where standard clinical measures of recovery after neurological injury are subjective, labor intensive, and graded on an ordinal scale.

A Multisensory Approach to Present Phonemes as Language Through a Wearable Haptic Device.

Communication is an important part of our daily interactions; however, communication can be hindered, either through visual or auditory impairment, or because usual communication channels are overloaded. When standard communication channels are not available, our sense of touch offers an alternative sensory modality for transmitting messages. Multi-sensory haptic cues that combine multiple types of haptic sensations have shown promise for applications, such as haptic communication, that require large discrete cue sets while maintaining a small, wearable form factor. This article presents language transmission using a multi-sensory haptic device that occupies a small footprint on the upper arm. In our approach, phonemes are encoded as multisensory haptic cues consisting of vibration, radial squeeze, and lateral skin stretch components. Participants learned to identify haptically transmitted phonemes and words after training across a four day training period. A subset of our participants continued their training to extend word recognition free response. Participants were able to identify words after four days using multiple choice with an accuracy of 89% and after eight days using free response with an accuracy of 70%. These results show promise for the use of multisensory haptics for haptic communication, demonstrating high word recognition performance with a small, wearable device.

Importance of Wrist Movement Direction in Performing Activities of Daily Living Efficiently.

The wrist is an essential component in performing the activities of daily living (ADLs) associated with a high quality of life. After a neurological disorder, motor function of the hand and wrist can be affected, reducing quality of life. Many experiments have illustrated that more wrist flexion/extension is required than radial/ulnar deviation when performing ADLs; however, how this result translates to efficiency in performing ADLs has not been investigated. Motivated by clinical assessment during neurorehabilitation, in this paper we investigate with able-bodied participants how performing tasks representative of the Jebsen-Taylor Hand Function Test are impacted when a splint constrains the user to a single rotational degree of freedom of the wrist. Twenty participants enrolled in the study, performing five tasks under five conditions, including constraint to pure flexion/extension and radial/ulnar deviation. The importance of wrist movement direction in performing ADLs efficiently found in this study could shape clinical wrist rehabilitation paradigms and wrist rehabilitation robot designs.