Implementation of Deep Deterministic Policy Gradients for Controlling Dynamic Bipedal Walking.

A control system for bipedal walking in the sagittal plane was developed in simulation. The biped model was built based on anthropometric data for a 1.8 m tall male of average build. At the core of the controller is a deep deterministic policy gradient (DDPG) neural network that was trained in GAZEBO, a physics simulator, to predict the ideal foot placement to maintain stable walking despite external disturbances. The complexity of the DDPG network was decreased through carefully selected state variables and a distributed control system. Additional controllers for the hip joints during their stance phases and the ankle joint during toe-off phase help to stabilize the biped during walking. The simulated biped can walk at a steady pace of approximately 1 m/s, and during locomotion it can maintain stability with a 30 kg·m/s impulse applied forward on the torso or a 40 kg·m/s impulse applied rearward. It also maintains stable walking with a 10 kg backpack or a 25 kg front pack. The controller was trained on a 1.8 m tall model, but also stabilizes models 1.4-2.3 m tall with no changes.

A muscle-driven approach to restore stepping with an exoskeleton for individuals with paraplegia.

BACKGROUND: Functional neuromuscular stimulation, lower limb orthosis, powered lower limb exoskeleton, and hybrid neuroprosthesis (HNP) technologies can restore stepping in individuals with paraplegia due to spinal cord injury (SCI). However, a self-contained muscle-driven controllable exoskeleton approach based on an implanted neural stimulator to restore walking has not been previously demonstrated, which could potentially result in system use outside the laboratory and viable for long term use or clinical testing. In this work, we designed and evaluated an untethered muscle-driven controllable exoskeleton to restore stepping in three individuals with paralysis from SCI. METHODS: The self-contained HNP combined neural stimulation to activate the paralyzed muscles and generate joint torques for limb movements with a controllable lower limb exoskeleton to stabilize and support the user. An onboard controller processed exoskeleton sensor signals, determined appropriate exoskeletal constraints and stimulation commands for a finite state machine (FSM), and transmitted data over Bluetooth to an off-board computer for real-time monitoring and data recording. The FSM coordinated stimulation and exoskeletal constraints to enable functions, selected with a wireless finger switch user interface, for standing up, standing, stepping, or sitting down. In the stepping function, the FSM used a sensor-based gait event detector to determine transitions between gait phases of double stance, early swing, late swing, and weight acceptance. RESULTS: The HNP restored stepping in three individuals with motor complete paralysis due to SCI. The controller appropriately coordinated stimulation and exoskeletal constraints using the sensor-based FSM for subjects with different stimulation systems. The average range of motion at hip and knee joints during walking were 8.5°-20.8° and 14.0°-43.6°, respectively. Walking speeds varied from 0.03 to 0.06 m/s, and cadences from 10 to 20 steps/min. CONCLUSIONS: A self-contained muscle-driven exoskeleton was a feasible intervention to restore stepping in individuals with paraplegia due to SCI. The untethered hybrid system was capable of adjusting to different individuals' needs to appropriately coordinate exoskeletal constraints with muscle activation using a sensor-driven FSM for stepping. Further improvements for out-of-the-laboratory use should include implantation of plantar flexor muscles to improve walking speed and power assist as needed at the hips and knees to maintain walking as muscles fatigue.

Improving stand-to-sit maneuver for individuals with spinal cord injury.

BACKGROUND: Users of neuroprostheses employing electrical stimulation (ES) generally complete the stand-to-sit (STS) maneuver with high knee angular velocities, increased upper limb support forces, and high peak impact forces at initial contact with the chair. Controlling the knee during STS descent is challenging in individuals with spinal cord injury (SCI) due to the decreasing joint moment available with increased knee angle in response to ES. METHODS: The goal of this study was to investigate the effects of incorporating either (1) a coupling mechanism that coordinates hip and knee flexion or (2) a mechanism that damps knee motion to keep the knee angular velocity constant during the STS transition. The coupling and damping were achieved by hydraulic orthotic mechanisms. Two subjects with SCI were enrolled and each served as their own controls when characterizing the performance of each mechanism during STS as compared to stimulation alone. Outcome measures such as hip-knee angle, knee angular velocity, upper limb support force, and impact force were analyzed to determine the effectiveness of the two mechanisms in providing controlled STS. RESULTS: The coordination between the hip and knee joints improved with each orthotic mechanism. The damping and hip-knee coupling mechanisms caused the hip and knee joint ratios of 1:1.1 and 1:0.99, respectively, which approached the 1:1 coordination ratio observed in nondisabled individuals during STS maneuver. The knee damping mechanism provided lower (p < 0.001) and a more constant knee angular velocity than the hip-knee coupling mechanism over the knee range of motion. Both the coupling and damping mechanisms were similarly effective at reducing upper limb support forces by 70 % (p < 0.001) and impact force by half (p ≤ 0.001) as compared to sitting down with stimulation alone. CONCLUSIONS: Orthoses imposing simple kinematic constraints, such as 1:1 hip-knee coupling or knee damping, can normalize upper limb support forces, peak knee angular velocity, and peak impact force during the STS maneuvers.

A stimulation-driven exoskeleton for walking after paraplegia.

An untethered version of a stimulation-driven exoskeleton was evaluated for its ability to restore walking after paralysis from spinal cord injury. The hybrid neuroprosthesis (HNP) combined a passive variable-constraint exoskeleton for stability and support with functional neuromuscular stimulation (FNS) to contract the paralyzed muscles to drive limb movement. This self-contained HNP was operated by an onboard controller that sampled sensor signals, generated appropriate commands to both the exoskeletal constraints and integrated stimulator, and transmitted data wirelessly via Bluetooth to an off-board computer for real-time monitoring and recording for offline analysis. The subject selected the desired function (i.e. standing up, stepping, or sitting down) by means of a wireless finger switch that communicated with the onboard controller. Within the stepping function, a gait event detector supervisory controller transitioned between the different phases of gait such as double stance, swing, and weight acceptance based on signals from sensors incorporated into the exoskeleton. The different states of the control system governed the locking and unlocking of the exoskeletal hip and knee joints as well as the stimulation patterns activating hip and knee flexor or extensor muscles at the appropriate times and intensities to enable stepping. This study was one of our first successful implementations of the self-contained "muscle-first" HNP and successfully restored gait to an individual with motor complete mid-thoracic paraplegia.

Feasibility of a Hydraulic Power Assist System for Use in Hybrid Neuroprostheses.

Feasibility of using pressurized hydraulic fluid as a source of on-demand assistive power for hybrid neuroprosthesis combining exoskeleton with functional neuromuscular stimulation was explored. Hydraulic systems were selected as an alternative to electric motors for their high torque/mass ratio and ability to be located proximally on the exoskeleton and distribute power distally to assist in moving the joints. The power assist system (PAS) was designed and constructed using off-the-shelf components to test the feasibility of using high pressure fluid from an accumulator to provide assistive torque to an exoskeletal hip joint. The PAS was able to provide 21 Nm of assistive torque at an input pressure of 3171 kPa with a response time of 93 ms resulting in 32° of hip flexion in an able-bodied test. The torque output was independent of initial position of the joint and was linearly related to pressure. Thus, accumulator pressure can be specified to provide assistive torque as needed in exoskeletal devices for walking or stair climbing beyond those possible either volitionally or with electrical stimulation alone.