Accelerated Gradient Approach For Deep Neural Network-Based Adaptive Control of Unknown Nonlinear Systems.

Recent connections in the adaptive control literature to continuous-time analogs of Nesterov's accelerated gradient method have led to the development of new real-time adaptation laws based on accelerated gradient methods. However, previous results assume that the system's uncertainties are linear-in-the-parameters (LIP). To compensate for non-LIP uncertainties, our preliminary results developed a neural network (NN)-based accelerated gradient adaptive controller to achieve trajectory tracking for nonlinear systems; however, the development and analysis only considered single-hidden-layer NNs. In this article, a generalized deep NN (DNN) architecture with an arbitrary number of hidden layers is considered, and a new DNN-based accelerated gradient adaptation scheme is developed to generate estimates of all the DNN weights in real-time. A nonsmooth Lyapunov-based analysis is used to guarantee the developed accelerated gradient-based DNN adaptation design achieves global asymptotic tracking error convergence for general nonlinear control affine systems subject to unknown (non-LIP) drift dynamics and exogenous disturbances. A comprehensive set of simulation studies are conducted on a two-state nonlinear system, a robotic manipulator, and a complex 20-D nonlinear system to demonstrate the improved performance of the developed method. Our simulation studies demonstrate enhanced tracking and function approximation performance from both DNN architectures and accelerated gradient adaptation.

Electromechanical delay during functional electrical stimulation induced cycling is a function of lower limb position.

INTRODUCTION: Functional electrical stimulation (FES) induced cycling has been shown to be an effective rehabilitation for those with lower limb movement disorders. However, a consequence of FES is an electromechanical delay (EMD) existing between the stimulation input and the onset of muscle force. The objective of this study is to determine if the cycle crank angle has an effect on the EMD. METHODS: Experiments were performed on 10 participants, five healthy and five with neurological conditions resulting in movement disorders. A motor fixed the crank arm of a FES-cycle in 10° increments and at each angle stimulation was applied in a random sequence to a combination of the quadriceps femoris and gluteal muscle groups. The EMD was examined by considering the contraction delay (CD) and the residual delay (RD), where the CD (RD) is the time latency between the start (end) of stimulation and the onset (cessation) of torque. Two different measurements were used to examine the CD and RD. Further, two multiple linear regressions were performed on each measurement, one for the left and one for the right muscle groups. RESULTS: The crank angle was determined to be statistically relevant for both the CD and RD. CONCLUSIONS: Since the crank angle has a significant effect on both the CD and RD, the angle has a significant effect on the EMD. Therefore, future efforts should consider the importance of the crank angle when modelling or estimating the EMD to improve control designs and ultimately improve rehabilitative treatments.Implications for rehabilitationNew model predicts the delayed response of muscle torque production to electrical stimulation as a function of limb position during FES cycling.The model can inform closed-loop electrical stimulation induced rehabilitative cycling.

Data-Based and Opportunistic Integral Concurrent Learning for Adaptive Trajectory Tracking During Switched FES-Induced Biceps Curls.

Hybrid exoskeletons, which combine functional electrical stimulation (FES) with a motorized testbed, can potentially improve the rehabilitation of people with movement disorders. However, hybrid exoskeletons have inherently nonlinear and uncertain dynamics, including combinations of discrete modes that switch between different continuous dynamic subsystems, which complicate closed-loop control. A particular complication is the uncertain muscle control effectiveness associated with FES. In this work, adaptive integral concurrent learning (ICL) motor and FES controllers are developed for a hybrid biceps curl exoskeleton, which are designed to achieve opportunistic and data-based learning of the uncertain human and electromechanical testbed parameters. Global exponential trajectory tracking and parameter estimation errors are proven through a Lyapunov-based stability analysis. The motor effectiveness is assumed to be unknown, and, to help with fatigue reduction, FES is enabled to switch between multiple electrodes on the biceps brachii, further complicating the analysis. A consequence of switching between the different uncertain subsystems is that the parameters must be opportunistically learned for each subsystem (i.e. each electrode and the motor), while that subsystem is active. Experiments were performed to validate the developed ICL controllers on twelve healthy participants. The average (± standard deviation) position tracking errors across each participant were 1.44 ± 5.32 deg, -0.25 ± 2.85 deg, and -0.17 ± 2.66 deg across biceps Curls 1-3, 4-7, and 8-10, respectively, where the average across the entire experiment was 0.28 ± 3.53 deg.

Encouraging Volitional Pedaling in Functional Electrical Stimulation-Assisted Cycling Using Barrier Functions.

Stationary motorized cycling assisted by functional electrical stimulation (FES) is a popular therapy for people with movement impairments. Maximizing volitional contributions from the rider of the cycle can lead to long-term benefits like increased muscular strength and cardiovascular endurance. This paper develops a combined motor and FES control system that tasks the rider with maintaining their cadence near a target point using their own volition, while assistance or resistance is applied gradually as their cadence approaches the lower or upper boundary, respectively, of a user-defined safe range. Safety-ensuring barrier functions are used to guarantee that the rider's cadence is constrained to the safe range, while minimal assistance is provided within the range to maximize effort by the rider. FES stimulation is applied before electric motor assistance to further increase power output from the rider. To account for uncertain dynamics, barrier function methods are combined with robust control tools from Lyapunov theory to develop controllers that guarantee safety in the worst-case. Because of the intermittent nature of FES stimulation, the closed-loop system is modeled as a hybrid system to certify that the set of states for which the cadence is in the safe range is asymptotically stable. The performance of the developed control method is demonstrated experimentally on five participants. The barrier function controller constrained the riders' cadence in a range of 50 ± 5 RPM with an average cadence standard deviation of 1.4 RPM for a protocol where cadence with minimal variance was prioritized and used minimal assistance from the motor (4.1% of trial duration) in a separate protocol where power output from the rider was prioritized.

Characterization of the Time-Varying Nature of Electromechanical Delay During FES-Cycling.

Functional electrical stimulation (FES) induced cycling is a common rehabilitative technique for people with neuromuscular disorders. A challenge for closed-loop FES control is that there exists a potentially destabilizing time-varying input delay, termed electromechanical delay (EMD), between the application of the electric field and the corresponding muscle contraction. In this article, the FES-induced torque production and EMD are quantified on an FES-cycle for the quadriceps femoris and gluteal muscle groups. Experiments were performed on five able-bodied individuals and five individuals with neurological conditions. Closed-loop FES-cycling was applied to induce fatigue and torque and EMD measurements were made during isometric conditions before and after each minute of cycling to quantify the effect of fatigue on EMD and torque production. A multiple linear regression and other descriptive statistics were performed to establish a range of expected EMD values and bounds on the rate of change of the EMD across a diverse population. The results from these experiments can be used to assist in the development of closed-loop controllers for FES-cycling that are robust to time-varying EMD and changes in torque production.

FES Cycling in Stroke: Novel Closed-Loop Algorithm Accommodates Differences in Functional Impairments.

OBJECTIVE: The objective of this paper was to develop and test a novel control algorithm that enables stroke survivors to pedal a cycle in a desired cadence range despite varying levels of functional abilities after stroke. METHODS: A novel algorithm was developed which automatically adjusts 1) the intensity of functional electrical stimulation (FES) delivered to the leg muscles, and 2) the current delivered to an electric motor. The algorithm automatically switches between assistive, uncontrolled, and resistive modes to accommodate for differences in functional impairment, based on the mismatch between the desired and actual cadence. Lyapunov-based methods were used to theoretically prove that the rider's cadence converges to the desired cadence range. To demonstrate the controller's real-world performance, nine chronic stroke survivors performed two cycling trials: 1) volitional effort only and 2) volitional effort accompanied by the control algorithm assisting and resisting pedaling as needed. RESULTS: With a desired cadence range of 50-55 r/min, the developed controller resulted in an average rms cadence error of 1.90 r/min, compared to 6.16 r/min during volitional-only trials. CONCLUSION: Using FES and an electric motor with a two-sided cadence control objective to assist and resist volitional efforts enabled stroke patients with varying strength and abilities to pedal within a desired cadence range. SIGNIFICANCE: A protocol design that constrains volitional movements with assistance and resistance from FES and a motor shows potential for FES cycles and other rehabilitation robots during stroke rehabilitation.

Distributed Repetitive Learning Control for Cooperative Cadence Tracking in Functional Electrical Stimulation Cycling.

Closed-loop control of functional electrical stimulation coupled with motorized assistance to induce cycling is a rehabilitative strategy that can improve the mobility of people with neurological conditions (NCs). However, robust control methods, which are currently pervasive in the cycling literature, have limited effectiveness due to the use of high stimulation intensity leading to accelerated fatigue during cycling protocols. This paper examines the design of a distributed repetitive learning controller (RLC) that commands an independent learning feedforward term to each of the six stimulated lower-limb muscle groups and an electric motor during the tracking of a periodic cadence trajectory. The switched controller activates lower limb muscles during kinematic efficient regions of the crank cycle and provides motorized assistance only when most needed (i.e., during the portions of the crank cycle where muscles evoke a low torque output). The controller exploits the periodicity of the desired cadence trajectory to learn from previous control inputs for each muscle group and electric motor. A Lyapunov-based stability analysis guarantees asymptotic tracking via an invariance-like corollary for nonsmooth systems. The switched distributed RLC was evaluated in experiments with seven able-bodied individuals and five participants with NCs. A mean root-mean-squared cadence error of 3.58 ± 0.43 revolutions per minute (RPM) (0.07 ± 7.35% average error) and 4.26 ± 0.84 RPM (0.1 ± 8.99% average error) was obtained for the healthy and neurologically impaired populations, respectively.

Controlling the Cadence and Admittance of a Functional Electrical Stimulation Cycle.

For an individual suffering from a neurological condition, such as spinal cord injury, traumatic brain injury, or stroke, motorized functional electrical stimulation (FES) cycling is a rehabilitation strategy, which offers numerous health benefits. Motorized FES cycling is an example of physical human-robot interaction in which both systems must be controlled; the human is actuated by applying neuromuscular electrical stimulation to the large leg muscle groups, and the cycle is actuated through its onboard electric motor. While the rider is stimulated using a robust sliding-mode controller, the cycle utilizes an admittance controller to preserve rider safety. The admittance controller is shown to be passive with respect to the rider, and the cadence controller is shown to be globally exponentially stable through a Lyapunov-like switched systems stability analysis. Experiments are conducted on three able-bodied participants and four participants with neurological conditions (NCs) to demonstrate the efficacy of the developed controller and investigate the effect of manipulating individual admittance parameters. Results demonstrate an average admittance cadence error of -0.06±1.47 RPM for able-bodied participants and -0.02 ± 0.93 RPM for participants with NCs.

The State Following Approximation Method.

A function approximation method is developed which aims to approximate a function in a small neighborhood of a state that travels within a compact set. The method provides a novel approximation strategy for the efficient approximation of nonlinear functions for real-time simulations and experiments. The development is based on the theory of universal reproducing kernel Hilbert spaces over the n -dimensional Euclidean space. Several theorems are introduced which support the development of this state following (StaF) method. In particular, it is shown that there is a bound on the number of kernel functions required for the maintenance of an accurate function approximation as a state moves through a compact set. In addition, a weight update law, based on gradient descent, is introduced where arbitrarily close accuracy can be achieved provided the weight update law is iterated at a sufficient frequency, as detailed in Theorem 4. An experience-based approximation method is presented which utilizes the samples of the estimations of the ideal weights to generate a global approximation of a function. The experience-based approximation interpolates the samples of the weight estimates using radial basis functions. To illustrate the StaF method, the method is utilized for derivative estimation, function approximation, and is applied to an adaptive dynamic programming problem where it is demonstrated that the stability is maintained with a reduced number of basis functions.

Approximate Dynamic Programming: Combining Regional and Local State Following Approximations.

An infinite-horizon optimal regulation problem for a control-affine deterministic system is solved online using a local state following (StaF) kernel and a regional model-based reinforcement learning (R-MBRL) method to approximate the value function. Unlike traditional methods such as R-MBRL that aim to approximate the value function over a large compact set, the StaF kernel approach aims to approximate the value function in a local neighborhood of the state that travels within a compact set. In this paper, the value function is approximated using a state-dependent convex combination of the StaF-based and the R-MBRL-based approximations. As the state enters a neighborhood containing the origin, the value function transitions from being approximated by the StaF approach to the R-MBRL approach. Semiglobal uniformly ultimately bounded (SGUUB) convergence of the system states to the origin is established using a Lyapunov-based analysis. Simulation results are provided for two, three, six, and ten-state dynamical systems to demonstrate the scalability and performance of the developed method.

Influence of Elbow Flexion and Stimulation Site on Neuromuscular Electrical Stimulation of the Biceps Brachii.

Functional electrical stimulation (FES) can help individuals with physical disabilities by assisting limb movement; however, the change in muscle geometry associated with limb movement may affect the response to stimulation. The aim of this paper was to quantify the effects of elbow flexion and stimulation site on muscle torque production. Contraction torque about the elbow was measured in 12 healthy individuals using a custom elbow flexion testbed and a transcutaneous electrode array. Stimulation was delivered to six distinct sites along the biceps brachii over 11 elbow flexion angles. Flexion angle was found to significantly influence the optimal (i.e., torque-maximizing) stimulation site ( ), with post hoc analysis indicating a proximal shift in optimal stimulation site with increased flexion. Similarly, the biceps stimulation site was found to significantly influence the flexion angle at which peak torque occurred ( ), with post hoc analysis indicating an increase in peak-torque flexion angle as stimulation site is moved proximally up the biceps. Since maximizing muscle force per unit stimulation is a common goal in rehabilitative FES, future efforts could examine methods which compensate for the shift in optimal stimulation site during FES-induced limb movement.

Synchronization of Uncertain Euler-Lagrange Systems With Uncertain Time-Varying Communication Delays.

A decentralized controller is designed for leader-based synchronization of communication-delayed networked agents. The agents have heterogeneous dynamics modeled by uncertain, nonlinear Euler-Lagrange equations of motion affected by heterogeneous, unknown, exogenous disturbances. The developed controller requires only one-hop (delayed) communication from network neighbors and the communication delays are assumed to be heterogeneous, uncertain, and time-varying. Each agent uses an estimate of communication delay to provide feedback of estimated recent tracking error. Simulation results are provided to demonstrate the improved performance of the developed controller over other popular control designs.

Position and torque control via rehabilitation robot and functional electrical stimulation.

Two common rehabilitation therapies for individuals possessing neurological conditions are functional electrical stimulation (FES) and robotic assistance. This paper focuses on combining the two rehabilitation strategies for use on the biceps brachii muscle group. FES is used to elicit muscle contractions to actuate the forearm and a rehabilitation robot is used to challenge the muscle group in its efforts. Two controllers were developed and implemented to accomplish the multifaceted objective, both of which achieve global exponential stability for position and torque tracking as proven through a Lyapunov stability analysis. Experiments performed on one able bodied individual demonstrate an average RMS error of 5.8 degrees for position tracking and 0.40 Newton-meters for torque tracking.

Model-Based Reinforcement Learning for Infinite-Horizon Approximate Optimal Tracking.

This brief paper provides an approximate online adaptive solution to the infinite-horizon optimal tracking problem for control-affine continuous-time nonlinear systems with unknown drift dynamics. To relax the persistence of excitation condition, model-based reinforcement learning is implemented using a concurrent-learning-based system identifier to simulate experience by evaluating the Bellman error over unexplored areas of the state space. Tracking of the desired trajectory and convergence of the developed policy to a neighborhood of the optimal policy are established via Lyapunov-based stability analysis. Simulation results demonstrate the effectiveness of the developed technique.

The Time-Varying Nature of Electromechanical Delay and Muscle Control Effectiveness in Response to Stimulation-Induced Fatigue.

Neuromuscular electrical stimulation (NMES) and Functional Electrical Stimulation (FES) are commonly prescribed rehabilitative therapies. Closed-loop NMES holds the promise to yield more accurate limb control, which could enable new rehabilitative procedures. However, NMES/FES can rapidly fatigue muscle, which limits potential treatments and presents several control challenges. Specifically, the stimulation intensity-force relation changes as the muscle fatigues. Additionally, the delayed response between the application of stimulation and muscle force production, termed electromechanical delay (EMD), may increase with fatigue. This paper quantifies these effects. Specifically, open-loop fatiguing protocols were applied to the quadriceps femoris muscle group of able-bodied individuals under isometric conditions, and the resulting torque was recorded. Short pulse trains were used to measure EMD with a thresholding method while long duration pulse trains were used to induce fatigue, measure EMD with a cross-correlation method, and construct recruitment curves. EMD was found to increase significantly with fatigue, and the control effectiveness (i.e., the linear slope of the recruitment curve) decreased with fatigue. Outcomes of these experiments indicate an opportunity for improved closed-loop NMES/FES control development by considering EMD to be time-varying and by considering the muscle recruitment curve to be a nonlinear, time-varying function of the stimulation input.

Switched Tracking Control of the Lower Limb During Asynchronous Neuromuscular Electrical Stimulation: Theory and Experiments.

Neuromuscular electrical stimulation (NMES) induces muscle contractions via electrical stimuli. NMES can be used for rehabilitation and to enable functional movements; however, a fundamental limitation is the early onset of fatigue. Asynchronous stimulation is a method that can reduce fatigue by utilizing multiple stimulation channels to segregate and switch between different sets of recruited motor units. However, switching between stimulation channels is challenging due to each channel's differing response to stimulation. To address this challenge, a switched systems analysis is used in the present work to design a controller that allows for instantaneous switching between stimulation channels. The developed controller yields semi-global exponential tracking of a desired angular trajectory for a person's knee-joint. Experiments were conducted in six able-bodied individuals. Compared to conventional stimulation, the results indicate that asynchronous stimulation with the developed controller yields longer durations of successful tracking despite different responses between the stimulation channels.

Switched Control of Cadence During Stationary Cycling Induced by Functional Electrical Stimulation.

Functional electrical stimulation (FES) can be used to activate the dysfunctional lower limb muscles of individuals with neurological disorders to produce cycling as a means of rehabilitation. However, previous literature suggests that poor muscle control and nonphysiological muscle fiber recruitment during FES-cycling causes lower efficiency and power output at the cycle crank than able-bodied cycling, thus motivating the investigation of improved control methods for FES-cycling. In this paper, a stimulation pattern is designed based on the kinematic effectiveness of the rider's hip and knee joints to produce a forward torque about the cycle crank. A robust controller is designed for the uncertain, nonlinear cycle-rider system with autonomous, state-dependent switching. Provided sufficient conditions are satisfied, the switched controller yields ultimately bounded tracking of a desired cadence. Experimental results on four able-bodied subjects demonstrate cadence tracking errors of 0.05 ±1.59 and 5.27 ±2.14 revolutions per minute during volitional and FES-induced cycling, respectively. To establish feasibility of FES-assisted cycling in subjects with Parkinson's disease, experimental results with one subject demonstrate tracking errors of 0.43 ± 4.06 and 0.17 ±3.11 revolutions per minute during volitional and FES-induced cycling, respectively.

Identification-Based Closed-Loop NMES Limb Tracking With Amplitude-Modulated Control Input.

An upper motor neuron lesion (UMNL) can be caused by various neurological disorders or trauma and leads to disabilities. Neuromuscular electrical stimulation (NMES) is a technique that is widely used for rehabilitation and restoration of motor function for people suffering from UMNL. Typically, stability analysis for closed-loop NMES ignores the modulated implementation of NMES. However, electrical stimulation must be applied to muscle as a modulated series of pulses. In this paper, a muscle activation model with an amplitude modulated control input is developed to capture the discontinuous nature of muscle activation, and an identification-based closed-loop NMES controller is designed and analyzed for the uncertain amplitude modulated muscle activation model. Semi-global uniformly ultimately bounded tracking is guaranteed. The stability of the closed-loop system is analyzed with Lyapunov-based methods, and a pulse frequency related gain condition is obtained. Experiments are performed with five able-bodied subjects to demonstrate the interplay between the control gains and the pulse frequency, and results are provided which indicate that control gains should be increased to maintain stability if the stimulation pulse frequency is decreased to mitigate muscle fatigue. For the first time, this paper brings together an analysis of the controller and modulation scheme.

Closed-Loop Asynchronous Neuromuscular Electrical Stimulation Prolongs Functional Movements in the Lower Body.

Neuromuscular electrical stimulation (NMES) is commonly used in rehabilitative settings and is also used for assistive purposes to create functional movements, where it is termed functional electrical stimulation (FES). One limitation of NMES/FES is early onset of muscle fatigue. NMES-induced fatigue can be reduced by switching between multiple stimulation channels that target different motor units or synergistic muscles (i.e., asynchronous stimulation). However, switching stimulation channels introduces additional complexity due to the need to consider the switching dynamics and differing muscle response to stimulation. The objective of this study was to develop and test a closed-loop controller for asynchronous stimulation. The developed closed-loop controller yields asymptotic tracking of a desired trajectory for a person's knee-shank complex despite switching between stimulation channels. The developed controller was implemented on four able-bodied individuals with four-channel asynchronous stimulation as well as single-channel conventional stimulation. The results indicate that asynchronous stimulation extends the duration that functional movements can be performed during feedback control. This result is promising for the implementation of asynchronous stimulation in closed-loop rehabilitative procedures and in assistive devices as a method to reduce muscle fatigue while maintaining a person's ability to track a desired limb trajectory.

Comparing the Induced Muscle Fatigue Between Asynchronous and Synchronous Electrical Stimulation in Able-Bodied and Spinal Cord Injured Populations.

Neuromuscular electrical stimulation (NMES) has been shown to impart a number of health benefits and can be used to produce functional outcomes. However, one limitation of NMES is the onset of NMES-induced fatigue. Multi-channel asynchronous stimulation has been shown to reduce NMES-induced fatigue compared to conventional single-channel stimulation. However, in previous studies in man, the effect of stimulation frequency on the NMES-induced fatigue has not been examined for asynchronous stimulation. Low stimulation frequencies are known to reduce fatigue during conventional stimulation. Therefore, the aim of this study was to examine the fatigue characteristics of high- and low-frequency asynchronous stimulation as well as high- and low-frequency conventional stimulation. Experiments were performed in both able-bodied and spinal cord injured populations. Low frequency asynchronous stimulation is found to have significant fatigue benefits over high frequency asynchronous stimulation as well as high- and low-frequency conventional stimulation, motivating its use for rehabilitation and functional electrical stimulation (FES).