Mobile Robotic Balance Assistant (MRBA): a gait assistive and fall intervention robot for daily living.

BACKGROUND: Aging degrades the balance and locomotion ability due to frailty and pathological conditions. This demands balance rehabilitation and assistive technologies that help the affected population to regain mobility, independence, and improve their quality of life. While many overground gait rehabilitation and assistive robots exist in the market, none are designed to be used at home or in community settings. METHODS: A device named Mobile Robotic Balance Assistant (MRBA) is developed to address this problem. MRBA is a hybrid of a gait assistive robot and a powered wheelchair. When the user is walking around performing activities of daily living, the robot follows the person and provides support at the pelvic area in case of loss of balance. It can also be transformed into a wheelchair if the user wants to sit down or commute. To achieve instability detection, sensory data from the robot are compared with a predefined threshold; a fall is identified if the value exceeds the threshold. The experiments involve both healthy young subjects and an individual with spinal cord injury (SCI). Spatial Parametric Mapping is used to assess the effect of the robot on lower limb joint kinematics during walking. The instability detection algorithm is evaluated by calculating the sensitivity and specificity in identifying normal walking and simulated falls. RESULTS: When walking with MRBA, the healthy subjects have a lower speed, smaller step length and longer step time. The SCI subject experiences similar changes as well as a decrease in step width that indicates better stability. Both groups of subjects have reduced joint range of motion. By comparing the force sensor measurement with a calibrated threshold, the instability detection algorithm can identify more than 93% of self-induced falls with a false alarm rate of 0%. CONCLUSIONS: While there is still room for improvement in the robot compliance and the instability identification, the study demonstrates the first step in bringing gait assistive technologies into homes. We hope that the robot can encourage the balance-impaired population to engage in more activities of daily living to improve their quality of life. Future research includes recruiting more subjects with balance difficulty to further refine the device functionalities.

Learning-Based Motion-Intention Prediction for End-Point Control of Upper-Limb-Assistive Robots.

The lack of intuitive and active human-robot interaction makes it difficult to use upper-limb-assistive devices. In this paper, we propose a novel learning-based controller that intuitively uses onset motion to predict the desired end-point position for an assistive robot. A multi-modal sensing system comprising inertial measurement units (IMUs), electromyographic (EMG) sensors, and mechanomyography (MMG) sensors was implemented. This system was used to acquire kinematic and physiological signals during reaching and placing tasks performed by five healthy subjects. The onset motion data of each motion trial were extracted to input into traditional regression models and deep learning models for training and testing. The models can predict the position of the hand in planar space, which is the reference position for low-level position controllers. The results show that using IMU sensor with the proposed prediction model is sufficient for motion intention detection, which can provide almost the same prediction performance compared with adding EMG or MMG. Additionally, recurrent neural network (RNN)-based models can predict target positions over a short onset time window for reaching motions and are suitable for predicting targets over a longer horizon for placing tasks. This study's detailed analysis can improve the usability of the assistive/rehabilitation robots.

Autonomous modeling of repetitive movement for rehabilitation exercise monitoring.

BACKGROUND: Insightful feedback generation for daily home-based stroke rehabilitation is currently unavailable due to the inefficiency of exercise inspection done by therapists. We aim to produce a compact anomaly representation that allows a therapist to pay attention to only a few specific sections in a long exercise session record and boost their efficiency in feedback generation. METHODS: This study proposes a data-driven technique to model a repetitive exercise using unsupervised phase learning on an artificial neural network and statistical learning on principal component analysis (PCA). After a model is built on a set of normal healthy movements, the model can be used to extract a sequence of anomaly scores from a movement of the same prescription. RESULTS: The method not only works on a standard marker-based motion capture system but also performs well on a more compact and affordable motion capture system based-on Kinect V2 and wrist-worn inertial measurement units that can be used at home. An evaluation of four different exercises shows its potential in separating anomalous movements from normal ones with an average area under the curve (AUC) of 0.9872 even on the compact motion capture system. CONCLUSIONS: The proposed processing technique has the potential to help clinicians in providing high-quality feedback for telerehabilitation in a more scalable way.

Real-Time Foot Tracking and Gait Evaluation with Geometric Modeling.

Gait evaluation is important in gait rehabilitation and assistance to monitor patient's balance status and assess recovery performance. Recent technologies leverage on vision-based systems with high portability and low operational complexity. In this paper, we propose a new vision-based foot tracking algorithm specially catering to overground gait assistive devices, which often have limited view of the users. The algorithm models the foot and the shank of the user using simple geometry. Through cost optimization, it then aligns the models to the point cloud, showing the back view of the user's lower limbs. The system outputs the poses of the feet, which are used to compute the spatial-temporal gait parameters. Seven healthy young subjects are recruited to perform overground and treadmill walking trials. The results of the algorithm are compared with the motion capture system and a third-party gait analysis software. The algorithm has a fitting rotational and translational errors of less than 20 degrees and 33 mm, respectively, for 0.4 m/s walking speed. The gait detection F1 score achieves more than 96.8%. The step length and step width errors are around 35 mm, while the cycle time error is less than 38 ms. The proposed algorithm provides a fast, contactless, portable, and cost-effective gait evaluation method without requiring the user to wear any customized footwear.

A Modified Prandtl-Ishlinskii Hysteresis Model for Modeling and Compensating Asymmetric Hysteresis of Piezo-Actuated Flexure-Based Systems.

Piezo-actuated flexure-based systems are widely used in applications with high accuracy requirements, but the intrinsic hysteresis has a detrimental effect on the performance which should be compensated. Conventional models were presented to model this undesired effect using additional dead-zone operators. This paper presents a new approach using two sets of operators with a distributed compensator to model and compensate for the asymmetric system hysteresis based on inversion calculation with a simplified digitized representation. The experimental results validate the effectiveness of the proposed model in modeling and compensating the asymmetric system hysteresis.

Fall inducing movable platform (FIMP) for overground trips and slips.

BACKGROUND: The study of falls and fall prevention/intervention devices requires the recording of true falls incidence. However, true falls are rare, random, and difficult to collect in real world settings. A system capable of producing falls in an ecologically valid manner will be very helpful in collecting the data necessary to advance our understanding of the neuro and musculoskeletal mechanisms underpinning real-world falls events. METHODS: A fall inducing movable platform (FIMP) was designed to arrest or accelerate a subject's ankle to induce a trip or slip. The ankle was arrested posteriorly with an electromagnetic brake and accelerated anteriorly with a motor. A power spring was connected in series between the ankle and the brake/motor to allow freedom of movement (system transparency) when a fall is not being induced. A gait phase detection algorithm was also created to enable precise activation of the fall inducing mechanisms. Statistical Parametric Mapping (SPM1D) and one-way repeated measure ANOVA were used to evaluate the ability of the FIMP to induce a trip or slip. RESULTS: During FIMP induced trips, the brake activates at the terminal swing or mid swing gait phase to induce the lowering or skipping strategies, respectively. For the lowering strategy, the characteristic leg lowering and subsequent contralateral leg swing was seen in all subjects. Likewise, for the skipping strategy, all subjects skipped forward on the perturbed leg. Slip was induced by FIMP by using a motor to impart unwanted forward acceleration to the ankle with the help of friction-reducing ground sliding sheets. Joint stiffening was observed during the slips, and subjects universally adopted the surfing strategy after the initial slip. CONCLUSION: The results indicate that FIMP can induce ecologically valid falls under controlled laboratory conditions. The use of SPM1D in conjunction with FIMP allows for the time varying statistical quantification of trip and slip reactive kinematics events. With future research, fall recovery anomalies in subjects can now also be systematically evaluated through the assessment of other neuromuscular variables such as joint forces, muscle activation and muscle forces.

Analysis of accuracy in pointing with redundant hand-held tools: a geometric approach to the uncontrolled manifold method.

This work introduces a coordinate-independent method to analyse movement variability of tasks performed with hand-held tools, such as a pen or a surgical scalpel. We extend the classical uncontrolled manifold (UCM) approach by exploiting the geometry of rigid body motions, used to describe tool configurations. In particular, we analyse variability during a static pointing task with a hand-held tool, where subjects are asked to keep the tool tip in steady contact with another object. In this case the tool is redundant with respect to the task, as subjects control position/orientation of the tool, i.e. 6 degrees-of-freedom (dof), to maintain the tool tip position (3dof) steady. To test the new method, subjects performed a pointing task with and without arm support. The additional dof introduced in the unsupported condition, injecting more variability into the system, represented a resource to minimise variability in the task space via coordinated motion. The results show that all of the seven subjects channeled more variability along directions not directly affecting the task (UCM), consistent with previous literature but now shown in a coordinate-independent way. Variability in the unsupported condition was only slightly larger at the endpoint but much larger in the UCM.

Anatomical-Marker-Driven 3D Markerless Human Motion Capture.

Marker-based motion capture (mocap) is a conventional method used in biomechanics research to precisely analyze human movement. However, the time-consuming marker placement process and extensive post-processing limit its wider adoption. Therefore, markerless mocap systems that use deep learning to estimate 2D keypoint from images have emerged as a promising alternative, but annotation errors in training datasets used by deep learning models can affect estimation accuracy. To improve the precision of 2D keypoint annotation, we present a method that uses anatomical landmarks based on marker-based mocap. Specifically, we use multiple RGB cameras synchronized and calibrated with a marker-based mocap system to create a high-quality dataset (RRIS40) of images annotated with surface anatomical landmarks. A deep neural network is then trained to estimate these 2D anatomical landmarks and a ray-distance-based triangulation is used to calculate the 3D marker positions. We conducted extensive evaluations on our RRIS40 test set, which consists of 10 subjects performing various movements. Compared against a marker-based system, our method achieves a mean Euclidean error of 13.23 mm in 3D marker position, which is comparable to the precision of marker placement itself. By learning directly to predict anatomical keypoints from images, our method outperforms OpenCap's augmentation of 3D anatomical landmarks from triangulated wild keypoints. This highlights the potential of facilitating wider integration of markerless mocap into biomechanics research. The RRIS40 test set is made publicly available for research purposes at koonyook.github.io/rris40.

Real-time Tracking of Handheld Object from Color or Depth Images.

Object tracking during rehabilitation could help a therapist to evaluate a patient's movement and progress. Hence, we present an image-based method for real-time tracking of handheld objects due to its ease of use and availability of color or depth cameras. We use an efficient projective point correspondence method and generalize the use of precomputed spare viewpoint information to allow real-time tracking of a rigid object. The method runs at more than 30 fps on a CPU while achieving submillimeter accuracy on synthetic datasets and robust tracking on a semi-synthetic dataset.Clinical relevance Real-time, accurate, and robust tracking of an object using an image-based method is a promising tool for rehabilitation applications as it is practical for clinical settings.

Wireless Pressure Sensor Array Module for Sensorized Object.

To complement rehabilitation assessments that involve hand-object interaction with additional information on the grasping parameters, we sensorized an object with a pressure sensor array module that can generate a pressure distribution map. The module can be customized for cylindrical and cuboid objects with up to 1024 sensing elements and it supports the efficient transfer of data wirelessly at more than 30 Hz. Although the module uses inexpensive materials, it is sensitive to changes in pressure distribution. It can also depict the shape of various objects with reasonable details as shown in the small errors for object pose estimation and high accuracy scores for hand grasp classification. The module's modular design and wireless functionality help to simplify integration with existing objects to create a smart sensing surface.Clinical relevance The resulting pressure distribution map allows the therapist to analyze grasping parameters that cannot be determined from visual observations alone.

Safety-Based Speed Control of a Wheelchair Using Robust Adaptive Model Predictive Control.

Electric-powered wheelchairs play a vital role in ensuring accessibility for individuals with mobility impairments. The design of controllers for tracking tasks must prioritize the safety of wheelchair operation across various scenarios and for a diverse range of users. In this study, we propose a safety-oriented speed tracking control algorithm for wheelchair systems that accounts for external disturbances and uncertain parameters at the dynamic level. We employ a set-membership approach to estimate uncertain parameters online in deterministic sets. Additionally, we present a model predictive control scheme with real-time adaptation of the system model and controller parameters to ensure safety-related constraint satisfaction during the tracking process. This proposed controller effectively guides the wheelchair speed toward the desired reference while maintaining safety constraints. In cases where the reference is inadmissible and violates constraints, the controller can navigate the system to the vicinity of the nearest admissible reference. The efficiency of the proposed control scheme is demonstrated through high-fidelity speed tracking results from two tasks involving both admissible and inadmissible references.

Creation and Evaluation of Human Models with Varied Walking Ability from Motion Capture for Assistive Device Development.

As the world ages, rehabilitation and assistive devices will play a key role in improving mobility. However, designing controllers for these devices presents several challenges, from varying degrees of impairment to unique adaptation strategies of users. To use computer simulation to address these challenges, simulating human motions is required. Recently, deep reinforcement learning (DRL) has been successfully applied to generate walking motions whose goal is to produce a stable human walking policy. However, from a rehabilitation perspective, it is more important to match the walking policy's ability to that of an impaired person with reduced ability. In this paper, we present the first attempt to investigate the correlation between DRL training parameters with the ability of the generated human walking policy to recover from perturbation. We show that the control policies can produce gait patterns resembling those of humans without perturbation and that varying perturbation parameters during training can create variation in the recovery ability of the human model. We also demonstrate that the control policy can produce similar behaviours when subjected to forces that users may experience while using a balance assistive device.

Versatile clinical movement analysis using statistical parametric mapping in MovementRx.

Clinical gait analysis is an important biomechanics field that is often influenced by subjectivity in time-varying analysis leading to type I and II errors. Statistical Parametric Mapping can operate on all time-varying joint dynamics simultaneously, thereby overcoming subjectivity errors. We present MovementRx, the first gait analysis modelling application that correctly models the deviations of joints kinematics and kinetics both in 3 and 1 degrees of freedom; presented with easy-to-understand color maps for clinicians with limited statistical training. MovementRx is a python-based versatile GUI-enabled movement analysis decision support system, that provides a holistic view of all lower limb joints fundamental to the kinematic/kinetic chain related to functional gait. The user can cascade the view from single 3D multivariate result down to specific single joint individual 1D scalar movement component in a simple, coherent, objective, and visually intuitive manner. We highlight MovementRx benefit by presenting a case-study of a right knee osteoarthritis (OA) patient with otherwise undetected postintervention contralateral OA predisposition. MovementRx detected elevated frontal plane moments of the patient's unaffected knee. The patient also revealed a surprising adverse compensation to the contralateral limb.

Adaptive Gravity Compensation Framework Based on Human Upper Limb Model for Assistive Robotic Arm Extender.

The Assistive Robotic Arm Extender (ARAE) is an upper limb assistive and rehabilitation robot that belongs to the end-effector type, enabling it to assist patients with upper limb movement disorders in three-dimensional space. However, the problem of gravity compensation for the human upper limb with this type of robot is crucial, which directly affects the deployment of the robot in the assistive or rehabilitation field. This paper presents an adaptive gravity compensation framework that calculates the compensated force based on the estimated human posture in 3D space. First, we estimated the human arm joint angles in real-time without any wearable sensors, such as inertial measurement unit (IMU) or magnetic sensors, only through the kinematic data of the robot and established human model. The performance of the estimation method was evaluated through a motion capture system, which validated the accuracy of joint angle estimation. Second, the estimated human joint angles were input to the rigid link model to demonstrate the support force profile generated by the robot. The force profile showed that the support force provided by the developed ARAE robot could adaptively change with human arm postures in 3D space. The adaptive gravity compensation framework can improve the usability and feasibility of the 3D end-effector rehabilitation or assistive robot.

A Micro Motion Sensing System for Micromanipulation Tasks.

An optical-based motion sensing system has been developed for real-time sensing of instrument motion in micromanipulation. The main components of the system consist of a pair of position sensitive detectors (PSD), lenses, an infrared (IR) diode that illuminates the workspace of the system, a non-reflective intraocular shaft, and a white reflective ball attached at the end of the shaft. The system calculates 3D displacement of the ball inside the workspace using the centroid position of the IR rays that are reflected from the ball and strike the PSDs. In order to eliminate inherent nonlinearity of the system, calibration using a feedforward neural network is proposed and presented. Handling of different ambient light and environment light conditions not to affect the system accuracy is described. Analyses of the whole optical system and effect of instrument orientation on the system accuracy are presented. Sensing resolution, dynamic accuracies at a few different frequencies, and static accuracies at a few different orientations of the instrument are reported. The system and the analyses are useful in assessing performance of hand-held microsurgical instruments and operator performance in micromanipulation tasks.

Current clinical practice in managing somatosensory impairments and the use of technology in stroke rehabilitation.

Stroke-induced somatosensory impairments seem to be clinically overlooked, despite their prevalence and influence on motor recovery post-stroke. Interest in technology has been gaining traction over the past few decades as a promising method to facilitate stroke rehabilitation. This questionnaire-based cross-sectional study aimed to identify current clinical practice and perspectives on the management of somatosensory impairments post-stroke and the use of technology in assessing outcome measures and providing intervention. Participants were 132 physiotherapists and occupational therapists currently working with stroke patients in public hospitals and rehabilitation centres in Singapore. It was found that the majority (64.4%) of the therapists spent no more than half of the time per week on somatosensory interventions. Functional or task-specific training was the primary form of intervention applied to retrain somatosensory functions in stroke survivors. Standardised assessments (43.2%) were used less frequently than non-standardised assessments (97.7%) in clinical practice, with the sensory subscale of the Fugl-Meyer Assessment being the most popular outcome measure, followed by the Nottingham Sensory Assessment. While the adoption of technology for assessment was relatively scarce, most therapists (87.1%) reported that they have integrated technology into intervention. There was a common agreement that proprioception is an essential component in stroke rehabilitation, and that robotic technology combined with conventional therapy is effective in enhancing stroke rehabilitation, particularly for retraining proprioception. Most therapists identified price, technology usability, and lack of available space as some of the biggest barriers to integrating robotic technology in stroke rehabilitation. Standardised assessments and interventions targeting somatosensory functions should be more clearly delineated in clinical guidelines. Although therapists were positive about technology-based rehabilitation, obstacles that make technology integration challenging ought to be addressed.

Identification of Secondary Biomechanical Abnormalities in the Lower Limb Joints after Chronic Transtibial Amputation: A Proof-of-Concept Study Using SPM1D Analysis.

SPM is a statistical method of analysis of time-varying human movement gait signal, depending on the random field theory (RFT). MovementRx is our inhouse-developed decision-support system that depends on SPM1D Python implementation of the SPM (spm1d.org). We present the potential application of MovementRx in the prediction of increased joint forces with the possibility to predispose to osteoarthritis in a sample of post-surgical Transtibial Amputation (TTA) patients who were ambulant in the community. We captured the three-dimensional movement profile of 12 males with TTA and studied them using MovementRx, employing the SPM1D Python library to quantify the deviation(s) they have from our corresponding reference data, using "Hotelling 2" and "T test 2" statistics for the 3D movement vectors of the 3 main lower limb joints (hip, knee, and ankle) and their nine respective components (3 joints × 3 dimensions), respectively. MovementRx results visually demonstrated a clear distinction in the biomechanical recordings between TTA patients and a reference set of normal people (ABILITY data project), and variability within the TTA patients' group enabled identification of those with an increased risk of developing osteoarthritis in the future. We conclude that MovementRx is a potential tool to detect increased specific joint forces with the ability to identify TTA survivors who may be at risk for osteoarthritis.

Estimation of physiological tremor from accelerometers for real-time applications.

Accurate filtering of physiological tremor is extremely important in robotics assisted surgical instruments and procedures. This paper focuses on developing single stage robust algorithms for accurate tremor filtering with accelerometers for real-time applications. Existing methods rely on estimating the tremor under the assumption that it has a single dominant frequency. Our time-frequency analysis on physiological tremor data revealed that tremor contains multiple dominant frequencies over the entire duration rather than a single dominant frequency. In this paper, the existing methods for tremor filtering are reviewed and two improved algorithms are presented. A comparative study is conducted on all the estimation methods with tremor data from microsurgeons and novice subjects under different conditions. Our results showed that the new improved algorithms performed better than the existing algorithms for tremor estimation. A procedure to separate the intended motion/drift from the tremor component is formulated.

Drift-free position estimation of periodic or quasi-periodic motion using inertial sensors.

Position sensing with inertial sensors such as accelerometers and gyroscopes usually requires other aided sensors or prior knowledge of motion characteristics to remove position drift resulting from integration of acceleration or velocity so as to obtain accurate position estimation. A method based on analytical integration has previously been developed to obtain accurate position estimate of periodic or quasi-periodic motion from inertial sensors using prior knowledge of the motion but without using aided sensors. In this paper, a new method is proposed which employs linear filtering stage coupled with adaptive filtering stage to remove drift and attenuation. The prior knowledge of the motion the proposed method requires is only approximate band of frequencies of the motion. Existing adaptive filtering methods based on Fourier series such as weighted-frequency Fourier linear combiner (WFLC), and band-limited multiple Fourier linear combiner (BMFLC) are modified to combine with the proposed method. To validate and compare the performance of the proposed method with the method based on analytical integration, simulation study is performed using periodic signals as well as real physiological tremor data, and real-time experiments are conducted using an ADXL-203 accelerometer. Results demonstrate that the performance of the proposed method outperforms the existing analytical integration method.

A Digitized Representation of the Modified Prandtl-Ishlinskii Hysteresis Model for Modeling and Compensating Piezoelectric Actuator Hysteresis.

Piezoelectric actuators are widely used in micromanipulation and miniature robots due to their rapid response and high repeatability. The piezoelectric actuators often have undesired hysteresis. The Prandtl-Ishlinskii (PI) hysteresis model is one of the most popular models for modeling and compensating the hysteresis behaviour. This paper presents an alternative digitized representation of the modified Prandtl-Ishlinskii with the dead-zone operators (MPI) hysteresis model to describe the asymmetric hysteresis behavior of piezoelectric actuators. Using a binary number with n digits to represent the classical Prandtl-Ishlinskii hysteresis model with n elementary operators, the inverse model can be easily constructed. A similar representation of the dead-zone operators is also described. With the proposed digitized representation, the model is more intuitive and the inversion calculation is avoided. An experiment with a piezoelectric stacked linear actuator is conducted to validate the proposed digitized MPI hysteresis model and it is shown that it has almost the same performance as compared to the classical representation.