Real-time Tracking of Handheld Object from Color or Depth Images.

Object tracking during rehabilitation could help a therapist to evaluate a patient's movement and progress. Hence, we present an image-based method for real-time tracking of handheld objects due to its ease of use and availability of color or depth cameras. We use an efficient projective point correspondence method and generalize the use of precomputed spare viewpoint information to allow real-time tracking of a rigid object. The method runs at more than 30 fps on a CPU while achieving submillimeter accuracy on synthetic datasets and robust tracking on a semi-synthetic dataset.Clinical relevance Real-time, accurate, and robust tracking of an object using an image-based method is a promising tool for rehabilitation applications as it is practical for clinical settings.

Wireless Pressure Sensor Array Module for Sensorized Object.

To complement rehabilitation assessments that involve hand-object interaction with additional information on the grasping parameters, we sensorized an object with a pressure sensor array module that can generate a pressure distribution map. The module can be customized for cylindrical and cuboid objects with up to 1024 sensing elements and it supports the efficient transfer of data wirelessly at more than 30 Hz. Although the module uses inexpensive materials, it is sensitive to changes in pressure distribution. It can also depict the shape of various objects with reasonable details as shown in the small errors for object pose estimation and high accuracy scores for hand grasp classification. The module's modular design and wireless functionality help to simplify integration with existing objects to create a smart sensing surface.Clinical relevance The resulting pressure distribution map allows the therapist to analyze grasping parameters that cannot be determined from visual observations alone.

Creation and Evaluation of Human Models with Varied Walking Ability from Motion Capture for Assistive Device Development.

As the world ages, rehabilitation and assistive devices will play a key role in improving mobility. However, designing controllers for these devices presents several challenges, from varying degrees of impairment to unique adaptation strategies of users. To use computer simulation to address these challenges, simulating human motions is required. Recently, deep reinforcement learning (DRL) has been successfully applied to generate walking motions whose goal is to produce a stable human walking policy. However, from a rehabilitation perspective, it is more important to match the walking policy's ability to that of an impaired person with reduced ability. In this paper, we present the first attempt to investigate the correlation between DRL training parameters with the ability of the generated human walking policy to recover from perturbation. We show that the control policies can produce gait patterns resembling those of humans without perturbation and that varying perturbation parameters during training can create variation in the recovery ability of the human model. We also demonstrate that the control policy can produce similar behaviours when subjected to forces that users may experience while using a balance assistive device.

Adaptive Gravity Compensation Framework Based on Human Upper Limb Model for Assistive Robotic Arm Extender.

The Assistive Robotic Arm Extender (ARAE) is an upper limb assistive and rehabilitation robot that belongs to the end-effector type, enabling it to assist patients with upper limb movement disorders in three-dimensional space. However, the problem of gravity compensation for the human upper limb with this type of robot is crucial, which directly affects the deployment of the robot in the assistive or rehabilitation field. This paper presents an adaptive gravity compensation framework that calculates the compensated force based on the estimated human posture in 3D space. First, we estimated the human arm joint angles in real-time without any wearable sensors, such as inertial measurement unit (IMU) or magnetic sensors, only through the kinematic data of the robot and established human model. The performance of the estimation method was evaluated through a motion capture system, which validated the accuracy of joint angle estimation. Second, the estimated human joint angles were input to the rigid link model to demonstrate the support force profile generated by the robot. The force profile showed that the support force provided by the developed ARAE robot could adaptively change with human arm postures in 3D space. The adaptive gravity compensation framework can improve the usability and feasibility of the 3D end-effector rehabilitation or assistive robot.

Safety-Based Speed Control of a Wheelchair Using Robust Adaptive Model Predictive Control.

Electric-powered wheelchairs play a vital role in ensuring accessibility for individuals with mobility impairments. The design of controllers for tracking tasks must prioritize the safety of wheelchair operation across various scenarios and for a diverse range of users. In this study, we propose a safety-oriented speed tracking control algorithm for wheelchair systems that accounts for external disturbances and uncertain parameters at the dynamic level. We employ a set-membership approach to estimate uncertain parameters online in deterministic sets. Additionally, we present a model predictive control scheme with real-time adaptation of the system model and controller parameters to ensure safety-related constraint satisfaction during the tracking process. This proposed controller effectively guides the wheelchair speed toward the desired reference while maintaining safety constraints. In cases where the reference is inadmissible and violates constraints, the controller can navigate the system to the vicinity of the nearest admissible reference. The efficiency of the proposed control scheme is demonstrated through high-fidelity speed tracking results from two tasks involving both admissible and inadmissible references.

Mobile Robotic Balance Assistant (MRBA): a gait assistive and fall intervention robot for daily living.

BACKGROUND: Aging degrades the balance and locomotion ability due to frailty and pathological conditions. This demands balance rehabilitation and assistive technologies that help the affected population to regain mobility, independence, and improve their quality of life. While many overground gait rehabilitation and assistive robots exist in the market, none are designed to be used at home or in community settings. METHODS: A device named Mobile Robotic Balance Assistant (MRBA) is developed to address this problem. MRBA is a hybrid of a gait assistive robot and a powered wheelchair. When the user is walking around performing activities of daily living, the robot follows the person and provides support at the pelvic area in case of loss of balance. It can also be transformed into a wheelchair if the user wants to sit down or commute. To achieve instability detection, sensory data from the robot are compared with a predefined threshold; a fall is identified if the value exceeds the threshold. The experiments involve both healthy young subjects and an individual with spinal cord injury (SCI). Spatial Parametric Mapping is used to assess the effect of the robot on lower limb joint kinematics during walking. The instability detection algorithm is evaluated by calculating the sensitivity and specificity in identifying normal walking and simulated falls. RESULTS: When walking with MRBA, the healthy subjects have a lower speed, smaller step length and longer step time. The SCI subject experiences similar changes as well as a decrease in step width that indicates better stability. Both groups of subjects have reduced joint range of motion. By comparing the force sensor measurement with a calibrated threshold, the instability detection algorithm can identify more than 93% of self-induced falls with a false alarm rate of 0%. CONCLUSIONS: While there is still room for improvement in the robot compliance and the instability identification, the study demonstrates the first step in bringing gait assistive technologies into homes. We hope that the robot can encourage the balance-impaired population to engage in more activities of daily living to improve their quality of life. Future research includes recruiting more subjects with balance difficulty to further refine the device functionalities.

Learning-Based Motion-Intention Prediction for End-Point Control of Upper-Limb-Assistive Robots.

The lack of intuitive and active human-robot interaction makes it difficult to use upper-limb-assistive devices. In this paper, we propose a novel learning-based controller that intuitively uses onset motion to predict the desired end-point position for an assistive robot. A multi-modal sensing system comprising inertial measurement units (IMUs), electromyographic (EMG) sensors, and mechanomyography (MMG) sensors was implemented. This system was used to acquire kinematic and physiological signals during reaching and placing tasks performed by five healthy subjects. The onset motion data of each motion trial were extracted to input into traditional regression models and deep learning models for training and testing. The models can predict the position of the hand in planar space, which is the reference position for low-level position controllers. The results show that using IMU sensor with the proposed prediction model is sufficient for motion intention detection, which can provide almost the same prediction performance compared with adding EMG or MMG. Additionally, recurrent neural network (RNN)-based models can predict target positions over a short onset time window for reaching motions and are suitable for predicting targets over a longer horizon for placing tasks. This study's detailed analysis can improve the usability of the assistive/rehabilitation robots.

Versatile clinical movement analysis using statistical parametric mapping in MovementRx.

Clinical gait analysis is an important biomechanics field that is often influenced by subjectivity in time-varying analysis leading to type I and II errors. Statistical Parametric Mapping can operate on all time-varying joint dynamics simultaneously, thereby overcoming subjectivity errors. We present MovementRx, the first gait analysis modelling application that correctly models the deviations of joints kinematics and kinetics both in 3 and 1 degrees of freedom; presented with easy-to-understand color maps for clinicians with limited statistical training. MovementRx is a python-based versatile GUI-enabled movement analysis decision support system, that provides a holistic view of all lower limb joints fundamental to the kinematic/kinetic chain related to functional gait. The user can cascade the view from single 3D multivariate result down to specific single joint individual 1D scalar movement component in a simple, coherent, objective, and visually intuitive manner. We highlight MovementRx benefit by presenting a case-study of a right knee osteoarthritis (OA) patient with otherwise undetected postintervention contralateral OA predisposition. MovementRx detected elevated frontal plane moments of the patient's unaffected knee. The patient also revealed a surprising adverse compensation to the contralateral limb.

A Modified Prandtl-Ishlinskii Hysteresis Model for Modeling and Compensating Asymmetric Hysteresis of Piezo-Actuated Flexure-Based Systems.

Piezo-actuated flexure-based systems are widely used in applications with high accuracy requirements, but the intrinsic hysteresis has a detrimental effect on the performance which should be compensated. Conventional models were presented to model this undesired effect using additional dead-zone operators. This paper presents a new approach using two sets of operators with a distributed compensator to model and compensate for the asymmetric system hysteresis based on inversion calculation with a simplified digitized representation. The experimental results validate the effectiveness of the proposed model in modeling and compensating the asymmetric system hysteresis.

Current clinical practice in managing somatosensory impairments and the use of technology in stroke rehabilitation.

Stroke-induced somatosensory impairments seem to be clinically overlooked, despite their prevalence and influence on motor recovery post-stroke. Interest in technology has been gaining traction over the past few decades as a promising method to facilitate stroke rehabilitation. This questionnaire-based cross-sectional study aimed to identify current clinical practice and perspectives on the management of somatosensory impairments post-stroke and the use of technology in assessing outcome measures and providing intervention. Participants were 132 physiotherapists and occupational therapists currently working with stroke patients in public hospitals and rehabilitation centres in Singapore. It was found that the majority (64.4%) of the therapists spent no more than half of the time per week on somatosensory interventions. Functional or task-specific training was the primary form of intervention applied to retrain somatosensory functions in stroke survivors. Standardised assessments (43.2%) were used less frequently than non-standardised assessments (97.7%) in clinical practice, with the sensory subscale of the Fugl-Meyer Assessment being the most popular outcome measure, followed by the Nottingham Sensory Assessment. While the adoption of technology for assessment was relatively scarce, most therapists (87.1%) reported that they have integrated technology into intervention. There was a common agreement that proprioception is an essential component in stroke rehabilitation, and that robotic technology combined with conventional therapy is effective in enhancing stroke rehabilitation, particularly for retraining proprioception. Most therapists identified price, technology usability, and lack of available space as some of the biggest barriers to integrating robotic technology in stroke rehabilitation. Standardised assessments and interventions targeting somatosensory functions should be more clearly delineated in clinical guidelines. Although therapists were positive about technology-based rehabilitation, obstacles that make technology integration challenging ought to be addressed.

Identification of Secondary Biomechanical Abnormalities in the Lower Limb Joints after Chronic Transtibial Amputation: A Proof-of-Concept Study Using SPM1D Analysis.

SPM is a statistical method of analysis of time-varying human movement gait signal, depending on the random field theory (RFT). MovementRx is our inhouse-developed decision-support system that depends on SPM1D Python implementation of the SPM (spm1d.org). We present the potential application of MovementRx in the prediction of increased joint forces with the possibility to predispose to osteoarthritis in a sample of post-surgical Transtibial Amputation (TTA) patients who were ambulant in the community. We captured the three-dimensional movement profile of 12 males with TTA and studied them using MovementRx, employing the SPM1D Python library to quantify the deviation(s) they have from our corresponding reference data, using "Hotelling 2" and "T test 2" statistics for the 3D movement vectors of the 3 main lower limb joints (hip, knee, and ankle) and their nine respective components (3 joints × 3 dimensions), respectively. MovementRx results visually demonstrated a clear distinction in the biomechanical recordings between TTA patients and a reference set of normal people (ABILITY data project), and variability within the TTA patients' group enabled identification of those with an increased risk of developing osteoarthritis in the future. We conclude that MovementRx is a potential tool to detect increased specific joint forces with the ability to identify TTA survivors who may be at risk for osteoarthritis.

Autonomous modeling of repetitive movement for rehabilitation exercise monitoring.

BACKGROUND: Insightful feedback generation for daily home-based stroke rehabilitation is currently unavailable due to the inefficiency of exercise inspection done by therapists. We aim to produce a compact anomaly representation that allows a therapist to pay attention to only a few specific sections in a long exercise session record and boost their efficiency in feedback generation. METHODS: This study proposes a data-driven technique to model a repetitive exercise using unsupervised phase learning on an artificial neural network and statistical learning on principal component analysis (PCA). After a model is built on a set of normal healthy movements, the model can be used to extract a sequence of anomaly scores from a movement of the same prescription. RESULTS: The method not only works on a standard marker-based motion capture system but also performs well on a more compact and affordable motion capture system based-on Kinect V2 and wrist-worn inertial measurement units that can be used at home. An evaluation of four different exercises shows its potential in separating anomalous movements from normal ones with an average area under the curve (AUC) of 0.9872 even on the compact motion capture system. CONCLUSIONS: The proposed processing technique has the potential to help clinicians in providing high-quality feedback for telerehabilitation in a more scalable way.

Real-Time Foot Tracking and Gait Evaluation with Geometric Modeling.

Gait evaluation is important in gait rehabilitation and assistance to monitor patient's balance status and assess recovery performance. Recent technologies leverage on vision-based systems with high portability and low operational complexity. In this paper, we propose a new vision-based foot tracking algorithm specially catering to overground gait assistive devices, which often have limited view of the users. The algorithm models the foot and the shank of the user using simple geometry. Through cost optimization, it then aligns the models to the point cloud, showing the back view of the user's lower limbs. The system outputs the poses of the feet, which are used to compute the spatial-temporal gait parameters. Seven healthy young subjects are recruited to perform overground and treadmill walking trials. The results of the algorithm are compared with the motion capture system and a third-party gait analysis software. The algorithm has a fitting rotational and translational errors of less than 20 degrees and 33 mm, respectively, for 0.4 m/s walking speed. The gait detection F1 score achieves more than 96.8%. The step length and step width errors are around 35 mm, while the cycle time error is less than 38 ms. The proposed algorithm provides a fast, contactless, portable, and cost-effective gait evaluation method without requiring the user to wear any customized footwear.

Robotic-based ACTive somatoSENSory (Act.Sens) retraining on upper limb functions with chronic stroke survivors: study protocol for a pilot randomised controlled trial.

BACKGROUND: Prior studies have established that senses of the limb position in space (proprioception and kinaesthesia) are important for motor control and learning. Although nearly one-half of stroke patients have impairment in the ability to sense their movements, somatosensory retraining focusing on proprioception and kinaesthesia is often overlooked. Interventions that simultaneously target motor and somatosensory components are thought to be useful for relearning somatosensory functions while increasing mobility of the affected limb. For over a decade, robotic technology has been incorporated in stroke rehabilitation for more controlled therapy intensity, duration, and frequency. This pilot randomised controlled trial introduces a compact robotic-based upper-limb reaching task that retrains proprioception and kinaesthesia concurrently. METHODS: Thirty first-ever chronic stroke survivors (> 6-month post-stroke) will be randomly assigned to either a treatment or a control group. Over a 5-week period, the treatment group will receive 15 training sessions for about an hour per session. Robot-generated haptic guidance will be provided along the movement path as somatosensory cues while moving. Audio-visual feedback will appear following every successful movement as a reward. For the same duration, the control group will complete similar robotic training but without the vision occluded and robot-generated cues. Baseline, post-day 1, and post-day 30 assessments will be performed, where the last two sessions will be conducted after the last training session. Robotic-based performance indices and clinical assessments of upper limb functions after stroke will be used to acquire primary and secondary outcome measures respectively. This work will provide insights into the feasibility of such robot-assisted training clinically. DISCUSSION: The current work presents a study protocol to retrain upper-limb somatosensory and motor functions using robot-based rehabilitation for community-dwelling stroke survivors. The training promotes active use of the affected arm while at the same time enhances somatosensory input through augmented feedback. The outcomes of this study will provide preliminary data and help inform the clinicians on the feasibility and practicality of the proposed exercise. TRIAL REGISTRATION: ClinicalTrials.gov NCT04490655 . Registered 29 July 2020.

A Digitized Representation of the Modified Prandtl-Ishlinskii Hysteresis Model for Modeling and Compensating Piezoelectric Actuator Hysteresis.

Piezoelectric actuators are widely used in micromanipulation and miniature robots due to their rapid response and high repeatability. The piezoelectric actuators often have undesired hysteresis. The Prandtl-Ishlinskii (PI) hysteresis model is one of the most popular models for modeling and compensating the hysteresis behaviour. This paper presents an alternative digitized representation of the modified Prandtl-Ishlinskii with the dead-zone operators (MPI) hysteresis model to describe the asymmetric hysteresis behavior of piezoelectric actuators. Using a binary number with n digits to represent the classical Prandtl-Ishlinskii hysteresis model with n elementary operators, the inverse model can be easily constructed. A similar representation of the dead-zone operators is also described. With the proposed digitized representation, the model is more intuitive and the inversion calculation is avoided. An experiment with a piezoelectric stacked linear actuator is conducted to validate the proposed digitized MPI hysteresis model and it is shown that it has almost the same performance as compared to the classical representation.

Neurorehabilitation From a Distance: Can Intelligent Technology Support Decentralized Access to Quality Therapy?

Current neurorehabilitation models primarily rely on extended hospital stays and regular therapy sessions requiring close physical interactions between rehabilitation professionals and patients. The current COVID-19 pandemic has challenged this model, as strict physical distancing rules and a shift in the allocation of hospital resources resulted in many neurological patients not receiving essential therapy. Accordingly, a recent survey revealed that the majority of European healthcare professionals involved in stroke care are concerned that this lack of care will have a noticeable negative impact on functional outcomes. COVID-19 highlights an urgent need to rethink conventional neurorehabilitation and develop alternative approaches to provide high-quality therapy while minimizing hospital stays and visits. Technology-based solutions, such as, robotics bear high potential to enable such a paradigm shift. While robot-assisted therapy is already established in clinics, the future challenge is to enable physically assisted therapy and assessments in a minimally supervized and decentralized manner, ideally at the patient's home. Key enablers are new rehabilitation devices that are portable, scalable and equipped with clinical intelligence, remote monitoring and coaching capabilities. In this perspective article, we discuss clinical and technological requirements for the development and deployment of minimally supervized, robot-assisted neurorehabilitation technologies in patient's homes. We elaborate on key principles to ensure feasibility and acceptance, and on how artificial intelligence can be leveraged for embedding clinical knowledge for safe use and personalized therapy adaptation. Such new models are likely to impact neurorehabilitation beyond COVID-19, by providing broad access to sustained, high-quality and high-dose therapy maximizing long-term functional outcomes.

Fall inducing movable platform (FIMP) for overground trips and slips.

BACKGROUND: The study of falls and fall prevention/intervention devices requires the recording of true falls incidence. However, true falls are rare, random, and difficult to collect in real world settings. A system capable of producing falls in an ecologically valid manner will be very helpful in collecting the data necessary to advance our understanding of the neuro and musculoskeletal mechanisms underpinning real-world falls events. METHODS: A fall inducing movable platform (FIMP) was designed to arrest or accelerate a subject's ankle to induce a trip or slip. The ankle was arrested posteriorly with an electromagnetic brake and accelerated anteriorly with a motor. A power spring was connected in series between the ankle and the brake/motor to allow freedom of movement (system transparency) when a fall is not being induced. A gait phase detection algorithm was also created to enable precise activation of the fall inducing mechanisms. Statistical Parametric Mapping (SPM1D) and one-way repeated measure ANOVA were used to evaluate the ability of the FIMP to induce a trip or slip. RESULTS: During FIMP induced trips, the brake activates at the terminal swing or mid swing gait phase to induce the lowering or skipping strategies, respectively. For the lowering strategy, the characteristic leg lowering and subsequent contralateral leg swing was seen in all subjects. Likewise, for the skipping strategy, all subjects skipped forward on the perturbed leg. Slip was induced by FIMP by using a motor to impart unwanted forward acceleration to the ankle with the help of friction-reducing ground sliding sheets. Joint stiffening was observed during the slips, and subjects universally adopted the surfing strategy after the initial slip. CONCLUSION: The results indicate that FIMP can induce ecologically valid falls under controlled laboratory conditions. The use of SPM1D in conjunction with FIMP allows for the time varying statistical quantification of trip and slip reactive kinematics events. With future research, fall recovery anomalies in subjects can now also be systematically evaluated through the assessment of other neuromuscular variables such as joint forces, muscle activation and muscle forces.

An Asian-centric human movement database capturing activities of daily living.

Assessment of human movement performance in activities of daily living (ADL) is a key component in clinical and rehabilitation settings. Motion capture technology is an effective method for objective assessment of human movement. Existing databases capture human movement and ADL performance primarily in the Western population, and there are no Asian databases to date. This is despite the fact that Asian anthropometrics influence movement kinematics and kinetics. This paper details the protocol in the first phase of the largest Asian normative human movement database. Data collection has commenced, and this paper reports 10 healthy participants. Twelve tasks were performed and data was collected using Qualisys motion capture system, force plates and instrumented table and chair. In phase two, human movement of individuals with stroke and knee osteoarthritis will be captured. This can have great potential for benchmarking with the normative human movement captured in phase one and predicting recovery and progression of movement for patients. With individualised progression, it will offer the development of personalised therapy protocols in rehabilitation.

Regenerative rehabilitation: exploring the synergistic effects of rehabilitation and implantation of a bio-functional scaffold in enhancing nerve regeneration.

Clinically, rehabilitation is one of the most common treatment options for traumatic injuries. Despite that, recovery remains suboptimal and recent breakthroughs in regenerative approaches may potentially improve clinical outcomes. To date, there have been numerous studies on the utilization of either rehabilitative or regenerative strategies for traumatic injury treatment. However, studies that document the combined effects of rehabilitation and regenerative tissue engineering options remain scarce. Here, in the context of traumatic nerve injury treatment, we use a rat spinal cord injury (SCI) model as a proof of concept to evaluate the synergistic effects of regenerative tissue engineering and rehabilitation. Specifically, we implanted a pro-regenerative hybrid fiber-hydrogel scaffold and subjected SCI rats to intensive rehabilitation. Of note, the rehabilitation session was augmented by a novel customized training device that imparts normal hindlimb gait movements to rats. Morphologically, more regenerated axons were observed when rats received rehabilitation (∼2.5 times and ∼2 times enhancement after 4 and 12 weeks of recovery, respectively, p < 0.05). Besides that, we also observed a higher percentage of anti-inflammatory cells (36.1 ± 12.9% in rehab rats vs. 3.31 ± 1.48% in non-rehab rats, p < 0.05) and perineuronal net formation in rehab rats at Week 4. Physically, rehab animals were also able to exert higher ankle flexion force (∼0.779 N vs. ∼0.495 N at Week 4 and ∼1.36 N vs. ∼0.647 N at Week 12 for rehab vs. non-rehab rats, p < 0.001) and performed better than non-rehab rats in the open field test. Taken together, we conclude that coupling rehabilitation with regenerative scaffold implantation strategies can further promote functional recovery after traumatic nerve injuries.

Application of Signed Distance Function Neural Network in Real-Time Feet Tracking.

Loss of locomotion is one major problem faced by the elderly currently. Various rehabilitation technologies have been developed to assist them in recovering walking capability. Gait monitoring is an important aspect of lower-limb function rehabilitation. By observing the walking behaviour of a patient, the stability and recovery progress can be evaluated. Despite gait is often measured by motion capture and force-based measurement, these methods are costly and non-portable, whereas inertial measurement units (IMUs) require the attachment of sensors onto the subject's body. A few contactless measurements have been proposed, however, none of them views the feet from the back, making it non-trivial to transfer the method to over ground rehabilitation robots. This paper proposes a method to track the poses of the feet in real time using a novel deep neural network, termed SDF-Net, that models the signed distance function (SDF) of an object. Independent of the viewing angle, the algorithm receives the colour and depth images of the feet as input and computes the pose of the feet. The tracking accuracy is evaluated by having a subject to perform various actions with the feet; the dynamic errors are found to be less than 9 mm and 8 degrees for position and orientation errors respectively, which are better than the state-of-arts reviewed.