Evaluating the Accuracy of the Azure Kinect and Kinect v2.

The Azure Kinect represents the latest generation of Microsoft Kinect depth cameras. Of interest in this article is the depth and spatial accuracy of the Azure Kinect and how it compares to its predecessor, the Kinect v2. In one experiment, the two sensors are used to capture a planar whiteboard at 15 locations in a grid pattern with laser scanner data serving as ground truth. A set of histograms reveals the temporal-based random depth error inherent in each Kinect. Additionally, a two-dimensional cone of accuracy illustrates the systematic spatial error. At distances greater than 2.5 m, we find the Azure Kinect to have improved accuracy in both spatial and temporal domains as compared to the Kinect v2, while for distances less than 2.5 m, the spatial and temporal accuracies were found to be comparable. In another experiment, we compare the distribution of random depth error between each Kinect sensor by capturing a flat wall across the field of view in horizontal and vertical directions. We find the Azure Kinect to have improved temporal accuracy over the Kinect v2 in the range of 2.5 to 3.5 m for measurements close to the optical axis. The results indicate that the Azure Kinect is a suitable substitute for Kinect v2 in 3D scanning applications.

Motion sensor-acquired reachable workspace correlates with patient-reported upper extremity activities of daily living (ADL) function in facioscapulohumeral dystrophy.

BACKGROUND: This study examines the correlation, and clinical meaningfulness, between reachable workspace outcome and reported activities of daily living (ADL) function of individuals with facioscapulohumeral dystrophy (FSHD). METHODS: Twenty-one FSHD subjects with various disease severity (clinical severity scores 1-4) underwent reachable workspace evaluation and completed the Quality of Life in Neurological Disorders (NeuroQoL) upper extremity questionnaire. Spearman and receiver operator curve analyses were performed. RESULTS: Moderate correlation was found between NeuroQoL scores and total (ρ = 0.7609; P < .01), and upper-quadrants relative surface areas (RSAs) (ρ = 0.6969; P < .01). Five specific items (ie, shirt on, shirt off, use spoon, pull on pants, pick-up clothes) demonstrated even higher correlations with total (ρ = 0.8397; P < .01) and above shoulder (ρ = 0.8082; P < .01) RSAs. A total RSA cuffoff value of 0.70 would achieve 100% sensitivity and 94% specificity (area under the curve = 0.975). CONCLUSIONS: Reachable workspace values identify when individuals have difficulties performing ADLs at home. This information improves patient monitoring, and clinical decision making by enabling more timely recommendations for medications, assistive devices, or considerations for clinical trial enrollments.

Usefulness of Kinect sensor-based reachable workspace system for assessing upper extremity dysfunction in breast cancer patients.

PURPOSE: Recently, the utility of the Kinect sensor-based reachable workspace analysis system for measuring upper extremity outcomes of neuromuscular and musculoskeletal diseases has been demonstrated. Here, we investigated its usefulness for assessing upper extremity dysfunction in breast cancer patients. METHODS: Twenty unilateral breast cancer patients were enrolled. Upper extremity active range of motion was captured by the Kinect sensor, and reachable workspace relative surface areas (RSAs) were obtained. The QuickDASH was completed to assess upper extremity disability. General and breast cancer-specific quality of life (QOL) were assessed by the EORTC QLQ-C30 and EORTC QLQ-BR23. RESULTS: The total RSA ratio of the affected and unaffected sides ranges from 0.64 to 1.11. Total RSA was significantly reduced on the affected versus unaffected side (0.659 ± 0.105 vs. 0.762 ± 0.065; p = 0.001). Quadrant 1 and 3 RSAs were significantly reduced (0.135 ± 0.039 vs. 0.183 ± 0.040, p < 0.001; 0.172 ± 0.058 vs. 0.217 ± 0.031, p = 0.006). Total RSA of the affected side was strongly correlated with the numeric pain rating scale during movement (r = - 0.812, p < 0.001) and moderately with the QuickDASH (r = - 0.494, p = 0.027). Further, quadrant 3 RSA was correlated with EORTC QLQ-C30 role functioning (r = 0.576, p = 0.008) and EORTC QLQ-BR23 arm symptoms (r = - 0.588, p = 0.006) scales. CONCLUSIONS: The Kinect sensor-based reachable workspace analysis system was effectively applied to assess upper extremity dysfunction in breast cancer patients. This system could potentially serve as a quick and simple outcome measure that provides quantitative data for breast cancer patients.

The Feasibility and Usability of RunningCoach: A Remote Coaching System for Long-Distance Runners.

Studies have shown that about half of the injuries sustained during long-distance running involve the knee. Cadence (steps per minute) has been identified as a factor that is strongly associated with these running-related injuries, making it a worthwhile candidate for further study. As such, it is critical for long-distance runners to minimize their risk of injury by running at an appropriate running cadence. In this paper, we present the results of a study on the feasibility and usability of RunningCoach, a mobile health (mHealth) system that remotely monitors running cadence levels of runners in a continuous fashion, among other variables, and provides immediate feedback to runners in an effort to help them optimize their running cadence.

Upper extremity 3-dimensional reachable workspace assessment in amyotrophic lateral sclerosis by Kinect sensor.

INTRODUCTION: Reachable workspace is a measure that provides clinically meaningful information regarding arm function. In this study, a Kinect sensor was used to determine the spectrum of 3-dimensional reachable workspace encountered in a cross-sectional cohort of individuals with amyotrophic lateral sclerosis (ALS). METHODS: Bilateral 3D reachable workspace was recorded from 10 subjects with ALS and 17 healthy controls. The data were normalized by each individual's arm length to obtain a reachable workspace relative surface area (RSA). Concurrent validity was assessed by correlation with scoring on the ALS Functional Rating Score-revised (ALSFRSr). RESULTS: The Kinect-measured reachable workspace RSA differed significantly between the ALS and control subjects (0.579 ± 0.226 vs. 0.786 ± 0.069; P < 0.001). The RSA demonstrated correlation with ALSFRSr upper extremity items (Spearman correlation ρ = 0.569; P = 0.009). With worsening upper extremity function, as categorized by the ALSFRSr, the reachable workspace also decreased progressively. CONCLUSIONS: This study demonstrates the feasibility and potential of using a novel Kinect-based reachable workspace outcome measure in ALS.

Reachable workspace and performance of upper limb (PUL) in duchenne muscular dystrophy.

INTRODUCTION: The Kinect-based reachable workspace relative surface area (RSA) is compared with the performance of upper limb (PUL) assessment in Duchenne muscular dystrophy (DMD). METHODS: 29 individuals with DMD (ages: 7-23; Brooke: 1-5) underwent both Kinect-based reachable workspace RSA and PUL assessments. RSAs were also collected from 24 age-matched controls. Total and quadrant RSAs were compared with the PUL total, shoulder-, middle-, and distal-dimension scores. RESULTS: The total reachable workspace RSA correlated well with the total PUL score (Spearman ρ = -0.602; P < 0.001), and with each of the PUL dimensional scores: shoulder (ρ = -0.624; P < 0.001), middle (ρ = -0.564; P = 0.001), and distal (ρ = -0.630; P < 0.001). With quadrant RSA, reachability in a particular quadrant was closely associated with respective PUL dimensional-level function (lateral-upper quadrant for shoulder-, lateral-upper/lower quadrants for middle-, and lateral-lower quadrant for distal-level function). CONCLUSIONS: This study demonstrates concurrent validity of the reachable workspace outcome measure (RSA) with the DMD-specific upper extremity outcome measure (PUL).

Reachable workspace reflects dynamometer-measured upper extremity strength in facioscapulohumeral muscular dystrophy.

INTRODUCTION: It is not known whether a reduction in reachable workspace closely reflects loss of upper extremity strength in facioscapulohumeral muscular dystrophy (FSHD). In this study we aimed to determine the relationship between reachable workspace and quantitative upper extremity strength measures. METHODS: Maximal voluntary isometric contraction (MVIC) testing of bilateral elbow flexion and shoulder abduction by hand-held dynamometry was performed on 26 FSHD and 27 control subjects. In addition, Kinect sensor-based 3D reachable workspace relative surface areas (RSAs) were obtained. Loading (500-g weight) effects on reachable workspace were also evaluated. RESULTS: Quantitative upper extremity strength (MVIC of elbow flexion and shoulder abduction) correlated with Kinect-acquired reachable workspace RSA (R = 0.477 for FSHD, P = 0.0003; R = 0.675 for the combined study cohort, P < 0.0001). Progressive reduction in RSA reflected worsening MVIC measures. Loading impacted the moderately weak individuals the most with additional reductions in RSA. CONCLUSIONS: Reachable workspace outcome measure is reflective of upper extremity strength impairment in FSHD.

Upper extremity 3-dimensional reachable workspace analysis in dystrophinopathy using Kinect.

INTRODUCTION: An innovative upper extremity 3-dimensional (3D) reachable workspace outcome measure acquired using the Kinect sensor is applied toward Duchenne/Becker muscular dystrophy (DMD/BMD). The validity, sensitivity, and clinical meaningfulness of this novel outcome measure are examined. METHODS: Upper extremity function assessment (Brooke scale and NeuroQOL questionnaire) and Kinect-based reachable workspace analyses were conducted in 43 individuals with dystrophinopathy (30 DMD and 13 BMD, aged 7-60 years) and 46 controls (aged 6-68 years). RESULTS: The reachable workspace measure reliably captured a wide range of upper extremity impairments encountered in both pediatric and adult, as well as ambulatory and non-ambulatory individuals with dystrophinopathy. Reduced reachable workspaces were noted for the dystrophinopathy cohort compared with controls, and they correlated with Brooke grades. In addition, progressive reduction in reachable workspace correlated directly with worsening ability to perform activities of daily living, as self-reported on the NeuroQOL. CONCLUSION: This study demonstrates the utility and potential of the novel sensor-acquired reachable workspace outcome measure in dystrophinopathy.

Reachable workspace in facioscapulohumeral muscular dystrophy (FSHD) by Kinect.

INTRODUCTION: A depth-ranging sensor (Kinect) based upper extremity motion analysis system was applied to determine the spectrum of reachable workspace encountered in facioscapulohumeral muscular dystrophy (FSHD). METHODS: Reachable workspaces were obtained from 22 individuals with FSHD and 24 age- and height-matched healthy controls. To allow comparison, total and quadrant reachable workspace relative surface areas (RSAs) were obtained by normalizing the acquired reachable workspace by each individual's arm length. RESULTS: Significantly contracted reachable workspace and reduced RSAs were noted for the FSHD cohort compared with controls (0.473 ± 0.188 vs. 0.747 ± 0.082; P < 0.0001). With worsening upper extremity function as categorized by the FSHD evaluation subscale II + III, the upper quadrant RSAs decreased progressively, while the lower quadrant RSAs were relatively preserved. There were no side-to-side differences in reachable workspace based on hand-dominance. CONCLUSIONS: This study demonstrates the feasibility and potential of using an innovative Kinect-based reachable workspace outcome measure in FSHD.

Recognizing the intensity of strength training exercises with wearable sensors.

In this paper we propose a system based on a network of wearable accelerometers and an off-the-shelf smartphone to recognize the intensity of stationary activities, such as strength training exercises. The system uses a hierarchical algorithm, consisting of two layers of Support Vector Machines (SVMs), to first recognize the type of exercise being performed, followed by recognition of exercise intensity. The first layer uses a single SVM to recognize the type of the performed exercise. Based on the recognized type a corresponding intensity prediction SVM is selected on the second layer, specializing in intensity prediction for the recognized type of exercise. We evaluate the system for a set of upper-body exercises using different weight loads. Additionally, we compare the most important features for exercise and intensity recognition tasks and investigate how different sliding window combinations, sensor configurations and number of training subjects impact the algorithm performance. We perform all of the experiments for two different types of features to evaluate the feasibility of implementation on resource constrained hardware. The results show the algorithm is able to recognize exercise types with approximately 85% accuracy and 6% intensity prediction error. Furthermore, due to similar performance using different types of features, the algorithm offers potential for implementation on resource constrained hardware.

Architecture of an automated coaching system for elderly population.

We present an automated coaching system for elderly population living in assisted homes. The system guides its users through a sequence of exercises and tests. Each exercise is demonstrated by a pre-recorded video of a coach, checked for correct execution and qualitatively evaluated. Automatic coaching advices are generated in order to improve the execution. Performance measurements are shown as an immediate feedback to the user, and stored and evaluated over time. The system is designed to allow for a remote interaction with a coach, and, to bolster social aspect of the exercise, for concurrent exercise of two (or eventually multiple) remote users.

Upper extremity reachable workspace evaluation with Kinect.

We propose a novel low-cost method for quantitative assessment of upper extremity workspace envelope using Microsoft Kinect camera. In clinical environment there are currently no practical and cost-effective methods available to provide arm-function evaluation in three-dimensional space. In this paper we examine the accuracy of the proposed technique for workspace estimation using Kinect in comparison with a motion capture system. The experimental results show that the developed system is capable of capturing the workspace with sufficient accuracy and robustness.

mHealth application for upper extremity range of motion and reachable workspace.

We present mobile health (mHealth) applications utilizing embedded phone sensors as an angle-measuring device for upper-limb range of motion (ROM) and estimation of reachable workspace to assist in evaluation of upper limb functional capacity. Our results show that the phone can record accurate measurements, as well as provide additional functionalities for clinicians.

Real-time human pose detection and tracking for tele-rehabilitation in virtual reality.

We present a real-time algorithm for human pose detection and tracking from vision-based 3D data and its application to tele-rehabilitation in virtual environments. We employ stereo camera(s) to capture 3D avatars of geographically dislocated patient and therapist in real-time, while sending the data remotely and displaying it in a virtual scene. A pose detection and tracking algorithm extracts kinematic parameters from each participant and determines pose similarity. The pose similarity score is used to quantify patient's performance and provide real-time feedback for remote rehabilitation.

Use of the Kinect sensor measured three-dimensional reachable workspace to assess the upper extremity function in older adults.

BACKGROUND: We explored the utility of Kinect sensor-based upper extremity reachable workspace measure in healthy adults aged over 65 years. METHODS: Forty-three healthy older subjects (19 men and 24 women) aged over 65 years and 22 healthy young subjects (11 men and 11 women) were included. All participants were ambulatory and perform the activities of daily living independently. Three-dimensional reachable workspace data were acquired for both arms using the Kinect sensor. We evaluated hand grip strength, manual muscle shoulder strength, and the active shoulder ranges of motion of the dominant and non-dominant sides. We assessed upper limb function using the Disabilities of Arm, Shoulder, and Hand (DASH) instrument and the health-related quality of life employing the descriptive EQ-5D-5L system. FINDINGS: The quadrant 3 relative surface area in older adults was significantly smaller than that of young adults (both dominant and non-dominant sides), while the total and quadrants 1, 2, and 4 relative surface areas did not differ between older and young adults. However, the quadrant 3 relative surface area did not correlate with the DASH or EQ5D scores. The total and quadrant 1, 2, and 4 relative surface areas of the dominant side significantly correlated with the DASH score. The quadrant 4 relative surface area of the dominant side significantly correlated with the EQ5D score. INTERPRETATION: Kinect sensor-based, three-dimensional, reachable workspace analysis may be useful to evaluate upper limb function in older adults.

The Black Box of Technological Outcome Measures: An Example in Duchenne Muscular Dystrophy.

BACKGROUND: Outcome measures for non-ambulant Duchenne muscular dystrophy (DMD) patients are limited, with only the Performance of the Upper Limb (PUL) approved as endpoint for clinical trials. OBJECTIVE: We assessed four outcome measures based on devices developed for the gaming industry, aiming to overcome disadvantages of observer-dependency and motivation. METHODS: Twenty-two non-ambulant DMD patients (range 8.6-24.1 years) and 14 healthy controls (HC; range 9.5-25.4 years) were studied at baseline and 16 patients at 12 months using Leap Motion to quantify wrist/hand active range of motion (aROM) and a Kinect sensor for reached volume with Ability Captured Through Interactive Video Evaluation (ACTIVE), Functional Workspace (FWS) summed distance to seven upper extremity body points, and trunk compensation (KinectTC). PUL 2.0 was performed in patients only. A stepwise approach assessed quality control, construct validity, reliability, concurrent validity, longitudinal change and patient perception. RESULTS: Leap Motion aROM distinguished patients and HCs for supination, radial deviation and wrist flexion (range p = 0.006 to <0.001). Reliability was low and the manufacturer's hand model did not match the sensor's depth images. ACTIVE differed between patients and HCs (p < 0.001), correlated with PUL (rho = 0.76), and decreased over time (p = 0.030) with a standardized response mean (SRM) of -0.61. It was appraised as fun on a 10-point numeric rating scale (median 9/10). PUL decreased over time (p < 0.001) with an SRM of -1.28, and was appraised as fun (median 7/10). FWS summed distance distinguished patients and HCs (p < 0.001), but reliability in patients was insufficient. KinectTC differed between patients and HCs (p < 0.01), but correlated insufficiently with PUL (rho = -0.69). CONCLUSIONS: Only ACTIVE qualified as potential outcome measure in non-ambulant DMD patients, although the SRM was below the commonly used threshold of 0.8. Lack of insight in technological constraints due to intellectual property and software updates made the technology behind these outcome measures a kind of black box that could jeopardize long-term use in clinical development.

Real-time 3D avatars for tele-rehabilitation in virtual reality.

We present work in progress on a tele-immersion system for telerehabilitation using real-time stereo vision and virtual environments. Stereo reconstruction is used to capture user's 3D avatar in real time and project it into a shared virtual environment, enabling a patient and therapist to interact remotely. Captured data can also be used to analyze the movement and provide feedback to the patient as we present in a preliminary study of stepping-in-place task. Such tele-presence system could in the future allow patients to interact remotely with remote physical therapist and virtual environment while objectively tracking their performance.

Reachable Workspace and Proximal Function Measures for Quantifying Upper Limb Motion.

There are a lack of quantitative measures for clinically assessing upper limb function. Conventional biomechanical performance measures are restricted to specialist labs due to hardware cost and complexity, while the resulting measurements require specialists for analysis. Depth cameras are low cost and portable systems that can track surrogate joint positions. However, these motions may not be biologically consistent, which can result in noisy, inaccurate movements. This paper introduces a rigid body modelling method to enforce biological feasibility of the recovered motions. This method is evaluated on an existing depth camera assessment: the reachable workspace (RW) measure for assessing gross shoulder function. As a rigid body model is used, position estimates of new proximal targets can be added, resulting in a proximal function (PF) measure for assessing a subject's ability to touch specific body landmarks. The accuracy, and repeatability of these measures is assessed on ten asymptomatic subjects, with and without rigid body constraints. This analysis is performed both on a low-cost depth camera system and a gold-standard active motion capture system. The addition of rigid body constraints was found to improve accuracy and concordance of the depth camera system, particularly in lateral reaching movements. Both RW and PF measures were found to be feasible candidates for clinical assessment, with future analysis needed to determine their ability to detect changes within specific patient populations.

Upper Limb Three-Dimensional Reachable Workspace Analysis Using the Kinect Sensor in Hemiplegic Stroke Patients: A Cross-Sectional Observational Study.

OBJECTIVE: A reachable workspace evaluation using the Kinect sensor was previously introduced as a novel upper limb outcome measure in neuromuscular and musculoskeletal conditions. This study investigated its usefulness in hemiplegic stroke patients. DESIGN: Forty-one patients with hemiplegic stroke were included. Kinect-based reachable workspace analysis was performed on both paretic and nonparetic sides. Upper limb impairment was measured using the Fugl-Meyer Assessment and the Motricity Index on the paretic side. Disability was assessed using the shortened Disabilities of the Arm, Shoulder, and Hand questionnaire. Correlations between the relative surface areas, impairment scores, and disability were analyzed. RESULTS: Quadrants 1, 3, and 4 as well as the total relative surface area of the paretic side were significantly reduced compared with the nonparetic side. The total relative surface area of the paretic side correlated with the Fugl-Meyer Assessment scores, the Motricity Index for Upper Extremity, and the Disabilities of the Arm, Shoulder, and Hand questionnaire score. Furthermore, quadrant 3 was the most important determinant of upper limb impairment and disability. CONCLUSIONS: A reachable workspace (a sensor-based measure that can be obtained relatively quickly and unobtrusively) could be a useful and alternative outcome measure for upper limb in hemiplegic stroke patients.

Teleimmersive environment for remote medical collaboration.

We present the work in progress on teleimmersive framework that would allow doctors to collaborate in more natural way by being immersed in the virtual space along with the medical data they could examine and discuss with remotely located colleagues. The system consists of multiple cameras, hand tracking with gesture recognition, and virtual reality system to support volume visualization and 3D interaction with medical data.