Simulation of Human Movement in Zero Gravity.

In the era of expanding manned space missions, understanding the biomechanical impacts of zero gravity on human movement is pivotal. This study introduces a novel and cost-effective framework that demonstrates the application of Microsoft's Azure Kinect body tracking technology as a motion input generator for subsequent OpenSim simulations in weightlessness. Testing rotations, locomotion, coordination, and martial arts movements, we validate the results' realism under the constraints of angular and linear momentum conservation. While complex, full-body coordination tasks face limitations in a zero gravity environment, our findings suggest possible approaches to device-free exercise routines for astronauts and reveal insights into the feasibility of hand-to-hand combat in space. However, some challenges remain in distinguishing zero gravity effects in the simulations from discrepancies in the captured motion input or forward dynamics calculations, making a comprehensive validation difficult. The paper concludes by highlighting the framework's practical potential for the future of space mission planning and related research endeavors, while also providing recommendations for further refinement.

How the Processing Mode Influences Azure Kinect Body Tracking Results.

The Azure Kinect DK is an RGB-D-camera popular in research and studies with humans. For good scientific practice, it is relevant that Azure Kinect yields consistent and reproducible results. We noticed the yielded results were inconsistent. Therefore, we examined 100 body tracking runs per processing mode provided by the Azure Kinect Body Tracking SDK on two different computers using a prerecorded video. We compared those runs with respect to spatiotemporal progression (spatial distribution of joint positions per processing mode and run), derived parameters (bone length), and differences between the computers. We found a previously undocumented converging behavior of joint positions at the start of the body tracking. Euclidean distances of joint positions varied clinically relevantly with up to 87 mm between runs for CUDA and TensorRT; CPU and DirectML had no differences on the same computer. Additionally, we found noticeable differences between two computers. Therefore, we recommend choosing the processing mode carefully, reporting the processing mode, and performing all analyses on the same computer to ensure reproducible results when using Azure Kinect and its body tracking in research. Consequently, results from previous studies with Azure Kinect should be reevaluated, and until then, their findings should be interpreted with caution.

A Serious Game for the Assessment of Visuomotor Adaptation Capabilities during Locomotion Tasks Employing an Embodied Avatar in Virtual Reality.

The study of visuomotor adaptation (VMA) capabilities has been encompassed in various experimental protocols aimed at investigating human motor control strategies and/or cognitive functions. VMA-oriented frameworks can have clinical applications, primarily in the investigation and assessment of neuromotor impairments caused by conditions such as Parkinson's disease or post-stroke, which affect the lives of tens of thousands of people worldwide. Therefore, they can enhance the understanding of the specific mechanisms of such neuromotor disorders, thus being a potential biomarker for recovery, with the aim of being integrated with conventional rehabilitative programs. Virtual Reality (VR) can be entailed in a framework targeting VMA since it allows the development of visual perturbations in a more customizable and realistic way. Moreover, as has been demonstrated in previous works, a serious game (SG) can further increase engagement thanks to the use of full-body embodied avatars. Most studies implementing VMA frameworks have focused on upper limb tasks and have utilized a cursor as visual feedback for the user. Hence, there is a paucity in the literature about VMA-oriented frameworks targeting locomotion tasks. In this article, the authors present the design, development, and testing of an SG-based framework that addresses VMA in a locomotion activity by controlling a full-body moving avatar in a custom VR environment. This workflow includes a set of metrics to quantitatively assess the participants' performance. Thirteen healthy children were recruited to evaluate the framework. Several quantitative comparisons and analyses were run to validate the different types of introduced visuomotor perturbations and to evaluate the ability of the proposed metrics to describe the difficulty caused by such perturbations. During the experimental sessions, it emerged that the system is safe, easy to use, and practical in a clinical setting. Despite the limited sample size, which represents the main limitation of the study and can be compensated for with future recruitment, the authors claim the potential of this framework as a useful instrument for quantitatively assessing either motor or cognitive impairments. The proposed feature-based approach gives several objective parameters as additional biomarkers that can integrate the conventional clinical scores. Future studies might investigate the relation between the proposed biomarkers and the clinical scores for specific disorders such as Parkinson's disease and cerebral palsy.

Assessment Tasks and Virtual Exergames for Remote Monitoring of Parkinson's Disease: An Integrated Approach Based on Azure Kinect.

Motor impairments are among the most relevant, evident, and disabling symptoms of Parkinson's disease that adversely affect quality of life, resulting in limited autonomy, independence, and safety. Recent studies have demonstrated the benefits of physiotherapy and rehabilitation programs specifically targeted to the needs of Parkinsonian patients in supporting drug treatments and improving motor control and coordination. However, due to the expected increase in patients in the coming years, traditional rehabilitation pathways in healthcare facilities could become unsustainable. Consequently, new strategies are needed, in which technologies play a key role in enabling more frequent, comprehensive, and out-of-hospital follow-up. The paper proposes a vision-based solution using the new Azure Kinect DK sensor to implement an integrated approach for remote assessment, monitoring, and rehabilitation of Parkinsonian patients, exploiting non-invasive 3D tracking of body movements to objectively and automatically characterize both standard evaluative motor tasks and virtual exergames. An experimental test involving 20 parkinsonian subjects and 15 healthy controls was organized. Preliminary results show the system's ability to quantify specific and statistically significant (p < 0.05) features of motor performance, easily monitor changes as the disease progresses over time, and at the same time permit the use of exergames in virtual reality both for training and as a support for motor condition assessment (for example, detecting an average reduction in arm swing asymmetry of about 14% after arm training). The main innovation relies precisely on the integration of evaluative and rehabilitative aspects, which could be used as a closed loop to design new protocols for remote management of patients tailored to their actual conditions.

RGB-D-Based Method for Measuring the Angular Range of Hip and Knee Joints during Home Care Rehabilitation.

Home-based rehabilitation is becoming a gold standard for patient who have undergone knee arthroplasty or full knee replacement, as it helps healthcare costs to be minimized. Nevertheless, there is a chance of increasing adverse health effects in case of home care, primarily due to the patients' lack of motivation and the doctors' difficulty in carrying out rigorous supervision. The development of devices to assess the efficient recovery of the operated joint is highly valued both for the patient, who feels encouraged to perform the proper number of activities, and for the doctor, who can track him/her remotely. Accordingly, this paper introduces an interactive approach to angular range calculation of hip and knee joints based on the use of low-cost devices which can be operated at home. First, the patient's body posture is estimated using a 2D acquisition method. Subsequently, the 3D posture is evaluated by using the depth information coming from an RGB-D sensor. Preliminary results show that the proposed method effectively overcomes many limitations by fusing the results obtained by the state-of-the-art robust 2D pose estimation algorithms with the 3D data of depth cameras by allowing the patient to be correctly tracked during rehabilitation exercises.

Custom IMU-Based Wearable System for Robust 2.4 GHz Wireless Human Body Parts Orientation Tracking and 3D Movement Visualization on an Avatar.

Recent studies confirm the applicability of Inertial Measurement Unit (IMU)-based systems for human motion analysis. Notwithstanding, high-end IMU-based commercial solutions are yet too expensive and complex to democratize their use among a wide range of potential users. Less featured entry-level commercial solutions are being introduced in the market, trying to fill this gap, but still present some limitations that need to be overcome. At the same time, there is a growing number of scientific papers using not commercial, but custom do-it-yourself IMU-based systems in medical and sports applications. Even though these solutions can help to popularize the use of this technology, they have more limited features and the description on how to design and build them from scratch is yet too scarce in the literature. The aim of this work is two-fold: (1) Proving the feasibility of building an affordable custom solution aimed at simultaneous multiple body parts orientation tracking; while providing a detailed bottom-up description of the required hardware, tools, and mathematical operations to estimate and represent 3D movement in real-time. (2) Showing how the introduction of a custom 2.4 GHz communication protocol including a channel hopping strategy can address some of the current communication limitations of entry-level commercial solutions. The proposed system can be used for wireless real-time human body parts orientation tracking with up to 10 custom sensors, at least at 50 Hz. In addition, it provides a more reliable motion data acquisition in Bluetooth and Wi-Fi crowded environments, where the use of entry-level commercial solutions might be unfeasible. This system can be used as a groundwork for developing affordable human motion analysis solutions that do not require an accurate kinematic analysis.

A Body Tracking-Based Low-Cost Solution for Monitoring Workers' Hygiene Best Practices during Pandemics.

Since its beginning at the end of 2019, the pandemic spread of the severe acute respiratory syndrome coronavirus 2 (Sars-CoV-2) caused more than one million deaths in only nine months. The threat of emerging and re-emerging infectious diseases exists as an imminent threat to human health. It is essential to implement adequate hygiene best practices to break the contagion chain and enhance society preparedness for such critical scenarios and understand the relevance of each disease transmission route. As the unconscious hand-face contact gesture constitutes a potential pathway of contagion, in this paper, the authors present a prototype system based on low-cost depth sensors able to monitor in real-time the attitude towards such a habit. The system records people's behavior to enhance their awareness by providing real-time warnings, providing for statistical reports for designing proper hygiene solutions, and better understanding the role of such route of contagion. A preliminary validation study measured an overall accuracy of 91%. A Cohen's Kappa equal to 0.876 supports rejecting the hypothesis that such accuracy is accidental. Low-cost body tracking technologies can effectively support monitoring compliance with hygiene best practices and training people in real-time. By collecting data and analyzing them with respect to people categories and contagion statistics, it could be possible to understand the importance of this contagion pathway and identify for which people category such a behavioral attitude constitutes a significant risk.

On Inertial Body Tracking in the Presence of Model Calibration Errors.

In inertial body tracking, the human body is commonly represented as a biomechanical model consisting of rigid segments with known lengths and connecting joints. The model state is then estimated via sensor fusion methods based on data from attached inertial measurement units (IMUs). This requires the relative poses of the IMUs w.r.t. the segments-the IMU-to-segment calibrations, subsequently called I2S calibrations-to be known. Since calibration methods based on static poses, movements and manual measurements are still the most widely used, potentially large human-induced calibration errors have to be expected. This work compares three newly developed/adapted extended Kalman filter (EKF) and optimization-based sensor fusion methods with an existing EKF-based method w.r.t. their segment orientation estimation accuracy in the presence of model calibration errors with and without using magnetometer information. While the existing EKF-based method uses a segment-centered kinematic chain biomechanical model and a constant angular acceleration motion model, the newly developed/adapted methods are all based on a free segments model, where each segment is represented with six degrees of freedom in the global frame. Moreover, these methods differ in the assumed motion model (constant angular acceleration, constant angular velocity, inertial data as control input), the state representation (segment-centered, IMU-centered) and the estimation method (EKF, sliding window optimization). In addition to the free segments representation, the optimization-based method also represents each IMU with six degrees of freedom in the global frame. In the evaluation on simulated and real data from a three segment model (an arm), the optimization-based method showed the smallest mean errors, standard deviations and maximum errors throughout all tests. It also showed the lowest dependency on magnetometer information and motion agility. Moreover, it was insensitive w.r.t. I2S position and segment length errors in the tested ranges. Errors in the I2S orientations were, however, linearly propagated into the estimated segment orientations. In the absence of magnetic disturbances, severe model calibration errors and fast motion changes, the newly developed IMU centered EKF-based method yielded comparable results with lower computational complexity.

Multimodal human motion dataset of 3D anatomical landmarks and pose keypoints.

In this paper, we present a dataset that takes 2D and 3D human pose keypoints estimated from images and relates them to the location of 3D anatomical landmarks. The dataset contains 51,051 poses obtained from 71 persons in A-Pose while performing 7 movements (walking, running, squatting, and four types of jumping). These poses were scanned to build a collection of 3D moving textured meshes with anatomical correspondence. Each mesh in that collection was used to obtain the 3D locations of 53 anatomical landmarks, and 48 images were created using virtual cameras with different perspectives. 2D pose keypoints from those images were obtained using the MediaPipe Human Pose Landmarker, and their corresponding 3D keypoints were calculated by linear triangulation. The dataset consists of a folder for each participant containing two Track Row Column (TRC) files and one JSON file for each movement sequence. One TRC file is used to store the 3D data of the triangulated 3D keypoints while the other contains the 3D anatomical landmarks. The JSON file is used to store the 2D keypoints and the calibration parameters of the virtual cameras. The anthropometric characteristics of the participants are annotated in a single CSV file. These data are intended to be used in developments that require the transformation of existing human pose solutions in computer vision into biomechanical applications or simulations. This dataset can also be used in other applications related to training neural networks for human motion analysis and studying their influence on anthropometric characteristics.

Markerless Motion Capture to Quantify Functional Performance in Neurodegeneration: Systematic Review.

BACKGROUND: Markerless motion capture (MMC) uses video cameras or depth sensors for full body tracking and presents a promising approach for objectively and unobtrusively monitoring functional performance within community settings, to aid clinical decision-making in neurodegenerative diseases such as dementia. OBJECTIVE: The primary objective of this systematic review was to investigate the application of MMC using full-body tracking, to quantify functional performance in people with dementia, mild cognitive impairment, and Parkinson disease. METHODS: A systematic search of the Embase, MEDLINE, CINAHL, and Scopus databases was conducted between November 2022 and February 2023, which yielded a total of 1595 results. The inclusion criteria were MMC and full-body tracking. A total of 157 studies were included for full-text screening, out of which 26 eligible studies that met the selection criteria were included in the review. . RESULTS: Primarily, the selected studies focused on gait analysis (n=24), while other functional tasks, such as sit to stand (n=5) and stepping in place (n=1), were also explored. However, activities of daily living were not evaluated in any of the included studies. MMC models varied across the studies, encompassing depth cameras (n=18) versus standard video cameras (n=5) or mobile phone cameras (n=2) with postprocessing using deep learning models. However, only 6 studies conducted rigorous comparisons with established gold-standard motion capture models. CONCLUSIONS: Despite its potential as an effective tool for analyzing movement and posture in individuals with dementia, mild cognitive impairment, and Parkinson disease, further research is required to establish the clinical usefulness of MMC in quantifying mobility and functional performance in the real world.

The whole-body motor skills of children with autism spectrum disorder taking goal-directed actions in virtual reality.

Many symptoms of the autism spectrum disorder (ASD) are evident in early infancy, but ASD is usually diagnosed much later by procedures lacking objective measurements. It is necessary to anticipate the identification of ASD by improving the objectivity of the procedure and the use of ecological settings. In this context, atypical motor skills are reaching consensus as a promising ASD biomarker, regardless of the level of symptom severity. This study aimed to assess differences in the whole-body motor skills between 20 children with ASD and 20 children with typical development during the execution of three tasks resembling regular activities presented in virtual reality. The virtual tasks asked to perform precise and goal-directed actions with different limbs vary in their degree of freedom of movement. Parametric and non-parametric statistical methods were applied to analyze differences in children's motor skills. The findings endorsed the hypothesis that when particular goal-directed movements are required, the type of action could modulate the presence of motor abnormalities in ASD. In particular, the ASD motor abnormalities emerged in the task requiring to take with the upper limbs goal-directed actions with low degree of freedom. The motor abnormalities covered (1) the body part mainly involved in the action, and (2) further body parts not directly involved in the movement. Findings were discussed against the background of atypical prospective control of movements and visuomotor discoordination in ASD. These findings contribute to advance the understanding of motor skills in ASD while deepening ecological and objective assessment procedures based on VR.

Using machine learning to track dogs' exploratory behaviour in the presence and absence of their caregiver.

Machine-learning-based behavioural tracking is a rapidly growing method in the behavioural sciences providing data with high spatial and temporal resolution while reducing the risk of observer bias. Nevertheless, only a few canine behaviour studies have applied this method. In the current study, we used three-dimensional (3D) tracking of the dogs' bodies to study how separation from the caregiver affected the dogs' behaviour in a novel environment. During the study, the dogs could move freely in a room equipped with trial-unique objects. We manipulated across trials whether the owner and/or a stranger was present in the room to evaluate the secure base effect, the tendency to explore and play more in the presence of the caregiver compared to another person. This secure base effect is considered a key characteristic of human attachment bonds and has also been described for the dog-caregiver relationship. The tracking data were internally consistent and highly correlated with human scorings and measurements. The results show that both the owner and stranger significantly increased the dogs' exploration; the dogs also spent more time in the proximity of the owner and stranger location when they were present. Even though the presence of both owner and stranger had a significant effect on the dogs' behaviour, the effect of the owner was more pronounced. Moreover, in the presence of the stranger the dogs spent more time close to their owner and showed a reduced tail-wagging asymmetry to the right side further supporting the distinct effect of owner and stranger on the dogs' behaviour. We conclude that machine-driven 3D tracking provides an efficient and reliable access for detailed behavioural analyses of dogs' exploration and attachment-related behaviours.

A dataset of human body tracking of walking actions captured using two Azure Kinect sensors.

A dataset of body tracking information is presented. The dataset consists of 315 captured walking sequences. Each sequence is simultaneously captured by two Azure Kinect devices. The two captures are interleaved to effectively double the frame rate. Fifteen participants partook in this experiment. Each experiment consists of seven walking actions, and having three predefined trajectories per experiment. That results in 21 sequences per participant. The data were collected using the Azure Kinect Sensor SDK. They were later processed using the official tools and libraries provided by Microsoft. For each sequence and trajectory, the positions and orientations of thirty-two tracked joints were obtained and saved. The dataset is structured as follows. The experiments from each subject are saved in a single directory. Each directory contains multiple JSON files of timestamped body tracking information to enable the fusion of the two device streams. A calibration file is also provided, enabling the mapping of the coordinates between the two Azure Kinect devices capturing the data (mapping the coordinates of the device known as the Subordinate device to the Master device coordinate system). This data can be used to train neural networks for human motion prediction tasks or test pre-existing algorithms on Azure Kinect data. This dataset could also aid in gait recognition and analysis, as well as in performing action recognition and other surveillance activities. The dataset can be found at https://zenodo.org/record/7997856.

Resilience of visually guided weight shifting to a proprioceptive perturbation depends on the complexity of the guidance stimulus.

BACKGROUND: Whole-body tracking of visual motion cues is used in balance training to improve weight shifting ability in old age and sports. RESEARCH QUESTION: How tracking of a complex (pink noise) and a periodic visual target motion during anteroposterior weight shifting affects postural and muscle responses to unilateral hip vibration. METHODS: Twenty-six participants performed 160 anteroposterior weight shifting cycles while tracking the vertical motion of a visual target, concurrently receiving Center of Pressure (CoP) feedback. They were randomly divided to groups; (a) the Constant group tracked a visual target motion constructed by 3 sinusoids of different amplitude, and (b) the Pink group tracked a complex visual target motion constructed by a pink noise generation process. Between the 60th and the 120th cycle, vibration was applied to the right gluteus medius, introducing a sideways CoP deviation. CoP displacement and electromyographic (EMG) responses of soleus, tibialis anterior and peroneus longus were recorded and summarized in blocks of 3 cycles. RESULTS: Sideways CoP deviation induced at the onset/offset of unilateral hip vibration was smaller for the Pink than the Constant group. The Pink group demonstrated greater tibialis anterior and peroneus longus EMG activity around the most anterior sway peak while soleus EMG was similar for the two groups. Both groups successfully coupled weight shifting amplitude to the target motion, but the Pink group tracked the target motion with a greater delay compared to the Constant group. SIGNIFICANCE: Whole body tracking of complex visual motions evokes perception-based action and increases ankle muscle co-activation making sway more resilient to a proprioceptive perturbation induced by unilateral hip vibration. Complex visual guidance motions should be considered when designing balance rehabilitation regimes, aiming at improving weight shifting ability and dynamic balance control.

Unobtrusive tracking of interpersonal orienting and distance predicts the subjective quality of social interactions.

Interpersonal coordination of behaviour is essential for smooth social interactions. Measures of interpersonal behaviour, however, often rely on subjective evaluations, invasive measurement techniques or gross measures of motion. Here, we constructed an unobtrusive motion tracking system that enables detailed analysis of behaviour at the individual and interpersonal levels, which we validated using wearable sensors. We evaluate dyadic measures of joint orienting and distancing, synchrony and gaze behaviours to summarize data collected during natural conversation and joint action tasks. Our results demonstrate that patterns of proxemic behaviours, rather than more widely used measures of interpersonal synchrony, best predicted the subjective quality of the interactions. Increased distance between participants predicted lower enjoyment, while increased joint orienting towards each other during cooperation correlated with increased effort reported by the participants. Importantly, the interpersonal distance was most informative of the quality of interaction when task demands and experimental control were minimal. These results suggest that interpersonal measures of behaviour gathered during minimally constrained social interactions are particularly sensitive for the subjective quality of social interactions and may be useful for interaction-based phenotyping for further studies.

Affordable Personalized, Immersive VR Motor Rehabilitation System with Full Body Tracking.

During the last decade, we have experienced fast development of virtual reality technology combined with various sensors and applications in different fields. The devices and applications are more easily accessible for the broader public. In this article, we describe a feasibility study of an affordable personalized, immersive VR motor rehabilitation system with full body tracking. While virtual reality rehabilitation is a relatively new field, several applications were already proven more effective than traditional rehabilitation programs. The applied methods utilize VR headset HTC Vive and HTC trackers together witch inverse kinematics algorithms to provide full body tracking. For that, we provide a framework for individual body calibration. The main result of the study is a developed virtual environment with guided rehabilitation scenarios based on personalized body calibration. We have proven that this approach can be used in personalized rehabilitation programs.

Towards an IMU Evaluation Framework for Human Body Tracking.

Existing full-body tracking systems, which use Inertial Measurement Units (IMUs) as sensing unit, require expert knowledge for setup and data collection. Thus, the daily application for human body tracking is difficult. In particular, in the field of active and assisted living (AAL), tracking human movements would enable novel insights not only into the quantity but also into the quality of human movement, for example by monitoring functional training. While the current market offers a wide range of products with vastly different properties, literature lacks guidelines for choosing IMUs for body tracking applications. Therefore, this paper introduces developments towards an IMU evaluation framework for human body tracking which compares IMUs against five requirement areas that consider device features and data quality. The data quality is assessed by conducting a static and a dynamic error analysis. In a first application to four IMUs of different component consumption, the IMU evaluation framework convinced as promising tool for IMU selection.

Dominant foot could affect the postural control in vestibular neuritis perceived by dynamic body balance.

During attacks of vestibular neuritis (VN), patients typically lose postural balance, with resultant postural inclination, gait deviation toward the lesion side, and tendency to fall. In this study, we examined and analyzed static and dynamic postural control during attacks of VN to characterize differences in postural control between right and left VN. Subjects were patients diagnosed with VN at the Department of Otolaryngology, Toho University Sakura Medical Center, and underwent in-patient treatment. Twenty-five patients who had spontaneous nystagmus were assessed within 3days after the onset; all were right-foot dominant. Right VN was detected in nine patients (men: 4, women: 5; mean age: 57.6±17.08years [range: 23-82]) and left VN in 16 patients (men: 10, women: 6; mean age: 58.4±14.08years [range: 23-85 years]); the percentages of canal paresis of right and left VN were 86.88±18.1% and 86.02±15.0%, respectively. Statistical comparisons were conducted using the independent t-test. In stabilometry, with eyes opened, no significant differences were found between patients with right and left VN. However, with eyes closed, the center of horizontal movement significantly shifted ipsilateral (p<0.01). The differences in the lateral and anteroposterior body tracking test (BTT) were statistically significant (p=0.0039 and p=0.0376, respectively), with greater changes in cases with right VN. Thus, the dominant foot might contribute to the postural control mechanism.

Evaluation of suitability of a micro-processing unit of motion analysis for upper limb tracking.

The aim of this study is to assess the suitability of a micro-processing unit of motion analysis (MPUMA), for monitoring, reproducing, and tracking upper limb movements. The MPUMA is based on an inertial measurement unit, a 16-bit digital signal controller and a customized algorithm. To validate the performance of the system, simultaneous recordings of the angular trajectory were performed with a video-based motion analysis system. A test of the flexo-extension of the shoulder joint during the active elevation in a complete range of 120º of the upper limb was carried out in 10 healthy volunteers. Additional tests were carried out to assess MPUMA performance during upper limb tracking. The first, a 3D motion reconstruction of three movements of the shoulder joint (flexo-extension, abduction-adduction, horizontal internal-external rotation), and the second, an upper limb tracking online during the execution of three movements of the shoulder joint followed by a continuous random movement without any restrictions by using a virtual model and a mechatronic device of the shoulder joint. Experimental results demonstrated that the MPUMA measured joint angles that are close to those from a motion-capture system with orientation RMS errors less than 3º.

Vision-based body tracking: turning Kinect into a clinical tool.

PURPOSE: Vision-based body tracking technologies, originally developed for the consumer gaming market, are being repurposed to form the core of a range of innovative healthcare applications in the clinical assessment and rehabilitation of movement ability. Vision-based body tracking has substantial potential, but there are technical limitations. METHOD: We use our "stories from the field" to articulate the challenges and offer examples of how these can be overcome. RESULTS: We illustrate that: (i) substantial effort is needed to determine the measures and feedback vision-based body tracking should provide, accounting for the practicalities of the technology (e.g. range) as well as new environments (e.g. home). (ii) Practical considerations are important when planning data capture so that data is analysable, whether finding ways to support a patient or ensuring everyone does the exercise in the same manner. (iii) Home is a place of opportunity for vision-based body tracking, but what we do now in the clinic (e.g. balance tests) or in the home (e.g. play games) will require modifications to achieve capturable, clinically relevant measures. CONCLUSIONS: This article articulates how vision-based body tracking works and when it does not to continue to inspire our clinical colleagues to imagine new applications. Implications for Rehabilitation Vision-based body tracking has quickly been repurposed to form the core of innovative healthcare applications in clinical assessment and rehabilitation, but there are clinical as well as practical challenges to make such systems a reality. Substantial effort needs to go into determining what types of measures and feedback vision-based body tracking should provide. This needs to account for the practicalities of the technology (e.g. range) as well as the opportunities of new environments (e.g. the home). Practical considerations need to be accounted for when planning capture in a particular environment so that data is analysable, whether it be finding a chair substitute, ways to support a patient or ensuring everyone does the exercise in the same manner. The home is a place of opportunity with vision-based body tracking, but it would be naïve to think that we can do what we do now in the clinic (e.g. balance tests) or in the home (e.g. play games), without appropriate modifications to what constitutes a practically capturable, clinically relevant measure.