Automated, Vision-Based Goniometry and Range of Motion Calculation in Individuals With Suspected Ehlers-Danlos Syndromes/Generalized Hypermobility Spectrum Disorders: A Comparison of Pose-Estimation Libraries to Goniometric Measurements.

Generalized joint hypermobility (GJH) often leads clinicians to suspect a diagnosis of Ehlers Danlos Syndrome (EDS), but it can be difficult to objectively assess. Video-based goniometry has been proposed to objectively estimate joint range of motion in hyperextended joints. As part of an exam of joint hypermobility at a specialized EDS clinic, a mobile phone was used to record short videos of 97 adults (89 female, 35.0 ± 9.9 years old) undergoing assessment of the elbows, knees, shoulders, ankles, and fifth fingers. Five body keypoint pose-estimation libraries (AlphaPose, Detectron, MediaPipe-Body, MoveNet - Thunder, OpenPose) and two hand keypoint pose-estimation libraries (AlphaPose, MediaPipe-Hands) were used to geometrically calculate the maximum angle of hyperextension or hyperflexion of each joint. A custom domain-specific model with a MobileNet-v2 backbone finetuned on data collected as part of this study was also evaluated for the fifth finger movement. Spearman's correlation was used to analyze the angles calculated from the tracked joint positions, the angles calculated from manually annotated keypoints, and the angles measured using a goniometer. Moderate correlations between the angles estimated using pose-tracked keypoints and the goniometer measurements were identified for the elbow (rho =.722; Detectron), knee (rho =.608; MoveNet - Thunder), shoulder (rho =.632; MoveNet - Thunder), and fifth finger (rho =.786; custom model) movements. The angles estimated from keypoints predicted by open-source libraries at the ankles were not significantly correlated with the goniometer measurements. Manually annotated angles at the elbows, knees, shoulders, and fifth fingers were moderately to strongly correlated to goniometer measurements but were weakly correlated for the ankles. There was not one pose-estimation library which performed best across all joints, so the library of choice must be selected separately for each joint of interest. This work evaluates several pose-estimation models as part of a vision-based system for estimating joint angles in individuals with suspected joint hypermobility. Future applications of the proposed system could facilitate objective assessment and screening of individuals referred to specialized EDS clinics.

Evaluating the ability of a predictive vision-based machine learning model to measure changes in gait in response to medication and DBS within individuals with Parkinson's disease.

INTRODUCTION: Gait impairments in Parkinson's disease (PD) are treated with dopaminergic medication or deep-brain stimulation (DBS), although the magnitude of the response is variable between individuals. Computer vision-based approaches have previously been evaluated for measuring the severity of parkinsonian gait in videos, but have not been evaluated for their ability to identify changes within individuals in response to treatment. This pilot study examines whether a vision-based model, trained on videos of parkinsonism, is able to detect improvement in parkinsonian gait in people with PD in response to medication and DBS use. METHODS: A spatial-temporal graph convolutional model was trained to predict MDS-UPDRS-gait scores in 362 videos from 14 older adults with drug-induced parkinsonism. This model was then used to predict MDS-UPDRS-gait scores on a different dataset of 42 paired videos from 13 individuals with PD, recorded while ON and OFF medication and DBS treatment during the same clinical visit. Statistical methods were used to assess whether the model was responsive to changes in gait in the ON and OFF states. RESULTS: The MDS-UPDRS-gait scores predicted by the model were lower on average (representing improved gait; p = 0.017, Cohen's d = 0.495) during the ON medication and DBS treatment conditions. The magnitude of the differences between ON and OFF state was significantly correlated between model predictions and clinician annotations (p = 0.004). The predicted scores were significantly correlated with the clinician scores (Kendall's tau-b = 0.301, p = 0.010), but were distributed in a smaller range as compared to the clinician scores. CONCLUSION: A vision-based model trained on parkinsonian gait did not accurately predict MDS-UPDRS-gait scores in a different PD cohort, but detected weak, but statistically significant proportional changes in response to medication and DBS use. Large, clinically validated datasets of videos captured in many different settings and treatment conditions are required to develop accurate vision-based models of parkinsonian gait.

Ambient Monitoring of Gait and Machine Learning Models for Dynamic and Short-Term Falls Risk Assessment in People With Dementia.

Falls are a leading cause of morbidity and mortality in older adults with dementia residing in long-term care. Having access to a frequently updated and accurate estimate of the likelihood of a fall over a short time frame for each resident will enable care staff to provide targeted interventions to prevent falls and resulting injuries. To this end, machine learning models to estimate and frequently update the risk of a fall within the next 4 weeks were trained on longitudinal data from 54 older adult participants with dementia. Data from each participant included baseline clinical assessments of gait, mobility, and fall risk at the time of admission, daily medication intake in three medication categories, and frequent assessments of gait performed via a computer vision-based ambient monitoring system. Systematic ablations investigated the effects of various hyperparameters and feature sets and experimentally identified differential contributions from baseline clinical assessments, ambient gait analysis, and daily medication intake. In leave-one-subject-out cross-validation, the best performing model predicts the likelihood of a fall over the next 4 weeks with a sensitivity and specificity of 72.8 and 73.2, respectively, and achieved an area under the receiver operating characteristic curve (AUROC) of 76.2. By contrast, the best model excluding ambient gait features achieved an AUROC of 56.2 with a sensitivity and specificity of 51.9 and 54.0, respectively. Future research will focus on externally validating these findings to prepare for the implementation of this technology to reduce fall and fall-related injuries in long-term care.

Feasibility of video-based joint hypermobility assessment in individuals with suspected Ehlers-Danlos syndromes/generalised hypermobility spectrum disorders: a single-site observational study protocol.

INTRODUCTION: Ehlers-Danlos syndromes (EDS)/generalised hypermobility spectrum disorders (G-HSD) affect the connective tissue of the body and present with a heterogeneous set of symptoms that pose a challenge for diagnosis. One of the main diagnostic criteria of EDS/G-HSD is generalised joint hypermobility, which is currently assessed by clinicians during a physical exam. However, the practice for measuring joint hypermobility is inconsistent between clinicians, leading to high inter-rater variability. Often patients are misdiagnosed with EDS/G-HSD based on an incorrect hypermobility assessment, leading to increased referral rates and resource utilisation at specialised EDS clinics that results in unnecessary emotional distress for patients. An objective, validated and scalable method for assessing hypermobility might mitigate these issues and result in improved EDS/G-HSD patient care. METHODS AND ANALYSIS: This study will examine the use of videos obtained using a smartphone camera to assess the range of motion (ROM) and hypermobility of the joints assessed in Beighton score and more (spine, shoulders, elbows, knees, ankles, thumbs and fifth fingers) in individuals with suspected EDS/G-HSD. Short videos of participants will be captured as they undergo a formal assessment of joint hypermobility at the GoodHope EDS Clinic at Toronto General Hospital. Clinicians will measure the ROM at each joint using a clinical-grade goniometer to establish ground truth measurements. Open-source human pose-estimation libraries will be used to extract the locations of key joints from the videos. Deterministic and machine learning systems will be developed and evaluated for estimating the ROM at each joint. Results will be analysed separately for each joint and human pose-estimation library. ETHICS AND DISSEMINATION: This study was approved by the Research Ethics Board of the University Health Network in Toronto on 26 April 2022. Participants will provide written informed consent. Findings from this study will be published in peer-reviewed journals and presented at conferences. TRIAL REGISTRATION NUMBER: NCT05366114.

Concurrent Validity of Zeno Instrumented Walkway and Video-Based Gait Features in Adults With Parkinson's Disease.

BACKGROUND: Parkinson's disease (PD) presents with motor symptoms such as bradykinesia, rigidity, and tremor that can affect gait. To monitor changes associated with disease progression or medication use, quantitative gait assessment is often performed during clinical visits. Conversely, vision-based solutions have been proposed for monitoring gait quality in non-clinical settings. METHODS: We use three 2D human pose-estimation libraries (AlphaPose, Detectron, OpenPose) and one 3D library (ROMP) to calculate gait features from color video, and correlate them with those extracted by a Zeno instrumented walkway in older adults with PD. We calculate video-based gait features using a manual and automated heel-strike detection algorithm, and compare the correlations when the participants walk towards and away from the camera separately. RESULTS: Based on analysis of 67 bidirectional walking bouts from 25 adults with PD, moderate to strong positive correlations were identified between the number of steps, cadence, as well as the mean and coefficient of variation of step width calculated from Zeno and video using 2D pose-estimation libraries. We noted that our automated heel-strike annotation method struggled to identify short steps. CONCLUSION: Gait features calculated from 2D joint trajectories are more strongly correlated with the Zeno than analogous gait features calculated from ROMP. Based on our analysis, videos processed with 2D pose-estimation libraries can be used for longitudinal gait monitoring in individuals with PD. Future work will seek to improve the prediction of gait features using a comprehensive machine learning model to predict gait features directly from color video without relying on intermediate extraction of joint trajectories.

The Toronto older adults gait archive: video and 3D inertial motion capture data of older adults' walking.

We introduce the Toronto Older Adults Gait Archive, a gait dataset of 14 older adults containing 2D video recordings, and 2D (video pose tracking algorithms) and 3D (inertial motion capture) joint locations of the lower body. Participants walked for 60 seconds. We also collected participants' scores on four clinical assessments of gait and balance, namely the Tinneti performance-oriented mobility assessment (POMA-gait and -balance), the Berg balance scale (BBS), and the timed-up-and-go (TUG). Three human pose tracking models (Alphapose, OpenPose, and Detectron) were used to detect body joint positions in 2D video frames and a number of gait parameters were computed using 2D video-based and 3D motion capture data. To show an example usage of our datasets, we performed a correlation analysis between the gait variables and the clinical scores. Our findings revealed that the temporal but not the spatial or variability gait variables from both systems had high correlations to clinical scores. This dataset can be used to evaluate, or to enhance vision-based pose-tracking models to the specifics of older adults' walking.

Technical Note: Quantifying music-dance synchrony during salsa dancing with a deep learning-based 2D pose estimator.

Dance interventions hold promise for improving gait and balance in people with neurological conditions. It is possible that synchronization of movement to the music is one of the mechanisms through which dance bestows physical benefits. This technical note will describe a novel method using a deep learning-based 2D pose estimator: OpenPose, alongside beat analysis of music to quantify movement-music synchrony during salsa dancing. This method has four components: i) camera setup and recording, ii) tempo/downbeat analysis and waveform cleanup, iii) OpenPose estimation and data extraction, and iv) synchronization analysis. Four participants performed a solo basic salsa step continuously for 90 s to a salsa track while their movements and the music were recorded with a webcam. Two conditions were recorded for each participant: one in which they danced on the beat of the music and one where they did not. This data was then extracted from OpenPose and analyzed. Median asynchrony values highlighted differences between participants with and without dance training and between on- and off-beat conditions, indicating that this method may be an effective means to quantify a dancer's asynchrony while performing a basic salsa step.

Estimating Parkinsonism Severity in Natural Gait Videos of Older Adults With Dementia.

Drug-induced parkinsonism affects many older adults with dementia, often causing gait disturbances. New advances in vision-based human pose- estimation have opened possibilities for frequent and unobtrusive analysis of gait in long-term care settings. This work leverages spatial-temporal graph convolutional network (ST-GCN) architectures and training procedures to predict clinical scores of parkinsonism in gait from video of individuals with dementia. We propose a two-stage training approach consisting of a self-supervised pretraining stage that encourages the ST-GCN model to learn about gait patterns before predicting clinical scores in the finetuning stage. The proposed ST-GCN models are evaluated on joint trajectories extracted from video and are compared against traditional (ordinal, linear, random forest) regression models and temporal convolutional network baselines. Three 2D human pose-estimation libraries (OpenPose, Detectron, AlphaPose) and the Microsoft Kinect (2D and 3D) are used to extract joint trajectories of 4787 natural walking bouts from 53 older adults with dementia. A subset of 399 walks from 14 participants is annotated with scores of parkinsonism severity on the gait criteria of the Unified Parkinson's Disease Rating Scale (UPDRS) and the Simpson-Angus Scale (SAS). Our results demonstrate that ST-GCN models operating on 3D joint trajectories extracted from the Kinect consistently outperform all other models and feature sets. Prediction of parkinsonism scores in natural walking bouts of unseen participants remains a challenging task, with the best models achieving macro-averaged F1-scores of 0.53 ± 0.03 and 0.40 ± 0.02 for UPDRS-gait and SAS-gait, respectively. Pre-trained model and demo code for this work is available.1.

Prediction of Parkinsonian Gait in Older Adults with Dementia using Joint Trajectories and Gait Features from 2D Video.

Older adults with dementia have a high risk of developing drug-induced parkinsonism; however, formal clinical gait assessments are too infrequent to capture fluctuations in their gait. Camera-based human pose estimation and tracking provides a means to frequently monitor gait in nonclinical settings. In this study, 2160 walking bouts from 49 participants were recorded using a ceiling-mounted camera. Recorded color videos were processed using AlphaPose to obtain 2D joint trajectories of the participant as they were walking down a hallway of the unit. A subset of 324 walking bouts from 14 participants were annotated with clinical scores of parkinsonism on the Unified Parkinson's Disease Rating Scale (UPDRS)-gait scale. Linear, random forest, and ordinal logistic regression models were evaluated for regression to UPDRS-gait scores using engineered 2D gait features calculated from the AlphaPose joint trajectories. Additionally, spatial temporal graph convolutional networks (ST-GCNs) were trained to predict UPDRS-gait scores from joint trajectories and gait features using a two-stage training scheme (self-supervised pretraining stage on all walks followed by a finetuning stage on labelled walks). All models were trained using leave-one-subject-out cross-validation to simulate testing on previously unseen participants. The macro-averaged F1-score was 0.333 for the best model operating on only gait features and 0.372 for the top ST-GCN model that used both joint trajectories and gait features as input. When accepting predicted scores that were only off by at most 1 point on the UPDRS-gait scale, the accuracy of the model that only used gait features was 82.8%, while the model that also used joint trajectories had an accuracy of 94.2%.Clinical Relevance- The combination of gait features and joint trajectories capture parkinsonian qualities in gait better than either group of data individually.

Gait changes over time in hospitalized older adults with advanced dementia: Predictors of mobility change.

People with dementia are at risk of mobility decline. In this study, we measured changes in quantitative gait measures over a maximum 10-week period during the course of a psychogeriatric admission in older adults with dementia, with the aims to describe mobility changes over the duration of the admission, and to determine which factors were associated with this change. Fifty-four individuals admitted to a specialized dementia inpatient unit participated in this study. A vision-based markerless motion capture system was used to record participants' natural gait. Mixed effect models were developed with gait measures as the dependent variables and clinical and demographic variables as predictors. We found that gait stability, step time, and step length decreased, and step time variability and step length variability increased over 10 weeks. Gait stability of men decreased more than that of women, associated with an increased sacrum mediolateral range of motion over time. In addition, the sacrum mediolateral range of motion decreased in those with mild neuropsychiatric symptoms over 10 weeks, but increased in those with more severe neuropsychiatric symptoms. Our study provides evidence of worsening of gait mechanics and control over the course of a hospitalization in older adults with dementia. Quantitative gait monitoring in hospital environments may provide opportunities to intervene to prevent adverse events, decelerate mobility decline, and monitor rehabilitation outcomes.

Concurrent validity of human pose tracking in video for measuring gait parameters in older adults: a preliminary analysis with multiple trackers, viewing angles, and walking directions.

BACKGROUND: Many of the available gait monitoring technologies are expensive, require specialized expertise, are time consuming to use, and are not widely available for clinical use. The advent of video-based pose tracking provides an opportunity for inexpensive automated analysis of human walking in older adults using video cameras. However, there is a need to validate gait parameters calculated by these algorithms against gold standard methods for measuring human gait data in this population. METHODS: We compared quantitative gait variables of 11 older adults (mean age = 85.2) calculated from video recordings using three pose trackers (AlphaPose, OpenPose, Detectron) to those calculated from a 3D motion capture system. We performed comparisons for videos captured by two cameras at two different viewing angles, and viewed from the front or back. We also analyzed the data when including gait variables of individual steps of each participant or each participant's averaged gait variables. RESULTS: Our findings revealed that, i) temporal (cadence and step time), but not spatial and variability gait measures (step width, estimated margin of stability, coefficient of variation of step time and width), calculated from the video pose tracking algorithms correlate significantly to that of motion capture system, and ii) there are minimal differences between the two camera heights, and walks viewed from the front or back in terms of correlation of gait variables, and iii) gait variables extracted from AlphaPose and Detectron had the highest agreement while OpenPose had the lowest agreement. CONCLUSIONS: There are important opportunities to evaluate models capable of 3D pose estimation in video data, improve the training of pose-tracking algorithms for older adult and clinical populations, and develop video-based 3D pose trackers specifically optimized for quantitative gait measurement.

Predicting Short-Term Risk of Falls in a High-Risk Group With Dementia.

OBJECTIVES: To develop a prognostic model to predict the probability of a short-term fall (within the next 7 to 30 days) in older adults with dementia. DESIGN: Prospective observational study. SETTING AND PARTICIPANTS: Fifty-one individuals with dementia at high risk of falls from a specialized dementia inpatient unit. METHODS: Clinical and demographic measures were collected and a vision-based markerless motion capture was used to record the natural gait of participants over a 2-week baseline. Falls were tracked throughout the length of stay. Cox proportional hazard regression analysis was used to build a prognostic model to determine fall-free survival probabilities at 7 days and at 30 days. The model's discriminative ability was also internally validated. RESULTS: Fall history and gait stability (estimated margin of stability) were statistically significant predictors of time to fall and included in the final prognostic model. The model's predicted survival probabilities were close to observed values at both 7 and 30 days. The area under the receiver operating curve was 0.80 at 7 days, and 0.67 at 30 days and the model had a discrimination performance (the Harrel concordance index) of 0.71. CONCLUSIONS AND IMPLICATIONS: Our short-term falls risk model had fair to good predictive and discrimination ability. Gait stability and recent fall history predicted an imminent fall in our population. This provides some preliminary evidence that the degree of gait instability may be measureable in natural everyday gait to allow dynamic falls risk monitoring. External validation of the model using a separate data set is needed to evaluate model's predictive performance.

Assessment of Parkinsonian gait in older adults with dementia via human pose tracking in video data.

BACKGROUND: Parkinsonism is common in people with dementia, and is associated with neurodegenerative and vascular changes in the brain, or with exposure to antipsychotic or other dopamine antagonist medications. The detection of parkinsonian changes to gait may provide an opportunity to intervene and address reversible causes. In this study, we investigate the use of a vision-based system as an unobtrusive means to assess severity of parkinsonism in gait. METHODS: Videos of walking bouts of natural gait were collected in a specialized dementia unit using a Microsoft Kinect sensor and onboard color camera, and were processed to extract sixteen 3D and eight 2D gait features. Univariate regression to gait quality, as rated on the Unified Parkinson's Disease Rating Scale (UPDRS) and Simpson-Angus Scale (SAS), was used to identify gait features significantly correlated to these clinical scores for inclusion in multivariate models. Multivariate ordinal logistic regression was subsequently performed and the relative contribution of each gait feature for regression to UPDRS-gait and SAS-gait scores was assessed. RESULTS: Four hundred one walking bouts from 14 older adults with dementia were included in the analysis. Multivariate ordinal logistic regression models incorporating selected 2D or 3D gait features attained similar accuracies: the UPDRS-gait regression models achieved accuracies of 61.4 and 62.1% for 2D and 3D features, respectively. Similarly, the SAS-gait models achieved accuracies of 47.4 and 48.5% with 2D or 3D gait features, respectively. CONCLUSIONS: Gait features extracted from both 2D and 3D videos are correlated to UPDRS-gait and SAS-gait scores of parkinsonism severity in gait. Vision-based systems have the potential to be used as tools for longitudinal monitoring of parkinsonism in residential settings.