Automating Video-Based Two-Dimensional Motion Analysis in Sport? Implications for Gait Event Detection, Pose Estimation, and Performance Parameter Analysis.

BACKGROUND: Two-dimensional (2D) video is a common tool used during sports training and competition to analyze movement. In these videos, biomechanists determine key events, annotate joint centers, and calculate spatial, temporal, and kinematic parameters to provide performance reports to coaches and athletes. Automatic tools relying on computer vision and artificial intelligence methods hold promise to reduce the need for time-consuming manual methods. OBJECTIVE: This study systematically analyzed the steps required to automate the video analysis workflow by investigating the applicability of a threshold-based event detection algorithm developed for 3D marker trajectories to 2D video data at four sampling rates; the agreement of 2D keypoints estimated by an off-the-shelf pose estimation model compared with gold-standard 3D marker trajectories projected to camera's field of view; and the influence of an offset in event detection on contact time and the sagittal knee joint angle at the key critical events of touch down and foot flat. METHODS: Repeated measures limits of agreement were used to compare parameters determined by markerless and marker-based motion capture. RESULTS: Results highlighted that a minimum video sampling rate of 100 Hz is required to detect key events, and the limited applicability of 3D marker trajectory-based event detection algorithms when using 2D video. Although detected keypoints showed good agreement with the gold-standard, misidentification of key events-such as touch down by 20 ms resulted in knee compression angle differences of up to 20°. CONCLUSION: These findings emphasize the need for de novo accurate key event detection algorithms to automate 2D video analysis pipelines.

Estimating Ground Reaction Forces from Two-Dimensional Pose Data: A Biomechanics-Based Comparison of AlphaPose, BlazePose, and OpenPose.

The adoption of computer vision pose estimation approaches, used to identify keypoint locations which are intended to reflect the necessary anatomical landmarks relied upon by biomechanists for musculoskeletal modelling, has gained increasing traction in recent years. This uptake has been further accelerated by keypoint use as inputs into machine learning models used to estimate biomechanical parameters such as ground reaction forces (GRFs) in the absence of instrumentation required for direct measurement. This study first aimed to investigate the keypoint detection rate of three open-source pose estimation models (AlphaPose, BlazePose, and OpenPose) across varying movements, camera views, and trial lengths. Second, this study aimed to assess the suitability and interchangeability of keypoints detected by each pose estimation model when used as inputs into machine learning models for the estimation of GRFs. The keypoint detection rate of BlazePose was distinctly lower than that of AlphaPose and OpenPose. All pose estimation models achieved a high keypoint detection rate at the centre of an image frame and a lower detection rate in the true sagittal plane camera field of view, compared with slightly anteriorly or posteriorly located quasi-sagittal plane camera views. The three-dimensional ground reaction force, instantaneous loading rate, and peak force for running could be estimated using the keypoints of all three pose estimation models. However, only AlphaPose and OpenPose keypoints could be used interchangeably with a machine learning model trained to estimate GRFs based on AlphaPose keypoints resulting in a high estimation accuracy when OpenPose keypoints were used as inputs and vice versa. The findings of this study highlight the need for further evaluation of computer vision-based pose estimation models for application in biomechanical human modelling, and the limitations of machine learning-based GRF estimation models that rely on 2D keypoints. This is of particular relevance given that machine learning models informing athlete monitoring guidelines are being developed for application related to athlete well-being.

Synthesising 2D Video from 3D Motion Data for Machine Learning Applications.

To increase the utility of legacy, gold-standard, three-dimensional (3D) motion capture datasets for computer vision-based machine learning applications, this study proposed and validated a method to synthesise two-dimensional (2D) video image frames from historic 3D motion data. We applied the video-based human pose estimation model OpenPose to real (in situ) and synthesised 2D videos and compared anatomical landmark keypoint outputs, with trivial observed differences (2.11−3.49 mm). We further demonstrated the utility of the method in a downstream machine learning use-case in which we trained and then tested the validity of an artificial neural network (ANN) to estimate ground reaction forces (GRFs) using synthesised and real 2D videos. Training an ANN to estimate GRFs using eight OpenPose keypoints derived from synthesised 2D videos resulted in accurate waveform GRF estimations (r > 0.9; nRMSE < 14%). When compared with using the smaller number of real videos only, accuracy was improved by adding the synthetic views and enlarging the dataset. The results highlight the utility of the developed approach to enlarge small 2D video datasets, or to create 2D video images to accompany 3D motion capture datasets to make them accessible for machine learning applications.

A Comparison of Three Neural Network Approaches for Estimating Joint Angles and Moments from Inertial Measurement Units.

The application of artificial intelligence techniques to wearable sensor data may facilitate accurate analysis outside of controlled laboratory settings-the holy grail for gait clinicians and sports scientists looking to bridge the lab to field divide. Using these techniques, parameters that are difficult to directly measure in-the-wild, may be predicted using surrogate lower resolution inputs. One example is the prediction of joint kinematics and kinetics based on inputs from inertial measurement unit (IMU) sensors. Despite increased research, there is a paucity of information examining the most suitable artificial neural network (ANN) for predicting gait kinematics and kinetics from IMUs. This paper compares the performance of three commonly employed ANNs used to predict gait kinematics and kinetics: multilayer perceptron (MLP); long short-term memory (LSTM); and convolutional neural networks (CNN). Overall high correlations between ground truth and predicted kinematic and kinetic data were found across all investigated ANNs. However, the optimal ANN should be based on the prediction task and the intended use-case application. For the prediction of joint angles, CNNs appear favourable, however these ANNs do not show an advantage over an MLP network for the prediction of joint moments. If real-time joint angle and joint moment prediction is desirable an LSTM network should be utilised.

Artificial Neural Networks in Motion Analysis-Applications of Unsupervised and Heuristic Feature Selection Techniques.

The use of machine learning to estimate joint angles from inertial sensors is a promising approach to in-field motion analysis. In this context, the simplification of the measurements by using a small number of sensors is of great interest. Neural networks have the opportunity to estimate joint angles from a sparse dataset, which enables the reduction of sensors necessary for the determination of all three-dimensional lower limb joint angles. Additionally, the dimensions of the problem can be simplified using principal component analysis. Training a long short-term memory neural network on the prediction of 3D lower limb joint angles based on inertial data showed that three sensors placed on the pelvis and both shanks are sufficient. The application of principal component analysis to the data of five sensors did not reveal improved results. The use of longer motion sequences compared to time-normalised gait cycles seems to be advantageous for the prediction accuracy, which bridges the gap to real-time applications of long short-term memory neural networks in the future.

Bridging the lab-to-field gap using machine learning: a narrative review.

This paper summarises recent advancement in applications of machine learning in sports biomechanics to bridge the lab-to-field gap as presented in the Hans Gros Emerging Researcher Award lecture at the annual conference of the International Society of Biomechanics in Sports 2022. One major challenge in machine learning applications is the need for large, high-quality datasets. Currently, most datasets, which contain kinematic and kinetic information, were collected using traditional laboratory-based motion capture despite wearable inertial sensors or standard video cameras being the hardware capable of on-field analysis. For both technologies, no high-quality large-scale databases exist. A second challenge is the lack of guidelines on how to use machine learning in biomechanics, where mostly small datasets collected on a particular population are available. This paper will summarise methods to re-purpose motion capture data for machine learning applications towards on-field motion analysis and give an overview of current applications in an attempt to derive guidelines on the most appropriate algorithm to use, an appropriate dataset size, suitable input data to estimate motion kinematics or kinetics, and how much variability should be in the dataset. This information will allow research to progress towards bridging the lab-to-field gap.

Prediction of ground reaction force and joint moments based on optical motion capture data during gait.

The standard camera- and force plate-based set-up for motion analysis suffers from the disadvantage of being limited to laboratory settings. Since adaptive algorithms are able to learn the connection between known inputs and outputs and generalise this knowledge to unknown data, these algorithms can be used to leverage motion analysis outside the laboratory. In most biomechanical applications, feedforward neural networks are used, although these networks can only work on time normalised data, while recurrent neural networks can be used for real time applications. Therefore, this study compares the performance of these two kinds of neural networks on the prediction of ground reaction force and joint moments of the lower limbs during gait based on joint angles determined by optical motion capture as input data. The accuracy of both networks when generalising to new data was assessed using the normalised root-mean-squared error, the root-mean-squared error and the correlation coefficient as evaluation metrics. Both neural networks demonstrated a high performance and good capabilities to generalise to new data. The mean prediction accuracy over all parameters applying a feedforward network was higher (r = 0.963) than using a recurrent long short-term memory network (r = 0.935).

Estimation of Gait Mechanics Based on Simulated and Measured IMU Data Using an Artificial Neural Network.

Enhancement of activity is one major topic related to the aging society. Therefore, it is necessary to understand people's motion and identify possible risk factors during activity. Technology can be used to monitor motion patterns during daily life. Especially the use of artificial intelligence combined with wearable sensors can simplify measurement systems and might at some point replace the standard motion capturing using optical measurement technologies. Therefore, this study aims to analyze the estimation of 3D joint angles and joint moments of the lower limbs based on IMU data using a feedforward neural network. The dataset summarizes optical motion capture data of former studies and additional newly collected IMU data. Based on the optical data, the acceleration and angular rate of inertial sensors was simulated. The data was augmented by simulating different sensor positions and orientations. In this study, gait analysis was undertaken with 30 participants using a conventional motion capture set-up based on an optoelectronic system and force plates in parallel with a custom IMU system consisting of five sensors. A mean correlation coefficient of 0.85 for the joint angles and 0.95 for the joint moments was achieved. The RMSE for the joint angle prediction was smaller than 4.8° and the nRMSE for the joint moment prediction was below 13.0%. Especially in the sagittal motion plane good results could be achieved. As the measured dataset is rather small, data was synthesized to complement the measured data. The enlargement of the dataset improved the prediction of the joint angles. While size did not affect the joint moment prediction, the addition of noise to the dataset resulted in an improved prediction accuracy. This indicates that research on appropriate augmentation techniques for biomechanical data is useful to further improve machine learning applications.

Prediction of lower limb joint angles and moments during gait using artificial neural networks.

In recent years, gait analysis outside the laboratory attracts more and more attention in clinical applications as well as in life sciences. Wearable sensors such as inertial sensors show high potential in these applications. Unfortunately, they can only measure kinematic motions patterns indirectly and the outcome is currently jeopardized by measurement discrepancies compared with the gold standard of optical motion tracking. The aim of this study was to overcome the limitation of measurement discrepancies and the missing information on kinetic motion parameters using a machine learning application based on artificial neural networks. For this purpose, inertial sensor data-linear acceleration and angular rate-was simulated from a database of optical motion tracking data and used as input for a feedforward and long short-term memory neural network to predict the joint angles and moments of the lower limbs during gait. Both networks achieved mean correlation coefficients higher than 0.80 in the minor motion planes, and correlation coefficients higher than 0.98 in the sagittal plane. These results encourage further applications of artificial intelligence to support gait analysis. Graphical Abstract The graphical abstract displays the processing of the data: IMU data is used as input to a feedforward and a long short-term memory neural network to predict the joint kinematics and kinetics of the lower limbs during gait.

Walking with rollator: a systematic review of gait parameters in older persons.

BACKGROUND: The aging population increasingly needs assistive technologies, such as rollators, to function and live less dependently. Rollators are designed to decrease the risk of falls by improving the gait mechanics of their users. However, data on the biomechanics of rollator assisted gait of older adults are limited, or mostly derived from experiments with younger adults. METHODS AND RESULTS: This review summarises the data from 18 independent studies on the kinematic and kinetic gait parameters of assisted gait of older persons. All of these studies evaluated spatio-temporal parameters, but not joint angles or moments. CONCLUSION: Due to the limited research on rollator supported gait in older adults, the number of parameters that could be analysed in this systematic review was restricted. Further research in the analysis of spatio-temporal parameters and a higher standardisation in clinical research will be necessary.

Assessment of the measurement accuracy of inertial sensors during different tasks of daily living.

The low cost and ease of use of inertial measurement units (IMUs) make them an attractive option for motion analysis tasks that cannot be easily measured in a laboratory. To date, only a limited amount of research has been conducted comparing commercial IMU systems to optoelectronic systems, the gold standard, for everyday tasks like stair climbing and inclined walking. In this paper, the 3D joint angles of the lower limbs are determined using both an IMU system and an optoelectronic system for twelve participants during stair ascent and descent, and inclined, declined and level walking. Three different datasets based on different hardware and anatomical models were collected for the same movement in an effort to determine the cause and quantify the errors involved with the analysis. Firstly, to calculate software errors, two different anatomical models were compared for one hardware system. Secondly, to calculate hardware errors, results were compared between two different measurement systems using the same anatomical model. Finally, the overall error between both systems with their native anatomical models was calculated. Statistical analysis was performed using statistical parametric mapping. When both systems were evaluated based on the same anatomical model, the number of trials with significant differences decreased markedly. Thus, the differences in joint angle measurement can mainly be attributed to the variability in the anatomical models used for calculations and not to the IMU hardware.

Intelligent prediction of kinetic parameters during cutting manoeuvres.

Due to its capabilities in analysing injury risk, the ability to analyse an athlete's ground reaction force and joint moments is of high interest in sports biomechanics. However, using force plates for the kinetic measurements influences the athlete's performance. Therefore, this study aims to use a feed-forward neural network to predict hip, knee and ankle joint moments as well as the ground reaction force from kinematic data during the execution and depart contact of a maximum effort 90° cutting manoeuvre. A total number of 525 cutting manoeuvres performed by 55 athletes were used to train and test neural networks. Either marker trajectories or joint angles were used as input data. The correlation coefficient between the measured and predicted data indicated strong correlations. By using joint angles as the input parameters, slightly but not significantly higher accuracy was found in joint moments predictions. The prediction of the ground reaction force showed significantly higher accuracy when using marker trajectories. Hence, the proposed feed-forward neural network method can be used to predict motion kinetics during a fast change of direction. This may allow for the simplification of cutting manoeuvres experimental set-ups for and through the use of inertial sensors. Graphical abstract The left part of the graphical abstract displays the angle progression of the hip, knee and ankle joint as an example of the kinematic input data and is supported by a stick figure of the motion task, a 90° cutting manoeuvre. This data is used to train a feed-forward neural network, which is displayed in the middle. The neural network's output is displayed on the right. As an example of the kinetic data, the joint moments of hip, knee and ankle joint are displayed and supported by a stick figure.

Is bone-cement augmentation of screw-anchor fixation systems superior in unstable femoral neck fractures? A biomechanical cadaveric study.

OBJECTIVES: Improved fixation techniques with optional use of bone cements for implant augmentation have been developed to enhance stability and reduce complication rates after osteosynthesis of femoral neck fractures. This biomechanical study aimed to evaluate the effect of cement augmentation on implant anchorage and overall performance of screw-anchor fixation systems in unstable femoral neck fractures. METHODS: Ten pairs of human cadaveric femora were used to create standardized femoral neck fractures (Pauwels type 3 fractures; AO/OTA 31-B2) with comminution and were fixed by means of a rotationally stable screw-anchor (RoSA) system. The specimens were assigned pairwise to two groups and either augmented with PMMA-based cement (Group 1, augmented) or left without such augmentation (Group 2, control). Biomechanical testing, simulating physiological loading at four distinct load levels, was performed over 10.000 cycles for each level with the use of a multidimensional force-transducer system. Data was analysed by means of motion tracking. RESULTS: Stiffness, femoral head rotation, implant migration, femoral neck shortening, and failure load did not differ significantly between the two groups (p ≥ .10). For both groups, the main failure type was dislocation in the frontal plane with consecutive varus collapse). In the cement-augmented specimens, implant migration and femoral neck shortening were significantly dependent on bone mineral density (BMD), with higher values in osteoporotic bones. There was a correlation between failure load and BMD in cement-augmented specimens. CONCLUSION: In screw-anchor fixation of unstable femoral neck fractures, bone-cement augmentation seems to show no additional advantages in regard to stiffness, rotational stability, implant migration, resistance to fracture displacement, femoral neck shortening or failure load.

Understanding cutting maneuvers - The mechanical consequence of preparatory strategies and foot strike pattern.

OBJECTIVES: This study investigated the relation of different previously reported preparatory strategies and musculo-skeletal loading during fast preplanned 90° cutting maneuvers (CM). The aim was to increase the understanding of the connection between whole body orientation, preparatory actions and the solution strategy to fulfil the requirements of a CM. METHODS: Three consecutive steps of anticipated 90° CMs were investigated in a 3D movement analysis setup. Pelvis orientation clustered the subjects in two groups, with minor and major pre-orientation. To understand the impact of body orientation on the specific movement strategy, joint angles, moments and energy as well as spatio-temporal parameters of the movement were analysed. RESULTS: Early rotation of the body was initiated by a small step width during braking resulting in a more constant path velocity of the centre of mass and less demands on the hip- and knee surrounding muscles. Minor pre-orientation required increased work of the hip muscles to decelerate, reaccelerate and in particular to rotate the body. This resulted in an increase of contact time. While pre-orientation in combination with fore-foot striking led to a strategy where energy absorption and generation is mainly generated by the ankle plantar flexors, less pre-orientation and rear-foot striking resulted in a knee- and hip dominant strategy. CONCLUSION: Step width before transition strongly determined pre-orientation and overall body position. Both strategies fulfil the requirements of a CM but induce different demands regarding muscular capacities. Pelvis orientation and step width are easy-to-use assessment parameters in the practical field.

A new in vitro spine test rig to track multiple vertebral motions under physiological conditions.

In vitro pure moment spine tests are commonly used to analyse surgical implants in cadaveric models. Most of the tests are performed at room temperature. However, some new dynamic instrumentation devices and soft tissues show temperature-dependent material properties. Therefore, the aim of this study is to develop a new test rig, which allows applying pure moments on lumbar spine specimens in a vapour-filled chamber at body temperature. As no direct sight is given in the vapour-filled closed chamber, a magnetic tracking (MT) system with implantable receivers was used. Four human cadaveric lumbar spines (L2-L5) were tested in a vapour atmosphere at body temperature with a native and rigid instrumented group. In conclusion, the experimental set-up allows vertebral motion tracking of multiple functional spinal units (FSUs) in a moisture environment at body temperature.

A systematic review of gait analysis methods based on inertial sensors and adaptive algorithms.

The conventional methods to assess human gait are either expensive or complex to be applied regularly in clinical practice. To reduce the cost and simplify the evaluation, inertial sensors and adaptive algorithms have been utilized, respectively. This paper aims to summarize studies that applied adaptive also called artificial intelligence (AI) algorithms to gait analysis based on inertial sensor data, verifying if they can support the clinical evaluation. Articles were identified through searches of the main databases, which were encompassed from 1968 to October 2016. We have identified 22 studies that met the inclusion criteria. The included papers were analyzed due to their data acquisition and processing methods with specific questionnaires. Concerning the data acquisition, the mean score is 6.1±1.62, what implies that 13 of 22 papers failed to report relevant outcomes. The quality assessment of AI algorithms presents an above-average rating (8.2±1.84). Therefore, AI algorithms seem to be able to support gait analysis based on inertial sensor data. Further research, however, is necessary to enhance and standardize the application in patients, since most of the studies used distinct methods to evaluate healthy subjects.

Biomechanical testing of a PEEK-based dynamic instrumentation device in a lumbar spine model.

BACKGROUND: The purpose of this study was to investigate the range-of-motion after posterior polyetheretherketone-based rod stabilisation combined with a dynamic silicone hinge in order to compare it with titanium rigid stabilisation. METHODS: Five human cadaveric lumbar spines with four vertebra each (L2 to L5) were tested in a temperature adjustable spine-testing set-up in four trials: (1) native measurement; (2) kinematics after rigid monosegmental titanium rod instrumentation with anterior intervertebral bracing of the segment L4/5; (3) kinematics after hybrid posterior polyetheretherketone rod instrumentation combined with a silicone hinge within the adjacent level (L3/4) and (4) kinematics after additional decompression with laminectomy of L4 and bilateral resection of the inferior articular processes (L3). During all steps, the specimens were loaded quasi-statically with 1°/s with pure moment up to 7.5Nm in flexion/extension, lateral bending and axial rotation. FINDINGS: In comparison to the native cadaveric spine, both the titanium device and polyetheretherketone-based device reduce the range-of-motion within the level L4/5 significantly (flexion/extension: reduction of 77%, p<0.001; lateral bending: reduction of 62%, p<0.001; axial rotation: reduction of 71%, p<0.001). There was a clear stabilisation effect after hybrid-instrumentation within the level L3/4, especially in flexion/extension (64%, p<0.001) and lateral bending (62%, p<0.001) but without any effect on the axial rotation. Any temperature dependency has not been observed. INTERPRETATION: Surprisingly, the hybrid device compensates for laminectomy L4 and destabilising procedure within the level L3/4 in comparison to other implants. Further studies must be performed to show its effectiveness regarding the adjacent segment instability.