A methodological framework to assess the accuracy of virtual reality hand-tracking systems: A case study with the Meta Quest 2.

Optical markerless hand-tracking systems incorporated into virtual reality (VR) headsets are transforming the ability to assess fine motor skills in VR. This promises to have far-reaching implications for the increased applicability of VR across scientific, industrial, and clinical settings. However, so far, there are little data regarding the accuracy, delay, and overall performance of these types of hand-tracking systems. Here we present a novel methodological framework based on a fixed grid of targets, which can be easily applied to measure these systems' absolute positional error and delay. We also demonstrate a method to assess finger joint-angle accuracy. We used this framework to evaluate the Meta Quest 2 hand-tracking system. Our results showed an average fingertip positional error of 1.1cm, an average finger joint angle error of 9.6∘ and an average temporal delay of 45.0 ms. This methodological framework provides a powerful tool to ensure the reliability and validity of data originating from VR-based, markerless hand-tracking systems.

Numerical instability of Hill-type muscle models.

Hill-type muscle models are highly preferred as phenomenological models for musculoskeletal simulation studies despite their introduction almost a century ago. The use of simple Hill-type models in simulations, instead of more recent cross-bridge models, is well justified since computationally 'light-weight'-although less accurate-Hill-type models have great value for large-scale simulations. However, this article aims to invite discussion on numerical instability issues of Hill-type muscle models in simulation studies, which can lead to computational failures and, therefore, cannot be simply dismissed as an inevitable but acceptable consequence of simplification. We will first revisit the basic premises and assumptions on the force-length and force-velocity relationships that Hill-type models are based upon, and their often overlooked but major theoretical limitations. We will then use several simple conceptual simulation studies to discuss how these numerical instability issues can manifest as practical computational problems. Lastly, we will review how such numerical instability issues are dealt with, mostly in an ad hoc fashion, in two main areas of application: musculoskeletal biomechanics and computer animation.

Muscle inertial contributions to ankle kinetics during the swing phase of running.

Skeletal muscles have inertia that leads to inertial forces acting around joints. Although these inertial muscle forces contribute to joint kinetics, they are not typically accounted for in musculoskeletal models used for human movement biomechanics research. Ignoring inertial forces can lead to errors in joint kinetics, but how large these errors are in inverse dynamics calculations of common movements is yet unclear. We, therefore, examined the role of shank muscle inertia on ankle joint moments during the swing phase of running at different speeds. A custom musculoskeletal modelling and simulation platform was used to perform inverse dynamics with a model that either combined muscle mass in the total shank mass, or considered the gastrocnemius lateralis/medialis, soleus, and tibialis anterior muscles as separate masses from the shank. Ankle moments were considerably affected when muscles were modelled as separate masses, with a general shift towards reduced dorsiflexion and higher plantarflexion moments. Differences between both modelling conditions increased with running speed and ranged between 0.8 and 1.6 Nm (ankle moment profile root mean square error), 8-18 % (peak dorsiflexion moment difference) and 24-42 % (peak plantarflexion moment difference). Moreover, we observed a complex combination of inertial forces, especially those due to rotation and translation of the shank, in which the direction of inertial force changed during the swing phase. These results show that ignoring muscle inertia in musculoskeletal models can lead to under- or overestimations of structure-specific loads and thus erroneous study conclusions. Our results suggest that muscle inertial forces should be carefully considered when using musculoskeletal models.

Deep-SAGA: a deep-learning-based system for automatic gaze annotation from eye-tracking data.

With continued advancements in portable eye-tracker technology liberating experimenters from the restraints of artificial laboratory designs, research can now collect gaze data from real-world, natural navigation. However, the field lacks a robust method for achieving this, as past approaches relied upon the time-consuming manual annotation of eye-tracking data, while previous attempts at automation lack the necessary versatility for in-the-wild navigation trials consisting of complex and dynamic scenes. Here, we propose a system capable of informing researchers of where and what a user's gaze is focused upon at any one time. The system achieves this by first running footage recorded on a head-mounted camera through a deep-learning-based object detection algorithm called Masked Region-based Convolutional Neural Network (Mask R-CNN). The algorithm's output is combined with frame-by-frame gaze coordinates measured by an eye-tracking device synchronized with the head-mounted camera to detect and annotate, without any manual intervention, what a user looked at for each frame of the provided footage. The effectiveness of the presented methodology was legitimized by a comparison between the system output and that of manual coders. High levels of agreement between the two validated the system as a preferable data collection technique as it was capable of processing data at a significantly faster rate than its human counterpart. Support for the system's practicality was then further demonstrated via a case study exploring the mediatory effects of gaze behaviors on an environment-driven attentional bias.

A Hybrid Method for Ultrasound-Based Tracking of Skeletal Muscle Architecture.

OBJECTIVE: Tracking skeletal muscle architecture using B-mode ultrasound is a widely used method in the field of human movement science and biomechanics. Sequential methods based on optical flow algorithms allow for smooth and coherent muscle tracking but are known to drift over time. Non-sequential feature detection methods on the other hand, do not suffer from drift, but are limited to tracking only lower-dimensional features. They are also known to be sensitive to image noise, and therefore often result in highly irregular tracking patterns. Building on the complimentary nature of both approaches, we present a novel automated hybrid muscle tracking approach that combines a sequential feature-point tracking method and a non-sequential method based on Hough transform. METHODS: Tibialis anterior fascicle pennation angle and length, and central aponeurosis displacement, were measured in five healthy individuals during isometric contractions at five different ankle angles. RESULTS: Our hybrid method was demonstrated to significantly (p < 0.001) reduce drift compared to two sequential methods, and curve irregularity was significantly (p < 0.001) decreased compared to a non-sequential method. CONCLUSION: These findings suggest that the proposed hybrid approach can uniquely mitigate drift and irregularity limitation of individual methods used for tracking skeletal muscle architecture. SIGNIFICANCE: Automated muscle tracking allows for convenient analysis of large datasets, whereas automatic drift correction opens the door for tracking muscle architecture in long ultrasound recordings during common movements, such as walking, running, and jumping without the need for manual intervention.

Application of subject-specific helmets for the study of human visuomotor behavior using transcranial focused ultrasound: a pilot study.

BACKGROUND AND OBJECTIVE: As a novel non-invasive human brain stimulation method, transcranial focused ultrasound (tFUS) is receiving growing attention due to its superior spatial specificity and depth penetrability. Since the focal point of tFUS needs to be fixated precisely to the target brain region during stimulation, a critical issue is to identify and maintain the accurate position and orientation of the tFUS transducer relative to the subject's head. This study aims to propose the entire framework of tFUS stimulation integrating the methods previously proposed by the authors for tFUS transducer configuration optimization and a subject-specific 3D-printed helmet, and to validate this complete setup in a human behavioral neuromodulation study. METHODS: To find the optimal configuration of the tFUS transducer, a numerical method based on subject-specific tFUS beamlines simulation was used. Then, the subject-specific 3D-printed helmet has been applied to effectively secure the transducer at the estimated optimal configuration. To validate this tFUS framework, a common behavioral neuromodulation paradigm was chosen; the effect of the dorsolateral prefrontal cortex (DLPFC) stimulation on anti-saccade (AS) behavior. While human participants (n=2) were performing AS tasks, tFUS stimulations were randomly applied to the left DLPFC right after the fixation target disappeared. RESULTS: The neuromodulation result strongly suggests that the cortical stimulation using the proposed tFUS setup is effective in significantly reducing the error rates of anti-saccades (about -10 %p for S1 and -16 %p for S2), whereas no significant effect was observed on their latencies. These observed behavioral effects are consistent with the previous results based on conventional brain stimulation or lesion studies. CONCLUSIONS: The proposed subject-specific tFUS framework has been effectively used in human neuromodulation study. The result suggests that the tFUS stimulation targeted to the DLPFC can generate a neuromodulatory effect on AS behavior.

On the encoding capacity of human motor adaptation.

Primitive-based models of motor learning suggest that adaptation occurs by tuning the responses of motor primitives. Based on this idea, we consider motor learning as an information encoding procedure, that is, a procedure of encoding a motor skill into primitives. The capacity of encoding is determined by the number of recruited primitives, which depends on how many primitives are "visited" by the movement, and this leads to a rather counterintuitive prediction that faster movement, where a larger number of motor primitives are involved, allows learning more complicated motor skills. Here, we provide a set of experimental results that support this hypothesis. First, we show that learning occurs only with movement, that is, only with nonzero encoding capacity. When participants were asked to counteract a rotating force applied to a robotic handle, they were unable to do so when maintaining a static posture but were able to adapt when making small circular movements. Our second experiment further investigated how adaptation is affected by movement speed. When adapting to a simple (low-information-content) force field, fast (high-capacity) movement did not have an advantage over slow (low-capacity) movement. However, for a complex (high-information-content) force field, the fast movement showed a significant advantage over slow movement. Our final experiment confirmed that the observed benefit of high-speed movement is only weakly affected by mechanical factors. Taken together, our results suggest that the encoding capacity is a genuine limiting factor of human motor adaptation.NEW & NOTEWORTHY We propose a novel concept called "encoding capacity" of motor adaptation, which describes an inherent limiting-factor of our brain's ability to learn new motor skills, just like any other storage system. By reinterpreting the existing primitive-based models of motor learning, we hypothesize that the encoding capacity is determined by the size of the movement, and present a set of experimental evidence suggesting that such limiting effect of encoding capacity does exist in human motor adaptation.

A Multimodal Analysis Combining Behavioral Experiments and Survey-Based Methods to Assess the Cognitive Effect of Video Game Playing: Good or Evil?

This study aims to bridge the gap between the discrepant views of existing studies in different modalities on the cognitive effect of video game play. To this end, we conducted a set of tests with different modalities within each participant: (1) Self-Reports Analyses (SRA) consisting of five popular self-report surveys, and (2) a standard Behavioral Experiment (BE) using pro- and antisaccade paradigms, and analyzed how their results vary between Video Game Player (VGP) and Non-Video Game Player (NVGP) participant groups. Our result showed that (1) VGP scored significantly lower in Behavioral Inhibition System (BIS) than NVGP (p = 0.023), and (2) VGP showed significantly higher antisaccade error rate than NVGP (p = 0.005), suggesting that results of both SRA and BE support the existing view that video game play has a maleficent impact on the cognition by increasing impulsivity. However, the following correlation analysis on the results across individual participants found no significant correlation between SRA and BE, indicating a complex nature of the cognitive effect of video game play.

When Optimal Feedback Control Is Not Enough: Feedforward Strategies Are Required for Optimal Control with Active Sensing.

Movement planning is thought to be primarily determined by motor costs such as inaccuracy and effort. Solving for the optimal plan that minimizes these costs typically leads to specifying a time-varying feedback controller which both generates the movement and can optimally correct for errors that arise within a movement. However, the quality of the sensory feedback during a movement can depend substantially on the generated movement. We show that by incorporating such state-dependent sensory feedback, the optimal solution incorporates active sensing and is no longer a pure feedback process but includes a significant feedforward component. To examine whether people take into account such state-dependency in sensory feedback we asked people to make movements in which we controlled the reliability of sensory feedback. We made the visibility of the hand state-dependent, such that the visibility was proportional to the component of hand velocity in a particular direction. Subjects gradually adapted to such a sensory perturbation by making curved hand movements. In particular, they appeared to control the late visibility of the movement matching predictions of the optimal controller with state-dependent sensory noise. Our results show that trajectory planning is not only sensitive to motor costs but takes sensory costs into account and argues for optimal control of movement in which feedforward commands can play a significant role.

Eye movement accuracy determines natural interception strategies.

Eye movements aid visual perception and guide actions such as reaching or grasping. Most previous work on eye-hand coordination has focused on saccadic eye movements. Here we show that smooth pursuit eye movement accuracy strongly predicts both interception accuracy and the strategy used to intercept a moving object. We developed a naturalistic task in which participants (n = 42 varsity baseball players) intercepted a moving dot (a "2D fly ball") with their index finger in a designated "hit zone." Participants were instructed to track the ball with their eyes, but were only shown its initial launch (100-300 ms). Better smooth pursuit resulted in more accurate interceptions and determined the strategy used for interception, i.e., whether interception was early or late in the hit zone. Even though early and late interceptors showed equally accurate interceptions, they may have relied on distinct tactics: early interceptors used cognitive heuristics, whereas late interceptors' performance was best predicted by pursuit accuracy. Late interception may be beneficial in real-world tasks as it provides more time for decision and adjustment. Supporting this view, baseball players who were more senior were more likely to be late interceptors. Our findings suggest that interception strategies are optimally adapted to the proficiency of the pursuit system.

Coordinate Representations for Interference Reduction in Motor Learning.

When opposing force fields are presented alternately or randomly across trials for identical reaching movements, subjects learn neither force field, a behavior termed 'interference'. Studies have shown that a small difference in the endpoint posture of the limb reduces this interference. However, any difference in the limb's endpoint location typically changes the hand position, joint angles and the hand orientation making it ambiguous as to which of these changes underlies the ability to learn dynamics that normally interfere. Here we examine the extent to which each of these three possible coordinate systems--Cartesian hand position, shoulder and elbow joint angles, or hand orientation--underlies the reduction in interference. Subjects performed goal-directed reaching movements in five different limb configurations designed so that different pairs of these configurations involved a change in only one coordinate system. By specifically assigning clockwise and counter-clockwise force fields to the configurations we could create three different conditions in which the direction of the force field could only be uniquely distinguished in one of the three coordinate systems. We examined the ability to learn the two fields based on each of the coordinate systems. The largest reduction of interference was observed when the field direction was linked to the hand orientation with smaller reductions in the other two conditions. This result demonstrates that the strongest reduction in interference occurred with changes in the hand orientation, suggesting that hand orientation may have a privileged role in reducing motor interference for changes in the endpoint posture of the limb.

Phenomenological models of the dynamics of muscle during isotonic shortening.

We investigated the effectiveness of simple, Hill-type, phenomenological models of the force-length-velocity relationship for simulating measured length trajectories during muscle shortening, and, if so, what forms of the model are most useful. Using isotonic shortening data from mouse soleus and toad depressor mandibulae muscles, we showed that Hill-type models can indeed simulate the shortening trajectories with sufficiently good accuracy. However, we found that the standard form of the Hill-type muscle model, called the force-scaling model, is not a satisfactory choice. Instead, the results support the use of less frequently used models, the f-max scaling model and force-scaling with parallel spring, to simulate the shortening dynamics of muscle.

Transducer and base compliance alter the in situ 6 dof force measured from muscle during an isometric contraction in a multi-joint limb.

Although musculoskeletal models are commonly used, validating the muscle actions predicted by such models is often difficult. In situ isometric measurements are a possible solution. The base of the skeleton is immobilized and the endpoint of the limb is rigidly attached to a 6-axis force transducer. Individual muscles are stimulated and the resulting forces and moments recorded. Such analyses generally assume idealized conditions. In this study we have developed an analysis taking into account the compliances due to imperfect fixation of the skeleton, imperfect attachment of the force transducer, and extra degrees of freedom (dof) in the joints that sometimes become necessary in fixed end contractions. We use simulations of the rat hindlimb to illustrate the consequences of such compliances. We show that when the limb is overconstrained, i.e., when there are fewer dof within the limb than are restrained by the skeletal fixation, the compliances of the skeletal fixation and of the transducer attachment can significantly affect measured forces and moments. When the limb dofs and restrained dofs are matched, however, the measured forces and moments are independent of these compliances. We also show that this framework can be used to model limb dofs, so that rather than simply omitting dofs in which a limb does not move (e.g., abduction at the knee), the limited motion of the limb in these dofs can be more realistically modeled as a very low compliance. Finally, we discuss the practical implications of these results to experimental measurements of muscle actions.

Is titin a 'winding filament'? A new twist on muscle contraction.

Recent studies have demonstrated a role for the elastic protein titin in active muscle, but the mechanisms by which titin plays this role remain to be elucidated. In active muscle, Ca(2+)-binding has been shown to increase titin stiffness, but the observed increase is too small to explain the increased stiffness of parallel elastic elements upon muscle activation. We propose a 'winding filament' mechanism for titin's role in active muscle. First, we hypothesize that Ca(2+)-dependent binding of titin's N2A region to thin filaments increases titin stiffness by preventing low-force straightening of proximal immunoglobulin domains that occurs during passive stretch. This mechanism explains the difference in length dependence of force between skeletal myofibrils and cardiac myocytes. Second, we hypothesize that cross-bridges serve not only as motors that pull thin filaments towards the M-line, but also as rotors that wind titin on the thin filaments, storing elastic potential energy in PEVK during force development and active stretch. Energy stored during force development can be recovered during active shortening. The winding filament hypothesis accounts for force enhancement during stretch and force depression during shortening, and provides testable predictions that will encourage new directions for research on mechanisms of muscle contraction.

Estimation of musculoskeletal models from in situ measurements of muscle action in the rat hindlimb.

Musculoskeletal models are often created by making detailed anatomical measurements of muscle properties. These measurements can then be used to determine the parameters of canonical models of muscle action. We describe here a complementary approach for developing and validating muscle models, using in situ measurements of muscle actions. We characterized the actions of two rat hindlimb muscles: the gracilis posticus (GRp) and the posterior head of biceps femoris (BFp; excluding the anterior head with vertebral origin). The GRp is a relatively simple muscle, with a circumscribed origin and insertion. The BFp is more complex, with an insertion distributed along the tibia. We measured the six-dimensional isometric forces and moments at the ankle evoked from stimulating each muscle at a range of limb configurations. The variation of forces and moments across the workspace provides a succinct characterization of muscle action. We then used this data to create a simple muscle model with a single point insertion and origin. The model parameters were optimized to best explain the observed force-moment data. This model explained the relatively simple muscle, GRp, very well (R(2)>0.85). Surprisingly, this simple model was also able to explain the action of the BFp, despite its greater complexity (R(2)>0.84). We then compared the actions observed here with those predicted using recently published anatomical measurements. Although the forces and moments predicted for the GRp were very similar to those observed here, the predictions for the BFp differed. These results show the potential utility of the approach described here for the development and refinement of musculoskeletal models based on in situ measurements of muscle actions.

Understanding complex muscles in the rat hindlimb: Activations and actions.

We present research examining the function of complex muscles in the rat hindlimb. Two related sets of experiments are described. In the first, we examine the degree of specificity in spinal pattern generators, assessing whether the pattern generators at birth are capable of differentially activating intramuscular subdivisions in the complex hindlimb muscle biceps femoris. In the second, we describe a novel approach for creating a musculoskeletal model to capture the mechanical actions of individual muscles and evaluate its ability to capture the action of both simple and complex muscles in the rat hindlimb.

Optimal design of musculoskeletal models using force field data.

We propose an optimal design framework for a musculoskeletal model. A six degrees of freedom generalized force field at an end effector is measured by activating a specific muscle in-situ. Given that we have a musculoskeletal simulation model with a set of undetermined design parameters, the framework chooses the optimal parameters which minimize the difference between simulated and measured force field data. To ensure generality, the framework is model independent, which means that any type of musculoskeletal model can be applied. As a generalized method, a PID (Proportional-Integral-Derivative) controller and an empirical Jacobian are used to correctly simulate the force field without suffering from numerical issues such as stability and indeterminacy. Case studies of the muscles of the rat hind limb show a satisfactory capability of the framework.