An observational study to assess validity and reliability of smartphone sensor-based gait and balance assessments in multiple sclerosis: Floodlight GaitLab protocol.

Background: Gait and balance impairments are often present in people with multiple sclerosis (PwMS) and have a significant impact on quality of life and independence. Gold-standard quantitative tools for assessing gait and balance such as motion capture systems and force plates usually require complex technical setups. Wearable sensors, including those integrated into smartphones, offer a more frequent, convenient, and minimally burdensome assessment of functional disability in a home environment. We developed a novel smartphone sensor-based application (Floodlight) that is being used in multiple research and clinical contexts, but a complete validation of this technology is still lacking. Methods: This protocol describes an observational study designed to evaluate the analytical and clinical validity of Floodlight gait and balance tests. Approximately 100 PwMS and 35 healthy controls will perform multiple gait and balance tasks in both laboratory-based and real-world environments in order to explore the following properties: (a) concurrent validity of the Floodlight gait and balance tests against gold-standard assessments; (b) reliability of Floodlight digital measures derived under different controlled gait and balance conditions, and different on-body sensor locations; (c) ecological validity of the tests; and (d) construct validity compared with clinician- and patient-reported assessments. Conclusions: The Floodlight GaitLab study (ISRCTN15993728) represents a critical step in the technical validation of Floodlight technology to measure gait and balance in PwMS, and will also allow the development of new test designs and algorithms.

Predicting critical state after COVID-19 diagnosis: model development using a large US electronic health record dataset.

As the COVID-19 pandemic is challenging healthcare systems worldwide, early identification of patients with a high risk of complication is crucial. We present a prognostic model predicting critical state within 28 days following COVID-19 diagnosis trained on data from US electronic health records (IBM Explorys), including demographics, comorbidities, symptoms, and hospitalization. Out of 15753 COVID-19 patients, 2050 went into critical state or deceased. Non-random train-test splits by time were repeated 100 times and led to a ROC AUC of 0.861 [0.838, 0.883] and a precision-recall AUC of 0.434 [0.414, 0.485] (median and interquartile range). The interpretability analysis confirmed evidence on major risk factors (e.g., older age, higher BMI, male gender, diabetes, and cardiovascular disease) in an efficient way compared to clinical studies, demonstrating the model validity. Such personalized predictions could enable fine-graded risk stratification for optimized care management.

A data-driven framework for selecting and validating digital health metrics: use-case in neurological sensorimotor impairments.

Digital health metrics promise to advance the understanding of impaired body functions, for example in neurological disorders. However, their clinical integration is challenged by an insufficient validation of the many existing and often abstract metrics. Here, we propose a data-driven framework to select and validate a clinically relevant core set of digital health metrics extracted from a technology-aided assessment. As an exemplary use-case, the framework is applied to the Virtual Peg Insertion Test (VPIT), a technology-aided assessment of upper limb sensorimotor impairments. The framework builds on a use-case-specific pathophysiological motivation of metrics, models demographic confounds, and evaluates the most important clinimetric properties (discriminant validity, structural validity, reliability, measurement error, learning effects). Applied to 77 metrics of the VPIT collected from 120 neurologically intact and 89 affected individuals, the framework allowed selecting 10 clinically relevant core metrics. These assessed the severity of multiple sensorimotor impairments in a valid, reliable, and informative manner. These metrics provided added clinical value by detecting impairments in neurological subjects that did not show any deficits according to conventional scales, and by covering sensorimotor impairments of the arm and hand with a single assessment. The proposed framework provides a transparent, step-by-step selection procedure based on clinically relevant evidence. This creates an interesting alternative to established selection algorithms that optimize mathematical loss functions and are not always intuitive to retrace. This could help addressing the insufficient clinical integration of digital health metrics. For the VPIT, it allowed establishing validated core metrics, paving the way for their integration into neurorehabilitation trials.

Age-Dependent Asymmetry of Wrist Position Sense Is Not Influenced by Stochastic Tactile Stimulation.

Stochastic stimulation has been shown to improve movement, balance, the sense of touch, and may also improve position sense. This stimulation can be non-invasive and may be a simple technology to enhance proprioception. In this study, we investigated whether sub-threshold stochastic tactile stimulation of mechanoreceptors reduces age-related errors in wrist position estimation. Fifteen young (24.5±1.5y) and 23 elderly (71.7±7.3y) unimpaired, right-handed adults completed a wrist position gauge-matching experiment. In each trial, the participant's concealed wrist was moved to a target position between 10 and 30° of wrist flexion or extension by a robotic manipulandum. The participant then estimated the wrist's position on a virtual gauge. During half of the trials, sub-threshold stochastic tactile stimulation was applied to the wrist muscle tendon areas. Stochastic stimulation did not significantly influence wrist position sense. In the elderly group, estimation errors decreased non-significantly when stimulation was applied compared to the trials without stimulation [mean constant error reduction Δ µ ( θ c o n o f ) = 0 . 8 ° in flexion and Δ µ ( θ c o n o e ) = 0 . 7 ° in extension direction, p = 0.95]. This effect was less pronounced in the young group [ Δ µ ( θ c o n y ) = 0 . 2 ° in flexion and in extension direction, p = 0.99]. These improvements did not yield a relevant effect size (Cohen's d < 0.1). Estimation errors increased with target angle magnitude in both movement directions. In young participants, estimation errors were non-symmetric, with estimations in flexion [ µ ( θ c o n y f ) = 1 . 8 ° , σ ( θ c o n y f ) = 7 . 0 ° ] being significantly more accurate than in extension [ µ ( θ c o n y e ) = 8 . 3 ° , σ ( θ c o n y e ) = 9 . 3 ° , p < 0.01]. This asymmetry was not present in the elderly group, where estimations in flexion [ µ ( θ c o n o f ) = 7 . 5 ° , σ ( θ c o n o f ) = 9 . 8 ° ] were similar to extension [ µ ( θ c o n o e ) = 7 . 7 ° , σ ( θ c o n o e ) = 9 . 3 ° ]. Hence, young and elderly participants performed equally in extension direction, whereas wrist position sense in flexion direction deteriorated with age (p < 0.01). Though unimpaired elderly adults did not benefit from stochastic stimulation, it cannot be deduced that individuals with more severe impairments of their sensory system do not profit from this treatment. While the errors in estimating wrist position are symmetric in flexion and extension in elderly adults, young adults are more accurate when estimating wrist flexion, an effect that has not been described before.

Design and Characterization of a Robotic Device for the Assessment of Hand Proprioceptive, Motor, and Sensorimotor Impairments.

Hand function is often impaired after neurological injuries such as stroke. In order to design patient-specific rehabilitation, it is essential to quantitatively assess those deficits. Current clinical scores cannot provide the required level of detail, and most assessment devices have been developed for the proximal joints of the upper limb. This paper presents a new robotic platform for the assessment of proprioceptive, motor, and sensorimotor hand impairments. A detailed technical evaluation demonstrated the capabilities to render different haptic environments required for a comprehensive assessment battery, and showed that the device is suitable for human interaction due to its ergonomic design. A preliminary study on proprioceptive assessment using a gauge position matching task with one healthy, one stroke, and one multiple sclerosis subject showed that the robotic system is able to rapidly and sensitively quantify proprioceptive deficits, and has the potential to be integrated into the clinical settings.

Automated and Quantitative Assessment of Tactile Mislocalization After Stroke.

Topesthesia, the recognition of tactile stimulation location on the skin, can be severely affected by neurological injuries, such as stroke. Despite topesthesia being crucial for manipulating objects and interacting with the environment during activities of daily living, deficits cannot be quantitatively captured with current clinical assessments and are, as a consequence, not well-understood. The present work describes a novel automated assessment tool for tactile mislocalization in neurological patients with somatosensory deficits. We present two cases of ischemic stroke patients, describe their tactile localization deficits with the automated assessment, and compare the results to a standard manual clinical assessment. Using the automated assessment tool, it was possible to identify, locate, precisely quantify, and depict the patients' deficits in topesthesia. In comparison, the clinical assessment was not sensitive enough and some deficits would remain undetected due to ceiling effects. In addition, an MRI structural analysis of the lesion supported the existence of somatosensory deficits. This novel and quantitative assessment may not only help to raise awareness of the implications of deficits in topesthesia, but would also allow monitoring recovery throughout the rehabilitation process, informing treatment design, and objectively evaluating treatment efficacy.

Performance metrics for an application-driven selection and optimization of psychophysical sampling procedures.

When estimating psychometric functions with sampling procedures, psychophysical assessments should be precise and accurate while being as efficient as possible to reduce assessment duration. The estimation performance of sampling procedures is commonly evaluated in computer simulations for single psychometric functions and reported using metrics as a function of number of trials. However, the estimation performance of a sampling procedure may vary for different psychometric functions. Therefore, the results of these type of evaluations may not be generalizable to a heterogeneous population of interest. In addition, the maximum number of trials is often imposed by time restrictions, especially in clinical applications, making trial-based metrics suboptimal. Hence, the benefit of these simulations to select and tune an ideal sampling procedure for a specific application is limited. We suggest to evaluate the estimation performance of sampling procedures in simulations covering the entire range of psychometric functions found in a population of interest, and propose a comprehensive set of performance metrics for a detailed analysis. To illustrate the information gained from these metrics in an application example, six sampling procedures were evaluated in a computer simulation based on prior knowledge on the population distribution and requirements from proprioceptive assessments. The metrics revealed limitations of the sampling procedures, such as inhomogeneous or systematically decreasing performance depending on the psychometric functions, which can inform the tuning process of a sampling procedure. More advanced metrics allowed directly comparing overall performances of different sampling procedures and select the best-suited sampling procedure for the example application. The proposed analysis metrics can be used for any sampling procedure and the estimation of any parameter of a psychometric function, independent of the shape of the psychometric function and of how such a parameter was estimated. This framework should help to accelerate the development process of psychophysical assessments.

Reliability, validity, and clinical feasibility of a rapid and objective assessment of post-stroke deficits in hand proprioception.

BACKGROUND: Proprioceptive function can be affected after neurological injuries such as stroke. Severe and persistent proprioceptive impairments may be associated with a poor functional recovery after stroke. To better understand their role in the recovery process, and to improve diagnostics, prognostics, and the design of therapeutic interventions, it is essential to quantify proprioceptive deficits accurately and sensitively. However, current clinical assessments lack sensitivity due to ordinal scales and suffer from poor reliability and ceiling effects. Robotic technology offers new possibilities to address some of these limitations. Nevertheless, it is important to investigate the psychometric and clinimetric properties of technology-assisted assessments. METHODS: We present an automated robot-assisted assessment of proprioception at the level of the metacarpophalangeal joint, and evaluate its reliability, validity, and clinical feasibility in a study with 23 participants with stroke and an age-matched group of 29 neurologically intact controls. The assessment uses a two-alternative forced choice paradigm and an adaptive sampling procedure to identify objectively the difference threshold of angular joint position. RESULTS: Results revealed a good reliability (ICC(2,1) = 0.73) for assessing proprioception of the impaired hand of participants with stroke. Assessments showed similar task execution characteristics (e.g., number of trials and duration per trial) between participants with stroke and controls and a short administration time of approximately 12 min. A difference in proprioceptive function could be found between participants with a right hemisphere stroke and control subjects (p<0.001). Furthermore, we observed larger proprioceptive deficits in participants with a right hemisphere stroke compared to a left hemisphere stroke (p=0.028), despite the exclusion of participants with neglect. No meaningful correlation could be established with clinical scales for different modalities of somatosensation. We hypothesize that this is due to their low resolution and ceiling effects. CONCLUSIONS: This study has demonstrated the assessment's applicability in the impaired population and promising integration into clinical routine. In conclusion, the proposed assessment has the potential to become a powerful tool to investigate proprioceptive deficits in longitudinal studies as well as to inform and adjust sensorimotor rehabilitation to the patient's deficits.

Algorithm for improving psychophysical threshold estimates by detecting sustained inattention in experiments using PEST.

Psychophysical procedures are applied in various fields to assess sensory thresholds. During experiments, sampled psychometric functions are usually assumed to be stationary. However, perception can be altered, for example by loss of attention to the presentation of stimuli, leading to biased data, which results in poor threshold estimates. The few existing approaches attempting to identify non-stationarities either detect only whether there was a change in perception, or are not suitable for experiments with a relatively small number of trials (e.g., [Formula: see text] 300). We present a method to detect inattention periods on a trial-by-trial basis with the aim of improving threshold estimates in psychophysical experiments using the adaptive sampling procedure Parameter Estimation by Sequential Testing (PEST). The performance of the algorithm was evaluated in computer simulations modeling inattention, and tested in a behavioral experiment on proprioceptive difference threshold assessment in 20 stroke patients, a population where attention deficits are likely to be present. Simulations showed that estimation errors could be reduced by up to 77% for inattentive subjects, even in sequences with less than 100 trials. In the behavioral data, inattention was detected in 14% of assessments, and applying the proposed algorithm resulted in reduced test-retest variability in 73% of these corrected assessments pairs. The novel algorithm complements existing approaches and, besides being applicable post hoc, could also be used online to prevent collection of biased data. This could have important implications in assessment practice by shortening experiments and improving estimates, especially for clinical settings.

Enhancing simulations with intra-subject variability for improved psychophysical assessments.

Psychometric properties of perceptual assessments, like reliability, depend on stochastic properties of psychophysical sampling procedures resulting in method variability, as well as inter- and intra-subject variability. Method variability is commonly minimized by optimizing sampling procedures through computer simulations. Inter-subject variability is inherent to the population of interest and cannot be influenced. Intra-subject variability introduced by confounds (e.g., inattention or lack of motivation) cannot be simply quantified from experimental data, as these data also include method variability. Therefore, this aspect is generally neglected when developing assessments. Yet, comparing method variability and intra-subject variability could give insights on whether effort should be invested in optimizing the sampling procedure, or in addressing potential confounds instead. We propose a new approach to estimate intra-subject variability of psychometric functions by combining computer simulations and behavioral data, and to account for it when simulating experiments. The approach was illustrated in a real-world scenario of proprioceptive difference threshold assessments. The behavioral study revealed a test-retest reliability of r = 0.212. Computer simulations without considering intra-subject variability predicted a reliability of r = 0.768, whereas the new approach including an intra-subject variability model lead to a realistic estimate of reliability (r = 0.207). Such a model also allows computing the theoretically maximally attainable reliability (r = 0.552) assuming an ideal sampling procedure. Comparing the reliability estimates when exclusively accounting for method variability versus intra-subject variability reveals that intra-subject variability should be reduced by addressing confounds and that only optimizing the sampling procedure may be insufficient to achieve a high reliability. This new approach allows computing the intra-subject variability with only two measurements per subject, and predicting the reliability for a larger number of subjects and retests based on simulations, without requiring additional experiments. Such a tool of predictive value is especially valuable for target populations where time is scarce, e.g., for assessments in clinical settings.

A novel pneumatic stimulator for the investigation of noise-enhanced proprioception.

Executing coordinated movements requires that motor and sensory systems cooperate to achieve a motor goal. Impairment of either system may lead to unstable and/or inaccurate movements. In rehabilitation training, however, most approaches have focused on the motor aspects of the control loop. We are examining mechanisms that may enhance the sensory system to improve motor control. More precisely, the effects of stochastic subliminal vibratory tactile stimulation on wrist proprioception. We developed a device - based on a novel soft pneumatic actuator skin technology - to stimulate multiple sites simultaneously and independently. This device applies vibratory stimulation (amplitude < 0.50 mm, bandwidth 20-120 Hz) to the skin overlaying the tendons of a joint to target the receptors in charge of position and movement encoding. It achieves high spatial resolution (< 1 mm2), uses a soft and flexible interface, and has the potential to be used in combination with additional rehabilitation interventions. We conducted a feasibility study with 16 healthy subjects (11 younger - 6 females; 5 older - 2 females) in which a robotic manipulandum moved the subject's wrist to defined positions that had to be matched with a gauge. Comparing trials with and without stimulation we found that stochastic stimulation influenced joint position sense. The device we developed can be readily used in psycho-physical experiments, and subsequently benefit physiotherapy and rehabilitation treatments.

Age-based model for metacarpophalangeal joint proprioception in elderly.

Neurological injuries such as stroke can lead to proprioceptive impairment. For an informed diagnosis, prognosis, and treatment planning, it is essential to be able to distinguish between healthy performance and deficits following the neurological injury. Since there is some evidence that proprioception declines with age and stroke occurs predominantly in the elderly population, it is important to create a healthy reference model in this specific age group. However, most studies investigate age effects by comparing young and elderly subjects and do not provide a model within a target age range. Moreover, despite the functional relevance of the hand in activities of daily living, age-based models of distal proprioception are scarce. Here, we present a proprioception model based on the assessment of the metacarpophalangeal joint angle difference threshold in 30 healthy elderly subjects, aged 55-80 years (median: 63, interquartile range: 58-66), using a robotic tool to apply passive flexion-extension movements to the index finger. A two-alternative forced-choice paradigm combined with an adaptive algorithm to define stimulus magnitude was used. The mixed-effects model analysis revealed that aging has a significant, increasing effect on the difference threshold at the metacarpophalangeal joint, whereas other predictors (eg, tested hand or sex) did not show a significant effect. The adaptive algorithm allowed reaching an average assessment duration <15 minutes, making its clinical applicability realistic. This study provides further evidence for an age-related decline in proprioception at the level of the hand. The established age-based model of proprioception in elderly may serve as a reference model for the proprioceptive performance of stroke patients, or of any other patient group with central or peripheral proprioceptive impairments. Furthermore, it demonstrates the potential of such automated robotic tools as a rapid and quantitative assessment to be used in research and clinical settings.

Reliable and Rapid Robotic Assessment of Wrist Proprioception Using a Gauge Position Matching Paradigm.

Quantitative assessments of position sense are essential for the investigation of proprioception, as well as for diagnosis, prognosis and treatment planning for patients with somatosensory deficits. Despite the development and use of various paradigms and robotic tools, their clinimetric properties are often poorly evaluated and reported. A proper evaluation of the latter is essential to compare results between different studies and to identify the influence of possible confounds on outcome measures. The aim of the present study was to perform a comprehensive evaluation of a rapid robotic assessment of wrist proprioception using a passive gauge position matching task. Thirty-two healthy subjects undertook six test-retests of proprioception of the right wrist on two different days. The constant error (CE) was 0.87°, the absolute error (AE) was 5.87°, the variable error (VE) was 4.59° and the total variability (E) was 6.83° in average for the angles presented in the range from 10° to 30°. The intraclass correlation analysis provided an excellent reliability for CE (0.75), good reliability for AE (0.68) and E (0.68), and fair reliability for VE (0.54). Tripling the assessment length had negligible effects on the reliabilities. Additional analysis revealed significant trends of larger overestimation (constant errors), as well as larger absolute and variable errors with increased flexion angles. No proprioceptive learning occurred, despite increased familiarity with the task, which was reflected in significantly decreased assessment duration by 30%. In conclusion, the proposed automated assessment can provide sensitive and reliable information on proprioceptive function of the wrist with an administration time of around 2.5 min, demonstrating the potential for its application in research or clinical settings. Moreover, this study highlights the importance of reporting the complete set of errors (CE, AE, VE, and E) in a matching experiment for the identification of trends and subsequent interpretation of results.

Learning and generalization in an isometric visuomotor task.

Adaptation is a prominent feature of the human motor system and has been studied extensively in reaching movements. This study characterizes adaptation and generalization during isometric reaching in which the arm remains stationary and the participant controls a virtual cursor via force applied by the hand. We measured how learning of a visual cursor rotation generalizes across workspace 1) to determine the coordinate system that predominates visual rotation learning, and 2) to ascertain whether mapping type, namely position or velocity control, influences transfer. Participants performed virtual reaches to one of two orthogonal training targets with the applied rotation. In a new workspace, participants reached to a single target, similar to the training target in either hand or joint space. Furthermore, a control experiment measured within-workspace generalization to an orthogonal target. Across position and velocity mappings, learning transferred predominantly in intrinsic (joint) space, although the transfer was incomplete. The velocity mapping resulted in significantly larger aftereffects and broader within-workspace generalization than the position mapping, potentially due to slower peak speeds, longer trial times, greater target overshoot, or other factors. Although we cannot rule out a mixed reference frame in our task, the predominance of intrinsic coding of cursor kinematics in the isometric environment opposes the extrinsic coding of arm kinematics in real reaching but matches the intrinsic coding of dynamics found in prior work. These findings have implications for the design of isometric control systems in human-machine interaction or in rehabilitation when coordinated multi-degree-of-freedom movement is difficult to achieve.