Test-Retest Reliability, Criterion-Related Validity, and Ecological Validity of the Test of Visual-Motor Skills, Third Edition, in Kindergarten Children With Developmental Coordination Disorder.

IMPORTANCE: Establishing empirical evidence on the psychometric properties of the Test of Visual-Motor Skills (3rd ed.; TVMS-3) is helpful for guiding its use as an assessment of visual-motor integration (VMI) skills in kindergarten children with developmental coordination disorder (DCD). OBJECTIVE: To investigate the test-retest reliability, criterion-related validity, and ecological validity of the TVMS-3 in Taiwanese kindergarten children with DCD. DESIGN: A nonexperimental, descriptive, correlational design. SETTING: A hospital in Central Taiwan. PARTICIPANTS: Fifty-seven kindergarten children with DCD were recruited in the study. OUTCOMES AND MEASURES: Intraclass correlation coefficient, percentage of minimal detectable change, and paired t test (Wilcoxon signed rank test) were used to investigate the test-retest reliability of the TVMS-3. The correlations (Pearson's r) between the TVMS-3 accuracy score and the scores of each of the four domains and the adaptive behavior composite score of the Vineland Adaptive Behavior Scales (3rd ed.; Vineland-3) were calculated, respectively, to examine criterion-related validity and ecological validity. RESULTS: The accuracy score of the TVMS-3 had excellent test-retest reliability and acceptable random measurement error. Moreover, it showed good criterion-related validity and sufficient ecological validity with the Vineland-3 in Taiwanese kindergarten children with DCD. CONCLUSIONS AND RELEVANCE: The accuracy score of the TVMS-3 is applicable to Taiwanese kindergarten children with DCD in clinical and research settings. Plain-Language Summary: The accuracy score of the Test of Visual-Motor Skills (3rd ed.; TVMS-3) is a useful assessment tool to detect deficits in visual-motor integration for Taiwanese kindergarten children with developmental coordination disorder. The TVMS-3 has excellent test-retest reliability, good criterion-related validity, and sufficient ecological validity.

Relationship between intraindividual auditory and visual attention in children with ADHD.

BACKGROUND AND AIM: Most previous attention-deficit/hyperactivity disorder (ADHD) studies have used only a single sensory modality (usually vision) to investigate attentional problems, although patients with ADHD might display deficits of auditory attention similar to their visual attention. This study explored intraindividual auditory and visual attention in children with and without ADHD to examine the relationship between these two dimensions of attention. METHODS: Attentional performances of 140 children (70 children with ADHD and 70 typically developing peers) were measured through the Test of Variables of Attention (TOVA) in the present study. RESULTS: For both groups, most attentional indices showed significant differences between the two modalities (d ranging from 0.32 to 0.72). The correlation coefficients of most of the attentional variables in children with ADHD were lower than their typically developing peers. All attentional indices of children with ADHD (ranging from 12.8%-55.7%) were much higher than those of their typically developing peers (ranging from 1.4%-8.6%). CONCLUSION: These results not only indicate that typically developing children display more consistent attentional performance, but also support the view that children with ADHD may show attention deficiency in one modality but not necessarily in the other.

Responsiveness and predictive validity of the Tablet-based Symbol Digit Modalities Test in patients with stroke.

BACKGROUND: The responsiveness and predictive validity of the Tablet-based Symbol Digit Modalities Test (T-SDMT) are unknown, which limits the utility of the T-SDMT in both clinical and research settings. AIM: The purpose of this study was to examine the responsiveness and predictive validity of the T-SDMT in inpatients with stroke. DESIGN: A follow-up, repeated-assessments design. SETTING: One rehabilitation unit at a local medical center. POPULATION: A total of 50 inpatients receiving rehabilitation completed T-SDMT assessments at admission to and discharge from a rehabilitation ward. METHODS: The median follow-up period was 14 days. The Barthel index (BI) was assessed at discharge and was used as the criterion of the predictive validity. RESULTS: The mean changes in the T-SDMT scores between admission and discharge were statistically significant (paired t-test = 3.46, P=0.001). The T-SDMT scores showed a nearly moderate standardized response mean (0.49). A moderate association (Pearson's r =0.47) was found between the scores of the T-SDMT at admission and those of the BI at discharge, indicating good predictive validity of the T-SDMT. CONCLUSIONS: Our results support the responsiveness and predictive validity of the T-SDMT in patients with stroke receiving rehabilitation in hospitals. CLINICAL REHABILITATION IMPACT: This study provides empirical evidence supporting the use of the T-SDMT as an outcome measure for assessing processing speed in inpatients with stroke. The scores of the T-SDMT could be used to predict basic activities of daily living function in inpatients with stroke.

Interrater and Intrarater Reliability of the Balance Computerized Adaptive Test in Patients With Stroke.

OBJECTIVE: To examine the interrater and intrarater reliability of the Balance Computerized Adaptive Test (Balance CAT) in patients with chronic stroke having a wide range of balance functions. DESIGN: Repeated assessments design (1wk apart). SETTING: Seven teaching hospitals. PARTICIPANTS: A pooled sample (N=102) including 2 independent groups of outpatients (n=50 for the interrater reliability study; n=52 for the intrarater reliability study) with chronic stroke. INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: Balance CAT. RESULTS: For the interrater reliability study, the values of intraclass correlation coefficient, minimal detectable change (MDC), and percentage of MDC (MDC%) for the Balance CAT were .84, 1.90, and 31.0%, respectively. For the intrarater reliability study, the values of intraclass correlation coefficient, MDC, and MDC% ranged from .89 to .91, from 1.14 to 1.26, and from 17.1% to 18.6%, respectively. CONCLUSIONS: The Balance CAT showed sufficient intrarater reliability in patients with chronic stroke having balance functions ranging from sitting with support to independent walking. Although the Balance CAT may have good interrater reliability, we found substantial random measurement error between different raters. Accordingly, if the Balance CAT is used as an outcome measure in clinical or research settings, same raters are suggested over different time points to ensure reliable assessments.

A comparison between the original and Tablet-based Symbol Digit Modalities Test in patients with schizophrenia: Test-retest agreement, random measurement error, practice effect, and ecological validity.

We aimed to compare the test-retest agreement, random measurement error, practice effect, and ecological validity of the original and Tablet-based Symbol Digit Modalities Test (T-SDMT) over five serial assessments, and to examine the concurrent validity of the T-SDMT in patients with schizophrenia. Sixty patients with chronic schizophrenia completed five serial assessments (one week apart) of the SDMT and T-SDMT and one assessment of the Activities of Daily Living Rating Scale III at the first time point. Both measures showed high test-retest agreement, similar levels of random measurement error over five serial assessments. Moreover, the practice effects of the two measures did not reach a plateau phase after five serial assessments in young and middle-aged participants. Nevertheless, only the practice effect of the T-SDMT became trivial after the first assessment. Like the SDMT, the T-SDMT had good ecological validity. The T-SDMT also had good concurrent validity with the SDMT. In addition, only the T-SDMT had discriminative validity to discriminate processing speed in young and middle-aged participants. Compared to the SDMT, the T-SDMT had overall slightly better psychometric properties, so it can be an alternative measure to the SDMT for assessing processing speed in patients with schizophrenia.

Test-retest reliability and responsiveness of the Barthel Index-based Supplementary Scales in patients with stroke.

BACKGROUND: A lack of evidence on the test-retest reliability and responsiveness limits the utility of the BI-based Supplementary Scales (BI-SS) in both clinical and research settings. AIM: To examine the test-retest reliability and responsiveness of the BI-based Supplementary Scales (BI-SS) in patients with stroke. DESIGN: A repeated-assessments design (1 week apart) was used to examine the test-retest reliability of the BI-SS. For the responsiveness study, the participants were assessed with the BI-SS and BI (treated as an external criterion) at admission to and discharge from rehabilitation wards. SETTING: Seven outpatient rehabilitation units and one inpatient rehabilitation unit. POPULATION: Outpatients with chronic stroke. METHODS: Eighty-four outpatients with chronic stroke participated in the test-retest reliability study. Fifty-seven inpatients completed baseline and follow-up assessments in the responsiveness study. RESULTS: For the test-retest reliability study, the values of the intra-class correlation coefficient and the overall percentage of minimal detectable change for the Ability Scale and Self-perceived Difficulty Scale were 0.97, 12.8%, and 0.78, 35.8%, respectively. For the responsiveness study, the standardized effect size and standardized response mean (representing internal responsiveness) of the Ability Scale and Self-perceived Difficulty Scale were 1.17 and 1.56, and 0.78 and 0.89, respectively. Regarding external responsiveness, the change in score of the Ability Scale had significant and moderate association with that of the BI (r=0.61, P<0.001). The change in score of the Self-perceived Difficulty Scale had non-significant and weak association with that of the BI (r=0.23, P=0.080). CONCLUSIONS: The Ability Scale of the BI-SS has satisfactory test-retest reliability and sufficient responsiveness for patients with stroke. However, the Self-perceived Difficulty Scale of the BI-SS has substantial random measurement error and insufficient external responsiveness, which may affect its utility in clinical settings. CLINICAL REHABILITATION IMPACT: The findings of this study provide empirical evidence of psychometric properties of the BI-SS for assessing ability and self-perceived difficulty of ADL in patients with stroke.

Intrarater and Interrater Reliability of the Hierarchical Balance Short Forms in Patients With Stroke.

OBJECTIVE: To examine the intrarater and interrater reliability of a quick balance measure, the Hierarchical Balance Short Forms (HBSF), in outpatients with stroke receiving rehabilitation. DESIGN: A repeated-assessments design (1wk apart) was used to examine the intrarater and interrater reliability of the HBSF. The HBSF was administered by a single rater in the intrarater reliability study and by 2 raters in the interrater reliability study. The raters had sufficient working experience in stroke rehabilitation. SETTING: Seven teaching hospitals. PARTICIPANTS: Two independent groups of outpatients (N=106; each group, n=53) with stroke in chronic stages and in stable medical condition were recruited. INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURE: HBSF. RESULTS: For the intrarater reliability study, the values of the intraclass correlation coefficient (ICC), minimal detectable change (MDC), and percentage of minimal detectable change (MDC%) were .95, 1.02, and 16.3%, respectively, for the HBSF. The 95% limits of agreement (LOA) of the HBSF ranged from -.69 to 1.19. For the interrater reliability study, the values of the ICC, MDC, and MDC% were .91, 1.22, and 18.3%, respectively, for the HBSF. The 95% LOA of the HBSF ranged from -1.01 to 1.35. CONCLUSIONS: Our results suggest that the HBSF has satisfactory intrarater and interrater reliability for assessing balance function in outpatients with stroke. The MDC values of the HBSF are useful for both researchers and clinicians to determine whether the change in balance function of an individual patient is real when administered by an individual rater or by different raters.

Development of a Tablet-based symbol digit modalities test for reliably assessing information processing speed in patients with stroke.

PURPOSE: To develop a Tablet-based Symbol Digit Modalities Test (T-SDMT) and to examine the test-retest reliability and concurrent validity of the T-SDMT in patients with stroke. METHODS: The study had two phases. In the first phase, six experts, nine college students and five outpatients participated in the development and testing of the T-SDMT. In the second phase, 52 outpatients were evaluated twice (2 weeks apart) with the T-SDMT and SDMT to examine the test-retest reliability and concurrent validity of the T-SDMT. RESULTS: The T-SDMT was developed via expert input and college student/patient feedback. Regarding test-retest reliability, the practise effects of the T-SDMT and SDMT were both trivial (d=0.12) but significant (p≦0.015). The improvement in the T-SDMT (4.7%) was smaller than that in the SDMT (5.6%). The minimal detectable changes (MDC%) of the T-SDMT and SDMT were 6.7 (22.8%) and 10.3 (32.8%), respectively. The T-SDMT and SDMT were highly correlated with each other at the two time points (Pearson's r=0.90-0.91). CONCLUSIONS: The T-SDMT demonstrated good concurrent validity with the SDMT. Because the T-SDMT had a smaller practise effect and less random measurement error (superior test-retest reliability), it is recommended over the SDMT for assessing information processing speed in patients with stroke. Implications for Rehabilitation The Symbol Digit Modalities Test (SDMT), a common measure of information processing speed, showed a substantial practise effect and considerable random measurement error in patients with stroke. The Tablet-based SDMT (T-SDMT) has been developed to reduce the practise effect and random measurement error of the SDMT in patients with stroke. The T-SDMT had smaller practise effect and random measurement error than the SDMT, which can provide more reliable assessments of information processing speed.

Obstacle crossing in 7-9-year-old children with Down syndrome.

This study aimed to investigate obstacle crossing in 7-9-year-old children with Down syndrome (DS). Fifteen children with DS, age- and gender-matched with 15 typically developing (TD) children, were recruited to walk and cross obstacles with heights of 10%, 20% and 30% of their leg lengths. End-point and kinematic variables of obstacle crossing were obtained using a three-dimensional motion analysis system. The results showed that children with DS tend to adopt a lower speed and larger step width when they perceive instability. Moreover, unlike TD children, children with DS adopt a pelvic strategy (i.e., greater pelvic leading-side listing and forward rotation) to achieve a higher leading toe clearance with a longer step length, presumably for safety reasons. This pelvic strategy increased the frontal plane motion of the whole leg and trunk, and thus possibly stability, during obstacle crossing. However, this strategy may be inefficient. Trailing toe clearance did not differ significantly between two groups. The results of this study suggest that children with DS tend to use inefficient and conservative strategies for obstacle crossing. Knowledge of both end-point and kinematic control of obstacle crossing in children with DS is useful for understanding the mechanisms of obstacle-related falls. Moreover, obstacle crossing can be used as a task-oriented rehabilitation program for children with DS.

Rasch Analysis of the General Self-Efficacy Scale in Workers with Traumatic Limb Injuries.

Purpose The purpose of this study was to apply Rasch analysis to examine the unidimensionality and reliability of the General Self-Efficacy Scale (GSE) in workers with traumatic limb injuries. Furthermore, if the items of the GSE fitted the Rasch model's assumptions, we transformed the raw sum ordinal scores of the GSE into Rasch interval scores. Methods A total of 1076 participants completed the GSE at 1 month post injury. Rasch analysis was used to examine the unidimensionality and person reliability of the GSE. The unidimensionality of the GSE was verified by determining whether the items fit the Rasch model's assumptions: (1) item fit indices: infit and outfit mean square (MNSQ) ranged from 0.6 to 1.4; and (2) the eigenvalue of the first factor extracted from principal component analysis (PCA) for residuals was <2. Person reliability was calculated. Results The unidimensionality of the 10-item GSE was supported in terms of good item fit statistics (infit and outfit MNSQ ranging from 0.92 to 1.32) and acceptable eigenvalues (1.6) of the first factor of the PCA, with person reliability = 0.89. Consequently, the raw sum scores of the GSE were transformed into Rasch scores. Conclusions The results indicated that the items of GSE are unidimensional and have acceptable person reliability in workers with traumatic limb injuries. Additionally, the raw sum scores of the GSE can be transformed into Rasch interval scores for prospective users to quantify workers' levels of self-efficacy and to conduct further statistical analyses.

Examining unidimensionality and improving reliability for the eight subscales of the SF-36 in opioid-dependent patients using Rasch analysis.

PURPOSE: The Medical Outcome Study Short Form 36 (SF-36) is one of the most commonly used questionnaires for monitoring the Health-Related Quality of Life (HRQOL) of opioid-dependent patients. However, the unidimensionality and reliability of the SF-36 have not been verified in opioid-dependent patients. The aim of this study was to examine the unidimensionality and to improve the test reliability of the SF-36 for use in opioid-dependent patients. METHODS: A total of 583 opioid-dependent patients were recruited in the study. Unidimensionality was examined by conducting unidimensional Rasch analysis. Item fit statistics and principle component analysis were used to check the item-model fit in each of the eight subscales of the SF-36. Reliability was evaluated by applying both unidimensional and multidimensional Rasch analyses. RESULTS: After three misfitting items were excluded, the remaining items of each subscale in the SF-36 represented a single construct. The test reliabilities (0.80-0.87) yielded by the multidimensional approach were much higher than those (0.68-0.82) produced by the unidimensional approach. CONCLUSION: The remaining 32 items of the SF-36 are appropriate for evaluating the HRQOL in opioid-dependent patients in terms of unidimensionality. Additionally, the test scores produced by the multidimensional approach were more accurate than those obtained by the unidimensional approach.

A prospective study of the responsiveness of the original and the short form Berg Balance Scale in people with stroke.

OBJECTIVE: The study aim was to examine the responsiveness of the Berg Balance Scale (BBS) and that of its short form (SFBBS) at both the individual person level and the group level. DESIGN: A repeated-measurements design. SETTING: Hospital and home setting. PARTICIPANTS: Patients with stroke. RESULTS: Totals of 226, 202, and 168 patients with stroke were assessed with the BBS at 14, 30, and 90 days after stroke, respectively. The SFBBS data were extracted from the patients' responses on the BBS. At the group level, the BBS and the SFBBS had sufficient and similar responsiveness. For the Rasch scores, the effect sizes of the three change scores for the BBS and the SFBBS, respectively, had similar ranges between 0.38 and 0.88 and between 0.39 and 0.85, respectively. The standardized response means of the three change scores for the BBS and the SFBBS ranged from 0.74 to 1.33 and from 0.72 to 1.13, respectively. At the individual person level, the BBS detected significant balance improvement in about twice as many patients as the SFBBS detected. CONCLUSION: The responsiveness of the BBS at the individual person level was better than that of the SFBBS in patients with stroke. The BBS is recommended as an outcome measure to better detect changes in individual patients.

Reliability and responsiveness of the activities of daily living computerized adaptive testing system in patients with stroke.

OBJECTIVE: To examine the intrarater reliability, interrater reliability, and responsiveness of the Activities of Daily Living Computerized Adaptive Testing system (ADL CAT) in patients with stroke. DESIGN: One repeated-measures design (at an interval of 7d) was used to examine the intrarater reliability and interrater reliability of the ADL CAT. For the responsiveness study, participants were assessed with the ADL CAT at admission to the rehabilitation ward and at discharge from the hospital. SETTING: Eight rehabilitation units. PARTICIPANTS: Three different (nonoverlapping) groups of patients (N=157) were recruited. Fifty-five and 42 outpatients with chronic stroke participated in the intrarater and interrater reliability studies, respectively; 60 inpatients who had recently had a stroke participated in the responsiveness study. INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURE ADL CAT RESULTS: The intraclass correlation coefficient values were .94 and .80 for the ADL CAT in the intrarater reliability and interrater reliability studies, respectively. The classical test theory-based minimal detectable change values were 6.5 and 9.5 for the ADL CAT in the intrarater reliability and interrater reliability studies, respectively. The Kazis' effect size and standardized response mean of the ADL CAT were moderate (.62-.73). CONCLUSIONS: The ADL CAT has good intrarater reliability and interrater reliability in outpatients with chronic stroke, and sufficient responsiveness in inpatients with stroke undergoing inpatient rehabilitation. Further investigations on the responsiveness of the ADL CAT in outpatients are needed to obtain more evidence on the utility of the ADL CAT.

Comparison of the test-retest reliability of the balance computerized adaptive test and a computerized posturography instrument in patients with stroke.

OBJECTIVE: To compare the test-retest reliabilities of the scores of the Balance Computerized Adaptive Test (CAT) and the Biodex Balance System in patients with stroke. DESIGN: A repeated-measures design (at a 1-wk interval) was used to examine the test-retest reliabilities of the scores of the Balance CAT and the Biodex Balance System. SETTING: One rehabilitation unit in a local hospital. PARTICIPANTS: Patients (N=50) with stroke for more than 6 months and undergoing outpatient rehabilitation completed the Balance CAT and the eyes open (EO)/closed (EC) tests, but only 17 patients finished the Limit of Stability (LOS) test because they were unable to reach all the targets. INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: The Balance CAT and 2 computerized tests of the Biodex Balance System, namely the EO/EC test and the LOS, were used to evaluate balance function. RESULTS: The test-retest reliabilities of the scores of the Balance CAT (Pearson r=.92, minimal detectable change [MDC] percent=12.8%) was excellent. Those of the EO/EC and LOS tests were poor to good (Pearson r=.56-.85, MDC%=50.8%-126.9%). CONCLUSIONS: The test-retest reliabilities of the scores of the Balance CAT were sufficient for assessing balance function in patients with stroke. Moreover, the test-retest reliabilities of the scores of the Balance CAT, one of the functional balance measures, were superior to those of the Biodex Balance System, 1 type of computerized posturography instrument. Therefore, the Balance CAT may be a more reliable measure for clinicians and researchers to use in assessing the balance function of patients with stroke for more than 6 months.

Responsiveness and predictive validity of the hierarchical balance short forms in people with stroke.

BACKGROUND: The lack of knowledge about the responsiveness and predictive validity of a set of Hierarchical Balance Short Forms (the HBSF) in people with stroke limits the utility of the HBSF in both clinical and research settings. OBJECTIVE: The purpose of this study was to investigate the responsiveness and predictive validity of the HBSF in people receiving inpatient rehabilitation after stroke. DESIGN: A prospective cohort study was conducted. METHODS: Sixty-six participants completed both the 6-item HBSF and the 12-item Postural Assessment Scale for Stroke (PASS) after admission to the rehabilitation ward and before hospital discharge. The standardized effect size (ES) and the standardized response mean (SRM) were used to investigate the internal responsiveness of the HBSF and the PASS. Changes in the Barthel Index and the mobility subscale of the Stroke Rehabilitation Assessment of Movement were used as the external criteria for examining external responsiveness. Moreover, the admission scores on the HBSF and the PASS and the discharge scores on the Barthel Index and mobility subscale of the Stroke Rehabilitation Assessment of Movement were analyzed to investigate the predictive validity of the 2 balance measures. RESULTS: The internal responsiveness of the HBSF was high (ES>0.9, SRM>1.6). The SRM of the HBSF was significantly larger than that of the PASS, whereas the ES of the HBSF was not significantly larger than that of the PASS. The external responsiveness and predictive validity of the HBSF were sufficient and similar to those of the PASS (external responsiveness: r≥.35; predictive validity: r≥.67). LIMITATIONS: The convenience sampling of people receiving inpatient rehabilitation after stroke may limit the generalization of the results. CONCLUSIONS: The HBSF has sufficient responsiveness and predictive validity in people receiving inpatient rehabilitation after stroke and is thus recommended for both clinicians and researchers.

A comparison of responsiveness and predictive validity of two balance measures in patients with stroke.

OBJECTIVE: To compare the responsiveness and predictive validity of the Balance Computerized Adaptive Test (Balance CAT) and the Postural Assessment Scale for Stroke patients (PASS) in inpatients with stroke receiving rehabilitation. DESIGN: A pre-post test design. SUBJECTS: Eighty-five inpatients after stroke. METHODS: Effect size d and Wilcoxon signed-rank test were used to assess the internal responsiveness of the Balance CAT and PASS. The changes in the Barthel Index (BI) and the mobility subscale of the Stroke Rehabilitation Assessment of Movement (MO-STREAM) scores were both chosen as the external criteria for examining external responsiveness. Moreover, to investigate the predictive validity, the admission scores of the two balance measures, and the discharge score of the BI/MO-STREAM, were examined by simple linear regression analysis. RESULTS: Both the Balance CAT and PASS had high internal responsiveness (effect size d ≥ 0.87) and fair external responsiveness (r(2) ≥ 0.20). The predictive validities of both measures were sufficient (r(2) ≥ 0.33). The Balance CAT took approximately 3 items (min-max = 2-4) to complete. CONCLUSION: The Balance CAT and PASS have sufficient responsiveness and predictive validity in inpatients with stroke receiving rehabilitation. The Balance CAT is more efficient to administer and is thus recommended over the PASS.