Item-Level Psychometrics of the Neonatal Eating Outcome Assessment in Orally Feeding Infants.

BACKGROUND: The Neonatal Eating Outcome Assessment determines feeding performance based on the infant's postmenstrual age (PMA). OBJECTIVE: To examine item-level measurement properties of this assessment's rating scale. METHODOLOGY: In this retrospective study, Rasch analysis was completed on clinical data from the Neonatal Eating Outcome Assessment for 100 infants (52 preterm and 48 full-term) using Winsteps version 3.93.1. Instead of PMA-based scores, ordered letters converted to numerical scores were analyzed. RESULTS: Analysis demonstrated that Section I (Pre-Feeding Skills) represents a separate construct from Sections II and III (Oral Feeding and End of Feeding, respectively). Sections II and III were adequately unidimensional to complete Rasch analysis. These sections fit the Rasch model overall, but rating scale category underuse was common, which may be attributed to sample characteristics. IMPLICATIONS: This analysis supports using validated ordered letter scoring of Sections II and III to measure oral feeding performance in preterm and full-term newborns.

Examining Measurement Properties of Items in the Neonatal Eating Outcome AssessmentThe Neonatal Eating Outcome Assessment is used to evaluate infant feeding skills, considering expected performance based on the infant's age. We completed Rasch analysis of this assessment using data from 100 infants (52 preterm and 48 full-term). Analysis used ordered letter scores (not based on infant age) which allowed comparison of the skill level of each infant and the difficulty level of each assessment item. This also provided information about how well each item measures the feeding behavior it intends to assess. Study limitations included underutilization of some rating scale categories and inability to analyze Section I. However, overall, results showed that items in Sections II and III of The Neonatal Eating Outcome Assessment provide good measurement of infants' oral feeding performance. This information will help clinicians and researchers understand this assessment's strengths and opportunities for improvement.

Piloting an empirical approach to link the international classification of functioning, disability, and health to job demand classification.

BACKGROUND: The Social Security Administration has a thorough disability claims process, though one goal is to process claims more efficiently. This pilot described the creation and trial of a web-based tool to assist this process. OBJECTIVE: To empirically link the International Classification of Functioning, Disability and Health (ICF) model to the Occupational Information Network (O*NET) with a self-reported activity measure (physical domain). METHODS: ICF Activity domains and item difficulty calibrations were obtained from a self-reported ICF-Activity Measure. Three work/disability experts matched measurement constructs, job names, and item difficulties with job demands. Item difficulties were linked for "Positioning and Transfer" with O*NET data values of "Stamina"; "Trunk Strength", and "Walking and Moving" with O*NET data values of "Stamina." The system was then pilot tested with 14 adults with more than one activity challenges. RESULTS: An average total of 613 jobs were generated per participant and each job was categorized into one of 16 job clusters. Person ability measures and constructs were significant predictors for numbers of job (R2 = 0.92, p < 0.05). Participants demonstrated moderate satisfaction with program usability, and reported capability in performing jobs. CONCLUSIONS: The system provides a feasible means to assist disability examiners and claimants in identifying relevant job abilities and options.

Interpreting Action Research Arm Test Assessment Scores to Plan Treatment.

Rasch keyforms can help interpret clinical assessment scores. The Action Research Arm Test (ARAT) is a commonly used assessment, yet no keyform currently exists. The aim is to provide a keyform for the ARAT and demonstrate how a clinician can use the keyform to design optimally challenging rehabilitation sessions. Secondary analysis of ARAT data ( n = 122) using confirmatory factor and Rasch analyses were used to examine the measurement properties and generate a keyform. The item standardized factor loadings were >0.40 (range = 0.82-0.96) and R2 values were >.60 (range = .65-.96). All items exhibited adequate infit statistics with point measure correlations >.60 (range = .72-.97). Person reliability was .98, and person separation was 7.07. Item-difficulty measures ranged from -2.78 logits to 2.64 logits. The ARAT has strong measurement properties, and a keyform was provided. We showed how the keyform can be utilized by clinicians to interpret scores, set goals, and plan treatment.

Linking Existing Instruments to Develop an Activity of Daily Living Item Bank.

This study examined dimensionality and item-level psychometric properties of an item bank measuring activities of daily living (ADL) across inpatient rehabilitation facilities and community living centers. Common person equating method was used in the retrospective veterans data set. This study examined dimensionality, model fit, local independence, and monotonicity using factor analyses and fit statistics, principal component analysis (PCA), and differential item functioning (DIF) using Rasch analysis. Following the elimination of invalid data, 371 veterans who completed both the Functional Independence Measure (FIM) and minimum data set (MDS) within 6 days were retained. The FIM-MDS item bank demonstrated good internal consistency (Cronbach's α = .98) and met three rating scale diagnostic criteria and three of the four model fit statistics (comparative fit index/Tucker-Lewis index = 0.98, root mean square error of approximation = 0.14, and standardized root mean residual = 0.07). PCA of Rasch residuals showed the item bank explained 94.2% variance. The item bank covered the range of θ from -1.50 to 1.26 (item), -3.57 to 4.21 (person) with person strata of 6.3. The findings indicated the ADL physical function item bank constructed from FIM and MDS measured a single latent trait with overall acceptable item-level psychometric properties, suggesting that it is an appropriate source for developing efficient test forms such as short forms and computerized adaptive tests.

The Safety Assessment Measure for persons with traumatic brain injury: Item pool development and content validity.

BACKGROUND: Persons with moderate to severe TBI are at increased risk for unintentional injury or harm in the home and community; however, there is currently no standard measure of safety risk they face now and in the future. OBJECTIVE: To develop comprehensive and content valid scales and item pools for assessing safety and risk for persons with moderate to severe traumatic brain injuries. METHOD: Qualitative psychometric methods for developing scales and items were used including literature review, item development and revision, focus groups with interdisciplinary rehabilitation staff (n = 26) for rating content validity, and cognitive interviewing of TBI family members (n = 9) for assuring item clarity. RESULTS: The Safety Assessment Measure is comprised of 6 primary scales - Cognitive Capacity, Visuomotor Capacity, Wheelchair Use, Risk Perception, Self-Regulation, and Compliance Failures with Safety Recommendations - in which family caregivers or clinicians rate the risk for unintentional injury or harm in adults who have sustained moderate or severe TBI. The scale item pools encompass a broad spectrum of everyday activities that pose risk in the home and community and were rated as having excellent levels of content validity. CONCLUSIONS: The Safety Assessment Measure scales and items cover a broad range of instrumental activities of daily living that can increase the risk of unintentional injuries or harm. Empirical evidence suggests that the Safety Assessment Measure items have excellent content validity. Future research should use modern psychometric methods to examine each scale unidimensionality, model fit, and precision.

Assessment of the psychometrics of a PROMIS item bank: self-efficacy for managing daily activities.

PURPOSE: The aim of this study is to investigate the psychometrics of the Patient-Reported Outcomes Measurement Information System self-efficacy for managing daily activities item bank. METHODS: The item pool was field tested on a sample of 1087 participants via internet (n = 250) and in-clinic (n = 837) surveys. All participants reported having at least one chronic health condition. The 35 item pool was investigated for dimensionality (confirmatory factor analyses, CFA and exploratory factor analysis, EFA), item-total correlations, local independence, precision, and differential item functioning (DIF) across gender, race, ethnicity, age groups, data collection modes, and neurological chronic conditions (McFadden Pseudo R (2) less than 10 %). RESULTS: The item pool met two of the four CFA fit criteria (CFI = 0.952 and SRMR = 0.07). EFA analysis found a dominant first factor (eigenvalue = 24.34) and the ratio of first to second eigenvalue was 12.4. The item pool demonstrated good item-total correlations (0.59-0.85) and acceptable internal consistency (Cronbach's alpha = 0.97). The item pool maintained its precision (reliability over 0.90) across a wide range of theta (3.70), and there was no significant DIF. CONCLUSION: The findings indicated the item pool has sound psychometric properties and the test items are eligible for development of computerized adaptive testing and short forms.

Dimensionality and Item-Difficulty Hierarchy of the Lower Extremity Fugl-Meyer Assessment in Individuals With Subacute and Chronic Stroke.

OBJECTIVE: To investigate the dimensionality and item-difficulty hierarchy of the Fugl-Meyer Assessment of the lower extremity (FMA-LE). DESIGN: Secondary analyses of data pooled from 4 existing datasets: a phase III randomized controlled trial investigating the effectiveness of body weight support and a treadmill for rehabilitation of walking poststroke, and 3 cross-sectional studies investigating the link between impaired motor performance poststroke and walking. SETTING: University research centers and rehabilitation centers. PARTICIPANTS: A pooled sample of individuals with a stroke (N=535, men=313; mean age ± SD, 61.91±12.42y). INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: Confirmatory factor analyses (CFA) and Rasch residual principal component analysis (PCA) investigated the dimensionality of the FMA-LE. The Rasch analysis rating scale model investigated item-difficulty hierarchy of the FMA-LE. RESULTS: The CFA showed adequate fit of a 3-factor model, with 2 out of 3 indices (CFA=.95; Tucker-Lewis Index=.94; root mean square error of approximation=.124) showing good model fit. Rasch PCA showed that removal of the reflex and coordination items explained 90.8% of variance in the data, suggesting that the abnormal synergy items contributed to the measurement of a unidimensional construct. However, rating scale model results revealed deviations in the item-difficulty hierarchy of the unidimensional abnormal synergy items from the originally proposed stepwise sequence of motor recovery. CONCLUSIONS: Our findings suggest that the FMA-LE might represent a multidimensional construct, challenging the use of a total score of the FMA-LE to predict lower extremity motor recovery. Removal of the misfit items resulted in creation of a unidimensional scale composed of the abnormal synergy items. However, this unidimensional scale deviates from the originally proposed hierarchical ordering.

Rasch analysis of a new hierarchical scoring system for evaluating hand function on the motor assessment scale for stroke.

Objectives. (1) To develop two independent measurement scales for use as items assessing hand movements and hand activities within the Motor Assessment Scale (MAS), an existing instrument used for clinical assessment of motor performance in stroke survivors; (2) To examine the psychometric properties of these new measurement scales. Design. Scale development, followed by a multicenter observational study. Setting. Inpatient and outpatient occupational therapy programs in eight hospital and rehabilitation facilities in the United States and Canada. Participants. Patients (N = 332) receiving stroke rehabilitation following left (52%) or right (48%) cerebrovascular accident; mean age 64.2 years (sd 15); median 1 month since stroke onset. Intervention. Not applicable. Main Outcome Measures. Data were tested for unidimensionality and reliability, and behavioral criteria were ordered according to difficulty level with Rasch analysis. Results. The new scales assessing hand movements and hand activities met Rasch expectations of unidimensionality and reliability. Conclusion. Following a multistep process of test development, analysis, and refinement, we have redesigned the two scales that comprise the hand function items on the MAS. The hand movement scale contains an empirically validated 10-behavior hierarchy and the hand activities item contains an empirically validated 8-behavior hierarchy.

Examination of the factor structure of the Disabilities of the Arm, Shoulder, and Hand questionnaire.

OBJECTIVE: The Disabilities of the Arm, Shoulder, and Hand (DASH) questionnaire is a global scale evaluating the impact of upper-extremity disorders. We sought to validate or challenge the unidimensional factor structure of the DASH. METHOD: Secondary analysis was performed on data collected from outpatient clinics. Factor analysis was performed in two steps, exploratory factor analysis (EFA) followed by confirmatory factor analysis (CFA). To provide further support for unidimensionality, fit statistics were calculated using the Andrich Rasch rating-scale model. RESULTS: EFA revealed three potential factors (eigenvalues = 18.40, 1.56, and 1.54). CFA was performed fitting a three-factor model. Tucker-Lewis Index (.99) and standardized root mean square residual (.05) values indicated good fit. Comparative fit index (.89) and root mean square error of approximation (.13) did not. When divided into three constructs, only one item misfit. CONCLUSION: More research is needed to determine situations in which division of the DASH may enhance interpretability.

Translating measurement findings into rehabilitation practice: an example using Fugl-Meyer Assessment-Upper Extremity with patients following stroke.

Standardized assessments are critical for advancing clinical rehabilitation, yet assessment scores often provide little information for rehabilitation treatment planning. A keyform recovery map is an innovative way for a therapist to record patient responses to standardized assessment items. The form enables a therapist to view the specific items that a patient can or cannot perform. This information can assist a therapist in tailoring treatments to a patient's individual ability level. We demonstrate how a keyform recovery map can be used to inform clinical treatment planning for individuals with stroke-related upper-limb motor impairment. A keyform map of poststroke upper-limb recovery defined by items of the Fugl-Meyer Assessment-Upper Extremity (FMA-UE) was generated by a previously published Rasch analysis. Three individuals with stroke enrolled in a separate research study were randomly selected from each of the three impairment strata of the FMA-UE. Their performance on each item was displayed on the FMA-UE keyform. The forms directly connected qualitative descriptions of patients' motor ability to assessment measures, thereby suggesting appropriate shorter and longer term rehabilitation goals. This study demonstrates how measurement theory can be used to translate a standardized assessment into a useful, evidence-based tool for making clinical practice decisions.

Longitudinal stability of the Fugl-Meyer Assessment of the upper extremity.

OBJECTIVE: To investigate the longitudinal stability of the Fugl-Meyer Assessment (FMA) of the upper-extremity item difficulties by using Rasch analysis. DESIGN: Secondary analysis of existing data from a cohort longitudinal study of stroke recovery. SETTING: University research center. PARTICIPANTS: A total of 377 people, ages 69.2+/-11.2 years, to whom the assessment was administered at 2 weeks and 6 months poststroke. INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: Differential item function analysis performed by using the Winsteps software program examined whether the item difficulty hierarchical order of a modified 30-item FMA for the upper extremity (reflex items removed) was invariant across 2 testing occasions. RESULTS: Only 2 items (shoulder flexion to 180 degrees, movement with normal speed) showed large differences in test-retest item difficulty calibration. Item instability had no practical consequences on the longitudinal measurement of person ability. CONCLUSIONS: The 30-item assessment shows a longitudinally stable item difficulty order and is valid for measuring volitional arm motor ability over time.

State-of-the-science on postacute rehabilitation: measurement and methodologies for assessing quality and establishing policy for postacute care.

We present an overview of commonly used postacute outcome measures and review new methodologies for postacute assessment. We question the impact that current measurement has had on improvement of quality of postacute care (PAC) and its utility in informing health policy. We suggest that Donabedian's model of health care quality should be endorsed for measurement. Specifically, measurement of outcomes and process should be used jointly in assessment of PAC.

Dimensionality and construct validity of the Fugl-Meyer Assessment of the upper extremity.

OBJECTIVE: To investigate the dimensionality and construct validity of the Fugl-Meyer Assessment of the upper extremity by using Rasch analysis. DESIGN: Secondary analysis of pooled data from 2 existing datasets: a randomized therapeutic exercise clinical trial and a cohort longitudinal study of stroke recovery. SETTING: University research center. PARTICIPANTS: A total of 512 subjects, ages 69.8+/-11.1 years, who were 0 to 145 days poststroke. INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: Dimensionality was examined with principal components analysis and Rasch item-fit statistics. The Rasch-derived item hierarchy was examined for consistency with the expected course of poststroke upper-extremity recovery suggested by the reflex-hierarchical conceptual model underlying the assessment. RESULTS: Factor loadings and item infit statistics suggested that the 3 reflex items were empirically disconnected from other assessment items. The reflex items were removed. The modified 30-item assessment showed a unidimensional structure. The Rasch-item-difficulty order was not consistent with the expected item order. CONCLUSIONS: The items testing resting-state reflexes may threaten the assessment's dimensionality. With reflex items removed, the assessment is a unidimensional measure of volitional movement. The Rasch-generated item-difficulty order challenges the hierarchical structure implied by the instrument's underlying conceptual framework.

Assessing arm and hand function after stroke: a validity test of the hierarchical scoring system used in the motor assessment scale for stroke.

OBJECTIVE: To evaluate the validity of the scoring hierarchy for the 3 upper-limb items on the Motor Assessment Scale (MAS). DESIGN: Application of Rasch analysis to 3 independent measurement scales, each representing the upper-arm function, hand movements, and advanced hand activities items of the MAS. SETTING: Inpatient and outpatient occupational therapy (OT) programs in a department of rehabilitation of an urban hospital center. PARTICIPANTS: One hundred patients (67 men, 33 women; average age, 54.3+/-14.4 y; average time since stroke onset, 104 d) attending OT for stroke rehabilitation. INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURE: The MAS for stroke (upper-arm function, hand movements, and advanced hand activities sections). RESULTS: Rasch analysis provided support for the validity of hierarchical scoring criteria for the upper-arm scale. This analysis, however, identified inconsistencies in the hierarchical scoring criteria for the hand function and advanced hand activities scales and, when considering measurement error, only small differences in difficulty level between several behavioral criteria. CONCLUSIONS: The findings lead to suggestions for changes in the behavioral criteria hierarchy for upper-limb items on the MAS and highlight the importance of using statistical analyses to test the validity of proposed hierarchies of behavioral criteria in functional assessments.

Rating scale analysis of the Berg Balance Scale.

OBJECTIVES: To examine, using Rasch analysis, the rating scale performance of the Berg Balance Scale (BBS) and to describe the 45/56 cutoff score in functional terms. DESIGN: Retrospective chart review of BBS scores. Rasch rating scale analysis was performed on these data. SETTING: Outpatient Veterans Affairs medical center. PARTICIPANTS: One hundred (99 men, 1 woman) community-dwelling veterans referred for balance deficits (age range, 64-88y). INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURE: The BBS. RESULTS: Condensing item-rating categories allowed the elimination of underutilized categories and constructed categories that better separated people of differing abilities. Rating pivot points were developed for each item to represent a transition between passing and failing. Following pivot-point development and rating scale rescoring, person and item measures became more evenly distributed across the BBS and resulted in changes in item difficulty order. In our sample, functional indicators of a score of at least 45/56 were a rating of passing the item "tandem stance," as well as passing 2 of the following 3 items: "alternating foot," "standing on one leg," and "look behind." CONCLUSIONS: Our findings provide direction for improving the rating scale structure for each of the items and establish a connection between the BBS cutoff score of 45/56 and functional ability.