Psychometric Evaluation of the Improved Work-Disability Functional Assessment Battery.

OBJECTIVE: To assess psychometric properties of the improved Work Disability Functional Assessment Battery (WD-FAB 2.0). DESIGN: Longitudinal study. SETTING: Community. PARTICIPANTS: Three samples of working-age (21-66) adults (N=1006): (1) unable to work because of a physical condition (n=375); (2) unable to work because of a mental health condition (n=296); (3) general United States working age sample (n=335). INTERVENTION: NA. MAIN OUTCOME MEASURES: All samples completed the WD-FAB 2.0; the second administration came 5 days after the first. Construct validity was examined by convergent and divergent correlational analysis using legacy measures. Test-retest reliability was assessed by intraclass correlation coefficients (ICC3,1). Standard error of the mean (SEM) and minimal detectable change (MDC90) were calculated to measure scale precision and sensitivity. RESULTS: Physical function ICCs ranged from 0.69 to 0.77 in the general sample, and 0.66 to 0.86 in the disability sample. Mental health function scales ICCs ranged from 0.62 to 0.73 in the general sample, and 0.74 to 0.76 in the disability sample. SEMs for all scales indicated good discrimination; those for the physical function scales were generally lower than those for the mental health scales. MDC90 values ranged from 3.41 to 10.55. Correlations between all WD-FAB 2.0 scales and legacy measures were in the expected direction. CONCLUSIONS: The study provides substantial support for the reliability and construct validity of the WD-FAB 2.0 among 3 diverse samples. Although initially developed for use within the Social Security Administration (SSA), these results suggest that the WD-FAB 2.0 could be used for assessment and measurement of work-related physical and mental health function in other contexts as well.

Validation of the Work-Disability Physical Functional Assessment Battery.

OBJECTIVE: To examine the validity of the self-report Work-Disability Functional Assessment Battery (WD-FAB) physical function scales relative to clinician ratings of function and a performance-based functional capacity evaluation called the Physical Work Performance Evaluation (PWPE). DESIGN: Cross-sectional. SETTING: Outpatient rehabilitation. PARTICIPANTS: Adults (N=50) participating in physical therapy for musculoskeletal conditions. INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: Patients completed the PWPE and the WD-FAB physical function scales including Changing and Maintaining Body Position, Whole Body Mobility, Upper Body Function, and Upper Extremity Fine Motor. The physical therapist also answered the WD-FAB questions on the patient's physical functioning. The WD-FAB computer-adaptive test version administered up to 10 items for each scale. The PWPE produces ratings from 0 to 5 indicating overall Level of Work ability: 0 (unable to work); 1 (sedentary); 2 (light); 3 (medium); 4 (heavy); 5 (very heavy). The PWPE also produces Level of Work ability ratings in the Dynamic Strength, Position Tolerance, and Mobility subsections. RESULTS: Participating in the study were 50 patients with 1 or more conditions (shoulder, n=21; knee, n=16; low back, n=13; ankle/foot, n=10; neck, n=8; hip, n=7). The patient-based WD-FAB scores demonstrated moderate, statistically significant correlations with the provider proxy WD-FAB report (R=.49-.65). The WD-FAB Upper Body Function scale scores demonstrated moderate strength relationships with the PWPE overall ratings. The Whole Body Mobility and Changing and Maintaining Body Position scales did not demonstrate statistically significant relationships with the PWPE overall ratings. CONCLUSIONS: We found moderate evidence for validity for the WD-FAB Upper Body Function, Whole Body Mobility, and Changing and Maintaining Body Position scales relative to clinician report and varied evidence relative to the PWPE in this clinical sample.

Improving measures of work-related physical functioning.

PURPOSE: To expand content of the physical function domain of the Work Disability Functional Assessment Battery (WD-FAB), developed for the US Social Security Administration's (SSA) disability determination process. METHODS: Newly developed questions were administered to 3532 recent SSA applicants for work disability benefits and 2025 US adults. Factor analyses and item response theory (IRT) methods were used to calibrate and link the new items to the existing WD-FAB, and computer-adaptive test simulations were conducted. RESULTS: Factor and IRT analyses supported integration of 44 new items into three existing WD-FAB scales and the addition of a new 11-item scale (Community Mobility). The final physical function domain consisting of: Basic Mobility (56 items), Upper Body Function (34 items), Fine Motor Function (45 items), and Community Mobility (11 items) demonstrated acceptable psychometric properties. CONCLUSIONS: The WD-FAB offers an important tool for enhancement of work disability determination. The FAB could provide relevant information about work-related functioning for initial assessment of claimants; identifying denied applicants who may benefit from interventions to improve work and health outcomes; enhancing periodic review of work disability beneficiaries; and assessing outcomes for policies, programs and services targeting people with work disability.

Work-related measures of physical and behavioral health function: Test-retest reliability.

BACKGROUND: The Work Disability Functional Assessment Battery (WD-FAB), developed for potential use by the US Social Security Administration to assess work-related function, currently consists of five multi-item scales assessing physical function and four multi-item scales assessing behavioral health function; the WD-FAB scales are administered as Computerized Adaptive Tests (CATs). OBJECTIVE: The goal of this study was to evaluate the test-retest reliability of the WD-FAB Physical Function and Behavioral Health CATs. METHODS: We administered the WD-FAB scales twice, 7-10 days apart, to a sample of 376 working age adults and 316 adults with work-disability. Intraclass correlation coefficients were calculated to measure the consistency of the scores between the two administrations. Standard error of measurement (SEM) and minimal detectable change (MDC90) were also calculated to measure the scales precision and sensitivity. RESULTS: For the Physical Function CAT scales, the ICCs ranged from 0.76 to 0.89 in the working age adult sample, and 0.77-0.86 in the sample of adults with work-disability. ICCs for the Behavioral Health CAT scales ranged from 0.66 to 0.70 in the working age adult sample, and 0.77-0.80 in the adults with work-disability. The SEM ranged from 3.25 to 4.55 for the Physical Function scales and 5.27-6.97 for the Behavioral Health function scales. For all scales in both samples, the MDC90 ranged from 7.58 to 16.27. CONCLUSION: Both the Physical Function and Behavioral Health CATs of the WD-FAB demonstrated good test-retest reliability in adults with work-disability and general adult samples, a critical requirement for assessing work related functioning in disability applicants and in other contexts.

Interpreting physical and behavioral health scores from new work disability instruments.

OBJECTIVE: To develop a system to guide interpretation of scores generated from 2 new instruments measuring work-related physical and behavioral health functioning (Work Disability - Physical Function (WD-PF) and WD - Behavioral Function (WD-BH)). DESIGN: Cross-sectional, secondary data from 3 independent samples to develop and validate the functional levels for physical and behavioral health functioning. SUBJECTS: Physical group: 999 general adult subjects, 1,017 disability applicants and 497 work-disabled subjects. Behavioral health group: 1,000 general adult subjects, 1,015 disability applicants and 476 work-disabled subjects. METHODS: Three-phase analytic approach including item mapping, a modified-Delphi technique, and known-groups validation analysis were used to develop and validate cut-points for functional levels within each of the WD-PF and WD-BH instrument's scales. RESULTS: Four and 5 functional levels were developed for each of the scales in the WD-PF and WD-BH instruments. Distribution of the comparative samples was in the expected direction: the general adult samples consistently demonstrated scores at higher functional levels compared with the claimant and work-disabled samples. CONCLUSION: Using an item-response theory-based methodology paired with a qualitative process appears to be a feasible and valid approach for translating the WD-BH and WD-PF scores into meaningful levels useful for interpreting a person's work-related physical and behavioral health functioning.

Work disability functional assessment battery: feasibility and psychometric properties.

OBJECTIVE: To assess the feasibility and psychometric properties of 8 scales covering 2 domains of the newly developed Work Disability Functional Assessment Battery (WD-FAB): physical function (PF) and behavioral health (BH) function. DESIGN: Cross-sectional study. SETTING: Community. PARTICIPANTS: Adults (N=973) unable to work because of a physical (n=497) or a mental (n=476) disability. INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: Each disability group responded to a survey consisting of the relevant WD-FAB scales and existing measures of established validity. The WD-FAB scales were evaluated with regard to data quality (score distribution, percentage of "I don't know" responses), efficiency of administration (number of items required to achieve reliability criterion, time required to complete the scale) by computerized adaptive testing (CAT), and measurement accuracy as tested by person fit. Construct validity was assessed by examining both convergent and discriminant correlations between the WD-FAB scales and scores on same-domain and cross-domain established measures. RESULTS: Data quality was good, and CAT efficiency was high across both WD-FAB domains. Measurement accuracy was very good for PF scales; BH scales demonstrated more variability. Construct validity correlations, both convergent and divergent, between all WD-FAB scales and established measures were in the expected direction and range of magnitude. CONCLUSIONS: The data quality, CAT efficiency, person fit, and construct validity of the WD-FAB scales were well supported and suggest that the WD-FAB could be used to assess PF and BH function related to work disability. Variation in scale performance suggests the need for future work on item replenishment and refinement, particularly with regard to the Self-Efficacy scale.

Conceptualizing disability in US national surveys: application of the World Health Organization's (WHO) International Classification of Functioning, Disability, and Health (ICF) framework.

PURPOSE: Disability data inform resource allocation and utilization, characterize functioning and changes over time, and provide a mechanism to monitor progress toward promoting and protecting the rights of individuals with disability. Data collection efforts, however, define and measure disability in varied ways. Our objective was to see how the content of disability measures differed in five US national surveys and over time. METHODS: Using the WHO ICF as a conceptual framework for measuring disability, we assessed the National Health Interview Survey (NHIS), Current Population Survey (CPS), Survey of Income and Program Participation (SIPP), National Survey of SSI Children and Families (NSCF), and American Community Survey (ACS) for their content coverage of disability relative to each of the four ICF components (i.e., body functions, body structures, activities and participation, and environment). We used second-level ICF three-digit codes to classify question content into categories within each ICF component and computed the proportion of categories within each ICF component that was represented in the questions selected from these five surveys. RESULTS: The disability measures varied across surveys and years. The NHIS captured a greater proportion of the ICF body functions and body structures components than did other surveys. The SIPP captured the most content of the ICF activities and participation component, and the NSCF contained the most content of the ICF environmental factors component. CONCLUSIONS: This research successfully illustrated demonstrated the utility of the ICF in examining the content of disability measures in five national surveys and over time.

Development of a computer-adaptive physical function instrument for Social Security Administration disability determination.

OBJECTIVES: To develop and test an instrument to assess physical function for Social Security Administration (SSA) disability programs, the SSA-Physical Function (SSA-PF) instrument. Item response theory (IRT) analyses were used to (1) create a calibrated item bank for each of the factors identified in prior factor analyses, (2) assess the fit of the items within each scale, (3) develop separate computer-adaptive testing (CAT) instruments for each scale, and (4) conduct initial psychometric testing. DESIGN: Cross-sectional data collection; IRT analyses; CAT simulation. SETTING: Telephone and Internet survey. PARTICIPANTS: Two samples: SSA claimants (n=1017) and adults from the U.S. general population (n=999). INTERVENTIONS: None. MAIN OUTCOME MEASURES: Model fit statistics, correlation, and reliability coefficients. RESULTS: IRT analyses resulted in 5 unidimensional SSA-PF scales: Changing & Maintaining Body Position, Whole Body Mobility, Upper Body Function, Upper Extremity Fine Motor, and Wheelchair Mobility for a total of 102 items. High CAT accuracy was demonstrated by strong correlations between simulated CAT scores and those from the full item banks. On comparing the simulated CATs with the full item banks, very little loss of reliability or precision was noted, except at the lower and upper ranges of each scale. No difference in response patterns by age or sex was noted. The distributions of claimant scores were shifted to the lower end of each scale compared with those of a sample of U.S. adults. CONCLUSIONS: The SSA-PF instrument contributes important new methodology for measuring the physical function of adults applying to the SSA disability programs. Initial evaluation revealed that the SSA-PF instrument achieved considerable breadth of coverage in each content domain and demonstrated noteworthy psychometric properties.

Development of an instrument to measure behavioral health function for work disability: item pool construction and factor analysis.

OBJECTIVES: To develop a broad set of claimant-reported items to assess behavioral health functioning relevant to the Social Security disability determination processes, and to evaluate the underlying structure of behavioral health functioning for use in development of a new functional assessment instrument. DESIGN: Cross-sectional. SETTING: Community. PARTICIPANTS: Item pools of behavioral health functioning were developed, refined, and field tested in a sample of persons applying for Social Security disability benefits (N=1015) who reported difficulties working because of mental or both mental and physical conditions. INTERVENTIONS: None. MAIN OUTCOME MEASURE: Social Security Administration Behavioral Health (SSA-BH) measurement instrument. RESULTS: Confirmatory factor analysis (CFA) specified that a 4-factor model (self-efficacy, mood and emotions, behavioral control, social interactions) had the optimal fit with the data and was also consistent with our hypothesized conceptual framework for characterizing behavioral health functioning. When the items within each of the 4 scales were tested in CFA, the fit statistics indicated adequate support for characterizing behavioral health as a unidimensional construct along these 4 distinct scales of function. CONCLUSIONS: This work represents a significant advance both conceptually and psychometrically in assessment methodologies for work-related behavioral health. The measurement of behavioral health functioning relevant to the context of work requires the assessment of multiple dimensions of behavioral health functioning. Specifically, we identified a 4-factor model solution that represented key domains of work-related behavioral health functioning. These results guided the development and scale formation of a new SSA-BH instrument.

Conceptual foundation for measures of physical function and behavioral health function for Social Security work disability evaluation.

Physical and mental impairments represent the 2 largest health condition categories for which workers receive Social Security disability benefits. Comprehensive assessment of physical and mental impairments should include aspects beyond medical conditions such as a person's underlying capabilities as well as activity demands relevant to the context of work. The objective of this article is to describe the initial conceptual stages of developing new measurement instruments of behavioral health and physical functioning relevant for Social Security work disability evaluation purposes. To outline a clear conceptualization of the constructs to be measured, 2 content models were developed using structured and informal qualitative approaches. We performed a structured literature review focusing on work disability and incorporating aspects of the International Classification of Functioning, Disability and Health as a unifying taxonomy for framework development. Expert interviews provided advice and consultation to enhance face validity of the resulting content models. The content model for work-related behavioral health function identifies 5 major domains: (1) behavior control, (2) basic interactions, (3) temperament and personality, (4) adaptability, and (5) workplace behaviors. The content model describing physical functioning includes 3 domains: (1) changing and maintaining body position, (2) whole-body mobility, and (3) carrying, moving, and handling objects. These content models informed subsequent measurement properties including item development and measurement scale construction, and provided conceptual coherence guiding future empirical inquiry. The proposed measurement approaches show promise to comprehensively and systematically assess physical and behavioral health functioning relevant to work.

Development of a self-report physical function instrument for disability assessment: item pool construction and factor analysis.

OBJECTIVES: To build a comprehensive item pool representing work-relevant physical functioning and to test the factor structure of the item pool. These developmental steps represent initial outcomes of a broader project to develop instruments for the assessment of function within the context of Social Security Administration (SSA) disability programs. DESIGN: Comprehensive literature review; gap analysis; item generation with expert panel input; stakeholder interviews; cognitive interviews; cross-sectional survey administration; and exploratory and confirmatory factor analyses to assess item pool structure. SETTING: In-person and semistructured interviews and Internet and telephone surveys. PARTICIPANTS: Sample of SSA claimants (n=1017) and a normative sample of adults from the U.S. general population (n=999). INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURE: Model fit statistics. RESULTS: The final item pool consisted of 139 items. Within the claimant sample, 58.7% were white; 31.8% were black; 46.6% were women; and the mean age was 49.7 years. Initial factor analyses revealed a 4-factor solution, which included more items and allowed separate characterization of: (1) changing and maintaining body position, (2) whole body mobility, (3) upper body function, and (4) upper extremity fine motor. The final 4-factor model included 91 items. Confirmatory factor analyses for the 4-factor models for the claimant and the normative samples demonstrated very good fit. Fit statistics for claimant and normative samples, respectively, were: Comparative Fit Index=.93 and .98; Tucker-Lewis Index=.92 and .98; and root mean square error approximation=.05 and .04. CONCLUSIONS: The factor structure of the physical function item pool closely resembled the hypothesized content model. The 4 scales relevant to work activities offer promise for providing reliable information about claimant physical functioning relevant to work disability.

Scale refinement and initial evaluation of a behavioral health function measurement tool for work disability evaluation.

OBJECTIVE: To use item response theory (IRT) data simulations to construct and perform initial psychometric testing of a newly developed instrument, the Social Security Administration Behavioral Health Function (SSA-BH) instrument, that aims to assess behavioral health functioning relevant to the context of work. DESIGN: Cross-sectional survey followed by IRT calibration data simulations. SETTING: Community. PARTICIPANTS: Sample of individuals applying for Social Security Administration disability benefits: claimants (n=1015) and a normative comparative sample of U.S. adults (n=1000). INTERVENTIONS: None. MAIN OUTCOME MEASURE: SSA-BH measurement instrument. RESULTS: IRT analyses supported the unidimensionality of 4 SSA-BH scales: mood and emotions (35 items), self-efficacy (23 items), social interactions (6 items), and behavioral control (15 items). All SSA-BH scales demonstrated strong psychometric properties including reliability, accuracy, and breadth of coverage. High correlations of the simulated 5- or 10-item computer adaptive tests with the full item bank indicated robust ability of the computer adaptive testing approach to comprehensively characterize behavioral health function along 4 distinct dimensions. CONCLUSIONS: Initial testing and evaluation of the SSA-BH instrument demonstrated good accuracy, reliability, and content coverage along all 4 scales. Behavioral function profiles of Social Security Administration claimants were generated and compared with age- and sex-matched norms along 4 scales: mood and emotions, behavioral control, social interactions, and self-efficacy. Using the computer adaptive test-based approach offers the ability to collect standardized, comprehensive functional information about claimants in an efficient way, which may prove useful in the context of the Social Security Administration's work disability programs.

Does postacute care site matter? A longitudinal study assessing functional recovery after a stroke.

OBJECTIVE: To determine the impact of postacute care site on stroke outcomes. DESIGN: Prospective cohort study. SETTING: Four northern California hospitals that are part of a single health maintenance organization. PARTICIPANTS: Patients with stroke (N=222) enrolled between February 2008 and July 2010. INTERVENTION: Not applicable. MAIN OUTCOME MEASURE: Baseline and 6-month assessments were performed using the Activity Measure for Post Acute Care (AM-PAC), a test of self-reported function in 3 domains: Basic Mobility, Daily Activities, and Applied Cognition. RESULTS: Of the 222 patients analyzed, 36% went home with no treatment, 22% received home health/outpatient care, 30% included an inpatient rehabilitation facility (IRF) in their care trajectory, and 13% included a skilled nursing facility (but not IRF) in their care trajectory. At 6 months, after controlling for important variables such as age, functional status at acute care discharge, and total hours of rehabilitation, patients who went to an IRF had functional scores that were at least 8 points higher (twice the minimally detectable change for the AM-PAC) than those who went to a skilled nursing facility in all 3 domains and in 2 of 3 functional domains compared with those who received home health/outpatient care. CONCLUSIONS: Patients with stroke may make more functional gains if their postacute care includes an IRF. This finding may have important implications as postacute care delivery is reshaped through health care reform.

Designing and implementing a system for tracking functional status after stroke: a feasibility study.

OBJECTIVE: To determine the feasibility of tracking stroke patients' functional outcomes in an integrated health system across a care continuum using the computer version of the Activity Measure of Post-Acute Care (AM-PAC). SETTING: A large integrated health care system in northern California. PARTICIPANTS: A total of 222 stroke patients (aged ≥18 years) who were hospitalized after an acute cerebrovascular accident. METHODS: An AM-PAC assessment was made at discharge from sites of care, including acute hospital, inpatient rehabilitation hospital, skilled nursing facility, home during home care, and outpatient settings. Assessments also were completed in the patient's home at 6 months. Data from the AM-PAC program were integrated with the health care system's databases. MAIN OUTCOME MEASUREMENTS: (1) AM-PAC administration time at the various sites of care; (2) assessment of a floor or a ceiling effect; and (3) administrative burden of tracking participants. RESULTS: AM-PAC assessment sessions averaged 7.9 minutes for data acquisition in 3 domains: Basic Mobility, Activities of Daily Living, and Applied Cognition. Participants answered, on average, 27 AM-PAC questions per session. A small ceiling effect was observed at 6 months, and there was a larger ceiling effect when the instrument was administered in an institution, ie, when the AM-PAC institutional item bank was used rather than the community item bank. It was feasible to track patients and to assess their function using the AM-PAC instrument from institutional to community settings. Implementation of the AM-PAC in clinical environments, and the success of the project, were influenced by instrumental, technological, operational, resource, and cultural factors. CONCLUSIONS: This study demonstrates the feasibility of implementing a single functional outcome instrument in clinical and community settings to measure rehabilitation functional outcomes of stroke patients. Integrating the AM-PAC measurement system into clinical workflows and the electronic medical record could provide assistance to clinicians for medical decision making, functional prognostication, and discharge planning.