Improving Assessment of Work Related Mental Health Function Using the Work Disability Functional Assessment Battery (WD-FAB).

Purpose To improve the mental health component of the Work Disability Functional Assessment Battery (WD-FAB), developed for the US Social Security Administration's (SSA) disability determination process. Specifically our goal was to expand the WD-FAB scales of mood & emotions, resilience, social interactions, and behavioral control to improve the depth and breadth of the current scales and expand the content coverage to include aspects of cognition & communication function. Methods Data were collected from a random, stratified sample of 1695 claimants applying for the SSA work disability benefits, and a general population sample of 2025 working age adults. 169 new items were developed to replenish the WD-FAB scales and analyzed using factor analysis and item response theory (IRT) analysis to construct unidimensional scales. We conducted computer adaptive test (CAT) simulations to examine the psychometric properties of the WD-FAB. Results Analyses supported the inclusion of four mental health subdomains: Cognition & Communication (68 items), Self-Regulation (34 items), Resilience & Sociability (29 items) and Mood & Emotions (34 items). All scales yielded acceptable psychometric properties. Conclusions IRT methods were effective in expanding the WD-FAB to assess mental health function. The WD-FAB has the potential to enhance work disability assessment both within the context of the SSA disability programs as well as other clinical and vocational rehabilitation settings.

Improving measures of work-related physical functioning.

PURPOSE: To expand content of the physical function domain of the Work Disability Functional Assessment Battery (WD-FAB), developed for the US Social Security Administration's (SSA) disability determination process. METHODS: Newly developed questions were administered to 3532 recent SSA applicants for work disability benefits and 2025 US adults. Factor analyses and item response theory (IRT) methods were used to calibrate and link the new items to the existing WD-FAB, and computer-adaptive test simulations were conducted. RESULTS: Factor and IRT analyses supported integration of 44 new items into three existing WD-FAB scales and the addition of a new 11-item scale (Community Mobility). The final physical function domain consisting of: Basic Mobility (56 items), Upper Body Function (34 items), Fine Motor Function (45 items), and Community Mobility (11 items) demonstrated acceptable psychometric properties. CONCLUSIONS: The WD-FAB offers an important tool for enhancement of work disability determination. The FAB could provide relevant information about work-related functioning for initial assessment of claimants; identifying denied applicants who may benefit from interventions to improve work and health outcomes; enhancing periodic review of work disability beneficiaries; and assessing outcomes for policies, programs and services targeting people with work disability.

Work disability functional assessment battery: feasibility and psychometric properties.

OBJECTIVE: To assess the feasibility and psychometric properties of 8 scales covering 2 domains of the newly developed Work Disability Functional Assessment Battery (WD-FAB): physical function (PF) and behavioral health (BH) function. DESIGN: Cross-sectional study. SETTING: Community. PARTICIPANTS: Adults (N=973) unable to work because of a physical (n=497) or a mental (n=476) disability. INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: Each disability group responded to a survey consisting of the relevant WD-FAB scales and existing measures of established validity. The WD-FAB scales were evaluated with regard to data quality (score distribution, percentage of "I don't know" responses), efficiency of administration (number of items required to achieve reliability criterion, time required to complete the scale) by computerized adaptive testing (CAT), and measurement accuracy as tested by person fit. Construct validity was assessed by examining both convergent and discriminant correlations between the WD-FAB scales and scores on same-domain and cross-domain established measures. RESULTS: Data quality was good, and CAT efficiency was high across both WD-FAB domains. Measurement accuracy was very good for PF scales; BH scales demonstrated more variability. Construct validity correlations, both convergent and divergent, between all WD-FAB scales and established measures were in the expected direction and range of magnitude. CONCLUSIONS: The data quality, CAT efficiency, person fit, and construct validity of the WD-FAB scales were well supported and suggest that the WD-FAB could be used to assess PF and BH function related to work disability. Variation in scale performance suggests the need for future work on item replenishment and refinement, particularly with regard to the Self-Efficacy scale.

Development of a computer-adaptive physical function instrument for Social Security Administration disability determination.

OBJECTIVES: To develop and test an instrument to assess physical function for Social Security Administration (SSA) disability programs, the SSA-Physical Function (SSA-PF) instrument. Item response theory (IRT) analyses were used to (1) create a calibrated item bank for each of the factors identified in prior factor analyses, (2) assess the fit of the items within each scale, (3) develop separate computer-adaptive testing (CAT) instruments for each scale, and (4) conduct initial psychometric testing. DESIGN: Cross-sectional data collection; IRT analyses; CAT simulation. SETTING: Telephone and Internet survey. PARTICIPANTS: Two samples: SSA claimants (n=1017) and adults from the U.S. general population (n=999). INTERVENTIONS: None. MAIN OUTCOME MEASURES: Model fit statistics, correlation, and reliability coefficients. RESULTS: IRT analyses resulted in 5 unidimensional SSA-PF scales: Changing & Maintaining Body Position, Whole Body Mobility, Upper Body Function, Upper Extremity Fine Motor, and Wheelchair Mobility for a total of 102 items. High CAT accuracy was demonstrated by strong correlations between simulated CAT scores and those from the full item banks. On comparing the simulated CATs with the full item banks, very little loss of reliability or precision was noted, except at the lower and upper ranges of each scale. No difference in response patterns by age or sex was noted. The distributions of claimant scores were shifted to the lower end of each scale compared with those of a sample of U.S. adults. CONCLUSIONS: The SSA-PF instrument contributes important new methodology for measuring the physical function of adults applying to the SSA disability programs. Initial evaluation revealed that the SSA-PF instrument achieved considerable breadth of coverage in each content domain and demonstrated noteworthy psychometric properties.

Development of an instrument to measure behavioral health function for work disability: item pool construction and factor analysis.

OBJECTIVES: To develop a broad set of claimant-reported items to assess behavioral health functioning relevant to the Social Security disability determination processes, and to evaluate the underlying structure of behavioral health functioning for use in development of a new functional assessment instrument. DESIGN: Cross-sectional. SETTING: Community. PARTICIPANTS: Item pools of behavioral health functioning were developed, refined, and field tested in a sample of persons applying for Social Security disability benefits (N=1015) who reported difficulties working because of mental or both mental and physical conditions. INTERVENTIONS: None. MAIN OUTCOME MEASURE: Social Security Administration Behavioral Health (SSA-BH) measurement instrument. RESULTS: Confirmatory factor analysis (CFA) specified that a 4-factor model (self-efficacy, mood and emotions, behavioral control, social interactions) had the optimal fit with the data and was also consistent with our hypothesized conceptual framework for characterizing behavioral health functioning. When the items within each of the 4 scales were tested in CFA, the fit statistics indicated adequate support for characterizing behavioral health as a unidimensional construct along these 4 distinct scales of function. CONCLUSIONS: This work represents a significant advance both conceptually and psychometrically in assessment methodologies for work-related behavioral health. The measurement of behavioral health functioning relevant to the context of work requires the assessment of multiple dimensions of behavioral health functioning. Specifically, we identified a 4-factor model solution that represented key domains of work-related behavioral health functioning. These results guided the development and scale formation of a new SSA-BH instrument.

Conceptual foundation for measures of physical function and behavioral health function for Social Security work disability evaluation.

Physical and mental impairments represent the 2 largest health condition categories for which workers receive Social Security disability benefits. Comprehensive assessment of physical and mental impairments should include aspects beyond medical conditions such as a person's underlying capabilities as well as activity demands relevant to the context of work. The objective of this article is to describe the initial conceptual stages of developing new measurement instruments of behavioral health and physical functioning relevant for Social Security work disability evaluation purposes. To outline a clear conceptualization of the constructs to be measured, 2 content models were developed using structured and informal qualitative approaches. We performed a structured literature review focusing on work disability and incorporating aspects of the International Classification of Functioning, Disability and Health as a unifying taxonomy for framework development. Expert interviews provided advice and consultation to enhance face validity of the resulting content models. The content model for work-related behavioral health function identifies 5 major domains: (1) behavior control, (2) basic interactions, (3) temperament and personality, (4) adaptability, and (5) workplace behaviors. The content model describing physical functioning includes 3 domains: (1) changing and maintaining body position, (2) whole-body mobility, and (3) carrying, moving, and handling objects. These content models informed subsequent measurement properties including item development and measurement scale construction, and provided conceptual coherence guiding future empirical inquiry. The proposed measurement approaches show promise to comprehensively and systematically assess physical and behavioral health functioning relevant to work.

Development of a self-report physical function instrument for disability assessment: item pool construction and factor analysis.

OBJECTIVES: To build a comprehensive item pool representing work-relevant physical functioning and to test the factor structure of the item pool. These developmental steps represent initial outcomes of a broader project to develop instruments for the assessment of function within the context of Social Security Administration (SSA) disability programs. DESIGN: Comprehensive literature review; gap analysis; item generation with expert panel input; stakeholder interviews; cognitive interviews; cross-sectional survey administration; and exploratory and confirmatory factor analyses to assess item pool structure. SETTING: In-person and semistructured interviews and Internet and telephone surveys. PARTICIPANTS: Sample of SSA claimants (n=1017) and a normative sample of adults from the U.S. general population (n=999). INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURE: Model fit statistics. RESULTS: The final item pool consisted of 139 items. Within the claimant sample, 58.7% were white; 31.8% were black; 46.6% were women; and the mean age was 49.7 years. Initial factor analyses revealed a 4-factor solution, which included more items and allowed separate characterization of: (1) changing and maintaining body position, (2) whole body mobility, (3) upper body function, and (4) upper extremity fine motor. The final 4-factor model included 91 items. Confirmatory factor analyses for the 4-factor models for the claimant and the normative samples demonstrated very good fit. Fit statistics for claimant and normative samples, respectively, were: Comparative Fit Index=.93 and .98; Tucker-Lewis Index=.92 and .98; and root mean square error approximation=.05 and .04. CONCLUSIONS: The factor structure of the physical function item pool closely resembled the hypothesized content model. The 4 scales relevant to work activities offer promise for providing reliable information about claimant physical functioning relevant to work disability.

Scale refinement and initial evaluation of a behavioral health function measurement tool for work disability evaluation.

OBJECTIVE: To use item response theory (IRT) data simulations to construct and perform initial psychometric testing of a newly developed instrument, the Social Security Administration Behavioral Health Function (SSA-BH) instrument, that aims to assess behavioral health functioning relevant to the context of work. DESIGN: Cross-sectional survey followed by IRT calibration data simulations. SETTING: Community. PARTICIPANTS: Sample of individuals applying for Social Security Administration disability benefits: claimants (n=1015) and a normative comparative sample of U.S. adults (n=1000). INTERVENTIONS: None. MAIN OUTCOME MEASURE: SSA-BH measurement instrument. RESULTS: IRT analyses supported the unidimensionality of 4 SSA-BH scales: mood and emotions (35 items), self-efficacy (23 items), social interactions (6 items), and behavioral control (15 items). All SSA-BH scales demonstrated strong psychometric properties including reliability, accuracy, and breadth of coverage. High correlations of the simulated 5- or 10-item computer adaptive tests with the full item bank indicated robust ability of the computer adaptive testing approach to comprehensively characterize behavioral health function along 4 distinct dimensions. CONCLUSIONS: Initial testing and evaluation of the SSA-BH instrument demonstrated good accuracy, reliability, and content coverage along all 4 scales. Behavioral function profiles of Social Security Administration claimants were generated and compared with age- and sex-matched norms along 4 scales: mood and emotions, behavioral control, social interactions, and self-efficacy. Using the computer adaptive test-based approach offers the ability to collect standardized, comprehensive functional information about claimants in an efficient way, which may prove useful in the context of the Social Security Administration's work disability programs.

Linking job duties, functioning, and employment status using the Work-Disability Functional Assessment Battery (WD-FAB): An expert coding and quantitative analysis.

BACKGROUND: An understanding of the link between specific occupational demands and individual worker functioning is limited, although such information could permit an assessment of the fit between the two in a manner that would inform national and state disability programs such as vocational rehabilitation and Social Security disability programs. OBJECTIVE: Our goal was to examine the utility of assessing physical and mental functioning relative to self-reported job duties to identify the domains of worker functioning most likely to create barriers to fulfilling an occupation's specific requirements. METHODS: Through primary survey data collection, 1770 participants completed the Work-Disability Functional Assessment Battery (WD-FAB) instrument after reporting details on their occupations (or most recent occupation if not working). Expert coders evaluated the level of function expected to successfully carry out each self-reported job duty with respect to six scales of physical and mental function. Quantitative analysis is used to examine the relationship between functioning and job duties. RESULTS: Those not working due to disability were more likely to fall short of the threshold of the physical and mental functioning requirements of their last job's three main job duties compared to those currently employed. Mental function scales were most likely to be the area experiencing a shortfall. CONCLUSIONS: Functional difficulties impede the ability to continue working in particular jobs that require that ability. This points to a need for specific accommodations to be implemented to bridge the gap between job requirements and functional capacity so that workers may remain engaged in their current work.

Information Extraction Framework for Disability Determination Using a Mental Functioning Use-Case.

Natural language processing (NLP) in health care enables transformation of complex narrative information into high value products such as clinical decision support and adverse event monitoring in real time via the electronic health record (EHR). However, information technologies for mental health have consistently lagged because of the complexity of measuring and modeling mental health and illness. The use of NLP to support management of mental health conditions is a viable topic that has not been explored in depth. This paper provides a framework for the advanced application of NLP methods to identify, extract, and organize information on mental health and functioning to inform the decision-making process applied to assessing mental health. We present a use-case related to work disability, guided by the disability determination process of the US Social Security Administration (SSA). From this perspective, the following questions must be addressed about each problem that leads to a disability benefits claim: When did the problem occur and how long has it existed? How severe is it? Does it affect the person's ability to work? and What is the source of the evidence about the problem? Our framework includes 4 dimensions of medical information that are central to assessing disability-temporal sequence and duration, severity, context, and information source. We describe key aspects of each dimension and promising approaches for application in mental functioning. For example, to address temporality, a complete functional timeline must be created with all relevant aspects of functioning such as intermittence, persistence, and recurrence. Severity of mental health symptoms can be successfully identified and extracted on a 4-level ordinal scale from absent to severe. Some NLP work has been reported on the extraction of context for specific cases of wheelchair use in clinical settings. We discuss the links between the task of information source assessment and work on source attribution, coreference resolution, event extraction, and rule-based methods. Gaps were identified in NLP applications that directly applied to the framework and in existing relevant annotated data sets. We highlighted NLP methods with the potential for advanced application in the field of mental functioning. Findings of this work will inform the development of instruments for supporting SSA adjudicators in their disability determination process. The 4 dimensions of medical information may have relevance for a broad array of individuals and organizations responsible for assessing mental health function and ability. Further, our framework with 4 specific dimensions presents significant opportunity for the application of NLP in the realm of mental health and functioning beyond the SSA setting, and it may support the development of robust tools and methods for decision-making related to clinical care, program implementation, and other outcomes.

Older Adult Productive Activity Participation Using the National Health and Aging Trends Study.

This study aims to characterize factors related to productive activity participation among community-dwelling older adults. Cross-sectional analyses using data from the National Health and Aging Trends Study were used to calculate weighted frequencies representative of the U.S. population of older adults. Multivariate logistic regression was used to explore factors related to participation outcomes (paid work, volunteering, caregiving). We found that 21% of community-dwelling older adults in the United States reported currently working. Older adults reported working in a wide range of occupations. Driving emerged as one of the most important factors related to increased odds of productive activity participation. Age, gender, and health factors were also significantly associated with increased odds of productive activity participation. By understanding the current profile of participation in activities including employment, caregiving, and volunteering among a national sample of community-dwelling older adults, we can effectively inform intervention programs and resource allocation to support productive aging.

Measuring Activity Limitations Within the National Health and Aging Trends Study (NHATS).

BACKGROUND AND OBJECTIVES: To better understand the disablement process among older adults, improved measures of activity limitations are needed. Traditional population-level measures lack the ability to distinguish precise gradations of activity limitation and are unable to detect degrees of differences over a wide range of ability levels. Therefore, we used contemporary measurement methods to improve upon current methodologies for characterizing activity limitations within the National Health and Aging Trends Study (NHATS) . RESEARCH DESIGN AND METHODS: We used the NHATS Round 1 cohort to assess the feasibility of constructing an Activity Limitations scale using Rasch item response theory methods. Factor analysis was used to develop the scale from a set of existing items in the NHATS Mobility, Self-Care, and Household Activity domains. Psychometric properties of the scale were evaluated and the scale was used to examine change in activity limitations among the sample from 2011 to 2015. RESULTS: Results supported an 18-item scale (N = 7,609). Rasch infit and outfit statistics were within acceptable range for all items (Cronbach's alpha = 0.95; sample score reliability = 0.83). From 2011 to 2015, 5.88% older adults demonstrated increase in function, 15% showed decrease in function, and 78% of the sample showed no change (did not exceed ± MDC90). DISCUSSION AND IMPLICATIONS: Findings demonstrate that a unidimensional, interval scale of activity limitations can be constructed using traditional survey measures nested within the NHATS. Results revealed concerns regarding ceiling effects within the current self-report items of activity limitations suggesting future work is needed to expand the range of ability currently represented in the NHATS Activity Limitation items.

Measuring Work Related Physical and Mental Health Function: Updating the Work Disability Functional Assessment Battery (WD-FAB) Using Item Response Theory.

OBJECTIVE: To further improve measurement of work-related physical and mental health by updating the Work Disability Assessment Battery (WD-FAB). METHODS: Cross-sectional study with 1024 disability claimants and 1000 working age (21 to 66 years) adults in the United States. Developed new items to replenish the WD-FAB and analyzed using factor analysis and item response theory (IRT). Computer adaptive testing (CAT) simulations evaluated the psychometric properties of the original versus updated WD-FAB. RESULTS: Analyses confirmed the structure of the WD-FAB. Twenty-three new items were added (basic mobility: 7, upper body function: 4, fine motor: 6, self-regulation: 1, resilience & sociability: 5 items). CONCLUSIONS: Findings support the WD-FAB as a robust, psychometrically sound assessment of work-related function. Extensive content coverage (331 items) represents eight physical and mental health domains. IRT/CAT methods allow administration in under 15 minutes. The WD-FAB may prove valuable for efficiently characterizing work-related function across work rehabilitation settings.

Do claimants over-report behavioral health dysfunction when filing for work disability benefits?

BACKGROND: Questions exist related to the best way to use medical evidence relative to self-report as part of the SSA disability determination process. OBJECTIVE: To examine concordance between provider and claimant responses along the four dimensions of work related behavioral health functioning: Social Interactions, Mood and Emotions, Behavioral Control, and Self-Efficacy. METHODS: Using secondary data from a larger study, which collected data on individuals reporting difficulties with work (claimants) due to mental conditions, 39 items were completed by claimants and their healthcare provider. Inter-rater agreement was assessed using three techniques: Cohen's kappa, percent absolute agreement, and folded mountain plots. RESULTS: A sample of 65 dyads was obtained. Inter-rater agreement was low for most items (k=0.0-0.20) with a minority of items having fair agreement (k=0.21-0.40) Percent agreement was fair: Mood and Emotions (46%), Self-Efficacy (44%), Behavioral Control (39%) and Social Interactions (38%). Overall, providers reported lower functioning compared to claimants for the Behavioral Control and Self-Efficacy scales; the reverse trend held for the Mood and Emotions scale. CONCLUSIONS: Results indicate discordance between provider and claimant report of behavioral health functioning. Understanding reasons for and approaches to reconciling the inconsistencies between claimant and provider perspectives is a complex task. These findings have implications for how best to assess mental and behavioral-health related work disability in the absence of an established gold standard measure.

Work-related measures of physical and behavioral health function: Test-retest reliability.

BACKGROUND: The Work Disability Functional Assessment Battery (WD-FAB), developed for potential use by the US Social Security Administration to assess work-related function, currently consists of five multi-item scales assessing physical function and four multi-item scales assessing behavioral health function; the WD-FAB scales are administered as Computerized Adaptive Tests (CATs). OBJECTIVE: The goal of this study was to evaluate the test-retest reliability of the WD-FAB Physical Function and Behavioral Health CATs. METHODS: We administered the WD-FAB scales twice, 7-10 days apart, to a sample of 376 working age adults and 316 adults with work-disability. Intraclass correlation coefficients were calculated to measure the consistency of the scores between the two administrations. Standard error of measurement (SEM) and minimal detectable change (MDC90) were also calculated to measure the scales precision and sensitivity. RESULTS: For the Physical Function CAT scales, the ICCs ranged from 0.76 to 0.89 in the working age adult sample, and 0.77-0.86 in the sample of adults with work-disability. ICCs for the Behavioral Health CAT scales ranged from 0.66 to 0.70 in the working age adult sample, and 0.77-0.80 in the adults with work-disability. The SEM ranged from 3.25 to 4.55 for the Physical Function scales and 5.27-6.97 for the Behavioral Health function scales. For all scales in both samples, the MDC90 ranged from 7.58 to 16.27. CONCLUSION: Both the Physical Function and Behavioral Health CATs of the WD-FAB demonstrated good test-retest reliability in adults with work-disability and general adult samples, a critical requirement for assessing work related functioning in disability applicants and in other contexts.

Interpreting physical and behavioral health scores from new work disability instruments.

OBJECTIVE: To develop a system to guide interpretation of scores generated from 2 new instruments measuring work-related physical and behavioral health functioning (Work Disability - Physical Function (WD-PF) and WD - Behavioral Function (WD-BH)). DESIGN: Cross-sectional, secondary data from 3 independent samples to develop and validate the functional levels for physical and behavioral health functioning. SUBJECTS: Physical group: 999 general adult subjects, 1,017 disability applicants and 497 work-disabled subjects. Behavioral health group: 1,000 general adult subjects, 1,015 disability applicants and 476 work-disabled subjects. METHODS: Three-phase analytic approach including item mapping, a modified-Delphi technique, and known-groups validation analysis were used to develop and validate cut-points for functional levels within each of the WD-PF and WD-BH instrument's scales. RESULTS: Four and 5 functional levels were developed for each of the scales in the WD-PF and WD-BH instruments. Distribution of the comparative samples was in the expected direction: the general adult samples consistently demonstrated scores at higher functional levels compared with the claimant and work-disabled samples. CONCLUSION: Using an item-response theory-based methodology paired with a qualitative process appears to be a feasible and valid approach for translating the WD-BH and WD-PF scores into meaningful levels useful for interpreting a person's work-related physical and behavioral health functioning.

Combining agreement and frequency rating scales to optimize psychometrics in measuring behavioral health functioning.

OBJECTIVE: The goal of this article was to investigate optimal functioning of using frequency vs. agreement rating scales in two subdomains of the newly developed Work Disability Functional Assessment Battery: the Mood & Emotions and Behavioral Control scales. STUDY DESIGN AND SETTING: A psychometric study comparing rating scale performance embedded in a cross-sectional survey used for developing a new instrument to measure behavioral health functioning among adults applying for disability benefits in the United States was performed. RESULTS: Within the sample of 1,017 respondents, the range of response category endorsement was similar for both frequency and agreement item types for both scales. There were fewer missing values in the frequency items than the agreement items. Both frequency and agreement items showed acceptable reliability. The frequency items demonstrated optimal effectiveness around the mean ± 1-2 standard deviation score range; the agreement items performed better at the extreme score ranges. CONCLUSION: Findings suggest an optimal response format requires a mix of both agreement-based and frequency-based items. Frequency items perform better in the normal range of responses, capturing specific behaviors, reactions, or situations that may elicit a specific response. Agreement items do better for those whose scores are more extreme and capture subjective content related to general attitudes, behaviors, or feelings of work-related behavioral health functioning.