Estimating Optimal Weights for Compound Scores: A Multidimensional IRT Approach.

A method is proposed for constructing indices as linear functions of variables such that the reliability of the compound score is maximized. Reliability is defined in the framework of latent variable modeling [i.e., item response theory (IRT)] and optimal weights of the components of the index are found by maximizing the posterior variance relative to the total latent variable variance. Three methods for estimating the weights are proposed. The first is a likelihood-based approach, that is, marginal maximum likelihood (MML). The other two are Bayesian approaches based on Markov chain Monte Carlo (MCMC) computational methods. One is based on an augmented Gibbs sampler specifically targeted at IRT, and the other is based on a general purpose Gibbs sampler such as implemented in OpenBugs and Jags. Simulation studies are presented to demonstrate the procedure and to compare the three methods. Results are very similar, so practitioners may be suggested the use of the easily accessible latter method. A real-data set pertaining to the 28-joint Disease Activity Score is used to show how the methods can be applied in a complex measurement situation with multiple time points and mixed data formats.

Automated Assessment of Patients' Self-Narratives for Posttraumatic Stress Disorder Screening Using Natural Language Processing and Text Mining.

Patients' narratives about traumatic experiences and symptoms are useful in clinical screening and diagnostic procedures. In this study, we presented an automated assessment system to screen patients for posttraumatic stress disorder via a natural language processing and text-mining approach. Four machine-learning algorithms-including decision tree, naive Bayes, support vector machine, and an alternative classification approach called the product score model-were used in combination with n-gram representation models to identify patterns between verbal features in self-narratives and psychiatric diagnoses. With our sample, the product score model with unigrams attained the highest prediction accuracy when compared with practitioners' diagnoses. The addition of multigrams contributed most to balancing the metrics of sensitivity and specificity. This article also demonstrates that text mining is a promising approach for analyzing patients' self-expression behavior, thus helping clinicians identify potential patients from an early stage.

Assessment of fatigue in rheumatoid arthritis: a psychometric comparison of single-item, multiitem, and multidimensional measures.

OBJECTIVE: To compare the psychometric functioning of multidimensional disease-specific, multiitem generic, and single-item measures of fatigue in patients with rheumatoid arthritis (RA). METHODS: Confirmatory factor analysis (CFA) and longitudinal item response theory (IRT) modeling were used to evaluate the measurement structure and local reliability of the Bristol RA Fatigue Multi-Dimensional Questionnaire (BRAF-MDQ), the Medical Outcomes Study Short Form-36 (SF-36) vitality scale, and the BRAF Numerical Rating Scales (BRAF-NRS) in a sample of 588 patients with RA. RESULTS: A 1-factor CFA model yielded a similar fit to a 5-factor model with subscale-specific dimensions, and the items from the different instruments adequately fit the IRT model, suggesting essential unidimensionality in measurement. The SF-36 vitality scale outperformed the BRAF-MDQ at lower levels of fatigue, but was less precise at moderate to higher levels of fatigue. At these levels of fatigue, the living, cognition, and emotion subscales of the BRAF-MDQ provide additional precision. The BRAF-NRS showed a limited measurement range with its highest precision centered on average levels of fatigue. CONCLUSION: The different instruments appear to access a common underlying domain of fatigue severity, but differ considerably in their measurement precision along the continuum. The SF-36 vitality scale can be used to measure fatigue severity in samples with relatively mild fatigue. For samples expected to have higher levels of fatigue, the multidimensional BRAF-MDQ appears to be a better choice. The BRAF-NRS are not recommended if precise assessment is required, for instance in longitudinal settings.

Extending the floor and the ceiling for assessment of physical function.

OBJECTIVE: To improve the assessment of physical function by enhancing precision of physical function assessment as it pertains to subjects at extreme ends of the health continuum (i.e., subjects with extremely poor function ["floor"] or extremely good health ["ceiling"]). METHODS: Under the Patient-Reported Outcomes Measurement Information System (PROMIS) (a National Institutes of Health initiative), we developed new items to assess floor and ceiling physical function in order to supplement the existing item bank. Using item response theory and standard PROMIS methodology, we developed 31 floor items and 31 ceiling items and administered the items during a 12-month prospective, observational study of 737 subjects whose health status was at either extreme. Effect size was calculated and change over time was compared across anchor instruments and across items. Using the observed changes in scores, we back-calculated sample size requirements for the new and comparison measures. RESULTS: We studied 444 subjects who had been diagnosed as having a chronic illness and/or were of old age and 293 generally fit subjects (including athletes in training). Item response theory analyses confirmed that the new floor and ceiling items outperformed reference items (P < 0.001). The estimated post hoc sample size requirements were reduced by a factor of 2-4 for the floor population and a factor of 2 for the ceiling population. CONCLUSION: Extending the range of items by which physical function is measured can substantially improve measurement quality, reduce sample size requirements, and improve research efficiency. The paradigm shift from assessing disability to assessing physical function focuses assessment on the entire spectrum of physical function, signals improvement in the conceptual base of outcome assessment, and may be transformative as medical goals more closely approach societal goals for health.

Linking physical function outcomes in rheumatology: performance of a crosswalk for converting Health Assessment Questionnaire scores to Short Form 36 physical functioning scale scores.

OBJECTIVE: To evaluate the reliability of a crosswalk, developed in The Netherlands, between the Health Assessment Questionnaire (HAQ) disability index (DI) and the Short Form 36 physical functioning scale (PF-10) in a sample of patients with various rheumatic diseases in the US. METHODS: Baseline data from patients with rheumatoid arthritis (RA; n = 29,020), fibromyalgia (FM; n = 3,776), and systemic lupus erythematosus (SLE; n = 1,609) participating in the National Data Bank for Rheumatic Diseases were analyzed. Reliability of the crosswalk was evaluated by calculating intraclass correlation coefficients (ICCs), and agreement between observed and predicted scores was evaluated using the Bland-Altman approach. RESULTS. The crosswalk produced reliable conversions for both the HAQ DI (ICC range 0.70-0.77) and PF-10 (ICC range 0.73-0.78) in all 3 disease groups. The mean difference between observed and expected scores was close to zero in US patients with RA. For all 3 disease groups, the limits of agreement were fairly wide and conversion at the level of individual patients is not recommended. CONCLUSION: The crosswalk produced reliable conversions at the group level in a crosscultural setting and can be used to convert HAQ DI to PF-10 scores and vice versa in US patients with RA, FM, or SLE.

Development and evaluation of a crosswalk between the SF-36 physical functioning scale and Health Assessment Questionnaire disability index in rheumatoid arthritis.

BACKGROUND: The SF-36 physical functioning scale (PF-10) and the Health Assessment Questionnaire disability index (HAQ-DI) are the most frequently used instruments for measuring self-reported physical function in rheumatoid arthritis (RA). The objective of this study was to develop a crosswalk between scores on the PF-10 and HAQ-DI in RA. METHODS: Item response theory (IRT) methods were used to co-calibrate both scales using data from 1791 RA patients. The appropriateness of a Rasch-based crosswalk was evaluated by comparing it with crosswalks based on a two-parameter and a multi-dimensional IRT model. The accuracy of the final crosswalk was cross-validated using baseline (n = 532) and 6-month follow-up (n = 276) data from an independent cohort of early RA patients. RESULTS: The PF-10 and HAQ-DI adequately fit a unidimensional Rasch model. Both scales measured a wide range of functioning, although the HAQ-DI tended to better target lower levels of functioning. The Rasch-based crosswalk performed similarly to crosswalks based on the two-parameter and multidimensional IRT models. Agreement between predicted and observed scale scores in the cross-validation sample was acceptable for group-level comparisons. The longitudinal validity in discriminating between disease response states was similar between observed and predicted scores. CONCLUSION: The crosswalk developed in this study allows for converting scores from one scale to the other and can be used for group-level analyses in patients with RA.

Crosscultural measurement equivalence of the Health Assessment Questionnaire II.

OBJECTIVE: To evaluate the crosscultural measurement equivalence of the US and Dutch Health Assessment Questionnaire II (HAQ-II) in rheumatoid arthritis (RA). METHODS: Item response theory (IRT) analyses were performed on US (n = 18,747) and Dutch (n = 1,022) HAQ-II data to evaluate the equivalence of crosscultural item performance. The observed inconsistencies were modeled by assigning country-specific item parameters to biased items. The impact of crosscultural item bias on the comparability of the Dutch and US total scores was analyzed by evaluating the agreement between physical function levels estimated from an IRT model with country-specific item parameters for biased items and physical function levels estimated from the original model that did not account for cultural bias. RESULTS: Two items showed significant crosscultural bias. However, the agreement in physical function estimates between the respecified and original models was very high, with an intraclass correlation coefficient >0.99 and Bland-Altman limits of agreement ranging from -0.08 to -0.01 on a latent scale with a mean of 0 and an SD of 1. CONCLUSION: This study suggests that the Dutch and US HAQ-II produce total scores that can be interpreted interchangeably across countries in RA studies, despite some minor bias at the item level.

Application of the health assessment questionnaire disability index to various rheumatic diseases.

PURPOSE: To investigate whether the Stanford Health Assessment Questionnaire Disability Index (HAQ-DI) can serve as a generic instrument for measuring disability across different rheumatic diseases and to propose a scoring method based on item response theory (IRT) modeling to support this goal. METHODS: The HAQ-DI was administered to a cross-sectional sample of patients with confirmed rheumatoid arthritis (n = 619), osteoarthritis (n = 125), or gout (n = 102). The results were analyzed using the generalized partial credit model as an IRT model. RESULTS: It was found that 4 out of 8 item categories of the HAQ-DI displayed substantial differential item functioning (DIF) over the three diseases. Further, it was shown that this DIF could be modeled using an IRT model with disease-specific item parameters, which produces measures that are comparable for the three diseases. CONCLUSION: Although the HAQ-DI partially functioned differently in the three disease groups, the measurement regarding the disability level of the patients can be made comparable using IRT methods.

Variance decomposition using an IRT measurement model.

Large scale research projects in behaviour genetics and genetic epidemiology are often based on questionnaire or interview data. Typically, a number of items is presented to a number of subjects, the subjects' sum scores on the items are computed, and the variance of sum scores is decomposed into a number of variance components. This paper discusses several disadvantages of the approach of analysing sum scores, such as the attenuation of correlations amongst sum scores due to their unreliability. It is shown that the framework of Item Response Theory (IRT) offers a solution to most of these problems. We argue that an IRT approach in combination with Markov chain Monte Carlo (MCMC) estimation provides a flexible and efficient framework for modelling behavioural phenotypes. Next, we use data simulation to illustrate the potentially huge bias in estimating variance components on the basis of sum scores. We then apply the IRT approach with an analysis of attention problems in young adult twins where the variance decomposition model is extended with an IRT measurement model. We show that when estimating an IRT measurement model and a variance decomposition model simultaneously, the estimate for the heritability of attention problems increases from 40% (based on sum scores) to 73%.