The Performance of Multivariate Methods for Two-Group Comparisons with Small Samples and Incomplete Data.

In intervention studies having multiple outcomes, researchers often use a series of univariate tests (e.g., ANOVAs) to assess group mean differences. Previous research found that this approach properly controls Type I error and generally provides greater power compared to MANOVA, especially under realistic effect size and correlation combinations. However, when group differences are assessed for a specific outcome, these procedures are strictly univariate and do not consider the outcome correlations, which may be problematic with missing outcome data. Linear mixed or multivariate multilevel models (MVMMs), implemented with maximum likelihood estimation, present an alternative analysis option where outcome correlations are taken into account when specific group mean differences are estimated. In this study, we use simulation methods to compare the performance of separate independent samples t tests estimated with ordinary least squares and analogous t tests from MVMMs to assess two-group mean differences with multiple outcomes under small sample and missingness conditions. Study results indicated that a MVMM implemented with restricted maximum likelihood estimation combined with the Kenward-Roger correction had the best performance. Therefore, for intervention studies with small N and normally distributed multivariate outcomes, the Kenward-Roger procedure is recommended over traditional methods and conventional MVMM analyses, particularly with incomplete data.

A comparison of computerized adaptive testing and fixed-length short forms for the Prosthetic Limb Users Survey of Mobility (PLUS-MTM).

BACKGROUND: New health status instruments can be administered by computerized adaptive test or short forms. The Prosthetic Limb Users Survey of Mobility (PLUS-MTM) is a self-report measure of mobility for prosthesis users with lower limb loss. This study used the PLUS-M to examine advantages and disadvantages of computerized adaptive test and short forms. OBJECTIVES: To compare scores obtained from computerized adaptive test to scores obtained from fixed-length short forms (7-item and 12-item) in order to provide guidance to researchers and clinicians on how to select the best form of administration for different uses. STUDY DESIGN: Cross-sectional, observational study. METHODS: Individuals with lower limb loss completed the PLUS-M by computerized adaptive test and short forms. Administration time, correlations between the scores, and standard errors were compared. RESULTS: Scores and standard errors from the computerized adaptive test, 7-item short form, and 12-item short form were highly correlated and all forms of administration were efficient. Computerized adaptive test required less time to administer than either paper or electronic short forms; however, time savings were minimal compared to the 7-item short form. CONCLUSION: Results indicate that the PLUS-M computerized adaptive test is most efficient, and differences in scores between administration methods are minimal. The main advantage of the computerized adaptive test was more reliable scores at higher levels of mobility compared to short forms. Clinical relevance Health-related item banks, like the Prosthetic Limb Users Survey of Mobility (PLUS-MTM), can be administered by computerized adaptive testing (CAT) or as fixed-length short forms (SFs). Results of this study will help clinicians and researchers decide whether they should invest in a CAT administration system or whether SFs are more appropriate.

The Development of MST Test Information for the Prediction of Test Performances.

The current study proposes novel methods to predict multistage testing (MST) performance without conducting simulations. This method, called MST test information, is based on analytic derivation of standard errors of ability estimates across theta levels. We compared standard errors derived analytically to the simulation results to demonstrate the validity of the proposed method in both measurement precision and classification accuracy. The results indicate that the MST test information effectively predicted the performance of MST. In addition, the results of the current study highlighted the relationship among the test construction, MST design factors, and MST performance.

Translating CESD-20 and PHQ-9 Scores to PROMIS Depression.

This study examined the accuracy of depression cross-walk tables in a sample of people with multiple sclerosis (MS). The tables link scores of two commonly used depression measures to the Patient Reported Outcome Measurement Information System Depression (PROMIS-D) scale metric. We administered the 8-item PROMIS-D (Short-Form 8b; PROMIS-D-8), the 20-item Center for Epidemiologic Studies Depression Scale (CESD-20), and the 9-item Patient Health Questionnaire (PHQ-9) to 459 survey participants with MS. We examined correlations between actual PROMIS-D-8 scores and the scores predicted by cross-walks based on PHQ-9 and CESD-20 scores. Intraclass correlation coefficients were used to assess correspondence. Consistency in severity classification was also calculated. Finally, we used Bland-Altman plots to graphically examine the levels of agreement. The correlations between actual and cross-walked PROMIS-D-8 scores were strong (CESD-20 = .82; PHQ-9 = .74). The intraclass correlation was moderate (.77). Participants were consistently classified as having or not having at least moderate depressive symptoms by both actual and cross-walked scores derived from the CESD-20 (90%) and PHQ-9 (85%). Bland-Altman plots suggested the smaller differences between actual and cross-walked scores with greater-than-average depression severity. PROMIS cross-walk tables can be used to translate depression scores of people with MS to the PROMIS-D metric, promoting continuity with previous research.

Minimally important differences for Patient Reported Outcomes Measurement Information System pain interference for individuals with back pain.

BACKGROUND: The minimally important difference (MID) refers to the smallest change that is sufficiently meaningful to carry implications for patients' care. MIDs are necessary to guide the interpretation of scores. This study estimated MID for the Patient Reported Outcomes Measurement Information System (PROMIS) pain interference (PI). METHODS: Study instruments were administered to 414 people who participated in two studies that included treatment with low back pain (LBP; n=218) or depression (n=196). Participants with LBP received epidural steroid injections and participants with depression received antidepressants, psychotherapy, or both. MIDs were estimated for the changes in LBP. MIDs were included only if a priori criteria were met (ie, sample size ≥10, Spearman correlation ≥0.3 between anchor measures and PROMIS-PI scores, and effect size range =0.2-0.8). The interquartile range (IQR) of MID estimates was calculated. RESULTS: The IQR ranged from 3.5 to 5.5 points. The lower bound estimate of the IQR (3.5) was greater than mean of standard error of measurement (SEM) both at time 1 (SEM =2.3) and at time 2 (SEM =2.5), indicating that the estimate of MID exceeded measurement error. CONCLUSION: Based on our results, researchers and clinicians using PROMIS-PI can assume that change of 3.5 to 5.5 points in comparisons of mean PROMIS-PI scores of people with LBP can be considered meaningful.

Testing the measurement invariance of the University of Washington Self-Efficacy Scale short form across four diagnostic subgroups.

PURPOSE: The University of Washington Self-Efficacy Scale (UW-SES) was originally developed for people with multiple sclerosis (MS) and spinal cord injury (SCI). This study evaluates the measurement invariance of the 6-item short form of the UW-SES across four disability subgroups. Evidence of measurement invariance would extend the UW-SES for use in two additional diagnostic groups: muscular dystrophy (MD) and post-polio syndrome (PPS). METHODS: Multi-group confirmatory factor analysis was used to evaluate successive levels of measurement invariance of the 6-item short form, the UW-SES: (a) configural invariance, i.e., equivalent item-factor structures between groups; (b) metric invariance, i.e., equivalent unstandardized factor loadings between groups; and (c) scalar invariance, i.e., equivalent item intercepts between groups. Responses from the four groups with different diagnostic disorders were compared: MD (n = 172), MS (n = 868), PPS (n = 225), and SCI (n = 242). RESULTS: The results of this study support that the most rigorous form of invariance (i.e., scalar) holds for the 6-item short form of the UW-SES across the four diagnostic subgroups. CONCLUSIONS: The current study suggests that the 6-item short form of the UW-SES has the same meaning across the four diagnostic subgroups. Thus, the 6-item short form is validated for people with MD, MS, PPS, and SCI.

Marginalized Maximum Likelihood Estimation for the 1PL-AG IRT Model.

Marginal maximum likelihood estimation based on the expectation-maximization algorithm (MML/EM) is developed for the one-parameter logistic model with ability-based guessing (1PL-AG) item response theory (IRT) model. The use of the MML/EM estimator is cross-validated with estimates from NLMIXED procedure (PROC NLMIXED) in Statistical Analysis System. Numerical data are provided for comparisons of results from MML/EM and PROC NLMIXED.

Symptoms and quality of life indicators among children with chronic medical conditions.

BACKGROUND: Children with chronic conditions often experience numerous symptoms, but few research studies examine patterns of symptoms and quality of life (QoL) indicators. OBJECTIVE: To examine if reliable latent classes of children with chronic medical conditions can be identified based on the clustering of symptoms and QoL indicators. METHODS: Structured interviews were conducted with children ages 9-21 living with chronic medical conditions (N = 90). Multiple symptoms (e.g., pain, sleep, fatigue, and depression) and QoL indicators (e.g., life satisfaction and social support) were measured. Physical health and emotional, social, and school functioning were measured using the Pediatric Quality of Life Inventory (PedsQL). Latent class analysis was used to classify each child into a latent class whose members report similar patterns of responses. RESULTS: A three-class solution had the best model fit. Class 1 (high-symptom group; n = 15, 16.7%) reported the most problems with symptoms and the lowest scores on the QoL indicators. Class 2 (moderate-symptom group; n = 39, 43.3%) reported moderate levels of both symptoms and QoL indicators. Class 3 (low-symptom group; n = 36, 40.0%) reported the lowest levels of symptoms and the highest scores on the QoL indicators. CONCLUSIONS: The three latent classes identified in this study were distributed along the severity continuum. All symptoms and QoL indicators appeared to move in the same direction (e.g., worse symptoms with lower QoL). The PedsQL psychosocial health summary score (combining emotional, social, and school functioning scores) discriminated well between children with different levels of disease burden.

A comparison of panel designs with routing methods in the multistage test with the partial credit model.

In this study, we compared panel designs applied with various routing methods in the multistage test (MST) based on the partial credit model in the context of classification testing. Simulations were performed to compare three routing methods and four panel structures. Conditions of two test lengths and three passing rates were also included. The results showed that, regardless of the routing method used, the same panel structure performed similarly in terms of the precision of the classification decision with the same test length condition. The longer test length produced higher accuracy, whereas the 50 % passing rate yielded the lowest accuracy. Finally, all MST conditions performed well in terms of test security.