Applying multidimensional computerized adaptive testing to the MSQOL-54: a simulation study.

BACKGROUND: The Multiple Sclerosis Quality of Life-54 (MSQOL-54) is one of the most commonly-used MS-specific health-related quality of life (HRQOL) measures. It is a multidimensional, MS-specific HRQOL inventory, which includes the generic SF-36 core items, supplemented with 18 MS-targeted items. Availability of an adaptive short version providing immediate item scoring may improve instrument usability and validity. However, multidimensional computerized adaptive testing (MCAT) has not been previously applied to MSQOL-54 items. We thus aimed to apply MCAT to the MSQOL-54 and assess its performance. METHODS: Responses from a large international sample of 3669 MS patients were assessed. We calibrated 52 (of the 54) items using bifactor graded response model (10 group factors and one general HRQOL factor). Then, eight simulations were run with different termination criteria: standard errors (SE) for the general factor and group factors set to different values, and change in factor estimates from one item to the next set at < 0.01 for both the general and the group factors. Performance of the MCAT was assessed by the number of administered items, root mean square difference (RMSD), and correlation. RESULTS: Eight items were removed due to local dependency. The simulation with SE set to 0.32 (general factor), and no SE thresholds (group factors) provided satisfactory performance: the median number of administered items was 24, RMSD was 0.32, and correlation was 0.94. CONCLUSIONS: Compared to the full-length MSQOL-54, the simulated MCAT required fewer items without losing precision for the general HRQOL factor. Further work is needed to add/integrate/revise MSQOL-54 items in order to make the calibration and MCAT performance efficient also on group factors, so that the MCAT version may be used in clinical practice and research.

The Development, Deployment, and Evaluation of the CLEFT-Q Computerized Adaptive Test: A Multimethods Approach Contributing to Personalized, Person-Centered Health Assessments in Plastic Surgery.

BACKGROUND: Routine use of patient-reported outcome measures (PROMs) and computerized adaptive tests (CATs) may improve care in a range of surgical conditions. However, most available CATs are neither condition-specific nor coproduced with patients and lack clinically relevant score interpretation. Recently, a PROM called the CLEFT-Q has been developed for use in the treatment of cleft lip or palate (CL/P), but the assessment burden may be limiting its uptake into clinical practice. OBJECTIVE: We aimed to develop a CAT for the CLEFT-Q, which could facilitate the uptake of the CLEFT-Q PROM internationally. We aimed to conduct this work with a novel patient-centered approach and make source code available as an open-source framework for CAT development in other surgical conditions. METHODS: CATs were developed with the Rasch measurement theory, using full-length CLEFT-Q responses collected during the CLEFT-Q field test (this included 2434 patients across 12 countries). These algorithms were validated in Monte Carlo simulations involving full-length CLEFT-Q responses collected from 536 patients. In these simulations, the CAT algorithms approximated full-length CLEFT-Q scores iteratively, using progressively fewer items from the full-length PROM. Agreement between full-length CLEFT-Q score and CAT score at different assessment lengths was measured using the Pearson correlation coefficient, root-mean-square error (RMSE), and 95% limits of agreement. CAT settings, including the number of items to be included in the final assessments, were determined in a multistakeholder workshop that included patients and health care professionals. A user interface was developed for the platform, and it was prospectively piloted in the United Kingdom and the Netherlands. Interviews were conducted with 6 patients and 4 clinicians to explore end-user experience. RESULTS: The length of all 8 CLEFT-Q scales in the International Consortium for Health Outcomes Measurement (ICHOM) Standard Set combined was reduced from 76 to 59 items, and at this length, CAT assessments reproduced full-length CLEFT-Q scores accurately (with correlations between full-length CLEFT-Q score and CAT score exceeding 0.97, and the RMSE ranging from 2 to 5 out of 100). Workshop stakeholders considered this the optimal balance between accuracy and assessment burden. The platform was perceived to improve clinical communication and facilitate shared decision-making. CONCLUSIONS: Our platform is likely to facilitate routine CLEFT-Q uptake, and this may have a positive impact on clinical care. Our free source code enables other researchers to rapidly and economically reproduce this work for other PROMs.

Speeding up Monte Carlo simulations for the adaptive sum of powered score test with importance sampling.

A central but challenging problem in genetic studies is to test for (usually weak) associations between a complex trait (e.g., a disease status) and sets of multiple genetic variants. Due to the lack of a uniformly most powerful test, data-adaptive tests, such as the adaptive sum of powered score (aSPU) test, are advantageous in maintaining high power against a wide range of alternatives. However, there is often no closed-form to accurately and analytically calculate the p-values of many adaptive tests like aSPU, thus Monte Carlo (MC) simulations are often used, which can be time consuming to achieve a stringent significance level (e.g., 5e-8) used in genome-wide association studies (GWAS). To estimate such a small p-value, we need a huge number of MC simulations (e.g., 1e+10). As an alternative, we propose using importance sampling to speed up such calculations. We develop some theory to motivate a proposed algorithm for the aSPU test, and show that the proposed method is computationally more efficient than the standard MC simulations. Using both simulated and real data, we demonstrate the superior performance of the new method over the standard MC simulations.

Single-stage, three-arm, adaptive test strategies for non-inferiority trials with an unstable reference.

For indications where only unstable reference treatments are available and use of placebo is ethically justified, three-arm "gold standard" designs with an experimental, reference and placebo arm are recommended for non-inferiority trials. In such designs, the demonstration of efficacy of the reference or experimental treatment is a requirement. They have the disadvantage that only little can be concluded from the trial if the reference fails to be efficacious. To overcome this, we investigate novel single-stage, adaptive test strategies where non-inferiority is tested only if the reference shows sufficient efficacy and otherwise δ $$ \delta $$ -superiority of the experimental treatment over placebo is tested. With a properly chosen superiority margin, δ $$ \delta $$ -superiority indirectly shows non-inferiority. We optimize the sample size for several decision rules and find that the natural, data driven test strategy, which tests non-inferiority if the reference's efficacy test is significant, leads to the smallest overall and placebo sample sizes. We proof that under specific constraints on the sample sizes, this procedure controls the family-wise error rate. All optimal sample sizes are found to meet this constraint. We finally show how to account for a relevant placebo drop-out rate in an efficient way and apply the new test strategy to a real life data set.

Psychometric properties of the spinal cord injury-quality of life (SCI-QOL) Resilience item bank in a sample with spinal cord injury and chronic pain.

PURPOSE: To describe the psychometric properties (e.g., data distribution characteristics, convergent/discriminant validity, internal consistency reliability, and test administration characteristics) of the spinal cord injury quality of life measurement system (SCI-QOL) Resilience item bank delivered as a computer adaptive test (CAT) in a sample of individuals with chronic pain and spinal cord injury (SCI). METHODS: Descriptive statistics were calculated to investigate variable data distribution characteristics. Correlation analyses were conducted for convergent and discriminant validity. Item response theory-derived reliability was calculated for the SCI-QOL Resilience CAT. RESULT: One hundred thirty-three adults with SCI (N = 133; 73.5% male, 26.5% female) were enrolled. Sample mean T score on the SCI-QOL Resilience measure was 48.40, SD = 8.60 (min = 29.4; max = 70.0). The CAT administered between 4 (most common, 41.4% of cases) and 12 (9% of cases) items with the Mean#items = 5.73, SD = 2.45. The SCI-QOL Resilience CAT scores were normally distributed, with very low ceiling (0%) and floor (3%) effects. The SCI-QOL Resilience CAT had a reliability of 0.89, and the mean length of time for respondents to complete the SCI-QOL Resilience CAT was 44.34 s. SCI-QOL Resilience CAT validity was supported by significant moderate correlations with pain acceptance, depressive symptoms, pain catastrophizing, positive affect and well-being, and pain interference (convergent validity) and small non-significant correlations with age, sex, injury level, pain intensity, mobility level, and years since injury (discriminant validity). CONCLUSION: The SCI-QOL Resilience CAT demonstrated good convergent and discriminant validity. The CAT administration characteristics were impressive: With few items (low response burden), the scale achieved good reliability.

Recursive Partitioning vs Computerized Adaptive Testing to Reduce the Burden of Health Assessments in Cleft Lip and/or Palate: Comparative Simulation Study.

BACKGROUND: Computerized adaptive testing (CAT) has been shown to deliver short, accurate, and personalized versions of the CLEFT-Q patient-reported outcome measure for children and young adults born with a cleft lip and/or palate. Decision trees may integrate clinician-reported data (eg, age, gender, cleft type, and planned treatments) to make these assessments even shorter and more accurate. OBJECTIVE: We aimed to create decision tree models incorporating clinician-reported data into adaptive CLEFT-Q assessments and compare their accuracy to traditional CAT models. METHODS: We used relevant clinician-reported data and patient-reported item responses from the CLEFT-Q field test to train and test decision tree models using recursive partitioning. We compared the prediction accuracy of decision trees to CAT assessments of similar length. Participant scores from the full-length questionnaire were used as ground truth. Accuracy was assessed through Pearson's correlation coefficient of predicted and ground truth scores, mean absolute error, root mean squared error, and a two-tailed Wilcoxon signed-rank test comparing squared error. RESULTS: Decision trees demonstrated poorer accuracy than CAT comparators and generally made data splits based on item responses rather than clinician-reported data. CONCLUSIONS: When predicting CLEFT-Q scores, individual item responses are generally more informative than clinician-reported data. Decision trees that make binary splits are at risk of underfitting polytomous patient-reported outcome measure data and demonstrated poorer performance than CATs in this study.

Bimodal cochlear implantation in elderly patients.

OBJECTIVE: Bimodal stimulation is a standard option for asymmetric hearing loss in adults. Questions have been raised whether receiving two stimulations may conflict in elderly listeners where the central integration of an acoustic/electrical signal may be very important to obtain benefit in terms of speech perception. DESIGN: Clinical retrospective study. STUDY SAMPLE: The outcomes from 17 bimodal cochlear implant (CI) users were analysed. The test material consisted of speech audiometry in quiet and in noise (STARR and Matrix). RESULTS: Bimodal PTA and speech perception both in quiet and in noise were significantly better than CI or HA alone. Age showed a significant effect on bimodal STARR outcomes. Similarly, bimodal STARR scores improved significantly in comparison to Better Ear. CONCLUSION: Both Matrix and STARR tests were very difficult for many elderly CI listeners from the present study group, especially in unilateral listening condition. The performance improved significantly, emphasising a good integration of acoustic and electric hearing in this group of elderly bimodal listeners. Overall results highlighted how a specific study, based on speech perception in noise in the elderly listeners, might shed light on the effect of speech test modality on bimodal outcomes.

Reliability of Telephone Acquisition of the PROMIS Upper Extremity Computer Adaptive Test.

PURPOSE: Our primary purpose was to evaluate the reliability of telephone administration of the Patient-Reported Outcomes Measurement Information System (PROMIS) Upper Extremity (UE) Computer Adaptive Test (CAT) version 2.0 in a hand and upper extremity population, and secondarily to make comparisons with the abbreviated version of the Disabilities of the Arm, Shoulder, and Hand (QuickDASH). METHODS: Patients more than 1 year out from hand surgeries performed at a single tertiary institution were enrolled. Half of the patients completed telephone PROMIS UE CAT and QuickDASH surveys first, followed by computer-based surveys 1 to 10 days later, and the other half completed them in the reverse order. Telephone surveys were readministered 2 to 6 weeks later to evaluate test-retest reliability. Concordance correlation coefficients (CCCs) were used to assess agreement between telephone and computer-based scores, and intraclass correlation coefficients (ICCs) were used to assess test-retest reliability. The proportion of patients with discrepancies in follow-up scores that exceeded estimates of the minimal clinically important difference (MCID) was evaluated. RESULTS: For the 89 enrolled patients, the PROMIS UE CAT CCC was 0.82 (83% confidence interval [83% CI], 0.77-0.86; good), which was significantly lower than 0.92 (83% CI, 0.89-0.94; good to excellent) for the QuickDASH. The PROMIS UE CAT ICC did not differ significantly from the QuickDASH (0.85 and 0.91, respectively). Differences in telephone versus computer scores exceeded 5 points (MCID estimate) for the PROMIS UE CAT in 34% of patients versus 5% of patients exceeding 14 points (MCID estimate) for the QuickDASH. CONCLUSIONS: Significantly better reliability was observed for the QuickDASH than the PROMIS UE CAT when comparing telephone with computer-based score acquisition. Over one-third of patients demonstrated a clinically relevant difference in scores between the telephone and the computer-administered tests. We conclude that the PROMIS UE CAT should only be administered through computer-based methods. CLINICAL RELEVANCE: These findings suggest that differences in collection methods for the PROMIS UE CAT may systematically affect the scores obtained, which may erroneously influence the interpretation of postoperative scores for hand surgery patients.

Establishing the Minimal Clinically Important Difference for the PROMIS Upper Extremity Computer Adaptive Test Version 2.0 in a Nonshoulder Hand and Upper Extremity Population.

PURPOSE: Our primary purpose was to calculate the minimal clinically important difference (MCID) for the Patient-Reported Outcomes Measurement Information System (PROMIS) Upper Extremity (UE) Computer Adaptive Test (CAT) version 2.0 (v2.0) for a nonshoulder hand and upper extremity population. Secondarily, we calculated the PROMIS Physical Function (PF) CAT v2.0 and the abbreviated version of the Disabilities of the Arm, Shoulder, and Hand (QuickDASH) MCID. METHODS: Adult patients treated by 1 of 5 fellowship-trained hand surgeons between March 2015 and September 2019 at an academic tertiary institution were identified. The PROMIS UE CAT v2.0, PROMIS PF CAT v2.0, and QuickDASH were collected via tablet computer. Inclusion required response to at least 1 of the instruments at both baseline and follow-up (6 ± 4 weeks), and a response to the anchor question: "Compared to your first evaluation at the University Orthopaedic Center, how would you describe your physical function level now?" An additional anchor question assessing treatment-related improvement was also asked. The MCID was calculated using an anchor-based approach using the mean change difference between groups reporting no change and slight change for both anchor questions, and with the 1/2 SD method. RESULTS: Of 2,106 participants, mean age was 48 ± 17 years, 53% were female, and 53% were recovering from surgery. Of these patients, 381 completed the PROMISE UE CAT v2.0, 497 completed the PROMIS PF CAT v2.0, and 2,018 completed the QuickDASH. The score change between baseline and follow-up was significantly different between anchor groups for both anchor-based MCID calculations. Anchor-based MCID values were 3.0 to 4.0 for the UE CAT, 2.1 to 3.6 for the PF CAT, and 10.3 for the QuickDASH. The MCID values per the 1/2 SD method were 4.1, 4.1, and 10.2, respectively. CONCLUSIONS: We propose MCID ranges of 3.0 to 4.1 for the PROMIS UE CAT v2.0, and 2.1 to 4.1 for the PROMIS PF CAT v2.0. The observed QuickDASH MCID values (10.2-10.3) are within the range of previously published values. CLINICAL RELEVANCE: These MCID estimates will aid in interpreting clinical outcomes and in powering clinical studies.

Evaluating the performance of PROMIS and QuickDASH instruments in an intercollegiate Division 1 athlete population.

BACKGROUND: Athletes demonstrate high levels of physical function, leading to difficulties in patient-reported outcome scoring and interpretation. In particular, the ability of patient-reported outcome (PRO) instruments to adequately discriminate between high levels of upper extremity function-that is, the ceiling effect-is limited. This study evaluated performance characteristics of the shortened version of the Disabilities of the Arm, Shoulder, and Hand (QuickDASH) survey and Patient-Reported Outcomes Measurement Information System (PROMIS) metrics in a population of Division 1 intercollegiate athletes, with specific attention to ceiling effects. METHODS: At a single institution, Division I intercollegiate athletes cleared for full participation in the 2018-2019 season were eligible. The following PROs were collected prospectively via tablet computer: PROMIS upper extremity (UE) computer adaptive test (CAT), PROMIS physical function (PF) CAT, QuickDASH, and QuickDASH Sports/Performing Arts Module. Descriptive statistics, and ceiling and floor effects, were calculated. The proportion of athletes with maximal scores on each PRO were compared to normative values using the 1-sample Wilcoxon signed rank test. RESULTS: A total of 268 participants were included; the mean age was 19.9 ± 1.5 years, 49% were female, and 61% were overhead athletes. Large ceiling effects were observed for the UE CAT (46%), QuickDASH (58%), and QuickDASH Sports Module (82%). The PF CAT demonstrated a relatively low ceiling effect of 6.7%. Athlete scores were all significantly better than published age-matched values from a normative population for all instruments, with the exception of no difference on the QuickDASH for males. DISCUSSION AND/OR CONCLUSION: The PROMIS UE CAT and QuickDash instruments are limited in their ability to assess and discriminate upper extremity function in highly functioning individuals such as Division I athletes. The PROMIS PF CAT, a measure of general physical function, did not suffer from a large ceiling effect.

Symptom burden trajectories experienced by patients with brain tumors.

BACKGROUND: Survivors of childhood brain tumors experience persistent health concerns across their lifespan. In the current study, the authors evaluated changes in symptom burden over the course of 12 months using pediatric Patient-Reported Outcomes Measurement Information System (PROMIS) measures. METHODS: Data from 202 survivors aged 8 to 21 years and 262 parents of survivors who were aged 5 to 21 years were analyzed. All completed a PROMIS Cognition short form and computerized adaptive tests of pediatric Anxiety, Depressive Symptoms, Fatigue, Mobility, Upper Extremity Function, and Peer Relationships. Approximately one-half of participants (223 participants: 97 survivors of childhood brain tumors and 126 parents) completed the 12-month follow-up. Linear mixed-effects models evaluated group-level symptoms over time. Cox proportional hazard models explored whether symptoms predicted survival, and latent class growth analysis investigated patterns of individual-level symptom changes over time. RESULTS: Linear mixed-effects models demonstrated that patient-reported Cognition and parent-reported Anxiety worsened over time. Latent class growth analysis results indicated that patient and parent reports diverged, both with regard to the number of classes identified and in the trends of these classes. Parents and patients reported similar patterns of depression over time. For the other areas, parents either were more likely to observe different patterns (Peer Relationships and Mobility) or less likely to observe different patterns (Upper Extremity Function, Cognition, Anxiety, and Fatigue). Baseline patient-reported Mobility and Upper Extremity Function were found to be associated with survival. CONCLUSIONS: Survivors of childhood brain tumors demonstrated different trajectory patterns of symptom burden. Along with baseline functioning status and days since treatment, patient-reported Mobility and Upper Extremity Function were associated with survival, suggesting a possible role for patient-reported outcomes in clinical care, especially individualized, tailored assessments such as PROMIS.

Maximizing the Potential of Patient-Reported Assessments by Using the Open-Source Concerto Platform With Computerized Adaptive Testing and Machine Learning.

Patient-reported assessments are transforming many facets of health care, but there is scope to modernize their delivery. Contemporary assessment techniques like computerized adaptive testing (CAT) and machine learning can be applied to patient-reported assessments to reduce burden on both patients and health care professionals; improve test accuracy; and provide individualized, actionable feedback. The Concerto platform is a highly adaptable, secure, and easy-to-use console that can harness the power of CAT and machine learning for developing and administering advanced patient-reported assessments. This paper introduces readers to contemporary assessment techniques and the Concerto platform. It reviews advances in the field of patient-reported assessment that have been driven by the Concerto platform and explains how to create an advanced, adaptive assessment, for free, with minimal prior experience with CAT or programming.

Development and Validation of an Item Bank for Drug Dependence Measurement Using Computer Adaptive Testing.

BACKGROUND: Currently, measurement tools to assess patient-reported outcomes for drug dependence are limited in their latent trait to adapt to the needs of individual patients while also maintaining comparability of scores across patients. Purpose/Objectives: To develop an item bank for computer adaptive testing (CAT) to measure severity of drug dependence. Methods: There were four phases: (1) review the literature of drug dependence measurement; (2) formulate an item list to be assessed by experts; (3) pretest our item list in two substance dependence treatment centers; and (4) field-test and conduct psychometric performance analysis with the final item bank. Additionally, based on our response data, a CAT simulation was used to validate the item bank, Drug Dependence CAT (DD-CAT). Results: The final drug dependence item bank - with a unidimensional configuration - contained 56 items with good item-fit, high discrimination, no differential item functioning, and covered all symptoms of diagnostic criteria for drug dependence. These results revealed that the final item bank was of good quality. Additionally, the results of a simulation CAT procedure with real response data indicated that the DD-CAT item bank exhibited acceptable and reasonable test reliability, content validity, and criterion-related validity. Conclusions/Importance: The proposed item bank for DD-CAT contained acceptable reliability and validity, and exhibited a shorter but efficient assessment of drug dependence. These psychometric properties can result in shorter test times, less information loss, and a reduction in the testing burden of patients.

Utilizing the Patient Reported Outcomes Measurement Information System (PROMIS®) to increase referral to ancillary support services for severely symptomatic patients with gynecologic cancer.

OBJECTIVE: The Patient-Reported Outcomes Measurement Information System (PROMIS®) Network has developed a comprehensive repository of electronic patient reported outcomes measures (ePROs) of major symptom domains that have been validated in cancer patients. Their use for patients with gynecologic cancer has been understudied. Our objective was to establish feasibility and acceptability of PROMIS ePRO integration in a gynecologic oncology outpatient clinic and assess if it can help identify severely symptomatic patients and increase referral to supportive services. METHODS: English-speaking patients with a confirmed history of gynecologic cancer completed PROMIS ePROs on iPads in the waiting area of an outpatient gynecologic oncology clinic. Symptom scores were calculated for each respondent and grouped using documented severity thresholds. Response data was compared with clinicopathologic characteristics across symptom domains. Severely symptomatic patients were offered referral to ancillary services and asked to complete post-exposure surveys assessing acceptability of the ePRO. RESULTS: Of the 336 patients who completed ePROs, 35% had active disease and 19% had experienced at least one disease recurrence. Sixty-nine percent of the cohort demonstrated moderate to severe physical dysfunction (60%), pain (36%), fatigue (28%), anxiety (9%), depression (8%), and sexual dysfunction (32%). Thirty-nine (12%) severely symptomatic patients were referred to services such as psychiatry, palliative care, pain management, social work or integrative oncology care. Most survey respondents identified the ePROs as helpful (78%) and easy to complete (92%). CONCLUSIONS: Outpatient PROMIS ePRO administration is feasible and acceptable to gynecologic oncology patients and can help identify severely symptomatic patients for referral to ancillary support services.

Calibration of an item bank in 474 orthopedic patients using Rasch analysis for computer-adaptive assessment of anxiety.

OBJECTIVE: To calibrate an item bank of anxiety-related questions for use in orthopedic patients within a computer-adaptive test. DESIGN: This is a psychometric study. SETTING: The sample of orthopedic patients was recruited in two orthopedic rehabilitation clinics in Germany. SUBJECTS: A total of 474 orthopedic rehabilitation patients were recruited for this study. INTERVENTIONS: Not applicable. MAIN MEASURES: The main measure is an adapted version of an existing anxiety item pool for cardiovascular rehabilitation patients. RESULTS: The results of the confirmatory factor analysis and Mokken analysis confirmed a one-factor structure and double monotonicity. An anxiety item bank (48 items) could be developed and calibrated using Rasch analysis. It fitted to the Rasch model with a non-significant item-trait interaction (χ2(203) = 172.59; P = .94) and was free of differential item functioning. Unidimensionality could be verified and the person separation reliability was .96. The category threshold parameters varied between 4.72 and 3.16 (7.88 logits). CONCLUSION: The unidimensional anxiety item bank provides the basis for a computer-adaptive test to assess a wide range of anxiety in rehabilitation patients with orthopedic diseases with very good psychometric characteristics.

Evaluation of Version 2.0 of the PROMIS Upper Extremity Computer Adaptive Test in Nonshoulder Upper Extremity Patients.

PURPOSE: The Patient-Reported Outcomes Measurement Information System (PROMIS) Upper Extremity Computer Adaptive Test (UE CAT) was recently updated to version 2.0 (v2.0). We hypothesized that the PROMIS UE CAT v2.0 would exhibit improved performance characteristics compared with the Quick Disabilities of the Arm, Shoulder, and Hand (QuickDASH) when administered to a nonshoulder upper extremity patient population. METHODS: The UE CAT v2.0, Physical Function (PF) CAT v2.0, and the QuickDASH were each prospectively administered via tablet computer to all patients presenting to a tertiary hand and upper extremity clinic between April 2017 and October 2017. Patient responses were analyzed, and the mean, range, floor and ceiling effect, and correlations between instruments were calculated. RESULTS: Among 825 patients, the mean UE CAT v2.0 score was 38.3 (SD 10.7) with a range of 15 to 61 and interquartile range of 15.4. The UE CAT v2.0 had a strong correlation with the QuickDASH (r = -0.749) and the PF CAT v2.0 (r = 0.719). No patient scored between 56 and 60, indicating a gap in scoring in that range. The UE CAT v2.0 demonstrated a floor effect of 1%, a ceiling effect of 6.9%, and a high internal consistency with a Cronbach alpha of 0.99. CONCLUSIONS: The PROMIS UE CAT v2.0 demonstrated improved ceiling effects, range, and a decreased gap in scoring compared with prior versions. Limitations of the PROMIS UE CAT v2.0 are still present, but updates have led to an incremental improvement over prior versions, demonstrating the ability to influence PROMIS instrument performance through upgrades. CLINICAL RELEVANCE: The updated PROMIS UE CAT v2.0 still demonstrates a ceiling effect and gap in scores at the upper end of the instrument, both of which may limit discrimination between different levels of upper extremity function for high-functioning patients.

PROMIS physical function underperforms psychometrically relative to American Shoulder and Elbow Surgeons score in patients undergoing anatomic total shoulder arthroplasty.

BACKGROUND: The purpose of this study was to evaluate the psychometric properties of the Patient-Reported Outcomes Measurement Information System (PROMIS) physical function computer adaptive test (PF-CAT) relative to the American Shoulder and Elbow Surgeons (ASES) score in patients with glenohumeral osteoarthritis undergoing primary anatomic total shoulder arthroplasty (TSA). METHODS: A retrospective study of an institutional TSA registry was performed. Preoperative PROMIS PF-CAT and ASES scores were collected. Floor and ceiling effects were determined, and convergent validity was established through Pearson correlations. Rasch partial credit modeling was used for psychometric analysis of the validity of PF-CAT and ASES question items. Person-item maps were generated to characterize the distribution of question responses along the latent dimension of shoulder disability. RESULTS: Responses from 179 patients (184 shoulders) were included. PF-CAT had a moderate correlation to ASES (r = 0.487; P < .001), with no floor or ceiling effects; ASES had a 1.1% floor effect and no ceiling effect. With iterative Rasch model item-reduction analysis eliminating poorly fitting question items, all possible PF-CAT items were eliminated after 6 iterations. With ASES, just 1 function question item was dropped. Person-item maps showed ASES to be superior to PROMIS PF-CAT psychometrically, with sequential and improved coverage of the latent dimension of shoulder disability. CONCLUSION: Despite moderate correlation with ASES, PROMIS PF-CAT demonstrated inferior validity and psychometric properties in patients undergoing TSA. PF-CAT should not replace the ASES in this population of patients.

Development of the life impact burn recovery evaluation (LIBRE) profile: assessing burn survivors' social participation.

PURPOSE: Measuring the impact burn injuries have on social participation is integral to understanding and improving survivors' quality of life, yet there are no existing instruments that comprehensively measure the social participation of burn survivors. This project aimed to develop the Life Impact Burn Recovery Evaluation Profile (LIBRE), a patient-reported multidimensional assessment for understanding the social participation after burn injuries. METHODS: 192 questions representing multiple social participation areas were administered to a convenience sample of 601 burn survivors. Exploratory factor analysis and confirmatory factor analysis (CFA) were used to identify the underlying structure of the data. Using item response theory methods, a Graded Response Model was applied for each identified sub-domain. The resultant multidimensional LIBRE Profile can be administered via Computerized Adaptive Testing (CAT) or fixed short forms. RESULTS: The study sample included 54.7% women with a mean age of 44.6 (SD 15.9) years. The average time since burn injury was 15.4 years (0-74 years) and the average total body surface area burned was 40% (1-97%). The CFA indicated acceptable fit statistics (CFI range 0.913-0.977, TLI range 0.904-0.974, RMSEA range 0.06-0.096). The six unidimensional scales were named: relationships with family and friends, social interactions, social activities, work and employment, romantic relationships, and sexual relationships. The marginal reliability of the full item bank and CATs ranged from 0.84 to 0.93, with ceiling effects less than 15% for all scales. CONCLUSIONS: The LIBRE Profile is a promising new measure of social participation following a burn injury that enables burn survivors and their care providers to measure social participation.

Item usage in a multidimensional computerized adaptive test (MCAT) measuring health-related quality of life.

PURPOSE: Examining item usage is an important step in evaluating the performance of a computerized adaptive test (CAT). We study item usage for a newly developed multidimensional CAT which draws items from three PROMIS domains, as well as a disease-specific one. METHODS: The multidimensional item bank used in the current study contained 194 items from four domains: the PROMIS domains fatigue, physical function, and ability to participate in social roles and activities, and a disease-specific domain (the COPD-SIB). The item bank was calibrated using the multidimensional graded response model and data of 795 patients with chronic obstructive pulmonary disease. To evaluate the item usage rates of all individual items in our item bank, CAT simulations were performed on responses generated based on a multivariate uniform distribution. The outcome variables included active bank size and item overuse (usage rate larger than the expected item usage rate). RESULTS: For average θ-values, the overall active bank size was 9-10%; this number quickly increased as θ-values became more extreme. For values of -2 and +2, the overall active bank size equaled 39-40%. There was 78% overlap between overused items and active bank size for average θ-values. For more extreme θ-values, the overused items made up a much smaller part of the active bank size: here the overlap was only 35%. CONCLUSIONS: Our results strengthen the claim that relatively short item banks may suffice when using polytomous items (and no content constraints/exposure control mechanisms), especially when using MCAT.

Rasch analysis of the Pediatric Evaluation of Disability Inventory-computer adaptive test (PEDI-CAT) item bank for children and young adults with spinal muscular atrophy.

INTRODUCTION: In this study we evaluated the suitability of a caregiver-reported functional measure, the Pediatric Evaluation of Disability Inventory-Computer Adaptive Test (PEDI-CAT), for children and young adults with spinal muscular atrophy (SMA). METHODS: PEDI-CAT Mobility and Daily Activities domain item banks were administered to 58 caregivers of children and young adults with SMA. Rasch analysis was used to evaluate test properties across SMA types. RESULTS: Unidimensional content for each domain was confirmed. The PEDI-CAT was most informative for type III SMA, with ability levels distributed close to 0.0 logits in both domains. It was less informative for types I and II SMA, especially for mobility skills. Item and person abilities were not distributed evenly across all types. CONCLUSIONS: The PEDI-CAT may be used to measure functional performance in SMA, but additional items are needed to identify small changes in function and best represent the abilities of all types of SMA. Muscle Nerve 54: 1097-1107, 2016.