Validation of the cognitive section of the Penn computerized adaptive test for neurocognitive and clinical psychopathology assessment (CAT-CCNB).

BACKGROUND: The Penn Computerized Neurocognitive Battery is an efficient tool for assessing brain-behavior domains, and its efficiency was augmented via computerized adaptive testing (CAT). This battery requires validation in a separate sample to establish psychometric properties. METHODS: In a mixed community/clinical sample of N = 307 18-to-35-year-olds, we tested the relationships of the CAT tests with the full-form tests. We compared discriminability among recruitment groups (psychosis, mood, control) and examined how their scores relate to demographics. CAT-Full relationships were evaluated based on a minimum inter-test correlation of 0.70 or an inter-test correlation within at least 0.10 of the full-form correlation with a previous administration of the full battery. Differences in criterion relationships were tested via mixed models. RESULTS: Most tests (15/17) met the minimum criteria for replacing the full-form with the updated CAT version (mean r = 0.67; range = 0.53-0.80) when compared to relationships of the full-forms with previous administrations of the full-forms (mean r = 0.68; range = 0.50-0.85). Most (16/17) CAT-based relationships with diagnostics and other validity criteria were indistinguishable (interaction p > 0.05) from their full-form counterparts. CONCLUSIONS: The updated CNB shows psychometric properties acceptable for research. The full-forms of some tests should be retained due to insufficient time savings to justify the loss in precision.

Computerized adaptive testing to screen pre-school children for emotional and behavioral problems.

Questionnaires to detect emotional and behavioral (EB) problems in preventive child healthcare (PCH) should be short; this potentially affects their validity and reliability. Computerized adaptive testing (CAT) could overcome this weakness. The aim of this study was to (1) develop a CAT to measure EB problems among pre-school children and (2) assess the efficiency and validity of this CAT. We used a Dutch national dataset obtained from parents of pre-school children undergoing a well-child care assessment by PCH (n = 2192, response 70%). Data regarded 197 items on EB problems, based on four questionnaires, the Strengths and Difficulties Questionnaire (SDQ), the Child Behavior Checklist (CBCL), the Ages and Stages Questionnaire: Social Emotional (ASQ:SE), and the Brief Infant-Toddler Social and Emotional Assessment (BITSEA). Using 80% of the sample, we calculated item parameters necessary for a CAT and defined a cutoff for EB problems. With the remaining part of the sample, we used simulation techniques to determine the validity and efficiency of this CAT, using as criterion a total clinical score on the CBCL. Item criteria were met by 193 items. This CAT needed, on average, 16 items to identify children with EB problems. Sensitivity and specificity compared to a clinical score on the CBCL were 0.89 and 0.91, respectively, for total problems; 0.80 and 0.93 for emotional problems; and 0.94 and 0.91 for behavioral problems.    Conclusion: A CAT is very promising for the identification of EB problems in pre-school children, as it seems to yield an efficient, yet high-quality identification. This conclusion should be confirmed by real-life administration of this CAT. What is Known: • Studies indicate the validity of using computerized adaptive test (CAT) applications to identify emotional and behavioral problems in school-aged children. • Evidence is as yet limited on whether CAT applications can also be used with pre-school children. What is New: • The results of this study show that a computerized adaptive test is very promising for the identification of emotional and behavior problems in pre-school children, as it appears to yield an efficient and high-quality identification.

Item selection methods in multidimensional computerized adaptive testing for forced-choice items using Thurstonian IRT model.

Multidimensional computerized adaptive testing for forced-choice items (MFC-CAT) combines the benefits of multidimensional forced-choice (MFC) items and computerized adaptive testing (CAT) in that it eliminates response biases and reduces administration time. Previous studies that explored designs of MFC-CAT only discussed item selection methods based on the Fisher information (FI), which is known to perform unstably at early stages of CAT. This study proposes a set of new item selection methods based on the KL information for MFC-CAT (namely MFC-KI, MFC-KB, and MFC-KLP) based on the Thurstonian IRT (TIRT) model. Three simulation studies, including one based on real data, were conducted to compare the performance of the proposed KL-based item selection methods against the existing FI-based methods in three- and five-dimensional MFC-CAT scenarios with various test lengths and inter-trait correlations. Results demonstrate that the proposed KL-based item selection methods are feasible for MFC-CAT and generate acceptable trait estimation accuracy and uniformity of item pool usage. Among the three proposed methods, MFC-KB and MFC-KLP outperformed the existing FI-based item selection methods and resulted in the most accurate trait estimation and relatively even utilization of the item pool.

Validity and reliability of the Patient-Reported Outcomes Measurement Information System (PROMIS®) using computerized adaptive testing in patients with advanced chronic kidney disease.

BACKGROUND: The Patient-Reported Outcomes Measurement Information System (PROMIS®) has been recommended for computerized adaptive testing (CAT) of health-related quality of life. This study compared the content, validity, and reliability of seven PROMIS CATs to the 12-item Short-Form Health Survey (SF-12) in patients with advanced chronic kidney disease. METHODS: Adult patients with chronic kidney disease and an estimated glomerular filtration rate under 30 mL/min/1.73 m2 who were not receiving dialysis treatment completed seven PROMIS CATs (assessing physical function, pain interference, fatigue, sleep disturbance, anxiety, depression, and the ability to participate in social roles and activities), the SF-12, and the PROMIS Pain Intensity single item and Dialysis Symptom Index at inclusion and 2 weeks. A content comparison was performed between PROMIS CATs and the SF-12. Construct validity of PROMIS CATs was assessed using Pearson's correlations. We assessed the test-retest reliability of all patient-reported outcome measures by calculating the intraclass correlation coefficient and minimal detectable change. RESULTS: In total, 207 patients participated in the study. A median of 45 items (10 minutes) were completed for PROMIS CATs. All PROMIS CATs showed evidence of sufficient construct validity. PROMIS CATs, most SF-12 domains and summary scores, and Dialysis Symptom Index showed sufficient test-retest reliability (intraclass correlation coefficient ≥ 0.70). PROMIS CATs had a lower minimal detectable change compared with the SF-12 (range, 5.7-7.4 compared with 11.3-21.7 across domains, respectively). CONCLUSION: PROMIS CATs showed sufficient construct validity and test-retest reliability in patients with advanced chronic kidney disease. PROMIS CATs required more items but showed better reliability than the SF-12. Future research is needed to investigate the feasibility of PROMIS CATs for routine nephrology care.

Applying multidimensional computerized adaptive testing to the MSQOL-54: a simulation study.

BACKGROUND: The Multiple Sclerosis Quality of Life-54 (MSQOL-54) is one of the most commonly-used MS-specific health-related quality of life (HRQOL) measures. It is a multidimensional, MS-specific HRQOL inventory, which includes the generic SF-36 core items, supplemented with 18 MS-targeted items. Availability of an adaptive short version providing immediate item scoring may improve instrument usability and validity. However, multidimensional computerized adaptive testing (MCAT) has not been previously applied to MSQOL-54 items. We thus aimed to apply MCAT to the MSQOL-54 and assess its performance. METHODS: Responses from a large international sample of 3669 MS patients were assessed. We calibrated 52 (of the 54) items using bifactor graded response model (10 group factors and one general HRQOL factor). Then, eight simulations were run with different termination criteria: standard errors (SE) for the general factor and group factors set to different values, and change in factor estimates from one item to the next set at < 0.01 for both the general and the group factors. Performance of the MCAT was assessed by the number of administered items, root mean square difference (RMSD), and correlation. RESULTS: Eight items were removed due to local dependency. The simulation with SE set to 0.32 (general factor), and no SE thresholds (group factors) provided satisfactory performance: the median number of administered items was 24, RMSD was 0.32, and correlation was 0.94. CONCLUSIONS: Compared to the full-length MSQOL-54, the simulated MCAT required fewer items without losing precision for the general HRQOL factor. Further work is needed to add/integrate/revise MSQOL-54 items in order to make the calibration and MCAT performance efficient also on group factors, so that the MCAT version may be used in clinical practice and research.

Measuring glaucoma quality of life in an Asian population using item banking: psychometric evaluation and computerized adaptive testing simulations.

PURPOSE: To assess the psychometric properties of glaucoma-specific health-related quality of life (HRQoL) item banks (IBs), and explore their efficiency using computerized adaptive testing (CAT) simulations. METHODS: In this cross-sectional, clinical study, 300 Asian glaucoma patients answered 221 items within seven IBs: Ocular Comfort Symptoms (OS); Activity Limitation (AL); Lighting (LT); Mobility (MB); Glaucoma Management (GM); Psychosocial (PSY); and Work (WK). Rasch analysis was conducted to assess each IB's psychometric properties (e.g., item "fit" to the construct; unidimensionality) and a set of analytic performance criteria guiding decision making relating to retaining or dropping domains and items was employed. CAT simulations determined the mean number of items for 'high' and 'moderate' measurement precision (stopping rule: SEM 0.3 and 0.387, respectively). RESULTS: Participants' mean age was 67.2 ± 9.2 years (62% male; 87% Chinese). LT, MB, and GM displayed good psychometric properties overall. To optimize AL's psychometric properties, 16 items were deleted due to poor "fit", high missing data, item bias, low discrimination and/or a low clinical/patient importance rating. To resolve multidimensionality in PSY, we rehomed 16 items into a "Concern (CN)" domain. PSY and CN required further amendment, including collapsing of response categories, and removal of poorly functioning items (N = 7). Due to poor measurement precision, low applicability and high ceiling effect, low test information indices, and low item separation index the WK IB was not considered further. In CAT simulations on the final seven IBs (n = 182 items total), an average of 12.1 and 15.7 items per IB were required for moderate and high precision measurement, respectively. CONCLUSIONS: After reengineering our seven IBs, they displayed robust psychometric properties and good efficiency in CAT simulations. Once finalized, GlauCAT™-Asian may enable comprehensive assessment of the HRQoL impact of glaucoma and associated treatments.

The Development, Deployment, and Evaluation of the CLEFT-Q Computerized Adaptive Test: A Multimethods Approach Contributing to Personalized, Person-Centered Health Assessments in Plastic Surgery.

BACKGROUND: Routine use of patient-reported outcome measures (PROMs) and computerized adaptive tests (CATs) may improve care in a range of surgical conditions. However, most available CATs are neither condition-specific nor coproduced with patients and lack clinically relevant score interpretation. Recently, a PROM called the CLEFT-Q has been developed for use in the treatment of cleft lip or palate (CL/P), but the assessment burden may be limiting its uptake into clinical practice. OBJECTIVE: We aimed to develop a CAT for the CLEFT-Q, which could facilitate the uptake of the CLEFT-Q PROM internationally. We aimed to conduct this work with a novel patient-centered approach and make source code available as an open-source framework for CAT development in other surgical conditions. METHODS: CATs were developed with the Rasch measurement theory, using full-length CLEFT-Q responses collected during the CLEFT-Q field test (this included 2434 patients across 12 countries). These algorithms were validated in Monte Carlo simulations involving full-length CLEFT-Q responses collected from 536 patients. In these simulations, the CAT algorithms approximated full-length CLEFT-Q scores iteratively, using progressively fewer items from the full-length PROM. Agreement between full-length CLEFT-Q score and CAT score at different assessment lengths was measured using the Pearson correlation coefficient, root-mean-square error (RMSE), and 95% limits of agreement. CAT settings, including the number of items to be included in the final assessments, were determined in a multistakeholder workshop that included patients and health care professionals. A user interface was developed for the platform, and it was prospectively piloted in the United Kingdom and the Netherlands. Interviews were conducted with 6 patients and 4 clinicians to explore end-user experience. RESULTS: The length of all 8 CLEFT-Q scales in the International Consortium for Health Outcomes Measurement (ICHOM) Standard Set combined was reduced from 76 to 59 items, and at this length, CAT assessments reproduced full-length CLEFT-Q scores accurately (with correlations between full-length CLEFT-Q score and CAT score exceeding 0.97, and the RMSE ranging from 2 to 5 out of 100). Workshop stakeholders considered this the optimal balance between accuracy and assessment burden. The platform was perceived to improve clinical communication and facilitate shared decision-making. CONCLUSIONS: Our platform is likely to facilitate routine CLEFT-Q uptake, and this may have a positive impact on clinical care. Our free source code enables other researchers to rapidly and economically reproduce this work for other PROMs.

Applying Item Response Theory to the Student Adaptation to College Questionnaire: Examining Psychometric Characteristics and Developing Computerized Adaptive Testing Version.

Incoming students have many difficulties adjusting to college, and selecting appropriate measures to effectively screen them is indispensable, especially in China, where there is insufficient research in this area. To enrich domestic research, this study seeks to examine psychometric characteristics and develop a computerized adaptive version of the Student Adaptation to College Questionnaire (SACQ-CAT) based on a sample of Chinese students. Under the framework of item response theory, the item bank of student adaptation to college was formulated after uni-dimensionality testing, model comparison, item fit testing, and local independence testing. Subsequently, a CAT simulation, including three termination rules, was performed using real data to evaluate and verify the SACQ-CAT. The results showed reliability values exceeding 0.90 when participants' latent traits were between -4 and 3, covering majority of the subjects. The SACQ-CAT administered an average of fewer than 10 items to participants compared to 67 items on the original scale. The correlation coefficient between latency estimated by the SACQ-CAT and the SACQ is greater than .85, whereas the correlation coefficient with the Symptom Checklist 90 (SCL-90) scores ranges from -.33 to -.55 (p < .001). The SACQ-CAT largely reduced the number of items administered to the participants and avoided losing measurement precision.

Efficient and precise Ultra-QuickDASH scale measuring lymphedema impact developed using computerized adaptive testing.

PURPOSE: This study aimed to evaluate and improve the accuracy and efficiency of the QuickDASH for use in assessment of limb function in patients with upper extremity lymphedema using modern psychometric techniques. METHOD: We conducted confirmative factor analysis (CFA) and Mokken analysis to examine the assumption of unidimensionality for IRT model on data from 285 patients who completed the QuickDASH, and then fit the data to Samejima's graded response model (GRM) and assessed the assumption of local independence of items and calibrated the item responses for CAT simulation. RESULTS: Initial CFA and Mokken analyses demonstrated good scalability of items and unidimensionality. However, the local independence of items assumption was violated between items 9 (severity of pain) and 11 (sleeping difficulty due to pain) (Yen's Q3 = 0.46) and disordered thresholds were evident for item 5 (cutting food). After addressing these breaches of assumptions, the re-analyzed GRM with the remaining 10 items achieved an improved fit. Simulation of CAT administration demonstrated a high correlation between scores on the CAT and the QuickDash (r = 0.98). Items 2 (doing heavy chores) and 8 (limiting work or daily activities) were the most frequently used. The correlation among factor scores derived from the QuickDASH version with 11 items and the Ultra-QuickDASH version with items 2 and 8 was as high as 0.91. CONCLUSION: By administering just these two best performing QuickDash items we can obtain estimates that are very similar to those obtained from the full-length QuickDash without the need for CAT technology.

Identifying the content for an item bank and computerized adaptive testing system to measure the impact of age-related macular degeneration on health-related quality of life.

PURPOSE: We are developing an age-related macular degeneration (AMD) health-related quality of life (HRQoL) item bank, applicable to Western and Asian populations. We report primarily on content generation and refinement, but also compare the HRQoL issues reported in our study with Western studies and current AMD-HRQoL questionnaires. METHODS: In this cross-sectional, qualitative study of AMD patients attending the Singapore National Eye Centre (May-December 2019), items/domains were generated from: (1) AMD-specific questionnaires; (2) published articles; (3) focus groups/semi-structured interviews with AMD patients (n = 27); and (4) written feedback from retinal experts. Following thematic analysis, items were systematically refined to a minimally representative set and pre-tested using cognitive interviews with 16 AMD patients. RESULTS: Of the 27 patients (mean ± standard deviation age 67.9 ± 7.0; 59.2% male), 18 (66.7%), two (7.4%), and seven (25.9%) had no, early-intermediate, and late/advanced AMD (better eye), respectively. Whilst some HRQoL issues, e.g. activity limitation, mobility, lighting, and concerns were similarly reported by Western patients and covered by other questionnaires, others like anxiety about intravitreal injections, work tasks, and financial dependency were novel. Overall, 462 items within seven independent HRQoL domains were identified: Activity limitation, Lighting, Mobility, Emotional, Concerns, AMD management, and Work. Following item refinement, items were reduced to 219, with 31 items undergoing amendment. CONCLUSION: Our 7-domain, 219-item AMD-specific HRQoL instrument will undergo psychometric testing and calibration for computerized adaptive testing. The future instrument will enable users to precisely, rapidly, and comprehensively quantify the HRQoL impact of AMD and associated treatments, with item coverage relevant across several populations.

Patient-Reported Outcomes Measurement Information System Mobility Computerized Adaptive Testing Maintains High Correlation and Low Test Burden Compared With Legacy Hip-Specific Instruments in Patients Undergoing Hip Arthroscopy for Femoroacetabular Impingement.

PURPOSE: To evaluate the reliability, construct validity, and responsiveness of the lower extremity-specific Patient-Reported Outcomes Measurement Information System (PROMIS) Mobility (MO) bank in patients who underwent hip arthroscopic surgery for femoroacetabular impingement. METHODS: Patients who underwent primary hip arthroscopic surgery at a large academic musculoskeletal specialty center between November 2019 and November 2020 completed the following baseline and 6-month measures: PROMIS MO, PROMIS Pain Interference (PI), PROMIS Physical Function (PF), modified Harris Hip Score, International Hip Outcome Tool 33, visual analog scale, and Single Assessment Numeric Evaluation. Construct validity was evaluated using Spearman correlation coefficients. The number of questions until completion was recorded as a marker of test burden. The percentage of patients scoring at the extreme high (ceiling) or low (floor) for each measure was recorded to measure inclusivity. Responsiveness was tested by comparing differences between baseline and 6-month measures, controlling for age and sex, using generalized estimating equations. Magnitudes of responsiveness were assessed through the effect size (Cohen d). RESULTS: In this study, 660 patients (50% female patients) aged 32 ± 14 years were evaluated. PROMIS MO showed a strong correlation with PROMIS PF (r = 0.84, P < .001), the International Hip Outcome Tool 33 (r = 0.73, P < .001), PROMIS PI (r = -0.76, P < .001), and the modified Harris Hip Score (r = 0.73, P < .001). Neither PROMIS MO, PROMIS PI, nor PROMIS PF met the conventional criteria for floor or ceiling effects (≥15%). The mean number of questions answered (± standard deviation) was 4.7 ± 2.1 for PROMIS MO, 4.1 ± 0.6 for PROMIS PI, and 4.1 ± 0.6 for PROMIS PF. From baseline to 6 months, the PROMIS and legacy measures exhibited significant responsiveness (P < .05), with similar effect sizes between the patient-reported outcome measures. CONCLUSIONS: This longitudinal study reveals that in patients undergoing hip arthroscopy, PROMIS MO computerized adaptive testing maintains high correlation with legacy hip-specific instruments, significant responsiveness to change, and low test burden compared with legacy measures, with no ceiling or floor effects at 6-month postoperative follow-up. LEVEL OF EVIDENCE: Level IV, retrospective case series.

Construct Validity of Pediatric PROMIS Computerized Adaptive Testing Measures in Children With Adolescent Idiopathic Scoliosis.

BACKGROUND: The use of patient-reported outcome measures, especially Patient-Reported Outcomes Measurement Information System (PROMIS) measures, has increased in recent years. Given this growth, it is imperative to ensure that the measures being used are validated for the intended population(s)/disease(s). Our objective was to assess the construct validity of 8 PROMIS computer adaptive testing (CAT) measures among children with adolescent idiopathic scoliosis (AIS). METHODS: We prospectively enrolled 200 children (aged 10 to 17 y) with AIS, who completed 8 PROMIS CATs (Anxiety, Depressive Symptoms, Mobility, Pain Behavior, Pain Interference, Peer Relationships, Physical Activity, Physical Stress Experiences) and the Scoliosis Research Society-22r questionnaire (SRS-22r) electronically. Treatment categories were observation, bracing, indicated for surgery, or postoperative from posterior spinal fusion. Construct validity was evaluated using known group analysis and convergent and discriminant validity analyses. Analysis of variance was used to identify differences in PROMIS T -scores by treatment category (known groups). The Spearman rank correlation coefficient ( rs ) was calculated between corresponding PROMIS and SRS-22r domains (convergent) and between unrelated PROMIS domains (discriminant). Floor/ceiling effects were calculated. RESULTS: Among treatment categories, significant differences were found in PROMIS Mobility, Pain Behavior, Pain Interference, and Physical Stress Experiences and in all SRS-22r domains ( P <0.05) except Mental Health ( P =0.15). SRS-22r Pain was strongly correlated with PROMIS Pain Interference ( rs =-0.72) and Pain Behavior ( rs =-0.71) and moderately correlated with Physical Stress Experiences ( rs =-0.57). SRS-22r Mental Health was strongly correlated with PROMIS Depressive Symptoms ( rs =-0.72) and moderately correlated with Anxiety ( rs =-0.62). SRS-22r Function was moderately correlated with PROMIS Mobility ( rs =0.64) and weakly correlated with Physical Activity ( rs =0.34). SRS-22r Self-Image was weakly correlated with PROMIS Peer Relationships ( rs =0.33). All unrelated PROMIS CATs were weakly correlated (| rs |<0.40). PROMIS Anxiety, Mobility, Pain Behavior, and Pain Interference and SRS-22r Function, Pain, and Satisfaction displayed ceiling effects. CONCLUSIONS: Evidence supports the construct validity of 6 PROMIS CATs in evaluating AIS patients. Ceiling effects should be considered when using specific PROMIS CATs. LEVEL OF EVIDENCE: Level II, prognostic.

Time to Completion of Pediatric PROMIS Computerized Adaptive Testing Measures and the SRS-22r in an Adolescent Idiopathic Scoliosis Population.

BACKGROUND: Patient-reported outcome measures are useful tools to quantify patients' pre-treatment and post-treatment symptoms. Historically used "legacy measures", such as the Scoliosis Research Society-22 revised questionnaire (SRS-22r), are often disease-specific and can be time-intensive. Recently developed Patient-Reported Outcomes Measurement Information System (PROMIS) computerized adaptive testing (CAT) measures may reduce administrative burdens and permit more efficient outcome collection within clinic workflows. In an era of medicine where payments are becoming tied to outcomes, we sought to assess the time to completion (TTC) of 8 pediatric PROMIS CAT measures and the SRS-22r in adolescents with idiopathic scoliosis. MATERIALS AND METHODS: Patients presenting to a large, urban tertiary referral hospital were prospectively enrolled into the study. Subjects were first-time survey respondents in various phases and types of treatment for adolescent idiopathic scoliosis. In total, 200 patients ranging from 10 to 17 years old completed 8 Pediatric PROMIS CATs and the SRS-22r. PROMIS CATs administered include Physical Activity, Mobility, Anxiety, Depressive symptoms, Peer Relationships, Physical Stress Experiences, Pain Behavior and Pain Interference. TTC was calculated using start and stop timestamps in the REDCap software. RESULTS: The mean (±SD) TTC for each PROMIS CAT was 1.1 (±0.9) minutes with physical activity, mobility, anxiety, depressive symptoms, peer relationships, physical stress experiences, pain behavior, and pain interference taking 1.2, 1.4, 1.0, 0.9, 1.2, 1.0, 1.0, and 1.2 minutes on average to complete, respectively. Mean TTC for the SRS-22r was 5.2 (±3.0) minutes. CONCLUSIONS: In this pediatric orthopaedic cohort, completion of 8 PROMIS CATs demonstrated minimal test-taker burden and time required for completion. These findings support rapid and easily integrable PROMIS CATs in clinical practice to aid in increased delivery of efficient, patient-centered care. LEVEL OF EVIDENCE: III, cross-sectional study.

A Brief Measure of Life Participation for People with COPD: Validation of the Computer Adaptive Test Version of the Late Life Disability Instrument.

Computer-adaptive tests use respondents' answers to previous questions to select the subsequent questions. They are gaining popularity for their increased measurement precision and decreased administration time compared to static questionnaires. The purpose of this study was to estimate the test-retest reliability and construct validity of the computer-adaptive test version of a participation measure, the Late Life Disability Instrument (LLDI-CAT) for people with COPD and to compare scores and administration time with those of the static LLDI. Among 76 older adults with COPD, scores on the LLDI-CAT were compared to scores on measures of related constructs, between groups based on symptom severity, prognosis and frailty phenotype, and to scores on the static LLDI. A subsample of 28 people completed the LLDI-CAT a second time within one week of the initial administration for test-retest reliability. The LLDI-CAT had very good test-retest reliability (ICC2,1 0.88; SEM 2.74 points), fair correlations with physical function (r = 0.37-0.50), anxiety (r=-0.42), and depression (r=-0.50), fair to moderately-strong correlations with quality of life (r = 0.48-0.63), and strong correlation with the static LLDI limitation domain (r = 0.80). The LLDI-CAT scores differed between people with different symptom severity, prognosis and frailty phenotype (p ≤ 0.004). The mean administration time for the LLDI-CAT was 3.3 (1.5) minutes, less than that of the static LLDI at 6.3 (2.8) minutes (p < 0.001). The LLDI-CAT demonstrates evidence of test-retest reliability and construct validity, and correlates well with the limitation domain of the static LLDI for people with COPD. The LLDI-CAT can be used to assess participation for this population.

Psychometric and Machine Learning Approaches to Reduce the Length of Scales.

Brief measures are important in psychology research because they reduce participant burden. Researchers can select items from longer measures either to build a short-form or to administer items conditional on a participant's previous responses. Researchers who carry out these item selection strategies either focus on estimating a precise score on the measure (typically carried out in a psychometric approach) or on predicting the score on the measure (possibly taking a machine learning approach). However, it is unclear how scores from the psychometric and machine learning approaches compare to each other. In this paper, the following four statistical approaches to select items are reviewed and illustrated: item response theory to build static short-forms, computerized adaptive testing, the genetic algorithm, and regression trees. Theoretical strengths and weaknesses between these four statistical approaches are discussed, and the overlap between the areas of psychometrics and machine learning is considered.

Establishing language and ethnic equivalence for health-related quality of life item banks and testing their efficiency via computerised adaptive testing simulations.

PURPOSE: We aimed to (1) establish linguistic and ethnic equivalence (i.e. lack of bias) for the items in the English and Chinese versions of the Singapore Health and Well Being (SHAWS) Physical Functioning (PF), Positive Mindset (PM) and Social Relationship (SR) item banks (IBs); and (2) evaluate the preliminary efficiency of these IBs using Computer Adaptive Testing (CAT) simulations. METHODS: In this cross-sectional study, 671, 670, and 672 subjects answered 55, 48 and 30 items of the PF, PM, and SR IBs, respectively. Rasch analysis was conducted to assess each IB's psychometric properties, particularly the presence of differential item functioning (DIF) for language and ethnicity. A set of performance criteria related to removing items that displayed notable DIF were employed. CAT simulations determined the mean number of items for high, moderate, and moderate-low measurement precisions (stopping rule: SEM 0.300, 0.387. 0.521, respectively). RESULTS: Half of subjects were >50 years old (40.9% PF, 42.1% PM, 41.4% SR), Chinese (50.7% PF, 51.0% PM, 50.6% SR) and female (50.0% PF. 49.4% PM, 52.8% SR) respectively. Rasch analysis revealed 4 items with DIF for the PF IB, 9 items with DIF for the PM IB and 2 items with DIF for the SR IB. In CAT simulations, the mean number of items administered was 8.5, 21.6 and 14.5 for the PF, PM and SR IBs, respectively (SEM 0.300), 5.1, 13.0, 8.0 for PF, PM and SR IBs, respectively (SEM 0.387) and 3.1, 5.3 and 4.1 for PF, PM and SR IBs, respectively (SEM 0.521). CONCLUSION: The PF, PM and SR IBs to measure health-related quality of life revealed minimal DIF for language and ethnicity after remedial efforts. CAT simulations demonstrated that these IBs were efficient, especially when the stopping rule was set at moderate precision, and support the implementation of the SHAWS IBs into routine clinical care.

The Effect of Computerized Adaptive Testing on Motivation and Anxiety: A Systematic Review and Meta-Analysis.

Although many studies have been carried out on the psychometric aspects of computerized adaptive testing (CAT), its psychological aspects are less researched. Early studies claimed that CAT can be more motivating and induce less anxiety than traditional fixed-item tests (FIT). The purpose of this systematic review and meta-analysis was to gain a comprehensive understanding of the effects of CAT on motivation and anxiety in comparison to traditional fixed-item testing. Seven databases were examined. Articles were eligible if they employed an empirical study containing a direct comparison between CAT and FIT. Meta-analytical results showed no overall effect of test type on anxiety and motivation when comparing CAT with FIT (k = 11, g+ = 0.06, p = .28). However, easier CAT had positive effect compared with FIT (k = 2, g+ = .22, p < .001). Certain modifications in CAT administration can provide positive psychological effects for test-takers.

Comparison of Patient-Reported Outcomes Measurement Information System Computerized Adaptive Testing Versus Fixed Short Forms in Juvenile Myositis.

OBJECTIVE: Patient-Reported Outcomes Measurement Information System (PROMIS) measures can be administered via computerized adaptive testing (CAT) or fixed short forms (FSFs), but the empirical benefits of CAT versus FSFs are unknown in juvenile myositis (JM). The present study was undertaken to assess whether PROMIS CAT is feasible, precise, correlated with FSFs, and less prone to respondent burden and floor/ceiling effects than FSFs in JM. METHODS: Patients 8-17 years of age (self-report and parent proxy) and parents of patients 5-7 years of age (only parent proxy) completed PROMIS fatigue, pain interference, upper extremity function, mobility, anxiety, and depressive symptoms measures. Pearson correlations, paired t-tests, and Cohen's d were calculated between PROMIS CAT and FSFs. McNemar's test assessed floor/ceiling effects between CAT and FSFs. Precision and respondent burden were examined across the T score range. RESULTS: Data from 67 patient-parent dyads were analyzed. CAT and FSF mean scores did not significantly differ except in parent proxy anxiety and fatigue (effect size 0.23 and 0.19, respectively). CAT had less pronounced floor/ceiling effects at the less symptomatic extreme in all domains except self-report anxiety. Increased item burden and higher SEs were seen in less symptomatic scorers for CAT. Modified stopping rules limiting CAT item administration did not decrease precision. CONCLUSION: PROMIS CAT appears to be feasible and correlated with FSFs. CAT had less pronounced floor/ceiling effects, allowing detection of individual differences in less symptomatic patients. Modified stopping rules for CAT may decrease respondent burden. CAT can be considered for long-term follow-up of JM patients.

Feasibility of PROMIS using computerized adaptive testing during inpatient rehabilitation.

BACKGROUND: There has been an increased significance on patient-reported outcomes in clinical settings. We aimed to evaluate the feasibility of administering patient-reported outcome measures by computerized adaptive testing (CAT) using a tablet computer with rehabilitation inpatients, assess workload demands on staff, and estimate the extent to which rehabilitation inpatients have elevated T-scores on six Patient Reported Outcomes Measurement Information System® (PROMIS®) measures. METHODS: Patients (N = 108) with stroke, spinal cord injury, traumatic brain injury, and other neurological disorders participated in this study. PROMIS computerized adaptive tests (CAT) were administered via a web-based platform. Summary scores were calculated for six measures: Pain Interference, Sleep Disruption, Anxiety, Depression, Illness Impact Positive, and Illness Impact Negative. We calculated the percent of patients with T-scores equivalent to 2 standard deviations or greater above the mean. RESULTS: During the first phase, we collected data from 19 of 49 patients; of the remainder, 61% were not available or had cognitive or expressive language impairments. In the second phase of the study, 40 of 59 patients participated to complete the assessment. The mean PROMIS T-scores were in the low 50 s, indicating an average symptom level, but 19-31% of patients had elevated T-scores where the patients needed clinical action. CONCLUSIONS: The study demonstrated that PROMIS assessment using a CAT administration during an inpatient rehabilitation setting is feasible with the presence of a research staff member to complete PROMIS assessment.

Developing Computerized Adaptive Testing for a National Health Professionals Exam: An Attempt from Psychometric Simulations.

Introduction: The accurate assessment of health professionals' competence is critical for ensuring public health safety and quality of care. Computerized Adaptive Testing (CAT) based on the Item Response Theory (IRT) has the potential to improve measurement accuracy and reduce respondent burden. In this study, we conducted psychometric simulations to develop a CAT for evaluating the candidates' competence of health professionals. Methods: The initial CAT item bank was sourced from the Standardized Competence Test for Clinical Medicine Undergraduates (SCTCMU), a nationwide summative test in China, consisting of 300 multiple-choice items. We randomly selected response data from 2000 Chinese clinical medicine undergraduates for analysis. Two types of analyses were performed: first, evaluating the psychometric properties of all items to meet the requirements of CAT; and second, conducting multiple CAT simulations using both simulated and real response data. Results: The final CAT item bank consisted of 121 items, for which item parameters were calculated using a two-parameter logistic model (2PLM). The CAT simulations, based on both simulated and real data, revealed sufficient marginal reliability (coefficient of marginal reliability above 0.750) and criterion-related validity (Pearson's correlations between CAT scores and aggregate scores of the SCTCMU exceeding 0.850). Discussion: In national-level medical education assessment, there is an increasing need for concise yet valid evaluations of candidates' competence of health professionals. The CAT developed in this study demonstrated satisfactory reliability and validity, offering a more efficient assessment of candidates' competence of health professionals. The psychometric properties of the CAT could lead to shorter test durations, reduced information loss, and a decreased testing burden for participants.