International test commission guidelines for test adaptation: a criterion checklist / Directrices de la Comisión Internacional de Test para la adaptación de test: un listado de verificación

BACKGROUND: To improve the quality of test translation and adaptation, and hence the comparability of scores across cultures, the International Test Commission (ITC) proposed a number of guidelines for the adaptation process. Although these guidelines are well-known, they are not implemented as often as they should be. One possible reason for this is the broad scope of the guidelines, which makes them difficult to apply in practice. The goal of this study was therefore to draw up an evaluative criterion checklist that would help test adapters to implement the ITC recommendations and which would serve as a model for assessing the quality of test adaptations. METHOD: Each ITC guideline was operationalized through a number of criteria. For each criterion, acceptable and excellent levels of accomplishment were proposed. The initial checklist was then reviewed by a panel of 12 experts in testing and test adaptation. The resulting checklist was applied to two different tests by two pairs of independent reviewers. RESULTS: The final evaluative checklist consisted of 29 criteria covering all phases of test adaptation: planning, development, confirmation, administration, score interpretation, and documentation. CONCLUSIONS: We believe that the proposed evaluative checklist will help to improve the quality of test adaptation

ANTECEDENTES: la Comisión Internacional de Test (ITC) propuso una serie de directrices para mejorar la calidad de la traducción y adaptación de los test y, consecuentemente, mejorar la comparabilidad de las puntuaciones a través de distintas culturas. Aunque estas directrices son bien conocidas, no se aplican tan frecuentemente como sería deseable. Este trabajo propone un listado de verificación de los criterios de cumplimiento asociados a las directrices de la ITC, que faciliten su implementación y sirvan de modelo para evaluar la calidad de las adaptaciones realizadas. MÉTODO: cada directriz de la ITC se operacionalizó a través de distintos criterios. Para cada criterio se propusieron niveles de aceptabilidad y excelencia. El listado inicial propuesto fue revisado por un panel de 12 expertos en el área de medición y adaptación de test. La versión resultante fue aplicada a dos test por dos pares de revisores independientes. RESULTADOS: el listado final de verificación del grado de cumplimiento de las directrices consistió en 29 criterios que cubren todas las fases del proceso de adaptación de un test: planificación, desarrollo, confirmación, administración, interpretación de las puntuaciones y documentación. CONCLUSIONES: se espera que el listado propuesto ayude a mejorar la calidad de los test adaptados

International Test Commission guidelines for test adaptation: A criterion checklist.

BACKGROUND: To improve the quality of test translation and adaptation, and hence the comparability of scores across cultures, the International Test Commission (ITC) proposed a number of guidelines for the adaptation process. Although these guidelines are well-known, they are not implemented as often as they should be. One possible reason for this is the broad scope of the guidelines, which makes them difficult to apply in practice. The goal of this study was therefore to draw up an evaluative criterion checklist that would help test adapters to implement the ITC recommendations and which would serve as a model for assessing the quality of test adaptations. METHOD: Each ITC guideline was operationalized through a number of criteria. For each criterion, acceptable and excellent levels of accomplishment were proposed. The initial checklist was then reviewed by a panel of 12 experts in testing and test adaptation. The resulting checklist was applied to two different tests by two pairs of independent reviewers. RESULTS: The final evaluative checklist consisted of 29 criteria covering all phases of test adaptation: planning, development, confirmation, administration, score interpretation, and documentation. CONCLUSIONS: We believe that the proposed evaluative checklist will help to improve the quality of test adaptation.

Profiles of Mathematics Anxiety Among 15-Year-Old Students: A Cross-Cultural Study Using Multi-Group Latent Profile Analysis.

Using PISA 2012 data, the present study explored profiles of mathematics anxiety (MA) among 15-year old students from Finland, Korea, and the United States to determine the similarities and differences of MA across the three national samples by applying a multi-group latent profile analysis (LPA). The major findings were that (a) three MA profiles were found in all three national samples, i.e., Low MA, Mid MA, and High MA profile, and (b) the percentages of students classified into each of the three MA profiles differed across the Finnish, Korean, and American samples, with United States having the highest prevalence of High MA, and Finland the lowest. Multi-group LPA also provided clear and useful latent profile separation. The High MA profile demonstrated significant poorer mathematics performance and lower mathematics interest, self-efficacy, and self-concept than the Mid and Low MA profiles. Same differences appeared between the Mid and Low MA profiles. The implications of the findings seem clear: (1) it is possible that there is some relative level of universality in MA among 15-year old students which is independent of cultural context; and (2) multi-group LPA could be a useful analytic tool for research on the study of classification and cultural differences of MA.

Practical Consequences of Item Response Theory Model Misfit in the Context of Test Equating with Mixed-Format Test Data.

In item response theory (IRT) models, assessing model-data fit is an essential step in IRT calibration. While no general agreement has ever been reached on the best methods or approaches to use for detecting misfit, perhaps the more important comment based upon the research findings is that rarely does the research evaluate IRT misfit by focusing on the practical consequences of misfit. The study investigated the practical consequences of IRT model misfit in examining the equating performance and the classification of examinees into performance categories in a simulation study that mimics a typical large-scale statewide assessment program with mixed-format test data. The simulation study was implemented by varying three factors, including choice of IRT model, amount of growth/change of examinees' abilities between two adjacent administration years, and choice of IRT scaling methods. Findings indicated that the extent of significant consequences of model misfit varied over the choice of model and IRT scaling methods. In comparison with mean/sigma (MS) and Stocking and Lord characteristic curve (SL) methods, separate calibration with linking and fixed common item parameter (FCIP) procedure was more sensitive to model misfit and more robust against various amounts of ability shifts between two adjacent administrations regardless of model fit. SL was generally the least sensitive to model misfit in recovering equating conversion and MS was the least robust against ability shifts in recovering the equating conversion when a substantial degree of misfit was present. The key messages from the study are that practical ways are available to study model fit, and, model fit or misfit can have consequences that should be considered when choosing an IRT model. Not only does the study address the consequences of IRT model misfit, but also it is our hope to help researchers and practitioners find practical ways to study model fit and to investigate the validity of particular IRT models for achieving a specified purpose, to assure that the successful use of the IRT models are realized, and to improve the applications of IRT models with educational and psychological test data.

Detecting Item Preknowledge Using a Predictive Checking Method.

Repeatedly using items in high-stake testing programs provides a chance for test takers to have knowledge of particular items in advance of test administrations. A predictive checking method is proposed to detect whether a person uses preknowledge on repeatedly used items (i.e., possibly compromised items) by using information from secure items that have zero or very low exposure rates. Responses on the secure items are first used to estimate a person's proficiency distribution, and then the corresponding predictive distribution for the person's responses on the possibly compromised items is constructed. The use of preknowledge is identified by comparing the observed responses to the predictive distribution. Different estimation methods for obtaining a person's proficiency distribution and different choices of test statistic in predictive checking are considered. A simulation study was conducted to evaluate the empirical Type I error and power rate of the proposed method. The simulation results suggested that the Type I error of this method is well controlled, and this method is effective in detecting preknowledge when a large proportion of items are compromised even with a short secure section. An empirical example is also presented to demonstrate its practical use.

Comparative Analyses of MIRT Models and Software (BMIRT and flexMIRT).

Application of MIRT modeling procedures is dependent on the quality of parameter estimates provided by the estimation software and techniques used. This study investigated model parameter recovery of two popular MIRT packages, BMIRT and flexMIRT, under some common measurement conditions. These packages were specifically selected to investigate the model parameter recovery of three item parameter estimation techniques, namely, Bock-Aitkin EM (BA-EM), Markov chain Monte Carlo (MCMC), and Metropolis-Hastings Robbins-Monro (MH-RM) algorithms. The results demonstrated that all estimation techniques had similar root mean square error values when larger sample size and higher test length were used. Depending on the number of dimensions, sample size, and test length, each estimation technique exhibited some strengths and weaknesses. Overall, the BA-EM technique was found to have shorter estimation time with all test specifications.

The Effect of Rating Unfamiliar Items on Angoff Passing Scores.

The Angoff standard setting method relies on content experts to review exam items and make judgments about the performance of the minimally proficient examinee. Unfortunately, at times content experts may have gaps in their understanding of specific exam content. These gaps are particularly likely to occur when the content domain is broad and/or highly technical, or when non-expert stakeholders are included in a standard setting panel (e.g., parents, administrators, or union representatives). When judges lack expertise regarding specific exam content, the ratings associated with those items may be bias. This study attempts to illustrate the impact of rating unfamiliar items on Angoff passing scores. The study presents a comparison of Angoff ratings for typical items with those identified by judges as containing unfamiliar content. The results indicate that judges tend to perceive unfamiliar items as being artificially difficult resulting in systematically lower Angoff ratings. The results suggest that when judges are forced to rate unfamiliar items, the validity of the resulting classification decision may be jeopardized.

An application of item response theory to psychological test development

Abstract Item response theory (IRT) has become a popular methodological framework for modeling response data from assessments in education and health; however, its use is not widespread among psychologists. This paper aims to provide a didactic application of IRT and to highlight some of these advantages for psychological test development. IRT was applied to two scales (a positive and a negative affect scale) of a self-report test. Respondents were 853 university students (57 % women) between the ages of 17 and 35 and who answered the scales. IRT analyses revealed that the positive affect scale has items with moderate discrimination and are measuring respondents below the average score more effectively. The negative affect scale also presented items with moderate discrimination and are evaluating respondents across the trait continuum; however, with much less precision. Some features of IRT are used to show how such results can improve the measurement of the scales. The authors illustrate and emphasize how knowledge of the features of IRT may allow test makers to refine and increase the validity and reliability of other psychological measures. (AU)

Tracking functional status across the spinal cord injury lifespan: linking pediatric and adult patient-reported outcome scores.

OBJECTIVE: To use item response theory (IRT) methods to link scores from 2 recently developed contemporary functional outcome measures, the adult Spinal Cord Injury-Functional Index (SCI-FI) and the Pedi SCI (both the parent version and the child version). DESIGN: Secondary data analysis of the physical functioning items of the adult SCI-FI and the Pedi SCI instruments. We used a nonequivalent group design with items common to both instruments and the Stocking-Lord method for the linking. Linking was conducted so that the adult SCI-FI and Pedi SCI scaled scores could be compared. SETTING: Community. PARTICIPANTS: This study included a total sample of 1558 participants. Pedi SCI items were administered to a sample of children (n=381) with SCI aged 8 to 21 years, and of parents/caregivers (n=322) of children with SCI aged 4 to 21 years. Adult SCI-FI items were administered to a sample of adults (n=855) with SCI aged 18 to 92 years. INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: Five scales common to both instruments were included in the analysis: Wheelchair, Daily Routine/Self-care, Daily Routine/Fine Motor, Ambulation, and General Mobility functioning. RESULTS: Confirmatory factor analysis and exploratory factor analysis results indicated that the 5 scales are unidimensional. A graded response model was used to calibrate the items. Misfitting items were identified and removed from the item banks. Items that function differently between the adult and child samples (ie, exhibit differential item functioning) were identified and removed from the common items used for linking. Domain scores from the Pedi SCI instruments were transformed onto the adult SCI-FI metric. CONCLUSIONS: This IRT linking allowed estimation of adult SCI-FI scale scores based on Pedi SCI scale scores and vice versa; therefore, it provides clinicians with a means of tracking long-term functional data for children with an SCI across their entire lifespan.

Directrices para la traducción y adaptación de los tests: segunda edición / International Test Commission Guidelines for test translation and adaptation

Antecedentes: en los últimos años la adaptación de los tests de unas culturas a otras se ha incrementado en todos los ámbitos evaluativos. Vivimos en un entorno cada vez más multicultural y multilingüe en el que los tests se utilizan como apoyo en la toma de decisiones. El objetivo de este trabajo es presentar la segunda edición de las directrices de la Comisión Internacional de Tests (ITC) para la adaptación de los tests de unas culturas a otras. Método: un grupo de seis expertos internacionales revisaron las directrices originales propuestas por la Comisión Internacional de Tests, teniendo en cuenta los avances habidos en el campo desde su formulación inicial. Resultados: la nueva edición está compuesta por veinte directrices agrupadas en seis apartados: directrices previas, desarrollo del test, confirmación, aplicación, puntuación e interpretación y documentación. Se analizan los diferentes apartados, y se estudian las posibles fuentes de error que pueden influir en el proceso de traducción y adaptación de los tests. Conclusiones: se proponen veinte directrices para guiar la traducción y adaptación de los tests de unas culturas a otras. Se discuten las perspectivas futuras de las directrices en relación con los nuevos desarrollos en el ámbito de la evaluación psicológica y educativa (AU)

Second edition. Background: Adapting tests across culture is a common practice that has increased in all evaluation areas in recent years. We live in an increasingly multicultural and multilingual world in which the tests are used to support decision-making in the educational, clinical, organizational and other areas, so the adaptation of tests becomes a necessity. The main goal of this paper is to present the second edition of the guidelines of the International Test Commission (ITC) for adapting tests across cultures. Method: A task force of six international experts reviewed the original guidelines proposed by the International Test Commission, taking into account the advances and developments of the field. Results: As a result of the revision this new edition consists of twenty guidelines grouped into six sections: Precondition, test development, confirmation, administration, score scales and interpretation, and document. The different sections are reviewed, and the possible sources of error influencing the tests translation and adaptation analyzed. Conclusions: Twenty guidelines are proposed for translating and adapting tests across cultures. Finally we discuss the future perspectives of the guidelines in relation to the new developments in the field of psychological and educational assessment (AU)

[International Test Commission Guidelines for test translation and adaptation: second edition]. / Directrices para la traducción y adaptación de los tests: segunda edición.

BACKGROUND: Adapting tests across cultures is a common practice that has increased in all evaluation areas in recent years. We live in an increasingly multicultural and multilingual world in which the tests are used to support decision-making in the educational, clinical, organizational and other areas, so the adaptation of tests becomes a necessity. The main goal of this paper is to present the second edition of the guidelines of the International Test Commission (ITC) for adapting tests across cultures. METHOD: A task force of six international experts reviewed the original guidelines proposed by the International Test Commission, taking into account the advances and developments of the field. RESULTS: As a result of the revision this new edition consists of twenty guidelines grouped into six sections: Precondition, test development, confirmation, administration, score scales and interpretation, and document. The different sections are reviewed, and the possible sources of error influencing the tests translation and adaptation analyzed. CONCLUSIONS: Twenty guidelines are proposed for translating and adapting tests across cultures. Finally we discuss the future perspectives of the guidelines in relation to the new developments in the field of psychological and educational assessment.

A functional difficulty and functional pain instrument for hip and knee osteoarthritis.

INTRODUCTION: The objectives of this study were to develop a functional outcome instrument for hip and knee osteoarthritis research (OA-FUNCTION-CAT) using item response theory (IRT) and computer adaptive test (CAT) methods and to assess its psychometric performance compared to the current standard in the field. METHODS: We conducted an extensive literature review, focus groups, and cognitive testing to guide the construction of an item bank consisting of 125 functional activities commonly affected by hip and knee osteoarthritis. We recruited a convenience sample of 328 adults with confirmed hip and/or knee osteoarthritis. Subjects reported their degree of functional difficulty and functional pain in performing each activity in the item bank and completed the Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC). Confirmatory factor analyses were conducted to assess scale uni-dimensionality, and IRT methods were used to calibrate the items and examine the fit of the data. We assessed the performance of OA-FUNCTION-CATs of different lengths relative to the full item bank and WOMAC using CAT simulation analyses. RESULTS: Confirmatory factor analyses revealed distinct functional difficulty and functional pain domains. Descriptive statistics for scores from 5-, 10-, and 15-item CATs were similar to those for the full item bank. The 10-item OA-FUNCTION-CAT scales demonstrated a high degree of accuracy compared with the item bank (r = 0.96 and 0.89, respectively). Compared to the WOMAC, both scales covered a broader score range and demonstrated a higher degree of precision at the ceiling and reliability across the range of scores. CONCLUSIONS: The OA-FUNCTION-CAT provided superior reliability throughout the score range and improved breadth and precision at the ceiling compared with the WOMAC. Further research is needed to assess whether these improvements carry over into superior ability to measure change.

Development of a parent-report computer-adaptive test to assess physical functioning in children with cerebral palsy I: lower-extremity and mobility skills.

The objective of this project was to develop computer-adaptive tests (CATs) using parent reports of physical function in children and adolescents with cerebral palsy (CP). The specific aims of this study were to (1) examine the psychometric properties of an item bank of lower-extremity and mobility skills for children with CP; (2) evaluate a CAT using this item bank; (3) examine the concurrent validity of the CAT with the Pediatric Outcomes Data Collection Instrument (PODCI) and the Functional Assessment Questionnaire (FAQ); and (4) establish the discriminant validity of simulated CATs with Gross Motor Function Classification System (GMFCS) levels and CP type (diplegia, hemiplegia, or quadriplegia). Parents (n=190) of children and adolescents with spastic diplegic (48%), hemiplegic (22%), or quadriplegic (30%) CP consisting of 108 males and 82 females with a mean age of 10 years 7 months (SD 4y 1mo, range 2-21y) and in GMFCS levels I to V participated in item pool calibration and completed the PODCI and FAQ. Confirmatory factor analyses supported a unidimensional model for the 45 basic lower-extremity and mobility items. Simulated CATs of 5, 10, and 15 items demonstrated excellent accuracy (intraclass correlation coefficients [ICCs] >0.91) with the full item bank and had high correlations with PODCI transfers and mobility (ICC = 0.86) and FAQ scores (ICC = 0.77). All CATs discriminated among GMFCS levels and CP type. The lower-extremity and mobility skills item bank and simulated CATs demonstrated excellent performance over a wide span of ages and severity levels.

Evaluation of an item bank for a computerized adaptive test of activity in children with cerebral palsy.

BACKGROUND: Contemporary clinical assessments of activity are needed across the age span for children with cerebral palsy (CP). Computerized adaptive testing (CAT) has the potential to efficiently administer items for children across wide age spans and functional levels. OBJECTIVE: The objective of this study was to examine the psychometric properties of a new item bank and simulated computerized adaptive test to assess activity level abilities in children with CP. DESIGN: This was a cross-sectional item calibration study. METHODS: The convenience sample consisted of 308 children and youth with CP, aged 2 to 20 years (X=10.7, SD=4.0), recruited from 4 pediatric hospitals. We collected parent-report data on an initial set of 45 activity items. Using an Item Response Theory (IRT) approach, we compared estimated scores from the activity item bank with concurrent instruments, examined discriminate validity, and developed computer simulations of a CAT algorithm with multiple stop rules to evaluate scale coverage, score agreement with CAT algorithms, and discriminant and concurrent validity. RESULTS: Confirmatory factor analysis supported scale unidimensionality, local item dependence, and invariance. Scores from the computer simulations of the prototype CATs with varying stop rules were consistent with scores from the full item bank (r=.93-.98). The activity summary scores discriminated across levels of upper-extremity and gross motor severity and were correlated with the Pediatric OUTCOMES: Data Collection Instrument (PODCI) physical function and sports subscale (r=.86), the Functional Independence Measure for Children (Wee-FIM) (r=.79), and the Pediatric Quality of Life Inventory-Cerebral Palsy version (r=.74). LIMITATIONS: The sample size was small for such IRT item banks and CAT development studies. Another limitation was oversampling of children with CP at higher functioning levels. CONCLUSIONS: The new activity item bank appears to have promise for use in a CAT application for the assessment of activity abilities in children with CP across a wide age range and different levels of motor severity.

Development of a parent-report computer-adaptive test to assess physical functioning in children with cerebral palsy II: upper-extremity skills.

The specific aims of this study were to (1) examine the psychometric properties (unidimensionality, differential item functioning, scale coverage) of an item bank of upper-extremity skills for children and adolescents with cerebral palsy (CP); (2) evaluate a simulated computer-adaptive test (CAT) using this item bank; (3) examine the concurrent validity of the CAT with the Pediatric Outcomes Data Collection Instrument (PODCI) upper-extremity core scale; and (4) determine the discriminant validity of the simulated CAT with Manual Ability Classification System (MACS) levels and CP type (i.e. diplegia, hemiplegia, or quadriplegia). Parents (n=180) of children and adolescents with CP (spastic diplegia 49%, hemiplegia 22%, or quadriplegia 28%) consisting of 102 males and 78 females with a mean age of 10 years 6 months (SD 4y 1mo, range 2-21y), and MACS levels I through V participated in calibration of an item pool and completed the PODCI. Confirmatory factor analyses supported a unidimensional model using 49 of the 53 upper-extremity items. Simulated CATs of 5, 10, and 15 items demonstrated excellent accuracy (intraclass correlation coefficient [ICCs] >0.93) with the full item bank, had high correlations with the PODCI upper-extremity core scale score (ICC 0.79), and discriminated among MACS levels. The simulated CATs demonstrated excellent overall content coverage over a wide age span and severity of upper-extremity involvement. The future development and refinement of CATs for parent report of physical function in children and adolescents with CP is supported by our work.

Measuring global physical health in children with cerebral palsy: illustration of a multidimensional bi-factor model and computerized adaptive testing.

PURPOSE: The purposes of this study were to apply a bi-factor model for the determination of test dimensionality and a multidimensional CAT using computer simulations of real data for the assessment of a new global physical health measure for children with cerebral palsy (CP). METHODS: Parent respondents of 306 children with cerebral palsy were recruited from four pediatric rehabilitation hospitals and outpatient clinics. We compared confirmatory factor analysis results across four models: (1) one-factor unidimensional; (2) two-factor multidimensional (MIRT); (3) bi-factor MIRT with fixed slopes; and (4) bi-factor MIRT with varied slopes. We tested whether the general and content (fatigue and pain) person score estimates could discriminate across severity and types of CP, and whether score estimates from a simulated CAT were similar to estimates based on the total item bank, and whether they correlated as expected with external measures. RESULTS: Confirmatory factor analysis suggested separate pain and fatigue sub-factors; all 37 items were retained in the analyses. From the bi-factor MIRT model with fixed slopes, the full item bank scores discriminated across levels of severity and types of CP, and compared favorably to external instruments. CAT scores based on 10- and 15-item versions accurately captured the global physical health scores. CONCLUSIONS: The bi-factor MIRT CAT application, especially the 10- and 15-item versions, yielded accurate global physical health scores that discriminated across known severity groups and types of CP, and correlated as expected with concurrent measures. The CATs have potential for collecting complex data on the physical health of children with CP in an efficient manner.

A computer-adaptive disability instrument for lower extremity osteoarthritis research demonstrated promising breadth, precision, and reliability.

OBJECTIVE: To develop and evaluate a prototype measure (OA-DISABILITY-CAT) for osteoarthritis research using item response theory (IRT) and computer-adaptive test (CAT) methodologies. STUDY DESIGN AND SETTING: We constructed an item bank consisting of 33 activities commonly affected by lower extremity (LE) osteoarthritis. A sample of 323 adults with LE osteoarthritis reported their degree of limitation in performing everyday activities, and completed the Health Assessment Questionnaire-II (HAQ-II). We used confirmatory factor analyses to assess scale unidimensionality and IRT methods to calibrate the items and examine the fit of the data. Using CAT simulation analyses, we examined the performance of OA-DISABILITY-CATs of different lengths compared with the full-item bank and the HAQ-II. RESULTS: One distinct disability domain was identified. The 10-item OA-DISABILITY-CAT demonstrated a high degree of accuracy compared with the full-item bank (r=0.99). The item bank and the HAQ-II scales covered a similar estimated scoring range. In terms of reliability, 95% of OA-DISABILITY reliability estimates were over 0.83 vs. 0.60 for the HAQ-II. Except at the highest scores, the 10-item OA-DISABILITY-CAT demonstrated superior precision to the HAQ-II. CONCLUSION: The prototype OA-DISABILITY-CAT demonstrated promising measurement properties compared with the HAQ-II, and is recommended for use in LE osteoarthritis research.

Psychometric evaluation and calibration of health-related quality of life item banks: plans for the Patient-Reported Outcomes Measurement Information System (PROMIS).

BACKGROUND: The construction and evaluation of item banks to measure unidimensional constructs of health-related quality of life (HRQOL) is a fundamental objective of the Patient-Reported Outcomes Measurement Information System (PROMIS) project. OBJECTIVES: Item banks will be used as the foundation for developing short-form instruments and enabling computerized adaptive testing. The PROMIS Steering Committee selected 5 HRQOL domains for initial focus: physical functioning, fatigue, pain, emotional distress, and social role participation. This report provides an overview of the methods used in the PROMIS item analyses and proposed calibration of item banks. ANALYSES: Analyses include evaluation of data quality (eg, logic and range checking, spread of response distribution within an item), descriptive statistics (eg, frequencies, means), item response theory model assumptions (unidimensionality, local independence, monotonicity), model fit, differential item functioning, and item calibration for banking. RECOMMENDATIONS: Summarized are key analytic issues; recommendations are provided for future evaluations of item banks in HRQOL assessment.

Computer adaptive testing improved accuracy and precision of scores over random item selection in a physical functioning item bank.

BACKGROUND AND OBJECTIVE: Measuring physical functioning (PF) within and across postacute settings is critical for monitoring outcomes of rehabilitation; however, most current instruments lack sufficient breadth and feasibility for widespread use. Computer adaptive testing (CAT), in which item selection is tailored to the individual patient, holds promise for reducing response burden, yet maintaining measurement precision. We calibrated a PF item bank via item response theory (IRT), administered items with a post hoc CAT design, and determined whether CAT would improve accuracy and precision of score estimates over random item selection. METHODS: 1,041 adults were interviewed during postacute care rehabilitation episodes in either hospital or community settings. Responses for 124 PF items were calibrated using IRT methods to create a PF item bank. We examined the accuracy and precision of CAT-based scores compared to a random selection of items. RESULTS: CAT-based scores had higher correlations with the IRT-criterion scores, especially with short tests, and resulted in narrower confidence intervals than scores based on a random selection of items; gains, as expected, were especially large for low and high performing adults. CONCLUSION: The CAT design may have important precision and efficiency advantages for point-of-care functional assessment in rehabilitation practice settings.