Empirical Evaluation of Computer-Adaptive Alternate Short Forms for the Assessment of Anomia Severity.

Purpose The purpose of this study was to verify the equivalence of 2 alternate test forms with nonoverlapping content generated by an item response theory (IRT)-based computer-adaptive test (CAT). The Philadelphia Naming Test (PNT; Roach, Schwartz, Martin, Grewal, & Brecher, 1996)was utilized as an item bank in a prospective, independent sample of persons with aphasia. Method Two alternate CAT short forms of the PNT were administered to a sample of 25 persons with aphasia who were at least 6 months postonset and received no treatment for 2 weeks before or during the study. The 1st session included administration of a 30-item PNT-CAT, and the 2nd session, conducted approximately 2 weeks later, included a variable-length PNT-CAT that excluded items administered in the 1st session and terminated when the modeled precision of the ability estimate was equal to or greater than the value obtained in the 1st session. The ability estimates were analyzed in a Bayesian framework. Results The 2 test versions correlated highly (r = .89) and obtained means and standard deviations that were not credibly different from one another. The correlation and error variance between the 2 test versions were well predicted by the IRT measurement model. Discussion The results suggest that IRT-based CAT alternate forms may be productively used in the assessment of anomia. IRT methods offer advantages for the efficient and sensitive measurement of change over time. Future work should consider the potential impact of differential item functioning due to person factors and intervention-specific effects, as well as expanding the item bank to maximize the clinical utility of the test. Supplemental Material https://doi.org/10.23641/asha.11368040.

Enhancing the Efficiency of Confrontation Naming Assessment for Aphasia Using Computer Adaptive Testing.

Purpose In this study, we investigated the agreement between the 175-item Philadelphia Naming Test (PNT; Roach, Schwartz, Martin, Grewal, & Brecher, 1996 ) and a 30-item computer adaptive PNT (PNT-CAT; Fergadiotis, Kellough, & Hula, 2015 ; Hula, Kellough, & Fergadiotis, 2015 ) created using item response theory (IRT) methods. Method The full PNT and the PNT-CAT were administered to 47 participants with aphasia in counterbalanced order. Latent trait-naming ability estimates for the 2 PNT versions were analyzed in a Bayesian framework, and the agreement between them was evaluated using correlation and measures of constant, variable, and total error. We also evaluated the extent to which individual pairwise differences were credibly greater than 0 and whether the IRT measurement model provided an adequate indication of the precision of individual score estimates. Results The agreement between the PNT and the PNT-CAT was strong, as indicated by high correlation ( r = .95, 95% CI [.92, .97]), negligible bias, and low variable and total error. The number of statistically robust pairwise score differences did not credibly exceed the Type I error rate, and the precision of individual score estimates was reasonably well predicted by the IRT model. Discussion The strong agreement between the full PNT and the PNT-CAT suggests that the latter is a suitable measurement of anomia in group studies. The relatively robust estimates of score precision also suggest that the PNT-CAT can be useful for the clinical assessment of anomia in individual cases. Finally, the IRT methods used to construct the PNT-CAT provide a framework for additional development to further reduce measurement error. Supplemental Material https://doi.org/10.23641/asha.8202176.

Item Response Theory Modeling of the Philadelphia Naming Test.

PURPOSE: In this study, we investigated the fit of the Philadelphia Naming Test (PNT; Roach, Schwartz, Martin, Grewal, & Brecher, 1996) to an item-response-theory measurement model, estimated the precision of the resulting scores and item parameters, and provided a theoretical rationale for the interpretation of PNT overall scores by relating explanatory variables to item difficulty. This article describes the statistical model underlying the computer adaptive PNT presented in a companion article (Hula, Kellough, & Fergadiotis, 2015). METHOD: Using archival data, we evaluated the fit of the PNT to 1- and 2-parameter logistic models and examined the precision of the resulting parameter estimates. We regressed the item difficulty estimates on three predictor variables: word length, age of acquisition, and contextual diversity. RESULTS: The 2-parameter logistic model demonstrated marginally better fit, but the fit of the 1-parameter logistic model was adequate. Precision was excellent for both person ability and item difficulty estimates. Word length, age of acquisition, and contextual diversity all independently contributed to variance in item difficulty. CONCLUSIONS: Item-response-theory methods can be productively used to analyze and quantify anomia severity in aphasia. Regression of item difficulty on lexical variables supported the validity of the PNT and interpretation of anomia severity scores in the context of current word-finding models.

Development and Simulation Testing of a Computerized Adaptive Version of the Philadelphia Naming Test.

PURPOSE: The purpose of this study was to develop a computerized adaptive test (CAT) version of the Philadelphia Naming Test (PNT; Roach, Schwartz, Martin, Grewal, & Brecher, 1996), to reduce test length while maximizing measurement precision. This article is a direct extension of a companion article (Fergadiotis, Kellough, & Hula, 2015), in which we fitted the PNT to a 1-parameter logistic item-response-theory model and examined the validity and precision of the resulting item parameter and ability score estimates. METHOD: Using archival data collected from participants with aphasia, we simulated two PNT-CAT versions and two previously published static PNT short forms, and compared the resulting ability score estimates to estimates obtained from the full 175-item PNT. We used a jackknife procedure to maintain independence of the samples used for item estimation and CAT simulation. RESULTS: The PNT-CAT recovered full PNT scores with equal or better accuracy than the static short forms. Measurement precision was also greater for the PNT-CAT than the static short forms, though comparison of adaptive and static nonoverlapping alternate forms showed minimal differences between the two approaches. CONCLUSION: These results suggest that CAT assessment of naming in aphasia has the potential to reduce test burden while maximizing the accuracy and precision of score estimates.

The Aphasia Communication Outcome Measure (ACOM): Dimensionality, Item Bank Calibration, and Initial Validation.

PURPOSE: The purpose of this study is to investigate the structure and measurement properties of the Aphasia Communication Outcome Measure (ACOM), a patient-reported outcome measure of communicative functioning for persons with aphasia. METHOD: Three hundred twenty-nine participants with aphasia responded to 177 items asking about communicative functioning. The data were analyzed using a categorical item factor analysis approach. Validity of ACOM scores on the basis of their convergence with performance-based, clinician-reported, and surrogate-reported assessments of communication was also assessed. RESULTS: Fifty-nine items that obtained adequate fit to a modified bifactor measurement model and functioned similarly across several demographic and clinical subgroupings were identified. The factor model estimates were transformed to item response theory graded response model parameters, and the resulting score estimates showed good precision and moderately strong convergence with other measures of communicative ability and functioning. A free software application for administration and scoring of the ACOM item bank is available from the first author. CONCLUSIONS: The ACOM provides reliable measurement of patient-reported communicative functioning in aphasia. The results supported the validity of ACOM scores insofar as (a) factor analyses provided support for a coherent measurement model, (b) items functioned similarly across demographic and clinical subgroups, and (c) scores showed good convergence with measures of related constructs.