Validation of the cognitive section of the Penn computerized adaptive test for neurocognitive and clinical psychopathology assessment (CAT-CCNB).

BACKGROUND: The Penn Computerized Neurocognitive Battery is an efficient tool for assessing brain-behavior domains, and its efficiency was augmented via computerized adaptive testing (CAT). This battery requires validation in a separate sample to establish psychometric properties. METHODS: In a mixed community/clinical sample of N = 307 18-to-35-year-olds, we tested the relationships of the CAT tests with the full-form tests. We compared discriminability among recruitment groups (psychosis, mood, control) and examined how their scores relate to demographics. CAT-Full relationships were evaluated based on a minimum inter-test correlation of 0.70 or an inter-test correlation within at least 0.10 of the full-form correlation with a previous administration of the full battery. Differences in criterion relationships were tested via mixed models. RESULTS: Most tests (15/17) met the minimum criteria for replacing the full-form with the updated CAT version (mean r = 0.67; range = 0.53-0.80) when compared to relationships of the full-forms with previous administrations of the full-forms (mean r = 0.68; range = 0.50-0.85). Most (16/17) CAT-based relationships with diagnostics and other validity criteria were indistinguishable (interaction p > 0.05) from their full-form counterparts. CONCLUSIONS: The updated CNB shows psychometric properties acceptable for research. The full-forms of some tests should be retained due to insufficient time savings to justify the loss in precision.

Construction of a computerized adaptive test (CAT-CCNB) for efficient neurocognitive and clinical psychopathology assessment.

BACKGROUND: Traditional paper-and-pencil neurocognitive evaluations and semi-structured mental health interviews can take hours to administer and score. Computerized assessment has decreased that burden substantially, and contemporary psychometric tools such as item response theory and computerized adaptive testing (CAT) allow even further abbreviation. NEW METHOD: The goal of this paper was to describe the application of CAT and related methods to the Penn Computerized Neurocognitive Battery (CNB) and a well-validated clinical assessment in order to increase efficiency in assessment and relevant domain coverage. To calibrate item banks for CAT, N = 5053 participants (63% female; mean age 45 years, range 18-80) were collected from across the United States via crowdsourcing, providing item parameters that were then linked to larger item banks and used in individual test construction. Tests not amenable to CAT were abbreviated using complementary short-form methods. RESULTS: The final "CAT-CCNB" battery comprised 21 cognitive tests (compared to 14 in the original) and five adaptive clinical scales (compared to 16 in the original). COMPARISON WITH EXISTING METHODS: This new battery, derived with contemporary psychometric approaches, provides further improvements over existing assessments that use collections of fixed-length tests developed for stand-alone administration. The CAT-CCNB provides an improved version of the CNB that shows promise as a maximally efficient tool for neuropsychiatric assessment. CONCLUSIONS: We anticipate CAT-CCNB will help satisfy the clear need for broad yet efficient measurement of cognitive and clinical domains, facilitating implementation of large-scale, "big science" approaches to data collection, and potential widespread clinical implementation.