Psychometric Modeling to Identify Examinees' Strategy Differences during Testing.

Aptitude test scores are typically interpreted similarly for examinees with the same overall score. However, research has found evidence of examinee differences in strategies, as well as in the continued application of appropriate procedures during testing. Such differences can impact the correlates of test scores, making similar interpretations for equivalent scores questionable. This study presents some item response theory (IRT) models that are relevant to identifying examinee differences in strategies and understanding of test-taking procedures. First, mixture IRT models that identify latent classes of examinees with different patterns of item responses are considered; these models have long been available but unfortunately are not routinely applied. Strategy differences between the classes can then be studied separately by modeling the response patterns with cognitive complexity variables within each class. Secondly, novel psychometric approaches that leverage response time information (in particular, response time residuals) in order to identify both inter and intraindividual variability in response processes are considered. In doing so, a general method for evaluating threats to validity is proposed. The utility of the approach, in terms of providing more interpretable performance estimates and improving the administration of psychological measurement instruments, is then demonstrated with an empirical example.

What item response theory can tell us about the complex span tasks.

Working memory capacity is an important construct in psychology because of its relationship with many higher-order cognitive abilities and psychopathologies. Working memory capacity is often measured using a type of paradigm known as complex span. Some recent work has focused on shortening the administration time of the complex span tasks, resulting in different versions of these tasks being used (Foster et al., 2015; Oswald, McAbee, Redick, & Hambrick, 2015). Variations in the complex span tasks, such as the number of set sizes, can lead to varying power to discriminate individuals at different ability levels. Thus, research findings may be inconsistent across populations due to differing appropriateness for the ability levels. The present study uses a combination of item response theory and correlational analyses to better understand the psychometric properties of the operation span, symmetry span, and rotation span. The findings show that the typical administration of these tasks, particularly the operation span, is not suitable for above average ability samples (Study 1; n = 573). When larger set sizes are added to the tasks (Study 2; n = 351), predictive validity and discriminability is improved for all complex span tasks, however the operation span is still inferior to the spatial tasks. The authors make several conclusions about which tasks and set sizes should be used depending on the intended population, and further suggest avoiding the standard-length operation span for average or higher ability populations. (PsycINFO Database Record

An Experimental Evaluation of Competing Age-Predictions of Future Time Perspective between Workplace and Retirement Domains.

Future time perspective (FTP) is defined as "perceptions of the future as being limited or open-ended" (Lang and Carstensen, 2002; p. 125). The construct figures prominently in both workplace and retirement domains, but the age-predictions are competing: Workplace research predicts decreasing FTP age-change, in contrast, retirement scholars predict increasing FTP age-change. For the first time, these competing predictions are pitted in an experimental manipulation of subjective life expectancy (SLE). A sample of N = 207 older adults (age 45-60) working full-time (>30-h/week) were randomly assigned to SLE questions framed as either 'Live-to' or 'Die-by' to evaluate competing predictions for FTP. Results indicate general support for decreasing age-change in FTP, indicated by independent-sample t-tests showing lower FTP in the 'Die-by' framing condition. Further general-linear model analyses were conducted to test for interaction effects of retirement planning with experimental framings on FTP and intended retirement; While retirement planning buffered FTP's decrease, simple-effects also revealed that retirement planning increased intentions for sooner retirement, but lack of planning increased intentions for later retirement. Discussion centers on practical implications of our findings and consequences validity evidence in future empirical research of FTP in both workplace and retirement domains.

The Multicomponent Latent Trait Model for Diagnosis: Applications to Heterogeneous Test Domains.

Heterogeneous item content is prevalent on psychological and educational tests that measure global traits or competencies, such as general intelligence and achievement tests. In such tests, the domain is broadly defined, so as to include many attributes and skills. Items often vary substantially in both the type and the number of attributes or skills that are involved in item solving. A hierarchical organization is often necessary to accommodate the heterogeneity of the test domain. For example, the mathematical achievement tests that are routinely administered in all U.S. states at the end of the school year typically have this hierarchical structure. The multicomponent latent trait model for diagnosis (MLTM-D) was developed for application to heterogeneous tests. MLTM-D is a confirmatory model that permits diagnosis at broad and more specific attribute or skill levels. In the current study, MLTM-D is applied to diagnose mastery at both the broad and specific skill levels in middle school mathematics. MLTM-D is presented, and methods involved in application are described.

A multicomponent latent trait model for diagnosis.

This paper presents a noncompensatory latent trait model, the multicomponent latent trait model for diagnosis (MLTM-D), for cognitive diagnosis. In MLTM-D, a hierarchical relationship between components and attributes is specified to be applicable to permit diagnosis at two levels. MLTM-D is a generalization of the multicomponent latent trait model (MLTM; Whitely in Psychometrika, 45:479-494, 1980; Embretson in Psychometrika, 49:175-186, 1984) to be applicable to measures of broad traits, such as achievement tests, in which component structure varies between items. Conditions for model identification are described and marginal maximum likelihood estimators are presented, along with simulation data to demonstrate parameter recovery. To illustrate how MLTM-D can be used for diagnosis, an application to a large-scale test of mathematics achievement is presented. An advantage of MLTM-D for diagnosis is that it may be more applicable to large-scale assessments with more heterogeneous items than are latent class models.

Predictors of exercise and effects of exercise on symptoms, function, aerobic fitness, and disease outcomes of rheumatoid arthritis.

OBJECTIVE: To determine the effects of participation in a low-impact aerobic exercise program on fatigue, pain, and depression; to examine whether intervention groups compared with a control group differed on functional (grip strength and walk time) and disease activity (total joint count, erythrocyte sedimentation rate, and C-reactive protein) measures and aerobic fitness at the end of the intervention; and to test which factors predicted exercise participation. METHODS: A convenience sample of 220 adults with rheumatoid arthritis (RA), ages 40-70, was randomized to 1 of 3 groups: class exercise, home exercise using a videotape, and control group. Measures were obtained at baseline (T1), after 6 weeks of exercise (T2), and after 12 weeks of exercise (T3). RESULTS: Using structural equation modeling, overall symptoms (latent variable for pain, fatigue, and depression) decreased significantly at T3 (P < 0.04) for the class exercise group compared with the control group. There were significant interaction effects of time and group for the functional measures of walk time and grip strength: the treatment groups improved more than the control group (P

Multicomponent latent trait models for complex tasks.

Contemporary views on cognitive theory (e.g., Sternberg and Perez, 2005) regard typical measurement tasks, such as ability and achievement test items, multidimensional, rather than unidimensional. Assessing the levels and the sources of multidimensionality in an item domain is important for item selection as well as for item revision and development. In this paper, multicomponent latent trait models (MLTM) and traditional multidimensional item response theory models are described mathematically and compared for the nature of the dimensions that can be estimated. Then, some applications are presented to provide examples of MLTM. Last, practical estimation procedures are described, along with syntax, for the estimation of MLTM and a related model.

The continued search for nonarbitrary metrics in psychology.

H. Blanton and J. Jaccard examined the arbitrariness of metrics in the context of 2 current issues: (a) the measurement of racial prejudice and (b) the establishment of clinically significant change. According to Blanton and Jaccard, although research findings are not undermined by arbitrary metrics, individual scores and score changes may not be meaningfully interpreted. The author believes that their points are mostly valid and that their examples were appropriate. However, Blanton and Jaccard's article does not lead directly to solutions, nor did it adequately describe the scope of the metric problem. This article has 2 major goals. First, some prerequisites for nonarbitrary metrics are presented and related to Blanton and Jaccard's issues. Second, the impact of arbitrary metrics on psychological research findings are described. In contrast to Blanton and Jaccard (2006), research findings suggest that metrics have direct impact on statistics for group comparisons and trend analysis.