General fluid/inductive reasoning battery for a high-ability population.

The validity of studies investigating interventions to enhance fluid intelligence (Gf) depends on the adequacy of the Gf measures administered. Such studies have yielded mixed results, with a suggestion that Gf measurement issues may be partly responsible. The purpose of this study was to develop a Gf test battery comprising tests meeting the following criteria: (a) strong construct validity evidence, based on prior research; (b) reliable and sensitive to change; (c) varying in item types and content; (d) producing parallel tests, so that pretest-posttest comparisons could be made; (e) appropriate time limits; (f) unidimensional, to facilitate interpretation; and (g) appropriate in difficulty for a high-ability population, to detect change. A battery comprising letter, number, and figure series and figural matrix item types was developed and evaluated in three large-N studies (N = 3,067, 2,511, and 801, respectively). Items were generated algorithmically on the basis of proven item models from the literature, to achieve high reliability at the targeted difficulty levels. An item response theory approach was used to calibrate the items in the first two studies and to establish conditional reliability targets for the tests and the battery. On the basis of those calibrations, fixed parallel forms were assembled for the third study, using linear programming methods. Analyses showed that the tests and test battery achieved the proposed criteria. We suggest that the battery as constructed is a promising tool for measuring the effectiveness of cognitive enhancement interventions, and that its algorithmic item construction enables tailoring the battery to different difficulty targets, for even wider applications.

Social-emotional skill assessment in children and adolescents: Advances and challenges in personality, clinical, and educational contexts.

The development and promotion of social-emotional skills in childhood and adolescence contributes to subsequent well-being and positive life outcomes. However, the assessment of these skills is associated with conceptual and methodological challenges. This review discusses how social-emotional skill measurement in youth could be improved in terms of skills' conceptualization and classification, and in terms of assessment techniques and methodologies. The first part of the review discusses various conceptualizations of social-emotional skills, demonstrates their overlap with related constructs such as emotional intelligence and the Big Five personality dimensions, and proposes an integrative set of social-emotional skill domains that has been developed recently. Next, methodological approaches that are innovative and may improve social-emotional assessments are presented, illustrated by concrete examples. We discuss how these innovations could advance social-emotional assessments, and demonstrate links to similar issues in related fields. We conclude the review by providing several concrete assessment recommendations that follow from this discussion. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Rasch measurement in developing faculty ratings of students applying to graduate school.

The Standardized Letter of Recommendation (SLR), a 28-item form, was created by ETS to supplement the qualitative rating of graduate school applicants' nonacademic qualities with a quantitative approach. The purpose of this study was to evaluate the following psychometric properties of the SLR using the Rasch rating scale model: dimensionality, reliability, item quality, and rating category effectiveness. Principal component and factor analyses were also conducted to examine the dimensionality of the SLR. Results revealed (a) two secondary factors underlay the data, along with a strong higher order factor, (b) item and person separation reliabilities were high, (c) noncognitive items tended to elicit higher endorsements than did cognitive items, and (d) a 5-point Likert scale functioned effectively. The psychometric properties of the SLR support the use of a composite score when reporting SLR scores and the utility of the SLR in higher education and in admissions.

Inequality, Education, Workforce Preparedness, and Complex Problem Solving.

Economic inequality has been described as the defining challenge of our time, responsible for a host of potential negative societal and individual outcomes including reduced opportunity, decreased health and life expectancy, and the destabilization of democracy. Education has been proposed as the "great equalizer" that has and can continue to play a role in reducing inequality. One means by which education does so is through the development of complex problem solving skills in students, skills used to solve novel, ill-defined problems in complex, real-world settings. These are highly valued in the workforce and will likely continue to be so in the future workforce. Their importance is evident in results from employer surveys, as well as by their inclusion in large scale international and domestic comparative assessments. In this paper, I review various definitions of complex problem solving and approaches for measuring it, along with findings from PISA 2003, 2012, and 2015. I also discuss prospects for teaching, assessing, and reporting on it, and discuss the emerging importance of collaborative problem solving. Developing and monitoring complex problem solving skills, broadly defined, is a critical challenge in preparing students for the future workforce, and in overcoming the negative effects of inequality and the diminishment of individual opportunity.

Ability Tests Measure Personality, Personality Tests Measure Ability: Disentangling Construct and Method in Evaluating the Relationship between Personality and Ability.

Although personality and cognitive ability are separate (sets of) constructs, we argue and demonstrate in this article that their effects are difficult to tease apart, because personality affects the performance on cognitive tests and cognitive ability affects the item responses on personality assessments. Cognitive ability is typically measured with tests of items with correct answers; personality is typically measured with rating-scale self-reports. Oftentimes conclusions regarding the personality-ability relationship have as much to do with measurement methods as with the construct similarities and differences. In this article, we review key issues that touch on the relationship between cognitive ability and personality. These include the construct-method distinction, sources of test score variance, the maximal vs. typical performance distinction, and the special role for motivation in low-stakes testing. We review a general response model for cognitive and personality tests that recognizes those sources of test score variance. We then review the approaches for measuring personality through performance (objective personality tests, grit game, coding speed, economic preferences, and confidence), test and survey behavior (survey effort, response time, and item position effects), and real-world behavior (study time, registration latency, behavior residue, and social media). We also discuss ability effects on personality tests, indicated by age and cognitive ability effects, anchoring vignette rating errors, and instructions to 'fake good'. We conclude with a discussion of the implications for our understanding of personality and ability differences, and suggestions for integrating the fields.

Fluid Ability (Gf) and Complex Problem Solving (CPS).

Complex problem solving (CPS) has emerged over the past several decades as an important construct in education and in the workforce. We examine the relationship between CPS and general fluid ability (Gf) both conceptually and empirically. A review of definitions of the two factors, prototypical tasks, and the information processing analyses of performance on those tasks suggest considerable conceptual overlap. We review three definitions of CPS: a general definition emerging from the human problem solving literature; a more specialized definition from the "German School" emphasizing performance in many-variable microworlds, with high domain-knowledge requirements; and a third definition based on performance in Minimal Complex Systems (MCS), with fewer variables and reduced knowledge requirements. We find a correlation of 0.86 between expert ratings of the importance of CPS and Gf across 691 occupations in the O*NET database. We find evidence that employers value both Gf and CPS skills, but CPS skills more highly, even after controlling for the importance of domain knowledge. We suggest that this may be due to CPS requiring not just cognitive ability but additionally skill in applying that ability in domains. We suggest that a fruitful future direction is to explore the importance of domain knowledge in CPS.