Examining the Existence of Cognitive Thresholds in Highly Quantitative College Courses.

While the dominant finding indicates a monotonic relationship between cognitive ability and academic performance, some researchers have suggested the existence of cognitive thresholds for challenging coursework, such that a certain level of cognitive ability is required for reaching a satisfactory level of academic achievement. Given the significance of finding a threshold for understanding the relationship between cognitive ability and academic performance, and the limited studies on the topic, it is worth further investigating the possibility of cognitive thresholds. Using a multi-institutional dataset and the necessary condition analysis (NCA), we attempted to replicate previous findings of cognitive thresholds on the major GPA of mathematics and physics-majored students, as well as the course grade of organic chemistry, to examine whether high SAT math scores constitute a necessary condition for obtaining satisfactory grades in these courses. The results from the two studies do not indicate an absolute cognitive threshold point below which students are doomed to fail regardless of the amount of effort they devote into learning. However, we did find that the chance of students with a low level of quantitative ability to succeed in highly quantitative courses is very small, which qualifies for the virtually necessary condition.

A contemporary look at the relationship between general cognitive ability and job performance.

The relationship between general cognitive ability (GCA) and overall job performance has been a long-accepted fact in industrial and organizational psychology. However, the most prominent data on this relationship date back more than 50 years. This meta-analysis examines the relationship between GCA and overall job performance using studies from the current century. Results across 153 samples and a total sample size of 40,740 show a mean observed validity of .16, with a residual SD of .09. Correcting for unreliability in the criterion and correcting predictive studies for range restriction produces a mean corrected validity of .22 and a residual SD of .11. While this is a much smaller estimate than the .51 value offered by Schmidt and Hunter (1998), that value has been critiqued by Sackett et al. (2022), who offered a mean corrected validity of .31 based on integrating findings from prior meta-analyses of 20th century data. We obtain a lower value (.22) for 21st century data. We conclude that GCA is related to job performance, but our estimate of the magnitude of the relationship is lower than prior estimates. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

What Was Not Said and What to Do About It.

The Woo et al. review (this issue) provides a foundation for considering the larger goals of higher education. We step back to consider the broader goals and ideals of higher education. Fundamentally, we want to admit a diverse set of students into graduate school and then produce the most accomplished scientists, artists, leaders, and innovators. In a world with inequality in preparation and finite resources, these ideals end up in tension without any easy resolution. The inability to provide opportunities and develop talent across all groups up to early adulthood is the fundamental problem we face. It is tempting to ignore it. We would be delighted if test and grade differences could be easily dismissed. Instead, we know that a great deal of potential is being wasted, and this waste represents a terrible loss for individuals, communities, and society. We believe that the greatest change will come from better and expanded investment in expanded gifted-and-talented programs, increasing the flow of underrepresented students into these programs, greatly improved assessment of psychosocial skills and talents at all levels, and career counseling and mentoring that begins early and continues through higher education.

The process of attrition in pre-medical studies: A large-scale analysis across 102 schools.

The important but difficult choice of vocational trajectory often takes place in college, beginning with majoring in a subject and taking relevant coursework. Of all possible disciplines, pre-medical studies are often not a formally defined major but pursued by a substantial proportion of the college population. Understanding students' experiences with pre-med coursework is valuable and understudied, as most research on medical education focuses on the later medical school and residency. We examined the pattern and predictors of attrition at various milestones along the pre-med coursework track during college. Using a College Board dataset, we analyzed a sample of 15,442 students spanning 102 institutions who began their post-secondary education in years between 2006 and 2009. We examined whether students fulfilled the required coursework to remain eligible for medical schools at several milestones: 1) one semester of general chemistry, biology, physics, 2) two semesters of general chemistry, biology, physics, 3) one semester of organic chemistry, and 4) either the second semester of organic chemistry or one semester of biochemistry, and predictors of persistence at each milestone. Only 16.5% of students who intended to major in pre-med graduate college with the required coursework for medical schools. Attrition rates are highest initially but drop as students take more advanced courses. Predictors of persistence include academic preparedness before college (e.g., SAT scores, high school GPA) and college performance (e.g., grades in pre-med courses). Students who perform better academically both in high school and in college courses are more likely to remain eligible for medical school.

Personality, attitude, and demographic correlates of academic dishonesty: A meta-analysis.

Over a century of research has focused on the relationship between student characteristics and academic dishonesty. Here we conduct a meta-analysis of this extensive literature across three broad domains: personality, academic attitudes, and historical information including demographic and prior accomplishments. After analyzing correlations from over 27,000 participants across 75 samples, several relationships stand out. Although measures of conscientiousness (ρ¯ = -.25) and agreeableness (ρ¯ = -.13) from the popular five-factor model demonstrate important relationships with dishonesty, equal or greater effects were found for specific personality scales including impulsivity (ρ¯ = .39) and psychopathy (ρ¯ = .40). Both morality (ρ¯ = -.24) and academic self-efficacy (ρ¯ = -.28) were related to less dishonest behavior but demonstrated substantially smaller effects than measures of neutralization (ρ¯ = .43), a tendency to justify and deflect immoral behavior as reasonable given the circumstances. Hypotheses were developed and results are framed by two competing models of academic dishonesty. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Some Roads Lead to Psychology, Some Lead Away: College Student Characteristics and Psychology Major Choice.

Understanding which students enter and leave psychology majors in college is critical to understanding the pipeline into the field. In this study, we compared psychology majors with nonpsychology majors on the basis of demographic, degree planning, academic preparedness, and academic performance variables using a unique longitudinal sample of nearly a million college students at 249 colleges and universities. Guided by prior research, we examined which students would persist in psychology, enter psychology from another major, or leave psychology for another major between three points in time: intended major before entering college, second-year college major, and fourth-year college major. Critically, most students who majored in psychology did not initially express interest in the field, but entering and persisting in the field was strongly associated with high school exposure and performance in psychology. Students with poorer performance in college often transfer into psychology from majors that may be perceived as difficult (e.g., science, technology, engineering, and math), whereas higher-performing students appear to leave psychology for these same majors, which may also be perceived as more lucrative. These results are concerning for the field of psychology if individuals with high potential who are originally interested in the field eventually leave.

The relation between parents' involvement in children's schooling and children's adjustment: A meta-analysis.

This quantitative synthesis of 448 independent studies including 480,830 families revealed small positive associations (rs = .13 to .23) between parents' naturally occurring involvement in children's schooling and children's academic adjustment (i.e., achievement, engagement, and motivation) that were maintained over time. Parents' involvement was also positively related to children's social (r = .12) and emotional adjustment (r = .17) and negatively related to their delinquency (r = -.15), concurrently. Analyses focusing on children's academic adjustment revealed that different types of involvement (e.g., parents' participation in school events and discussion of school with children) were similarly positively associated with such adjustment. The only exception was that parents' homework assistance was negatively associated with children's achievement (r = -.15), but not engagement (r = .07) or motivation (r = .05). There was little variation due to age, ethnicity, or socioeconomic status in the links between different types of involvement and children's academic adjustment. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Effects of range restriction and criterion contamination on differential validity of the SAT by race/ethnicity and sex.

We illustrate the effects of range restriction and a form of criterion contamination (individual differences in course-taking patterns) on the validity of SAT scores for predicting college academic performance. College data facilitate exploration of differential validity's determinants because they (a) permit the use multivariate range-restriction corrections to more accurately account for differential range restriction across subgroups and (b) allow for separate examinations of composite performance and specific performance episodes, the latter of which controls for ecological contamination of composite performance due to individuals' choices of performance opportunities. Using data from 363,004 students at 107 U.S. institutions, we found that controlling for course-taking patterns resulted in validity coefficients that were appreciably larger than predictors' correlations with obtained grade point averages (GPAs). The validities of SAT scores for predicting the first-year college performance of Black and Hispanic students were not significantly different from the validity for White students after correcting for both course-taking patterns and differential range restriction, but significant Black-White differences were detected for predicting 4-year cumulative performance. Validity estimates for predicting both first-year and 4-year cumulative performance were significantly smaller among Asian students than White students after making these corrections. The SAT's observed validity for predicting college GPAs was substantially lower for males than females and, unexpectedly, controlling for course-taking patterns increased male-female validity differences. Implications for personnel selection research are discussed. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Changing abilities vs. changing tasks: Examining validity degradation with test scores and college performance criteria both assessed longitudinally.

We explore potential explanations for validity degradation using a unique predictive validation data set containing up to four consecutive years of high school students' cognitive test scores and four complete years of those students' college grades. This data set permits analyses that disentangle the effects of predictor-score age and timing of criterion measurements on validity degradation. We investigate the extent to which validity degradation is explained by criterion dynamism versus the limited shelf-life of ability scores. We also explore whether validity degradation is attributable to fluctuations in criterion variability over time and/or GPA contamination from individual differences in course-taking patterns. Analyses of multiyear predictor data suggest that changes to the determinants of performance over time have much stronger effects on validity degradation than does the shelf-life of cognitive test scores. The age of predictor scores had only a modest relationship with criterion-related validity when the criterion measurement occasion was held constant. Practical implications and recommendations for future research are discussed. (PsycINFO Database Record

Effects of predictor weighting methods on incremental validity.

It is common to add an additional predictor to a selection system with the goal of increasing criterion-related validity. Research on the incremental validity of a second predictor is generally based on forming a regression-weighted composite of the predictors. However, in practice predictors are commonly used in ways other than regression-weighted composites, and we examine the robustness of incremental validity findings to other ways of using predictors, namely, unit weighting and multiple hurdles. We show that there are settings in which the incremental value of a second predictor disappears, and can even produce lower validity than the first predictor alone, when these alternatives to regression weighting are used. First, we examine conditions under which unit weighting will negate gain in predictive power attainable via regression weights. Second, we revisit Schmidt and Hunter's (1998) summary of incremental validity of predictors over cognitive ability, evaluating whether the reported incremental value of a second predictor is different when predictors are unit weighted rather than regression weighted. Third, we analyze data reported in the published literature to discern the frequency with which unit weighting might affect conclusions about whether there is value in adding a second predictor to a first. Finally, we shift from unit weighting to multiple hurdle selection, examining conditions under which conclusions about incremental validity differ when regression weighting is replaced by multiple-hurdle selection. (PsycINFO Database Record

Individual differences and their measurement: A review of 100 years of research.

This article reviews 100 years of research on individual differences and their measurement, with a focus on research published in the Journal of Applied Psychology. We focus on 3 major individual differences domains: (a) knowledge, skill, and ability, including both the cognitive and physical domains; (b) personality, including integrity, emotional intelligence, stable motivational attributes (e.g., achievement motivation, core self-evaluations), and creativity; and (c) vocational interests. For each domain, we describe the evolution of the domain across the years and highlight major theoretical, empirical, and methodological developments, including relationships between individual differences and variables such as job performance, job satisfaction, and career development. We conclude by discussing future directions for individual differences research. Trends in the literature include a growing focus on substantive issues rather than on the measurement of individual differences, a differentiation between constructs and measurement methods, and the use of innovative ways of assessing individual differences, such as simulations, other-reports, and implicit measures. (PsycINFO Database Record

Why women perform better in college than admission scores would predict: Exploring the roles of conscientiousness and course-taking patterns.

Women typically obtain higher subsequent college GPAs than men with the same admissions test score. A common reaction is to attribute this to a flaw in the admissions test. We explore the possibility that this underprediction of women's performance reflects gender differences in conscientiousness and college course-taking patterns. In Study 1, we focus on using the ACT to predict performance in a single, large course where performance is decomposed into cognitive (exam and quiz scores) and less cognitive, discretionary components (discussion and extra credit points). The ACT does not underpredict female's cognitive performance, but it does underpredict female performance on the less cognitive, discretionary components of academic performance, because it fails to measure and account for the personality trait of conscientiousness. In Study 2, we create 2 course-difficulty indices (Course Challenge and Mean Aptitude in Course) and add them to an HLM regression model to see if they reduce the degree to which SAT scores underpredict female performance. Including Course Challenge does result in a modest reduction of the gender coefficient; however, including Mean Aptitude in Course does not. Thus, differences in course-taking patterns is a partial (albeit small) explanation for the common finding of differential prediction by gender.

Resolving the assessment center construct validity problem (as we know it).

Ongoing concern about the construct validity of assessment center dimensions has focused on postexercise dimension ratings (PEDRs) that are consistently found to reflect exercise variance to a greater degree than dimension variance. Here, we present a solution to this problem. Based on the argument that PEDRs are an intermediate step toward an overall dimension rating, and that the overall dimension rating should be the focus of inquiry, we demonstrate that correlated sources of dimension variance accumulate and increasingly displace uncorrelated sources of both systematic variance and error. Viewing overall dimension ratings as a composite of PEDRs, we show dimension variance will commonly quickly overtake exercise-specific variance as the dominant source of variance as ratings from multiple exercises are combined. We embed our results in a new framework for categorizing different levels of construct variance dominance, and our results indicate that with as few as two exercises, dimension variance can reach our lowest level of construct variance dominance. However, the largest source of dimension variance is a general factor. We conclude that the construct validity problem in assessments centers never existed as historically framed, but the presence of a general factor may limit interpretation for developmental purposes.

Mechanical versus clinical data combination in selection and admissions decisions: a meta-analysis.

In employee selection and academic admission decisions, holistic (clinical) data combination methods continue to be relied upon and preferred by practitioners in our field. This meta-analysis examined and compared the relative predictive power of mechanical methods versus holistic methods in predicting multiple work (advancement, supervisory ratings of performance, and training performance) and academic (grade point average) criteria. There was consistent and substantial loss of validity when data were combined holistically-even by experts who are knowledgeable about the jobs and organizations in question-across multiple criteria in work and academic settings. In predicting job performance, the difference between the validity of mechanical and holistic data combination methods translated into an improvement in prediction of more than 50%. Implications for evidence-based practice are discussed.

The role of socioeconomic status in SAT-grade relationships and in college admissions decisions.

This article examines the role of socioeconomic status (SES) in the relationships among college admissions-test scores, secondary school grades, and subsequent academic performance. Scores on the SAT (a test widely used in the admissions process in the United States), secondary school grades, college grades, and SES measures from 143,606 students at 110 colleges and universities were examined, and results of these analyses were compared with results obtained using a 41-school data set including scores from the prior version of the SAT and using University of California data from prior research on the role of SES. In all the data sets, the SAT showed incremental validity over secondary school grades in predicting subsequent academic performance, and this incremental relationship was not substantially affected by controlling for SES. The SES of enrolled students was very similar to that of specific schools' applicant pools, which suggests that the barrier to college for low-SES students in the United States is a lower rate of entering the college admissions process, rather than exclusion on the part of colleges.

A Plea for Process in Personality Prevarication.

We make a series recommendations for focusing research on personality test faking. Overall we suggest that a focus on the response process test takers go through will accelerate our understanding of faking behavior. We argue that the decision making process for faking must be simple and dependent on a modest set of decision rules or heuristics. The set of heuristics used by any given test taker will, in turn, be the result of test taker goals and situational press. By focusing in on what the test taker is doing we will avoid adopting the wrong frame of reference and, hopefully, make ever more rapid progress.

The predictive validity of nursing admission measures for performance on the National Council Licensure Examination: a meta-analysis.

The purpose of this meta-analysis was to examine the power of key admission and nursing school variables for predicting performance on the National Council Licensure Examination for Registered Nurses (NCLEX-RN). A meta-analysis of 31 independent samples across 7,159 participants yielded correlation estimates for 13 different predictors. Results indicated that standardized admissions tests (SAT and ACT) were effective predictors of performance. Prenursing grade point average (GPA) was also predictive, but to a lesser extent. Grades earned during baccalaureate degree nursing education were especially strong predictors. The strongest predictors were grades earned during the second year. Overall, multiple effective measures are available to predict success during admissions and diagnose risk of failure during nursing education.

Does socioeconomic status explain the relationship between admissions tests and post-secondary academic performance?

Critics of educational admissions tests assert that tests measure nothing more than socioeconomic status (SES) and that their apparent validity in predicting academic performance is an artifact of SES. The authors examined multiple large data sets containing data on admissions and related tests, SES, and grades showing that (a) SES is related to test scores (r = .42 among the population of SAT takers), (b) test scores are predictive of academic performance, and (c) statistically controlling for SES reduces the estimated test?grade correlation from r = .47 to r = .44. Thus, the vast majority of the test?academic performance relationship was independent of SES: The authors concluded that the test?grade relationship is not an artifact of common influences of SES on both test scores and grades.

Study Habits, Skills, and Attitudes: The Third Pillar Supporting Collegiate Academic Performance.

Study habit, skill, and attitude inventories and constructs were found to rival standardized tests and previous grades as predictors of academic performance, yielding substantial incremental validity in predicting academic performance. This meta-analysis (N = 72,431, k = 344) examines the construct validity and predictive validity of 10 study skill constructs for college students. We found that study skill inventories and constructs are largely independent of both high school grades and scores on standardized admissions tests but moderately related to various personality constructs; these results are inconsistent with previous theories. Study motivation and study skills exhibit the strongest relationships with both grade point average and grades in individual classes. Academic specific anxiety was found to be an important negative predictor of performance. In addition, significant variation in the validity of specific inventories is shown. Scores on traditional study habit and attitude inventories are the most predictive of performance, whereas scores on inventories based on the popular depth-of-processing perspective are shown to be least predictive of the examined criteria. Overall, study habit and skill measures improve prediction of academic performance more than any other noncognitive individual difference variable examined to date and should be regarded as the third pillar of academic success.

Application patterns when applicants know the odds: implications for selection research and practice.

Unlike previous research that found small differences between population standard deviations and applicant pool standard deviations (P. R. Sackett & D. J. Ostgaard, 1994; D. S. Ones & C. Viswesvaran, 2003), this study revealed a 23% disparity between Law School Admission Test (LSAT) scores of all LSAT test takers and those of LSAT test takers who applied to law school. This study also illustrated robust applicant self-selection behavior across different law school ranks. These findings are important, because predictor scores of applicants who know their scores in advance and perceive small selection ratios necessitate substantially smaller range restriction corrections than those that would be required by population standard deviations. Furthermore, these findings more generally reveal that applicants who know their scores in advance behave quite differently from applicants who do not.