A nuanced view of the extent to which samples from narrow populations are scientifically problematic.

Psychologists have a traditional concern with participant samples from narrow populations and deleterious effects on researchers' ability to generalize findings. Recently, both individuals and authoritative organizations, such as the American Psychological Association, have merged this external validity concern with diversity and inclusion concerns. The American Psychological Association directive for researchers to include diverse samples seems obviously well-taken as it purports to mitigate these problems at once; it simultaneously increases external validity and promotes diversity and inclusion. However, we show that there are complications. These include problems with internal and external validity conceptualizations; that sometimes generalization failures can support, rather than detract from, external validity; the crucial role auxiliary assumptions play in impacting internal and external validity; Lakatosian degenerative science and its problematic application; and distinguishing between merely including diverse groups in research samples versus analyzing for group differences. These complications imply a nuanced perspective of whether samples from narrow populations are undesirable. That a sample is from a narrow population might, or might not, preclude strong support or disconfirmation for the theory, including its ability to generalize. Our nuanced perspective militates against the current trend of journal directives to require diverse samples. Sample suitability for particular researcher goals should be judged on a case-by-case basis that takes into account that sometimes samples from narrow populations can nevertheless engender impressive scientific progress and sometimes not. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

On the psychometric evaluation of cognitive control tasks: An Investigation with the Dual Mechanisms of Cognitive Control (DMCC) battery.

The domain of cognitive control has been a major focus of experimental, neuroscience, and individual differences research. Currently, however, no theory of cognitive control successfully unifies both experimental and individual differences findings. Some perspectives deny that there even exists a unified psychometric cognitive control construct to be measured at all. These shortcomings of the current literature may reflect the fact that current cognitive control paradigms are optimized for the detection of within-subject experimental effects rather than individual differences. In the current study, we examine the psychometric properties of the Dual Mechanisms of Cognitive Control (DMCC) task battery, which was designed in accordance with a theoretical framework that postulates common sources of within-subject and individual differences variation. We evaluated both internal consistency and test-retest reliability, and for the latter, utilized both classical test theory measures (i.e., split-half methods, intraclass correlation) and newer hierarchical Bayesian estimation of generative models. Although traditional psychometric measures suggested poor reliability, the hierarchical Bayesian models indicated a different pattern, with good to excellent test-retest reliability in almost all tasks and conditions examined. Moreover, within-task, between-condition correlations were generally increased when using the Bayesian model-derived estimates, and these higher correlations appeared to be directly linked to the higher reliability of the measures. In contrast, between-task correlations remained low regardless of theoretical manipulations or estimation approach. Together, these findings highlight the advantages of Bayesian estimation methods, while also pointing to the important role of reliability in the search for a unified theory of cognitive control.

An item response theory approach to the measurement of working memory capacity.

Complex span tasks are perhaps the most widely used paradigm to measure working memory capacity (WMC). Researchers assume that all types of complex span tasks assess domain-general WM. However, most research supporting this claim comes from factor analysis approaches that do not examine task performance at the item level, thus not allowing comparison of the characteristics of verbal and spatial complex span tasks. Item response theory (IRT) can help determine the extent to which different complex span tasks assess domain-general WM. In the current study, spatial and verbal complex span tasks were examined using IRT. The results revealed differences between verbal and spatial tasks in terms of item difficulty and block difficulty, and showed that most subjects with below-average ability were able to answer most items correctly across all tasks. In line with previous research, the findings suggest that examining domain-general WM by using only one task might elicit skewed scores based on task domain. Further, visuospatial complex span tasks should be prioritized as a measure of WMC if resources are limited.

The Structure of Cognitive Abilities and Associations with Problem Behaviors in Early Adolescence: An Analysis of Baseline Data from the Adolescent Brain Cognitive Development Study.

Using baseline data (n = 9875) from the Adolescent Brain Cognitive Development (ABCD) Study examining children aged 9 to 10 years, the current analyses included: (1) exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) of neurocognitive measures administered during baseline collection, and (2) linear regression analyses on the Child Behavior Checklist (CBCL), controlling for demographic and socioeconomic factors. The neurocognitive tasks measured episodic memory, executive function (EF; attention), language skills, processing speed, working memory, visuospatial ability, and reasoning. The CBCL included composite scores of parent-reported internalizing, externalizing, and stress-related behavior problems. The study reported here serves as an extension of prior research using a principal components analysis (PCA) of the ABCD baseline data. We propose an alternative solution using factor analysis. Analyses revealed a three-factor structure: verbal ability (VA), executive function/processing speed (EF/PS), and working memory/episodic memory (WM/EM). These factors were significantly correlated with the CBCL scores, albeit with small effect sizes. These findings provide a novel three-factor solution to the structure of cognitive abilities measured in the ABCD Study, offering new insights into the association between cognitive function and problem behaviors in early adolescence.

The impact of auditory distraction on reading comprehension: An individual differences investigation.

Background noise disrupts auditory selective attention and impairs performance on cognitive tasks, but the degree to which it is disruptive depends on the task and the individual. According to the load theory of attention and cognitive control, selective attention is influenced by both the perceptual load and the cognitive load of the primary task. Recent studies suggest that hard-to-read font in a reading task may shield attention against background noise and auditory distraction. The current study examined the disruptive effect of background noise on reading comprehension as a function of perceptual load and cognitive load. Perceptual load was manipulated by introducing task disfluency (hard-to-read or easy-to-read font), and cognitive load was manipulated by varying the type of background noise and investigating individual differences in working memory capacity. The results suggest that high perceptual load and high working memory capacity both facilitate reading comprehension. However, contrary to previous research, neither perceptual load nor capacity moderates the disruptive effect of background noise. These results failed to support the generalizability and applicability of the shield effect of perceptual disfluency against auditory distraction during reading but supported the beneficial effect of perceptual disfluency on reading comprehension.

From the Incoming Editor.

First, let me say that I am honored to take on the role of Editor in Chief at the Journal of Intelligence [...].

Individual Differences in Attention and Intelligence: A United Cognitive/Psychometric Approach.

Process overlap theory (POT) is a new theoretical framework designed to account for the general factor of intelligence (g). According to POT, g does not reflect a general cognitive ability. Instead, g is the result of multiple domain-general executive attention processes and multiple domain-specific processes that are sampled in an overlapping manner across a battery of intelligence tests. POT explains several benchmark findings on human intelligence. However, the precise nature of the executive attention processes underlying g remains unclear. In the current paper, we discuss challenges associated with building a theory of individual differences in attention and intelligence. We argue that the conflation of psychological theories and statistical models, as well as problematic inferences based on latent variables, impedes research progress and prevents theory building. Two studies designed to illustrate the unique features of POT relative to previous approaches are presented. In Study 1, a simulation is presented to illustrate precisely how POT accounts for the relationship between executive attention processes and g. In Study 2, three datasets from previous studies are reanalyzed (N = 243, N = 234, N = 945) and reveal a discrepancy between the POT simulated model and the unity/diversity model of executive function. We suggest that this discrepancy is largely due to methodological problems in previous studies but also reflects different goals of research on individual differences in attention. The unity/diversity model is designed to facilitate research on executive function and dysfunction associated with cognitive and neural development and disease. POT is uniquely suited to guide and facilitate research on individual differences in cognitive ability and the investigation of executive attention processes underlying g.

The Relationship between Theory of Mind and Intelligence: A Formative g Approach.

Theory of Mind (ToM) is the ability understand that other people's mental states may be different from one's own. Psychometric models have shown that individual differences in ToM can largely be attributed to general intelligence (g) (Coyle et al. 2018). Most psychometric models specify g as a reflective latent variable, which is interpreted as a general ability that plays a causal role in a broad range of cognitive tasks, including ToM tasks. However, an alternative approach is to specify g as a formative latent variable, that is, an overall index of cognitive ability that does not represent a psychological attribute (Kovacs and Conway 2016). Here we consider a formative g approach to the relationship between ToM and intelligence. First, we conducted an SEM with reflective g to test the hypothesis that ToM is largely accounted for by a general ability. Next, we conducted a model with formative g to determine whether the relationship between ToM and intelligence is influenced by domain-specific tasks. Finally, we conducted a redundancy analysis to examine the contribution of each g variable. Results suggest that the relationship between ToM and intelligence in this study was influenced by language-based tasks, rather than solely a general ability.

Investigating the Structure of Intelligence Using Latent Variable and Psychometric Network Modeling: A Commentary and Reanalysis.

In a recent publication in the Journal of Intelligence, Dennis McFarland mischaracterized previous research using latent variable and psychometric network modeling to investigate the structure of intelligence. Misconceptions presented by McFarland are identified and discussed. We reiterate and clarify the goal of our previous research on network models, which is to improve compatibility between psychological theories and statistical models of intelligence. WAIS-IV data provided by McFarland were reanalyzed using latent variable and psychometric network modeling. The results are consistent with our previous study and show that a latent variable model and a network model both provide an adequate fit to the WAIS-IV. We therefore argue that model preference should be determined by theory compatibility. Theories of intelligence that posit a general mental ability (general intelligence) are compatible with latent variable models. More recent approaches, such as mutualism and process overlap theory, reject the notion of general mental ability and are therefore more compatible with network models, which depict the structure of intelligence as an interconnected network of cognitive processes sampled by a battery of tests. We emphasize the importance of compatibility between theories and models in scientific research on intelligence.

The Role of Non-Cognitive Factors in the SAT Remains Unclear: A Commentary on Hannon (2019).

In the current issue of the Journal of Intelligence, Hannon (2019) reports a novel and intriguing pattern of results that could be interpreted as evidence that the SAT is biased against Hispanic students. Specifically, Hannon's analyses suggest that non-cognitive factors, such as test anxiety, contribute to SAT performance and the impact of test anxiety on the SAT is stronger among Hispanic students than European-American students. Importantly, this pattern of results was observed after controlling for individual differences in cognitive abilities. We argue that there are multiple issues with Hannon's investigation and interpretation. For instance, Hannon did not include an adequate number or variety of measures of cognitive ability. In addition, the measure of test anxiety was a retrospective self-report survey on evaluated anxiety rather than a direct measure of situational test anxiety associated with the SAT. Based on these and other observations, we conclude that Hannon's current results do not provide sufficient evidence to suggest that non-cognitive factors play a significant role in the SAT or that they impact European-American and Hispanic students differently.

Psychometric Network Analysis of the Hungarian WAIS.

The positive manifold-the finding that cognitive ability measures demonstrate positive correlations with one another-has led to models of intelligence that include a general cognitive ability or general intelligence (g). This view has been reinforced using factor analysis and reflective, higher-order latent variable models. However, a new theory of intelligence, Process Overlap Theory (POT), posits that g is not a psychological attribute but an index of cognitive abilities that results from an interconnected network of cognitive processes. These competing theories of intelligence are compared using two different statistical modeling techniques: (a) latent variable modeling and (b) psychometric network analysis. Network models display partial correlations between pairs of observed variables that demonstrate direct relationships among observations. Secondary data analysis was conducted using the Hungarian Wechsler Adult Intelligence Scale Fourth Edition (H-WAIS-IV). The underlying structure of the H-WAIS-IV was first assessed using confirmatory factor analysis assuming a reflective, higher-order model and then reanalyzed using psychometric network analysis. The compatibility (or lack thereof) of these theoretical accounts of intelligence with the data are discussed.

The evolution of fluid intelligence meets formative g.

The argument by Burkart et al. in the target article relates to fluid (not general) intelligence: a domain-general ability involved in complex, novel problem solving, and strongly related to working memory and executive functions. A formative framework, under which the general factor of intelligence is the common consequence, not the common cause of the covariance among tests is more in line with an evolutionary approach.

Inducing Proactive Control Shifts in the AX-CPT.

The Dual Mechanisms of Control (DMC) account (Braver, 2012) proposes two distinct mechanisms of cognitive control, proactive and reactive. This account has been supported by a large number of studies using the AX-CPT paradigm that have demonstrated not only between-group differences, but also within-subjects variability in the use of the two control mechanisms. Yet there has been little investigation of task manipulations that can experimentally modulate the use of proactive control in healthy young adults; such manipulations could be useful to better understand the workings of cognitive control mechanisms. In the current study, a series of three experiments demonstrate how individuals can be systematically biased toward and away from the utilization of proactive control, via strategy training and no-go manipulations, respectively. These results provide increased support for the DMC framework, and provide a new basis from which to examine group-based differences and neural mechanisms underlying the two control modes.

Phonological similarity in working memory span tasks.

In a series of four experiments, we explored what conditions are sufficient to produce a phonological similarity facilitation effect in working memory span tasks. By using the same set of memoranda, but differing the secondary-task requirements across experiments, we showed that a phonological similarity facilitation effect is dependent upon the semantic relationship between the memoranda and the secondary-task stimuli, and is robust to changes in the representation, ordering, and pool size of the secondary-task stimuli. These findings are consistent with interference accounts of memory (Brown, Neath, & Chater, Psychological Review, 114, 539-576, 2007; Oberauer, Lewandowsky, Farrell, Jarrold, & Greaves, Psychonomic Bulletin & Review, 19, 779-819, 2012), whereby rhyming stimuli provide a form of categorical similarity that allows distractors to be excluded from retrieval at recall.

New and emerging models of human intelligence.

In the last decade, new models of human intelligence have altered the theoretical landscape in psychometrics and cognitive science. In the current article, we provide an overview of key distinguishing features of these new models. Compared to 20th century models of intelligence, the new models proposed in the 21st century are unique for three primary reasons; (1) new models interpret the general factor, or g, as an emergent property reflecting the pattern of positive correlations observed among test scores, not as a causal latent variable, and therefore challenge the notion of general ability, (2) new models bridge correlational and experimental psychology and account for inter-individual differences in behavior in terms of intra-individual psychological processes, and (3) new models make novel predictions about the neural correlates of intelligent behavior.

An ecological approach to cognitive enhancement: complex motor training.

Cognitive training has received a lot of attention recently, yielding findings that can be conflicting and controversial. In this paper, we present a novel approach to cognitive training based on complex motor activities. In a randomized controlled design, participants were assigned to one of three conditions: aerobic exercise, working memory training or designed sport--an intervention specifically tailored to include both physical and cognitive demands. After training for eight weeks, the designed sport group showed the largest gains in all cognitive measures, illustrating the efficacy of complex motor activities to enhance cognition. Designed sport training also revealed impressive health benefits, namely decreased heart rate and blood pressure. In this period of skepticism over the efficacy of computerized cognitive training, we discuss the potential of ecological interventions targeting both cognition and physical fitness, and propose some possible applications.

The scope and control of attention: Sources of variance in working memory capacity.

Working memory capacity is a strong positive predictor of many cognitive abilities, across various domains. The pattern of positive correlations across domains has been interpreted as evidence for a unitary source of inter-individual differences in behavior. However, recent work suggests that there are multiple sources of variance contributing to working memory capacity. The current study (N = 71) investigates individual differences in the scope and control of attention, in addition to the number and resolution of items maintained in working memory. Latent variable analyses indicate that the scope and control of attention reflect independent sources of variance and each account for unique variance in general intelligence. Also, estimates of the number of items maintained in working memory are consistent across tasks and related to general intelligence whereas estimates of resolution are task-dependent and not predictive of intelligence. These results provide insight into the structure of working memory, as well as intelligence, and raise new questions about the distinction between number and resolution in visual short-term memory.

Sequential dynamics in visual short-term memory.

Visual short-term memory (VSTM) is thought to help bridge across changes in visual input, and yet many studies of VSTM employ static displays. Here we investigate how VSTM copes with sequential input. In particular, we characterize the temporal dynamics of several different components of VSTM performance, including: storage probability, precision, variability in precision, guessing, and swapping. We used a variant of the continuous-report VSTM task developed for static displays, quantifying the contribution of each component with statistical likelihood estimation, as a function of serial position and set size. In Experiments 1 and 2, storage probability did not vary by serial position for small set sizes, but showed a small primacy effect and a robust recency effect for larger set sizes; precision did not vary by serial position or set size. In Experiment 3, the recency effect was shown to reflect an increased likelihood of swapping out items from earlier serial positions and swapping in later items, rather than an increased rate of guessing for earlier items. Indeed, a model that incorporated responding to non-targets provided a better fit to these data than alternative models that did not allow for swapping or that tried to account for variable precision. These findings suggest that VSTM is updated in a first-in-first-out manner, and they bring VSTM research into closer alignment with classical working memory research that focuses on sequential behavior and interference effects.

The case for an ecological approach to cognitive training.

Strong claims have been made about the efficacy of cognitive training. In particular, the idea that working memory (WM) training enhances intelligence initially generated enthusiasm but, on further inspection, is now met with skepticism. In our view, this is an unfortunate setback due to inappropriate task design and does not refute the core idea of cognitive enhancement. Growing evidence suggests that successful training programs integrate complexity, novelty, and diversity to maximize ecological validity.

Capacity estimates in working memory: Reliability and interrelationships among tasks.

The concept of capacity has become increasingly important in discussions of working memory (WM), in so far as most models of WM conceptualize it as a limited-capacity mechanism for maintaining information in an active state, and as capacity estimates from at least one type of WM task-complex span-are valid predictors of real-world cognitive performance. However, the term capacity is also often used in the context of a distinct set of WM tasks, change detection, and may or may not refer to the same cognitive capability. We here develop maximum-likelihood models of capacity from each of these tasks-as well as from a third WM task that places heavy demands on cognitive control, the self-ordered WM task (SOT)-and show that the capacity estimates from change detection and complex span tasks are not correlated with each other, although capacity estimates from change detection tasks do correlate with those from the SOT. Furthermore, exploratory factor analysis confirmed that performance on the SOT and change detection load on the same factor, with performance on our complex span task loading on its own factor. These findings suggest that at least two distinct cognitive capabilities underlie the concept of WM capacity as it applies to each of these three tasks.