Revisiting the remember-know task: Replications of Gardiner and Java (1990).

One of the most evidential behavioral results for two memory processes comes from Gardiner and Java (Memory & Cognition, 18, 23-30 1990). Participants provided more "remember" than "know" responses for old words but more know than remember responses for old nonwords. Moreover, there was no effect of word/nonword status for new items. The combination of a crossover interaction for old items with an invariance for new items provides strong evidence for two distinct processes while ruling out criteria or bias explanations. Here, we report a modern replication of this study. In three experiments, (Experiments 1, 2, and 4) with larger numbers of items and participants, we were unable to replicate the crossover. Instead, our data are more consistent with a single-process account. In a fourth experiment (Experiment 3), we were able to replicate Gardiner and Java's baseline results with a sure-unsure paradigm supporting a single-process explanation. It seems that Gardiner and Java's remarkable crossover result is not replicable.

Are divergence point analyses suitable for response time data?

Estimating the time course of the influence of different factors in human performance is one of the major topics of research in cognitive psychology/neuroscience. Over the past decades, researchers have proposed several methods to tackle this question using latency data. Here we examine a recently proposed procedure that employs survival analyses on latency data to provide precise estimates of the timing of the first discernible influence of a given factor (e.g., word frequency on lexical access) on performance (e.g., fixation durations or response times). A number of articles have used this method in recent years, and hence an exploration of its strengths and its potential weaknesses is in order. Unfortunately, our analysis revealed that the technique has conceptual flaws, and it might lead researchers into believing that they are obtaining a measurement of processing components when, in fact, they are obtaining an uninterpretable measurement.

Null Effects of Game Violence, Game Difficulty, and 2D:4D Digit Ratio on Aggressive Behavior.

Researchers have suggested that acute exposure to violent video games is a cause of aggressive behavior. We tested this hypothesis by using violent and nonviolent games that were closely matched, collecting a large sample, and using a single outcome. We randomly assigned 275 male undergraduates to play a first-person-shooter game modified to be either violent or less violent and hard or easy. After completing the game-play session, participants were provoked by a confederate and given an opportunity to behave aggressively. Neither game violence nor game difficulty predicted aggressive behavior. Incidentally, we found that 2D:4D digit ratio, thought to index prenatal testosterone exposure, did not predict aggressive behavior. Results do not support acute violent-game exposure and low 2D:4D ratio as causes of aggressive behavior.

Calibrated Bayes Factors Should Not Be Used: A Reply to Hoijtink, van Kooten, and Hulsker.

Hoijtink, Kooten, and Hulsker ( 2016 ) present a method for choosing the prior distribution for an analysis with Bayes factor that is based on controlling error rates, which they advocate as an alternative to our more subjective methods (Morey & Rouder, 2014 ; Rouder, Speckman, Sun, Morey, & Iverson, 2009 ; Wagenmakers, Wetzels, Borsboom, & van der Maas, 2011 ). We show that the method they advocate amounts to a simple significance test, and that the resulting Bayes factors are not interpretable. Additionally, their method fails in common circumstances, and has the potential to yield arbitrarily high Type II error rates. After critiquing their method, we outline the position on subjectivity that underlies our advocacy of Bayes factors.

Developmental Foundations of Children's Fraction Magnitude Knowledge.

The conceptual insight that fractions represent magnitudes is a critical yet daunting step in children's mathematical development, and the knowledge of fraction magnitudes influences children's later mathematics learning including algebra. In this study, longitudinal data were analyzed to identify the mathematical knowledge and domain-general competencies that predicted 8th and 9th graders' (n=122) knowledge of fraction magnitudes and its cross-grade gains. Performance on the fraction magnitude measures predicted 9th grade algebra achievement. Understanding and fluently identifying the numerator-denominator relation in 7th grade emerged as the key predictor of later fraction magnitudes knowledge in both 8th and 9th grades. Competence at using fraction procedures, knowledge of whole number magnitudes, and the central executive contributed to 9th but not 8th graders' fraction magnitude knowledge, and knowledge of whole number magnitude contributed to cross-grade gains. The key results suggest fluent processing of numerator-denominator relations presages students' understanding of fractions as magnitudes and that the integration of whole number and fraction magnitudes occurs gradually.

The what, why, and how of born-open data.

Although many researchers agree that scientific data should be open to scrutiny to ferret out poor analyses and outright fraud, most raw data sets are not available on demand. There are many reasons researchers do not open their data, and one is technical. It is often time consuming to prepare and archive data. In response, my laboratory has automated the process such that our data are archived the night they are created without any human approval or action. All data are versioned, logged, time stamped, and uploaded including aborted runs and data from pilot subjects. The archive is GitHub, github.com, the world's largest collection of open-source materials. Data archived in this manner are called born open. In this paper, I discuss the benefits of born-open data and provide a brief technical overview of the process. I also address some of the common concerns about opening data before publication.

Effects of Violent-Video-Game Exposure on Aggressive Behavior, Aggressive-Thought Accessibility, and Aggressive Affect Among Adults With and Without Autism Spectrum Disorder.

Recent mass shootings have prompted the idea among some members of the public that exposure to violent video games can have a pronounced effect on individuals with autism spectrum disorder (ASD). Empirical evidence for or against this claim has been missing, however. To address this issue, we assigned adults with and without ASD to play a violent or nonviolent version of a customized first-person shooter video game. After they played the game, we assessed three aggression-related outcome variables (aggressive behavior, aggressive-thought accessibility, and aggressive affect). Results showed strong evidence that adults with ASD, compared with typically developing adults, are not differentially affected by acute exposure to violent video games. Moreover, model comparisons provided modest evidence against any effect of violent game content whatsoever. Findings from this experiment suggest that societal concerns that exposure to violent games may have a unique effect on adults with autism are not supported by evidence.

Individual differences in algebraic cognition: Relation to the approximate number and semantic memory systems.

The relation between performance on measures of algebraic cognition and acuity of the approximate number system (ANS) and memory for addition facts was assessed for 171 ninth graders (92 girls) while controlling for parental education, sex, reading achievement, speed of numeral processing, fluency of symbolic number processing, intelligence, and the central executive component of working memory. The algebraic tasks assessed accuracy in placing x,y pairs in the coordinate plane, speed and accuracy of expression evaluation, and schema memory for algebra equations. ANS acuity was related to accuracy of placements in the coordinate plane and expression evaluation but not to schema memory. Frequency of fact retrieval errors was related to schema memory but not to coordinate plane or expression evaluation accuracy. The results suggest that the ANS may contribute to or be influenced by spatial-numerical and numerical-only quantity judgments in algebraic contexts, whereas difficulties in committing addition facts to long-term memory may presage slow formation of memories for the basic structure of algebra equations. More generally, the results suggest that different brain and cognitive systems are engaged during the learning of different components of algebraic competence while controlling for demographic and domain general abilities.

Evidence for discrete-state processing in recognition memory.

We provide evidence that recognition memory is mediated by a detect-or-guess mental-state model without recourse to concepts of latent-strength or multiple-memory systems. We assess performance in a two-alternative forced-choice recognition memory task with confidence ratings. The key manipulation is that sometimes participants are asked which of two new items is old, and the resulting confidence distribution is unambiguously interpreted as arising from a guessing state. The confidence ratings for other conditions are seemingly the resultant of mixing this stable guessing state with an additional stable detect state. Formal model comparison supports this observation, and an analysis of associated response times reveals a mixture signature as well.

A power fallacy.

The power fallacy refers to the misconception that what holds on average -across an ensemble of hypothetical experiments- also holds for each case individually. According to the fallacy, high-power experiments always yield more informative data than do low-power experiments. Here we expose the fallacy with concrete examples, demonstrating that a particular outcome from a high-power experiment can be completely uninformative, whereas a particular outcome from a low-power experiment can be highly informative. Although power is useful in planning an experiment, it is less useful-and sometimes even misleading-for making inferences from observed data. To make inferences from data, we recommend the use of likelihood ratios or Bayes factors, which are the extension of likelihood ratios beyond point hypotheses. These methods of inference do not average over hypothetical replications of an experiment, but instead condition on the data that have actually been observed. In this way, likelihood ratios and Bayes factors rationally quantify the evidence that a particular data set provides for or against the null or any other hypothesis.

Children's cognitive representation of the mathematical number line.

Learning of the mathematical number line has been hypothesized to be dependent on an inherent sense of approximate quantity. Children's number line placements are predicted to conform to the underlying properties of this system; specifically, placements are exaggerated for small numerals and compressed for larger ones. Alternative hypotheses are based on proportional reasoning; specifically, numerals are placed relative to set anchors such as end points on the line. Traditional testing of these alternatives involves fitting group medians to corresponding regression models which assumes homogenous residuals and thus does not capture useful information from between- and within-child variation in placements across the number line. To more fully assess differential predictions, we developed a novel set of hierarchical statistical models that enable the simultaneous estimation of mean levels of and variation in performance, as well as developmental transitions. Using these techniques we fitted the number line placements of 224 children longitudinally assessed from first to fifth grade, inclusive. The compression pattern was evident in mean performance in first grade, but was the best fit for only 20% of first graders when the full range of variation in the data are modeled. Most first graders' placements suggested use of end points, consistent with proportional reasoning. Developmental transition involved incorporation of a mid-point anchor, consistent with a modified proportional reasoning strategy. The methodology introduced here enables a more nuanced assessment of children's number line representation and learning than any previous approaches and indicates that developmental improvement largely results from midpoint segmentation of the line.

Does every study? Implementing ordinal constraint in meta-analysis.

The most prominent goal when conducting a meta-analysis is to estimate the true effect size across a set of studies. This approach is problematic whenever the analyzed studies have qualitatively different results; that is, some studies show an effect in the predicted direction while others show no effect and still others show an effect in the opposite direction. In case of such qualitative differences, the average effect may be a product of different mechanisms, and therefore uninterpretable. The first question in any meta-analysis should therefore be whether all studies show an effect in the same, expected direction. To tackle this question a model with ordinal constraints is proposed where the ordinal constraint holds each study in the set. This "every study" model is compared with a set of alternative models, such as an unconstrained model that predicts effects in both directions. If the ordinal constraints hold, one underlying mechanism may suffice to explain the results from all studies, and this result could be supported by reduced between-study heterogeneity. A major implication is then that average effects become interpretable. We illustrate the model comparison approach using Carbajal et al.'s (2021) meta-analysis on the familiar-word-recognition effect, show how predictor analyses can be incorporated in the approach, and provide R-code for interested researchers. As common in meta-analysis, only surface statistics (such as effect size and sample size) are provided from each study, and the modeling approach can be adapted to suit these conditions. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Why many studies of individual differences with inhibition tasks may not localize correlations.

Individual difference exploration of cognitive domains is predicated on being able to ascertain how well performance on tasks covary. Yet, establishing correlations among common inhibition tasks such as Stroop or flanker tasks has proven quite difficult. It remains unclear whether this difficulty occurs because there truly is a lack of correlation or whether analytic techniques to localize correlations perform poorly real-world contexts because of excessive measurement error from trial noise. In this paper, we explore how well correlations may localized in large data sets with many people, tasks, and replicate trials. Using hierarchical models to separate trial noise from true individual variability, we show that trial noise in 24 extant tasks is about 8 times greater than individual variability. This degree of trial noise results in massive attenuation in correlations and instability in Spearman corrections. We then develop hierarchical models that account for variation across trials, variation across individuals, and covariation across individuals and tasks. These hierarchical models also perform poorly in localizing correlations. The advantage of these models is not in estimation efficiency, but in providing a sense of uncertainty so that researchers are less likely to misinterpret variability in their data. We discuss possible improvements to study designs to help localize correlations.

An assessment of fixed-capacity models of visual working memory.

Visual working memory is often modeled as having a fixed number of slots. We test this model by assessing the receiver operating characteristics (ROC) of participants in a visual-working-memory change-detection task. ROC plots yielded straight lines with a slope of 1.0, a tell-tale characteristic of all-or-none mnemonic representations. Formal model assessment yielded evidence highly consistent with a discrete fixed-capacity model of working memory for this task.

Are There Reliable Qualitative Individual Difference in Cognition?

In this paper we propose a new set of questions that focus on the direction of effects. In almost all studies the direction is important. For example, in a Stroop task we expect responses to incongruent items to be slower than those to congruent ones, and this direction implies one theoretical explanation. Yet, if congruent words are slowed down relative to incongruent words we would have a completely different theoretical explanation. We ask a 'does everybody' question, such as, 'does every individual show a Stroop effect in the same direction?' Or, 'does every individual respond faster to loud tones than soft tones?' If all individuals truly have effects in the same direction that implicate a common theory, we term the differences among them as quantitative individual differences. Conversely, if all individuals truly have effects in different directions that implicate different theories, we term the differences among them as qualitative individual differences. Here, we provide a users guide to the question of whether individual differences are qualitative or quantitative. We discuss theoretical issues, methodological advances, new software for assessment, and, most importantly, how the question impacts theory development in cognitive science. Our hope is that this mode of analysis is a productive tool in researchers' toolkits.

The truth revisited: Bayesian analysis of individual differences in the truth effect.

The repetition-induced truth effect refers to a phenomenon where people rate repeated statements as more likely true than novel statements. In this paper, we document qualitative individual differences in the effect. While the overwhelming majority of participants display the usual positive truth effect, a minority are the opposite-they reliably discount the validity of repeated statements, what we refer to as negative truth effect. We examine eight truth-effect data sets where individual-level data are curated. These sets are composed of 1105 individuals performing 38,904 judgments. Through Bayes factor model comparison, we show that reliable negative truth effects occur in five of the eight data sets. The negative truth effect is informative because it seems unreasonable that the mechanisms mediating the positive truth effect are the same that lead to a discounting of repeated statements' validity. Moreover, the presence of qualitative differences motivates a different type of analysis of individual differences based on ordinal (i.e., Which sign does the effect have?) rather than metric measures. To our knowledge, this paper reports the first such reliable qualitative differences in a cognitive task.

Beneath the surface: Unearthing within-person variability and mean relations with Bayesian mixed models.

Mixed-effects models are becoming common in psychological science. Although they have many desirable features, there is still untapped potential. It is customary to view homogeneous variance as an assumption to satisfy. We argue to move beyond that perspective, and to view modeling within-person variance as an opportunity to gain a richer understanding of psychological processes. The technique to do so is based on the mixed-effects location scale model that can simultaneously estimate mixed-effects submodels to both the mean (location) and within-person variance (scale). We develop a framework that goes beyond assessing the submodels in isolation of one another and introduce a novel Bayesian hypothesis test for mean-variance correlations in the distribution of random effects. We first present a motivating example, which makes clear how the model can characterize mean-variance relations. We then apply the method to reaction times (RTs) gathered from 2 cognitive inhibition tasks. We find there are more individual differences in the within-person variance than the mean structure, as well as a complex web of structural mean-variance relations. This stands in contrast to the dominant view of within-person variance (i.e., "noise"). The results also point toward paradoxical within-person, as opposed to between-person, effects: several people had slower and less variable incongruent responses. This contradicts the typical pattern, wherein larger means tend to be associated with more variability. We conclude with future directions, spanning from methodological to theoretical inquires, that can be answered with the presented methodology. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Detection of patient risk by nurses: a theoretical framework.

AIM: This paper is a description of a theoretical framework of how nurses detect and interpret patient risk signals in the context of organizational attitudes and procedures related to patient safety. BACKGROUND: The ability to detect when patients are at increased risk for harm is a challenge faced by nurses worldwide. How nurses are able to discriminate patient risk warning signals from background noise is not well understood. Also, the impact of system-level factors on nurses' signal detection capabilities has not been investigated. DATA SOURCES: Computerized database searches were used to identify nursing, organizational science, and cognitive psychology literature from 1964 to 2009 pertinent to the framework. DISCUSSION: The patient risk detection theory synthesizes concepts of signal detection theory and high reliability theory. Signal detection theory explains the decision-making processes of nurses as they scan for signals of potential patient harm. High reliability theory explains how nurses' signal detection capacities are facilitated when healthcare settings operate as high reliability organizations making patient safety the top priority. CONCLUSION: The patient risk detection theory facilitates understanding of both individual and organizational factors that influence nurses' ability to detect risk in complex healthcare settings. It can be used to guide research on interventions to enhance signal detection by nurses and increase patient safety in today's complex care environments. The theory can also be used to guide design of training programmes that permit nurses to develop practical skills in signal detection.

The nature of psychological thresholds.

Following G. T. Fechner (1966), thresholds have been conceptualized as the amount of intensity needed to transition between mental states, such as between states of unconsciousness and consciousness. With the advent of the theory of signal detection, however, discrete-state theory and the corresponding notion of threshold have been discounted. Consequently, phenomena such as subliminal priming and perception have a reduced theoretical basis. The authors propose a process-neutral definition of threshold that allows for graded perception and activation throughout the system. Thresholds correspond to maximum stimulus intensities such that the distribution of mental states does not differ from that when an appropriate baseline stimulus is presented. In practice, thresholds are maximum intensities such that the probability distribution on behavioral events does not differ from that from baseline. These thresholds, which the authors call task thresholds, may be estimated with modified item response psychometric measurement models.