Most quantifiers have many meanings.

In this paper, we investigate, by means of a computational model, how individuals map quantifiers onto numbers and how they order quantifiers on a mental line. We selected five English quantifiers (few, fewer than half, many, more than half, and most) which differ in truth conditions and vagueness. We collected binary truth value judgment data in an online quantifier verification experiment. Using a Bayesian three-parameter logistic regression model, we separated three sources of individual differences: truth condition, vagueness, and response error. Clustering on one of the model's parameter that corresponds to truth conditions revealed four subgroups of participants with different quantifier-to-number mappings and different ranges of the mental line of quantifiers. Our findings suggest multiple sources of individual differences in semantic representations of quantifiers and support a conceptual distinction between different types of imprecision in quantifier meanings. We discuss the consequence of our findings for the main theoretical approaches to quantifiers: the bivalent truth-conditional approach and the fuzzy logic approach. We argue that the former approach neither can explain inter-individual differences nor intra-individual differences in truth conditions of vague quantifiers. The latter approach requires further specification to fully account for individual differences demonstrated in this study.

Refining Bayesian hierarchical MPT modeling: Integrating prior knowledge and ordinal expectations.

Multinomial processing tree (MPT) models are a broad class of statistical models used to test sophisticated psychological theories. The research questions derived from these theories often go beyond simple condition effects on parameters and involve ordinal expectations (e.g., the same-direction effect on the memory parameter is stronger in one experimental condition than another) or disordinal expectations (e.g., the effect reverses in one experimental condition). Here, we argue that by refining common modeling practices, Bayesian hierarchical models are well suited to estimate and test these expectations. Concretely, we show that the default priors proposed in the literature lead to nonsensical predictions for individuals and the population distribution, leading to problems not only in model comparison but also in parameter estimation. Rather than relying on these priors, we argue that MPT modelers should determine priors that are consistent with their theoretical knowledge. In addition, we demonstrate how Bayesian model comparison may be used to test ordinal and disordinal interactions by means of Bayes factors. We apply the techniques discussed to empirical data from Bell et al. Journal of Experimental Psychology: Learning, Memory, and Cognition, 41, 456-472 (2015).

A tutorial on Bayesian model-averaged meta-analysis in JASP.

Researchers conduct meta-analyses in order to synthesize information across different studies. Compared to standard meta-analytic methods, Bayesian model-averaged meta-analysis offers several practical advantages including the ability to quantify evidence in favor of the absence of an effect, the ability to monitor evidence as individual studies accumulate indefinitely, and the ability to draw inferences based on multiple models simultaneously. This tutorial introduces the concepts and logic underlying Bayesian model-averaged meta-analysis and illustrates its application using the open-source software JASP. As a running example, we perform a Bayesian meta-analysis on language development in children. We show how to conduct a Bayesian model-averaged meta-analysis and how to interpret the results.

"This behavior strikes us as ideal": assessment and anticipations of Huisman (2022).

Huisman (Psychonomic Bulletin & Review, 1-10. 2022) argued that a valid measure of evidence should indicate more support in favor of a true alternative hypothesis when sample size is large than when it is small. Bayes factors may violate this pattern and hence Huisman concluded that Bayes factors are invalid as a measure of evidence. In this brief comment we call attention to the following: (1) Huisman's purported anomaly is in fact dictated by probability theory; (2) Huisman's anomaly has been discussed and explained in the statistical literature since 1939; the anomaly was also highlighted in the Psychonomic Bulletin & Review article by Rouder et al. (2009), who interpreted the anomaly as "ideal": an interpretation diametrically opposed to that of Huisman. We conclude that when intuition clashes with probability theory, chances are that it is intuition that needs schooling.

Predictions and rewards affect decision-making but not subjective experience.

To survive, organisms constantly make decisions to avoid danger and maximize rewards in information-rich environments. As a result, decisions about sensory input are not only driven by sensory information but also by other factors, such as the expected rewards of a decision (known as the payoff matrix) or by information about temporal regularities in the environment (known as cognitive priors or predictions). However, it is unknown to what extent these different types of information affect subjective experience or whether they merely result in nonperceptual response criterion shifts. To investigate this question, we used three carefully matched manipulations that typically result in behavioral shifts in decision criteria: a visual illusion (Müller-Lyer condition), a punishment scheme (payoff condition), and a change in the ratio of relevant stimuli (base rate condition). To gauge shifts in subjective experience, we introduce a task in which participants not only make decisions about what they have just seen but are also asked to reproduce their experience of a target stimulus. Using Bayesian ordinal modeling, we show that each of these three manipulations affects the decision criterion as intended but that the visual illusion uniquely affects sensory experience as measured by reproduction. In a series of follow-up experiments, we use computational modeling to show that although the visual illusion results in a distinct drift-diffusion (DDM) parameter profile relative to nonsensory manipulations, reliance on DDM parameter estimates alone is not sufficient to ascertain whether a given manipulation is perceptual or nonperceptual.

Bayesian hierarchical modeling: an introduction and reassessment.

With the recent development of easy-to-use tools for Bayesian analysis, psychologists have started to embrace Bayesian hierarchical modeling. Bayesian hierarchical models provide an intuitive account of inter- and intraindividual variability and are particularly suited for the evaluation of repeated-measures designs. Here, we provide guidance for model specification and interpretation in Bayesian hierarchical modeling and describe common pitfalls that can arise in the process of model fitting and evaluation. Our introduction gives particular emphasis to prior specification and prior sensitivity, as well as to the calculation of Bayes factors for model comparisons. We illustrate the use of state-of-the-art software programs Stan and brms. The result is an overview of best practices in Bayesian hierarchical modeling that we hope will aid psychologists in making the best use of Bayesian hierarchical modeling.

Why many studies of individual differences with inhibition tasks may not localize correlations.

Individual difference exploration of cognitive domains is predicated on being able to ascertain how well performance on tasks covary. Yet, establishing correlations among common inhibition tasks such as Stroop or flanker tasks has proven quite difficult. It remains unclear whether this difficulty occurs because there truly is a lack of correlation or whether analytic techniques to localize correlations perform poorly real-world contexts because of excessive measurement error from trial noise. In this paper, we explore how well correlations may localized in large data sets with many people, tasks, and replicate trials. Using hierarchical models to separate trial noise from true individual variability, we show that trial noise in 24 extant tasks is about 8 times greater than individual variability. This degree of trial noise results in massive attenuation in correlations and instability in Spearman corrections. We then develop hierarchical models that account for variation across trials, variation across individuals, and covariation across individuals and tasks. These hierarchical models also perform poorly in localizing correlations. The advantage of these models is not in estimation efficiency, but in providing a sense of uncertainty so that researchers are less likely to misinterpret variability in their data. We discuss possible improvements to study designs to help localize correlations.

Multibridge: an R package to evaluate informed hypotheses in binomial and multinomial models.

The multibridge R package allows a Bayesian evaluation of informed hypotheses [Formula: see text] applied to frequency data from an independent binomial or multinomial distribution. multibridge uses bridge sampling to efficiently compute Bayes factors for the following hypotheses concerning the latent category proportions 𝜃: (a) hypotheses that postulate equality constraints (e.g., 𝜃1 = 𝜃2 = 𝜃3); (b) hypotheses that postulate inequality constraints (e.g., 𝜃1 < 𝜃2 < 𝜃3 or 𝜃1 > 𝜃2 > 𝜃3); (c) hypotheses that postulate combinations of inequality constraints and equality constraints (e.g., 𝜃1 < 𝜃2 = 𝜃3); and (d) hypotheses that postulate combinations of (a)-(c) (e.g., 𝜃1 < (𝜃2 = 𝜃3),𝜃4). Any informed hypothesis [Formula: see text] may be compared against the encompassing hypothesis [Formula: see text] that all category proportions vary freely, or against the null hypothesis [Formula: see text] that all category proportions are equal. multibridge facilitates the fast and accurate comparison of large models with many constraints and models for which relatively little posterior mass falls in the restricted parameter space. This paper describes the underlying methodology and illustrates the use of multibridge through fully reproducible examples.

Bayes Factors for Mixed Models: Perspective on Responses.

In van Doorn et al. (2021), we outlined a series of open questions concerning Bayes factors for mixed effects model comparison, with an emphasis on the impact of aggregation, the effect of measurement error, the choice of prior distributions, and the detection of interactions. Seven expert commentaries (partially) addressed these initial questions. Surprisingly perhaps, the experts disagreed (often strongly) on what is best practice-a testament to the intricacy of conducting a mixed effect model comparison. Here, we provide our perspective on these comments and highlight topics that warrant further discussion. In general, we agree with many of the commentaries that in order to take full advantage of Bayesian mixed model comparison, it is important to be aware of the specific assumptions that underlie the to-be-compared models.

Does every study? Implementing ordinal constraint in meta-analysis.

The most prominent goal when conducting a meta-analysis is to estimate the true effect size across a set of studies. This approach is problematic whenever the analyzed studies have qualitatively different results; that is, some studies show an effect in the predicted direction while others show no effect and still others show an effect in the opposite direction. In case of such qualitative differences, the average effect may be a product of different mechanisms, and therefore uninterpretable. The first question in any meta-analysis should therefore be whether all studies show an effect in the same, expected direction. To tackle this question a model with ordinal constraints is proposed where the ordinal constraint holds each study in the set. This "every study" model is compared with a set of alternative models, such as an unconstrained model that predicts effects in both directions. If the ordinal constraints hold, one underlying mechanism may suffice to explain the results from all studies, and this result could be supported by reduced between-study heterogeneity. A major implication is then that average effects become interpretable. We illustrate the model comparison approach using Carbajal et al.'s (2021) meta-analysis on the familiar-word-recognition effect, show how predictor analyses can be incorporated in the approach, and provide R-code for interested researchers. As common in meta-analysis, only surface statistics (such as effect size and sample size) are provided from each study, and the modeling approach can be adapted to suit these conditions. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Evaluating multinomial order restrictions with bridge sampling.

Hypotheses concerning the distribution of multinomial proportions typically entail exact equality constraints that can be evaluated using standard tests. Whenever researchers formulate inequality constrained hypotheses, however, they must rely on sampling-based methods that are relatively inefficient and computationally expensive. To address this problem we developed a bridge sampling routine that allows an efficient evaluation of multinomial inequality constraints. An empirical application showcases that bridge sampling outperforms current Bayesian methods, especially when relatively little posterior mass falls in the restricted parameter space. The method is extended to mixtures between equality and inequality constrained hypotheses. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Creative or not? Hierarchical diffusion modeling of the creative evaluation process.

When producing creative ideas (i.e., ideas that are original and useful) two main processes occur: ideation, where people brainstorm ideas, and evaluation, where they decide if the ideas are creative or not. While much is known about the ideation phase, the cognitive processes involved in creativity evaluation are less clear. In this article, we present a novel modeling approach for the evaluation phase of creativity. We apply the drift diffusion model (DDM) to the Creative-or-Not task (CON-task) to study the cognitive basis of evaluation and to examine individual differences in the extent to which people take originality and utility into account when evaluating creative ideas. The CON-task is a timed decision-making task where participants indicate whether they find uses for certain objects creative or not (e.g., using a book as a buoy). The different uses vary on the two creativity dimensions "originality" and "utility." In two studies (n = 293, 17,806 trials; n = 152, 9,291 trials), we found that stimulus originality was strongly related to participants' drift rates but found only weak evidence for an association between stimulus utility and the drift rate. However, participants differed substantially in the effects of originality and utility. Furthermore, the implicit weights assigned to originality and utility on the CON-task were associated with self-reported importance ratings of originality and utility and with divergent thinking performance in the Alternative Uses task (AUT). This research provides a cognitive modeling approach to creativity evaluation and underlines the importance of communicating rating criteria in divergent thinking tasks to ensure a fair assessment of creative ability. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Informed Bayesian survival analysis.

BACKGROUND: We provide an overview of Bayesian estimation, hypothesis testing, and model-averaging and illustrate how they benefit parametric survival analysis. We contrast the Bayesian framework to the currently dominant frequentist approach and highlight advantages, such as seamless incorporation of historical data, continuous monitoring of evidence, and incorporating uncertainty about the true data generating process. METHODS: We illustrate the application of the outlined Bayesian approaches on an example data set, retrospective re-analyzing a colon cancer trial. We assess the performance of Bayesian parametric survival analysis and maximum likelihood survival models with AIC/BIC model selection in fixed-n and sequential designs with a simulation study. RESULTS: In the retrospective re-analysis of the example data set, the Bayesian framework provided evidence for the absence of a positive treatment effect of adding Cetuximab to FOLFOX6 regimen on disease-free survival in patients with resected stage III colon cancer. Furthermore, the Bayesian sequential analysis would have terminated the trial 10.3 months earlier than the standard frequentist analysis. In a simulation study with sequential designs, the Bayesian framework on average reached a decision in almost half the time required by the frequentist counterparts, while maintaining the same power, and an appropriate false-positive rate. Under model misspecification, the Bayesian framework resulted in higher false-negative rate compared to the frequentist counterparts, which resulted in a higher proportion of undecided trials. In fixed-n designs, the Bayesian framework showed slightly higher power, slightly elevated error rates, and lower bias and RMSE when estimating treatment effects in small samples. We found no noticeable differences for survival predictions. We have made the analytic approach readily available to other researchers in the RoBSA R package. CONCLUSIONS: The outlined Bayesian framework provides several benefits when applied to parametric survival analyses. It uses data more efficiently, is capable of considerably shortening the length of clinical trials, and provides a richer set of inferences.

Same stimulus, same temporal context, different percept? Individual differences in hysteresis and adaptation when perceiving multistable dot lattices.

How we perceptually organize a visual stimulus depends not only on the stimulus itself, but also on the temporal and spatial context in which the stimulus is presented and on the individual processing the stimulus and context. Earlier research found both attractive and repulsive context effects in perception: tendencies to organize visual input similarly to preceding context stimuli (i.e., hysteresis, attraction) co-exist with tendencies that repel the current percept from the organization that is most dominant in these contextual stimuli (i.e., adaptation, repulsion). These processes have been studied mostly on a group level (e.g., Schwiedrzik et al., 2014). Using a Bayesian hierarchical model comparison approach, the present study (N = 75) investigated whether consistent individual differences exist in these attractive and repulsive temporal context effects, with multistable dot lattices as stimuli. In addition, the temporal stability of these individual differences in context effects was investigated, and it was studied how the strength of these effects related to the strength of individual biases for absolute orientations. The results demonstrate that large individual differences in the size of attractive and repulsive context effects exist. Furthermore, these individual differences are highly consistent across timepoints (one to two weeks apart). Although almost everyone showed both effects in the expected direction, not every single individual did. In sum, the study reveals differences in how individuals combine previous input and experience with current input in their perception, and more generally, this teaches us that different individuals can perceive identical stimuli differently, even within a similar context.

The Einstein effect provides global evidence for scientific source credibility effects and the influence of religiosity.

People tend to evaluate information from reliable sources more favourably, but it is unclear exactly how perceivers' worldviews interact with this source credibility effect. In a large and diverse cross-cultural sample (N = 10,195 from 24 countries), we presented participants with obscure, meaningless statements attributed to either a spiritual guru or a scientist. We found a robust global source credibility effect for scientific authorities, which we dub 'the Einstein effect': across all 24 countries and all levels of religiosity, scientists held greater authority than spiritual gurus. In addition, individual religiosity predicted a weaker relative preference for the statement from the scientist compared with the spiritual guru, and was more strongly associated with credibility judgements for the guru than the scientist. Independent data on explicit trust ratings across 143 countries mirrored our experimental findings. These findings suggest that irrespective of one's religious worldview, across cultures science is a powerful and universal heuristic that signals the reliability of information.

Are There Reliable Qualitative Individual Difference in Cognition?

In this paper we propose a new set of questions that focus on the direction of effects. In almost all studies the direction is important. For example, in a Stroop task we expect responses to incongruent items to be slower than those to congruent ones, and this direction implies one theoretical explanation. Yet, if congruent words are slowed down relative to incongruent words we would have a completely different theoretical explanation. We ask a 'does everybody' question, such as, 'does every individual show a Stroop effect in the same direction?' Or, 'does every individual respond faster to loud tones than soft tones?' If all individuals truly have effects in the same direction that implicate a common theory, we term the differences among them as quantitative individual differences. Conversely, if all individuals truly have effects in different directions that implicate different theories, we term the differences among them as qualitative individual differences. Here, we provide a users guide to the question of whether individual differences are qualitative or quantitative. We discuss theoretical issues, methodological advances, new software for assessment, and, most importantly, how the question impacts theory development in cognitive science. Our hope is that this mode of analysis is a productive tool in researchers' toolkits.

Revisiting the remember-know task: Replications of Gardiner and Java (1990).

One of the most evidential behavioral results for two memory processes comes from Gardiner and Java (Memory & Cognition, 18, 23-30 1990). Participants provided more "remember" than "know" responses for old words but more know than remember responses for old nonwords. Moreover, there was no effect of word/nonword status for new items. The combination of a crossover interaction for old items with an invariance for new items provides strong evidence for two distinct processes while ruling out criteria or bias explanations. Here, we report a modern replication of this study. In three experiments, (Experiments 1, 2, and 4) with larger numbers of items and participants, we were unable to replicate the crossover. Instead, our data are more consistent with a single-process account. In a fourth experiment (Experiment 3), we were able to replicate Gardiner and Java's baseline results with a sure-unsure paradigm supporting a single-process explanation. It seems that Gardiner and Java's remarkable crossover result is not replicable.

The JASP guidelines for conducting and reporting a Bayesian analysis.

Despite the increasing popularity of Bayesian inference in empirical research, few practical guidelines provide detailed recommendations for how to apply Bayesian procedures and interpret the results. Here we offer specific guidelines for four different stages of Bayesian statistical reasoning in a research setting: planning the analysis, executing the analysis, interpreting the results, and reporting the results. The guidelines for each stage are illustrated with a running example. Although the guidelines are geared towards analyses performed with the open-source statistical software JASP, most guidelines extend to Bayesian inference in general.

Beyond overall effects: A Bayesian approach to finding constraints in meta-analysis.

Most meta-analyses focus on the behavior of meta-analytic means. In many cases, however, this mean is difficult to defend as a construct because the underlying distribution of studies reflects many factors, including how we as researchers choose to design studies. We present an alternative goal for meta-analysis. The analyst may ask about relations that are stable across all the studies. In a typical meta-analysis, there is a hypothesized direction (e.g., that violent video games increase, rather than decrease, aggressive behavior). We ask whether all studies in a meta-analysis have true effects in the hypothesized direction. If so, this is an example of a stable relation across all the studies. We propose 4 models: (a) all studies are truly null; (b) all studies share a single true nonzero effect; (c) studies differ, but all true effects are in the same direction; and (d) some study effects are truly positive, whereas others are truly negative. We develop Bayes factor model comparison for these models and apply them to 4 extant meta-analyses to show their usefulness. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Common or distinct attention mechanisms for contrast and assimilation?

The ability to inhibit distractors while focusing on specific targets is crucial. In most tasks, like Stroop or priming, the to-be-ignored distractors affect the response to be more like the distractors. We call this assimilation. Yet, in some tasks, the opposite holds. Constrast occurs when the response is caused to be least like the distractors. Contrast and assimilation are opposing behavioral effects, but they both occur when to-be-ignored information affects judgments. We ask here whether inhibition across contrastive and assimilative tasks is common or distinct. Assimilation and contrast are often thought to have different underlying psychological mechanisms, and we use a correlational analysis with hierarchical Bayesian models as a test of this hypothesis. We designed tasks with large assimilation or contrast effects. The stimuli are morphed letters, and whether there is contrast or assimilation depends on whether the surrounding information is a letter field (contrast) or a word field (assimilation). Critically, a positive correlation was found-individuals who better inhibited contrast-inducing contexts also better inhibited assimilation-inducing contexts. These results indicate that inhibition is common, at least in part, across contrast and assimilation tasks.