Power priors for replication studies.

The ongoing replication crisis in science has increased interest in the methodology of replication studies. We propose a novel Bayesian analysis approach using power priors: The likelihood of the original study's data is raised to the power of α, and then used as the prior distribution in the analysis of the replication data. Posterior distribution and Bayes factor hypothesis tests related to the power parameter α quantify the degree of compatibility between the original and replication study. Inferences for other parameters, such as effect sizes, dynamically borrow information from the original study. The degree of borrowing depends on the conflict between the two studies. The practical value of the approach is illustrated on data from three replication studies, and the connection to hierarchical modeling approaches explored. We generalize the known connection between normal power priors and normal hierarchical models for fixed parameters and show that normal power prior inferences with a beta prior on the power parameter α align with normal hierarchical model inferences using a generalized beta prior on the relative heterogeneity variance I2. The connection illustrates that power prior modeling is unnatural from the perspective of hierarchical modeling since it corresponds to specifying priors on a relative rather than an absolute heterogeneity scale.

A conditional judgment procedure for probing evaluative conditioning effects in the absence of feelings of remembering.

Attitude research has capitalized on evaluative conditioning procedures to gain insight into how evaluations are formed and may be changed. In evaluative conditioning, a conditioned stimulus (CS; e.g., an unfamiliar soda brand) is paired with an unconditioned stimulus (US) of affective value (e.g., a pleasant picture). Following this pairing, a change in CS liking may be observed (e.g., the soda brand is liked better). A question with far-reaching theoretical and practical implications is whether the change in CS liking is found when participants feel they do not remember the CS-US pairings at the time an evaluation is produced about the CS. Here, we introduce a new conditional judgment procedure-the two-button-sets (TBS) task-for probing evaluative conditioning effects without feelings of remembering about the valence of the US paired with the CS. In three experiments, the TBS is (1) is successfully validated; it is also used to (2) provide preliminary information on the feeling of remembering question, and (3) to examine an affect-consistent bias in memory judgments for CS-US pairings. Results do not support evaluative effects in the absence of feelings of remembering, and they oppose the view that affect-consistent bias is limited to memory uncertainty. We discuss these findings in light of previous evidence and of dual-learning models of attitudes. We also discuss limitations and research avenues related to the new procedure.

Refining Bayesian hierarchical MPT modeling: Integrating prior knowledge and ordinal expectations.

Multinomial processing tree (MPT) models are a broad class of statistical models used to test sophisticated psychological theories. The research questions derived from these theories often go beyond simple condition effects on parameters and involve ordinal expectations (e.g., the same-direction effect on the memory parameter is stronger in one experimental condition than another) or disordinal expectations (e.g., the effect reverses in one experimental condition). Here, we argue that by refining common modeling practices, Bayesian hierarchical models are well suited to estimate and test these expectations. Concretely, we show that the default priors proposed in the literature lead to nonsensical predictions for individuals and the population distribution, leading to problems not only in model comparison but also in parameter estimation. Rather than relying on these priors, we argue that MPT modelers should determine priors that are consistent with their theoretical knowledge. In addition, we demonstrate how Bayesian model comparison may be used to test ordinal and disordinal interactions by means of Bayes factors. We apply the techniques discussed to empirical data from Bell et al. Journal of Experimental Psychology: Learning, Memory, and Cognition, 41, 456-472 (2015).

Multibridge: an R package to evaluate informed hypotheses in binomial and multinomial models.

The multibridge R package allows a Bayesian evaluation of informed hypotheses [Formula: see text] applied to frequency data from an independent binomial or multinomial distribution. multibridge uses bridge sampling to efficiently compute Bayes factors for the following hypotheses concerning the latent category proportions 𝜃: (a) hypotheses that postulate equality constraints (e.g., 𝜃1 = 𝜃2 = 𝜃3); (b) hypotheses that postulate inequality constraints (e.g., 𝜃1 < 𝜃2 < 𝜃3 or 𝜃1 > 𝜃2 > 𝜃3); (c) hypotheses that postulate combinations of inequality constraints and equality constraints (e.g., 𝜃1 < 𝜃2 = 𝜃3); and (d) hypotheses that postulate combinations of (a)-(c) (e.g., 𝜃1 < (𝜃2 = 𝜃3),𝜃4). Any informed hypothesis [Formula: see text] may be compared against the encompassing hypothesis [Formula: see text] that all category proportions vary freely, or against the null hypothesis [Formula: see text] that all category proportions are equal. multibridge facilitates the fast and accurate comparison of large models with many constraints and models for which relatively little posterior mass falls in the restricted parameter space. This paper describes the underlying methodology and illustrates the use of multibridge through fully reproducible examples.

Informed Bayesian survival analysis.

BACKGROUND: We provide an overview of Bayesian estimation, hypothesis testing, and model-averaging and illustrate how they benefit parametric survival analysis. We contrast the Bayesian framework to the currently dominant frequentist approach and highlight advantages, such as seamless incorporation of historical data, continuous monitoring of evidence, and incorporating uncertainty about the true data generating process. METHODS: We illustrate the application of the outlined Bayesian approaches on an example data set, retrospective re-analyzing a colon cancer trial. We assess the performance of Bayesian parametric survival analysis and maximum likelihood survival models with AIC/BIC model selection in fixed-n and sequential designs with a simulation study. RESULTS: In the retrospective re-analysis of the example data set, the Bayesian framework provided evidence for the absence of a positive treatment effect of adding Cetuximab to FOLFOX6 regimen on disease-free survival in patients with resected stage III colon cancer. Furthermore, the Bayesian sequential analysis would have terminated the trial 10.3 months earlier than the standard frequentist analysis. In a simulation study with sequential designs, the Bayesian framework on average reached a decision in almost half the time required by the frequentist counterparts, while maintaining the same power, and an appropriate false-positive rate. Under model misspecification, the Bayesian framework resulted in higher false-negative rate compared to the frequentist counterparts, which resulted in a higher proportion of undecided trials. In fixed-n designs, the Bayesian framework showed slightly higher power, slightly elevated error rates, and lower bias and RMSE when estimating treatment effects in small samples. We found no noticeable differences for survival predictions. We have made the analytic approach readily available to other researchers in the RoBSA R package. CONCLUSIONS: The outlined Bayesian framework provides several benefits when applied to parametric survival analyses. It uses data more efficiently, is capable of considerably shortening the length of clinical trials, and provides a richer set of inferences.

Incidental Attitude Formation via the Surveillance Task: A Preregistered Replication of the Olson and Fazio (2001) Study.

Evaluative conditioning is one of the most widely studied procedures for establishing and changing attitudes. The surveillance task is a highly cited evaluative-conditioning paradigm and one that is claimed to generate attitudes without awareness. The potential for evaluative-conditioning effects to occur without awareness continues to fuel conceptual, theoretical, and applied developments. Yet few published studies have used this task, and most are characterized by small samples and small effect sizes. We conducted a high-powered (N = 1,478 adult participants), preregistered close replication of the original surveillance-task study (Olson & Fazio, 2001). We obtained evidence for a small evaluative-conditioning effect when "aware" participants were excluded using the original criterion-therefore replicating the original effect. However, no such effect emerged when three other awareness criteria were used. We suggest that there is a need for caution when using evidence from the surveillance-task effect to make theoretical and practical claims about "unaware" evaluative-conditioning effects.

The enhancing effect of 200†mg caffeine on mnemonic discrimination is at best small.

Mnemonic discrimination is the ability to discriminate among similar memories, which requires separable representations of similar information. The neurocomputational process that assumedly decorrelates representations during encoding and consolidation is referred to as pattern separation. Deficits in pattern separation contribute to age-related declines in mnemonic functioning, which has motivated the development of targeted interventions. We followed-up a recent report that one 200 mg-dose of caffeine administered post-study enhances mnemonic discrimination [Borota, D., Murray, E., Keceli, G., Chang, A., Watabe, J. M., Ly, M., Toscano, J. P., & Yassa, M. A. (2014). Post-study caffeine administration enhances memory consolidation in humans. Nature Neuroscience, 17(2), 201-203. https://doi.org/10.1038/nn.3623]. To test whether the reported enhancements are an artifact of performance-impairing withdrawal symptoms in the control group, we did not restrict preexperimental caffeine intake and statistically adjusted treatment effects for habitual caffeine consumption. We detected no effects of caffeine and nonsuperiority testing ruled out medium and large enhancements in both average (1200 mg per week) and low-consumers (50 mg per week). Our results raise doubts about a caffeine-mediated enhancement of mnemonic discrimination on two counts: If the effect exists, it (1) is substantially smaller than originally reported and (2) may reflect an offset of performance-impairing withdrawal symptoms rather than genuinely enhanced consolidation. We recommend that future studies employ an alternating exposure-abstinence protocol, use an active control group, and verify posttreatment caffeine abstinence via saliva or blood samples.

Changing-state irrelevant speech disrupts visual-verbal but not visual-spatial serial recall.

In an influential article, Jones et al. (1995) provide evidence that auditory distraction by changing relative to repetitive auditory distracters (the changing-state effect) did not differ between a visual-verbal and visual-spatial serial recall task, providing evidence for an amodal mechanism for the representation of serial order in short-term memory that transcends modalities. This finding has been highly influential for theories of short-term memory and auditory distraction. However, evidence vis-à-vis the robustness of this result is sorely lacking. Here, two high-powered replications of Jones et al.'s (1995) crucial Experiment 4 were undertaken. In the first partial replication (n = 64), a fully within-participants design was adopted, wherein participants undertook both the visual-verbal and visual-spatial serial recall tasks under different irrelevant sound conditions, without a retention period. The second near-identical replication (n = 128), incorporated a retention period and implemented the task-modality manipulation as a between-participants factor, as per the original Jones et al. (1995; Experiment 4) study. In both experiments, the changing-state effect was observed for visual-verbal serial recall but not for visual-spatial serial recall. The results are consistent with modular and interference-based accounts of distraction and challenge some aspects of functional equivalence accounts. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Evaluating the robustness of parameter estimates in cognitive models: A meta-analytic review of multinomial processing tree models across the multiverse of estimation methods.

Researchers have become increasingly aware that data-analysis decisions affect results. Here, we examine this issue systematically for multinomial processing tree (MPT) models, a popular class of cognitive models for categorical data. Specifically, we examine the robustness of MPT model parameter estimates that arise from two important decisions: the level of data aggregation (complete-pooling, no-pooling, or partial-pooling) and the statistical framework (frequentist or Bayesian). These decisions span a multiverse of estimation methods. We synthesized the data from 13,956 participants (164 published data sets) with a meta-analytic strategy and analyzed the magnitude of divergence between estimation methods for the parameters of nine popular MPT models in psychology (e.g., process-dissociation, source monitoring). We further examined moderators as potential sources of divergence. We found that the absolute divergence between estimation methods was small on average (<.04; with MPT parameters ranging between 0 and 1); in some cases, however, divergence amounted to nearly the maximum possible range (.97). Divergence was partly explained by few moderators (e.g., the specific MPT model parameter, uncertainty in parameter estimation), but not by other plausible candidate moderators (e.g., parameter trade-offs, parameter correlations) or their interactions. Partial-pooling methods showed the smallest divergence within and across levels of pooling and thus seem to be an appropriate default method. Using MPT models as an example, we show how transparency and robustness can be increased in the field of cognitive modeling. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Seriousness checks are useful to improve data validity in online research.

Nonserious answering behavior increases noise and reduces experimental power; it is therefore one of the most important threats to the validity of online research. A simple way to address the problem is to ask respondents about the seriousness of their participation and to exclude self-declared nonserious participants from analysis. To validate this approach, a survey was conducted in the week prior to the German 2009 federal election to the Bundestag. Serious participants answered a number of attitudinal and behavioral questions in a more consistent and predictively valid manner than did nonserious participants. We therefore recommend routinely employing seriousness checks in online surveys to improve data validity.

Bayes Factors for Mixed Models: Perspective on Responses.

In van Doorn et al. (2021), we outlined a series of open questions concerning Bayes factors for mixed effects model comparison, with an emphasis on the impact of aggregation, the effect of measurement error, the choice of prior distributions, and the detection of interactions. Seven expert commentaries (partially) addressed these initial questions. Surprisingly perhaps, the experts disagreed (often strongly) on what is best practice-a testament to the intricacy of conducting a mixed effect model comparison. Here, we provide our perspective on these comments and highlight topics that warrant further discussion. In general, we agree with many of the commentaries that in order to take full advantage of Bayesian mixed model comparison, it is important to be aware of the specific assumptions that underlie the to-be-compared models.

Documenting contributions to scholarly articles using CRediT and tenzing.

Scholars traditionally receive career credit for a paper based on where in the author list they appear, but position in an author list often carries little information about what the contribution of each researcher was. "Contributorship" refers to a movement to formally document the nature of each researcher's contribution to a project. We discuss the emerging CRediT standard for documenting contributions and describe a web-based app and R package called tenzing that is designed to facilitate its use. tenzing can make it easier for researchers on a project to plan and record their planned contributions and to document those contributions in a journal article.

Attitudes from mere co-occurrences are guided by differentiation.

People often form attitudes about objects, individuals, or groups by examining and comparing their attributes. Such attribute-based attitude formation is guided by a differentiation principle: Whether people come to like or dislike an attitude object depends on the object's attributes that differentiate it from other objects. Attributes that are redundant with previously encountered attitude objects are typically cancelled out. We tested whether the same differentiation principle applies to co-occurrence-based attitude formation, also known as Evaluative Conditioning. This form of attitude formation describes the phenomenon that attitudes are influenced by positive or negative stimuli that have co-occurred with attitude object, but which are not inherent attributes of the attitude object itself. Across 7 experiments (N = 1611), we consistently found that co-occurrence-based attitude formation is guided by the same differentiation principle as attribute-based attitude formation. Specifically, participants' attitudes toward unknown brands were most strongly influenced by positive or negative stimuli that distinctly co-occurred with a specific brand, and that differentiated that brand from previously encountered ones. Stimuli that redundantly co-occurred with multiple brands had weaker influences on brand attitudes. The results further suggest that differentiation operates at the learning stage during which distinct stimulus co-occurrences enjoy a processing advantage. We discuss the present findings' theoretical and practical implications for attitude formation and identify differentiation as a possible cause of biased attitudes. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

A memory-based judgment account of expectancy-liking dissociations in evaluative conditioning.

Evaluative conditioning (EC) is a change in liking of neutral conditioned stimuli (CS) following pairings with positive or negative stimuli (unconditioned stimulus, US). A dissociation has been reported between US expectancy and CS evaluation in extinction learning: When CSs are presented alone subsequent to CS-US pairings, participants cease to expect USs but continue to exhibit EC effects. This dissociation is typically interpreted as demonstration that EC is resistant to extinction, and consequently, that EC is driven by a distinct learning process. We tested whether expectancy-liking dissociations are instead caused by different judgment strategies afforded by the dependent measures: CS evaluations are by default integrative judgments-summaries of large portions of the learning history-whereas US expectancy reflects momentary judgments that focus on recent events. In a counterconditioning and two extinction experiments, we eliminated the expectancy-liking dissociation by inducing nondefault momentary evaluative judgments, and demonstrated a reversed dissociation when we additionally induced nondefault integrative expectancy judgments. Our findings corroborated a priori predictions derived from the formal memory model MINERVA 2. Hence, dissociations between US expectancy and CS evaluation are consistent with a single-process learning model; they reflect different summaries of the learning history. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Subliminal influence on preferences? A test of evaluative conditioning for brief visual conditioned stimuli using auditory unconditioned stimuli.

In the field of evaluative conditioning (EC), two opposing theories-propositional single-process theory versus dual-process theory-are currently being discussed in the literature. The present set of experiments test a crucial prediction to adjudicate between these two theories: Dual-process theory postulates that evaluative conditioning can occur without awareness of the contingency between conditioned stimulus (CS) and unconditioned stimulus (US); in contrast, single-process propositional theory postulates that EC requires CS-US contingency awareness. In a set of three studies, we experimentally manipulate contingency awareness by presenting the CSs very briefly, thereby rendering it unlikely to be processed consciously. We address potential issues with previous studies on EC with subliminal or near-threshold CSs that limited their interpretation. Across two experiments, we consistently found an EC effect for CSs presented for 1000 ms and consistently failed to find an EC effect for briefly presented CSs. In a third pre-registered experiment, we again found evidence for an EC effect with CSs presented for 1000 ms, and we found some indication for an EC effect for CSs presented for 20 ms.

Incremental validity of Useful Field of View subtests for the prediction of instrumental activities of daily living.

INTRODUCTION: The Useful Field of View Test (UFOV®) is a cognitive measure that predicts older adults' ability to perform a range of everyday activities. However, little is known about the individual contribution of each subtest to these predictions, and the underlying constructs of UFOV performance remain a topic of debate. METHOD: We investigated the incremental validity of UFOV subtests for the prediction of Instrumental Activities of Daily Living (IADL) performance in two independent datasets, the SKILL (n = 828) and ACTIVE (n = 2426) studies. We then explored the cognitive and visual abilities assessed by UFOV using a range of neuropsychological and vision tests administered in the SKILL study. RESULTS: In the four subtest variant of UFOV, only Subtests 2 and 3 consistently made independent contributions to the prediction of IADL performance across three different behavioral measures. In all cases, the incremental validity of UFOV Subtests 1 and 4 was negligible. Furthermore, we found that UFOV was related to processing speed, general nonspeeded cognition, and visual function; the omission of Subtests 1 and 4 from the test score did not affect these associations. CONCLUSIONS: UFOV Subtests 1 and 4 appear to be of limited use to predict IADL and possibly other everyday activities. Future research should investigate whether shortening UFOV by omitting these subtests is a reliable and valid assessment approach.