Evaluating the robustness of parameter estimates in cognitive models: A meta-analytic review of multinomial processing tree models across the multiverse of estimation methods.

Researchers have become increasingly aware that data-analysis decisions affect results. Here, we examine this issue systematically for multinomial processing tree (MPT) models, a popular class of cognitive models for categorical data. Specifically, we examine the robustness of MPT model parameter estimates that arise from two important decisions: the level of data aggregation (complete-pooling, no-pooling, or partial-pooling) and the statistical framework (frequentist or Bayesian). These decisions span a multiverse of estimation methods. We synthesized the data from 13,956 participants (164 published data sets) with a meta-analytic strategy and analyzed the magnitude of divergence between estimation methods for the parameters of nine popular MPT models in psychology (e.g., process-dissociation, source monitoring). We further examined moderators as potential sources of divergence. We found that the absolute divergence between estimation methods was small on average (<.04; with MPT parameters ranging between 0 and 1); in some cases, however, divergence amounted to nearly the maximum possible range (.97). Divergence was partly explained by few moderators (e.g., the specific MPT model parameter, uncertainty in parameter estimation), but not by other plausible candidate moderators (e.g., parameter trade-offs, parameter correlations) or their interactions. Partial-pooling methods showed the smallest divergence within and across levels of pooling and thus seem to be an appropriate default method. Using MPT models as an example, we show how transparency and robustness can be increased in the field of cognitive modeling. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

On the Roles of Stereotype Activation and Application in Diminishing Implicit Bias.

Stereotypes can influence social perception in undesirable ways. However, activated stereotypes are not always applied in judgments. The present research investigated how stereotype activation and application processes impact social judgments as a function of available resources for control over stereotypes. Specifically, we varied the time available to intervene in the stereotyping process and used multinomial modeling to independently estimate stereotype activation and application. As expected, social judgments were less stereotypic when participants had more time to intervene. In terms of mechanisms, stereotype application, and not stereotype activation, corresponded with reductions in stereotypic biases. With increasing time, stereotype application was reduced, reflecting the fact that controlling application is time-dependent. In contrast, stereotype activation increased with increasing time, apparently due to increased engagement with stereotypic material. Stereotype activation was highest when judgments were least stereotypical, and thus, reduced stereotyping may coincide with increased stereotype activation if stereotype application is simultaneously decreased.

When scoring algorithms matter: effects of working memory load on different IAT scores.

In most process accounts of the Implicit Association Test (IAT), it is assumed that compatible and incompatible IAT blocks require different amounts of working memory capacity (WMC) and recruit executive functions such as task switching and inhibition to different extents. In the present study (N= 120), cognitive load during the completion of an IAT was experimentally manipulated by means of an oral random-number generation secondary task. Cognitive load led to slower latencies and more errors, especially in the incompatible block. However, different IAT scores, including conventional scores and D-scores, were affected differentially by the load manipulation: scores based on raw data of task performance such as latencies and errors were increased whereas scores that use transformations such as log-latency scores and D-scores were decreased. A number of analyses shed light on the reasons for the unexpected dissociation between scoring algorithms. Remarkably, external correlations of the IAT scores were not affected by the experimental manipulation.