Inclusion Bayes factors for mixed hierarchical diffusion decision models.

Cognitive models provide a substantively meaningful quantitative description of latent cognitive processes. The quantitative formulation of these models supports cumulative theory building and enables strong empirical tests. However, the nonlinearity of these models and pervasive correlations among model parameters pose special challenges when applying cognitive models to data. Firstly, estimating cognitive models typically requires large hierarchical data sets that need to be accommodated by an appropriate statistical structure within the model. Secondly, statistical inference needs to appropriately account for model uncertainty to avoid overconfidence and biased parameter estimates. In the present work, we show how these challenges can be addressed through a combination of Bayesian hierarchical modeling and Bayesian model averaging. To illustrate these techniques, we apply the popular diffusion decision model to data from a collaborative selective influence study. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

A review of applications of the Bayes factor in psychological research.

The last 25 years have shown a steady increase in attention for the Bayes factor as a tool for hypothesis evaluation and model selection. The present review highlights the potential of the Bayes factor in psychological research. We discuss six types of applications: Bayesian evaluation of point null, interval, and informative hypotheses, Bayesian evidence synthesis, Bayesian variable selection and model averaging, and Bayesian evaluation of cognitive models. We elaborate what each application entails, give illustrative examples, and provide an overview of key references and software with links to other applications. The article is concluded with a discussion of the opportunities and pitfalls of Bayes factor applications and a sketch of corresponding future research lines. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Efficient numerical approximation of a non-regular Fokker-Planck equation associated with first-passage time distributions.

In neuroscience, the distribution of a decision time is modelled by means of a one-dimensional Fokker-Planck equation with time-dependent boundaries and space-time-dependent drift. Efficient approximation of the solution to this equation is required, e.g., for model evaluation and parameter fitting. However, the prescribed boundary conditions lead to a strong singularity and thus to slow convergence of numerical approximations. In this article we demonstrate that the solution can be related to the solution of a parabolic PDE on a rectangular space-time domain with homogeneous initial and boundary conditions by transformation and subtraction of a known function. We verify that the solution of the new PDE is indeed more regular than the solution of the original PDE and proceed to discretize the new PDE using a space-time minimal residual method. We also demonstrate that the solution depends analytically on the parameters determining the boundaries as well as the drift. This justifies the use of a sparse tensor product interpolation method to approximate the PDE solution for various parameter ranges. The predicted convergence rates of the minimal residual method and that of the interpolation method are supported by numerical simulations.

Consensus-based guidance for conducting and reporting multi-analyst studies.

Any large dataset can be analyzed in a number of ways, and it is possible that the use of different analysis strategies will lead to different results and conclusions. One way to assess whether the results obtained depend on the analysis strategy chosen is to employ multiple analysts and leave each of them free to follow their own approach. Here, we present consensus-based guidance for conducting and reporting such multi-analyst studies, and we discuss how broader adoption of the multi-analyst approach has the potential to strengthen the robustness of results and conclusions obtained from analyses of datasets in basic and applied research.

An Attention-Based Diffusion Model for Psychometric Analyses.

The emergence of computer-based assessments has made response times, in addition to response accuracies, available as a source of information about test takers' latent abilities. The development of substantively meaningful accounts of the cognitive process underlying item responses is critical to establishing the validity of psychometric tests. However, existing substantive theories such as the diffusion model have been slow to gain traction due to their unwieldy functional form and regular violations of model assumptions in psychometric contexts. In the present work, we develop an attention-based diffusion model based on process assumptions that are appropriate for psychometric applications. This model is straightforward to analyse using Gibbs sampling and can be readily extended. We demonstrate our model's good computational and statistical properties in a comparison with two well-established psychometric models.

Prior information use and response caution in perceptual decision-making: No evidence for a relationship with autistic-like traits.

Interpreting the world around us requires integrating incoming sensory signals with prior information. Autistic individuals have been proposed to rely less on prior information and make more cautious responses than non-autistic individuals. Here, we investigated whether these purported features of autistic perception vary as a function of autistic-like traits in the general population. We used a diffusion model framework, whereby decisions are modelled as noisy evidence accumulation processes towards one of two bounds. Within this framework, prior information can bias the starting point of the evidence accumulation process. Our pre-registered hypotheses were that higher autistic-like traits would relate to reduced starting point bias caused by prior information and increased response caution (wider boundary separation). 222 participants discriminated the direction of coherent motion stimuli as quickly and accurately as possible. Stimuli were preceded with a neutral cue (square) or a directional cue (arrow). 80% of the directional cues validly predicted the upcoming motion direction. We modelled accuracy and response time data using a hierarchical Bayesian model in which starting point varied with cue condition. We found no evidence for our hypotheses, with starting point bias and response caution seemingly unrelated to Adult Autism Spectrum Quotient (AQ) scores. Alongside future research applying this paradigm to autistic individuals, our findings will help refine theories regarding the role of prior information and altered decision-making strategies in autistic perception. Our study also has implications for models of bias in perceptual decision-making, as the most plausible model was one that incorporated bias in both decision-making and sensory processing.

Real-time prediction of short-timescale fluctuations in cognitive workload.

Human operators often experience large fluctuations in cognitive workload over seconds timescales that can lead to sub-optimal performance, ranging from overload to neglect. Adaptive automation could potentially address this issue, but to do so it needs to be aware of real-time changes in operators' spare cognitive capacity, so it can provide help in times of peak demand and take advantage of troughs to elicit operator engagement. However, it is unclear whether rapid changes in task demands are reflected in similarly rapid fluctuations in spare capacity, and if so what aspects of responses to those demands are predictive of the current level of spare capacity. We used the ISO standard detection response task (DRT) to measure cognitive workload approximately every 4 s in a demanding task requiring monitoring and refueling of a fleet of simulated unmanned aerial vehicles (UAVs). We showed that the DRT provided a valid measure that can detect differences in workload due to changes in the number of UAVs. We used cross-validation to assess whether measures related to task performance immediately preceding the DRT could predict detection performance as a proxy for cognitive workload. Although the simple occurrence of task events had weak predictive ability, composite measures that tapped operators' situational awareness with respect to fuel levels were much more effective. We conclude that cognitive workload does vary rapidly as a function of recent task events, and that real-time predictive models of operators' cognitive workload provide a potential avenue for automation to adapt without an ongoing need for intrusive workload measurements.

Perceptual Decision-Making in Children: Age-Related Differences and EEG Correlates.

Children make faster and more accurate decisions about perceptual information as they get older, but it is unclear how different aspects of the decision-making process change with age. Here, we used hierarchical Bayesian diffusion models to decompose performance in a perceptual task into separate processing components, testing age-related differences in model parameters and links to neural data. We collected behavioural and EEG data from 96 6- to 12-year-old children and 20 adults completing a motion discrimination task. We used a component decomposition technique to identify two response-locked EEG components with ramping activity preceding the response in children and adults: one with activity that was maximal over centro-parietal electrodes and one that was maximal over occipital electrodes. Younger children had lower drift rates (reduced sensitivity), wider boundary separation (increased response caution) and longer non-decision times than older children and adults. Yet, model comparisons suggested that the best model of children's data included age effects only on drift rate and boundary separation (not non-decision time). Next, we extracted the slope of ramping activity in our EEG components and covaried these with drift rate. The slopes of both EEG components related positively to drift rate, but the best model with EEG covariates included only the centro-parietal component. By decomposing performance into distinct components and relating them to neural markers, diffusion models have the potential to identify the reasons why children with developmental conditions perform differently to typically developing children and to uncover processing differences inapparent in the response time and accuracy data alone.

A theoretical analysis of the reward rate optimality of collapsing decision criteria.

A standard assumption of most sequential sampling models is that decision-makers rely on a decision criterion that remains constant throughout the decision process. However, several authors have recently suggested that, in order to maximize reward rates in dynamic environments, decision-makers need to rely on a decision criterion that changes over the course of the decision process. We used dynamic programming and simulation methods to quantify the reward rates obtained by constant and dynamic decision criteria in different environments. We further investigated what influence a decision-maker's uncertainty about the stochastic structure of the environment has on reward rates. Our results show that in most dynamic environments, both types of decision criteria yield similar reward rates, across different levels of uncertainty. This suggests that a static decision criterion might provide a robust default setting.

On the importance of avoiding shortcuts in applying cognitive models to hierarchical data.

Psychological experiments often yield data that are hierarchically structured. A number of popular shortcut strategies in cognitive modeling do not properly accommodate this structure and can result in biased conclusions. To gauge the severity of these biases, we conducted a simulation study for a two-group experiment. We first considered a modeling strategy that ignores the hierarchical data structure. In line with theoretical results, our simulations showed that Bayesian and frequentist methods that rely on this strategy are biased towards the null hypothesis. Secondly, we considered a modeling strategy that takes a two-step approach by first obtaining participant-level estimates from a hierarchical cognitive model and subsequently using these estimates in a follow-up statistical test. Methods that rely on this strategy are biased towards the alternative hypothesis. Only hierarchical models of the multilevel data lead to correct conclusions. Our results are particularly relevant for the use of hierarchical Bayesian parameter estimates in cognitive modeling.

Using Bayesian regression to test hypotheses about relationships between parameters and covariates in cognitive models.

An important tool in the advancement of cognitive science are quantitative models that represent different cognitive variables in terms of model parameters. To evaluate such models, their parameters are typically tested for relationships with behavioral and physiological variables that are thought to reflect specific cognitive processes. However, many models do not come equipped with the statistical framework needed to relate model parameters to covariates. Instead, researchers often revert to classifying participants into groups depending on their values on the covariates, and subsequently comparing the estimated model parameters between these groups. Here we develop a comprehensive solution to the covariate problem in the form of a Bayesian regression framework. Our framework can be easily added to existing cognitive models and allows researchers to quantify the evidential support for relationships between covariates and model parameters using Bayes factors. Moreover, we present a simulation study that demonstrates the superiority of the Bayesian regression framework to the conventional classification-based approach.

Bayesian inference for psychology, part III: Parameter estimation in nonstandard models.

We demonstrate the use of three popular Bayesian software packages that enable researchers to estimate parameters in a broad class of models that are commonly used in psychological research. We focus on WinBUGS, JAGS, and Stan, and show how they can be interfaced from R and MATLAB. We illustrate the use of the packages through two fully worked examples; the examples involve a simple univariate linear regression and fitting a multinomial processing tree model to data from a classic false-memory experiment. We conclude with a comparison of the strengths and weaknesses of the packages. Our example code, data, and this text are available via https://osf.io/ucmaz/ .

A tutorial on bridge sampling.

The marginal likelihood plays an important role in many areas of Bayesian statistics such as parameter estimation, model comparison, and model averaging. In most applications, however, the marginal likelihood is not analytically tractable and must be approximated using numerical methods. Here we provide a tutorial on bridge sampling (Bennett, 1976; Meng & Wong, 1996), a reliable and relatively straightforward sampling method that allows researchers to obtain the marginal likelihood for models of varying complexity. First, we introduce bridge sampling and three related sampling methods using the beta-binomial model as a running example. We then apply bridge sampling to estimate the marginal likelihood for the Expectancy Valence (EV) model-a popular model for reinforcement learning. Our results indicate that bridge sampling provides accurate estimates for both a single participant and a hierarchical version of the EV model. We conclude that bridge sampling is an attractive method for mathematical psychologists who typically aim to approximate the marginal likelihood for a limited set of possibly high-dimensional models.

The computations that support simple decision-making: A comparison between the diffusion and urgency-gating models.

We investigate a question relevant to the psychology and neuroscience of perceptual decision-making: whether decisions are based on steadily accumulating evidence, or only on the most recent evidence. We report an empirical comparison between two of the most prominent examples of these theoretical positions, the diffusion model and the urgency-gating model, via model-based qualitative and quantitative comparisons. Our findings support the predictions of the diffusion model over the urgency-gating model, and therefore, the notion that evidence accumulates without much decay. Gross qualitative patterns and fine structural details of the data are inconsistent with the notion that decisions are based only on the most recent evidence. More generally, we discuss some strengths and weaknesses of scientific methods that investigate quantitative models by distilling the formal models to qualitative predictions.

Of monkeys and men: Impatience in perceptual decision-making.

For decades sequential sampling models have successfully accounted for human and monkey decision-making, relying on the standard assumption that decision makers maintain a pre-set decision standard throughout the decision process. Based on the theoretical argument of reward rate maximization, some authors have recently suggested that decision makers become increasingly impatient as time passes and therefore lower their decision standard. Indeed, a number of studies show that computational models with an impatience component provide a good fit to human and monkey decision behavior. However, many of these studies lack quantitative model comparisons and systematic manipulations of rewards. Moreover, the often-cited evidence from single-cell recordings is not unequivocal and complimentary data from human subjects is largely missing. We conclude that, despite some enthusiastic calls for the abandonment of the standard model, the idea of an impatience component has yet to be fully established; we suggest a number of recently developed tools that will help bring the debate to a conclusive settlement.

Trial-by-trial fluctuations in CNV amplitude reflect anticipatory adjustment of response caution.

The contingent negative variation, a slow cortical potential, occurs when humans are warned by a stimulus about an upcoming task. The cognitive processes that give rise to this EEG potential are not yet well understood. To explain these processes, we adopt a recently developed theoretical framework from the area of perceptual decision-making. This framework assumes that the basal ganglia control the tradeoff between fast and accurate decision-making in the cortex. It suggests that an increase in cortical excitability serves to lower response caution, which results in faster but more error prone responding. We propose that the CNV reflects this increased cortical excitability. To test this hypothesis, we conducted an EEG experiment in which participants performed the random dot motion task either under speed or under accuracy stress. Our results show that trial-by-trial fluctuations in participants' response speed as well as model-based estimates of response caution correlated with single-trial CNV amplitude under conditions of speed but not accuracy stress. We conclude that the CNV might reflect adjustments of response caution, which serves to enhance quick decision-making.