Against naïve induction from experimental data.

This commentary argues against the indictment of current experimental practices such as piecemeal testing, and the proposed integrated experiment design (IED) approach, which we see as yet another attempt at automating scientific thinking. We identify a number of undesirable features of IED that lead us to believe that its broad application will hinder scientific progress.

Is all mental effort equal? The role of cognitive demand-type on effort avoidance.

Humans are often termed "cognitive misers" for their aversion to mental effort. Both in and outside the laboratory people often show preferences for low-effort tasks and are willing to forgo financial reward to avoid more demanding alternatives. Mental effort, however, does not seem to be ubiquitously avoided: people play crosswords, board games, and read novels, all as forms of leisure. While such activities undoubtedly require effort, the type of cognitive demands they impose appear markedly different from the tasks typically used in psychological research on mental effort (e.g., N-Back, Stroop Task, vigilance tasks). We investigate the effect disparate demands, such as tasks which require problem solving (e.g., solve the missing number: 1, 3, 7, 15, 31,?) compared to those which require rule-implementation (e.g., N-Back task), have on people's aversion to or preference for increased mental effort. Across four experiments using three different tasks, and a mixture of online and lab-based settings, we find that aversion to effort remains largely stable regardless of the types of cognitive demands a task imposes. The results are discussed in terms of other factors that might induce the pursuit of mental effort over and above the type of cognitive demands imposed by a task.

Managing Interruptions to Improve Diagnostic Decision-Making: Strategies and Recommended Research Agenda.

Interruptions are an inevitable occurrence in health care. Interruptions in diagnostic decision-making are no exception and can have negative consequences on both the decision-making process and well-being of the decision-maker. This may result in inaccurate or delayed diagnoses. To date, research specific to interruptions on diagnostic decision-making has been limited, but strategies to help manage the negative impacts of interruptions need to be developed and implemented. In this perspective, we first present a modified model of interruptions to visualize the interruption process and illustrate where potential interventions can be implemented. We then consider several empirically tested strategies from the fields of health care and cognitive psychology that can lay the groundwork for additional research to mitigate effects of interruptions during diagnostic decision-making. We highlight strategies to minimize the negative impacts of interruptions as well as strategies to prevent interruptions altogether. Additionally, we build upon these strategies to propose specific research priorities within the field of diagnostic safety. Identifying effective interventions to help clinicians better manage interruptions has the potential to minimize diagnostic errors and improve patient outcomes.

Toward nonprobabilistic explanations of learning and decision-making.

Referring to probabilistic concepts (such as randomness, sampling, and probability distributions among others) is commonplace in contemporary explanations of how people learn and make decisions in the face of environmental unknowns. Here, we critically evaluate this practice and argue that such concepts should only play a relatively minor part in psychological explanations. To make this point, we provide a theoretical analysis of what people need to do in order to deal with unknown aspects of a typical decision-making task (a repeated-choice gamble). This analysis reveals that the use of probabilistic concepts in psychological explanations may and often does conceal essential, nonprobabilistic steps that people need to take to attempt to solve the problems that environmental unknowns present. To give these steps a central role, we recast how people solve these problems as a type of hypothesis generation and evaluation, of which using probabilistic concepts to deal with unknowns is one of many possibilities. We also demonstrate some immediate practical consequences of our proposed approach in two experiments. This perspective implies a shift in focus toward nonprobabilistic aspects of psychological explanations. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

The mazing race: Effects of interruptions and benefits of interruption lags in a novel maze-like decision-making paradigm.

Interruptions are an inevitable, and often negative, part of everyday life that increase both errors and the time needed to complete even menial tasks. However, existing research suggests that being given time to prepare for a pending interruption-a lag time-can mitigate some of the interruption costs. To understand better why interruption lags are effective, we present a series of three experiments in which we develop and test a novel sequential decision-making paradigm, the mazing race. We find that interruption lags were only beneficial when participants had a clear strategy for how to complete the task, allowing them to avoid specific errors. In the final experiment, we attempted to use what we learned about the kinds of errors introduced by interruptions to develop a feedback-based intervention, aimed at dealing with situations in which interruption lags are not possible. We found that feedback was, only in certain situations, an effective replacement for an interruption lag. Overall, however, because the usefulness of interruption lags depend on the specific strategy a participant adopts, developing generic interventions to replace interruption lags is likely to be difficult. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

MEM-EX: An exemplar memory model of decisions from experience.

Many real-world decisions must be made on basis of experienced outcomes. However, there is little consensus about the mechanisms by which people make these decisions from experience (DfE). Across five experiments, we identified several factors influencing DfE. We also introduce a novel computational modeling framework, the memory for exemplars model (MEM-EX), which posits that decision makers rely on memory for previously experienced outcomes to make choices. Using MEM-EX, we demonstrate how cognitive mechanisms provide intuitive and parsimonious explanations for the effects of value-ignorance, salience, outcome order, and sample size. We also conduct a cross-validation analysis of several models within the MEM-EX framework. We compare these to three alternative models; two baseline models built on the principle of expected value maximization, and another employing a suite of choice methods previously shown to perform well in prediction tournaments. We find that MEM-EX consistently outperforms these competitors, demonstrating its value as a tool for making quantitative predictions without overfitting. We discuss the implications of these findings for our understanding of the interplay between attention, memory, and experience-based choice.

Observing effects in various contexts won't give us general psychological theories.

Generalization does not come from repeatedly observing phenomena in numerous settings, but from theories explaining what is general in those phenomena. Expecting future behavior to look like past observations is especially problematic in psychology, where behaviors change when people's knowledge changes. Psychology should thus focus on theories of people's capacity to create and apply new representations of their environments.

Consensus-based guidance for conducting and reporting multi-analyst studies.

Any large dataset can be analyzed in a number of ways, and it is possible that the use of different analysis strategies will lead to different results and conclusions. One way to assess whether the results obtained depend on the analysis strategy chosen is to employ multiple analysts and leave each of them free to follow their own approach. Here, we present consensus-based guidance for conducting and reporting such multi-analyst studies, and we discuss how broader adoption of the multi-analyst approach has the potential to strengthen the robustness of results and conclusions obtained from analyses of datasets in basic and applied research.

Arrested Theory Development: The Misguided Distinction Between Exploratory and Confirmatory Research.

Science progresses by finding and correcting problems in theories. Good theories are those that help facilitate this process by being hard to vary: They explain what they are supposed to explain, they are consistent with other good theories, and they are not easily adaptable to explain anything. Here we argue that, rather than a lack of distinction between exploratory and confirmatory research, an abundance of flexible theories is a better explanation for the current replicability problems of psychology. We also explain why popular methods-oriented solutions fail to address the real problem of flexibility. Instead, we propose that a greater emphasis on theory criticism by argument might improve replicability.

An evaluation and comparison of models of risky intertemporal choice.

Risky intertemporal choices involve choosing between options that can differ in outcomes, their probability of receipt, and the delay until receipt. To date, there has been no attempt to systematically test, compare, and evaluate theoretical models of such choices. We contribute to theory development by generating predictions from 7 models for 3 common manipulations-magnitude, certainty, and immediacy-across 6 different types of risky intertemporal choices. Qualitative and quantitative comparisons of model predictions to data from an experiment involving almost 4,000 individual choices revealed that an attribute comparison-model, newly modified to incorporate risky intertemporal choices, (the risky intertemporal choice heuristic or RITCH) provided the best account of the data. Results are consistent with growing evidence in support of attribute comparison models in the risky and intertemporal choice literatures, and suggest that the relatively poorer fits of translation-based models reflect their inability to predict the differential impact of certainty and immediacy manipulations. Future theories of risky intertemporal choice may benefit from treating risk and time as independent dimensions, and focusing on attribute-comparison rather than value-comparison processes. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

How to Change the Weight of Rare Events in Decisions From Experience.

When people make risky choices, two kinds of information are crucial: outcome values and outcome probabilities. Here, we demonstrate that the juncture at which value and probability information is provided has a fundamental effect on choice. Across four experiments involving 489 participants, we compared two decision-making scenarios: one in which value information was revealed during sampling (standard) and one in which value information was revealed after sampling (value ignorance). On average, participants made riskier choices when value information was provided after sampling. Moreover, parameter estimates from a hierarchical Bayesian implementation of cumulative-prospect theory suggested that participants overweighted rare events when value information was absent during sampling but did not overweight such events in the standard condition. This suggests that the impact of rare events on choice relies crucially on the timing of probability and value integration. We provide paths toward mechanistic explanations of our results based on frameworks that assume different underlying cognitive architectures.

How well do women understand and remember information in labour versus in late pregnancy? A pilot randomised study.

Medical informed consent is the process by which a 'competent', non-coerced individual receives sufficient information including risks of a medical procedure and gives permission for it to occur. The capacity to give an informed consent might be impaired during labour. This study aimed to examine women's abilities to understand and remember during labour. Women were prospectively recruited at 36 weeks of gestation and randomised to undertake questionnaires which assessed their ability to understand and remember information. They were randomised to: (1) information given in labour only, written format (2) information in labour, verbal (3) information at 36 weeks plus labour, written (4) information at 36 weeks plus labour, verbal. Immediate comprehension and retention was assessed at 36 weeks, in labour, and 24-72 hours after birth. Forty-nine women completed the questionnaires regarding understanding and retention of information at 36 weeks, six intrapartum, and five postpartum (90% attrition). Women receiving information at 36 weeks and in labour versus in labour had a higher comprehension of pregnancy-related information, its retention, and total score. Women receiving information in late pregnancy and labour may comprehend and retain it better than women only receiving information during labour. Given small sample size, further research is needed to support these preliminary findings. Impact statement What is already known on this subject? The evidence regarding the capacity of labouring women to give informed consent is largely based on women's self-reported experiences or expert opinions and has mixed findings. Existing guidelines recommend that an informed consent should be given antenatally for both clinical practice and research. Studies show that obtaining an informed consent antenatally is neither feasible nor widely implemented. What do the results of this study add? A novel approach to providing empirical evidence regarding women's capacity to comprehend and retain information during labour. Our study confirms the difficulty with antenatal recruitment for intrapartum research. What are the implications of these findings for clinical practice and/further research? This raises ethical concerns regarding the current intrapartum research in which consent is largely sought at the time of the study. Emphasises the need to explore the question 'Do labouring women have the capacity to consent to research?' in order to ensure that women are protected during labour.

Simultaneous underweighting and overestimation of rare events: Unpacking a paradox.

We investigated previous findings suggesting a paradoxical inconsistency of people's beliefs and choices: When making decisions under uncertainty, people seem to both overestimate the probability of rare events in their judgments and underweight the probability of the same rare events in their choices. In our reexamination, we found that people's beliefs are consistent with their decisions, but they do not necessarily correspond with the environment. Both overestimation and underweighting of the rare event seemed to result from (most, but not all) participants' mistaken belief that they can infer and exploit sequential patterns in a static environment. In addition, we found that such inaccurate representations can be improved through incentives. Finally, detailed analysis suggested a mixture of individual-level response patterns, which can give rise to an erroneous interpretation of group-level patterns. Our results offer an explanation for why beliefs and decisions can appear contradictory and present challenges to some current models of decisions under uncertainty. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

The Quality of Response Time Data Inference: A Blinded, Collaborative Assessment of the Validity of Cognitive Models.

Most data analyses rely on models. To complement statistical models, psychologists have developed cognitive models, which translate observed variables into psychologically interesting constructs. Response time models, in particular, assume that response time and accuracy are the observed expression of latent variables including 1) ease of processing, 2) response caution, 3) response bias, and 4) non-decision time. Inferences about these psychological factors, hinge upon the validity of the models' parameters. Here, we use a blinded, collaborative approach to assess the validity of such model-based inferences. Seventeen teams of researchers analyzed the same 14 data sets. In each of these two-condition data sets, we manipulated properties of participants' behavior in a two-alternative forced choice task. The contributing teams were blind to the manipulations, and had to infer what aspect of behavior was changed using their method of choice. The contributors chose to employ a variety of models, estimation methods, and inference procedures. Our results show that, although conclusions were similar across different methods, these "modeler's degrees of freedom" did affect their inferences. Interestingly, many of the simpler approaches yielded as robust and accurate inferences as the more complex methods. We recommend that, in general, cognitive models become a typical analysis tool for response time data. In particular, we argue that the simpler models and procedures are sufficient for standard experimental designs. We finish by outlining situations in which more complicated models and methods may be necessary, and discuss potential pitfalls when interpreting the output from response time models.

Empirical comparison of the adjustable spanner and the adaptive toolbox models of choice.

Past research indicates that individuals respond adaptively to contextual factors in multiattribute choice tasks. Yet it remains unclear how this adaptation is cognitively governed. In this article, empirically testable implementations of two prominent competing theoretical frameworks are developed and compared across two multiattribute choice experiments: the adaptive toolbox framework assuming discrete choice strategies and the adjustable spanner framework assuming one comprehensive adaptive strategy. Results from two experiments indicate that in the environments we tested, in which all cue information was presented openly, the toolbox makes better predictions than the adjustable spanner both in- and out-of-sample. Follow-up simulation studies indicate that it is difficult to discriminate the models based on choice outcomes alone but allowed the identification of a small subset of cases where the predictions of both models diverged. Our results suggest that people adapt their decision strategies by flexibly switching between using as little information as possible and use of all of the available information. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

When Extremists Win: Cultural Transmission Via Iterated Learning When Populations Are Heterogeneous.

How does the process of information transmission affect the cultural or linguistic products that emerge? This question is often studied experimentally and computationally via iterated learning, a procedure in which participants learn from previous participants in a chain. Iterated learning is a powerful tool because, when all participants share the same priors, the stationary distributions of the iterated learning chains reveal those priors. In many situations, however, it is unreasonable to assume that all participants share the same prior beliefs. We present four simulation studies and one experiment demonstrating that when the population of learners is heterogeneous, the behavior of an iterated learning chain can be unpredictable and is often systematically distorted by the learners with the most extreme biases. This results in group-level outcomes that reflect neither the behavior of any individuals within the population nor the overall population average. We discuss implications for the use of iterated learning as a methodological tool as well as for the processes that might have shaped cultural and linguistic evolution in the real world.

Metastudies for robust tests of theory.

We describe and demonstrate an empirical strategy useful for discovering and replicating empirical effects in psychological science. The method involves the design of a metastudy, in which many independent experimental variables-that may be moderators of an empirical effect-are indiscriminately randomized. Radical randomization yields rich datasets that can be used to test the robustness of an empirical claim to some of the vagaries and idiosyncrasies of experimental protocols and enhances the generalizability of these claims. The strategy is made feasible by advances in hierarchical Bayesian modeling that allow for the pooling of information across unlike experiments and designs and is proposed here as a gold standard for replication research and exploratory research. The practical feasibility of the strategy is demonstrated with a replication of a study on subliminal priming.

The elimination of positive priming with increasing prime duration reflects a transition from perceptual fluency to disfluency rather than bias against primed words.

With immediate repetition priming of forced choice perceptual identification, short prime durations produce positive priming (i.e., priming the target leads to higher accuracy, while priming the foil leads to lower accuracy). Many theories explain positive priming following short duration primes as reflecting increased perceptual fluency for the primed target (i.e., decreased identification latency). However, most studies only examine either accuracy or response times, rather than considering the joint constraints of response times and accuracy to properly address the role of decision biases and response caution. This is a critical oversight because several theories propose that the transition to negative priming following a long duration prime reflects a decision strategy to compensate for the effect of increased perceptual fluency. In contrast, the nROUSE model of Huber and O'Reilly (2003) explains this transition as reflecting perceptual habituation, and thus a change to perceptual disfluency. We confirmed this prediction by applying a sequential sampling model (the diffusion race model) to accuracy and response time distributions from a new single item same-different version of the priming task. In this way, we measured strategic biases and perceptual fluency in each condition for each subject. The nROUSE model was only applied to accuracy from the original forced-choice version of the priming task. This application of nROUSE produced separate predictions for each subject regarding the degree of fluency and disfluency in each condition, and these predictions were confirmed by the drift rate parameters (i.e., fluency) from the response time model in contrast to the threshold parameters (i.e., bias).

Can a single model account for both risky choices and inter-temporal choices? Testing the assumptions underlying models of risky inter-temporal choice.

There is growing interest in modelling how people make choices that involve both risks and delays, i.e., risky inter-temporal choices. We investigated an untested assumption underlying several proposed risky inter-temporal choice models: that pure risky choices and pure inter-temporal choices are special cases of risky inter-temporal choice. We tested this assumption by presenting a single group of participants with risky choices and inter-temporal choices. We then compared the performance of a model that is fit to both choice types simultaneously, with the performance of separate models fit to the risky choice and inter-temporal choice data. We find, using Bayesian model comparison, that the majority of participants are best fit by a single model that incorporates both risky and inter-temporal choices. This result supports the assumption that risky choices and inter-temporal choices may be special cases of risky inter-temporal choice. Our results also suggest that, under the conditions of our experiment, interpretation of monetary value is very similar in risky choices and inter-temporal choices.