Estimating the proportions and latencies of reaction time outliers: A pooling method and case study of lexical decision tasks.

A methodological problem in most reaction time (RT) studies is that some measured RTs may be outliers-that is, they may be very fast or very slow for reasons unconnected to the task-related processing of interest. Numerous ad hoc methods have been suggested to discriminate between such outliers and the valid RTs of interest, but it is extremely difficult to determine how well these methods work in practice because virtually nothing is known about the actual characteristics of outliers in real RT datasets. This article proposes a new method of pooling cumulative distribution function values for examining empirical RT distributions to assess both the proportions of outliers and their latencies relative to those of the valid RTs. As the method is developed, its strengths and weaknesses are examined using simulations based on previously suggested ad hoc models for RT outliers with particular assumed proportions and distributions of valid RTs and outliers. The method is then applied to several large RT datasets from lexical decision tasks, and the results provide the first empirically based description of outlier RTs. For these datasets, fewer than 1% of the RTs seem to be outliers, and the median outlier latency appears to be approximately 4-6 standard deviations of RT above the mean of the valid RT distribution.

Percentile rank pooling: A simple nonparametric method for comparing group reaction time distributions with few trials.

Although many studies of reaction time (RT) focus on a single measure of central tendency such as the mean RT, a more detailed picture of the underlying processes can be gained by looking at full distributions of RTs. Unfortunately, for practical reasons it is sometimes difficult to obtain enough trials per participant in a condition of interest to construct such a distribution with existing methods. The purpose of this article is to propose a method of forming group RT distributions that can be used to compare the full distributions of RTs even in an infrequent condition with only a few trials per participant. In brief, the percentile ranks of each participant's infrequent-condition RTs are scored relative to a larger pool including that participant's RTs in other conditions, and a histogram of the infrequent-condition's percentile ranks is then formed by pooling across participants. The resulting histogram of infrequent-condition RT ranks shows where the RTs in that condition tend to fall relative to the other conditions, and this histogram can reveal systematic patterns in the infrequent-condition's RT distribution. To illustrate the method, I present histograms of the ranks of infrequent error RTs (~ 5% of trials), ranked relative to correct responses, in real data sets from Simon and lexical decision tasks.

The Worst Performance Rule, or the Not-Best Performance Rule? Latent-Variable Analyses of Working Memory Capacity, Mind-Wandering Propensity, and Reaction Time.

The worst performance rule (WPR) is a robust empirical finding reflecting that people's worst task performance shows numerically stronger correlations with cognitive ability than their average or best performance. However, recent meta-analytic work has proposed this be renamed the "not-best performance" rule because mean and worst performance seem to predict cognitive ability to similar degrees, with both predicting ability better than best performance. We re-analyzed data from a previously published latent-variable study to test for worst vs. not-best performance across a variety of reaction time tasks in relation to two cognitive ability constructs: working memory capacity (WMC) and propensity for task-unrelated thought (TUT). Using two methods of assessing worst performance-ranked-binning and ex-Gaussian-modeling approaches-we found evidence for both the worst and not-best performance rules. WMC followed the not-best performance rule (correlating equivalently with mean and longest response times (RTs)) but TUT propensity followed the worst performance rule (correlating more strongly with longest RTs). Additionally, we created a mini-multiverse following different outlier exclusion rules to test the robustness of our findings; our findings remained stable across the different multiverse iterations. We provisionally conclude that the worst performance rule may only arise in relation to cognitive abilities closely linked to (failures of) sustained attention.

Visual versus auditory Simon effect: A behavioural and physiological investigation.

This study investigated whether the visual and auditory Simon effects could be accounted for by the same mechanism. In a single experiment, we performed a detailed comparison of the visual and the auditory Simon effects arising in behavioural responses and in pupil dilation, a psychophysiological measure considered as a marker of the cognitive effort induced by conflict processing. To address our question, we performed sequential and distributional analyses on both reaction times and pupil dilation. Results confirmed that the mechanisms underlying the visual and auditory Simon effects are functionally equivalent in terms of the interaction between unconditional and conditional response processes. The two modalities, however, differ with respect to the strength of their activation and inhibition. Importantly, pupillary data mirrored the pattern observed in behavioural data for both tasks, adding physiological evidence to the current literature on the processing of visual and auditory information in a conflict task.