Dual-process modeling of sequential decision making in the balloon analogue risk task.

People are often faced with repeated risky decisions that involve uncertainty. In sequential risk-taking tasks, like the Balloon Analogue Risk Task (BART), the underlying decision process is not yet fully understood. Dual-process theory proposes that human cognition involves two main families of processes, often referred to as System 1 (fast and automatic) and System 2 (slow and conscious). We cross models of the BART with different architectures of the two systems to yield a pool of computational dual-process models that are evaluated on multiple performance measures (e.g., parameter identifiability, model recovery, and predictive accuracy). Results show that the best-performing model configuration assumes the two systems are competitively connected, an evaluation process based on the Scaled Target Learning model of the BART, and an assessment rate that incorporates sensitivity to the trial number, pumping opportunity, and bias to engage in System 1. Findings also shed light on how modeling choices and response times in a dual-process framework can benefit our understanding of sequential risk-taking behavior.

Implementation of an online spacing flanker task and evaluation of its test-retest reliability using measures of inhibitory control and the distribution of spatial attention.

The flanker task (Eriksen & Eriksen, Perception & Psychophysics, 16(1), 143-149, 1974) has been highly influential and widely used in studies of visual attention. Its simplicity has made it popular to include it in experimental software packages and online platforms. The spacing flanker task (SFT), in which the distance between the target and flankers varies, is useful for studying the distribution of attention across space as well as inhibitory control. Use of the SFT requires that the viewing environment (e.g., stimulus size and viewing distance) be controlled, which is a challenge for online delivery. We implement and evaluate an online version of the SFT that includes two calibration pretests to provide the necessary control. Test-retest and split-half reliability of the online version was compared with a laboratory version on measures of inhibitory control and measures of the distribution of attention across space. Analyses show that the online SFT is comparable to laboratory testing on all measures. Results also identify two measures with good test-retest reliability that hold promise for studying performance in the SFT: the mean flanker effect (ICC = 0.745) and RTs on incongruent trials across distances (ICC = 0.65-0.71).

Assessing the distortions introduced when calculating d': A simulation approach.

The discriminability measure d ' is widely used in psychology to estimate sensitivity independently of response bias. The conventional approach to estimate d ' involves a transformation from the hit rate and the false-alarm rate. When performance is perfect, correction methods must be applied to calculate d ' , but these corrections distort the estimate. In three simulation studies, we show that distortion in d ' estimation can arise from other properties of the experimental design (number of trials, sample size, sample variance, task difficulty) that, when combined with application of the correction method, make d ' distortion in any specific experiment design complex and can mislead statistical inference in the worst cases (Type I and Type II errors). To address this problem, we propose that researchers simulate d ' estimation to explore the impact of design choices, given anticipated or observed data. An R Shiny application is introduced that estimates d ' distortion, providing researchers the means to identify distortion and take steps to minimize its impact.

The scaled target learning model: Revisiting learning in the balloon analogue risk task.

The Balloon Analogue Risk Task (BART) is a sequential decision making paradigm that assesses risk-taking behavior. Several computational models have been proposed for the BART that characterize risk-taking propensity. An aspect of task performance that has proven challenging to model is the learning that develops from experiencing wins and losses across trials, which has the potential to provide further insight into risky decision making. We developed the Scaled Target Learning (STL) model for this purpose. STL describes learning as adjustments to an individual's strategy in reaction to outcomes in the task, with the size of adjustments reflecting an individual's sensitivity to wins and losses. STL is shown to be sensitive to the learning elicited by experimental manipulations. In addition, the model matches or bests the performance of three competing models in traditional model comparison tests (e.g., parameter recovery performance, predictive accuracy, sensitivity to risk-taking propensity). Findings are discussed in the context of the learning process involved in the task. By characterizing the extent to which people are willing to adapt their strategies based on past experience, STL is a step toward a complete depiction of the psychological processes underlying sequential risk-taking behavior.

Data-driven experimental design and model development using Gaussian process with active learning.

Interest in computational modeling of cognition and behavior continues to grow. To be most productive, modelers should be equipped with tools that ensure optimal efficiency in data collection and in the integrity of inference about the phenomenon of interest. Traditionally, models in cognitive science have been parametric, which are particularly susceptible to model misspecification because their strong assumptions (e.g. parameterization, functional form) may introduce unjustified biases in data collection and inference. To address this issue, we propose a data-driven nonparametric framework for model development, one that also includes optimal experimental design as a goal. It combines Gaussian Processes, a stochastic process often used for regression and classification, with active learning, from machine learning, to iteratively fit the model and use it to optimize the design selection throughout the experiment. The approach, dubbed Gaussian process with active learning (GPAL), is an extension of the parametric, adaptive design optimization (ADO) framework (Cavagnaro, Myung, Pitt, & Kujala, 2010). We demonstrate the application and features of GPAL in a delay discounting task and compare its performance to ADO in two experiments. The results show that GPAL is a viable modeling framework that is noteworthy for its high sensitivity to individual differences, identifying novel patterns in the data that were missed by the model-constrained ADO. This investigation represents a first step towards the development of a data-driven cognitive modeling framework that serves as a middle ground between raw data, which can be difficult to interpret, and parametric models, which rely on strong assumptions.

ADOpy: a python package for adaptive design optimization.

Experimental design is fundamental to research, but formal methods to identify good designs are lacking. Advances in Bayesian statistics and machine learning offer algorithm-based ways to identify good experimental designs. Adaptive design optimization (ADO; Cavagnaro, Myung, Pitt, & Kujala, 2010; Myung, Cavagnaro, & Pitt, 2013) is one such method. It works by maximizing the informativeness and efficiency of data collection, thereby improving inference. ADO is a general-purpose method for conducting adaptive experiments on the fly and can lead to rapid accumulation of information about the phenomenon of interest with the fewest number of trials. The nontrivial technical skills required to use ADO have been a barrier to its wider adoption. To increase its accessibility to experimentalists at large, we introduce an open-source Python package, ADOpy, that implements ADO for optimizing experimental design. The package, available on GitHub, is written using high-level modular-based commands such that users do not have to understand the computational details of the ADO algorithm. In this paper, we first provide a tutorial introduction to ADOpy and ADO itself, and then illustrate its use in three walk-through examples: psychometric function estimation, delay discounting, and risky choice. Simulation data are also provided to demonstrate how ADO designs compare with other designs (random, staircase).

Neurophysiology underlying influence of stimulus reliability on audiovisual integration.

We tested the predictions of the dynamic reweighting model (DRM) of audiovisual (AV) speech integration, which posits that spectrotemporally reliable (informative) AV speech stimuli induce a reweighting of processing from low-level to high-level auditory networks. This reweighting decreases sensitivity to acoustic onsets and in turn increases tolerance to AV onset asynchronies (AVOA). EEG was recorded while subjects watched videos of a speaker uttering trisyllabic nonwords that varied in spectrotemporal reliability and asynchrony of the visual and auditory inputs. Subjects judged the stimuli as in-sync or out-of-sync. Results showed that subjects exhibited greater AVOA tolerance for non-blurred than blurred visual speech and for less than more degraded acoustic speech. Increased AVOA tolerance was reflected in reduced amplitude of the P1-P2 auditory evoked potentials, a neurophysiological indication of reduced sensitivity to acoustic onsets and successful AV integration. There was also sustained visual alpha band (8-14 Hz) suppression (desynchronization) following acoustic speech onsets for non-blurred vs. blurred visual speech, consistent with continuous engagement of the visual system as the speech unfolds. The current findings suggest that increased spectrotemporal reliability of acoustic and visual speech promotes robust AV integration, partly by suppressing sensitivity to acoustic onsets, in support of the DRM's reweighting mechanism. Increased visual signal reliability also sustains the engagement of the visual system with the auditory system to maintain alignment of information across modalities.

Evaluating the performance of the quick CSF method in detecting contrast sensitivity function changes.

The contrast sensitivity function (CSF) has shown promise as a functional vision endpoint for monitoring the changes in functional vision that accompany eye disease or its treatment. However, detecting CSF changes with precision and efficiency at both the individual and group levels is very challenging. By exploiting the Bayesian foundation of the quick CSF method (Lesmes, Lu, Baek, & Albright, 2010), we developed and evaluated metrics for detecting CSF changes at both the individual and group levels. A 10-letter identification task was used to assess the systematic changes in the CSF measured in three luminance conditions in 112 naïve normal observers. The data from the large sample allowed us to estimate the test-retest reliability of the quick CSF procedure and evaluate its performance in detecting CSF changes at both the individual and group levels. The test-retest reliability reached 0.974 with 50 trials. In 50 trials, the quick CSF method can detect a medium 0.30 log unit area under log CSF change with 94.0% accuracy at the individual observer level. At the group level, a power analysis based on the empirical distribution of CSF changes from the large sample showed that a very small area under log CSF change (0.025 log unit) could be detected by the quick CSF method with 112 observers and 50 trials. These results make it plausible to apply the method to monitor the progression of visual diseases or treatment effects on individual patients and greatly reduce the time, sample size, and costs in clinical trials at the group level.

A hierarchical Bayesian approach to adaptive vision testing: A case study with the contrast sensitivity function.

Measurement efficiency is of concern when a large number of observations are required to obtain reliable estimates for parametric models of vision. The standard entropy-based Bayesian adaptive testing procedures addressed the issue by selecting the most informative stimulus in sequential experimental trials. Noninformative, diffuse priors were commonly used in those tests. Hierarchical adaptive design optimization (HADO; Kim, Pitt, Lu, Steyvers, & Myung, 2014) further improves the efficiency of the standard Bayesian adaptive testing procedures by constructing an informative prior using data from observers who have already participated in the experiment. The present study represents an empirical validation of HADO in estimating the human contrast sensitivity function. The results show that HADO significantly improves the accuracy and precision of parameter estimates, and therefore requires many fewer observations to obtain reliable inference about contrast sensitivity, compared to the method of quick contrast sensitivity function (Lesmes, Lu, Baek, & Albright, 2010), which uses the standard Bayesian procedure. The improvement with HADO was maintained even when the prior was constructed from heterogeneous populations or a relatively small number of observers. These results of this case study support the conclusion that HADO can be used in Bayesian adaptive testing by replacing noninformative, diffuse priors with statistically justified informative priors without introducing unwanted bias.

Putative mechanisms mediating tolerance for audiovisual stimulus onset asynchrony.

Audiovisual (AV) speech perception is robust to temporal asynchronies between visual and auditory stimuli. We investigated the neural mechanisms that facilitate tolerance for audiovisual stimulus onset asynchrony (AVOA) with EEG. Individuals were presented with AV words that were asynchronous in onsets of voice and mouth movement and judged whether they were synchronous or not. Behaviorally, individuals tolerated (perceived as synchronous) longer AVOAs when mouth movement preceded the speech (V-A) stimuli than when the speech preceded mouth movement (A-V). Neurophysiologically, the P1-N1-P2 auditory evoked potentials (AEPs), time-locked to sound onsets and known to arise in and surrounding the primary auditory cortex (PAC), were smaller for the in-sync than the out-of-sync percepts. Spectral power of oscillatory activity in the beta band (14-30 Hz) following the AEPs was larger during the in-sync than out-of-sync perception for both A-V and V-A conditions. However, alpha power (8-14 Hz), also following AEPs, was larger for the in-sync than out-of-sync percepts only in the V-A condition. These results demonstrate that AVOA tolerance is enhanced by inhibiting low-level auditory activity (e.g., AEPs representing generators in and surrounding PAC) that code for acoustic onsets. By reducing sensitivity to acoustic onsets, visual-to-auditory onset mapping is weakened, allowing for greater AVOA tolerance. In contrast, beta and alpha results suggest the involvement of higher-level neural processes that may code for language cues (phonetic, lexical), selective attention, and binding of AV percepts, allowing for wider neural windows of temporal integration, i.e., greater AVOA tolerance.