A quantitative confidence signal detection model: 2. Confidence analysis.

Decision making is a fundamental subfield within neuroscience. While recent findings have yielded major advances in our understanding of decision making, confidence in such decisions remains poorly understood. In this paper, we present a confidence signal detection (CSD) model that combines a standard signal detection model yielding a noisy decision variable with a model of confidence. The CSD model requires quantitative measures of confidence obtained by recording confidence probability judgments. Specifically, we model confidence probability judgments for binary direction recognition (e.g., did I move left or right) decisions. We use our CSD model to study both confidence calibration (i.e., how does confidence compare with performance) and the distributions of confidence probability judgments. We evaluate two variants of our CSD model: a conventional model with two free parameters (CSD2) that assumes that confidence is well calibrated and our new model with three free parameters (CSD3) that includes an additional confidence scaling factor. On average, our CSD2 and CSD3 models explain 73 and 82%, respectively, of the variance found in our empirical data set. Furthermore, for our large data sets consisting of 3,600 trials per subject, correlation and residual analyses suggest that the CSD3 model better explains the predominant aspects of the empirical data than the CSD2 model, especially for subjects whose confidence is not well calibrated. Moreover, simulations show that asymmetric confidence distributions can lead traditional confidence calibration analyses to suggest "underconfidence" even when confidence is perfectly calibrated. These findings show that this CSD model can be used to help improve our understanding of confidence and decision making.NEW & NOTEWORTHY We make life-or-death decisions each day; our actions depend on our "confidence." Though confidence, accuracy, and response time are the three pillars of decision making, we know little about confidence. In a previous paper, we presented a new model - dependent on a single scaling parameter - that transforms decision variables to confidence. Here we show that this model explains the empirical human confidence distributions obtained during a vestibular direction recognition task better than standard signal detection models.

When uncertain, does human self-motion decision-making fully utilize complete information?

When forced to choose humans often feel uncertain. Investigations of human perceptual decision-making often employ signal detection theory, which assumes that even when uncertain all available information is fully utilized. However, other studies have suggested or assumed that, when uncertain, human subjects guess totally at random, ignoring available information. When uncertain, do humans simply guess totally at random? Or do humans fully utilize complete information? Or does behavior fall between these two extremes yielding "above chance" performance without fully utilizing complete information? While it is often assumed complete information is fully utilized, even when uncertain, to our knowledge this has never been experimentally confirmed. To answer this question, we combined numerical simulations, theoretical analyses, and human studies performed using a self-motion direction-recognition perceptual decision-making task (did I rotate left or right?). Subjects were instructed to make forced-choice binary (left/right) and trinary (left/right/uncertain) decisions when cued following each stimulus. Our results show that humans 1) do not guess at random when uncertain and 2) make binary and trinary decisions equally well. These findings show that humans fully utilize complete information when uncertain for our perceptual decision-making task. This helps unify signal detection theory and other models of forced-choice decision-making which allow for uncertain responses. NEW & NOTEWORTHY Humans make many perceptual decisions every day. But what if we are uncertain? While many studies assume that humans fully utilize complete information, other studies have suggested and/or assumed that when we're uncertain and forced to decide, information is not fully utilized. While humans tend to perform above chance when uncertain, no earlier study has tested whether available information is fully utilized. Our results show that humans make fully informed decisions even when uncertain.

A quantitative confidence signal detection model: 1. Fitting psychometric functions.

Perceptual thresholds are commonly assayed in the laboratory and clinic. When precision and accuracy are required, thresholds are quantified by fitting a psychometric function to forced-choice data. The primary shortcoming of this approach is that it typically requires 100 trials or more to yield accurate (i.e., small bias) and precise (i.e., small variance) psychometric parameter estimates. We show that confidence probability judgments combined with a model of confidence can yield psychometric parameter estimates that are markedly more precise and/or markedly more efficient than conventional methods. Specifically, both human data and simulations show that including confidence probability judgments for just 20 trials can yield psychometric parameter estimates that match the precision of those obtained from 100 trials using conventional analyses. Such an efficiency advantage would be especially beneficial for tasks (e.g., taste, smell, and vestibular assays) that require more than a few seconds for each trial, but this potential benefit could accrue for many other tasks.

Determining thresholds using adaptive procedures and psychometric fits: evaluating efficiency using theory, simulations, and human experiments.

When measuring thresholds, careful selection of stimulus amplitude can increase efficiency by increasing the precision of psychometric fit parameters (e.g., decreasing the fit parameter error bars). To find efficient adaptive algorithms for psychometric threshold ("sigma") estimation, we combined analytic approaches, Monte Carlo simulations, and human experiments for a one-interval, binary forced-choice, direction-recognition task. To our knowledge, this is the first time analytic results have been combined and compared with either simulation or human results. Human performance was consistent with theory and not significantly different from simulation predictions. Our analytic approach provides a bound on efficiency, which we compared against the efficiency of standard staircase algorithms, a modified staircase algorithm with asymmetric step sizes, and a maximum likelihood estimation (MLE) procedure. Simulation results suggest that optimal efficiency at determining threshold is provided by the MLE procedure targeting a fraction correct level of 0.92, an asymmetric 4-down, 1-up staircase targeting between 0.86 and 0.92 or a standard 6-down, 1-up staircase. Psychometric test efficiency, computed by comparing simulation and analytic results, was between 41 and 58% for 50 trials for these three algorithms, reaching up to 84% for 200 trials. These approaches were 13-21% more efficient than the commonly used 3-down, 1-up symmetric staircase. We also applied recent advances to reduce accuracy errors using a bias-reduced fitting approach. Taken together, the results lend confidence that the assumptions underlying each approach are reasonable and that human threshold forced-choice decision making is modeled well by detection theory models and mimics simulations based on detection theory models.

Effect of reduced cutaneous cues on motion perception and postural control.

To investigate whether the sensory perception could be a more direct assessment of sensory deficit as oppose to the postural performance, we examined the effect of reduced cutaneous cues on motion perception and motion control. The subject was translated in a mediolateral direction with a single sinusoidal acceleration at a stimulus frequency of 0.25 Hz with a peak acceleration magnitude ranging from 0.25 to 8 mG in the dark. Two different plantar cutaneous conditions were provided: the control condition (barefoot) and the reduced cutaneous condition (foot on a spongy surface). For each foot-sole sensory condition, the subject completed six sets of 33 randomly ordered translation stimuli. After each translational stimulus, the subject reported their perceived direction of motion by pressing a hand-held button. The center of pressure (COP) and joint kinematics of the quiet stance were also measured. The results showed a significant increase in perception threshold as well as COP variation in the anteroposterior direction in the reduced cutaneous cue trials. However, a non-significant increase in COP in the mediolateral direction was shown. Multivariate covariance analysis of joint kinematics showed changes in postural coordination, such as increased reliance on hip strategy under reduced cutaneous cues condition, that have not been differentiated by univariate measures. The observed discrepancy in the significance of the contribution of plantar cutaneous cues to the detection threshold and the COP variation implies that the 'perception' could provide more direct and sensitive assessment of the sensory degradation than the 'action'.