Dissecting Bayes: Using influence measures to test normative use of probability density information derived from a sample.

Bayesian decision theory (BDT) is frequently used to model normative performance in perceptual, motor, and cognitive decision tasks where the possible outcomes of actions are associated with rewards or penalties. The resulting normative models specify how decision makers should encode and combine information about uncertainty and value-step by step-in order to maximize their expected reward. When prior, likelihood, and posterior are probabilities, the Bayesian computation requires only simple arithmetic operations: addition, etc. We focus on visual cognitive tasks where Bayesian computations are carried out not on probabilities but on (1) probability density functions and (2) these probability density functions are derived from samples. We break the BDT model into a series of computations and test human ability to carry out each of these computations in isolation. We test three necessary properties of normative use of pdf information derived from a sample-accuracy, additivity and influence. Influence measures allow us to assess how much weight each point in the sample is assigned in making decisions and allow us to compare normative use (weighting) of samples to actual, point by point. We find that human decision makers violate accuracy and additivity systematically but that the cost of failure in accuracy or additivity would be minor in common decision tasks. However, a comparison of measured influence for each sample point with normative influence measures demonstrates that the individual's use of sample information is markedly different from the predictions of BDT. We will show that the normative BDT model takes into account the geometric symmetries of the pdf while the human decision maker does not. An alternative model basing decisions on a single extreme sample point provided a better account for participants' data than the normative BDT model.

Lightness constancy in reality, in virtual reality, and on flat-panel displays.

Virtual reality (VR) displays are being used in an increasingly wide range of applications. However, previous work shows that viewers often perceive scene properties very differently in real and virtual environments and so realistic perception of virtual stimuli should always be a carefully tested conclusion, not an assumption. One important property for realistic scene perception is surface color. To evaluate how well virtual platforms support realistic perception of achromatic surface color, we assessed lightness constancy in a physical apparatus with real lights and surfaces, in a commercial VR headset, and on a traditional flat-panel display. We found that lightness constancy was good in all three environments, though significantly better in the real environment than on the flat-panel display. We also found that variability across observers was significantly greater in VR and on the flat-panel display than in the physical environment. We conclude that these discrepancies should be taken into account in applications where realistic perception is critical but also that in many cases VR can be used as a flexible alternative to flat-panel displays and a reasonable proxy for real environments.

A comparison of human and GPT-4 use of probabilistic phrases in a coordination game.

English speakers use probabilistic phrases such as likely to communicate information about the probability or likelihood of events. Communication is successful to the extent that the listener grasps what the speaker means to convey and, if communication is successful, individuals can potentially coordinate their actions based on shared knowledge about uncertainty. We first assessed human ability to estimate the probability and the ambiguity (imprecision) of twenty-three probabilistic phrases in a coordination game in two different contexts, investment advice and medical advice. We then had GPT-4 (OpenAI), a Large Language Model, complete the same tasks as the human participants. We found that GPT-4's estimates of probability both in the Investment and Medical Contexts were as close or closer to that of the human participants as the human participants' estimates were to one another. However, further analyses of residuals disclosed small but significant differences between human and GPT-4 performance. Human probability estimates were compressed relative to those of GPT-4. Estimates of probability for both the human participants and GPT-4 were little affected by context. We propose that evaluation methods based on coordination games provide a systematic way to assess what GPT-4 and similar programs can and cannot do.

Detecting visual texture patterns in binary sequences through pattern features.

We measured human ability to detect texture patterns in a signal detection task. Observers viewed sequences of 20 blue or yellow tokens placed horizontally in a row. They attempted to discriminate sequences generated by a random generator ("a fair coin") from sequences produced by a disrupted Markov sequence (DMS) generator. The DMSs were generated in two stages: first a sequence was generated using a Markov chain with probability, pr = 0.9, that a token would be the same color as the token to its left. The Markov sequence was then disrupted by flipping each token from blue to yellow or vice versa with probability, pd-the probability of disruption. Disruption played the role of noise in signal detection terms. We can frame what observers are asked to do as detecting Markov texture patterns disrupted by noise. The experiment included three conditions differing in pd (0.1, 0.2, 0.3). Ninety-two observers participated, each in only one condition. Overall, human observers' sensitivities to texture patterns (d' values) were markedly less than those of an optimal Bayesian observer. We considered the possibility that observers based their judgments not on the entire texture sequence but on specific features of the sequences such as the length of the longest repeating subsequence. We compared human performance with that of multiple optimal Bayesian classifiers based on such features. We identify the single- and multiple-feature models that best match the performance of observers across conditions and develop a pattern feature pool model for the signal detection task considered.

Two sources of uncertainty independently modulate temporal expectancy.

The environment is shaped by two sources of temporal uncertainty: the discrete probability of whether an event will occur and-if it does-the continuous probability of when it will happen. These two types of uncertainty are fundamental to every form of anticipatory behavior including learning, decision-making, and motor planning. It remains unknown how the brain models the two uncertainty parameters and how they interact in anticipation. It is commonly assumed that the discrete probability of whether an event will occur has a fixed effect on event expectancy over time. In contrast, we first demonstrate that this pattern is highly dynamic and monotonically increases across time. Intriguingly, this behavior is independent of the continuous probability of when an event will occur. The effect of this continuous probability on anticipation is commonly proposed to be driven by the hazard rate (HR) of events. We next show that the HR fails to account for behavior and propose a model of event expectancy based on the probability density function of events. Our results hold for both vision and audition, suggesting independence of the representation of the two uncertainties from sensory input modality. These findings enrich the understanding of fundamental anticipatory processes and have provocative implications for many aspects of behavior and its neural underpinnings.

The bounded rationality of probability distortion.

In decision making under risk (DMR) participants' choices are based on probability values systematically different from those that are objectively correct. Similar systematic distortions are found in tasks involving relative frequency judgments (JRF). These distortions limit performance in a wide variety of tasks and an evident question is, Why do we systematically fail in our use of probability and relative frequency information? We propose a bounded log-odds model (BLO) of probability and relative frequency distortion based on three assumptions: 1) log-odds: probability and relative frequency are mapped to an internal log-odds scale, 2) boundedness: the range of representations of probability and relative frequency are bounded and the bounds change dynamically with task, and 3) variance compensation: the mapping compensates in part for uncertainty in probability and relative frequency values. We compared human performance in both DMR and JRF tasks to the predictions of the BLO model as well as 11 alternative models, each missing one or more of the underlying BLO assumptions (factorial model comparison). The BLO model and its assumptions proved to be superior to any of the alternatives. In a separate analysis, we found that BLO accounts for individual participants' data better than any previous model in the DMR literature. We also found that, subject to the boundedness limitation, participants' choice of distortion approximately maximized the mutual information between objective task-relevant values and internal values, a form of bounded rationality.

Measuring and Modeling Visual Appearance.

In studying visual perception, we seek to develop models of processing that accurately predict perceptual judgments. Much of this work is focused on judgments of discrimination, and there is a large literature concerning models of visual discrimination. There are, however, non-threshold visual judgments, such as judgments of the magnitude of differences between visual stimuli, that provide a means to bridge the gap between threshold and appearance. We describe two such models of suprathreshold judgments, maximum likelihood difference scaling and maximum likelihood conjoint measurement, and review recent literature that has exploited them.

The anticipation of events in time.

Humans anticipate events signaled by sensory cues. It is commonly assumed that two uncertainty parameters modulate the brain's capacity to predict: the hazard rate (HR) of event probability and the uncertainty in time estimation which increases with elapsed time. We investigate both assumptions by presenting event probability density functions (PDFs) in each of three sensory modalities. We show that perceptual systems use the reciprocal PDF and not the HR to model event probability density. We also demonstrate that temporal uncertainty does not necessarily grow with elapsed time but can also diminish, depending on the event PDF. Previous research identified neuronal activity related to event probability in multiple levels of the cortical hierarchy (sensory (V4), association (LIP), motor and other areas) proposing the HR as an elementary neuronal computation. Our results-consistent across vision, audition, and somatosensation-suggest that the neurobiological implementation of event anticipation is based on a different, simpler and more stable computation than HR: the reciprocal PDF of events in time.

Sub-optimality in motor planning is not improved by explicit observation of motor uncertainty.

To make optimal decisions under risk, one must correctly weight potential rewards and penalties by the probabilities of receiving them. In motor decision tasks, the uncertainty in outcome is a consequence of motor uncertainty. When participants perform suboptimally as they often do in such tasks, it could be because they have insufficient information about their motor uncertainty: with more information, their performance could converge to optimal as they learn their own motor uncertainty. Alternatively, their suboptimal performance may reflect an inability to make use of the information they have or even to perform the correct computations. To discriminate between these two possibilities, we performed an experiment spanning two days. On the first day, all participants performed a reaching task with trial-by-trial feedback of motor error. At the end of the day, their aim points were still typically suboptimal. On the second day participants were divided into two groups one of which repeated the task of the first day and the other of which repeated the task but were intermittently given additional information summarizing their motor errors. Participants receiving additional information did not perform significantly better than those who did not.

Generality of likelihood ratio decisions.

A basic assumption of Signal Detection Theory - a special case of Bayesian Decision Theory - is that decisions are based on likelihood ratios (the likelihood ratio hypothesis). In a preceding paper, Glanzer et al. (2009) tested this assumption in recognition memory tasks. The tests consisted of formal proofs and computational demonstrations that decisions based on likelihood ratios produce three regularities (1. the Mirror Effect, 2. the Variance Effect, and 3. the z-ROC Length Effect). Glanzer et al. found that the three implied regularities do indeed hold for a wide range of item recognition memory studies taken from the literature. We now claim that the likelihood ratio regularities hold for decisions generally: decisions about sensory events, reasoning, weather forecasting, etc. An examination of past decision studies supports the generalization. We also report new experimental studies of decisions in two additional areas, semantic memory and mental rotation, further supporting the generalization. The results highlight the optimal characteristics of decision making in contrast to the current emphasis on its inefficiencies.

Efficient visual information sampling develops late in childhood.

It is often unclear which course of action gives the best outcome. We can reduce this uncertainty by gathering more information, but gathering information always comes at a cost. For example, a sports player waiting too long to judge a ball's trajectory will run out of time to intercept it. Efficient samplers must therefore optimize a trade-off: when the costs of collecting further information exceed the expected benefits, they should stop sampling and start acting. In visually guided tasks, adults can make these trade-offs efficiently, correctly balancing any reductions in visuomotor uncertainty against cost factors associated with increased sampling. To investigate how this ability develops during childhood, we tested 6- to 11-year-olds, adolescents, and adults on a visual localization task in which the costs and benefits of sampling were formalized in a quantitative framework. This allowed us to compare participants to each other and to an ideal observer who maximizes expected reward. Visual sampling became substantially more efficient between 6 and 11 years, converging onto adult performance in adolescence. Younger children systematically undersampled information relative to the ideal observer and varied their sampling strategy more. Further analyses suggested that young children used a suboptimal decision rule that insufficiently accounted for the chance of task failure, in line with a late developing ability to compute with probabilities and costs. We therefore propose that late development of efficient information sampling, a crucial element of real-world decision-making under risk, may form an important component of suboptimality in child perception, action, and decision-making. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

One mirror effect: The regularities of recognition memory.

The mirror effect is a pattern of results generally found in two-condition recognition memory experiments that is consistent with normative signal detection theory as a model of recognition. However, the claim has been made that there is a distinct mirror effect, the "strength mirror effect," that differs from the normative one. This claim is based on experiments on recognition memory in which repetition or study time is varied to produce differences in accuracy, where typically the ordinary mirror effect pattern is absent. If this claim is correct, it has major implications for theories of recognition memory. Therefore, a full examination of the data that support the claim was called for. To do that, we replicated the basic demonstration of the no-mirror-effect data and analyzed it further in a series of experiments. The analysis showed the following: (1) Whether or not the mirror effect occurs is determined by whether the experimenter furnishes effective discriminanda that distinguish the weak and strong conditions for the participant. (2) Once Finding 1 is taken into account, no adjustments of or additions to the normative signal detection theory explanations are necessary. (3) There is only one mirror effect, and no separate "strength mirror effect."

Information sampling behavior with explicit sampling costs.

The decision to gather information should take into account both the value of information and its accrual costs in time, energy and money. Here we explore how people balance the monetary costs and benefits of gathering additional information in a perceptual-motor estimation task. Participants were rewarded for touching a hidden circular target on a touch-screen display. The target's center coincided with the mean of a circular Gaussian distribution from which participants could sample repeatedly. Each "cue" - sampled one at a time - was plotted as a dot on the display. Participants had to repeatedly decide, after sampling each cue, whether to stop sampling and attempt to touch the hidden target or continue sampling. Each additional cue increased the participants' probability of successfully touching the hidden target but reduced their potential reward. Two experimental conditions differed in the initial reward associated with touching the hidden target and the fixed cost per cue. For each condition we computed the optimal number of cues that participants should sample, before taking action, to maximize expected gain. Contrary to recent claims that people gather less information than they objectively should before taking action, we found that participants over-sampled in one experimental condition, and did not significantly under- or over-sample in the other. Additionally, while the ideal observer model ignores the current sample dispersion, we found that participants used it to decide whether to stop sampling and take action or continue sampling, a possible consequence of imperfect learning of the underlying population dispersion across trials.

Gambling on visual performance: neural correlates of metacognitive choice between visual lotteries.

A lottery is a list of mutually exclusive outcomes together with their associated probabilities of occurrence. Decision making is often modeled as choices between lotteries and-in typical research on decision under risk-the probabilities are given to the subject explicitly in numerical form. In this study, we examined lottery decision task where the probabilities of receiving various rewards are contingent on the subjects' own visual performance in a random-dot-motion (RDM) discrimination task, a metacognitive or second order judgment. While there is a large literature concerning the RDM task and there is also a large literature on decision under risk, little is known about metacognitive decisions when the source of uncertainty is visual. Using fMRI with humans, we found distinct fronto-striatal and fronto-parietal networks representing subjects' estimates of his or her performance, reward value, and the expected value (EV) of the lotteries. The fronto-striatal network includes the dorsomedial prefrontal cortex and the ventral striatum, involved in reward processing and value-based decision-making. The fronto-parietal network includes the intraparietal sulcus and the ventrolateral prefrontal cortex, which was shown to be involved in the accumulation of sensory evidence during visual decision making and in metacognitive judgments on visual performance. These results demonstrate that-while valuation of performance-based lotteries involves a common fronto-striatal valuation network-an additional network unique to the estimation of task-related performance is recruited for the integration of probability and reward information when probability is inferred from visual performance.

Allocentric kin recognition is not affected by facial inversion.

Typical judgments involving faces are disrupted by inversion, with the Thatcher illusion serving as a compelling example. In two experiments, we examined how inversion affects allocentric kin recognition-the ability to judge the degree of genetic relatedness of others. In the first experiment, participants judged whether pairs of photographs of children portrayed siblings or unrelated children. Half of the pairs were siblings, half were unrelated. In three experimental conditions, photographs were viewed in upright orientation, flipped around a horizontal axis, or rotated 180°. Neither rotation nor flipping had any detectable effect on allocentric kin recognition. In the second experiment, participants judged pairs of photographs of adult women. Half of the pairs were sisters, half were unrelated. We again found no significant effect of facial inversion. Unlike almost all other face judgments, judgments of kinship from facial appearance do not rely on perceptual cues disrupted by inversion, suggesting that they rely more on spatially localized cues rather than "holistic" cues. We conclude that kin recognition is not simply a byproduct of other face perception abilities. We discuss the implications for cue combination models of other facial judgments that are affected by inversion.

Acquisition, representation, and transfer of models of visuo-motor error.

We examined how human subjects acquire and represent models of visuo-motor error and how they transfer information about visuo-motor error from one task to a closely related one. The experiment consisted of three phases. In the training phase, subjects threw beanbags underhand towards targets displayed on a wall-mounted touch screen. The distribution of their endpoints was a vertically elongated bivariate Gaussian. In the subsequent choice phase, subjects repeatedly chose which of two targets varying in shape and size they would prefer to attempt to hit. Their choices allowed us to investigate their internal models of visuo-motor error distribution, including the coordinate system in which they represented visuo-motor error. In the transfer phase, subjects repeated the choice phase from a different vantage point, the same distance from the screen but with the throwing direction shifted 45°. From the new vantage point, visuo-motor error was effectively expanded horizontally by √2. We found that subjects incorrectly assumed an isotropic distribution in the choice phase but that the anisotropy they assumed in the transfer phase agreed with an objectively correct transfer. We also found that the coordinate system used in coding two-dimensional visuo-motor error in the choice phase was effectively one-dimensional.

Human representation of visuo-motor uncertainty as mixtures of orthogonal basis distributions.

In many laboratory visuo-motor decision tasks, subjects compensate for their own visuo-motor error, earning close to the maximum reward possible. To do so, they must combine information about the distribution of possible error with values associated with different movement outcomes. The optimal solution is a potentially difficult computation that presupposes knowledge of the probability density function (pdf) of visuo-motor error associated with each possible planned movement. It is unclear how the brain represents such pdfs or computes with them. In three experiments, we used a forced-choice method to reveal subjects' internal representations of their spatial visuo-motor error in a speeded reaching movement. Although subjects' objective distributions were unimodal, close to Gaussian, their estimated internal pdfs were typically multimodal and were better described as mixtures of a small number of distributions differing only in location and scale. Mixtures of a small number of uniform distributions outperformed other mixture distributions, including mixtures of Gaussians.

Three regularities of recognition memory: the role of bias.

A basic assumption of Signal Detection Theory is that decisions are made on the basis of likelihood ratios. In a preceding paper, Glanzer, Hilford, and Maloney (Psychonomic Bulletin & Review, 16, 431-455, 2009) showed that the likelihood ratio assumption implies that three regularities will occur in recognition memory: (1) the Mirror Effect, (2) the Variance Effect, (3) the normalized Receiver Operating Characteristic (z-ROC) Length Effect. The paper offered formal proofs and computational demonstrations that decisions based on likelihood ratios produce the three regularities. A survey of data based on group ROCs from 36 studies validated the likelihood ratio assumption by showing that its three implied regularities are ubiquitous. The study noted, however, that bias, another basic factor in Signal Detection Theory, can obscure the Mirror Effect. In this paper we examine how bias affects the regularities at the theoretical level. The theoretical analysis shows: (1) how bias obscures the Mirror Effect, not the other two regularities, and (2) four ways to counter that obscuring. We then report the results of five experiments that support the theoretical analysis. The analyses and the experimental results also demonstrate: (1) that the three regularities govern individual, as well as group, performance, (2) alternative explanations of the regularities are ruled out, and (3) that Signal Detection Theory, correctly applied, gives a simple and unified explanation of recognition memory data.

Revealing individual differences in strategy selection through visual motion extrapolation.

Humans are constantly challenged to make use of internal models to fill in missing sensory information. We measured human performance in a simple motion extrapolation task where no feedback was provided in order to elucidate the models of object motion incorporated into observers' extrapolation strategies. There was no "right" model for extrapolation in this task. Observers consistently adopted one of two models, linear or quadratic, but different observers chose different models. We further demonstrate that differences in motion sensitivity impact the choice of internal models for many observers. These results demonstrate that internal models and individual differences in those models can be elicited by unconstrained, predictive-based psychophysical tasks.