The visual environment and attention in decision making.

Visual attention is a fundamental aspect of most everyday decisions, and governments and companies spend vast resources competing for the attention of decision makers. In natural environments, choice options differ on a variety of visual factors, such as salience, position, or surface size. However, most decision theories ignore such visual factors, focusing on cognitive factors such as preferences as determinants of attention. To provide a systematic review of how the visual environment guides attention we meta-analyze 122 effect sizes on eye movements in decision making. A psychometric meta-analysis and Top10 sensitivity analysis show that visual factors play a similar or larger role than cognitive factors in determining attention. The visual factors that most influence attention are positioning information centrally, ρ = .43 (Top10 = .67), increasing the surface size, ρ = .35 (Top10 = .43), reducing the set size of competing information elements, ρ = .24 (Top10 = .24), and increasing visual salience, ρ = .13 (Top10 = .24). Cognitive factors include attending more to preferred choice options and attributes, ρ = .36 (Top10 = .31), effects of task instructions on attention, ρ = .35 (Top10 = .21), and attending more to the ultimately chosen option, ρ = .59 (Top10 = .26). Understanding real-world decision making will require the integration of both visual and cognitive factors in future theories of attention and decision making. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

An empirical evaluation of active inference in multi-armed bandits.

A key feature of sequential decision making under uncertainty is a need to balance between exploiting-choosing the best action according to the current knowledge, and exploring-obtaining information about values of other actions. The multi-armed bandit problem, a classical task that captures this trade-off, served as a vehicle in machine learning for developing bandit algorithms that proved to be useful in numerous industrial applications. The active inference framework, an approach to sequential decision making recently developed in neuroscience for understanding human and animal behaviour, is distinguished by its sophisticated strategy for resolving the exploration-exploitation trade-off. This makes active inference an exciting alternative to already established bandit algorithms. Here we derive an efficient and scalable approximate active inference algorithm and compare it to two state-of-the-art bandit algorithms: Bayesian upper confidence bound and optimistic Thompson sampling. This comparison is done on two types of bandit problems: a stationary and a dynamic switching bandit. Our empirical evaluation shows that the active inference algorithm does not produce efficient long-term behaviour in stationary bandits. However, in the more challenging switching bandit problem active inference performs substantially better than the two state-of-the-art bandit algorithms. The results open exciting venues for further research in theoretical and applied machine learning, as well as lend additional credibility to active inference as a general framework for studying human and animal behaviour.

Analysis of task-based functional MRI data preprocessed with fMRIPrep.

Functional magnetic resonance imaging (fMRI) is a standard tool to investigate the neural correlates of cognition. fMRI noninvasively measures brain activity, allowing identification of patterns evoked by tasks performed during scanning. Despite the long history of this technique, the idiosyncrasies of each dataset have led to the use of ad-hoc preprocessing protocols customized for nearly every different study. This approach is time consuming, error prone and unsuitable for combining datasets from many sources. Here we showcase fMRIPrep (http://fmriprep.org), a robust tool to prepare human fMRI data for statistical analysis. This software instrument addresses the reproducibility concerns of the established protocols for fMRI preprocessing. By leveraging the Brain Imaging Data Structure to standardize both the input datasets (MRI data as stored by the scanner) and the outputs (data ready for modeling and analysis), fMRIPrep is capable of preprocessing a diversity of datasets without manual intervention. In support of the growing popularity of fMRIPrep, this protocol describes how to integrate the tool in a task-based fMRI investigation workflow.

Linear integration and lexicographic models of choice: A cue weight learning perspective.

Choosing between options characterized by multiple cues can be a daunting task. People may integrate all information at hand or just use lexicographic strategies that ignore most of it. Notably, integrative strategies require knowing exact cue weights, whereas lexicographic heuristics can operate by merely knowing the importance order of cues. Here we study how using integrative or lexicographic strategies interacts with learning about cues. In our choice-learning-estimation paradigm people first make choices, learning about cues from the experienced qualities of chosen options, and then estimate qualities of new options. We developed delta-elimination (DE), a new lexicographic strategy that generalizes previous heuristics to any type of environment, and compared it to the integrative weighted-additive (WADD) strategy. Our results show that participants learned cue weights, regardless of whether the DE strategy or the WADD strategy described their choices the best. Still, there was an interaction between the adopted strategy and the cue weight learning process: the DE users learned cue weights slower than the WADD users. This work advances the study of lexicographic choice strategies, both empirically and theoretically, and deepens our understanding of strategy selection, in particular the interaction between the strategy used and learning the structure of the environment. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

It's new, but is it good? How generalization and uncertainty guide the exploration of novel options.

How do people decide whether to try out novel options as opposed to tried-and-tested ones? We argue that they infer a novel option's reward from contextual information learned from functional relations and take uncertainty into account when making a decision. We propose a Bayesian optimization model to describe their learning and decision making. This model relies on similarity-based learning of functional relationships between features and rewards, and a choice rule that balances exploration and exploitation by combining predicted rewards and the uncertainty of these predictions. Our model makes 2 main predictions. First, decision makers who learn functional relationships will generalize based on the learned reward function, choosing novel options only if their predicted reward is high. Second, they will take uncertainty about the function into account, and prefer novel options that can reduce this uncertainty. We test these predictions in 3 preregistered experiments in which we examine participants' preferences for novel options using a feature-based multiarmed bandit task in which rewards are a noisy function of observable features. Our results reveal strong evidence for functional exploration and moderate evidence for uncertainty-guided exploration. However, whether or not participants chose a novel option also depended on their attention, as well as reflecting on the value of the options. These results advance our understanding of people's reactions in the face of novelty. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Uncertainty in learning, choice, and visual fixation.

Uncertainty plays a critical role in reinforcement learning and decision making. However, exactly how it influences behavior remains unclear. Multiarmed-bandit tasks offer an ideal test bed, since computational tools such as approximate Kalman filters can closely characterize the interplay between trial-by-trial values, uncertainty, learning, and choice. To gain additional insight into learning and choice processes, we obtained data from subjects' overt allocation of gaze. The estimated value and estimation uncertainty of options influenced what subjects looked at before choosing; these same quantities also influenced choice, as additionally did fixation itself. A momentary measure of uncertainty in the form of absolute prediction errors determined how long participants looked at the obtained outcomes. These findings affirm the importance of uncertainty in multiple facets of behavior and help delineate its effects on decision making.