Explaining the flaws in human random generation as local sampling with momentum.

In many tasks, human behavior is far noisier than is optimal. Yet when asked to behave randomly, people are typically too predictable. We argue that these apparently contrasting observations have the same origin: the operation of a general-purpose local sampling algorithm for probabilistic inference. This account makes distinctive predictions regarding random sequence generation, not predicted by previous accounts-which suggests that randomness is produced by inhibition of habitual behavior, striving for unpredictability. We verify these predictions in two experiments: people show the same deviations from randomness when randomly generating from non-uniform or recently-learned distributions. In addition, our data show a novel signature behavior, that people's sequences have too few changes of trajectory, which argues against the specific local sampling algorithms that have been proposed in past work with other tasks. Using computational modeling, we show that local sampling where direction is maintained across trials best explains our data, which suggests it may be used in other tasks too. While local sampling has previously explained why people are unpredictable in standard cognitive tasks, here it also explains why human random sequences are not unpredictable enough.

Understanding the structure of cognitive noise.

Human cognition is fundamentally noisy. While routinely regarded as a nuisance in experimental investigation, the few studies investigating properties of cognitive noise have found surprising structure. A first line of research has shown that inter-response-time distributions are heavy-tailed. That is, response times between subsequent trials usually change only a small amount, but with occasional large changes. A second, separate, line of research has found that participants' estimates and response times both exhibit long-range autocorrelations (i.e., 1/f noise). Thus, each judgment and response time not only depends on its immediate predecessor but also on many previous responses. These two lines of research use different tasks and have distinct theoretical explanations: models that account for heavy-tailed response times do not predict 1/f autocorrelations and vice versa. Here, we find that 1/f noise and heavy-tailed response distributions co-occur in both types of tasks. We also show that a statistical sampling algorithm, developed to deal with patchy environments, generates both heavy-tailed distributions and 1/f noise, suggesting that cognitive noise may be a functional adaptation to dealing with a complex world.

How could we make a social robot? A virtual bargaining approach.

What is required to allow an artificial agent to engage in rich, human-like interactions with people? I argue that this will require capturing the process by which humans continually create and renegotiate 'bargains' with each other. These hidden negotiations will concern topics including who should do what in a particular interaction, which actions are allowed and which are forbidden, and the momentary conventions governing communication, including language. Such bargains are far too numerous, and social interactions too rapid, for negotiation to be conducted explicitly. Moreover, the very process of communication presupposes innumerable momentary agreements concerning the meaning of communicative signals, thus raising the threat of circularity. Thus, the improvised 'social contracts' that govern our interactions must be implicit. I draw on the recent theory of virtual bargaining, according to which social partners mentally simulate a process of negotiation, to outline how these implicit agreements can be made, and note that this viewpoint raises substantial theoretical and computational challenges. Nonetheless, I suggest that these challenges must be met if we are ever to create AI systems that can work collaboratively alongside people, rather than serving primarily as valuable special-purpose computational tools. This article is part of a discussion meeting issue 'Cognitive artificial intelligence'.

The ephemeral stories of our lives.

Johnson et al. make a persuasive case that qualitative, story-like reasoning plays a crucial role in everyday thought and decision-making. This commentary questions the cohesiveness of this type of reasoning and the representations that generate it. Perhaps narratives do not underpin, but are ephemeral products of thought, created when we need to justify our actions to ourselves and others.

Where next for behavioral public policy?

Our target article distinguishes between policy approaches that seek to address societal problems through intervention at the level of the individual (adopting the "i-frame") and those that seek to change the system within which those individuals live (adopting the "s-frame"). We stress also that a long-standing tactic of corporations opposing systemic change is to promote the i-frame perspective, presumably hoping that i-frame interventions will be largely ineffective and more importantly will be seen by the public and some policy makers as a genuine alternative to systemic change. We worry that the i-frame focus of much of behavioral science has inadvertently reinforced this unhelpful focus on the individual. In this response to commentators, we identify common themes, build on the many constructive suggestions to extend our approach, and reply to concerns. We argue, along with several commentators, that a key role of behavioral public policy is to clarify how to build support for systemic reforms for which there is a broad consensus in the policy community, but which are opposed by powerful special interests.

From the pragmatics of charades to the creation of language.

We agree with Heintz & Scott-Phillips that pragmatics does not supplement, but is prior to and underpins, language. Indeed, human non-linguistic communication is astonishingly rich, flexible, and subtle, as we illustrate through the game of charades, where people improvise communicative signals when linguistic channels are blocked. The route from non-linguistic charade-like communication to combinatorial language involves (1) local processes of conventionalization and grammaticalization and (2) spontaneous order arising from mutual constraints between different communicative signals.

Perceptual and Cognitive Judgments Show Both Anchoring and Repulsion.

One of the most robust effects in cognitive psychology is anchoring, in which judgments show a bias toward previously viewed values. However, in what is essentially the same task as used in anchoring research, a perceptual illusion demonstrates the opposite effect of repulsion. Here, we united these two literatures, testing in two experiments with adults (total N = 200) whether prior comparative decisions bias cognitive and perceptual judgments in opposing directions or whether anchoring and repulsion are two domain-general biases whose co-occurrence has so far gone undetected. We found that in both perceptual and cognitive tasks, anchoring and repulsion co-occur. Further, the direction of the bias depends on the comparison value: Distant values attract judgments, whereas nearby values repulse judgments. Because none of the leading theories for either effect account for both biases, theoretical integration is needed. As a starting point, we describe one such joint theory based on sampling models of cognition.

Rationality as the end of thought.

Bermúdez convincingly argues that framing effects are ubiquitous and that this is not a sign of human irrationality, but an unavoidable feature of any intelligent system. The commentary adds that framing effects arise even in formal domains, such as chess and mathematics, which appear paradigms of rational thought. Indeed, finding and attempting to resolve clashes between different frames is a major impetus for deliberative cognition.

The i-frame and the s-frame: How focusing on individual-level solutions has led behavioral public policy astray.

An influential line of thinking in behavioral science, to which the two authors have long subscribed, is that many of society's most pressing problems can be addressed cheaply and effectively at the level of the individual, without modifying the system in which the individual operates. We now believe this was a mistake, along with, we suspect, many colleagues in both the academic and policy communities. Results from such interventions have been disappointingly modest. But more importantly, they have guided many (though by no means all) behavioral scientists to frame policy problems in individual, not systemic, terms: To adopt what we call the "i-frame," rather than the "s-frame." The difference may be more consequential than i-frame advocates have realized, by deflecting attention and support away from s-frame policies. Indeed, highlighting the i-frame is a long-established objective of corporate opponents of concerted systemic action such as regulation and taxation. We illustrate our argument briefly for six policy problems, and in depth with the examples of climate change, obesity, retirement savings, and pollution from plastic waste. We argue that the most important way in which behavioral scientists can contribute to public policy is by employing their skills to develop and implement value-creating system-level change.

New Paradigms in the Psychology of Reasoning.

The psychology of verbal reasoning initially compared performance with classical logic. In the last 25 years, a new paradigm has arisen, which focuses on knowledge-rich reasoning for communication and persuasion and is typically modeled using Bayesian probability theory rather than logic. This paradigm provides a new perspective on argumentation, explaining the rational persuasiveness of arguments that are logical fallacies. It also helps explain how and why people stray from logic when given deductive reasoning tasks. What appear to be erroneous responses, when compared against logic, often turn out to be rationally justified when seen in the richer rational framework of the new paradigm. Moreover, the same approach extends naturally to inductive reasoning tasks, in which people extrapolate beyond the data they are given and logic does not readily apply. We outline links between social and individual reasoning and set recent developments in the psychology of reasoning in the wider context of Bayesian cognitive science.

Sampling as a resource-rational constraint.

Resource rationality is useful for choosing between models with the same cognitive constraints but cannot settle fundamental disagreements about what those constraints are. We argue that sampling is an especially compelling constraint, as optimizing accumulation of evidence or hypotheses minimizes the cost of time, and there are well-established models for doing so which have had tremendous success explaining human behavior.

Economic irrationality is optimal during noisy decision making.

According to normative theories, reward-maximizing agents should have consistent preferences. Thus, when faced with alternatives A, B, and C, an individual preferring A to B and B to C should prefer A to C. However, it has been widely argued that humans can incur losses by violating this axiom of transitivity, despite strong evolutionary pressure for reward-maximizing choices. Here, adopting a biologically plausible computational framework, we show that intransitive (and thus economically irrational) choices paradoxically improve accuracy (and subsequent economic rewards) when decision formation is corrupted by internal neural noise. Over three experiments, we show that humans accumulate evidence over time using a "selective integration" policy that discards information about alternatives with momentarily lower value. This policy predicts violations of the axiom of transitivity when three equally valued alternatives differ circularly in their number of winning samples. We confirm this prediction in a fourth experiment reporting significant violations of weak stochastic transitivity in human observers. Crucially, we show that relying on selective integration protects choices against "late" noise that otherwise corrupts decision formation beyond the sensory stage. Indeed, we report that individuals with higher late noise relied more strongly on selective integration. These findings suggest that violations of rational choice theory reflect adaptive computations that have evolved in response to irreducible noise during neural information processing.

The social character of moral reasoning.

May provides a compelling case that reasoning is central to moral psychology. In practice, many morally significant decisions involve several moral agents whose actions are interdependent - and agents embedded in society. We suggest that social life and the rich patterns of reasoning that underpin it are ethical through and through.

Simpler grammar, larger vocabulary: How population size affects language.

Languages with many speakers tend to be structurally simple while small communities sometimes develop languages with great structural complexity. Paradoxically, the opposite pattern appears to be observed for non-structural properties of language such as vocabulary size. These apparently opposite patterns pose a challenge for theories of language change and evolution. We use computational simulations to show that this inverse pattern can depend on a single factor: ease of diffusion through the population. A population of interacting agents was arranged on a network, passing linguistic conventions to one another along network links. Agents can invent new conventions, or replicate conventions that they have previously generated themselves or learned from other agents. Linguistic conventions are either Easy or Hard to diffuse, depending on how many times an agent needs to encounter a convention to learn it. In large groups, only linguistic conventions that are easy to learn, such as words, tend to proliferate, whereas small groups where everyone talks to everyone else allow for more complex conventions, like grammatical regularities, to be maintained. Our simulations thus suggest that language, and possibly other aspects of culture, may become simpler at the structural level as our world becomes increasingly interconnected.

Theories or fragments?

Lake et al. argue persuasively that modelling human-like intelligence requires flexible, compositional representations in order to embody world knowledge. But human knowledge is too sparse and self-contradictory to be embedded in "intuitive theories." We argue, instead, that knowledge is grounded in exemplar-based learning and generalization, combined with high flexible generalization, a viewpoint compatible both with non-parametric Bayesian modelling and with sub-symbolic methods such as neural networks.

Numerical magnitude evaluation as a foundation for decision making.

The evaluation of magnitudes serves as a foundation not only for numerical and mathematical cognition, but also for decision making. Recent theoretical developments and empirical studies have linked numerical magnitude evaluation to a wide variety of core phenomena in decision making and challenge the idea that preferences are driven by an innate, universal, and stable sense of number or value.

Instantaneous Conventions.

Humans can communicate even with few existing conventions in common (e.g., when they lack a shared language). We explored what makes this phenomenon possible with a nonlinguistic experimental task requiring participants to coordinate toward a common goal. We observed participants creating new communicative conventions using the most minimal possible signals. These conventions, furthermore, changed on a trial-by-trial basis in response to shared environmental and task constraints. Strikingly, as a result, signals of the same form successfully conveyed contradictory messages from trial to trial. Such behavior is evidence for the involvement of what we term joint inference, in which social interactants spontaneously infer the most sensible communicative convention in light of the common ground between them. Joint inference may help to elucidate how communicative conventions emerge instantaneously and how they are modified and reshaped into the elaborate systems of conventions involved in human communication, including natural languages.

Squeezing through the Now-or-Never bottleneck: Reconnecting language processing, acquisition, change, and structure.

If human language must be squeezed through a narrow cognitive bottleneck, what are the implications for language processing, acquisition, change, and structure? In our target article, we suggested that the implications are far-reaching and form the basis of an integrated account of many apparently unconnected aspects of language and language processing, as well as suggesting revision of many existing theoretical accounts. With some exceptions, commentators were generally supportive both of the existence of the bottleneck and its potential implications. Many commentators suggested additional theoretical and linguistic nuances and extensions, links with prior work, and relevant computational and neuroscientific considerations; some argued for related but distinct viewpoints; a few, though, felt traditional perspectives were being abandoned too readily. Our response attempts to build on the many suggestions raised by the commentators and to engage constructively with challenges to our approach.

The Now-or-Never bottleneck: A fundamental constraint on language.

Memory is fleeting. New material rapidly obliterates previous material. How, then, can the brain deal successfully with the continual deluge of linguistic input? We argue that, to deal with this "Now-or-Never" bottleneck, the brain must compress and recode linguistic input as rapidly as possible. This observation has strong implications for the nature of language processing: (1) the language system must "eagerly" recode and compress linguistic input; (2) as the bottleneck recurs at each new representational level, the language system must build a multilevel linguistic representation; and (3) the language system must deploy all available information predictively to ensure that local linguistic ambiguities are dealt with "Right-First-Time"; once the original input is lost, there is no way for the language system to recover. This is "Chunk-and-Pass" processing. Similarly, language learning must also occur in the here and now, which implies that language acquisition is learning to process, rather than inducing, a grammar. Moreover, this perspective provides a cognitive foundation for grammaticalization and other aspects of language change. Chunk-and-Pass processing also helps explain a variety of core properties of language, including its multilevel representational structure and duality of patterning. This approach promises to create a direct relationship between psycholinguistics and linguistic theory. More generally, we outline a framework within which to integrate often disconnected inquiries into language processing, language acquisition, and language change and evolution.