Exploration, exploitation, and development: Developmental shifts in decision-making.

Decision-making requires balancing exploration with exploitation, yet children are highly exploratory, with exploration decreasing with development. Less is known about what drives these changes. We examined the development of decision-making in 188 three- to eight-year-old children (M = 64 months; 98 girls) and 26 adults (M = 19 years; 13 women). Children were recruited from ethnically diverse suburban middle-class neighborhoods of Columbus, Ohio, USA. Results indicate that mature reward-based choices emerge relatively late in development, with children tending to over-explore. Computational modeling suggests that this exploration is systematic rather than random, as children tend to avoid repeating choices made on the previous trial. This pattern of exploration (reminiscent of novelty preference) decreased with development, whereas the tendency to exploit increased.

The benefits of immature cognitive control: How distributed attention guards against learning traps.

Cognitive control allows one to focus one's attention efficiently on relevant information while filtering out irrelevant information. This ability provides a means of rapid and effective learning, but using this control also brings risks. Importantly, useful information may be ignored and missed, and learners may fall into "learning traps" (e.g., learned inattention) wherein they fail to realize that what they ignore carries important information. Previous research has shown that adults may be more prone to such traps than young children, but the mechanisms underlying this difference are unclear. The current study used eye tracking to examine the role of attentional control during learning in succumbing to these learning traps. The participants, 4-year-old children and adults, completed a category learning task in which an unannounced switch occurred wherein the feature dimensions most relevant to correct categorization became irrelevant and formerly irrelevant dimensions became relevant. After the switch, adults were more likely than children to ignore the new highly relevant dimension and settle on a suboptimal categorization strategy. Furthermore, eye-tracking analyses reveal that greater attentional selectivity during learning (i.e., optimizing attention to focus only on the most relevant sources of information) predicted this tendency to miss important information later. Children's immature cognitive control, leading to broadly distributed attention, appears to protect children from this trap-although at the cost of less efficient and slower learning. These results demonstrate the double-edged sword of cognitive control and suggest that immature control may serve an adaptive function early in development.

Systematic exploration and uncertainty dominate young children's choices.

Organisms need to constantly balance the competing demands of gathering information and using previously acquired information to obtain rewarding outcomes (i.e., the "exploration-exploitation" dilemma). Exploration is critical to obtain information to discover how the world works, which should be particularly important for young children. While studies have shown that young children explore in response to surprising events, little is known about how they balance exploration and exploitation across multiple decisions or about how this process changes with development. In this study, we compare decision-making patterns of children and adults and evaluate the relative influences of reward seeking, random exploration, and systematic switching (which approximates uncertainty-directed exploration). In a second experiment, we directly test the effect of uncertainty on children's choices. Influential models of decision-making generally describe systematic exploration as a computationally refined capacity that relies on top-down cognitive control. We demonstrate that (a) systematic patterns dominate young children's behavior (facilitating exploration), despite protracted development of cognitive control; and (b) that uncertainty plays a major, but complicated, role in determining children's choices. We conclude that while young children's immature top-down control should hinder adult-like systematic exploration, other mechanisms may pick up the slack, facilitating broad information gathering in a systematic fashion to build a foundation of knowledge for use later in life.

Attentional mechanisms drive systematic exploration in young children.

Exploration is critical for discovering how the world works. Exploration should be particularly valuable for young children, who have little knowledge about the world. Theories of decision-making describe systematic exploration as being primarily driven by top-down cognitive control, which is immature in young children. Recent research suggests that a type of systematic exploration predominates in young children's choices, despite immature control, suggesting that it may be driven by different mechanisms. We hypothesize that young children's tendency to distribute attention widely promotes elevated exploration, and that interrupting distributed attention allocation through bottom-up attentional capture would also disrupt systematic exploration. We test this hypothesis by manipulating saliency of the options in a simple choice task. Saliency disrupted systematic exploration, thus indicating that attentional mechanisms may drive children's systematic exploratory behavior. We suggest that both may be part of a larger tendency toward broad information gathering in young children.

Adaptive flexibility in category learning? Young children exhibit smaller costs of selective attention than adults.

Previous research has shown that when learning categories, adults and young children allocate attention differently. Adults tend to attend selectively, focusing primarily on the most relevant information, whereas young children tend to distribute their attention broadly. Although selective attention is useful in many situations, it also has costs. In addition to ignoring information that may turn out to be useful later, selective attention can have long-term costs, such as learned inattention-ignoring formerly irrelevant sources of information in novel situations. In 2 reported experiments, adults and 4-year-old children completed a category learning task in which an unannounced shift occurred such that information that was most relevant became irrelevant, whereas formerly irrelevant information became relevant. Costs stemming from this shift were assessed. The results indicate that adults exhibit greater costs due to learned inattention than young children. Distributing attention may be adaptive in young children, making them flexible to changing contingencies in the world and facilitating broad information gathering, both of which are useful when general knowledge about the environment is limited. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Transcranial infrared laser stimulation improves rule-based, but not information-integration, category learning in humans.

This is the first randomized, controlled study comparing the cognitive effects of transcranial laser stimulation on category learning tasks. Transcranial infrared laser stimulation is a new non-invasive form of brain stimulation that shows promise for wide-ranging experimental and neuropsychological applications. It involves using infrared laser to enhance cerebral oxygenation and energy metabolism through upregulation of the respiratory enzyme cytochrome oxidase, the primary infrared photon acceptor in cells. Previous research found that transcranial infrared laser stimulation aimed at the prefrontal cortex can improve sustained attention, short-term memory, and executive function. In this study, we directly investigated the influence of transcranial infrared laser stimulation on two neurobiologically dissociable systems of category learning: a prefrontal cortex mediated reflective system that learns categories using explicit rules, and a striatally mediated reflexive learning system that forms gradual stimulus-response associations. Participants (n=118) received either active infrared laser to the lateral prefrontal cortex or sham (placebo) stimulation, and then learned one of two category structures-a rule-based structure optimally learned by the reflective system, or an information-integration structure optimally learned by the reflexive system. We found that prefrontal rule-based learning was substantially improved following transcranial infrared laser stimulation as compared to placebo (treatment X block interaction: F(1, 298)=5.117, p=0.024), while information-integration learning did not show significant group differences (treatment X block interaction: F(1, 288)=1.633, p=0.202). These results highlight the exciting potential of transcranial infrared laser stimulation for cognitive enhancement and provide insight into the neurobiological underpinnings of category learning.

Improving executive function using transcranial infrared laser stimulation.

Transcranial infrared laser stimulation is a new non-invasive form of low-level light therapy that may have a wide range of neuropsychological applications. It entails using low-power and high-energy-density infrared light from lasers to increase metabolic energy. Preclinical work showed that this intervention can increase cortical metabolic energy, thereby improving frontal cortex-based memory function in rats. Barrett and Gonzalez-Lima (2013, Neuroscience, 230, 13) discovered that transcranial laser stimulation can enhance sustained attention and short-term memory in humans. We extend this line of work to executive function. Specifically, we ask whether transcranial laser stimulation enhances performance in the Wisconsin Card Sorting Task that is considered the gold standard of executive function and is compromised in normal ageing and a number of neuropsychological disorders. We used a laser of a specific wavelength (1,064 nm) that photostimulates cytochrome oxidase - the enzyme catalysing oxygen consumption for metabolic energy production. Increased cytochrome oxidase activity is considered the primary mechanism of action of this intervention. Participants who received laser treatment made fewer errors and showed improved set-shifting ability relative to placebo controls. These results suggest that transcranial laser stimulation improves executive function and may have exciting potential for treating or preventing deficits resulting from neuropsychological disorders or normal ageing.

Framing matters: Effects of framing on older adults' exploratory decision-making.

We examined framing effects on exploratory decision-making. In Experiment 1 we tested older and younger adults in two decision-making tasks separated by one week, finding that older adults' decision-making performance was preserved when maximizing gains, but it declined when minimizing losses. Computational modeling indicates that younger adults in both conditions, and older adults in gains maximization, utilized a decreasing threshold strategy (which is optimal), but older adults in losses were better fit by a fixed-probability model of exploration. In Experiment 2 we examined within-subject behavior in older and younger adults in the same exploratory decision-making task, but without a time separation between tasks. We replicated the older adult disadvantage in loss minimization from Experiment 1 and found that the older adult deficit was significantly reduced when the loss-minimization task immediately followed the gains-maximization task. We conclude that older adults' performance in exploratory decision-making is hindered when framed as loss minimization, but that this deficit is attenuated when older adults can first develop a strategy in a gains-framed task. (PsycINFO Database Record

To not settle for small losses: evidence for an ecological aspiration level of zero in dynamic decision-making.

This work aimed to investigate how one's aspiration level is set in decision-making involving losses and how people respond when all alternatives appear to be below the aspiration level. We hypothesized that the zero point would serve as an ecological aspiration level where losses cause participants to focus on improvements in payoffs. In two experiments, we investigated these issues by combining behavioral studies and computational modeling. Participants chose from two alternatives on each trial. A decreasing option consistently gave a larger immediate payoff, although it caused future payoffs for both options to decrease. Selecting an increasing option caused payoffs for both options to increase on future trials. We manipulated the incentive structure such that in the losses condition the smallest payoff for the decreasing option was a loss, whereas in the gains condition the smallest payoff for the decreasing option was a gain, while the differences in outcomes for the two options were kept equivalent across conditions. Participants selected the increasing option more often in the losses condition than in the gains condition, regardless of whether the increasing option was objectively optimal (Experiment 1) or suboptimal (Experiment 2). Further, computational modeling results revealed that participants in the losses condition exhibited heightened weight to the frequency of positive versus negative prediction errors, suggesting that they were more attentive to improvements and reductions in outcomes than to expected values. This supports our assertion that losses induce aspiration for larger payoffs. We discuss our results in the context of recent theories of how losses shape behavior.

Exploratory decision-making as a function of lifelong experience, not cognitive decline.

Older adults perform worse than younger adults in some complex decision-making scenarios, which is commonly attributed to age-related declines in striatal and frontostriatal processing. Recently, this popular account has been challenged by work that considered how older adults' performance may differ as a function of greater knowledge and experience, and by work showing that, in some cases, older adults outperform younger adults in complex decision-making tasks. In light of this controversy, we examined the performance of older and younger adults in an exploratory choice task that is amenable to model-based analyses and ostensibly not reliant on prior knowledge. Exploration is a critical aspect of decision-making poorly understood across the life span. Across 2 experiments, we addressed (a) how older and younger adults differ in exploratory choice and (b) to what extent observed differences reflect processing capacity declines. Model-based analyses suggested that the strategies used by the 2 groups were qualitatively different, resulting in relatively worse performance for older adults in 1 decision-making environment but equal performance in another. Little evidence was found that differences in processing capacity drove performance differences. Rather the results suggested that older adults' performance might result from applying a strategy that may have been shaped by their wealth of real-word decision-making experience. While this strategy is likely to be effective in the real world, it is ill suited to some decision environments. These results underscore the importance of taking into account effects of experience in aging studies, even for tasks that do not obviously tap past experiences.

Enhanced procedural learning of speech sound categories in a genetic variant of FOXP2.

A mutation of the forkhead box protein P2 (FOXP2) gene is associated with severe deficits in human speech and language acquisition. In rodents, the humanized form of FOXP2 promotes faster switching from declarative to procedural learning strategies when the two learning systems compete. Here, we examined a polymorphism of FOXP2 (rs6980093) in humans (214 adults; 111 females) for associations with non-native speech category learning success. Neurocomputational modeling results showed that individuals with the GG genotype shifted faster to procedural learning strategies, which are optimal for the task. These findings support an adaptive role for the FOXP2 gene in modulating the function of neural learning systems that have a direct bearing on human speech category learning.

A frontal dopamine system for reflective exploratory behavior.

The COMT gene modulates dopamine levels in prefrontal cortex with Met allele carriers having lower COMT enzyme activity and, therefore, higher dopamine levels compared to Val/Val homozygotes. Concordantly, Val/Val homozygotes tend to perform worse and display increased (interpreted as inefficient) frontal activation in certain cognitive tasks. In a sample of 209 participants, we test the hypothesis that Met carriers will be advantaged in a decision-making task that demands sequencing exploratory and exploitive choices to minimize uncertainty about the reward structure in the environment. Previous work suggests that optimal performance depends on limited cognitive resources supported by prefrontal systems. If so, Met carriers should outperform Val/Val homozygotes, particularly under dual-task conditions that tax limited cognitive resources. In accord with these a priori predictions, Met carriers were more resilient in the face of cognitive load, continuing to explore in a sophisticated manner. We fit computational models that embody sophisticated reflective and simple reflexive strategies to further evaluate participants' exploration behavior. The Ideal Actor model reflectively updates beliefs and plans ahead, taking into account the information gained by each choice and making choices that maximize long-term payoffs. In contrast, the Naïve Reinforcement Learning (RL) model instantiates the reflexive account of choice, in which the values of actions are based only on the rewards experienced so far. Its beliefs are updated reflexively in response to observed changes in rewards. Converging with standard analyses, Met carriers were best characterized by the Ideal Actor model, whereas Val/Val homozygotes were best characterized by the Naive RL model, particularly under dual-task conditions.

The influence of depression symptoms on exploratory decision-making.

People with symptoms of depression show impairments in decision-making. One explanation is that they have difficulty maintaining rich representations of the task environment. We test this hypothesis in the context of exploratory choice. We analyze depressive and non-depressive participants' exploration strategies by comparing their choices to two computational models: (1) an "Ideal Actor" model that reflectively updates beliefs and plans ahead, employing a rich representation of the environment and (2) a "Naïve Reinforcement Learning" (RL) model that updates beliefs reflexively utilizing a minimal task representation. Relative to non-depressive participants, we find that depressive participants' choices are better described by the simple RL model. Further, depressive participants were more exploratory than non-depressives in their decision-making. Depressive symptoms appear to influence basic mechanisms supporting choice behavior by reducing use of rich task representations and hindering performance during exploratory decision-making.