Neural representations of statistical and rule-based predictions in Gilles de la Tourette syndrome.

Gilles de la Tourette syndrome (GTS) is a disorder characterised by motor and vocal tics, which may represent habitual actions as a result of enhanced learning of associations between stimuli and responses (S-R). In this study, we investigated how adults with GTS and healthy controls (HC) learn two types of regularities in a sequence: statistics (non-adjacent probabilities) and rules (predefined order). Participants completed a visuomotor sequence learning task while EEG was recorded. To understand the neurophysiological underpinnings of these regularities in GTS, multivariate pattern analyses on the temporally decomposed EEG signal as well as sLORETA source localisation method were conducted. We found that people with GTS showed superior statistical learning but comparable rule-based learning compared to HC participants. Adults with GTS had different neural representations for both statistics and rules than HC adults; specifically, adults with GTS maintained the regularity representations longer and had more overlap between them than HCs. Moreover, over different time scales, distinct fronto-parietal structures contribute to statistical learning in the GTS and HC groups. We propose that hyper-learning in GTS is a consequence of the altered sensitivity to encode complex statistics, which might lead to habitual actions.

Language statistical learning responds to reinforcement learning principles rooted in the striatum.

Statistical learning (SL) is the ability to extract regularities from the environment. In the domain of language, this ability is fundamental in the learning of words and structural rules. In lack of reliable online measures, statistical word and rule learning have been primarily investigated using offline (post-familiarization) tests, which gives limited insights into the dynamics of SL and its neural basis. Here, we capitalize on a novel task that tracks the online SL of simple syntactic structures combined with computational modeling to show that online SL responds to reinforcement learning principles rooted in striatal function. Specifically, we demonstrate-on 2 different cohorts-that a temporal difference model, which relies on prediction errors, accounts for participants' online learning behavior. We then show that the trial-by-trial development of predictions through learning strongly correlates with activity in both ventral and dorsal striatum. Our results thus provide a detailed mechanistic account of language-related SL and an explanation for the oft-cited implication of the striatum in SL tasks. This work, therefore, bridges the long-standing gap between language learning and reinforcement learning phenomena.

Impaired implicit learning in adults with anorexia nervosa.

OBJECTIVE: Cognitive alterations play an important role in the pathophysiology and treatment of anorexia nervosa (AN). Previous studies suggest that some implicit learning processes may be inhibited in AN. However, this has not yet been fully explored. The purpose of this study is to analyze implicit learning in patients with AN in comparison to healthy controls. METHODS: In this pilot-study, a total of 21 patients diagnosed with AN and 21 matched controls were administered the weather prediction task (WPT), a probabilistic implicit category learning task that consists of two sub-variants. During the feedback (FB) version of the task, participants learn associations between tarot cards and weather outcomes via an operant learning model through which they receive immediate FB on their answers, whereas during the paired associate (PA) variant, participants are directly asked to memorize given associations. RESULTS: AN patients showed selective impairment on the FB task where they scored significantly lower both in comparison to controls (p = .001) who completed the same task and when compared to their own performance on the PA variant (p = .006). Clinical measures showed no significant correlations with test scores. DISCUSSION: Our results demonstrate implicit FB learning deficiencies in adult patients with AN. These impairments may have an impact on the effect of psychotherapeutic interventions and could partially explain the lack of treatment response in AN. Further studies are necessary to derive when and through which mechanisms these alterations originate, and to what extent they should be considered during treatment of the disorder. PUBLIC SIGNIFICANCE: Cognitive impairments pose a challenge in the management of anorexia nervosa. Improved comprehension of cognitive alterations could lead to a greater understanding of the disease and adaptation of psychotherapeutic treatments. In this study, we found that implicit feedback learning in anorexia nervosa is impaired compared to healthy controls. This could indicate the necessity of treatment adaptations in the form of therapy tools without feedback and a larger focus on psychoeducation.

A characterization of the neural representation of confidence during probabilistic learning.

Learning in a stochastic and changing environment is a difficult task. Models of learning typically postulate that observations that deviate from the learned predictions are surprising and used to update those predictions. Bayesian accounts further posit the existence of a confidence-weighting mechanism: learning should be modulated by the confidence level that accompanies those predictions. However, the neural bases of this confidence are much less known than the ones of surprise. Here, we used a dynamic probability learning task and high-field MRI to identify putative cortical regions involved in the representation of confidence about predictions during human learning. We devised a stringent test based on the conjunction of four criteria. We localized several regions in parietal and frontal cortices whose activity is sensitive to the confidence of an ideal observer, specifically so with respect to potential confounds (surprise and predictability), and in a way that is invariant to which item is predicted. We also tested for functionality in two ways. First, we localized regions whose activity patterns at the subject level showed an effect of both confidence and surprise in qualitative agreement with the confidence-weighting principle. Second, we found neural representations of ideal confidence that also accounted for subjective confidence. Taken together, those results identify a set of cortical regions potentially implicated in the confidence-weighting of learning.

Interactions between human orbitofrontal cortex and hippocampus support model-based inference.

Internal representations of relationships between events in the external world can be utilized to infer outcomes when direct experience is lacking. This process is thought to involve the orbitofrontal cortex (OFC) and hippocampus (HPC), but there is little evidence regarding the relative role of these areas and their interactions in inference. Here, we used a sensory preconditioning task and pattern-based neuroimaging to study this question. We found that associations among value-neutral cues were acquired in both regions during preconditioning but that value-related information was only represented in the OFC at the time of the probe test. Importantly, inference was accompanied by representations of associated cues and inferred outcomes in the OFC, as well as by increased HPC-OFC connectivity. These findings suggest that the OFC and HPC represent only partially overlapping information and that interactions between the two regions support model-based inference.

State anxiety alters the neural oscillatory correlates of predictions and prediction errors during reward-based learning.

Anxiety influences how the brain estimates and responds to uncertainty. The consequences of these processes on behaviour have been described in theoretical and empirical studies, yet the associated neural correlates remain unclear. Rhythm-based accounts of Bayesian predictive coding propose that predictions in generative models of perception are represented in alpha (8-12 Hz) and beta oscillations (13-30 Hz). Updates to predictions are driven by prediction errors weighted by precision (inverse variance) encoded in gamma oscillations (>30 Hz) and associated with the suppression of beta activity. We tested whether state anxiety alters the neural oscillatory activity associated with predictions and precision-weighted prediction errors (pwPE) during learning. Healthy human participants performed a probabilistic reward-based learning task in a volatile environment. In our previous work, we described learning behaviour in this task using a hierarchical Bayesian model, revealing more precise (biased) beliefs about the tendency of the reward contingency in state anxiety, consistent with reduced learning in this group. The model provided trajectories of predictions and pwPEs for the current study, allowing us to assess their parametric effects on the time-frequency representations of EEG data. Using convolution modelling for oscillatory responses, we found that, relative to a control group, state anxiety increased beta activity in frontal and sensorimotor regions during processing of pwPE, and in fronto-parietal regions during encoding of predictions. No effects of state anxiety on gamma modulation were found. Our findings expand prior evidence on the oscillatory representations of predictions and pwPEs into the reward-based learning domain. The results suggest that state anxiety modulates beta-band oscillatory correlates of pwPE and predictions in generative models, providing insights into the neural processes associated with biased belief updating and poorer learning.

The Event-Related Potentials Responding to Outcome Valence and Expectancy Violation during Feedback Processing.

Feedback-related negativity (FRN) is believed to encode reward prediction error (RPE), a term describing whether the outcome is better or worse than expected. However, some studies suggest that it may reflect unsigned prediction error (UPE) instead. Some disagreement remains as to whether FRN is sensitive to the interaction of outcome valence and prediction error (PE) or merely responsive to the absolute size of PE. Moreover, few studies have compared FRN in appetitive and aversive domains to clarify the valence effect or examine PE's quantitative modulation. To investigate the impact of valence and parametrical PE on FRN, we varied the prediction and feedback magnitudes within a probabilistic learning task in valence (gain and loss domains, Experiment 1) and non-valence contexts (pure digits, Experiment 2). Experiment 3 was identical to Experiment 1 except that some blocks emphasized outcome valence, while others highlighted predictive accuracy. Experiments 1 and 2 revealed a UPE encoder; Experiment 3 found an RPE encoder when valence was emphasized and a UPE encoder when predictive accuracy was highlighted. In this investigation, we demonstrate that FRN is sensitive to outcome valence and expectancy violation, exhibiting a preferential response depending on the dimension that is emphasized.

Visual object recognition is facilitated by temporal community structure.

Humans and others primates are highly attuned to temporal consistencies and regularities in their sensory environment and learn to predict such statistical structure. Moreover, in several instances, the presence of temporal structure has been found to facilitate procedural learning and to improve task performance. Here we extend these findings to visual object recognition and to presentation sequences in which mutually predictive objects form distinct clusters or "communities." Our results show that temporal community structure accelerates recognition learning and affects the order in which objects are learned ("onset of familiarity").

State anxiety biases estimates of uncertainty and impairs reward learning in volatile environments.

Clinical and subclinical (trait) anxiety impairs decision making and interferes with learning. Less understood are the effects of temporary anxious states on learning and decision making in healthy populations, and whether these can serve as a model for clinical anxiety. Here we test whether anxious states in healthy individuals elicit a pattern of aberrant behavioural, neural, and physiological responses comparable with those found in anxiety disorders-particularly when processing uncertainty in unstable environments. In our study, both a state anxious and a control group learned probabilistic stimulus-outcome mappings in a volatile task environment while we recorded their electrophysiological (EEG) signals. By using a hierarchical Bayesian model of inference and learning, we assessed the effect of state anxiety on Bayesian belief updating with a focus on uncertainty estimates. State anxiety was associated with an underestimation of environmental uncertainty, and informational uncertainty about the reward tendency. Anxious individuals' beliefs about reward contingencies were more precise (had smaller uncertainty) and thus more resistant to updating, ultimately leading to impaired reward-based learning. State anxiety was also associated with greater uncertainty about volatility. We interpret this pattern as evidence that state anxious individuals are less tolerant to informational uncertainty about the contingencies governing their environment and more willing to be uncertain about the level of stability of the world itself. Further, we tracked the neural representation of belief update signals in the trial-by-trial EEG amplitudes. In control participants, lower-level precision-weighted prediction errors (pwPEs) about reward tendencies were represented in the ERP signals across central and parietal electrodes peaking at 496 ms, overlapping with the late P300 in classical ERP analysis. The state anxiety group did not exhibit a significant representation of low-level pwPEs, and there were no significant differences between the groups. Smaller variance in low-level pwPE about reward tendencies in state anxiety could partially account for the null results. Expanding previous computational work on trait anxiety, our findings establish that temporary anxious states in healthy individuals impair reward-based learning in volatile environments, primarily through changes in uncertainty estimates, which play a central role in current Bayesian accounts of perceptual inference and learning.

Neurophysiological and functional neuroanatomical coding of statistical and deterministic rule information during sequence learning.

Humans are capable of acquiring multiple types of information presented in the same information stream. It has been suggested that at least two parallel learning processes are important during learning of sequential patterns-statistical learning and rule-based learning. Yet, the neurophysiological underpinnings of these parallel learning processes are not fully understood. To differentiate between the simultaneous mechanisms at the single trial level, we apply a temporal EEG signal decomposition approach together with sLORETA source localization method to delineate whether distinct statistical and rule-based learning codes can be distinguished in EEG data and can be related to distinct functional neuroanatomical structures. We demonstrate that concomitant but distinct aspects of information coded in the N2 time window play a role in these mechanisms: mismatch detection and response control underlie statistical learning and rule-based learning, respectively, albeit with different levels of time-sensitivity. Moreover, the effects of the two learning mechanisms in the different temporally decomposed clusters of neural activity also differed from each other in neural sources. Importantly, the right inferior frontal cortex (BA44) was specifically implicated in visuomotor statistical learning, confirming its role in the acquisition of transitional probabilities. In contrast, visuomotor rule-based learning was associated with the prefrontal gyrus (BA6). The results show how simultaneous learning mechanisms operate at the neurophysiological level and are orchestrated by distinct prefrontal cortical areas. The current findings deepen our understanding on the mechanisms of how humans are capable of learning multiple types of information from the same stimulus stream in a parallel fashion.

Translational Assessments of Reward Responsiveness in the Marmoset.

BACKGROUND: Anhedonia, the loss of pleasure in previously rewarding activities, is a prominent feature of major depressive disorder and often resistant to first-line antidepressant treatment. A paucity of translatable cross-species tasks to assess subdomains of anhedonia, including reward learning, presents a major obstacle to the development of effective therapeutics. One assay of reward learning characterized by orderly behavioral and pharmacological findings in both humans and rats is the probabilistic reward task. In this computerized task, subjects make discriminations across numerous trials in which correct responses to one alternative are rewarded more often (rich) than correct responses to the other (lean). Healthy control subjects reliably develop a response bias to the rich alternative. However, participants with major depressive disorder as well as rats exposed to chronic stress typically exhibit a blunted response bias. METHODS: The present studies validated a touchscreen-based probabilistic reward task for the marmoset, a small nonhuman primate with considerable translational value. First, probabilistic reinforcement contingencies were parametrically examined. Next, the effects of ketamine (1.0-10.0 mg/kg), a US Food and Drug Administration-approved rapid-acting antidepressant, and phencyclidine (0.01-0.1 mg/kg), a pharmacologically similar N-methyl-D-aspartate receptor antagonist with no known antidepressant efficacy, were evaluated. RESULTS: Increases in the asymmetry of rich:lean probabilistic contingencies produced orderly increases in response bias. Consistent with their respective clinical profiles, ketamine but not phencyclidine produced dose-related increases in response bias at doses that did not reduce task discriminability. CONCLUSIONS: Collectively, these findings confirm task and pharmacological sensitivity in the marmoset, which may be useful in developing medications to counter anhedonia across neuropsychiatric disorders.

From learning to creativity: Identifying the behavioural and neural correlates of learning to predict human judgements of musical creativity.

Human creativity is intricately linked to acquired knowledge. However, to date learning a new musical style and subsequent musical creativity have largely been studied in isolation. We introduced a novel experimental paradigm combining behavioural, electrophysiological, and computational methods, to examine the neural correlates of unfamiliar music learning, and to investigate how neural and computational measures can predict human creativity. We investigated music learning by training non-musicians (N = 40) on an artificial music grammar. Participants' knowledge of the grammar was tested before and after three training sessions on separate days by assessing explicit recognition of the notes of the grammar, while additionally recording their EEG. After each training session, participants created their own musical compositions, which were later evaluated by human experts. A computational model of auditory expectation was used to quantify the statistical properties of both the grammar and the compositions. Results showed that participants successfully learned the new grammar. This was also reflected in the N100, P200, and P3a components, which were higher in response to incorrect than correct notes. The delta band (2.5-4.5 Hz) power in response to grammatical notes during first exposure to the grammar positively correlated with learning, suggesting a potential neural mechanism of encoding. On the other hand, better learning was associated with lower alpha and higher beta band power after training, potentially reflecting neural mechanisms of retrieval. Importantly, learning was a significant predictor of creativity, as judged by experts. There was also an inverted U-shaped relationship between percentage of correct intervals and creativity, as compositions with an intermediate proportion of correct intervals were associated with the highest creativity. Finally, the P200 in response to incorrect notes was predictive of creativity, suggesting a link between the neural correlates of learning, and creativity. Overall, our findings shed light on the neural mechanisms of learning an unfamiliar music grammar, and offer novel contributions to the associations between learning measures and creative compositions based on learned materials.

Functional brain network architecture supporting the learning of social networks in humans.

Most humans have the good fortune to live their lives embedded in richly structured social groups. Yet, it remains unclear how humans acquire knowledge about these social structures to successfully navigate social relationships. Here we address this knowledge gap with an interdisciplinary neuroimaging study drawing on recent advances in network science and statistical learning. Specifically, we collected BOLD MRI data while participants learned the community structure of both social and non-social networks, in order to examine whether the learning of these two types of networks was differentially associated with functional brain network topology. We found that participants learned the community structure of the networks, as evidenced by a slower reaction time when a trial moved between communities than when a trial moved within a community. Learning the community structure of social networks was also characterized by significantly greater functional connectivity of the hippocampus and temporoparietal junction when transitioning between communities than when transitioning within a community. Furthermore, temporoparietal regions of the default mode were more strongly connected to hippocampus, somatomotor, and visual regions for social networks than for non-social networks. Collectively, our results identify neurophysiological underpinnings of social versus non-social network learning, extending our knowledge about the impact of social context on learning processes. More broadly, this work offers an empirical approach to study the learning of social network structures, which could be fruitfully extended to other participant populations, various graph architectures, and a diversity of social contexts in future studies.

EEG correlates of physical effort and reward processing during reinforcement learning.

Effort-based decision making is often modeled using subjective value, a function of reward discounted by effort. We asked whether EEG event-related potential (ERP) correlates of reward processing are also modulated by physical effort. Human participants performed a task in which they were required to accurately produce target levels of muscle activation to receive rewards. Quadriceps muscle activation was recorded with electromyography (EMG) during isometric knee extension. On a given trial, the target muscle activation required either low or high effort. The effort was determined probabilistically according to a binary choice, such that the responses were associated with 20% and 80% probability of high effort. This contingency could only be learned through experience, and it reversed periodically. Binary reinforcement feedback depended on accurately producing the target muscle activity. Participants adaptively avoided effort by switching responses more frequently after choices that resulted in hard effort. Feedback after participants' choices that revealed the resulting effort requirement did not elicit modulation of the feedback-related negativity/reward positivity (FRN/RP). However, the neural response to reinforcement outcome after effort production was increased by preceding physical effort. Source decomposition revealed separable early and late positive deflections contributing to the ERP. The main effect of reward outcome, characteristic of the FRN/RP, loaded onto the earlier component, whereas the reward × effort interaction was observed only in the later positivity, which resembled the P300. Thus, retrospective effort modulates reward processing. This may explain paradoxical behavioral findings whereby rewards requiring more effort to obtain can become more powerful reinforcers.NEW & NOTEWORTHY Choices probabilistically determined the physical effort requirements for a subsequent task, and reward depended on task performance. Feedback revealing whether choices resulted in easy or hard effort did not elicit reinforcement learning signals. However, the neural responses to reinforcement were modulated by preceding effort. Thus, effort itself was not treated as loss or punishment, but it affected the responses to subsequent reinforcement outcomes. This may explain how effort can enhance the motivational effect of reward.

Decision-related activity in sensory neurons: correlations among neurons and with behavior.

Neurons in early sensory cortex show weak but systematic correlations with perceptual decisions when trained animals perform at psychophysical threshold. These correlations are observed across repeated presentations of identical stimuli and cannot be explained by variation in external factors. The relationship between the activity of individual sensory neurons and the animal's behavioral choice means that even neurons in early sensory cortex carry information about an upcoming decision. This relationship, termed choice probability, may reflect the effect of fluctuations in neuronal firing rate on the animal's decision, but it can also reflect modulation of sensory responses by cognitive factors, or network properties such as variability that is shared among populations of neurons. Here, we review recent work clarifying the relationship among fluctuations in the responses of individual neurons, correlated variability, and behavior in a variety of tasks and cortical areas. We also discuss the possibility that choice probability may in part reflect the influence of cognitive factors on sensory neurons and explore the situations in which choice probability can be used to make inferences about the role of particular sensory neurons in the decision-making process.

Statistical Learning of Orthographic Regularities in Chinese Children With and Without Dyslexia.

This study examined distributional statistical learning of positional, phonetic, and semantic regularities of an artificial orthography in Chinese children aged 8-10 years: 29 with dyslexia, 29 age-matched controls, and 30 reading-level matched controls. Despite having positional regularity learning performance comparable to the controls, the children with dyslexia were poorer at learning left-right structured characters than top-bottom structured characters in high- and low-consistency conditions. Moreover, they showed difficulties in mapping a given sound or meaning to a specific character compared with the typically developing controls. These findings suggest that children with dyslexia have deficits in some, though not all, aspects of statistical learning of character orthography, which may reflect their difficulties in coping with distractors and inconsistency of orthographic input.

Human free-operant performance varies with a concurrent task: Probability learning without a task, and schedule-consistent with a task.

Three experiments examined human rates and patterns of responding during exposure to various schedules of reinforcement with or without a concurrent task. In the presence of the concurrent task, performances were similar to those typically noted for nonhumans. Overall response rates were higher on medium-sized ratio schedules than on smaller or larger ratio schedules (Experiment 1), on interval schedules with shorter than longer values (Experiment 2), and on ratio compared with interval schedules with the same rate of reinforcement (Experiment 3). Moreover, bout-initiation responses were more susceptible to influence by rates of reinforcement than were within-bout responses across all experiments. In contrast, in the absence of a concurrent task, human schedule performance did not always display characteristics of nonhuman performance, but tended to be related to the relationship between rates of responding and reinforcement (feedback function), irrespective of the schedule of reinforcement employed. This was also true of within-bout responding, but not bout-initiations, which were not affected by the presence of a concurrent task. These data suggest the existence of two strategies for human responding on free-operant schedules, relatively mechanistic ones that apply to bout-initiation, and relatively explicit ones, that tend to apply to within-bout responding, and dominate human performance when other demands are not made on resources.

Simultaneous training on overlapping grapheme phoneme correspondences augments learning and retention.

An important component of learning to read is the acquisition of letter-to-sound mappings. The sheer quantity of mappings and many exceptions in opaque languages such as English suggests that children may use a form of statistical learning to acquire them. However, whereas statistical models of reading are item-based, reading instruction typically focuses on rule-based approaches involving small sets of regularities. This discrepancy poses the question of how different groupings of regularities, an unexamined factor of most reading curricula, may affect learning. Exploring the interplay between item statistics and rules, this study investigated how consonant variability, an item-level factor, and the degree of overlap among the to-be-trained vowel strings, a group-level factor, influence learning. English-speaking first graders (N = 361) were randomly assigned to be trained on vowel sets with high overlap (e.g., EA, AI) or low overlap (e.g., EE, AI); this was crossed with a manipulation of consonant frame variability. Whereas high vowel overlap led to poorer initial performance, it resulted in more learning when tested immediately and after a 2-week-delay. There was little beneficial effect of consonant variability. These findings indicate that online letter/sound processing affects how new knowledge is integrated into existing information. Moreover, they suggest that vowel overlap should be considered when designing reading curricula.

Specificity of representations in infants' visual statistical learning.

Past work has demonstrated infants' robust statistical learning across visual and auditory modalities. However, the specificity of representations produced via visual statistical learning has not been fully explored. The current study addressed this by investigating infants' abilities to identify previously learned object sequences when some object features (e.g., shape, face) aligned with prior learning and other features did not. Experiment 1 replicated past work demonstrating that infants can learn statistical regularities across sequentially presented objects and extended this finding to 16-month-olds. In Experiment 2, infants viewed test sequences in which one object feature (e.g., face) had been removed but the other feature (e.g., shape) was maintained, resulting in failure to identify familiar sequences. We further probed learning specificity by assessing infants' recognition of sequences when one feature was altered rather than removed (Experiment 3) and when one feature was uncorrelated with the original sequence structure (Experiment 4). In both cases, infants failed to identify sequences in which object features were not identical between learning and test. These findings suggest that infants are limited in their ability to generalize the statistical structure of an object sequence when the objects' features do not align between learning and test.

Electrophysiological signatures of visual statistical learning in 3-month-old infants at familial and low risk for autism spectrum disorder.

Visual statistical learning (VSL) refers to the ability to extract associations and conditional probabilities within the visual environment. It may serve as a precursor to cognitive and social communication development. Quantifying VSL in infants at familial risk (FR) for Autism Spectrum Disorder (ASD) provides opportunities to understand how genetic predisposition can influence early learning processes which may, in turn, lay a foundation for cognitive and social communication delays. We examined electroencephalography (EEG) signatures of VSL in 3-month-old infants, examining whether EEG correlates of VSL differentiated FR from low-risk (LR) infants. In an exploratory analysis, we then examined whether EEG correlates of VSL at 3 months relate to cognitive function and ASD symptoms at 18 months. Infants were exposed to a continuous stream of looming shape pairs with varying probability that the shapes would occur in sequence (high probability-deterministic condition; low probability-probabilistic condition). EEG was time-locked to shapes based on their transitional probabilities. EEG analysis examined group-level characteristics underlying specific components, including the late frontal positivity (LFP) and N700 responses. FR infants demonstrated increased LFP and N700 response to the probabilistic condition, whereas LR infants demonstrated increased LFP and N700 response to the deterministic condition. LFP at 3 months predicted 18-month visual reception skills and not ASD symptoms. Our findings thus provide evidence for distinct VSL processes in FR and LR infants as early as 3 months. Atypical pattern learning in FR infants may lay a foundation for later delays in higher level, nonverbal cognitive skills, and predict ASD symptoms well before an ASD diagnosis is made.