Designing optimal behavioral experiments using machine learning.

Computational models are powerful tools for understanding human cognition and behavior. They let us express our theories clearly and precisely and offer predictions that can be subtle and often counter-intuitive. However, this same richness and ability to surprise means our scientific intuitions and traditional tools are ill-suited to designing experiments to test and compare these models. To avoid these pitfalls and realize the full potential of computational modeling, we require tools to design experiments that provide clear answers about what models explain human behavior and the auxiliary assumptions those models must make. Bayesian optimal experimental design (BOED) formalizes the search for optimal experimental designs by identifying experiments that are expected to yield informative data. In this work, we provide a tutorial on leveraging recent advances in BOED and machine learning to find optimal experiments for any kind of model that we can simulate data from, and show how by-products of this procedure allow for quick and straightforward evaluation of models and their parameters against real experimental data. As a case study, we consider theories of how people balance exploration and exploitation in multi-armed bandit decision-making tasks. We validate the presented approach using simulations and a real-world experiment. As compared to experimental designs commonly used in the literature, we show that our optimal designs more efficiently determine which of a set of models best account for individual human behavior, and more efficiently characterize behavior given a preferred model. At the same time, formalizing a scientific question such that it can be adequately addressed with BOED can be challenging and we discuss several potential caveats and pitfalls that practitioners should be aware of. We provide code to replicate all analyses as well as tutorial notebooks and pointers to adapt the methodology to different experimental settings.

Can computational models help elucidate the link between complex trauma and hallucinations?

Recently, a number of predictive coding models have been proposed to account for post-traumatic stress disorder (PTSD)'s symptomatology, including intrusions, flashbacks and hallucinations. These models were usually developed to account for traditional/type-1 PTSD. We here discuss whether these models also apply or can be translated to the case of complex/type-2 PTSD and childhood trauma (cPTSD). The distinction between PTSD and cPTSD is important because the disorders differ in terms of symptomatology and potential mechanisms, how they relate to developmental stages, but also in terms of illness trajectory and treatment. Models of complex trauma could give us insights on hallucinations in physiological/pathological conditions or more generally on the development of intrusive experiences across diagnostic classes.

A primer on the use of computational modelling to investigate affective states, affective disorders and animal welfare in non-human animals.

Objective measures of animal emotion-like and mood-like states are essential for preclinical studies of affective disorders and for assessing the welfare of laboratory and other animals. However, the development and validation of measures of these affective states poses a challenge partly because the relationships between affect and its behavioural, physiological and cognitive signatures are complex. Here, we suggest that the crisp characterisations offered by computational modelling of the underlying, but unobservable, processes that mediate these signatures should provide better insights. Although this computational psychiatry approach has been widely used in human research in both health and disease, translational computational psychiatry studies remain few and far between. We explain how building computational models with data from animal studies could play a pivotal role in furthering our understanding of the aetiology of affective disorders, associated affective states and the likely underlying cognitive processes involved. We end by outlining the basic steps involved in a simple computational analysis.

Definition of early age at onset in bipolar disorder according to distinctive neurodevelopmental pathways: insights from the FACE-BD study.

BACKGROUND: Converging evidence suggests that a subgroup of bipolar disorder (BD) with an early age at onset (AAO) may develop from aberrant neurodevelopment. However, the definition of early AAO remains unprecise. We thus tested which age cut-off for early AAO best corresponds to distinguishable neurodevelopmental pathways. METHODS: We analyzed data from the FondaMental Advanced Center of Expertise-Bipolar Disorder cohort, a naturalistic sample of 4421 patients. First, a supervised learning framework was applied in binary classification experiments using neurodevelopmental history to predict early AAO, defined either with Gaussian mixture models (GMM) clustering or with each of the different cut-offs in the range 14 to 25 years. Second, an unsupervised learning approach was used to find clusters based on neurodevelopmental factors and to examine the overlap between such data-driven groups and definitions of early AAO used for supervised learning. RESULTS: A young cut-off, i.e. 14 up to 16 years, induced higher separability [mean nested cross-validation test AUROC = 0.7327 (± 0.0169) for ⩽16 years]. Predictive performance deteriorated increasing the cut-off or setting early AAO with GMM. Similarly, defining early AAO below 17 years was associated with a higher degree of overlap with data-driven clusters (Normalized Mutual Information = 0.41 for ⩽17 years) relatively to other definitions. CONCLUSIONS: Early AAO best captures distinctive neurodevelopmental patterns when defined as ⩽17 years. GMM-based definition of early AAO falls short of mapping to highly distinguishable neurodevelopmental pathways. These results should be used to improve patients' stratification in future studies of BD pathophysiology and biomarkers.

10 years of Bayesian theories of autism: A comprehensive review.

Ten years ago, Pellicano and Burr published one of the most influential articles in the study of autism spectrum disorders, linking them to aberrant Bayesian inference processes in the brain. In particular, they proposed that autistic individuals are less influenced by their brains' prior beliefs about the environment. In this systematic review, we investigate if this theory is supported by the experimental evidence. To that end, we collect all studies which included comparisons across diagnostic groups or autistic traits and categorise them based on the investigated priors. Our results are highly mixed, with a slight majority of studies finding no difference in the integration of Bayesian priors. We find that priors developed during the experiments exhibited reduced influences more frequently than priors acquired previously, with various studies providing evidence for learning differences between participant groups. Finally, we focus on the methodological and computational aspects of the included studies, showing low statistical power and often inconsistent approaches. Based on our findings, we propose guidelines for future research.

The effect of body image dissatisfaction on goal-directed decision making in a population marked by negative appearance beliefs and disordered eating.

Eating disorders are associated with one of the highest mortality rates among all mental disorders, yet there is very little research about them within the newly emerging and promising field of computational psychiatry. As such, we focus on investigating a previously unexplored, yet core aspect of eating disorders-body image dissatisfaction. We continue a freshly opened debate about model-based learning and its trade-off against model-free learning-a proxy for goal-directed and habitual behaviour. We perform a behavioural study that utilises a two-step decision-making task and a reinforcement learning model to understand the effect of body image dissatisfaction on model-based learning in a population characterised by high scores of disordered eating and negative appearance beliefs, as recruited using Prolific. We find a significantly reduced model-based contribution in the body image dissatisfaction task condition in the population of interest as compared to a healthy control. This finding suggests general deficits in deliberate control in this population, leading to habitual, compulsive-like behaviours (body checking) dominating the experience. Importantly, the results may inform treatment approaches, which could focus on enhancing the reliance on goal-directed decision making to help cope with unwanted behaviours.

Prediction of depression symptoms in individual subjects with face and eye movement tracking.

BACKGROUND: Depression is a challenge to diagnose reliably and the current gold standard for trials of DSM-5 has been in agreement between two or more medical specialists. Research studies aiming to objectively predict depression have typically used brain scanning. Less expensive methods from cognitive neuroscience may allow quicker and more reliable diagnoses, and contribute to reducing the costs of managing the condition. In the current study we aimed to develop a novel inexpensive system for detecting elevated symptoms of depression based on tracking face and eye movements during the performance of cognitive tasks. METHODS: In total, 75 participants performed two novel cognitive tasks with verbal affective distraction elements while their face and eye movements were recorded using inexpensive cameras. Data from 48 participants (mean age 25.5 years, standard deviation of 6.1 years, 25 with elevated symptoms of depression) passed quality control and were included in a case-control classification analysis with machine learning. RESULTS: Classification accuracy using cross-validation (within-study replication) reached 79% (sensitivity 76%, specificity 82%), when face and eye movement measures were combined. Symptomatic participants were characterised by less intense mouth and eyelid movements during different stages of the two tasks, and by differences in frequencies and durations of fixations on affectively salient distraction words. CONCLUSIONS: Elevated symptoms of depression can be detected with face and eye movement tracking during the cognitive performance, with a close to clinically-relevant accuracy (~80%). Future studies should validate these results in larger samples and in clinical populations.

Inflexible Updating of the Self-Other Divide During a Social Context in Autism: Psychophysical, Electrophysiological, and Neural Network Modeling Evidence.

BACKGROUND: Autism spectrum disorder (ASD) affects many aspects of life, from social interactions to (multi)sensory processing. Similarly, the condition expresses at a variety of levels of description, from genetics to neural circuits and interpersonal behavior. We attempt to bridge between domains and levels of description by detailing the behavioral, electrophysiological, and putative neural network basis of peripersonal space (PPS) updating in ASD during a social context, given that the encoding of this space relies on appropriate multisensory integration, is malleable by social context, and is thought to delineate the boundary between the self and others. METHODS: Fifty (20 male/30 female) young adults, either diagnosed with ASD or age- and sex-matched individuals, took part in a visuotactile reaction time task indexing PPS, while high-density electroencephalography was continuously recorded. Neural network modeling was performed in silico. RESULTS: Multisensory psychophysics demonstrates that while PPS in neurotypical individuals shrinks in the presence of others-as to "give space"-this does not occur in ASD. Likewise, electroencephalography recordings suggest that multisensory integration is altered by social context in neurotypical individuals but not in individuals with ASD. Finally, a biologically plausible neural network model shows, as a proof of principle, that PPS updating may be inflexible in ASD owing to the altered excitatory/inhibitory balance that characterizes neural circuits in animal models of ASD. CONCLUSIONS: Findings are conceptually in line with recent statistical inference accounts, suggesting diminished flexibility in ASD, and further these observations by suggesting within an example relevant for social cognition that such inflexibility may be due to excitatory/inhibitory imbalances.

Influence of E/I balance and pruning in peri-personal space differences in schizophrenia: A computational approach.

The encoding of the space close to the body, named peri-personal space (PPS), is thought to play a crucial role in the unusual experiences of the self observed in schizophrenia (SCZ). However, it is unclear why SCZ patients and high schizotypal (H-SPQ) individuals present a narrower PPS and why the boundaries of the PPS are more sharply defined in patients. We hypothesise that the unusual PPS representation observed in SCZ is caused by an imbalance of excitation and inhibition (E/I) in recurrent synapses of unisensory neurons or an impairment of bottom-up and top-down connectivity between unisensory and multisensory neurons. These hypotheses were tested computationally by manipulating the effects of E/I imbalance, feedback weights and synaptic density in the network. Using simulations we explored the effects of such impairments in the PPS representation generated by the network and fitted the model to behavioural data. We found that increased excitation of sensory neurons could account for the smaller PPS observed in SCZ and H-SPQ, whereas a decrease of synaptic density caused the sharp definition of the PPS observed in SCZ. We propose a novel conceptual model of PPS representation in the SCZ spectrum that can account for alterations in self-world demarcation, failures in tactile discrimination and symptoms observed in patients.

No increased circular inference in adults with high levels of autistic traits or autism.

Autism spectrum disorders have been proposed to arise from impairments in the probabilistic integration of prior knowledge with sensory inputs. Circular inference is one such possible impairment, in which excitation-to-inhibition imbalances in the cerebral cortex cause the reverberation and amplification of prior beliefs and sensory information. Recent empirical work has associated circular inference with the clinical dimensions of schizophrenia. Inhibition impairments have also been observed in autism, suggesting that signal reverberation might be present in that condition as well. In this study, we collected data from 21 participants with self-reported diagnoses of autism spectrum disorders and 155 participants with a broad range of autistic traits in an online probabilistic decision-making task (the fisher task). We used previously established Bayesian models to investigate possible associations between autistic traits or autism and circular inference. There was no correlation between prior or likelihood reverberation and autistic traits across the whole sample. Similarly, no differences in any of the circular inference model parameters were found between autistic participants and those with no diagnosis. Furthermore, participants incorporated information from both priors and likelihoods in their decisions, with no relationship between their weights and psychiatric traits, contrary to what common theories for both autism and schizophrenia would suggest. These findings suggest that there is no increased signal reverberation in autism, despite the known presence of excitation-to-inhibition imbalances. They can be used to further contrast and refine the Bayesian theories of schizophrenia and autism, revealing a divergence in the computational mechanisms underlying the two conditions.

Visual statistical learning and integration of perceptual priors are intact in attention deficit hyperactivity disorder.

BACKGROUND: Deficits in visual statistical learning and predictive processing could in principle explain the key characteristics of inattention and distractibility in attention deficit hyperactivity disorder (ADHD). Specifically, from a Bayesian perspective, ADHD may be associated with flatter likelihoods (increased sensory processing noise), and/or difficulties in generating or using predictions. To our knowledge, such hypotheses have never been directly tested. METHODS: We here test these hypotheses by evaluating whether adults diagnosed with ADHD (n = 17) differed from a control group (n = 30) in implicitly learning and using low-level perceptual priors to guide sensory processing. We used a visual statistical learning task in which participants had to estimate the direction of a cloud of coherently moving dots. Unbeknown to the participants, two of the directions were more frequently presented than the others, creating an implicit bias (prior) towards those directions. This task had previously revealed differences in other neurodevelopmental disorders, such as autistic spectrum disorder and schizophrenia. RESULTS: We found that both groups acquired the prior expectation for the most frequent directions and that these expectations substantially influenced task performance. Overall, there were no group differences in how much the priors influenced performance. However, subtle group differences were found in the influence of the prior over time. CONCLUSION: Our findings suggest that the symptoms of inattention and hyperactivity in ADHD do not stem from broad difficulties in developing and/or using low-level perceptual priors.

Blunted medial prefrontal cortico-limbic reward-related effective connectivity and depression.

Major depressive disorder is a leading cause of disability and significant mortality, yet mechanistic understanding remains limited. Over the past decade evidence has accumulated from case-control studies that depressive illness is associated with blunted reward activation in the basal ganglia and other regions such as the medial prefrontal cortex. However it is unclear whether this finding can be replicated in a large number of subjects. The functional anatomy of the medial prefrontal cortex and basal ganglia has been extensively studied and the former has excitatory glutamatergic projections to the latter. Reduced effect of glutamatergic projections from the prefrontal cortex to the nucleus accumbens has been argued to underlie motivational disorders such as depression, and many prominent theories of major depressive disorder propose a role for abnormal cortico-limbic connectivity. However, it is unclear whether there is abnormal reward-linked effective connectivity between the medial prefrontal cortex and basal ganglia related to depression. While resting state connectivity abnormalities have been frequently reported in depression, it has not been possible to directly link these findings to reward-learning studies. Here, we tested two main hypotheses. First, mood symptoms are associated with blunted striatal reward prediction error signals in a large community-based sample of recovered and currently ill patients, similar to reports from a number of studies. Second, event-related directed medial prefrontal cortex to basal ganglia effective connectivity is abnormally increased or decreased related to the severity of mood symptoms. Using a Research Domain Criteria approach, data were acquired from a large community-based sample of subjects who participated in a probabilistic reward learning task during event-related functional MRI. Computational modelling of behaviour, model-free and model-based functional MRI, and effective connectivity dynamic causal modelling analyses were used to test hypotheses. Increased depressive symptom severity was related to decreased reward signals in areas which included the nucleus accumbens in 475 participants. Decreased reward-related effective connectivity from the medial prefrontal cortex to striatum was associated with increased depressive symptom severity in 165 participants. Decreased striatal activity may have been due to decreased cortical to striatal connectivity consistent with glutamatergic and cortical-limbic related theories of depression and resulted in reduced direct pathway basal ganglia output. Further study of basal ganglia pathophysiology is required to better understand these abnormalities in patients with depressive symptoms and syndromes.

Acquisition of visual priors and induced hallucinations in chronic schizophrenia.

Prominent theories suggest that symptoms of schizophrenia stem from learning deficiencies resulting in distorted internal models of the world. To test these theories further, we used a visual statistical learning task known to induce rapid implicit learning of the stimulus statistics. In this task, participants are presented with a field of coherently moving dots and are asked to report the presented direction of the dots (estimation task), and whether they saw any dots or not (detection task). Two of the directions were more frequently presented than the others. In controls, the implicit acquisition of the stimuli statistics influences their perception in two ways: (i) motion directions are perceived as being more similar to the most frequently presented directions than they really are (estimation biases); and (ii) in the absence of stimuli, participants sometimes report perceiving the most frequently presented directions (a form of hallucinations). Such behaviour is consistent with probabilistic inference, i.e. combining learnt perceptual priors with sensory evidence. We investigated whether patients with chronic, stable, treated schizophrenia (n = 20) differ from controls (n = 23) in the acquisition of the perceptual priors and/or their influence on perception. We found that although patients were slower than controls, they showed comparable acquisition of perceptual priors, approximating the stimulus statistics. This suggests that patients have no statistical learning deficits in our task. This may reflect our patients' relative wellbeing on antipsychotic medication. Intriguingly, however, patients experienced significantly fewer (P = 0.016) hallucinations of the most frequently presented directions than controls when the stimulus was absent or when it was very weak (prior-based lapse estimations). This suggests that prior expectations had less influence on patients' perception than on controls when stimuli were absent or below perceptual threshold.

Major Depression Impairs the Use of Reward Values for Decision-Making.

Depression is a debilitating condition with a high prevalence. Depressed patients have been shown to be diminished in their ability to integrate their reinforcement history to adjust future behaviour during instrumental reward learning tasks. Here, we tested whether such impairments could also be observed in a Pavlovian conditioning task. We recruited and analysed 32 subjects, 15 with depression and 17 healthy controls, to study behavioural group differences in learning and decision-making. Participants had to estimate the probability of some fractal stimuli to be associated with a binary reward, based on a few passive observations. They then had to make a choice between one of the observed fractals and another target for which the reward probability was explicitly given. Computational modelling was used to succinctly describe participants' behaviour. Patients performed worse than controls at the task. Computational modelling revealed that this was caused by behavioural impairments during both learning and decision phases. Depressed subjects showed lower memory of observed rewards and had an impaired ability to use internal value estimations to guide decision-making in our task.

Neurons That Update Representations of the Future.

A recent article shows that the brain automatically estimates the probabilities of possible future actions before it has even received all the information necessary to decide what to do next.

Autistic traits, but not schizotypy, predict increased weighting of sensory information in Bayesian visual integration.

Recent theories propose that schizophrenia/schizotypy and autistic spectrum disorder are related to impairments in Bayesian inference that is, how the brain integrates sensory information (likelihoods) with prior knowledge. However existing accounts fail to clarify: (i) how proposed theories differ in accounts of ASD vs. schizophrenia and (ii) whether the impairments result from weaker priors or enhanced likelihoods. Here, we directly address these issues by characterizing how 91 healthy participants, scored for autistic and schizotypal traits, implicitly learned and combined priors with sensory information. This was accomplished through a visual statistical learning paradigm designed to quantitatively assess variations in individuals' likelihoods and priors. The acquisition of the priors was found to be intact along both traits spectra. However, autistic traits were associated with more veridical perception and weaker influence of expectations. Bayesian modeling revealed that this was due, not to weaker prior expectations, but to more precise sensory representations.

Performance-monitoring integrated reweighting model of perceptual learning.

Perceptual learning (PL) has been traditionally thought of as highly specific to stimulus properties, task and retinotopic position. This view is being progressively challenged, with accumulating evidence that learning can generalize (transfer) across various parameters under certain conditions. For example, retinotopic specificity can be diminished when the proportion of easy to hard trials is high, such as when multiple short staircases, instead of a single long one, are used during training. To date, there is a paucity of mechanistic explanations of what conditions affect transfer of learning. Here we present a model based on the popular Integrated Reweighting Theory model of PL but departing from its one-layer architecture by including a novel key feature: dynamic weighting of retinotopic-location-specific vs location-independent representations based on internal performance estimates of these representations. This dynamic weighting is closely related to gating in a mixture-of-experts architecture. Our dynamic performance-monitoring model (DPMM) unifies a variety of psychophysical data on transfer of PL, such as the short-vs-long staircase effect, as well as several findings from the double-training literature. Furthermore, the DPMM makes testable predictions and ultimately helps understand the mechanisms of generalization of PL, with potential applications to vision rehabilitation and enhancement.

The Influence of Feedback on Task-Switching Performance: A Drift Diffusion Modeling Account.

Task-switching is an important cognitive skill that facilitates our ability to choose appropriate behavior in a varied and changing environment. Task-switching training studies have sought to improve this ability by practicing switching between multiple tasks. However, an efficacious training paradigm has been difficult to develop in part due to findings that small differences in task parameters influence switching behavior in a non-trivial manner. Here, for the first time we employ the Drift Diffusion Model (DDM) to understand the influence of feedback on task-switching and investigate how drift diffusion parameters change over the course of task switch training. We trained 316 participants on a simple task where they alternated sorting stimuli by color or by shape. Feedback differed in six different ways between subjects groups, ranging from No Feedback (NFB) to a variety of manipulations addressing trial-wise vs. Block Feedback (BFB), rewards vs. punishments, payment bonuses and different payouts depending upon the trial type (switch/non-switch). While overall performance was found to be affected by feedback, no effect of feedback was found on task-switching learning. Drift Diffusion Modeling revealed that the reductions in reaction time (RT) switch cost over the course of training were driven by a continually decreasing decision boundary. Furthermore, feedback effects on RT switch cost were also driven by differences in decision boundary, but not in drift rate. These results reveal that participants systematically modified their task-switching performance without yielding an overall gain in performance.