Deep latent variable joint cognitive modeling of neural signals and human behavior.

As the field of computational cognitive neuroscience continues to expand and generate new theories, there is a growing need for more advanced methods to test the hypothesis of brain-behavior relationships. Recent progress in Bayesian cognitive modeling has enabled the combination of neural and behavioral models into a single unifying framework. However, these approaches require manual feature extraction, and lack the capability to discover previously unknown neural features in more complex data. Consequently, this would hinder the expressiveness of the models. To address these challenges, we propose a Neurocognitive Variational Autoencoder (NCVA) to conjoin high-dimensional EEG with a cognitive model in both generative and predictive modeling analyses. Importantly, our NCVA enables both the prediction of EEG signals given behavioral data and the estimation of cognitive model parameters from EEG signals. This novel approach can allow for a more comprehensive understanding of the triplet relationship between behavior, brain activity, and cognitive processes.

A tutorial on fitting joint models of M/EEG and behavior to understand cognition.

We present motivation and practical steps necessary to find parameter estimates of joint models of behavior and neural electrophysiological data. This tutorial is written for researchers wishing to build joint models of human behavior and scalp and intracranial electroencephalographic (EEG) or magnetoencephalographic (MEG) data, and more specifically those researchers who seek to understand human cognition. Although these techniques could easily be applied to animal models, the focus of this tutorial is on human participants. Joint modeling of M/EEG and behavior requires some knowledge of existing computational and cognitive theories, M/EEG artifact correction, M/EEG analysis techniques, cognitive modeling, and programming for statistical modeling implementation. This paper seeks to give an introduction to these techniques as they apply to estimating parameters from neurocognitive models of M/EEG and human behavior, and to evaluate model results and compare models. Due to our research and knowledge on the subject matter, our examples in this paper will focus on testing specific hypotheses in human decision-making theory. However, most of the motivation and discussion of this paper applies across many modeling procedures and applications. We provide Python (and linked R) code examples in the tutorial and appendix. Readers are encouraged to try the exercises at the end of the document.

The latency of a visual evoked potential tracks the onset of decision making.

Encoding of a sensory stimulus is believed to be the first step in perceptual decision making. Previous research has shown that electrical signals recorded from the human brain track evidence accumulation during perceptual decision making (Gold and Shadlen, 2007; O'Connell et al., 2012; Philiastides et al., 2014). In this study we directly tested the hypothesis that the latency of the N200 recorded by EEG (a negative peak occurring between 150 and 275 ms after stimulus presentation in human participants) reflects the visual encoding time (VET) required for completion of figure-ground segregation before evidence accumulation. We show that N200 latencies vary across individuals, are modulated by external visual noise, and increase response time by x milliseconds when they increase by x milliseconds, reflecting a linear regression slope of 1. Simulations of cognitive decision-making theory show that variation in human response times not related to evidence accumulation (non-decision time; NDT), including VET, are tracked by the fastest response times. Evidence that VET is tracked by N200 latencies was found by fitting a linear model between trial-averaged N200 latencies and the 10th percentiles of response times, a model-independent estimate of NDT. Fitting a novel neuro-cognitive model of decision making also yielded a slope of 1 between N200 latency and model-estimated NDT in multiple visual noise conditions, indicating that N200 latencies track the completion of visual encoding and the onset of evidence accumulation. The N200 waveforms were localized to the cortical surface at distributed temporal and extrastriate locations, consistent with a distributed network engaged in figure-ground segregation of the target stimulus.

Multi-Scale Neural Sources of EEG: Genuine, Equivalent, and Representative. A Tutorial Review.

A biophysical framework needed to interpret electrophysiological data recorded at multiple spatial scales of brain tissue is developed. Micro current sources at membrane surfaces produce local field potentials, electrocorticography, and electroencephalography (EEG). We categorize multi-scale sources as genuine, equivalent, or representative. Genuine sources occur at the micro scale of cell surfaces. Equivalent sources provide identical experimental outcomes over a range of scales and applications. In contrast, each representative source distribution is just one of many possible source distributions that yield similar experimental outcomes. Macro sources ("dipoles") may be defined at the macrocolumn (mm) scale and depend on several features of the micro sources-magnitudes, micro synchrony within columns, and distribution through the cortical depths. These micro source properties are determined by brain dynamics and the columnar structure of cortical tissue. The number of representative sources underlying EEG data depends on the spatial scale of neural tissue under study. EEG inverse solutions (e.g. dipole localization) and high resolution estimates (e.g. Laplacian, dura imaging) have both strengths and limitations that depend on experimental conditions. The proposed theoretical framework informs studies of EEG source localization, source characterization, and low pass filtering. It also facilitates interpretations of brain dynamics and cognition, including measures of synchrony, functional connections between cortical locations, and other aspects of brain complexity.

Beyond rates: time-varying dynamics of high frequency oscillations as a biomarker of the seizure onset zone.

Objective. High frequency oscillations (HFOs) recorded by intracranial electrodes have generated excitement for their potential to help localize epileptic tissue for surgical resection. However, the number of HFOs per minute (i.e. the HFO 'rate') is not stable over the duration of intracranial recordings; for example, the rate of HFOs increases during periods of slow-wave sleep. Moreover, HFOs that are predictive of epileptic tissue may occur in oscillatory patterns due to phase coupling with lower frequencies. Therefore, we sought to further characterize between-seizure (i.e. 'interictal') HFO dynamics both within and outside the seizure onset zone (SOZ).Approach. Using long-term intracranial EEG (mean duration 10.3 h) from 16 patients, we automatically detected HFOs using a new algorithm. We then fit a hierarchical negative binomial model to the HFO counts. To account for differences in HFO dynamics and rates between sleep and wakefulness, we also fit a mixture model to the same data that included the ability to switch between two discrete brain states that were automatically determined during the fitting process. The ability to predict the SOZ by model parameters describing HFO dynamics (i.e. clumping coefficients and coefficients of variation) was assessed using receiver operating characteristic curves.Main results. Parameters that described HFO dynamics were predictive of SOZ. In fact, these parameters were found to be more consistently predictive than HFO rate. Using concurrent scalp EEG in two patients, we show that the model-found brain states corresponded to (1) non-REM sleep and (2) awake and rapid eye movement sleep. However the brain state most likely corresponding to slow-wave sleep in the second model improved SOZ prediction compared to the first model for only some patients.Significance. This work suggests that delineation of SOZ with interictal data can be improved by the inclusion of time-varying HFO dynamics.

Causal role for the primate superior colliculus in the computation of evidence for perceptual decisions.

Trained monkeys performed a two-choice perceptual decision-making task in which they reported the perceived orientation of a dynamic Glass pattern, before and after unilateral, reversible, inactivation of a brainstem area-the superior colliculus (SC)-involved in preparing eye movements. We found that unilateral SC inactivation produced significant decision biases and changes in reaction times consistent with a causal role for the primate SC in perceptual decision-making. Fitting signal detection theory and sequential sampling models to the data showed that SC inactivation produced a decrease in the relative evidence for contralateral decisions, as if adding a constant offset to a time-varying evidence signal for the ipsilateral choice. The results provide causal evidence for an embodied cognition model of perceptual decision-making and provide compelling evidence that the SC of primates (a brainstem structure) plays a causal role in how evidence is computed for decisions-a process usually attributed to the forebrain.

Timing of readiness potentials reflect a decision-making process in the human brain.

Decision-making in two-alternative forced choice tasks has several underlying components including stimulus encoding, perceptual categorization, response selection, and response execution. Sequential sampling models of decision-making are based on an evidence accumulation process to a decision boundary. Animal and human studies have focused on perceptual categorization and provide evidence linking brain signals in parietal cortex to the evidence accumulation process. In this exploratory study, we use a task where the dominant contribution to response time is response selection and model the response time data with the drift-diffusion model. EEG measurement during the task show that the Readiness Potential (RP) recorded over motor areas has timing consistent with the evidence accumulation process. The duration of the RP predicts decision-making time, the duration of evidence accumulation, suggesting that the RP partly reflects an evidence accumulation process for response selection in the motor system. Thus, evidence accumulation may be a neural implementation of decision-making processes in both perceptual and motor systems. The contributions of perceptual categorization and response selection to evidence accumulation processes in decision-making tasks can be potentially evaluated by examining the timing of perceptual and motor EEG signals.

Automated Detection of High Frequency Oscillations in Human Scalp Electroencephalogram.

High frequency oscillations (HFOs) > 80 Hz are a promising biomarker of epileptic tissue. Recent evidence has shown that spontaneous HFOs can be recorded from the scalp, but detection of these electrographic events remains a challenge. Here, we modified a simple automatic detector, used originally for intracranial EEG (iEEG) recordings, to detect ripples and fast ripples in scalp EEG. We analyzed scalp EEG recordings of seven subjects and validated our detector and artifact rejection algorithm via visual review. Of the candidate events marked by the detector, 40% and 60% were confirmed to be ripples and fast ripples, respectively, by human visual review, making this algorithm suitable for supervised detection. Detected HFOs occurred at a rate of <1/min in most channels, and the average duration was 47 and 24 ms for ripples and fast ripples, respectively. The simplicity of the algorithm, with only a single parameter, enables the consistent application of automatic detection across recording modalities, and it could therefore be a tool for the assessment and localization of epileptic activity.

Moving Beyond ERP Components: A Selective Review of Approaches to Integrate EEG and Behavior.

Relationships between neuroimaging measures and behavior provide important clues about brain function and cognition in healthy and clinical populations. While electroencephalography (EEG) provides a portable, low cost measure of brain dynamics, it has been somewhat underrepresented in the emerging field of model-based inference. We seek to address this gap in this article by highlighting the utility of linking EEG and behavior, with an emphasis on approaches for EEG analysis that move beyond focusing on peaks or "components" derived from averaging EEG responses across trials and subjects (generating the event-related potential, ERP). First, we review methods for deriving features from EEG in order to enhance the signal within single-trials. These methods include filtering based on user-defined features (i.e., frequency decomposition, time-frequency decomposition), filtering based on data-driven properties (i.e., blind source separation, BSS), and generating more abstract representations of data (e.g., using deep learning). We then review cognitive models which extract latent variables from experimental tasks, including the drift diffusion model (DDM) and reinforcement learning (RL) approaches. Next, we discuss ways to access associations among these measures, including statistical models, data-driven joint models and cognitive joint modeling using hierarchical Bayesian models (HBMs). We think that these methodological tools are likely to contribute to theoretical advancements, and will help inform our understandings of brain dynamics that contribute to moment-to-moment cognitive function.

How attention influences perceptual decision making: Single-trial EEG correlates of drift-diffusion model parameters.

Perceptual decision making can be accounted for by drift-diffusion models, a class of decision-making models that assume a stochastic accumulation of evidence on each trial. Fitting response time and accuracy to a drift-diffusion model produces evidence accumulation rate and non-decision time parameter estimates that reflect cognitive processes. Our goal is to elucidate the effect of attention on visual decision making. In this study, we show that measures of attention obtained from simultaneous EEG recordings can explain per-trial evidence accumulation rates and perceptual preprocessing times during a visual decision making task. Models assuming linear relationships between diffusion model parameters and EEG measures as external inputs were fit in a single step in a hierarchical Bayesian framework. The EEG measures were features of the evoked potential (EP) to the onset of a masking noise and the onset of a task-relevant signal stimulus. Single-trial evoked EEG responses, P200s to the onsets of visual noise and N200s to the onsets of visual signal, explain single-trial evidence accumulation and preprocessing times. Within-trial evidence accumulation variance was not found to be influenced by attention to the signal or noise. Single-trial measures of attention lead to better out-of-sample predictions of accuracy and correct reaction time distributions for individual subjects.

Individual differences in attention influence perceptual decision making.

Sequential sampling decision-making models have been successful in accounting for reaction time (RT) and accuracy data in two-alternative forced choice tasks. These models have been used to describe the behavior of populations of participants, and explanatory structures have been proposed to account for between individual variability in model parameters. In this study we show that individual differences in behavior from a novel perceptual decision making task can be attributed to (1) differences in evidence accumulation rates, (2) differences in variability of evidence accumulation within trials, and (3) differences in non-decision times across individuals. Using electroencephalography (EEG), we demonstrate that these differences in cognitive variables, in turn, can be explained by attentional differences as measured by phase-locking of steady-state visual evoked potential (SSVEP) responses to the signal and noise components of the visual stimulus. Parameters of a cognitive model (a diffusion model) were obtained from accuracy and RT distributions and related to phase-locking indices (PLIs) of SSVEPs with a single step in a hierarchical Bayesian framework. Participants who were able to suppress the SSVEP response to visual noise in high frequency bands were able to accumulate correct evidence faster and had shorter non-decision times (preprocessing or motor response times), leading to more accurate responses and faster response times. We show that the combination of cognitive modeling and neural data in a hierarchical Bayesian framework relates physiological processes to the cognitive processes of participants, and that a model with a new (out-of-sample) participant's neural data can predict that participant's behavior more accurately than models without physiological data.