Events in context-The HED framework for the study of brain, experience and behavior.

The brain is a complex dynamic system whose current state is inextricably coupled to awareness of past, current, and anticipated future threats and opportunities that continually affect awareness and behavioral goals and decisions. Brain activity is driven on multiple time scales by an ever-evolving flow of sensory, proprioceptive, and idiothetic experience. Neuroimaging experiments seek to isolate and focus on some aspect of these complex dynamics to better understand how human experience, cognition, behavior, and health are supported by brain activity. Here we consider an event-related data modeling approach that seeks to parse experience and behavior into a set of time-delimited events. We distinguish between event processes themselves, that unfold through time, and event markers that record the experiment timeline latencies of event onset, offset, and any other event phase transitions. Precise descriptions of experiment events (sensory, motor, or other) allow participant experience and behavior to be interpreted in the context either of the event itself or of all or any experiment events. We discuss how events in neuroimaging experiments have been, are currently, and should best be identified and represented with emphasis on the importance of modeling both events and event context for meaningful interpretation of relationships between brain dynamics, experience, and behavior. We show how text annotation of time series neuroimaging data using the system of Hierarchical Event Descriptors (HED; https://www.hedtags.org) can more adequately model the roles of both events and their ever-evolving context than current data annotation practice and can thereby facilitate data analysis, meta-analysis, and mega-analysis. Finally, we discuss ways in which the HED system must continue to expand to serve the evolving needs of neuroimaging research.

Building FAIR Functionality: Annotating Events in Time Series Data Using Hierarchical Event Descriptors (HED).

Human electrophysiological and related time series data are often acquired in complex, event-rich environments. However, the resulting recorded brain or other dynamics are often interpreted in relation to more sparsely recorded or subsequently-noted events. Currently a substantial gap exists between the level of event description required by current digital data archiving standards and the level of annotation required for successful analysis of event-related data across studies, environments, and laboratories. Manifold challenges must be addressed, most prominently ontological clarity, vocabulary extensibility, annotation tool availability, and overall usability, to allow and promote sharing of data with an effective level of descriptive detail for labeled events. Motivating data authors to perform the work needed to adequately annotate their data is a key challenge. This paper describes new developments in the Hierarchical Event Descriptor (HED) system for addressing these issues. We recap the evolution of HED and its acceptance by the Brain Imaging Data Structure (BIDS) movement, describe the recent release of HED-3G, a third generation HED tools and design framework, and discuss directions for future development. Given consistent, sufficiently detailed, tool-enabled, field-relevant annotation of the nature of recorded events, prospects are bright for large-scale analysis and modeling of aggregated time series data, both in behavioral and brain imaging sciences and beyond.

Capturing the nature of events and event context using hierarchical event descriptors (HED).

Event-related data analysis plays a central role in EEG and MEG (MEEG) and other neuroimaging modalities including fMRI. Choices about which events to report and how to annotate their full natures significantly influence the value, reliability, and reproducibility of neuroimaging datasets for further analysis and meta- or mega-analysis. A powerful annotation strategy using the new third-generation formulation of the Hierarchical Event Descriptors (HED) framework and tools (hedtags.org) combines robust event description with details of experiment design and metadata in a human-readable as well as machine-actionable form, making event annotation relevant to the full range of neuroimaging and other time series data. This paper considers the event design and annotation process using as a case study the well-known multi-subject, multimodal dataset of Wakeman and Henson made available by its authors as a Brain Imaging Data Structure (BIDS) dataset (bids.neuroimaging.io). We propose a set of best practices and guidelines for event annotation integrated in a natural way into the BIDS metadata file architecture, examine the impact of event design decisions, and provide a working example of organizing events in MEEG and other neuroimaging data. We demonstrate how annotations using HED can document events occurring during neuroimaging experiments as well as their interrelationships, providing machine-actionable annotation enabling automated within- and across-experiment analysis and comparisons. We discuss the evolution of HED software tools and have made available an accompanying HED-annotated BIDS-formated edition of the MEEG data of the Wakeman and Henson dataset (openneuro.org, ds003645).

How Sensitive Are EEG Results to Preprocessing Methods: A Benchmarking Study.

Although several guidelines for best practices in EEG preprocessing have been released, even studies that strictly adhere to those guidelines contain considerable variation in the ways that the recommended methods are applied. An open question for researchers is how sensitive the results of EEG analyses are to variations in preprocessing methods and parameters. To address this issue, we analyze the effect of preprocessing methods on downstream EEG analysis using several simple signal and event-related measures. Signal measures include recording-level channel amplitudes, study-level channel amplitude dispersion, and recording spectral characteristics. Event-related methods include ERPs and ERSPs and their correlations across methods for a diverse set of stimulus events. Our analysis also assesses differences in residual signals both in the time and spectral domains after blink artifacts have been removed. Using fully automated pipelines, we evaluate these measures across 17 EEG studies for two ICA-based preprocessing approaches (LARG, MARA) plus two variations of Artifact Subspace Reconstruction (ASR). Although the general structure of the results is similar across these preprocessing methods, there are significant differences, particularly in the low-frequency spectral features and in the residuals left by blinks. These results argue for detailed reporting of processing details as suggested by most guidelines, but also for using a federation of automated processing pipelines and comparison tools to quantify effects of processing choices as part of the research reporting.

Automated EEG mega-analysis II: Cognitive aspects of event related features.

We present the results of a large-scale analysis of event-related responses based on raw EEG data from 17 studies performed at six experimental sites associated with four different institutions. The analysis corpus represents 1,155 recordings containing approximately 7.8 million event instances acquired under several different experimental paradigms. Such large-scale analysis is predicated on consistent data organization and event annotation as well as an effective automated preprocessing pipeline to transform raw EEG into a form suitable for comparative analysis. A key component of this analysis is the annotation of study-specific event codes using a common vocabulary to describe relevant event features. We demonstrate that Hierarchical Event Descriptors (HED tags) capture statistically significant cognitive aspects of EEG events common across multiple recordings, subjects, studies, paradigms, headset configurations, and experimental sites. We use representational similarity analysis (RSA) to show that EEG responses annotated with the same cognitive aspect are significantly more similar than those that do not share that cognitive aspect. These RSA similarity results are supported by visualizations that exploit the non-linear similarities of these associations. We apply temporal overlap regression, reducing confounds caused by adjacent event instances, to extract time and time-frequency EEG features (regressed ERPs and ERSPs) that are comparable across studies and replicate findings from prior, individual studies. Likewise, we use second-level linear regression to separate effects of different cognitive aspects on these features across all studies. This work demonstrates that EEG mega-analysis (pooling of raw data across studies) can enable investigations of brain dynamics in a more generalized fashion than single studies afford. A companion paper complements this event-based analysis by addressing commonality of the time and frequency statistical properties of EEG across studies at the channel and dipole level.

Automated EEG mega-analysis I: Spectral and amplitude characteristics across studies.

Significant achievements have been made in the fMRI field by pooling statistical results from multiple studies (meta-analysis). More recently, fMRI standardization efforts have focused on enabling the joint analysis of raw fMRI data across studies (mega-analysis), with the hope of achieving more detailed insights. However, it has not been clear if such analyses in the EEG field are possible or equally fruitful. Here we present the results of a large-scale EEG mega-analysis using 18 studies from six sites representing several different experimental paradigms. We demonstrate that when meta-data are consistent across studies, both channel-level and source-level EEG mega-analysis are possible and can provide insights unavailable in single studies. The analysis uses a fully-automated processing pipeline to reduce line noise, interpolate noisy channels, perform robust referencing, remove eye-activity, and further identify outlier signals. We define several robust measures based on channel amplitude and dispersion to assess the comparability of data across studies and observe the effect of various processing steps on these measures. Using ICA-based dipolar sources, we also observe consistent differences in overall frequency baseline amplitudes across brain areas. For example, we observe higher alpha in posterior vs anterior regions and higher beta in temporal regions. We also detect consistent differences in the slope of the aperiodic portion of the EEG spectrum across brain areas. In a companion paper, we apply mega-analysis to assess commonalities in event-related EEG features across studies. The continuous raw and preprocessed data used in this analysis are available through the DataCatalog at https://cancta.net.

An 18-subject EEG data collection using a visual-oddball task, designed for benchmarking algorithms and headset performance comparisons.

This data note describes an 18-subject EEG (electroencephalogram) data collection from an experiment in which subjects performed a standard visual oddball task. Several research projects have used this data to test artifact detection, classification, transfer learning, EEG preprocessing, blink detection, and automated annotation algorithms. We are releasing the data in three formats to enable benchmarking of EEG algorithms in many areas. The data was acquired using a Biosemi Active 2 EEG headset and includes 64 channels of EEG, 4 channels of EOG (electrooculogram), and 2 mastoid reference channels.

Single-Trial Classification of Disfluent Brain States in Adults Who Stutter.

Normal human speech requires precise coordination between motor planning and sensory processing. Speech disfluencies are common when children learn to talk, but usually abate with time. About 5% of children experience stuttering. For most, this resolves within a year. However, for approximately 1% of the world population, stuttering continues into adulthood, which is termed 'persistent developmental stuttering'. Most stuttering events occur at the beginning of an utterance. So, in principle, brain activity before speaking should differ between fluent and stuttered speech. Here we present a method for classifying brain network states associated with fluent vs. stuttered speech on a single trial basis. Brain activity was recorded with EEG before people who stutter read aloud pseudo-word pairs. Offline independent component analysis (ICA) was used to identify the independent neural sources that underlie speech preparation. A time window selection algorithm extracted spectral power and coherence data from salient windows specific to each neural source. A stepwise linear discriminant analysis (sLDA) algorithm predicted fluent vs. stuttered speech for 81% of trials in two subjects. These results support the feasibility of developing a brain-computer interface (BCI) system to detect stuttering before it occurs, with potential for therapeutic application.

EEG-Annotate: Automated identification and labeling of events in continuous signals with applications to EEG.

BACKGROUND: In controlled laboratory EEG experiments, researchers carefully mark events and analyze subject responses time-locked to these events. Unfortunately, such markers may not be available or may come with poor timing resolution for experiments conducted in less-controlled naturalistic environments. NEW METHOD: We present an integrated event-identification method for identifying particular responses that occur in unlabeled continuously recorded EEG signals based on information from recordings of other subjects potentially performing related tasks. We introduce the idea of timing slack and timing-tolerant performance measures to deal with jitter inherent in such non-time-locked systems. We have developed an implementation available as an open-source MATLAB toolbox (http://github.com/VisLab/EEG-Annotate) and have made test data available in a separate data note. RESULTS: We applied the method to identify visual presentation events (both target and non-target) in data from an unlabeled subject using labeled data from other subjects with good sensitivity and specificity. The method also identified actual visual presentation events in the data that were not previously marked in the experiment. COMPARISON WITH EXISTING METHODS: Although the method uses traditional classifiers for initial stages, the problem of identifying events based on the presence of stereotypical EEG responses is the converse of the traditional stimulus-response paradigm and has not been addressed in its current form. CONCLUSIONS: In addition to identifying potential events in unlabeled or incompletely labeled EEG, these methods also allow researchers to investigate whether particular stereotypical neural responses are present in other circumstances. Timing-tolerance has the added benefit of accommodating inter- and intra- subject timing variations.

BLINKER: Automated Extraction of Ocular Indices from EEG Enabling Large-Scale Analysis.

Electroencephalography (EEG) offers a platform for studying the relationships between behavioral measures, such as blink rate and duration, with neural correlates of fatigue and attention, such as theta and alpha band power. Further, the existence of EEG studies covering a variety of subjects and tasks provides opportunities for the community to better characterize variability of these measures across tasks and subjects. We have implemented an automated pipeline (BLINKER) for extracting ocular indices such as blink rate, blink duration, and blink velocity-amplitude ratios from EEG channels, EOG channels, and/or independent components (ICs). To illustrate the use of our approach, we have applied the pipeline to a large corpus of EEG data (comprising more than 2000 datasets acquired at eight different laboratories) in order to characterize variability of certain ocular indicators across subjects. We also investigate dependence of ocular indices on task in a shooter study. We have implemented our algorithms in a freely available MATLAB toolbox called BLINKER. The toolbox, which is easy to use and can be applied to collections of data without user intervention, can automatically discover which channels or ICs capture blinks. The tools extract blinks, calculate common ocular indices, generate a report for each dataset, dump labeled images of the individual blinks, and provide summary statistics across collections. Users can run BLINKER as a script or as a plugin for EEGLAB. The toolbox is available at https://github.com/VisLab/EEG-Blinks. User documentation and examples appear at http://vislab.github.io/EEG-Blinks/.

BLASST: Band Limited Atomic Sampling With Spectral Tuning With Applications to Utility Line Noise Filtering.

OBJECTIVE: In this paper, we present and test a new method for the identification and removal of nonstationary utility line noise from biomedical signals. METHODS: The method, band limited atomic sampling with spectral tuning (BLASST), is an iterative approach that is designed to 1) fit nonstationarities in line noise by searching for best-fit Gabor atoms at predetermined time points, 2) self-modulate its fit by leveraging information from frequencies surrounding the target frequency, and 3) terminate based on a convergence criterion obtained from the same surrounding frequencies. To evaluate the performance of the proposed algorithm, we generate several simulated and real instances of nonstationary line noise and test BLASST along with alternative filtering approaches. RESULTS: We find that BLASST is capable of fitting line noise well and/or preserving local signal features relative to tested alternative filtering techniques. CONCLUSION: BLASST may present a useful alternative to bandpass, notch, or other filtering methods when experimentally relevant features have significant power in a spectrum that is contaminated by utility line noise, or when the line noise in question is highly nonstationary. SIGNIFICANCE: This is of particular significance in electroencephalography experiments, where line noise may be present in the frequency bands of neurological interest and measurements are typically of low enough strength that induced line noise can dominate the recorded signals. In conjunction with this paper, the authors have released a MATLAB toolbox that performs BLASST on real, vector-valued signals (available at https://github.com/VisLab/blasst).

Hierarchical Event Descriptors (HED): Semi-Structured Tagging for Real-World Events in Large-Scale EEG.

Real-world brain imaging by EEG requires accurate annotation of complex subject-environment interactions in event-rich tasks and paradigms. This paper describes the evolution of the Hierarchical Event Descriptor (HED) system for systematically describing both laboratory and real-world events. HED version 2, first described here, provides the semantic capability of describing a variety of subject and environmental states. HED descriptions can include stimulus presentation events on screen or in virtual worlds, experimental or spontaneous events occurring in the real world environment, and events experienced via one or multiple sensory modalities. Furthermore, HED 2 can distinguish between the mere presence of an object and its actual (or putative) perception by a subject. Although the HED framework has implicit ontological and linked data representations, the user-interface for HED annotation is more intuitive than traditional ontological annotation. We believe that hiding the formal representations allows for a more user-friendly interface, making consistent, detailed tagging of experimental, and real-world events possible for research users. HED is extensible while retaining the advantages of having an enforced common core vocabulary. We have developed a collection of tools to support HED tag assignment and validation; these are available at hedtags.org. A plug-in for EEGLAB (sccn.ucsd.edu/eeglab), CTAGGER, is also available to speed the process of tagging existing studies.

PWC-ICA: A Method for Stationary Ordered Blind Source Separation with Application to EEG.

Independent component analysis (ICA) is a class of algorithms widely applied to separate sources in EEG data. Most ICA approaches use optimization criteria derived from temporal statistical independence and are invariant with respect to the actual ordering of individual observations. We propose a method of mapping real signals into a complex vector space that takes into account the temporal order of signals and enforces certain mixing stationarity constraints. The resulting procedure, which we call Pairwise Complex Independent Component Analysis (PWC-ICA), performs the ICA in a complex setting and then reinterprets the results in the original observation space. We examine the performance of our candidate approach relative to several existing ICA algorithms for the blind source separation (BSS) problem on both real and simulated EEG data. On simulated data, PWC-ICA is often capable of achieving a better solution to the BSS problem than AMICA, Extended Infomax, or FastICA. On real data, the dipole interpretations of the BSS solutions discovered by PWC-ICA are physically plausible, are competitive with existing ICA approaches, and may represent sources undiscovered by other ICA methods. In conjunction with this paper, the authors have released a MATLAB toolbox that performs PWC-ICA on real, vector-valued signals.

Preparing Laboratory and Real-World EEG Data for Large-Scale Analysis: A Containerized Approach.

Large-scale analysis of EEG and other physiological measures promises new insights into brain processes and more accurate and robust brain-computer interface models. However, the absence of standardized vocabularies for annotating events in a machine understandable manner, the welter of collection-specific data organizations, the difficulty in moving data across processing platforms, and the unavailability of agreed-upon standards for preprocessing have prevented large-scale analyses of EEG. Here we describe a "containerized" approach and freely available tools we have developed to facilitate the process of annotating, packaging, and preprocessing EEG data collections to enable data sharing, archiving, large-scale machine learning/data mining and (meta-)analysis. The EEG Study Schema (ESS) comprises three data "Levels," each with its own XML-document schema and file/folder convention, plus a standardized (PREP) pipeline to move raw (Data Level 1) data to a basic preprocessed state (Data Level 2) suitable for application of a large class of EEG analysis methods. Researchers can ship a study as a single unit and operate on its data using a standardized interface. ESS does not require a central database and provides all the metadata data necessary to execute a wide variety of EEG processing pipelines. The primary focus of ESS is automated in-depth analysis and meta-analysis EEG studies. However, ESS can also encapsulate meta-information for the other modalities such as eye tracking, that are increasingly used in both laboratory and real-world neuroimaging. ESS schema and tools are freely available at www.eegstudy.org and a central catalog of over 850 GB of existing data in ESS format is available at studycatalog.org. These tools and resources are part of a larger effort to enable data sharing at sufficient scale for researchers to engage in truly large-scale EEG analysis and data mining (BigEEG.org).

The PREP pipeline: standardized preprocessing for large-scale EEG analysis.

The technology to collect brain imaging and physiological measures has become portable and ubiquitous, opening the possibility of large-scale analysis of real-world human imaging. By its nature, such data is large and complex, making automated processing essential. This paper shows how lack of attention to the very early stages of an EEG preprocessing pipeline can reduce the signal-to-noise ratio and introduce unwanted artifacts into the data, particularly for computations done in single precision. We demonstrate that ordinary average referencing improves the signal-to-noise ratio, but that noisy channels can contaminate the results. We also show that identification of noisy channels depends on the reference and examine the complex interaction of filtering, noisy channel identification, and referencing. We introduce a multi-stage robust referencing scheme to deal with the noisy channel-reference interaction. We propose a standardized early-stage EEG processing pipeline (PREP) and discuss the application of the pipeline to more than 600 EEG datasets. The pipeline includes an automatically generated report for each dataset processed. Users can download the PREP pipeline as a freely available MATLAB library from http://eegstudy.org/prepcode.

Classification of imperfectly time-locked image RSVP events with EEG device.

Classification based on EEG data in an RSVP experiment is considered. Although the latency in neural response relative to the stimulus onset time may be more realistically considered to vary across trials due to factors such as subject fatigue and environmental distractions, it is nevertheless assumed to be time-locked to the stimulus in most of the existing work as a means to alleviate the computational complexity. We consider here a more practical scenario that allows variation in response latency and develop a rigorous statistical formulation for modeling the uncertainty within the varying latency coupled with a likelihood ratio test (LRT) for classification. The new model not only improves the EEG classification performance, but also may predict the true stimulus onset time when this information is not precisely available. We test the proposed LRT algorithm on an EEG data set from an image RSVP experiment and show that, by admitting the latency variation, the proposed approach consistently outperforms a method that relies on perfect time-locking (AUC: 0.88 vs 0.86), especially when the stimulus onset time is not precisely available (AUC: 0.84 vs 0.71). Furthermore, the predicted stimulus onset times are highly enriched around the true onset time with p-value = 5.2 x 10(44).

MOBBED: a computational data infrastructure for handling large collections of event-rich time series datasets in MATLAB.

Experiments to monitor human brain activity during active behavior record a variety of modalities (e.g., EEG, eye tracking, motion capture, respiration monitoring) and capture a complex environmental context leading to large, event-rich time series datasets. The considerable variability of responses within and among subjects in more realistic behavioral scenarios requires experiments to assess many more subjects over longer periods of time. This explosion of data requires better computational infrastructure to more systematically explore and process these collections. MOBBED is a lightweight, easy-to-use, extensible toolkit that allows users to incorporate a computational database into their normal MATLAB workflow. Although capable of storing quite general types of annotated data, MOBBED is particularly oriented to multichannel time series such as EEG that have event streams overlaid with sensor data. MOBBED directly supports access to individual events, data frames, and time-stamped feature vectors, allowing users to ask questions such as what types of events or features co-occur under various experimental conditions. A database provides several advantages not available to users who process one dataset at a time from the local file system. In addition to archiving primary data in a central place to save space and avoid inconsistencies, such a database allows users to manage, search, and retrieve events across multiple datasets without reading the entire dataset. The database also provides infrastructure for handling more complex event patterns that include environmental and contextual conditions. The database can also be used as a cache for expensive intermediate results that are reused in such activities as cross-validation of machine learning algorithms. MOBBED is implemented over PostgreSQL, a widely used open source database, and is freely available under the GNU general public license at http://visual.cs.utsa.edu/mobbed. Source and issue reports for MOBBED are maintained at http://vislab.github.com/MobbedMatlab/

Detecting alpha spindle events in EEG time series using adaptive autoregressive models.

BACKGROUND: Rhythmic oscillatory activity is widely observed during a variety of subject behaviors and is believed to play a central role in information processing and control. A classic example of rhythmic activity is alpha spindles, which consist of short (0.5-2 s) bursts of high frequency alpha activity. Recent research has shown that alpha spindles in the parietal/occipital area are statistically related to fatigue and drowsiness. These spindles constitute sharp changes in the underlying statistical properties of the signal. Our hypothesis is that change point detection models can be used to identify the onset and duration of spindles in EEG. In this work we develop an algorithm that accurately identifies sudden bursts of narrowband oscillatory activity in EEG using techniques derived from change point analysis. Our motivating example is detection of alpha spindles in the parietal/occipital areas of the brain. Our goal is to develop an algorithm that can be applied to any type of rhythmic oscillatory activity of interest for accurate online detection. METHODS: In this work we propose modeling the alpha band EEG time series using discounted autoregressive (DAR) modeling. The DAR model uses a discounting rate to weigh points measured further in the past less heavily than points more recently observed. This model is used together with predictive loss scoring to identify periods of EEG data that are statistically significant. RESULTS: Our algorithm accurately captures changes in the statistical properties of the alpha frequency band. These statistical changes are highly correlated with alpha spindle occurrences and form a reliable measure for detecting alpha spindles in EEG. We achieve approximately 95% accuracy in detecting alpha spindles, with timing precision to within approximately 150 ms, for two datasets from an experiment of prolonged simulated driving, as well as in simulated EEG. Sensitivity and specificity values are above 0.9, and in many cases are above 0.95, for our analysis. CONCLUSION: Modeling the alpha band EEG using discounted AR models provides an efficient method for detecting oscillatory alpha activity in EEG. The method is based on statistical principles and can generally be applied to detect rhythmic activity in any frequency band or brain region.

DETECT: a MATLAB toolbox for event detection and identification in time series, with applications to artifact detection in EEG signals.

Recent advances in sensor and recording technology have allowed scientists to acquire very large time-series datasets. Researchers often analyze these datasets in the context of events, which are intervals of time where the properties of the signal change relative to a baseline signal. We have developed DETECT, a MATLAB toolbox for detecting event time intervals in long, multi-channel time series. Our primary goal is to produce a toolbox that is simple for researchers to use, allowing them to quickly train a model on multiple classes of events, assess the accuracy of the model, and determine how closely the results agree with their own manual identification of events without requiring extensive programming knowledge or machine learning experience. As an illustration, we discuss application of the DETECT toolbox for detecting signal artifacts found in continuous multi-channel EEG recordings and show the functionality of the tools found in the toolbox. We also discuss the application of DETECT for identifying irregular heartbeat waveforms found in electrocardiogram (ECG) data as an additional illustration.