Evaluation of artificial intelligence systems for assisting neurologists with fast and accurate annotations of scalp electroencephalography data.

BACKGROUND: Assistive automatic seizure detection can empower human annotators to shorten patient monitoring data review times. We present a proof-of-concept for a seizure detection system that is sensitive, automated, patient-specific, and tunable to maximise sensitivity while minimizing human annotation times. The system uses custom data preparation methods, deep learning analytics and electroencephalography (EEG) data. METHODS: Scalp EEG data of 365 patients containing 171,745 s ictal and 2,185,864 s interictal samples obtained from clinical monitoring systems were analysed as part of a crowdsourced artificial intelligence (AI) challenge. Participants were tasked to develop an ictal/interictal classifier with high sensitivity and low false alarm rates. We built a challenge platform that prevented participants from downloading or directly accessing the data while allowing crowdsourced model development. FINDINGS: The automatic detection system achieved tunable sensitivities between 75.00% and 91.60% allowing a reduction in the amount of raw EEG data to be reviewed by a human annotator by factors between 142x, and 22x respectively. The algorithm enables instantaneous reviewer-managed optimization of the balance between sensitivity and the amount of raw EEG data to be reviewed. INTERPRETATION: This study demonstrates the utility of deep learning for patient-specific seizure detection in EEG data. Furthermore, deep learning in combination with a human reviewer can provide the basis for an assistive data labelling system lowering the time of manual review while maintaining human expert annotation performance. FUNDING: IBM employed all IBM Research authors. Temple University employed all Temple University authors. The Icahn School of Medicine at Mount Sinai employed Eren Ahsen. The corresponding authors Stefan Harrer and Gustavo Stolovitzky declare that they had full access to all the data in the study and that they had final responsibility for the decision to submit for publication.

Automatic Analysis of EEGs Using Big Data and Hybrid Deep Learning Architectures.

Brain monitoring combined with automatic analysis of EEGs provides a clinical decision support tool that can reduce time to diagnosis and assist clinicians in real-time monitoring applications (e.g., neurological intensive care units). Clinicians have indicated that a sensitivity of 95% with specificity below 5% was the minimum requirement for clinical acceptance. In this study, a high-performance automated EEG analysis system based on principles of machine learning and big data is proposed. This hybrid architecture integrates hidden Markov models (HMMs) for sequential decoding of EEG events with deep learning-based post-processing that incorporates temporal and spatial context. These algorithms are trained and evaluated using the Temple University Hospital EEG, which is the largest publicly available corpus of clinical EEG recordings in the world. This system automatically processes EEG records and classifies three patterns of clinical interest in brain activity that might be useful in diagnosing brain disorders: (1) spike and/or sharp waves, (2) generalized periodic epileptiform discharges, (3) periodic lateralized epileptiform discharges. It also classifies three patterns used to model the background EEG activity: (1) eye movement, (2) artifacts, and (3) background. Our approach delivers a sensitivity above 90% while maintaining a specificity below 5%. We also demonstrate that this system delivers a low false alarm rate, which is critical for any spike detection application.

Application of identity vectors for EEG classification.

BACKGROUND: Finding an optimal EEG subject verification algorithm is a long standing goal within the EEG community. For every advancement made, another feature set, classifier, or dataset is often introduced; tracking improvements in classification without a consistent benchmark, such as a classifier-feature pairing tested on a publicly available dataset, makes it difficult to understand how and why these improvements occur. NEW METHOD: Following on previous biometric experiments, I-Vectors and Gaussian Mixture Model-Universal Background Models are compared to an established Mahalanobis classifier. A second experiment then addresses the impact of epoch duration as a function of classification performance across all three classifiers. RESULTS: The experimental classification results indicate that I-Vectors are more robust than the other classifiers displaying less sensitivity to epoch duration, data composition, and feature selection. COMPARISON WITH EXISTING METHODS: This I-Vector based approach is compared against commonly used EEG classifiers, such as Mahalanobis and Gaussian mixture models. These classifiers are benchmarked using the publicly available PhysioNet database converted into three feature sets, spectral coherence, power spectral density, and cepstral coefficients. CONCLUSIONS: The experimental results suggests I-Vectors provide reliable baseline performance by leveling the field between feature set and datasets making them well suited for EEG signal processing tasks.

Validity of an Automated Balance Error Scoring System.

The Balance Error Scoring System (BESS) is a human-scored, field-based balance test used in cases of suspected concussion. Recently developed instrumented alternatives to human scoring carry substantial advantages over traditional testing, but thus far report relatively abstract outcomes which may not be useful to clinicians or coaches. In contrast, the Automated Assessment of Postural Stability (AAPS) is a computerized system that tabulates error events in accordance with the original description of the BESS. This study compared AAPS and human-based BESS scores. Twenty-five healthy adults performed the modified BESS. Tests were scored twice each by human raters (3) and the computerized system. Interrater (between-human) and inter-method (AAPS vs. human) agreement (ICC(2,1)) were calculated alongside Bland-Altman limits of agreement (LOA). Interrater analyses were significant (p<0.005) and demonstrated good to excellent agreement. Inter-method agreement analyses were significant (p<0.005), with agreement ranging from poor to excellent. Computerized scores were equivalent across rating occasions. LOA ranges for AAPS vs. the Human Average exceeded the average LOA ranges between human raters. Coaches and clinicians may consider a system such as AAPS to automate balance testing while maintaining the familiarity of human-based scoring, although scores should not yet be considered interchangeable with those of a human rater.

The Automated Assessment of Postural Stability: Balance Detection Algorithm.

Impaired balance is a common indicator of mild traumatic brain injury, concussion and musculoskeletal injury. Given the clinical relevance of such injuries, especially in military settings, it is paramount to develop more accurate and reliable on-field evaluation tools. This work presents the design and implementation of the automated assessment of postural stability (AAPS) system, for on-field evaluations following concussion. The AAPS is a computer system, based on inexpensive off-the-shelf components and custom software, that aims to automatically and reliably evaluate balance deficits, by replicating a known on-field clinical test, namely, the Balance Error Scoring System (BESS). The AAPS main innovation is its balance error detection algorithm that has been designed to acquire data from a Microsoft Kinect® sensor and convert them into clinically-relevant BESS scores, using the same detection criteria defined by the original BESS test. In order to assess the AAPS balance evaluation capability, a total of 15 healthy subjects (7 male, 8 female) were required to perform the BESS test, while simultaneously being tracked by a Kinect 2.0 sensor and a professional-grade motion capture system (Qualisys AB, Gothenburg, Sweden). High definition videos with BESS trials were scored off-line by three experienced observers for reference scores. AAPS performance was assessed by comparing the AAPS automated scores to those derived by three experienced observers. Our results show that the AAPS error detection algorithm presented here can accurately and precisely detect balance deficits with performance levels that are comparable to those of experienced medical personnel. Specifically, agreement levels between the AAPS algorithm and the human average BESS scores ranging between 87.9% (single-leg on foam) and 99.8% (double-leg on firm ground) were detected. Moreover, statistically significant differences in balance scores were not detected by an ANOVA test with alpha equal to 0.05. Despite some level of disagreement between human and AAPS-generated scores, the use of an automated system yields important advantages over currently available human-based alternatives. These results underscore the value of using the AAPS, that can be quickly deployed in the field and/or in outdoor settings with minimal set-up time. Finally, the AAPS can record multiple error types and their time course with extremely high temporal resolution. These features are not achievable by humans, who cannot keep track of multiple balance errors with such a high resolution. Together, these results suggest that computerized BESS calculation may provide more accurate and consistent measures of balance than those derived from human experts.

Comparative Analysis of Human and Rodent Brain Primary Neuronal Culture Spontaneous Activity Using Micro-Electrode Array Technology.

Electrical activity in embryonic brain tissue has typically been studied using Micro Electrode Array (MEA) technology to make dozens of simultaneous recordings from dissociated neuronal cultures, brain stem cell progenitors, or brain slices from fetal rodents. Although these rodent neuronal primary culture electrical properties are mostly investigated, it has not been yet established to what extent the electrical characteristics of rodent brain neuronal cultures can be generalized to those of humans. A direct comparison of spontaneous spiking activity between rodent and human primary neurons grown under the same in vitro conditions using MEA technology has never been carried out before and will be described in the present study. Human and rodent dissociated fetal brain neuronal cultures were established in-vitro by culturing on a glass grid of 60 planar microelectrodes neurons under identical conditions. Three different cultures of human neurons were produced from tissue sourced from a single aborted fetus (at 16-18 gestational weeks) and these were compared with seven different cultures of embryonic rat neurons (at 18 gestational days) originally isolated from a single rat. The results show that the human and rodent cultures behaved significantly differently. Whereas the rodent cultures demonstrated robust spontaneous activation and network activity after only 10 days, the human cultures required nearly 40 days to achieve a substantially weaker level of electrical function. These results suggest that rat neuron preparations may yield inferences that do not necessarily transfer to humans.

Applications of UBMs and I-vectors in EEG subject verification.

The processing of electroencephalograms (EEGs) is a growing field where mature speech processing techniques are able to rapidly progress development and understanding of the associated neuroscience. I-vectors and Joint Factor Analysis (JFA), along with their foundational universal background models (UBMs) have progressed to a level of understanding that makes them prime for transition to the EEG community. To prove the capability of these techniques they are tested against two contrasting EEG data sets, PhysioNet's EEG Motor Movement/Imagery Dataset and the Temple University Hospital EEG Corpus, to highlight the effectiveness of the techniques with minimal domain knowledge modifications. The initial results, presented as equal error rates as low as 20%, support the development of these techniques as a viable approach to addressing subject verification within and across subjects.

Automated assessment of postural stability system.

The Balance Error Scoring System (BESS) is one of the most commonly used clinical tests to evaluate static postural stability deficits resulting from traumatic brain events and musculoskeletal injury. This test requires a trained operator to visually assess balance and give the subject a performance score based on the number of balance "errors" they committed. Despite being regularly used in several real-world situations, the BESS test is scored by clinician observation and is therefore (a) potentially susceptible to biased and inaccurate test scores and (b) cannot be administered in the absence of a trained provider. The purpose of this research is to develop, calibrate and field test a computerized version of the BESS test using low-cost commodity motion tracking technology. This `Automated Assessment of Postural Stability' (AAPS) system will quantify balance control in field conditions. This research goal is to overcome the main limitations of both the commercially available motion capture systems and the standard BESS test. The AAPS system has been designed to be operated by a minimally trained user and it requires little set-up time with no sensor calibration necessary. These features make the proposed automated system a valuable balance assessment tool to be utilized in the field.

Investigating brain functional evolution and plasticity using microelectrode array technology.

The aim of this work was to investigate long and short-term plasticity responsible for memory formation in dissociated neuronal networks. In order to address this issue, a set of experiments was designed and implemented in which the microelectrode array electrode grid was divided into four quadrants, two of which were chronically stimulated, every two days for one hour with a stimulation paradigm that varied over time. Overall network and quadrant responses were then analyzed to quantify what level of plasticity took place in the network and how this was due to the stimulation interruption. The results demonstrate that there were no spatial differences in the stimulus-evoked activity within quadrants. Furthermore, the implemented stimulation protocol induced depression effects in the neuronal networks as demonstrated by the consistently lower network activity following stimulation sessions. Finally, the analysis demonstrated that the inhibitory effects of the stimulation decreased over time, thus suggesting a habituation phenomenon. These findings are sufficient to conclude that electrical stimulation is an important tool to interact with dissociated neuronal cultures, but localized stimuli are not enough to drive spatial synaptic potentiation or depression. On the contrary, the ability to modulate synaptic temporal plasticity was a feasible task to achieve by chronic network stimulation.

Understanding the temporal evolution of neuronal connectivity in cultured networks using statistical analysis.

BACKGROUND: Micro-Electrode Array (MEA) technology allows researchers to perform long-term non-invasive neuronal recordings in-vitro while actively interacting with the cultured neurons. Despite numerous studies carried out using MEAs, many functional, chemical and structural mechanisms of how dissociated cortical neurons develop and respond to external stimuli are not yet well understood because of the lack of quantitative studies that assess how their development can be affected by chronic external stimulation. METHODS: To investigate network changes, we analyzed a large MEA data set composed of neuron spikes recorded from cultures of dissociated rat cortical neurons plated on MEA dishes with 59 recording electrodes each. Neural network activity was recorded during the first five weeks of each culture's in-vitro development. Stimulation sessions were delivered to each of the 59 electrodes. The False Discovery Rate technique was used to quantify the temporal evolution of dissociated cortical neurons. Our analysis focused on network responses that occurred within selected time window durations, namely 50 ms, 100 ms and 150 ms after stimulus onset. RESULTS: Our results show an evolution in dissociated cortical neuronal network activity over time, that reflects the network synaptic evolution. Furthermore, we tested the sensitivity of our technique to different observation time windows and found that varying the time windows, allows us to capture different dynamics of the observed responses. In addition, when selecting a 150 ms observation time window, our findings indicate that cultures dissociated from the same brain tissue display trends in their temporal evolution that are more similar than those obtained from different brains. CONCLUSION: Our results emphasize that the FDR technique can be implemented without the need to make any particular assumptions about the data a priori. The proposed technique was able to capture the well-known dissociated cortical neuron networks' temporal evolution, that has been previously observed in in-vivo and in intact brain tissue studies. Furthermore, our findings suggest that the time window that is used to capture the stimulus-evoked network responses is a critical parameter to analyze the electrical behavioral and temporal evolution of dissociated cortical neurons.

Fuzzy logic-based spike sorting system.

We present a new method for autonomous real-time spike sorting using a fuzzy logic inference engine. The engine assigns each detected event a 'spikiness index' from zero to one that quantifies the extent to which the detected event is like an ideal spike. Spikes can then be sorted by simply clustering the spikiness indices. The sorter is defined in terms of natural language rules that, once defined, are static and thus require no user intervention or calibration. The sorter was tested using extracellular recordings from three animals: a macaque, an owl monkey and a rat. Simulation results show that the fuzzy sorter performed equal to or better than the benchmark principal component analysis (PCA) based sorter. Importantly, there was no degradation in fuzzy sorter performance when the spikes were not temporally aligned prior to sorting. In contrast, PCA sorter performance dropped by 27% when sorting unaligned spikes. Since the fuzzy sorter is computationally trivial and requires no spike alignment, it is suitable for scaling into large numbers of parallel channels where computational overhead and the need for operator intervention would preclude other spike sorters.

Massively parallel neural signal processing: System-on-Chip design with FPGAs.

This work discusses the architectural layout and performance results of a SoC design for parallel neural signal processing. Architectural framework for scalability and scalar reconfigurability are presented. Architectural requirements for massive parallelism in neural recordings are presented. Prototype architecture with dual processors and multi-level reconfigurable platform design is presented. Functional modules of the platform include real-time spike detector and sorter for several hundreds of neural channels. Performance of the platform for a 300 channel interface is also discussed.

Modular particle filtering FPGA hardware architecture for brain machine interfaces.

As the computational complexities of neural decoding algorithms for brain machine interfaces (BMI) increase, their implementation through sequential processors becomes prohibitive for real-time applications. This work presents the field programmable gate array (FPGA) as an alternative to sequential processors for BMIs. The reprogrammable hardware architecture of the FPGA provides a near optimal platform for performing parallel computations in real-time. The scalability and reconfigurability of the FPGA accommodates diverse sets of neural ensembles and a variety of decoding algorithms. Throughput is significantly increased by decomposing computations into independent parallel hardware modules on the FPGA. This increase in throughput is demonstrated through a parallel hardware implementation of the auxiliary particle filtering signal processing algorithm.

Parallel field programmable gate array particle filtering architecture for real-time neural signal processing.

Both linear and nonlinear estimation algorithms have been successfully applied as neural decoding techniques in brain machine interfaces. Nonlinear approaches such as Bayesian auxiliary particle filters offer improved estimates over other methodologies seemingly at the expense of computational complexity. Real-time implementation of particle filtering algorithms for neural signal processing may become prohibitive when the number of neurons in the observed ensemble becomes large. By implementing a parallel hardware architecture, filter performance can be improved in terms of throughput over conventional sequential processing. Such an architecture is presented here and its FPGA resource utilization is reported.

Reconfigurable embedded system architecture for next-generation Neural Signal Processing.

This work presents a new architectural framework for next generation Neural Signal Processing (NSP). The essential features of the NSP hardware platform include scalability, reconfigurability, real-time processing ability and data storage. This proposed framework has been implemented in a proof-of-concept NSP prototype using an embedded system architecture synthesized in a Xilinx(®)Virtex(®)5 development board. The prototype includes a threshold-based spike detector and a fuzzy logic-based spike sorter.

Estimation of phosphene spatial variability for visual prosthesis applications.

Visual prostheses are the focus of intensive research efforts to restore some measure of useful vision to blind or near-blind patients. The development of such technology is being guided to an extent by tools that simulate prosthesis behavior for healthy sighted subjects in order to assess system requirements and configurations. These simulators, however, typically assume purely deterministic phosphene properties and thus do not apply any variability to phosphene size, intensity, or location. We address this issue by presenting data on phosphene variability measured in a blind human subject fitted with an optic nerve prosthesis. In order to correct for normal limitations in human-pointing accuracy, the experimental conditions were repeated with sighted subjects. We conclude that identical optic nerve stimulations can result in phosphenes whose perceived locations vary by up to 5 degrees of deviation angle and 10 degrees of position angle. The consistency of phosphenes presented in the peripheral field of view can vary by an additional 3 degrees.

Bayesian auxiliary particle filters for estimating neural tuning parameters.

A common challenge in neural engineering is to track the dynamic parameters of neural tuning functions. This work introduces the application of Bayesian auxiliary particle filters for this purpose. Based on Monte-Carlo filtering, Bayesian auxiliary particle filters use adaptive methods to model the prior densities of the state parameters being tracked. The observations used are the neural firing times, modeled here as a Poisson process, and the biological driving signal. The Bayesian auxiliary particle filter was evaluated by simultaneously tracking the three parameters of a hippocampal place cell and compared to a stochastic state point process filter. It is shown that Bayesian auxiliary particle filters are substantially more accurate and robust than alternative methods of state parameter estimation. The effects of time-averaging on parameter estimation are also evaluated.

A single-chip signal processing and telemetry engine for an implantable 96-channel neural data acquisition system.

A fully implantable neural data acquisition system is a key component of a clinically viable cortical brain-machine interface. We present the design and implementation of a single-chip device that serves the processing needs of such a system. Our device processes 96 channels of multi-unit neural data and performs all digital processing necessary for bidirectional wireless communication. The implementation utilizes a single programmable logic device that is responsible for performing data reduction on the 96 channels of neural data, providing a bidirectional telemetry interface to a transceiver and performing command interpretation and system supervision. The device takes as input neural data sampled at 31.25 kHz and outputs a line-encoded serial bitstream containing the information to be transmitted by the transceiver. Data can be output in one of the following four modes: (1) streaming uncompressed data from a single channel, (2) extracted spike waveforms from any subset of the 96 channels, (3) 1 ms bincounts for each channel or (4) streaming data along with extracted spikes from a single channel. The device can output up to 2000 extracted spikes per second with latencies suitable for a brain-machine interface application. This device provides all of the digital processing components required by a fully implantable system.