Science in the cloud (SIC): A use case in MRI connectomics.

Modern technologies are enabling scientists to collect extraordinary amounts of complex and sophisticated data across a huge range of scales like never before. With this onslaught of data, we can allow the focal point to shift from data collection to data analysis. Unfortunately, lack of standardized sharing mechanisms and practices often make reproducing or extending scientific results very difficult. With the creation of data organization structures and tools that drastically improve code portability, we now have the opportunity to design such a framework for communicating extensible scientific discoveries. Our proposed solution leverages these existing technologies and standards, and provides an accessible and extensible model for reproducible research, called 'science in the cloud' (SIC). Exploiting scientific containers, cloud computing, and cloud data services, we show the capability to compute in the cloud and run a web service that enables intimate interaction with the tools and data presented. We hope this model will inspire the community to produce reproducible and, importantly, extensible results that will enable us to collectively accelerate the rate at which scientific breakthroughs are discovered, replicated, and extended.

An automated images-to-graphs framework for high resolution connectomics.

Reconstructing a map of neuronal connectivity is a critical challenge in contemporary neuroscience. Recent advances in high-throughput serial section electron microscopy (EM) have produced massive 3D image volumes of nanoscale brain tissue for the first time. The resolution of EM allows for individual neurons and their synaptic connections to be directly observed. Recovering neuronal networks by manually tracing each neuronal process at this scale is unmanageable, and therefore researchers are developing automated image processing modules. Thus far, state-of-the-art algorithms focus only on the solution to a particular task (e.g., neuron segmentation or synapse identification). In this manuscript we present the first fully-automated images-to-graphs pipeline (i.e., a pipeline that begins with an imaged volume of neural tissue and produces a brain graph without any human interaction). To evaluate overall performance and select the best parameters and methods, we also develop a metric to assess the quality of the output graphs. We evaluate a set of algorithms and parameters, searching possible operating points to identify the best available brain graph for our assessment metric. Finally, we deploy a reference end-to-end version of the pipeline on a large, publicly available data set. This provides a baseline result and framework for community analysis and future algorithm development and testing. All code and data derivatives have been made publicly available in support of eventually unlocking new biofidelic computational primitives and understanding of neuropathologies.

Fast approximate quadratic programming for graph matching.

Quadratic assignment problems arise in a wide variety of domains, spanning operations research, graph theory, computer vision, and neuroscience, to name a few. The graph matching problem is a special case of the quadratic assignment problem, and graph matching is increasingly important as graph-valued data is becoming more prominent. With the aim of efficiently and accurately matching the large graphs common in big data, we present our graph matching algorithm, the Fast Approximate Quadratic assignment algorithm. We empirically demonstrate that our algorithm is faster and achieves a lower objective value on over 80% of the QAPLIB benchmark library, compared with the previous state-of-the-art. Applying our algorithm to our motivating example, matching C. elegans connectomes (brain-graphs), we find that it efficiently achieves performance.

Demonstration of a semi-autonomous hybrid brain-machine interface using human intracranial EEG, eye tracking, and computer vision to control a robotic upper limb prosthetic.

To increase the ability of brain-machine interfaces (BMIs) to control advanced prostheses such as the modular prosthetic limb (MPL), we are developing a novel system: the Hybrid Augmented Reality Multimodal Operation Neural Integration Environment (HARMONIE). This system utilizes hybrid input, supervisory control, and intelligent robotics to allow users to identify an object (via eye tracking and computer vision) and initiate (via brain-control) a semi-autonomous reach-grasp-and-drop of the object by the MPL. Sequential iterations of HARMONIE were tested in two pilot subjects implanted with electrocorticographic (ECoG) and depth electrodes within motor areas. The subjects performed the complex task in 71.4% (20/28) and 67.7% (21/31) of trials after minimal training. Balanced accuracy for detecting movements was 91.1% and 92.9%, significantly greater than chance accuracies (p < 0.05). After BMI-based initiation, the MPL completed the entire task 100% (one object) and 70% (three objects) of the time. The MPL took approximately 12.2 s for task completion after system improvements implemented for the second subject. Our hybrid-BMI design prevented all but one baseline false positive from initiating the system. The novel approach demonstrated in this proof-of-principle study, using hybrid input, supervisory control, and intelligent robotics, addresses limitations of current BMIs.

Simultaneous neural control of simple reaching and grasping with the modular prosthetic limb using intracranial EEG.

Intracranial electroencephalographic (iEEG) signals from two human subjects were used to achieve simultaneous neural control of reaching and grasping movements with the Johns Hopkins University Applied Physics Lab (JHU/APL) Modular Prosthetic Limb (MPL), a dexterous robotic prosthetic arm. We performed functional mapping of high gamma activity while the subject made reaching and grasping movements to identify task-selective electrodes. Independent, online control of reaching and grasping was then achieved using high gamma activity from a small subset of electrodes with a model trained on short blocks of reaching and grasping with no further adaptation. Classification accuracy did not decline (p < 0.05, one-way ANOVA) over three blocks of testing in either subject. Mean classification accuracy during independently executed overt reach and grasp movements for (Subject 1, Subject 2) were (0.85, 0.81) and (0.80, 0.96), respectively, and during simultaneous execution they were (0.83, 0.88) and (0.58, 0.88), respectively. Our models leveraged knowledge of the subject's individual functional neuroanatomy for reaching and grasping movements, allowing rapid acquisition of control in a time-sensitive clinical setting. We demonstrate the potential feasibility of verifying functionally meaningful iEEG-based control of the MPL prior to chronic implantation, during which additional capabilities of the MPL might be exploited with further training.

Restoring the sense of touch with a prosthetic hand through a brain interface.

Our ability to manipulate objects dexterously relies fundamentally on sensory signals originating from the hand. To restore motor function with upper-limb neuroprostheses requires that somatosensory feedback be provided to the tetraplegic patient or amputee. Given the complexity of state-of-the-art prosthetic limbs and, thus, the huge state space they can traverse, it is desirable to minimize the need for the patient to learn associations between events impinging on the limb and arbitrary sensations. Accordingly, we have developed approaches to intuitively convey sensory information that is critical for object manipulation--information about contact location, pressure, and timing--through intracortical microstimulation of primary somatosensory cortex. In experiments with nonhuman primates, we show that we can elicit percepts that are projected to a localized patch of skin and that track the pressure exerted on the skin. In a real-time application, we demonstrate that animals can perform a tactile discrimination task equally well whether mechanical stimuli are delivered to their native fingers or to a prosthetic one. Finally, we propose that the timing of contact events can be signaled through phasic intracortical microstimulation at the onset and offset of object contact that mimics the ubiquitous on and off responses observed in primary somatosensory cortex to complement slowly varying pressure-related feedback. We anticipate that the proposed biomimetic feedback will considerably increase the dexterity and embodiment of upper-limb neuroprostheses and will constitute an important step in restoring touch to individuals who have lost it.

Graph classification using signal-subgraphs: applications in statistical connectomics.

This manuscript considers the following "graph classification" question: Given a collection of graphs and associated classes, how can one predict the class of a newly observed graph? To address this question, we propose a statistical model for graph/class pairs. This model naturally leads to a set of estimators to identify the class-conditional signal, or "signal-subgraph," defined as the collection of edges that are probabilistically different between the classes. The estimators admit classifiers which are asymptotically optimal and efficient, but which differ by their assumption about the "coherency" of the signal-subgraph (coherency is the extent to which the signal-edges "stick together" around a common subset of vertices). Via simulation, the best estimator is shown to be not just a function of the coherency of the model, but also the number of training samples. These estimators are employed to address a contemporary neuroscience question: Can we classify "connectomes" (brain-graphs) according to sex? The answer is yes, and significantly better than all benchmark algorithms considered. Synthetic data analysis demonstrates that even when the model is correct, given the relatively small number of training samples, the estimated signal-subgraph should be taken with a grain of salt. We conclude by discussing several possible extensions.

The Open Connectome Project Data Cluster: Scalable Analysis and Vision for High-Throughput Neuroscience.

We describe a scalable database cluster for the spatial analysis and annotation of high-throughput brain imaging data, initially for 3-d electron microscopy image stacks, but for time-series and multi-channel data as well. The system was designed primarily for workloads that build connectomes- neural connectivity maps of the brain-using the parallel execution of computer vision algorithms on high-performance compute clusters. These services and open-science data sets are publicly available at openconnecto.me. The system design inherits much from NoSQL scale-out and data-intensive computing architectures. We distribute data to cluster nodes by partitioning a spatial index. We direct I/O to different systems-reads to parallel disk arrays and writes to solid-state storage-to avoid I/O interference and maximize throughput. All programming interfaces are RESTful Web services, which are simple and stateless, improving scalability and usability. We include a performance evaluation of the production system, highlighting the effec-tiveness of spatial data organization.

Fast neuromimetic object recognition using FPGA outperforms GPU implementations.

Recognition of objects in still images has traditionally been regarded as a difficult computational problem. Although modern automated methods for visual object recognition have achieved steadily increasing recognition accuracy, even the most advanced computational vision approaches are unable to obtain performance equal to that of humans. This has led to the creation of many biologically inspired models of visual object recognition, among them the hierarchical model and X (HMAX) model. HMAX is traditionally known to achieve high accuracy in visual object recognition tasks at the expense of significant computational complexity. Increasing complexity, in turn, increases computation time, reducing the number of images that can be processed per unit time. In this paper we describe how the computationally intensive and biologically inspired HMAX model for visual object recognition can be modified for implementation on a commercial field-programmable aate Array, specifically the Xilinx Virtex 6 ML605 evaluation board with XC6VLX240T FPGA. We show that with minor modifications to the traditional HMAX model we can perform recognition on images of size 128 × 128 pixels at a rate of 190 images per second with a less than 1% loss in recognition accuracy in both binary and multiclass visual object recognition tasks.

Are mental properties supervenient on brain properties?

The "mind-brain supervenience" conjecture suggests that all mental properties are derived from the physical properties of the brain. To address the question of whether the mind supervenes on the brain, we frame a supervenience hypothesis in rigorous statistical terms. Specifically, we propose a modified version of supervenience (called ε-supervenience) that is amenable to experimental investigation and statistical analysis. To illustrate this approach, we perform a thought experiment that illustrates how the probabilistic theory of pattern recognition can be used to make a one-sided determination of ε-supervenience. The physical property of the brain employed in this analysis is the graph describing brain connectivity (i.e., the brain-graph or connectome). ε-supervenience allows us to determine whether a particular mental property can be inferred from one's connectome to within any given positive misclassification rate, regardless of the relationship between the two. This may provide further motivation for cross-disciplinary research between neuroscientists and statisticians.

Designing tyrosine-derived polycarbonate polymers for biodegradable regenerative type neural interface capable of neural recording.

Next-generation neuroprosthetic limbs will require a reliable long-term neural interface to residual nerves in the peripheral nervous system (PNS). To this end, we have developed novel biocompatible materials and a fabrication technique to create high site-count microelectrodes for stimulating and recording from regenerated peripheral nerves. Our electrodes are based on a biodegradable tyrosine-derived polycarbonate polymer system with suitable degradation and erosion properties and a fabrication technique for deployment of the polymer in a porous, degradable, regenerative, multiluminal, multielectrode conduit. The in vitro properties of the polymer and the electrode were tuned to retain mechanical strength for over 24 days and to completely degrade and erode within 220 days. The fabrication technique resulted in a multiluminal conduit with at least 10 functioning electrodes maintaining recording site impedance in the single-digit kOhm range. Additionally, in vivo results showed that neural signals could be recorded from these devices starting at four weeks postimplantation and that signal strength increased over time. We conclude that our biodegradable regenerative-type neural interface is a good candidate for chronic high fidelity recording electrodes for integration with regenerated peripheral nerves.

Particle swarm optimization-based feature selection for cognitive state detection.

This manuscript proposes a particle swarm-based feature extraction to monitors brain activity with the goal of identifying correlate cognitive states and intensity of a task. This in turn would allow us to develop a pattern recognition system that will classify such cognitive states and thus to redistribute the workload to other subjects. In this abstract, we present a recognition system that employ multiple features from different domains, a feature selection method using a Particle Swarm Optimization (PSO) search algorithm while the classification is provided using a k-nearest neighbor. Through this approach, we are able to achieve an averaged classification accuracy of 90.25% on held-out, cross-validated data among the eight subjects.

An embedded controller for a 7-degree of freedom prosthetic arm.

We present results from an embedded real-time hardware system capable of decoding surface myoelectric signals (sMES) to control a seven degree of freedom upper limb prosthesis. This is one of the first hardware implementations of sMES decoding algorithms and the most advanced controller to-date. We compare decoding results from the device to simulation results from a real-time PC-based operating system. Performance of both systems is shown to be similar, with decoding accuracy greater than 90% for the floating point software simulation and 80% for fixed point hardware and software implementations.

A real-time virtual integration environment for the design and development of neural prosthetic systems.

We have developed a virtual integration environment (VIE) for the development of neural prosthetic systems. The VIE is a software environment that modularizes the core functions of a neural prosthetic system--receiving signals, decoding signals and controlling a real or simulated device. Complete prosthetic systems can be quickly assembled by linking pre-existing modules together through standard interfaces. Systems can be simulated in real-time, and simulated components can be swapped out for real hardware. This paper is the first of two companion papers that describe the VIE and its use. In this paper, we first describe the architecture of the VIE and review implemented modules. We then describe the use of the VIE for the real-time validation of neural decode algorithms from pre-recorded data, the use of the VIE in closed loop primate experiments and the use of the VIE in the clinic.

The use of a virtual integration environment for the real-time implementation of neural decode algorithms.

We have developed a virtual integration environment (VIE) for the development of neural prosthetic systems. This paper, the second of two companion articles, describes the use of the VIE as a common platform for the implementation of neural decode algorithms. In this paper, a linear filter decode and a recursive Bayesian algorithm are implemented as separate signal analysis modules of the VIE for the real-time decode of end effector trajectory. The process of implementing each algorithm is described and the real-time behavior as well as computational cost for each algorithm is examined. This is the first report of the real-time implementation of the Mixture of Trajectory Models decode [10]. These real-time algorithms can be easily interfaced with pre-existing modules of the VIE to control simulated and real devices.

A multichip neuromorphic system for spike-based visual information processing.

We present a multichip, mixed-signal VLSI system for spike-based vision processing. The system consists of an 80 x 60 pixel neuromorphic retina and a 4800 neuron silicon cortex with 4,194,304 synapses. Its functionality is illustrated with experimental data on multiple components of an attention-based hierarchical model of cortical object recognition, including feature coding, salience detection, and foveation. This model exploits arbitrary and reconfigurable connectivity between cells in the multichip architecture, achieved by asynchronously routing neural spike events within and between chips according to a memory-based look-up table. Synaptic parameters, including conductance and reversal potential, are also stored in memory and are used to dynamically configure synapse circuits within the silicon neurons.

Dynamically reconfigurable silicon array of spiking neurons with conductance-based synapses.

A mixed-signal very large scale integration (VLSI) chip for large scale emulation of spiking neural networks is presented. The chip contains 2400 silicon neurons with fully programmable and reconfigurable synaptic connectivity. Each neuron implements a discrete-time model of a single-compartment cell. The model allows for analog membrane dynamics and an arbitrary number of synaptic connections, each with tunable conductance and reversal potential. The array of silicon neurons functions as an address-event (AE) transceiver, with incoming and outgoing spikes communicated over an asynchronous event-driven digital bus. Address encoding and conflict resolution of spiking events are implemented via a randomized arbitration scheme that ensures balanced servicing of event requests across the array. Routing of events is implemented externally using dynamically programmable random-access memory that stores a postsynaptic address, the conductance, and the reversal potential of each synaptic connection. Here, we describe the silicon neuron circuits, present experimental data characterizing the 3 mm x 3 mm chip fabricated in 0.5-microm complementary metal-oxide-semiconductor (CMOS) technology, and demonstrate its utility by configuring the hardware to emulate a model of attractor dynamics and waves of neural activity during sleep in rat hippocampus.

Dynamic control of the central pattern generator for locomotion.

We show that an ongoing locomotor pattern can be dynamically controlled by applying discrete pulses of electrical stimulation to the central pattern generator (CPG) for locomotion. Data are presented from a pair of experiments on biological (wetware) and electrical (hardware) models of the CPG demonstrating that stimulation causes brief deviations from the CPG's limit cycle activity. The exact characteristics of the deviation depend strongly on the phase of stimulation. Applications of this work are illustrated by examples showing how locomotion can be controlled by using a feedback loop to monitor CPG activity and applying stimuli at the appropriate times to modulate motor output. Eventually, this approach could lead to development of a neuroprosthetic device for restoring locomotion after paralysis.

Phase-dependent effects of spinal cord stimulation on locomotor activity.

This paper examines how electrical stimulation of the spinal cord can modulate the output of the central pattern generator (CPG) for locomotion. Application of discrete current pulses to a single spinal segment was shown to affect multiple parameters of an ongoing locomotor pattern in an in vitro spinal cord. For any given stimulus, the effects on frequency, duration, and symmetry of locomotor output were strongly dependent on the phase at which stimulation was applied within the CPG cycle. Additionally, most stimuli had an immediate impact and evinced no effects on subsequent cycles. The most dramatic changes were seen when stimulation was applied during motor bursting: stimuli applied to the ipsilateral spinal hemicord increased the burst length, while stimuli applied to the contralateral spinal hemicord decreased the burst length. Smaller changes were observed when stimulating during delays between motor bursts. Thus, phasic stimulation was shown to influence the behavior of the CPG and spinal locomotion circuits on a cycle-by-cycle basis. This work represents the first step toward our ultimate goal of developing a neuroprosthetic device to restore locomotion after a severe spinal cord injury.