Analyzing complex single-molecule emission patterns with deep learning.

A fluorescent emitter simultaneously transmits its identity, location, and cellular context through its emission pattern. We developed smNet, a deep neural network for multiplexed single-molecule analysis to retrieve such information with high accuracy. We demonstrate that smNet can extract three-dimensional molecule location, orientation, and wavefront distortion with precision approaching the theoretical limit, and therefore will allow multiplexed measurements through the emission pattern of a single molecule.

Embedded Streaming Deep Neural Networks Accelerator With Applications.

Deep convolutional neural networks (DCNNs) have become a very powerful tool in visual perception. DCNNs have applications in autonomous robots, security systems, mobile phones, and automobiles, where high throughput of the feedforward evaluation phase and power efficiency are important. Because of this increased usage, many field-programmable gate array (FPGA)-based accelerators have been proposed. In this paper, we present an optimized streaming method for DCNNs' hardware accelerator on an embedded platform. The streaming method acts as a compiler, transforming a high-level representation of DCNNs into operation codes to execute applications in a hardware accelerator. The proposed method utilizes maximum computational resources available based on a novel-scheduled routing topology that combines data reuse and data concatenation. It is tested with a hardware accelerator implemented on the Xilinx Kintex-7 XC7K325T FPGA. The system fully explores weight-level and node-level parallelizations of DCNNs and achieves a peak performance of 247 G-ops while consuming less than 4 W of power. We test our system with applications on object classification and object detection in real-world scenarios. Our results indicate high-performance efficiency, outperforming all other presented platforms while running these applications.

Spatially Compact Neural Clusters in the Dorsal Striatum Encode Locomotion Relevant Information.

An influential striatal model postulates that neural activities in the striatal direct and indirect pathways promote and inhibit movement, respectively. Normal behavior requires coordinated activity in the direct pathway to facilitate intended locomotion and indirect pathway to inhibit unwanted locomotion. In this striatal model, neuronal population activity is assumed to encode locomotion relevant information. Here, we propose a novel encoding mechanism for the dorsal striatum. We identified spatially compact neural clusters in both the direct and indirect pathways. Detailed characterization revealed similar cluster organization between the direct and indirect pathways, and cluster activities from both pathways were correlated with mouse locomotion velocities. Using machine-learning algorithms, cluster activities could be used to decode locomotion relevant behavioral states and locomotion velocity. We propose that neural clusters in the dorsal striatum encode locomotion relevant information and that coordinated activities of direct and indirect pathway neural clusters are required for normal striatal controlled behavior. VIDEO ABSTRACT.

Continuous Time Level Crossing Sampling ADC for Bio-Potential Recording Systems.

In this paper we present a fixed window level crossing sampling analog to digital convertor for bio-potential recording sensors. This is the first proposed and fully implemented fixed window level crossing ADC without local DACs and clocks. The circuit is designed to reduce data size, power, and silicon area in future wireless neurophysiological sensor systems. We built a testing system to measure bio-potential signals and used it to evaluate the performance of the circuit. The bio-potential amplifier offers a gain of 53 dB within a bandwidth of 200 Hz-20 kHz. The input-referred rms noise is 2.8 µV. In the asynchronous level crossing ADC, the minimum delta resolution is 4 mV. The input signal frequency of the ADC is up to 5 kHz. The system was fabricated using the AMI 0.5 µm CMOS process. The chip size is 1.5 mm by 1.5 mm. The power consumption of the 4-channel system from a 3.3 V supply is 118.8 µW in the static state and 501.6 µW with a 240 kS/s sampling rate. The conversion efficiency is 1.6 nJ/conversion.

CMOS low current measurement system for biomedical applications.

We present a micro-chip implementation of a low current measurement system for biomedical applications using capacitive feedback that exhibits 190 fA of RMS noise in a 1 kHz bandwidth. The sampling rate is selectable up to 100 kHz. When measuring the amplifier noise with a 10 GΩ resistor and a 47 pF capacitor at the input, typical of cell membrane capacitance in DNA and patch clamp experiments, the measured RMS noise was 2.44 pA on a 50 pA signal in a 10 kHz bandwidth. Two channels were implemented on 630 × 440 µm² using a 0.5-µm 3-metal 2-poly CMOS process. Each channel consumes 1.5 mW of power from a 3.3 V supply. We measured the characteristics of an artificial lipid bilayer similar to the ones used in DNA sequencing experiments via nanopores.

Design constraints for mobile, high-speed fluorescence brain imaging in awake animals.

In this paper we present a fully self-contained imaging instrument (30 mm overall length) that is capable of recording high speed and detect relatively small fluorescent signals (0.1% ΔF/F) from brain tissues potentially containing genetically-encoded sensors or dyes. This device potentially enables the study of neuronal activity in awake and mobile animals during natural behaviors without the stress and suppression of anesthesia and restraint. The device is a fully self-contained illumination system, wide field fluorescence microscope (~ 4.8 mm² FOV-25 um lateral resolution-1.8 × magnification-0.39 NA) and CMOS image sensor (32 × 32). The total weight of the system is 10 g and is capable of imaging up to 900 fps. We present voltage dye RH1692 experiments using the system to study the somatosensory cortex of mice during whisker movements using an air puff.

Efficient feedforward categorization of objects and human postures with address-event image sensors.

This paper proposes an algorithm for feedforward categorization of objects and, in particular, human postures in real-time video sequences from address-event temporal-difference image sensors. The system employs an innovative combination of event based hardware and bio-inspired software architecture. An event-based temporal difference image sensor is used to provide input video sequences, while a software module extracts size and position invariant line features inspired by models of the primate visual cortex. The detected line features are organized into vectorial segments. After feature extraction, a modified line segment Hausdorff distance classifier combined with on-the-fly cluster-based size and position invariant categorization. The system can achieve about 90 percent average success rate in the categorization of human postures, while using only a small number of training samples. Compared to state-of-the-art bio-inspired categorization methods, the proposed algorithm requires less hardware resource, reduces the computation complexity by at least five times, and is an ideal candidate for hardware implementation with event-based circuits.

Comparison between Frame-Constrained Fix-Pixel-Value and Frame-Free Spiking-Dynamic-Pixel ConvNets for Visual Processing.

Most scene segmentation and categorization architectures for the extraction of features in images and patches make exhaustive use of 2D convolution operations for template matching, template search, and denoising. Convolutional Neural Networks (ConvNets) are one example of such architectures that can implement general-purpose bio-inspired vision systems. In standard digital computers 2D convolutions are usually expensive in terms of resource consumption and impose severe limitations for efficient real-time applications. Nevertheless, neuro-cortex inspired solutions, like dedicated Frame-Based or Frame-Free Spiking ConvNet Convolution Processors, are advancing real-time visual processing. These two approaches share the neural inspiration, but each of them solves the problem in different ways. Frame-Based ConvNets process frame by frame video information in a very robust and fast way that requires to use and share the available hardware resources (such as: multipliers, adders). Hardware resources are fixed- and time-multiplexed by fetching data in and out. Thus memory bandwidth and size is important for good performance. On the other hand, spike-based convolution processors are a frame-free alternative that is able to perform convolution of a spike-based source of visual information with very low latency, which makes ideal for very high-speed applications. However, hardware resources need to be available all the time and cannot be time-multiplexed. Thus, hardware should be modular, reconfigurable, and expansible. Hardware implementations in both VLSI custom integrated circuits (digital and analog) and FPGA have been already used to demonstrate the performance of these systems. In this paper we present a comparison study of these two neuro-inspired solutions. A brief description of both systems is presented and also discussions about their differences, pros and cons.

Head-mountable high speed camera for optical neural recording.

We report a head-mountable CMOS camera for recording rapid neuronal activity in freely moving rodents using fluorescent activity reporters. This small, lightweight camera is capable of detecting small changes in light intensity (0.2% ΔI/I) at 500fps. The camera has a resolution of 32×32, sensitivity of 0.62V/lxs, conversion gain of 0.52µV/e(-) and well capacity of 2.1Me(-). The camera, containing intensity offset subtraction circuitry within the imaging chip, is part of a miniaturized epi-fluorescent microscope and represents a first generation, mobile scientific-grade, physiology imaging camera.

Patch-clamp amplifiers on a chip.

We present the first, fully integrated, two-channel implementation of a patch-clamp measurement system. With this "PatchChip" two simultaneous whole-cell recordings can be obtained with rms noise of 8pA in a 10kHz bandwidth. The capacitance and series-resistance of the electrode can be compensated up to 10pF and 100MΩ respectively under computer control. Recordings of hERG and Na(v) 1.7 currents demonstrate the system's capabilities, which are on par with large, commercial patch-clamp instrumentation. By reducing patch-clamp amplifiers to a millimeter size micro-chip, this work paves the way to the realization of massively parallel, high-throughput patch-clamp systems for drug screening and ion-channel research. The PatchChip is implemented in a 0.5µm silicon-on-sapphire process; its size is 3×3mm(2) and the power consumption is 5mW per channel with a 3.3V power supply.

A multichip neuromorphic system for spike-based visual information processing.

We present a multichip, mixed-signal VLSI system for spike-based vision processing. The system consists of an 80 x 60 pixel neuromorphic retina and a 4800 neuron silicon cortex with 4,194,304 synapses. Its functionality is illustrated with experimental data on multiple components of an attention-based hierarchical model of cortical object recognition, including feature coding, salience detection, and foveation. This model exploits arbitrary and reconfigurable connectivity between cells in the multichip architecture, achieved by asynchronously routing neural spike events within and between chips according to a memory-based look-up table. Synaptic parameters, including conductance and reversal potential, are also stored in memory and are used to dynamically configure synapse circuits within the silicon neurons.