A Computationally Efficient Visual Saliency Algorithm Suitable for an Analog CMOS Implementation.

Computer vision algorithms are often limited in their application by the large amount of data that must be processed. Mammalian vision systems mitigate this high bandwidth requirement by prioritizing certain regions of the visual field with neural circuits that select the most salient regions. This work introduces a novel and computationally efficient visual saliency algorithm for performing this neuromorphic attention-based data reduction. The proposed algorithm has the added advantage that it is compatible with an analog CMOS design while still achieving comparable performance to existing state-of-the-art saliency algorithms. This compatibility allows for direct integration with the analog-to-digital conversion circuitry present in CMOS image sensors. This integration leads to power savings in the converter by quantizing only the salient pixels. Further system-level power savings are gained by reducing the amount of data that must be transmitted and processed in the digital domain. The analog CMOS compatible formulation relies on a pulse width (i.e., time mode) encoding of the pixel data that is compatible with pulse-mode imagers and slope based converters often used in imager designs. This letter begins by discussing this time-mode encoding for implementing neuromorphic architectures. Next, the proposed algorithm is derived. Hardware-oriented optimizations and modifications to this algorithm are proposed and discussed. Next, a metric for quantifying saliency accuracy is proposed, and simulation results of this metric are presented. Finally, an analog synthesis approach for a time-mode architecture is outlined, and postsynthesis transistor-level simulations that demonstrate functionality of an implementation in a modern CMOS process are discussed.

Light-controlled modulation of gene expression by chemical optoepigenetic probes.

Epigenetic gene regulation is a dynamic process orchestrated by chromatin-modifying enzymes. Many of these master regulators exert their function through covalent modification of DNA and histone proteins. Aberrant epigenetic processes have been implicated in the pathophysiology of multiple human diseases. Small-molecule inhibitors have been essential to advancing our understanding of the underlying molecular mechanisms of epigenetic processes. However, the resolution offered by small molecules is often insufficient to manipulate epigenetic processes with high spatiotemporal control. Here we present a generalizable approach, referred to as 'chemo-optical modulation of epigenetically regulated transcription' (COMET), enabling high-resolution, optical control of epigenetic mechanisms based on photochromic inhibitors of human histone deacetylases using visible light. COMET probes may be translated into new therapeutic strategies for diseases where conditional and selective epigenome modulation is required.

CIDER: Enhancing the Performance of Computational Eyeglasses.

The human eye offers a fascinating window into an individual's health, cognitive attention, and decision making, but we lack the ability to continually measure these parameters in the natural environment. We demonstrate CIDER, a system that operates in a highly optimized low-power mode under indoor settings by using a fast Search-Refine controller to track the eye, but detects when the environment switches to more challenging outdoor sunlight and switches models to operate robustly under this condition. Our design is holistic and tackles a) power consumption in digitizing pixels, estimating pupillary parameters, and illuminating the eye via near-infrared and b) error in estimating pupil center and pupil dilation. We demonstrate that CIDER can estimate pupil center with error less than two pixels (0.6°), and pupil diameter with error of one pixel (0.22mm). Our end-to-end results show that we can operate at power levels of roughly 7mW at a 4Hz eye tracking rate, or roughly 32mW at rates upwards of 250Hz.

CIDER: Enabling Robustness-Power Tradeoffs on a Computational Eyeglass.

The human eye offers a fascinating window into an individual's health, cognitive attention, and decision making, but we lack the ability to continually measure these parameters in the natural environment. The challenges lie in: a) handling the complexity of continuous high-rate sensing from a camera and processing the image stream to estimate eye parameters, and b) dealing with the wide variability in illumination conditions in the natural environment. This paper explores the power-robustness tradeoffs inherent in the design of a wearable eye tracker, and proposes a novel staged architecture that enables graceful adaptation across the spectrum of real-world illumination. We propose CIDER, a system that operates in a highly optimized low-power mode under indoor settings by using a fast Search-Refine controller to track the eye, but detects when the environment switches to more challenging outdoor sunlight and switches models to operate robustly under this condition. Our design is holistic and tackles a) power consumption in digitizing pixels, estimating pupillary parameters, and illuminating the eye via near-infrared, b) error in estimating pupil center and pupil dilation, and c) model training procedures that involve zero effort from a user. We demonstrate that CIDER can estimate pupil center with error less than two pixels (0.6°), and pupil diameter with error of one pixel (0.22mm). Our end-to-end results show that we can operate at power levels of roughly 7mW at a 4Hz eye tracking rate, or roughly 32mW at rates upwards of 250Hz.

A portable time-domain LED fluorimeter for nanosecond fluorescence lifetime measurements.

Fluorescence lifetime measurements are becoming increasingly important in chemical and biological research. Time-domain lifetime measurements offer fluorescence multiplexing and improved handling of interferers compared with the frequency-domain technique. In this paper, an all solid-state, filterless, and highly portable light-emitting-diode based time-domain fluorimeter (LED TDF) is reported for the measurement of nanosecond fluorescence lifetimes. LED based excitation provides more wavelengths options compared to laser diode based excitation, but the excitation is less effective due to the uncollimated beam, less optical power, and longer latency in state transition. Pulse triggering and pre-bias techniques were implemented in our LED TDF to improve the peak optical power to over 100 mW. The proposed pulsing circuit achieved an excitation light fall time of less than 2 ns. Electrical resetting technique realized a time-gated photo-detector to remove the interference of the excitation light with fluorescence. These techniques allow the LED fluorimeter to accurately measure the fluorescence lifetime of fluorescein down to concentration of 0.5 µM. In addition, all filters required in traditional instruments are eliminated for the non-attenuated excitation/emission light power. These achievements make the reported device attractive to biochemical laboratories seeking for highly portable lifetime detection devices for developing sensors based on fluorescence lifetime changes. The device was initially validated by measuring the lifetimes of three commercial fluorophores and comparing them with reported lifetime data. It was subsequently used to characterize a ZnSe quantum dot based DNA sensor.

Lensless fluorescence imaging with height calculation.

Lensless fluorescence imaging (LFI) is the imaging of fluorescence from cells or microspheres using an image sensor with no external lenses or filters. The simplicity of the hardware makes it well suited to replace fluorescence microscopes and flow cytometers in lab-on-a-chip applications, but the images captured by LFI are highly dependent on the distance between the sample and the sensor. This work demonstrates that not only can samples be accurately detected across a range of sample-sensor separations using LFI, but also that the separation can be accurately estimated based on the shape of fluorescence in the LFI image. First, a theoretical model that accurately predicts LFI images of microspheres is presented. Then, the experimental results are compared to the model and an image processing method for accurately predicting sample-sensor separation from LFI images is presented. Finally, LFI images of microspheres and cells passing through a microfluidic channel are presented.

iShadow: Design of a Wearable, Real-Time Mobile Gaze Tracker.

Continuous, real-time tracking of eye gaze is valuable in a variety of scenarios including hands-free interaction with the physical world, detection of unsafe behaviors, leveraging visual context for advertising, life logging, and others. While eye tracking is commonly used in clinical trials and user studies, it has not bridged the gap to everyday consumer use. The challenge is that a real-time eye tracker is a power-hungry and computation-intensive device which requires continuous sensing of the eye using an imager running at many tens of frames per second, and continuous processing of the image stream using sophisticated gaze estimation algorithms. Our key contribution is the design of an eye tracker that dramatically reduces the sensing and computation needs for eye tracking, thereby achieving orders of magnitude reductions in power consumption and form-factor. The key idea is that eye images are extremely redundant, therefore we can estimate gaze by using a small subset of carefully chosen pixels per frame. We instantiate this idea in a prototype hardware platform equipped with a low-power image sensor that provides random access to pixel values, a low-power ARM Cortex M3 microcontroller, and a bluetooth radio to communicate with a mobile phone. The sparse pixel-based gaze estimation algorithm is a multi-layer neural network learned using a state-of-the-art sparsity-inducing regularization function that minimizes the gaze prediction error while simultaneously minimizing the number of pixels used. Our results show that we can operate at roughly 70mW of power, while continuously estimating eye gaze at the rate of 30 Hz with errors of roughly 3 degrees.

High-throughput imaging of adult fluorescent zebrafish with an LED fluorescence macroscope.

Zebrafish are a useful vertebrate model for the study of development, behavior, disease and cancer. A major advantage of zebrafish is that large numbers of animals can be economically used for experimentation; however, high-throughput methods for imaging live adult zebrafish had not been developed. Here, we describe protocols for building a light-emitting diode (LED) fluorescence macroscope and for using it to simultaneously image up to 30 adult animals that transgenically express a fluorescent protein, are transplanted with fluorescently labeled tumor cells or are tagged with fluorescent elastomers. These protocols show that the LED fluorescence macroscope is capable of distinguishing five fluorescent proteins and can image unanesthetized swimming adult zebrafish in multiple fluorescent channels simultaneously. The macroscope can be built and used for imaging within 1 day, whereas creating fluorescently labeled adult zebrafish requires 1 hour to several months, depending on the method chosen. The LED fluorescence macroscope provides a low-cost, high-throughput method to rapidly screen adult fluorescent zebrafish and it will be useful for imaging transgenic animals, screening for tumor engraftment, and tagging individual fish for long-term analysis.

Upconverting luminescent nanomaterials: application to in vivo bioimaging.

In this report, the development of multi-channel anti-Stokes luminescent Y2O3 nanoparticles for application to in vivo upconversion imaging is detailed.

Quantitative Measurement of Protease-Activity with Correction of Probe Delivery and Tissue Absorption Effects.

Proteases play important roles in a variety of pathologies from heart disease to cancer. Quantitative measurement of protease activity is possible using a novel spectrally matched dual fluorophore probe and a small animal lifetime imager. The recorded fluorescence from an activatable fluorophore, one that changes its fluorescent amplitude after biological target interaction, is also influenced by other factors including imaging probe delivery and optical tissue absorption of excitation and emission light.Fluorescence from a second spectrally matched constant (non-activatable) fluorophore on each nanoparticle platform can be used to correct for both probe delivery and tissue absorption. The fluorescence from each fluorophore is separated using fluorescence lifetime methods.

Design and demonstration of a small-animal up-conversion imager.

This first small-animal up-conversion imager (SAUCI) was developed and used for in-vivo imaging of up-converting nanoparticles (UCNs.) Unlike traditional fluorophores, UCNs absorb multiple lower-energy photons and emit a single higher-energy photon. This unique physical process makes it possible to image deeper into tissue with lower background signals. In vivo imaging of particle accumulation in the liver was demonstrated following intravenous injection of particles.

Development of a time domain fluorimeter for fluorescent lifetime multiplexing analysis.

We show that a portable, inexpensive USB-powered time domain fluorimeter (TDF) and analysis scheme were developed for use in evaluating a new class of fluorescent lifetime multiplexed dyes. Fluorescent proteins, organic dyes, and quantum dots allow the labeling of more and more individual features within biological systems, but the wide absorption and emission spectra of these fluorophores limit the number of distinct processes which may be simultaneously imaged using spectral separation alone. By additionally separating reporters in a second dimension, fluorescent lifetime multiplexing provides a means to multiply the number of available imaging channels.

An ultra-low-power programmable analog bionic ear processor.

We report a programmable analog bionic ear (cochlear implant) processor in a 1.5-microm BiCMOS technology with a power consumption of 211 microW and 77-dB dynamic range of operation. The 9.58 mm x 9.23 mm processor chip runs on a 2.8 V supply and has a power consumption that is lower than state-of-the-art analog-to-digital (A/D)-then-DSP designs by a factor of 25. It is suitable for use in fully implanted cochlear-implant systems of the future which require decades of operation on a 100-mAh rechargeable battery with a finite number of charge-discharge cycles. It may also be used as an ultra-low-power spectrum-analysis front end in portable speech-recognition systems. The power consumption of the processor includes the 100 microW power consumption of a JFET-buffered electret microphone and an associated on-chip microphone front end. An automatic gain control circuit compresses the 77-dB input dynamic range into a narrower internal dynamic range (IDR) of 57 dB at which each of the 16 spectral channels of the processor operate. The output bits of the processor are scanned and reported off chip in a format suitable for continuous-interleaved-sampling stimulation of electrodes. Power-supply-immune biasing circuits ensure robust operation of the processor in the high-RF-noise environment typical of cochlear implant systems.