Wavelet Transforms Significantly Sparsify and Compress Tactile Interactions.

As higher spatiotemporal resolution tactile sensing systems are being developed for prosthetics, wearables, and other biomedical applications, they demand faster sampling rates and generate larger data streams. Sparsifying transformations can alleviate these requirements by enabling compressive sampling and efficient data storage through compression. However, research on the best sparsifying transforms for tactile interactions is lagging. In this work we construct a library of orthogonal and biorthogonal wavelet transforms as sparsifying transforms for tactile interactions and compare their tradeoffs in compression and sparsity. We tested the sparsifying transforms on a publicly available high-density tactile object grasping dataset (548 sensor tactile glove, grasping 26 objects). In addition, we investigated which dimension wavelet transform-1D, 2D, or 3D-would best compress these tactile interactions. Our results show that wavelet transforms are highly efficient at compressing tactile data and can lead to very sparse and compact tactile representations. Additionally, our results show that 1D transforms achieve the sparsest representations, followed by 3D, and lastly 2D. Overall, the best wavelet for coarse approximation is Symlets 4 evaluated temporally which can sparsify to 0.5% sparsity and compress 10-bit tactile data to an average of 0.04 bits per pixel. Future studies can leverage the results of this paper to assist in the compressive sampling of large tactile arrays and free up computational resources for real-time processing on computationally constrained mobile platforms like neuroprosthetics.

Optical coherence tomography using physical domain data compression to achieve MHz A-scan rates.

The three-dimensional volumetric imaging capability of optical coherence tomography (OCT) leads to the generation of large amounts of data, which necessitates high speed acquisition followed by high dimensional image processing and visualization. This signal acquisition and processing pipeline demands high A-scan rates on the front end, which has driven researchers to push A-scan acquisition rates into the MHz regime. To this end, the optical time-stretch approach uses a mode locked laser (MLL) source, dispersion in optical fiber, and a single analog-to-digital converter (ADC) to achieve multi-MHz A-scan rates. While enabling impressive performance this Nyquist sampling approach is ultimately constrained by the sampling rate and bandwidth of the ADC. Additionally such an approach generates massive amounts of data. Here we present a compressed sensing (CS) OCT system that uses a MLL, electro-optic modulation, and optical dispersion to implement data compression in the physical domain and rapidly acquire real-time compressed measurements of the OCT signals. Compression in the analog domain prior to digitization allows for the use of lower bandwidth ADCs, which reduces cost and decreases the required data capacity of the sampling interface. By leveraging a compressive A-scan optical sampling approach and the joint sparsity of C-scan data we demonstrate 14.4-MHz to 144-MHz A-scan acquisition speeds using a sub-Nyquist 1.44 Gsample/sec ADC sampling rate. Furthermore we evaluate the impact of data compression and resulting imaging speed on image quality.

Deep Learning-Based Target Tracking and Classification for Low Quality Videos Using Coded Aperture Cameras.

Compressive sensing has seen many applications in recent years. One type of compressive sensing device is the Pixel-wise Code Exposure (PCE) camera, which has low power consumption and individual control of pixel exposure time. In order to use PCE cameras for practical applications, a time consuming and lossy process is needed to reconstruct the original frames. In this paper, we present a deep learning approach that directly performs target tracking and classification in the compressive measurement domain without any frame reconstruction. In particular, we propose to apply You Only Look Once (YOLO) to detect and track targets in the frames and we propose to apply Residual Network (ResNet) for classification. Extensive simulations using low quality optical and mid-wave infrared (MWIR) videos in the SENSIAC database demonstrated the efficacy of our proposed approach.

Widefield compressive multiphoton microscopy.

A single-pixel compressively sensed architecture is exploited to simultaneously achieve a 10× reduction in acquired data compared with the Nyquist rate, while alleviating limitations faced by conventional widefield temporal focusing microscopes due to scattering of the fluorescence signal. Additionally, we demonstrate an adaptive sampling scheme that further improves the compression and speed of our approach.

Compact all-CMOS spatiotemporal compressive sensing video camera with pixel-wise coded exposure.

We present a low power all-CMOS implementation of temporal compressive sensing with pixel-wise coded exposure. This image sensor can increase video pixel resolution and frame rate simultaneously while reducing data readout speed. Compared to previous architectures, this system modulates pixel exposure at the individual photo-diode electronically without external optical components. Thus, the system provides reduction in size and power compare to previous optics based implementations. The prototype image sensor (127 × 90 pixels) can reconstruct 100 fps videos from coded images sampled at 5 fps. With 20× reduction in readout speed, our CMOS image sensor only consumes 14µW to provide 100 fps videos.

High-speed flow microscopy using compressed sensing with ultrafast laser pulses.

We demonstrate an imaging system employing continuous high-rate photonically-enabled compressed sensing (CHiRP-CS) to enable efficient microscopic imaging of rapidly moving objects with only a few percent of the samples traditionally required for Nyquist sampling. Ultrahigh-rate spectral shaping is achieved through chirp processing of broadband laser pulses and permits ultrafast structured illumination of the object flow. Image reconstructions of high-speed microscopic flows are demonstrated at effective rates up to 39.6 Gigapixel/sec from a 720-MHz sampling rate.

Ultrawideband compressed sensing of arbitrary multi-tone sparse radio frequencies using spectrally encoded ultrafast laser pulses.

We demonstrate a photonic system for pseudorandom sampling of multi-tone sparse radio-frequency (RF) signals in an 11.95-GHz bandwidth using <1% of the measurements required for Nyquist sampling. Pseudorandom binary sequence (PRBS) patterns are modulated onto highly chirped laser pulses, encoding the patterns onto the optical spectra. The pulses are partially compressed to increase the effective sampling rate by 2.07×, modulated with the RF signal, and fully compressed yielding optical integration of the PRBS-RF inner product prior to photodetection. This yields a 266× reduction in the required electronic sampling rate. We introduce a joint-sparsity-based matching-pursuit reconstruction via bagging to achieve accurate recovery of tones at arbitrary frequencies relative to the reconstruction basis.

Detection and Confirmation of Multiple Human Targets Using Pixel-Wise Code Aperture Measurements.

Compressive video measurements can save bandwidth and data storage. However, conventional approaches to target detection require the compressive measurements to be reconstructed before any detectors are applied. This is not only time consuming but also may lose information in the reconstruction process. In this paper, we summarized the application of a recent approach to vehicle detection and classification directly in the compressive measurement domain to human targets. The raw videos were collected using a pixel-wise code exposure (PCE) camera, which condensed multiple frames into one frame. A combination of two deep learning-based algorithms (you only look once (YOLO) and residual network (ResNet)) was used for detection and confirmation. Optical and mid-wave infrared (MWIR) videos from a well-known database (SENSIAC) were used in our experiments. Extensive experiments demonstrated that the proposed framework was feasible for target detection up to 1500 m, but target confirmation needs more research.

Deep Learning to Obtain Simultaneous Image and Segmentation Outputs From a Single Input of Raw Ultrasound Channel Data.

Single plane wave transmissions are promising for automated imaging tasks requiring high ultrasound frame rates over an extended field of view. However, a single plane wave insonification typically produces suboptimal image quality. To address this limitation, we are exploring the use of deep neural networks (DNNs) as an alternative to delay-and-sum (DAS) beamforming. The objectives of this work are to obtain information directly from raw channel data and to simultaneously generate both a segmentation map for automated ultrasound tasks and a corresponding ultrasound B-mode image for interpretable supervision of the automation. We focus on visualizing and segmenting anechoic targets surrounded by tissue and ignoring or deemphasizing less important surrounding structures. DNNs trained with Field II simulations were tested with simulated, experimental phantom, and in vivo data sets that were not included during training. With unfocused input channel data (i.e., prior to the application of receive time delays), simulated, experimental phantom, and in vivo test data sets achieved mean ± standard deviation Dice similarity coefficients of 0.92 ± 0.13, 0.92 ± 0.03, and 0.77 ± 0.07, respectively, and generalized contrast-to-noise ratios (gCNRs) of 0.95 ± 0.08, 0.93 ± 0.08, and 0.75 ± 0.14, respectively. With subaperture beamformed channel data and a modification to the input layer of the DNN architecture to accept these data, the fidelity of image reconstruction increased (e.g., mean gCNR of multiple acquisitions of two in vivo breast cysts ranged 0.89-0.96), but DNN display frame rates were reduced from 395 to 287 Hz. Overall, the DNNs successfully translated feature representations learned from simulated data to phantom and in vivo data, which is promising for this novel approach to simultaneous ultrasound image formation and segmentation.

Multiple description coding with prediction compensation.

A new multiple description coding paradigm is proposed by combining the time-domain lapped transform, block level source splitting, linear prediction, and prediction residual encoding. The method provides effective redundancy control and fully utilizes the source correlation. The joint optimization of all system components and the asymptotic performance analysis are presented. Image coding results demonstrate the superior performance of the proposed method, especially at low redundancies.

Supervised Deep Sparse Coding Networks for Image Classification.

In this paper, we propose a novel deep sparse coding network (SCN) capable of efficiently adapting its own regularization parameters for a given application. The network is trained end-to-end with a supervised task-driven learning algorithm via error backpropagation. During training, the network learns both the dictionaries and the regularization parameters of each sparse coding layer so that the reconstructive dictionaries are smoothly transformed into increasingly discriminative representations. In addition, the adaptive regularization also offers the network more flexibility to adjust sparsity levels. Furthermore, we have devised a sparse coding layer utilizing a 'skinny' dictionary. Integral to computational efficiency, these skinny dictionaries compress the high dimensional sparse codes into lower dimensional structures. The adaptivity and discriminability of our fifteen-layer sparse coding network are demonstrated on five benchmark datasets, namely Cifar-10, Cifar-100, STL-10, SVHN and MNIST, most of which are considered difficult for sparse coding models. Experimental results show that our architecture overwhelmingly outperforms traditional one-layer sparse coding architectures while using much fewer parameters. Moreover, our multilayer architecture exploits the benefits of depth with sparse coding's characteristic ability to operate on smaller datasets. In such data-constrained scenarios, our technique demonstrates highly competitive performance compared to the deep neural networks.

A minimally invasive lens-free computational microendoscope.

Ultra-miniaturized microendoscopes are vital for numerous biomedical applications. Such minimally invasive imagers allow for navigation into hard-to-reach regions and observation of deep brain activity in freely moving animals. Conventional solutions use distal microlenses. However, as lenses become smaller and less invasive, they develop greater aberrations and restricted fields of view. In addition, most of the imagers capable of variable focusing require mechanical actuation of the lens, increasing the distal complexity and weight. Here, we demonstrate a distal lens-free approach to microendoscopy enabled by computational image recovery. Our approach is entirely actuation free and uses a single pseudorandom spatial mask at the distal end of a multicore fiber. Experimentally, this lensless approach increases the space-bandwidth product, i.e., field of view divided by resolution, by threefold over a best-case lens-based system. In addition, the microendoscope demonstrates color resolved imaging and refocusing to 11 distinct depth planes from a single camera frame without any actuated parts.

Robust Short-Lag Spatial Coherence Imaging.

Short-lag spatial coherence (SLSC) imaging displays the spatial coherence between backscattered ultrasound echoes instead of their signal amplitudes and is more robust to noise and clutter artifacts when compared with traditional delay-and-sum (DAS) B-mode imaging. However, SLSC imaging does not consider the content of images formed with different lags, and thus does not exploit the differences in tissue texture at each short-lag value. Our proposed method improves SLSC imaging by weighting the addition of lag values (i.e., M-weighting) and by applying robust principal component analysis (RPCA) to search for a low-dimensional subspace for projecting coherence images created with different lag values. The RPCA-based projections are considered to be denoised versions of the originals that are then weighted and added across lags to yield a final robust SLSC (R-SLSC) image. Our approach was tested on simulation, phantom, and in vivo liver data. Relative to DAS B-mode images, the mean contrast, signal-to-noise ratio (SNR), and contrast-to-noise ratio (CNR) improvements with R-SLSC images are 21.22 dB, 2.54, and 2.36, respectively, when averaged over simulated, phantom, and in vivo data and over all lags considered, which corresponds to mean improvements of 96.4%, 121.2%, and 120.5%, respectively. When compared with SLSC images, the corresponding mean improvements with R-SLSC images were 7.38 dB, 1.52, and 1.30, respectively (i.e., mean improvements of 14.5%, 50.5%, and 43.2%, respectively). Results show great promise for smoothing out the tissue texture of SLSC images and enhancing anechoic or hypoechoic target visibility at higher lag values, which could be useful in clinical tasks such as breast cyst visualization, liver vessel tracking, and obese patient imaging.

An Unsupervised Compressed Sensing Algorithm for Multi-Channel Neural Recording and Spike Sorting.

We propose an unsupervised compressed sensing (CS)-based framework to compress, recover, and cluster neural action potentials. This framework can be easily integrated into high-density multi-electrode neural recording VLSI systems. Embedding spectral clustering and group structures in dictionary learning, we extend the proposed framework to unsupervised spike sorting without prior label information. Additionally, we incorporate group sparsity concepts in the dictionary learning to enable the framework for multi-channel neural recordings, as in tetrodes. To further improve spike sorting success rates in the CS framework, we embed template matching in sparse coding to jointly predict clusters of spikes. Our experimental results demonstrate that the proposed CS-based framework can achieve a high compression ratio (8:1 to 20:1), with a high quality reconstruction performance (>8 dB) and a high spike sorting accuracy (>90%).

Undersampled boundary pre-/postfilters for low bit-rate DCT-based block coders.

It has been well established that critically sampled boundary pre-/postfiltering operators can improve the coding efficiency and mitigate blocking artifacts in traditional discrete cosine transform-based block coders at low bit rates. In these systems, both the prefilter and the postfilter are square matrices. This paper proposes to use undersampled boundary pre- and postfiltering modules, where the pre-/postfilters are rectangular matrices. Specifically, the prefilter is a "fat" matrix, while the postfilter is a "tall" one. In this way, the size of the prefiltered image is smaller than that of the original input image, which leads to improved compression performance and reduced computational complexities at low bit rates. The design and VLSI-friendly implementation of the undersampled pre-/postfilters are derived. Their relations to lapped transforms and filter banks are also presented. Two design examples are also included to demonstrate the validity of the theory. Furthermore, image coding results indicate that the proposed undersampled pre-/postfiltering systems yield excellent and stable performance in low bit-rate image coding.

Wiener filter-based error resilient time-domain lapped transform.

In this paper, the design of the error resilient time-domain lapped transform is formulated as a linear minimal mean-squared error problem. The optimal Wiener solution and several simplifications with different tradeoffs between complexity and performance are developed. We also prove the persymmetric structure of these Wiener filters. The existing mean reconstruction method is proven to be a special case of the proposed framework. Our method also includes as a special case the linear interpolation method used in DCT-based systems when there is no pre/postfiltering and when the quantization noise is ignored. The design criteria in our previous results are scrutinized and improved solutions are obtained. Various design examples and multiple description image coding experiments are reported to demonstrate the performance of the proposed method.

Error resilient pre/post-filtering for DCT-based block coding systems.

Block coding based on the discrete cosine transform (DCT) is very popular in image and video compression. Pre/post-filtering can be attached to a DCT-based block coding system to improve coding efficiency as well as to mitigate blocking artifacts. Previously designed pre/post-filters are optimized to maximize coding efficiency solely. For image and video communication over unreliable channels, those pre/post-filters are sensitive to transmission errors. This paper addresses the problem of designing pre/post-filters which are more error resilient. Reconstruction performance is measured by how low the average reconstruction error is, and how uniformly the reconstruction error is distributed. A family of pre/post-filters is designed to provide desired tradeoffs between coding efficiency and robustness to transmission errors. Experiments show that these filtering operators can achieve superior reconstruction performance without sacrificing much coding performance.

Segmentation of Thalamus from MR images via Task-Driven Dictionary Learning.

Automatic thalamus segmentation is useful to track changes in thalamic volume over time. In this work, we introduce a task-driven dictionary learning framework to find the optimal dictionary given a set of eleven features obtained from T1-weighted MRI and diffusion tensor imaging. In this dictionary learning framework, a linear classifier is designed concurrently to classify voxels as belonging to the thalamus or non-thalamus class. Morphological post-processing is applied to produce the final thalamus segmentation. Due to the uneven size of the training data samples for the non-thalamus and thalamus classes, a non-uniform sampling scheme is proposed to train the classifier to better discriminate between the two classes around the boundary of the thalamus. Experiments are conducted on data collected from 22 subjects with manually delineated ground truth. The experimental results are promising in terms of improvements in the Dice coefficient of the thalamus segmentation over state-of-the-art atlas-based thalamus segmentation algorithms.

A Fully Integrated Wireless Compressed Sensing Neural Signal Acquisition System for Chronic Recording and Brain Machine Interface.

Reliable, multi-channel neural recording is critical to the neuroscience research and clinical treatment. However, most hardware development of fully integrated, multi-channel wireless neural recorders to-date, is still in the proof-of-concept stage. To be ready for practical use, the trade-offs between performance, power consumption, device size, robustness, and compatibility need to be carefully taken into account. This paper presents an optimized wireless compressed sensing neural signal recording system. The system takes advantages of both custom integrated circuits and universal compatible wireless solutions. The proposed system includes an implantable wireless system-on-chip (SoC) and an external wireless relay. The SoC integrates 16-channel low-noise neural amplifiers, programmable filters and gain stages, a SAR ADC, a real-time compressed sensing module, and a near field wireless power and data transmission link. The external relay integrates a 32 bit low-power microcontroller with Bluetooth 4.0 wireless module, a programming interface, and an inductive charging unit. The SoC achieves high signal recording quality with minimized power consumption, while reducing the risk of infection from through-skin connectors. The external relay maximizes the compatibility and programmability. The proposed compressed sensing module is highly configurable, featuring a SNDR of 9.78 dB with a compression ratio of 8×. The SoC has been fabricated in a 180 nm standard CMOS technology, occupying 2.1 mm × 0.6 mm silicon area. A pre-implantable system has been assembled to demonstrate the proposed paradigm. The developed system has been successfully used for long-term wireless neural recording in freely behaving rhesus monkey.

Optimal block boundary pre/postfiltering for wavelet-based image and video compression.

This paper presents a pre/postfiltering framework to reduce the reconstruction errors near block boundaries in wavelet-based image and video compression. Two algorithms are developed to obtain the optimal filter, based on boundary filter bank and polyphase structure, respectively. A low-complexity structure is employed to approximate the optimal solution. Performances of the proposed method in the removal of JPEG 2000 tiling artifact and the jittering artifact of three-dimensional wavelet video coding are reported. Comparisons with other methods demonstrate the advantages of our pre/postfiltering framework.