Deep Learning to Obtain Simultaneous Image and Segmentation Outputs From a Single Input of Raw Ultrasound Channel Data.

Single plane wave transmissions are promising for automated imaging tasks requiring high ultrasound frame rates over an extended field of view. However, a single plane wave insonification typically produces suboptimal image quality. To address this limitation, we are exploring the use of deep neural networks (DNNs) as an alternative to delay-and-sum (DAS) beamforming. The objectives of this work are to obtain information directly from raw channel data and to simultaneously generate both a segmentation map for automated ultrasound tasks and a corresponding ultrasound B-mode image for interpretable supervision of the automation. We focus on visualizing and segmenting anechoic targets surrounded by tissue and ignoring or deemphasizing less important surrounding structures. DNNs trained with Field II simulations were tested with simulated, experimental phantom, and in vivo data sets that were not included during training. With unfocused input channel data (i.e., prior to the application of receive time delays), simulated, experimental phantom, and in vivo test data sets achieved mean ± standard deviation Dice similarity coefficients of 0.92 ± 0.13, 0.92 ± 0.03, and 0.77 ± 0.07, respectively, and generalized contrast-to-noise ratios (gCNRs) of 0.95 ± 0.08, 0.93 ± 0.08, and 0.75 ± 0.14, respectively. With subaperture beamformed channel data and a modification to the input layer of the DNN architecture to accept these data, the fidelity of image reconstruction increased (e.g., mean gCNR of multiple acquisitions of two in vivo breast cysts ranged 0.89-0.96), but DNN display frame rates were reduced from 395 to 287 Hz. Overall, the DNNs successfully translated feature representations learned from simulated data to phantom and in vivo data, which is promising for this novel approach to simultaneous ultrasound image formation and segmentation.

Optical coherence tomography using physical domain data compression to achieve MHz A-scan rates.

The three-dimensional volumetric imaging capability of optical coherence tomography (OCT) leads to the generation of large amounts of data, which necessitates high speed acquisition followed by high dimensional image processing and visualization. This signal acquisition and processing pipeline demands high A-scan rates on the front end, which has driven researchers to push A-scan acquisition rates into the MHz regime. To this end, the optical time-stretch approach uses a mode locked laser (MLL) source, dispersion in optical fiber, and a single analog-to-digital converter (ADC) to achieve multi-MHz A-scan rates. While enabling impressive performance this Nyquist sampling approach is ultimately constrained by the sampling rate and bandwidth of the ADC. Additionally such an approach generates massive amounts of data. Here we present a compressed sensing (CS) OCT system that uses a MLL, electro-optic modulation, and optical dispersion to implement data compression in the physical domain and rapidly acquire real-time compressed measurements of the OCT signals. Compression in the analog domain prior to digitization allows for the use of lower bandwidth ADCs, which reduces cost and decreases the required data capacity of the sampling interface. By leveraging a compressive A-scan optical sampling approach and the joint sparsity of C-scan data we demonstrate 14.4-MHz to 144-MHz A-scan acquisition speeds using a sub-Nyquist 1.44 Gsample/sec ADC sampling rate. Furthermore we evaluate the impact of data compression and resulting imaging speed on image quality.

A minimally invasive lens-free computational microendoscope.

Ultra-miniaturized microendoscopes are vital for numerous biomedical applications. Such minimally invasive imagers allow for navigation into hard-to-reach regions and observation of deep brain activity in freely moving animals. Conventional solutions use distal microlenses. However, as lenses become smaller and less invasive, they develop greater aberrations and restricted fields of view. In addition, most of the imagers capable of variable focusing require mechanical actuation of the lens, increasing the distal complexity and weight. Here, we demonstrate a distal lens-free approach to microendoscopy enabled by computational image recovery. Our approach is entirely actuation free and uses a single pseudorandom spatial mask at the distal end of a multicore fiber. Experimentally, this lensless approach increases the space-bandwidth product, i.e., field of view divided by resolution, by threefold over a best-case lens-based system. In addition, the microendoscope demonstrates color resolved imaging and refocusing to 11 distinct depth planes from a single camera frame without any actuated parts.

Supervised Deep Sparse Coding Networks for Image Classification.

In this paper, we propose a novel deep sparse coding network (SCN) capable of efficiently adapting its own regularization parameters for a given application. The network is trained end-to-end with a supervised task-driven learning algorithm via error backpropagation. During training, the network learns both the dictionaries and the regularization parameters of each sparse coding layer so that the reconstructive dictionaries are smoothly transformed into increasingly discriminative representations. In addition, the adaptive regularization also offers the network more flexibility to adjust sparsity levels. Furthermore, we have devised a sparse coding layer utilizing a 'skinny' dictionary. Integral to computational efficiency, these skinny dictionaries compress the high dimensional sparse codes into lower dimensional structures. The adaptivity and discriminability of our fifteen-layer sparse coding network are demonstrated on five benchmark datasets, namely Cifar-10, Cifar-100, STL-10, SVHN and MNIST, most of which are considered difficult for sparse coding models. Experimental results show that our architecture overwhelmingly outperforms traditional one-layer sparse coding architectures while using much fewer parameters. Moreover, our multilayer architecture exploits the benefits of depth with sparse coding's characteristic ability to operate on smaller datasets. In such data-constrained scenarios, our technique demonstrates highly competitive performance compared to the deep neural networks.

Widefield compressive multiphoton microscopy.

A single-pixel compressively sensed architecture is exploited to simultaneously achieve a 10× reduction in acquired data compared with the Nyquist rate, while alleviating limitations faced by conventional widefield temporal focusing microscopes due to scattering of the fluorescence signal. Additionally, we demonstrate an adaptive sampling scheme that further improves the compression and speed of our approach.

An Unsupervised Compressed Sensing Algorithm for Multi-Channel Neural Recording and Spike Sorting.

We propose an unsupervised compressed sensing (CS)-based framework to compress, recover, and cluster neural action potentials. This framework can be easily integrated into high-density multi-electrode neural recording VLSI systems. Embedding spectral clustering and group structures in dictionary learning, we extend the proposed framework to unsupervised spike sorting without prior label information. Additionally, we incorporate group sparsity concepts in the dictionary learning to enable the framework for multi-channel neural recordings, as in tetrodes. To further improve spike sorting success rates in the CS framework, we embed template matching in sparse coding to jointly predict clusters of spikes. Our experimental results demonstrate that the proposed CS-based framework can achieve a high compression ratio (8:1 to 20:1), with a high quality reconstruction performance (>8 dB) and a high spike sorting accuracy (>90%).

Segmentation of Thalamus from MR images via Task-Driven Dictionary Learning.

Automatic thalamus segmentation is useful to track changes in thalamic volume over time. In this work, we introduce a task-driven dictionary learning framework to find the optimal dictionary given a set of eleven features obtained from T1-weighted MRI and diffusion tensor imaging. In this dictionary learning framework, a linear classifier is designed concurrently to classify voxels as belonging to the thalamus or non-thalamus class. Morphological post-processing is applied to produce the final thalamus segmentation. Due to the uneven size of the training data samples for the non-thalamus and thalamus classes, a non-uniform sampling scheme is proposed to train the classifier to better discriminate between the two classes around the boundary of the thalamus. Experiments are conducted on data collected from 22 subjects with manually delineated ground truth. The experimental results are promising in terms of improvements in the Dice coefficient of the thalamus segmentation over state-of-the-art atlas-based thalamus segmentation algorithms.

A Fully Integrated Wireless Compressed Sensing Neural Signal Acquisition System for Chronic Recording and Brain Machine Interface.

Reliable, multi-channel neural recording is critical to the neuroscience research and clinical treatment. However, most hardware development of fully integrated, multi-channel wireless neural recorders to-date, is still in the proof-of-concept stage. To be ready for practical use, the trade-offs between performance, power consumption, device size, robustness, and compatibility need to be carefully taken into account. This paper presents an optimized wireless compressed sensing neural signal recording system. The system takes advantages of both custom integrated circuits and universal compatible wireless solutions. The proposed system includes an implantable wireless system-on-chip (SoC) and an external wireless relay. The SoC integrates 16-channel low-noise neural amplifiers, programmable filters and gain stages, a SAR ADC, a real-time compressed sensing module, and a near field wireless power and data transmission link. The external relay integrates a 32 bit low-power microcontroller with Bluetooth 4.0 wireless module, a programming interface, and an inductive charging unit. The SoC achieves high signal recording quality with minimized power consumption, while reducing the risk of infection from through-skin connectors. The external relay maximizes the compatibility and programmability. The proposed compressed sensing module is highly configurable, featuring a SNDR of 9.78 dB with a compression ratio of 8×. The SoC has been fabricated in a 180 nm standard CMOS technology, occupying 2.1 mm × 0.6 mm silicon area. A pre-implantable system has been assembled to demonstrate the proposed paradigm. The developed system has been successfully used for long-term wireless neural recording in freely behaving rhesus monkey.

Ultrawideband compressed sensing of arbitrary multi-tone sparse radio frequencies using spectrally encoded ultrafast laser pulses.

We demonstrate a photonic system for pseudorandom sampling of multi-tone sparse radio-frequency (RF) signals in an 11.95-GHz bandwidth using <1% of the measurements required for Nyquist sampling. Pseudorandom binary sequence (PRBS) patterns are modulated onto highly chirped laser pulses, encoding the patterns onto the optical spectra. The pulses are partially compressed to increase the effective sampling rate by 2.07×, modulated with the RF signal, and fully compressed yielding optical integration of the PRBS-RF inner product prior to photodetection. This yields a 266× reduction in the required electronic sampling rate. We introduce a joint-sparsity-based matching-pursuit reconstruction via bagging to achieve accurate recovery of tones at arbitrary frequencies relative to the reconstruction basis.

High-speed flow microscopy using compressed sensing with ultrafast laser pulses.

We demonstrate an imaging system employing continuous high-rate photonically-enabled compressed sensing (CHiRP-CS) to enable efficient microscopic imaging of rapidly moving objects with only a few percent of the samples traditionally required for Nyquist sampling. Ultrahigh-rate spectral shaping is achieved through chirp processing of broadband laser pulses and permits ultrafast structured illumination of the object flow. Image reconstructions of high-speed microscopic flows are demonstrated at effective rates up to 39.6 Gigapixel/sec from a 720-MHz sampling rate.

A closed-loop compressive-sensing-based neural recording system.

OBJECTIVE: This paper describes a low power closed-loop compressive sensing (CS) based neural recording system. This system provides an efficient method to reduce data transmission bandwidth for implantable neural recording devices. By doing so, this technique reduces a majority of system power consumption which is dissipated at data readout interface. The design of the system is scalable and is a viable option for large scale integration of electrodes or recording sites onto a single device. APPROACH: The entire system consists of an application-specific integrated circuit (ASIC) with 4 recording readout channels with CS circuits, a real time off-chip CS recovery block and a recovery quality evaluation block that provides a closed feedback to adaptively adjust compression rate. Since CS performance is strongly signal dependent, the ASIC has been tested in vivo and with standard public neural databases. MAIN RESULTS: Implemented using efficient digital circuit, this system is able to achieve >10 times data compression on the entire neural spike band (500-6KHz) while consuming only 0.83uW (0.53 V voltage supply) additional digital power per electrode. When only the spikes are desired, the system is able to further compress the detected spikes by around 16 times. Unlike other similar systems, the characteristic spikes and inter-spike data can both be recovered which guarantes a >95% spike classification success rate. The compression circuit occupied 0.11mm(2)/electrode in a 180nm CMOS process. The complete signal processing circuit consumes <16uW/electrode. SIGNIFICANCE: Power and area efficiency demonstrated by the system make it an ideal candidate for integration into large recording arrays containing thousands of electrode. Closed-loop recording and reconstruction performance evaluation further improves the robustness of the compression method, thus making the system more practical for long term recording.

Structured sparse priors for image classification.

Model-based compressive sensing (CS) exploits the structure inherent in sparse signals for the design of better signal recovery algorithms. This information about structure is often captured in the form of a prior on the sparse coefficients, with the Laplacian being the most common such choice (leading to l1 -norm minimization). Recent work has exploited the discriminative capability of sparse representations for image classification by employing class-specific dictionaries in the CS framework. Our contribution is a logical extension of these ideas into structured sparsity for classification. We introduce the notion of discriminative class-specific priors in conjunction with class specific dictionaries, specifically the spike-and-slab prior widely applied in Bayesian sparse regression. Significantly, the proposed framework takes the burden off the demand for abundant training image samples necessary for the success of sparsity-based classification schemes. We demonstrate this practical benefit of our approach in important applications, such as face recognition and object categorization.

Interpretable exemplar-based shape classification using constrained sparse linear models.

Many types of diseases manifest themselves as observable changes in the shape of the affected organs. Using shape classification, we can look for signs of disease and discover relationships between diseases. We formulate the problem of shape classification in a holistic framework that utilizes a lossless scalar field representation and a non-parametric classification based on sparse recovery. This framework generalizes over certain classes of unseen shapes while using the full information of the shape, bypassing feature extraction. The output of the method is the class whose combination of exemplars most closely approximates the shape, and furthermore, the algorithm returns the most similar exemplars along with their similarity to the shape, which makes the result simple to interpret. Our results show that the method offers accurate classification between three cerebellar diseases and controls in a database of cerebellar ataxia patients. For reproducible comparison, promising results are presented on publicly available 2D datasets, including the ETH-80 dataset where the method achieves 88.4% classification accuracy.

Energy-efficient multi-mode compressed sensing system for implantable neural recordings.

Widely utilized in the field of Neuroscience, implantable neural recording devices could capture neuron activities with an acquisition rate on the order of megabytes per second. In order to efficiently transmit neural signals through wireless channels, these devices require compression methods that reduce power consumption. Although recent Compressed Sensing (CS) approaches have successfully demonstrated their power, their full potential is yet to be explored. Built upon our previous on-chip CS implementation, we propose an energy efficient multi-mode CS framework that focuses on improving the off-chip components, including (i) a two-stage sensing strategy, (ii) a sparsifying dictionary directly using data, (iii) enhanced compression performance from Full Signal CS mode and Spike Restoration mode to Spike CS + Restoration mode and; (iv) extension of our framework to the Tetrode CS recovery using joint sparsity. This new framework achieves energy efficiency, implementation simplicity and system flexibility simultaneously. Extensive experiments are performed on simulation and real datasets. For our Spike CS + Restoration mode, we achieve a compression ratio of 6% with a reconstruction SNDR > 10 dB and a classification accuracy > 95% for synthetic datasets. For real datasets, we get a 10% compression ratio with  âˆ¼  10 dB for Spike CS + Restoration mode.

An efficient and compact compressed sensing microsystem for implantable neural recordings.

Multi-Electrode Arrays (MEA) have been widely used in neuroscience experiments. However, the reduction of their wireless transmission power consumption remains a major challenge. To resolve this challenge, an efficient on-chip signal compression method is essential. In this paper, we first introduce a signal-dependent Compressed Sensing (CS) approach that outperforms previous works in terms of compression rate and reconstruction quality. Using a publicly available database, our simulation results show that the proposed system is able to achieve a signal compression rate of 8 to 16 while guaranteeing almost perfect spike classification rate. Finally, we demonstrate power consumption measurements and area estimation of a test structure implemented using TSMC 0.18 µm process. We estimate the proposed system would occupy an area of around 200 µm ×300 µm per recording channel, and consumes 0.27 µ W operating at 20 KHz .

Multiple description coding with prediction compensation.

A new multiple description coding paradigm is proposed by combining the time-domain lapped transform, block level source splitting, linear prediction, and prediction residual encoding. The method provides effective redundancy control and fully utilizes the source correlation. The joint optimization of all system components and the asymptotic performance analysis are presented. Image coding results demonstrate the superior performance of the proposed method, especially at low redundancies.

Undersampled boundary pre-/postfilters for low bit-rate DCT-based block coders.

It has been well established that critically sampled boundary pre-/postfiltering operators can improve the coding efficiency and mitigate blocking artifacts in traditional discrete cosine transform-based block coders at low bit rates. In these systems, both the prefilter and the postfilter are square matrices. This paper proposes to use undersampled boundary pre- and postfiltering modules, where the pre-/postfilters are rectangular matrices. Specifically, the prefilter is a "fat" matrix, while the postfilter is a "tall" one. In this way, the size of the prefiltered image is smaller than that of the original input image, which leads to improved compression performance and reduced computational complexities at low bit rates. The design and VLSI-friendly implementation of the undersampled pre-/postfilters are derived. Their relations to lapped transforms and filter banks are also presented. Two design examples are also included to demonstrate the validity of the theory. Furthermore, image coding results indicate that the proposed undersampled pre-/postfiltering systems yield excellent and stable performance in low bit-rate image coding.

Wiener filter-based error resilient time-domain lapped transform.

In this paper, the design of the error resilient time-domain lapped transform is formulated as a linear minimal mean-squared error problem. The optimal Wiener solution and several simplifications with different tradeoffs between complexity and performance are developed. We also prove the persymmetric structure of these Wiener filters. The existing mean reconstruction method is proven to be a special case of the proposed framework. Our method also includes as a special case the linear interpolation method used in DCT-based systems when there is no pre/postfiltering and when the quantization noise is ignored. The design criteria in our previous results are scrutinized and improved solutions are obtained. Various design examples and multiple description image coding experiments are reported to demonstrate the performance of the proposed method.

Error resilient pre/post-filtering for DCT-based block coding systems.

Block coding based on the discrete cosine transform (DCT) is very popular in image and video compression. Pre/post-filtering can be attached to a DCT-based block coding system to improve coding efficiency as well as to mitigate blocking artifacts. Previously designed pre/post-filters are optimized to maximize coding efficiency solely. For image and video communication over unreliable channels, those pre/post-filters are sensitive to transmission errors. This paper addresses the problem of designing pre/post-filters which are more error resilient. Reconstruction performance is measured by how low the average reconstruction error is, and how uniformly the reconstruction error is distributed. A family of pre/post-filters is designed to provide desired tradeoffs between coding efficiency and robustness to transmission errors. Experiments show that these filtering operators can achieve superior reconstruction performance without sacrificing much coding performance.

Optimal block boundary pre/postfiltering for wavelet-based image and video compression.

This paper presents a pre/postfiltering framework to reduce the reconstruction errors near block boundaries in wavelet-based image and video compression. Two algorithms are developed to obtain the optimal filter, based on boundary filter bank and polyphase structure, respectively. A low-complexity structure is employed to approximate the optimal solution. Performances of the proposed method in the removal of JPEG 2000 tiling artifact and the jittering artifact of three-dimensional wavelet video coding are reported. Comparisons with other methods demonstrate the advantages of our pre/postfiltering framework.