Neuronavigation using three-dimensional proton magnetic resonance spectroscopy data.

BACKGROUND: Applications in clinical medicine can benefit from fusion of spectroscopy data with anatomical imagery. For example, new 3-dimensional (3D) spectroscopy techniques allow for improved correlation of metabolite profiles with specific regions of interest in anatomical tumor images, which can be useful in characterizing and treating heterogeneous tumors that appear structurally homogeneous. OBJECTIVES: We sought to develop a clinical workflow and uniquely capable custom software tool to integrate advanced 3-tesla 3D proton magnetic resonance spectroscopic imaging ((1)H-MRSI) into industry standard image-guided neuronavigation systems, especially for use in brain tumor surgery. METHODS: (1)H-MRSI spectra from preoperative scanning on 15 patients with recurrent or newly diagnosed meningiomas were processed and analyzed, and specific voxels were selected based on their chemical contents. 3D neuronavigation overlays were then generated and applied to anatomical image data in the operating room. The proposed 3D methods fully account for scanner calibration and comprise tools that we have now made publicly available. RESULTS: The new methods were quantitatively validated through a phantom study and applied successfully to mitigate biopsy uncertainty in a clinical study of meningiomas. CONCLUSIONS: The proposed methods improve upon the current state of the art in neuronavigation through the use of detailed 3D (1)H-MRSI data. Specifically, 3D MRSI-based overlays provide comprehensive, quantitative visual cues and location information during neurosurgery, enabling a progressive new form of online spectroscopy-guided neuronavigation.

Reduced-complexity Convolutional Neural Network in the compressed domain.

Deep neural networks have achieved outstanding performance in computer vision tasks. Convolutional Neural Networks (CNNs) typically operate in the spatial domain with raw images, but in practice, images are usually stored and transmitted in their compressed representation where JPEG is one of the most widely used encoder. Also, these networks are computationally intensive and slow. This paper proposes performing the learning and inference processes in the compressed domain in order to reduce the computational complexity and improve the speed of popular CNNs. For this purpose, a novel graph-based frequency channel selection method is proposed to identify and select the most important frequency channels. The computational complexity is reduced by retaining the important frequency components and discarding the insignificant ones as well as eliminating the unnecessary layers of the network. Experimental results show that the modified ResNet-50 operating in the compressed domain is up to 70% faster than the spatial-based traditional ResNet-50 while resulting in similar classification accuracy. Moreover, this paper proposes a preprocessing step with partial encoding to improve the resilience to distortions caused by low-quality encoded images. Finally, we show that training a network with highly compressed data can achieve a good classification accuracy with up to 93% reduction in the storage requirements of the training data.

Frequency-Tuned Universal Adversarial Attacks on Texture Recognition.

Although deep neural networks (DNNs) have been shown to be susceptible to image-agnostic adversarial attacks on natural image classification problems, the effects of such attacks on DNN-based texture recognition have yet to be explored. As part of our work, we find that limiting the perturbation's lp norm in the spatial domain may not be a suitable way to restrict the perceptibility of universal adversarial perturbations for texture images. Based on the fact that human perception is affected by local visual frequency characteristics, we propose a frequency-tuned universal attack method to compute universal perturbations in the frequency domain. Our experiments indicate that our proposed method can produce less perceptible perturbations yet with a similar or higher white-box fooling rates on various DNN texture classifiers and texture datasets as compared to existing universal attack techniques. We also demonstrate that our approach can improve the attack robustness against defended models as well as the cross-dataset transferability for texture recognition problems.

It GAN Do Better: GAN-Based Detection of Objects on Images With Varying Quality.

In this paper, we propose a novel generative framework which uses Generative Adversarial Networks (GANs) to generate features that provide robustness for object detection on reduced-quality images. The proposed GAN-based Detection of Objects (GAN-DO) framework is not restricted to any particular architecture and can be generalized to several deep neural network (DNN) based architectures. The resulting deep neural network maintains the exact architecture as the selected baseline model without adding to the model parameter complexity or inference speed. We first evaluate the effect of image quality on both object classification and object bounding box regression. We then test the models resulting from our proposed GAN-DO framework, using two state-of-the-art object detection architectures as the baseline models. We also evaluate the effect of the number of re-trained parameters in the generator of GAN-DO on the accuracy of the final trained model. Performance results provided using GAN-DO on object detection datasets establish an improved robustness to varying image quality and a higher mAP compared to the existing approaches.

A no-reference objective image sharpness metric based on the notion of just noticeable blur (JNB).

This work presents a perceptual-based no-reference objective image sharpness/blurriness metric by integrating the concept of just noticeable blur into a probability summation model. Unlike existing objective no-reference image sharpness/blurriness metrics, the proposed metric is able to predict the relative amount of blurriness in images with different content. Results are provided to illustrate the performance of the proposed perceptual-based sharpness metric. These results show that the proposed sharpness metric correlates well with the perceived sharpness being able to predict with high accuracy the relative amount of blurriness in images with different content.

DeepCorrect: Correcting DNN Models Against Image Distortions.

In recent years, the widespread use of deep neural networks (DNNs) has facilitated great improvements in performance for computer vision tasks like image classification and object recognition. In most realistic computer vision applications, an input image undergoes some form of image distortion such as blur and additive noise during image acquisition or transmission. Deep networks trained on pristine images perform poorly when tested on such distortions. In this paper, we evaluate the effect of image distortions like Gaussian blur and additive noise on the activations of pre-trained convolutional filters. We propose a metric to identify the most noise susceptible convolutional filters and rank them in order of the highest gain in classification accuracy upon correction. In our proposed approach called DeepCorrect, we apply small stacks of convolutional layers with residual connections at the output of these ranked filters and train them to correct the worst distortion affected filter activations, while leaving the rest of the pre-trained filter outputs in the network unchanged. Performance results show that applying DeepCorrect models for common vision tasks like image classification (ImageNet), object recognition (Caltech-101, Caltech-256), and scene classification (SUN-397), significantly improves the robustness of DNNs against distorted images and outperforms other alternative approaches.

Selective error detection for error-resilient wavelet-based image coding.

This paper introduces the concept of a similarity check function for error-resilient multimedia data transmission. The proposed similarity check function provides information about the effects of corrupted data on the quality of the reconstructed image. The degree of data corruption is measured by the similarity check function at the receiver, without explicit knowledge of the original source data. The design of a perceptual similarity check function is presented for wavelet-based coders such as the JPEG2000 standard, and used with a proposed "progressive similarity-based ARQ" (ProS-ARQ) scheme to significantly decrease the retransmission rate of corrupted data while maintaining very good visual quality of images transmitted over noisy channels. Simulation results with JPEG2000-coded images transmitted over the Binary Symmetric Channel, show that the proposed ProS-ARQ scheme significantly reduces the number of retransmissions as compared to conventional ARQ-based schemes. The presented results also show that, for the same number of retransmitted data packets, the proposed ProS-ARQ scheme can achieve significantly higher PSNR and better visual quality as compared to the selective-repeat ARQ scheme.

JPEG2000 encoding with perceptual distortion control.

In this paper, a new encoding approach is proposed to control the JPEG2000 encoding in order to reach a desired perceptual quality. The new method is based on a vision model that incorporates various masking effects of human visual perception and a perceptual distortion metric that takes spatial and spectral summation of individual quantization errors into account. Compared with the conventional rate-based distortion minimization JPEG2000 encoding, the new method provides a way to generate consistent quality images at a lower bit rate.

A Locally Weighted Fixation Density-Based Metric for Assessing the Quality of Visual Saliency Predictions.

With the increased focus on visual attention (VA) in the last decade, a large number of computational visual saliency methods have been developed over the past few years. These models are traditionally evaluated by using performance evaluation metrics that quantify the match between predicted saliency and fixation data obtained from eye-tracking experiments on human observers. Though a considerable number of such metrics have been proposed in the literature, there are notable problems in them. In this paper, we discuss shortcomings in the existing metrics through illustrative examples and propose a new metric that uses local weights based on fixation density, which overcomes these flaws. To compare the performance of our proposed metric at assessing the quality of saliency prediction with other existing metrics, we construct a ground-truth subjective database in which saliency maps obtained from 17 different VA models are evaluated by 16 human observers on a five-point categorical scale in terms of their visual resemblance with corresponding ground-truth fixation density maps obtained from eye-tracking data. The metrics are evaluated by correlating metric scores with the human subjective ratings. The correlation results show that the proposed evaluation metric outperforms all other popular existing metrics. In addition, the constructed database and corresponding subjective ratings provide an insight into which of the existing metrics and future metrics are better at estimating the quality of saliency prediction and can be used as a benchmark.

Mutual information-based analysis of JPEG2000 contexts.

Context-based arithmetic coding has been widely adopted in image and video compression and is a key component of the new JPEG2000 image compression standard. In this paper, the contexts used in JPEG2000 are analyzed using the mutual information, which is closely related to the compression performance. We first show that, when combining the contexts, the mutual information between the contexts and the encoded data will decrease unless the conditional probability distributions of the combined contexts are the same. Given I, the initial number of contexts, and F, the final desired number of contexts, there are S(I, F) possible context classification schemes where S(I, F) is called the Stirling number of the second kind. The optimal classification scheme is the one that gives the maximum mutual information. Instead of using an exhaustive search, the optimal classification scheme can be obtained through a modified generalized Lloyd algorithm with the relative entropy as the distortion metric. For binary arithmetic coding, the search complexity can be reduced by using dynamic programming. Our experimental results show that the JPEG2000 contexts capture the correlations among the wavelet coefficients very well. At the same time, the number of contexts used as part of the standard can be reduced without loss in the coding performance.

A No-Reference Texture Regularity Metric Based on Visual Saliency.

This paper presents a no-reference perceptual metric that quantifies the degree of perceived regularity in textures. The metric is based on the similarity of visual attention (VA) of the textural primitives and the periodic spatial distribution of foveated fixation regions throughout the image. A ground-truth eye-tracking database for textures is also generated as part of this paper and is used to evaluate the performance of the most popular VA models. Using the saliency map generated by the best VA model, the proposed texture regularity metric is computed. It is shown through subjective testing that the proposed metric has a strong correlation with the mean opinion score for the perceived regularity of textures. The proposed texture regularity metric can be used to improve the quality and performance of many image processing applications like texture synthesis, texture compression, and content-based image retrieval.

Adaptive image coding with perceptual distortion control.

This paper presents a discrete cosine transform (DCT)-based locally adaptive perceptual image coder, which discriminates between image components based on their perceptual relevance for achieving increased performance in terms of quality and bit rate. The new coder uses a locally adaptive perceptual quantization scheme based on a tractable perceptual distortion metric. Our strategy is to exploit human visual masking properties by deriving visual masking thresholds in a locally adaptive fashion. The derived masking thresholds are used in controlling the quantization stage by adapting the quantizer reconstruction levels in order to meet the desired target perceptual distortion. The proposed coding scheme is flexible in that it can be easily extended to work with any subband-based decomposition in addition to block-based transform methods. Compared to existing perceptual coding methods, the proposed perceptual coding method exhibits superior performance in terms of bit rate and distortion control. Coding results are presented to illustrate the performance of the presented coding scheme.

A distributed Canny edge detector: algorithm and FPGA implementation.

The Canny edge detector is one of the most widely used edge detection algorithms due to its superior performance. Unfortunately, not only is it computationally more intensive as compared with other edge detection algorithms, but it also has a higher latency because it is based on frame-level statistics. In this paper, we propose a mechanism to implement the Canny algorithm at the block level without any loss in edge detection performance compared with the original frame-level Canny algorithm. Directly applying the original Canny algorithm at the block-level leads to excessive edges in smooth regions and to loss of significant edges in high-detailed regions since the original Canny computes the high and low thresholds based on the frame-level statistics. To solve this problem, we present a distributed Canny edge detection algorithm that adaptively computes the edge detection thresholds based on the block type and the local distribution of the gradients in the image block. In addition, the new algorithm uses a nonuniform gradient magnitude histogram to compute block-based hysteresis thresholds. The resulting block-based algorithm has a significantly reduced latency and can be easily integrated with other block-based image codecs. It is capable of supporting fast edge detection of images and videos with high resolutions, including full-HD since the latency is now a function of the block size instead of the frame size. In addition, quantitative conformance evaluations and subjective tests show that the edge detection performance of the proposed algorithm is better than the original frame-based algorithm, especially when noise is present in the images. Finally, this algorithm is implemented using a 32 computing engine architecture and is synthesized on the Xilinx Virtex-5 FPGA. The synthesized architecture takes only 0.721 ms (including the SRAM READ/WRITE time and the computation time) to detect edges of 512 × 512 images in the USC SIPI database when clocked at 100 MHz and is faster than existing FPGA and GPU implementations.

A no-reference image blur metric based on the cumulative probability of blur detection (CPBD).

This paper presents a no-reference image blur metric that is based on the study of human blur perception for varying contrast values. The metric utilizes a probabilistic model to estimate the probability of detecting blur at each edge in the image, and then the information is pooled by computing the cumulative probability of blur detection (CPBD). The performance of the metric is demonstrated by comparing it with existing no-reference sharpness/blurriness metrics for various publicly available image databases.

An efficient selective perceptual-based super-resolution estimator.

In this paper, a selective perceptual-based (SELP) framework is presented to reduce the complexity of popular super-resolution (SR) algorithms while maintaining the desired quality of the enhanced images/video. A perceptual human visual system model is proposed to compute local contrast sensitivity thresholds. The obtained thresholds are used to select which pixels are super-resolved based on the perceived visibility of local edges. Processing only a set of perceptually significant pixels reduces significantly the computational complexity of SR algorithms without losing the achievable visual quality. The proposed SELP framework is integrated into a maximum-a posteriori-based SR algorithm as well as a fast two-stage fusion-restoration SR estimator. Simulation results show a significant reduction on average in computational complexity with comparable signal-to-noise ratio gains and visual quality.