The Multi-Focus-Image-Fusion Method Based on Convolutional Neural Network and Sparse Representation.

Multi-focus-image-fusion is a crucial embranchment of image processing. Many methods have been developed from different perspectives to solve this problem. Among them, the sparse representation (SR)-based and convolutional neural network (CNN)-based fusion methods have been widely used. Fusing the source image patches, the SR-based model is essentially a local method with a nonlinear fusion rule. On the other hand, the direct mapping between the source images follows the decision map which is learned via CNN. The fusion is a global one with a linear fusion rule. Combining the advantages of the above two methods, a novel fusion method that applies CNN to assist SR is proposed for the purpose of gaining a fused image with more precise and abundant information. In the proposed method, source image patches were fused based on SR and the new weight obtained by CNN. Experimental results demonstrate that the proposed method clearly outperforms existing state-of-the-art methods in addition to SR and CNN in terms of both visual perception and objective evaluation metrics, and the computational complexity is greatly reduced. Experimental results demonstrate that the proposed method not only clearly outperforms the SR and CNN methods in terms of visual perception and objective evaluation indicators, but is also significantly better than other state-of-the-art methods since our computational complexity is greatly reduced.

Anisotropic Diffusion Based Multiplicative Speckle Noise Removal.

Multiplicative speckle noise removal is a challenging task in image processing. Motivated by the performance of anisotropic diffusion in additive noise removal and the structure of the standard deviation of a compressed speckle noisy image, we address this problem with anisotropic diffusion theories. Firstly, an anisotropic diffusion model based on image statistics, including information on the gradient of the image, gray levels, and noise standard deviation of the image, is proposed. Although the proposed model can effectively remove multiplicative speckle noise, it does not consider the noise at the edge during the denoising process. Hence, we decompose the divergence term in order to make the diffusion at the edge occur along the boundaries rather than perpendicular to the boundaries, and improve the model to meet our requirements. Secondly, the iteration stopping criteria based on kurtosis and correlation in view of the lack of ground truth in real image experiments, is proposed. The optimal values of the parameters in the model are obtained by learning. To improve the denoising effect, post-processing is performed. Finally, the simulation results show that the proposed model can effectively remove the speckle noise and retain minute details of the images for the real ultrasound and RGB color images.

Image decomposition fusion method based on sparse representation and neural network.

For noisy images, in most existing sparse representation-based models, fusion and denoising proceed simultaneously using the coefficients of a universal dictionary. This paper proposes an image fusion method based on a cartoon + texture dictionary pair combined with a deep neural network combination (DNNC). In our model, denoising and fusion are carried out alternately. The proposed method is divided into three main steps: denoising + fusion + network denoising. More specifically, (1) denoise the source images using external/internal methods separately; (2) fuse these preliminary denoised results with external/internal cartoon and texture dictionary pair to obtain the external cartoon + texture sparse representation result (E-CTSR) and internal cartoon + texture sparse representation result (I-CTSR); and (3) combine E-CTSR and I-CTSR using DNNC (EI-CTSR) to obtain the final result. Experimental results demonstrate that EI-CTSR outperforms not only the stand-alone E-CTSR and I-CTSR methods but also state-of-the-art methods such as sparse representation (SR) and adaptive sparse representation (ASR) for isomorphic images, and E-CTSR outperforms SR and ASR for heterogeneous multi-mode images.

Weight Decay With Tailored Adam on Scale-Invariant Weights for Better Generalization.

Weight decay (WD) is a fundamental and practical regularization technique in improving generalization of current deep learning models. However, it is observed that the WD does not work effectively for an adaptive optimization algorithm (such as Adam), as it works for SGD. Specifically, the solution found by Adam with the WD often generalizes unsatisfactorily. Though efforts have been made to mitigate this issue, the reason for such deficiency is still vague. In this article, we first show that when using the Adam optimizer, the weight norm increases very fast along with the training procedure, which is in contrast to SGD where the weight norm increases relatively slower and tends to converge. The fast increase of weight norm is adverse to WD; in consequence, the Adam optimizer will lose efficacy in finding solution that generalizes well. To resolve this problem, we propose to tailor Adam by introducing a regularization term on the adaptive learning rate, such that it is friendly to WD. Meanwhile, we introduce first moment on the WD to further enhance the regularization effect. We show that the proposed method is able to find solution with small norm and generalizes better than SGD. We test the proposed method on general image classification and fine-grained image classification tasks with different networks. Experimental results on all these cases substantiate the effectiveness of the proposed method in help improving the generalization. Specifically, the proposed method improves the test accuracy of Adam by a large margin and even improves the performance of SGD by 0.84% on CIFAR 10 and 1.03 % on CIFAR 100 with ResNet-50. The code of this article is public available at xxx.

Deep RED Unfolding Network for Image Restoration.

The deep unfolding network (DUN) provides an efficient framework for image restoration. It consists of a regularization module and a data fitting module. In existing DUN models, it is common to directly use a deep convolution neural network (DCNN) as the regularization module, and perform data fitting before regularization in each iteration/stage. In this work, we present a DUN by incorporating a new regularization module, and putting the regularization module before the data fitting module. The proposed regularization model is deducted by using the regularization by denoing (RED) and plugging in it a newly designed DCNN. For the data fitting module, we use the closed-form solution with Faster Fourier Transform (FFT). The resulted DRED-DUN model has some major advantages. First, the regularization model inherits the flexibility of learned image-adaptive and interpretability of RED. Second, the DRED-DUN model is an end-to-end trainable DUN, which learns the regularization network and other parameters jointly, thus leads to better restoration performance than the plug-and-play framework. Third, extensive experiments show that, our proposed model significantly outperforms the-state-of-the-art model-based methods and learning based methods in terms of PSNR indexes as well as the visual effects. In particular, our method has much better capability in recovering salient image components such as edges and small scale textures.

Generalized Unitarily Invariant Gauge Regularization for Fast Low-Rank Matrix Recovery.

Spectral regularization is a widely used approach for low-rank matrix recovery (LRMR) by regularizing matrix singular values. Most of the existing LRMR solvers iteratively compute the singular values via applying singular value decomposition (SVD) on a dense matrix, which is computationally expensive and severely limits their applications to large-scale problems. To address this issue, we present a generalized unitarily invariant gauge (GUIG) function for LRMR. The proposed GUIG function does not act on the singular values; however, we show that it generalizes the well-known spectral functions, including the rank function, the Schatten- p quasi-norm, and logsum of singular values. The proposed GUIG regularization model can be formulated as a bilinear variational problem, which can be efficiently solved without computing SVD. Such a property makes it well suited for large-scale LRMR problems. We apply the proposed GUIG model to matrix completion and robust principal component analysis and prove the convergence of the algorithms. Experimental results demonstrate that the proposed GUIG method is not only more accurate but also much faster than the state-of-the-art algorithms, especially on large-scale problems.

Unified Discriminative and Coherent Semi-supervised Subspace Clustering.

The ubiquitous large, complex and high dimensional datasets in computer vision and machine learning generates the problem of subspace clustering, which aims to partition the data into several low dimensional subspaces. By utilizing relatively limited labeled data and sufficient unlabeled data, the semi-supervised subspace clustering is more effective, practical and become more popular. In this work, we present a new regularity combing the labels and the affinity to ensure the coherence of the affinity between data points from the same subspace as well as the discrimination of cluster labels for data points from different subspaces. We combine it with the manifold smoothing term of the existing methods and the Gaussian fields and harmonic functions method to give a new unified optimization framework for semi-supervised subspace clustering. Analysis shows the proposed model fully combines the affinity and the labels to guide each other so that both are discriminative between clusters and coherent within clusters. Extensive experiments show that our method outperforms the existing state-of-the-art methods, thus suggests that the property of discriminative between clusters and coherent within clusters of our method is advantageous to semi-supervised subspace clustering.

Fractional-order anisotropic diffusion for image denoising.

This paper introduces a new class of fractional-order anisotropic diffusion equations for noise removal. These equations are Euler-Lagrange equations of a cost functional which is an increasing function of the absolute value of the fractional derivative of the image intensity function, so the proposed equations can be seen as generalizations of second-order and fourth-order anisotropic diffusion equations. We use the discrete Fourier transform to implement the numerical algorithm and give an iterative scheme in the frequency domain. It is one important aspect of the algorithm that it considers the input image as a periodic image. To overcome this problem, we use a folded algorithm by extending the image symmetrically about its borders. Finally, we list various numerical results on denoising real images. Experiments show that the proposed fractional-order anisotropic diffusion equations yield good visual effects and better signal-to-noise ratio.

Cross-Domain Visual Matching via Generalized Similarity Measure and Feature Learning.

Cross-domain visual data matching is one of the fundamental problems in many real-world vision tasks, e.g., matching persons across ID photos and surveillance videos. Conventional approaches to this problem usually involves two steps: i) projecting samples from different domains into a common space, and ii) computing (dis-)similarity in this space based on a certain distance. In this paper, we present a novel pairwise similarity measure that advances existing models by i) expanding traditional linear projections into affine transformations and ii) fusing affine Mahalanobis distance and Cosine similarity by a data-driven combination. Moreover, we unify our similarity measure with feature representation learning via deep convolutional neural networks. Specifically, we incorporate the similarity measure matrix into the deep architecture, enabling an end-to-end way of model optimization. We extensively evaluate our generalized similarity model in several challenging cross-domain matching tasks: person re-identification under different views and face verification over different modalities (i.e., faces from still images and videos, older and younger faces, and sketch and photo portraits). The experimental results demonstrate superior performance of our model over other state-of-the-art methods.

Multiplicative noise removal using primal-dual and reweighted alternating minimization.

Multiplicative noise removal is an important research topic in image processing field. An algorithm using reweighted alternating minimization to remove this kind of noise is proposed in our preliminary work. While achieving good results, a small parameter is needed to avoid the denominator vanishing. We find that the parameter has important influence on numerical results and has to be chosen carefully. In this paper a primal-dual algorithm is designed without the artificial parameter. Numerical experiments show that the new algorithm can get a good visual quality, overcome staircase effects and preserve the edges, while maintaining high signal-to-noise ratio.

Blind image quality assessment using joint statistics of gradient magnitude and Laplacian features.

Blind image quality assessment (BIQA) aims to evaluate the perceptual quality of a distorted image without information regarding its reference image. Existing BIQA models usually predict the image quality by analyzing the image statistics in some transformed domain, e.g., in the discrete cosine transform domain or wavelet domain. Though great progress has been made in recent years, BIQA is still a very challenging task due to the lack of a reference image. Considering that image local contrast features convey important structural information that is closely related to image perceptual quality, we propose a novel BIQA model that utilizes the joint statistics of two types of commonly used local contrast features: 1) the gradient magnitude (GM) map and 2) the Laplacian of Gaussian (LOG) response. We employ an adaptive procedure to jointly normalize the GM and LOG features, and show that the joint statistics of normalized GM and LOG features have desirable properties for the BIQA task. The proposed model is extensively evaluated on three large-scale benchmark databases, and shown to deliver highly competitive performance with state-of-the-art BIQA models, as well as with some well-known full reference image quality assessment models.

Two-direction nonlocal model for image denoising.

Similarities inherent in natural images have been widely exploited for image denoising and other applications. In fact, if a cluster of similar image patches is rearranged into a matrix, similarities exist both between columns and rows. Using the similarities, we present a two-directional nonlocal (TDNL) variational model for image denoising. The solution of our model consists of three components: one component is a scaled version of the original observed image and the other two components are obtained by utilizing the similarities. Specifically, by using the similarity between columns, we get a nonlocal-means-like estimation of the patch with consideration to all similar patches, while the weights are not the pairwise similarities but a set of clusterwise coefficients. Moreover, by using the similarity between rows, we also get nonlocal-autoregression-like estimations for the center pixels of the similar patches. The TDNL model leads to an alternative minimization algorithm. Experiments indicate that the model can perform on par with or better than the state-of-the-art denoising methods.