Improved Multiple-Image-Based Reflection Removal Algorithm Using Deep Neural Networks.

When imaging through a semi-reflective medium such as glass, the reflection of another scene can often be found in the captured images. It degrades the quality of the images and affects their subsequent analyses. In this paper, a novel deep neural network approach for solving the reflection problem in imaging is presented. Traditional reflection removal methods not only require long computation time for solving different optimization functions, their performance is also not guaranteed. As array cameras are readily available in nowadays imaging devices, we first suggest in this paper a multiple-image based depth estimation method using a convolutional neural network (CNN). The proposed network avoids the depth ambiguity problem due to the reflection in the image, and directly estimates the depths along the image edges. They are then used to classify the edges as belonging to the background or reflection. Since edges having similar depth values are error prone in the classification, they are removed from the reflection removal process. We suggest a generative adversarial network (GAN) to regenerate the removed background edges. Finally, the estimated background edge map is fed to another auto-encoder network to assist the extraction of the background from the original image. Experimental results show that the proposed reflection removal algorithm achieves superior performance both quantitatively and qualitatively as compared to the state-of-the-art methods. The proposed algorithm also shows much faster speed compared to the existing approaches using the traditional optimization methods.

A Framework of Reversible Color-tograyscale Conversion with Watermarking Feature.

Reversible color-to-grayscale conversion (RCGC) is a method that embeds the chromatic information of a full color image into its grayscale version such that the original color image can be reconstructed in the future when necessary. In practical applications, it is required to provide a means to authenticate an information-embedded image such that its integrity can be guaranteed. However, none of the current RCGC algorithms take this factor into account. In this paper, to address this issue, we develop an information-embedding framework based on a vector quantization-based (VQ-based) RCGC algorithm recently proposed by us. Under this framework, we propose a RCGC algorithm that can embed both chromatic information and fragile watermark simultaneously into a grayscale image with the same technique to reduce the complexity and improve the efficiency. Like other VQ-based RCGC algorithms, the performance of the proposed RCGC algorithm highly relies on the palette it uses. We also propose a palette generation algorithm in this paper to support the information embedding process such that the visual quality of the color-embedded grayscale images and the reconstructed color images can be significantly improved.

Robust Reflection Removal Based on Light Field Imaging.

In daily photography, it is common to capture images in the reflection of an unwanted scene. This circumstance arises frequently when imaging through a semi-reflecting material such as glass. The unwanted reflection will affect the visibility of the background image and introduce ambiguity that perturbs the subsequent analysis on the image. It is a very challenging task to remove the reflection of an image since the problem is severely ill-posed. In this paper, we propose a novel algorithm to solve the reflection removal problem based on light field (LF) imaging. For the proposed algorithm, we first show that the strong gradient points of an LF epipolar plane image (EPI) are preserved after adding to the EPI of another LF image. We can then make use of these strong gradient points to give a rough estimation of the background and reflection. Rather than assuming that the background and reflection have absolutely different disparity ranges, we propose a sandwich layer model to allow them to have common disparities, which is more realistic in practical situations. Then, the background image is refined by recovering the components in the shared disparity range using an iterative enhancement process. Our experimental results show that the proposed algorithm achieves superior performance over traditional approaches both qualitatively and quantitatively. These results verify the robustness of the proposed algorithm when working with images captured from real-life scenes.

A lossless compression scheme for Bayer color filter array images.

In most digital cameras, Bayer color filter array (CFA) images are captured and demosaicing is generally carried out before compression. Recently, it was found that compression-first schemes outperform the conventional demosaicing-first schemes in terms of output image quality. An efficient prediction-based lossless compression scheme for Bayer CFA images is proposed in this paper. It exploits a context matching technique to rank the neighboring pixels when predicting a pixel, an adaptive color difference estimation scheme to remove the color spectral redundancy when handling red and blue samples, and an adaptive codeword generation technique to adjust the divisor of Rice code for encoding the prediction residues. Simulation results show that the proposed compression scheme can achieve a better compression performance than conventional lossless CFA image coding schemes.

Robust Single-shot Fringe Projection Profilometry Based on Morphological Component Analysis.

In a fringe projection profilometry (FPP) process, the captured fringe images can be modeled as the superimposition of the projected fringe patterns on the texture of the objects. Extracting the fringe patterns from the captured fringe images is an essential procedure in FPP; but traditional single-shot FPP methods often fail to perform if the objects have a highly textured surface. In this paper, a new single-shot FPP algorithm which allows the object texture and fringe pattern to be estimated simultaneously is proposed. The heart of the proposed algorithm is an enhanced morphological component analysis (MCA) tailored for FPP problems. Conventional MCA methods which use a uniform threshold in an iterative optimization process are inefficient to separate fringe-like patterns from image texture. We extend the conventional MCA by taking advantage of the low-rank structure of the fringe's sparse representation to enable an adaptive thresholding process. It ends up with a robust single-shot FPP algorithm that can extract the fringe pattern even if the object has a highly textured surface. The proposed approach has a side benefit that the object texture can be simultaneously obtained in the fringe pattern estimation process, which is useful in many FPP applications. Experimental results have demonstrated the improved performance of the proposed algorithm over the conventional single-shot FPP approaches.

A low-complexity joint color demosaicking and zooming algorithm for digital camera.

This paper presents a low complexity joint color demosaicking and digital zooming algorithm for single-sensor digital cameras. The proposed algorithm directly extracts edge information from raw sensor data for interpolation in both demosaicking and zooming to preserve edge features in its output. This allows the extracted information to be exploited consistently in both stages and also efficiently, as no separate extraction process is required in different stages. The proposed algorithm can produce a zoomed full-color image as well as a zoomed Bayer color filter array image with outstanding performance as compared with conventional approaches which generally combine separate color demosaicking and digital zooming schemes.

A POCS-based restoration algorithm for restoring halftoned color-quantized images.

This paper studies the restoration of images which are color-quantized with error diffusion. Though there are many reported algorithms proposed for restoring noisy blurred color images and inverse halftoning, restoration of color-quantized images is rarely addressed in the literature especially when the images are color-quantized with halftoning. Direct application of existing restoration techniques are generally inadequate to deal with this problem. In this paper, a restoration algorithm based on projection onto convex sets is proposed. This algorithm makes use of the available color palette and the mechanism of a halftoning process to derive useful a priori information for restoration. Simulation results showed that it could improve the quality of a halftoned color-quantized image remarkably in terms of both SNR and CIELAB color difference metric.

Color demosaicing using variance of color differences.

This paper presents an adaptive demosaicing algorithm. Missing green samples are first estimated based on the variances of the color differences along different edge directions. The missing red and blue components are then estimated based on the interpolated green plane. This algorithm can effectively preserve the details in texture regions and, at the same time, it can significantly reduce the color artifacts. As compared with the latest demosaicing algorithms, the proposed algorithm produces the best average demosaicing performance both objectively and subjectively.

A technique for producing scalable color-quantized images with error diffusion.

To reliably and efficiently deliver media information to diverse clients over heterogeneous networks, the media involved must be scalable. In this paper, a color quantization algorithm for generating scalable color-indexed images is proposed based on a multiscale error diffusion framework. Images of lower resolutions are embedded in the outputs such that a simple down-sampling process can extract images of any desirable resolutions. Images possessing this scalable property support transmission over the Internet which contains clients with different display resolutions, systems with different caching resources and networks with varying bandwidths and QoS capabilities. Unlike most of the color halftoning algorithms available nowadays, the proposed algorithm is not dedicated for printing applications but for color-indexed displays. It works with any arbitrary palettes of different size.

Efficient blind image restoration using discrete periodic radon transform.

Restoring an image from its convolution with an unknown blur function is a well-known ill-posed problem in image processing. Many approaches have been proposed to solve the problem and they have shown to have good performance in identifying the blur function and restoring the original image. However, in actual implementation, various problems incurred due to the large data size and long computational time of these approaches are undesirable even with the current computing machines. In this paper, an efficient algorithm is proposed for blind image restoration based on the discrete periodic Radon transform (DPRT). With DPRT, the original two-dimensional blind image restoration problem is converted into one-dimensional ones, which greatly reduces the memory size and computational time required. Experimental results show that the resulting approach is faster in almost an order of magnitude as compared with the traditional approach, while the quality of the restored image is similar.

Optimizing the error diffusion filter for blue noise halftoning with multiscale error diffusion.

A good halftoning output should bear a blue noise characteristic contributed by isotropically-distributed isolated dots. Multiscale error diffusion (MED) algorithms try to achieve this by exploiting radially symmetric and noncausal error diffusion filters to guarantee spatial homogeneity. In this brief, an optimized diffusion filter is suggested to make the diffusion close to isotropic. When it is used with MED, the resulting output has a nearly ideal blue noise characteristic.

Green noise digital halftoning with multiscale error diffusion.

Multiscale error diffusion (MED) is superior to conventional error diffusion algorithms as it can eliminate directional hysteresis completely and possesses a good blue noise characteristic. However, due to its filter design, it is not suitable for systems with poor isolated dot generation and instable dot gain. In this paper, we propose a MED algorithm to produce halftones of desirable green noise characteristics. This algorithm allows one to adjust the desirable cluster size freely through a single parameter and supports a linear relationship between the cluster size and the input gray level. With a close-to-isotropic diffusion filter, the algorithm can effectively remove pattern artifacts, eliminate directional artifacts and preserve original image details. Analysis and simulation results show that it provides better performance in terms of various aspects including dot distribution, anisotropy and output image quality as compared with other conventional green noise error diffusion algorithms.