PS-NET: an end-to-end phase space depth estimation approach for computer-generated holograms.

In the present work, an end-to-end approach is proposed for recovering an RGB-D scene representation directly from a hologram using its phase space representation. The proposed method involves four steps. First, a set of silhouette images is extracted from the hologram phase space representation. Second, a minimal 3D volume that describes these silhouettes is extracted. Third, the extracted 3D volume is decomposed into horizontal slices, and each slice is processed using a neural network to generate a coarse estimation of the scene geometry. Finally, a third neural network is employed to refine the estimation for higher precision applications. Experimental results demonstrate that the proposed approach yields faster and more accurate results compared to numerical reconstruction-based methods. Moreover, the obtained RGB-D representation can be directly utilized for alternative applications such as motion estimation.

EDDMF: An Efficient Deep Discrepancy Measuring Framework for Full-Reference Light Field Image Quality Assessment.

The increasing demand for immersive experience has greatly promoted the quality assessment research of Light Field Image (LFI). In this paper, we propose an efficient deep discrepancy measuring framework for full-reference light field image quality assessment. The main idea of the proposed framework is to efficiently evaluate the quality degradation of distorted LFIs by measuring the discrepancy between reference and distorted LFI patches. Firstly, a patch generation module is proposed to extract spatio-angular patches and sub-aperture patches from LFIs, which greatly reduces the computational cost. Then, we design a hierarchical discrepancy network based on convolutional neural networks to extract the hierarchical discrepancy features between reference and distorted spatio-angular patches. Besides, the local discrepancy features between reference and distorted sub-aperture patches are extracted as complementary features. After that, the angular-dominant hierarchical discrepancy features and the spatial-dominant local discrepancy features are combined to evaluate the patch quality. Finally, the quality of all patches is pooled to obtain the overall quality of distorted LFIs. To the best of our knowledge, the proposed framework is the first patch-based full-reference light field image quality assessment metric based on deep-learning technology. Experimental results on four representative LFI datasets show that our proposed framework achieves superior performance as well as lower computational complexity compared to other state-of-the-art metrics.

H-Seg: a horizontal reconstruction volume segmentation method for accurate depth estimation in a computer-generated hologram.

In this work, we introduce a novel approach for depth estimation in a computer-generated hologram by employing horizontal segmentation of the reconstruction volume instead of conventional vertical segmentation. The reconstruction volume is divided into horizontal slices and each slice is processed using a residual U-net architecture to identify in-focus lines, enabling determination of the slice's intersection with the 3D scene. The individual slice results are then combined to generate a dense depth map of the scene. Our experiments demonstrate the effectiveness of our method, with improved accuracy, faster processing times, lower graphics processing unit (GPU) utilization, and smoother predicted depth maps than existing state-of-the-art models.

Automatic depth map retrieval from digital holograms using a depth-from-focus approach.

Recovering the scene depth map from a computer-generated hologram is a problem that remains unsolved, despite the growing interest in the subject. In this paper, we propose to study the application of depth-from-focus (DFF) methods to retrieve the depth information from the hologram. We discuss the different hyperparameters that are required for the application of the method and their impact on the final result. The obtained results show that DFF methods can be used for depth estimation from the hologram if the set of hyperparameters is well chosen.

Automatic depth map retrieval from digital holograms using a deep learning approach.

Information extraction from computer-generated holograms using learning-based methods is a topic that has not received much research attention. In this article, we propose and study two learning-based methods to extract the depth information from a hologram and compare their performance with that of classical depth from focus (DFF) methods. We discuss the main characteristics of a hologram and how these characteristics can affect model training. The obtained results show that it is possible to extract depth information from a hologram if the problem formulation is well-posed. The proposed methods are faster and more accurate than state-of-the-art DFF methods.

Optimal Adaptive Quantization Based on Temporal Distortion Propagation Model for HEVC.

Optimal adaptive quantization is one of the key points to optimize the coding efficiency of video encoders. The latest block-based video compression standards, such as high-efficiency video coding (HEVC), extensively use predictive coding techniques that create dependencies between blocks and increase the complexity of optimal block quantizers search. Specifically, the motion compensation is responsible for a dependency network connecting all blocks of the same GOP together. In this paper, this dependency network is estimated by a temporal distortion propagation model and an accurate estimation of Inter and Skip modes probabilities. Optimal quantizers are then designed per block in order to achieve global optimization in terms of rate-distortion efficiency. By implementing the algorithm into the HEVC reference model (HM), we report -16.51% PSNR-based and -26.26% SSIM-based average bitrate savings compared to no adaptive quantization. The proposed algorithm outperforms several related methods from the state-of-the-art. Moreover, along with the demonstration of an optimal quantizer solution, we propose an in-depth analysis of the algorithm behavior. This analysis includes, among others, the relative distribution of rates between frames and the control of quantizers dynamic range.

NIQSV+: A No-Reference Synthesized View Quality Assessment Metric.

Benefiting from multi-view video plus depth and depth-image-based-rendering technologies, only limited views of a real 3-D scene need to be captured, compressed, and transmitted. However, the quality assessment of synthesized views is very challenging, since some new types of distortions, which are inherently different from the texture coding errors, are inevitably produced by view synthesis and depth map compression, and the corresponding original views (reference views) are usually not available. Thus the full-reference quality metrics cannot be used for synthesized views. In this paper, we propose a novel no-reference image quality assessment method for 3-D synthesized views (called NIQSV+). This blind metric can evaluate the quality of synthesized views by measuring the typical synthesis distortions: blurry regions, black holes, and stretching, with access to neither the reference image nor the depth map. To evaluate the performance of the proposed method, we compare it with four full-reference 3-D (synthesized view dedicated) metrics, five full-reference 2-D metrics, and three no-reference 2-D metrics. In terms of their correlations with subjective scores, our experimental results show that the proposed no-reference metric approaches the best of the state-of-the-art full reference and no-reference 3-D metrics; and outperforms the widely used no-reference and full-reference 2-D metrics significantly. In terms of its approximation of human ranking, the proposed metric achieves the best performance in the experimental test.

Hybrid approach for fast occlusion processing in computer-generated hologram calculation.

A hybrid approach for fast occlusion processing in computer-generated hologram calculation is studied in this paper. The proposed method is based on the combination of two commonly used approaches that complement one another: the point-source and wave-field approaches. By using these two approaches together, the proposed method thus takes advantage of both of them. In this method, the 3D scene is first sliced into several depth layers parallel to the hologram plane. Light scattered by the scene is then propagated and shielded from one layer to another using either a point-source or a wave-field approach according to a threshold criterion on the number of points within the layer. Finally, the hologram is obtained by computing the propagation of light from the nearest layer to the hologram plane. Experimental results reveal that the proposed method does not produce any visible artifact and outperforms both the point-source and wave-field approaches.

3-D model-based frame interpolation for distributed video coding of static scenes.

This paper addresses the problem of side information extraction for distributed coding of videos captured by a camera moving in a 3-D static environment. Examples of targeted applications are augmented reality, remote-controlled robots operating in hazardous environments, or remote exploration by drones. It explores the benefits of the structure-from-motion paradigm for distributed coding of this type of video content. Two interpolation methods constrained by the scene geometry, based either on block matching along epipolar lines or on 3-D mesh fitting, are first developed. These techniques are based on a robust algorithm for sub-pel matching of feature points, which leads to semi-dense correspondences between key frames. However, their rate-distortion (RD) performances are limited by misalignments between the side information and the actual Wyner-Ziv (WZ) frames due to the assumption of linear motion between key frames. To cope with this problem, two feature point tracking techniques are introduced, which recover the camera parameters of the WZ frames. A first technique, in which the frames remain encoded separately, performs tracking at the decoder and leads to significant RD performance gains. A second technique further improves the RD performances by allowing a limited tracking at the encoder. As an additional benefit, statistics on tracks allow the encoder to adapt the key frame frequency to the video motion content.

One-dimensional dense disparity estimation for three-dimensional reconstruction.

We present a method for fully automatic three-dimensional (3D) reconstruction from a pair of weakly calibrated images in order to deal with the modeling of complex rigid scenes. A two-dimensional (2D) triangular mesh model of the scene is calculated using a two-step algorithm mixing sparse matching and dense motion estimation approaches. The 2D mesh is iteratively refined to fit any arbitrary 3D surface. At convergence, each triangular patch corresponds to the projection of a 3D plane. The proposed algorithm relies first on a dense disparity field. The dense field estimation modelized within a robust framework is constrained by the epipolar geometry. The resulting field is then segmented according to homographic models using iterative Delaunay triangulation. In association with a weak calibration and camera motion estimation algorithm, this 2D planar model is used to obtain a VRML-compatible 3D model of the scene.