Optical three-dimensional refocusing from elemental images based on a sifting property of the periodic δ-function array in integral-imaging.

We propose a novel approach to optically refocus three-dimensional (3-D) objects on their real depth from the captured elemental image array (EIA) by using a sifting property of the periodic δ-function array (PDFA) in integral-imaging. By convolving the PDFAs whose spatial periods correspond to each object's depth with the sub-image array (SIA) transformed from the EIA, a set of spatially filtered-SIAs (SF-SIAs) for each object's depth can be extracted. These SF-SIAs are then inverse-transformed into the corresponding versions of the EIAs, and from these, 3-D objects with their own perspectives can be reconstructed to be refocused on their depth in the space. The feasibility of the proposed method has been confirmed through optical experiments as well as ray-optical analysis.

Resolution-enhanced integral imaging in focal mode with a time-multiplexed electrical mask array.

The inferior resolution of the three-dimensional (3D) image is one of the main problems to be resolved for realizing a commercial autosteresosopic 3D display device. In this paper, a time-multiplexing technique using electrically moving masks is proposed to enhance the resolution of the 3D image realized by integral imaging in a focal mode while preserving the viewing angle of it.

Multi-projection integral imaging by use of a convex mirror array.

We propose a new multi-projection integral imaging scheme using a convex mirror array. In the proposed scheme, to overcome the resolution limitation of the conventional method due to observing the single aperture imaging point (AIP) from each convex mirror, we introduce the multi-projection to obtain multiple AIPs per convex mirror so that the viewer observes the resolution-improved 3D reconstructed images. We validate the theoretical analysis of the proposed scheme and confirm its feasibility through the optical experiments. To our best knowledge, this is the first report to generate multiple AIPs per convex mirror in a projection integral imaging system.

Three-dimensional imaging and visualization using off-axially distributed image sensing.

This Letter presents an off-axially distributed image sensing (ODIS) system for three-dimensional (3D) imaging and visualization. The off-axially distributed sensing method provides both lateral and longitudinal perspectives for 3D scenes even though the sensor moves along a slanted, one-dimensional path. A 3D volume is generated from a set of recorded images by use of a computational algorithm based on ray backprojection. Preliminary experimental results are presented to illustrate the feasibility of the proposed system. To the best of our knowledge, this is the first report on 3D imaging and visualization using ODIS.

Depth extraction of 3D objects using axially distributed image sensing.

Axially distributed image sensing (ADS) technique is capable of capturing 3D objects and reconstructing high-resolution slice plane images for 3D objects. In this paper, we propose a computational method for depth extraction of 3D objects using ADS. In the proposed method, the high-resolution elemental images are recorded by simply moving the camera along the optical axis and the recorded elemental images are used to generate a set of 3D slice images using the computational reconstruction algorithm based on ray back-projection. To extract depth of 3D object, we propose the simple block comparison algorithm between the first elemental image and a set of 3D slice images. This provides a simple computation process and robustness for depth extraction. To demonstrate our method, we carry out the preliminary experiments of three scenarios for 3D objects and the results are presented. To our best knowledge, this is the first report to extract the depth information using an ADS method.

Three-dimensional imaging and visualization of partially occluded objects using axially distributed stereo image sensing.

In this Letter, we propose a multiperspective three-dimensional (3D) imaging system using axially distributed stereo image sensing. In this proposed method, the stereo camera is translated along its optical axis and multiple axial elemental image pairs for a 3D scene are collected. The captured elemental images are reconstructed in 3D using a computational reconstruction algorithm based on ray back-projection. The proposed method is applied to partially occluded object visualization. Optical experiments are performed to verify the approach.

Three-dimensional integral imaging with improved visualization using subpixel optical ray sensing.

In this Letter, we propose an improved three-dimensional (3D) image reconstruction method for integral imaging. We use subpixel sensing of the optical rays of the 3D scene projected onto the image sensor. When reconstructing the 3D image, we use a calculated minimum subpixel distance for each sensor pixel instead of the average pixel value of integrated pixels from elemental images. The minimum subpixel distance is defined by measuring the distance between the center of the sensor pixel and the physical position of the imaging lens point spread function onto the sensor, which is projected from each reconstruction point for all elemental images. To show the usefulness of the proposed method, preliminary 3D imaging experiments are presented. Experimental results reveal that the proposed method may improve 3D imaging visualization because of the superior sensing and reconstruction of optical ray direction and intensity information for 3D objects.

Generalization of three-dimensional N-ocular imaging systems under fixed resource constraints.

The performance of multiview three-dimensional imaging systems depends on several factors, including the number of sensors, sensor pixel size, relative sensor configuration, imaging optics, and computational reconstruction algorithm. Therefore, it is important to compare the performance of such systems under equally constrained resources. In this Letter, we develop a unifying framework to evaluate the lateral and axial resolution of N-ocular imaging systems ranging from stereo (two cameras) to multiple sensors (integral imaging) under fixed resource constraints. The proposed framework enables one to evaluate the system performance as a function of sensing parameters such as the number of cameras, the number of pixels, parallax, pixel size, lens aperture, and focal length. We carry out Monte Carlo simulations based on this framework to evaluate system performance as a function of sensing parameters. To the best of our knowledge, this is the first report on quantitative analysis of N-ocular imaging systems under common resource constraints.

Optofluidic system for three-dimensional sensing and identification of micro-organisms with digital holographic microscopy.

Optofluidic devices offer flexibility for a variety of tasks involving biological specimen. We propose a system for three-dimensional (3D) sensing and identification of biological micro-organisms. This system consists of a microfluidic device along with a digital holographic microscope and relevant statistical recognition algorithms. The microfluidic channel is used to house the micro-organisms, while the holographic microscope and a CCD camera record their digital holograms. The holograms can be computationally reconstructed in 3D using a variety of algorithms, such as the Fresnel transform. Statistical recognition algorithms are used to analyze and identify the micro-organisms from the reconstructed wavefront. Experimental results are presented. Because of computational reconstruction of wavefronts in holographic imaging, this technique offers unique advantages that allow one to image micro-organisms within a deep channel while removing the inherent microfluidic-induced aberration through interferometery.

Three-dimensional optical microscopy using axially distributed image sensing.

We propose three-dimensional (3D) optical microscopy using axially distributed image sensing. In the proposed method, the micro-objects are optically magnified and their axially distributed images are recorded by moving the image sensor along a common optical axis. The 3D volumetric images are generated from the recorded axial image set using a computational reconstruction algorithm based on ray backprojection. Preliminary experimental results are presented. To the best of our knowledge, this is the first report on 3D optical microscopy using axially distributed sensing.

Simple correction method of distorted elemental images using surface markers on lenslet array for computational integral imaging reconstruction.

In this paper, we propose a simple correction method of distorted elemental images for computational integral imaging reconstruction (CIIR) method by using surface markers on lenslet array. The position information of surface markers is extracted from distorted elemental images with geometric misalignments such as skew, rotation and so on. Then the elemental images can be corrected simply when applying linear transformation calculated from the extracted positions. Therefore, the proposed method can simply correct geometric misalignments such as skew and rotation. The corrected elemental images can provide the precise reconstruction of 3D plane images in CIIR. To show the usefulness of the proposed method, the preliminary experiments are carried out and the experimental results are presented. To the best of our knowledge, this is the first report to deal with compensating for the distorted elemental images recorded by using computational integral imaging.

Signal model and granular-noise analysis of computational image reconstruction for curved integral imaging systems.

In this paper, we propose an improved analysis on the signal property of curved computational integral imaging reconstruction (C-CIIR). In the proposed model and analysis, we explain a general analysis of computational integral imaging by introducing a curvature effect that is obtained by the additional use of a large-aperture (LA) lens. Based on the proposed signal model in C-CIIR, we analyze the characteristics of the granular noise (GN) and conduct preliminary experiments to show the feasibility of our model. Experimental results indicate that the GN caused by the nonuniform overlapping gets reduced and that the GN is diminished as the focal length of the additional LA lens used decreases in C-CIIR. Also, the proposed model and analysis are considered to be generalized versions of the signal model and analysis of the previous computational integral imaging systems.

Scale-variant magnification for computational integral imaging and its application to 3D object correlator.

In this paper, we present a novel volumetric computational reconstruction (VCR) method for improved 3D object correlator. Basically, VCR consists of magnification and superposition. This paper presents new scale-variant magnification as a technique for VCR. To introduce our technique, we discuss an interference problem among elemental images in VCR. We find that a large magnification causes interference among elemental images when they are applied to the superposition. Thus, the resolution of reconstructed images should be limited by this interference. To overcome the interference problem, we propose a method to calculate a minimum magnification factor while VCR is still valid. Magnification by a new factor enables the proposed method to reconstruct resolution-enhanced images. To confirm the feasibility of the proposed method, we apply our method to a VCR-based 3D object correlator. Experimental results indicate that our method outperforms the conventional VCR method.

Occlusion removal method of partially occluded 3D object using sub-image block matching in computational integral imaging.

In this paper, we propose an occlusion removal method using sub-image block matching for improved recognition of partially occluded 3D objects in computational integral imaging (CII). When 3D plane images are reconstructed in CII, occlusion degrades the resolution of reconstructed images. To overcome this problem, we apply the sub-image transform to elemental image array (EIA) and these sub-images are employed using block matching method for depth estimation. Based on the estimated depth information, we remove the unknown occlusion. After completing the occlusion removal for all sub-images, we obtain the modified EIA without occlusion information through the inverse sub-image transform. Finally, the 3D plane images are reconstructed by using a computational integral imaging reconstruction method with the modified EIA. The proposed method can provide a substantial gain in terms of the visual quality of 3D reconstructed images. To show the usefulness of the proposed method we carry out some experiments and the results are presented.

Resolution-enhanced three-dimensional image reconstruction by use of smart pixel mapping in computational integral imaging.

Computational integral imaging method can digitally provide a series of plane images of three-dimensional (3D) objects. However, the resolution of 3D reconstructed images is dramatically degraded as the distance from the lenslet array increases. In this paper, to overcome this problem, we propose a novel computational integral imaging reconstruction (CIIR) method based on smart pixel mapping (SPM). Since SPM is a computational process in which elemental images recorded at long distances are convertible to ones recorded near lenslet array, this can give us the improved resolution of plane images for 3D objects located at a long distance range from a lenslet array. For the effective use of the SPM-based CIIR method, we design a novel two-stage CIIR method by the combined use of the conventional CIIR and the SPM-based one. The conventional CIIR method is applied over a short distance range, while the SPM-based CIIR is used over a long distance range. We carry out some experiments to verify the performance of the two-stage CIIR system. From the experimental results, the proposed system can provide a substantial gain over a long distance range in terms of the resolution of reconstructed plane images.

Image quality enhancement in 3D computational integral imaging by use of interpolation methods.

In this paper, we propose a computational integral imaging reconstruction (CIIR) method by use of image interpolation algorithms to improve the visual quality of 3D reconstructed images. We investigate the characteristics of the conventional CIIR method along the distance between lenslet and objects. What we observe is that the visual quality of reconstructed images is periodically degraded. The experimentally observed period is half size of the elemental image. To remedy this problem, we focus on the interpolation methods in computational integral imaging. Several interpolation methods are applied to the conventional CIIR method and their performances are analyzed. To objectively evaluate the proposed CIIR method, we introduce an experimental framework for the computational pickup process and the CIIR process using a Gaussian function. We also carry out experiments on real objects to subjectively evaluate the proposed method. Experimental results indicate that our method outperforms the conventional CIIR method. In addition, our method reduces the grid noise that the conventional CIIR method suffers from.

Improved analysis on the signal property of computational integral imaging system.

In this paper, we introduce an improved signal analysis of the computational integral imaging (CII) system having a pickup process of three-dimensional object and a volumetric computational reconstruction (VCR) process. We propose a signal model for the CII system. From the signal model and its analysis, we can define a granular noise caused by the non-uniform overlapping. We also analyze the characteristics of the granular noise. According to our model and analysis, there is a condition that the granular noise cancels out. To show the feasibility of our model, the preliminary experiments are carried out and the result is presented. This is the first time, to our knowledge, that a signal model for the analysis of CII systems is provided.

Depth extraction of three-dimensional objects in space by the computational integral imaging reconstruction technique.

A novel approach to extract the depth data of 3D objects in space by using the computational integral imaging reconstruction (CIIR) technique is proposed. With elemental images of 3D objects captured by the CCD camera through a pinhole array, depth-dependent object images can be reconstructed on the output plane by the CIIR technique. Only the images reconstructed on the output planes where 3D objects were located are clearly focused; so the depth data of 3D objects in space can be extracted by discriminating these focused output images from the others by using an image separation technique. A feasibility test of the proposed CIIR-based depth extraction method is carried out, and its results are discussed as well.

Curved computational integral imaging reconstruction technique for resolution-enhanced display of three-dimensional object images.

A novel curved computational integral imaging reconstruction (C-CIIR) technique for the virtually curved integral imaging (VCII) system is proposed, and its performances are analyzed. In the C-CIIR model, an additional virtual large-aperture lens is included to provide a multidirectional curving effect in the reconstruction process, and its effect is analyzed in detail by using the ABCD matrix. With this method, resolution-enhanced 3D object images can be computationally reconstructed from the picked-up elemental images of the VCII system. To confirm the feasibility of the proposed model, some experiments are carried out. Experiments revealed that the sampling rate in the VCII system could be kept at a maximum value within some range of the distance z, whereas in the conventional integral imaging system it linearly decreased as the distance z increased. It is also shown that resolutions of the object images reconstructed by the C-CIIR method have been significantly improved compared with those of the conventional CIIR method.

Multidirectional curved integral imaging with large depth by additional use of a large-aperture lens.

We propose a curved integral imaging system with large depth achieved by the additional use of a large-aperture lens in a conventional large-depth integral imaging system. The additional large-aperture lens provides a multidirectional curvature effect and improves the viewing angle. The proposed system has a simple structure due to the use of well-fabricated, unmodified flat devices. To calculate the proper elemental images for the proposed system, we explain a modified computer-generated pickup technique based on an ABCD matrix and analyze an effective viewing zone in the proposed system. From experiments, we show that the proposed system has an improved viewing angle of more than 7 degrees compared with conventional integral imaging.