Learning-based compensation of spatially varying aberrations for holographic display [Invited].

We propose a hologram generation technique to compensate for spatially varying aberrations of holographic displays through machine learning. The image quality of the holographic display is severely degraded when there exist optical aberrations due to misalignment of optical elements or off-axis projection. One of the main advantages of holographic display is that aberrations can be compensated for without additional optical elements. Conventionally, computer-generated holograms for compensation are synthesized through a point-wise integration method, which requires large computational loads. Here, we propose to replace the integration with a combination of fast-Fourier-transform-based convolutions and forward computation of a deep neural network. The point-wise integration method took approximately 95.14 s to generate a hologram of 1024×1024pixels, while the proposed method took about 0.13 s, which corresponds to ×732 computation speed improvement. Furthermore, the aberration compensation by the proposed method is verified through experiments.

Retinal projection type lightguide-based near-eye display with switchable viewpoints.

We present a retinal-projection-based near-eye display with switchable multiple viewpoints by polarization-multiplexing. Active switching of viewpoints is provided by the polarization grating, multiplexed holographic optical elements and polarization-dependent eyepiece lens that can generate one of the dual-divided focus groups according to the pupil position. The lightguide-combined optical devices have a potential to enable a wide field of view (FOV) and short eye relief with compact form factor. Our proposed system can support a pupil movement with an extended eyebox and mitigate image problem caused by duplicated viewpoints. We discuss the optical design for guiding system and demonstrate that proof-of-concept system provides all-in-focus images with 37 degrees FOV and 16 mm eyebox in horizontal direction.

Aberration-corrected full-color holographic augmented reality near-eye display using a Pancharatnam-Berry phase lens.

We present a full-color holographic augmented reality near-eye display using a Pancharatnam-Berry phase lens (PBP lens) and its aberration correction method. Monochromatic and chromatic aberrations of the PBP lens are corrected by utilizing complex wavefront modulation of the holographic display. A hologram calculation method incorporating the phase profile of the PBP lens is proposed to correct the monochromatic aberration. Moreover, the chromatic aberration is corrected by warping the image using the mapping function obtained from ray tracing. The proposed system is demonstrated with the benchtop prototype, and the experimental results show that the proposed system offers 50° field of view full-color holographic images without optical aberrations.

Foveated display system based on a doublet geometric phase lens.

We propose a new concept of a foveated display with a single display module. A multi-resolution and wide field of view (FOV) can be simultaneously achieved using only a single display, based on temporal polarization-multiplexing. The polarization-dependent lens set functions as an optical window or beam expander system depending on the polarization state, which can provide two operating modes: fovea mode for a high-resolution and peripheral mode for a wide viewing angle. By superimposing two-mode images, the proposed system supports a foveated and wide FOV image without an ultra-high-resolution display. We demonstrate the feasibility of the proposed configuration through the proof-of-concept system.

Holographically customized optical combiner for eye-box extended near-eye display.

We propose a customizing method of a holographic optical element (HOE) by using a holographic printer, which extends the eye-box with high field of view (FOV) for a holographic augmented reality near-eye display (AR NED). The holographic printer setup to manufacture HOE is presented and a prototype of the AR NED is implemented. To make a simple AR NED system, we propose a total internal reflection holographic printing method using an index-matching optical frame. As a result, the eye-box of the AR NED is extended in both vertical and horizontal directions and FOV of 50° is achieved at the center of the eye-box. Through the simulations and the experimental results, the feasibility of the proposed method is verified.

Hybrid multi-layer displays providing accommodation cues.

Hybrid multi-layer displays are proposed as the system combines additive light field (LF) displays and multiplicative LF displays. The system is implemented by integrating the multiplicative LF displays with a half mirror to expand the overall depth of field. The hybrid displays are advantageous in that the form factor is competitive with existing additive LF displays with 2 layers implemented by a half mirror and two panels, only half of brightness loss is experienced compared to multiplicative LF displays with 2 layers, and no time-division is required to provide images for multi-layer displays. The images for presentation planes are processed by light field factorization and optimized with the presented algorithm. Retinal images are reconstructed based on various accommodation states and display types to check the accommodation response and utilized to compare the proposed displays with existing displays. With ray tracing method, retinal images generated by the proposed displays can be obtained. To verify the feasibility of the system, a prototype of hybrid multi-layer displays was implemented and display photographs were captured with different accommodation states of camera. With the simulation results and experimental results, this system was confirmed to support accommodation cues in a range of 1.8 diopters.

Viewing zone duplication of multi-projection 3D display system using uniaxial crystal.

We propose a novel multiplexing technique for increasing the viewing zone of a multi-view based multi-projection 3D display system by employing double refraction in uniaxial crystal. When linearly polarized images from projector pass through the uniaxial crystal, two possible optical paths exist according to the polarization states of image. Therefore, the optical paths of the image could be changed, and the viewing zone is shifted in a lateral direction. The polarization modulation of the image from a single projection unit enables us to generate two viewing zones at different positions. For realizing full-color images at each viewing zone, a polarization-based temporal multiplexing technique is adopted with a conventional polarization switching device of liquid crystal (LC) display. Through experiments, a prototype of a ten-view multi-projection 3D display system presenting full-colored view images is implemented by combining five laser scanning projectors, an optically clear calcite (CaCO3) crystal, and an LC polarization rotator. For each time sequence of temporal multiplexing, the luminance distribution of the proposed system is measured and analyzed.

Computational multi-projection display.

A computational multi-projection display is proposed by employing a multi-projection system combining with compressive light field displays. By modulating the intensity of light rays from a spatial light modulator inside a single projector, the proposed system can offer several compact views to observer. Since light rays are spread to all directions, the system can provide flexible positioning of viewpoints without stacking projectors in vertical direction. Also, if the system is constructed properly, it is possible to generate view images with inter-pupillary gap and satisfy the super multi-view condition. We explain the principle of the proposed system and verify its feasibility with simulations and experimental results.

Compact three-dimensional head-mounted display system with Savart plate.

We propose three-dimensional (3D) head-mounted display (HMD) providing multi-focal and wearable functions by using polarization-dependent optical path switching in Savart plate. The multi-focal function is implemented as micro display with high pixel density of 1666 pixels per inches is optically duplicated in longitudinal direction according to the polarization state. The combination of micro display, fast switching polarization rotator and Savart plate retains small form factor suitable for wearable function. The optical aberrations of duplicated panels are investigated by ray tracing according to both wavelength and polarization state. Astigmatism and lateral chromatic aberration of extraordinary wave are compensated by modification of the Savart plate and sub-pixel shifting method, respectively. To verify the feasibility of the proposed system, a prototype of the HMD module for monocular eye is implemented. The module has the compact size of 40 mm by 90 mm by 40 mm and the weight of 131 g with wearable function. The micro display and polarization rotator are synchronized in real-time as 30 Hz and two focal planes are formed at 640 and 900 mm away from eye box, respectively. In experiments, the prototype also provides augmented reality function by combining the optically duplicated panels with a beam splitter. The multi-focal function of the optically duplicated panels without astigmatism and color dispersion compensation is verified. When light field optimization for two additive layers is performed, perspective images are observed, and the integration of real world scene and high quality 3D images is confirmed.

See-through multi-projection three-dimensional display using transparent anisotropic diffuser.

We propose a see-through multi-projection three-dimensional (3D) display using a transparent anisotropic diffuser. By immersing a metal-coated anisotropic diffuser into index matching oil which has the same refractive index of anisotropic diffuser, a transparent anisotropic diffuser is implemented. The reflectance of the transparent anisotropic diffuser is analyzed with the transfer matrix. Two multi-projection methods are proposed based on reflection type integral imaging and multi-view method. Especially, the reflection type multi-view-based system is realized with a curved anisotropic diffuser. High resolution see-through 3D display can be realized with the proposed methods. They can be used in various applications with the two multi-projection methods. In order to show the augmented reality features, real objects and virtual 3D images are presented at the same time in the experimental setup.

Compact multi-projection 3D display system with light-guide projection.

We propose a compact multi-projection based multi-view 3D display system using an optical light-guide, and perform an analysis of the characteristics of the image for distortion compensation via an optically equivalent model of the light-guide. The projected image traveling through the light-guide experiences multiple total internal reflections at the interface. As a result, the projection distance in the horizontal direction is effectively reduced to the thickness of the light-guide, and the projection part of the multi-projection based multi-view 3D display system is minimized. In addition, we deduce an equivalent model of such a light-guide to simplify the analysis of the image distortion in the light-guide. From the equivalent model, the focus of the image is adjusted, and pre-distorted images for each projection unit are calculated by two-step image rectification in air and the material. The distortion-compensated view images are represented on the exit surface of the light-guide when the light-guide is located in the intended position. Viewing zones are generated by combining the light-guide projection system, a vertical diffuser, and a Fresnel lens. The feasibility of the proposed method is experimentally verified and a ten-view 3D display system with a minimized structure is implemented.

Diffraction-engineered holography: Beyond the depth representation limit of holographic displays.

Holography is one of the most prominent approaches to realize true-to-life reconstructions of objects. However, owing to the limited resolution of spatial light modulators compared to static holograms, reconstructed objects exhibit various coherent properties, such as content-dependent defocus blur and interference-induced noise. The coherent properties severely distort depth perception, the core of holographic displays to realize 3D scenes beyond 2D displays. Here, we propose a hologram that imitates defocus blur of incoherent light by engineering diffracted pattern of coherent light with adopting multi-plane holography, thereby offering real world-like defocus blur and photorealistic reconstruction. The proposed hologram is synthesized by optimizing a wave field to reconstruct numerous varifocal images after propagating the corresponding focal distances where the varifocal images are rendered using a physically-based renderer. Moreover, to reduce the computational costs associated with rendering and optimizing, we also demonstrate a network-based synthetic method that requires only an RGB-D image.

Augmented reality near-eye display using Pancharatnam-Berry phase lenses.

An augmented reality (AR) near-eye display using Pancharatnam-Berry (PB) phase lenses is proposed. PB phase lenses provide different optical effects depending on the polarization state of the incident light. By exploiting this characteristic, it is possible to manufacture an AR combiner with a small form factor and a large numerical aperture value. The AR combiner adopted in the proposed system operates as a convex lens for right-handed circularly polarized light and operates as transparent glass for left-handed circularly polarized light. By merging this combiner with a transparent screen, such as diffuser-holographic optical elements (DHOEs), it is possible to make an AR near-eye display with a small form factor and a wide field of view. In addition, the proposed AR system compensates the chromatic aberration that occurs in PB phase lens by adopting three-layered DHOEs. The operating principle of the proposed system is covered, and its feasibility is verified with experiments and analysis.

Dual-dimensional microscopy: real-time in vivo three-dimensional observation method using high-resolution light-field microscopy and light-field display.

Here, we present dual-dimensional microscopy that captures both two-dimensional (2-D) and light-field images of an in-vivo sample simultaneously, synthesizes an upsampled light-field image in real time, and visualizes it with a computational light-field display system in real time. Compared with conventional light-field microscopy, the additional 2-D image greatly enhances the lateral resolution at the native object plane up to the diffraction limit and compensates for the image degradation at the native object plane. The whole process from capturing to displaying is done in real time with the parallel computation algorithm, which enables the observation of the sample's three-dimensional (3-D) movement and direct interaction with the in-vivo sample. We demonstrate a real-time 3-D interactive experiment with Caenorhabditis elegans.

Metasurface eyepiece for augmented reality.

Recently, metasurfaces composed of artificially fabricated subwavelength structures have shown remarkable potential for the manipulation of light with unprecedented functionality. Here, we first demonstrate a metasurface application to realize a compact near-eye display system for augmented reality with a wide field of view. A key component is a see-through metalens with an anisotropic response, a high numerical aperture with a large aperture, and broadband characteristics. By virtue of these high-performance features, the metalens can overcome the existing bottleneck imposed by the narrow field of view and bulkiness of current systems, which hinders their usability and further development. Experimental demonstrations with a nanoimprinted large-area see-through metalens are reported, showing full-color imaging with a wide field of view and feasibility of mass production. This work on novel metasurface applications shows great potential for the development of optical display systems for future consumer electronics and computer vision applications.