Neural compression for hologram images and videos.

Holographic near-eye displays can deliver high-quality three-dimensional (3D) imagery with focus cues. However, the content resolution required to simultaneously support a wide field of view and a sufficiently large eyebox is enormous. The consequent data storage and streaming overheads pose a big challenge for practical virtual and augmented reality (VR/AR) applications. We present a deep-learning-based method for efficiently compressing complex-valued hologram images and videos. We demonstrate superior performance over the conventional image and video codecs.

Neurofeedback learning for mental practice rather than repetitive practice improves neural pattern consistency and functional network efficiency in the subsequent mental motor execution.

During brain modulation, repeated mental practice may not always result in efficient learning. Particularly, the effectiveness of mental motor practice depends on how well one induces neural activity in a desired state consistently across mental trials, which calls for feedbacks to adjust one's performance. We hypothesized that even a brief experience of neurofeedback learning enhances trial-by-trial neural pattern consistency during subsequent mental motor execution and that this experience would change recruitment of functional connectivity in the motor imagery and default mode networks. To test this hypothesis, we conducted an experiment with two sessions of mental motor practice before and after a neurofeedback training session, in which participants conducted four types of first-person mental motor execution tasks (walking forward, turning left, turning right, and touching a tree). During the neurofeedback training session, in which participants conducted a virtual navigation game, 10 experimental participants received real-time fMRI neuro-feedbacks, while 10 control participants simply repeated the same mental task according to given cues without feedbacks. The experimental group showed significantly higher effects of neuro-feedback training on trial-by-trial consistencies and classification accuracies of activated neural patterns than the control group. Task-performing global node strength and network efficiency were increased in the motor imagery network but decreased in the default mode network only in the experimental group. These results demonstrate that even a brief experience of feedback learning is more effective than simple practice repetitions without evaluation, which was reflected in increased neural pattern consistency and task-dependent functional connectivity during a mental motor execution task.

Compact noise-filtering volume gratings for holographic displays.

A compact noise filtering method for holographic head-mounted displays (HMDs) is proposed. Conventionally, twin, DC and high-order noise from the spatial light modulators is filtered by a spatial stop filter with a 4-f system. Since the 4-f system requires a long optical path length, the noise filtering system usually occupies most of the volume in holographic displays. In the proposed method, only thin angular stop filters (ASFs) are used for noise filtering. The ASFs do not require a bulky 4-f system while performing the same function as the conventional method. The proposed method is verified experimentally. Our study provides an important solution for realization of compact holographic HMDs.

Dual-focal waveguide see-through near-eye display with polarization-dependent lenses.

A waveguide near-eye display (NED) with a dual-focal plane using a polarization-dependent lens device is proposed. The novel optical device is composed of a geometric phase holographic lens, a wave plate, and a circular polarizer, which is operating as a concave lens or a see-through optical window, depending on the polarization state of the input beam. Such property and ultra-thinness of about 1.5 mm can be applied to a combiner-eyepiece lens for augmented reality. This optical device attached to the waveguide provides two depth planes with polarization multiplexing. We have demonstrated that our proof-of-concept system has image planes at infinity and 20 diopters. The devised system can be expected to offer a better immersive experience, compared to a NED system with a single-focal plane.

Viewing angle enhancement of an integral imaging display using Bragg mismatched reconstruction of holographic optical elements.

Holographic-optical-element (HOE)-based integral imaging display can be applied to augmented reality. However, a narrow viewing angle is a bottleneck for commercialization. Here, we propose a method to enhance the viewing angle of the integral imaging display using Bragg mismatched reconstruction of HOEs. The viewing angle of the integral imaging display can be enlarged with two probe waves, which form two different viewing zones. The effect of Bragg mismatched reconstruction is analyzed with simulation and experiment. In order to show feasibility of the proposed method, a display experiment is demonstrated.

Recent progress in see-through three-dimensional displays using holographic optical elements [Invited].

The principles and characteristics of see-through 3D displays are presented. We especially focus on the integral-imaging display system using a holographic optical element (IDHOE), which is able to display 3D images and satisfy the see-through property at the same time. The technique has the advantage of the high transparency and capability of displaying autostereoscopic 3D images. We have analyzed optical properties of IDHOE for both recording and displaying stages. Furthermore, various studies of new applications and system improvements for IDHOE are introduced. Thanks to the characteristics of holographic volume grating, it is possible to implement a full-color lens-array holographic optical element and conjugated reconstruction as well as 2D/3D convertible IDHOE. Studies on the improvements of viewing characteristics including a viewing angle, fill factor, and resolution are also presented. Lastly, essential issues and their possible solutions are discussed as future work.

Multivariate detrending of fMRI signal drifts for real-time multiclass pattern classification.

Signal drift in functional magnetic resonance imaging (fMRI) is an unavoidable artifact that limits classification performance in multi-voxel pattern analysis of fMRI. As conventional methods to reduce signal drift, global demeaning or proportional scaling disregards regional variations of drift, whereas voxel-wise univariate detrending is too sensitive to noisy fluctuations. To overcome these drawbacks, we propose a multivariate real-time detrending method for multiclass classification that involves spatial demeaning at each scan and the recursive detrending of drifts in the classifier outputs driven by a multiclass linear support vector machine. Experiments using binary and multiclass data showed that the linear trend estimation of the classifier output drift for each class (a weighted sum of drifts in the class-specific voxels) was more robust against voxel-wise artifacts that lead to inconsistent spatial patterns and the effect of online processing than voxel-wise detrending. The classification performance of the proposed method was significantly better, especially for multiclass data, than that of voxel-wise linear detrending, global demeaning, and classifier output detrending without demeaning. We concluded that the multivariate approach using classifier output detrending of fMRI signals with spatial demeaning preserves spatial patterns, is less sensitive than conventional methods to sample size, and increases classification performance, which is a useful feature for real-time fMRI classification.

Space bandwidth product enhancement of holographic display using high-order diffraction guided by holographic optical element.

A space bandwidth product (SBP) enhancement method for holographic display using high-order diffraction of a spatial light modulator (SLM) is proposed. Among numerous high order diffraction terms, the plus-minus first and the zeroth are adopted and guided by holographic optical elements (HOEs) to an identical direction with the same intensity. By using a set of electro-shutters synchronized with corresponding order component, the system acts as if three SLMs are tiled in the horizontal direction. To confirm the feasibility of using HOE as the guiding optics for the system, several optical characteristics of the recording material are measured before using them. Furthermore, a computer generated hologram algorithm is proposed for compensating the wavefront distortion caused by use of the HOE. The demonstrated system achieves a three-fold increase in SBP of a single SLM. The results are verified experimentally.

Three-dimensional/two-dimensional convertible projection screen using see-through integral imaging based on holographic optical element.

We propose a 3D/2D convertible screen using a holographic optical element and angular multiplexing method of volume hologram. The proposed screen, named a multiplexed holographic optical element screen (MHOES), is composed of passive optical components, and displaying modes between 3D and 2D modes are converted according to projection directions. In a recording process, the angular multiplexing method by using two reference waves with different incidence angles enables the functions of 3D and 2D screens to be recorded in a single holographic material. Also, in order to avoid the bulky experimental setup due to adopting different projectors for the 3D and 2D modes, the projection part is realized based on a prism. The designed projection part enables the single projector to present 3D on 2D mode, where the 3D and 2D contents are simultaneously displayed in one scene, without active components. The optical characteristics of MHOES are experimentally analyzed, and displaying experiments with a full-color MHOES are presented in order to verify the 3D/2D convertibility and see-through properties.

See-through integral imaging display using a resolution and fill factor-enhanced lens-array holographic optical element.

We report on the development of a high-resolution see-through integral imaging system with a resolution and fill factor-enhanced lens-array holographic optical element (HOE). We propose a procedure for fabricating of a lens pitch controllable lens-array HOE. By controlling the recording plane and performing repetitive recordings process, the lens pitch of the lens-array HOE could be substantially reduced, with a high fill factor and the same numerical aperture compared to the reference lens-array. We demonstrated the feasibility by fabricating a lens-array HOE with a 500 micrometer pitch. Since the pixel pitch of the projected image can be easily controlled in projection type integral imaging, the small lens pitch enhances the quality of the displayed 3D image very effectively. The enhancement of visibility of the 3D images is verified in experimental results.

Two-dimensional and three-dimensional transparent screens based on lens-array holographic optical elements.

Two-dimensional (2D) and three-dimensional (3D) transparent screens can be created using lens-array holographic optical elements (HOEs). Lens-array HOEs can be used to perform 2D and 3D imaging for Bragg matched images while maintaining the transparent properties of the images in the background scenes. 2D or 3D imaging on the proposed screen is determined by the relative size of an elemental-lens on the lens-array to a pixel on the projected image. The 2D and 3D displays on the lens-array HOEs are implemented by the diffusion of light on each elemental-lens and by taking advantage of reflection-type integral imaging, respectively. We constructed an HOE recording setup and recorded two lens-array HOEs having different optical specifications, permitting them to function as 2D and 3D transparent screens. Experiments regarding 2D and 3D imaging on the proposed transparent screens are carried out and the viewing characteristics in both cases are discussed. The experimental results show that the proposed screens are capable of providing 2D and 3D images properly while satisfying the see-through properties.

Solution for pseudoscopic problem in integral imaging using phase-conjugated reconstruction of lens-array holographic optical elements.

We propose an optical pseudoscopic to orthoscopic conversion method for integral imaging using a lens-array holographic optical element (LAHOE), which solves the pseudoscopic problem. The LAHOE reconstructs an array of diverging spherical waves when a probe wave with the phase-conjugated condition is imposed on it, while an array of converging spherical waves is reconstructed in ordinary reconstruction. For given pseudoscopic elemental images, the array of the diverging spherical waves integrates the orthoscopic three-dimensional images without a distortion. The principle of the proposed method is verified by the experiments of displaying the integral imaging on the LAHOE using computer generated and optically acquired elemental images.

Reflection-type integral imaging system using a diffuser holographic optical element.

A reflection-type integral imaging (InIm) system using a diffuser holographic optical element (DHOE) is proposed for improving the fill factor of displayed three-dimensional images. The DHOE performs an optical function similar to that for a conventional diffuser only for Bragg matched light, while Bragg mismatched light passes through the DHOE. Elemental images projected under Bragg matching condition are scattered by the DHOE. Meanwhile, light reflected by a concave mirror-array becomes Bragg mismatched light, and is integrated into three-dimensional images without the fill factor problem. The optical characteristics of the DHOE are examined by measuring diffraction efficiencies, and the feasibility of the fill-factor-improved InIm is verified by a concave mirror-array and DHOE.

Full-color lens-array holographic optical element for three-dimensional optical see-through augmented reality.

A novel system of optical see-through augmented reality (AR) is proposed by making use of a holographic optical element (HOE) with full-color and lens-array functions. The full-color lens-array HOE provides see-through property with three-dimensional (3D) virtual images, for it functions as a conventional lens array only for Bragg-matched lights. An HOE recording setup was built, and it recorded a 30 mm × 60 mm sized full-color lens-array HOE by using the techniques of spatial multiplexing for large-area recording and wavelength multiplexing for full-color imaging. The experimental results confirm that the suggested full-color lens-array HOE can provide the full-color 3D virtual images in the optical see-through AR system.

Waveguide holography for 3D augmented reality glasses.

Near-eye displays are fundamental technology in the next generation computing platforms for augmented reality and virtual reality. However, there are remaining challenges to deliver immersive and comfortable visual experiences to users, such as compact form factor, solving vergence-accommodation conflict, and achieving a high resolution with a large eyebox. Here we show a compact holographic near-eye display concept that combines the advantages of waveguide displays and holographic displays to overcome the challenges towards true 3D holographic augmented reality glasses. By modeling the coherent light interactions and propagation via the waveguide combiner, we demonstrate controlling the output wavefront using a spatial light modulator located at the input coupler side. The proposed method enables 3D holographic displays via exit-pupil expanding waveguide combiners, providing a large software-steerable eyebox. It also offers additional advantages such as resolution enhancement capability by suppressing phase discontinuities caused by pupil replication process. We build prototypes to verify the concept with experimental results and conclude the paper with discussion.

Real-time capturing and 3D visualization method based on integral imaging.

We propose a real-time capturing and 3D visualization method based on integral imaging. We applied real-time conversion algorithm to conventional integral imaging pickup system. Gap control method with depth plane adjustment is also applied to improve image quality. Implemented system provides real-time 3D images with ultra high definition resolution in 20 frames per second, and the observer can change depth planes freely. Simulations and experimental results show the validity of proposed system.

The impact of comorbid anxiety on quantitative EEG heterogeneity in children with attention-deficit/hyperactivity disorder.

Objective: The objective of this study was to compare quantitative electroencephalography (Q-EEG) characteristics of children with Attention-deficit/hyperactivity disorder (ADHD), taking into account the presence of a comorbidity for anxiety disorder. It also sought to investigate the impact of comorbid anxiety on the Q-EEG heterogeneity of children with ADHD. Method: A total of 141 children with ADHD but without comorbid anxiety (ADHD-Only), 25 children with a comorbidity for anxiety disorder (ADHD-ANX) and 43 children in the control group were assessed. To compare Q-EEG characteristics between groups, we performed ANCOVA (Analysis of Covariance) on relative power and theta/beta ratio (TBR) controlling for covariates such as age, sex, and FSIQ. Relative power values from 19 electrodes were averaged for three regions (frontal, central and posterior). Furthermore, cluster analysis (Ward's method) using the squared Euclidian distance was conducted on participants with ADHD to explore the impact of anxiety on the heterogeneity of Q-EEG characteristics in ADHD. Results: There were no significant group differences in cognitive and behavioral measures. However, significant differences between groups were observed in the theta values in the central region, and the beta values in the frontal, central and posterior regions. In post hoc analyses, It was found that the ADHD-ANX group has significantly higher beta power values than the ADHD-Only group in all regions. For the theta/beta ratio, the ADHD-Only group had significantly higher values than the ADHD-ANX group in frontal, central and posterior regions. However, the control group did not show significant differences compared to both the ADHD-Only and ADHD-ANX group. Through clustering analysis, the participants in the ADHD-Only and ADHD-ANX groups were classified into four clusters. The ratios of children with comorbidities for anxiety disorder within each cluster were significantly different (χ2 = 10.018, p = 0.019). Conclusion: Attention-deficit/hyperactivity disorder children with comorbid anxiety disorder showed lower theta power in the central region, higher beta power in all regions and lower TBR in all regions compared to those without comorbid anxiety disorder. The ratios of children with comorbidities for anxiety disorder within each cluster were significantly different.

Foveated near-eye display for mixed reality using liquid crystal photonics.

Foveated near-eye display is one of the most promising approaches to deliver immersive experience of mixed reality. However, it is challenged to conceive a compact optical system. Here, we introduce a method to use polarization optics via liquid crystal photonics to improve the foveated display performance. We demonstrate a benchtop prototype of this idea. We implement and combine two display modules for peripheral and foveal visions. A peripheral display consists of a polarization selective lens (PSL) module, a polarization selective diffuser (PSD), and a slanted projection system. An 80[Formula: see text] diagonal field of view is achieved by on-axis optical configuration of the PSL module and the PSD. A foveal holographic display is composed of a spatial light modulator (SLM), a volume grating lens, and a microelectromechanical system mirror possibly in combination with a switchable polarization selective grating module. The holographic reconstruction using the SLM enables accurate focus cue generation and high resolution above 30 cycles per degree within 15[Formula: see text] by 15[Formula: see text] field of view. We explore and discuss the liquid crystal photonics in the prototype that has a novel optical design using volume gratings with polarization selectivity.

Augmented reality near-eye display using Pancharatnam-Berry phase lenses.

An augmented reality (AR) near-eye display using Pancharatnam-Berry (PB) phase lenses is proposed. PB phase lenses provide different optical effects depending on the polarization state of the incident light. By exploiting this characteristic, it is possible to manufacture an AR combiner with a small form factor and a large numerical aperture value. The AR combiner adopted in the proposed system operates as a convex lens for right-handed circularly polarized light and operates as transparent glass for left-handed circularly polarized light. By merging this combiner with a transparent screen, such as diffuser-holographic optical elements (DHOEs), it is possible to make an AR near-eye display with a small form factor and a wide field of view. In addition, the proposed AR system compensates the chromatic aberration that occurs in PB phase lens by adopting three-layered DHOEs. The operating principle of the proposed system is covered, and its feasibility is verified with experiments and analysis.

Dual-dimensional microscopy: real-time in vivo three-dimensional observation method using high-resolution light-field microscopy and light-field display.

Here, we present dual-dimensional microscopy that captures both two-dimensional (2-D) and light-field images of an in-vivo sample simultaneously, synthesizes an upsampled light-field image in real time, and visualizes it with a computational light-field display system in real time. Compared with conventional light-field microscopy, the additional 2-D image greatly enhances the lateral resolution at the native object plane up to the diffraction limit and compensates for the image degradation at the native object plane. The whole process from capturing to displaying is done in real time with the parallel computation algorithm, which enables the observation of the sample's three-dimensional (3-D) movement and direct interaction with the in-vivo sample. We demonstrate a real-time 3-D interactive experiment with Caenorhabditis elegans.