An Encoder-Decoder Architecture within a Classical Signal-Processing Framework for Real-Time Barcode Segmentation.

In this work, two methods are proposed for solving the problem of one-dimensional barcode segmentation in images, with an emphasis on augmented reality (AR) applications. These methods take the partial discrete Radon transform as a building block. The first proposed method uses overlapping tiles for obtaining good angle precision while maintaining good spatial precision. The second one uses an encoder-decoder structure inspired by state-of-the-art convolutional neural networks for segmentation while maintaining a classical processing framework, thus not requiring training. It is shown that the second method's processing time is lower than the video acquisition time with a 1024 × 1024 input on a CPU, which had not been previously achieved. The accuracy it obtained on datasets widely used by the scientific community was almost on par with that obtained using the most-recent state-of-the-art methods using deep learning. Beyond the challenges of those datasets, the method proposed is particularly well suited to image sequences taken with short exposure and exhibiting motion blur and lens blur, which are expected in a real-world AR scenario. Two implementations of the proposed methods are made available to the scientific community: one for easy prototyping and one optimised for parallel implementation, which can be run on desktop and mobile phone CPUs.

Evaluation of eye response using a wearable display with automatic interpupillary distance adjustment.

In this study, we introduce a design for a near-eye, wearable display (HMD: head mounted display) that can automatically control the user's interpupillary distance (IPD). In addition, we demonstrate a test-bed module for the wearable AR display based on proposed design. Both the adjustment accuracy and the viewing effect through distance matching between the user's eyes are evaluated by the user's experience in actual wearing of the module. We demonstrate that the distance between the left and right eye pupils can be measured and adjusted using a set of IR camera sensors and a micro-actuator module that we proposed. A half-mirror unit to be mechanically controlled for each eye is designed to combine the image displayed from the projector and an image taken by the IR camera, leading to fine adjustment of the user's IPD. A set of images taken by the IR camera sensors is image-processed in real time to determine each pupil's position with high accuracy under infrared light illumination. Based on the measured information, a micro-actuator module we fabricated for the test bed can automatically adjust the binocular distance to fit each viewer's IPD. The maximum movement distance of each micro-actuator motor is ±10 mm with precision control of at least 0.5 mm. It takes about 18 seconds to calculate the user's IPD from two IR photographs and then to accurately adjust the actual binocular distance of the module that the participant wears. Using the demonstrated test bed, a total of 50 subjects participated to confirm the accuracy in the automatic IPD adjustment with an error of 0.25% as well as the improvement of the displayed image quality and 3D immersive experience.

Moiré patterns in non-parallel surfaces such as wedge.

The moiré effect in 3D objects with planar facets is considered. The projected period of the inclined periodic grating was found. The formula for the period of the moiré patterns in inclined plain surfaces was obtained for objects with arbitrary oriented plain facets, namely, the parallelepiped and the prism (parallel and non-parallel facets). The similarity between the projected period and the moiré period was demonstrated. The direction to the longest moiré pattern in the wedge was found theoretically and observed in experiments. The results can be used in the alignment of flat surfaces.

Hand-held aperture-sharing camera for a multiview image set.

A hand-held camera that is capable of acquiring full parallax multiview images is introduced. Since the camera is based on spatial sharing of the aperture stop of a digital camera, it is easy to manufacture. The aperture stop is spatially shared by a 2D high-speed liquid crystal (LC) shutter array that is fixed on the front surface of the camera objective. The shutter array consists of 8×8 LC shutters to acquire 8×8 multiview images. The equivalent optical configuration of the camera is not different from that of a parallel-type multiview camera array. Each of the multiview images has an equal disparity from its neighbors and has the full detector resolution of the camera. The disparity is small enough to be used in light field imaging. The depth resolution and the resolvable number of depth layers calculated from the configuration are not different from those obtained experimentally.

Electro-holographic display using a ZBLAN glass as the image space.

An Er3+-doped ZBLAN glass is used to display a 360° viewable reconstructed image from a hologram on a DMD. The reconstructed image, when the hologram is illuminated by a 852 nm wavelength laser beam, is situated at the inside of the glass, and then a 1530 nm wavelength laser beam is crossed through the image to light it with an upconversion green light, which is viewable at all surrounding directions. This enables us to eliminate the limitation of the viewing zone angle imposed by the finite size of pixels in electro-holographic displays based on digital display chips/panels. The amount of the green light is much higher than that known previously. This is partly caused by the upconversion luminescence induced by 852 and 1530 nm laser beams.

Fringe periods of color moirés in contact-type 3-D displays.

A mathematical formula of calculating the fringe periods of the color moirés appearing at the contact-type 3-D displays is derived. It is typical that the color moirés are chirped and the period of the line pattern in viewing zone forming optics is more than two times of that of the pixel pattern in the display panel. These make impossible to calculate the fringe periods of the color moirés with the conventional beat frequency formula. The derived formula work very well for any combination of two line patterns having either a same line period or different line periods. This is experimentally proved. Furthermore, it is also shown that the fringe period can be expressed in terms of the viewing distance and focal length of the viewing zone forming optics.

The effects of MPE on electro-holographic displays.

The effects of MPE (maximum permissible exposure) on the characteristics of electro-holographic displays are analyzed. The main effect is the reduction in the spectral range to be presented by the displays. The range will be reduced more as the pixel and hologram sizes of the displays become smaller and larger, respectively. The spectral range for the hologram size 0.5m and pixel size 0.8 µm will be about 50 nm less than that of the visible. In addition to the spectral range, the reconstructed image size should not be smaller than the value defined by 0.1 times of the diffracted intensity of the image to meet the MPE requirement for eye.

A floating type holographic display.

A floating image type holographic display which projects an electronically generated holographic image together with a background image displayed on a monitor/TV to enhance the visual effects of the former image is introduced. This display can display a holographic image with a spatial volume floating in the front space of the display with use of PDLC sheets as the focused plane of the image. This display can preserve and enhance the main property of holographic image from a display chip, i.e., a spatial image with a volume. This property had not been appealed by the previous holographic displays due to the much brighter active surface image accompanied with the reconstructed image and the diffuser used for viewing the image.

Holographic display based on a spatial DMD array.

The image space of the reconstructed image from the hologram displayed on a digital micromirror device (DMD) is defined by the diffraction pattern induced by the 2D pixel pattern of the DMD, which works as a 2D blazed grating. Within this space, a reconstructed image of 100 mm × 20 mm is spatially multiplexed by a 2 × 5 DMD array that is aligned on a board, without using any extra optics. Each DMD chip reconstructs an image piece of the size 20 mm (width) × 10 mm (height). The reconstructed image looks somewhat noisy but regenerates the original object image faithfully.

Depth resolution in three-dimensional images.

The depth of field of a camera defines the depth range to be covered by the camera. In 3D images, the resolvable depth range is also determined by the depth of field (DOF). Hence the depth resolution and resolvable number of depth layers obtainable with a given 3D display will be defined within the DOF when the display has the same resolution as the total camera resolution of the array in the horizontal direction. The depth resolution and resolvable number of depth layers are mathematically derived in terms of the circle of confusion. The resolvable number of depth layers is approximately linearly proportional to the camera distance and inversely proportional to the aperture diameter of the camera objective. The accuracies of the derivations are examined experimentally. The results show that the DOF extends slightly and the depth resolution improves up to 20% more than that predicted by theory for the given experimental condition. This means that the depth resolution derived has more than 80% accuracy.

Impacts of residential environment on residents' place attachment, satisfaction, WOM, and pro-environmental behavior: evidence from the Korean housing industry.

This study considers seven residential environment elements and examines their effect on residents' place attachment (place dependence and place identity), satisfaction, word-of-mouth behavior, and pro-environmental behavior. The study also examines whether gender moderates the proposed relationships. The data were collected from 603 respondents who owned a condominium in Seoul, South Korea. We analyzed the data using structural equation modeling with SmartPLS 4. The finding shows that all seven elements of the residential environment have a significant impact on either dimension of place attachment, except for the insignificant effect of social environment on place dependence. Both dimensions of place attachment have a significant effect on satisfaction, WOM, and pro-environmental behavior except for the insignificant effect of place dependence on pro-environmental behavior. The interaction effect test of gender shows that males consider eco-friendly materials and green/recreational areas more than females. On the other hand, females are found to weigh and social environments more heavily than males. The finding shows that pro-environmental behavior is influenced by place identity (not by place dependence) and satisfaction, indicating a key role of affective response.

Wavefront Characteristics of a Digital Holographic Optical Element.

In this study, a 50 × 50 mm holographic optical element (HOE) with the property of a spherical mirror was recorded digitally on a silver halide photoplate using a wavefront printing method. It consisted of 51 × 96 hologram spots with each spot measuring 0.98 × 0.52 mm. The wavefronts and optical performance of the HOE were compared with those of reconstructed images from a point hologram displayed on DMDs of different pixel structures. The same comparison was also performed with an analog-type HOE for a heads-up display and with a spherical mirror. A Shack-Hartmann wavefront sensor was used to measure the wavefronts of the diffracted beams from the digital HOE and the holograms as well as the reflected beam from the analog HOE and the mirror when a collimated beam was incident on them. These comparisons revealed that the digital HOE could perform as a spherical mirror, but they also revealed astigmatism-as in the reconstructed images from the holograms on DMDs-and that its focusability was worse than that of the analog HOE and the spherical mirror. A phase map, i.e., the polar coordinate-type presentation of the wavefront, could visualize the wavefront distortions more clearly than the reconstructed wavefronts obtained using Zernike polynomials. The phase map revealed that the wavefront of the digital HOE was more distorted than those of the analog HOE and the spherical mirror.

Characteristics of composite images in multiview imaging and integral photography.

The compositions of images projected to a viewer's eyes from the various viewing regions of the viewing zone formed in one-dimensional integral photography (IP) and multiview imaging (MV) are identified. These compositions indicate that they are made up of pieces from different view images. Comparisons of the composite images with images composited at various regions of imaging space formed by camera arrays for multiview image acquisition reveal that the composite images do not involve any scene folding in the central viewing zone for either MV or IP. However, in the IP case, compositions from neighboring viewing regions aligned in the horizontal direction have reversed disparities, but in the viewing regions between the central and side viewing zones, no reversed disparities are expected. However, MV does exhibit them.

Real-time outdoor concealed-object detection with passive millimeter wave imaging.

Millimeter wave imaging is finding rapid adoption in security applications such as the detection of objects concealed under clothing. A passive imaging system can be realized as a stand-off type sensor that can operate in open spaces, both indoors and outdoors. In this paper, we address real-time outdoor concealed-object detection and segmentation with a radiometric imaging system operating in the W-band. The imaging system is equipped with a dielectric lens and a receiver array operating at around 94 GHz. Images are analyzed by multilevel segmentation to identify a concealed object. Each level of segmentation comprises vector quantization, expectation-maximization, and Bayesian decision making to cluster pixels on the basis of a Gaussian mixture model. In addition, we describe a faster process that adopts only vector quantization for the first level segmentation. Experiments confirm that the proposed methods provide fast and reliable detection and segmentation for a moving human subject carrying a concealed gun.

Characteristics of stereo images from detectors in focal plane array.

The equivalent ray geometry of two horizontally aligned detectors at the focal plane of the main antenna in a millimeter wave imaging system is analyzed to reveal the reason why the images from the detectors are fused as an image with a depth sense. Scanning the main antenna in both horizontal and vertical directions makes each detector perform as a camera, and the two detectors can work like a stereo camera in the millimeter wave range. However, the stereo camera geometry is different from that of the stereo camera used in the visual spectral range because the detectors' viewing directions are diverging to each other and they are a certain distance apart. The depth sense is mainly induced by the distance between detectors. The images obtained from the detectors in the millimeter imaging system are perceived with a good depth sense. The disparities responsible for the depth sense are identified in the images.

Automatic image segmentation for concealed object detection using the expectation-maximization algorithm.

We address an image segmentation method to detect concealed objects captured by passive millimeter wave (MMW) imaging. Passive MMW imaging can create interpretable imagery on the objects concealed under clothing, which gives the great advantage to the security system. In this paper, we propose the multi-level expectation maximization (EM) method to separate the concealed objects from the other area in the image. We apply the EM method to obtain a Gaussian mixture model (GMM) of the acquired image. In the experiments, we evaluate the performance by the average probability of error. We will show that the consecutive EM processes separates the object area more accurately than the conventional EM method.

Static moiré patterns in moving grids.

We describe an optical phenomenon of unmovable moiré patterns in sliding (moving) grids and gratings. The phenomenon was observed visually in the planar straight movement of the black-and-white gratings with a period of several mm. This is a velocity-independent effect confirmed analytically and in a computer simulation based on the spatial averaging. We found the static directions of the moiré patterns in the regular grids, but our technique can be also applied to other objects. The orientation and period of the static moiré patterns are not obvious, especially in the presence of the distance effect. The phenomenon can be practically used in security applications.

Stereoscopic imaging and display for a 3-D mobile phone.

There is growing interest in imaging and display of stereoscopic images on a mobile phone. Mobile phone users can capture a stereo image pair by taking two pictures, one after another, with a single camera at the left and right eye positions. Such a handheld setup makes it difficult to take the stereo image pair at the exact positions. In this respect, although either radial (toed-in) or parallel arrangements can be used, the former is better suited for a short distance subject, such as a self-portrait, because a single reference point can be placed at the center of two pictures. We describe the optimum distance of camera shifting in the horizontal direction, allowable differences of the radial arrangement for taking stereo images at a short distance, and the alignment procedure of viewing zone forming optics on the display panel through experiments and human subjective evaluation.

Evaluation of the monocular depth cue in 3D displays.

Binocular disparity and monocular depth information are the principal functions of ideal 3D displays. 3D display systems such as stereoscopic or multi-view, super multi-view (SMV), and multi-focus (MF) displays were considered for the testing of the satisfaction level with the monocular accommodation of three different depths of 3D object points. The numerical simulation and experimental results show that the MF 3D display gives a monocular depth cue. In addition, the experimental results of the monocular MF 3D display show clear monocular focus on four different depths. Therefore, we can apply the MF 3D display to monocular 3D displays.