Hologram classification of occluded and deformable objects with speckle noise contamination by deep learning.

Advancements in optical, computing, and electronic technologies have enabled holograms of physical three-dimensional (3D) objects to be captured. The hologram can be displayed with a spatial light modulator to reconstruct a visible image. Although holography is an ideal solution for recording 3D images, a hologram comprises high-frequency fringe patterns that are almost impossible to recognize with traditional computer vision methods. Recently, it has been shown that holograms can be classified with deep learning based on convolution neural networks. However, the method can only achieve a high success classification rate if the image represented in the hologram is without speckle noise and occlusion. Minor occlusion of the image generally leads to a substantial drop in the success rate. This paper proposes a method known as ensemble deep-learning invariant occluded hologram classification to overcome this problem. The proposed new method attains over 95% accuracy in the classification of holograms of partially occluded handwritten numbers contaminated with speckle noise. To achieve the performance, a new augmentation scheme and a new enhanced ensemble structure are necessary. The new augmentation process includes occluded objects and simulates the worst-case scenario of speckle noise.

Optimal sampled phase-only hologram (OSPOH).

A sampled phase-only hologram (SPOH) is the phase component of the hologram of an object image with pixels being sampled with a periodic grid-cross pattern. The reconstructed image of a SPOH is a sparse image with abundant empty voids and degradation in sharpness and contrast. In this paper we proposed a method based on a new sampling scheme, together with stochastic binary search (SBS), to obtain an optimal sampling lattice that can be applied to generate phase-only holograms with enhanced reconstructed image. Experimental results show that with our proposed method, the fidelity and quality of the reconstructed image are increased.

Ensemble convolutional neural network for classifying holograms of deformable objects.

Recently, a method known as "ensemble deep learning invariant hologram classification" (EDL-IHC) for classifying of holograms of deformable objects with deep learning network (DLN) has been demonstrated. However DL-IHC requires substantial computational resources to attain near perfect success rate (≥99%). In practice, it is always desirable to have higher success rate with a low complexity DLN. In this paper we propose a low complexity DLN known as "ensemble deep learning invariant hologram classification" (EDL-IHC). In comparison with DL-IHC, our proposed hologram classifier has promoted the success rate by 2.86% in the classification of holograms of handwritten numerals.

Generation of patterned-phase-only holograms (PPOHs).

A fast and non-iterative method for generating a phase-only hologram, hereafter referred to as the patterned-phase-only hologram (PPOH), is reported in this paper. Briefly, a phase mask with a periodic phase pattern is added to the source image, and converted into a hologram. Subsequently, only the phase component is retained as a phase-only hologram. Experimental evaluation reveals that the visual quality of the reconstructed images of the PPOH generated with our proposed method is favorable, and superior to that obtained with existing methods.

Generation of complementary sampled phase-only holograms.

If an image is uniformly down-sampled into a sparse form and converted into a hologram, the phase component alone will be adequate to reconstruct the image. However, the appearance of the reconstructed image is degraded with numerous empty holes. In this paper, we present a low complexity and non-iterative solution to this problem. Briefly, two phase-only holograms are generated for an image, each based on a different down-sampling lattice. Subsequently, the holograms are displayed alternately at high frame rate. The reconstructed images of the 2 holograms will appear to be a single, densely sampled image with enhance visual quality.

Nonlinearity compensation and complex-to-phase conversion of complex incoherent digital holograms for optical reconstruction.

Incoherent digital holography (IDH) can be realized by optical scanning holography or self-interference incoherent holography. Although IDH can exhibit high quality reconstruction due to its inherently speckle-free property, direct display of an incoherent hologram is a challenge because of its amplitude nonlinearity and the demand of complex modulation. In this paper we propose to compensate the amplitude nonlinearity at the object plane, and use bidirectional error-diffusion method to convert the complex-type incoherent Fresnel hologram to a phase-only Fresnel hologram for display. A spatial light modulator is used to reconstruct the phase-only hologram optically to demonstrate the validity of our proposed method.

Fast generation of digital holograms based on warping of the wavefront recording plane.

This paper reports a fast method for generating a 2048x2048 digital Fresnel hologram at a rate of over 100 frames per second. Briefly, the object wave of an image is nonuniformally sampled and generated on a wavefront recording plane (WPR) that is close to the object scene. The sampling interval at each point on the WRP image is then modulated according to the depth map. Subsequently, the WRP image is converted into a hologram. The hologram generated with our proposed method, which is referred to as the warped WRP (WWRP) hologram, is capable of presenting a 3-D object with faster speed as compared with existing methods.

Generation of phase-only Fresnel hologram based on down-sampling.

We present a novel non-iterative method for generating phase-only Fresnel holograms. The intensity image of the source object scene is first down-sampled with uniform grid-cross lattices. A Fresnel hologram is then generated from the intensity and the depth information of the sampled object points. Subsequently, only the phase component of the hologram is preserved, resulting in a pure phase hologram that we call the sampled-phase-only hologram (SPOH). Experimental evaluation reveals that the numerical, as well as the optical reconstructed images of the proposed phase-only hologram derived with our method are of high visual quality. Moreover, the reconstructed optical image is brighter, and less affected by phase noise contamination on the hologram as compared with those generated with existing error-diffusion approaches.

Fast conversion of digital Fresnel hologram to phase-only hologram based on localized error diffusion and redistribution.

Past research has demonstrated that a digital, complex Fresnel hologram can be converted into a phase-only hologram with the use of the bi-direction error diffusion (BERD) algorithm. However, the recursive nature error diffusion process is lengthy and increases monotonically with hologram size. In this paper, we propose a method to overcome this problem. Briefly, each row of a hologram is partitioned into short non-overlapping segments, and a localized error diffusion algorithm is applied to convert the pixels in each segment into phase only values. Subsequently, the error signal is redistributed with low-pass filtering. As the operation on each segment is independent of others, the conversion process can be conducted at high speed with the graphic processing unit. The hologram obtained with the proposed method, known as the Localized Error Diffusion and Redistribution (LERDR) hologram, is over two orders of magnitude faster than that obtained by BERD for a 2048×2048 hologram, exceeding the capability of generating quality phase-only holograms in video rate.

Review of holographic-based three-dimensional object recognition techniques [invited].

With the advancement of computing and optical technologies, it is now possible to capture digital holograms of real-life object scenes. Theoretically, through the analysis of a hologram, the three-dimensional (3D) objects coded on the hologram can be identified. However, being different from an optical image, a hologram is composed of complicated fringes that cannot be analyzed easily with traditional computer vision methods. Over the years, numerous important research investigations have been attempted to provide viable solutions to this problem. The aim of this work is three-fold. First, we provide a quick walkthrough on the overall development of holographic-based 3D object recognition (H3DOR) in the past five decades, from film-based approaches to digital-based innovation. Second, we describe in more detail a number of selected H3DOR methods that are introduced at different timelines, starting from the late sixties and then from the seventies, where viable digital holographic-based 3D recognition methods began to emerge. Finally, we present our work on digital holographic, pose-invariant 3D object recognition that is based on a recently introduced virtual diffraction plane framework. As our method has not been reported elsewhere, we have included some experimental results to demonstrate the feasibility of the approach.

Displaying a high-resolution digital hologram on a low-resolution spatial light modulator with the same resolution obtained from the hologram.

In this paper, a fast method for displaying a digital, real and off-axis Fresnel hologram on a lower resolution device is reported. Preserving the original resolution of the hologram upon display is one of the important attributes of the proposed method. Our method can be divided into 3 stages. First, a digital hologram representing a given three dimensional (3D) object is down-sampled based on a fix, jitter down-sampling lattice. Second, the down-sampled hologram is interpolated, through pixel duplication, into a low resolution hologram that can be displayed with a low-resolution spatial light modulator (SLM). Third, the SLM is overlaid with a grating which is generated based on the same jitter down-sampling lattice that samples the hologram. The integration of the grating and the low-resolution hologram results in, to a good approximation, the resolution of the original hologram. As such, our proposed method enables digital holograms to be displayed with lower resolution SLMs, paving the way for the development of low-cost holographic video display.

Novel method for converting digital Fresnel hologram to phase-only hologram based on bidirectional error diffusion.

We report a novel and fast method for converting a digital, complex Fresnel hologram into a phase-only hologram. Briefly, the pixels in the complex hologram are scanned sequentially in a row by row manner. The odd and even rows are scanned from opposite directions, constituting to a bidirectional error diffusion process. The magnitude of each visited pixel is forced to be a constant value, while preserving the exact phase value. The resulting error is diffused to the neighboring pixels that have not been visited before. The resulting novel phase-only hologram is called the bidirectional error diffusion (BERD) hologram. The reconstructed image from the BERD hologram exhibits high fidelity as compared with those obtained with the original complex hologram.

Comparison of adaptive optical scanning holography based on new evaluation methods.

Adaptive Optical Scanning Holography (AOSH) represents a powerful technique that employs an adaptive approach to selectively omit certain lines within holograms, guided by the utilization of Normalized-Mean-Error (NME) as a predictive measure. This approach effectively diminishes scanning time and conserves the storage space required for data preservation. However, there exists alternative methods superior to NME in terms of evaluating the model's efficacy. This paper introduces two novel methods, namely Normalized-Root-Mean-Square-Error (NRMSE) and Normalized-Mean-Square-Error (NMSE), into the AOSH system, leading to the development of NRMSE-AOSH and NMSE-AOSH. These new systems aim to further minimize duration of holographic recording. Through a comparative analysis of hologram lines between the two newly proposed AOSH systems and the original AOSH, we demonstrate that both NRMSE-AOSH and NMSE-AOSH effectively reduce the number of hologram lines while maintaining the hologram's informational content. Among the three methods, our two new methods exhibit better performance compared with the original method.

Enhancing the pictorial content of digital holograms at 100 frames per second.

We report a low complexity, non-iterative method for enhancing the sharpness, brightness, and contrast of the pictorial content that is recorded in a digital hologram, without the need of re-generating the latter from the original object scene. In our proposed method, the hologram is first back-projected to a 2-D virtual diffraction plane (VDP) which is located at close proximity to the original object points. Next the field distribution on the VDP, which shares similar optical properties as the object scene, is enhanced. Subsequently, the processed VDP is expanded into a full hologram. We demonstrate two types of enhancement: a modified histogram equalization to improve the brightness and contrast, and localized high-boost-filtering (LHBF) to increase the sharpness. Experiment results have demonstrated that our proposed method is capable of enhancing a 2048x2048 hologram at a rate of around 100 frames per second. To the best of our knowledge, this is the first time real-time image enhancement is considered in the context of digital holography.

Real-time relighting of digital holograms based on wavefront recording plane method.

Relighting is an important technique in photography which enables the optical properties of a picture to be modified without retaking it again. However, different from an optical image, a digital hologram cannot be relit by simply varying the value of individual pixel, as each of them is representing holistic information of the entire object scene. In this paper, we propose a fast method for the relighting of a digital hologram. First, the latter is projected to a virtual wavefront recording plane (WRP) that is located close to the object scene. Next, the WRP is relit, and subsequently expanded into a full hologram. Experiment results have demonstrated that our proposed method is capable of relighting a 2048x2048 hologram at a rate of over 50 frames per second. To the best of our knowledge, this is the first time relighting is considered in the context of holography.

Binary mask programmable hologram.

We report, for the first time, the concept and generation of a novel Fresnel hologram called the digital binary mask programmable hologram (BMPH). A BMPH is comprised of a static, high resolution binary grating that is overlaid with a lower resolution binary mask. The reconstructed image of the BMPH can be programmed to approximate a target image (including both intensity and depth information) by configuring the pattern of the binary mask with a simple genetic algorithm (SGA). As the low resolution binary mask can be realized with less stringent display technology, our method enables the development of simple and economical holographic video display.

A new in vivo model for luteolysis using systemic pulsatile infusions of PGF(2α).

A new in vivo model for studying luteolysis was developed in sheep to provide a convenient method for collecting corpora lutea for molecular, biochemical, and histological analysis during a procedure that mimics natural luteolysis. It was found that the infusion of prostaglandin F(2α) (PGF(2α)) at 20 µg/min/h into the systemic circulation during the mid luteal phase of the cycle allowed sufficient PGF(2α) to escape across the lungs and thus mimic the transient 40% decline in the concentration of progesterone in peripheral plasma seen at the onset of natural luteolysis in sheep. Additional 1h-long systemic infusions of PGF(2α), given at physiological intervals, indicated that two infusions were not sufficient to induce luteolysis. However, an early onset of luteolysis and estrus was induced in one out of three sheep with three infusions, two out of three sheep with four infusions, and three out of three sheep with five infusions. Reducing the duration of each systemic infusion of PGF(2α) from 1h to 30 min failed to induce luteolysis and estrus even after six systemic infusions indicating that, not only are the amplitude and frequency of PGF(2α) pulses essential for luteolysis, but the actual duration of each pulse is also critical. We conclude that a minimum of five systemic pulses of PGF(2α), given in an appropriate amount and at a physiological frequency and duration, are required to mimic luteolysis consistently in all sheep. The five pulse regimen thus provides a new accurate in vivo model for studying molecular mechanisms of luteolysis.

Reassessment of spiny dogfish Squalus acanthias age and growth using vertebrae and dorsal-fin spines.

Male and female spiny dogfish Squalus acanthias were collected in the western North Atlantic Ocean in the Gulf of Maine between July 2006 and June 2009. Squalus acanthias ranged from 25 to 102 cm stretch total length and were caught during all months of the year except January. Age estimates derived from banding patterns visible in both the vertebrae and second dorsal-fin spines were compared. Vertebral growth increments were visualized using a modified histological staining technique, which was verified as appropriate for obtaining age estimates. Marginal increment analysis of vertebrae verified the increment periodicity, suggesting annual band deposition. Based on increased precision and accuracy of age estimates, as well as more biologically realistic parameters generated in growth models, the current study found that vertebrae provided a more reliable and accurate means of estimating age in S. acanthias than the second dorsal-fin spine. Age estimates obtained from vertebrae ranged from <1 year-old to 17 years for male and 24 years for female S. acanthias. The two-parameter von Bertalanffy growth model fit to vertebrae-derived age estimates produced parameters of L∞ = 94·23 cm and k = 0·11 for males and L∞ = 100·76 cm and k = 0·12 for females. While these growth parameters differed from those previously reported for S. acanthias in the western North Atlantic Ocean, the causes of such differences were beyond the scope of the current study and remain to be determined.

Visualization of fine-scale genomic structure by oligonucleotide-based high-resolution FISH.

The discovery of complex structural variations that exist within individual genomes has prompted a need to visualize chromosomes at a higher resolution than previously possible. To address this concern, we established a robust, high-resolution fluorescence in situ hybridization (FISH) method that utilizes probes derived from high complexity libraries of long oligonucleotides (>150 mers) synthesized in massively parallel reactions. In silico selected oligonucleotides, targeted to only the most informative elements in 18 genomic regions of interest, eliminated the need for suppressive hybridization reagents. Because of the inherent flexibility in our probe design methods, we readily visualized regions as small as 6.7 kb with high specificity on human metaphase chromosomes, resulting in an overall success rate of 94%. Two-color FISH over a 479-kb duplication, initially reported as being identical in 2 individuals, revealed distinct 2-color patterns representing direct and inverted duplicons, demonstrating that visualization by high-resolution FISH provides further insight in the fine-scale complexity of genomic structures. The ability to design FISH probes for any sequenced genome along with the ease, reproducibility, and high level of accuracy of this technique suggests that it will be powerful for routine analysis of previously difficult genomic regions and structures.

Fast reconstruction of sectional images in digital holography.

Past research has demonstrated that a three-dimensional object scene can be converted into a digital hologram. Subsequently, the object scene can be reconstructed from the hologram with an iterative blind sectional image reconstruction (BSIR) method. However, the computation is extremely intensive, and escalated with the size of holograms. To overcome this problem, we propose a fast BSIR method that reconstructs sectional images with less out-of-focus haze. While the technique proposed here is applicable in general to holography for sectioning, we use holograms acquired by optical scanning holography as examples to show the method's effectiveness.