Deep learning based coherence holography reconstruction of 3D objects.

We propose a reconstruction method for coherence holography using deep neural networks. cGAN and U-NET models were developed to reconstruct 3D complex objects from recorded interferograms. Our proposed methods, dubbed deep coherence holography (DCH), predict the non-diffracted fields or the sub-objects included in the 3D object from the captured interferograms, yielding better reconstructed objects than the traditional analytical imaging methods in terms of accuracy, resolution, and time. The DCH needs one image per sub-object as opposed to N images for the traditional sin-fit algorithm, and hence the total reconstruction time is reduced by N×. Furthermore, with noisy interferograms the DCH amplitude mean square reconstruction error (MSE) is 5×104× and 104× and phase MSE is 102× and 3×103× better than Fourier fringe and sin-fit algorithms, respectively. The amplitude peak signal to noise ratio (PSNR) is 3× and 2× and phase PSNR is 5× and 3× better than Fourier fringe and sin-fit algorithms, respectively. The reconstruction resolution is the same as sin-fit but 2× better than the Fourier fringe analysis technique.

Three-dimensional refractive index estimation based on deep-inverse non-interferometric optical diffraction tomography (ODT-Deep).

Optical diffraction tomography (ODT) solves an inverse scattering problem to obtain label-free, 3D refractive index (RI) estimation of biological specimens. This work demonstrates 3D RI retrieval methods suitable for partially-coherent ODT systems supported by intensity-only measurements consisting of axial and angular illumination scanning. This framework allows for access to 3D quantitative RI contrast using a simplified non-interferometric technique. We consider a traditional iterative tomographic solver based on a multiple in-plane representation of the optical scattering process and gradient descent optimization adapted for focus-scanning systems, as well as an approach that relies solely on 3D convolutional neural networks (CNNs) to invert the scattering process. The approaches are validated using simulations of the 3D scattering potential for weak phase 3D biological samples.

Computational multi-wavelength phase synthesis using convolutional neural networks [Invited].

Multi-wavelength digital holographic microscopy (MWDHM) provides indirect measurements of the refractive index for non-dispersive samples. Successive-shot MWDHM is not appropriate for dynamic samples and single-shot MWDHM significantly increases the complexity of the optical setup due to the need for multiple lasers or a wavelength tunable source. Here we consider deep learning convolutional neural networks for computational phase synthesis to obtain high-speed simultaneous phase estimates on different wavelengths and thus single-shot estimates of the integral refractive index without increased experimental complexity. This novel, to the best of our knowledge, computational concept is validated using cell phantoms consisting of internal refractive index variations representing cytoplasm and membrane-bound organelles, respectively, and a simulation of a realistic holographic recording process. Specifically, in this work we employed data-driven computational techniques to perform accurate dual-wavelength hologram synthesis (hologram-to-hologram prediction), dual-wavelength phase synthesis (unwrapped phase-to-phase prediction), direct phase-to-index prediction using a single wavelength, hologram-to-phase prediction, and 2D phase unwrapping with sharp discontinuities (wrapped-to-unwrapped phase prediction).

Simulation of digital holographic recording and reconstruction using a generalized matrix method.

In recent years, research efforts in the field of digital holography have expanded significantly, due to the ability to obtain high-resolution intensity and phase images. The information contained in these images have become of great interest to the machine learning community, with applications spanning a wide portfolio of research areas, including bioengineering. In this work, we seek to demonstrate a high-fidelity simulation of holographic recording. By accurately and numerically simulating the propagation of a coherent light source through a series of optical elements and the object itself, we accurately predict the optical interference of the object and reference wave at the recording plane, including diffraction effects, aberrations, and speckle. We show that the optical transformation that predicts the complex field at the recording plane can be generalized for arbitrary holographic recording configurations using a matrix method. In addition, we provide a detailed description of digital phase reconstruction and aberration compensation for a variety of off-axis holographic configurations. Reconstruction errors are presented for the various holographic recording geometries and complex field objects. While the primary objective of this work is not to evaluate phase reconstruction approaches, the reconstruction of simulated holograms provides validation of the generalized simulation method. The long-term goal of this work is that the generalized holographic simulation motivates the use of phase reconstruction of the simulated holograms to populate databases for training machine-learning algorithms aimed at classifying relevant objects recorded through a variety of holographic setups.

Machine Learning with Optical Phase Signatures for Phenotypic Profiling of Cell Lines.

Robust and reproducible profiling of cell lines is essential for phenotypic screening assays. The goals of this study were to determine robust and reproducible optical phase signatures of cell lines for classification with machine learning and to correlate optical phase parameters to motile behavior. Digital holographic microscopy (DHM) reconstructed phase maps of cells from two pairs of cancer and non-cancer cell lines. Seventeen image parameters were extracted from each cell's phase map, used for linear support vector machine learning, and correlated to scratch wound closure and Boyden chamber chemotaxis. The classification accuracy was between 90% and 100% for the six pairwise cell line comparisons. Several phase parameters correlated with wound closure rate and chemotaxis across the four cell lines. The level of cell confluence in culture affected phase parameters in all cell lines tested. Results indicate that optical phase features of cell lines are a robust set of quantitative data of potential utility for phenotypic screening and prediction of motile behavior. © 2019 International Society for Advancement of Cytometry.

Experimentally measuring a detectability index of a computational imaging system.

Computational imaging (CI) systems are an enabling technology for multifunctional cameras capable of performing a wide variety of imaging tasks. However, given the complexity of CI systems, it is often difficult to characterize their performance. In this research, a novel measurement technique is proposed and tested to evaluate the performance of complex non-shift invariant linear CI systems performing a detection task at the system level. The performance is characterized using detectability indexes such as an average Hotelling's statistic (t2). The proposed measurement technique relies on a previously developed general CI system framework. The detectability predicts the upper-bounded signal-to-noise ratio of a linear algorithm through evaluation of a matched filter. The experimental results are compared with theoretical expected values through the Night Vision Integrated Performance Model (NV-IPM) and Monte Carlo simulations. We demonstrate the experimental results for a variety of target sizes, colors, and brightnesses on different colored flat backgrounds. Our results demonstrate how the detectability indexes can provide valuable insight into the final system performance. Finally, the measurement technique is used to compare the detection performance of two different cameras.

Quantitative assessment of cancer cell morphology and motility using telecentric digital holographic microscopy and machine learning.

The noninvasive, fast acquisition of quantitative phase maps using digital holographic microscopy (DHM) allows tracking of rapid cellular motility on transparent substrates. On two-dimensional surfaces in vitro, MDA-MB-231 cancer cells assume several morphologies related to the mode of migration and substrate stiffness, relevant to mechanisms of cancer invasiveness in vivo. The quantitative phase information from DHM may accurately classify adhesive cancer cell subpopulations with clinical relevance. To test this, cells from the invasive breast cancer MDA-MB-231 cell line were cultured on glass, tissue-culture treated polystyrene, and collagen hydrogels, and imaged with DHM followed by epifluorescence microscopy after staining F-actin and nuclei. Trends in cell phase parameters were tracked on the different substrates, during cell division, and during matrix adhesion, relating them to F-actin features. Support vector machine learning algorithms were trained and tested using parameters from holographic phase reconstructions and cell geometric features from conventional phase images, and used to distinguish between elongated and rounded cell morphologies. DHM was able to distinguish between elongated and rounded morphologies of MDA-MB-231 cells with 94% accuracy, compared to 83% accuracy using cell geometric features from conventional brightfield microscopy. This finding indicates the potential of DHM to detect and monitor cancer cell morphologies relevant to cell cycle phase status, substrate adhesion, and motility. © 2017 International Society for Advancement of Cytometry.

Large-volume, low-cost, high-precision FMCW tomography using stitched DFBs.

Optical frequency-modulated continuous-wave (FMCW) reflectometry is a ranging technique that allows for high-resolution distance measurements over long ranges. Similarly, swept-source optical coherence tomography (SS-OCT) provides high-resolution depth imaging over typically shorter distances and higher scan speeds. In this work, we demonstrate a low-cost, low-bandwidth 3D imaging system that provides the high axial resolution imaging capability normally associated with SS-OCT over typical FMCW ranging depths. The imaging system combines 12 distributed feedback laser (DFB) elements from a single butterfly module to provide an axial resolution of 27.1 µm over 6 m of depth and up to 14 cubic meters of volume. Active sweep linearization is used, greatly reducing the signal processing overhead. Various sub-surface, OCT-style tomograms of semi-transparent objects are shown, as well as 3D maps of various objects over depths ranging from sub-millimeter to several meters. Such imaging capability would make long-distance, high-resolution surface interrogation possible in a low-cost, compact package.

Deep learning approach for Fourier ptychography microscopy.

Convolutional neural networks (CNNs) have gained tremendous success in solving complex inverse problems. The aim of this work is to develop a novel CNN framework to reconstruct video sequences of dynamic live cells captured using a computational microscopy technique, Fourier ptychographic microscopy (FPM). The unique feature of the FPM is its capability to reconstruct images with both wide field-of-view (FOV) and high resolution, i.e. a large space-bandwidth-product (SBP), by taking a series of low resolution intensity images. For live cell imaging, a single FPM frame contains thousands of cell samples with different morphological features. Our idea is to fully exploit the statistical information provided by these large spatial ensembles so as to make predictions in a sequential measurement, without using any additional temporal dataset. Specifically, we show that it is possible to reconstruct high-SBP dynamic cell videos by a CNN trained only on the first FPM dataset captured at the beginning of a time-series experiment. Our CNN approach reconstructs a 12800×10800 pixel phase image using only ∼25 seconds, a 50× speedup compared to the model-based FPM algorithm. In addition, the CNN further reduces the required number of images in each time frame by ∼ 6×. Overall, this significantly improves the imaging throughput by reducing both the acquisition and computational times. The proposed CNN is based on the conditional generative adversarial network (cGAN) framework. We further propose a mixed loss function that combines the standard image domain loss and a weighted Fourier domain loss, which leads to improved reconstruction of the high frequency information. Additionally, we also exploit transfer learning so that our pre-trained CNN can be further optimized to image other cell types. Our technique demonstrates a promising deep learning approach to continuously monitor large live-cell populations over an extended time and gather useful spatial and temporal information with sub-cellular resolution.

Multiaxis heterodyne vibrometer for simultaneous observation of 5 degrees of dynamic freedom from a single beam.

A multiaxis heterodyne interferometer concept is under development for observations of 5 deg of dynamic freedom using a single illumination source. This Letter presents a laboratory system that combines elements of heterodyne Doppler vibrometry, holography, and digital image correlation to simultaneously quantify in-plane translation, out-of-plane rotation, and out-of-plane displacement. The sensor concept observes a dynamic object by mixing a single optical field with heterodyne reference beams and collecting these combined fields at the image and Fourier planes, simultaneously. Polarization and frequency multiplexing are applied to separate two segments of a receive Mach-Zehnder interferometer. Different optical configurations are utilized; one segment produces a focused image of the optical field scattered off the object while the other segment produces an optical Fourier transform of the optical field scattered off the object. Utilizing the amplitude and phase from each plane allows quantification of multiple components of transient motion using a single, orthogonal beam.

Phase-noise model for actively linearized frequency-modulated continuous-wave ladar.

Understanding the effects of laser phase noise on frequency-modulated continuous-wave distance measurements is important in evaluating ranging accuracy. The standard white-frequency-noise assumption is commonly used to predict the ranging performance. However, other noise sources are typically present that can further degrade the heterodyne beat signal and make this assumption invalid. In addition, many ranging systems employ active sweep linearization techniques that can impact the phase noise. Here, we present a phase-noise model for assessing the accuracy of a phase-locked swept laser source.

Multi-terahertz frequency sweeps for high-resolution, frequency-modulated continuous wave ladar using a distributed feedback laser array.

Swept-wavelength reflectometry is an absolute distance measurement technique with significant sensitivity and detector bandwidth advantages over normal pulsed, time-of-flight methods. Although several tunable laser sources exist, many exhibit short coherence lengths or require mechanical tuning components. Semiconductor distributed feedback laser diodes (DFBs) are advantageous as a swept source because they exhibit a narrow instantaneous linewidth and can be frequency-swept simply via a single injection current. Here, we present a novel bandwidth generation technique that uses a compact, monolithic, 12-element DFB array to create an effectively continuous, gap-free sweep. Each DFB is sequentially swept over 3.5 nm at 1,600 THz/s using a shaped current pulse, ensuring spectral overlap between each element. After combining the self-heterodyned return signatures, the transform-limited resolution of the 43.6 nm sweep is demonstrated to be ~27.4 µm in air with a precision of 0.18 µm at a distance of 1.4 m.

Phase-resolved heterodyne shearographic vibrometer for observation of transient surface motion: theory and model.

A phase-resolved heterodyne shearing interferometer concept is under development for high-rate, whole field observations of transient surface motion. The sensor utilizes frequency and polarization multiplexing with two temporal carrier frequencies to separate each segment of a shearing Mach-Zehnder interferometer. Post-processing routines have been developed to recombine the segments by extracting the scattered object phase from Doppler shifted intermediate carrier frequencies. The processing routines provide quantitative relative phase changes and information required to generate phase resolved shearographic fringe patterns without temporal or spatial phase shifting. Separation of each segment allows for adjustment of shearing distance and direction as well as simultaneous whole field Doppler velocity (LDV) measurements. This paper presents background theory and numerical model results leading to a sensor concept.

Automatic phase aberration compensation for digital holographic microscopy based on deep learning background detection.

We propose a fully automatic technique to obtain aberration free quantitative phase imaging in digital holographic microscopy (DHM) based on deep learning. The traditional DHM solves the phase aberration compensation problem by manually detecting the background for quantitative measurement. This would be a drawback in real time implementation and for dynamic processes such as cell migration phenomena. A recent automatic aberration compensation approach using principle component analysis (PCA) in DHM avoids human intervention regardless of the cells' motion. However, it corrects spherical/elliptical aberration only and disregards the higher order aberrations. Traditional image segmentation techniques can be employed to spatially detect cell locations. Ideally, automatic image segmentation techniques make real time measurement possible. However, existing automatic unsupervised segmentation techniques have poor performance when applied to DHM phase images because of aberrations and speckle noise. In this paper, we propose a novel method that combines a supervised deep learning technique with convolutional neural network (CNN) and Zernike polynomial fitting (ZPF). The deep learning CNN is implemented to perform automatic background region detection that allows for ZPF to compute the self-conjugated phase to compensate for most aberrations.

Standardized target-specific detectivity metric for computational imaging systems.

Multifunctional cameras capable of performing a wide variety of nearly simultaneous imaging tasks are expected to play a major role in the near future. Computational imaging (CI) systems will serve as one of the main enabling technologies for multifunctional cameras, especially due to the abundance of low-cost, high-speed computational processing available today. An important aspect of these systems is to be able to quantify their performance with respect to specific imaging tasks. However, the non-traditional design of CI systems, both available and proposed, presents a considerable challenge to modeling, comparing, specifying, and measuring their performance. To solve this problem, this paper presents a standardized detection signal-to-noise ratio, referred to as a detectivity metric, along with a general CI system framework. This metric has the flexibility to handle various types of CI systems and specific targets while minimizing the complexity and assumptions needed. The detectivity metric is designed to assess the performance of a CI system searching for a specific known target or signal of interest. An analytical version of the detectivity metric is also presented for a compressive sensing CI system. Special considerations for standardization are also discussed.

Digital Holography and 3D Imaging: introduction to the joint feature issue in Applied Optics and Journal of the Optical Society of America B.

The OSA Topical Meeting on Digital Holography and 3D Imaging (DH) was held 25-28 July 2016 in Heidelberg, Germany, as part of the Imaging Congress. Feature issues based on the DH meeting series have been released by Applied Optics (AO) since 2007. This year, AO and the Journal of the Optical Society of America B (JOSA B) jointly decided to have one such feature issue in each journal. This feature issue includes 31 papers in AO and 11 in JOSA B, and covers a large range of topics, reflecting the rapidly expanding techniques and applications of digital holography and 3D imaging. The upcoming DH meeting (DH 2017) will be held from 29 May to 1 June in Jeju Island, South Korea.

Accurate quantitative phase digital holographic microscopy with single- and multiple-wavelength telecentric and nontelecentric configurations.

In this work, we investigate, both theoretically and experimentally, single-wavelength and multiwavelength digital holographic microscopy (DHM) using telecentric and nontelecentric configurations in transmission and reflection modes. A single-wavelength telecentric imaging system in DHM was originally proposed to circumvent the residual parabolic phase distortion due to the microscope objective (MO) in standard nontelecentric DHM configurations. However, telecentric configurations cannot compensate for higher order phase aberrations. As an extension to the telecentric and nontelecentric arrangements in single-wavelength DHM (SW-DHM), we propose multiple-wavelength telecentric DHM (MW-TDHM) in reflection and transmission modes. The advantages of MW-TDHM configurations are to extend the vertical measurement range without phase ambiguity and optically remove the parabolic phase distortion caused by the MO in traditional MW-DHM. These configurations eliminate the need for a second reference hologram to subtract the two-phase maps and make digital automatic aberration compensation easier to apply compared to nontelecentric configurations. We also discuss a reconstruction algorithm that eliminates the zero-order and virtual images using spatial filtering and another algorithm that minimizes the intensity of fluctuations using apodization. In addition, we employ two polynomial models using 2D surface fitting to compensate digitally for chromatic aberration (in the multiwavelength case) and for higher order phase aberrations. A custom-developed user-friendly graphical user interface is employed to automate the reconstruction processes for all configurations. Finally, TDHM is used to visualize cells from the highly invasive MDA-MB-231 cultured breast cancer cells.

Accurate testing of aspheric surfaces using the transport of intensity equation by properly selecting the defocusing distance.

In the last decade, the transport of intensity has been increasingly used in microscopy, wavefront sensing, and metrology. In this study, we verify by simulation and experiment the use of the transport of intensity equation (TIE) in the accurate testing of optical aspheric surfaces. Guided by simulation results and assuming that the experimental setup parameters and the conic constants are known, one can estimate an appropriate defocusing distance Δz that leads to an accurate solution of the TIE. In this paper, this method is verified through the construction of a non-nulled experiment for testing the 2D profile of an aspheric surface. The theoretical method and experimental results are compared to validate the results. Finally, to validate the TIE methodology, the phase distribution obtained by TIE is compared with the phase distribution obtained by a Shack-Hartmann sensor.

Adaptive defect and pattern detection in amplitude and phase structures via photorefractive four-wave mixing.

This work comprises the theoretical and numerical validations of experimental work on pattern and defect detection of periodic amplitude and phase structures using four-wave mixing in photorefractive materials. The four-wave mixing optical processor uses intensity filtering in the Fourier domain. Specifically, the nonlinear transfer function describing four-wave mixing is modeled, and the theory for detection of amplitude and phase defects and dislocations are developed. Furthermore, numerical simulations are performed for these cases. The results show that this technique successfully detects the slightest defects clearly even with no prior enhancement. This technique should prove to be useful in quality control systems, production-line defect inspection, and e-beam lithography.

Fully automated, high speed, tomographic phase object reconstruction using the transport of intensity equation in transmission and reflection configurations.

While traditional transport of intensity equation (TIE) based phase retrieval of a phase object is performed through axial translation of the CCD, in this work a tunable lens TIE is employed in both transmission and reflection configurations. These configurations are extended to a 360° tomographic 3D reconstruction through multiple illuminations from different angles by a custom fabricated rotating assembly of the phase object. Synchronization circuitry is developed to control the CCD camera and the Arduino board, which in its turn controls the tunable lens and the stepper motor to automate the tomographic reconstruction process. Finally, a MATLAB based user friendly graphical user interface is developed to control the whole system and perform tomographic reconstruction using both multiplicative and inverse radon based techniques.