Fast non-line-of-sight imaging with high-resolution and wide field of view using synthetic wavelength holography.

The presence of a scattering medium in the imaging path between an object and an observer is known to severely limit the visual acuity of the imaging system. We present an approach to circumvent the deleterious effects of scattering, by exploiting spectral correlations in scattered wavefronts. Our Synthetic Wavelength Holography (SWH) method is able to recover a holographic representation of hidden targets with sub-mm resolution over a nearly hemispheric angular field of view. The complete object field is recorded within 46 ms, by monitoring the scattered light return in a probe area smaller than 6 cm × 6 cm. This unique combination of attributes opens up a plethora of new Non-Line-of-Sight imaging applications ranging from medical imaging and forensics, to early-warning navigation systems and reconnaissance. Adapting the findings of this work to other wave phenomena will help unlock a wider gamut of applications beyond those envisioned in this paper.

Active computational imaging for circumventing resolution limits at macroscopic scales.

Macroscopic imagers are subject to constraints imposed by the wave nature of light and the geometry of image formation. The former limits the resolving power while the latter results in a loss of absolute size and shape information. The suite of methods outlined in this work enables macroscopic imagers the unique ability to capture unresolved spatial detail while recovering topographic information. The common thread connecting these methods is the notion of imaging under patterned illumination. The notion is advanced further to develop computational imagers with resolving power that is decoupled from the constraints imposed by the collection optics and the image sensor. These imagers additionally feature support for multiscale reconstruction.

Geometric model for an independently tilted lens and sensor with application for omnifocus imaging.

Optical imaging systems in which the lens and sensor are free to rotate about independent pivots offer greater degrees of freedom for controlling and optimizing the process of image gathering. However, to benefit from the expanded possibilities, we need an imaging model that directly incorporates the essential parameters. In this work, we propose a model of imaging which can accurately predict the geometric properties of the image in such systems. Furthermore, we introduce a new method for synthesizing an omnifocus (all-in-focus) image from a sequence of images captured while rotating a lens. The crux of our approach lies in insights gained from the new model.

Blind Super Resolution of Real-Life Video Sequences.

Super resolution (SR) for real-life video sequences is a challenging problem due to complex nature of the motion fields. In this paper, a novel blind SR method is proposed to improve the spatial resolution of video sequences, while the overall point spread function of the imaging system, motion fields, and noise statistics are unknown. To estimate the blur(s), first, a nonuniform interpolation SR method is utilized to upsample the frames, and then, the blur(s) is(are) estimated through a multi-scale process. The blur estimation process is initially performed on a few emphasized edges and gradually on more edges as the iterations continue. Also for faster convergence, the blur is estimated in the filter domain rather than the pixel domain. The high-resolution frames are estimated using a cost function that has the fidelity and regularization terms of type Huber-Markov random field to preserve edges and fine details. The fidelity term is adaptively weighted at each iteration using a masking operation to suppress artifacts due to inaccurate motions. Very promising results are obtained for real-life videos containing detailed structures, complex motions, fast-moving objects, deformable regions, or severe brightness changes. The proposed method outperforms the state of the art in all performed experiments through both subjective and objective evaluations. The results are available online at http://lyle.smu.edu/~rajand/Video_SR/.

Review of multiscale optical design.

Multiscale optical design is an approach that has been successfully utilized for over 100 years by optical designers and engineers to overcome challenges and achieve desired optical system performance. The benefits of the design paradigm include improving light collection, creating specific symmetries that can be exploited, collecting additional information about the object space, partitioning the optical field to enable piecewise correction of aberrations, and alleviating packing constraints. The purpose of this paper is to review the historical emergence of the use of multiscale optical design and present key examples of developments that have expanded its capabilities over the years.

Unified blind method for multi-image super-resolution and single/multi-image blur deconvolution.

This paper presents, for the first time, a unified blind method for multi-image super-resolution (MISR or SR), single-image blur deconvolution (SIBD), and multi-image blur deconvolution (MIBD) of low-resolution (LR) images degraded by linear space-invariant (LSI) blur, aliasing, and additive white Gaussian noise (AWGN). The proposed approach is based on alternating minimization (AM) of a new cost function with respect to the unknown high-resolution (HR) image and blurs. The regularization term for the HR image is based upon the Huber-Markov random field (HMRF) model, which is a type of variational integral that exploits the piecewise smooth nature of the HR image. The blur estimation process is supported by an edge-emphasizing smoothing operation, which improves the quality of blur estimates by enhancing strong soft edges toward step edges, while filtering out weak structures. The parameters are updated gradually so that the number of salient edges used for blur estimation increases at each iteration. For better performance, the blur estimation is done in the filter domain rather than the pixel domain, i.e., using the gradients of the LR and HR images. The regularization term for the blur is Gaussian (L2 norm), which allows for fast noniterative optimization in the frequency domain. We accelerate the processing time of SR reconstruction by separating the upsampling and registration processes from the optimization procedure. Simulation results on both synthetic and real-life images (from a novel computational imager) confirm the robustness and effectiveness of the proposed method.

Applications of the phase transfer function of digital incoherent imaging systems.

The phase of the optical transfer function is advocated as an important tool in the characterization of modern incoherent imaging systems. It is shown that knowledge of the phase transfer function (PTF) can benefit a diverse array of applications involving both traditional and computational imaging systems. Areas of potential benefits are discussed, and three applications are presented, demonstrating the utility of the phase of the complex frequency response in practical scenarios. In traditional imaging systems, the PTF is shown via simulation results to be strongly coupled with odd-order aberrations and hence useful in misalignment detection and correction. In computational imaging systems, experimental results confirm that the PTF can be successfully applied to subpixel shift estimation and wavefront coding characterization tasks.

Prototype development and field-test results of an adaptive multiresolution PANOPTES imaging architecture.

The design, development, and field-test results of a visible-band, folded, multiresolution, adaptive computational imaging system based on the Processing Arrays of Nyquist-limited Observations to Produce a Thin Electro-optic Sensor (PANOPTES) concept is presented. The architectural layout that enables this imager to be adaptive is described, and the control system that ensures reliable field-of-view steering for precision and accuracy in subpixel target registration is explained. A digital superresolution algorithm introduced to obtain high-resolution imagery from field tests conducted in both nighttime and daytime imaging conditions is discussed. The digital superresolution capability of this adaptive PANOPTES architecture is demonstrated via results in which resolution enhancement by a factor of 4 over the detector Nyquist limit is achieved.

Experimental evidence of the theoretical spatial frequency response of cubic phase mask wavefront coding imaging systems.

The optical transfer function of a cubic phase mask wavefront coding imaging system is experimentally measured across the entire range of defocus values encompassing the system's functional limits. The results are compared against mathematical expressions describing the spatial frequency response of these computational imagers. Experimental data shows that the observed modulation and phase transfer functions, available spatial frequency bandwidth and design range of this imaging system strongly agree with previously published mathematical analyses. An imaging system characterization application is also presented wherein it is shown that the phase transfer function is more robust than the modulation transfer function in estimating the strength of the cubic phase mask.

Effects of sampling on the phase transfer function of incoherent imaging systems.

With the advent of modern-day computational imagers, the phase of the optical transfer function may no longer be summarily ignored. This study discusses some important properties of the phase transfer function (PTF) of digital incoherent imaging systems and their implications on the performance and characterization of these systems. The effects of aliasing and sub-pixel image shifts on the phase of the complex frequency response of these sampled systems are described, including an examination of the specific case of moderate aliasing. Key properties of this function in aliased imaging systems are derived and their potential treatment to a range of diverse applications encompassing traditional and computational imaging systems is discussed.

Active integrated filters for RF-photonic channelizers.

A theoretical study of RF-photonic channelizers using four architectures formed by active integrated filters with tunable gains is presented. The integrated filters are enabled by two- and four-port nano-photonic couplers (NPCs). Lossless and three individual manufacturing cases with high transmission, high reflection, and symmetric couplers are assumed in the work. NPCs behavior is dependent upon the phenomenon of frustrated total internal reflection. Experimentally, photonic channelizers are fabricated in one single semiconductor chip on multi-quantum well epitaxial InP wafers using conventional microelectronics processing techniques. A state space modeling approach is used to derive the transfer functions and analyze the stability of these filters. The ability of adapting using the gains is demonstrated. Our simulation results indicate that the characteristic bandpass and notch filter responses of each structure are the basis of channelizer architectures, and optical gain may be used to adjust filter parameters to obtain a desired frequency magnitude response, especially in the range of 1-5 GHz for the chip with a coupler separation of ∼9 mm. Preliminarily, the measurement of spectral response shows enhancement of quality factor by using higher optical gains. The present compact active filters on an InP-based integrated photonic circuit hold the potential for a variety of channelizer applications. Compared to a pure RF channelizer, photonic channelizers may perform both channelization and down-conversion in an optical domain.

Experimentally validated computational imaging with adaptive multiaperture folded architecture.

We present experimental results of imaging and digital superresolution in a multiaperture miniature folded imaging architecture called PANOPTES. We prove the feasibility of integrating a low f-number folded imagers within a steerable multiaperture framework while maintaining a thin profile. Stringent requirements including low f-number and compact form factor, combined with the need for an ability to steer individual fields of view necessitate an off-axis design, resulting in a plane symmetric optical system. We present a detailed description of the ensuing optical design and its performance. The feasibility of this architecture is demonstrated through experiments and preliminary reconstruction results.

Integrated photonic coupler based on frustrated total internal reflection.

An optical coupler for integrated photonic circuits is presented and analyzed. The coupler is based on frustrated total internal reflection (FTIR) and offers high efficiency in a compact footprint. Analytic expressions for the transmission and reflection coefficients of the coupler are obtained using a plane-wave theory and experimentally verified. Finite-difference time-domain modeling of FTIR is discussed and modeling results of the coupler are presented. A parametric discussion of the FTIR coupler provides design tools for making 3 dB couplers.

Adaptive framework for robust high-resolution image reconstruction in multiplexed computational imaging architectures.

In multiplexed computational imaging schemes, high-resolution images are reconstructed by fusing the information in multiple low-resolution images detected by a two-dimensional array of low-resolution image sensors. The reconstruction procedure assumes a mathematical model for the imaging process that could have generated the low-resolution observations from an unknown high-resolution image. In practical settings, the parameters of the mathematical imaging model are known only approximately and are typically estimated before the reconstruction procedure takes place. Violations to the assumed model, such as inaccurate knowledge of the field of view of the imagers, erroneous estimation of the model parameters, and/or accidental scene or environmental changes can be detrimental to the reconstruction quality, even if they are small in number. We present an adaptive algorithm for robust reconstruction of high-resolution images in multiplexed computational imaging architectures. Using robust M-estimators and incorporating a similarity measure, the proposed scheme adopts an adaptive estimation strategy that effectively deals with violations to the assumed imaging model. Comparisons with nonadaptive reconstruction techniques demonstrate the superior performance of the proposed algorithm in terms of reconstruction quality and robustness.

Computational imaging systems: joint design and end-to-end optimality.

A framework is proposed for optimal joint design of the optical and reconstruction filters in a computational imaging system. First, a technique for the design of a physically unconstrained system is proposed whose performance serves as a universal bound on any realistic computational imaging system. Increasing levels of constraints are then imposed to emulate a physically realizable optical filter. The proposed design employs a generalized Benders' decomposition method to yield multiple globally optimal solutions to the nonconvex optimization problem. Structured, closed-form solutions for the design of observation and reconstruction filters, in terms of the system input and noise autocorrelation matrices, are presented. Numerical comparison with a state-of-the-art optical system shows the advantage of joint optimization and concurrent design.

Impact of measurement precision and noise on superresolution image reconstruction.

The performance of uniform and nonuniform detector arrays for application to the PANOPTES (processing arrays of Nyquist-limited observations to produce a thin electro-optic sensor) flat camera design is analyzed for measurement noise environments including quantization noise and Gaussian and Poisson processes. Image data acquired from a commercial camera with 8 bit and 14 bit output options are analyzed, and estimated noise levels are computed. Noise variances estimated from the measurement values are used in the optimal linear estimators for superresolution image reconstruction.

Improving photon count and flat profiles of multiplex imaging systems with the odd-symmetric quadratic phase modulation mask.

The odd-symmetric quadratic (OSQ) phase mask is examined as a candidate for reduction of working distance and enhancement of light collection in multiplex imaging systems. The knowledge gained from the exact mathematical representation of the optical transfer function of the OSQ phase mask imager is exploited to explain the limits of system performance and quantify the upper bound on the magnitude of defocus within which this wavefront coding imager can successfully operate. The sensitivity of this imaging system to defocus about the special imaging condition that yields an enhanced dynamic range is examined, and it is shown that the modulation transfer function (MTF) degradation when the magnitude of misfocus is increased past this condition is much more gradual than the degradation of a conventional imager past a zero-misfocus state. The condition required for the spatial frequency and angular resolution of this OSQ phase mask imager to exceed that of its counterpart scaled imager is established, and results of simulated imaging under a reduced working distance configuration are presented.

Frequency analysis of the wavefront-coding odd-symmetric quadratic phase mask.

A mathematical analysis of the frequency response of the wavefront-coding odd-symmetric quadratic phase mask is presented. An exact solution for the optical transfer function of a wavefront-coding imager using this type of mask is derived from first principles, whose result applies over all misfocus values. The misfocus-dependent spatial filtering property of this imager is described. The available spatial frequency bandwidth for a given misfocus condition is quantified. A special imaging condition that yields an increased dynamic range is identified.

Composition methods for four-port couplers in photonic integrated circuitry.

Planar photonic integrated circuits based on four-port couplers offer enhanced sophistication and functionality. Each four-port coupler is characterized by sixteen signal coupling coefficients governed by ten energy constraints. The ability to generate the constrained sixteen coupling coefficients is needed in the analysis of the four-port coupler. However, the energy constraint equations are nonlinear and cumbersome to solve directly. We introduce two techniques to reduce these signal coupling coefficients to a set of six free parameters. Hence we can characterize all possible couplers in terms of their sixteen constrained coupling coefficients, or either of two sets of six free parameters. This reduction in parameters has significant ramifications for the design, specification, and empirical characterization of these useful building blocks.

Trade-offs between lens complexity and real estate utilization in a free-space multichip global interconnection module.

The FAST-Net (Free-space Accelerator for Switching Terabit Networks) concept uses an array of wide-field-of-view imaging lenses to realize a high-density shuffle interconnect pattern across an array of smart-pixel integrated circuits. To simplify the optics we evaluated the efficiency gained in replacing spherical surfaces with aspherical surfaces by exploiting the large disparity between narrow vertical cavity surface emitting laser (VCSEL) beams and the wide field of view of the imaging optics. We then analyzed trade-offs between lens complexity and chip real estate utilization and determined that there exists an optimal numerical aperture for VCSELs that maximizes their area density. The results provide a general framework for the design of wide-field-of-view free-space interconnection systems that incorporate high-density VCSEL arrays.