Compressive video via IR-pulsed illumination.

We propose and demonstrate a compressive temporal imaging system based on pulsed illumination to encode temporal dynamics into the signal received by the imaging sensor during exposure time. Our approach enables >10x increase in effective frame rate without increasing camera complexity. To mitigate the complexity of the inverse problem during reconstruction, we introduce two keyframes: one before and one after the coded frame. We also craft what we believe to be a novel deep learning architecture for improved reconstruction of the high-speed scenes, combining specialized convolutional and transformer architectures. Simulation and experimental results clearly demonstrate the reconstruction of high-quality, high-speed videos from the compressed data.

Snapshot ptychography on array cameras.

We use convolutional neural networks to recover images optically down-sampled by 6.7 × using coherent aperture synthesis over a 16 camera array. Where conventional ptychography relies on scanning and oversampling, here we apply decompressive neural estimation to recover full resolution image from a single snapshot, although as shown in simulation multiple snapshots can be used to improve signal-to-noise ratio (SNR). In place training on experimental measurements eliminates the need to directly calibrate the measurement system. We also present simulations of diverse array camera sampling strategies to explore how snapshot compressive systems might be optimized.

Photon-limited bounds for phase retrieval.

We show that the optimal Cramér-Rao lower bound on the mean-square error for the estimation of a coherent signal from photon-limited intensity measurements is equal to the number of signal elements, or the number of signal elements minus one when we account for the unobservable reference phase. Whereas this bound is attained by phase-quadrature holography, we also show that it can be attained through a phase-retrieval system that does not require a coherent reference. We also present the bounds for classic phase-retrieval and ptychography, and show that practical coding strategies can approach optimal performance.

Noise suppression for ballistic-photons based on compressive in-line holographic imaging through an inhomogeneous medium.

Noise suppression is one of the most important tasks in imaging through inhomogeneous mediums. Here, we proposed a denoising approach based on compressive in-line holography for imaging through an inhomogeneous medium. A reference-beam-free system with a low-cost continuous-wave laser is presented. The suppression against the noise, which is brought by the scattering photons, is presented in simulations using the proposed algorithm. The noise immunity is demonstrated in lensless imaging behind a random phase mask with an optical depth of 1.42 by single exposure, as well as behind a ground glass with an optical depth of 6.38 by multiple exposures.

Twin-Image-Free Holography: A Compressive Sensing Approach.

Holographic reconstruction is troubled by the phase-conjugate wave front arising from Hermitian symmetry of the complex field. The so-called twin image obfuscates the reconstruction in solving the inverse problem. Here we quantitatively reveal how and how much the twin image affects the reconstruction and propose a compressive sensing (CS) approach to reconstruct a hologram completely free from the twin image. Using the canonical basis, the incoherence condition of CS is naturally satisfied by the Fourier transformation associated with wave propagation. With the propagation kernel function related to the distance, the object wave diffracts into a sharp pattern while the phase-conjugate wave diffracts into a diffuse pattern. An iterative algorithm using a total variation sparsity constraint could filter out the diffuse conjugated signal and overcome the inherent physical symmetry of holographic reconstruction. The feasibility is verified by simulation and experimental results, as well as a comparative study to an existing phase retrieval method.

Field of view in monocentric multiscale cameras.

Conventionally, the field of view of a camera is understood as the angular extent of a convex circular or rectangular region. Parallel camera architectures with computational image stitching, however, allow implementation of a field of view with an arbitrary shape. Monocentric multiscale lenses further allow the implementation of an arbitrary field of view in camera volumes comparable to conventional single-lens systems. In contrast with conventional wide-field-of-view systems, multiscale design can also achieve nearly uniform resolution across the entire field of view. This paper presents several design studies obtaining unconventional fields of view using this approach.

Single-sensor multispeaker listening with acoustic metamaterials.

Designing a "cocktail party listener" that functionally mimics the selective perception of a human auditory system has been pursued over the past decades. By exploiting acoustic metamaterials and compressive sensing, we present here a single-sensor listening device that separates simultaneous overlapping sounds from different sources. The device with a compact array of resonant metamaterials is demonstrated to distinguish three overlapping and independent sources with 96.67% correct audio recognition. Segregation of the audio signals is achieved using physical layer encoding without relying on source characteristics. This hardware approach to multichannel source separation can be applied to robust speech recognition and hearing aids and may be extended to other acoustic imaging and sensing applications.

Galilean monocentric multiscale optical systems.

The first generation of monocentric multiscale gigapixel cameras used Keplerian designs to enable full field coverage. This paper considers alternative designs that remove the requirement that adjacent subimages overlap. Removing this constraint enables Galilean designs that reduce system volume and improve relative illumination and image quality. The entrance aperture can also be moved to more closely approximate telecentricity and gaps in the field of view can be filled using multiple co-boresighted MMS cameras. Even with multiple cameras, Galilean systems can still reduce the total volume by 10 times relative to previous Keplerian designs.

Efficient block-wise algorithm for compressive holography.

Compressive holography is a relatively time-consuming image estimation in convex optimized problem. We propose an efficient block-wise algorithm to limit the searching space and reduce the calculation time while keeping the reconstruction quality. The effective anti-aliasing boundary of the sub-hologram is located to determine the block size for compressive reconstruction in the total-variation two-step iterative shrinkage/thresholding algorithm. Padded sub-holograms could be reconstructed in parallel by using multi-core processors. Compared with the traditional compressive holography, the block-wise algorithm could take approximately 1/50 of the reconstruction time and achieve an improved reconstruction quality.

Compressive sensing and adaptive sampling applied to millimeter wave inverse synthetic aperture imaging.

In order to improve speed and efficiency over traditional scanning methods, a Bayesian compressive sensing algorithm using adaptive spatial sampling is developed for single detector millimeter wave synthetic aperture imaging. The application of this algorithm is compared to random sampling to demonstrate that the adaptive algorithm converges faster for simple targets and generates more reliable reconstructions for complex targets.

Heterogeneous camera array for multispectral light field imaging.

Multispectral light field acquisition is challenging due to the increased dimensionality of the problem. In this paper, inspired by anaglyph theory (i.e. the ability of human eyes to synthesize colored stereo perception from color-complementary (such as red and cyan) views), we propose to capture the multispectral light field using multiple cameras with different wide band filters. A convolutional neural network is used to extract the joint information of different spectral channels and to pair the cross-channel images. In our experiment, results on both synthetic data and real data captured by our prototype system validate the effectiveness and accuracy of proposed method.

Snapshot fan beam coded aperture coherent scatter tomography.

We use coherently scattered X-rays to measure the molecular composition of an object throughout its volume. We image a planar slice of the object in a single snapshot by illuminating it with a fan beam and placing a coded aperture between the object and the detectors. We characterize the system and demonstrate a resolution of 13 mm in range and 2 mm in cross-range and a fractional momentum transfer resolution of 15%. In addition, we show that this technique allows a 100x speedup compared to previously-studied pencil beam systems using the same components. Finally, by scanning an object through the beam, we image the full 4-dimensional data cube (3 spatial and 1 material dimension) for complete volumetric molecular imaging.

Spatial light modulator based color polarization imaging.

We describe a compressive snapshot color polarization imager that encodes spatial, spectral, and polarization information using a liquid crystal modulator. We experimentally show that polarization imaging is compressible by multiplexing polarization states and present the reconstruction results. This compressive camera captures the spatial distribution of four polarizations and three color channels. It achieves <0.027° spatial resolution, 10(3) average extinction ratio, and >30 PSNR.

Spectral-temporal compressive imaging.

This Letter presents a compressive camera that integrates mechanical translation and spectral dispersion to compress a multi-spectral, high-speed scene onto a monochrome, video-rate detector. Experimental reconstructions of 17 spectral channels and 11 temporal channels from a single measurement are reported for a megapixel-scale monochrome camera.

Echelle crossed grating millimeter wave beam scanner.

We present a two-dimensional, active, millimeter-wave, electronic beam scanner, with Doppler capabilities for stand-off imaging. The two-dimensional scan is achieved by mapping the millimeter wave spectrum to space using a pair of crossed gratings. The active transceiver and heterodyne quadrature detection allow the measurement of the relative phase between two consecutive measurements and the synthesis of the scene's Doppler signature. The frame rate of the imager is currently limited by the sweep rate of the vector network analyzer which is used to drive the millimeter wave extenders. All of the beam steering components are passive and can be designed to operate at any wavelength. The system design, characterization and measurements are presented and further uses and improvements are suggested.

Adaptive millimeter-wave synthetic aperture imaging for compressive sampling of sparse scenes.

We apply adaptive sensing techniques to the problem of locating sparse metallic scatterers using high-resolution, frequency modulated continuous wave W-band RADAR. Using a single detector, a frequency stepped source, and a lateral translation stage, inverse synthetic aperture RADAR reconstruction techniques are used to search for one or two wire scatterers within a specified range, while an adaptive algorithm determined successive sampling locations. The two-dimensional location of each scatterer is thereby identified with sub-wavelength accuracy in as few as 1/4 the number of lateral steps required for a simple raster scan. The implications of applying this approach to more complex scattering geometries are explored in light of the various assumptions made.

Optical performance test and validation of microcameras in multiscale, gigapixel imagers.

Wide field-of-view gigapixel imaging systems capable of diffraction-limited resolution and video-rate acquisition have a broad range of applications, including sports event broadcasting, security surveillance, astronomical observation, and bioimaging. The complexity of the system integration of such devices demands precision optical components that are fully characterized and qualified before being integrated into the final system. In this work, we present component and assembly level characterizations of microcameras in our first gigapixel camera, the AWARE-2. Based on the results of these measurements, we revised the optical design and assembly procedures to construct the second generation system, the AWARE-2 Retrofit, which shows significant improvement in image quality.

Metamaterial microwave holographic imaging system.

We demonstrate a microwave imaging system that combines advances in metamaterial aperture design with emerging computational imaging techniques. The flexibility inherent to guided-wave, complementary metamaterials enables the design of a planar antenna that illuminates a scene with dramatically varying radiation patterns as a function of frequency. As frequency is swept over the K-band (17.5-26.5 GHz), a sequence of pseudorandom radiation patterns interrogates a scene. Measurements of the return signal versus frequency are then acquired and the scene is reconstructed using computational imaging methods. The low-cost, frequency-diverse static aperture allows three-dimensional images to be formed without mechanical scanning or dynamic beam-forming elements. The metamaterial aperture is complementary to a variety of computational imaging schemes, and can be used in conjunction with other sensors to form a multifunctional imaging platform. We illustrate the potential of multisensor fusion by integrating an infrared structured-light and optical image sensor to accelerate the microwave scene reconstruction and to provide a simultaneous visualization of the scene.

Compressed sampling strategies for tomography.

We investigate new sampling strategies for projection tomography, enabling one to employ fewer measurements than expected from classical sampling theory without significant loss of information. Inspired by compressed sensing, our approach is based on the understanding that many real objects are compressible in some known representation, implying that the number of degrees of freedom defining an object is often much smaller than the number of pixels/voxels. We propose a new approach based on quasi-random detector subsampling, whereas previous approaches only addressed subsampling with respect to source location (view angle). The performance of different sampling strategies is considered using object-independent figures of merit, and also based on reconstructions for specific objects, with synthetic and real data. The proposed approach can be implemented using a structured illumination of the interrogated object or the detector array by placing a coded aperture/mask at the source or detector side, respectively. Advantages of the proposed approach include (i) for structured illumination of the detector array, it leads to fewer detector pixels and allows one to integrate detectors for scattered radiation in the unused space; (ii) for structured illumination of the object, it leads to a reduced radiation dose for patients in medical scans; (iii) in the latter case, the blocking of rays reduces scattered radiation while keeping the same energy in the transmitted rays, resulting in a higher signal-to-noise ratio than that achieved by lowering exposure times or the energy of the source; (iv) compared to view-angle subsampling, it allows one to use fewer measurements for the same image quality, or leads to better image quality for the same number of measurements. The proposed approach can also be combined with view-angle subsampling.

Continuous 3D Myocardial Motion Tracking via Echocardiography.

Myocardial motion tracking stands as an essential clinical tool in the prevention and detection of cardiovascular diseases (CVDs), the foremost cause of death globally. However, current techniques suffer from incomplete and inaccurate motion estimation of the myocardium in both spatial and temporal dimensions, hindering the early identification of myocardial dysfunction. To address these challenges, this paper introduces the Neural Cardiac Motion Field (NeuralCMF). NeuralCMF leverages implicit neural representation (INR) to model the 3D structure and the comprehensive 6D forward/backward motion of the heart. This method surpasses pixel-wise limitations by offering the capability to continuously query the precise shape and motion of the myocardium at any specific point throughout the cardiac cycle, enhancing the detailed analysis of cardiac dynamics beyond traditional speckle tracking. Notably, NeuralCMF operates without the need for paired datasets, and its optimization is self-supervised through the physics knowledge priors in both space and time dimensions, ensuring compatibility with both 2D and 3D echocardiogram video inputs. Experimental validations across three representative datasets support the robustness and innovative nature of the NeuralCMF, marking significant advantages over existing state-of-the-art methods in cardiac imaging and motion tracking. Code is available at: https://njuvision.github.io/NeuralCMF.