Deep learning sheds new light on non-orthogonal optical multiplexing.

A deep neural network for non-orthogonal input channel encoding is proposed to recover speckle images through a multimode fiber. This novel approach could shed new light on the non-orthogonal optical multiplexing over a scattering medium.

Decoding of compressive data pages for optical data storage utilizing FFDNet.

Coded aperture-based compression has proven to be an effective approach for high-density cold data storage. Nevertheless, its limited decoding speed represents a significant challenge for its broader application. We introduce a novel, to the best of our knowledge, decoding method leveraging the fast and flexible denoising network (FFDNet), capable of decoding a coded aperture-based compressive data page within 30.64 s. The practicality of the method has been confirmed in the decoding of monochromatic photo arrays, full-color photos, and dynamic videos. In experimental trials, the variance between decoded results obtained via the FFDNet-based method and the FFDNet-absent method in terms of average PSNR is less than 1 dB, while realizing a decoding speed enhancement of over 100-fold when employing the FFDNet-based method.

Adaptive layer-based computer-generated holograms.

We propose a novel, to the best of our knowledge, and fast adaptive layer-based (ALB) method for generating a computer-generated hologram (CGH) with accurate depth information. A complex three-dimensional (3D) object is adaptively divided into layers along the depth direction according to its own non-uniformly distributed depth coordinates, which reduces the depth error caused by the conventional layer-based method. Each adaptive layer generates a single-layer hologram using the angular spectrum method for diffraction, and the final hologram of a complex three-dimensional object is obtained by superimposing all the adaptive layer holograms. A hologram derived with the proposed method is referred to as an adaptive layer-based hologram (ALBH). Our demonstration shows that the desired reconstruction can be achieved with 52 adaptive layers in 8.7 s, whereas the conventional method requires 397 layers in 74.9 s.

Super-resolution lensless on-chip microscopy based on array illumination and sub-pixel shift search.

The resolution of a lensless on-chip microscopy system is constrained by the pixel size of image sensors. This Letter introduces a super-resolution on-chip microscopy system based on a compact array light source illumination and sub-pixel shift search. The system utilizes a closely spaced array light source composed by four RGB LED modules, sequentially illuminating the sample. A sub-pixel shift search algorithm is proposed, which determines the sub-pixel shift by comparing the frequency of captured low-resolution holograms. Leveraging this sub-pixel shift, a super-resolution reconstruction algorithm is introduced, building upon a multi-wavelength phase retrieval method, enabling the rapid super-resolution reconstruction of holograms with the region-of-interest. The system and algorithms presented herein obviate the need for a displacement control platform and calibration of the illumination angles of the light source, facilitating a super-resolution phase reconstruction under partially coherent illumination.

Computational Imaging: The Next Revolution for Biophotonics and Biomedicine.

This Editorial is the preface for the topical collection of "Computational Imaging for Biophotonics and Biomedicine", which collates the 12 contributions listed in Table 1 [...].

Lensless imaging in one shot using the complex degree of coherence obtained by multiaperture interferences.

The van Cittert-Zernike theorem states that the Fourier transform of the intensity distribution function of a distant, incoherent source is equal to the complex degree of coherence. In this Letter, we present a method for measuring the complex degree of coherence in one shot by recording the interference patterns produced by multiple aperture pairs. The intensity of the sample is obtained by Fourier transforming the complex degree of coherence. The experimental verification by using a simple object is presented together with a discussion on how the method could be improved for imaging more complex samples.

Single-shot deterministic complex amplitude imaging with a single-layer metalens.

Conventional imaging systems can only capture light intensity. Meanwhile, the lost phase information may be critical for a variety of applications such as label-free microscopy and optical metrology. Existing phase retrieval techniques typically require a bulky setup, multiframe measurements, or prior information of the target scene. Here, we proposed an extremely compact system for complex amplitude imaging, leveraging the extreme versatility of a single-layer metalens to generate spatially multiplexed and polarization phase-shifted point spread functions. Combining the metalens with a polarization camera, the system can simultaneously record four polarization shearing interference patterns along both in-plane directions, thus allowing the deterministic reconstruction of the complex amplitude light field in a single shot. Using an incoherent light-emitting diode as the illumination, we experimentally demonstrated speckle-noise-free complex amplitude imaging for both static and moving objects with tailored magnification ratio and field of view. The miniaturized and robust system may open the door for complex amplitude imaging in portable devices for point-of-care applications.

Deep nonlocal low-rank regularization for complex-domain pixel super-resolution.

Pixel super-resolution (PSR) has emerged as a promising technique to break the sampling limit for phase imaging systems. However, due to the inherent nonconvexity of phase retrieval problem and super-resolution process, PSR algorithms are sensitive to noise, leading to reconstruction quality inevitably deteriorating. Following the plug-and-play framework, we introduce the nonlocal low-rank (NLR) regularization for accurate and robust PSR, achieving a state-of-the-art performance. Inspired by the NLR prior, we further develop the complex-domain nonlocal low-rank network (CNLNet) regularization to perform nonlocal similarity matching and low-rank approximation in the deep feature domain rather than the spatial domain of conventional NLR. Through visual and quantitative comparisons, CNLNet-based reconstruction shows an average 1.4 dB PSNR improvement over conventional NLR, outperforming existing algorithms under various scenarios.

Coded aperture-based compressive data page for optical data storage.

In an era of data explosion, optical data storage provides an alternative solution for cold data storage due to its energy-saving and cost-effective features. However, its data density is still insufficient for zettabyte-scale cold data storage. Here, a coded aperture-based compressive data page with a compression ratio of ≤0.125 is proposed. Based on two frameworks-weighted nuclear norm minimization (WNNM) and alternating direction method of multipliers (ADMM)-the decoded quality of the compressive data page is ensured by utilizing sparsity priors. In experiments, compressive data pages of a monochromatic photo-array, full-color photo, and dynamic video are accurately decoded.

DGE-CNN: 2D-to-3D holographic display based on a depth gradient extracting module and ZCNN network.

Holography is a crucial technique for the ultimate three-dimensional (3D) display, because it renders all optical cues from the human visual system. However, the shortage of 3D contents strictly restricts the extensive application of holographic 3D displays. In this paper, a 2D-to-3D-display system by deep learning-based monocular depth estimation is proposed. By feeding a single RGB image of a 3D scene into our designed DGE-CNN network, a corresponding display-oriented 3D depth map can be accurately generated for layer-based computer-generated holography. With simple parameter adjustment, our system can adapt the distance range of holographic display according to specific requirements. The high-quality and flexible holographic 3D display can be achieved based on a single RGB image without 3D rendering devices, permitting potential human-display interactive applications such as remote education, navigation, and medical treatment.

Amp-vortex edge-camera: a lensless multi-modality imaging system with edge enhancement.

We demonstrate a lensless imaging system with edge-enhanced imaging constructed with a Fresnel zone aperture (FZA) mask placed 3 mm away from a CMOS sensor. We propose vortex back-propagation (vortex-BP) and amplitude vortex-BP algorithms for the FZA-based lensless imaging system to remove the noise and achieve the fast reconstruction of high contrast edge enhancement. Directionally controlled anisotropic edge enhancement can be achieved with our proposed superimposed vortex-BP algorithm. With different reconstruction algorithms, the proposed amp-vortex edge-camera in this paper can achieve 2D bright filed imaging, isotropic, and directional controllable anisotropic edge-enhanced imaging with incoherent light illumination, by a single-shot captured hologram. The effect of edge detection is the same as optical edge detection, which is the re-distribution of light energy. Noise-free in-focus edge detection can be achieved by using back-propagation, without a de-noise algorithm, which is an advantage over other lensless imaging technologies. This is expected to be widely used in autonomous driving, artificial intelligence recognition in consumer electronics, etc.

Polarimetric calibrated robust dual-SLM complex-amplitude computer-generated holography.

Liquid crystal on silicon (LCoS) is a widely used spatial light modulator (SLM) in computer-generated holography (CGH). However, the phase-modulating profile of LCoS is often not ideally uniform in application, bringing about undesired intensity fringes. In this study, we overcome this problem by proposing a highly robust dual-SLM complex-amplitude CGH technique, which incorporates a polarimetric mode and a diffractive mode. The polarimetric mode linearizes the general phase modulations of the two SLMs separately, while the diffractive mode uses camera-in-the-loop optimization to achieve improved holographic display. Experimental results show the effectiveness of our proposal in improving reconstructing accuracy by 21.12% in peak signal-to-noise ratio (PSNR) and 50.74% in structure similarity index measure (SSIM), using LCoS SLMs with originally non-uniform phase-modulating profiles.

Analysis of reconstruction quality for computer-generated holograms using a model free of circular-convolution error.

Continuous complex-amplitude computer-generated holograms (CGHs) are converted to discrete amplitude-only or phase-only ones in practical applications to cater for the characteristics of spatial light modulators (SLMs). To describe the influence of the discretization correctly, a refined model that eliminates the circular-convolution error is proposed to emulate the propagation of the wavefront during the formation and reconstruction of a CGH. The effects of several significant factors, including quantized amplitude and phase, zero-padding rate, random phase, resolution, reconstruction distance, wavelength, pixel pitch, phase modulation deviation and pixel-to-pixel interaction, are discussed. Based on evaluations, the optimal quantization for both available and future SLM devices is suggested.

Toward a see-through camera via AR lightguide.

As the foundation of virtual content generation, cameras are crucial for augmented reality (AR) applications, yet their integration with transparent displays has remained a challenge. Prior efforts to develop see-through cameras have struggled to achieve high resolution and seamless integration with AR displays. In this work, we present LightguideCam, a compact and flexible see-through camera based on an AR lightguide. To address the overlapping artifacts in measurement, we present a compressive sensing algorithm based on an equivalent imaging model that minimizes computational consumption and calibration complexity. We validate our design using a commercial AR lightguide and demonstrate a field of view of 23.1° and an angular resolution of 0.1° in the prototype. Our LightguideCam has great potential as a plug-and-play extensional imaging component in AR head-mounted displays, with promising applications for eye-gaze tracking, eye-position perspective photography, and improved human-computer interaction devices, such as full-screen mobile phones.

Adaptive constraints by morphological operations for single-shot digital holography.

Digital holography provides access to quantitative measurement of the entire complex field, which is indispensable for the investigation of wave-matter interactions. The emerging iterative phase retrieval approach enables to solve the inverse imaging problem only from the given intensity measurements and physical constraints. However, enforcing imprecise constraints limits the reconstruction accuracy and convergence speed. Here, we propose an advanced iterative phase retrieval framework for single-shot in-line digital holography that incorporates adaptive constraints, which achieves optimized convergence behavior, high-fidelity and twin-image-free reconstruction. In conjunction with morphological operations which can extract the object structure while eliminating the irrelevant part such as artifacts and noise, adaptive constraints allow the support region to be accurately estimated and automatically updated at each iteration. Numerical reconstruction of complex-valued objects and the capability of noise immunity are investigated. The improved reconstruction performance of this approach is experimentally validated. Such flexible and versatile framework has promising applications in biomedicine, X-ray coherent diffractive imaging and wavefront sensing.

Hough transform-based multi-object autofocusing compressive holography.

Reconstruction of multiple objects from one hologram can be affected by the focus metric judgment of autofocusing. Various segmentation algorithms are applied to obtain a single object in the hologram. Each object is unambiguously reconstructed to acquire its focal position, which produces complicated calculations. Herein, Hough transform (HT)-based multi-object autofocusing compressive holography is presented. The sharpness of each reconstructed image is computed by using a focus metric such as entropy or variance. According to the characteristics of the object, the standard HT is further used for calibration to remove redundant extreme points. The compressive holographic imaging framework with a filter layer can eliminate the inherent noise in in-line reconstruction including cross talk noise of different depth layers, two-order noise, and twin image noise. The proposed method can effectively obtain 3D information on multiple objects and achieve noise elimination by only reconstructing from one hologram.

Phase aberration separation for holographic microscopy by alternating direction sparse optimization.

The morphology and dynamics of label-free tissues can be exploited by sample-induced changes in the optical field from quantitative phase imaging. Its sensitivity to subtle changes in the optical field makes the reconstructed phase susceptible to phase aberrations. We import variable sparse splitting framework on quantitative phase aberration extraction based on alternating direction aberration free method. The optimization and regularization in the reconstructed phase are decomposed into object terms and aberration terms. By formulating the aberration extraction as a convex quadratic problem, the background phase aberration can be fast and directly decomposed with the specific complete basis functions such as Zernike or standard polynomials. Faithful phase reconstruction can be obtained by eliminating global background phase aberration. The aberration-free two-dimensional and three-dimensional imaging experiments are demonstrated, showing the relaxation of the strict alignment requirements for the holographic microscopes.

Autofocusing of Fresnel zone aperture lensless imaging for QR code recognition.

Fresnel zone aperture (FZA) lensless imaging encodes the incident light into a hologram-like pattern, so that the scene image can be numerically focused at a long imaging range by the back propagation method. However, the target distance is uncertain. The inaccurate distance causes blurs and artifacts in the reconstructed images. This brings difficulties for the target recognition applications, such as quick response code scanning. We propose an autofocusing method for FZA lensless imaging. By incorporating the image sharpness metrics into the back propagation reconstruction process, the method can acquire the desired focusing distance and reconstruct noise-free high-contrast images. By combining the Tamura of the gradient metrics and nuclear norm of gradient, the relative error of estimated object distance is only 0.95% in the experiment. The proposed reconstruction method significantly improves the mean recognition rate of QR code from 4.06% to 90.00%. It paves the way for designing intelligent integrated sensors.

Non-Invasive and Minute-Frequency 3D Tomographic Imaging Enabling Long-Term Spatiotemporal Observation of Single Cell Fate.

Non-invasive and rapid imaging technique at subcellular resolution is significantly important for multiple biological applications such as cell fate study. Label-free refractive-index (RI)-based 3D tomographic imaging constitutes an excellent candidate for 3D imaging of cellular structures, but its full potential in long-term spatiotemporal cell fate observation is locked due to the lack of an efficient integrated system. Here, a long-term 3D RI imaging system incorporating a cutting-edge white light diffraction phase microscopy module with spatiotemporal stability, and an acoustofluidic device to roll and culture single cells in a customized live cell culture chamber is reported. Using this system, 3D RI imaging experiments are conducted for 250 cells and demonstrate efficient cell identification with high accuracy. Importantly, long-term and frequency-on-demand 3D RI imaging of K562 and MCF-7 cancer cells reveal different characteristics during normal cell growth, drug-induced cell apoptosis, and necrosis of drug-treated cells. Overall, it is believed that the proposed 3D tomographic imaging technique opens up a new avenue for visualizing intracellular structures and will find many applications such as disease diagnosis and nanomedicine.

Monocular metasurface camera for passive single-shot 4D imaging.

It is a grand challenge for an imaging system to simultaneously obtain multi-dimensional light field information, such as depth and polarization, of a scene for the accurate perception of the physical world. However, such a task would conventionally require bulky optical components, time-domain multiplexing, and active laser illumination. Here, we experimentally demonstrate a compact monocular camera equipped with a single-layer metalens that can capture a 4D image, including 2D all-in-focus intensity, depth, and polarization of a target scene in a single shot under ambient illumination conditions. The metalens is optimized to have a conjugate pair of polarization-decoupled rotating single-helix point-spread functions that are strongly dependent on the depth of the target object. Combined with a straightforward, physically interpretable image retrieval algorithm, the camera can simultaneously perform high-accuracy depth sensing and high-fidelity polarization imaging over an extended depth of field for both static and dynamic scenes in both indoor and outdoor environments. Such a compact multi-dimensional imaging system could enable new applications in diverse areas ranging from machine vision to microscopy.