Sparse single-pixel imaging via optimization in nonuniform sampling sparsity.

Reducing the imaging time while maintaining reconstruction accuracy remains challenging for single-pixel imaging. One cost-effective approach is nonuniform sparse sampling. The existing methods lack intuitive and intrinsic analysis in sparsity. The lack impedes our comprehension of the form's adjustable range and may potentially limit our ability to identify an optimal distribution form within a confined adjustable range, consequently impacting the method's overall performance. In this Letter, we report a sparse sampling method with a wide adjustable range and define a sparsity metric to guide the selection of sampling forms. Through a comprehensive analysis and discussion, we select a sampling form that yields satisfying accuracy. These works will make up for the existing methods' lack of sparsity analysis and help adjust methods to accommodate different situations and needs.

Deep nonlocal low-rank regularization for complex-domain pixel super-resolution.

Pixel super-resolution (PSR) has emerged as a promising technique to break the sampling limit for phase imaging systems. However, due to the inherent nonconvexity of phase retrieval problem and super-resolution process, PSR algorithms are sensitive to noise, leading to reconstruction quality inevitably deteriorating. Following the plug-and-play framework, we introduce the nonlocal low-rank (NLR) regularization for accurate and robust PSR, achieving a state-of-the-art performance. Inspired by the NLR prior, we further develop the complex-domain nonlocal low-rank network (CNLNet) regularization to perform nonlocal similarity matching and low-rank approximation in the deep feature domain rather than the spatial domain of conventional NLR. Through visual and quantitative comparisons, CNLNet-based reconstruction shows an average 1.4 dB PSNR improvement over conventional NLR, outperforming existing algorithms under various scenarios.

High-resolution single-photon imaging with physics-informed deep learning.

High-resolution single-photon imaging remains a big challenge due to the complex hardware manufacturing craft and noise disturbances. Here, we introduce deep learning into SPAD, enabling super-resolution single-photon imaging with enhancement of bit depth and imaging quality. We first studied the complex photon flow model of SPAD electronics to accurately characterize multiple physical noise sources, and collected a real SPAD image dataset (64 × 32 pixels, 90 scenes, 10 different bit depths, 3 different illumination flux, 2790 images in total) to calibrate noise model parameters. With this physical noise model, we synthesized a large-scale realistic single-photon image dataset (image pairs of 5 different resolutions with maximum megapixels, 17250 scenes, 10 different bit depths, 3 different illumination flux, 2.6 million images in total) for subsequent network training. To tackle the severe super-resolution challenge of SPAD inputs with low bit depth, low resolution, and heavy noise, we further built a deep transformer network with a content-adaptive self-attention mechanism and gated fusion modules, which can dig global contextual features to remove multi-source noise and extract full-frequency details. We applied the technique in a series of experiments including microfluidic inspection, Fourier ptychography, and high-speed imaging. The experiments validate the technique's state-of-the-art super-resolution SPAD imaging performance.

Robust Kramers-Kronig holographic imaging with Hilbert-Huang transform.

Holography based on Kramers-Kronig relations (KKR) is a promising technique due to its high-space-bandwidth product. However, the absence of an iterative process limits its noise robustness, primarily stemming from the lack of a regularization constraint. This Letter reports a generalized framework aimed at enhancing the noise robustness of KKR holography. Our proposal involves employing the Hilbert-Huang transform to connect the real and imaginary parts of an analytic function. The real part is initially processed by bidimensional empirical mode decomposition into a series of intrinsic mode functions (IMFs) and a residual term. They are then selected to remove the noise and bias terms. Finally, the imaginary part can be obtained using the Hilbert transform. In this way, we efficiently suppress the noise in the synthetic complex function, facilitating high-fidelity wavefront reconstruction using ∼20% of the exposure time required by existing methods. Our work is expected to expand the applications of KKR holography, particularly in low phototoxicity biological imaging and other related scenarios.

Masked autoencoder for highly compressed single-pixel imaging.

The single-pixel imaging technique uses multiple patterns to modulate the entire scene and then reconstructs a two-dimensional (2-D) image from the single-pixel measurements. Inspired by the statistical redundancy of natural images that distinct regions of an image contain similar information, we report a highly compressed single-pixel imaging technique with a decreased sampling ratio. This technique superimposes an occluded mask onto modulation patterns, realizing that only the unmasked region of the scene is modulated and acquired. In this way, we can effectively decrease 75% modulation patterns experimentally. To reconstruct the entire image, we designed a highly sparse input and extrapolation network consisting of two modules: the first module reconstructs the unmasked region from one-dimensional (1-D) measurements, and the second module recovers the entire scene image by extrapolation from the neighboring unmasked region. Simulation and experimental results validate that sampling 25% of the region is enough to reconstruct the whole scene. Our technique exhibits significant improvements in peak signal-to-noise ratio (PSNR) of 1.5 dB and structural similarity index measure (SSIM) of 0.2 when compared with conventional methods at the same sampling ratios. The proposed technique can be widely applied in various resource-limited platforms and occluded scene imaging.

PeakForce AFM Analysis Enhanced with Model Reduction Techniques.

PeakForce quantitative nanomechanical AFM mode (PF-QNM) is a popular AFM technique designed to measure multiple mechanical features (e.g., adhesion, apparent modulus, etc.) simultaneously at the exact same spatial coordinates with a robust scanning frequency. This paper proposes compressing the initial high-dimensional dataset obtained from the PeakForce AFM mode into a subset of much lower dimensionality by a sequence of proper orthogonal decomposition (POD) reduction and subsequent machine learning on the low-dimensionality data. A substantial reduction in user dependency and subjectivity of the extracted results is obtained. The underlying parameters, or "state variables", governing the mechanical response can be easily extracted from the latter using various machine learning techniques. Two samples are investigated to illustrate the proposed procedure (i) a polystyrene film with low-density polyethylene nano-pods and (ii) a PDMS film with carbon-iron particles. The heterogeneity of material, as well as the sharp variation in topography, make the segmentation challenging. Nonetheless, the underlying parameters describing the mechanical response naturally offer a compact representation allowing for a more straightforward interpretation of the high-dimensional force-indentation data in terms of the nature (and proportion) of phases, interfaces, or topography. Finally, those techniques come with a low processing time cost and do not require a prior mechanical model.

Three-phase material mapping with incomplete X-ray diffraction spectral information.

An equiatomic nickel-titanium shape-memory alloy specimen subjected to a uniaxial tensile load undergoes a two-step phase transformation under stress, from austenite (A) to a rhombohedral phase (R) and further to martensite (M) variants. The pseudo-elasticity that goes accompanies the phase transformation induces spatial inhomogeneity. To unravel the spatial distribution of the phases, in situ X-ray diffraction analyses are performed while the sample is under tensile load. However, the diffraction spectra of the R phase, as well as the extent of potential martensite detwinning, are not known. A novel algorithm, based on a proper orthogonal decomposition and incorporating inequality constraints, is proposed in order to map out the different phases and simultaneously yield the missing diffraction spectral information. An experimental case study illustrates the methodology.

Self-supervised learning for single-pixel imaging via dual-domain constraints.

Deep-learning-augmented single-pixel imaging (SPI) provides an efficient solution for target compressive sensing. However, the conventional supervised strategy suffers from laborious training and poor generalization. In this Letter, we report a self-supervised learning method for SPI reconstruction. It introduces dual-domain constraints to integrate the SPI physics model into a neural network. Specifically, in addition to the traditional measurement constraint, an extra transformation constraint is employed to ensure target plane consistency. The transformation constraint uses the invariance of reversible transformation to impose an implicit prior, which avoids the non-uniqueness of measurement constraint. A series of experiments validate that the reported technique realizes self-supervised reconstruction in various complex scenes without any paired data, ground truth, or pre-trained prior. It can tackle the underdetermined degradation and noise, with ∼3.7-dB improvement on the PSNR index compared with the existing method.

Phase retrieval via nonlocal complex-domain sparsity.

Phase retrieval is indispensable for a number of coherent imaging systems. Owing to limited exposure, it is a challenge for traditional phase retrieval algorithms to reconstruct fine details in the presence of noise. In this Letter, we report an iterative framework for noise-robust phase retrieval with high fidelity. In the framework, we investigate nonlocal structural sparsity in the complex domain by low-rank regularization, which effectively suppresses artifacts caused by measurement noise. The joint optimization of sparsity regularization and data fidelity with forward models enables satisfying detail recovery. To further improve computational efficiency, we develop an adaptive iteration strategy that automatically adjusts matching frequency. The effectiveness of the reported technique has been validated for coherent diffraction imaging and Fourier ptychography, with ≈7 dB higher peak SNR (PSNR) on average, compared with conventional alternating projection reconstruction.

Plug-and-play pixel super-resolution phase retrieval for digital holography.

In order to increase signal-to-noise ratio in optical imaging, most detectors sacrifice resolution to increase pixel size in a confined area, which impedes further development of high throughput holographic imaging. Although the pixel super-resolution technique (PSR) enables resolution enhancement, it suffers from the trade-off between reconstruction quality and super-resolution ratio. In this work, we report a high-fidelity PSR phase retrieval method with plug-and-play optimization, termed PNP-PSR. It decomposes PSR reconstruction into independent sub-problems based on generalized alternating projection framework. An alternating projection operator and an enhancing neural network are employed to tackle the measurement fidelity and statistical prior regularization, respectively. PNP-PSR incorporates the advantages of individual operators, achieving both high efficiency and noise robustness. Extensive experiments show that PNP-PSR outperforms the existing techniques in both resolution enhancement and noise suppression.

Deep distributed optimization for blind diffuser-modulation ptychography.

Blind diffuser-modulation ptychography has emerged as a low-cost technique for micro-nano holographic imaging, which enables breaking the resolution limit of optical systems. However, the existing reconstruction method requires thousands of measurements to recover object and diffuser profile simultaneously, which makes the data acquisition time-consuming and cumbersome. In this Letter, we report a novel, to the best of our knowledge, blind ptychography technique with deep distributed optimization, termed BPD2O. It decomposes the complicated optimization task into subproblems, then introduces extended ptychographical iterative engine and enhanced network solver to optimize each in a distributed strategy. In this way, BPD2O combines the advantages of both model-driven and data-driven strategies, realizing high-fidelity robust ptychography imaging. Extensive experiments validate that BPD2O can realize better resolution and lead to a reduction of more than one order of magnitude in the number of measurements.