Light-field photography using differential high-speed aperture coding.

Programmable aperture light-field photography enables the acquisition of angular information without compromising spatial resolution. However, direct current (DC) background noise is unavoidable in images recorded by programmable aperture light-field photography, leading to reducing the contrast of reconstructed images. In addition, it requires sacrificing temporal resolution to obtain angular information, making it a challenge to capture dynamic scenes. In this paper, we propose programmable aperture light-field photography using differential high-speed aperture coding. This method effectively reduces DC noise and produces high-contrast refocused images. Furthermore, we build a light-field camera based on a 1250 Hz spatial light modulator and a 1250 fps high-speed camera, achieving dynamic light-field photography at 1110(H)×800(V) resolution and 24 fps. Our results demonstrate significant improvements in image contrast and exhibit considerable promise for diverse applications.

Single-shot differential phase contrast microscopy using ring-shaped polarisation multiplexing illumination.

We propose a differential phase contrast microscopy that enables single-shot phase imaging for unstained biological samples. The proposed approach employs a ring-shaped LED array for polarisation multiplexing illumination and a polarisation camera for image acquisition. As such, multiple images of different polarisation angles can be simultaneously captured with a single shot. Through polarisation demultiplexing, the sample phase can therefore be recovered from the single-shot measurement. Both simulations and experiments demonstrate the effectiveness of the approach. We also demonstrate that ring-shaped illumination enables higher contrast and lower-distortion imaging results than disk-shaped illumination does. The proposed single-shot approach potentially enables phase contrast imaging for live cell samples in vitro. Lay Description: We propose a microscopy that enables imaging of transparent samples, unstained cells, etc. We demonstrate that the proposed method enables higher contrast and lower-distortion imaging results than conventional methods, and significantly improves imaging efficiency. The proposed method potentially enables dynamic imaging for live cell samples in vitro.

Adaptive real-time single-pixel imaging.

For most imaging systems, there is a trade-off between spatial resolution, temporal resolution, and signal-to-noise ratio. Such a trade-off is particularly severe in single-pixel imaging systems, given the limited throughput of the only one available pixel. Here we report a real-time single-pixel imaging method that can adaptively balance the spatial resolution, temporal resolution, and signal-to-noise ratio of the imaging system according to the changes in the target scene. When scene changes are detected, the dynamic imaging mode will be activated. The temporal resolution will be given high priority and real-time single-pixel imaging will be conducted at a video frame rate (30 frames/s) to visualize the object motion. When no scene changes are detected, the static imaging mode will be activated. The spatial resolution and the signal-to-noise ratio will be progressively built up to resolve fine structures and to improve image quality. The proposed method not only adds practicability to single-pixel imaging, but also generates a new, to the best of our knowledge, insight in data redundancy reduction and information capacity improvement for other computational imaging schemes.

Autofocus Fourier single-pixel microscopy.

Single-pixel microscopy enables observation of micro samples in invisible wave bands. Finding the focus position is essential to capture a clear image of a sample but could be difficult for single-pixel microscopy particularly in invisible wave bands. It is because the structured patterns projected onto the sample would be invisible and searching for the focus position manually could be exhausting. Here, we report an autofocus method for Fourier single-pixel microscopy. The reported method allows one to find the focus position without recording or reconstructing a complete image. The focus position is determined by the magnitude summation of a small number of Fourier coefficients, which enables fast autofocus. The reported method is experimentally demonstrated in imaging various objects in both visible and near-infrared wave bands. The method adds practicability to a single-pixel microscopy.

Contrast-enhanced, single-shot LED array microscopy based on Fourier ptychographic algorithm and deep learning.

LED array microscopes have the advantages of miniaturisation and low cost. It has been demonstrated that LED array microscopes outperform Köhler illumination microscopes in some applications. A LED array allows for a large numerical aperture of illumination. The larger numerical aperture of illumination brings the higher spatial resolution, but the lower image contrast as well. Therefore, there is a tradeoff between resolution and contrast for LED array microscopes. The Fourier ptychographic algorithm can overcome this tradeoff by increasing image contrast without sacrificing spatial resolution. However, the Fourier ptychographic algorithm requires acquisition of multiple images, which is time-consuming and results in live sample imaging challenging. To solve this problem, we develop contrast-enhanced, single-shot LED array microscopy based on the Fourier ptychographic algorithm and deep learning. The sample to be imaged is under illumination by all LEDs of the array simultaneously. The image captured is fed to several trained convolutional neural networks to generate the same number of images that are required by the Fourier ptychographic algorithm. We experimentally present that the image contrast of the final reconstruction is remarkably improved in comparison with the image captured. The proposed method can also produce chromatic-aberration-free results, even when an objective without aberration correction is used. We believe the method might provide live sample imaging with a low-cost approach.

Image-free active autofocusing with dual modulation and its application to Fourier single-pixel imaging.

Autofocusing is widely used in applications where sharp image acquisition or projection is needed. Here we report an active autofocusing method for sharp image projection. The method works with wide-field structured illumination and single-pixel detection. To find the focus position, the method illuminates the target object with a set of 3-step phase-shifting Fourier basis patterns repeatedly and collects the backscattered light by using a single-pixel detector through a grating. Dual modulation-dynamic modulation by the time-varying structured illumination and static modulation by the grating-embeds the depth information for the target object in the resulting single-pixel measurements. As such, the focus position can be determined by recovering the Fourier coefficients from the single-pixel measurements and searching for the coefficient with the maximum magnitude. High-speed spatial light modulation not only enables rapid autofocusing but also makes the method work even when the lens system is in continuous motion or the focal length of the lens is continuously adjusted. We experimentally validate the reported method in a self-built digital projector and demonstrate the application of the method in Fourier single-pixel imaging.

Full-resolution, full-field-of-view, and high-quality fast Fourier single-pixel imaging.

Fourier single-pixel imaging (FSI) uses Fourier basis patterns for spatial light modulation to acquire the Fourier spectrum of the object image. The object image can be reconstructed via an inverse Fourier transform. However, the Fourier basis patterns are inherently gray scale, which results in the difficulty that the patterns can hardly be generated at a high speed by using a commonly used spatial light modulator-digital micromirrors device. To tackle this problem, fast FSI, which uses upsampled and dithered Fourier basis patterns to approximate the gray scale patterns, has been reported, but the achievable spatial resolution has to be sacrificed in the pattern upsampling process. Here we propose a method that can achieve not only full-resolution but also full-field-of-view and high-quality FSI. The key to the proposed method is to use a new, to the best of our knowledge, error diffusion dithering algorithm combined with two different scanning strategies to generate two sets of binarized Fourier basis patterns for spatial light modulation. As a result, two images with a sub-pixel shift from each other are reconstructed. It results in the final high-quality reconstruction by synthesizing the two images. We experimentally demonstrate the method can produce a high-quality 1024 × 768-pixel and full resolution image with a digital micromirror device with 1024 × 768 micromirrors.

Robust multi-angle structured illumination lensless microscopy via illumination angle calibration.

Multi-angle structured illumination lensless (MASIL) microscopy enables high-resolution image recovery over a large field of view. Successful image recovery of MASIL microscopy, however, relies on an accurate knowledge of the multi-angle illumination. System misalignments and slight deviations from the true illumination angle may result in image artifacts in reconstruction. Here we report a MASIL microscopy system that is robust against illumination misalignment. To calibrate the illumination angles, we design and use a double-sided mask, which is a glass wafer fabricated with a ring-array pattern on the upper surface and a disk-array pattern on the lower surface. As such, the illumination angles can be decoded from the captured images by estimating the relative displacement of the two patterns. We experimentally demonstrate that this system can achieve successful image recovery without any prior knowledge of the illumination angles. The reported approach provides a simple yet robust resolution for wide-field lensless microscopy. It can solve the LED array misalignment problem and calibrate angle-varied illumination for a variety of applications.

Ringing-free fast Fourier single-pixel imaging.

Fourier single-pixel imaging (FSI) allows an image to be reconstructed by acquiring the Fourier spectrum of the image using a single-pixel detector. Fast FSI is typically achieved by acquiring a truncated Fourier spectrum, that is, only low-frequency Fourier coefficients are acquired, with the high-frequency coefficients discarded. However, the truncation of the Fourier spectrum leads to undesirable ringing artifacts in the resulting image. Ringing artifacts produce false edges in the image and reduce the image contrast, resulting in image quality degeneration. The artifact is particularly severe in dynamic FSI, where the sampling ratio is generally ultra-low. We propose an effective and fast deringing algorithm to achieve ringing-free fast FSI. The algorithm eliminates ringing artifacts through 2D sub-pixel shifting and preserves image details through image fusion. Both static and dynamic imaging results demonstrate that the proposed method can reconstruct ringing-free images from under-sampled data in FSI. The deringing algorithm not only provides FSI with the capability of fast high-quality single-pixel imaging but also might prove its applicability in other areas, such as Fourier-based data compression algorithms.

Mask-modulated lensless imaging via translated structured illumination.

Lensless microscopy technique enables high-resolution image recovery over a large field of view. By integrating the concept of phase retrieval, it can also retrieve the lost phase information from intensity-only measurements. Here we report a mask-modulated lensless imaging platform based on translated structured illumination. In the reported platform, we sandwich the object in-between a coded mask and a naked image sensor for lensless data acquisition. An LED array is used to provide angle-varied illumination for projecting a translated structured pattern without involving mechanical scanning. For different LED elements, we acquire the lensless intensity data for recovering the complex-valued object. In the reconstruction process, we employ the regularized ptychographic iterative engine and implement an up-sampling process in the reciprocal space. As demonstrated by experimental results, the reported platform is able to recover complex-valued object images with higher resolution and better quality than previous implementations. Our approach may provide a cost-effective solution for high-resolution and wide field-of-view ptychographic imaging without involving mechanical scanning.

Image-free real-time 3-D tracking of a fast-moving object using dual-pixel detection.

Real-time 3-D tracking of a fast-moving object has found important applications in industry, traffic control, sports, biomedicine, defense, etc. However, it is difficult to adopt typical image-based object tracking systems in a fast-moving object tracking in real time and for a long duration, because reliable and robust image processing and analysis algorithms are often computationally exhausted, and limited storage and bandwidth can hardly fulfill the great demand of high-speed photography. Here we report an image-free 3-D tracking approach. The approach uses only two single-pixel detectors and a high-speed spatial light modulator for data acquisition. By illuminating the target moving object with six single-period Fourier basis patterns, the approach is able to analytically calculate the position of the object with the corresponding single-pixel measurements. The approach is low-cost, and data- and computation-efficient. We experimentally demonstrate that the proposed approach can detect and track a fast-moving object at a frame rate of 1666 frames per second by using a 10,000 Hz digital micromirror device. Benefiting from the wide working spectrum of single-pixel detectors, the reported approach might be applicable for hidden fast-moving object tracking.

Efficient full-color single-pixel imaging based on the human vision property-"giving in to the blues".

Single-pixel imaging is a novel, to the best of our knowledge, computational imaging scheme, but a large number of measurements are typically required in data acquisition. Full-color single-pixel imaging takes many more measurements than does monochromatic single-pixel imaging. Utilizing the fact that human eyes have a poorer spatial resolution to blues than reds and greens, we propose to sample the blue component of color images with an ultra-low sampling ratio so as to reduce the number of measurements. We demonstrate our method with simulations and experiments, concluding that 95% of the measurements can be reduced in the acquisition of the blue component of natural color images in the size of 256×256 pixels, and the resulting images are without remarkable visual loss. Moreover, utilizing the sparsity of natural images, the sampling ratios of the red and green components can be reduced to 15% and 50%, respectively. This Letter may generate a new insight of how to optimize the imaging efficiency by utilizing human vision properties. The proposed method can be adopted by other full-color computational imaging techniques.

Image-free classification of fast-moving objects using "learned" structured illumination and single-pixel detection.

Object classification generally relies on image acquisition and subsequent analysis. Real-time classification of fast-moving objects is a challenging task. Here we propose an approach for real-time classification of fast-moving objects without image acquisition. The key to the approach is to use structured illumination and single-pixel detection to acquire the object features directly. A convolutional neural network (CNN) is trained to learn the object features. The "learned" object features are then used as structured patterns for structured illumination. Object classification can be achieved by picking up the resulting light signals by a single-pixel detector and feeding the single-pixel measurements to the trained CNN. In our experiments, we show that accurate and real-time classification of fast-moving objects can be achieved. Potential applications of the proposed approach include rapid classification of flowing cells, assembly-line inspection, and aircraft classification in defense applications. Benefiting from the use of a single-pixel detector, the approach might be applicable for hidden moving object classification.

Full-color light-field microscopy via single-pixel imaging.

Light-field microscopy is a scanless volumetric imaging technique. Conventional color light microscope employs a micro-lens array at the image plane and samples the spatial, angular, and color information by a pixelated two-dimensional (2D) sensor (such as CCD). However, the space bandwidth product of the pixelated 2D sensor is a fixed value determined by its parameters, leading to the trade-offs between the spatial, angular, and color resolutions. In addition, the inherent chromatic aberration of the micro-lens array also reduces the viewing quality. Here we propose full-color light-field microscopy via single-pixel imaging that can distribute the sampling tasks of the spatial, angular, and color information to both illumination and detection sides, rather than condense on the detection side. Therefore, the space bandwidth product of the light-field microscope is increased and the spatial resolution of the reconstructed light-field can be improved. In addition, the proposed method can reconstruct full-color light-field without using a micro-lens array, thereby the chromatic aberration induced by the micro-lens array is avoided. Because distributing the three sampling tasks to both the illumination and detection sides has different possible sampling schemes, we present two sampling schemes and compare their advantages and disadvantages via several experiments. Our work provides insight for developing a high-resolution full-color light-field microscope. It may find potential applications in the biomedical and material sciences.

Experimental observation of differential self-mixing interference signals using a randomly polarized laser: a differential self-mixing interferometry.

Differential measurement has strong anti-interference ability. We report the first, to the best of our knowledge, experimental demonstration of differential self-mixing interference signals using a randomly polarized laser for differential self-mixing interferometry (SMI). In the differential SMI system, the detection light can be divided into interference signals of $p$p-polarized and $s$s-polarized light with identical intensity and pi-shift phase difference. By exploiting such a new experimental phenomenon, we propose a noise-robust and low-cost self-mixing interferometer. Experiments show that the proposed approach can effectively suppress both periodic and aperiodic noise. Thus, the reported phenomenon and approach has a good application prospect in self-mixing interferometers.

In-vivo full-field measurement of microcirculatory blood flow velocity based on intelligent object identification.

Microcirculation plays a crucial role in delivering oxygen and nutrients to living tissues and in removing metabolic wastes from the human body. Monitoring the velocity of blood flow in microcirculation is essential for assessing various diseases, such as diabetes, cancer, and critical illnesses. Because of the complex morphological pattern of the capillaries, both In-vivo capillary identification and blood flow velocity measurement by conventional optical capillaroscopy are challenging. Thus, we focused on developing an In-vivo optical microscope for capillary imaging, and we propose an In-vivo full-field flow velocity measurement method based on intelligent object identification. The proposed method realizes full-field blood flow velocity measurements in microcirculation by employing a deep neural network to automatically identify and distinguish capillaries from images. In addition, a spatiotemporal diagram analysis is used for flow velocity calculation. In-vivo experiments were conducted, and the images and videos of capillaries were collected for analysis. We demonstrated that the proposed method is highly accurate in performing full-field blood flow velocity measurements in microcirculation. Further, because this method is simple and inexpensive, it can be effectively employed in clinics.

Model-Free Lens Distortion Correction Based on Phase Analysis of Fringe-Patterns.

The existing lens correction methods deal with the distortion correction by one or more specific image distortion models. However, distortion determination may fail when an unsuitable model is used. So, methods based on the distortion model would have some drawbacks. A model-free lens distortion correction based on the phase analysis of fringe-patterns is proposed in this paper. Firstly, the mathematical relationship of the distortion displacement and the modulated phase of the sinusoidal fringe-pattern are established in theory. By the phase demodulation analysis of the fringe-pattern, the distortion displacement map can be determined point by point for the whole distorted image. So, the image correction is achieved according to the distortion displacement map by a model-free approach. Furthermore, the distortion center, which is important in obtaining an optimal result, is measured by the instantaneous frequency distribution according to the character of distortion automatically. Numerical simulation and experiments performed by a wide-angle lens are carried out to validate the method.

Reflection light-field microscope with a digitally tunable aperture by single-pixel imaging.

Reflected light microscope is a tool for imaging opaque specimens. However, most of the existing reflected light microscopes can only obtain the two-dimensional image of the specimen. Here we demonstrate that with the help of single-pixel imaging, we can develop a reflection light-field microscopy for volumetric imaging. Importantly, using single-pixel imaging, we can digitally adjust the size of the aperture diaphragm of the proposed reflection light-field microscope for changing the depth of field and for achieving three-dimensional differential phase-contrast imaging in an arbitrary direction, without a hardware change. Our approach may benefit various reflective specimens with wide depth information in the semiconductor industry and material science.

Image-free real-time detection and tracking of fast moving object using a single-pixel detector.

Real-time detection and tracking for fast moving object has important applications in various fields. However, available methods, especially low-cost ones, can hardly achieve real-time and long-duration object detection and tracking. Here we report an image-free and cost-effective method for detecting and tracking a fast moving object in real time and for long duration. The method employs a spatial light modulator and a single-pixel detector for data acquisition. It uses Fourier basis patterns to illuminate the target moving object and collects the resulting light signal with a single-pixel detector. The proposed method is able to detect and track the object with the single-pixel measurements directly without image reconstruction. The detection and tracking algorithm of the proposed method is computationally efficient. We experimentally demonstrate that the method can achieve a temporal resolution of 1,666 frames per second by using a 10,000 Hz digital micro-mirror device. The latency time of the method is on the order of microseconds. Additionally, the method acquires only 600 bytes of data for each frame. The method therefore allows fast moving object detection and tracking in real time and for long duration. This image-free approach might open up a new avenue for spatial information acquisition in a highly efficient manner.

Label-free 3D imaging of weakly absorbing samples using spatially-incoherent annular illumination microscopy.

In this paper, we present a simple but effective label-free three dimensional (3D) microscopy for weakly absorbing samples. The proposed technique employs spatially-incoherent annular illumination to enhance absorption contrast of in-focus images and reduce phase contrast. We also employ mechanical scanning along axial direction to acquire a volume of the sample images. A 3D gradient operation is further adopted to remove the background with defocused shadows caused by oblique illumination. As such, a sequence of background-free sectioning images is acquired. The 3D gradient operation results in that only the structural edges of the weakly absorbing sample are visible in the images. We can therefore reconstruct the 3D skeleton structure of the sample from the sectioning image sequence. A label-free diatom is used to verify our technique experimentally. The 3D skeleton structure of the diatom is reconstructed and presented. The proposed technique would find applications in various fields, such as life science, materials science, etc.