Optical single-pixel volumetric imaging by three-dimensional light-field illumination.

Three-dimensional single-pixel imaging (3D SPI) has become an attractive imaging modality for both biomedical research and optical sensing. 3D-SPI techniques generally depend on time-of-flight or stereovision principle to extract depth information from backscattered light. However, existing implementations for these two optical schemes are limited to surface mapping of 3D objects at depth resolutions, at best, at the millimeter level. Here, we report 3D light-field illumination single-pixel microscopy (3D-LFI-SPM) that enables volumetric imaging of microscopic objects with a near-diffraction-limit 3D optical resolution. Aimed at 3D space reconstruction, 3D-LFI-SPM optically samples the 3D Fourier spectrum by combining 3D structured light-field illumination with single-element intensity detection. We build a 3D-LFI-SPM prototype that provides an imaging volume of ∼390 × 390 × 3,800 µm3 and achieves 2.7-µm lateral resolution and better than 37-µm axial resolution. Its capability of 3D visualization of label-free optical absorption contrast is demonstrated by imaging single algal cells in vivo. Our approach opens broad perspectives for 3D SPI with potential applications in various fields, such as biomedical functional imaging.

Single-Shot Intensity- and Phase-Sensitive Compressive Sensing-Based Coherent Modulation Ultrafast Imaging.

Ultrafast imaging can capture the dynamic scenes with a nanosecond and even femtosecond temporal resolution. Complementarily, phase imaging can provide the morphology, refractive index, or thickness information that intensity imaging cannot represent. Therefore, it is important to realize the simultaneous ultrafast intensity and phase imaging for achieving as much information as possible in the detection of ultrafast dynamic scenes. Here, we report a single-shot intensity- and phase-sensitive compressive sensing-based coherent modulation ultrafast imaging technique, shortened as CS-CMUI, which integrates coherent modulation imaging, compressive imaging, and streak imaging. We theoretically demonstrate through numerical simulations that CS-CMUI can obtain both the intensity and phase information of the dynamic scenes with ultrahigh fidelity. Furthermore, we experimentally build a CS-CMUI system and successfully measure the intensity and phase evolution of a multimode Q-switched laser pulse and the dynamical behavior of laser ablation on an indium tin oxide thin film. It is anticipated that CS-CMUI enables a profound comprehension of ultrafast phenomena and promotes the advancement of various practical applications, which will have substantial impact on fundamental and applied sciences.

Temporal resolution of ultrafast compressive imaging using a single-chirped optical probe.

Ultrafast compressive imaging captures three-dimensional spatiotemporal information of transient events in a single shot. When a single-chirped optical probe is applied, the temporal information is obtained from the probe modulated in amplitude or phase using a direct frequency-time mapping method. Here, we extend the analysis of the temporal resolution of conventional one-dimensional ultrafast measurement techniques such as spectral interferometry to that in three-dimensional ultrafast compressive imaging. In this way, both the amplitude and phase of the probe are necessary for a full Fourier transform method, which obtains temporal information with an improved resolution determined by probe spectral bandwidth. The improved temporal resolution potentially enables ultrafast compressive imaging with an effective imaging speed at the quadrillion-frames-per-second level.

Multi-scale band-limited illumination profilometry for robust three-dimensional surface imaging at video rate.

Dynamic three-dimensional (3D) surface imaging by phase-shifting fringe projection profilometry has been widely implemented in diverse applications. However, existing techniques fall short in simultaneously providing the robustness in solving spatially isolated 3D objects, the tolerance of large variation in surface reflectance, and the flexibility of tunable working distances with meter-square-level fields of view (FOVs) at video rate. In this work, we overcome these limitations by developing multi-scale band-limited illumination profilometry (MS-BLIP). Supported by the synergy of dual-level intensity projection, multi-frequency fringe projection, and an iterative method for distortion compensation, MS-BLIP can accurately discern spatially separated 3D objects with highly varying reflectance. MS-BLIP is demonstrated by dynamic 3D imaging of a translating engineered box and a rotating vase. With an FOV of up to 1.7 m × 1.1 m and a working distance of up to 2.8 m, MS-BLIP is applied to capturing full human-body movements at video rate.

High-speed super-resolution imaging with compressive imaging-based structured illumination microscopy.

Structured illumination microscopy (SIM) has been widely applied to investigating fine structures of biological samples by breaking the optical diffraction limitation. So far, video-rate imaging has been obtained in SIM, but the imaging speed was still limited due to the reconstruction of a super-solution image through multi-sampling, which hindered the applications in high-speed biomedical imaging. To overcome this limitation, here we develop compressive imaging-based structured illumination microscopy (CISIM) by synergizing SIM and compressive sensing (CS). Compared with conventional SIM, CISIM can greatly improve the super-resolution imaging speed by extracting multiple super-resolution images from one compressed image. Based on CISIM, we successfully reconstruct the super-resolution images in biological dynamics, and analyze the effect factors of image reconstruction quality, which verify the feasibility of CISIM. CISIM paves a way for high-speed super-resolution imaging, which may bring technological breakthroughs and significant applications in biomedical imaging.

A cellular endolysosome-modulating pore-forming protein from a toad is negatively regulated by its paralog under oxidizing conditions.

Endolysosomes are key players in cell physiology, including molecular exchange, immunity, and environmental adaptation. They are the molecular targets of some pore-forming aerolysin-like proteins (ALPs) that are widely distributed in animals and plants and are functionally related to bacterial toxin aerolysins. ßγ-CAT is a complex of an ALP (BmALP1) and a trefoil factor (BmTFF3) in the firebelly toad (Bombina maxima). It is the first example of a secreted endogenous pore-forming protein that modulates the biochemical properties of endolysosomes by inducing pore formation in these intracellular vesicles. Here, using a large array of biochemical and cell biology methods, we report the identification of BmALP3, a paralog of BmALP1 that lacks membrane pore-forming capacity. We noted that both BmALP3 and BmALP1 contain a conserved cysteine in their C-terminal regions. BmALP3 was readily oxidized to a disulfide bond-linked homodimer, and this homodimer then oxidized BmALP1 via disulfide bond exchange, resulting in the dissociation of ßγ-CAT subunits and the elimination of biological activity. Consistent with its behavior in vitro, BmALP3 sensed environmental oxygen tension in vivo, leading to modulation of ßγ-CAT activity. Interestingly, we found that this C-terminal cysteine site is well conserved in numerous vertebrate ALPs. These findings uncover the existence of a regulatory ALP (BmALP3) that modulates the activity of an active ALP (BmALP1) in a redox-dependent manner, a property that differs from those of bacterial toxin aerolysins.

Compressed ultrafast tomographic imaging by passive spatiotemporal projections.

Existing streak-camera-based two-dimensional (2D) ultrafast imaging techniques are limited by long acquisition time, the trade-off between spatial and temporal resolutions, and a reduced field of view. They also require additional components, customization, or active illumination. Here we develop compressed ultrafast tomographic imaging (CUTI), which passively records 2D transient events with a standard streak camera. By grafting the concept of computed tomography to the spatiotemporal domain, the operations of temporal shearing and spatiotemporal integration in a streak camera's data acquisition can be equivalently expressed as the spatiotemporal projection of an (x,y,t) datacube from a certain angle. Aided by a new, to the best of our knowledge, compressed-sensing reconstruction algorithm, the 2D transient event can be accurately recovered in a few measurements. CUTI is exhibited as a new imaging mode universally adaptable to most streak cameras. Implemented in an image-converter streak camera, CUTI captures the sequential arrival of two spatially modulated ultrashort ultraviolet laser pulses at 0.5 trillion frames per second. Applied to a rotating-mirror streak camera, CUTI records an amination of fast-bouncing balls at 5,000 frames per second.

Punching holes in light: recent progress in single-shot coded-aperture optical imaging.

Single-shot coded-aperture optical imaging physically captures a code-aperture-modulated optical signal in one exposure and then recovers the scene via computational image reconstruction. Recent years have witnessed dazzling advances in various modalities in this hybrid imaging scheme in concomitant technical improvement and widespread applications in physical, chemical and biological sciences. This review comprehensively surveys state-of-the-art single-shot coded-aperture optical imaging. Based on the detected photon tags, this field is divided into six categories: planar imaging, depth imaging, light-field imaging, temporal imaging, spectral imaging, and polarization imaging. In each category, we start with a general description of the available techniques and design principles, then provide two representative examples of active-encoding and passive-encoding approaches, with a particular emphasis on their methodology and applications as well as their advantages and challenges. Finally, we envision prospects for further technical advancement in this field.

Camera-free three-dimensional dual photography.

We report camera-free three-dimensional (3D) dual photography. Inspired by the linkage between fringe projection profilometry (FPP) and dual photography, we propose to implement coordinate mapping to simultaneously sense the direct component of the light transport matrix and the surface profiles of 3D objects. By exploiting Helmholtz reciprocity, dual photography and scene relighting can thus be performed on 3D images. To verify the proposed imaging method, we have developed a single-pixel imaging system based on two digital micromirror devices (DMDs). Binary cyclic S-matrix patterns and binary sinusoidal fringe patterns are loaded on each DMD for scene encoding and virtual fringe projection, respectively. Using this system, we have demonstrated viewing and relighting 3D images at user-selectable perspectives. Our work extends the conceptual scope and the imaging capability of dual photography.

Real-time high-speed three-dimensional surface imaging using band-limited illumination profilometry with a CoaXPress interface.

High-speed three-dimensional (3D) surface imaging by structured-light profilometry is currently driven by numerous applications. However, the limited speeds in fringe pattern projection, image acquisition, and data transmission have strained the existing methods from reaching kilohertz-level acquisition, processing, and display of 3D information during the occurrence of dynamic events (i.e., in real time). To overcome these limitations, we have developed band-limited illumination profilometry (BLIP) with a CoaXPress interface (CI), which enables real-time high-speed 3D surface imaging. We have demonstrated the system's performance by imaging various static and fast-moving 3D objects in real time. We have also applied this system in fluid mechanics by imaging dynamics of a flag, which allowed observation of the wave propagation, gravity-induced phase mismatch, and asymmetric flapping motion. We expect CI-BLIP to find diverse scientific and industrial applications.

Single-shot compressed ultrafast photography at one hundred billion frames per second.

The capture of transient scenes at high imaging speed has been long sought by photographers, with early examples being the well known recording in 1878 of a horse in motion and the 1887 photograph of a supersonic bullet. However, not until the late twentieth century were breakthroughs achieved in demonstrating ultrahigh-speed imaging (more than 10(5) frames per second). In particular, the introduction of electronic imaging sensors based on the charge-coupled device (CCD) or complementary metal-oxide-semiconductor (CMOS) technology revolutionized high-speed photography, enabling acquisition rates of up to 10(7) frames per second. Despite these sensors' widespread impact, further increasing frame rates using CCD or CMOS technology is fundamentally limited by their on-chip storage and electronic readout speed. Here we demonstrate a two-dimensional dynamic imaging technique, compressed ultrafast photography (CUP), which can capture non-repetitive time-evolving events at up to 10(11) frames per second. Compared with existing ultrafast imaging techniques, CUP has the prominent advantage of measuring an x-y-t (x, y, spatial coordinates; t, time) scene with a single camera snapshot, thereby allowing observation of transient events with temporal resolution as tens of picoseconds. Furthermore, akin to traditional photography, CUP is receive-only, and so does not need the specialized active illumination required by other single-shot ultrafast imagers. As a result, CUP can image a variety of luminescent--such as fluorescent or bioluminescent--objects. Using CUP, we visualize four fundamental physical phenomena with single laser shots only: laser pulse reflection and refraction, photon racing in two media, and faster-than-light propagation of non-information (that is, motion that appears faster than the speed of light but cannot convey information). Given CUP's capability, we expect it to find widespread applications in both fundamental and applied sciences, including biomedical research.

Single-shot compressed optical-streaking ultra-high-speed photography.

Single-shot ultra-high-speed imaging is of great significance to capture transient phenomena in physics, biology, and chemistry in real time. Existing techniques, however, have a restricted application scope, a low sequence depth, or a limited pixel count. To overcome these limitations, we developed single-shot compressed optical-streaking ultra-high-speed photography (COSUP) with an imaging speed of 1.5 million frames per second, a sequence depth of 500 frames, and an (x,y) pixel count of 0.5 megapixels per frame. COSUP's single-shot ultra-high-speed imaging ability was demonstrated by recording single laser pulses illuminating through transmissive targets and by tracing a fast-moving object. As a universal imaging platform, COSUP is capable of increasing imaging speeds of a wide range of CCD and complementary metal-oxide-semiconductor cameras by four orders of magnitude. We envision COSUP to be applied in widespread applications in biomedicine and materials science.

Amplitude-masked photoacoustic wavefront shaping and application in flowmetry.

Optical-resolution photoacoustic flowmetry (PAF) allows noninvasive single-cell flow measurements. However, its operational depth is limited by optical diffusion, which prevents focusing beyond shallow depths in scattering media, as well as reducing the measurement signal-to-noise ratio (SNR). To overcome this limitation, we used binary-amplitude wavefront shaping to enhance light focusing in the presence of scattering. Here, the transmission modes that contributed constructively to the intensity at the optical focus were identified and selectively illuminated, resulting in a 14-fold intensity increase and a corresponding increase in SNR. This technique can potentially extend the operational depth of optical-resolution PAF beyond 1 mm in tissue.

Spatially Fourier-encoded photoacoustic microscopy using a digital micromirror device.

We have developed spatially Fourier-encoded photoacoustic (PA) microscopy using a digital micromirror device. The spatial intensity distribution of laser pulses is Fourier-encoded, and a series of such encoded PA measurements allows one to decode the spatial distribution of optical absorption. The throughput and Fellgett advantages were demonstrated by imaging a chromium target. By using 63 spatial elements, the signal-to-noise ratio in the recovered PA signal was enhanced by ∼4×. The system was used to image two biological targets, a monolayer of red blood cells and melanoma cells.

Spatial bandwidth analysis of fast backward Fresnel diffraction for precise computer-generated hologram design.

Designing near-field computer-generated holograms (CGHs) for a spatial light modulator (SLM) requires backward diffraction calculations. However, direct implementation of the discrete computational model of the Fresnel diffraction integral often produces inaccurate reconstruction. Finite sizes of the SLM and the target image, as well as aliasing, are major sources of error. Here we present a new design prescription for precise near-field CGHs based on comprehensive analysis of the spatial bandwidth. We demonstrate that, by controlling two free variables related to the target image, the designed hologram is free from aliasing and can have minimum error. To achieve this, we analyze the geometry of the target image, hologram, and Fourier transform plane of the target image to derive conditions for minimizing reconstruction error due to truncation of spatial frequencies lying outside of the hologram. The design prescription is verified by examples showing reconstruction error versus controlled parameters. Finally, it is applied to precise three-dimensional image reconstruction.

Tutorial on compressed ultrafast photography.

Significance: Compressed ultrafast photography (CUP) is currently the world's fastest single-shot imaging technique. Through the integration of compressed sensing and streak imaging, CUP can capture a transient event in a single camera exposure with imaging speeds from thousands to trillions of frames per second, at micrometer-level spatial resolutions, and in broad sensing spectral ranges. Aim: This tutorial aims to provide a comprehensive review of CUP in its fundamental methods, system implementations, biomedical applications, and prospect. Approach: A step-by-step guideline to CUP's forward model and representative image reconstruction algorithms is presented with sample codes and illustrations in Matlab and Python. Then, CUP's hardware implementation is described with a focus on the representative techniques, advantages, and limitations of the three key components-the spatial encoder, the temporal shearing unit, and the two-dimensional sensor. Furthermore, four representative biomedical applications enabled by CUP are discussed, followed by the prospect of CUP's technical advancement. Conclusions: CUP has emerged as a state-of-the-art ultrafast imaging technology. Its advanced imaging ability and versatility contribute to unprecedented observations and new applications in biomedicine. CUP holds great promise in improving technical specifications and facilitating the investigation of biomedical processes.

Short-wave Infrared Photoluminescence Lifetime Mapping of Rare-Earth Doped Nanoparticles Using All-Optical Streak Imaging.

The short-wave infrared (SWIR) photoluminescence lifetimes of rare-earth doped nanoparticles (RENPs) have found diverse applications in fundamental and applied research. Despite dazzling progress in the novel design and synthesis of RENPs with attractive optical properties, existing optical systems for SWIR photoluminescence lifetime imaging are still considerably restricted by inefficient photon detection, limited imaging speed, and low sensitivity. To overcome these challenges, SWIR photoluminescence lifetime imaging microscopy using an all-optical streak camera (PLIMASC) is developed. Synergizing scanning optics and a high-sensitivity InGaAs CMOS camera, SWIR-PLIMASC has a 1D imaging speed of up to 138.9 kHz in the spectral range of 900-1700 nm, which quantifies the photoluminescence lifetime of RENPs in a single shot. A 2D photoluminescence lifetime map can be acquired by 1D scanning of the sample. To showcase the power of SWIR-PLIMASC, a series of core-shell RENPs with distinct SWIR photoluminescence lifetimes is synthesized. In particular, using Er3+ -doped RENPs, SWIR-PLIMASC enables multiplexed anti-counterfeiting. Leveraging Ho3+ -doped RENPs as temperature indicators, this system is applied to SWIR photoluminescence lifetime-based thermometry. Opening up a new avenue for efficient SWIR photoluminescence lifetime mapping, this work is envisaged to contribute to advanced materials characterization, information science, and biomedicine.

Spatial-temporal characterization of photoemission in a streak-mode dynamic transmission electron microscope.

A long-standing motivation driving high-speed electron microscopy development is to capture phase transformations and material dynamics in real time with high spatial and temporal resolution. Current dynamic transmission electron microscopes (DTEMs) are limited to nanosecond temporal resolution and the ability to capture only a few frames of a transient event. With the motivation to overcome these limitations, we present our progress in developing a streak-mode DTEM (SM-DTEM) and demonstrate the recovery of picosecond images with high frame sequence depth. We first demonstrate that a zero-dimensional (0D) SM-DTEM can provide temporal information on any local region of interest with a 0.37 µm diameter, a 20-GHz sampling rate, and 1200 data points in the recorded trace. We use this method to characterize the temporal profile of the photoemitted electron pulse, finding that it deviates from the incident ultraviolet laser pulse and contains an unexpected peak near its onset. Then, we demonstrate a two-dimensional (2D) SM-DTEM, which uses compressed-sensing-based tomographic imaging to recover a full spatiotemporal photoemission profile over a 1.85-µm-diameter field of view with nanoscale spatial resolution, 370-ps inter-frame interval, and 140-frame sequence depth in a 50-ns time window. Finally, a perspective is given on the instrumental modifications necessary to further develop this promising technique with the goal of decreasing the time to capture a 2D SM-DTEM dataset.

Swept coded aperture real-time femtophotography.

Single-shot real-time femtophotography is indispensable for imaging ultrafast dynamics during their times of occurrence. Despite their advantages over conventional multi-shot approaches, existing techniques confront restricted imaging speed or degraded data quality by the deployed optoelectronic devices and face challenges in the application scope and acquisition accuracy. They are also hindered by the limitations in the acquirable information imposed by the sensing models. Here, we overcome these challenges by developing swept coded aperture real-time femtophotography (SCARF). This computational imaging modality enables all-optical ultrafast sweeping of a static coded aperture during the recording of an ultrafast event, bringing full-sequence encoding of up to 156.3 THz to every pixel on a CCD camera. We demonstrate SCARF's single-shot ultrafast imaging ability at tunable frame rates and spatial scales in both reflection and transmission modes. Using SCARF, we image ultrafast absorption in a semiconductor and ultrafast demagnetization of a metal alloy.

High-brightness X-ray free-electron laser with an optical undulator by pulse shaping.

A normal-incident flattop laser with a tapered end is proposed as an optical undulator to achieve a high-gain and high-brightness X-ray free electron laser (FEL). The synchronic interaction of an electron bunch with the normal incident laser is realized by tilting the laser pulse front. The intensity of the flattop laser is kept constant during the interaction time of the electron bunch and the laser along the focal plane of a cylindrical lens. Optical shaping to generate the desired flattop pulse with a tapered end from an original Gaussian pulse distribution is designed and simulated. The flattop laser with a tapered end can enhance the X-ray FEL beyond the exponential growth saturation power by one order to reach 1 Gigawatt as compared to that without a tapered end. The peak brightness can reach 1030 photons/mm2/mrad2/s/0.1% bandwidth, more than 10 orders brighter than the conventional incoherent Thompson Scattering X-ray source.