Neural network enabled fringe projection through scattering media.

The projection of fringes plays an essential role in many applications, such as fringe projection profilometry and structured illumination microscopy. However, these capabilities are significantly constrained in environments affected by optical scattering. Although recent developments in wavefront shaping have effectively generated high-fidelity focal points and relatively simple structured images amidst scattering, the ability to project fringes that cover half of the projection area has not yet been achieved. To address this limitation, this study presents a fringe projector enabled by a neural network, capable of projecting fringes with variable periodicities and orientation angles through scattering media. We tested this projector on two types of scattering media: ground glass diffusers and multimode fibers. For these scattering media, the average Pearson's correlation coefficients between the projected fringes and their designed configurations are 86.9% and 79.7%, respectively. These results demonstrate the effectiveness of the proposed neural network enabled fringe projector. This advancement is expected to broaden the scope of fringe-based imaging techniques, making it feasible to employ them in conditions previously hindered by scattering effects.

Single-Shot Intensity- and Phase-Sensitive Compressive Sensing-Based Coherent Modulation Ultrafast Imaging.

Ultrafast imaging can capture the dynamic scenes with a nanosecond and even femtosecond temporal resolution. Complementarily, phase imaging can provide the morphology, refractive index, or thickness information that intensity imaging cannot represent. Therefore, it is important to realize the simultaneous ultrafast intensity and phase imaging for achieving as much information as possible in the detection of ultrafast dynamic scenes. Here, we report a single-shot intensity- and phase-sensitive compressive sensing-based coherent modulation ultrafast imaging technique, shortened as CS-CMUI, which integrates coherent modulation imaging, compressive imaging, and streak imaging. We theoretically demonstrate through numerical simulations that CS-CMUI can obtain both the intensity and phase information of the dynamic scenes with ultrahigh fidelity. Furthermore, we experimentally build a CS-CMUI system and successfully measure the intensity and phase evolution of a multimode Q-switched laser pulse and the dynamical behavior of laser ablation on an indium tin oxide thin film. It is anticipated that CS-CMUI enables a profound comprehension of ultrafast phenomena and promotes the advancement of various practical applications, which will have substantial impact on fundamental and applied sciences.

Multimode fiber-based greyscale image projector enabled by neural networks with high generalization ability.

Multimode fibers (MMFs) are emerging as promising transmission media for delivering images. However, strong mode coupling inherent in MMFs induces difficulties in directly projecting two-dimensional images through MMFs. By training two subnetworks named Actor-net and Model-net synergetically, [Nature Machine Intelligence2, 403 (2020)10.1038/s42256-020-0199-9] alleviated this issue and demonstrated projecting images through MMFs with high fidelity. In this work, we make a step further by improving the generalization ability to greyscale images. The modified projector network contains three subnetworks, namely forward-net, backward-net, and holography-net, accounting for forward propagation, backward propagation, and the phase-retrieval process. As a proof of concept, we experimentally trained the projector network using randomly generated phase maps and their corresponding resultant speckle images output from a 1-meter-long MMF. With the network being trained, we successfully demonstrated projecting binary images from MNIST and EMNIST and greyscale images from Fashion-MNIST, exhibiting averaged Pearson's correlation coefficients of 0.91, 0.92, and 0.87, respectively. Since all these projected images have never been seen by the projector network before, a strong generalization ability in projecting greyscale images is confirmed.

Projecting colorful images through scattering media via deep learning.

The existence of scatterers in the optical path has been the major obstacle that prohibits one from projecting images through solid walls, turbid water, clouds, and fog. Recent developments in wavefront shaping and neural networks demonstrate effective compensation for scattering effects, showing the promise to project clear images against strong scattering. However, previous studies were mainly restricted to projecting greyscale images using monochromatic light, mainly due to the increased complexity of simultaneously controlling multiple wavelengths. In this work, we fill this blank by developing a projector network, which enables the projection of colorful images through scattering media with three primary colors. To validate the performance of the projector network, we experimentally demonstrated projecting colorful images obtained from the MINST dataset through two stacked diffusers. Quantitatively, the averaged intensity Pearson's correlation coefficient for 1,000 test colorful images reaches about 90.6%, indicating the superiority of the developed network. We anticipate that the projector network can be beneficial to a variety of display applications in scattering environments.

Single photon single pixel imaging into thick scattering medium.

Imaging into thick scattering medium is a long-standing challenge. Beyond the quasi-ballistic regime, multiple scattering scrambles the spatiotemporal information of incident/emitted light, making canonical imaging based on light focusing nearly impossible. Diffusion optical tomography (DOT) is one of the most popular approach to look inside scattering medium, but quantitatively inverting the diffusion equation is ill-posed, and prior information of the medium is typically necessary, which is nontrivial to obtain. Here, we show theoretically and experimentally that, by synergizing the one-way light scattering characteristic of single pixel imaging with ultrasensitive single photon detection and a metric-guided image reconstruction, single photon single pixel imaging can serve as a simple and powerful alternative to DOT for imaging into thick scattering medium without prior knowledge or inverting the diffusion equation. We demonstrated an image resolution of 12 mm inside a 60 mm thick (∼ 78 mean free paths) scattering medium.

Optical focusing inside scattering media with iterative time-reversed ultrasonically encoded near-infrared light.

Focusing light inside scattering media is a long-sought goal in optics. Time-reversed ultrasonically encoded (TRUE) focusing, which combines the advantages of biological transparency of the ultrasound and the high efficiency of digital optical phase conjugation (DOPC) based wavefront shaping, has been proposed to tackle this problem. By invoking repeated acousto-optic interactions, iterative TRUE (iTRUE) focusing can further break the resolution barrier imposed by the acoustic diffraction limit, showing great potential for deep-tissue biomedical applications. However, stringent requirements on system alignment prohibit the practical use of iTRUE focusing, especially for biomedical applications at the near-infrared spectral window. In this work, we fill this blank by developing an alignment protocol that is suitable for iTRUE focusing with a near-infrared light source. This protocol mainly contains three steps, including rough alignment with manual adjustment, fine-tuning with a high-precision motorized stage, and digital compensation through Zernike polynomials. Using this protocol, an optical focus with a peak-to-background ratio (PBR) of up to 70% of the theoretical value can be achieved. By using a 5-MHz ultrasonic transducer, we demonstrated the first iTRUE focusing using near-infrared light at 1053 nm, enabling the formation of an optical focus inside a scattering medium composed of stacked scattering films and a mirror. Quantitatively, the size of the focus decreased from roughly 1 mm to 160 µm within a few consecutive iterations and a PBR up to 70 was finally achieved. We anticipate that the capability of focusing near-infrared light inside scattering media, along with the reported alignment protocol, can be beneficial to a variety of applications in biomedical optics.

Single-exposure ultrasound-modulated optical tomography with a quaternary phase encoded mask.

Ultrasound-modulated optical tomography (UOT) is a deep-tissue imaging modality that provides optical contrast with acoustic resolution. Among existing implementations, camera-based UOT improves modulation depth through parallel detection but suffers from a low camera frame rate. The condition prohibits this technique from being applied to in vivo applications where speckles decorrelate on a time scale of 1 ms or less. To overcome this challenge, we developed single-exposure camera-based UOT by employing a quaternary phase encoded mask (QPEM). As a proof of concept, we demonstrated imaging of an absorptive target buried inside a dynamic scattering medium with a speckle correlation time as short as 0.49 ms, typical of living biological tissues. Benefiting from the QPEM-enabled single-exposure wavefront measurement (5.5 ms) and GPU-assisted wavefront reconstruction (0.97 ms), the point scanning and result update speed can reach up to 150 Hz. We envision that the QPEM-enabled single-exposure scheme paves the way for in vivo UOT imaging, which holds promise for a variety of medical and biological applications.

Switching between topological edge states in plasmonic systems using phase-change materials.

We introduce non-Hermitian plasmonic waveguide-cavity structures based on the Aubry-Andre-Harper model to realize switching between right and left topological edge states (TESs) using the phase-change material Ge2Sb2Te5 (GST). We show that switching between the crystalline and amorphous phases of GST leads to a shift of the dispersion relation of the optimized structure so that a right TES for the crystalline phase, and a left TES for the amorphous phase occur at the same frequency. Thus, we realize switching between right and left TESs at that frequency by switching between the crystalline and amorphous phases of GST. Our results could be potentially important for developing compact reconfigurable topological photonic devices.

Coaxial interferometry for camera-based ultrasound-modulated optical tomography with paired illumination.

Ultrasound-modulated optical tomography (UOT), which combines the advantages of both light and ultrasound, is a promising imaging modality for deep-tissue high-resolution imaging. Among existing implementations, camera-based UOT gains huge advances in modulation depth through parallel detection. However, limited by the long exposure time and the slow framerate of modern cameras, the measurement of UOT signals always requires holographic methods with additional reference beams. This requirement increases system complexity and is susceptible to environmental disturbances. To overcome this challenge, we develop coaxial interferometry for camera-based UOT in this work. Such a coaxial scheme is enabled by employing paired illumination with slightly different optical frequencies. To measure the UOT signal, the conventional phase-stepping method in holography can be directly transplanted into coaxial interferometry. Specifically, we performed both numerical investigations and experimental validations for camera-based UOT under the proposed coaxial scheme. One-dimensional imaging for an absorptive target buried inside a scattering medium was demonstrated. With coaxial interferometry, this work presents an effective way to reduce system complexity and cope with environmental disturbances for camera-based UOT.

Modeling of iterative time-reversed ultrasonically encoded optical focusing in a reflection mode.

Time-reversed ultrasonically-encoded (TRUE) optical focusing is a promising technique to realize deep-tissue optical focusing by employing ultrasonic guide stars. However, the sizes of the ultrasound-induced optical focus are determined by the wavelengths of the ultrasound, which are typically tens of microns. To satisfy the need for high-resolution imaging and manipulation, iterative TRUE (iTRUE) was proposed to break this limit by triggering repeated interactions between light and ultrasound and compressing the optical focus. However, even for the best result reported to date, the resolutions along the ultrasound axial and lateral direction were merely improved by only 2-fold to 3-fold. This observation leads to doubt whether iTRUE can be effective in reducing the size of the optical focus. In this work, we address this issue by developing a physical model to investigate iTRUE in a reflection mode numerically. Our numerical results show that, under the influence of shot noises, iTRUE can reduce the optical focus to a single speckle within a finite number of iterations. This model also allows numerical investigations of iTRUE in detail. Quantitatively, based on the parameters set, we show that the optical focus can be reduced to a size of 1.6 µm and a peak-to-background ratio over 104 can be realized. It is also shown that iTRUE cannot significantly advance the focusing depth. We anticipate that this work can serve as useful guidance for optimizing iTRUE system for future biomedical applications, including deep-tissue optical imaging, laser surgery, and optogenetics.

Genetic-algorithm-assisted coherent enhancement absorption in scattering media by exploiting transmission and reflection matrices.

The investigations on coherent enhancement absorption (CEA) inside scattering media are critically important in biophotonics. CEA can deliver light to the targeted position, thus enabling deep-tissue optical imaging by improving signal strength and imaging resolution. In this work, we develop a numerical framework that employs the method of finite-difference time-domain. Both the transmission and reflection matrices of scattering media with open boundaries are constructed, allowing the studies on the eigenvalues and eigenchannels. To realize CEA for scattering media with local absorption, we develop a genetic-algorithm-assisted numerical model. By minimizing the total transmittance and reflectance simultaneously, different realizations of CEA are observed and, without setting internal monitors, can be differentiated with cases of light leaked from sides. By modulating the incident wavefront at only one side of the scattering medium, it is shown that for a 5-µm-diameter absorber buried inside a scattering medium of 15 µm × 12 µm, more than half of the incident light can be delivered and absorbed at the target position. The enhancement in absorption is more than four times higher than that with random input. This value can be even higher for smaller absorption regions. We also quantify the effectiveness of the method and show that it is inversely proportional to the openness of the scattering medium. This result is potentially useful for targeted light delivery inside scattering media with local absorption.

Characterization of the spectral memory effect of scattering media.

The optical memory effect is an interesting phenomenon exploited for deep-tissue optical imaging. Besides the widely studied memory effects in the spatial domain to accelerate point scanning speed, the spectral memory effect is also important in multispectral wavefront shaping. Although being theoretically analyzed for decades, quantitative studies of spectral memory effect on a variety of scattering media including biological tissue were rarely reported. In practice, quantifying the range of the spectral memory effect is essential in efficiently shaping broadband light, as it determines the optimum spectral resolution in realizing spatiotemporal focus through scattering media. In this work, we analyze the spectral memory effect based on a diffusion model. An explicit analytical expression involves the illumination wavelength, the diffusion constant, and the sample thickness is derived, which is consistent with the one in the literature. We experimentally quantified the range of spectral correlation for two types of biological tissue, tissue-mimicking phantoms with different concentrations, and diffusers. Specifically, for tissue-mimicking phantoms with calibrated scattering parameters, we show that a correction factor of more than 20 should be inserted, indicating that the range of spectral correlation is much larger than one would expect. This finding is particularly beneficial to multispectral wavefront shaping, as stringent requirements on the spectral resolution could be alleviated by at least one order of magnitude.

Single-shot ultrasound-modulated optical tomography with enhanced speckle contrast.

Ultrasound-modulated optical tomography (UOT) images optical contrast deep inside biological tissue. Among existing approaches, camera-based parallel detection is beneficial in modulation depth but is limited to the relatively slow framerate of cameras. This condition prevents such a scheme from achieving maturity to image live animals with sub-millisecond speckle correlation time. In this work, we developed on-axis single-shot UOT by investigating the statistics of speckles, breaking the restriction imposed by the slow camera framerate. As a proof of concept, we experimentally imaged a one-dimensional absorptive object buried inside a moving scattering medium with speckle correlation time down to 0.48 ms. We envision that this single-shot UOT is promising to cope with live animals with fast speckle decorrelation.

Feedback-assisted transmission matrix measurement of a multimode fiber in a referenceless system.

Recent development in wavefront shaping shows the promise to employ multimode fibers (MMFs) to deliver images in endoscopy. In these applications, retrieving the transmission matrix (TM) of the MMF is especially important. Among existing non-holographic approaches, feedback-based wavefront shaping requires a large number of measurements, while directly measuring the TM can be easily trapped into local optimums if the constraints are insufficient. To reduce the required number of measurements, we combine the concepts of these two approaches and develop a scheme termed feedback-assisted TM measurements. We show that under such a hybrid scheme, less than 3N intensity measurements are sufficient to accurately retrieve one row of the TM that contains N unknown complex elements. As a proof of concept, we experimentally demonstrated retrieving multiple rows of the TM of an MMF using the proposed scheme with high fidelity. In particular, a single focus and dual foci through the MMF with enhancements larger than 75% of the theoretical values were reported.

Coherent laser detection of the femtowatt-level frequency-shifted optical feedback based on a DFB fiber laser.

The theoretical basis and experimental realization of an all-fiber self-mixing laser Doppler velocimetry based on frequency-shifted feedback in a distributed feedback (DFB) fiber laser are presented, which employs a pair of fiber-coupled acousto-optic modulators to adjust the modulation intensity and frequency of the laser self-mixing effect. Moreover, the minimum optical feedback intensity for the velocity signal successfully measured by the interferometer is 5.12 fW, corresponding to 0.16 photons per Doppler cycle. The results demonstrate that the proposed scheme can adapt to the non-contact measurement requirements of the wide-range speed and weak feedback level in the complex environment.

Retrieving the optical transmission matrix of a multimode fiber using the extended Kalman filter.

Characterizing the transmission matrix (TM) of a multimode fiber (MMF) benefits many fiber-based applications and allows in-depth studies on the physical properties. For example, by modulating the incident field, the knowledge of the TM allows one to synthesize any optical field at the distill end of the MMF. However, the extraction of optical fields usually requires holographic measurements with interferometry, which complicates the system design and introduces additional noise. In this work, we developed an efficient method to retrieve the TM of the MMF in a referenceless optical system. With pure intensity measurements, this method uses the extended Kalman filter (EKF) to recursively search for the optimum solution. To facilitate the computational process, a modified speckle-correlation scatter matrix (MSSM) is constructed as a low-fidelity initial estimation. This method, termed EKF-MSSM, only requires 4N intensity measurements to precisely solve for N unknown complex variables in the TM. Experimentally, we successfully retrieved the TM of the MMF with high precision, which allows optical focusing with the enhancement (>70%) close to the theoretical value. We anticipate that this method will serve as a useful tool for studying physical properties of the MMFs and potentially open new possibilities in a variety of applications in fiber optics.

Delivering targeted color light through a multimode fiber by field synthesis.

Recent developments of wavefront shaping make the multimode fiber (MMF) as a promising tool to deliver images in endoscopy. However, previous studies using the MMF were limited to monochromatic light or polychromatic light with narrow bandwidth. The desires for colored imaging stimulate us to deliver multi-wavelength light that covers the entire visible spectrum through the MMF. In this work, we demonstrated delivering targeted color light through the MMF by mixing three primary colors (red, green, and blue) with a single spatial light modulator. The optimum phase map that considers all three colors was generated through field synthesis (FS), which requires every pixel of the SLM to partially account for all colors. With both theoretical and numerical approaches, we showed that FS exhibited much better performance than the previously developed spatial segmentation method that employs different pixels to represent different colors. Moreover, by computationally adjusting the compositions of the weight for each color, the colors of the delivered focus can be switched at video framerate. We anticipate that our work paves a way for future applications of delivering color images through the MMF in endoscopy.

Non-PT-symmetric two-layer cylindrical waveguide for exceptional-point-enhanced optical devices.

We investigate the exceptional points in a two-layer cylindrical waveguide structure consisting of absorbing and non-absorbing dielectrics. We show that, by tuning the core to total radius ratio and the refractive index of the core layer in such a structure, the complex effective indices of two waveguide modes can coalesce so that an exceptional point is formed. We show that the sensitivity of the effective index of the waveguide mode to variations of the refractive index of the material filling the shell layer is enhanced at the exceptional point. In addition, we show that larger sensitivity enhancement is obtained for smaller perturbations. Our results could potentially contribute to the development of a new generation of chip-scale exceptional-point-enhanced optical waveguide devices for modulation, switching, and sensing.

Controlling 1550-nm light through a multimode fiber using a Hadamard encoding algorithm.

To meet the demand for higher capacity fiber-optic communication, multimode fibers have gradually attracted attention, but they introduce spatial distortions. To overcome this limitation, wavefront shaping technology promises to control scattered light after it is transmitted through multimode fibers. In this work, we introduce a Hadamard encoding algorithm (HEA) to control 1550-nm light that has passed through a multimode fiber. A series of Hadamard bases is iteratively added to the current optimum phase map, and the coefficient of each order is determined through a simple four-step phase-shifting mechanism. Using a laser source at 1550-nm wavelength, we experimentally achieved an optical focus through a 2-meter-long multimode fiber. With 1024 orders, the experimental enhancement reached 690, which is 86% of the theoretical value. As far as we know, this is the best result ever reported in focusing 1550-nm light through a multimode fiber. Moreover, we note that the HEA can also be used to reduce the intensity of the targeted light, suggesting broad applications in glare suppression. These results demonstrate superior performance in controlling targeted light transport through a multimode fiber at a telecommunication wavelength. We anticipate that this work will open new possibilities in a variety of applications in fiber optics.

Optimization of photonic nanojets generated by multilayer microcylinders with a genetic algorithm.

We employ a genetic algorithm coupled with Mie theory to optimize the magnetic field intensity profile of photonic nanojets (PNJs) generated by multilayer microcylinders at visible wavelengths in free space. We first optimize five-layer microcylinders to elongate the PNJs. We show that a properly designed five-layer microcylinder structure can generate an ultra-long PNJ with a beam length ~ 107.5 times the illumination wavelength λ0. We then optimize five-layer microcylinders to narrow the waist of PNJs. We show that a PNJ with a full-width at half maximum waist of ~ 0.22λ0 can be obtained outside the surface of the optimized microcylinder. In addition, curved PNJs with subwavelength waist are also obtained. We finally optimize the five-layer structures for refractive index sensing based on the beam length of PNJs. The estimated minimum detectable refractive index variation when using this sensing method is ultra-small. Our results could potentially contribute to the development of a new generation of devices for optical nanoscopy and biophotonics, and greatly promote the practical applications of PNJs.