Neural network enabled fringe projection through scattering media.

The projection of fringes plays an essential role in many applications, such as fringe projection profilometry and structured illumination microscopy. However, these capabilities are significantly constrained in environments affected by optical scattering. Although recent developments in wavefront shaping have effectively generated high-fidelity focal points and relatively simple structured images amidst scattering, the ability to project fringes that cover half of the projection area has not yet been achieved. To address this limitation, this study presents a fringe projector enabled by a neural network, capable of projecting fringes with variable periodicities and orientation angles through scattering media. We tested this projector on two types of scattering media: ground glass diffusers and multimode fibers. For these scattering media, the average Pearson's correlation coefficients between the projected fringes and their designed configurations are 86.9% and 79.7%, respectively. These results demonstrate the effectiveness of the proposed neural network enabled fringe projector. This advancement is expected to broaden the scope of fringe-based imaging techniques, making it feasible to employ them in conditions previously hindered by scattering effects.

Single-Shot Intensity- and Phase-Sensitive Compressive Sensing-Based Coherent Modulation Ultrafast Imaging.

Ultrafast imaging can capture the dynamic scenes with a nanosecond and even femtosecond temporal resolution. Complementarily, phase imaging can provide the morphology, refractive index, or thickness information that intensity imaging cannot represent. Therefore, it is important to realize the simultaneous ultrafast intensity and phase imaging for achieving as much information as possible in the detection of ultrafast dynamic scenes. Here, we report a single-shot intensity- and phase-sensitive compressive sensing-based coherent modulation ultrafast imaging technique, shortened as CS-CMUI, which integrates coherent modulation imaging, compressive imaging, and streak imaging. We theoretically demonstrate through numerical simulations that CS-CMUI can obtain both the intensity and phase information of the dynamic scenes with ultrahigh fidelity. Furthermore, we experimentally build a CS-CMUI system and successfully measure the intensity and phase evolution of a multimode Q-switched laser pulse and the dynamical behavior of laser ablation on an indium tin oxide thin film. It is anticipated that CS-CMUI enables a profound comprehension of ultrafast phenomena and promotes the advancement of various practical applications, which will have substantial impact on fundamental and applied sciences.

Projecting colorful images through scattering media via deep learning.

The existence of scatterers in the optical path has been the major obstacle that prohibits one from projecting images through solid walls, turbid water, clouds, and fog. Recent developments in wavefront shaping and neural networks demonstrate effective compensation for scattering effects, showing the promise to project clear images against strong scattering. However, previous studies were mainly restricted to projecting greyscale images using monochromatic light, mainly due to the increased complexity of simultaneously controlling multiple wavelengths. In this work, we fill this blank by developing a projector network, which enables the projection of colorful images through scattering media with three primary colors. To validate the performance of the projector network, we experimentally demonstrated projecting colorful images obtained from the MINST dataset through two stacked diffusers. Quantitatively, the averaged intensity Pearson's correlation coefficient for 1,000 test colorful images reaches about 90.6%, indicating the superiority of the developed network. We anticipate that the projector network can be beneficial to a variety of display applications in scattering environments.

Optical focusing inside scattering media with iterative time-reversed ultrasonically encoded near-infrared light.

Focusing light inside scattering media is a long-sought goal in optics. Time-reversed ultrasonically encoded (TRUE) focusing, which combines the advantages of biological transparency of the ultrasound and the high efficiency of digital optical phase conjugation (DOPC) based wavefront shaping, has been proposed to tackle this problem. By invoking repeated acousto-optic interactions, iterative TRUE (iTRUE) focusing can further break the resolution barrier imposed by the acoustic diffraction limit, showing great potential for deep-tissue biomedical applications. However, stringent requirements on system alignment prohibit the practical use of iTRUE focusing, especially for biomedical applications at the near-infrared spectral window. In this work, we fill this blank by developing an alignment protocol that is suitable for iTRUE focusing with a near-infrared light source. This protocol mainly contains three steps, including rough alignment with manual adjustment, fine-tuning with a high-precision motorized stage, and digital compensation through Zernike polynomials. Using this protocol, an optical focus with a peak-to-background ratio (PBR) of up to 70% of the theoretical value can be achieved. By using a 5-MHz ultrasonic transducer, we demonstrated the first iTRUE focusing using near-infrared light at 1053 nm, enabling the formation of an optical focus inside a scattering medium composed of stacked scattering films and a mirror. Quantitatively, the size of the focus decreased from roughly 1 mm to 160 µm within a few consecutive iterations and a PBR up to 70 was finally achieved. We anticipate that the capability of focusing near-infrared light inside scattering media, along with the reported alignment protocol, can be beneficial to a variety of applications in biomedical optics.

Single-exposure ultrasound-modulated optical tomography with a quaternary phase encoded mask.

Ultrasound-modulated optical tomography (UOT) is a deep-tissue imaging modality that provides optical contrast with acoustic resolution. Among existing implementations, camera-based UOT improves modulation depth through parallel detection but suffers from a low camera frame rate. The condition prohibits this technique from being applied to in vivo applications where speckles decorrelate on a time scale of 1 ms or less. To overcome this challenge, we developed single-exposure camera-based UOT by employing a quaternary phase encoded mask (QPEM). As a proof of concept, we demonstrated imaging of an absorptive target buried inside a dynamic scattering medium with a speckle correlation time as short as 0.49 ms, typical of living biological tissues. Benefiting from the QPEM-enabled single-exposure wavefront measurement (5.5 ms) and GPU-assisted wavefront reconstruction (0.97 ms), the point scanning and result update speed can reach up to 150 Hz. We envision that the QPEM-enabled single-exposure scheme paves the way for in vivo UOT imaging, which holds promise for a variety of medical and biological applications.

Parallel interrogation of the chalcogenide-based micro-ring sensor array for photoacoustic tomography.

Photoacoustic tomography (PAT), also known as optoacoustic tomography, is an attractive imaging modality that provides optical contrast with acoustic resolutions. Recent progress in the applications of PAT largely relies on the development and employment of ultrasound sensor arrays with many elements. Although on-chip optical ultrasound sensors have been demonstrated with high sensitivity, large bandwidth, and small size, PAT with on-chip optical ultrasound sensor arrays is rarely reported. In this work, we demonstrate PAT with a chalcogenide-based micro-ring sensor array containing 15 elements, while each element supports a bandwidth of 175 MHz (-6 dB) and a noise-equivalent pressure of 2.2 mPaHz-1/2. Moreover, by synthesizing a digital optical frequency comb (DOFC), we further develop an effective means of parallel interrogation to this sensor array. As a proof of concept, parallel interrogation with only one light source and one photoreceiver is demonstrated for PAT with this sensor array, providing images of fast-moving objects, leaf veins, and live zebrafish. The superior performance of the chalcogenide-based micro-ring sensor array and the effectiveness of the DOFC-enabled parallel interrogation offer great prospects for advancing applications in PAT.

Single photon single pixel imaging into thick scattering medium.

Imaging into thick scattering medium is a long-standing challenge. Beyond the quasi-ballistic regime, multiple scattering scrambles the spatiotemporal information of incident/emitted light, making canonical imaging based on light focusing nearly impossible. Diffusion optical tomography (DOT) is one of the most popular approach to look inside scattering medium, but quantitatively inverting the diffusion equation is ill-posed, and prior information of the medium is typically necessary, which is nontrivial to obtain. Here, we show theoretically and experimentally that, by synergizing the one-way light scattering characteristic of single pixel imaging with ultrasensitive single photon detection and a metric-guided image reconstruction, single photon single pixel imaging can serve as a simple and powerful alternative to DOT for imaging into thick scattering medium without prior knowledge or inverting the diffusion equation. We demonstrated an image resolution of 12 mm inside a 60 mm thick (∼ 78 mean free paths) scattering medium.

Multimode fiber-based greyscale image projector enabled by neural networks with high generalization ability.

Multimode fibers (MMFs) are emerging as promising transmission media for delivering images. However, strong mode coupling inherent in MMFs induces difficulties in directly projecting two-dimensional images through MMFs. By training two subnetworks named Actor-net and Model-net synergetically, [Nature Machine Intelligence2, 403 (2020)10.1038/s42256-020-0199-9] alleviated this issue and demonstrated projecting images through MMFs with high fidelity. In this work, we make a step further by improving the generalization ability to greyscale images. The modified projector network contains three subnetworks, namely forward-net, backward-net, and holography-net, accounting for forward propagation, backward propagation, and the phase-retrieval process. As a proof of concept, we experimentally trained the projector network using randomly generated phase maps and their corresponding resultant speckle images output from a 1-meter-long MMF. With the network being trained, we successfully demonstrated projecting binary images from MNIST and EMNIST and greyscale images from Fashion-MNIST, exhibiting averaged Pearson's correlation coefficients of 0.91, 0.92, and 0.87, respectively. Since all these projected images have never been seen by the projector network before, a strong generalization ability in projecting greyscale images is confirmed.

Switching between topological edge states in plasmonic systems using phase-change materials.

We introduce non-Hermitian plasmonic waveguide-cavity structures based on the Aubry-Andre-Harper model to realize switching between right and left topological edge states (TESs) using the phase-change material Ge2Sb2Te5 (GST). We show that switching between the crystalline and amorphous phases of GST leads to a shift of the dispersion relation of the optimized structure so that a right TES for the crystalline phase, and a left TES for the amorphous phase occur at the same frequency. Thus, we realize switching between right and left TESs at that frequency by switching between the crystalline and amorphous phases of GST. Our results could be potentially important for developing compact reconfigurable topological photonic devices.

High-speed single-exposure time-reversed ultrasonically encoded optical focusing against dynamic scattering.

Focusing light deep inside live scattering tissue promises to revolutionize biophotonics by enabling deep tissue noninvasive optical imaging, manipulation, and therapy. By combining with guide stars, wavefront shaping is emerging as a powerful tool to make scattering media optically transparent. However, for in vivo biomedical applications, the speeds of existing techniques are still too slow to accommodate the fast speckle decorrelation of live tissue. To address this key bottleneck, we develop a quaternary phase encoding scheme to enable single-exposure time-reversed ultrasonically encode optical focusing with full-phase modulations. Specifically, we focus light inside dynamic scattering media with an average mode time down to 29 ns, which indicates that more than 104 effective spatial modes can be controlled within 1 millisecond. With this technique, we demonstrate in vivo light focusing in between a highly opaque adult zebrafish of 5.1 millimeters in thickness and a ground glass diffuser. Our work presents an important step toward in vivo deep tissue applications of wavefront shaping.

Coaxial interferometry for camera-based ultrasound-modulated optical tomography with paired illumination.

Ultrasound-modulated optical tomography (UOT), which combines the advantages of both light and ultrasound, is a promising imaging modality for deep-tissue high-resolution imaging. Among existing implementations, camera-based UOT gains huge advances in modulation depth through parallel detection. However, limited by the long exposure time and the slow framerate of modern cameras, the measurement of UOT signals always requires holographic methods with additional reference beams. This requirement increases system complexity and is susceptible to environmental disturbances. To overcome this challenge, we develop coaxial interferometry for camera-based UOT in this work. Such a coaxial scheme is enabled by employing paired illumination with slightly different optical frequencies. To measure the UOT signal, the conventional phase-stepping method in holography can be directly transplanted into coaxial interferometry. Specifically, we performed both numerical investigations and experimental validations for camera-based UOT under the proposed coaxial scheme. One-dimensional imaging for an absorptive target buried inside a scattering medium was demonstrated. With coaxial interferometry, this work presents an effective way to reduce system complexity and cope with environmental disturbances for camera-based UOT.

Wavefront shaping: A versatile tool to conquer multiple scattering in multidisciplinary fields.

Optical techniques offer a wide variety of applications as light-matter interactions provide extremely sensitive mechanisms to probe or treat target media. Most of these implementations rely on the usage of ballistic or quasi-ballistic photons to achieve high spatial resolution. However, the inherent scattering nature of light in biological tissues or tissue-like scattering media constitutes a critical obstacle that has restricted the penetration depth of non-scattered photons and hence limited the implementation of most optical techniques for wider applications. In addition, the components of an optical system are usually designed and manufactured for a fixed function or performance. Recent advances in wavefront shaping have demonstrated that scattering- or component-induced phase distortions can be compensated by optimizing the wavefront of the input light pattern through iteration or by conjugating the transmission matrix of the scattering medium. This offers unprecedented opportunities in many applications to achieve controllable optical delivery or detection at depths or dynamically configurable functionalities by using scattering media to substitute conventional optical components. In this article, the recent progress of wavefront shaping in multidisciplinary fields is reviewed, from optical focusing and imaging with scattering media, functionalized devices, modulation of mode coupling, and nonlinearity in multimode fiber to multimode fiber-based applications. Apart from insights into the underlying principles and recent advances in wavefront shaping implementations, practical limitations and roadmap for future development are discussed in depth. Looking back and looking forward, it is believed that wavefront shaping holds a bright future that will open new avenues for noninvasive or minimally invasive optical interactions and arbitrary control inside deep tissues. The high degree of freedom with multiple scattering will also provide unprecedented opportunities to develop novel optical devices based on a single scattering medium (generic or customized) that can outperform traditional optical components.

Feedback-assisted transmission matrix measurement of a multimode fiber in a referenceless system.

Recent development in wavefront shaping shows the promise to employ multimode fibers (MMFs) to deliver images in endoscopy. In these applications, retrieving the transmission matrix (TM) of the MMF is especially important. Among existing non-holographic approaches, feedback-based wavefront shaping requires a large number of measurements, while directly measuring the TM can be easily trapped into local optimums if the constraints are insufficient. To reduce the required number of measurements, we combine the concepts of these two approaches and develop a scheme termed feedback-assisted TM measurements. We show that under such a hybrid scheme, less than 3N intensity measurements are sufficient to accurately retrieve one row of the TM that contains N unknown complex elements. As a proof of concept, we experimentally demonstrated retrieving multiple rows of the TM of an MMF using the proposed scheme with high fidelity. In particular, a single focus and dual foci through the MMF with enhancements larger than 75% of the theoretical values were reported.

Modeling of iterative time-reversed ultrasonically encoded optical focusing in a reflection mode.

Time-reversed ultrasonically-encoded (TRUE) optical focusing is a promising technique to realize deep-tissue optical focusing by employing ultrasonic guide stars. However, the sizes of the ultrasound-induced optical focus are determined by the wavelengths of the ultrasound, which are typically tens of microns. To satisfy the need for high-resolution imaging and manipulation, iterative TRUE (iTRUE) was proposed to break this limit by triggering repeated interactions between light and ultrasound and compressing the optical focus. However, even for the best result reported to date, the resolutions along the ultrasound axial and lateral direction were merely improved by only 2-fold to 3-fold. This observation leads to doubt whether iTRUE can be effective in reducing the size of the optical focus. In this work, we address this issue by developing a physical model to investigate iTRUE in a reflection mode numerically. Our numerical results show that, under the influence of shot noises, iTRUE can reduce the optical focus to a single speckle within a finite number of iterations. This model also allows numerical investigations of iTRUE in detail. Quantitatively, based on the parameters set, we show that the optical focus can be reduced to a size of 1.6 µm and a peak-to-background ratio over 104 can be realized. It is also shown that iTRUE cannot significantly advance the focusing depth. We anticipate that this work can serve as useful guidance for optimizing iTRUE system for future biomedical applications, including deep-tissue optical imaging, laser surgery, and optogenetics.

Characterization of the spectral memory effect of scattering media.

The optical memory effect is an interesting phenomenon exploited for deep-tissue optical imaging. Besides the widely studied memory effects in the spatial domain to accelerate point scanning speed, the spectral memory effect is also important in multispectral wavefront shaping. Although being theoretically analyzed for decades, quantitative studies of spectral memory effect on a variety of scattering media including biological tissue were rarely reported. In practice, quantifying the range of the spectral memory effect is essential in efficiently shaping broadband light, as it determines the optimum spectral resolution in realizing spatiotemporal focus through scattering media. In this work, we analyze the spectral memory effect based on a diffusion model. An explicit analytical expression involves the illumination wavelength, the diffusion constant, and the sample thickness is derived, which is consistent with the one in the literature. We experimentally quantified the range of spectral correlation for two types of biological tissue, tissue-mimicking phantoms with different concentrations, and diffusers. Specifically, for tissue-mimicking phantoms with calibrated scattering parameters, we show that a correction factor of more than 20 should be inserted, indicating that the range of spectral correlation is much larger than one would expect. This finding is particularly beneficial to multispectral wavefront shaping, as stringent requirements on the spectral resolution could be alleviated by at least one order of magnitude.

Imaging biological tissue with high-throughput single-pixel compressive holography.

Single-pixel holography (SPH) is capable of generating holographic images with rich spatial information by employing only a single-pixel detector. Thanks to the relatively low dark-noise production, high sensitivity, large bandwidth, and cheap price of single-pixel detectors in comparison to pixel-array detectors, SPH is becoming an attractive imaging modality at wavelengths where pixel-array detectors are not available or prohibitively expensive. In this work, we develop a high-throughput single-pixel compressive holography with a space-bandwidth-time product (SBP-T) of 41,667 pixels/s, realized by enabling phase stepping naturally in time and abandoning the need for phase-encoded illumination. This holographic system is scalable to provide either a large field of view (~83 mm2) or a high resolution (5.80 µm × 4.31 µm). In particular, high-resolution holographic images of biological tissues are presented, exhibiting rich contrast in both amplitude and phase. This work is an important step towards multi-spectrum imaging using a single-pixel detector in biophotonics.

Genetic-algorithm-assisted coherent enhancement absorption in scattering media by exploiting transmission and reflection matrices.

The investigations on coherent enhancement absorption (CEA) inside scattering media are critically important in biophotonics. CEA can deliver light to the targeted position, thus enabling deep-tissue optical imaging by improving signal strength and imaging resolution. In this work, we develop a numerical framework that employs the method of finite-difference time-domain. Both the transmission and reflection matrices of scattering media with open boundaries are constructed, allowing the studies on the eigenvalues and eigenchannels. To realize CEA for scattering media with local absorption, we develop a genetic-algorithm-assisted numerical model. By minimizing the total transmittance and reflectance simultaneously, different realizations of CEA are observed and, without setting internal monitors, can be differentiated with cases of light leaked from sides. By modulating the incident wavefront at only one side of the scattering medium, it is shown that for a 5-µm-diameter absorber buried inside a scattering medium of 15 µm × 12 µm, more than half of the incident light can be delivered and absorbed at the target position. The enhancement in absorption is more than four times higher than that with random input. This value can be even higher for smaller absorption regions. We also quantify the effectiveness of the method and show that it is inversely proportional to the openness of the scattering medium. This result is potentially useful for targeted light delivery inside scattering media with local absorption.

Single-shot ultrasound-modulated optical tomography with enhanced speckle contrast.

Ultrasound-modulated optical tomography (UOT) images optical contrast deep inside biological tissue. Among existing approaches, camera-based parallel detection is beneficial in modulation depth but is limited to the relatively slow framerate of cameras. This condition prevents such a scheme from achieving maturity to image live animals with sub-millisecond speckle correlation time. In this work, we developed on-axis single-shot UOT by investigating the statistics of speckles, breaking the restriction imposed by the slow camera framerate. As a proof of concept, we experimentally imaged a one-dimensional absorptive object buried inside a moving scattering medium with speckle correlation time down to 0.48 ms. We envision that this single-shot UOT is promising to cope with live animals with fast speckle decorrelation.

Coherent laser detection of the femtowatt-level frequency-shifted optical feedback based on a DFB fiber laser.

The theoretical basis and experimental realization of an all-fiber self-mixing laser Doppler velocimetry based on frequency-shifted feedback in a distributed feedback (DFB) fiber laser are presented, which employs a pair of fiber-coupled acousto-optic modulators to adjust the modulation intensity and frequency of the laser self-mixing effect. Moreover, the minimum optical feedback intensity for the velocity signal successfully measured by the interferometer is 5.12 fW, corresponding to 0.16 photons per Doppler cycle. The results demonstrate that the proposed scheme can adapt to the non-contact measurement requirements of the wide-range speed and weak feedback level in the complex environment.

Harnessing a multi-dimensional fibre laser using genetic wavefront shaping.

The multi-dimensional laser is a fascinating platform not only for the discovery and understanding of new higher-dimensional coherent lightwaves but also for the frontier study of the complex three-dimensional (3D) nonlinear dynamics and solitary waves widely involved in physics, chemistry, biology and materials science. Systemically controlling coherent lightwave oscillation in multi-dimensional lasers, however, is challenging and has largely been unexplored; yet, it is crucial for both designing 3D coherent light fields and unveiling any underlying nonlinear complexities. Here, for the first time, we genetically harness a multi-dimensional fibre laser using intracavity wavefront shaping technology such that versatile lasing characteristics can be manipulated. We demonstrate that the output power, mode profile, optical spectrum and mode-locking operation can be genetically optimized by appropriately designing the objective function of the genetic algorithm. It is anticipated that this genetic and systematic intracavity control technology for multi-dimensional lasers will be an important step for obtaining high-performance 3D lasing and presents many possibilities for exploring multi-dimensional nonlinear dynamics and solitary waves that may enable new applications.