Projecting colorful images through scattering media via deep learning.

The existence of scatterers in the optical path has been the major obstacle that prohibits one from projecting images through solid walls, turbid water, clouds, and fog. Recent developments in wavefront shaping and neural networks demonstrate effective compensation for scattering effects, showing the promise to project clear images against strong scattering. However, previous studies were mainly restricted to projecting greyscale images using monochromatic light, mainly due to the increased complexity of simultaneously controlling multiple wavelengths. In this work, we fill this blank by developing a projector network, which enables the projection of colorful images through scattering media with three primary colors. To validate the performance of the projector network, we experimentally demonstrated projecting colorful images obtained from the MINST dataset through two stacked diffusers. Quantitatively, the averaged intensity Pearson's correlation coefficient for 1,000 test colorful images reaches about 90.6%, indicating the superiority of the developed network. We anticipate that the projector network can be beneficial to a variety of display applications in scattering environments.

Multimode fiber-based greyscale image projector enabled by neural networks with high generalization ability.

Multimode fibers (MMFs) are emerging as promising transmission media for delivering images. However, strong mode coupling inherent in MMFs induces difficulties in directly projecting two-dimensional images through MMFs. By training two subnetworks named Actor-net and Model-net synergetically, [Nature Machine Intelligence2, 403 (2020)10.1038/s42256-020-0199-9] alleviated this issue and demonstrated projecting images through MMFs with high fidelity. In this work, we make a step further by improving the generalization ability to greyscale images. The modified projector network contains three subnetworks, namely forward-net, backward-net, and holography-net, accounting for forward propagation, backward propagation, and the phase-retrieval process. As a proof of concept, we experimentally trained the projector network using randomly generated phase maps and their corresponding resultant speckle images output from a 1-meter-long MMF. With the network being trained, we successfully demonstrated projecting binary images from MNIST and EMNIST and greyscale images from Fashion-MNIST, exhibiting averaged Pearson's correlation coefficients of 0.91, 0.92, and 0.87, respectively. Since all these projected images have never been seen by the projector network before, a strong generalization ability in projecting greyscale images is confirmed.