Inference in artificial intelligence with deep optics and photonics.

Artificial intelligence tasks across numerous applications require accelerators for fast and low-power execution. Optical computing systems may be able to meet these domain-specific needs but, despite half a century of research, general-purpose optical computing systems have yet to mature into a practical technology. Artificial intelligence inference, however, especially for visual computing applications, may offer opportunities for inference based on optical and photonic systems. In this Perspective, we review recent work on optical computing for artificial intelligence applications and discuss its promise and challenges.

Fast wavefront shaping for two-photon brain imaging with multipatch correction.

Nonlinear fluorescence microscopy promotes in-vivo optical imaging of cellular structure at diffraction-limited resolution deep inside scattering biological tissues. Active compensation of tissue-induced aberrations and light scattering through adaptive wavefront correction further extends the accessible depth by restoring high resolution at large depth. However, those corrections are only valid over a very limited field of view within the angular memory effect. To overcome this limitation, we introduce an acousto-optic light modulation technique for fluorescence imaging with simultaneous wavefront correction at pixel scan speed. Biaxial wavefront corrections are first learned by adaptive optimization at multiple locations in the image field. During image acquisition, the learned corrections are then switched on the fly according to the position of the excitation focus during the raster scan. The proposed microscope is applied to in vivo transcranial neuron imaging and demonstrates multi-patch correction of thinned skull-induced aberrations and scattering at 40-kHz data acquisition speed.

Direct imaging of micrometer-thick interfaces in salt-salt aqueous biphasic systems.

Unlike the interface between two immiscible electrolyte solutions (ITIES) formed between water and polar solvents, molecular understanding of the liquid-liquid interface formed for aqueous biphasic systems (ABSs) is relatively limited and mostly relies on surface tension measurements and thermodynamic models. Here, high-resolution Raman imaging is used to provide spatial and chemical resolution of the interface of lithium chloride - lithium bis(trifluoromethanesulfonyl)imide - water (LiCl-LiTFSI-water) and HCl-LiTFSI-water, prototypical salt-salt ABSs found in a range of electrochemical applications. The concentration profiles of both TFSI anions and water are found to be sigmoidal thus not showing any signs of a positive adsorption for both salts and solvent. More striking, however, is the length at which the concentration profiles extend, ranging from 11 to 2 µm with increasing concentrations, compared to a few nanometers for ITIES. We thus reveal that unlike ITIES, salt-salt ABSs do not have a molecularly sharp interface but rather form an interphase with a gradual change of environment from one phase to the other. This knowledge represents a major stepping-stone in the understanding of aqueous interfaces, key for mastering ion or electron transfer dynamics in a wide range of biological and technological settings including novel battery technologies such as membraneless redox flow and dual-ion batteries.

Linear optical random projections without holography.

We introduce what we believe to be a novel method to perform linear optical random projections without the need for holography. Our method consists of a computationally trivial combination of multiple intensity measurements to mitigate the information loss usually associated with the absolute-square non-linearity imposed by optical intensity measurements. Both experimental and numerical findings demonstrate that the resulting matrix consists of real-valued, independent, and identically distributed (i.i.d.) Gaussian random entries. Our optical setup is simple and robust, as it does not require interference between two beams. We demonstrate the practical applicability of our method by performing dimensionality reduction on high-dimensional data, a common task in randomized numerical linear algebra with relevant applications in machine learning.

Quantifying high-dimensional spatial entanglement with a single-photon-sensitive time-stamping camera.

High-dimensional entanglement is a promising resource for quantum technologies. Being able to certify it for any quantum state is essential. However, to date, experimental entanglement certification methods are imperfect and leave some loopholes open. Using a single-photon-sensitive time-stamping camera, we quantify high-dimensional spatial entanglement by collecting all output modes and without background subtraction, two critical steps on the route toward assumptions-free entanglement certification. We show position-momentum Einstein-Podolsky-Rosen (EPR) correlations and quantify the entanglement of formation of our source to be larger than 2.8 along both transverse spatial axes, indicating a dimension higher than 14. Our work overcomes important challenges in photonic entanglement quantification and paves the way toward the development of practical quantum information processing protocols based on high-dimensional entanglement.

Physics-based neural network for non-invasive control of coherent light in scattering media.

Optical imaging through complex media, such as biological tissues or fog, is challenging due to light scattering. In the multiple scattering regime, wavefront shaping provides an effective method to retrieve information; it relies on measuring how the propagation of different optical wavefronts are impacted by scattering. Based on this principle, several wavefront shaping techniques were successfully developed, but most of them are highly invasive and limited to proof-of-principle experiments. Here, we propose to use a neural network approach to non-invasively characterize and control light scattering inside the medium and also to retrieve information of hidden objects buried within it. Unlike most of the recently-proposed approaches, the architecture of our neural network with its layers, connected nodes and activation functions has a true physical meaning as it mimics the propagation of light in our optical system. It is trained with an experimentally-measured input/output dataset built from a series of incident light patterns and corresponding camera snapshots. We apply our physics-based neural network to a fluorescence microscope in epi-configuration and demonstrate its performance through numerical simulations and experiments. This flexible method can include physical priors and we show that it can be applied to other systems as, for example, non-linear or coherent contrast mechanisms.

Scattering correcting wavefront shaping for three-photon microscopy.

Three-photon (3P) microscopy is getting traction due to its superior performance in deep tissues. Yet, aberrations and light scattering still pose one of the main limitations in the attainable depth ranges for high-resolution imaging. Here, we show scattering correcting wavefront shaping with a simple continuous optimization algorithm, guided by the integrated 3P fluorescence signal. We demonstrate focusing and imaging behind scattering layers and investigate convergence trajectories for different sample geometries and feedback non-linearities. Furthermore, we show imaging through a mouse skull and demonstrate a novel, to the best of our knowledge, fast phase estimation scheme that substantially increases the speed at which the optimal correction can be found.

Non-invasive chemically selective energy delivery and focusing inside a scattering medium guided by Raman scattering.

Raman scattering is a chemically selective probing mechanism with diverse applications in industry and clinical settings. Yet, most samples are optically opaque limiting the applicability of Raman probing at depth. Here, we demonstrate chemically selective energy deposition behind a scattering medium by combining prior information on the chemical's spectrum with the measurement of a spectrally resolved Raman speckle as a feedback mechanism for wavefront shaping. We demonstrate unprecedented sixfold signal enhancement in an epi-geometry, realizing targeted energy deposition and focusing on individual Raman active particles.

Three-dimensional broadband light beam manipulation in forward scattering samples.

Focusing light into highly disordered biological tissue is a major challenge in optical microscopy and biomedical imaging due to scattering. However, correlations in the scattering matrix, known as "memory effects", can be used to improve imaging capabilities. Here we discuss theoretically and numerically the possibility to achieve three-dimensional ultrashort laser focusing and scanning inside forward scattering media, beyond the scattering mean free path, by simultaneously taking advantage of the angular and the chromato-axial memory effects. The numerical model is presented in details, is validated within the state of the art theoretical and experimental framework and is finally used to propose a scheme for focusing ultra-short laser pulses in depth through forward scattering media.

Spectrally resolved point-spread-function engineering using a complex medium.

Propagation of an ultrashort pulse of light through strongly scattering media generates an intricate spatio-spectral speckle that can be described by means of the multi-spectral transmission matrix (MSTM). In conjunction with a spatial light modulator, the MSTM enables the manipulation of the pulse leaving the medium; in particular focusing it at any desired spatial position and/or time. Here, we demonstrate how to engineer the point-spread-function of the focused beam both spatially and spectrally, from the measured MSTM. It consists of numerically filtering the spatial content at each wavelength of the matrix prior to focusing. We experimentally report on the versatility of the technique through several examples, in particular as an alternative to simultaneous spatial and temporal focusing, with potential applications in multiphoton microscopy.

Engineering spatial correlations of entangled photon pairs by pump beam shaping.

The ability to engineer the properties of quantum optical states is essential for quantum information processing applications. Here, we demonstrate tunable control of spatial correlations between photon pairs produced by spontaneous parametric down-conversion, and measure them using an electron multiplying charge coupled device (EMCCD) camera. By shaping the spatial pump beam profile in a type-I collinear configuration, we tailor the spatial structure of coincidences between photon pairs entangled in high dimensions without effect on intensity. The results highlight fundamental aspects of spatial coherence and hold potential for the development of quantum technologies based on high-dimensional spatial entanglement.

Accelerating ptychographic reconstructions using spectral initializations.

Ptychography is a promising phase retrieval technique for label-free quantitative phase imaging. Recent advances in phase retrieval algorithms witnessed the development of spectral methods to accelerate gradient descent algorithms. Using spectral initializations on experimental data, for the first time, we report three times faster ptychographic reconstructions than with a standard gradient descent algorithm and improved resilience to noise. Coming at no additional computational cost compared to gradient-descent-based algorithms, spectral methods have the potential to be implemented in large-scale iterative ptychographic algorithms.

Deeply Subwavelength Localization with Reverberation-Coded Aperture.

Accessing subwavelength information about a scene from the far-field without invasive near-field manipulations is a fundamental challenge in wave engineering. Yet it is well understood that the dwell time of waves in complex media sets the scale for the waves' sensitivity to perturbations. Modern coded-aperture imagers leverage the degrees of freedom (d.o.f.) offered by complex media as natural multiplexor but do not recognize and reap the fundamental difference between placing the object of interest outside or within the complex medium. Here, we show that the precision of localizing a subwavelength object can be improved by several orders of magnitude simply by enclosing it in its far field with a reverberant passive chaotic cavity. We identify deep learning as a suitable noise-robust tool to extract subwavelength localization information encoded in multiplexed measurements, achieving resolutions well beyond those available in the training data. We demonstrate our finding in the microwave domain: harnessing the configurational d.o.f. of a simple programmable metasurface, we localize a subwavelength object along a curved trajectory inside a chaotic cavity with a resolution of λ/76 using intensity-only single-frequency single-pixel measurements. Our results may have important applications in photoacoustic imaging as well as human-machine interaction based on reverberating elastic waves, sound, or microwaves.

Speckle Engineering through Singular Value Decomposition of the Transmission Matrix.

Speckle patterns are ubiquitous in optics and have multiple applications for which the control of their spatial correlations is essential. Here, we report on a method to engineer speckle correlations behind a scattering medium through the singular value decomposition of the transmission matrix. We not only demonstrate control over the speckle grain size and shape but also realize patterns with nonlocal correlations. Moreover, we show that the reach of our method extends also along the axial dimension, allowing volumetric speckle engineering behind scattering layers.

Far-Field Wavefront Control of Nonlinear Luminescence in Disordered Gold Metasurfaces.

We demonstrate the local optimization of nonlinear luminescence from disordered gold metasurfaces by shaping the phase of femtosecond excitation. This process is enabled by the far-field wavefront control of plasmonic modes delocalized over the sample surface, leading to a coherent enhancement of subwavelength electric fields. In practice, the increase in nonlinear luminescence is strongly sensitive to both the nanometer-scale morphology and the level of structural complexity of the gold metasurface. We typically observe a 2 orders of magnitude enhancement of the luminescence signal for an optimized excitation wavefront compared to a random one. These results demonstrate how disordered metasurfaces made of randomly coupled plasmonic resonators, together with wavefront shaping, provide numerous degrees of freedom to program locally optimized nonlinear responses and optical hotspots.

Non-invasive single-shot recovery of a point-spread function of a memory effect based scattering imaging system.

Accessing the point-spread function (PSF) of a complex optical system is important for a variety of imaging applications. However, placing an invasive point source is often impractical, and estimating it blindly with multiple frames is slow and requires a complex nonlinear optimization. Here, we introduce a simple single-shot method to noninvasively recover the accurate PSF of an isoplanatic imaging system, in the context of multiple light scattering. Our approach is based on the reconstruction of any unknown sparse hidden object using the autocorrelation imaging technique, followed by a deconvolution with a blur kernel derived from the statistics of a speckle pattern. A deconvolution on the camera image then retrieves the accurate PSF of the system, enabling further imaging applications. We demonstrate numerically and experimentally the effectiveness of this approach compared to previous deconvolution techniques.

Classical optics in France: introduction.

This special issue of the Journal of the Optical Society of America A (JOSA A) is devoted to the wide array of French researchers from universities and state research organisms, offering them the opportunity to share and showcase their current research in the fields of optics and imaging sciences to the global community.

Snapshot fiber spectral imaging using speckle correlations and compressive sensing.

Snapshot spectral imaging is rapidly gaining interest for remote sensing applications. Acquiring spatial and spectral data within one image promotes fast measurement times, and reduces the need for stabilized scanning imaging systems. Many current snapshot technologies, which rely on gratings or prisms to characterize wavelength information, are difficult to reduce in size for portable hyperspectral imaging. Here, we show that a multicore multimode fiber can be used as a compact spectral imager with sub-nanometer resolution, by encoding spectral information within a monochrome CMOS camera. We characterize wavelength-dependent speckle patterns for up to 3000 fiber cores over a broad wavelength range. A clustering algorithm is employed in combination with l1-minimization to limit data collection at the acquisition stage for the reconstruction of spectral images that are sparse in the wavelength domain. We also show that in the non-compressive regime these techniques are able to accurately reconstruct broadband information.

Transmission matrix approaches for nonlinear fluorescence excitation through multiple scattering media.

Several matrix approaches were developed to control light propagation through multiple scattering media under illumination of ultrashort pulses of light. These matrices can be recorded with either spectral or temporal resolution. Thanks to wavefront shaping, temporal and spatial refocusing has been demonstrated. In this Letter, we study how these different methods can be exploited to enhance a two-photon excitation fluorescence process. We first compare the different techniques on micrometer-size isolated fluorescent beads. We then demonstrate point-scanning imaging of these fluorescent microbeads located after a thick scattering medium at a depth where conventional imaging would be impossible because of scattering effects.

Imaging through a thin scattering layer and jointly retrieving the point-spread-function using phase-diversity.

Recently introduced angular-memory-effect based techniques enable non-invasive imaging of objects hidden behind thin scattering layers. However, both the speckle-correlation and the bispectrum analysis are based on the statistical average of large amounts of speckle grains, which determines that they can hardly access the important information of the point-spread-function (PSF) of a highly scattering imaging system. Here, inspired by notions used in astronomy, we present a phase-diversity speckle imaging scheme, based on recording a sequence of intensity speckle patterns at various imaging planes, and experimentally demonstrate that in addition to being able to retrieve the image of hidden objects, we can also simultaneously estimate the pupil function and the PSF of a highly scattering imaging system without any guide-star nor reference.