Nonlinear optical encoding enabled by recurrent linear scattering.

Optical information processing and computing can potentially offer enhanced performance, scalability and energy efficiency. However, achieving nonlinearity-a critical component of computation-remains challenging in the optical domain. Here we introduce a design that leverages a multiple-scattering cavity to passively induce optical nonlinear random mapping with a continuous-wave laser at a low power. Each scattering event effectively mixes information from different areas of a spatial light modulator, resulting in a highly nonlinear mapping between the input data and output pattern. We demonstrate that our design retains vital information even when the readout dimensionality is reduced, thereby enabling optical data compression. This capability allows our optical platforms to offer efficient optical information processing solutions across applications. We demonstrate our design's efficacy across tasks, including classification, image reconstruction, keypoint detection and object detection, all of which are achieved through optical data compression combined with a digital decoder. In particular, high performance at extreme compression ratios is observed in real-time pedestrian detection. Our findings open pathways for novel algorithms and unconventional architectural designs for optical computing.

Roadmap on computational methods in optical imaging and holography [invited].

Computational methods have been established as cornerstones in optical imaging and holography in recent years. Every year, the dependence of optical imaging and holography on computational methods is increasing significantly to the extent that optical methods and components are being completely and efficiently replaced with computational methods at low cost. This roadmap reviews the current scenario in four major areas namely incoherent digital holography, quantitative phase imaging, imaging through scattering layers, and super-resolution imaging. In addition to registering the perspectives of the modern-day architects of the above research areas, the roadmap also reports some of the latest studies on the topic. Computational codes and pseudocodes are presented for computational methods in a plug-and-play fashion for readers to not only read and understand but also practice the latest algorithms with their data. We believe that this roadmap will be a valuable tool for analyzing the current trends in computational methods to predict and prepare the future of computational methods in optical imaging and holography. Supplementary Information: The online version contains supplementary material available at 10.1007/s00340-024-08280-3.

Concurrent oxygen evolution reaction pathways revealed by high-speed compressive Raman imaging.

Transition metal oxides are state-of-the-art materials for catalysing the oxygen evolution reaction (OER), whose slow kinetics currently limit the efficiency of water electrolysis. However, microscale physicochemical heterogeneity between particles, dynamic reactions both in the bulk and at the surface, and an interplay between particle reactivity and electrolyte makes probing the OER challenging. Here, we overcome these limitations by applying state-of-the-art compressive Raman imaging to uncover concurrent bias-dependent pathways for the OER in a dense, crystalline electrocatalyst, α-Li2IrO3. By spatially and temporally tracking changes in stretching modes we follow catalytic activation and charge accumulation following ion exchange under various electrolytes and cycling conditions, comparing our observations with other crystalline catalysts (IrO2, LiCoO2). We demonstrate that at low overpotentials the reaction between water and the oxidized catalyst surface is compensated by bulk ion exchange, as usually only found for amorphous, electrolyte permeable, catalysts. At high overpotentials the charge is compensated by surface redox active sites, as in other crystalline catalysts such as IrO2. Hence, our work reveals charge compensation can extend beyond the surface in crystalline catalysts. More generally, the results highlight the power of compressive Raman imaging for chemically specific tracking of microscale reaction dynamics in catalysts, battery materials, or memristors.

Demixing fluorescence time traces transmitted by multimode fibers.

Optical methods based on thin multimode fibers (MMFs) are promising tools for measuring neuronal activity in deep brain regions of freely moving mice thanks to their small diameter. However, current methods are limited: while fiber photometry provides only ensemble activity, imaging techniques using of long multimode fibers are very sensitive to bending and have not been applied to unrestrained rodents yet. Here, we demonstrate the fundamentals of a new approach using a short MMF coupled to a miniscope. In proof-of-principle in vitro experiments, we disentangled spatio-temporal fluorescence signals from multiple fluorescent sources transmitted by a thin (200 µm) and short (8 mm) MMF, using a general unconstrained non-negative matrix factorization algorithm directly on the raw video data. Furthermore, we show that low-cost open-source miniscopes have sufficient sensitivity to image the same fluorescence patterns seen in our proof-of-principle experiment, suggesting a new avenue for novel minimally invasive deep brain studies using multimode fibers in freely behaving mice.

Large-scale photonic computing with nonlinear disordered media.

Neural networks find widespread use in scientific and technological applications, yet their implementations in conventional computers have encountered bottlenecks due to ever-expanding computational needs. Photonic computing is a promising neuromorphic platform with potential advantages of massive parallelism, ultralow latency and reduced energy consumption but mostly for computing linear operations. Here we demonstrate a large-scale, high-performance nonlinear photonic neural system based on a disordered polycrystalline slab composed of lithium niobate nanocrystals. Mediated by random quasi-phase-matching and multiple scattering, linear and nonlinear optical speckle features are generated as the interplay between the simultaneous linear random scattering and the second-harmonic generation, defining a complex neural network in which the second-order nonlinearity acts as internal nonlinear activation functions. Benchmarked against linear random projection, such nonlinear mapping embedded with rich physical computational operations shows improved performance across a large collection of machine learning tasks in image classification, regression and graph classification. Demonstrating up to 27,648 input and 3,500 nonlinear output nodes, the combination of optical nonlinearity and random scattering serves as a scalable computing engine for diverse applications.

Speckle-enabled in vivo demixing of neural activity in the mouse brain.

Functional imaging of neuronal activity in awake animals, using a combination of fluorescent reporters of neuronal activity and various types of microscopy modalities, has become an indispensable tool in neuroscience. While various imaging modalities based on one-photon (1P) excitation and parallel (camera-based) acquisition have been successfully used for imaging more transparent samples, when imaging mammalian brain tissue, due to their scattering properties, two-photon (2P) microscopy systems are necessary. In 2P microscopy, the longer excitation wavelengths reduce the amount of scattering while the diffraction-limited 3D localization of excitation largely eliminates out-of-focus fluorescence. However, this comes at the cost of time-consuming serial scanning of the excitation spot and more complex and expensive instrumentation. Thus, functional 1P imaging modalities that can be used beyond the most transparent specimen are highly desirable. Here, we transform light scattering from an obstacle into a tool. We use speckles with their unique patterns and contrast, formed when fluorescence from individual neurons propagates through rodent cortical tissue, to encode neuronal activity. Spatiotemporal demixing of these patterns then enables functional recording of neuronal activity from a group of discriminable sources. For the first time, we provide an experimental, in vivo characterization of speckle generation, speckle imaging and speckle-assisted demixing of neuronal activity signals in the scattering mammalian brain tissue. We found that despite an initial fast speckle decorrelation, substantial correlation was maintained over minute-long timescales that contributed to our ability to demix temporal activity traces in the mouse brain in vivo. Informed by in vivo quantifications of speckle patterns from single and multiple neurons excited using 2P scanning excitation, we recorded and demixed activity from several sources excited using 1P oblique illumination. In our proof-of-principle experiments, we demonstrate in vivo speckle-assisted demixing of functional signals from groups of sources in a depth range of 220-320 µm in mouse cortex, limited by available speckle contrast. Our results serve as a basis for designing an in vivo functional speckle imaging modality and for maximizing the key resource in any such modality, the speckle contrast. We anticipate that our results will provide critical quantitative guidance to the community for designing techniques that overcome light scattering as a fundamental limitation in bioimaging.

Neurophotonics beyond the Surface: Unmasking the Brain's Complexity Exploiting Optical Scattering.

The intricate nature of the brain necessitates the application of advanced probing techniques to comprehensively study and understand its working mechanisms. Neurophotonics offers minimally invasive methods to probe the brain using optics at cellular and even molecular levels. However, multiple challenges persist, especially concerning imaging depth, field of view, speed, and biocompatibility. A major hindrance to solving these challenges in optics is the scattering nature of the brain. This perspective highlights the potential of complex media optics, a specialized area of study focused on light propagation in materials with intricate heterogeneous optical properties, in advancing and improving neuronal readouts for structural imaging and optical recordings of neuronal activity. Key strategies include wavefront shaping techniques and computational imaging and sensing techniques that exploit scattering properties for enhanced performance. We discuss the potential merger of the two fields as well as potential challenges and perspectives toward longer term in vivo applications.

Neurophotonics beyond the surface: unmasking the brain's complexity exploiting optical scattering.

The intricate nature of the brain necessitates the application of advanced probing techniques to comprehensively study and understand its working mechanisms. Neurophotonics offers minimally invasive methods to probe the brain using optics at cellular and even molecular levels. However, multiple challenges persist, especially concerning imaging depth, field of view, speed, and biocompatibility. A major hindrance to solving these challenges in optics is the scattering nature of the brain. This perspective highlights the potential of complex media optics, a specialized area of study focused on light propagation in materials with intricate heterogeneous optical properties, in advancing and improving neuronal readouts for structural imaging and optical recordings of neuronal activity. Key strategies include wavefront shaping techniques and computational imaging and sensing techniques that exploit scattering properties for enhanced performance. We discuss the potential merger of the two fields as well as potential challenges and perspectives toward longer term in vivo applications.

Single-pixel transmission matrix recovery via two-photon fluorescence.

Imaging at depth in opaque materials has long been a challenge. Recently, wavefront shaping has enabled notable advance for deep imaging. Nevertheless, most noninvasive wavefront-shaping methods require cameras, lack the sensitivity for deep imaging under weak optical signals, or can only focus on a single "guidestar." Here, we retrieve the transmission matrix (TM) noninvasively using two-photon fluorescence exploiting a single-pixel detection combined with a computational framework, allowing to achieve single-target focus on multiple guidestars spread beyond the memory effect range. In addition, if we assume that memory effect correlations exist in the TM, we are able to substantially reduce the number of measurements needed.

Fast wavefront shaping for two-photon brain imaging with multipatch correction.

Nonlinear fluorescence microscopy promotes in-vivo optical imaging of cellular structure at diffraction-limited resolution deep inside scattering biological tissues. Active compensation of tissue-induced aberrations and light scattering through adaptive wavefront correction further extends the accessible depth by restoring high resolution at large depth. However, those corrections are only valid over a very limited field of view within the angular memory effect. To overcome this limitation, we introduce an acousto-optic light modulation technique for fluorescence imaging with simultaneous wavefront correction at pixel scan speed. Biaxial wavefront corrections are first learned by adaptive optimization at multiple locations in the image field. During image acquisition, the learned corrections are then switched on the fly according to the position of the excitation focus during the raster scan. The proposed microscope is applied to in vivo transcranial neuron imaging and demonstrates multi-patch correction of thinned skull-induced aberrations and scattering at 40-kHz data acquisition speed.

Linear optical random projections without holography.

We introduce what we believe to be a novel method to perform linear optical random projections without the need for holography. Our method consists of a computationally trivial combination of multiple intensity measurements to mitigate the information loss usually associated with the absolute-square non-linearity imposed by optical intensity measurements. Both experimental and numerical findings demonstrate that the resulting matrix consists of real-valued, independent, and identically distributed (i.i.d.) Gaussian random entries. Our optical setup is simple and robust, as it does not require interference between two beams. We demonstrate the practical applicability of our method by performing dimensionality reduction on high-dimensional data, a common task in randomized numerical linear algebra with relevant applications in machine learning.

Three-dimensional operando optical imaging of particle and electrolyte heterogeneities inside Li-ion batteries.

Understanding (de)lithiation heterogeneities in battery materials is key to ensure optimal electrochemical performance. However, this remains challenging due to the three-dimensional morphology of electrode particles, the involvement of both solid- and liquid-phase reactants and a range of relevant timescales (seconds to hours). Here we overcome this problem and demonstrate the use of confocal microscopy for the simultaneous three-dimensional operando measurement of lithium-ion dynamics in individual agglomerate particles, and the electrolyte in batteries. We examine two technologically important cathode materials: LixCoO2 and LixNi0.8Mn0.1Co0.1O2. The surface-to-core transport velocity of Li-phase fronts and volume changes are captured as a function of cycling rate. Additionally, we visualize heterogeneities in the bulk and at agglomerate surfaces during cycling, and image microscopic liquid electrolyte concentration gradients. We discover that surface-limited reactions and intra-agglomerate competing rates control (de)lithiation and structural heterogeneities in agglomerate-based electrodes. Importantly, the conditions under which optical imaging can be performed inside the complex environments of battery electrodes are outlined.

Quantifying high-dimensional spatial entanglement with a single-photon-sensitive time-stamping camera.

High-dimensional entanglement is a promising resource for quantum technologies. Being able to certify it for any quantum state is essential. However, to date, experimental entanglement certification methods are imperfect and leave some loopholes open. Using a single-photon-sensitive time-stamping camera, we quantify high-dimensional spatial entanglement by collecting all output modes and without background subtraction, two critical steps on the route toward assumptions-free entanglement certification. We show position-momentum Einstein-Podolsky-Rosen (EPR) correlations and quantify the entanglement of formation of our source to be larger than 2.8 along both transverse spatial axes, indicating a dimension higher than 14. Our work overcomes important challenges in photonic entanglement quantification and paves the way toward the development of practical quantum information processing protocols based on high-dimensional entanglement.

Direct imaging of micrometer-thick interfaces in salt-salt aqueous biphasic systems.

Unlike the interface between two immiscible electrolyte solutions (ITIES) formed between water and polar solvents, molecular understanding of the liquid-liquid interface formed for aqueous biphasic systems (ABSs) is relatively limited and mostly relies on surface tension measurements and thermodynamic models. Here, high-resolution Raman imaging is used to provide spatial and chemical resolution of the interface of lithium chloride - lithium bis(trifluoromethanesulfonyl)imide - water (LiCl-LiTFSI-water) and HCl-LiTFSI-water, prototypical salt-salt ABSs found in a range of electrochemical applications. The concentration profiles of both TFSI anions and water are found to be sigmoidal thus not showing any signs of a positive adsorption for both salts and solvent. More striking, however, is the length at which the concentration profiles extend, ranging from 11 to 2 µm with increasing concentrations, compared to a few nanometers for ITIES. We thus reveal that unlike ITIES, salt-salt ABSs do not have a molecularly sharp interface but rather form an interphase with a gradual change of environment from one phase to the other. This knowledge represents a major stepping-stone in the understanding of aqueous interfaces, key for mastering ion or electron transfer dynamics in a wide range of biological and technological settings including novel battery technologies such as membraneless redox flow and dual-ion batteries.

Physics-based neural network for non-invasive control of coherent light in scattering media.

Optical imaging through complex media, such as biological tissues or fog, is challenging due to light scattering. In the multiple scattering regime, wavefront shaping provides an effective method to retrieve information; it relies on measuring how the propagation of different optical wavefronts are impacted by scattering. Based on this principle, several wavefront shaping techniques were successfully developed, but most of them are highly invasive and limited to proof-of-principle experiments. Here, we propose to use a neural network approach to non-invasively characterize and control light scattering inside the medium and also to retrieve information of hidden objects buried within it. Unlike most of the recently-proposed approaches, the architecture of our neural network with its layers, connected nodes and activation functions has a true physical meaning as it mimics the propagation of light in our optical system. It is trained with an experimentally-measured input/output dataset built from a series of incident light patterns and corresponding camera snapshots. We apply our physics-based neural network to a fluorescence microscope in epi-configuration and demonstrate its performance through numerical simulations and experiments. This flexible method can include physical priors and we show that it can be applied to other systems as, for example, non-linear or coherent contrast mechanisms.

Non-invasive chemically selective energy delivery and focusing inside a scattering medium guided by Raman scattering.

Raman scattering is a chemically selective probing mechanism with diverse applications in industry and clinical settings. Yet, most samples are optically opaque limiting the applicability of Raman probing at depth. Here, we demonstrate chemically selective energy deposition behind a scattering medium by combining prior information on the chemical's spectrum with the measurement of a spectrally resolved Raman speckle as a feedback mechanism for wavefront shaping. We demonstrate unprecedented sixfold signal enhancement in an epi-geometry, realizing targeted energy deposition and focusing on individual Raman active particles.

Large field-of-view non-invasive imaging through scattering layers using fluctuating random illumination.

Non-invasive optical imaging techniques are essential diagnostic tools in many fields. Although various recent methods have been proposed to utilize and control light in multiple scattering media, non-invasive optical imaging through and inside scattering layers across a large field of view remains elusive due to the physical limits set by the optical memory effect, especially without wavefront shaping techniques. Here, we demonstrate an approach that enables non-invasive fluorescence imaging behind scattering layers with field-of-views extending well beyond the optical memory effect. The method consists in demixing the speckle patterns emitted by a fluorescent object under variable unknown random illumination, using matrix factorization and a novel fingerprint-based reconstruction. Experimental validation shows the efficiency and robustness of the method with various fluorescent samples, covering a field of view up to three times the optical memory effect range. Our non-invasive imaging technique is simple, neither requires a spatial light modulator nor a guide star, and can be generalized to a wide range of incoherent contrast mechanisms and illumination schemes.

Scattering correcting wavefront shaping for three-photon microscopy.

Three-photon (3P) microscopy is getting traction due to its superior performance in deep tissues. Yet, aberrations and light scattering still pose one of the main limitations in the attainable depth ranges for high-resolution imaging. Here, we show scattering correcting wavefront shaping with a simple continuous optimization algorithm, guided by the integrated 3P fluorescence signal. We demonstrate focusing and imaging behind scattering layers and investigate convergence trajectories for different sample geometries and feedback non-linearities. Furthermore, we show imaging through a mouse skull and demonstrate a novel, to the best of our knowledge, fast phase estimation scheme that substantially increases the speed at which the optimal correction can be found.

Engineering spatial correlations of entangled photon pairs by pump beam shaping.

The ability to engineer the properties of quantum optical states is essential for quantum information processing applications. Here, we demonstrate tunable control of spatial correlations between photon pairs produced by spontaneous parametric down-conversion, and measure them using an electron multiplying charge coupled device (EMCCD) camera. By shaping the spatial pump beam profile in a type-I collinear configuration, we tailor the spatial structure of coincidences between photon pairs entangled in high dimensions without effect on intensity. The results highlight fundamental aspects of spatial coherence and hold potential for the development of quantum technologies based on high-dimensional spatial entanglement.

Speckle Engineering through Singular Value Decomposition of the Transmission Matrix.

Speckle patterns are ubiquitous in optics and have multiple applications for which the control of their spatial correlations is essential. Here, we report on a method to engineer speckle correlations behind a scattering medium through the singular value decomposition of the transmission matrix. We not only demonstrate control over the speckle grain size and shape but also realize patterns with nonlocal correlations. Moreover, we show that the reach of our method extends also along the axial dimension, allowing volumetric speckle engineering behind scattering layers.