Calibration-free quantitative phase imaging in multi-core fiber endoscopes using end-to-end deep learning.

Quantitative phase imaging (QPI) through multi-core fibers (MCFs) has been an emerging in vivo label-free endoscopic imaging modality with minimal invasiveness. However, the computational demands of conventional iterative phase retrieval algorithms have limited their real-time imaging potential. We demonstrate a learning-based MCF phase imaging method that significantly reduced the phase reconstruction time to 5.5 ms, enabling video-rate imaging at 181 fps. Moreover, we introduce an innovative optical system that automatically generated the first, to the best of our knowledge, open-source dataset tailored for MCF phase imaging, comprising 50,176 paired speckles and phase images. Our trained deep neural network (DNN) demonstrates a robust phase reconstruction performance in experiments with a mean fidelity of up to 99.8%. Such an efficient fiber phase imaging approach can broaden the applications of QPI in hard-to-reach areas.

Chromatic aberration correction employing reinforcement learning.

In fluorescence microscopy a multitude of labels are used that bind to different structures of biological samples. These often require excitation at different wavelengths and lead to different emission wavelengths. The presence of different wavelengths can induce chromatic aberrations, both in the optical system and induced by the sample. These lead to a detuning of the optical system, as the focal positions shift in a wavelength dependent manner and finally to a decrease in the spatial resolution. We present the correction of chromatic aberrations by using an electrical tunable achromatic lens driven by reinforcement learning. The tunable achromatic lens consists of two lens chambers filled with different optical oils and sealed with deformable glass membranes. By deforming the membranes of both chambers in a targeted manner, the chromatic aberrations present in the system can be manipulated to tackle both systematic and sample induced aberrations. We demonstrate chromatic aberration correction of up to 2200 mm and shift of the focal spot positions of 4000 mm. For control of this non-linear system with four input voltages, several reinforcement learning agents are trained and compared. The experimental results show that the trained agent can correct system and sample induced aberration and thereby improve the imaging quality, this is demonstrated using biomedical samples. In this case human thyroid was used for demonstration.

Fully refractive telecentric f-theta microscope based on adaptive elements for 3D raster scanning of biological tissues.

Various techniques in microscopy are based on point-wise acquisition, which provides advantages in acquiring sectioned images, for example in confocal or two-photon microscopy. The advantages come along with the need to perform three-dimensional scanning, which is often realized by mechanical movement achieved by stage-scanning or piezo-based scanning in the axial direction. Lateral scanning often employs galvo-mirrors, leading to a reflective setup and hence to a folded beam path. In this paper, we introduce a fully refractive microscope capable of three-dimensional scanning, which employs the combination of an adaptive lens, an adaptive prism, and a tailored telecentric f-theta objective. Our results show that this microscope is capable to perform flexible three-dimensional scanning, with low scan-induced aberrations, at a uniform resolution over a large tuning range of X=Y=6300 µ m and Z=480 µ m with only transmissive components. We demonstrate the capabilities at the example of volumetric measurements on the transgenic fluorescence of the thyroid of a zebrafish embryo and mixed pollen grains. This is the first step towards flexible aberration-free volumetric smart microscopy of three-dimensional samples like embryos and organoids, which could be exploited for the demands in both lateral and axial dimensions in biomedical samples without compromising image quality.

Multimode Optical Interconnects on Silicon Interposer Enable Confidential Hardware-to-Hardware Communication.

Following Moore's law, the density of integrated circuits is increasing in all dimensions, for instance, in 3D stacked chip networks. Amongst other electro-optic solutions, multimode optical interconnects on a silicon interposer promise to enable high throughput for modern hardware platforms in a restricted space. Such integrated architectures require confidential communication between multiple chips as a key factor for high-performance infrastructures in the 5G era and beyond. Physical layer security is an approach providing information theoretic security among network participants, exploiting the uniqueness of the data channel. We experimentally project orthogonal and non-orthogonal symbols through 380 µm long multimode on-chip interconnects by wavefront shaping. These interconnects are investigated for their uniqueness by repeating these experiments across multiple channels and samples. We show that the detected speckle patterns resulting from modal crosstalk can be recognized by training a deep neural network, which is used to transform these patterns into a corresponding readable output. The results showcase the feasibility of applying physical layer security to multimode interconnects on silicon interposers for confidential optical 3D chip networks.

Nonlinear microscopy using impulsive stimulated Brillouin scattering for high-speed elastography.

The impulsive stimulated Brillouin microscopy promises fast, non-contact measurements of the elastic properties of biological samples. The used pump-probe approach employs an ultra-short pulse laser and a cw laser to generate Brillouin signals. Modeling of the microscopy technique has already been carried out partially, but not for biomedical applications. The nonlinear relationship between pulse energy and Brillouin signal amplitude is proven with both simulations and experiments. Tayloring of the excitation parameters on the biologically relevant polyacrylamide hydrogels outline sub-ms temporal resolutions at a relative precision of <1%. Brillouin microscopy using the impulsive stimulated scattering therefore exhibits high potential for the measurements of viscoelastic properties of cells and tissues.

Surveillance of few-mode fiber-communication channels with a single hidden layer neural network.

Multi- and few-mode fibers (FMFs) promise to enhance the capacity of optical communication networks by orders of magnitude. The key for this evolution was the strong advancement of computational approaches that allowed inherent complex light transmission to be surpassed, learned, or controlled, reined in by modal crosstalk and mode-dependent losses. However, complex light transmission through FMFs can be learned by a single hidden layer neural network (NN). The emerging developments in NNs additionally allow the implementation of novel concepts for security enhancements in optical communication. Once the transmission characteristics of FMFs are learned, it is possible to survey the incoming and outgoing light fields via monitoring channels during data transmission. If an eavesdropper tries to gain unauthorized access to the FMF, its transmission properties are impaired through sensitive modal crosstalk. This process is registered by the NN and thus the eavesdropper is revealed. With our solution, the security of optical communication can be improved.

Benchmarking analysis of computer generated holograms for complex wavefront shaping using pixelated phase modulators.

Wavefront shaping with spatial light modulators (SLMs) enables aberration correction, especially for light control through complex media, like biological tissues and multimode fibres. High-fidelity light field shaping is associated with the calculation of computer generated holograms (CGHs), of which there are a variety of algorithms. The achievable performance of CGH algorithms depends on various parameters. In this paper, four different algorithms for CGHs are presented and compared for complex light field generation. Two iterative, double constraint Gerchberg-Saxton and direct search, and the two analytical, superpixel and phase encoding, algorithms are investigated. For each algorithm, a parameter study is performed varying the modulator's pixel number and phase resolution. The analysis refers to mode field generation in multimode fibre endoscopes and communication. This enables generality by generating specific mode combinations according to certain spatial frequency power spectra. Thus, the algorithms are compared varying spatial frequencies applied to different implementation scenarios. Our results demonstrate that the choice of algorithms has a significant impact on the achievable performance. This comprehensive study provides the required guide for CGH algorithm selection, improving holographic systems towards multimode fibre endoscopy and communications.

Impulsive stimulated Brillouin microscopy for non-contact, fast mechanical investigations of hydrogels.

The mechanical properties of tissues and cells are increasingly recognized as an important feature for the understanding of pathological processes and as a diagnostic tool in biomedicine. Impulsive stimulated Brillouin scattering (ISBS) is promising to overcome shortcomings of other measurement methods such as invasiveness, low spatial resolution and long acquisition time. In this paper, we present for the first time ISBS measurements of hydrogels, which are model materials for biological samples. We demonstrate ISBS measurements discriminating hydrogels of different stiffness. ISBS measurements with lateral resolution close to cellular level are presented. These results underline that ISBS microscopy has a high potential for biomedical applications.

Self-calibration of lensless holographic endoscope using programmable guide stars.

Coherent fiber bundle (CFB)-based endoscopes enable optical keyhole access in applications such as biophotonics. In conjunction with objective lenses, CFBs allow imaging of intensity patterns. In contrast, digital optical phase conjugation enables lensless holographic endoscopes for the generation of pixelation-free arbitrary light patterns. For real-world applications, however, this requires a non-invasive in situ calibration of the complex optical transfer function of the CFB with only single-sided access. We show that after an initial calibration in a forward direction, a differential phase measurement of the back-reflected light allows for tracking and compensating of bending-induced phase distortions. Furthermore, we present a novel in situ calibration procedure based on a programmable guide star, which requires access to only one side of the fiber.

Transmission of independent signals through a multimode fiber using digital optical phase conjugation.

Multimode fibers are attractive for a variety of applications such as communication engineering and biophotonics. However, a major hurdle for the optical transmission through multimode fibers is the inherent mode mixing. Although an image transmission was successfully accomplished using wavefront shaping, the image information was not transmitted individually for each of the independent pixels. We demonstrate a transmission of independent signals using individually shaped wavefronts employing a single segmented spatial light modulator for optical phase conjugation regarding each light signal. Our findings pave the way towards transferring independent signals through strongly scattering media.

Wavefront shaping for imaging-based flow velocity measurements through distortions using a Fresnel guide star.

Imaging-based flow measurement techniques, like particle image velocimetry (PIV), are vulnerable to time-varying distortions like refractive index inhomogeneities or fluctuating phase boundaries. Such distortions strongly increase the velocity error, as the position assignment of the tracer particles and the decrease of image contrast exhibit significant uncertainties. We demonstrate that wavefront shaping based on spatially distributed guide stars has the potential to significantly reduce the measurement uncertainty. Proof of concept experiments show an improvement by more than one order of magnitude. Possible applications for the wavefront shaping PIV range from measurements in jets and film flows to biomedical applications.

Volumetric HiLo microscopy employing an electrically tunable lens.

Electrically tunable lenses exhibit strong potential for fast motion-free axial scanning in a variety of microscopes. However, they also lead to a degradation of the achievable resolution because of aberrations and misalignment between illumination and detection optics that are induced by the scan itself. Additionally, the typically nonlinear relation between actuation voltage and axial displacement leads to over- or under-sampled frame acquisition in most microscopic techniques because of their static depth-of-field. To overcome these limitations, we present an Adaptive-Lens-High-and-Low-frequency (AL-HiLo) microscope that enables volumetric measurements employing an electrically tunable lens. By using speckle-patterned illumination, we ensure stability against aberrations of the electrically tunable lens. Its depth-of-field can be adjusted a-posteriori and hence enables to create flexible scans, which compensates for irregular axial measurement positions. The adaptive HiLo microscope provides an axial scanning range of 1 mm with an axial resolution of about 4 µm and sub-micron lateral resolution over the full scanning range. Proof of concept measurements at home-built specimens as well as zebrafish embryos with reporter gene-driven fluorescence in the thyroid gland are shown.

Optical dynamic deformation measurements at translucent materials.

Due to their high stiffness-to-weight ratio, glass fiber-reinforced polymers are an attractive material for rotors, e.g., in the aerospace industry. A fundamental understanding of the material behavior requires non-contact, in-situ dynamic deformation measurements. The high surface speeds and particularly the translucence of the material limit the usability of conventional optical measurement techniques. We demonstrate that the laser Doppler distance sensor provides a powerful and reliable tool for monitoring radial expansion at fast rotating translucent materials. We find that backscattering in material volume does not lead to secondary signals as surface scattering results in degradation of the measurement volume inside the translucent medium. This ensures that the acquired signal contains information of the rotor surface only, as long as the sample surface is rough enough. Dynamic deformation measurements of fast-rotating fiber-reinforced polymer composite rotors with surface speeds of more than 300 m/s underline the potential of the laser Doppler sensor.

Axial scanning in confocal microscopy employing adaptive lenses (CAL).

In this paper we analyze the capability of adaptive lenses to replace mechanical axial scanning in confocal microscopy. The adaptive approach promises to achieve high scan rates in a rather simple implementation. This may open up new applications in biomedical imaging or surface analysis in micro- and nanoelectronics, where currently the axial scan rates and the flexibility at the scan process are the limiting factors. The results show that fast and adaptive axial scanning is possible using electrically tunable lenses but the performance degrades during the scan. This is due to defocus and spherical aberrations introduced to the system by tuning of the adaptive lens. These detune the observation plane away from the best focus which strongly deteriorates the axial resolution by a factor of ~2.4. Introducing balancing aberrations allows addressing these influences. The presented approach is based on the employment of a second adaptive lens, located in the detection path. It enables shifting the observation plane back to the best focus position and thus creating axial scans with homogeneous axial resolution. We present simulated and experimental proof-of-principle results.

On the speckle number of interferometric velocity and distance measurements of moving rough surfaces.

The minimum achievable systematic uncertainty of interferometric measurements is fundamentally limited due to speckle noise. Numerical and physical experiments, regarding the achievable measurement uncertainty of Mach-Zehnder based velocity and position sensors, are presented at the example of the laser Doppler distance sensor with phase evaluation. The results show that the measurement uncertainty depends on the number of speckles on the photo detectors. However, while the systematic uncertainty due to the speckle effect decreases, the random uncertainty due to noise from the photo detector increases with increasing speckle number. This results in a minimal total measurement uncertainty for an optimal speckle number on the photo detector, which is achieved by adjusting the aperture of the detection optics.

Multiwavelength phase unwrapping and aberration correction using depth filtered digital holography.

In this Letter, we present a new approach to processing data from a standard spectral domain optical coherence tomography (OCT) system using depth filtered digital holography (DFDH). Intensity-based OCT processing has an axial resolution of the order of a few micrometers. When the phase information is used to obtain optical path length differences, subwavelength accuracy can be achieved, but this limits the resolvable step heights to half of the wavelength of the system. Thus there is a metrology gap between phase- and intensity-based methods. Our concept addresses this metrology gap by combining DFHD with multiwavelength phase unwrapping. Additionally, the measurements are corrected for aberrations. Here, we present proof of concept measurements of a structured semiconductor sample.

AI-driven projection tomography with multicore fibre-optic cell rotation.

Optical tomography has emerged as a non-invasive imaging method, providing three-dimensional insights into subcellular structures and thereby enabling a deeper understanding of cellular functions, interactions, and processes. Conventional optical tomography methods are constrained by a limited illumination scanning range, leading to anisotropic resolution and incomplete imaging of cellular structures. To overcome this problem, we employ a compact multi-core fibre-optic cell rotator system that facilitates precise optical manipulation of cells within a microfluidic chip, achieving full-angle projection tomography with isotropic resolution. Moreover, we demonstrate an AI-driven tomographic reconstruction workflow, which can be a paradigm shift from conventional computational methods, often demanding manual processing, to a fully autonomous process. The performance of the proposed cell rotation tomography approach is validated through the three-dimensional reconstruction of cell phantoms and HL60 human cancer cells. The versatility of this learning-based tomographic reconstruction workflow paves the way for its broad application across diverse tomographic imaging modalities, including but not limited to flow cytometry tomography and acoustic rotation tomography. Therefore, this AI-driven approach can propel advancements in cell biology, aiding in the inception of pioneering therapeutics, and augmenting early-stage cancer diagnostics.

Securing Data in Multimode Fibers by Exploiting Mode-Dependent Light Propagation Effects.

Multimode fibers hold great promise to advance data rates in optical communications but come with the challenge to compensate for modal crosstalk and mode-dependent losses, resulting in strong distortions. The holographic measurement of the transmission matrix enables not only correcting distortions but also harnessing these effects for creating a confidential data connection between legitimate communication parties, Alice and Bob. The feasibility of this physical-layer-security-based approach is demonstrated experimentally for the first time on a multimode fiber link to which the eavesdropper Eve is physically coupled. Once the proper structured light field is launched at Alice's side, the message can be delivered to Bob, and, simultaneously, the decipherment for an illegitimate wiretapper Eve is destroyed. Within a real communication scenario, we implement wiretap codes and demonstrate confidentiality by quantifying the level of secrecy. Compared to an uncoded data transmission, the amount of securely exchanged data is enhanced by a factor of 538. The complex light transportation phenomena that have long been considered limiting and have restricted the widespread use of multimode fiber are exploited for opening new perspectives on information security in spatial multiplexing communication systems.

Depth-filtered digital holography.

We introduce depth-filtered digital holography (DFDH) as a method for quantitative tomographic phase imaging of buried layers in multilayer samples. The procedure is based on the acquisition of multiple holograms for different wavelengths. Analyzing the intensity over wavelength pixel wise and using an inverse Fourier transform leads to a depth-profile of the multilayered sample. Applying a windowed Fourier transform with a narrow window, we choose a depth-of interest (DOI) which is used to synthesize filtered interference patterns that just contain information of this limited depth. We use the angular spectrum method to introduce an additional spatial filtering and to reconstruct the corresponding holograms. After a short theoretical framework we show experimental proof-of-principle results for the method.

Real-time complex light field generation through a multi-core fiber with deep learning.

The generation of tailored complex light fields with multi-core fiber (MCF) lensless microendoscopes is widely used in biomedicine. However, the computer-generated holograms (CGHs) used for such applications are typically generated by iterative algorithms, which demand high computation effort, limiting advanced applications like fiber-optic cell manipulation. The random and discrete distribution of the fiber cores in an MCF induces strong spatial aliasing to the CGHs, hence, an approach that can rapidly generate tailored CGHs for MCFs is highly demanded. We demonstrate a novel deep neural network-CoreNet, providing accurate tailored CGHs generation for MCFs at a near video rate. The CoreNet is trained by unsupervised learning and speeds up the computation time by two magnitudes with high fidelity light field generation compared to the previously reported CGH algorithms for MCFs. Real-time generated tailored CGHs are on-the-fly loaded to the phase-only spatial light modulator (SLM) for near video-rate complex light fields generation through the MCF microendoscope. This paves the avenue for real-time cell rotation and several further applications that require real-time high-fidelity light delivery in biomedicine.