Computational optical sensing and imaging: introduction to the feature issue.

This joint feature issue of Optics Express and Applied Optics showcases technical innovations by participants of the 2023 topical meeting on Computational Optical Sensing and Imaging and the computational imaging community. The articles included in the feature issue highlight advances in imaging science that emphasize synergistic activities in optics, signal processing and machine learning. The issue features 26 contributed articles that cover multiple themes including non line-of-sight imaging, imaging through scattering media, compressed sensing, lensless imaging, ptychography, computational microscopy, spectroscopy and optical metrology.

Removal of algorithmic stagnation by augmented iterative phase retrieval.

Retrieving the phase of an optical field using intensity measurements is one of the most widespread and studied inverse problems in classical optics. However, common iterative approaches such as the Gerchberg-Saxton algorithm and its derivatives suffer from the twin-image problem - the iterative minimisation stagnates and the recovered field contains features from both the target field and its point-reflection. We present a technique that leverages mathematical properties of the stagnated field, to constrain the problem and remove the twin image artefacts. This improvement in reconstruction robustness has implications in a range of fields, including applications in adaptive optics, holography and optical communications.

Single-sample image-fusion upsampling of fluorescence lifetime images.

Fluorescence lifetime imaging microscopy (FLIM) provides detailed information about molecular interactions and biological processes. A major bottleneck for FLIM is image resolution at high acquisition speeds due to the engineering and signal-processing limitations of time-resolved imaging technology. Here, we present single-sample image-fusion upsampling, a data-fusion approach to computational FLIM super-resolution that combines measurements from a low-resolution time-resolved detector (that measures photon arrival time) and a high-resolution camera (that measures intensity only). To solve this otherwise ill-posed inverse retrieval problem, we introduce statistically informed priors that encode local and global correlations between the two "single-sample" measurements. This bypasses the risk of out-of-distribution hallucination as in traditional data-driven approaches and delivers enhanced images compared, for example, to standard bilinear interpolation. The general approach laid out by single-sample image-fusion upsampling can be applied to other image super-resolution problems where two different datasets are available.

Adaptive optical imaging with entangled photons.

Adaptive optics (AO) has revolutionized imaging in fields from astronomy to microscopy by correcting optical aberrations. In label-free microscopes, however, conventional AO faces limitations because of the absence of a guide star and the need to select an optimization metric specific to the sample and imaging process. Here, we propose an AO approach leveraging correlations between entangled photons to directly correct the point spread function. This guide star-free method is independent of the specimen and imaging modality. We demonstrate the imaging of biological samples in the presence of aberrations using a bright-field imaging setup operating with a source of spatially entangled photon pairs. Our approach performs better than conventional AO in correcting specific aberrations, particularly those involving substantial defocus. Our work improves AO for label-free microscopy and could play a major role in the development of quantum microscopes.

A comprehensive multimodal dataset for contactless lip reading and acoustic analysis.

Small-scale motion detection using non-invasive remote sensing techniques has recently garnered significant interest in the field of speech recognition. Our dataset paper aims to facilitate the enhancement and restoration of speech information from diverse data sources for speakers. In this paper, we introduce a novel multimodal dataset based on Radio Frequency, visual, text, audio, laser and lip landmark information, also called RVTALL. Specifically, the dataset consists of 7.5 GHz Channel Impulse Response (CIR) data from ultra-wideband (UWB) radars, 77 GHz frequency modulated continuous wave (FMCW) data from millimeter wave (mmWave) radar, visual and audio information, lip landmarks and laser data, offering a unique multimodal approach to speech recognition research. Meanwhile, a depth camera is adopted to record the landmarks of the subject's lip and voice. Approximately 400 minutes of annotated speech profiles are provided, which are collected from 20 participants speaking 5 vowels, 15 words, and 16 sentences. The dataset has been validated and has potential for the investigation of lip reading and multimodal speech recognition.

Fluorescence lifetime Hong-Ou-Mandel sensing.

Fluorescence Lifetime Imaging Microscopy in the time domain is typically performed by recording the arrival time of photons either by using electronic time tagging or a gated detector. As such the temporal resolution is limited by the performance of the electronics to 100's of picoseconds. Here, we demonstrate a fluorescence lifetime measurement technique based on photon-bunching statistics with a resolution that is only dependent on the duration of the reference photon or laser pulse, which can readily reach the 1-0.1 picosecond timescale. A range of fluorescent dyes having lifetimes spanning from 1.6 to 7 picoseconds have been here measured with only ~1 s measurement duration. We corroborate the effectiveness of the technique by measuring the Newtonian viscosity of glycerol/water mixtures by means of a molecular rotor having over an order of magnitude variability in lifetime, thus introducing a new method for contact-free nanorheology. Accessing fluorescence lifetime information at such high temporal resolution opens a doorway for a wide range of fluorescent markers to be adopted for studying yet unexplored fast biological processes, as well as fundamental interactions such as lifetime shortening in resonant plasmonic devices.

Single-shot time-folded fluorescence lifetime imaging.

Fluorescence lifetime imaging is an important tool in bioimaging that allows one to detect subtle changes in cell dynamics and their environment. Most time-domain approaches currently involve scanning a single illumination point across the sample, which can make imaging dynamic scenes challenging, while single-shot "rapid lifetime determination" can suffer from large uncertainties when the lifetime is not appropriately sampled. Here, we propose a time-folded fluorescence lifetime imaging microscopy (TFFLIM) approach, whereby a time-folding cavity provides multiple spatially sheared replicas of the lifetime, each shifted temporally with respect to a fixed time gate. This provides a robust, single-shot FLIM approach that we experimentally validate across a broad lifetime range on fluorescent beads and Convallaria samples.

Electro-Optical Sampling of Single-Cycle THz Fields with Single-Photon Detectors.

Electro-optical sampling of Terahertz fields with ultrashort pulsed probes is a well-established approach for directly measuring the electric field of THz radiation. This technique usually relies on balanced detection to record the optical phase shift brought by THz-induced birefringence. The sensitivity of electro-optical sampling is, therefore, limited by the shot noise of the probe pulse, and improvements could be achieved using quantum metrology approaches using, e.g., NOON states for Heisenberg-limited phase estimation. We report on our experiments on THz electro-optical sampling using single-photon detectors and a weak squeezed vacuum field as the optical probe. Our approach achieves field sensitivity limited by the probe state statistical properties using phase-locked single-photon detectors and paves the way for further studies targeting quantum-enhanced THz sensing.

Real-Time Scene Monitoring for Deaf-Blind People.

It is estimated that at least 15 million people worldwide live with severe deaf-blindness, with many more experiencing varying degrees of deaf-blindness. The existing options of assistance are mostly limited to walking canes, guide dogs and human care. We propose a wearable device which harnesses a multi-antenna mmWave radar transceiver and a haptic feedback array for real time detection of a person moving within a scene. We present our findings from a series of workshops with participants classed with multi-sensory impairments (MSI), to demonstrate the relative success of this approach and its potential for integration into existing assistance for the MSI of the future.

An All-Dielectric Metasurface Polarimeter.

The polarization state of light is a key parameter in many imaging systems. For example, it can image mechanical stress and other physical properties that are not seen with conventional imaging and can also play a central role in quantum sensing. However, polarization is more difficult to image, and polarimetry typically involves several independent measurements with moving parts in the measurement device. Metasurfaces with interleaved designs have demonstrated sensitivity to either linear or circular/elliptical polarization states. Here, we present an all-dielectric meta-polarimeter for direct measurement of any arbitrary polarization state from a single-unit-cell design. By engineering a completely asymmetric design, we obtained a metasurface that can excite eigenmodes of the nanoresonators, thus displaying a unique diffraction pattern for not only any linear polarization state but all elliptical polarization states (and handedness) as well. The unique diffraction patterns are quantified into Stokes parameters with a resolution of 5° and with a polarization state fidelity of up to 99 ± 1%. This holds promise for applications in polarization imaging and quantum state tomography.

Remote laser-speckle sensing of heart sounds for health assessment and biometric identification.

Assessment of heart sounds is the cornerstone of cardiac examination, but it requires a stethoscope, skills and experience, and a direct contact with the patient. We developed a contactless, machine-learning assisted method for heart-sound identification and quantification based on the remote measurement of the reflected laser speckle from the neck skin surface in healthy individuals. We compare the performance of this method to standard digital stethoscope recordings on an example task of heart-beat sound biometric identification. We show that our method outperforms the stethoscope even allowing identification on the test data taken on different days. This method might allow development of devices for remote monitoring of cardiovascular health in different settings.

To make a mirrorless laser.

Periodic temporal modulation of a photonic crystal can be used to produce laser light.

Pixel super-resolution with spatially entangled photons.

Pixelation occurs in many imaging systems and limits the spatial resolution of the acquired images. This effect is notably present in quantum imaging experiments with correlated photons in which the number of pixels used to detect coincidences is often limited by the sensor technology or the acquisition speed. Here, we introduce a pixel super-resolution technique based on measuring the full spatially-resolved joint probability distribution (JPD) of spatially-entangled photons. Without shifting optical elements or using prior information, our technique increases the pixel resolution of the imaging system by a factor two and enables retrieval of spatial information lost due to undersampling. We demonstrate its use in various quantum imaging protocols using photon pairs, including quantum illumination, entanglement-enabled quantum holography, and in a full-field version of N00N-state quantum holography. The JPD pixel super-resolution technique can benefit any full-field imaging system limited by the sensor spatial resolution, including all already established and future photon-correlation-based quantum imaging schemes, bringing these techniques closer to real-world applications.

Light detection and ranging with entangled photons.

Single-photon light detection and ranging (LiDAR) is a key technology for depth imaging through complex environments. Despite recent advances, an open challenge is the ability to isolate the LiDAR signal from other spurious sources including background light and jamming signals. Here we show that a time-resolved coincidence scheme can address these challenges by exploiting spatio-temporal correlations between entangled photon pairs. We demonstrate that a photon-pair-based LiDAR can distill desired depth information in the presence of both synchronous and asynchronous spurious signals without prior knowledge of the scene and the target object. This result enables the development of robust and secure quantum LiDAR systems and paves the way to time-resolved quantum imaging applications.

Measurement of Penrose Superradiance in a Photon Superfluid.

The superradiant amplification in the scattering from a rotating medium was first elucidated by Sir Roger Penrose over 50 years ago as a means by which particles could gain energy from rotating black holes. Despite this fundamental process being ubiquitous also in wave physics, it has only been observed once experimentally, in a water tank. Here, we measure this amplification for a nonlinear optics experiment in the superfluid regime. In particular, by focusing a weak optical beam carrying orbital angular momentum onto the core of a strong pump vortex beam, negative norm modes are generated and trapped inside the vortex core, allowing for amplification of a reflected beam. Our experiment demonstrates amplified reflection due to a novel form of nonlinear optical four-wave mixing, whose phase-relation coincides with the Zel'dovich-Misner condition for Penrose superradiance in our photon superfluid, and unveil the role played by negative frequency modes in the process.

Generation of Entanglement from Mechanical Rotation.

Many phenomena and fundamental predictions, ranging from Hawking radiation to the early evolution of the Universe rely on the interplay between quantum mechanics and gravity or more generally, quantum mechanics in curved spacetimes. However, our understanding is hindered by the lack of experiments that actually allow us to probe quantum mechanics in curved spacetime in a repeatable and accessible way. Here we propose an experimental scheme for a photon that is prepared in a path superposition state across two rotating Sagnac interferometers that have different diameters and thus represent a superposition of two different spacetimes. We predict the generation of genuine entanglement even at low rotation frequencies and show how these effects could be observed even due to the Earth's rotation. These predictions provide an accessible platform in which to study the role of the underlying spacetime in the generation of entanglement.

Visible photon generation via four-wave mixing in near-infrared near-zero-index thin films.

Optical nonlinearities can be strongly enhanced by operating in the so-called near-zero-index (NZI) regime, where the real part of the refractive index of the system under investigation approaches zero. Here we experimentally demonstrate semi-degenerate four-wave mixing (FWM) in aluminum zinc oxide thin films generating radiation tunable in the visible spectral region, where the material is highly transparent. To this end, we employed an intense pump (787 nm) and a seed tunable in the NIR window (1100-1500 nm) to generate a visible idler wave (530-620 nm). Experiments show enhancement of the frequency conversion efficiency with a maximum of 2% and a signal-to-pump detuning of 360 nm. Effective idler wavelength tuning has also been demonstrated by operating on the temporal delay between the pump and signal.

3D Imaging from Multipath Temporal Echoes.

Echo location is a broad approach to imaging and sensing that includes both manmade RADAR, LIDAR, SONAR, and also animal navigation. However, full 3D information based on echo location requires some form of scanning of the scene in order to provide the spatial location of the echo origin-points. Without this spatial information, imaging objects in three-dimensional (3D) is a very challenging task as the inverse retrieval problem is strongly ill-posed. Here, we show that the temporal information encoded in the return echoes that are reflected multiple times within a scene is sufficient to faithfully render an image in 3D. Numerical modeling and an information theoretic perspective prove the concept and provide insight into the role of the multipath information. We experimentally demonstrate the concept by using both radio frequency and acoustic waves for imaging individuals moving in a closed environment.

Fluorescence lifetime imaging with a megapixel SPAD camera and neural network lifetime estimation.

Fluorescence lifetime imaging microscopy (FLIM) is a key technology that provides direct insight into cell metabolism, cell dynamics and protein activity. However, determining the lifetimes of different fluorescent proteins requires the detection of a relatively large number of photons, hence slowing down total acquisition times. Moreover, there are many cases, for example in studies of cell collectives, where wide-field imaging is desired. We report scan-less wide-field FLIM based on a 0.5 MP resolution, time-gated Single Photon Avalanche Diode (SPAD) camera, with acquisition rates up to 1 Hz. Fluorescence lifetime estimation is performed via a pre-trained artificial neural network with 1000-fold improvement in processing times compared to standard least squares fitting techniques. We utilised our system to image HT1080-human fibrosarcoma cell line as well as Convallaria. The results show promise for real-time FLIM and a viable route towards multi-megapixel fluorescence lifetime images, with a proof-of-principle mosaic image shown with 3.6 MP.

Penrose Superradiance in Nonlinear Optics.

Particles or waves scattered from a rotating black hole can be amplified through the process of Penrose superradiance, although this cannot currently be observed in an astrophysical setting. Here we theoretically show that analog Penrose superradiance arises naturally in the field of nonlinear optics. A loosely focused signal beam can experience gain or amplification as it glances off a strong vortex pump beam in a nonlinear defocusing medium. Amplification occurs only with the generation and trapping of negative norm modes in the core of the pump vortex, as predicted by Penrose. Our results elucidate a new regime of nonlinear optics involving the notion of an ergoregion, providing further insight into the processes and transient dynamics involved in Penrose superradiance.