High-speed object detection with a single-photon time-of-flight image sensor.

3D time-of-flight (ToF) imaging is used in a variety of applications such as augmented reality (AR), computer interfaces, robotics and autonomous systems. Single-photon avalanche diodes (SPADs) are one of the enabling technologies providing accurate depth data even over long ranges. By developing SPADs in array format with integrated processing combined with pulsed, flood-type illumination, high-speed 3D capture is possible. However, array sizes tend to be relatively small, limiting the lateral resolution of the resulting depth maps and, consequently, the information that can be extracted from the image for applications such as object detection. In this paper, we demonstrate that these limitations can be overcome through the use of convolutional neural networks (CNNs) for high-performance object detection. We present outdoor results from a portable SPAD camera system that outputs 16-bin photon timing histograms with 64×32 spatial resolution, with each histogram containing thousands of photons. The results, obtained with exposure times down to 2 ms (equivalent to 500 FPS) and in signal-to-background (SBR) ratios as low as 0.05, point to the advantages of providing the CNN with full histogram data rather than point clouds alone. Alternatively, a combination of point cloud and active intensity data may be used as input, for a similar level of performance. In either case, the GPU-accelerated processing time is less than 1 ms per frame, leading to an overall latency (image acquisition plus processing) in the millisecond range, making the results relevant for safety-critical computer vision applications which would benefit from faster than human reaction times.

Multifocal imaging for precise, label-free tracking of fast biological processes in 3D.

Many biological processes happen on a nano- to millimeter scale and within milliseconds. Established methods such as confocal microscopy are suitable for precise 3D recordings but lack the temporal or spatial resolution to resolve fast 3D processes and require labeled samples. Multifocal imaging (MFI) allows high-speed 3D imaging but is limited by the compromise between high spatial resolution and large field-of-view (FOV), and the requirement for bright fluorescent labels. Here, we provide an open-source 3D reconstruction algorithm for multi-focal images that allows using MFI for fast, precise, label-free tracking spherical and filamentous structures in a large FOV and across a high depth. We characterize fluid flow and flagellar beating of human and sea urchin sperm with a z-precision of 0.15 µm, in a volume of 240 × 260 × 21 µm, and at high speed (500 Hz). The sampling volume allowed to follow sperm trajectories while simultaneously recording their flagellar beat. Our MFI concept is cost-effective, can be easily implemented, and does not rely on object labeling, which renders it broadly applicable.

3D Imaging from Multipath Temporal Echoes.

Echo location is a broad approach to imaging and sensing that includes both manmade RADAR, LIDAR, SONAR, and also animal navigation. However, full 3D information based on echo location requires some form of scanning of the scene in order to provide the spatial location of the echo origin-points. Without this spatial information, imaging objects in three-dimensional (3D) is a very challenging task as the inverse retrieval problem is strongly ill-posed. Here, we show that the temporal information encoded in the return echoes that are reflected multiple times within a scene is sufficient to faithfully render an image in 3D. Numerical modeling and an information theoretic perspective prove the concept and provide insight into the role of the multipath information. We experimentally demonstrate the concept by using both radio frequency and acoustic waves for imaging individuals moving in a closed environment.

Fluorescence lifetime imaging with a megapixel SPAD camera and neural network lifetime estimation.

Fluorescence lifetime imaging microscopy (FLIM) is a key technology that provides direct insight into cell metabolism, cell dynamics and protein activity. However, determining the lifetimes of different fluorescent proteins requires the detection of a relatively large number of photons, hence slowing down total acquisition times. Moreover, there are many cases, for example in studies of cell collectives, where wide-field imaging is desired. We report scan-less wide-field FLIM based on a 0.5 MP resolution, time-gated Single Photon Avalanche Diode (SPAD) camera, with acquisition rates up to 1 Hz. Fluorescence lifetime estimation is performed via a pre-trained artificial neural network with 1000-fold improvement in processing times compared to standard least squares fitting techniques. We utilised our system to image HT1080-human fibrosarcoma cell line as well as Convallaria. The results show promise for real-time FLIM and a viable route towards multi-megapixel fluorescence lifetime images, with a proof-of-principle mosaic image shown with 3.6 MP.

Light scattering control in transmission and reflection with neural networks.

Scattering often limits the controlled delivery of light in applications such as biomedical imaging, optogenetics, optical trapping, and fiber-optic communication or imaging. Such scattering can be controlled by appropriately shaping the light wavefront entering the material. Here, we develop a machine-learning approach for light control. Using pairs of binary intensity patterns and intensity measurements we train neural networks (NNs) to provide the wavefront corrections necessary to shape the beam after the scatterer. Additionally, we demonstrate that NNs can be used to find a functional relationship between transmitted and reflected speckle patterns. Establishing the validity of this relationship, we focus and scan in transmission through opaque media using reflected light. Our approach shows the versatility of NNs for light shaping, for efficiently and flexibly correcting for scattering, and in particular the feasibility of transmission control based on reflected light.

Generation of reconfigurable optical traps for microparticles spatial manipulation through dynamic split lens inspired light structures.

We present an experimental method, based on the use of dynamic split-lens configurations, useful for the trapping and spatial control of microparticles through the photophoretic force. In particular, the concept of split-lens configurations is exploited to experimentally create customized and reconfigurable three-dimensional light structures, in which carbon coated glass microspheres, with sizes in a range of 63-75 µm, can be captured. The generation of light spatial structures is performed by properly addressing phase distributions corresponding to different split-lens configurations onto a spatial light modulator (SLM). The use of an SLM allows a dynamic variation of the light structures geometry just by modifying few control parameters of easy physical interpretation. We provide some examples in video format of particle trapping processes. What is more, we also perform further spatial manipulation, by controlling the spatial position of the particles in the axial direction, demonstrating the generation of reconfigurable three-dimensional photophoretic traps for microscopic manipulation of absorbing particles.

Extreme Ultraviolet Fractional Orbital Angular Momentum Beams from High Harmonic Generation.

We investigate theoretically the generation of extreme-ultraviolet (EUV) beams carrying fractional orbital angular momentum. To this end, we drive high-order harmonic generation with infrared conical refraction (CR) beams. We show that the high-order harmonic beams emitted in the EUV/soft x-ray regime preserve the characteristic signatures of the driving beam, namely ringlike transverse intensity profile and CR-like polarization distribution. As a result, through orbital and spin angular momentum conservation, harmonic beams are emitted with fractional orbital angular momentum, and they can be synthesized into structured attosecond helical beams -or "structured attosecond light springs"- with rotating linear polarization along the azimuth. Our proposal overcomes the state of the art limitations for the generation of light beams far from the visible domain carrying non-integer orbital angular momentum and could be applied in fields such as diffraction imaging, EUV lithography, particle trapping, and super-resolution imaging.

Complete snapshot Stokes polarimeter based on a single biaxial crystal.

We propose a new complete snapshot Stokes polarimeter based on a single biaxial crystal. It presents different strengths (snapshot, complete polarimetric measurements, large data redundancy, and high sensitivity) in a simple and compact optical arrangement. The polarimeter is experimentally implemented and analyzed in terms of accuracy and repeatability.

Interferometric characterization of the structured polarized light beam produced by the conical refraction phenomenon.

The interest on the conical refraction (CR) phenomenon in biaxial crystals has revived in the last years due to its prospective for generating structured polarized light beams, i.e. vector beams. While the intensity and the polarization structure of the CR beams are well known, an accurate experimental study of their phase structure has not been yet carried out. We investigate the phase structure of the CR rings by means of a Mach-Zehnder interferometer while applying the phase-shifting interferometric technique to measure the phase at the focal plane. In general the two beams interfering correspond to different states of polarization (SOP) which locally vary. To distinguish if there is an additional phase added to the geometrical one we have derived the appropriate theoretical expressions using the Jones matrix formalism. We demonstrate that the phase of the CR rings is equivalent to that one introduced by an azimuthally segmented polarizer with CR-like polarization distribution. Additionally, we obtain direct evidence that the Poggendorff dark ring is an annular singularity, with a π phase change between the inner and outer bright rings.

Conical refraction as a tool for polarization metrology.

A method for polarization metrology based on the conical refraction (CR) phenomenon, occurring in biaxial crystals, is reported. CR transforms an input Gaussian beam into a light ring whose intensity distribution is linked to the incoming polarization. We present the design of a division-of-amplitude complete polarimeter composed of two biaxial crystals, whose measurement principle is based on the CR phenomenon. This design corresponds to a static polarimeter, that is, without mechanical movements or electrical signal addressing. Only one division-of-amplitude device is required, besides the two biaxial crystals, to completely characterize any state of polarization, including partially polarized and unpolarized states. In addition, a mathematical model describing the system is included. Experimental images of the intensity distribution related to different input polarization states are provided. These intensity patterns are compared with simulated values, proving the potential of polarimeters based on biaxial crystals.

Type I and type II second harmonic generation of conically refracted beams.

Type I and type II second harmonic generation (SHG) of a beam transformed by the conical refraction phenomenon are presented. We show that, for type I, the second harmonic intensity pattern is a light ring with a point of null intensity while, for type II, the light ring possesses two dark regions. Taking into account the different two-photon processes involved in SHG, we have derived analytical expressions for the resulting transverse intensity patterns that are in good agreement with the experimental data. Finally, we have investigated the spatial evolution of the second harmonic signals, showing that they behave as conically refracted beams.

Multiple rings formation in cascaded conical refraction.

When a light beam passes through a cascade of biaxial crystals (BCs) with aligned optic axes, the resulting transverse intensity pattern consists of multiple concentric rings. We provide a simple formulation for the pattern formation for both circularly and linearly polarized input beams, that could be applied for a cascade of an arbitrary number of BCs. We have experimentally investigated multiple ring formation with up to three cascade BCs, showing that the theoretical formulation is in full agreement with the experimental results.

Free-space optical polarization demultiplexing and multiplexing by means of conical refraction.

Polarization demultiplexing and multiplexing by means of conical refraction is proposed to increase the channel capacity for free-space optical communication applications. The proposed technique is based on the forward-backward optical transform occurring when a light beam propagates consecutively along the optic axes of two identical biaxial crystals with opposite orientations of their conical refraction characteristic vectors. We present an experimental proof of usefulness of the conical refraction demultiplexing and multiplexing technique by increasing in one order of magnitude the channel capacity at optical frequencies in a propagation distance of 4 m.