Human activity recognition using a single-photon direct time-of-flight sensor.

Single-Photon Avalanche Diode (SPAD) direct Time-of-Flight (dToF) sensors provide depth imaging over long distances, enabling the detection of objects even in the absence of contrast in colour or texture. However, distant objects are represented by just a few pixels and are subject to noise from solar interference, limiting the applicability of existing computer vision techniques for high-level scene interpretation. We present a new SPAD-based vision system for human activity recognition, based on convolutional and recurrent neural networks, which is trained entirely on synthetic data. In tests using real data from a 64×32 pixel SPAD, captured over a distance of 40 m, the scheme successfully overcomes the limited transverse resolution (in which human limbs are approximately one pixel across), achieving an average accuracy of 89% in distinguishing between seven different activities. The approach analyses continuous streams of video-rate depth data at a maximal rate of 66 FPS when executed on a GPU, making it well-suited for real-time applications such as surveillance or situational awareness in autonomous systems.

Video super-resolution for single-photon LIDAR.

3D time-of-flight (ToF) image sensors are used widely in applications such as self-driving cars, augmented reality (AR), and robotics. When implemented with single-photon avalanche diodes (SPADs), compact, array format sensors can be made that offer accurate depth maps over long distances, without the need for mechanical scanning. However, array sizes tend to be small, leading to low lateral resolution, which combined with low signal-to-background ratio (SBR) levels under high ambient illumination, may lead to difficulties in scene interpretation. In this paper, we use synthetic depth sequences to train a 3D convolutional neural network (CNN) for denoising and upscaling (×4) depth data. Experimental results, based on synthetic as well as real ToF data, are used to demonstrate the effectiveness of the scheme. With GPU acceleration, frames are processed at >30 frames per second, making the approach suitable for low-latency imaging, as required for obstacle avoidance.

Fundamental limits to depth imaging with single-photon detector array sensors.

Single-Photon Avalanche Detector (SPAD) arrays are a rapidly emerging technology. These multi-pixel sensors have single-photon sensitivities and pico-second temporal resolutions thus they can rapidly generate depth images with millimeter precision. Such sensors are a key enabling technology for future autonomous systems as they provide guidance and situational awareness. However, to fully exploit the capabilities of SPAD array sensors, it is crucial to establish the quality of depth images they are able to generate in a wide range of scenarios. Given a particular optical system and a finite image acquisition time, what is the best-case depth resolution and what are realistic images generated by SPAD arrays? In this work, we establish a robust yet simple numerical procedure that rapidly establishes the fundamental limits to depth imaging with SPAD arrays under real world conditions. Our approach accurately generates realistic depth images in a wide range of scenarios, allowing the performance of an optical depth imaging system to be established without the need for costly and laborious field testing. This procedure has applications in object detection and tracking for autonomous systems and could be easily extended to systems for underwater imaging or for imaging around corners.

Pixels2Pose: Super-resolution time-of-flight imaging for 3D pose estimation.

Single-photon-sensitive depth sensors are being increasingly used in next-generation electronics for human pose and gesture recognition. However, cost-effective sensors typically have a low spatial resolution, restricting their use to basic motion identification and simple object detection. Here, we perform a temporal to spatial mapping that drastically increases the resolution of a simple time-of-flight sensor, i.e., an initial resolution of 4 × 4 pixels to depth images of resolution 32 × 32 pixels. The output depth maps can then be used for accurate three-dimensional human pose estimation of multiple people. We develop a new explainable framework that provides intuition to how our network uses its input data and provides key information about the relevant parameters. Our work greatly expands the use cases of simple single-photon avalanche detector time-of-flight sensors and opens up promising possibilities for future super-resolution techniques applied to other types of sensors with similar data types, i.e., radar and sonar.

General design principle for structured light lasers.

Using custom laser cavities to produce as the output some desired structured light field has seen tremendous advances lately, but there is no universal approach to designing such cavities for arbitrarily defined field structures within the cavity, e.g., at both the output and gain ends. Here we outline a general design approach for structured light from lasers which allows us to specify the required cavity for any selected structured light fields at both ends. We verify the approach by numerical simulation as well as by an unwrapped cavity experiment. The power of this approach is that the cavity can be designed to maximise the overlap with the available pump for higher powers, minimise thermal effects for higher brightness, and at the same time output a desired structured light field that may differ substantially from the gain-end profile. These benefits make this work appeal to the large laser communities interested in cavities for high brightness and/or customized output beams.

Lossless reshaping of structured light.

Structured light concerns the control of light in its spatial degrees of freedom (amplitude, phase, and polarization), and has proven instrumental in many applications. The creation of structured light usually involves the conversion of a Gaussian mode to a desired structure in a single step, while the detection is often the reverse process, both fundamentally lossy or imperfect. Here we show how to ideally reshape structured light in a lossless manner in a simple two-step process using conformal mapping. We outline the core theoretical arguments, and experimentally demonstrate reshaping of arbitrary structured light patterns with correlations in excess of 90%. Further, we highlight when the technique is applicable and when not, and how best to implement it. This work will be a useful addition to the structured light toolkit, and particularly relevant to those wishing to use the spatial modes of light as a basis in classical and quantum communication.

Improving the beam quality factor (M2) by phase-only reshaping of structured light.

Laser brightness is crucial in many optical processes, and is optimized by high power, high beam quality (low M2) beams. Here we show how to improve the laser beam quality factor (reducing the M2) of arbitrary structured light fields in a lossless manner using continuous phase-only elements, thus allowing for the increase in brightness by a simple linear optical transformation. We demonstrate the principle with four high M2 initial beams, converting each to a Gaussian (M2≈1) with a dramatic increase in brightness of >10×. This work puts a new (to the best of our knowledge) perspective on the old debate of improving laser beam quality with binary diffractive optics, while providing a practical approach to enhancing laser brightness for arbitrary input beams.

A deterministic detector for vector vortex states.

Encoding information in high-dimensional degrees of freedom of photons has led to new avenues in various quantum protocols such as communication and information processing. Yet to fully benefit from the increase in dimension requires a deterministic detection system, e.g., to reduce dimension dependent photon loss in quantum key distribution. Recently, there has been a growing interest in using vector vortex modes, spatial modes of light with entangled degrees of freedom, as a basis for encoding information. However, there is at present no method to detect these non-separable states in a deterministic manner, negating the benefit of the larger state space. Here we present a method to deterministically detect single photon states in a four dimensional space spanned by vector vortex modes with entangled polarisation and orbital angular momentum degrees of freedom. We demonstrate our detection system with vector vortex modes from the |[Formula: see text]| = 1 and |[Formula: see text]| = 10 subspaces using classical and weak coherent states and find excellent detection fidelities for both pure and superposition vector states. This work opens the possibility to increase the dimensionality of the state-space used for encoding information while maintaining deterministic detection and will be invaluable for long distance classical and quantum communication.