Efficient non-line-of-sight tracking with computational neuromorphic imaging.

Non-line-of-sight (NLOS) sensing is an emerging technique that is capable of detecting objects hidden behind a wall, around corners, or behind other obstacles. However, NLOS tracking of moving objects is challenging due to signal redundancy and background interference. Here, we demonstrate computational neuromorphic imaging with an event camera for NLOS tracking, unaffected by the relay surface, which can efficiently obtain non-redundant information. We show how this sensor, which responds to changes in luminance within dynamic speckle fields, allows us to capture the most relevant events for direct motion estimation. The experimental results confirm that our method has superior performance in terms of efficiency, and accuracy, which greatly benefits from focusing on well-defined NLOS object tracking.

Lens-free motion analysis via neuromorphic laser speckle imaging.

Laser speckle imaging (LSI) is a powerful tool for motion analysis owing to the high sensitivity of laser speckles. Traditional LSI techniques rely on identifying changes from the sequential intensity speckle patterns, where each pixel performs synchronous measurements. However, a lot of redundant data of the static speckles without motion information in the scene will also be recorded, resulting in considerable resources consumption for data processing and storage. Moreover, the motion cues are inevitably lost during the "blind" time interval between successive frames. To tackle such challenges, we propose neuromorphic laser speckle imaging (NLSI) as an efficient alternative approach for motion analysis. Our method preserves the motion information while excluding the redundant data by exploring the use of the neuromorphic event sensor, which acquires only the relevant information of the moving parts and responds asynchronously with a much higher sampling rate. This neuromorphic data acquisition mechanism captures fast-moving objects on the order of microseconds. In the proposed NLSI method, the moving object is illuminated using a coherent light source, and the reflected high frequency laser speckle patterns are captured with a bare neuromorphic event sensor. We present the data processing strategy to analyze motion from event-based laser speckles, and the experimental results demonstrate the feasibility of our method at different motion speeds.

Event-based laser speckle correlation for micro motion estimation: erratum.

We present an erratum to our Letter [Opt. Lett.46, 3885 (2021)OPLEDP0146-959210.1364/OL.430419]. This erratum corrects an inadvertent error in Eq. (4). The corrections have no influence on the results and conclusions of the original Letter.

Event-based laser speckle correlation for micro motion estimation.

Micro motion estimation has important applications in various fields such as microfluidic particle detection and biomedical cell imaging. Conventional methods analyze the motion from intensity images captured using frame-based imaging sensors such as the complementary metal-oxide semiconductor (CMOS) and the charge-coupled device (CCD). Recently, event-based sensors have evolved with the special capability to record asynchronous light changes with high dynamic range, high temporal resolution, low latency, and no motion blur. In this Letter, we explore the potential of using the event sensor to estimate the micro motion based on the laser speckle correlation technique.

Dynamic laser speckle analysis using the event sensor.

Dynamic laser speckle analysis (DLSA) can obtain useful information about the scene dynamics. Traditional implementations use intensity-based imaging sensors such as a complementary metal oxide semiconductor and charge-coupled device to capture time-varying intensity frames. We use an event sensor that measures pixel-wise asynchronous brightness changes to record speckle pattern sequences. Our approach takes advantage of the low latency and high contrast sensitivity of the event sensor to implement DLSA with high temporal resolution. We also propose two evaluation metrics designed especially for event data. Comparison experiments are conducted in identical conditions to demonstrate the feasibility of our proposed approach.

Neuromorphic Imaging With Joint Image Deblurring and Event Denoising.

Neuromorphic imaging reacts to per-pixel brightness changes of a dynamic scene with high temporal precision and responds with asynchronous streaming events as a result. It also often supports a simultaneous output of an intensity image. Nevertheless, the raw events typically involve a large amount of noise due to the high sensitivity of the sensor, while capturing fast-moving objects at low frame rates results in blurry images. These deficiencies significantly degrade human observation and machine processing. Fortunately, the two information sources are inherently complementary - events with microsecond-level temporal resolution, which are triggered by the edges of objects recorded in a latent sharp image, can supply rich motion details missing from the blurry one. In this work, we bring the two types of data together and introduce a simple yet effective unifying algorithm to jointly reconstruct blur-free images and noise-robust events in an iterative coarse-to-fine fashion. Specifically, an event-regularized prior offers precise high-frequency structures and dynamic features for blind deblurring, while image gradients serve as a kind of faithful supervision in regulating neuromorphic noise removal. Comprehensively evaluated on real and synthetic samples, such a synergy delivers superior reconstruction quality for both images with severe motion blur and raw event streams with a storm of noise, and also exhibits greater robustness to challenging realistic scenarios such as varying levels of illumination, contrast and motion magnitude. Meanwhile, it can be driven by much fewer events and holds a competitive edge at computational time overhead, rendering itself preferable as available computing resources are limited. Our solution gives impetus to the improvement of both sensing data and paves the way for highly accurate neuromorphic reasoning and analysis.

Dual Alternating Direction Method of Multipliers for Inverse Imaging.

Inverse imaging covers a wide range of imaging applications, including super-resolution, deblurring, and compressive sensing. We propose a novel scheme to solve such problems by combining duality and the alternating direction method of multipliers (ADMM). In addition to a conventional ADMM process, we introduce a second one that solves the dual problem to find the estimated nontrivial lower bound of the objective function, and the related iteration results are used in turn to guide the primal iterations. We call this D-ADMM, and show that it converges to the global minimum when the regularization function is convex and the optimization problem has at least one optimizer. Furthermore, we show how the scheme can give rise to two specific algorithms, called D-ADMM-L2 and D-ADMM-TV, by having different regularization functions. We compare D-ADMM-TV with other methods on image super-resolution and demonstrate comparable or occasionally slightly better quality results. This paves the way of incorporating advanced operators and strategies designed for basic ADMM into the D-ADMM method as well to further improve the performances of those methods.

Flexible and Hierarchical 3D Interconnected Silver Nanowires/Cellulosic Paper-Based Thermoelectric Sheets with Superior Electrical Conductivity and Ultrahigh Thermal Dispersion Capability.

To date, various electronic devices have been strategically fabricated, and simultaneous realization of high electrical conductivity, sensing property, and heat-conducting property by a simple, efficient, and accurate approach is significant but still challenging. Here, cellulosic fiber supported 3D interconnected silver nanowire (AgNW) networks with hierarchical structures are rationally designed to achieve excellent electrical conductivity and superior thermal dispersion capability. In particular, thermal annealing at the junctions enables both phonon and electron transfer as well as impedes interfacial slippage. In the current study, the AgNW/cellulosic paper with the low Ag content (1.55 wt %) exhibits a low sheet resistance of 0.51 Ω sq-1. More importantly, the AgNW/cellulosic paper-based flexible strain sensor has been reasonably developed, which can be applied to monitor various microstructural changes and human motions with high sensitivity and robust stability (fast response/relaxation time of ∼100 ms and high stability >2000 bending-stretching cycles). The AgNW/cellulosic paper-based device also displays efficient thermal dispersion property, which offers exciting opportunities for thermal management application. Furthermore, the obtained hybrid paper exhibits superior heat dispersion capacity for thermal management devices. Overall, uniform dispersion and 3D interconnected junctions of AgNW among the fibers inside the cellulosic papers lead to the combination of high mechanical strength, highly efficient electrical conductivity, and ultrahigh heat dispersion property. The AgNW/cellulosic paper has promising potentials in the flexible and wearable sensing elements, thermal management materials, and artificial intelligence devices.