Quantization-Aware NN Layers with High-throughput FPGA Implementation for Edge AI.

Over the past few years, several applications have been extensively exploiting the advantages of deep learning, in particular when using convolutional neural networks (CNNs). The intrinsic flexibility of such models makes them widely adopted in a variety of practical applications, from medical to industrial. In this latter scenario, however, using consumer Personal Computer (PC) hardware is not always suitable for the potential harsh conditions of the working environment and the strict timing that industrial applications typically have. Therefore, the design of custom FPGA (Field Programmable Gate Array) solutions for network inference is gaining massive attention from researchers and companies as well. In this paper, we propose a family of network architectures composed of three kinds of custom layers working with integer arithmetic with a customizable precision (down to just two bits). Such layers are designed to be effectively trained on classical GPUs (Graphics Processing Units) and then synthesized to FPGA hardware for real-time inference. The idea is to provide a trainable quantization layer, called Requantizer, acting both as a non-linear activation for neurons and a value rescaler to match the desired bit precision. This way, the training is not only quantization-aware, but also capable of estimating the optimal scaling coefficients to accommodate both the non-linear nature of the activations and the constraints imposed by the limited precision. In the experimental section, we test the performance of this kind of model while working both on classical PC hardware and a case-study implementation of a signal peak detection device running on a real FPGA. We employ TensorFlow Lite for training and comparison, and use Xilinx FPGAs and Vivado for synthesis and implementation. The results show an accuracy of the quantized networks close to the floating point version, without the need for representative data for calibration as in other approaches, and performance that is better than dedicated peak detection algorithms. The FPGA implementation is able to run in real time at a rate of four gigapixels per second with moderate hardware resources, while achieving a sustained efficiency of 0.5 TOPS/W (tera operations per second per watt), in line with custom integrated hardware accelerators.

Three-dimensional imaging of waves and floes in the marginal ice zone during a cyclone.

The marginal ice zone is the dynamic interface between the open ocean and consolidated inner pack ice. Surface gravity waves regulate marginal ice zone extent and properties, and, hence, atmosphere-ocean fluxes and ice advance/retreat. Over the past decade, seminal experimental campaigns have generated much needed measurements of wave evolution in the marginal ice zone, which, notwithstanding the prominent knowledge gaps that remain, are underpinning major advances in understanding the region's role in the climate system. Here, we report three-dimensional imaging of waves from a moving vessel and simultaneous imaging of floe sizes, with the potential to enhance the marginal ice zone database substantially. The images give the direction-frequency wave spectrum, which we combine with concurrent measurements of wind speeds and reanalysis products to reveal the complex multi-component wind-plus-swell nature of a cyclone-driven wave field, and quantify evolution of large-amplitude waves in sea ice.

Deep Demosaicing for Polarimetric Filter Array Cameras.

Polarisation Filter Array (PFA) cameras allow the analysis of light polarisation state in a simple and cost-effective manner. Such filter arrays work as the Bayer pattern for colour cameras, sharing similar advantages and drawbacks. Among the others, the raw image must be demosaiced considering the local variations of the PFA and the characteristics of the imaged scene. Non-linear effects, like the cross-talk among neighbouring pixels, are difficult to explicitly model and suggest the potential advantage of a data-driven learning approach. However, the PFA cannot be removed from the sensor, making it difficult to acquire the ground-truth polarization state for training. In this work we propose a novel CNN-based model which directly demosaics the raw camera image to a per-pixel Stokes vector. Our contribution is twofold. First, we propose a network architecture composed by a sequence of Mosaiced Convolutions operating coherently with the local arrangement of the different filters. Second, we introduce a new method, employing a consumer LCD screen, to effectively acquire real-world data for training. The process is designed to be invariant by monitor gamma and external lighting conditions. We extensively compared our method against algorithmic and learning-based demosaicing techniques, obtaining a consistently lower error especially in terms of polarisation angle.

A data set of sea surface stereo images to resolve space-time wave fields.

Stereo imaging of the sea surface elevation provides unique field data to investigate the geometry and dynamics of oceanic waves. Typically, this technique allows retrieving the 4-D ocean topography (3-D space + time) at high frequency (up to 15-20 Hz) over a sea surface region of area ~104 m2. Stereo data fill the existing wide gap between sea surface elevation time-measurements, like the local observation provided by wave-buoys, and large-scale ocean observations by satellites. The analysis of stereo images provides a direct measurement of the wavefield without the need of any linear-wave theory assumption, so it is particularly interesting to investigate the nonlinearities of the surface, wave-current interaction, rogue waves, wave breaking, air-sea interaction, and potentially other processes not explored yet. In this context, this open dataset aims to provide, for the first time, valuable stereo measurements collected in different seas and wave conditions to invite the ocean-wave scientific community to continue exploring these data and to contribute to a better understanding of the nature of the sea surface dynamics.

On the shape and likelihood of oceanic rogue waves.

We consider the observation and analysis of oceanic rogue waves collected within spatio-temporal (ST) records of 3D wave fields. This class of records, allowing a sea surface region to be retrieved, is appropriate for the observation of rogue waves, which come up as a random phenomenon that can occur at any time and location of the sea surface. To verify this aspect, we used three stereo wave imaging systems to gather ST records of the sea surface elevation, which were collected in different sea conditions. The wave with the ST maximum elevation (happening to be larger than the rogue threshold 1.25H s) was then isolated within each record, along with its temporal profile. The rogue waves show similar profiles, in agreement with the theory of extreme wave groups. We analyze the rogue wave probability of occurrence, also in the context of ST extreme value distributions, and we conclude that rogue waves are more likely than previously reported; the key point is coming across them, in space as well as in time. The dependence of the rogue wave profile and likelihood on the sea state conditions is also investigated. Results may prove useful in predicting extreme wave occurrence probability and strength during oceanic storms.

An Accurate and Robust Artificial Marker Based on Cyclic Codes.

Artificial markers are successfully adopted to solve several vision tasks, ranging from tracking to calibration. While most designs share the same working principles, many specialized approaches exist to address specific application domains. Some are specially crafted to boost pose recovery accuracy. Others are made robust to occlusion or easy to detect with minimal computational resources. The sheer amount of approaches available in recent literature is indeed a statement to the fact that no silver bullet exists. Furthermore, this is also a hint to the level of scholarly interest that still characterizes this research topic. With this paper we try to add a novel option to the offer, by introducing a general purpose fiducial marker which exhibits many useful properties while being easy to implement and fast to detect. The key ideas underlying our approach are three. The first one is to exploit the projective invariance of conics to jointly find the marker and set a reading frame for it. Moreover, the tag identity is assessed by a redundant cyclic coded sequence implemented using the same circular features used for detection. Finally, the specific design and feature organization of the marker are well suited for several practical tasks, ranging from camera calibration to information payload delivery.