Road User Position and Speed Estimation via Deep Learning from Calibrated Fisheye Videos.

In this paper, we present a deep learning processing flow aimed at Advanced Driving Assistance Systems (ADASs) for urban road users. We use a fine analysis of the optical setup of a fisheye camera and present a detailed procedure to obtain Global Navigation Satellite System (GNSS) coordinates along with the speed of the moving objects. The camera to world transform incorporates the lens distortion function. YOLOv4, re-trained with ortho-photographic fisheye images, provides road user detection. All the information extracted from the image by our system represents a small payload and can easily be broadcast to the road users. The results show that our system is able to properly classify and localize the detected objects in real time, even in low-light-illumination conditions. For an effective observation area of 20 m × 50 m, the error of the localization is in the order of one meter. Although an estimation of the velocities of the detected objects is carried out by offline processing with the FlowNet2 algorithm, the accuracy is quite good, with an error below one meter per second for urban speed range (0 to 15 m/s). Moreover, the almost ortho-photographic configuration of the imaging system ensures that the anonymity of all street users is guaranteed.

Multi-View Learning for Material Classification.

Material classification is similar to texture classification and consists in predicting the material class of a surface in a color image, such as wood, metal, water, wool, or ceramic. It is very challenging because of the intra-class variability. Indeed, the visual appearance of a material is very sensitive to the acquisition conditions such as viewpoint or lighting conditions. Recent studies show that deep convolutional neural networks (CNNs) clearly outperform hand-crafted features in this context but suffer from a lack of data for training the models. In this paper, we propose two contributions to cope with this problem. First, we provide a new material dataset with a large range of acquisition conditions so that CNNs trained on these data can provide features that can adapt to the diverse appearances of the material samples encountered in real-world. Second, we leverage recent advances in multi-view learning methods to propose an original architecture designed to extract and combine features from several views of a single sample. We show that such multi-view CNNs significantly improve the performance of the classical alternatives for material classification.

Real-time skin chromophore estimation from hyperspectral images using a neural network.

BACKGROUND: Hyperspectral imaging for in vivo human skin study has shown great potential by providing non-invasive measurement from which information usually invisible to the human eye can be revealed. In particular, maps of skin parameters including oxygen rate, blood volume fraction, and melanin concentration can be estimated from a hyperspectral image by using an optical model and an optimization algorithm. These applications, relying on hyperspectral images acquired with a high-resolution camera especially dedicated to skin measurement, have yielded promising results. However, the data analysis process is relatively expensive in terms of computation cost, with calculation of full-face skin property maps requiring up to 5 hours for 3-megapixels hyperspectral images. Such a computation time prevents punctual previewing and quality assessment of the maps immediately after acquisition. METHODS: To address this issue, we have implemented a neural network that models the optimization-based analysis algorithm. This neural network has been trained on a set of hyperspectral images, acquired from 204 patients and their corresponding skin parameter maps, which were calculated by optimization. RESULTS: The neural network is able to generate skin parameter maps that are visually very faithful to the reference maps much more quickly than the optimization-based algorithm, with computation times as short as 2 seconds for a 3-megapixel image representing a full face and 0.5 seconds for a 1-megapixel image representing a smaller area of skin. The average deviation calculated on selected areas shows the network's promising generalization ability, even on wide-field full-face images. CONCLUSION: Currently, the network is adequate for preview purposes, providing relatively accurate results in a few seconds.

Three-dimensional maps of human skin properties on full face with shadows using 3-D hyperspectral imaging.

Hyperspectral imaging has shown great potential for optical skin analysis by providing noninvasive, pixel-by-pixel surface measurements from which, applying an optical model, information such as melanin concentration and total blood volume fraction can be mapped. Such applications have been successfully performed on small flat skin areas, but existing methods are not suited to large areas such as an organ or a face, due to the difficulty of ensuring homogeneous illumination on complex three-dimensional (3-D) objects, which leads to errors in the maps. We investigate two methods to account for these irradiance variations on a face. The first one relies on a radiometric correction of the irradiance, using 3-D information on the face's shape acquired by combining the hyperspectral camera with a 3-D scanner; the second relies on an optimization metric used in the map computation, which is invariant to irradiance. We discuss the advantages and drawbacks of the two methods, after having presented in detail the whole acquisition setup, which has been designed to provide high-resolution images with a short acquisition time, as required for live surface measurements of complex 3-D objects such as the face.

Deep spectral reflectance and illuminant estimation from self-interreflections.

In this work, we propose a convolutional neural network based approach to estimate the spectral reflectance of a surface and spectral power distribution of light from a single RGB image of a V-shaped surface. Interreflections happening in a concave surface lead to gradients of RGB values over its area. These gradients carry a lot of information concerning the physical properties of the surface and the illuminant. Our network is trained with only simulated data constructed using a physics-based interreflection model. Coupling interreflection effects with deep learning helps to retrieve the spectral reflectance under an unknown light and to estimate spectral power distribution of this light as well. In addition, it is more robust to the presence of image noise than classical approaches. Our results show that the proposed approach outperforms state-of-the-art learning-based approaches on simulated data. In addition, it gives better results on real data compared to other interreflection-based approaches.

Traffic sign recognition method for intelligent vehicles.

Traffic sign recognition is one of the main components of intelligent transportation systems (ITS). It improves safety by informing the driver of the current state of the road, e.g., warnings, prohibitions, restrictions, and other information useful for driving. This paper presents a new road sign recognition method that is achieved in three main steps. The first step maps the input image from the Cartesian coordinate system to the log-polar one. The second step computes the histogram of oriented gradients, local binary pattern, and local self-similarity characteristics from the image represented in the log-polar coordinate system. The third step performs classification on the basis of the random forest classifier and the features computed in the second step. The proposed method has been tested on the German Traffic Sign Recognition Benchmark dataset, and the results obtained are satisfactory when compared to the state-of-the-art approaches.

Spectral reflectance estimation from one RGB image using self-interreflections in a concave object.

Light interreflections occurring in a concave object generate a color gradient that is characteristic of the object's spectral reflectance. In this paper, we use this property in order to estimate the spectral reflectance of matte, uniformly colored, V-shaped surfaces from a single RGB image taken under directional lighting. First, simulations show that using one image of the concave object is equivalent to, and can even outperform, the state-of-the-art approaches based on three images taken under three lightings with different colors. Experiments on real images of folded papers were performed under unmeasured direct sunlight. The results show that our interreflection-based approach outperforms existing approaches, even when the latter are improved by a calibration step. The mathematical solution for the interreflection equation and the effect of surface parameters on the performance of the method are also discussed in this paper.

Joint Color-Spatial-Directional Clustering and Region Merging (JCSD-RM) for Unsupervised RGB-D Image Segmentation.

Recent advances in depth imaging sensors provide easy access to the synchronized depth with color, called RGB-D image. In this paper, we propose an unsupervised method for indoor RGB-D image segmentation and analysis. We consider a statistical image generation model based on the color and geometry of the scene. Our method consists of a joint color-spatial-directional clustering method followed by a statistical planar region merging method. We evaluate our method on the NYU depth database and compare it with existing unsupervised RGB-D segmentation methods. Results show that, it is comparable with the state of the art methods and it needs less computation time. Moreover, it opens interesting perspectives to fuse color and geometry in an unsupervised manner.

Color calibration of an RGB camera mounted in front of a microscope with strong color distortion.

This paper aims at showing that performing color calibration of an RGB camera can be achieved even in the case where the optical system before the camera introduces strong color distortion. In the present case, the optical system is a microscope containing a halogen lamp, with a nonuniform irradiance on the viewed surface. The calibration method proposed in this work is based on an existing method, but it is preceded by a three-step preprocessing of the RGB images aiming at extracting relevant color information from the strongly distorted images, taking especially into account the nonuniform irradiance map and the perturbing texture due to the surface topology of the standard color calibration charts when observed at micrometric scale. The proposed color calibration process consists first in computing the average color of the color-chart patches viewed under the microscope; then computing white balance, gamma correction, and saturation enhancement; and finally applying a third-order polynomial regression color calibration transform. Despite the nonusual conditions for color calibration, fairly good performance is achieved from a 48 patch Lambertian color chart, since an average CIE-94 color difference on the color-chart colors lower than 2.5 units is obtained.

Quality image metrics for synthetic images based on perceptual color differences.

Due to the improvement of image rendering processes, and the increasing importance of quantitative comparisons among synthetic color images, it is essential to define perceptually based metrics which enable to objectively assess the visual quality of digital simulations. In response to this need, this paper proposes a new methodology for the determination of an objective image quality metric, and gives an answer to this problem through three metrics. This methodology is based on the LLAB color space for perception of color in complex images, a modification of the CIELab1976 color space. The first metric proposed is a pixel by pixel metric which introduces a local distance map between two images. The second metric associates, to a pair of images, a global value. Finally, the third metric uses a recursive subdivision of the images to obtain an adaptative distance map, rougher but less expensive to compute than the first method.