Improving RGB illuminant estimation exploiting spectral average radiance.

We introduce a method that enhances RGB color constancy accuracy by combining neural network and k-means clustering techniques. Our approach stands out from previous works because we combine multispectral and color information together to estimate illuminants. Furthermore, we investigate the combination of the illuminant estimation in the RGB color and in the spectral domains, as a strategy to provide a refined estimation in the RGB color domain. Our investigation can be divided into three main points: (1) identify the spatial resolution for sampling the input image in terms of RGB color and spectral information that brings the highest performance; (2) determine whether it is more effective to predict the illuminant in the spectral or in the RGB color domain, and finally, (3) assuming that the illuminant is in fact predicted in the spectral domain, investigate if it is better to have a loss function defined in the RGB color or spectral domain. Experimental results are carried out on NUS: a standard dataset of multispectral radiance images with an annotated spectral global illuminant. Among the several considered options, the best results are obtained with a model trained to predict the illuminant in the spectral domain using an RGB color loss function. In terms of comparison with the state of the art, this solution improves the recovery angular error metric by 66% compared to the best tested spectral method, and by 41% compared to the best tested RGB method.

Raw Spectral Filter Array Imaging for Scene Recognition.

Scene recognition is the task of identifying the environment shown in an image. Spectral filter array cameras allow for fast capture of multispectral images. Scene recognition in multispectral images is usually performed after demosaicing the raw image. Along with adding latency, this makes the classification algorithm limited by the artifacts produced by the demosaicing process. This work explores scene recognition performed on raw spectral filter array images using convolutional neural networks. For this purpose, a new raw image dataset is collected for scene recognition with a spectral filter array camera. The classification is performed using a model constructed based on the pretrained Places-CNN. This model utilizes all nine channels of spectral information in the images. A label mapping scheme is also applied to classify the new dataset. Experiments are conducted with different pre-processing steps applied on the raw images and the results are compared. Higher-resolution images are found to perform better even if they contain mosaic patterns.

Spectral Reconstruction from RGB Imagery: A Potential Option for Infinite Spectral Data?

Spectral imaging has revolutionisedvarious fields by capturing detailed spatial and spectral information. However, its high cost and complexity limit the acquisition of a large amount of data to generalise processes and methods, thus limiting widespread adoption. To overcome this issue, a body of the literature investigates how to reconstruct spectral information from RGB images, with recent methods reaching a fairly low error of reconstruction, as demonstrated in the recent literature. This article explores the modification of information in the case of RGB-to-spectral reconstruction beyond reconstruction metrics, with a focus on assessing the accuracy of the reconstruction process and its ability to replicate full spectral information. In addition to this, we conduct a colorimetric relighting analysis based on the reconstructed spectra. We investigate the information representation by principal component analysis and demonstrate that, while the reconstruction error of the state-of-the-art reconstruction method is low, the nature of the reconstructed information is different. While it appears that the use in colour imaging comes with very good performance to handle illumination, the distribution of information difference between the measured and estimated spectra suggests that caution should be exercised before generalising the use of this approach.

Logistic splicing correction for VNIR-SWIR reflectance imaging spectroscopy.

In the field of spectroscopy, a splicing correction is a process by which two spectra captured with different sensors in adjacent or overlapping electromagnetic spectrum ranges are smoothly connected. In our study, we extend this concept to the case of reflectance imaging spectroscopy in the visible-near-infrared (VNIR) and short-wave infrared (SWIR), accounting for additional sources of noise that arise at the pixel level. The proposed approach exploits the adaptive fitting of a logistic function to compute correcting coefficients that harmonize the two spectral sets. This short Letter addresses usage conditions and compares results against the existing state of the art.

Impact of Exposure and Illumination on Texture Classification Based on Raw Spectral Filter Array Images.

Spectral Filter Array cameras provide a fast and portable solution for spectral imaging. Texture classification from images captured with such a camera usually happens after a demosaicing process, which makes the classification performance rely on the quality of the demosaicing. This work investigates texture classification methods applied directly to the raw image. We trained a Convolutional Neural Network and compared its classification performance to the Local Binary Pattern method. The experiment is based on real SFA images of the objects of the HyTexiLa database and not on simulated data as are often used. We also investigate the role of integration time and illumination on the performance of the classification methods. The Convolutional Neural Network outperforms other texture classification methods even with a small amount of training data. Additionally, we demonstrated the model's ability to adapt and scale for different environmental conditions such as illumination and exposure compared to other methods. In order to explain these results, we analyze the extracted features of our method and show the ability of the model to recognize different shapes, patterns, and marks in different textures.

Measuring the Optical Properties of Highly Diffuse Materials.

Measuring the optical properties of highly diffuse materials is a challenge as it could be related to the white colour or an oversaturation of pixels in the acquisition system. We used a spatially resolved method and adapted a nonlinear trust-region algorithm to the fit Farrell diffusion theory model. We established an inversion method to estimate two optical properties of a material through a single reflectance measurement: the absorption and the reduced scattering coefficient. We demonstrate the validity of our method by comparing results obtained on milk samples, with a good fitting and a retrieval of linear correlations with the fat content, given by R2 scores over 0.94 with low p-values. The values of absorption coefficients retrieved vary between 1 × 10-3 and 8 × 10-3 mm-1, whilst the values of the scattering coefficients obtained from our method are between 3 and 8 mm-1 depending on the percentage of fat in the milk sample, and under the assumption of the anisotropy factor g>0.8. We also measured and analyzed the results on white paint and paper, although the paper results were difficult to relate to indicators. Thus, the method designed works for highly diffuse isotropic materials.

Modification and evaluation of the peripheral contrast sensitivity function models.

We propose a series of modifications to the Barten contrast sensitivity function model for peripheral vision based on anatomical and psychophysical studies. These modifications result in a luminance pattern detection model that could quantitatively describe the extent of veridical pattern resolution and the aliasing zone. We evaluated our model against psychophysical measurements in peripheral vision. Our numerical assessment shows that the modified Barten leads to lower estimate errors than its original version.

Image quality and perception: introduction.

This feature issue focuses on image quality and perception, including image and video quality, subjective and objective quality, and enhancement. The feature issue contains papers on several important topics, such as contrast discrimination, analysis of color imaging in cameras, image quality assessment, and more. The papers represent different important aspects in image quality and perception, contributing to the advancement of the field.

A System for Autonomous Seaweed Farm Inspection with an Underwater Robot.

This paper outlines challenges and opportunities in operating underwater robots (so-called AUVs) on a seaweed farm. The need is driven by an emerging aquaculture industry on the Swedish west coast where large-scale seaweed farms are being developed. In this paper, the operational challenges are described and key technologies in using autonomous systems as a core part of the operation are developed and demonstrated. The paper presents a system and methods for operating an AUV in the seaweed farm, including initial localization of the farm based on a prior estimate and dead-reckoning navigation, and the subsequent scanning of the entire farm. Critical data from sidescan sonars for algorithm development are collected from real environments at a test site in the ocean, and the results are demonstrated in a simulated seaweed farm setup.

Translucency perception: A review.

Translucency is an optical and a perceptual phenomenon that characterizes subsurface light transport through objects and materials. Translucency as an optical property of a material relates to the radiative transfer inside and through this medium, and translucency as a perceptual phenomenon describes the visual sensation experienced by humans when observing a given material under given conditions. The knowledge about the visual mechanisms of the translucency perception remains limited. Accurate prediction of the appearance of the translucent objects can have a significant commercial impact in the fields such as three-dimensional printing. However, little is known how the optical properties of a material relate to a perception evoked in humans. This article overviews the knowledge status about the visual perception of translucency and highlights the applications of the translucency perception research. Furthermore, this review summarizes current knowledge gaps, fundamental challenges and existing ambiguities with a goal to facilitate translucency perception research in the future.

Comparison of Imaging Models for Spectral Unmixing in Oil Painting.

The radiation captured in spectral imaging depends on both the complex light-matter interaction and the integration of the radiant light by the imaging system. In order to obtain material-specific information, it is important to define and invert an imaging process that takes into account both aspects. In this article, we investigate the use of several mixing models and evaluate their performances in the study of oil paintings. We propose an evaluation protocol, based on different features, i.e., spectral reconstruction, pigment mapping, and concentration estimation, which allows investigating the different properties of those mixing models in the context of spectral imaging. We conduct our experiment on oil-painted mockup samples of mixtures and show that models based on subtractive mixing perform the best for those materials.

Correlations in Joint Spectral and Polarization Imaging.

Recent imaging techniques enable the joint capture of spectral and polarization image data. In order to permit the design of computational imaging techniques and future processing of this information, it is interesting to describe the related image statistics. In particular, in this article, we present observations for different correlations between spectropolarimetric channels. The analysis is performed on several publicly available databases that are unified for joint processing. We perform global investigation and analysis on several specific clusters of materials or reflection types. We observe that polarization channels generally have more inter-channel correlation than the spectral channels.

Multispectral camera as spatio-spectrophotometer under uncontrolled illumination.

Multispectral constancy enables the illuminant invariant representation of multi-spectral data. This article proposes an experimental investigation of multispectral constancy through the use of multispectral camera as a spectrophotometer for the reconstruction of surface reflectance. Three images with varying illuminations are captured and the spectra of material surfaces is reconstructed. The acquired images are transformed into canonical representation through the use of diagonal transform and spectral adaptation transform. Experimental results show that use of multispectral constancy is beneficial for both filter-wheel and snapshot multi-spectral cameras. The proposed concept is robust to errors in illuminant estimation and is able to perform well with linear spectral reconstruction method. This work makes us one step closer to the use of multispectral imaging for computer vision.

An Evaluation Framework for Spectral Filter Array Cameras to Optimize Skin Diagnosis.

Comparing and selecting an adequate spectral filter array (SFA) camera is application-specific and usually requires extensive prior measurements. An evaluation framework for SFA cameras is proposed and three cameras are tested in the context of skin analysis. The proposed framework does not require application-specific measurements and spectral sensitivities together with the number of bands are the main focus. An optical model of skin is used to generate a specialized training set to improve spectral reconstruction. The quantitative comparison of the cameras is based on reconstruction of measured skin spectra, colorimetric accuracy, and oxygenation level estimation differences. Specific spectral sensitivity shapes influence the results directly and a 9-channel camera performed best regarding the spectral reconstruction metrics. Sensitivities at key wavelengths influence the performance of oxygenation level estimation the strongest. The proposed framework allows to compare spectral filter array cameras and can guide their application-specific development.

Illuminant estimation in multispectral imaging.

With the advancement in sensor technology, the use of multispectral imaging is gaining wide popularity for computer vision applications. Multispectral imaging is used to achieve better discrimination between the radiance spectra, as compared to the color images. However, it is still sensitive to illumination changes. This study evaluates the potential evolution of illuminant estimation models from color to multispectral imaging. We first present a state of the art on computational color constancy and then extend a set of algorithms to use them in multispectral imaging. We investigate the influence of camera spectral sensitivities and the number of channels. Experiments are performed on simulations over hyperspectral data. The outcomes indicate that extension of computational color constancy algorithms from color to spectral gives promising results and may have the potential to lead towards efficient and stable representation across illuminants. However, this is highly dependent on spectral sensitivities and noise. We believe that the development of illuminant invariant multispectral imaging systems will be a key enabler for further use of this technology.

High Dynamic Range Spectral Imaging Pipeline For Multispectral Filter Array Cameras.

Spectral filter arrays imaging exhibits a strong similarity with color filter arrays. This permits us to embed this technology in practical vision systems with little adaptation of the existing solutions. In this communication, we define an imaging pipeline that permits high dynamic range (HDR)-spectral imaging, which is extended from color filter arrays. We propose an implementation of this pipeline on a prototype sensor and evaluate the quality of our implementation results on real data with objective metrics and visual examples. We demonstrate that we reduce noise, and, in particular we solve the problem of noise generated by the lack of energy balance. Data are provided to the community in an image database for further research.

Spectral Characterization of a Prototype SFA Camera for Joint Visible and NIR Acquisition.

Multispectral acquisition improves machine vision since it permits capturing more information on object surface properties than color imaging. The concept of spectral filter arrays has been developed recently and allows multispectral single shot acquisition with a compact camera design. Due to filter manufacturing difficulties, there was, up to recently, no system available for a large span of spectrum, i.e., visible and Near Infra-Red acquisition. This article presents the achievement of a prototype of camera that captures seven visible and one near infra-red bands on the same sensor chip. A calibration is proposed to characterize the sensor, and images are captured. Data are provided as supplementary material for further analysis and simulations. This opens a new range of applications in security, robotics, automotive and medical fields.

Multispectral filter arrays: recent advances and practical implementation.

Thanks to some technical progress in interferencefilter design based on different technologies, we can finally successfully implement the concept of multispectral filter array-based sensors. This article provides the relevant state-of-the-art for multispectral imaging systems and presents the characteristics of the elements of our multispectral sensor as a case study. The spectral characteristics are based on two different spatial arrangements that distribute eight different bandpass filters in the visible and near-infrared area of the spectrum. We demonstrate that the system is viable and evaluate its performance through sensor spectral simulation.

Relationship between reflectance and degree of polarization in the VNIR-SWIR: A case study on art paintings with polarimetric reflectance imaging spectroscopy.

We study the relationship between reflectance and the degree of linear polarization of radiation that bounces off the surface of an unvarnished oil painting. We design a VNIR-SWIR (400 nm to 2500 nm) polarimetric reflectance imaging spectroscopy setup that deploys unpolarized light and allows us to estimate the Stokes vector at the pixel level. We observe a strong negative correlation between the S0 component of the Stokes vector (which can be used to represent the reflectance) and the degree of linear polarization in the visible interval (average -0.81), while the correlation is weaker and varying in the infrared range (average -0.50 in the NIR range between 780 and 1500 nm, and average -0.87 in the SWIR range between 1500 and 2500 nm). By tackling the problem with multi-resolution image analysis, we observe a dependence of the correlation on the local complexity of the surface. Indeed, we observe a general trend that strengthens the negative correlation for the effect of artificial flattening provoked by low image resolutions.

Governance hurdles for expansion of low trophic mariculture production in Sweden.

The study examines the governance of low trophic species mariculture (LTM) using Sweden as a case study. LTM, involving species such as seaweeds and mollusks, offers ecosystem services and nutritious foods. Despite its potential to contribute to blue growth and Sustainable Development Goals, LTM development in the EU and OECD countries has stagnated. A framework for mapping governance elements (institutions, structures, and processes) and analyzing governance objective (effective, equitable, responsive, and robust) was combined with surveys addressed to the private entrepreneurs in the sector. Analysis reveals ineffective institutions due to lack of updated legislation and guidance, resulting in ambiguous interpretations. Governance structures include multiple decision-making bodies without a clear coordination agency. Licensing processes were lengthy and costly for the private entrepreneurs, and the outcomes were uncertain. To support Sweden's blue bioeconomy, LTM governance requires policy integration, clearer direction, coordinated decision-making, and mechanisms for conflict resolution and learning.