Metamer mismatching underlies color difference sensitivity.

Color difference sensitivity as represented by the size of discrimination ellipsoids is known to depend on where the colors reside within color space. In the past, various color spaces and color difference formulas have been developed as parametric fits to the experimental data with the goal of establishing a color coordinate system in which equally discriminable colors are equal distances apart. These empirical models, however, provide no explanation as to why color discrimination varies in the way it does. This article considers the hypothesis that the variation in color discrimination tolerances reflects the uncertainty created by the degree of metamer mismatching for a given color. Specifically, the greater the degree of metamer mismatching for a color, the wider the range of spectral reflectances that could have led to it and, hence, the more finely a color needs to be discriminated in order to reliably identify materials and objects. To test this hypothesis, the available color discrimination data sets for surface colors are gathered and analyzed. A strong correlation between color discrimination and the degree of metamer mismatching is found. This correlation provides evidence that metamer mismatching provides an explanation as to why color discrimination varies throughout color space as it does.

Color Sensor Accuracy Index Utilizing Metamer Mismatch Radii.

A novel method is described for evaluating the colorimetric accuracy of digital color cameras based on a new measure of the metamer mismatch body (MMB) that is induced by the change from the camera as an 'observer' to the human standard observer. In comparison to the majority of existing methods for evaluating colorimetric accuracy, the advantage of using the MMB is that it is based on the theory of metamer mismatching and, therefore, shows how much color error can arise in principle. A new measure of colorimetric accuracy based on the shape of the camera-induced MMB is proposed and tested. MMB shape is measured in terms of the moments of inertia of the MMB treated as a mass of uniform density. Since colorimetric accuracy is independent of any linear transformation of the sensor space, the MMB measure needs to be as well. Normalization by the moments of inertia of the object color solid is introduced to provide this independence.

Computational color prediction versus least-dissimilar matching.

The performance of color prediction methods CIECAM02, KSM2, Waypoint, Best Linear, Metamer Mismatch Volume Center, and Relit color signal are compared in terms of how well they explain Logvinenko and Tokunaga's asymmetric color matching results [Seeing Perceiving24, 407 (2011)]. In their experiment, four observers were asked to determine (three repeats) for a given Munsell paper under a test illuminant which of 22 other Munsell papers was the least-dissimilar under a match illuminant. Their use of "least-dissimilar" as opposed to "matching" is an important aspect of their experiment. Their results raise several questions. Question 1: Are observers choosing the original Munsell paper under the match illuminant? If they are, then the average (over 12 matches) color signal (i.e., cone LMS or CIE XYZ) made under a given illuminant condition should correspond to that of the test paper's color signal under the match illuminant. Computation shows that the mean color signal of the matched papers is close to the color signal of the physically identical paper under the match illuminant. Question 2: Which color prediction method most closely predicts the observers' average least-dissimilar match? Question 3: Given the variability between observers, how do individual observers compare to the computational methods in predicting the average observer matches? A leave-one-observer-out comparison shows that individual observers, somewhat surprisingly, predict the average matches of the remaining observers better than any of the above color prediction methods.

Metamer mismatching in practice versus theory.

Metamer mismatching (the phenomenon that two objects matching in color under one illuminant may not match under a different illuminant) potentially has important consequences for color perception. Logvinenko et al. [PLoS ONE10, e0135029 (2015)] show that in theory the extent of metamer mismatching can be very significant. This paper examines metamer mismatching in practice by computing the volumes of the empirical metamer mismatch bodies and comparing them to the volumes of the theoretical mismatch bodies. A set of more than 25 million unique reflectance spectra is assembled using datasets from several sources. For a given color signal (e.g., CIE XYZ) recorded under a given first illuminant, its empirical metamer mismatch body for a change to a second illuminant is computed as follows: the reflectances having the same color signal when lit by the first illuminant (i.e., reflect metameric light) are computationally relit by the second illuminant, and the convex hull of the resulting color signals then defines the empirical metamer mismatch body. The volume of these bodies is shown to vary systematically with Munsell value and chroma. The empirical mismatch bodies are compared to the theoretical mismatch bodies computed using the algorithm of Logvinenko et al. [IEEE Trans. Image Process.23, 34 (2014)]. There are three key findings: (1) the empirical bodies are found to be substantially smaller than the theoretical ones; (2) the sizes of both the empirical and theoretical bodies show a systematic variation with Munsell value and chroma; and (3) applied to the problem of color-signal prediction, the centroid of the empirical metamer mismatch body is shown to be a better predictor of what a given color signal might become under a specified illuminant than state-of-the-art methods.

Outlier modeling for spectral data reduction.

The spectra in spectral reflectance datasets tend to be quite correlated and therefore they can be represented more compactly using standard techniques such as principal components analysis (PCA) as part of a lossy compression strategy. However, the presence of outlier spectra can often increase the overall error of the reconstructed spectra. This paper introduces a new outlier modeling (OM) method that detects, clusters, and separately models outliers with their own set of basis vectors. Outliers are defined in terms of the robust Mahalanobis distance using the fast minimum covariance determinant algorithm as a robust estimator of the multivariate mean and covariance from which it is computed. After removing the outliers from the main dataset, the performance of PCA on the remaining data improves significantly; however, since outlier spectra are a part of the image, they cannot simply be ignored. The solution is to cluster the outliers into a small number of clusters and then model each cluster separately using its own cluster-specific PCA-derived bases. Tests show that OM leads to lower spectral reconstruction errors of reflectance spectra in terms of both normalized RMS and goodness of fit.

Object-color-signal prediction using wraparound Gaussian metamers.

Alexander Logvinenko introduced an object-color atlas based on idealized reflectances called rectangular metamers in 2009. For a given color signal, the atlas specifies a unique reflectance that is metameric to it under the given illuminant. The atlas is complete and illuminant invariant, but not possible to implement in practice. He later introduced a parametric representation of the object-color atlas based on smoother "wraparound Gaussian" functions. In this paper, these wraparound Gaussians are used in predicting illuminant-induced color signal changes. The method proposed in this paper is based on computationally "relighting" that reflectance to determine what its color signal would be under any other illuminant. Since that reflectance is in the metamer set the prediction is also physically realizable, which cannot be guaranteed for predictions obtained via von Kries scaling. Testing on Munsell spectra and a multispectral image shows that the proposed method outperforms the predictions of both those based on von Kries scaling and those based on the Bradford transform.

Illumination estimation via thin-plate spline interpolation.

Thin-plate spline interpolation is used to interpolate the chromaticity of the color of the incident scene illumination across a training set of images. Given the image of a scene under unknown illumination, the chromaticity of the scene illumination can be found from the interpolated function. The resulting illumination-estimation method can be used to provide color constancy under changing illumination conditions and automatic white balancing for digital cameras. A thin-plate spline interpolates over a nonuniformly sampled input space, which in this case is a training set of image thumbnails and associated illumination chromaticities. To reduce the size of the training set, incremental k medians are applied. Tests on real images demonstrate that the thin-plate spline method can estimate the color of the incident illumination quite accurately, and the proposed training set pruning significantly decreases the computation.

Providing a Single Ground-Truth for Illuminant Estimation for the ColorChecker Dataset.

The ColorChecker dataset is one of the most widely used image sets for evaluating and ranking illuminant estimation algorithms. However, this single set of images has at least 3 different sets of ground-truth (i.e., correct answers) associated with it. In the literature it is often asserted that one algorithm is better than another when the algorithms in question have been tuned and tested with the different ground-truths. In this short correspondence we present some of the background as to why the 3 existing ground-truths are different and go on to make a new single and recommended set of correct answers. Experiments reinforce the importance of this work in that we show that the total ordering of a set of algorithms may be reversed depending on whether we use the new or legacy ground-truth data.

A multilinear constraint on dichromatic planes for illumination estimation.

A new multilinear constraint on the color of the scene illuminant based on the dichromatic reflection model is proposed. The formulation avoids the problem, common to previous dichromatic methods, of having to first identify pixels corresponding to the same surface material. Once pixels from two or more materials have been identified, their corresponding dichromatic planes can be intersected to yield the illuminant color. However, it is not always easy to determine which pixels from an arbitrary region of an image belong to which dichromatic plane. The image region may cover an area of the scene encompassing several different materials and, hence, pixels from several different dichromatic planes. The new multilinear constraint accounts for this multiplicity of materials and provides a mechanism for choosing the most plausible illuminant from a finite set of candidate illuminants. The performance of this new method is tested on a database of real images.

Gaussian-Based Hue Descriptors.

A robust and accurate hue descriptor that is useful in modeling human color perception and for computer vision applications is explored. The hue descriptor is based on the peak wavelength of a Gaussian-like function (called a wraparound Gaussian) and is shown to correlate as well as CIECAM02 hue to the hue designators of papers from the Munsell and Natural Color System color atlases and to the hue names found in Moroney's Color Thesaurus. The new hue descriptor is also shown to be significantly more stable under a variety of illuminants than CIECAM02. The use of wraparound Gaussians as a hue model is similar in spirit to the use of subtractive Gaussians proposed by Mizokami et al., but overcomes many of their limitations.

How metamer mismatching decreases as the number of colour mechanisms increases with implications for colour and lightness constancy.

Metamer mismatching has been previously found to impose serious limitations on colour constancy. The extent of metamer mismatching is shown here to be considerably smaller for trichromats than for dichromats, and maximal for monochromats. The implications for achromatic colour perception are discussed.

Rethinking Colour Constancy.

Colour constancy needs to be reconsidered in light of the limits imposed by metamer mismatching. Metamer mismatching refers to the fact that two objects reflecting metameric light under one illumination may reflect non-metameric light under a second; so two objects appearing as having the same colour under one illuminant can appear as having different colours under a second. Yet since Helmholtz, object colour has generally been believed to remain relatively constant. The deviations from colour constancy registered in experiments are usually thought to be small enough that they do not contradict the notion of colour constancy. However, it is important to determine how the deviations from colour constancy relate to the limits metamer mismatching imposes on constancy. Hence, we calculated metamer mismatching's effect for the 20 Munsell papers and 8 pairs of illuminants employed in the colour constancy study by Logvinenko and Tokunaga and found it to be so extensive that the two notions-metamer mismatching and colour constancy-must be mutually exclusive. In particular, the notion of colour constancy leads to some paradoxical phenomena such as the possibility of 20 objects having the same colour under chromatic light dispersing into a hue circle of colours under neutral light. Thus, colour constancy refers to a phenomenon, which because of metamer mismatching, simply cannot exist. Moreover, it obscures the really important visual phenomenon; namely, the alteration of object colours induced by illumination change. We show that colour is not an independent, intrinsic attribute of an object, but rather an attribute of an object/light pair, and then define a concept of material colour in terms of equivalence classes of such object/light pairs. We suggest that studying the shift in material colour under a change in illuminant will be more fruitful than pursuing colour constancy's false premise that colour is an intrinsic attribute of an object.

A comparison of computational color constancy algorithms--part I: methodology and experiments with synthesized data.

We introduce a context for testing computational color constancy, specify our approach to the implementation of a number of the leading algorithms, and report the results of three experiments using synthesized data. Experiments using synthesized data are important because the ground truth is known, possible confounds due to camera characterization and pre-processing are absent, and various factors affecting color constancy can be efficiently investigated because they can be manipulated individually and precisely. The algorithms chosen for close study include two gray world methods, a limiting case of a version of the Retinex method, a number of variants of Forsyth's gamut-mapping method, Cardei et al.'s neural net method, and Finlayson et al.'s color by correlation method. We investigate the ability of these algorithms to make estimates of three different color constancy quantities: the chromaticity of the scene illuminant, the overall magnitude of that illuminant, and a corrected, illumination invariant, image. We consider algorithm performance as a function of the number of surfaces in scenes generated from reflectance spectra, the relative effect on the algorithms of added specularities, and the effect of subsequent clipping of the data. All data is available on-line at http://www.cs.sfu.ca/(tilde)color/data, and implementations for most of the algorithms are also available (http://www.cs.sfu.ca/(tilde)color/code).

A comparison of computational color constancy algorithms--part II: experiments with image data.

We test a number of the leading computational color constancy algorithms using a comprehensive set of images. These were of 33 different scenes under 11 different sources representative of common illumination conditions. The algorithms studied include two gray world methods, a version of the Retinex method, several variants of Forsyth's gamut-mapping method, Cardei et al.'s neural net method, and Finlayson et al.'s Color by Correlation method. We discuss a number of issues in applying color constancy ideas to image data, and study in depth the effect of different preprocessing strategies. We compare the performance of the algorithms on image data with their performance on synthesized data. All data used for this study are available online at http://www.cs.sfu.ca/(tilde)color/data, and implementations for most of the algorithms are also available (http://www.cs.sfu.ca/(tilde)color/code). Experiments with synthesized data (part one of this paper) suggested that the methods which emphasize the use of the input data statistics, specifically color by correlation and the neural net algorithm, are potentially the most effective at estimating the chromaticity of the scene illuminant. Unfortunately, we were unable to realize comparable performance on real images. Here exploiting pixel intensity proved to be more beneficial than exploiting the details of image chromaticity statistics, and the three-dimensional (3-D) gamut-mapping algorithms gave the best performance.

Evaluating combinational illumination estimation methods on real-world images.

Illumination estimation is an important component of color constancy and automatic white balancing. A number of methods of combining illumination estimates obtained from multiple subordinate illumination estimation methods now appear in the literature. These combinational methods aim to provide better illumination estimates by fusing the information embedded in the subordinate solutions. The existing combinational methods are surveyed and analyzed here with the goals of determining: 1) the effectiveness of fusing illumination estimates from multiple subordinate methods; 2) the best method of combination; 3) the underlying factors that affect the performance of a combinational method; and 4) the effectiveness of combination for illumination estimation in multiple-illuminant scenes. The various combinational methods are categorized in terms of whether or not they require supervised training and whether or not they rely on high-level scene content cues (e.g., indoor versus outdoor). Extensive tests and enhanced analyzes using three data sets of real-world images are conducted. For consistency in testing, the images were labeled according to their high-level features (3D stages, indoor/outdoor) and this label data is made available on-line. The tests reveal that the trained combinational methods (direct combination by support vector regression in particular) clearly outperform both the non-combinational methods and those combinational methods based on scene content cues.

Metamer mismatching.

A new algorithm for calculating the metamer mismatch volumes that arise in colour vision and colour imaging is introduced. Unlike previous methods, the proposed method places no restrictions on the set of possible object reflectance spectra. As a result of such restrictions, previous methods have only been able to provide approximate solutions to the mismatch volume. The proposed new method is the first to characterize precisely the metamer mismatch volume for any possible reflectance.

Estimating the scene illumination chromaticity by using a neural network.

A neural network can learn color constancy, defined here as the ability to estimate the chromaticity of a scene's overall illumination. We describe a multilayer neural network that is able to recover the illumination chromaticity given only an image of the scene. The network is previously trained by being presented with a set of images of scenes and the chromaticities of the corresponding scene illuminants. Experiments with real images show that the network performs better than previous color constancy methods. In particular, the performance is better for images with a relatively small number of distinct colors. The method has application to machine vision problems such as object recognition, where illumination-independent color descriptors are required, and in digital photography, where uncontrolled scene illumination can create an unwanted color cast in a photograph.