Effects of light map orientation and shape on the visual perception of canonical materials.

We previously presented a systematic optics-based canonical approach to test material-lighting interactions in their full natural ecology, combining canonical material and lighting modes. Analyzing the power of the spherical harmonics components of the lighting allowed us to predict the lighting effects on material perception for generic natural illumination environments. To further understand how material properties can be brought out or communicated visually, in the current study, we tested whether and how light map orientation and shape affect these interactions in a rating experiment: For combinations of four materials, three shapes, and three light maps, we rotated the light maps in 15 different configurations. For the velvety objects, there were main and interaction effects of lighting and light map orientation. The velvety ratings decreased when the main light source was coming from the back of the objects. For the specular objects, there were main and interaction effects of lighting and shape. The specular ratings increased when the environment in the specular reflections was clearly visible in the stimuli. For the glittery objects, there were main and interaction effects of shape and light map orientation. The glittery ratings correlated with the coverage of the glitter reflections as the shape and light map orientation varied. For the matte objects, results were robust across all conditions. Last, we propose combining the canonical modes approach with so-called importance maps to analyze the appearance features of the proximal stimulus, the image, in contradistinction to the physical parameters as an approach for optimization of material communication.

A systematic approach to testing and predicting light-material interactions.

Photographers and lighting designers set up lighting environments that best depict objects and human figures to convey key aspects of the visual appearance of various materials, following rules drawn from experience. Understanding which lighting environment is best adapted to convey which key aspects of materials is an important question in the field of human vision. The endless range of natural materials and lighting environments poses a major problem in this respect. Here we present a systematic approach to make this problem tractable for lighting-material interactions, using optics-based models composed of canonical lighting and material modes. In two psychophysical experiments, different groups of inexperienced observers judged the material qualities of the objects depicted in the stimulus images. In the first experiment, we took photographs of real objects as stimuli under canonical lightings. In a second experiment, we selected three generic natural lighting environments on the basis of their predicted lighting effects and made computer renderings of the objects. The selected natural lighting environments have characteristics similar to the canonical lightings, as computed using a spherical harmonic analysis. Results from the two experiments correlate strongly, showing (a) how canonical material and lighting modes associate with perceived material qualities; and (b) which lighting is best adapted to evoke perceived material qualities, such as softness, smoothness, and glossiness. Our results demonstrate that a system of canonical modes spanning the natural range of lighting and materials provides a good basis to study lighting-material interactions in their full natural ecology.

The perception of hazy gloss.

Most previous work on gloss perception has examined the strength and sharpness of specular reflections in simple bidirectional reflectance distribution functions (BRDFs) having a single specular component. However, BRDFs can be substantially more complex and it is interesting to ask how many additional perceptual dimensions there could be in the visual representation of surface reflectance qualities. To address this, we tested materials with two specular components that elicit an impression of hazy gloss. Stimuli were renderings of irregularly shaped objects under environment illumination, with either a single Ward specular BRDF component (Ward, 1992), or two such components, with the same total specular reflectance but different sharpness parameters, yielding both sharp and blurry highlights simultaneously. Differently shaped objects were presented side by side in matching, discrimination, and rating tasks. Our results show that observers mainly attend to the sharpest reflections in matching tasks, but they can indeed discriminate between single-component and two-component specular materials in discrimination and rating tasks. The results reveal an additional perceptual dimension of gloss-beyond strength and sharpness-akin to "haze gloss" (Hunter & Harold, 1987). However, neither the physical measurements of Hunter and Harold nor the kurtosis of the specular term predict perception in our tasks. We suggest the visual system may use a decomposition of specular reflections in the perception of hazy gloss, and we compare two possible candidates: a physical representation made of two gloss components, and an alternative representation made of a central gloss component and a surrounding halo component.

Specular motion and 3D shape estimation.

Dynamic visual information facilitates three-dimensional shape recognition. It is still unclear, however, whether the motion information generated by moving specularities across a surface is congruent to that available from optic flow produced by a matte-textured shape. Whereas the latter is directly linked to the first-order properties of the shape and its motion relative to the observer, the specular flow, the image flow generated by a specular object, is less sensitive to the object's motion and is tightly related to second-order properties of the shape. We therefore hypothesize that the perceived bumpiness (a perceptual attribute related to curvature magnitude) is more stable to changes in the type of motion in specular objects compared with their matte-textured counterparts. Results from two two-interval forced-choice experiments in which observers judged the perceived bumpiness of perturbed spherelike objects support this idea and provide an additional layer of evidence for the capacity of the visual system to exploit image information for shape inference.

Material category of visual objects computed from specular image structure.

Recognizing materials and their properties visually is vital for successful interactions with our environment, from avoiding slippery floors to handling fragile objects. Yet there is no simple mapping of retinal image intensities to physical properties. Here, we investigated what image information drives material perception by collecting human psychophysical judgements about complex glossy objects. Variations in specular image structure-produced either by manipulating reflectance properties or visual features directly-caused categorical shifts in material appearance, suggesting that specular reflections provide diagnostic information about a wide range of material classes. Perceived material category appeared to mediate cues for surface gloss, providing evidence against a purely feedforward view of neural processing. Our results suggest that the image structure that triggers our perception of surface gloss plays a direct role in visual categorization, and that the perception and neural processing of stimulus properties should be studied in the context of recognition, not in isolation.

Compatible embedding for 2D shape animation.

We present new algorithms for the compatible embedding of 2D shapes. Such embeddings offer a convenient way to interpolate shapes having complex, detailed features. Compared to existing techniques, our approach requires less user input, and is faster, more robust, and simpler to implement, making it ideal for interactive use in practical applications. Our new approach consists of three parts. First, our boundary matching algorithm locates salient features using the perceptually motivated principles of scale-space and uses these as automatic correspondences to guide an elastic curve matching algorithm. Second, we simplify boundaries while maintaining their parametric correspondence and the embedding of the original shapes. Finally, we extend the mapping to shapes' interiors via a new compatible triangulation algorithm. The combination of our algorithms allows us to demonstrate 2D shape interpolation with instant feedback. The proposed algorithms exhibit a combination of simplicity, speed, and accuracy that has not been achieved in previous work.

Visual Features in the Perception of Liquids.

Perceptual constancy-identifying surfaces and objects across large image changes-remains an important challenge for visual neuroscience [1-8]. Liquids are particularly challenging because they respond to external forces in complex, highly variable ways, presenting an enormous range of images to the visual system. To achieve constancy, the brain must perform a causal inference [9-11] that disentangles the liquid's viscosity from external factors-like gravity and object interactions-that also affect the liquid's behavior. Here, we tested whether the visual system estimates viscosity using "midlevel" features [12-14] that respond more to viscosity than other factors. Observers reported the perceived viscosity of simulated liquids ranging from water to molten glass exhibiting diverse behaviors (e.g., pouring, stirring). A separate group of observers rated the same animations for 20 midlevel 3D shape and motion features. Applying factor analysis to the feature ratings reveals that a weighted combination of four underlying factors (distribution, irregularity, rectilinearity, and dynamics) predicted perceived viscosity very well across this wide range of contexts (R2 = 0.93). Interestingly, observers unknowingly ordered their midlevel judgments according to the one common factor across contexts: variation in viscosity. Principal component analysis reveals that across the features, the first component lines up almost perfectly with the viscosity (R2 = 0.96). Our findings demonstrate that the visual system achieves constancy by representing stimuli in a multidimensional feature space-based on complementary, midlevel features-which successfully cluster very different stimuli together and tease similar stimuli apart, so that viscosity can be read out easily.

Improving shape depiction under arbitrary rendering.

Based on the observation that shading conveys shape information through intensity gradients, we present a new technique called Radiance Scaling that modifies the classical shading equations to offer versatile shape depiction functionalities. It works by scaling reflected light intensities depending on both surface curvature and material characteristics. As a result, diffuse shading or highlight variations become correlated with surface feature variations, enhancing concavities and convexities. The first advantage of such an approach is that it produces satisfying results with any kind of material for direct and global illumination: we demonstrate results obtained with Phong and Ashikmin-Shirley BRDFs, Cartoon shading, sub-Lambertian materials, perfectly reflective or refractive objects. Another advantage is that there is no restriction to the choice of lighting environment: it works with a single light, area lights, and interreflections. Third, it may be adapted to enhance surface shape through the use of precomputed radiance data such as Ambient Occlusion, Prefiltered Environment Maps or Lit Spheres. Finally, our approach works in real time on modern graphics hardware making it suitable for any interactive 3D visualization.