Abstract: Many problems in computer vision and graphics require compact and accurate representations of the visual appearance of objects, and the mathematical algorithms to manipulate them. For instance, being able to recognize objects or faces independent of lighting in computer vison requires understanding the space of different appearances with changing illumination. High quality real-time rendering in computer graphics needs models for appearance effects like natural illumination from wide-area light sources such as skylight, realistic material properties like velvet, satin, paints, or wood, and shading effects like soft shadows. Similarly, capturing the appearance of real materials for rendering also requires understanding the variation in appearance with lighting direction, viewing direction and spatial location. In many of these problems, we must operate in a very high-dimensional space. However, one can often find lower-dimensional and more compact structures that lead to efficient algorithms.
In this talk, we discuss appropriate mathematical representations and computational models for a number of challenging problems in visual appearance. First we describe our early work on a signal-processing framework for reflection, formalizing it as a spherical convolution of the lighting and reflectance. This enables handling complex illumination in computer vision applications, as well as a new theory of frequency domain invariants. Next, we develop a factorization method and a wavelet triple product algorithm for high quality real-time rendering with complex lighting, materials and cast shadows. Finally, we discuss the acquisition and representation of space and time-varying visual processes such as drying, burning, decay and corrosion, describing a novel non-linear space-time appearance factorization.