Lighting Sensitive Display: Lighting Sensitive Display of David
Project Page: http://www.cs.columbia.edu/CAVE/proje...
Although display devices have been used for decades, they have functioned without taking into account the illumination of their environment. In this project, an initial step has been taken towards addressing this limitation. We are exploring the concept of a lighting sensitive display (LSD) -- a display that measures the surrounding illumination and modifies its content accordingly. In particular, we are interested in a system that renders a 3D scene such that it always appears to be lit by the real environment that the display resides in. In addition, we would like the appearance of the scene to quickly adapt to changes in the environmental lighting. This is a challenging rendering problem as it requires the subtle appearance details (shadings, specularities, shadows and interreflections) of the scene to be rendered with high efficiency.
We have proposed several sensing techniques for measuring the illumination field incident on a display. While it is desirable to measure the complete 4D light field incident upon the display, our current system uses a 2D measurement that is adequate when the light sources in the environment are distant from the display. A novel relighting algorithm has been developed that uses a compact representation of a large set of images of the scene that correspond to different lighting conditions. Unlike previous relighting algorithms, this one exploits not only image correlations over the illumination dimensions but also coherences over the spatial dimensions of the image. This enables the LSD to achieve high quality relighting in real time. Our prototype LSD can render 640x480 images of scenes under complex and varying illuminations at 15 frames per second using a 2GHz processor. This project was done in collaboration with Peter Belhumeur(http://www1.cs.columbia.edu/~belhumeur/) at Columbia University and Terry Boult(http://www.vast.uccs.edu/~tboult/inde...) at the University of Colorado, Colorado Springs.
In this video the LSD displays a synthetic model of Michelangelo's David (courtesy of Marc Levoy, Stanford University). As a lamp is moved around in front of the display, the LSD changes the appearance of the synthetic David so as to keep it consistent with the illumination of the real environment.