DIANA PHILLIPS MAHONEY
Long the de facto standard for adding 2D detail to computer-generated images at minimal computational expense, traditional texture mapping falls flat, literally, when it comes to adding 3D detail to an object's surface. While texture mapping can change the appearance of the surface of an object in terms of its color, it does not affect the perception of three-dimensional shape.
The two-dimensionality of texture-mapped surfaces is particularly evident when the object or scene is being viewed by a moving observer, as in a virtual walkthrough, or when the static scene is viewed from an oblique angle.
To raise texture maps to a new level, researchers Manuel Oliveira of the State University of New York at Stony Brook and Gary Bishop of the University of North Carolina at Chapel Hill have developed an image-based rendering technique that enables the representation of 3D surface details and depth.
As we move around the real world, depth variations across a surface or among objects in a scene affect our perception of three-dimensionality because of changes in visibility. For instance, when we walk past a brick wall, some mortar regions may be occluded, then become visible, then become occluded again. This effect, called motion parallax, is absent from a CG wall represented as a single polygon mapped with a 2D brick texture.
 |
| As in the real world, the spatial relationships among the bricks in this wall appear to change with various perspectives. This effect is re-created digitally by pre-warping 2D texture maps with height fields, then pasting the pre-warped images onto flat p |
The new approach, called relief texture mapping, integrates conventional texture maps with height fields to achieve a more convincing 3D appearance with only small computational load on the processor, as would be the case with detailed geometric models requiring tens of thousands of polygons.
A relief texture differs from its flat counterpart in that each texture element, or texel, stores some depth information expressing how much the represented surface deviates from a the plane of the texture. To describe the effect, Oliveira uses an example of a poster with a picture of a car. "With traditional texture mapping, we can simulate a poster on the wall by taking a picture of a car and pasting it onto a polygon. In this case, no matter where we look at the poster from, nothing changes." A different effect results when we watch the actual car through a glass window of the same size as the poster. In this case, from each viewpoint, we see a different image. "If for every viewpoint we pass on the window, we see a 2D image that exactly matches the image of the 3D car from that viewpoint, we will have the illusion of looking at the 3D object."
This effect is re-created by warping the original texture based on the observer's viewpoint, so the resulting image always matches the view the observer would have from a given perspective. The resulting warped textures are then pasted on top of flat polygons. To some extent, says Oliveira, "we can think of a relief texture-mapped polygon as a hologram: it looks different from different viewpoints."
The warping process relies on depth information stored at each texel. The warping is carried out as a two-phase process. The first phase consists of shifting texels along the rows of the original image. In the second phase, shifts take place along the columns of the image resulting from the first phase. Next, the algorithm maps the new texture onto one or a few polygons using conventional techniques.
Not only does relief texture mapping significantly increase the expressive power of conventional texture mapping, says Oliveira, "it also dramatically reduces the polygonal count required to model a scene while preserving its realistic appearance."
 |
| A 35,280-polygon model is represented for a fraction of the computational cost using relief texture maps (top right). Textures that have been pre-warped with height fields (bottom) are mapped to two faces of a box that contain the statue (top left). The |
In one example, a 35,280-polygon mod el of a statue is represented using six relief texture mapped polygons, which are the faces of a box that contains the statue. Ultimately, the appearance of the geometry is maintained, even from viewpoints almost parallel to the texture.
Relief textures are intended to complement, not supplant, traditional techniques. "Relief texture mapping preserves all features and benefits of the original texture-mapping technique," says Oliveira. For example, he says, "when a viewer is far away from the represented surface, a relief texture can be rendered as a regular texture by simply ignoring its depth information. As the viewer approaches the surface, we render it as relief texture-mapped polygons." This flexibility is useful for applications requiring dynamic perspectives on a scene, such as games and architectural walkthroughs.
In some applications, says Oliveira, it may not be possible to constrain the viewpoint from crossing the plane of a relief texture, which means that the relief texture-mapped polygon will not be rendered even if the represented surface may still be visible. In such a case, the surface can still be rendered as a mesh of micro-polygons implied by the depth information. Also, because relief texture maps are single-layer image representations, they are ill-suited for rendering objects in which multiple layers of surfaces are needed, such as the leaves of a tree, as well as some concave objects, such as holes.
An important aspect of the new technology, says Oliveira, is that it provides "a framework for combining the photorealistic promise of image-based modeling and rendering with the advantages of polygonal rendering."
Among the areas ripe for further exploration, according to Oliveira, are the design of efficient methods for implementing the technique in hardware as well as automatic or semi-automatic techniques for extracting relief textures from 3D environments. Another area the researchers are considering is the development of dynamic relief textures for use in morphing and animation. For such applications, the colors and texel depth would have to change over time.
In terms of commercial potential, says Oliveira, "The ideal solution would be the incorporation of the technique into graphics chips, in which case it would become widely available and potentially as popular as conventional texture mapping."