Diana Phillips Mahoney
There's no question that computer graphics has had a tremendous impact on the business and practice of artistic expression. And while the consequences have been largely positive, a significant casualty of the digital revolution has been the sensory deprivation the technology imposes on its users.
Artists and designers are a notoriously touchy-feely lot to whom the ideal creative process is a multisensory one that supports direct, physical control of the artistic media. Traditional computer graphics doesn't allow this. By adopting a 2D mouse or stylus as their design implement, artists have had to stifle their natural impulse to tactically explore and manipulate their creations.
 |
| Two heads are better than one, and 3D is better than 2D for the interface used to model them. To create this unusual penguin, the designer interactively deformed and edited a polygonal mesh, then painted its surface using the inTouch haptic-modeling inter |
In recent years, researchers have begun addressing this discrepancy by developing interfaces that close the gap between physical and digital creative processes. One such system under development at the University of North Carolina holds particular promise. Called inTouch, the tool is a 3D interface that lets users interactively create, edit, and paint a 3D multiresolution polygonal mesh using a force-feedback device.
The objective of this development effort is the creation of an intuitive tool that lets even novice users painlessly create a model, edit it with finer details, and directly paint it with colors and textures. This is fundamentally different than the approach taken with most commercial modeling systems. Because the latter typically rely on 2D input and output technologies, there is often a steep learning curve involved in mastering the ability to translate conceptual designs to digital form, which in turn can limit artists' creative freedom. In contrast, inTouch takes a "hands-on" approach by integrating the modeling software with a stereo display and a haptic arm, inviting uninitiated users to grab onto the technology.
Using a standard 2D mouse to create 3D models is counterintuitive, says inTouch developer Arth ur Gregory. "It's almost as bad as trying to create a real model from the other side of a glass window." The goal of in Touch is to allow artists to understand and interact with their virtual models in 3D.
The inTouch system integrates three main functions: direct interaction with 3D models using a haptic interface, multiresolution editing, and interactive 3D painting. The haptic interaction is achieved using a Phantom force-feedback device and Ghost haptic-development toolkit from SensAble Technologies, along with UNC's own collision-detection library called H-Collide. With these tools, users are able to directly manipulate points on the surface of a virtual model and physically place digital paint on the desired locations.
Multiresolution editing is en abled by the system's reliance on subdivision surfaces to represent the geometric models. Subdivision surface techniques use a mesh of polygonal shapes-in this case triangles-to describe a surface. With inTouch, a user can interactively de form the polygonal mesh by choosing the edit level (resolution) and probe-constraint mode, then attaching the haptic probe to the surface. By pushing or pulling on the surface, the user can deform the geometry into a desired shape. The force vector of the probe moves the surface point at the selected edit level. The resulting geometric changes propagate according to predefined subdivision rules to the highest level of the mesh, and the graphical and haptic representations are updated in real time accordingly.
To paint the 3D model, the user chooses the color, saturation, and luminance of the brush stroke, as well as its radius and falloff. The haptic stylus inputs the 3D location of the brush, which is drawn on the screen as a sphere with a radius equal to that of the virtual brush. The radius increases or decreases relative to the force exerted by the user.
The inTouch prototype system, written in C++ using multiple graphics libraries, uses an SGI R10000 InfinityReality engine for the graphical display, a dual-processor Pentium III PC as the haptic server, and a UNC-developed network-transparent interface to link the application programs and the haptic system. As with most force-feedback applications, the major implementation challenge the researchers faced was achieving the real-time haptic display of a deformable 3D surface. Unlike a graphical display, which requires the processing of 30 frames per second to achieve a realistic representation, says Gregory, "a haptic device must be updated at least 1000 times per second." They were able to achieve this by dedicating one of the haptic server's two basic processes entirely to the Ghost toolkit and the H-Collide library in order to update the force displayed by the Phantom. The second process handles message-passing across the network to the client application and the model deformations.
 |
| To paint the 3D model, the user chooses the color, saturation, and luminance of the brush stroke, as well as its radius and falloff. The haptic stylus inputs the 3D location of the brush, which is drawn on the screen as a sphere with a radius equal to tha |
While there are commercial technologies available to achieve each of the main components of inTouch, none addresses them all. For example, traditional modeling software can be used to create complex shapes, but without the force-feedback component, doing so is not an easy, intuitive pro cess. "The force feedback provides an un der standing of the location and proportions of the model relative to a user's hand. This allows users to easily predict the effects of their movements when sculpting or painting," says Gregory. "The force feedback also provides valuable resistance when sculpting or painting the model. When deforming the model, the more force applied, the more dramatic the deformation-just like in the real world."
The same benefit distinguishes the painting capability from that available in standard 3D paint software. "As the user applies more force to the virtual paintbrush, the brush stroke becomes wider. If you've ever painted anything before, imagine trying to perform that same task without the force applied by the object to your paintbrush," says Gregory, noting that such an interface makes the technology more accessible to a broader range of users. "We've had untrained users come in and sculpt and paint cool-looking models from scratch in just a couple of hours. This is hardly enough time to figure out how to transform the models and define construction planes in traditional software, and then to actually sculpt something and paint its surface."
The one commercial product similar in concept to inTouch is the recently introduced haptic-based 3D digital modeling tool called FreeForm from SensAble technologies. The FreeForm modeler differs from inTouch in that the former relies on an internal volumetric representation that lets users sculpt their models as if working with clay. According to Gregory, while such an approach lets artists and designers take advantage of force-feedback technology to express the creativity, it limits users' ability to easily create fine details and sharp features, as well as global modifications, relative to that which can be achieved using subdivision surfaces. In addition, the 3D painting capability is unique to inTouch.
 |
| Modeled and painted with inTouch, a young insect evolved from a 20,480-triangle polyhedron with 5 subdivision levels into a 78,846-triangle object (below, in white). The various perspectives on the finished model demonstrate the fine detail the system ena |
Currently, there are no plans to commercialize the inTouch technology. Gregory points out that it is a proof-of-concept system still under development. Among the potential areas the researchers would like to pursue are the use of haptics for animation and the integration of a six-degrees-of-freedom haptic device to enhance the flexibility of the modeler and allow more interesting brush functions by applied torque.
Diana Phillips Mahoney is chief technology editor of Computer Graphics World.