Each year computer graphics artists nudge the state of their art closer toward the creation of a virtual film star that could work alongside human actors. On one branch of the evolving digital family tree we find animal stars created with computer graphics that are real enough to fool the eye-cockroaches, penguins, bats, birds, lions, zebras, elephants, monkeys, apes, and even digital humans seen from a distance. On another branch, we find such characters as Casper, the dragon in Dragonheart, and Jar Jar Binks, Watto, Boss Nass, and others in Star Wars Episode I, who stepped into starring roles, said their lines, and worked successfully alongside human counterparts in feature films. But Casper is a cartoon. Draco, the dragon, is a photorealistic fantasy animal. And while the Star Wars characters act something like humans and even wear clothes, they don't look like anyone recognizable. Thus, on one branch we have the evolution of photorealistic digital beings from the real world and on another, the evolution of digital fantasy beings with starring roles and speaking parts. This month, a digital character named Stuart will step onto the big screen and bend the branches a little closer together.
Created at Sony Pictures Imageworks (Culver City, CA), Stuart Little looks like a mouse but walks, talks, and dresses like a human child. Based on E. B. White's children's book, Stuart Little, the Columbia Pictures/Sony Pictures Entertainment film of the same name is directed by Rob Minkoff, who co-directed The Lion King. It stars Hugh Laurie and Geena Davis as Mr. and Mrs. Little, and Jonathan Lipnicki as their young son George.
 |
| Making it possible to drive Stuart Little's performance were hundreds of controls that helped animators and artists manipulate every detail of this digital character, from his whiskers to his five little fingers to the glint in his eye. Stuart's i |
As the movie begins, George is excitedly saying good-bye to his parents, who have promised to bring home a brother. But at the adoption agency, the roomful of orphans overwhelms the Littles-until a small mouse tries to help. Stuart, the mouse-child, wins the Littles' hearts, and despite a warning from a counselor that it would be better to adopt within their own species, they adopt him. Thus, Stuart finds a new home in a brownstone on Fifth Avenue, to the dismay of his new younger brother George and the horror of the house cat Snowbell, who grunts, "They go for a son and come back with a mouse. I need a drink." From the first moment Stuart appears onscreen until the last minute of the movie, he and his human co-stars must convince the audience that he is as plausible a member of the family as the Littles believe him to be.
"In visual effects, each film has to be a big challenge, or it's not fun," says John Dykstra, senior visual effects supervisor, who won an Oscar in 1977 for creating special photographic effects in Star Wars. "If I know how to do everything when I start on a film, then I'm using obsolete technology."
Stuart's performance would be a big animation challenge; his fur and clothes, big technical challenges. He had the title role; it all had to work. "It was a huge risk," Dykstra says. "We said, 'Of course we can do it,' but in the back of my mind, I'm thinking, 'how the hell am I going to do this?' " He hedged his bet by having animatronic figures ready, but the puppets were rarely used. "It turned out that the CGI character was more cost- and time-effective and more flexible," Dykstra says.
In fact, the digital Stuart appears in 500 shots. "We have a couple shots with the animatronic stunt double in the radio-controlled model car," says Jay Redd, one of four CG supervisors. "But when he turns his head, he's CG." The CG Stuart is in daytime and nighttime shots, indoors and outdoors, up close and personal, and appears alongside human actors and real cats with real fur. Stuart's fur and his costumes must be believable, and to make that more difficult, sometimes they're wet. If that weren't enough, Stuart's "real parents," Mr. and Mrs. Stout, had to be modeled, animated, furred, and costumed as well.
 |
| Particularly difficult to animate were the oddly shaped Mr. and Mrs. Stout, Stuart's "real parents," shown here meeting the Littles: Hugh Laurie, Geena Davis, Jonathan Lipnicki, and Stuart (Michael J. Fox). |
"There were six aspects that we had to get right," says Dykstra. "We hoped we could get four, any four. I think we maybe hit all six." One aspect was creating lip synch for the cats, whose mouths were modified and animated `a la Babe for the scenes in which they talked. For this, the effects team relied on Centropolis Effects (Culver City, CA) and Rhythm & Hues (Los Angeles, CA), two studios with techniques and artists in place for making live animals look like they're talking.
The other five aspects centered on Stuart and the handful of shots with the Stouts. Good match-moving tools and techniques gave the director of photography freedom to film live-action scenes as he wished, yet provided the technical staff enough data to replicate the scene in 3D for the animators. The fur had to make Stuart look like a living mouse, not a stuffed animal. The clothes had to fit and move properly, or he would seem like a doll. Character-animation tools had to give animators the freedom to focus on creating a personality and performance for Stuart. And rendering, lighting, and compositing techniques were needed to blend Stuart flawlessly into the live-action scenes.
Some 127 computer artists, CG supervisors, and technical people were on the dedicated Stuart crew. Stuart's personality and performance were the responsibility of animation director Henry Anderson, who managed a team of 30 animators. Jerome Chen, visual effects supervisor, managed the rest of the CG process, the so-called "pipeline." The four CG supervisors worked directly under him with the help of 32 color, lighting, and composite artists, 15 cloth technical directors, five programming technical directors, 12 technical assistants, and "probably a dozen" people from R&D, according to Chen. In addition, the effects crew included another 40 to 50 artists helping out on occasion.
The principal software used to create and animate Stuart was Alias|Wavefront's (Toronto) Maya, which ran on both SGI (Mountain View) Irix and various Intel-based machines running NT for character animation and exclusively on SGI Irix workstations for everything else. Additional software used at Imageworks included Avid's (Tewksbury, MA) Matador for texture painting, Side Effects' (Toronto) Houdini for effects, Science.D.Visions' (Dortmund, Germany) 3D Equalizer for 3D camera tracking, Pixar's (Pt. Richmond, CA) RenderMan for rendering, Alias|Wavefront's Composer for compositing shots with Stuart, and Discreet's (Montreal) Inferno for compositing the cats. In addition, the studio developed stand-alone code and many proprietary plug-ins and macros to extend and customize the commercial packages.
The process began with principal photography. On the set, the actors practiced with mouse maquettes, but during filming the maquettes were replaced with laser dots. The laser was, cleverly, synchronized with the camera's shutter so that no dots would be recorded on film. After each take, three reference balls were filmed: a chrome ball to capture highlights and environment maps that were used in Stuart's eyes, and white and gray balls that provided lighting information. As it turned out, the technical directors lighted Stuart like the other stars, using rim lights, for example, instead of the information from the white and gray balls. "When we matched the lighting on the balls, Stuart looked like a prop," says Chen.
The negatives from the live-action shoot were scanned using a 10-bit logarithmic color space, color timed to provide color consistency, then given to the match movers. Using Maya to model the scene, and 3D Equalizer to track the position of the live-action camera, the match movers created a 3D environment with a virtual camera that provided an exact match of the scene as it was filmed by the real camera. With this, the stage was set for the animators.
 |
| Although props that Stuart held, such as his toothbrush, were modeled, his vanity is a miniature physical model. |
The animators worked with a modified version of Maya that had specific controls for Stuart's body (or the Stouts) and with dozens of face shapes. Once the animation was approved in gray-shaded form, the match-move data was turned off, Stuart was converted to a high-resolution NURBS model and skinned, then given fur, a costume, and lighting.The last step in the 3D process was the rendering of layers of data as 2D images that were composited with the live-action footage. Depending on the shot, Stuart's head, eyes, whiskers, mouth, teeth, shirt, hands, and three types of shadows might be rendered separately. "We wanted to control each layer in compositing," Chen says.
Long before filming began, research and development were well under way, with some of the crew moving directly from their work on the movie Contact in December 1997 to begin developing Stuart. Stuart had to look like a mouse, but he also had to look cute. By giving Stuart bigger eyes than a real mouse, rounding the shape of the ears and making them furry, and putting fur on his tail, the artists, designers, and effects team managed to keep him looking like a real mouse-but not too real.
Fur, a big technical challenge, became as important as shape in influencing Stuart's cuteness. "The choices of length, color, thickness, whether it's oily and coarse or soft and fuzzy, all help create the perception of Stuart as charming and cute," Chen says. Although it was hard to predict which combinations would work, once they hit the right mix, they all knew it. "I remember the day when everyone ooh'd and aah'd," he says. "Stuart changed very little after that." That final mix put about 450,000 hairs on Stuart's head, with nearly one-third on his ears alone. As in real animals, the length of Stuart's hair varies, his coat has slight variations in its whitish color, and he even has guard hairs. Unlike real mice, the hair above Stuart's eyes is a little darker to give a hint of eyebrows. The Stouts' hair is similar, although their fur is about 10% redder.
To create the hair, Imageworks' R&D team ruled out the few commercial hair and fur solutions available at the time and began developing tools specifically for Stuart. The resulting toolset, comprised of stand-alone code plus RenderMan DSOs [dynamically shared objects] and MEL scripts/plug-ins for Maya, provided a flexible mix of modeling and rendering techniques. The MEL scripts and Maya plug-ins helped define base geometry-a parametric curve with data specifying width and taper at each control point; the stand-alone code then calculated the features of individual hairs. Final hairs were drawn and rendered using RenderMan 3.7's RiCurves primitive calls and special hair shaders. The RiCurves primitive is a series of micropolygons along a curve in a ribbon-like shape that looks 3D and always has its widest part pointing at the camera. "We found we could create about 700 RiCurves [hairs] with one procedure call," says Armin Bruderlin, senior software engineer. "And by using RenderMan's DSOs to create hair at render time, the individual hairs don't show up in files. RenderMan 3.7 saved us a lot of headaches. If we had explicitly created all the hundreds of thousands of hairs for each frame, we would have had lots of large files."
 |
| Making Stuart's fur and clothes look realistic was difficult in this washing machine sequence, in which he's both wet (with clothes floating on and around him) and dry. |
For hair styling, the studio developed its own code. Using the Imageworks software, the hair stylists can specify the curl and orientation of the hairs and can vary the color, length, wave, twist, transparency, and density of hairs in particular areas.
As for animation, the underlying skin usually controls the movement of Stuart's hair. However, hair that gets wet is directly controlled. If the wet hair is animated, the amount, shape, size, and density of clumps are controlled through Maya. If it's static, texture maps are used to specify clumping areas, percentages, and rates over a series of patches; a stand-alone module translates the shades of gray painted in these maps into clumping controls. With this combination of tools, the artists can do something as specific as change the hair under a teardrop as it rolls down Stuart's cheek, or dot his fur with water when he's splashed while sailing on the lake in Central Park.
A second big technical challenge was Stuart's clothes. Working with a maquette, costume designer Joseph Porro tailored 12 costumes for Stuart and the Stouts, many with multiple layers of clothing, then delivered patterns and cloth samples for all the costumes to the cloth technical directors (TDs). The cloth and the patterns were scanned and traced, according to Jim Berney, CG supervisor, who helped develop Maya plug-ins to stitch the pieces together and fit them onto Stuart's model. Then, Maya Cloth's simulation software moved the clothes dynamically. Once the dynamics were tested, the TDs added textures, all fashioned procedurally with custom RenderMan shaders. To create Stuart's plaid jacket, for example, a shader would weave colored threads into cloth, even adding bumps to simulate texture.
Rather than run one lengthy simulation, which would have to be run again if something didn't look right, the technical directors ran several variations at the same time on separate processors." We rarely got a perfect result from one, so we'd take what worked from each and blend them together," Chen says. "Going with Maya Cloth was a gamble because we had to work with alpha code. But we decided to because it wouldn't limit Stuart's performance." And Stuart's performance is, of course, critical to the success of the film.
 |
| Wearing doll clothes that fit better, Stuart strikes a typically concerned-yet-determined pose (left). On the boat (above), Stuart's animation and effects crew had to deal with the impact of wind and water on his fur and sailor suit. |
Henry Anderson, who directed Stuart's animation, says of Stuart: "He doesn't really look at himself as a mouse. He is never driven by his limitations. I always thought of him as a little guy." Unfortunately for the animators, although Stuart needed to emote like a human, his face is shaped differently. "We had dozens and dozens and dozens of face shapes," Anderson says. "By mixing and matching and blending these shapes using sliders, we could create an infinite number of expressions."
As for overall performance, although Michael J. Fox provides Stuart's voice, Bill Irwin, a mime brought into the studio and videotaped, provided the vocabulary for his body language. The animators also used Stuart's hands, tail, and whiskers to convey emotion. "We worked a lot with the fingers," Anderson says. In one scene, for example, Stuart is about to be introduced to the relatives. He comes to the railing of the staircase and as he says, "Boy, that's a lot of Littles," he drums his fingers on the rail. When Stuart is sad, his tail drags on the ground. If he is agitated, it flicks. Similarly, his whiskers droop when he's sad or concerned.
All the animation was done in Maya using what CG supervisor Scott Stokdyk calls a fairly traditional skeleton setup with joints. "We used a combination of forward and inverse kinematics, bound the skin to the skeleton, and used deformers to deform the skin based on the movement of joints," he explains. However, the clothes sometimes made a difference. "We cared about interpenetration of the clothes and we didn't care what the body looked like underneath, so, for example, if an arm came close to penetrating the chest, we'd put a deformer in to cave the chest," he says.
To make it easier for Stuart to interact with his environment, many of the props he touched were created with computer graphics. To integrate Stuart into his environment, a combination of effects, lighting, shading, and compositing were used.
For example, when Stuart touches something soft, he makes an impression. For effects work such as this, Stokdyk's team used Houdini, which, in this case, helped deform the real world in the live-action plate. Most of the integration was accomplished with lighting, though. The TDs extended the illumination model in RenderMan so that surfaces and lights could react to each other, creating, for example, translucent skin and more natural-looking hair. "Phong shading is not enough," Redd says. "Light is diffused by every piece of hair and if the hair is transparent, it's diffused again." Also helping to put Stuart into the environment were multiple layers of shadows. And once all those elements were rendered, the compositors began their work. "We'd get the rendering close, but the render times were so heavy, we set up little tools in Composer to do things like brighten and darken different areas of the mice in post," says Stokdyk.
"This was meticulous work," says Redd. "Even the rivets in Stuart's shoes had reflection maps. Sometimes you just have to keep trying, and that can be tough on artists."
"It is tweaky stuff, 'finesse-y' stuff," agrees Dykstra. "There are so many subliminal cues that can make it seem wrong."
But all indications are that this crew got it right. If so, it's possible that Stuart will achieve something few digital characters have accomplished: a strong emotional tie with the audience. If you see anyone watching the movie get a tear in his or her eye, you'll know that the evolution of digital characters has most definitely taken another step.
Barbara Robertson is Senior Editor, West Coast, for Computer Graphics World.