You may have seen James Brown-the "Godfatha' Of Soul"-who hasn't? But you can be sure you've never seen him quite like this. At Paul Allen's Experience Music Project, a new museum in Seattle designed to look like a smashed Jimi Hendrix guitar, Brown stars in a ride film that transports ticket holders back to a 1970s funkadelic scene. During the film, we get a four-minute glimpse of a young James Brown dancing in the street and singing "Sex Machine" ... but it's an illusion. An effects crew at Digital Do main (Venice, CA) captured, modeled, animated, and rendered James Brown's face, subtracting 30 years in the process, then put the CG face on a dance double who performed the song, using the younger James Brown's moves.
Creating the photorealistic CG face, doing realistic lip sync animation for the singer, and then pasting that face onto the dance double's head in the 70mm film required innovation in several areas. The crew used subdivision surfaces to prepare a model for facial animation, created a facial animation system that blended keyframe and motion capture techniques in a clever way, and developed a global illumination technique and implemented it within RenderMan.
The ride starts, as do most motion-based attractions, with a film shown in a waiting area to set the stage for the ride. In the film, two boys who would like to be performers are at a funk music concert, where they meet the Archangel of Funk. The Archangel offers to take them on a journey that will help them learn to feel the music, not just play the notes. As the boys are led into a brightly colored tunnel, the pre-show film ends and the ticket holders move on to the main theater. Here they sit in motion-base seats in front of a 120-foot-wide by 30-foot tall screen, curved horizontally and vertically, onto which the 70mm ride film is projected.
Onscreen, the boys fall through a psychedelic tunnel, and the motion base follows the action. The path they take and the brilliant images inside the tunnel were designed and created by Ron Gress, art director, who used Corel's (Ottawa, Canada) Bryce to create storyboards. During the ride, the music takes the audience back in time and at one point drives an animated landscape via channel operators in Side Effects' (Toronto) Houdini. Gress's Bryce "paintings" were used as texture maps in Houdini.
|The digital model of young James Brown's head at right was created in Maya using polygons and subdivision surfaces. The model at far right was rendered with Pixar's Render Man using texture paintings, bump maps, displacement maps, and shaders to c|
When the boys pop out of the tunnel, they land on a dark, deserted street and look up to see a two-story billboard of James Brown on the side of a building. Suddenly, Brown jumps out of the sign, hits the street, and throws a comet of glittering "funk energy" that turns the scene into a wild 1970s block party with bright colors and mylar confetti falling like snow.
The project started, for the effects crew that created the digital James Brown, with a trip to his hometown (Augusta, GA) last fall. There, with the help of the House of Moves (Los Angeles), they motion captured James Brown's face using a Vicon (Oxford, UK) system. The crew also filmed James Brown giving a private concert, made a plaster cast of his face, and digitized his face with a Polhemus (Burlington, VT) Fast Scan. "I felt that although his face has changed over the years, there are some critical poses and timing that he's still doing," says Andre Bustanoby, visual effects supervisor. "Who better to act as a foundation of a younger him than him?"
This reference material went to Todd Masters of MastersFX (Arleta, CA) who sculpted a bust of a young James Brown. Once approved, the sculpture was scanned by Viewpoint Digital (Draper, UT). That scan became a starting point for Digital Domain artist Beau Cameron, who created a digital model using polygonal tools in Alias|Wavefront's Maya and then used subdivision surfaces to make the surface easier to animate realistically.
"By using subdivision surfaces, we can give the surface innate characteristics," Cameron says. He did this in part by determining how to subdivide the polygons. "In deciding how I wanted the mesh to subdivide, I looked at the stresses on the face and followed the contours in areas like around the eyes," he says. After pinpointing where muscles and tendons would cause stress on the skin, he began adding detail in those areas. "The surface will look the same whether you have one five-point, one four-point, two three-point, or three three-point polygons, but the stresses on the surfaces and how it animates will be different," he says.
To create the subdivision surfaces while he modeled, he used a proprietary plug-in for Maya; to fine-tune the surface, he used Maya's polygon smoothing tools, which gave him an approximation of the final surface. Pixar's Photorealistic Ren derMan 3.9 did the final subdivision calculations at render time.
|At top, the tracking dots on the dance double make it possible to track his face in the live action scene and replace it with the CG face of James Brown, below. |
Helping Cameron determine where to put the detail was Caleb Owens, lead technical director who would "skin" the model, that is, set up the model so that the skin would move properly for the animators. "We wanted to go with a pseudo-muscular approach because we needed more subtlety than blend shapes alone could get us," says Owens. In this way, he created the illusion of muscles under the skin without having to model and animate them. "Also, we wanted to incorporate motion capture," Owens says. To accomplish this, Owens' crew adopted a layered approach that allowed animators to blend many methods for creating facial expressions.
Using photographs taken during the motion capture session, the team put points on the model's surface that roughly matched the motion capture dots on James Brown's face. After that, they drew curves through those points, putting one point at each end and one in the middle. The motion capture points could then drive the curves. Also, the team added animation controls via sliders for the same curves. Thus, each curve could be driven by a blend of each type of motion-mocap data or hand animation.
Then, using Maya's Artisan tools, they described how much influence each curve would have on the entire face-that is, how much each curve would tug on a particular group or cluster of control vertices-by painting the areas of influence in shades of gray. The whiter the area, the more weight in that area, which meant that moving the curve would have a big impact there. For fine detail, the animators could grab and move each point. To create the impression of bones and muscles moving under the skin and to change the shape of the lower lid relative to the convex shape of the eye, they added Maya sculpt deformers.
|The House of Moves used a Vicon system (left) to capture James Brown singing. Digital Domain created Maya tools (right) that mix keyframe and mocap data.|
Because James Brown is singing, specific controls for his mouth and lips became especially important. Continuing with the layered approach, the team created 20 shapes for the lips that could be blended before the deformation tools were used.
Mark Brown and a team of five animators had the job of making the CG face of James Brown look like it was singing and reacting to what was happening on the set. For the former, they had reference film and digital video from the Augusta shoot. For the latter, they used the facial expressions of the dance double on the set.
To make it easier to see if what they were doing worked in the scene and to make changes, Owens' team created two faces. They used the same basic model for both, but one had the deformation controls and shape blending tools and the other had only a simple blend shape. They rigged the faces so that the complex face would drive the simple face. The animators worked with the complex face; a change made on that face would cause the simple blend shape to mimic the expression. The simple face was "tracked onto" the dancer's body in the live action scene so it would follow the body as the dancer did his splits, flips, and turns.
With this setup, the animators could work with the complex deformation tools on a face that remained in one position on the screen and while they worked, the simple face on the dancer's body would mimic the expressions they created whether it was turned to the left, the right, or upside down. Procedural, secondary animation, like a slight jiggle to the cheeks created with Maya soft body dynamics, was added to the simple face.
To attach the face to the young dancer's body, the effects crew had to isolate the dancer's face in each frame of the film. To do this, the crew glued tracking dots on the dancer's face, using a mask to register the dots so they'd be in the same position each day. This allowed the tracking team, led by John Paszkiewicz, to find the position of the dancer's head relative to the camera and to the scene, and with this information, they could put James Brown's face onto the double. It would be up to the compositers to smooth the transition from James Brown's face to the dancer's neck.
|Both of these images, created in Bryce and applied to Houdini models, are from a tunnel that takes the viewer into the past and provides action for the motion base. |
Because the CG face was being applied to a body that was reflecting lights in the scene, it was important to have lighting on the face match lights on the body to make the illusion work. For the lighting, Joshua Kolden, CG supervisor, developed a method of global illumination based on research by Paul Debevec and Jitendral Malik, and implemented in RenderMan, that greatly reduced the need for CG lights. Starting with 360-degree photographs of a chrome sphere, he was able to capture the entire dynamic range of light on the set and apply that to radiance maps, which were then used as light sources. Because he was able to use the actual lights in the scene in this way to light the CG face, the face reflected exactly the same light as the dancer's body. Judith Adamson, lighting artist, combined Kolden's global illumination shader with CG lights used for spotlights to create the final renders of the scene.
The film also includes a variation of the "frozen moment" camera technique in which James Brown throws a comet of funk energy that blasts down the street past people who are frozen in place. "The director wanted to integrate some Matrix kind of stuff but in a linear move mimick ing a camera crane," says Bustanoby.
In all, Digital Domain created 40 shots for the ride film that takes people on a journey into the history of funk music. In doing so, the studio has given us a view into a future in which computer graphics can help aging performers act and look as if they are still young. Barbara Robertson is Senior Editor, West Coast, for Computer Graphics World