What if everyone had to leave Earth and no one remembered to turn off the last robot? That kernel of a story idea from Pixar’s Andrew Stanton, who won an Oscar for directing Finding Nemo, grew to become Wall-e—a love story, a science-fiction film, and the latest feature animation in Disney/Pixar’s mega-successful series of CG hits. It’s also the most unusual film Pixar has produced, and arguably the studio’s biggest creative risk.
“Andrew pitched the idea to me when I met him,” says producer Jim Morris, who left LucasFilm to join the Wall-e team. “It had an almost haunting quality, like a Robinson Crusoe story. Why would everyone leave Earth and forget to turn the robot off? Where does this lead? What might cause him not to be lonely? The more we got into the story, the more it appealed to me. I’m a sci-fi fan, and after being on the business side for years, I was hankering to get back into production.”
The robot left behind is named Wall-e, of course, an acronym for Waste Allocation Load Lifters–Earth Class. He’s a rusty little machine that rolls around the dusty planet on triangular tank treads. When the humans’ rampant consumerism trashed Earth, they all moved to a giant spaceship, the Axiom, leaving him behind with the junk. It made sense: Wall-e’s job was to compact all that stuff into cubes, and his program didn’t change. He still motors along and stuffs detritus into his metal belly. When he’s full, out pops a cube that he stacks to create ever-growing towers of trash.
And here’s the risk. Wall-e has no mouth, no nose, and no head. He’s a real robot; he doesn’t talk—in English, anyway. He only makes machine sounds. His expressions come entirely from his body language and his eyes, which are a pair of binoculars that sit atop a long “neck.” And that means Stanton built an entire feature film around a character who doesn’t speak one line of dialog. It doesn’t mean the film is entirely silent, however, although the first third largely is. And, it doesn’t mean that Wall-e hasn’t changed since the people left Earth. He has.
Meet Wall-e
The first third of the feature takes place on Earth’s dusty, debris-filled environment, where billboards on abandoned buildings still broadcast messages from Axiom’s CEO and, conveniently, help set the stage for the audience. But the billboards are in the background. Our attention is on Wall-e. As he rumbles around compacting junk, we see him picking out favorite bits and bobs—a Rubik’s cube, an iPod, bubble wrap—and stashing them in a Styrofoam container. As he does so, we see his personality develop. He flicks open a cigarette lighter, and the flame surprises him. He covers his eyes with a bra. He’s a character. And, he has a friend of sorts: a cockroach.
At the end of his day, Wall-e rolls inside a maintenance truck, his little home, and adds his new treasures to his collection. The detail in Wall-e’s rubbish-filled world outside and inside his maintenance truck is amazing. We can identify household items, electronic gear, car parts, all manner of stuff in the trash towers and in Wall-e’s personal collection—and it’s all CG.
A shallow depth of fi eld helps focus the audience’s attention on the sunbathing Wall-e within his vast CG world.
Inside the truck, the musical Hello, Dolly! plays on Wall-e’s TV set, and we watch him discover how two people in love interact. He taps his “fingers” together in front of him like a nervous little man, and we sympathize with the lonely robot.
Stanton’s mandate was to give his animated feature a different look from Pixar’s previous films, and he succeeded. The blown-out, gritty, garbage-filled Earth is about as far from Finding Nemo as you could imagine, and integrating such live-action elements as the billboards and the movie into the animated world give it a cinematic feeling. Axiom, where most of the second third of the film takes place, is closer in style to previous Pixar films: a colorful, clean, bright space filled with thousands of people and robots. Creating that detail was one problem. Focusing attention on Wall-e was another.
“Because there is no traditional dialog in the first third of the movie and not a lot of dialog in the second two-thirds, it put more pressure on the camera and the lighting than before to tell you what to pay attention to in the world,” says Danielle Feinberg, DP for lighting. For that reason, and because Stanton wanted to create an animation with a cinematic feel, many of the technical innovations for Wall-e centered on photography and lighting.
Photography
“The thing that was aesthetically so enticing about [Wall-e] was that Andrew wanted to create the feeling that it was filmed, not recorded in the computer,” says Morris. “I had spent much time in the live-action universe, worked with a lot of DPs, and was a camera operator myself. So we got a Panavision camera similar to the one used for the original Star Wars, shot film, and analyzed it.” They realized that most tools they had created to imitate the aberrations in live-action photography weren’t correct.
“We ran a battery of tests with a crude model of Wall-e and propagated the data back into our existing camera,” says Nigel Hardwidge, supervising TD. “A lot of things were off, so we redesigned how we wanted our camera model to work.”
The new virtual camera imitates the anamorphic lenses used to film such sci-fi epics as Close Encounters of the Third Kind and the first Star Wars. These lenses squeeze an image horizontally to occupy the full height of 35mm film, and then during projection, a second lens expands the image to fill a wide screen. “The difference in quality is almost subliminal,” says Hardwidge.
In addition, Pixar added such common lens aberrations as barrel distortion, flares, oval-shaped circles of confusion, and lens “breathing” (the way the field of view changes during a rack focus), to help give the computer images the look of photographed film.
Director of photography Jeremy Lasky supervised the 12 layout artists at Pixar who used the new camera and lenses to design camera moves for the film. The artists worked in Pixar’s 3D animation software, Marionette, from 2D storyboards. “It’s similar to live action in a way,” Lasky says. “We shot coverage for the sequences. Then, as the shots get assembled in editing and the sequence gets polished, one idea wins out.”
For example, once the artists started working with Wall-e in the 3D world, they might offer Stanton such choices as an over-the-shoulder shot, a wide shot, and a close-up with the foreground out of focus. “The layout artists could plug in the lenses and see the view change right in front of their eyes,” Lasky says. A limited set of lenses helped maintain consistency through the film.
On Earth, the camera is always moving, panning, tilting, and sometimes acting as if it is on a camera operator’s shoulder. Often, the camera watches the little robot wander through the frame. “We’d deliberately put him on one side of the frame and something else on the other side to give a bigger sense of the world around him,” Lasky explains. On Earth, Wall-e is always in his element.
The artists used different techniques for the spaceship Axiom, where thousands of refugee humans, tended by thousands of robots, float through the scenes on motorized hover chairs. The humans can’t leave their chairs; a constant diet of inaction has turned their bones to mush. Here, reflecting the ship’s orderliness, the camera moves on virtual dolly tracks and on cranes, not on a camera operator’s shoulder. “Anytime the camera moves in a more handheld way, we replicated a steadicam look,” Lasky says. “We took the rough edges off everything.”
To reinforce the notion that Wall-e is out of his element on Axiom, the layout artists framed those shots to include chaos. “We were always trying to capture something else going on in frame,” Lasky says.
Wall-e lands on Axiom by following a beautiful robot, Eve, and this is the love-story part of the film. When Eve arrives on Earth in a spaceship, it’s love at first sight for Wall-e. The sleek, white, egg-shaped ’bot with the sparkling blue eyes and the ability to fly is his Dolly. He shares his treasures with her even though she is not very interested in him. When the spaceship returns, she flies onboard, and lovesick Wall-e stows away. The scene when she leaves Earth is one in which the layout artists added drama with camera moves.
“It was boarded with Wall-e working, a cut to the ship, and then Wall-e at the top of the ramp telling the cockroach to stay,” explains Lasky. “We thought it would be really cool if the camera raced behind him as he ran to the ship and you could see Eve moving inside.”
To help the layout artists design live-action camera moves, Pixar brought in seven-time Oscar nominee Roger Deakins, a DP for such films as No Country for Old Men, O Brother, Where Art Thou?, Fargo, and The Shawshank Redemption. “He’s very good at helping you take an idea and simplify it down,” Lasky says.
Wall-e and Eve run away from the robots chasing them on Axiom. In this world, the camera captures chaos to reinforce the feeling that Wall-e is out of his element.
For example, Deakins worked with Lasky on scenes in which Wall-e and Eve are together in the maintenance truck. “We talked about how you’d coordinate camera moves in that tight space,” Lasky says. “Too many cuts would take you out of the moment, so rather than cutting, we adjusted the camera to keep things moving. It allowed us to stay with the actors.”
Deakins also contributed an idea that radically altered how the layout artists worked. Lasky explains: “We were looking at a layout on the computer for half a sequence. It had basic models, no shading yet. He said, ‘I don’t see many lights.’ I said, ‘Right. Lighting comes later.’ He looked at me and said, ‘It would drive me nuts. Lighting is half my job.’”
After that, Feinberg provided the layout artists with a simple lighting setup. Then, as they composed shots, they could see light, for the appropriate time of day, with colors and shadows. And, as they moved the camera, the shadows and the light falloff changed.
“It was literally like we had been working in the dark,” Lasky says. “It opened so many options.”
An example: During a sequence in which Eve tries to resuscitate Wall-e, he’s in shadow. She sings to him a little, and lifts his face into the light for a second. He falls back into the shadow. When he revives, he steps forward into the light, and it changes the dynamic of the scene.
“We discovered that setup during layout and showed it to Andrew,” Lasky says. Stanton approved the shot. Before, they would have created the sequence without considering the interplay between light and shadow.
Lighting
To help give the film a more cinematic look, Pixar also rewrote the illumination model used for lighting the 3D world. “We wanted the materials to feel more realistic in the way they reacted to light, and the lights to be more physically driven,” says Feinberg. “We’d had the same illumination model since A Bug’s Life. Ratatouille even used the same code but with big pieces added to change the color space.”
The new code builds energy conservation into the lighting and shading model. “We have one knob that takes the materials from rough and diffuse at one end and, at the other, highly reflective metal,” Feinberg says. “Before, we tuned diffuse, specular, and reflection separately. Now, they’re all on a continuum.”
Pixar modelers work in Autodesk’s Maya, with all the shading happening through the Slim interface to RenderMan within Maya. “Our old shading set relied on a TD or an artist to make explicit choices about materials,” Hardwidge says. “For this film, we wanted the degraded pieces of metal and plastic to respond as accurately as possible.”
The shading coefficients now incorporate judgments about how much various materials preserve the energy of light hitting the surface. In addition, because the new lighting and shading model made manual tweaking to produce high-quality materials less necessary, it helped maintain consistency.
“Any prop, whether hastily built for the background or a hero prop, still has the same level of quality in its response to light,” Hardwidge says. “We wanted to build this integration and believability through the whole image.”
In addition, new lights with a built-in reflection component and a falloff set to mimic reality helped the lighting artists do their jobs. To keep render costs within reason, they avoided raytraced reflections, relying on environment maps instead. “We used some RenderMan point-based occlusion,” Feinberg says. “If we had done raytraced occlusions, we would have been in a world of hurt.”
Pixar developed a new illumination model for Wall-e to cause all the materials, from highly reflective plastic to rusty iron, to respond realistically to lights that are more physically driven.
As a result, on Earth, the junk looks real; on Axiom, the environment reflects light accurately, including the light from the colorful, animated billboards that advertise the latest drink to consume and things to buy.
“Axiom is a more reflective and clinical environment, and the shading model allowed us to leverage that,” Hardwidge says. “Rather than a round, white highlight on white plastic, we see the light from the billboards reflecting and stretching, and the way the light falls off and diffuses is much closer to what you would expect.”
Epic Scale
In addition to lighting and photography, the other significant challenge for the artists at Pixar was the scale of the film. On planet Earth, the first act, which extends for the first 25 minutes of the film, takes place in a large cityscape. “We needed five or six square miles of set,” says Hardwidge. “And, when we were planning it, the story wasn’t defined enough to know where specific locations would be.”
So rather than build the Earth only from camera view, the studio modeled a huge set, into which the director and layout artists could place the camera where they wanted.
Because the set extends for miles, disappearing eventually into the horizon, matte paintings sometimes added subtleties in the distance, but the sets were largely 3D. Within these sets, tall towers built from trash cubes rise from the dusty ground, and huge piles of litter collect against buildings. The detail is astounding.
“Clearly, we couldn’t dress the sets with geometry, but you can go only so far with displacement, so we needed to balance the two,” Hardwidge says. Early tests on 50-foot piles of junk helped them determine how to blend the different techniques.
Hardwidge explains: “We built pieces of trash as geometry and turned some of that trash into displacement shaders we controlled with paint and procedural techniques. Then, we put geometric trash on top, depending on the angle of the shot.”
Wall-e, seen here with his cockroach friend, doesn’t have real eyes; he sees through binoculars. Pixar lit the gray aperture rings in the binoculars to help give the impression of eyes.
Adding the geometric debris on top of the displacement shaders gave the final piles of junk some nooks and crannies so lighters could add shadows, depth, and occlusion. “The displacement shader was awesome,” Feinberg says. “The set dressers piled up the right amount of big, little, and medium pieces so the towers don’t look like buildings or rock pillars; they look like they’re made from cubes of trash.”
To scatter rubbish with varying densities, the set dressers used procedural paint tools, and on the shading side, procedural shaders intelligently lit the litter depending on surface angles and on how much dirt or dust had collected.
Effective Details
The effects department also helped dress the sets. The wind sends bits of paper and plastic swirling in the dusty air. Within the trash cubes, small bits of stuff move slightly and catch glints of light. Everything keeps changing in the junkyard landscape.
“We knew we had a big effects job on this show,” Hardwidge says. “We had dust storms, steam coming off the spaceship when it lands, dirt, paper caught in the wind—and a lot of these effects were full-screen. We re-engineered the effects pipeline for more flexibility and to have more powerful tool sets for using our volume shader.”
A new nodal-based tool set named Dynamo acted as the interface between such software applications as Side Effects’ Houdini, Maya, RenderMan, and Marionette. “We could feed into Dynamo any kind of particle through plug-ins—blobbies, spheres, points, sprites, or curves,” Hardwidge says. “It became the framework the effects TDs used to insert particle data into the scene and decide how to output it.”
The particles ranged from hard pieces of dirt thrown off Wall-e’s tire treads, to large-scale nebulas in space, to the low-lying dust that constantly blows across the Earth’s surface. The effects TDs generated 90 percent of these effects and others using various types of particles. They turned to fluid simulations for only a few shots—when Wall-e travels through oily sludge, for instance, and when the spaceship lands.
The detail increased render times, of course, and one of Hardwidge’s jobs was managing the computational load in Pixar’s 2600-processor renderfarm. “You always run into strange things in some shots, and we had a few that took 30 to 40 hours per frame, but we also rendered complex imagery in three to four hours,” he says. “Our goal was to keep it down to eight hours. We achieved an average render time of seven hours per processor for a film-resolution frame with all the elements in there.”
On Axiom, which is approximately two miles from head to tail, the detail is largely in the huge numbers of people and robots that populate the spaceship. To build the enormous variety of robots that constantly serve the people, a team of modelers used component parts. “The articulation belonged with the part,” Hardwidge says. In addition, a unified shader helped keep the designs consistent.
Wall-e stores his collection of interesting stuff in hundreds of bins inside a maintenance truck. To manage the detail, Pixar used a system of displacement shaders topped with geometry.
For the lazy, fat humans, who have lost bone mass, Pixar created a rig with varying thicknesses of skin that responded to a simulation system. “We needed to have the body deform if it fell on the floor,” Hardwidge says. “So we leveraged the Physbam system developed at Stanford to create a volumetric system. If you pushed one area of the human’s skin, you’d see an appropriate response based on the thickness of the skin in that area.”
To animate the crowds, the effects team used a combination of systems. For the robots on Axiom, particularly those in the mechanical areas rather than the human areas, Pixar used Massive to apply various motion cycles created in the animation department. Similarly, Massive moved the 10,000 flaccid humans on their hover chairs. “We had a complex network of lines on the floor,” describes Hardwidge.
But, when the ship tilts and the roly-poly people tumble onto the floor, Pixar pulled in a simulation based on the open-source Open Dynamics Engine implemented in Maya, called MODE, to add physics-based motion. “We generated the rigid-body simulation for the 10,000 people as they hit the chairs and kept kerplunking along the deck,” Hardwidge points out. Then, based on the motion and the speed generated from the simulation, Massive’s fuzzy-logic brain applied cycles from the animation department to the characters.
“We’d bake the simulation and pass it into Massive to choose files of animation based on what the poses from the simulation were doing,” Hardwidge says. “The nice thing about MODE is that you can scale up the number of elements in the simulation in a linear fashion, so the simulation times don’t become excessive and you still get good behaviors.”
All of this—the new cameras and lighting models, the simulation, the attention to details—helped Stanton realize his dream of creating an animated film unlike any other Pixar feature; in fact, unlike any other feature animation. The haunting images of the gritty but fascinating debris-filled planet Earth will stay with audiences long after they leave the theater. The futuristic spaceship will delight them. They’ll laugh at the fat people on their silly hover chairs. They’ll cheer little Wall-e in his attempts to woo the cool Eve, and applaud his heroism. And, they’ll do all this without hearing one complete line of dialog from Wall-e or Eve, and without, for the most part, having any idea of the risks Pixar took in making this remarkable film or the technology that made it possible. And that’s the Pixar magic.
Barbara Robertson is an award-winning writer and a contributing editor for Computer Graphics World. She can be reached at BarbaraRR@comcast.net.