When it comes to dressing for success, weapons manufacturer and sometimes arms dealer Tony Stark hits the mark, as any fan of Marvel Comics' Iron Man knows and others will learn as the latest superhero blasts into theaters. Directed by Jon Favreau, the Paramount Pictures action/adventure film Iron Man stars Robert Downey Jr. as Stark/Iron Man, Gwyneth Paltrow as his secretary Virginia "Pepper" Potts, and Jeff Bridges as Obadiah Stane, otherwise known as Iron Monger, the enemy.
Iron Man is a rarity in the superhero world. He isn't an alien, and he doesn't have magical superpowers. Stark, the man behind the metal mask, is a regular person, albeit a filthy-rich genius inventor who takes the concept of "power suit" into a whole new dimension. In directing Iron Man, Favreau emphasized that reality. "Not only did the effects have to look real, they had to move realistically and they had to seem as if they were photographed from a real camera with photographic conventions," says John Nelson, senior VFX supervisor, who won a visual effects Oscar for Gladiator and received an Oscar nomination for I, Robot.
Moreover, Favreau made no secret that he didn't like CGI. "He'd flat out say, 'The cameras are never real, it never looks real,'" Nelson says. "So we had to prove this guy wrong. We had to go the extra mile."
The film follows Tony Stark from his capture in Afghanistan, where he cobbles together his first suit, the Mark I, through his emergence as a superhero in his red and gold Mark III. Concept artist Ryan Meinerding designed the Mark I; Phil Saunders, the Mark II and III. Stan Winston Studios built practical versions of these suits, a third prototype silver suit that Robert Downey Jr. and stunt actors wore, and the top half of the evil Iron Monger's suit. Of the eight vendors that created tests for the visual effects work in the film, Nelson and the studio chose Industrial Light & Magic (ILM), The Orphanage, and The Embassy.
ILM did 410 shots, around half, and what Nelson calls the hardest ones: digital versions of the Mark II and III, Stark's "suit-up machine," the digital Iron Monger, and jet aircraft for a dogfight. In addition, ILM created several digital environments, including Los Angeles from skyward, Tony Stark's house, and the Stark Industries campus. The Orphanage contributed the Stark heads-up display (HUD), the missile test, environmental work for the village of Gulmira, and a few shots with the Mark III. The Embassy created the digital Mark I.
Winston Helgason led the team at The Embassy who built the Mark I suit. The practical version of this early Iron Man suit is rugged, with obvious welds, torch burns, and other roughly hewn details, because Stark forged it while being held prisoner in Afghanistan. It was so rugged and stiff that the stunt actor wearing the metal suit couldn't move. So, the Embassy's crew modeled the Mark I in Softimage's XSI and Luxology's Modo from scans of the Stan Winston suit provided by Gentle Giant, added details with Pixologic's ZBrush, animated it in XSI, texture-mapped the suit using high-resolution digital stills of the practical suit, and rendered it in NewTek's LightWave. The two suits appear in back-to-back shots during the first 15 minutes of the film. "This was our first feature," says Helgason. "John Nelson was in Vancouver shooting I, Robot a few years ago and saw our robot cop [Tetra Vaal] and Citroen dancing-car ad. We've done a lot of hard-surface things in 3D, so this was a perfect sequence for us."
The brushed metal of Iron Man's Mark II suit presented an especially difficult rendering problem for ILM's CG supervisors to solve.
At ILM, Ben Snow led a crew of approximately 100 people who worked for 13 months on the show using Autodesk's Maya, ILM's Zeno and Sabre, Apple's Shake, Pixar's RenderMan, Adobe's Photoshop, and proprietary shaders. Snow, who received visual effects Oscar nominations for Pearl Harbor and Star Wars: Episode II, singles out two particular areas in which he believes the studio raised the visual effects bar: first, matching the brushed metal of the practical suit, which helped allow the second, digital costuming.
Both Downey and the stunt actors wore Winston's practical suit on set, but in the end, the CG suit replaced the real metal suits in most of the shots; the CG suit became the character. "We tried to get the realism of the real suit when we could, but [the actors] couldn't move like we needed them to, so we used the practical suit to extend the realism of the CG suit," says Nelson. "Every now and then we popped in a practical suit; it grounds the movie." But, that meant the match needed to be exact. No small problem.
"Iron Man's silver prototype suit (the Mark II) was tricky for us to create in CG because it's brushed metal and we're really close to it," Snow says. "It has a peculiar way of reflecting light."
Digital production supervisors Doug Smythe and Pat Myers wrestled with the solution. Smythe explains: "Throughout the history of computer graphics, we've done silver guys, starting with the silver dinosaurs that PDI did in the early '80s in 'Chromasaurus,' and it turns out that doing chrome is pretty easy. But, trying to get anisotropic brushed-metal surfaces is incredibly hard. One reason is the specular highlights are stretched more in one direction than the other. There are a number of SIGGRAPH papers about anisotropic specular reflection that provide part of the solution, but in the real world, the highlights don't respond in exactly the same way the mathematical SIGGRAPH functions tell you."
The problem, Smythe explains, is that brushing the metal creates microscopic grooves and imperfections that break up the reflections and alter the surface's response to light in various ways. "And this is where the second problem comes in," he says. "The brushed metal lines are so microscopic that if you try to physically model them, you end up with aliasing problems like crazy, so even a procedural approach doesn't work."
In fact, Smythe tried that. "I did some render tests with incredibly high-quality settings," he says. "It took several days to render a single frame, and we still had horribly bad aliasing problems. We had to figure out a different approach."
Smythe also considered using measured BRDFs (Bidirectional Reflectance Distribution Functions). "You get an accurate model of one section, but you can't use that function over the entire surface because it doesn't give you variation, the part of the surface that has a chip, the part that's dirty, the part with half a smudge."
The hero Iron Man flies using thrusters in his boots and "repulsor technology" in his gauntlets that help him steer. The villain Iron Monger, following behind, rockets into the air with an Apollo 13-like burst.
Instead, rather than a table of measured values to tweak the BRDF, they used texture maps. "We used a combination of an anisotropic specular model for the macroscopic-highlight stretching effects, tuned those parameters, and combined them with textured breakup functions so that when we were close to the surface, we'd see the brushed-metal lines," Smythe says. "Texture maps are as old as CG, but being able to use the textures to control localized parameters of shader functions and modulate the results that the basic shader functions give you is really important to make things look like real-world objects rather than perfect CG objects."
RenderMan handled all the raytracing for the show, and Smythe estimates that the shader writers created between 5000 and 6000 lines of RenderMan code to accommodate the surface-collecting variations, such as those caused by primary rays, reflected rays, lying in shadow, and other variations. For example, in one of the first shots of Iron Man in the complete Mark II, we see the lights in his hands illuminating the sides of the silvery, brushed-metal suit. "That's a complicated BRDF because we use multiple specular lobes with different colors and falloffs," Smythe says. "In fact, it responds differently in each color channel."
A second variation: During Iron Man's first flight at night, we see the lights of Los Angeles-a digimatte painting created by ILM artists-reflecting on the brushed-metal suit. The 3D digital matte painting is one of 14 used in 200 shots during the film.
Stark creates the silvery Mark II after escaping from Afghanistan; he vows to stop selling arms and to start fighting evil. Repulsor technology (RT) in his boots, which we watch him build in his workshop, thrust him into the air; RT in his gauntlets helps him control his flight. The boots represent one reason why the CG suit is important: We see Stark working with a prop on a workshop bench. The boot is complicated, but it doesn't do anything. ILM added moving parts and packed the rocket thrusters inside. When Stark pushes a button during the shot, the CG boot, not the prop, opens.
Is It Real, Or...
In a shot soon after, when Stark tests the rocket boots to see if he can get liftoff, he wears a real gauntlet on his right arm and a CG gauntlet on his left (screen right). The new techniques for creating the brushed-metal surface made it possible to believe the digital costume is real. New iMocap technology helped the artists at ILM fit the digital gauntlet onto Downey's arm.
"On Pirates, we used our on-set iMocap tool to capture the performance of the actors that we re-targeted to a CG creature, which completely replaced the actors," Snow says. "On Iron Man, we used it differently because we were doing digital costuming. We had to track the performance exactly. There was no forgiveness." Moreover, the crew wanted to reduce their on-set presence.
"We used all the tricks in the book to get as much information as we could from the suit and a single camera," says Steve Sullivan, director of R&D. "We didn't need to be pixel-accurate on Pirates, but for Iron Man, we needed better algorithms. Production is always chaotic, so we had a lot going on in the solvers to get the human motion." On set, Downey or the stunt actor wore red iMocap "pajamas." Any time either felt constrained by the practical suit, he could remove any piece of it knowing that ILM could replace that part later with digital costume parts. iMocap data captured from the images caught on film made it possible to re-create Downey's motion precisely.
This worked so well it almost became a problem. The client soon forgot which parts were real and which were CG. So, when the actors realized they didn't need to wear the cumbersome parts, they didn't want to. "We tried to work out ways to get them to wear the suit," Snow laughs. "We considered bribing them to save [us from] doing a CG shot."
The geometrically complex "suit-up" machine's robotic arms fuse Iron Man's Mark III suit onto an underlayer garment, starting with his boots. Animators used ILM's dynamic rigging, developed originally for Transformers, to help control all the parts.
During that first flight, we see Stark's house on a cliff overlooking the Pacific Ocean. ILM artists working in San Francisco and in Singapore altered the landscape and created the house from plans provided by the studio for day and night shots. "We were like architects working with a client," says Chris Stosky, matte-painting supervisor. "Any time you see the house, it's a 3D matte painting." In one scene, the camera pulls back to reveal a girl standing at the window in a bedroom and details inside the room behind her.
"I'm proud to say you can see that the bed is made and the pillows are plumped up in rooms that didn't exist," Nelson says. "It's amazing."
Iron Man's Mark II suit has a nearly fatal flaw that becomes apparent as Stark flies higher and higher: The suit ices up, he falls, and crashes through the roof of his house, through a piano, to land on a car in his workshop. The crash causes him to gild the suit so it won't ice up and, to be sporty, paint parts of it red. He also adds weapons; Gulmira, the village of a man who had helped him escape from the terrorists is under attack.
A sequence added late in the process shows how he dons his new red and gold Mark III: Complex 3D robotic arms fit the powerful thousand-pound suit around him. Because Favreau decided to add the sequence after principal photography had wrapped, ILM pieced the workshop background together using a 360-degree steadicam plate shot earlier and tweaked by the digimatte department.
An animatic-created by Pixel Liberation Front and amped up by ILM art director Aaron McBride and an animation crew led by Hal Hickel-choreographed the action. High-resolution stills taken of Downey and shots of a stand-in in a wet suit put Stark into the scene-and the suit. Creature development supervisor James Tooley rigged the so-called suit-up machine, and creature TD Keiji Yamaguchi, who had most recently worked on Transformers, created the shots of a dozen robotic arms fusing the armor-plated pieces onto Stark's under suit. "We wanted the arms to look menacing," McBride says, "to give the idea that he was willing to sacrifice himself for his own invention. He throws himself at their mercy."
Once fused into the new suit, Iron Man flies to the village. The Orphanage picked this three-part rescue sequence for its test, and the production team liked it so much, the test essentially became the animatic for the final shots. ILM created most of the sequence. The Orphanage created backgrounds for the village, an exploding tank, and shots of Iron Man crawling out of a hole. For the tank explosion, the team used Autodesk's 3ds Max. For Iron Man, it used ILM's model and textures, working in Maya for rigging and animation, and Mental Images' Mental Ray for rendering. "The sequence was delayed, and by the time it came to us, we didn't have the capacity to get it done, so we split it with ILM," says Jonathan Rothbart.
By then, ILM was, as Nelson puts it, "completely dialed in" to the Iron Man look. In fact, one of Nelson's favorite shots in the film is ILM's extreme close-up of Iron Man at Gulmira. "You can see the dents and dirt and dust all interrupting the reflections and breaking up the specular sheen," he says. "That's my idea of photorealistic CGI to the max."
At top, Iron Man hangs onto footage of a Learjet applied to a CG model of an F-22. At bottom, CG F-22s, replaced aerial footage of a MIG and an F85, and real clouds interspersed with CG clouds helped make the environment believable.
In addition to extending live-action footage to create Gulmira, The Orphanage handled a weapons test seen at the beginning of the movie in which their CG cluster missiles blow up a mountain. For this, they again used a combination of Maya, Mental Ray, and 3ds Max, with FumeFX fluid-dynamics software and AfterBurn (both from Sitni Sati), and The Foundry's Nuke for compositing. In addition, The Orphanage artists helped Stark fire the RT in the Mark III's red gloves using a series of volumetric lights and particles, and created Iron Man's heads-up display (see "Heads Up!," pg. 14).
During Iron Man's flight, he finds himself in a dogfight with a pair of F-22s, a sequence ILM created using footage of a MIG and an F85 that Nelson had a cameraman shoot from a Lear jet, and stills that he shot of F-22s on the ground. ILM replaced the MIG and F85 with its models of F-22s, using the footage for animation and lighting reference. "With the CG planes, if we cheat the choreography and if the cheat is toward the camera, the smoke trail gives it away," says Hickel. "So, the animators set up a simple contrail in Maya."
Sometimes the environments and the clouds are live action, but often the action moves through CG clouds and CG environments that helped the team accomplished otherwise impossible shots. To create CG environments for lighting Iron Man, Snow shot HDRIs from the top of a mountain in the middle of the desert.
Similarly, during the final battle sequence between Iron Man and his nemesis, Iron Monger, ILM's virtual background team played an important role, allowing Favreau to reframe shots long after principal photography. "We repurposed more footage on this movie than I have ever done before," Nelson says. "We didn't stop changing things until we couldn't change them any more."
Stosky's digimatte painters created the enormous Stark Industries campus, including the freeway and a rooftop where much of the final battle takes place, working from plates and McBride's concept art. During the battle, the crew wheels Jeff Bridges around the set inside a practical "teapot," the top half of the Iron Monger suit, which ILM enhanced and, when the shots demanded more action, replaced. "Our creature dev people built in a lot of secondary motion," says Snow. "You can see bits moving inside."
Digimatte painters at ILM created the Stark Industries campus where the final battle takes place by adding water to original footage of an industrial park, changing and adding buildings, and removing trees from a median strip on the freeway.
To create lightning-like electrical effects during that battle, Smythe's team developed a procedural system based on new hair tools created by the R&D department that let artists place lightning arcs that could split and rejoin anywhere as directed. "Creating art-directable, animatable electricity has always been a problem, but this time, I think we nailed it," says Snow.
With effects such as those, the inventive brushed-metal shaders, and the new and improved iMocap technology that makes believable digital costuming possible, ILM has thrown down a technical gauntlet with this film. And, Iron Man's focus on realism by Favreau that Nelson and all the studios helped realize presented an artistic challenge.
"At the beginning of the movie, I knew it would make our job harder to work this way, but, ultimately, it would make it more satisfying," Nelson says. "The spectacle doesn't come from something that can't be done. It comes from doing something that can be done, and doing it well."