AFTER Tony Stark (Robert Downey Jr.) crashed inside Digital Domain’s Mark 42, he dragged the damaged suit to a garage and parked it there.
“Sometimes the best-laid plans…” begins a saying that always implies a disastrous or, at least, unexpected ending. That was often true for Tony Stark, the protagonist in Marvel’s latest superhero action/adventure Iron Man 3 who would call for his metal suit and have only a glove or a boot arrive, or have it land in pieces beside him.
No matter. As always, Tony Stark (Robert Downey Jr.) handled the problem with aplomb, saved Pepper Potts (Gwyneth Paltrow), and sent the film into the box-office record books to the tune of $175 million opening weekend in the US – the second-highest opening ever. After the first week, the film’s worldwide box office totaled more than $600 million. Shane Black wrote and directed the Disney Pictures release, which stars, in addition, Don Cheadle as Rhodey, Guy Pearce as Killian, and Ben Kingsley as the Mandarin.
But, Tony Stark wasn’t the only one associated with this film who had to think quick and deal with unexpected events. Unforeseen changes peppered postproduction’s best-laid plans, as well.
“We had a few unfortunate instances,” says Chris Townsend, who led visual effects teams around the globe that created much of the action in the highly acclaimed blockbuster.
Principal photography started a month later than originally planned.
During production, Robert Downey Jr. broke his leg, so the filmmakers took a six-week hiatus. As a result, the postproduction schedule shrank from 28 weeks to 20 weeks.
Then, the 800 shots originally planned for more than doubled.
“Once we wrapped photography, we had 19 or 20 weeks to do over 2,000 shots,” Townsend says.
Complicating that problem, Digital Domain (DD), the cornerstone vendor, and Fuel VFX, another vendor on the show, filed for bankruptcy. Animal Logic rescued Fuel, but DD had a bumpy road. “Very unfortunately, we had to scramble to find companies to take up work they couldn’t do,” Townsend says.
And, as if Townsend needed another challenge, Director Shane Black was new to visual effects. “It was a perfect storm of events,” Townsend says. “The complexities of making an Iron Man film or any film of this size and grandeur are immense, and we had complications because the director hadn’t been there and done this before. Shane is talented and smart, but the only movie he had directed before was a small movie, Kiss Kiss Bang Bang, so it was a journey for him. To get the picture he wanted, the studio wanted to make, Robert [Downey Jr.] wanted to make, and mold the picture into a cohesive, fun film was exciting. We worked up until the last minute.”
To meet the deadline, Townsend and Visual Effects Producer Mark Soper drew on small, nimble studios and small teams within large studios, as well as the larger, more muscular facilities.
“We had 17 vendors, one of which was a handful of artists working in-house to do compositing work,” Townsend says. “It was a crazy amount of vendors, but it was required because of the sheer volume of work, the short time frame, and the complexity.”
At Cinesite, where a team of 18 artists created 100 shots in nine weeks toward the end of production, Visual Effects Supervisor Simon Stanley-Clutterbuck joked, “Chris had more vendors than we had crew on the film.” Cinesite artists produced digital environments that extended plate photography leading into a final battle created at Weta Digital and fireworks after.
The original plan was to have Digital Domain handle several major elements, including the final battle. “I wanted the skill set and look they brought to Real Steel,” Townsend says. “But they were struggling. When they started having the troubles, Mark Soper, Victoria Alonzo at Marvel, and I met, and we decided not to pull the work from them. It was a difficult decision from a business point of view. You can’t sacrifice the film; you have to make sure it gets what it needs. But we also needed to consider what was good for the visual effects industry. It made sense to stand by them, support them, and not pull the work, and that’s what we chose to do.”
Under Visual Effects Supervisor Erik Nash’s leadership, artists at Digital Domain created 300 shots, of which approximately 250 stayed in the movie, and they made suits for the final battle. But, when the final battle alone doubled in size to 500 shots, it became more than the besieged studio could handle.
“The production delays and the other factors put us in a bind where we couldn’t deliver that sequence on what was already an aggressive post schedule,” says Nash. “Thankfully, Weta Digital had the bandwidth and can expand rapidly. They took a lot of pressure off us.”
“Weta Digital was a cornerstone vendor, but they weren’t even involved until the last few months,” Townsend says. “They rose to every challenge.”
Fun with Previs
For Townsend, the work began in August 2012, with early drafts of the script. “We went straight into prep mode,” he says. “The beauty of my job happens when I’m integral to the process from the beginning. It’s an honor to be that close to the director, the production designers, and the rest of the team.”
Artists working with the production team had created animatics for two key sequences: one in which the villain blasts Tony Stark’s house off a cliff, and another in which individual pieces of Tony’s suit fly off a workbench and attach to his body.
“We brought on The Third Floor to do the previs,” Townsend says. “You can get a sense of drama and energy from the animatics, but we need to make sure everything works and the camera positions are logical in the real world. We need to know exactly what the camera can see.”
It was clear from the beginning that the house-attack sequence would rely heavily on computer graphics, and after analyzing the animatics, Townsend realized it needed to be fully digital.
“We couldn’t photograph Point Doom in Malibu [where Tony Stark’s house is located] because of an ordinance,” Townsend says. “So, we had to re-create it digitally based on stills and previous films’ footage.”
As for the “suit-on” sequence, the challenge was in turning an approved animatic into a workable plan. “So, previs worked on the camera and lenses, how the pieces would fly around and attach to Tony’s body, where he would stand, and how he would perform,” Townsend says.
But, rather than traditional previs, which often looks like a video game, Townsend tried something new. “I had The Third Floor artists work in black and white, in an illustrative style,” he says. “In my experience, sometimes directors and producers dislike previs because it has a cartoony look. Also, sometimes people fall in love with a temporary look, and suddenly everything from lighting to directing the camera has been established by a previs artist, not a cinematographer. So, I instructed The Third Floor that I wanted previs to be about line and movement, not color and lighting. By having them work in a black-and-white, impressionistic style, the previs didn’t look like a CG world. We worked closely with the director of photography and the director, and they embraced it somewhat. It wasn’t people in a dark room creating and then telling the director, ‘This is what you’re shooting.’ ”
It worked. Perhaps, too well.
“The marketing folks said [the suit-on sequence] would be great for Comic-Con,” Townsend says.
But, Comic-Con was five weeks after principal photography started. To fit the digital suit parts around Robert Downey Jr., the visual effects studio needed his performance.
“I told them it would take months and months of work,” Townsend says. “They said, ‘We really want it.’ ” For the suit-on sequence, Townsend went to Trixter, based in Munich, Berlin, and Los Angeles. For the house-attack sequence, he called on Scanline artists working in Los Angeles and Vancouver.
“I had worked with Trixter several times before and knew they could react quickly and work efficiently,” Townsend says. “And, I was impressed with their animation. That’s where their strength lies.”
TRIXTER artists animated pieces of the new Mark 42 suit as they attached to Tony Stark.
Trixter
In the sequence, we see pieces of the new Iron Man 42 suit on Tony Stark’s table. Robert Downey Jr. strikes a pose, flicks his arm, and expects the suit pieces to fly toward him and attach properly to his body. It’s a new suit, though. It doesn’t quite work the way he expects.
“He bounces all over the place and then finally lands in his three-point Iron Man pose in his suit,” Kraus says. “And then the last piece kicks him off the platform and all the pieces fall off his body.”
To meet the Comic-Con deadline, Townsend persuaded the studio to film Downey Jr. in this sequence first. “He had dots all over him, witness cameras surrounding him, and it was a very complex scene,” Townsend says. “We’d tell him to mime a piece coming onto his leg and to lift it one way, then he’d need to duck to avoid another piece, then stand up. The choreography was complex, but he was great. We shot for three days and then shipped the footage to Trixter, where they tracked, painted out wires, dots, and cables, built assets, animated, and added lighting based on HDRs. They did incredibly complex work in a short time.”
At Trixter, Alessandro Cioffi was the visual effects supervisor and Simone Kraus, the animation supervisor. Trixter’s main pipeline includes Autodesk’s Maya, The Foundry’s Katana and Nuke, and Pixar’s RenderMan. For trailing smoke and other VFX, they used the FumeFX plug-in from Ideate for Autodesk’s 3ds Max and The Chaos Group’s V-Ray for rendering, creating holdouts through Alembic caches.
“Comic-Con was in July,” Kraus says. “That gave us five weeks of production. So, to get agreement before we could begin production, we started on concept animation in early April.” Once they received approval for designs on paper, they moved the concept into Maya and, with further approval, into production.
“We had a design for the Mark 42,” Cioffi says. “But from there, we had to design how the single parts would go from non-distinguished shapes while flying onto Robert Downey Jr.’s body, connect, and assemble into the final suit. It was frantic working on these assets to create the shots. We were doing compositing on some shots while some assets were still in modeling.”
After meeting the Comic-Con deadline, they started over. “We could partially use the assets,” Cioffi says. “But we had to do heavy modifications for the final release. We had new versions of every part of the rig for every asset. A good 65 percent of the shots had to be redone almost completely.”
In addition to the suit-on sequence, Trixter artists animated the suit in part and in whole for other shots, including a “date night” sequence that involved more subtle animation. In that sequence, Pepper finds Tony in his Iron Man suit sitting on the sofa. She tries to get him out of the suit. Instead, he massages – that is, the animated suit massages – her shoulders.
“It was interesting to do this and keep him looking like a man, but not too soft,” Kraus says. “That balance was quite a challenge. We used a human walk as a reference to make it look real; otherwise, it would have been difficult to keyframe.”
In a dream sequence, Tony’s nightmare sends the suit hovering over Pepper, threatening her. He wakes up, shuts the suit off, and pieces fall down. Later, when the bad guys have tied Tony to a bed frame, he calls the suit, but only one glove and one boot arrive. “There are a lot of effects in the glove and boot fight,” Kraus says. “We re-created an entire room because it was too small to have the movement they wanted with a camera crane. They shot a stunt actor in front of a greenscreen. We added the digital version of the room, and put a CG face on the stunt actor.”
Simultaneous Suit Design
At the same time Trixter artists were working on the multiple-part Mark 42 suit, artists at Scanline and Digital Domain were building suits for their shots. “They all had to work at the same time and build slightly different suits,” Townsend says. “So we had three versions of the same asset being built at the same time that all had to ultimately work together and look the same so we could cut from one sequence, sometimes one shot, to another.”
Trixter’s multiple pieces needed to join together, so those artists had to swiftly build the undersuit for the Comic-Con footage. Scanline also had to start early on the house-attack sequence and quickly needed a suit that walked around. Digital Domain could work more slowly to build a highly-detailed main suit.
Trixter’s undersuit design went to Digital Domain and Scanline. “Legacy Effects later created a practical glove and boot based on Trixter’s CG and used their undersuit design,” Townsend says. “Looking back, it was a crazy way to do it, but it was the only way. Everything had to be done immediately. But, everyone was very collaborative. If one company came up with something that looked great, say a deep-red cherry finish, they’d send it to the others. You can’t think too much about this stuff. You just need to run forward.”
SCANLINE artists destroyed Tony Stark’s house, sending pieces of the structure and Tony into the ocean below.
Broken Home
Also running forward were the artists at Scanline who created the house-attack sequence and, like Trixter, needed to provide shots for Comic-Con. The sequence is action-packed. Three helicopters blast Tony’s house with missiles until it tumbles from a steep cliff into the ocean. When the helicopters fire the first missile into the house, Tony sends the Iron Man suit onto Pepper to protect her. He takes it back when she’s safe, but by then, the heavily damaged suit malfunctions. He slides into the ocean with his house, as debris falls into the water around him and lands on the ocean floor. The suit rescues him in the nick of time – and flies him to Tennessee.
“We thought long and hard about who could do the work, and picked Scanline,” Townsend says. “They are one of the best, perhaps the best, at doing this level of destruction, particularly when it involves water. We turned shots over to them even before we started principal photography. Shane [Black, the director] asked how we could turn shots over when we hadn’t shot anything, but they had to create the entire dramatic collapse digitally.”
Like Trixter, Scanline’s first job was creating shots for Comic-Con. “We looked at the previs and picked out 10 shots,” says Bryan Grill, who supervised the studio’s work. Stephan Trojansky was co-supervisor. “Because the shots were all-CG, we didn’t have to wait for plates. That gave us a jump start on creating the assets we would need.”
Over the next year, approximately 60 people in Los Angeles and 80 in Vancouver would produce 250 shots at Scanline. “Two hundred of those shots were in the house-attack sequence,” Grill says. “And every one of those shots had some sort of simulation. Chris Townsend and Mark Soper came here and looked at our reel. They were impressed, but they pointed out things they thought looked good and things they didn’t like. They really put us to the test. Any time there were missile trails and explosions, Chris had video that Dan Suddick, the special effects supervisor, had taken. They’d say, ‘This is how we want it to look.’ The biggest thing, though, was making Point Doom, the mansion, and the surrounding environment look photoreal.”
The team used the studio’s own Flowline for the water, smoke, and fire simulations, and Cebas’s Thinking Particles for 3ds Max for the rigid-body destruction. V-Ray handled the rendering; Nuke, the compositing.
On set, the actors worked on a gimbaled living room set that tilted down. The floor cracked, and the ceiling fell. “We enlarged the crack and changed the timing when Pepper runs across the room in the Iron Man suit,” Grill explains. “There’s a cute moment where the debris is about to crash on Tony and she blocks it and saves him.”
Scanline replaced the glass in the windows and broke it, added the garage in CG, and throughout added more destruction and debris, built the helicopters and fired missiles from them, destroyed a helicopter when Iron Man throws a missile back, tossed a piano into a helicopter to destroy another one, sent cars from Tony’s garage rolling out and down the cliff, built the Iron Man Mark 42 suit and then damaged it with bullet hits, and, finally, dropped Iron Man to the bottom of the ocean.
“He gets trapped on the ocean floor, and the rest of the garage falls on top of him, creating a grave of sorts,” Grill says. “He’s surrounded by rubble, and there’s a leak in his suit. So we had lots of bubble simulation, particulate, raining debris, dynamic simulations of things falling in water. One of the tricky parts was keeping it from feeling like we were looking in an aquarium. We had to play with visibility and light, and added some debris and particulate that felt like it was hitting the camera lens.”
At the end of the sequence, the suit’s glove detaches, grabs Tony’s real hand, pulls him from the rubble, reattaches, and propels Tony out of the water.
“One of the biggest things was making sure the holdouts and motion from Thinking Particles and Flowline matched up,” Grill says. “If we wanted a certain dynamic in a hero piece of debris, we’d have an animator do that and then let Thinking Particles take over, but most of the time, we’d let it run and whatever happened, happened. Getting the resistance, the weight, and the drag right in the water was tricky. But always, as things came closer to camera, they’d move faster, so if they didn’t have the right sense of scale, we’d delete them.”
Diving with Digital Domain
While Trixter and Scanline worked on suits for their sequences, artists at Digital Domain began modeling a main Iron Man suit as well as a dozen or more other suits used in the film. The team also created a key skydiving sequence called “barrel of monkeys,” a sequence in which Iron Man crashes into a snowy forest, and a close-up shot with an “extremis” effect.
In the skydiving sequence, Iron Man rescues people propelled out a hole in the side of an airplane at 30,000 feet by grabbing one person, having that one grab the next, and so forth.
“The skydiving sequence and the house attack were the most advanced in terms of animatics,” Townsend says. “We broke the skydiving sequence down in previs, to base it in reality, as if we were shooting with a real camera, again working in black and white to keep it impressionistic. We’ve all seen footage of skydivers falling out of a plane and looking into a camera of someone falling with them, though. It isn’t exciting. You don’t get a sense of people falling, no sense of distance. So we tried to maintain that sense of urgency and excitement.”
Often stunt actors working on wires against greenscreen perform the action for shots such as these. “When I saw the animatic, I said, ‘Look, I’ve had experience with this,’” Townsend says. “It always looks hokey on greenscreen. Let’s throw people out of plane and shoot it. Then, we’ll solve the other problems.”
The Red Bull Sky Dive Team did the jump with one wearing a costume patterned after the Iron Man suit and a helmet, the others in civilian clothes and military uniforms. For continuity, the crew filmed all the skydivers except the Iron Man character in the plane before the jump. When they jumped, two skydivers with RED cameras on their helmets filmed the others.
“They worked hard to re-enact what we were building in previs,” Townsend says. “We shot them for a week. They did between seven and 10 jumps per day on five days. Each shot was orchestrated.”
Digital Domain solved the other problems: removing parachute packs, adding the background environments, changing the lighting on the characters, creating a sense of distance.
“The biggest challenge was replacing the sky background and the terrain below the divers,” Nash says. “They shot it in North Carolina, but in the story, it happens over the coast of Florida. And they filmed from morning to late afternoon, so the lighting and weather conditions varied dramatically. Also, the free-fall photography happened between 12,000 and 3,000 feet, but in the shot we go from 30,000 feet to sea level. We had to build in that progression.”
Once the artists had created digital backgrounds, clouds, shoreline, and so forth, the harder work began. “There was an unbelievable amount of painstaking rotoscoping work to separate each individual with frantically flapping cloth and hair, and to place them into our synthetic backgrounds,” Nash explains. “I wish there had been some sort of magic application that would have made this easier, but it was just old-school hard work, delicate edge work that was different from one frame to the next. It’s a very tricky skill.”
DIGITAL DOMAIN'S Iron Man Mark 42 suit rescued people falling from a plane.
For each character on each frame, the artists rotoscoped two sets of images, working pixel by pixel. “We did an inside outline and an outside outline,” Nash says. “The inside outline defines the solid part of the figure. The outer represents the outer extreme of the flapping fabric. Because the fabric and hair are flapping at such a high frequency, it creates a kind of transparent edge outside the solid part of the figure. We can’t throw away that transparent boundary area. We have to retain it and put new material behind it. Traditionally, you define a shape that changes a bit from frame to frame. In this case, the outer outline changed radically from one frame to the next because everything was flapping in the wind.”
The rotoscoping and painting took hundreds of man-hours to accomplish. “We had 20 artists working on it internally and two outsourced vendors over the course of five months,” Nash says. “Thankfully, we got that sequence early.”
Once the people are falling through the air, a simulation created at Digital Domain by artists using Side Effects Software’s Houdini blows up the plane.
To build the new Mark 42 that Iron Man wears in this sequence, Digital Domain artists worked from the practical suit built at Legacy Effects and from concept art supplied by Marvel. “For the skydiving sequence, we developed a lot of flaps and used plates on the suit as air brakes to sell the idea he’s in control,” Nash says.
After Scanline's house-attack sequence, Digital Domain's suit crash-lands in the snowy Tennessee woods. Although there was practical snow in North Carolina for the July shoot, much of the environment is a huge 3D matte painting created at Digital Domain.
“You see the suit skipping across the meadow and into the woods,” Nash says. “It comes to a rest and you see Tony Stark get out of the suit, and he drags it into a nearby small town where it winds up in a kid’s garage.”
When Downey Jr. wore some pieces of the practical suit during the shot, Digital Domain artists added the rest. “But even then, we wound up replacing the practical pieces rather than filling in missing pieces, partly for the proportions,” Nash says. “The idealized Iron Man doesn’t really match the practical suit. So we usually wound up replacing it all, keeping the face visible. It was a point of pride, as we got our look development done on the new Mark 42. It was the one with the most screen time. Chris shared our turntables and look development images with the other vendors and told them to make it look like that. That was nice. We’ve gotten good at metal men. We used the same rendering pipeline as on Real Steel.”
To create the suits, modelers worked in Maya, texture artists in The Foundry’s Mari, and lighters in the in-house software Atomic. V-Ray rendered the shots.
DIGITAL DOMAIN artists added the glowing “extremis” effect to this live-action actor. Other studios added the effect in other sequences.
Extremis
The third key sequence for Digital Domain involved the so-called extremis effect that infects one plant and several characters in the film.
“It’s a superdrug that turns a human into a superhuman who can re-grow limbs and heal injured parts of the body,” Townsend says. “A side effect is that it makes people glow from within and, occasionally, explode. We worked closely with Digital Domain and, particularly initially, Framestore to create a look that was complex and volumetric. We wanted one company to do all the extremis work because it was so specific, but with the time constraints and various bankruptcies, we ended up with six companies working on the extremis effect. But, the hero look was the one Framestore created.”
Digital Domain applied the extremis effect to a character with a broken nose. “It heals in front of our eyes when an orange glow comes up from the core,” Townsend says. “Hopefully by the end of the movie, you don’t question it.”
Method artists also used extremis to re-generate a branch of a plant early in the film, open the Iron Patriot suit, and melt the girders holding up a water tank.
“We came on the show toward the end,” says Matt Dessero, visual effects supervisor. “We got plates on January 8 and finished 80-plus shots March 15. It was a busy two months.”
In the film, the character Happy Hogan [Jon Favreau] rips the limb off a plant injected with the extremis chemical. Method artists working in Vancouver used Houdini to re-generate the limb from the main stalk. Then, they exploded the plant and disintegrated everything around it. “It’s only two shots,” Dessero says, “but they’re cool shots. We modeled a wall in the room but went with as many 2D and practical elements as we could. The wall is singed. We have embers and burning ash floating in the air, and little fires all around it. Our big challenge, though, was locking extremis to Killian in another shot.”
AT LEFT, Method artists re-generated the limb of an extremis-infected plant. At right, the artists melted a girder, toppled a water tank, and created digital water.
Killian, played by Guy Pearce, is the primary bad guy in the film and dangerously infected with extremis. In the sequence Method worked on, Tony Stark’s friend Rhodey (Don Cheadle) is inside the former War Machine suit, now painted red, white, and blue and named the Iron Patriot. Killian has captured him and he wants Rhodey out; he has other uses for the suit. Killian uses extremis to heat the suit until it automatically opens.
Working from an Iron Patriot suit model provided by Digital Domain, the Method artists added geometry and textures where needed in particular shots. “We had a limited amount of time,” Dessero says. “We built only what was necessary for the shots. We couldn’t have everything happen at the end because we were already at the end.”
Animators began with a camera track and a matchmove of Cheadle’s performance. “We roughed-in the timing with a few simple exterior shapes for arms and the head,” Dessero says. “Once we had approval on the primary animation, we added more geometry and complexity when the suit opens up. There are lots of moving pieces, all hand-animated.”
To infect Killian with extremis, the artists built and rigged a digital double based on a cyberscan of Pearce. Then, they matchmoved the actor’s performance using custom blendshapes per shot to lock the matchmove. Effects artists working in Houdini created extremis using layers of glow maps and a pulsing noise through the body.
“The extremis look had been established,” Dessero says. “Chris [Townsend] gave us target images to match. Because Killian is angry and at the peak of his control over extremis, we added skin blisters that come and go as the skin heals.”
To have the extremis glow hit and reflect on the metal suit, compositing artists and lighting artists worked together passing render layers and composites back and forth to achieve a final look.
For a third sequence, Method artists helped Savin (James Badge Dale), an extremis-infected character, melt a girder supporting a water tower, causing it to crash to the ground and destroy a construction trailer. “They shot a 100-foot-tall water tower on set that we used for base geometry,” Dessero says. “Our task was to create a wide establishing shot and collapse the tower as Savin heats up the leg, and send CG water under the camera. Chris [Townsend] scanned every set and gave us beautifully prepped data with HDRs, Lidar scans, and measurements. We built a full-CG tower, ran water simulations, rigid-body simulations, and had a ton of animation.”
To melt the girder, the crew matchmoved a practical plate of Savin touching the girder, then inserted a CG girder and ran a viscous fluid simulation. “The metal starts to sag and embers spark,” Dessero says. For the tower destruction, modelers pre-scored the geometry and then used rigid-body dynamics from within Houdini to break it apart when it crashed to the ground.
“Chris filmed some beautiful water tank shots with multiple cameras,” Dessero says. “We added debris and part of the construction trailer in the water. The water under the camera is all-CG, simulated with Houdini.”
WETA DIGITAL built 36 iron suits for the final battle from 10 hero suits.
Extreme Weta
These shots all lead to the finale created at Weta Digital, where an army of CG iron men fight CG digital doubles filled with the extremis effect.
“When we talked to Weta, they were finishing The Hobbit, so they had hundreds of people available,” Townsend says. “We gave them the least-developed 500 shots and the last we would shoot. Their deadline was the end of March. So we needed a heavy hitter. They had a tremendous amount of work.”
The big battle takes place in a seaport around giant container cranes. The evil extremis mercenaries have captured the president and plan to sacrifice him during a television broadcast. Tony Stark arrives with his own army of iron suits.
“We had to hit the ground running,” says Matt Aitken, who, along with Guy Williams, supervised the visual effects at Weta. “We have suits flying around, taking on the extremis mercenaries. We have a lot of destruction. And, I think we might have had the tightest deadline ever. We didn’t have plates until mid-December.”
To lace the digital doubles with the orange, glowing extremis effect, Weta relied on their “genman” rigs (see “Of Gollum and Wargs and Goblins, Oh My!,” January/February 2013) configured with a skeleton, muscles that drive the skin, networks of veins, more finely detailed muscles, and organs. “We needed a correct representation of what it looked like to have an internal glowing energy source,” Aitken says. “We used a technically correct approach to lighting, raytracing with interior light sources, which really paid off. The light sources are just like another light. The geometry provides the complexity and visual detail. We didn’t get too anatomically correct – we didn’t want it to look gruesome – but, we illuminate biological detail: the muscles, veins, layers of skin, the interior of the body. We had the extremis effect in hundreds of shots, and it was largely automated. If we had created it by hand, we’d still be compositing.”
For Tony’s remotely controlled iron suits, the artists began with 10 hero models from Digital Domain, and then derived additional suits from hero components, on which they varied the textures to create a total of 36 suits. A new shading system dubbed “Gen2” provided energy conservation between layers on the surface so that the aluminum, emulsion, and clear coat have a different response based on the effect each has on light passing through the layers.
The lighting artists also called on Weta’s new real-time “pre-lighting” tool called Gazebo (see “Shaping Middle-Earth,” January/February 2013) to help speed animation approvals. “We used Gazebo to render animation for our client reviews,” Aitken says. “We could render shots in stereo while we were still developing final lighting for the shots.” Final rendering was through Pixar's RenderMan.
Aaron Gilman led a team of animators who performed the suits, the army of extremis mercenaries, the moving cranes, and even the destruction. “It was a massive aerial battle with mercenaries jumping off cranes, hand-to-hand combat, blaster fire, suits thrusting all over the background,” Gilman says, “32 suits beating the crap out of mercenaries and getting the crap beaten out of them. We had previs for shots with hero performances, but nothing that really defined the nature of the aerial battle – how many suits, where they’d fly, how fast. We needed to choreograph those performances.”
Once the battle begins, everything is digital, and within this chaos, the animators had several basic challenges. They needed to make sure the audience knew the mercenaries could win. They needed to have interactive performances between the mercenaries and the iron suits; they couldn’t just have suits flying around. And, they needed to create a battle that didn’t detract from the star performances.
“We needed to do the opposite of hero performances,” Gilman says. “We had to learn how to design performances that didn’t take center stage. Guy Williams told us to take a camera, stick it on a tripod, and put it in the middle of a school courtyard at lunchtime. That’s what we wanted; we wanted the viewer in the middle of the fight. And, we wanted to tell the story without drawing the viewer’s eye, so we broke a lot of rules in terms of composition, staging, and timing. We broke the rule of thirds. Sometimes you see only the thruster boots of a suit.”
The fundamental challenge, though, was that Tony Stark would jump from suit to suit. That is, as he ran, jumped, and fought, various suits formed around him in turn. “He’d do these skydiving moves and a suit would fly around him,” Gilman says.
That created a pipeline problem: The animators had to break apart the suit model – change the geometry – to fit it around Downey Jr. in the transformation shots. But, traditionally, animators work with a puppet that references a high-res model and textures that live elsewhere in the pipeline.
“We can’t change the topology on the global model,” Gilman says. Thus, the animators would have had to show modelers what they wanted, and wait for the modeling and creature departments to create the assets they needed before they could animate each transformation.
“It would have been incredibly cumbersome,” Gilman says. “So we established a guide rig system that circumvents the traditional workflow. We could work on a non-referenced puppet, so animators could block all the animation without dealing with the other departments. We could generate controls on the fly. A chest panel could become four or five pieces that subdivide, slide, and break apart. Animators could do that on the fly without leaving their shots.” Because the non-referenced guide rig had high-resolution panels attached to the joints, animators could be certain their cuts would work on the final models.
The guide rig also made it possible for an animator to separate limbs from the puppet without creating two separate puppets. And, they could damage the suits without sending the suits to the character effects department.
“It gave us an enormous amount of control,” Gilman says. Once approved, the guide-rigged animation made its way into the standard pipeline, but having this alternate way of working made iterations on the way to approval faster and easier.
Animators were also able to art-direct much of the destruction. “We built on the work we did for the destruction sequence in Tintin,” Aitken says. “We set up events so they are physically believable and directable. A lot of work has gone into enabling animators to run simulations. Animators have simulation tools, so they can run explosions and rigid-body dynamic simulations. And, if they drop an object, it falls with gravity. A shipping container knows how heavy it is.”
In one series of shots, for example, an enormous crane collapses after an explosion breaks the moorings that held up the main deck, the boom arm. Animator Julia Chung created a 600-frame animation of the crane toppling from beginning to end. “It looked like a simulation,” Gilman says, “but she animated it by hand.” Once Williams, the supervisor on the shot, and Marvel approved the animation, the animators gave the master file to the effects team to run the simulation.
At the peak of production, close to 40 animators worked on the shots, with the majority starting in mid-December. “We did 420 shots, a 20-minute battle, in three months,” Gilman says. “That is not to be believed. But with our new technology development, it was a ride.”
SEVERAL of the 17 studios that worked on the film created Iron Man’s Mark 42 suit. VFX Supe Chris Townsend made sure the suits looked the same and the damage was consistent even though different lighting and rendering packages were used.
…And More
“Framestore did a major sequence with Iron Man confronting an extremis character,” Townsend says. “Luma did great suit work. They did shots with Iron Patriot looking for Tony, creating full-digital versions of the suits that matched and intercut seamlessly with practical suits. Cantina did all our HUDs, taking those one or two steps further than the work they had done for previous films. Fuel did a couple sequences with holographic images. One was in Tony’s war room where they created a wall of graphics as a hologram that was really interesting. They created something new but that related to what we accept and know. They also did a massive holographic brain that our characters get inside. It looks magical and beautiful.”
And, in addition to the beginning of the end battle, Cinesite did car comps. “I detest doing car comps,” Townsend says. “They are one of the hardest things to do in visual effects and often done badly. You have nodding heads in a car and you can tell it’s greenscreen outside. But they had light changing over faces, shadows, reflections, little camera bobs and weaves. They did a great job.”
Throughout postproduction, it was Townsend’s job to maintain consistency among the many vendors, making sure extremis created in one studio matched extremis in another, that the Mark 42 looked like the same suit in every shot, albeit with increasing damage, despite the fact that the studios used different textures and renderers. “Of course they do,” he laughs. “I wouldn’t have it any other way.”
“It was very long hours for a long time,” Townsend adds. “In the final three or four months, I had one day off and I was working 14-, 16-, 18-hour days. All the visual effects companies were working crazy hours to get what they needed in. It was brutal. But, the production team was incredible. Everyone at Marvel was full on. I saw the president, who is also a producer, in editorial at 2:00 in the morning.
“There was a moment five weeks before the end of our postproduction schedule. I was on a FaceTime call with my 12-year-old daughter at 9:30 on a Wednesday night. My production manager poked her head in the door and said, ‘We’re going to a bar to celebrate.’ I told my daughter I was going to go have a drink because we had just finaled the first 1,000 shots. We were halfway through. She said, ‘Dad, you’ve been on the show for a year and a half and you only have a few weeks left.’ I said, ‘Yeah. That’s why I’m going to have a drink.’ We had five weeks to do full-CG environments, assets, digital characters, full-screen in-your-face shots – tremendously hard shots. But we had 17 companies working furiously around the clock. Because we had spread the work and shared assets, theories, and techniques, we were able to get it done on time, on schedule. We finished on a Saturday night at 1:00 in the morning. It was incredible.”
Barbara Robertson
is an award-winning writer and a contributing editor for CGW. She can be reached at BarbaraRR@comcast.net.