It’s all about growing up, really, isn’t it? Harry Potter and his wizardly friends in the series of books by JK Rowling. The actors in the seven film versions who have aged in concert with the characters they play. And, no less, the visual effects crews who have been making screen magic for the past 10 years.
Think of the effects in the first Harry Potter film. They may have been state of the art at the time, but by today’s standards, they seem almost, well, if not childish, then “tweenish.” Back then, there were no GPUs for accelerated graphics, no 3D compositing programs, no Mova for real-time facial capture, no subsurface scattering. And the list goes on.
“Effects have definitely gotten more sophisticated in the 10 years or so since the first film,” says visual effects supervisor Tim Burke, who had just completed work on Warner Bros. Harry Potter and the Deathly Hallows: Part 1, his sixth trip to Hogwarts. Except that Hogwarts doesn’t appear in this film.
“Harry and his friends travel in search of Horcruxes, and they end up in a chase the scale of which we wouldn’t have taken on before,” Burke says. “From animation to environments, it was a huge task. And the quality of the characters—the tools the animators are using now give such realism.”
Many of the same UK-based studios that had worked on previous Harry Potter films drove the effects in Deathly Hallows, along with Rising Sun from Australia returning as well. Industrial Light & Magic (ILM), which had worked on the previous six films, dropped out of this one. “For various reasons, when we put it out to tender again, things switched around,” Burke says.
For example, Cinesite rather than The Moving Picture Company (MPC) reshaped Voldemort’s nose, and Rising Sun rather than MPC created the Ministry of Magic (for an in-depth look at this work and more, see the online feature “A Magic Touch” on www.cgw.com). Perhaps this was because the MPC artists had their hands full dealing with one of the largest and most complex sequences.
Of the 170 shots created at MPC, more than half take place during this sequence.
MPC: Seven Harry Potters
The MPC sequence begins with Harry Potter (Daniel Radcliffe) in the Dursley’s house on Privet Drive, where he had lived in a cupboard under the stairs as a child. He’s on the run from Voldemort and that evil wizard’s soul-sucking Dementors. To Harry’s surprise, six friends appear armed with a plan for getting Harry to a safe house. “Their technique was to use a Polyjuice potion to turn everyone into Harry Potter,” Burke says.
The magic happens quickly in the film, but at MPC, 15 artists worked for five months on the transformation shot. Because director David Yates wanted something unique, the artists couldn’t do quick morphs from the various actors’ faces into Dan Radcliffe’s face. “He wanted to keep them in the process until the camera comes all the way around,” explains Nicolas Aithadi, visual effects supervisor at MPC. Only when the camera has circled and moved behind Harry Potter, giving us his point of view, do we see six identical Harry Potters looking back. Before that, we see hybrids.
Working from live-action plates of the actors, MPC’s artists began by creating target concepts for each of the hybrids. “The artwork ranged from subtle to cartoonish,” Aithadi says. “Because Tim [Burke] went for something more subtle, our next step was to create CG humans for all the actors and do the transformation on the digital double.”
This chase sequence, which takes place in a digitally altered tunnel, used a combination of real and digital vehicles, digital Death Eaters, the actors, stunt doubles, and digital doubles, all created at The Moving Picture Company.
To create the CG doubles, the crew decided to use Mova’s Contour System and capture the actors’ faces as they took breaks from filming. Before making that decision, though, they tested the system for a month. “We tried to break it, but it worked really well,” Aithadi says.
On set, each of the actors stepped into the Contour rig and acted out a transformation. Some acted surprised, others expressed pain. Mova then processed the data and sent it to MPC, where riggers applied it to a facial rig.
“The facial features were really accurate,” Aithadi says, “And, the skin deformation. The only problem was that they didn’t give us the eyes, so we ended up with a very good face and dead eyes that didn’t blink.”
And, the crew discovered that the accuracy demanded was greater than anyone expected. “We underestimated how important the eyelids were,” Aithadi says. “Because we were mixing characters together, every piece had to be good. It isn’t so much of a problem when you do one face—one George or one Harry. But you have to have good eyes to register Harry’s eyes with George’s nose. If you don’t fold the eyelids right, the eye looks weird. And the eyelashes. One millimeter can make a huge difference.” Blendshapes added to the rig gave animators the means to add blinks, squints, twitches, and eyelid folds to, in this example, Harry Potter’s (Radcliffe’s) eyes placed above George Weasley’s (Oliver Phelps’) nose.
The facial skin tones also had to be accurate, though most of the actors had darker skin than Radcliffe. “We changed the skin shader we usually use to get something that looked good close to camera,” Aithadi explains. “Then it was a lot of trial and error to find out which shades worked best.”
Once they had created a good blend, they deformed the faces. “Fleur [Clémence Poésy] was hardest,” Aithadi says. “She’s too perfect. We didn’t have much to grab onto.”
After the transformation was complete, to create the illusion of seven different personalities “inside” Harry Potter’s face, Burke had each actor play out each role as if they were Harry Potter. Then, Radcliffe copied those performances. “He did a fantastic job of picking up all their characteristics,” Burke says. “We shot him on a greenscreen set with motion control, and composited all the personalities together.”
With the transformation complete, and the split screens composited, MPC mounted the six fake Harry Potters on brooms and Thestrals, and slipped Harry into the side car of Hagrid’s (Robbie Coltrane’s) magical motorbike.
ILM, which had previously created the skeletal horse-like Thestrals, provided MPC with the original models and textures, which MPC modified to accommodate two riders rather than one. On set, the actors rode real horses that MPC later replaced with the CG animals.
As they take off, the point of view is Harry’s, and he sees chaos. “We have maybe 100 Death Eaters and a dozen good guys,” Aithadi says. “Everyone is fighting. And the sequence happens within a big aerial environment.”
To create the environment, the team analyzed all the shots and divided them into backgrounds, mid-ground, and foregrounds. For extreme backgrounds, they produced matte paintings. For the mid-ground, they flew digital doubles through clouds created with Autodesk’s Maya fluids rendered in Mental Images’ Mental Ray, and through volumetric clouds rendered in Pixar’s RenderMan.
“RenderMan gave us better detail for flying through clouds,” Aithadi says. “The details define the distance. From a distance, a cloud is fluffy, but when you fly through, it’s wispy, so we had to find ways to do that transition. When Harry flies away from Privet Drive, the environment is calm, serene. But, when he goes through a cloud, it’s hell. It’s a World War II extreme dogfight.”
Inside the clouds, the young wizards—digital doubles from afar and live-action elements when close—and the evil, digital Death Eaters fire spells at one another, and the spells light the volumetric clouds. Simulation times for the cloud volumes ranged from 10 to 20 hours per frame, with rendering often taking two days. The crew rendered the clouds with passes that had different directions for lighting, and then composited the shots in The Foundry’s Nuke. “We didn’t want to get a cloud plate and stick people in there,” Aithadi says. “We really wanted to play with the light and the depth. Nuke allowed us to create more of a 3D feel with the 2D elements as well. The compositors became more like environment technical directors.”
Road Rage
To escape, Hagrid uses dragon fire, a massive burst of CG flame nearly a mile long that rockets the bike and sidecar. To create it, MPC used Scanline’s Flowline. “It’s the longest simulation I have experienced in my life,” Aithadi says. “It took ages to simulate, maybe a week and a half. It’s extremely detailed.”
The camera moves into the fire, and when it emerges, we see Hagrid and Harry speeding on a real-world highway toward the Dartford Tunnel outside London. MPC built the environment in CG using photographs of the entrance and the tunnel. “The only real things are the cars and the tarmac,” Aithadi says. “They closed the tunnel for two nights so we could shoot stunt cars. We also shot stunt doubles dressed as Hagrid and Harry swerving between cars on an airfield dressed like a highway.”
In one shot, a Death Eater fires at Harry, misses, and hits a car pulling a caravan (trailer). The caravan spins, the car loses control, and they barrel roll down the highway. MPC created that stunt entirely in CG. “We don’t usually have these kinds of effects on a real-world highway with real people in a Harry Potter movie,” Aithadi says. “It was quite fun.”
To make the action inside the tunnel look more dangerous, MPC narrowed the four lanes to two by rotoscoping out the live-action cars, removing the walls, re-creating the walls digitally, and then replacing the cars. At one point, a CG Hagrid drives a CG bike up the walls and along the ceiling to avoid hitting a bus and truck, and a real Harry—that is, a bluescreen element of Radcliffe—falls out of the sidecar and hangs by his hands until Hagrid rights the bike.
“When we shoot live action with Hagrid and Harry, they’re never together,” Aithadi says. “We had a tall stunt man in an oversized costume. We had Robbie Coltrane in a costume to make him seem bigger. We had bikes in different sizes, [including] a small one to make Hagrid look bigger. We did a lot of face replacement.”
As they leave the tunnel, Hagrid is injured, and one Death Eater still follows them. Harry is driving the bike from the sidecar, so his owl Hedwig attacks the Death Eater, which shoots her. When she dies, Harry screams, and Voldemort arrives in a cloud of black smoke to finish him off. “The smoke is evil, dark, and weird, and it creeps into the shot,” Aithadi says. Harry faints. But, as Voldemort sticks out his wand, Harry’s wand reacts for him, and the two spells collide in a massive ball of fire and energy.
Shots such as this, for which MPC turned six actors into Daniel Radcliffe clones, helped overall VFX supe Tim Burke and MPC VFX supe Nicolas Aithadi earn Oscar nominations in January 2011 for Best Achievement in Visual Effects.
“As with the clouds, we used a Maya fluid volume driving particles rendered in RenderMan,” Aithadi says. “We used whatever we had in the bag, really. The camera travels along the stream of the Voldemort spell, so we couldn’t stick with one thing. Volumes are good but don’t give you enough detail when close. Particles have more detail, but they don’t give you a fluffy feel, so we mixed them with elements. And, the environment artists created geometry with animated textures. We gave all that to compositors, who combined the big jigsaw puzzle.”
The crew created the outside fireball by simulating volumes. For the center of the fireball, they used volume simulation and geometry textured with elements, like water and fire, distorted to feel more magical. A mixture of water and fire particles added an organic feel. And, at the end, a massive volume simulation with particles rushes toward the camera.
When Voldemort realizes he’s defeated, he screams and emits a shock wave that destroys all the electrical pylons around him—in the real world. MPC’s environment artists created the scene by uprooting the giant pylons and adding electrical bolts and flying cables. And all the lights in the Muggles’ world go out.
Hagrid and Harry use the bike’s dragon fire again to escape, and they crash-land in a swamp near the Weasleys’ house. Double Negative (DNeg) created this environment; MPC provided the CG bikes, CG Harry and Hagrid, and CG splashes in the water.
Thus begins the latest Harry Potter film. Indeed, the children in Harry Potter aren’t the only ones who have grown up. You can see the evolution of the visual effects industry through the decade of Harry Potter films. It’s like magic.
Barbara Robertson is an award-winning writer and a contributing editor for Computer Graphics World. She can be reached at BarbaraRR@comcast.net.