Before the wizard of Oz became wonderful, he, too, needed to leave black-and-white Kansas, follow a yellow brick road, determine who was good and evil, and discover greatness within himself. In Oz the Great and Powerful, an imagined prequel to the Wizard of Oz, the would-be wizard is a cunning magician, Oscar Diggs (played by James Franco), in a small-time circus. A tornado (of course) sucks his hot-air balloon inside, and he emerges from the vortex in the colorful but threatened Land of Oz. Can he save it from the wicked witch? Or, is his magic only an illusion?
The e biggest illusions in the film, however, are the Land of Oz itself and Oz’s traveling companions: the monkey Finley voiced by Zach Braff , and a tiny China Girl voiced by Joey King. Sam Raimi, known for Darkman and the Spider-Man trilogy, directed the Walt Disney Pictures production. A crew at Sony Pictures Imageworks provided a majority of the visual eff ects, as they had done for Raimi’s Spider-Man trilogy and, perhaps more akin to this film, Alice in Wonderland.
Visual Eff ects Supervisor Scott Stokdyk, who won an Oscar for the visual eff ects in Spider- Man 2 and Oscar nominations for
Spider-Man and
Hollow Man, was overall supervisor for the film’s approximately 1,500 shots. Imageworks took on between 1,000 and 1,100 shots, including the main digital characters and most of the environments. Artists at Luma created the tornado and a flying action scene between two witches at the end. Digiscope, Evil Eye, and Method artists handled much of the 2D compositing work.
“On this movie, we art-directed every single thing with an eye to consistent detail,” Stokdyk says. “And along the way, there was a massive amount of hand-done, artdirected work. We had massive environments and a massive amount of animation.”
At Imageworks, a crew of approximately 300 created the effects, including 50 animators at peak who performed the two digital stars that appear in the film: the little China Girl and the ff ying monkey Finley. Because these characters appear with Oz throughout the film, and for consistency, it made sense to have Imageworks handle the environments.
“flis is a traveling buddy movie,” says Digital Effects Supervisor Francisco De Jesus. “Oz starts in one place and walks to the Emerald City on the yellow brick road, and all this time you see all the way to the horizon in daylight.” All along the way, the Land of Oz is a fabricated world.
“There was no location photography,” Stokdyk says. “flis film is very art-designed, very, very sculpted. Sam Raimi, Robert Stromberg [production designer], and Peter Deming [director of photography] wanted a stagebased look. A classic Hollywood look. So, they decided to shoot it all on stages.”
The production team chose shooting on stages to create a style based on 1920s films. they wanted to harken back to the 1900 publication of Frank Baum’s book the Wonderful Wizard of Oz, and the 1902 stage play rather than the famous 1939 film. “We tried to build sets for as much as we could and use bluescreens, our modernized version of backdrops,” Stokdyk says. “Our goal was to be stage-based. We fought to get on-set builds and not be in a sea of blue. To a large extent, though, this is a traveling movie, so once you go 200 or 300 feet from one environment to another, it didn’t make sense to build a set. And we couldn’t build a set for only one shot. So we ended up having probably five to 10 allblue environments.”
Artists at Sony Pictures Imageworks created all the fantastical landscapes for Oz the Great and Powerful as well as China Girl (top) and the monkey Finley (not pictured).
Landscape Design
Including the set extensions, the crew at Image works built approximately 50 environments – interiors, exteriors, the Emerald City, the witch Glinda’s courtyard, and many others in a variety of shapes and forms.
“We start in Kansas during the Dust Bowl,” Stokdyk. “I’m familiar with the landscape – I grew up in Kansas – so to add interest, we gave Kansas rolling hills like the Flint Hills [in eastern Kansas]. then, we go into a fantastical landscape, a land of jagged peaks and waterfalls, and into a lush area. All through the movie, we explore different corners of Oz.”
Production Designer Robert Stromberg, who won Oscars for the art direction on Avatar and Alice in Wonderland, created the concept art for the film. To further guide the postproduction crew, he painted over dailies shot on the elaborate sets.
“Robert married the concept art with the set,” says De Jesus. “Because his paint-overs would have gone through Sam [Raimi], they gave us the first solid approved look of a scene or a shot. We paid very, very close attention to those paint-overs. He provided lighting cues in addition to what they built.”
On set, the Imageworks team scanned and photographed each set and collected HDRIs at two heights for each lighting setup. “We had almost 1,000 HDRI pairs at the end of the shoot,” Stokdyk says. “We could fully reconstruct what Deming lit for any set. But, you always art-direct a bit away from that or put special lighting in to make it look more artistically beautiful. On set, the DP is concerned with making a shot look beautiful. Real is automatic. But in visual effects, we have a dual challenge. We have to make the shot look beautiful and look real. If we cheat the lighting and put a rim light around our monkey, he is lit more beautifully even though the lighting isn’t ‘real.’ So, it’s something we have to weigh constantly. This is a fantasy movie with a flying monkey in a fantastic world. We wanted it to look real, but also fantastically beautiful.”
Most of those fantastic environments were hybrids: A set for the near-ground, CG extensions for the mid-ground to background, and digital matte paintings in the far background. Working from Stromberg’s paint-overs, a team at Imageworks mapped out the assets the crew would need to build.
The first step was matchmoving. Layout artists positioned the stereo virtual cameras in the same position as the on-set camera, albeit inside a digital environment that matched the outlines of the set. “Layout worked in [stereo] 3D all the time,” De Jesus says. “This film and Spider-Man were our first native shows. Because we had native stereo from day one and had the [digital] stereo cameras from the matchmove, there was no reason not to look at stereo in layout and in animation – sometimes even in effects as early as possible to get a sense of where to play up the stereo.”
Then the layout department looked at the previs for each shot and the images that Stromberg had painted over the dailies. “We had to reconcile how they cheated with what we could do,” De Jesus says. “For example, they might have used forced perspective, but when we had to create the shot in stereo, it would break.”
Next, the layout artists set the stage for the animators, and ordered assets. “Our layout and model departments worked hand-inhand,” De Jesus says. “Ordering props was an ongoing process: Modeling was constantly meeting layout’s requirements.” The final layout with the shot fully set-dressed with upgraded models would happen after animation.
Oz was filmed entirely on stage. There were no exterior locations. Most of the landscapes were hybrids with a set piece in the foreground, CG set extensions in the mid-ground and background, and matte paintings in the far background.
Flowers and Trees
Modelers built all the props to work from any angle so the assets could be re-used, and they made the set extensions from scans of the sets so that the digital replicas would perfectly match. “We always tried to be efficient and build only to camera, but the reality is that this is the director’s playground and directors like to move around,” De Jesus says. “Although many models could be re-used, the texture painting was done more to camera. If we had shots only from a particular angle, we’d paint only those assets in high detail.”
To link the unique landscapes, the visual effects artists populated areas in each with similar assets, and used a similar layout for the ground cover.
“We’d mix and match the assets, but they were the thread that takes us from start to end,” Stokdyk says. “For example, we might have a cornfield next to the yellow brick road, with a unique lighting, layout, and feel. But, the flowers in the foreground would appear in three or four other places, as would the fence posts along the yellow brick road.”
The cloth and hair teams created the ground cover, the grass, and flowers. The layout team managed the trees, with effects adding dynamics. By the end of the film, the teams had created a forest library with 500 unique trees modeled, textured, and rigged.
“We explored using L-systems and more procedural or dedicated software for the trees, but our modelers and texture painters are very particular about the spans and how they want the UVs laid out,” De Jesus says. “It proved to be as much work to change an out-of-the-box tree model as to do one from scratch. Most of our trees were geometry, but for the distant ones, we used textures on flat cards. We kept our renders as efficient as we possibly could.”
Trees not on flat cards had animation cycles that could be scaled up or down in intensity or speed to help bring the landscape alive. “Depending on the shot, the lighting team could adjust the movement with a slider,” De Jesus says. Why the lighting team? “There’s a simple fact that any director can comment on certain details only when a shot is close to completion,” De Jesus says. “The director might have reviewed a shot 10 or 20 times for animation and have the character performance he wants. But only after lighting and compositing can he consider the whole balance of the frame. Something on the right might distract from a character in frame left.”
Having the sets lessened the amount of building the Imageworks crew needed to do, but it also made their work harder. “When you have an all-CG shot, it’s a really well-understood pipeline these days,” Stokdyk says. “You track the actors on bluescreen, track the camera, rotomate to cast shadows. But when you have set extensions, you have to make sure you match-move 100 percent in every part of the frame. You have to figure out where to split the lines and blend the two, especially in [stereo] 3D. If the lighting in part of the set is too hot or out of the sweet spot of lighting, if there’s an errant shadow, you have to replace that part of the set. If something accidentally covers the set, you have to remove it. The more you build of a set, the more challenges there are. You trade the beauty of getting something in camera with the technical challenges in post.”
The bubbles moving across Oz were geometric spheres deformed by the characters inside and then converted into volumes.
Flighty Bubbles
During one sequence, Oz, the good witch Glinda (Michelle Williams), China Girl, and Finley travel inside enormous bubbles across a landscape. “The bubble voyage begins at dawn,” says De Jesus. “They are running from the bad witch, jump off a precipice into the clouds, and when they emerge, they are in the bubbles. They travel on bubbles to her castle, going around, through, above, and below CG clouds.”
With clouds designed to mimic the landscape below, the crew needed to art-direct the volumes. “We worked out a way to turn any geometric model into a cloud volume,” De Jesus says. “The layout team could place the shape and it would turn into a full-on volumetric.”
The bubbles were spheres deformed by the characters inside when they pushed against the surface with their hands and feet. “Glinda is elegant and poised, so her bubble has that character,” De Jesus says. “Oz stumbles and bounces around inside his bubble.”
De Jesus considers these scenes with characters inside bubbles to be one of the technical achievements on the film. For reference, the crew took stills and videos of giant soap bubbles several feet across in various lighting conditions. “On past shows, we used our volume renderer SVEA and we had to do a lot of cheats. Sometimes effects TDs would light the volumes and the compositors tried to balance the passes to create a look. But about two or three years ago, we incorporated SVEA into Arnold, our global illumination renderer, and that did away with holdouts and lighting issues. It means, for example, if we have smoke next to a red wall, the smoke automatically gets the red bounce. All these subtle things take a scene to the next level. It makes it look real and, in some cases, look beautiful.”
Thus, for Oz, the artists could light the entire environment and render the scene together with the volumetrics. “The shadowing, global illumination, and bounce light all work out of the render,” De Jesus says. “Our render servers have 24gb of RAM, so we piled on details until we hit that limit, and then we were done. You can never have enough detail; the complexity is what makes the scenes look real and interesting.”
Some of that complexity came from effects – that is, from elements inserted into the atmosphere. “Every scene has an atmospheric theme,” Stokdyk says. “In Kansas, we had straw and dust blowing through the foreground, which gave us texture and allowed us to art-direct through the [stereo] depth. And, throughout the movie, we have snow, clouds, dandelion puffs, and sparks. We had the 3D photography and the concept artwork, but there are still a lot of art-direction decisions we can make in visual effects. Where to put trees to dapple shadows. Where to position a matte painting for the sky and have clouds that complement the foreground. And we made dozens and dozens of depth choices. It was a great creative outlet.”
China Town
Two of the most creative innovations devised by the crew solved the problem of giving James Franco a way to interact on set with his small CG buddies: China Girl and Finley, the monkey.
Oz discovers China Girl when he stumbles across her destroyed village. On set, James Franco crouched down and peered into a small-scale set built by a crew at New Deal.
“We had New Deal, which is a miniature company, build the sets for this little world,” Stokdyk says. “But we shot it as one-to-one scale. It has giant teapots, which are the homes for China Girl, her family, and her village. Inside the teapots is miniature-scale furniture.”
On set, Franco could hold a China Girl puppet in his hand, but 99 percent of the time China Girl is CG. Her size, only 18 inches tall, made it difficult to give him useful eyelines and interaction. A tennis ball on a stick or a laser pointer could provide an eyeline, but not the interaction, and she was too short to have young voice actor Joey King play the doll’s part on set.
“We couldn’t squeeze Joey King down into a corner,” Stokdyk says. “So, we had a marionette version of China Girl on set and a master puppeteer, Philip Huber, who has been doing this for 40 years. He was able to get great nuances out of a simple model, even though he could only open and close its eyes. In post, we painted out the puppet and Philip when he was in frame, and replaced the puppet with the CG version.
Easy to say, but time-consuming and tedious for the painters. “It was painful for all involved,” De Jesus says. “The puppeteer was in a blue suit in the scene, casting a shadow, crossing in front of the actor. So, the amount of paint work we had to do was staggering. It took a long time to get there. After we finished the animation and renders, we waited another three weeks for the paint work. When it was done, though, the shots actually did work.”
On set, as James Franco acted with the marionette, he wore ear buds to hear King – who watched from a nearby booth – perform China Girl’s voice. “Sam [Raimi] directed the performances, so later we had this great source material to draw from,” Stokdyk says. “Tory Salida, our animation supervisor, could pick and choose with Sam how much to take from the puppet performance and how much from Joey.”
Modelers created the CG character with ball and socket joints like the real puppet. After the marionette had appeared on set and in dailies for five months, the puppeteered version had become the China Doll character in people’s minds.
“Philip’s [Huber] instincts were good,” Saliba says. “The performance was simple, subtle, and poignant. And because of the puppet’s limitations, his China Girl had quirky things we wouldn’t have invented as we animated. Whether we used the surreal and awkward puppet or Joey, the little girl, depended on which Sam fell in love with. But even when he leaned toward Joey, we’d go toward the surreal puppet.”
The big challenge for the animators was in performing the face of the porcelain doll. Despite her brittle face, China Girl needed to be a living, breathing character. “We didn’t want her to move her mouth like a ventriloquist’s dummy,” Saliba says. “We needed to feel that she was speaking, and we wanted that sweet little girl face. But, we didn’t want her face to feel like rubber. Her face has that crackled surface like old china, and we wanted that definitive texture. So, we worked with Scott [Stokdyk] to minimize the fleshiness without making her look old and haggard.”
To have emotions play across her face in a scene without breaking the feeling that she was made of porcelain, the animators devised a unique solution. “Her expressions had to be readable, so when they change within a shot, we tried to hide the change in the gross movement of her head,” Saliba says. “Or, we’d cut away to James Franco. When we cut back, the expression was there. We didn’t modulate. That limitation sometimes made our job difficult. But, it was exciting. Usually we fight a tendency to rely on formula. This time, we couldn’t pull from our old bags of tricks. China Doll feels like a character you don’t usually see. I think she’s endearing. The scene when you first meet her in Chinatown is heart-wrenching.”
At top, the china village was a miniature set built at New Deal Studios. On set, a puppeteer performed China Girl, but in the film, she was computer-generated 99 percent of the time. At bottom, the monkey Finley was always digital.
Monkey Business
Like China Girl, the CG monkey Finley’s on-set presence was a challenge, but the challenge was distinct. Finley stands three feet rather than 18 inches tall, so in scenes when the monkey is on a rock or a tree stump, Franco could have been at eye level with Zach Braff, who was the digital character’s voice. But not when Oz (Franco) walked and talked with Finley at his side. And, the monkey flies.
“When Oz is walking on the yellow brick road, you want the CG monkey to interact with him,” Stokdyk says. “We wanted that interaction between James [Franco] and Zach [Braff].”
Rather than using a marionette for the monkey as they had for China Girl, though, the crew devised a unique solution. “We had a puppeteer in a blue suit with a monitor on a pole,” Stokdyk says. “We called it ‘puppet cam.’ ” In addition to the small monitor, the puppet cam pole had a lipstick camera, all covered in blue neoprene. The puppet cam operators were from the KNB EFX Group, which provided special effects.
On set, Franco and Braff would rehearse scenes and block out action, and then Braff would move to a soundproof booth equipped with two monitors. The main camera fed the footage the DP shot to one monitor. The other monitor showed footage from the lipstick camera on the puppet cam pole; in other words, from the monkey’s point of view, the view Braff would have had if he were on set rather than in the booth.
Also in the booth was a camera aimed at Braff’s face that fed images to the monitor on the puppet cam pole. Thus, Franco could see Braff’s face on the small monitor on the pole. And, Braff could see Franco’s face. In addition, both actors had ear buds so they could hear each other.
“The puppet cam operator would put the monitor and lipstick camera where the monkey’s head was supposed to be, so James always had a live actor’s face and eyeline,” Saliba says. “If the monkey was on the ground, the operator had a three-foot rope, so he knew where the monkey’s head would be. He could move the stick up and down with a lever, and he had a joystick to swivel the monitor on the end of the stick, like a little head. I don’t know if anyone expected this, but it gave the thing a little personality as well.”
Certainly more personality than a tennis ball on a stick and a laser pointer. “It ended up working very well,” Saliba says. “It’s a big stretch for an actor to visualize a character when we have a personal assistant reading lines off camera and a tennis ball on set. Zach and James could play off each other, and we got a much more tactile performance from James on stage. Even when they brought back Zach post-shoot to change a line, it was worth it because James Franco’s performance was so vibrant.”
Animators had reference from the rehearsal to see how Braff gestured, footage from Braff in the booth for facial expressions and lip sync, and footage of monkeys. “If the monkey was doing something physical, we leaned toward reference of the animal behavior,” Saliba says. “If he was acting, we tended to rely more on the videotape of Zach.”
There was no reference, however, that showed how a three-foot-tall monkey would fly. “He couldn’t fly like a bird,” Saliba says. “He was a monkey with wings. We didn’t have a benchmark. We had to figure out how the physics would work.”
Once the animators had created a flight cycle that felt right for Finley given his weight and shape, they realized the same physics wouldn’t produce a good image when he hovered. “He could propel himself with the perfect amount of agility, but when he hovered, his head bobbed up and down, which didn’t work for conversation or with the eyeline,” Saliba says. “We had to figure out a way to make his physics believable without having him bob. It took trial and error.”
Finley is small and cute, and has bird-like wings with feathers. The baboons in the movie, which fly in hoards, have one purpose: kill. “The baboons have big, boney bat wings,” Saliba says. “We had to experiment. We used all the reference we could find from nature, but nothing that big flies.”
Artists at Sony Pictures Imageworks created all the fantastical landscapes for Oz the Great and Powerful as well as China Girl (top) and the monkey Finley (not pictured).
Digital Actors
In addition to the two main characters, the animators created swarms of butterflies, digital doubles for the main characters, and put thousands of people in the Emerald City. “When James Franco ran into a big vista, once he got to a certain distance, our digidoubles took over,” Saliba says. “We had volunteers run up ramps on a motion-capture stage calculated to match the environments. So, with a bit of adapting in the computer, we could continue having the actor run into a vast land.”
The crew also used motion capture for the crowds of characters in the Emerald city. “We scanned all the hero extras on set and photographed the rest of the extras for weeks,” Saliba says. “Then, once we got back to Imageworks, we captured artists and volunteers from our team doing actions needed for the citizens, like cheering and loading carts for battle. When the crowds got really big, we had help from the guys in our Massive department who used bits of motion capture or, in the case of butterflies, bits of animation.”
As the animators worked, the director and Editor Bob Murawski took notes. “They would ask why we didn’t have a particular person working on a character,” Saliba says. “They were very conscientious. They’d see things they liked and note who did it. I’ve never experienced that before. Sam had done Spider-Man, of course. But, this was the first movie for them in which the two main characters who emote and carry chunks of the film are completely digital. They worried about whether they could carry the performance.”
In the 1939 film The Wizard of Oz, the main character Dorothy Gale (Judy Garland) had three buddies, three actors wearing suits, any of which would have been eligible for Oscar nominations. Filmmaking magic has adventured a long way since then: In this movie, the main character travels with two digital buddies.
We can trace the first digital companions back to the dragon created 17 years ago at ILM for the 1996 DragonHeart, and Imageworks’ own Stuart Little in the 1999 film of the same name – both of which received Oscar nominations for Best Visual Effects. Indeed, early this year, two of the five films nominated for visual effects had main digital characters with dialog – The Avengers and The Hobbit – and all five nominees had realistic, believable digital characters.
Perhaps someday we’ll see an Oscar award for best digital actor in a supporting role. It’s time to pull back the curtain and fully honor the great and powerful wizards in the wonderful world of visual effects.
Barbara Robertson is an award-winning writer and a contributing editor for Computer Graphics World. She can be reached at BarbaraRR@comcast.net.