Artists create CG effects that reflect the director's style and vision for a new science-fiction series
For the artists at Zoic Studios, realism is the top priority when it comes to creating digital effects for broadcast and film projects. But to these folks, realism means more than creating photorealistic 3D models and seamless greenscreen composites. It also means ensuring that the style of digital cinematography they reflect in their effects work perfectly matches the style intended by the director and achieved by the camera crew shooting the live-action footage.
Zoic's approach is most evident in a new science-fiction series on Fox Television called Firefly, developed by Buffy the Vampire Slayer's creator Joss Whedon. Set 500 years in the future after a universal civil war, Firefly tells the story of Serenity, a small transport spaceship whose inhabitants have left our known galaxy for its border planets—many of which are barely inhabited—in search of the "new frontier." Along the way, Serenity and its crew must avoid the Alliance cruiser, a giant spacecraft manned by the intergalactic police, who are trying to stop the Serenity crew from exploring and inhabiting these outer planets.
According to visual effects supervisor Loni Peristere, Firefly is different from other science-fiction programs because of its early-.1970s documentary-style approach to cinematography. The result of using handheld cameras rather than locked-off cameras is film footage that looks as if it were shot from a helicopter. The team also shoots footage with screens in front of the lights for dappled effects, and lets lens flares intrude into the field of view. "The live-action cinematography is purposefully rough so it feels like the camera is part of the action," Peristere says.
Because Whedon has chosen this method for the live sequences in Firefly, the Zoic artists must ensure that the digital shots they create for each episode are designed with this style in mind. "If the production crew shoots an action sequence from a helicopter, for example, they'll get artifacts like bounce and shake," Peristere explains. Therefore, if that sequence requires digital effects, the artists need to incorporate those artifacts into their shots, whether they're created entirely in the computer or composited. "This brings the viewers into the story, and they forget they're looking at a digital shot."
According to Peristere, most CG effects shots today do not blend seamlessly with the live-action shots surrounding them. "So much time and effort is spent on making effects shots look photoreal and perfect. But the problem is that the live shots appearing before and after the effects aren't usually perfect in the same sense," explains Emile E. Smith, Zoic's CG supervisor. "For instance, you may be starting the sequence with a handheld camera move and then moving to a CG effect with a locked-off plate, so the CG effect ends up sticking out like a sore thumb. The continuity of the sequence is ruined."
Instead, Zoic provides what it calls "freeform digital cinematography," which is based on storytelling rather than technology. To accomplish this, the artists first read the portion of the script that calls for their digital work. Next, they create an animatic in NewTek's LightWave to explore the issues and challenges involved in making the sequence. Once they've ironed out such issues as the type and placement of the cameras and rigs, Zoic cinematographers shoot the live action for the scenes requiring the digital elements. After completing the CG portions of the sequences, they incorporate the live action into their CG scenes.
Firefly, which is scheduled for a 13-week run, is currently being considered for a second-seasonal renewal. On average, each episode features approximately 30 effects shots, including highly detailed models of the Serenity and the Alliance, as well as an additional spaceship manned by a group of villains, various 3D environments, digital replicas of the live-action sets, and a digital version of each principal in the cast.
Many of those digital elements have debuted in some fairly challenging sequences. In one segment, which appeared in the show's premier episode, the screenplay called for a train robbery to occur on a "hover" train barreling through the desert. One of the show's main characters, Jayne (played by Adam Baldwin), was to jump from Serenity onto the top of the train, enter the train, steal its cargo, and return to Serenity.
"Whedon wanted the camera to follow the course of the train, in a handheld, documentary fashion," says Smith. "But the train was moving at about 300 miles an hour, and it's hard to get a helicopter to fly at that speed while shooting usable plates. Therefore, we created the sequence digitally."
The artists began by creating the environment, using desert film footage shot in Sedona, Arizona, and later digitized it. Next, Smith and artist Lee Stringer added a mountain backdrop, rocks, and bushes built and textured in LightWave, which resulted in a finished model of 1 million polygons. Using Adobe Systems' After Effects, the artists then composited digital replicas of Serenity and the hover train—which Stringer built and animated in LightWave and textured in Adobe's Photoshop—into the live footage.
For the shots showing Jayne jumping onto the train, the artists shot greenscreen footage of the actor and used After Effects and Discreet's Combustion to composite it into the CG background. However, for the shots showing Jayne being hoisted back onto the spacecraft, the artists inserted a digital double of the actor, which was created by Gentle Giant Studios (Burbank, CA) using a Cyberware WB4 full-body scanner and head scanner. A character modeler at Zoic later cleaned the data and set it up as an IK rig in LightWave. The team then "dressed" the digital actor using photographic textures garnered from digital pictures of Baldwin.
According to Peristere and Smith, this train sequence presented the artists with two challenges—mimicking Whedon's style of cinematography using CGI, and ensuring that the sequence looked real. To achieve both goals, the artists used a technique within LightWave in which radiosity rendering is "baked" into the models' texture maps. "You're baking highly expensive renders into committed lighting setups so that when you change a scene, you can re-render only those changes rather than the model's entire lighting and shading setup," explains Peristere.
In another difficult sequence, also from the show's premier episode, the screenplay called for a crew member to be pushed backward into the spacecraft's exhaust system, and then lifted up and sucked into one of its engines, all from a distance of 3 to 6 feet from the camera. "Whedon wanted this to occur unnaturally fast," Peristere recalls. "Yet, even the show's stunt people felt that with cinematography shot at 8 frames per second and with a wire [setup], the sequence would still look awkward."
Alternatively, the artists concocted some fancy digital camera work of their own to pull off the effect. As Peristere explains, the first 12 frames of the sequence are live action, with the real actor. Then, there is a seamless transition to a CG sequence using a digital double for the actor in the remaining 42 frames. According to Peristere, the challenge here was to create realistic facial expressions on the digital actor as it was being sucked into the engine. The artists didn't have the time to animate the facial expressions in LightWave, so they used After Effects to project footage of the real actor's expressions onto the 3D model. "Using these camera and projection techniques, we got the style and performance Whedon wanted," notes Peristere.
 |
| For this search sequence, artists integrated camera moves from two different physical sets, and combined it with CG camera data. |
A third challenging sequence—this one from the show's second episode—required seamlessly combining multiple camera moves, both live and digital, while preserving Whedon's documentary camera style. In this scene, members of the Alliance are searching Serenity's dining room for two stowaways. "Whedon wanted the scene shot with a handheld style, and then he wanted the camera to pull out through a window in Serenity to reveal the characters, clad in space suits, clinging to the outside of the ship," Peristere explains. "Finally, he wanted the camera to pull back even farther to reveal the fact that the smaller Serenity ship is actually being shadowed by the larger Alliance cruiser, which is supposed to be about the length of 50 football fields."
For this sequence, the dining room and Serenity exterior sets were shot separately with a real camera, while the Alliance scene is a digital creation. According to Peristere, the artists had to devise a way to stitch together the two live camera moves, along with the digital camera move, so it looked like one continuous pullback that incorporated the handheld camera style. After reading the script, the artists created an animatic in LightWave to determine what kind of camera rig and lens they would use, the placement of the camera rig, and how much of the Serenity set piece had to be constructed. "Using this information on set enabled us to obtain the necessary live footage," says Peristere.
The artists also digitally tracked the actors using 2d3's boujou as they hung from the side of ship in the second live shot. They imported that camera tracking information into LightWave to establish a 3D camera move that matched that of the live camera. They also tracked the camera that shot the dining room sequence, and linked the two live cameras using a CG interstitial camera in LightWave.
 |
| Zoic creates an average of 30 effects shots for each Firefly episode, including digital enhancements to the live-action sets. |
According to Peristere, this is just a snapshot of the effects Zoic is creating for the series. In future episodes, he adds, viewers will continue to see the extensive digital environments, sets, and characters the team has created. The artists also will strive to make their sequences look even more realistic by continuing to integrate the qualities of live-action cinematography into their digital shots. "In an upcoming episode, we were trying to examine how we could realistically zoom in on a spacecraft in outer space," Peristere say. "Where should the camera be? How do we get that handheld feel?"
After some experimentation, the artists saw that although the camera setup was correct, something was missing. "We were zooming in on something that was supposed to be miles and miles away—like the moon," Peristere says. "But the shot looked fake because we were making it too easy to see the object. The minute we added instability to the focus, the shot felt grounded."
These are the kinds of techniques viewers will see employed on Firefly as the series progresses. And, they're techniques the Zoic artists believe make the show compelling, and the facility stand out. "We're trying to give television a sense of realism in cinematography that hasn't been seen before now," concludes Peristere. ..
Contributing editor Audrey Doyle is a freelance writer and ..editor based in the Boston area.
2d3 www.2d3.com
Adobe Systems www.adobe.com
Cyberware www.cyberware.com
Discreet www.discreet.com
NewTek www.newtek.com