Doggie Doubles

Showing at the local multiplex: a new method in the creation of digital animal doubles that not only makes the characters indistinguishable from their real-life counterparts, but saves the studio considerable time and money.

This latest feat in photorealism can be seen in Underdog, Spyglass Entertainment and Walt Disney Pictures’ live-action adaptation of the 2D cartoon of the same name that ran on television from 1964 to 1973. In a throwback to the premise of Superman, an unassuming beagle named Shoeshine Boy (voiced by actor-comedian Jason Lee) develops an extraordinary ability to speak following an accident in the mysterious lab of maniacal scientist Dr. Simon Barsinister (Peter Dinklage). He transforms into a rhyming canine superhero that goes by the moniker Underdog and vows to protect the beleaguered citizens of Capitol City by proclaiming in doggone earnest bravado: “There’s no need to fear. Underdog is here!”

Hoyt Yeatman served as the film’s visual effects supervisor; he teamed with Realscan 3D in Redondo Beach, California, a mobile 3D scanning house whose proprietary technology is used to build exact digital doubles of talent and objects that can be matched quickly to live-action plates. The partnership was born of necessity. Director Frederik Du Chau’s vision for Underdog was to create a photoreal canine that could fly, in hopes of wooing a younger audience for whom realism is critical. “I think kids will love the way he looks, talks, and moves,” Yeatman says.

To this end, the group used Realscan’s portable, suitcase-sized RealCapture NexGen 3D scanning technology to capture unprecedented levels of detail and color in the pack of dogs on which the digital doggie doubles were based. With the technology, an animal stands on a table, just as it has been trained to do in other camera shoots. The process also cut painstaking creative and approval processes from months to weeks and even days because the models are built using real, exact information. The economical approach enabled Yeatman, who has worked on more than 100 motion-picture, television, and commercial projects, and won an Oscar for The Abyss, to focus on more strategic issues rather than worry about perfecting the animal likeness.

Digtizing dogs
The difficult task at hand involved translating a stylized 2D lead character into a live-action photoreal action star. Yeatman was concerned that the jerry-rigging associated with previous automated attempts at creating digital animal doubles would prove too difficult for use at the time of principal photography or too time-consuming in postproduction. Such techniques included body casts or laser scans of the talent, which required filling in the undercuts with inexact data, or creating all the geometry directly from reference shots, resulting in an embellished rendition rather than exact replica of the subject.

 

Realscan digitized a number of dogs for the film Underdog. The company’s mobile scanning device captured high levels of detail and color, allowing animators to substitute the CG canines into scenes that would have been impossible for a real animal to handle, such as the one above.

Also, modelers often faced an approval process involving CG versions of the animal that was lengthy, expensive, and prone to varying degrees of success. Knowing that it can take up to half a year to build, fur, groom, light, animate, and capture the persona of a CG canine model before it is put into production, Yeatman sought to avoid a half-baked solution that could pose creative and logistical problems as revisions were made.

Digitally scanning live animals has always proven difficult in the past. For example, some automated scanners required an animal to stand completely still for at least 30 to 60 seconds. Using laser scanners meant wetting the critters with water and covering them with white chalk or baby powder, or both. So studios often opted to hire sculptors or creature shops to build scale or life-size animal replicas. Not so on this project.

When the Realscan crew came aboard in December 2005, Yeatman had the luxury of plugging Realscan into the VFX pipeline before the animal or human talent had been cast. In the initial scanning session at Boone’s Animals for Hollywood (Castaic, CA), full-body animal scans were done of Shoeshine (a cooperative beagle named Leo), “Sweet” Polly Purebred (his love interest, voiced by Academy Award nominee Amy Adams), and one other prominent canine (voiced by Brad Garrett). Other work included a head scan of a Rottweiler named Riff Raff. Shoeshine, the first such model, was delivered in just a matter of days. The group returned to Boone’s a few months later to shoot head scans of the entire Riff Raff Gang consisting of an old basset hound, Shoeshine’s Mother, Riff Raff Dog No. 1, and Riff Raff Dog No. 2.

The next step came in April 2006, when a full-body people scan of Dinklage, the film’s villain, and a head scan of a Chinese crested crescent pooch known as “The Crazy Dog” were done on location in Rhode Island between takes so that they wouldn’t interfere with principal photography. The scan team returned in August 2006 to capture digital replicas of four 12-foot statues that were part of the set in order to match certain live-action footage. A final trip was made to the Disney lot in Burbank, California, to scan a cat during additional shoots this past February. 

 

Artists paid close attention to the details of the dog’s mouth, especially how his teeth, tongue, and throat worked, particularly during speaking scenes. Elaborate face controls allowed the group to achieve subtle motions for the digital dogs. 
 
The Virtual Process

 

Realscan’s portability factor came in handy for scenes that introduced new dogs from Rhode Island. In general, scanning live animals is similar to scanning human toddlers in that the process involves indirect communication. So it was imperative to establish a friendly rapport to help the dogs relax, given their sensitivity to industrial equipment. In the end, the scanning sessions were completed in a matter of minutes for both head and full-body scans.

CG techniques were used to augment Leo’s image whenever Shoeshine Boy spoke or flew through the air as his altar ego, Underdog. To facilitate the construction of his digital double, Leo had to be properly scanned. Quite a bit of attention was paid to the inside of the dog’s mouth and how his tongue, teeth, and throat worked, for the speaking scenes. A digital mask featuring elaborate face controls captured subtleties such as frowning. “It was important to get an accurate model,” says Yeatman.

The CG dog had to be designed with musculature and pivot points that mirrored a real pooch. Caught between the worlds of physics and imagination, the chief creative challenge for Yeatman and his VFX crew was to visually bring the script’s action to life with regard to crashing through walls or scaling impossibly large objects so that it would be both believable and breathtaking.

Realscan’s use of multiple custom-camera calibrations to more accurately capture the details of each animal won over Yeatman when he put the VFX services out to bid. Tests showed the realism that could be acquired for hybrid filmmaking involving CG and live action, making this technology well worth the investment in his eyes.

As a result, Yeatman was able to spend more time on an HDRI lighting rig he developed to speed the lighting and compositing. Yeatman recruited a former partner from Dream Quest Images and his brother in Switzerland, whose specialty is programming computer systems for equipment, to create a customized solution. Together, they built a prototype unit to capture HDRI at the time of photography, while the film’s VFX facilities, Cinesite and Framestore CFC, came up with various postproduction pipelines using the HDRI information.

This allowed David Eggby, the film’s director of photography, to light scenes not only in the physical world, but also to a large extent in the CG realm. Color temperature, exposure value, and other variables lose their meaning in a CG environment because of the virtual technology used to replicate the live action—which means most, if not all, of the assessments are done by the naked eye. “That takes a lot of time, and it doesn’t always capture the subtleties, especially with global illumination,” observes Yeatman, who is building eight more HDRI composite units made of much lighter carbon fiber material for future projects.
  

According to Yeatman, Cinesite reduced what typically would have been a six-day process to approximately three hours. For the film, Cinesite handled the muzzle-replacement shots, while Framestore turned its attention to the 3D CGI full-body double shots. Cinesite replaced the nose, lips, teeth, and lower jaw, as well as patches above those features and eyebrows, for each of the dogs. The studio completed 372 shots (a dozen of which were of the full-CG dog, and the rest CG muzzles), created using a blend of the original tracked plates with the matchmoved CG performance—a process that unfolded with the help of Pixel Farm’s PFTrack software. 

 

Framestore, which turned in more than 300 fully CG shots (about 250 involving the dogs), received cyberscans of Shoeshine, Polly, and SuperShep, from which the facility created and animated fully CG versions of Polly and the three German “super” shepherds.

Given the digital stunt-double nature of the shots, one popular effect Cinesite worked on, known by the production as a “whoosh,” captured Underdog flying through the air at breakneck speed to a point where he appears as a blur on the screen. Nevertheless, he needed to be modeled accurately and animated to match subsequent sequences that cut back to the real dog.

Framestore also created three action sequences in fully CG environments. They involved flying German shepherds chasing Underdog at the top of the Capitol, Underdog flying away from an explosion after being buried in a tunnel deep inside the Earth, and Underdog blasting into space by the force of an explosion before falling back to Earth as part of a re-entry effect. Other shots that required a CG stunt double for Leo involved various crashing, fighting, landing, and takeoff sequences.

Since Du Chau wanted the canine mouths to be as flexible as a 2D animator’s line while staying within the realm of realism, Cinesite’s detailed anatomical model defined and limited movement appropriately. A bespoke system was used to blend animation between different muscle shapes and form phonemes. Animation details in the dogs’ eyebrows, subtle movements, and 2D warps matched the dialog, thereby enhancing and improving the dogs’ original performances.

Cinesite also used photogrammetry to derive the 3D model from original footage of each dog, which was independently modeled using the naked eye. The modeling team studied anatomical references to create highly realistic skeletal and muscle systems for the dogs, which were rigged and ready for animating. In terms of tools, Cinesite used Autodesk’s Maya for animation and lighting, while the program’s particle system handled a fur-explosion shot. Adobe’s Photoshop and Right Hemisphere’s Deep Paint 3D were used for texture, Pixar’s RenderMan for rendering, and Apple’s Shake for compositing.

 

To better integrate the digital dogs into the scenes, Realscan captured HDRI lighting that could be used by Cinesite and Framestore CFC, two facilities that handled everything from muzzle replacements all the way through to all-CG dog replacements in the film.

Framestore used a combination of NewTek’s LightWave and Maya for all of its modeling work. The studio also used Maya for animation and lighting, along with many proprietary plug-ins. Texturing was achieved with a combination of Photoshop and Maxon’s BodyPaint 3D, and compositing was done using RenderMan and Shake.

 

Once all the elements were brought together, Leo and his digital double were indistinguishable as they ran around in circles on a front lawn, for example-a must for today’s audiences, which are growing more sophisticated and demanding more convincing digital doubles. The aim for Underdog was to please not only kids who routinely expect to be dazzled by imagery, but also their nostalgic parents who grew up watching the cartoon on which this film is based.

“Our technical and creative contribution to films is so much more important than it was five or six years ago,” says Yeatman. “These characters coming out now are doing amazing things.”