CG-animated films hit the third dimension in a big way
IMAX and CG animation are natural partners. Animation creates immersive worlds of fantasy; with its almost eight-story-high screen, multi-channel digital surround sound, and 15-perf 70mm prints, IMAX creates the ne plus ultra of immersive environments. With the emergence of the 3D CG-animated feature, the idea of seeing all those three dimensions on the IMAX screen became an irresistible challenge.
That idea became a reality in 2004 when Warner Bros. Pictures released The Polar Express in IMAX 3D. This was the first full-length Hollywood feature to be converted to IMAX 3D—and it was a smash hit, with some critics preferring the stereoscopic version to the more widely released 2D version. And box office wasn’t bad either: After a disappointing opening, the film gained momentum, in large part buoyed by the IMAX 3D version. During some weeks, the 3D IMAX version accounted for 24 to 30 percent of each day’s gross take. For IMAX, The Polar Express became the highest and fastest grossing digitally re-mastered IMAX release.
With one icy train ride, IMAX 3D gained an infusion of enthusiasm that has resulted in two more CG-animated titles, both of them from among the crop of 2006 animated features: The Ant Bully and Open Season. "There has been a resurgence of interest in stereoscopic films," says IMAX vice president of technical production Hugh Murray, who also played the role of IMAX producer for The Polar Express, The Ant Bully, and Open Season. "The digital stereoscopic films have all been in the works since The Polar Express, so it’s either a big coincidence or The Polar Express has ignited a new wave of interest."
IMAX, which was founded in Toronto in 1967, has a long history in stereoscopic filmmaking. "3D has been a part of IMAX since the very beginning," says Murray. The first stereoscopic CG-animated IMAX film was the 11-minute We Are Born of Stars, in 1985, and in 1989, Echoes of the Sun, a 20-minute, partially CG stereoscopic film, was created for the 1990 World Expo in Osaka, Japan. IMAX’s first live-action 3D film was Transitions, for the 1986 Vancouver World Expo.
"A lot of IMAX’s early history was about special-venue attractions like world fairs and expos," explains Murray. "From the beginning, there was a drive to do something better than was ever done before. The next ‘better’ was to do it in 3D." In fact, 3D live-action films have steadily been a hit on IMAX screens. James Cameron’s Aliens of the Deep, Bugs!, Deep Sea 3D, and many others have been popular at IMAX theaters everywhere (see "Atypical Stereo," July 2003).
Hollywood Goes IMAX
In the live-action realm, IMAX made a breakthrough with Apollo 13, released in September 2002, the first film to benefit from a new IMAX technology. IMAX DMR is a proprietary technology that enables the conventional Hollywood feature film to be digitally re-mastered into IMAX’s big-screen format as a so-called IMAX Experience. With the IMAX DMR (Digital Re-Mastering) technology, the IMAX theater network became a distribution venue for Hollywood features, drawing in audiences seeking entertainment rather than education. IMAX Experience films include Star Wars: Episode II—Attack of the Clones, The Matrix Reloaded, The Matrix Revolutions, Harry Potter and the Prisoner of Azkaban, Batman Begins, Charlie and the Chocolate Factory, and Harry Potter and the Goblet of Fire.
The IMAX DMR process starts by scanning each 35mm frame, at the highest resolution possible, into the digital realm. Proprietary image enhancement tools optimize each image for 70mm film, including sharpening, color correction, grain removal, and stabilization. These digital images are then recorded onto 15/70 film. The 35mm film’s original soundtrack is also re-mastered for IMAX’s multi-channel digital surround sound system.
IMAX 3D: The Process
This proprietary system laid the foundation for developing the IMAX 3D DMR process. IMAX executives were interested in the possibility of transforming 3D computer-generated films into stereoscopic IMAX films the moment they saw Toy Story. "We thought, here’s a film already made in 3D but not rendered in 3D," says Murray. "We approached Pixar with the idea of doing it [in IMAX 3D], but they were already busy on their next feature. So we decided to do a demonstration on our own."
IMAX’s CyberWorld, released in 2000, is a museum tour—led by cyber Phig (Jenna Elfman)—of existing 3D clips, from the 3D episode of The Simpsons to the movie Antz. During the tour, three unwelcome guests barge into the museum, and Phig is forced into a battle to save the museum’s 3D environment. "It was intended to show people how amazing it would look to take 3D CG animation and show it stereoscopically," says Murray, who was responsible for the original story and co-wrote the screenplay with Charlie Rubin and Steve Hoban.
The intention bore fruit very quickly. Steve Oedekirk created the IMAX 3D holiday film Santa vs. the Snowman in 2001, in which a lonely snowman is first captivated by Santa’s village and then, out of jealousy, wages war on him and his elves in a battle that features hot chocolate squirt guns and a 50-foot-tall toy soldier.
However, IMAX 3D didn’t gain much traction after that film. "We talked to a lot of studios about taking 3D CG and rendering it stereoscopically," recalls Murray. "Many of them were enthusiastic. The problem is that a CG animated film is a huge undertaking and, with an IMAX 3D film, they’d be making two films. Even though many studios were fascinated by the look, it became difficult for them to take that leap of faith to get things done."
That is, until powerhouse director Robert Zemeckis decided that The Polar Express would be the perfect film to roll out in both 35mm theatrical release and the highly immersive IMAX 3D. The Warner Bros. film was created at Sony Pictures Imageworks (see "Locomotion," December 2004). "I think that when the people at Imageworks first met me, they thought their worst nightmare had happened," Murray recalls. "But they quickly became converts when they saw how it looked."
Often filmmakers create an offset camera view for stereo, but for a scene in The PolarExpress, they had to reposition both views due to a composition change in the film.
At Imageworks, senior CG supervisor Rob Engle was put in charge of the 3D project and credited as digital effects supervisor. "IMAX wanted to make a movie in 3D, and Bob Zemeckis is always trying to push the craft of filmmaking and was already interested in IMAX," says Engle. "So it was a good meeting of the minds."
In facing the challenge of turning a 2D CGI film into stereoscopic IMAX, Engle saw two issues. "The first thing you have to talk about is the content and how it lends itself to stereoscopic 3D," he says. "With a live-action movie, you’re often telling a story with the physics and realities of the real world. When you talk about creating a CG feature, you’re trying to create a different world. In [the stereoscopic] Polar Express, when snow is falling, people feel like snow is falling in the theater. I believe these created worlds lend themselves to immersion. They help make people part of the whole experience—the IMAX 3D Experience."
The second issue is a technical one. "Digital technology allows you to create a second movie, which is for another eye, at a reasonably small incremental cost compared to producing a movie from scratch or shooting a live-action movie with a stereo rig," Engle notes. "To move a camera and take another picture doesn’t really cost a lot in a virtual world."
Creating a Stereo CG Film
The process for creating a stereoscopic film is based on how the brain and the eyes work together to see in three dimensions. Our two eyes see a single point from two slightly different positions, and the brain fuses together those two images. An IMAX 3D film comprises two separate strips of film, one showing the viewpoint of the right eye and the other the left eye, projected onto a special silver IMAX 3D screen. Audience members wear custom-designed polarized IMAX 3D glasses, which channel the right-eye image to the right eye and the left-eye image to the left eye. (Some IMAX theaters use electronic liquid crystal shutter glasses, which sense an infrared signal from the projection system. The projector sequentially projects the left-eye image and the right-eye image without overlap, and the glasses, which are synchronized to the projector, block light to each eye in the sequence.)
In producing a stereoscopic IMAX film from a CG animated feature, the filmmakers start with the initial camera used for the theatrical release and, on a per-shot basis, create an "offset" that represents the other eye. "On a per-shot basis, we can decide to make the theatrical release the right- or left-eye [render]," explains Engle. "Our general rule of thumb is that it’s the left eye, but we can switch back and forth depending on the composition of the shot."
For the scene in The Polar Express whereby the train comes to a screeching halt in front of the screen, the filmmakers had to reposition both eyes to keep the train from veering to the left or right. "The composition can change in 3D, so you have to watch for that," Engle says.
The Polar Express helped move CG animation into IMAX 3D. However, certain 2D techniques used in producing a CG theatrical release, especially those affecting the image plane, become issues in 3D space.
© Warner Bros. Pictures.
Dos and Don’ts
Though simply creating an offset of the original movie—a fairly uncomplicated task in the virtual world—sounds simple, the transformation of a 2D theatrical release into a stereoscopic film holds numerous pitfalls. "On paper, you create a separate camera, take another shot of the scene, re-render all the elements, and then re-composite it, producing another eye," says Engle. "The questions become, how do you make this more efficient, how do you streamline the process? So many of the techniques used to create the theatrical release are 2D techniques, like matte painting or rotoscoping. Anything associated with the image plane becomes problematic when going into the 3D world."
In one Polar Express shot, the shadow of a character was rotoscoped into the ground plane. "When converted to 3D, that roto naturally didn’t stick to the surface," says Engle. "It floated off the surface and appeared as a dark mass, like a card sticking up in space. Since each shot has a life of its own, once you get it into the 3D realm, you inherit all those little problems." Engle reports that Imageworks uses a variety of techniques to overcome these "little problems," including warping or re-projecting the image.
Digital 3D experience provider Real D uses a StereoGraphics-based solution that offers an easy, cost-efficient method for making theaters stereo-capable.Image courtesy Real D.
"If you were to look at each eye, neither of them exactly match the theatrical release," Engle says. "We do a few other things to make the 3D experience better. One of them, we tend to dial back on the use of depth of field and blur. Everything is a little sharper in the 3D version, and that allows the viewers to look at things in the foreground and background, instead of having a narrow depth of field where they’re only able to look at one object in the scene. The result is that it becomes a more immersive experience. People are less aware that they’re watching a movie and feel more a part of the world."
In fact, depth of field—an accepted convention in the ordinary theatrical release—can actually look rather strange in a stereoscopic IMAX movie. "CG cameras don’t have depth of field, and it’s added as a forced process to make [the production] look more filmic," says Murray. "With the stereoscopic film, if something in the foreground is soft, it looks wrong. Depth of field can feel like a mistake in the stereoscopic film."
In the world of 35mm theatrical releases, directors use depth of field to direct the viewers’ eyes toward something or someone specific. "But a lot of times, that isn’t truly necessary, because other cues will force you to look at whatever you’re supposed to look at," says Engle. "Inevitably when someone is talking, you’ll look at that person. Another methodology is lighting. If the main character is lit brighter, you’re going to look at that person."
The solution? "Ideally, you’d just turn off the depth of field everywhere," says Murray. "But you can’t always do that. We leave the out-of-focus background alone. It’s only the foreground where it is an issue." As a result, depth-of-field issues resulted in a re-render for both eyes in about 20 percent of The Polar Express.
The re-render wasn’t strictly for depth of field. 2D cheats often used in 3D CG films is also an issue that has to be dealt with in the 2D-to-3D transformation. "3D is always slightly more complicated because there is a mathematical aspect to it that is not present in 2D," says Murray. "You have to do calculations for every shot to see if it’s going to work, and then you need to check it. Because CG films are rendered in layers, for some things 2D cheats are used instead of real 3D elements. If you’re making a stereo vision, you have to put in the geometry in place of those cheats."
This summer, the CG animated feature The Ant Bully was also released in IMAX 3D format, in which the human world appeared even larger, especially to the main characters, nearly all of whom are insects.
Image courtesy Warner Bros. Pictures.
In creating a left- and right-eye render, many aspects of the two images are identical. Yet, some are slightly different, and lighting is one of them. "You can take advantage of the fact that these images are 99 percent the same," says Engle. "You’ve already produced one eye—why not characterize what’s different and then transfer the information that isn’t? That way, you don’t have to re-render everything." How you handle the information that isn’t the same, that’s the proprietary part."
Another issue that can arise when translating a 2D CG film for stereoscopic viewing is that the new camera angle can reveal things not found in the original point of view; new CG work can be required to extend an object found in the 2D version.
Making the two eyes work takes some tweaking. Though productions are usually able to retain a vast majority of the information contained in the original 2D version, all the changes made in creating the second eye has to be finessed. "They probably render each shot five or six times to get it right," says Murray. In an ideal world, a director who knows in advance that the film also will have an IMAX 3D release would plan for that in production. Not using 2D cheats, for example, or foreground objects out of focus are good decisions. "But first, the person has to make a good 2D movie," says Engle.
At the 2005 ShoWest motion-picture exhibition, stereoscopic films got a round of applause from some A-list directors, including James Cameron and Robert Rodriguez, both of whom have produced stereoscopic or partially stereoscopic films in the past. Likewise, exhibitors—worried about flagging attendance—showed their support and remain intrigued by the possibilities of 3D as a way to show films that can’t be seen at home.
The Polar Express: The IMAX 3D Experience was such a hit for Warner Bros. and for Sony that WB ventured again into that territory with The Ant Bully this past summer. Open Season, a production of the new Sony Pictures Animation, also had an IMAX 3D release, as both films utilized Imageworks’ successfully efficient pipeline and customized tools developed during the work for The Polar Express: IMAX 3D Experience. (The Warner Bros./Animal Logic movie Happy Feet is showing in IMAX as well, but not in stereoscopy.) Most recently, the live-action Superman Returns was transformed into the 3D IMAX format through the same DMR technology.
Last Spring, Sony announced the creation of Imageworks 3D, a division that put a stake in the ground with regard to future stereoscopic work. Now, in principle, every CG production produced at Sony Pictures Animation could be converted for stereoscopic viewing via IMAX or other means (see "Another Big Picture," pg. 26). Yet, not all the changes are occurring at the studios. IMAX is developing a digital product that would replace the current cumbersome projection system, by which two strips of 15/70 film are spooled out simultaneously. IMAX also intends to deploy a high-end digital projector in the second half of 2008.
With the growing popularity of large-format stereo films, Sony Pictures Image works recently created a 3D division, paving the way for more ofits CG movies to follow inthe tracks of Open Season.
© Sony Pictures Imageworks.
"We hope IMAX 3D grows," concludes Murray. "It certainly looks spectacular, and clearly the public has responded well to it."
Debra Kaufman is a freelance writer in the entertainment industry. She can be reached at email@example.com
ANOTHER BIG PICTURE
The Polar Express certainly got stereo CGI on the right track. Overnight, the stereoscopic film genre transformed from a novelty to a viable way of bringing viewers back to the movie theater. IMAX 3D—with its 170-plus number of 3D-equipped theaters—has drawn in thousands of people eager for the large-screen, large-format immersive experience.
Michael V. Lewis, who produced the 1999 IMAX 3D Siegfried & Roy: The Magic Box and co-produced the 1998 T-Rex: Back to the Cretaceous, says he fell in love with the realism of 3D—as long as it’s done right. "We keep coming back to it because it’s the last frontier to have visual depth," he says. In 2002, in partnership with Joshua Greer, Lewis started Real D, a digital 3D experience provider, with the intent of developing technology for 3D in cinema.
The first step was licensing ZScreen cinema technology from StereoGraphics, a company founded in 1980 that produced shuttering eyewear for telerobotics, the first flicker-free 3D display, and Crystal Eyes eyewear, which became a standard in industrial virtual reality. (NASA used Crystal Eyes to pilot the Mars Rover spacecraft.)
In 2005, Real D acquired StereoGraphics. Taking the science from StereoGraphics’ 26 years of R&D, Real D created an entertainment-oriented solution that would make it easy and affordable enough to install in hundreds of theaters across the country.
What the company came up with was a system for exhibitors that required an upgrade to a digital projector, a silver screen, and disposable glasses. Real D’s digital projector partners are Christie, Barco, and NEC; its server partners are Kodak Digital Cinema, Doremi, and QuVis. Theater owners, many of whom already possess a DLP digital projector, simply added Real D’s shuttering device, the ZScreen. This hardware/software device fits on the front of the projector, running 144 flashes a second, alternating left-eye/right-eye views. For viewing, audience members wear disposable polarized glasses.
"It takes 15 minutes to install the hardware/software upgrade," says Lewis. "We supply the installation, maintenance, and constant upgrades." In return, the exhibitor pays Real D a combination of an annual license fee and a per-ticket percentage.
Real D’s first stereoscopic film was the 2005 Chicken Little, and its first Real D-equipped theater, the Mann Theatre’s Grauman’s Chinese Complex, was completed in 2005 as well. Since then, exhibitors have profited from the technology. "In Real D-enabled theaters, exhibitors had three times the box office than their theaters with 2D," says Lewis.
Following the 3D version of Chicken Little was Monster House, which, says Lewis, played in Real D on 4 percent of the screens but accounted for 15 percent of the overall box office. The stop-motion Nightmare Before Christmas (which, like Chicken Little, was transformed from 2D into stereoscopy by Industrial Light & Magic) is also a Real D film. Robert Zemeckis’ Beowulf, being produced at Sony Pictures Animation, is expected to open November 2007 in 1000 Real D-enabled theaters.
The number of Real D-enabled screens exploded recently when exhibitor Carmike announced plans to equip 500 screens with Real D’s stereoscopic gear. Of these, 200 screens are expected to be ready for the release of Disney’s stereoscopic version of Meet the Robinsons in March 2007. That deal follows another with exhibitor Cinemark, to equip 150 screens with Real D technology.
Next year, says Lewis, his firm expects to release five stereoscopic films in Real D-enabled theaters, as well as begin to provide live-streaming events and concerts in 3D. "Content is blowing up," says Lewis. "If you build it [in 3D], they will come—and they’ve been coming pretty quickly." —Debra Kaufman