By Barbara Robertson
There aren't many islands in the world where you can find a beautiful beachfront mansion surrounded by low-rent houses. Yet in the film Bad Boys II, you see exactly that in a scene crafted at Sony Pictures Imageworks.
A camera mounted on a helicopter flies toward a beach supposedly in Cuba, cranes up as it nears a mansion, circles closer, goes behind the mansion to look at it from a "safe house," and slides past a row of small houses. The landscape looks real, but this scene of a diamond in the rough is an illusion fashioned from a live-action shot of a mansion in Miami; row houses photographed in Puerto Rico and placed on 3D geometry; digital trees, photographs and paintings of buildings and landscape elements placed on a series of cards; and in the far background, a digital painting of mountains.
As is typical of matte paintings, you'd never know. You might not even think to call it a matte painting if you did.
"It's difficult to categorize what a matte-painting shot is today," says Craig Barron, co-founder of Matte World Digital and co-author of the book The Invisible Art, which offers a history of matte painting as told by many of the painters who created the genre. "The computer graphics techniques we use are constantly changing. There are no rules; we do what it takes to produce a realistic illusion. Most filmmakers still call what we do matte shots, and we like that because we see our work as an extension of the original craft. But, it's more accurate to say we are involved in environment creation." Indeed, recent projects at Matte World Digital include crafting a coastline and nearby landscape for The Ring, realizing an idealized Manhattan for Down With Love, and transforming a summer scene in the flatlands into winter in the mountains for a new film version of The Alamo, which is currently in production.
 |
| Images © 20th Century Fox. Courtesy Matte World Digital. |
 |
| Matte World Digital's painting of the United Nations building transformed the greenscreen shot at Universal Studios (bottom) into a NYC scene (top) for the movie Down With Love. |
Similarly, Jonathan Harb, supervisor of the digital matte department at Industrial Light & Magic, notes that matte artists at this studio might touch up a still frame, create an interior as the department did for Terminator 3, build the background for a pod race scene in Star Wars: Episode I, or construct entire cityscapes as in Episodes I and II. "Matte paintings are often good choices when existing reference materials or locations are incomplete, so that much (or all) of a shot has to be created from scratch," he says. "Often, creating content from disparate source materials is a key part of this process."
But even though matte painting projects range widely, there are some common threads. The camera move in a matte painting is often fairly limited, with the scene shot from one angle (although not necessarily); the environments tend to be created for only a few shots and not reused; and one person often creates the paintings—and thus, controls the shot.
"You could have a shot with a big city in it that you create in the CG pipeline," explains Harb, describing a process in which all the people in the pipeline—modelers, texture painters, technical directors, animators, renderers, compositors, and so forth—work on a shot. "Or, you could give the shot to a matte painter. If you were going to see [an environment] once, you'd probably give it to the matte painter. "If you were going to use it for 300 shots, all from different camera angles, you'd have the pipeline team work on it or maybe build a miniature."
Although matte artists might as easily construct a matte painting from 3D elements as from 2D, the environments they create are usually image-based. Rather than write shaders that calculate complex textures and light, they project photographs and paintings onto 3D geometry and onto 2D planes in a 3D environment to create scenes. Even when it works best to use computer-generated renderings, they often keep the geometry simple and use paintings to create the final look. "One of the first shots I did was an underground complex for Cats and Dogs," says Chris Stoski, an artist at Matte World Digital. "There was no live-action set; it was a complete CG interior. I painted the aesthetics onto 3D renderings."
 |
The Ring's landscape (top) was crafted with filmed elements and paintings placed on 3D terrain and models (bottom).
Images © DreamWorks SKG. Courtesy Matte World Digital.
|
Adds Barron: "This was a last-minute shot, and our post schedule was too short to use global illumination. A cleverly executed painting is often not only the best trick, it also offers some real advantages. Painters are equipped with visual solutions that build on the history of 100 years of matte-painting art and on classical painting. We rely on the artistry of the individual matte artist working on the shot."
The advantages extend beyond quick turnaround. "Often you want a certain mood in a shot," says ILM's Harb. "But 3D tools have a certain inflexibility. You can write shaders, but how do you code mood? This might oversimplify things, but it gets to the point that a lot of techniques we use can quickly get us the last one percent of a shot."
Traditionally, matte painters at ILM work separately from the main 3D pipeline, in a studio within the studio, and use different tools. ILM's pipeline is based on proprietary software and such commercial software as Softimage and Alias Systems' Maya; the matte painting department, which was Mac-based until recently, uses PC-based software. "We're using [auto.des.sys's] form Z for modeling, [Discreet's] 3ds max with [SplutterFish's] Brazil for rendering, and [Adobe's] Photoshop and After Effects," Harb says.
The artists at Matte World Digital use similar tools. "Photoshop mostly, but also [Corel's] Painter and [CinePaint's] Gimp," says Barron. For 3D, Glen Cotter, who specializes in 3D elements for the studio, has Maya and Softimage 3D available, but often uses 3ds max. "Software created with an architectural lineage in mind is better for us," he says.
For example, Cotter put a wireframe model of a lighthouse on a 3D terrain map to create the underpinnings for an island used in an establishing shot for The Ring. Because the island would look as if it were filmed from a helicopter, Stoski blended helicopter footage of Washington and Oregon for the coast and still photography shot over California farmlands for the surrounding landscape. He then projected 10 matte paintings of the land, light poles, trees, and the lighthouse sequentially. "The camera looked at one side only, so I created the model only for the front side," said Cotter. "It was like a façade for an old-time set." To disguise the transition between a CG truck traveling through the synthetic landscape and a live-action car, the artists added passing power poles and drifting mist.
"[Matte paintings] give directors so much more control to get the exact shot they want," says Barron. If the perfect location with perfect, unchanging weather doesn't exist, directors can order up a synthetic environment or film on a backlot, as they did in the '50s. However, with today's tools, skilled matte artists can craft locations that are more realistic and flexible than those '50's backdrops. This, in fact, was exactly what happened for the movie Down With Love.
"[Down With Love] was made with new tools, but in the spirit of how a studio picture would have been made in the early '50s," says Barron. "It takes place in New York, but it was filmed in California on the Universal backlot with actors on greenscreen stages."
Indeed, because the film was set in the '60s, Matte World Digital used background plates from Universal's library of films made at that time rather than alter live-action plates shot in New York today. "We restored stock Technorama film elements to look like modern-day photography and used them as texture maps on low-polygon models," says Barron. For some scenes, an artist might project images onto 2-1/2D planes set in as many as 10 layers that would all move as the camera would pan or tilt. Seen from the CG camera, the layers blended into a cityscape. For others, flat paintings with moving elements such as cars or clouds fostered the illusion. And when the scene required, images were projected onto simple 3D models.
 |
The painted background in this Down With Love scene includes the Empire State Building, moved for effect. All cars in the distance are CG.
Images © 20th Century Fox. Courtesy Matte World Digital.
|
The techniques are similar to those used at Sony Pictures Imageworks for the shot of the mansion amid shanties in Bad Boys II. "The thing I like most about digital matte paintings is that you're using photographic material," says Carey Villegas, visual effects supervisor for the film. "I've always favored mapping photographs onto basic geometry over writing complex shaders, but it used to be that we could do this kind of matte work only in high-end post. Now, 3D packages like Maya and [NewTek's] LightWave are more commonly available to artists."
In fact, because Imageworks' resources were not available for this type of work at the time, according to Villegas, he hired artists at smaller studios: Xye at Yannix Technologies did the 3D match-move to derive the camera, and Robert Stromberg from Digital Backlot helped create the environment. "We replicated the helicopter camera with a 3D camera that we gave to Robert, and he was able to use the camera move in LightWave," says Villegas. "He created most of the 3D elements using photos I gave him as the basis for artwork that he projected onto the models. And then we brought the elements and the same camera move into [Discreet's] inferno at Imageworks, where compositor Christian Boudman built the environment."
"We started with the helicopter plates of the approach from the ocean to the house, the way [director] Michael Bay envisioned the shot," Villegas says. "We kept the ocean, the sand, and the main mansion, but cleaned out and replaced everything on either side of and behind the structure." To find buildings suitable for the shot, Villegas traveled to several small towns in Puerto Rico. "I would stay the whole day, taking pictures of houses," he says. "We used those photos for the houses behind the main house in the virtual environment where we needed to have 3D geometry. The 2-1/2D part happens in the next layer where we didn't need as much parallax. For this, we mapped a river and other elements onto a series of cards. Then in deep background, we used flat artwork."
To create the new landscape to the side and behind the mansion, Boudman rotoscoped and removed elements from the live-action shot using inferno and Avid's Matador and added the new elements. "The real mansion was surrounded by trees and bushes," says Villegas, "so in some cases, rather than removing trees, we put our own 3D versions on top of the real ones. Also, to add realism to the shot, we gave the big clouds, which were matte-painting elements, dramatic movement by shearing and displacing them, and added glints, highlight kicks, and reflections from the sun and ocean in the windows of the buildings. The shot took many months to do, but I don't think that in a million years you'd ever know it was a matte painting."
In addition to creating exterior environments, matte artists often paint interiors and set extensions. In Terminator 3, for example, there's a scene in which Kate Brewster's father, the military mucky-muck, gazes through a window at a gigantic laboratory. Inside, a tethered Hunter-Killer swings back and forth while scientists move around in the background. In reality, Kate's father was filmed on a bluescreen stage, the Hunter-Killer was a 3D model, and the laboratory was a matte painting created at ILM with actors composited in.
 |
For Bad Boys II, Imageworks started with helicopter footage of a Miami mansion (top) and then replaced the background with 3D houses, 2-1/2D painted layers, and paintings.
© 2003 Columbia Pictures, Industries, Inc. All rights reserved.
|
To help make shots such as this—that have elements created in the pipeline and in the matte department—flow more smoothly, ILM has begun working on new software for the matte painters. "The R&D group is building a tool that will take camera mapping to a new level," says Harb. "It will ease the rift between matte painters and the rest of the pipeline. We'll get access to the pipeline, and they'll get access to our tools. Plus, it will make what we do easier."
Dubbed Zenviro, the new tool works within ILM's Zeno software and offers the artists more interactive editing of geometry and painted maps. "The artists can project multiple maps from multiple angles while viewing all of the maps with correct transparency at the same time on the same object," says Steve Sullivan, director of R&D. "Furthermore, many clusters of projections and objects can exist in the same scene at the same time. By seeing the results 'live,' artists can quickly hone in on trouble spots without wasting effort on areas that look fine."
 |
For Terminator 3, ILM artists painted a laboratory (top) then added a 3D Hunter-Killer (second from top) and bluescreen elements of actors (third from top and fourth from top shown with matte) to form the final image (below).
Images © 2003 IMF
|
When movies were first made, because it was difficult to go on location, filmmakers had little choice but to use painted backdrops. Now, experienced artists using today's sophisticated tools can create synthetic environments that are as believable as real environments, and filmmakers are taking notice.
"I think filmmakers might be rediscovering the old studio idea of making their films on the backlot again," Barron says. "It's not the right idea for every film, but it absolutely was on Down With Love because we had to create a New York that doesn't exist anymore. You don't need to go on location if it's not really there."
Barbara Robertson is a contributing editor of Computer Graphics World and a freelance journalist specializing in computer graphics, visual effects, and animation. She can be reached at BarbaraRR@comcast.net.
Adobe Systems www.adobe.com
Alias Systems www.alias.com
Avid Technology www.avid.com
auto.des.sys www.formz.com
CinePaint www.cinepaint.com
Corel www.corel.com
Discreet www.discreet.com
NewTek www.newtek.com
Splutterfish www.splutterfish.com