New-Age Enetertainment
Filmmakers—no matter if they are from small, independent studios or from the largest facilities, or somewhere in between—are discovering the advantages of using digital sets. Creating CG backgrounds are saving untold dollars in production costs, since filmmakers, crew, and talent no longer have to travel to exotic locales to have such imagery in the background of their production. Nor do they have to rent endless hours of time on an expensive sound stage. Instead, rich, imaginative 3D imagery is accomplishing the same goals or, in many instances, surpassing them.
The implementation of CG sets is hardly new. However, recent uses—in particular, for the film 300, which employs a unique graphics style thanks to the all-CG backgrounds—has placed the technology in the spotlight.
In May, a small independent studio, Stage 3 Media in Vancouver, British Columbia, debuted a unique Webisodic series, called Sanctuary, which was filmed in HD using greenscreens and CG backgrounds (see “Creating Sanctuary,” pg. 22). In another fairly recent application, a small, indie facility fashioned digital sets to create something new by using something old. Open Sky Entertainment created a remix of the 1919 horror film The Cabinet of Dr. Caligari, whereby modern actors were filmed against greenscreen and inserted in front of digital versions of the original movie sets (see “Something Old, Something New,” pg. 27).
While each production places a new spin on the use of CG sets, the end result is the same. The technology provided the only means for the small, creative groups to achieve their artistic vision.
—Karen Moltenbrey
Creating Sanctuary
A Unique Website uses all-CG sets to create drama on the internet.
A new science-fiction drama recently made its worldwide screen debut. The episodic series is called Sanctuary, and it is fast becoming a hit. The program touts a number of well-known stars, including Amanda Tapping (Stargate: SG-1, Stargate Atlantis) and Robin Dunne (Dawson’s Creek, Species 3), and features an abundance of advanced visual effects. But don’t look for it on cable television; it can be seen only on the Internet. Unique for a drama, Sanctuary also offers a number of innovative interactive elements. Moreover, an entire season was filmed in a mere 20 days. And the overall production costs have been a fraction of those for a comparable television series of the same depth and breadth.
Taking advantage of many new digital trends and technologies, the creators of the Internet series Sanctuary were able to deliver a compelling episodic story, along with rich, HD imagery at a reasonable cost for an indie facility. Sanctuary, which takes viewers into a thrilling world where science meets the supernatural, was filmed using greenscreen, with lavish, detailed CG sets serving as the backdrops to the action and drama.
Shooting live-action footage against greenscreen is a growing trend, particularly in light of some recent successes—in particular, Sin City and 300, the movie adaptations of Frank Miller’s graphic novels (see “The Art of War,” March 2007, pg. 21). “I feel like we’re riding the crest of an amazing wave,” says Tapping, who also serves as an executive producer for Sanctuary.
The reasons for this trend are simple: Using 3D sets costs less than filming on expensive sound stages and allows small, independent companies to make their own mark in the entertainment world. Digital sets also allow for more creative freedom, as there are virtually no limitations placed on the camera in the virtual space or on the artist when crafting the sets. Furthermore, the technology enables the director to try new things, and by bringing the series to the Internet (as in this case), the property can be augmented with interactive elements and more.
For a series like Sanctuary, these advantages are invaluable. According to visual effects producer Ron Martin, without the use of virtual environments, it would have been nearly impossible for an independent production company like Stage 3 Media to produce such a compelling show, let alone do so in HD. In addition, Sanctuary features unworldly creatures and imaginative locales that are far easier to create using computer graphics.
The show follows the exploits of Dr. Helen Magnus (Tapping) as she seeks out all types of terrifying and monstrous creatures. She is aided in her quest by her reluctant protégé, Will Zimmerman (Dunne). The world they inhabit is mysterious and populated by beings that defy explanation. “We’ve heard people say that Sanctuary is sort of like Buffy the Vampire Slayer meets X-Files,” says Martin. “We get goose bumps because those are two pedigree products we are being compared to, and it heightens our enthusiasm just knowing we are venturing into the same genre.”
While Sanctuary is only about a year old in production terms, the script is approximately nine years old, generated by the show’s writer/creator and executive producer, Damian Kindler (Stargate). A few years back, Kindler entered a casual conversation with his friend, Electronic Arts VP and executive producer Marc Aubanel concerning the difficulty and expense of producing media, particularly at large conglomerates. When idle talk turned into more serious consideration, they approached Martin, who was working as director of cinematics at EA, and he offered his insight into using a digital approach to storytelling. Soon thereafter, the group approached director Martin Wood, who contributed his expertise in television production. “We realized that creating a quality product would require utilizing every one of our faculties and our connections,” says Martin.
The Virtual World Founded in 2006, Stage 3 Media began production on Sanctuary with an extensive previsualization process. Using Autodesk’s MotionBuilder, a group of artists blocked scenes with virtual characters and a virtual camera. “Because we shot greenscreen, all our sets were modeled in CG, so we could import those dimensions and sets right into our previz. In previz, we could manipulate the position of our sets easily for a more efficient production,” explains Martin. By using digital actors, or digital dummies, the artists blocked out the shots, giving them a clear picture of what had to be modeled in 3D and where details had to be added to make the shots more interesting.
Then, the artists, led by art director Todd Van Hulzen (scenic designer for Girl With the Pearl Earring), began building the virtual sets that would transport viewers into the strange, Gothic-style world of Sanctuary, which is populated by things that are both seen and unseen. “The shooting method that helps convey our visual style is ‘a thousand eyes’—that, at any point, you can be watched by a creature of sorts or by those from a much higher level of society,” Martin says. “That affects how we use our camera. In our visual style, we break the boundaries of conventional TV.”
Because the group is using virtual sets, it is not confined by what would normally be a TV stage, or set, with three walls and no ceilings. “In that conventional shooting methodology, you would be pointing the camera at characters and would cut between the characters,” explains Martin. “We can shoot very low shots or crane up very high and extend not only our sets into the x and z axes, but also on the y axis, so we can create a much more dramatic camera move than we could by using conventional television sets. We can also shoot 360 degrees, and have been doing that with our cuts, working around a set to show all 360 degrees of it.”
Strange activity occurs in The Sanctuary, located beneath an old cathedral, which, like all the structures in the series, was created with Softimage XSI.
With digital sets, the art team was virtually unlimited in its ability to explore concepts and designs that were representations and extensions of the characters, thereby furthering the Sanctuary story line. For instance, the time period of Sanctuary is in the near alternate future, and the story takes place in two very different locations: Old Town, a dark, ominous, rundown, nearly deserted area, and New Town, a big, bright, modern section across the river. Linking the two locales is an elevated train whose ride through Old Town is as risky as a middle-of-the-night walk through the dark alleys in the same part of town.
“When designing this world, we took an approach that there are two different levels of beings. There are people who live in New Town who embrace technology and use it to fulfill their lust for power. Old Town is a dilapidated, more conventional part of the city along the river that was abandoned for New Town because of some incident,” explains Martin, careful to not give away a story spoiler. “It is as contrasting as the Emerald City is to the land of the Munchkins.” This gap extends to the differences between the people who live in New Town and those who inhabit Old Town.
Situated in Old Town is The Sanctuary, located six stories underneath a burned-out Gothic cathedral. It is here where the seemingly ageless 157-year-old Dr. Magnus has established her secret lab; inside shows a contrast of old and new, with high-tech gear situated beside 1880s bottles and devices. The people of Old Town are aware that something suspicious is occurring at the lab under what was once a picture of the city’s architectural height and grace, but exactly what that is escapes them.
The Sanctuary, like all the other locations in the series—whether an indoor or outdoor setting, from the bowels of the cathedral to the heights of the 50-foot skyscrapers—are completely computer-generated, created in Softimage’s XSI. By creating the sets in 3D rather than using 2D matte paintings, the artists are able to reuse them in every which way possible—for example, for successive storytelling in future episodes. Additionally, 3D provides the freedom of camera in which to show a rich environment by panning and tilting for multiple perspectives.
Likewise setting the scene are a range of CG effects, such as fog, atmospheric elements, fluid dynamics, particle and hair simulations, and more, also created within XSI. In fact, XSI is the modeling, animation, texturing, and rendering tool at the center of the series’ production. Unlike a typical studio production pipeline, Stage 3 Media is designed around a multi-faceted group of artists who perform a multitude of different functions. “That is one of the reasons why we have been so successful,” Martin explains.
In addition, the artists use Pixologic’s ZBrush for painting, Autodesk’s Combustion for rotoscoping and compositing, and Mental Images’ Mental Ray for rendering. Nvidia’s Quadro FX 5600 cards facilitated the modeling and rendering of the content. As Martin points out, the team maintains its core competencies throughout the production. “When we add more tools, it tends to dilute the ability for people to jump from scene to scene,” he says. “I would rather be much more clever with the tools we have than to throw technology at a problem and try to train a talent to wrangle with the technology. Instead, we keep a focused approach to what we are developing, and that ultimately keeps our production costs lower. We don’t need more tools; we need the tools to get better.”
To augment that key software, the group created its own asset management system, coined DAMS, that is a client/server-based tool for previewing 2D renders, 3D objects, and artwork created using Python, HTML, and PHP. It can be accessed remotely from any locale.
The Real World
Filming for the series takes place at the Bridge Studio soundstage in Vancouver, British Columbia, not far from Stage 3 Media. The group employed a Panasonic HDX-900 720p-1080i HD camera with a custom lens mount for prime lenses that provided a more cinematic appeal to the image quality. “We needed a little more than the 4:2:0 color space,” notes Martin. The footage was shot directly to hard disc using a Focus Enhancements FireStore portable FireWire drive. As a result of digitizing the imagery on set and off-loading the material directly onto an editing timeline, the group managed to stay within a day and a half of the shooting schedule with the edit.
Above left shows one of the actors alongside a vehicle, filmed against greenscreen. Above right is the final shot, with the actor and car composited onto an all-CG background. With the exception of the actors and some props, all the imagery in the series is CG. Filming of the entire first season (which comprises eight 15-minute episodes, or two hours of content) occurred over 20 days. As a result, technology and production costs were minimal. Editing and color correction was accomplished on dual-quad 3.0ghz Mac Pros (with Quadro FX 5600 cards) and running Apple’s Final Cut Pro.
In the near future, the team plans to place actors and vehicles in front of greenscreen atop a turntable, which will rotate as the crew tracks the camera. “We are expanding our concept of a digital set,” Martin says. For now, though, the group uses a 3D Inserter, developed by Mosys, which enables the group to take the live feed from the camera and do a live composite on set to see where the actors are situated in relation to the actual physical set. As a result, they can make adjustments accordingly on set.
Unlike the sets themselves, most of the props that are handled by the actors are practical, though from time to time prop extensions and prop effects are added in CG. While most of the actors are real, some are not. Seeking shelter in the sanctuary are a number of monsters, or abnormals. Sometimes these are live characters with prosthetics or CG appendages; other times they are all-CG.
Web Advantages
Stage 3 Media’s use of digital technology is not limited to the sets and characters. Rather, the company is employing a Web-based delivery model to “air” the episodes. Every two weeks starting this past May, a new, roughly 15-minute segment was made available for $2 each at
www.santuaryforall.com; previous episodes are also available. The firm hopes to begin a next “season” this fall, following a brief hiatus.
Stage 3 Media currently has teamed with VitalStream, a content distribution network with servers worldwide, to efficiently deliver Sanctuary from the nearest node to the user. Revenue is acquired solely through subscription, so users will not experience annoying pop-ups or other distracting advertisements. Presently, Stage 3 Media is developing its own platform to distribute and play its own media, a Flash-type technology. Meanwhile, the company is looking into linear distribution methods as well.
By creating the sets in 3D rather than 2D, the producers are able to reuse the imagery in subsequent episodes. And, Stage 3 Media is not stopping its technical development at the point of being able to deliver the show. “We have tracked the camera data off the set on every single shot that we are going to use as part of our player. This will be a 3D engine that will allow viewers to pause the show and walk onto the digital set and move around within the set,” Martin says. Indeed, the viewer would be locked to the camera plane while watching the show and would not be able to run the video while being situated within the set. Nevertheless, this capability would provide a novel, enhanced experience.
In addition, the company is building a social networking engine, and will be hosting contests and games within the 3D space on the site, all of which will be available in the near future. “We started the process with the show, but we are not stopping there,” Martin promises. “The goal is to make an enriched media experience with the content from Sanctuary and its story line. Then, while writing and finishing the script, we have the potential for other methods of involvement, for example, through interactivity with three-dimensional objects or sets, or by meeting someone in the world of Sanctuary similar to what occurs in Second Life.”
From a production standpoint, the company is focused on advancing the interactive producer ideals of the show by creating a multi-format user experience. As a result, the group will focus on a user-driven model whereby the producers will listen to the fans and react accordingly. “That feedback is instantaneous on the Web, which is ideal for this type of a project because after we write and develop scripts, we can hear what people like and don’t like, and anticipate the viewers’ desires and then pleasantly surprise them by using the information they provide in ways that they never would have guessed,” says Martin.
New Beginnings
Using these atypical methodologies, Stage 3 Media was able to do the unthinkable: produce its own episodic series. Though Martin could not provide final figures, he notes that the pilot—an engaging 23-minute peek at the intriguing world of Sanctuary—cost close to $3 million. In contrast, the production fees alone (sans postproduction and to-air costs) to do the same work for television would have been more than $10 million. That ratio holds throughout the rest of production, he says.
“We can do this because we are our own full-service production facility. We shot, produced, wrote, designed, built the VFX, and distributed the content in an HD format,” says Martin. “It would have been impossible to do any other way given our budget and time frame.”
Yet, employing a digital delivery model poses a risk. Are there enough viewers who, at this time, are ready and willing to try something different? Martin believes so, pointing to how digital downloads have revolutionized the music industry. Moreover, he believes that the sci-fi nature of the show will draw more technically oriented viewers. “Our target audience comprises those who use computers and appreciate an action story that can suspend their belief system within an alternate universe,” he says.
Adds Kindler: “This is the next evolution of series-based entertainment. Sanctuary is the first series developed exclusively for an online audience that offers both standard- and high-definition resolutions, plus direct communication between the fans and the Sanctuary creative team.”
And there is no better time for an independent filmmaker to enter the entertainment space, particularly with directors like David Fincher and Robert Rodriquez—both whom Martin lauds—leading the charge. So, it’s hardly surprising that Stage 3 Media is planning other productions. “We have proven that for a small amount of money, we can produce a broadcast-quality production in HD that we believe rivals the quality of production on TV right now,” Martin says, stopping short of a reveal.
Just as audiences are no longer limited in their entertainment selections by the large television networks, so, too, are those tuning in to the Web, where a number a small, independent studios like Stage 3 Media are offering “sanctuary” from cookie-cutter productions.
Karen Moltenbrey is the chief editor for Computer Graphics World.
Something Old, Something New
Remixing an old silent film: Using today's actors on yesteryear's sets
by Karen Moltenbrey 
A small facility remakes an old film classic by using original backgrounds. Above is a completed shot; below shows the actors and props, shot against greenscreen.
Many movie enthusiasts love the look of the old silent movies—the subtitles, the grainy and jumpy black-and-white film, the locked-down camera, and the over-dramatization. These features account for the charm, they say; to others, they make the film difficult to watch. Recently an effects artist decided to eliminate those obstacles in the classic 1919 horror film The Cabinet of Dr. Caligari by remixing the movie, using the original sets as digital backgrounds. As a result, the project set a milestone, marking the first time that original sets from a silent-era film were used as backdrops that integrated present-day actors.
The director contemporized the original story and film by adding new dialog and using a new technique in on-set compositing whereby the actors were shot completely on greenscreen and composited into the originals sets of the original film.
The project is the brainchild of David Fisher, a VFX artist and former game cinematics creator, who directed the remix for Highlander Films. Christopher Duddy, cofounder and executive producer at the film production company Open Sky Entertainment, served as cinematographer. The remix was released on DVD this past summer by Image Entertainment.
“As evidenced by the extraordinary success of the film 300 (see “The Art of War,” March 2007, pg. 20), the blending of live action against CG backgrounds is certain to become a bigger and bigger part of the future of cinema,” says Duddy. “The major difference between our film and a film like 300 is the time and budget. We shot our movie in nine days for less than $150,000.”
To acquire the backdrops, the group digitally scanned a goodquality print of the original movie and then tried to select frames that didn’t have any actors in them. During a two-week period, the present-day filming was done, with the contemporary actors. Then, during several months of postproduction work, the actors were digitally inserted into the scenes.
Making the Cut
So, what made Dr. Caligari an ideal candidate for this exercise? “The original movie is one of my favorites,” says Fisher. He loves the aesthetic of German Expressionism—in particular, the crazy, tilted, skewed worlds of Dr. Caligari. On a story level, it has a classic detective-esque story line running on the top, along with a lot of metaphor and psychological horror at the sub-story level. Moreover, Fisher also likes the modern versions of works such as Tim Burton’s Edward Scissorhands and The Nightmare Before Christmas. “It’s a style I like,” he says simply.
One day, while watching Dr. Caligari, the idea came to him. “I thought, this is such a great movie, but it is not accessible to modern audiences,” says Fisher. “The hard-core fans hate to hear that, but it is a slow-plotting, older movie, and it is silent. Then I began to think how nice it would be if the movie were updated without usurping the original, but rather, exemplifying how great it is.”
Yet, Fisher also realized that redoing this film again independently would be out of the question due to cost and manpower. “It would have been impossible to build all those incredible sets again,” he adds.
Undeterred, Fisher came up with a solution. In the 1990s, he had created cut-scenes for a game called Metal Works that used greenscreens. It was a garage-type 40-minute movie that won first place at the San Francisco SIGGRAPH competition. Because Dr. Caligari is public domain, he reasoned that if he was able to locate a good print, he could pull the backgrounds while shooting actors on greenscreen, and then composite them into the original film. In this way, an epic version of the movie could be created, as opposed to a modern update.
Test shots and composites proved this could be done. Fisher then used those tests to persuade Leonard McLeod and Paula Elins to join the venture as producers. Next, Fisher wrote a script based on the original story outline he located on the Internet, augmenting it with new dialog and a richer exploration of theme and character.
Production
The group cast the new actors within two weeks, while Fisher did the preproduction work. This involved printing the original film frames onto paper and making notations. Yet, there was no camera data from the original production. “I took all the stills I collected for each scene and built a system whereby I would run a tap out of the camera and do a quick and dirty composite, so we could see where people were standing in the set—previz, basically,” says Fisher.
Filming lasted nine days on a stage in Burbank, California, and involved the use of two Panasonic AG-DVX100A cameras that didn’t move more than four feet during the entire movie. “Once I had the shot lined up, I incrementally placed people in the background,” explains Fisher. The group averaged 54 setups a day, all of which were greenscreen shots with the exception of a handful of 2D sets. “A few of the interior sets were flats that we kept repainting and repurposing,” he adds. These included one for the inspector’s office and another for the interior of the caravan. Stairs were green, and anything anyone touched was a prop.
The biggest job of all occurred in post, where Fisher and VFX supervisor Josiah Holmes Howison spent 10 months digitally inserting the actors into the background imagery. First, the duo made a scanned print of the original, working in on uncompressed format. “We got an excellent original print that was spirited out of Nazi Germany in the 1940s. Nevertheless, it was like we had gotten it out of the ocean,” describes Fisher. “It had a strange frame rate, like 25 fps; basically it was hand-cranked, and the shutter had half-frames. I went through and found the best frames I could, and we cleaned those up.” Sometimes there was a clean frame available without any characters, and other times the two men had to stitch pieces of frames together. When neither option was available, Fisher and Howison used Adobe’s Photoshop to paint out an arm or leg, for example.
The team cleaned every frame in the original footage to ensure that the new imagery blended seamlessly with the old.
The task became more complicated in the new scenes as the actors moved through walls and doors. “We never built green partitions, so we did a lot of masking,” says Fisher.
All the compositing work and motion masks were done using Adobe’s After Effects. Fisher and Howison also used some new tools that were just coming out at the time, including dvGarage’s DV Matte Pro, a keyer with light-wrapping tools: “My secret weapon,” says Fisher. “The program was so good that we couldn’t have done the movie without it.”
In addition, the compositors used what Fisher’s assistant calls “the secret sauce,” the ideal combination of settings from eight plug-ins. “The original film has about 10 percent color, though it looks black and white. And we needed a way to comp people into those backgrounds so they didn’t look pasted on,” Fisher continues. “We used off-the-shelf plug-ins that we messed around with for a long time until we finally got the right look to seamlessly punch people into the backgrounds.” On the downside, this resulted in high rendering times.
“The entire movie is basically a 76-minute effects shot,” says Fisher. The film was shot at 724/80 resolution and rendered on eight homegrown Pentium 4 PCs. “Every single frame from the original was cleaned up,” says Fisher. “If you look at the number of passes we did, we probably rendered the whole movie at least eight times over. We would come up with innovations and then go back and redo shots.”
Because of the greenscreen, very little roto work was required. Similarly, there was no tracking needed because the style of the original was locked-down. “If we would have had to track people, there would have been a whole new technical level to address,” says Fisher. While filming the new footage, the group retained the stillness of the movie, though they did add a few camera moves in post to the updated version.
The overall goal was not to replicate the original shot for shot, but rather give it a modern edge while keeping the film in the era. “Some might think the acting is a little choppy, a new film with characters acting old. Some think it’s genius, and others do not understand it, and that’s OK,” says Fisher.
Spark of Innovation
“Just a few years ago, who would have thought it was possible for two guys in a room to digitally make a feature film by placing new people in a very old place using off-the-shelf tools?” asks Fisher.
The director first became interested in digital backgrounds when he saw Tron. “I didn’t know what it was, but I knew I wanted to do it.” Fisher’s dad said it was not possible without millions of dollars’ worth of computer equipment; and back then that may have been the case. But a few decades later, Fisher proved him wrong.
“There are some purists in the world who say, ‘How dare you touch something that’s sacred?’ But I think the majority of people will think it is very cool and doesn’t detract from the old version, and it actually piques their interest in the original and shines light on the classics by making them more accessible,” says Fisher.
This fall Fisher will apply many of the same techniques he used for the remix of Dr. Caligari in his newest project: a Vincent Price-style horror feature based on Edgar Allan Poe, with an indie movie twist. He will be using newly created backdrops in that latest greenscreen project—a process that is adding new color to feature films.
Karen Moltenbrey is the chief editor of Computer Graphics World.