Getting Unreal

Leave it to a “former game developer, turned Hollywood executive, turned filmmaker” to utilize UE3 in a whole new way. Jason (Jace) Hall, founder of Hollywood entertainment company HDFilms, has spent the past year or so creating a pipeline that he hopes will change the way CG entertainment is developed for the Web and television.

A short time back, Hall had overseen the creation of the Lithtech engine at Monolith Productions, developed in conjunction with Microsoft, and was the primary person responsible for licensing the technology to other developers. “Because of my background in developing games, I have a lot of experience understanding how in-game cinematics come together, what the limitations can be, and once you set the limitations, how rapidly you can make changes or get someone up to speed who has limited experience in constructing cinematics,” explains Hall. “I thought, in theory, you could take game technology and create a one-hour story.”

Today, Hall is putting that theory to the test—two times over. First, he is working on “Chadam,” an original 10-part Web series featuring characters by Alex Pardee that he created for the band The Used. More than a year ago, Warner Bros. Television Group (WBTVG) announced plans to offer the series through its Studio 2.0 production division. According to Hall, that should occur sometime this year. Hall’s second project is still undecided, though he says he would like to try something less violent and less dark than “Chadam,” suggesting a sugary girl-­focused story with music as one possible idea. The concept would be to show two diverse examples of how the development pipeline he’s laying—on top of the UE3 foundation—can be implemented for any type of CG project. Hall is especially interested in bridging the gap between linear and interactive entertainment. After all, he got his start developing first-person shooters.

Hall founded development studio Monolith in 1994, which went on to create games such as Shogo, No One Lives Forever, FEAR, and The Matrix Online. Seven months after Warner Bros. Interactive Entertainment (WBIE) hired Hall as its senior vice president in January 2004, WBIE purchased Monolith. During his tenure, Hall oversaw a number of game hits (FEAR, Batman Begins) and misses (Catwoman, Superman Returns). In February 2007, he left the executive world to seek out new creative opportunities at HDFilms.

The story that Hall and his team are telling through HDFilms focuses on the character Chadam’s battle against Viceroy, a serial killer who’s out to destroy the world. The whole concept revolves around the use of the imagination and an overarching story about how someone’s imagination can be so powerful that it can affect the lives of those around that person.

Laying the Pipeline
Hall has an in-house core team of eight people working on world construction for “Chadam.” The art pipelines were designed similar to the way games are built today, except the team didn’t have to be concerned with gameplay or frame rate. Instead, it’s all about telling a story and pushing resolution. 

“Our goal wasn’t to create the most realistic, detailed world, but more on developing a story,” says Hall. “However, some of the sets built for ‘Chadam’ are pushing well over hundreds of thousand triangles, along with some textures at 4096x4096 pixels. Since our concerns were renders rather than frame rates, our limitations, as far as quality was concerned, is pretty much, at the moment, limited to our computer specs.” 

Intended as a Web and television series, “Chadam” is a CG project whose production pipeline is built on top of the Unreal Engine 3 game engine.

HDFilms is working with Exigent, former game developer Paul Steed’s company, to re-create the original character models designed by Pardee for use in the Unreal engine. “Paul Steed is well known for being the key character modeler in top video game franchises, such as Quake,” says Hall. “Taking two-dimensional characters that were not designed for 3D space has its challenges, and it is important to have someone building your models who understands and brings an essential creative component to the mix to help make it all work.”

A good portion of the work is actually done outside the game engine. The artists build and animate the characters mainly using Autodesk’s 3ds Max. Because many of the characters have odd designs and proportions, keyframing offers the most suitable animation option. In addition to 3ds Max, the group also is doing some keyframing within Unreal’s Matinee. According to Hall, there’s a lot that can be done in Matinee that helps development on the fly; for instance, the team is able to review many of its changes in real time, rather than waiting for new renders within 3ds Max.

For the facial animation, the artists used OC3 Entertainment’s FaceFX. 

For some action sequences, the group turned to motion capture, provided by House of Moves (HOM). The goal was to shoot everything in one day, but in the end, an additional day was required for pickups. The main stage at HOM is equipped with 64 Vicon MX-F40 cameras for capturing body and basic finger articulation.For the “Chadam” project, the team focused solely on the movements of the characters’ bodies, which was done inside the 70x40x22-foot volume. Later, the data was applied to the characters with Autodesk’s MotionBuilder.

Hall says collaboration has been a key component of this project, even when it came to outsourcing the mocap work. Rather than walking in and asking for a list of animations, he instead requested that the HOM team read the “Chadam” script to determine which actions would be handled more efficiently and creatively through motion capture as opposed to keyframing.

Within the ‘Chadam’ World
Digital ToyBox, led by Landon Pascual, is filling in the environments and creating the special effects. The artists use primarily 3ds Max, along with Pixologic’s ZBrush and Ryan Clark’s CrazyBump, for the environments. Most of the special effects, on the other hand, are handled by the particle system within the Unreal engine.

The lighting, meanwhile, also is handled within Unreal. “We’re using a heavy amount of dynamic lighting and high-quality ambient occlusion. There’s no need to optimize based on what we’re doing. The results are in real time and look beautiful,” Hall explains. “What you’re going to see will be in-game renders.”

The size of the “Chadam” team is actually a little smaller than those Hall assembled in the early days of Monolith’s game development. When creating a game and cinematics for in-game presentation, developers don’t have the advantage of controlling exactly what the player will see at all times because of the interactivity. Yet, they have to live within the game-specific environments because [the scenes] will run from a level to an in-game cinematic. In contrast, HDFilms operates much more like a film studio. There doesn’t have to be a back to the buildings in the world of “Chadam.” The crew has the luxury of knowing it can go back and edit these UE3-rendered scenes and switch back and forth from camera angle to camera angle with a nonlinear editing system.

All the camera work is handled in Unreal through Matinee. Once the scenes are set up, the renders are dumped as single images and composited in Adobe’s Premiere.

“We’re designing our show around whatever the limitations are for game technology,” explains Hall. “That makes the size of the team manageable because we don’t need 50 people or an army of programmers. When you’re doing a game, you’re trying to make something that surpasses what’s previously been done with the Unreal Engine and distinguish yourself that way. In this case, [we don’t believe anyone has] ever done anything like this with the Unreal Engine.”

That said, Hall emphasizes that he’s not out to create the next Shrek, nor does he have the budget to do so. (Confidentiality agreements prohibit Hall from revealing the sum.) Visually speaking, there’s a certain level of realism in “Chadam,” a look that is in step with the story: Chadam, whose strong imagination can physically alter his environment, lives in a hyper-stylized, metropolitan island city of Vulture.
All the imagery is being captured in high definition at 1920x1080. Hall reasons that the Web series should be an improvement over traditional television content done in CG because there will be more content generated with HDFilms’ game-development process than a “traditional” production at the same budget.

Hall offers this analogy to support his point: “If I only give you $10 for gas, you will go much farther in a Prius than a Ferrari. So, when you are trying to get the most distance with limited money, you would use a Prius because it processes fuel differently,” he explains. “If you had as much gas as you need no matter what, or if you need to go fast, well then, you would use the Ferrari.”

Even so, Hall acknowledges that in terms of his analogy, a Prius is not supposed to replace a Ferrari. “We are living with a controlled budget and building a machine that can get the most distance while still looking great. We are not trying to negate the need for [traditional 3D software] or traditional 3D productions.”

Some Bumps
Like with any project, the “Chadam” crew has encountered its share of bumps along the road. One of the major limitations was not due to a lack of realism, contends Hall. Rather, it was the number of sets and the total number of animations being made.

Former founder of game developer Monolith, Jace Hall is trying to bring game techniques to the Hollywood stage in an attempt to revolutionize and streamline the production process for television and online projects.

“With any 3D animated project, artists essentially have an infinite amount of resolution in terms of how detailed their animations, textures, and models are going to be,” explains Hall. “Part of the genius of what we’re trying to do is control the overarching budget. The trick is figuring out what you can accomplish with a fixed budget, and how compelling you can make that content. It’s hard because it is a subjective scale. For every piece, whether it’s the special effects, the models, or the level of detail in the animation, they can go on forever.”

As of press time, Hall and his team were still tweaking and learning as they fine-tuned their creation and the new pipeline. WBTVG has given the project plenty of time to gestate in the hope of getting it right the first time.

An Unreal Editor
Selection of UE3 was an economical move from the beginning. One of the reasons Hall chose to buy the UE3 license was because Epic Games releases its complete UE3 tool set to gamers through PC titles like Gears of War and Unreal Tourna­­ment 3. So, in effect, he got the kit for free. Hall then partnered with a graduate from The Art Institute who had a UE3 background to lead this project and assemble a team, thus further controlling costs.

“You have this tool set and all these people who know how to use it,” says Hall. “Unreal has become pervasive. It turns out you only need a few experts in terms of payroll, and they can guide others who are newer to this, and you can create something like ‘Chadam.’”

Epic Games designed UE3 this way, and the key tool that has opened the door once segregated to programmers is indeed Matinee, which HDFilms is using extensively for keyframing and camera work. In addition, this software within UE3 gives artists control over lights and particles. In addition, it offers the ability to edit between shots.

Artists are using primarily 3ds Max to model and animate the characters, and Max and ZBrush are the tools of choice for the backgrounds.

“Anyone who has worked on any type of software in Hollywood, whether it is Adobe After Effects or Premiere, Apple Final Cut Pro, or Avid Media Composer, can easily jump in and see how Matinee works,” says Greg Mitchell, digital cinematographer at Epic Games. “Hollywood is used to working in a system whereby scenes are rendered out shot by shot. In one particular film scene, there can be 20 shots numbered individually. UE3 has the director track in it, which allows you to plot out the entire scene in the engine and then actually cut your cameras from shot to shot. You can have an entire scene in one file and then maintain it in the one UE3 scene, instead of having to break it up into small scenes.”

Although obviously biased, it is Mitchell’s opinion that the traditional Hollywood method of progressing from a moving storyboard or animatic phase to a final master offers a road map with little flexibility. With Matinee, creators can look at things with extra camera shots and then lock down what they want, as opposed to having it planned out from the beginning. It’s this on-the-fly nature that has allowed “Chadam” to progress so smoothly—even if this isn’t exactly Hollywood, yet.

“As we’ve seen in game development, when you give artists tools that allow them to make modifications instantly and see results instantly, they have an entirely higher level of productivity,” claims Tim Sweeney, founder of Epic Games and creator of the Unreal Engine technology. “Even if our tools are simpler and less powerful than [traditional modeling and animation tools], they allow artists to do cool stuff faster. I think in many cases that’s better.”

Sweeney continues: “We’re about to see a complete inversion of that marketplace where real-time game tools take over. With artists’ time being so valuable, they don’t want to wait around for a day for previsualization of their scenes; they want to see it right now. If they can see it right now, they don’t need as many special effects as they would have had in the offline render.”

Nevertheless, even HDFilms did not completely go real time for “Chadam,” and the artists continue to use traditional 3D modeling and animation software for the core work. Rather, the gaming tools are supplementing these packages.

A Virtual Backlot
That does not negate any of the advantages of HDFilms’ new system, including the ability to create a virtual backlot of character models, environments, and animations that can be fine-tuned and ported to television or video games in the future. The fact that “Chadam” was created inside a video game engine makes the transition to interactive entertainment natural. According to Hall, all the money that is being spent creating these assets translates almost directly into a video game production.

“You come out ahead of the game producing something like this and dovetailing into a game, and vice versa,” contends Hall.

If you create a game, you can go into the kind of production we’re doing with “Chadam,” Hall says, but notes that you’ll have to make some changes. In games, you’re living with certain memory constraints concerning texture sizes and the number of animation types, and we’re not, he explains. The artists can load up a scene with huge textures, knowing that’s the only place viewers are going to see, and the PC is not going to have to render a whole city behind it.

Also, the group can be extremely detailed and specific in its animations; they know they are just going to use the animation for that scene. “You have to run a balance in a game,” Hall adds. “However, there’s no question that it’s a huge advantage moving laterally.”

UE3’s Matinee enables the artists to review changes in real time. The group also used the tool for some keyframing, camera work, lighting, and particles.

Hall certainly hopes “Chadam” finds an online audience, whether broadband or wireless, and hopes to bring these characters to both television and video games in the future, as well. But the ultimate goal with this project, outside of entertaining an online audience, is to prove that UE3 can be used for much more than creating blockbuster games.

“Our goal is to set up a way of producing these kinds of animations that are incredibly cost-effective and offer opportunities for beginning to intermediate artists to get involved and produce something that’s great under the supervision of an advanced artist,” explains Hall. “Essentially, we’ll have created a cost-effective way to create a type of 3D animation.

Hall continues: “Although you can do cost-effective animation with [traditional DCC software], the issue is how it looks at a budget of, let’s say, $100,000. If you do the same thing using Unreal and our process, I believe our version will almost always contain more content under that compressed budget. We’re not trying to replace [those other tools]; it’s just a different avenue for a controlled budget to produce these things for the Internet or television.”

So far, Hall has had an open informational policy regarding this process, but he cautions that it may turn out that everything his team has learned becomes so significant that he won’t want to give away the golden goose. Even if he does decide to share this new CG process in more detail, it will be a while before he can actually articulate this to others. The team is still learning, not just with “Chadam,” but with its second test case. Hall believes that “Chadam” will open Hollywood’s eyes to what Unreal can do. And he’s happy to be leading the new wave of CG storytelling using this game engine technology.

John Gaudiosi has been covering the video game business for more than 15 years for outlets such as The Washington Post, Wired Magazine, Reuters, and AOL Games. He can be reached at JGaudiosi@aol.com .