Seeing Double
Issue: Volume 34 Issue 6: (June/July 2011)

Seeing Double

The old joke is: In LA, everyone has a screenplay to pitch. Here’s the new joke: Everyone with a game to pitch goes to LA. The City of Angels is not just home to Hollywood, but also home to E3, the annual expo where game developers, game publishers, console makers, and their fans come together and willingly endure sensory overload.

Last year at the exposition, Alex Neuse and Mike Roush from the independent studio Gaijin Games did something many industry insiders would consider ill-advised. They pitched their game concept for Laserlife (a mix of music, rhythm, motion-triggered action, and adventure) to Nintendo, even though they knew it wouldn’t work on the Wii, Nintendo’s star console. Perhaps because of the list of projects the pair had worked on at LucasArts and Santa Cruz Games, they felt they could get away with it. Perhaps the fellowship of game developers influenced someone at Nintendo to give them an audience. Whatever the reason, Nintendo decided to hear them out. After­ward, instead of shutting the doors on them, Nintendo opened a new door. The console maker suggested they should pitch their Bit.Trip game series, already available on the Wii, for the Nintendo 3DS, the company’s new handheld device with a glasses-free stereo 3D display.

Here’s another sea change. In the past, talented game developers looked to Hollywood visual effects as the next step in their career. Today, when a successful game company needs a spruced-up cinematic, they come knocking on the doors of top Hollywood talent. Vernon Wilbert Jr.’s CG and VFX have appeared in XXX and I, Robot, to name but two films. In March, he got a call from Guerilla Games to direct the opening sequence of Kill­zone 3, featuring a futuristic world under a Third Reich-style dictatorship. Killzone 3 marks Guerilla Games’ first assault on Sony’s stereo 3D-enabled PS3 console.

Roush and Wilbert are among the first wave of digital artists playing in another dimension, unlocked by a new generation of handhelds and consoles. Along the way, they’ll figure out what works well and what hampers storytelling and gameplay when their target audience is literally seeing double at all times—two simulated views of the same scene.

Tomorrow’s Tech, Yesterday’s Aesthetics

Between March 2009 and December 2010, Gaijin Games released six games—Bit.Trip Beat, Bit.Trip Core, Bit.Trip Void, Bit.Trip Runner, Bit.Trip Fate, and Bit.Trip Flux—for the Nintendo Wii. (Two of the games, Beat and Runner, are now also available for the PC and Mac.) Deliberately designed to resemble classic arcade games, the Bit.Trip saga (as the collection is now called) leads players into a psychedelic-colored landscape populated with Atari-style geometric shapes. Adam Rosenberg, a contributor to the Web site Digital Trends, summed up the Bit.Trip gameplay in a single word: “hypnotic.” One of the games, Runner, claimed the Excellence in Visual Art award at the 13th Independent Games Festival (run by those who oversee the annual Game Developers Conference).

Roush acknowledges the Bit.Trip saga’s look is an homage to the type of games he and many others grew up playing in their youth (a bit-trip down memory lane, so to speak). “We’re using the same kind of state-of-the-art hardware used to make movies like Star Wars,” notes Roush with a chuckle. “But we’re trying to emulate the type of Atari games from 1979.”

When developing for the Wii, the simplicity of the arcade style primarily driven by geometric blocks and bright backgrounds allowed them to keep their games within Wii’s stringent file-size limit. “WiiWare has a 40mb limit per game,” recalls Roush. “Our music took up 35mb, so there wasn’t a lot left for textures.”

EA has embraced stereo 3D gaming with its recent releases (on opposite page) American McGee’s action-adventure title Alice: Madness Returns and (above) Crytek’s Crysis 2 first-person shooter, which supports stereo 3D on all platforms—the Xbox 360, PS3, and PC.

After getting their cues—and a development kit—from Nintendo, Gaijin Games set to work on transforming the Bit.Trip games into stereo 3D titles. Roush found out that the simplistic style that served them well on the development for user-downloadable Wii­Ware titles also works well for Nintendo 3DS: Big blocks of floating geometry set against colored backgrounds emphasize the simulated depth of field. Gaijin Games relied primarily on Autodesk Maya and Adobe Photoshop and Illustrator to create the game assets for these games. To prototype and test stereo effects, the team employed Nintendo’s 3DS developer kit.

“We usually like to save our stuff out, compile it, and view it in the dev kit,” says Roush. “We can change the 3D level in real time in the kit. So, essentially, we can tune the depth of field.” In the case of the Bit.Trip games, Gaijin Games was porting titles previously developed for the Wii to a new hardware platform. If they were developing a game specifically for stereo 3D from the ground up, Roush notes, there are certain things the group might have done differently.

Since 3DS’s basic character is slightly different from the heavily motion-driven Wii, certain gameplay strategies have to be reconfigured. “We will use the stylus controls for Beat, Fate, and Flux,” explains Roush. “We tested the motion controls and opted for stylus control. This choice was made because of the fragility of 3D on the device; at a certain angle you just loose the 3D.”

At the present time, the debate over glasses-free stereo 3D versus glasses-enabled stereo 3D remains unsettled. Ken Tidwell, HP’s worldwide graphics product manager in the workstation global business unit, explains the downside with glasses-free stereo, sometimes called auto-stereoscopy: “There’s the sweet-spot issue. You need to be at a certain location or angle to get the effect. If you leave the sweet spot, you’re seeing both images [left-eye view and right-eye view], so it becomes a blur.”

Guerilla Games’ interplanetary shooter Killzone 3, a first-person shooter for the Sony PS3, is the first title in the popular game series to be presented in stereo 3D and the first to embrace the new PlayStation Move motion-sensing controller.

Big, chunky geometry, Roush has observed, works better than granular elements. “At a certain point, small things in the background just turn to noise [in stereo 3D],” he remarks.

From TRON’s Universe to Killzone 3’s Helghan EmpireDigital Domain’s Vernon Wilbert had delved into stereo 3D before—on the TRON: Legacy movie teaser. “When working on the TRON: Legacy teaser, we started the animation in Maya, then moved it into [Autodesk’s] 3ds Max. Maya at the time did not have what it has today, which was the stereoscopic camera capability,” says Wilbert. “Now, we’re dealing with a piece of software that understands two cameras. That increased our productivity dramatically.”

Beginning with Maya 2010, users have the option to activate a stereoscopic camera in the modeling window. With this function, artists can view their scenes in anaglyph mode-simultaneously looking at both the left-eye and right-eye views (they will see objects in two overlapping shades, as if they were slightly misaligned, but that is, in fact, how stereoscopic content looks to normal eyes).

“You’re dealing with another set of images,” says Wilbert of stereo 3D. “But not only that, you’re also dealing with how they relate to each other. We need to be able to judge them at key points in the production line to maintain a quality.”

Working in Maya, Wilbert’s team could adjust the interaxial separation, representing the distance between the virtual lenses that create the stereo effects.

“We could actually look at the shots in Maya, with the ability to tune the anaglyphic effects, convergence points, and interocular distances. That gave us the ability to create the dramatic effect we need right there, in the open scene file,” Wilbert explains.

Inspired by the works and aesthetics of German filmmaker Leni Riefenstahl (best remembered for her controversial documentary Triumph of the Will), the dramatic opening cinematic sequence in Killzone 3 includes sweeping wide-angle shots of marches and military assemblies. Stereo 3D, Wilbert believes, should be “intrinsic to the storytelling process, as opposed to something you tag on in the end. Some think of stereo as a postproduction process. To us, it’s part of the story.”

Gaijin Games is developing its 1980s-style Bit.Trip games, currently available on the Nintendo Wii, for the portable Nintendo DS3, which has a stereo 3D display.

“The careful planning of a stereo 3D production is even more important than in 2D, and, ideally, the use of depth should be addressed even in previs,” observes Brad Peebler, president and co-founder of Luxology. “As production begins, tools like the parallax offset probe inside Luxology’s Modo software help the ‘stereo budget’ of the shot, which is the total amount of parallax offset between the foreground and background.”

Non-conventional 3D

Since stereo 3D is a young medium and launching a brand-new title or franchise is costly, many developers and publishers may scour their existing franchises for conversion to stereo 3D. But it won’t be as simple as a push-button conversion, as Roush and Wilbert have learned.

Roush cautions that, while first-person shooters and role-playing games that fully commit themselves to 3D camera movement, character design, and environment (like Tomb Raider: Underworld, for instance) would work well when converted to stereo 3D, games with 2.5D and 2.75D looks (as exemplified by Blizzard’s Diablo series, for instance) may not fully take advantage of the simulated depth of field that is possible in stereo 3D.

According to Wilbert, fire and smoke effects—imagery that, in flat cinematic, you can just grab the texture from a source file and place it somewhere—would not work well in stereo 3D, unless you understand where that fire or smoke is in the 3D space, where the convergence point is.

The stereoscopic camera capability in Autodesk’s Maya (above) as well as other popular DCC software makes it easier for artists to view their scenes in anaglyph mode for simultaneous right- and left-eye views.

It’s not easy to come up with a formula to decide the added cost for implementing stereo 3D, Wilbert observes. “Some think, ‘Well, for stereo, just add 2 percent to the budget—or some other number.’ That’s dangerous,” he points out. “[The added workload] depends on your content, on how much live action you want to incorporate, for example. When you add live action, it adds another layer of complexity,” as was the case with his work on the TRON: Legacy movie teaser.

The extra camera you can work with in stereo 3D, Wilbert points out, is both an opportunity and a burden. “Be mindful of your focus point—especially if you’re following it,” he advises. “Your convergence has to play a role in [your scene], but don’t become a slave to convergence. My suggestion is to watch it.”

We Need More Trucks

The finished products—Killzone 3, BIT.TRIP games, and other stereo 3D titles—play on different devices, some powered by shutter glasses, others by a glasses-free stereo 3D display. But most development efforts are bound to take place on personal computers. Powerful multicore workstations with professional graphics cards have made it possible for developers to view game assets and animation sequences in real time. “The high-end graphics cards from our two GPU partners—Nvidia and AMD—have stereo output jacks,” says HP’s Tidwell. “You can connect them to your stereo glasses or stereo monitor.”

If the game developer’s hardware were a supermarket and the stereo 3D data were grocery items, Wilbert thinks it’s time to consider the Costco model, better designed for wholesale processing and transfer. “In a typical supermarket,” he observes, “people are buying milk, bread, and eggs by the basketful. At Costco, they’re buying them by the cartful. So if you’re using the same trucks [to restock inventory], you’d need more of them coming in more frequently. Faster machines are always nicer, but the machines, the hardware, and the artists need to be able to maintain a certain rate of throughput to make the production reliable.”

SOCOM 4, Zipper Interactive’s tactical third-person shooter, is another title that will take advantage of stereo 3D play on the PS3. By the end of the year, there could be 30 3D-capable titles available, including many notable sequels.

“Generally, the main need for stereo production is more rendering horsepower,” says Luxology’s Peebler. “Stereo rendering of final frames for a cut sequence using Modo takes no more RAM than single-frame rendering. It’s basically two renders back to back, so there’s only one frame in memory at a time. But doubling the amount of time to render while still maintaining the same rendering quality means people working with stereo 3D rendering will either need faster equipment or more time to get it done.”

PC Game Resurgence?

It might be difficult for younger game players to believe, but once, in the not-so-distant past, the PC was considered a superior game platform. Up until 2001 or 2002, the type of photorealistic visuals delivered by PC games wasn’t possible on consoles because the former offered greater memory, storage, and processing power than the consoles available at the time.

But Microsoft Xbox, Sony PlayStation 2, and Nintendo GameCube changed that. With a simpler hardware setup and graphics that rival (and often surpass) those on the PC, consoles now seem to own a greater share of the consumer’s wallet. With motion-triggered gameplay (Microsoft Kinect and Nintendo Wii) and out-of-the-box stereo 3D (Sony PS3 and Nintendo 3DS), the latest generation of game consoles continue to challenge the PC.

According to a report by the NPD Group, a market research firm, the total console, portable, and PC game software industry generated $10.5 billion in 2009. Of that, PC game software accounted only for $538 million.

Newzoo, which publishes the yearly “Total Consumer Spend” report, which is based on the National Gamers Surveys, reports that total revenues spent on games in the US reached $24 billion in 2010. PC/Mac game downloads and retail box sales accounted for approximately 19 percent of that ($4.6 billion). By contrast, console games accounted for close to 43 percent ($10.6 billion). The rest was distributed among casual game portals, massively multiplayer games, social networks, and mobile devices.

So, how will stereo 3D change the market’s dynamics? Harry McCraken, former editor in chief of PC World magazine, has stated, “I’m skeptical about 3D gaming being a huge deal, simply because it’s been pretty good for several years now, and hasn’t caught on big time. It certainly has niche appeal, and Nvidia has put a lot of energy into it.” Graphics card maker Nvidia has a vested interest in PC games’ return to dominance. Promoting its $149 stereo 3D kit, called 3D Vision, the company states, “Nvidia 3D Vision automatically transforms hundreds of PC games into full-3D right out of the box, without the need for special game patches. By leveraging its ‘The Way It’s Meant to be Played’ program and its close relationships with game developers, Nvidia provides a terrific 3D gaming experience.”

That means while you wait for new stereo 3D games for the PC to be developed, you may invest in a $149 kit to be able to play a vast collection of existing games in stereo 3D mode. Nvidia maintains a list of games recommended for its 3D Vision treatment, along with ratings. According to its chart, you can get excellent stereo 3D performance from titles such as Prince of Persia: The Sands of Time, Assassin’s Creed, and Tomb Raider: Legend. Then there are other titles, like Battlefield 2142, The Sims 3, and others that fall into the Good, Fair, Poor, and Not Recommended categories.

Closer to the Holodeck

Gaijin Games’ Roush, who wears glasses to aid his eyesight, acknowledges that he has difficulty experiencing stereo 3D with shutter glasses. He is thankful that he happens to be working on a stereo 3D platform that doesn’t require glasses. “3D is such a young medium right now. It’s just going to get bigger,” he says. “Being in the first round of game developers to work on stereo 3D is an honor.” From the blocky, pixelated graphics on Atari to the latest breed of stereo 3D games, the evolution of interactive entertainment points to simulating reality as closely as possible. Perhaps that’s why Wilbert calls stereo 3D games “a push toward the holodeck,” the reality simulation facility in the fictional universe of Star Trek. “It’s getting you one step closer to the holodeck,” Wilbert says. “Stereo is the gateway to something else.”

Kenneth Wong is a freelance writer who covers the CAD, DCC, and game development industries. A recovering Age of Empires addict, he was nearly banned from a local Best Buy for hogging the Nintendo 3DS display unit for hours (doing research for this article). He can be reached at