“It’s all about the storytelling, and this, right here,” James Cameron has stated, pointing two fingers directly at his eternally impassioned, turquoise eyes. The legendary director is stressing the importance of adhering strictly to a character’s point of view during filmmaking, forcing the audience to look through their eyes. More importantly, he’s stressing the theme of Avatar, which is to look beyond the prism of our own life experience and, in his words, “see and understand the world through others’ eyes.”
The world of which he speaks is, of course, Pandora, the lush, bioluminescent planet inhabited by the indigenous, 10-foot-tall Na’vi, and despoiled against their will by human mining for a priceless mineral called unobtainium—a superconductor for energy. While the air is unbreathable on Pandora, the mining corporation, called the Resource Development Administration (RDA), creates a human-Na’vi hybrid—the Vishnu-blue Avatar. An RDA soldier, lying in a sarcophagus-like vessel, can project his or her consciousness into the Avatar, controlling the Avatar’s body remotely while interacting with the native Na’vi.
From these interactions spring a moral conflict, centered on their opposing perspectives, which the team at Ubisoft’s Montreal studio seemed to grasp instinctively from the outset and managed to encapsulate in its initial pitch to the Titanic director. The director was so impressed that the studio immediately won the rights to develop James Cameron’s Avatar: The Game.
Working in parallel and using the same digital models, animations, and textures, Ubisoft and Weta forged a relationship that Avatar film director James Cameron called a “perfect consonance” between the film and game crews, in which game artists borrowed models and mocap data for the blue Avatars (above), while Weta borrowed ideas for Pandoran bioluminescence.
|
“They came up with the idea of allowing the player to choose either a Na’vi perspective or a human perspective,” Cameron has said during one of several industry conferences. “In other words, the good guys and the bad guys are entirely a matter of the player’s choice. I think that’s really cool. And there a number of thresholds throughout the game where, if you feel like you’ve made the initial choice incorrectly, you can switch sides further down throughout all the levels. It’s really pretty remarkable, and it tracks beautifully with the moral message of the movie.”
Cameron has likened the Pandora journey to Dorothy’s journey to Oz, and the experience of watching Avatar as “dreaming with your eyes open.” If the film follows the yellow-brick road, the game, at Cameron’s behest, allows the viewer to stray from that narrative brick road into the farthest, most exotic reaches of that open-eyed dream—into 16 diverse environments, each filled with luminescent forests, gorges, gullies, beautiful, floating mountain ranges, and stunning alien flora and fauna.
(Top) Avatar: the Game features 16 rich, diverse environments teeming with exotic flowers and beasts. (Bottom) When a plant is injured in a firefight, the Dunia engine decreases the intensity of its glow shader to simulate its waning life.
|
Perfect Consonance
To accomplish the daunting task of fully realizing Cameron’s blue-bathed, luridly painted alien biosphere in the interactive medium, Ubisoft began its collaboration with the director and the Weta crew in New Zealand, which was tasked with a great deal of the film’s digital imagery, over two and a half years ago. “This is where the whole game and movie interface usually breaks down,” Cameron has said, “because movies are often on a one-year track screaming to the theater, and a year after, someone pushes the button on the game, and it’s just not enough time to develop a game properly.”
Cameron has noted that he set out with Ubisoft to create the ideal model for how a game and a movie should be co-developed, with neither being the redheaded stepchild of the other. Under Cameron’s model, production on the game began concurrently with that of the film. In addition, he has stated, “I proposed that the game should not be a slave to the movie, but should follow its own story line; it should be developed fully in parallel, so that they exist in the same world—using the same creatures and environments—and yet have its own story.” Ubisoft rose to that challenge, creating new characters, vehicles, and weapons, embellishing settings, even enhancing the way the Na’vi interact with creatures and plants, and obtain their powers and poisons.
“As a result,” Cameron has said, “the world of the Avatar game is considerably richer and more extensive than what you’ll see in the film, and at the same time, it doesn’t contain any spoilers that will ruin the movie experience for you, which allows us to put it on the street before the film comes out. This is really the perfect consonance between the two mediums.”
The bulkhead in Cameron’s model for achieving this “perfect consonance” was aggressive asset sharing and two-way collaboration, in which both the film and game teams could develop and share designs and digital assets for each other’s projects. “We received just about every type of reference we could from Lightstorm [Cameron’s production company], from concept designs, characters, creature meshes, and mocap data, to early renders for animation reference,” says Ubisoft artistic director Pascal Blanche. Meanwhile, Ubisoft’s artists—working behind double-locked doors and under high-security cameras and strict confidentiality clauses in their veritable bunker in downtown Montreal—designed everything from vehicles to costumes, even sounds, that Cameron eventually incorporated into his film, marking a watershed moment in film-game convergence.
“We even received entire animatic sequences of scenes depicted in the film,” says Blanche. “Nevertheless, it’s important to understand that for creating the game, they still served somewhat as references. The models needed to be able to work within our Dunia game engine, and within the tighter constraints that we’re given (we don’t yet have Weta-strength über computers, obviously). Even if the game industry gets close to the movie industry’s pipeline, we still have technical obstacles to overcome that place certain demands on our models and textures, so we often had to rebuild Weta’s models based on the various film assets that we received.”
A Bioluminescent World
Spike Tears. Cliff Slouchers. Stinger Ivy. These are but a few of the alien plant species that inhabit the tropical wonderland of Pandora, all of which Cameron based on the strange bioluminescent marine life he discovered while exploring the kelp forests and coral reefs of the deep sea. Modeled, rigged with bones and IK, and animated in Autodesk’s 3ds Max, these plants are living, sentient beings that can cooperate with the Na’vi or antagonize the humans. To re-create the plants for the game, Weta provided Ubisoft with QuickTime turntables of the plant meshes used in the film, so the artists could analyze them from every point of view.
“They also gave us high-res textures from the film, which we used while painting the in-game shaders in Pixologic’s Zbrush,” says Blanche.
At night, the plants luminesce, setting the forest alight like some neon Garden of Eden. To achieve this effect, Ubisoft developed special shaders in Zbrush that were then modified in the studio’s Dunia game engine. The engine, which had been developed for Ubisoft’s Far Cry 2, was heavily modified for Avatar to accommodate such effects and to expand its vegetation technology. “Because the Dunia game engine is time-based, we had the opportunity to increase specific layers of glow values and textures in the shaders,” says Blanche, “giving the whole bioluminescent effect an extraordinary natural, organic feel, depending on the time of day.”
Cameron, a firm believer that the devil is in the details, also insisted that the intensity of the glow should wane when a plant is injured, or wax as it recovers its strength. “For example, when you cut a leaf with a blade or explode one of the many varieties of plants, their bioluminescence will slowly fade as the plant dies,” says Blanche. “We dug pretty deep, pushing Dunia to its limits, to get these effects just right.” Indeed, so extraordinary were some of Ubisoft’s bioluminescent effects that Cameron often asked his Weta team to incorporate them into his film. “I’d often see something really cool that Ubisoft had done, and I’d say to my crew, I want that in the movie—that particular treatment for the bioluminescence, for example,” Cameron has said.
In addition to the nocturnal bioluminescence, the player can also use a flamethrower at night to set the forest ablaze, further complicating the lighting challenges. “A lot of the fire effects had already been implemented and tested in the Dunia engine back when we made Far Cry 2. However, we made some significant changes to improve on the under-canopy atmosphere during these fires,” says Blanche, referring specifically to technological advancements that simulate the dancing, spectral shadows and bounce lighting from the fire and luminescent plants.
The RDA battles the Hammerhead, modeled and rigged in 3ds Max based on assets provided by Weta.
|
“In addition, Pandora has many expansive and dense rain forests, and we wanted to make sure the light rendering of this under-canopy atmosphere matched Cameron’s artistic direction,” notes Blanche. “If you look closely while you’re playing the game—or in certain screenshots—you’ll notice that we’ve added subtle coloration that simulates reflected light on characters, specular lighting in the shadow of the foliage, and ambient colors within the shadows, too. Most of those rendering effects are built into the shaders or into the lighting system itself, using gradients and intensity curves in the Dunia engine.”
Moving in Stereo
“Because Avatar was being made in stereoscopic 3D, I urged Ubisoft to offer the game in 3D,” director James Cameron has stated. “At first they were dubious, but then they came back six months later and showed us a demo in stereo that blew us away. And then, it was on.”
Indeed it was, for Avatar is the first major next-gen game to be developed for stereoscopic 3D. Thick smoke and mist swirl and drift beyond the screen, as the Banshees and twin-propellered helicopters reach out toward you. Ubisoft designed the 3D version of the game specifically for 3D-enabled televisions utilizing stereoscopic technology from Sensio, another growing Montreal company, but it will also work on TVs supporting DLP or RealD technologies. These pricey $4000 sets are capable of running at the 120 hz refresh rate required to project two images at once.
To create the stereoscopic effect, the game engine uses two cameras, one for each eye, placed in parallel with a controllable interocular distance—the space between the left- and the right-eye cameras. The images from the two cameras are then rendered in real time.
Along with the two cameras that are parallel one to the other, the game developers also used the vertex shader’s equivalent of “off-centered projections” to help control the convergence distance—that is, the distance where there is no on-screen separation. Controlling separation and convergence in the vertex shaders enables the artists to correct some special effects and makes them work properly in 3D.
“The convergence is positioned farther than our character to have him slightly out of the screen and to enhance the 3D effect,” says artistic director Pascal Blanche. “As for the left and right images, they are rendered in off-screen buffers and combined for each 3D TV’s specific format.” –Martin McEachern
|
Pandoran Wildlife
While meeting Cameron’s exacting attention to detail in the plant life was difficult, equally challenging was his demand for the most minute details in the Na’vi, humans, Avatars, and wildlife. The notoriously fastidious director wanted the Na’vi to have a specific, graceful way of gliding through the forest, for example; their ears to glow a specific shade of red when backlit; their skin to have a particular, tactile, bioluminescent texture; and they had to shoot their bows with a kind of two-fingered, inverted draw beyond the head.
Here again, Ubisoft was provided with mocap data from the performance-capture volume at Giant Studios in Los Angeles, and digital assets from Weta. They had both photos and turntable scans of the actors from the set, including Sigourney Weaver as Dr. Grace Augustine, Giovanni Ribisi as arrogant RDA soldier Parker Selfridge, Michelle Rodriguez as Trudy Chacon, who functions as the player’s chopper pilot, and Zoe Saldana as the Na’vi princess—most of whom reprise their roles for the game. Referencing the turntables and photos, the game artists hand-modeled each character using 3ds Max.
“The RDA troopers and scientists were easier to animate because of our familiarity with human movement. The Na’vi, however, provided us with a completely new challenge,” says animation project lead Xavier Rang. “At first, we weren’t sure how to animate them. Poring over reference animations, we studied the Na’vi proportions, skeletal structure, and musculature. Their walk and run cycles were some of the hardest to achieve. Luckily, Lightstorm gave us direct access to the animation director from the film, Richard Baneham, who came to Montreal and provided us with a great amount of constructive feedback.”
Working in 3ds Max, animators keyframed the in-game animations for all the characters, creatures, and plants. For several of the scripted cinematics involving humans and Na’vi, however, Ubisoft motion-captured actors at its Montreal studio using Vicon mocap cameras, processing the data in Autodesk’s MotionBuilder and exporting it into the game engine via 3ds Max. “The mocapped performances gave the scripted events a bit more of an edge over the in-game animations, which was crucial to differentiating them,” says Rang.
But animating the humans and Na’vi were a cakewalk compared to the non-bipedal wildlife that stalk the forest floor. Indeed, the forest is a veritable menagerie of ferocious beasts, like the Banshee, the flying blue dragon the player sits astride as it soars through the sky; the slinking, oil-black Viper Wolves; and the six-legged black panther-like Thanator. But none proved more nightmarish to rig and animate than the Leonopteryx, an orange dragon and the largest animal on Pandora. “For the Leonopteryx, we spent nearly a month just re-creating Weta’s animation rig using a custom rig in 3ds Max, so it resembled what we saw in the video references we received from Lightstorm,” says Rang. “We had to learn a lot.”
In terms of modeling, texturing, and rigging the Leonopteryx, Rang says the team put a lot of love into properly capturing this amazing flying beast. There are more than 100 bones in the rig, one skeleton for the data export, and another for controlling the rig. “The main question we constantly asked ourselves was, ‘What makes the Leo stand out from all other creatures, and how can we make it truly recognizable?’????” he relays. To do just that, the artists looked over the length of the wings, the shape of the head and jaw, and the dynamics of liftoff and landing to make sure they got all the details right.
“It was one the most complex challenges we had in creating the game,” Rang adds.
Alien Botany
The almost endless diversity in Pandora’s glowing, coral-reef inspired plant life made procedural methods of creating the foliage difficult to employ. According to lead texture artist Pierre Theriault, those trees and plants—which followed a standard form of stem-with-leaf or trunk-and-branches with leaves—were created using an in-engine plant-creation tool. However, the more complex or alien-shaped plant life had to be hand-modeled by Theriault and his colleagues using a variety of techniques, usually involving Zbrush sculpting of a high-res base mesh or polygon modeling using photo-sourced textures. Therefore, the forests are usually a mix of procedurally or hand-placed objects. But, given the highly organic nature of the alien plants in Pandora, Theriault found himself taking the Zbrush route more often than not.
During gameplay, “foliage zones” are defined in-engine, which are then procedurally populated with the appropriate plant-life assets. “Level artists then did extra detail work, adding focal pieces by hand and adjusting settings and zones for the desired look,” explains Theriault.
The plant shaders were similar to the general-use shaders, encompassing the standard map channels, but sometimes with additional special parameters requested by the artists for translucency effects and glow maps that were controlled by the time of day. Due to memory constraints, sometimes a plant had to forego this special shader and use the general shader when a certain effect was needed, like a cube map or texture blending. In general, anything leafy used the translucent leaf shader, and anything with mass to it used the general-use shader. The shaders went through many experimental changes, during which the crew added and removed features, trying to zero-in on the most efficient version to meet as many of the project’s special demands as possible.
The key to success, says Theriault, was close collaboration with the shader programmer, who managed to accommodate all the group’s requests with great results.
Carpeting the Forest Floor
From the lush, densely foliated landscape of Verdant Pinnacle, to the darker, burned-out palette of Stalker Valley, one of the primary challenges of the game was finding a way to completely carpet the forest floor with grass that looked natural and believable from every angle, high and low, and, moreover, hid each level’s tiling ground texture. “That was the first big task given to me on the project: Hide the ground!” exclaims Theriault.
It took a lot of trial and error to find the right approach. The first attempts used dense grass meshes to hide the ground plane. “Unfortunately, it looked great horizontally, but wasn’t believable when looking down from above, Theriault recalls. After other passes, he ended up taking inspiration from the moss and fur technique used in [Team Ico’s] Shadow of the Colossus, thereby creating mossy patches in 3ds Max that could be used in Ubisoft’s procedural population tool. This quickly carpeted most of the ground with uneven, lumpy moss that effectively obscured the level’s tiling ground texture. The earlier grass meshes were salvaged and put into the mix as well, along with filler plants and rock debris.
No amount of detail was spared in the game, including that used for the operations center.
|
“In the end, a little of everything ended up in the final ground cover, giving a very natural mishmash effect that was interesting when looking down or forward upon it,” Theriault says. “It’s easy to overlook this effort with all the impressive sights of Pandora, but until we got this part right, the environments looked extremely sparse and sterile.”
Creating assets for the numerous Na’vi camps nestled throughout the forest was another challenge. Many types of props and structures were needed for gameplay that naturally did not exist in the movie’s reference materials or story line, according to Theriault. After taking time to study all the references the team had available, the common traits in the Na’vi style of construction became clearer. With that style in mind, Theriault was able to design several original Na’vi structures consistent with what would be seen in the movie. “Cameron and his team were very cool in encouraging and welcoming the creation of original assets on our end, which added more richness to the Avatar universe,” he says.
Retrofitted Armory
Cameron purposely made Pandora hot and humid, while engulfing it in such a powerful magnetic field so that the RDA couldn’t use sophisticated energy weapons. This was an artistic conceit that enabled him to make all the vehicles and weaponry look retrofitted, like they came right out of the late 20th century. Of course, this conceit also guided Ubisoft’s weapon and vehicle creation process, from the military helicopters, to the Samson armored recovery vehicle the player pilots, to the AMP (Armored Military Platform), which resembles Ripley’s loader from the movie Aliens.
“All of the six-wheeled vehicles are controlled by in-game physics, but the AMP Suit was really challenging. It’s far more agile than people would expect,” says Blanche. Rigged in 3ds Max like a regular humanoid character, it’s meant to replicate the movement of its pilot through gesture recognition, so it has an endless combination of articulations and extensions that are especially hard to keep track of. When the AMP Suit is jumping, running, or attacking, it was difficult to show that it was a heavy piece of machinery, yet still had the requisite agility, he points out. To this end, the team adjusted the weighting on the bones to give the suit’s mechanized movements a palpable sense of inertia.
The Game is Convergence
Avatar: The Game represents certainly the most intensive collaboration between a game developer and a film director, and, moreover, a trifecta in heavyweight film collaborations for Ubisoft, after working first with Peter Jackson on King Kong and then with both Jackson and Steven Spielberg on a game adaptation of the upcoming Tintin. Now with 56 games under its belt, including system-sellers Assassin’s Creed, Prince of Persia, and Splinter Cell, Ubisoft has become an industry behemoth that’s managed to harness Quebec’s massive pool of creative talent, the province’s incredibly generous tax credit system, and its software development infrastructure—which includes Autodesk’s Softimage—to rack up annual worldwide sales totaling more than $1.7 billion.
In addition, Ubisoft has drawn a considerable amount of talent from Quebec’s thriving animation schools, hiring 20 percent of the 1500 students who have graduated from Montreal’s National Animation and Design Centre. The goal now for Ubisoft, and in particular, its four Canadian studios (two in Quebec, one in Vancouver, and the recently announced Toronto studio) is convergence. “The game is all about convergence now,” says Yannis Mallat, CEO of the Montreal and Toronto studios, “and allowing directors to explore the universe they create in their movies in a whole new way through interactivity.”
Mallat’s vision is of a future where film and game developers can merge their pipelines and share tools, while people with different skill sets can work on the same products across a broad spectrum of entertainment media, from movies and video games to online experiences. “We need people with the technical know-how to make that happen,” he adds.
Cameron also agrees that the number one challenge facing the industry today is convergence. “I believe that when entertainment succeeds in today’s market, it does so by converging movies, video games, books, toys, graphic novels, and online experiences, to create this kind of greater universe in which you can visit and enjoy the characters and settings in multiple ways,” he says. “The story of the movie doesn’t have to be retold in those other outlets, but there should be this sense of enlarging the world and enhancing the viewers’ knowledge of the history, culture, and all these things within the fantasy world.”
Indeed, so committed was Cameron to convergence that not only did he borrow many of Ubisoft’s designs—for the Pandora’s bioluminescence, for example—but he also had his own design teams develop vehicles for the game that aren’t in the movie. Avatar: The Game is a seminal moment in the history of film-game convergence, one that Ubisoft is determined to nurture and grow, as it sets its sights now on Spielberg’s Tintin.
Martin McEachern is an award-winning writer and contributing editor for Computer Graphics World. He can be reached at martinmceachern@hotmail.com.