A digital Revolution

The Wachowski brothers’ film noir-styled “kung-fu meets sci-fi” movie, The Matrix, won a visual effects Oscar in 2000 for, among other effects, the stylish “bullet-time” shots in which Keanu Reeves’s actions happened in extremely slow motion while the camera swung around him. Since then, so many films-from Charlie’s Angels to Shrek-have imitated and parodied the bullet-time shots that the effect has become a cliché.

But, what happened behind the slo-mo action arguably had a more widespread and lasting impact on visual effects. To create digital reproductions of the locations surrounding the action, the Matrix crew used photogrammetric modeling with projected texture mapping, and state-of-the-art image-based lighting techniques based on Paul Debevec’s research at the University of California. Today, most films with visual effects use lighting techniques based Debevec’s research, and digital matte paintings that include projected texture maps on image-based models are increasingly common.

It’s unlikely that the 500 visual effects shots in the Wachowski brothers’ latest film, V for Vendetta (which they wrote and co-produced), will win a visual effects Oscar-the timeline was tight, and the story didn’t demand huge effects. However, as in The Matrix, something that few people will notice on screen could be a harbinger for changes in the way postproduction houses and colorists work with film in the future.

James McTeigue, who was first assistant director on all three Matrix films and on Star Wars: Episode II-Attack of the Clones, directed V for Vendetta. Hugo Weaving, who played Agent Smith in the Matrix trilogy and Elrond in The Lord of the Rings trilogy, is V, a vigilante who wears a white Guy Fawkes mask, black hat, and black cape. And, Natalie Portman is the mild-mannered young woman he rescues-and radicalizes. Based on a graphic novel illustrated by David Lloyd and published by Vertigo/CD Comics, the UK-German production takes place in a future, totalitarian Britain. The tag line for the film, which tackles the question of armed resistance to totalitarian oppression, is: “People should not be afraid of their governments. Governments should be afraid of their people.”

Cinesite used its React crowd-simulation software to handle the motion for the thousands of digital people dressed like V in this scene.

Dan Glass, who was visual effects supervisor for Batman Begins and for the final two Matrix films, was V for Vendetta’s visual effects supervisor. Unlike the Matrix trilogy, only a few of Vendetta’s effects are obvious, and the visual effects crew often relied on practical elements rather than CG. “The majority of the effects are subtle,” Glass says. “It was a real pleasure to work on it from that point of view, to play with little details in a way that I don’t think anyone will notice, but that finessed the movie as a whole.”

Cinesite created most of the effects, with Framestore CFC handling the digital intermediate (DI) work. In addition, Double Negative put videos on television monitors and video screens throughout the film. All three studios are in London’s Soho district.

“Our most obvious visual effects were the explosions and a knife fight scene,” says Glass. “Cinesite handled those and smaller bits and pieces. Framestore CFC changed the lighting on the key character’s mask in postproduction and balanced the atmosphere throughout one sequence [see “VFX in the DI Suite,” pg. 16]. I found that work particularly interesting-the whole process often blurred the line between what we did in visual effects and what was dealt with in the DI.”

Early in the film, V makes his mark by exploding big buildings. For this, the buildings were miniatures that the crew photographed. “We had seven weeks allotted for visual effects, and we didn’t have a big budget,” says Glass. “We tried to find or photograph real elements.”

For the fire, Glass had Pacific Title in Los Angeles scan fire and firework elements from the Warner Bros. archives. “I was keen to use real fireworks,” Glass says. “CG tends to be slightly clean, but with the real elements, you get all the details for free.”

Cinesite relied on CG for the knife effects, however, during a sequence in which V shows off his superhuman power by moving at hyperspeed. “It’s the Vendetta version of bullet time,” says Matt Johnson, visual effects supervisor at Cinesite. To create the feeling of hyperspeed, the crew filmed stunt performers at high speed; the stunt performer acting as V handled bladeless knives. The CG knife trails, which looked like cigarette smoke, created the illusion of speed in the slow-motion scenes.

To create the trails, Cinesite first tracked the camera with 2d3’s Boujou software and Science-D-Visions’ 3D Equalizer, and rotoscoped knife strikes in the shots, adding CG knife blades to handles and, sometimes, complete knives modeled in Autodesk Media and Entertainment’s Maya (formerly from Alias). A Maya plug-in extruded layers of geometry from the tracking curves to create geometry planes based on the knives’ motion, “but more complex,” says Thrain Shadbolt, 3D supervisor. “[The plug-in] also cut these planes into several pieces, some moving quicker than others. The geometry was created in Pixar’s RenderMan and layered in different ways with textures, some moving and some not, and then taken into the composite.”

Layers of geometry extruded from tracking curves and rendered with textures were composited with various levels of transparency to create the knife trails left by V in this image.

Compositors working in Apple’s Shake used optical-distortion nodes to break up the passes and mix various levels of transparency, especially when V walked through a knife trail. “We used lots of 2D plug-ins for Shake,” says Johnson. “Everyone had to maintain continuity through scenes.”

Cinesite also created digital crowds for a sequence in which thousands of people appear on the streets dressed like V. Although the production crew filmed real people in costume gathering at the Houses of Parliament, Cinesite increased their numbers by replicating those real performers in compositing and by creating fully CG characters. The studio’s crowd-simulation software, called React, handled the motion by applying animation created from motion-capture data to 15 different crowd models. To animate the cloaks, the crew integrated Syflex’s cloth simulations into the crowd software, prebaking the animation when the crowds were shot from a long distance.

In addition, Cinesite created large matte paintings by using RealViz’s Stitcher to quilt photographic tiles enhanced in Adobe’s Photoshop. A shot of Natalie Portman standing on a balcony in the rain, for example, was filmed on a set piece in a studio. To create the rain, Cinesite used Maya particles rendered through RenderMan. The crew added depth of field by splitting the rainstorm into multiple layers rendered at different speeds, and then blurring highlights on some layers in Shake, during compositing. To control a hero raindrop that splashes in Portman’s face, Shadbolt used a Maya expression to attach one particle to a locator.


“Hopefully, these effects won’t draw attention to themselves,” says Johnson. “This isn’t a popcorn movie. The visual effects were not the end; they were the means to the end.”

Barbara Robertson is an award-winning journalist and a contributing editor for Computer Graphics World. She can be reached at BarbaraRR@comcast.net.

---

During V for Vendetta, V’s pure-white Guy Fawkes mask is always partly in shadow, a stylistic device that wasn’t possible to achieve all the time during filming, particularly in fast-moving scenes. Also, because the main character is masked, the filmmakers used shadows to create expressions on the unchanging mask.

At first, colorists in Framestore CFC’s digital intermediate (DI) suite tried drawing relevant shapes on the mask and moving them with the tracking system in the grading software. “The drawn shapes only looked realistic enough on a few occasions,” says Adam Glasman, colorist.

To increase the realism, the VFX crew built a 3D model in Autodesk’s Maya from a scan of the real mask and, using 2d3’s Boujou and RealViz’s MatchMover, tracked it onto the character. Then, they lit the mask, rendered out a gray-scale matte in Pixar’s RenderMan, and imported that matte into the grading suite. There, with senior compositing artist Jonathan Fawkner at his side, colorist Adam Inglis applied corrections through the 3D matte to increase the density and create shadows on the filmed mask.

Framestore CFC uses the Baselight grading system from FilmLight, which provides real-time processing on films scanned at 2k. For this movie, the digital effects studio began by making a digital version of the feature. Then, Glasman sat in a grading suite with the director of photography, the director, and others, correcting the color. Thanks to mattes created by the visual effects crew, correcting shadows on the mask utilized a similar process.

“It gave us all the advantages of an interactive grading session to do what would otherwise be a visual effects job,” says Fawkner. “We could have done this in compositing, but we would have had to deliver it for comments, go back and change it, deliver it again, and so forth. Instead, we basically had a client session in which we were able to increase and decrease the contrast in the mask, and make it lighter or darker so that it not only matched the rest of the lighting, it did its job in telling more of a story and giving V more character.”


The colorists turned to the visual effects crew for help with another grading problem: adding atmosphere to a long sequence. “Usually, we’d grade everything, send it to visual effects, and they’d add atmosphere,” says Inglis. “We tried that at first, but we felt it wasn’t working. It was a long sequence with a lot of cuts, and we needed to see the atmosphere in context.”

The visual effects crew rotoscoped the foreground characters and rough shapes of the alleyway to create mattes, and generated visual effects plates of smoke. The smoke plates were a combination of practical elements from Framestore CFC’s library and smoky noise generated in Shake.

“We mixed the smoke, graded the background through the smoke using holdout mattes from the rotoscoped characters and alleyway, and then combined everything with keys we pulled in Baselight and with shapes we drew in Baselight,” says Fawkner. “It’s remarkable how much you can get away with when you have a swift work flow.”

Although they might have accomplished the task for that sequence using a Flame or Inferno system from Discreet, it would have required many iterations because the compositing was grade dependant: The grade changed as the artists added the smoky atmosphere. Also, by using the Baselight system, the colorist-compositing duo could access the entire film at 2k.

“On the Inferno, we could have made the changes on a background plate, but then the grade would have been bluer, so we would have had to redo the smoke,” explains Fawkner. “On the grading system, we just dialed down the smoke. So, it was slightly more interactive.” And, as a result, the client could tweak the effect until the day before the film was finished.

Fawkner believes DI software will begin including more effects capabilities. Inglis points out, for example, that once clients are in the grading suite, they notice such things as rigging wires that they didn’t spot when looking at sequences offline on an Avid system…things they want painted out.

“Everyone knows you can color-correct,” Inglis says. “Now clients are asking what else can be added on top [of that]. When we first started doing DI, we’d send shots to visual effects to have images flopped. Nowadays that would be absurd.”

Nevertheless, none of the people on the team expects that colorists will handle complicated effects in grading. “The process we did for V was quite unusual,” says Inglis. “Not many productions would be prepared to run two grading suites for two months.”
— Barbara Robertson

Real performers were replicated at Cinesite, and digital extras were added to create the huge crowds shown in these shots.