At the Crossroads
Kathleen Maher
Issue: Volume: 32 Issue: 7 (July 2009)

At the Crossroads

Stereo 3D can be an exciting adventure—if it is done properly.

They’re running on guts, and gall, and pure, unadulterated fear—their usual jet fuel. But there’s a little extra juice flowing nowadays as the studios make big bets on 3D animation, stereo 3D, and digitizing the production pipeline all the way through to distribution, while the US economy shivers like a kid with some serious monsters under the bed.

The stereo team on Disney/Pixar's Up—Pixar's first stereoscopic film—worked in tandem with the director.

The problem is the audiences: They’re being more discriminating about how they spend their money. They’ve got choices, and lots of them. Hollywood has been nagging and pushing for the digital transition to get people in the theaters, and all the while, audiences are choosing to stay home. Attendance numbers have flattened out in the past three years, and worse, DVD sales and rentals are also down. The studios and the theaters have finally organized themselves into a mutual admiration society, and both are driving toward the worldwide transition to digital. And they’re realizing that they might see all the theaters turn digital, only to find attendance lacking. So, rather than digital becoming a pleasant dream, it could easily turn into a nightmare—from which Hollywood awakes screaming.

Instead, the industry has found its lullaby: stereographic 3D. Big-screen movies drawing more people into the theaters and sucking more money out of pockets is a beautiful vision for the moguls, and it’s enough to bring stereo 3D back from exile after several disastrous attempts to go mainstream. Suddenly, studio execs are talking as if they invented stereoscopic 3D movies. And, the next step is for stereo 3D to find its way to TVs in the home, so that getting a new DVD in the house will be a good reason to gather around in front of the ol’ electronic hearth—this time with the folks wearing glasses. The studios have gotten to like the extra dollars that DVD sales and rentals provide.

The latest rebirth of stereo 3D for entertainment might look like a revolution for some, an evolution for others, but there’s a population of technologists and artists who have been working away in the niches and who have been exploring and pushing the technology because they are flat desperately in love with 3D, and most of them have been since they were little kids and put on their first pair of red and green glasses.

These guys are scared, too. They’re seeing the prize right in front of them—3D movies for everyone, everywhere—and they’re afraid the same terrible nightmare will happen again. Bad content coupled with bad technology will kill stereo 3D before it even gets started. The problem is, there’s not a lot of agreement about which way is the right way forward.

The Good and the Bad

RealD and 3ality Digital Systems are the Johnny Appleseeds of stereo 3D. RealD is the leading technology company for projecting 3D, and so, in some ways, it is the tip of the spear pushing the technology out into the theaters. 3ality has shouldered the burden of live-action 3D production. The firm has been fairly successful so far, producing the U2 3D concert film—which, by all that’s holy, should have ended Bono’s career as a result of his earnest histrionics. It didn’t, though, because the 3D put audiences within Bono’s urgent reach and gave viewers the immediacy of being there, if being there means you can fly all around the theater for the best look at everything.

Bono’s career chugs on unchecked, and the movie has become a textbook for live-action 3D. The film gets regular theatrical showings and garners decent box office every time. 3ality followed that triumph with Hannah Montana, The Movie, which brought little girls into the theater in droves and got them to pay a premium for watching their idol in 3D. The third offering in the 3D rock documentary trilogy, Jonas Brothers: The 3D Concert Experience, tanked, proving that even hysterical little girls have their limits.

This 3D camera rig uses a mirror as a beam splitter so that cameras can be mounted perpendicular, but the image is transmitted as side by side. This arrangement is not pretty, but it allows stereographers to get a closer interocular distance than they could with two cameras side by side.

There are seemingly no limits to the demand for 3D, however. It has hit its elbow curve. At NAB 2009, Steve Schklair, founder of 3ality, noted that the demand has outstripped the capabilities of the specialist companies and trained professionals to make the movies. “We’re concerned about the quality,” he stated. Demand for product is high, yet almost every director in the world is a first-time 3D director.   

In an example of what not to do, Fly Me to the Moon, a European production by director Ben Stassen, was not only an uninteresting (or, cutting to the chase, a lousy) movie in this writer’s (and many others’) opinion, but the 3D has been criticized as painful to watch. Early in production, the 3D community reached out to Stassen and offered advice. “Lots of people tried to help him. He didn’t want any help,” says one 3D expert who asked not to be identified. But even as a failure, Fly Me to the Moon has made money in its theatrical release. Furthermore, it has been re-purposed for theme parks all over the world. So far, Fly Me to the Moon has not killed 3D, but proponents are afraid a few more films like it could do the trick.

Eartly in the year, the SoBe lizards went stereo in a TV commercial that aired during the Super Bowl. In the spot, the lizards were joined by some NFL players and characters from Monsters vs. Aliens.

The problem is there is so much more that can go wrong, and some of it is downright ineffable. Sometimes, says Buzz Hays of Sony Pictures Imageworks, something just doesn’t look right. The problem is more complicated in live-action movies, where the issues might lie with the camera—there might be asymmetry between the lenses—or they might have been introduced in post. “There’s a whole bunch of elements, and something can easily get misaligned,” he adds. In some cases, the tools are still part of the problem, albeit an entirely fixable one.

It’s not an accident that most movies being made in stereo 3D are animated features, because there, the director has absolute control—though, as Fly Me to the Moon amply illustrated, there’s a lot that can still go wrong.

Jonas Brothers: The 3D Concert Experience and Hannah Montana, The Movie were presented in stereo by 3ality.

One of the most common issues, according to the experts, is busting the parallax budget. In 3D, the distance between the images for the left and right eye and the distance from the screen creates depth backward and forward. If something seems to move beyond that depth, the illusion is broken. “You can’t break the rules of physics,” says Ray Hannisian, stereographer/depth-balancing artist at 3ality.

The trick is dependent on the ability to create a similar experience to what we see in reality. The view from the left and right eye have a point of convergence where the view is balanced, and around that area is a field of view that is in focus and makes sense to the brain. Bring an object in too close, and the eyes have to “toe in” too much to focus. Move it too far out and the eyes move apart. Either way, it can be uncomfortable. This is known as retinal disparity, and it is also a problem for cinematographers and audiences: It’s uncomfortable.

The ability to make objects seem to pop from the screen is a crowd pleaser. It can also break the illusion if an object seems to cross the limits of the screen, which is referred to as an edge violation. Some filmmakers are criticizing the over-enthusiastic use of depth for in-your-face illusions and blame it on first-time directors and the lack of technical expertise. The illusion can be safely created by using “floating windows” within the frame so that objects that come forward do not break that psychological barrier of the edges of the screen.

Peter Docter, director of Up, Pixar’s first 3D stereo film (see “The Shape of Animation,” June 2009), notes that the CG feature was created with a stereographic team who worked in tandem with him as he created Up using traditional film techniques.

Docter describes the use of 3D in Up as an effect that gives the audience the feeling of looking into a window at something. They also avoided the effect of having objects leap out of the screen at the audience because, he says, then the audience becomes aware that it is sitting in a theater wearing glasses.

The Here and Now

It’s only very, very recently that the concept of 3D is in front of directors when they’re in the early stages of making a film. Until better tools arrived to enable the filmmakers to get an idea of what they were going to see on the screen, they essentially were working with a 2D mind-set. Simple tools—such as Nvidia’s GeForce 3D Vision system (with 3D board, Samsung 120hz monitor, and glasses) developed for gamers—are turning out to be a real boon for filmmakers simply because they enable more people on the set to see the 3D effects (see “Game-Changing Technology,” March 2009). At this year’s NAB, almost every company selling post software for 3D talked about the ability to see work in real time using the Nvidia setup as a part of the process.

“The Nvidia monitors are good,” says stereoscopic expert Lenny Lipton, “but you have to see it on a large screen.” Hayes agrees, adding that it is harder to see what’s not working on a 20-inch screen. Furthermore, as a consumer monitor, the Samsung display can’t be calibrated to the point needed to see true color reproduction, so the team is only checking the 3D and not the way the shots look as far as lighting and color go. However, that situation is temporary: New wide-gamut 120hz displays are on the way from Sony, Samsung, HP, and others. They’ll have color-correction capabilities, as Portrait Displays has already developed color-correction tools for these monitors. It’s all just a matter of time.

Eyeon recently added 3D compositing capability to Fusion.

Still, when U2 3D was made in 2006, it was edited in 2D and then handed off to the Quantel system for finishing. In some senses, the editors were working blind. At NAB this year, both Avid and Quantel were demonstrating new tools that let editors work in 3D, to see what they’re getting and to heighten the effects. Eyeon has taken advantage of graphics acceleration and has added new tools for 3D as well, including 3D compositing in its flagship compositing product, Fusion.

Better tools mean that directors have more tricks in their arsenal. In one sense, directors have to deal with the limitations of the medium. It’s common wisdom that you can’t have a lot of fast cuts because the eye will have trouble adjusting if the focal point changes between the cuts. U2 3D tended to use layered dissolves to ease the audience into cuts, so that the filmmakers could have lots of transitions, as a rock-and-roll film deserves, but not to where it induces actual vertigo in an audience while it is listening to the song “Vertigo.”  

At the same time, talented and creative technologists defy common wisdom, and limitations drop away. Avid used My Bloody Valentine as its showcase for the new 3D editing tools in Media Composer. (Now there’s a movie with in-your-face 3D effects.) In reality, the Media Composer tools from Avid weren’t available to edit Valentine. Rather, the piece was cut in 2D just like any movie and handed over to the Quantel system, where it was finished in 2k and 4k. The pieces, however, are quickly being built to accomplish a 3D workflow. Media Composer puts tools in directors’ hands earlier, while Quantel is working on the back end; as a matter of fact, the two companies are forging a partnership to enable easier data flow between the two systems. Autodesk, Eyeon, Assimilate, and others are all working feverishly to supply 3D tools throughout the entire pipeline.

More control over the process gives filmmakers the ability to experiment and push limits. In this, My Bloody Valentine marks another milestone—3D dailies. The film was shot with the Red One camera from Red Digital Cinema Camera Company and the smaller and lighter SI-2K from Silicon Imaging, for situations that favored mobility. Technicolor developed the 3D workflow.

In a recent magazine article, Dan Lion of Technicolor Creative Bridge described how the company did the 3D postproduction as well as created 3D dailies that could be viewed at the Lionsgate offices. Technicolor devised a way to take the left- and right-eye footage, put it through the company’s system, attach a look to it, and output the material to a drive. Again, the 3D-capable Quantel Pablo system came into play to correct the color. The footage was screened on a 46-inch Hyundai display running the RealD system.  

Stereo 3D gives directors another dimension, literally. There is more screen to work with, and 3D effects can be used to further direct (or misdirect) the eye to the screen. This is one of the reasons why the true believers in 3D don’t see a barrier to any genre for 3D.

What’s Next?

Josh Greer, president and CEO of RealD, believes that the U2 movie represents a major milestone for cinematic history. It was a live-action film that exploited 3D to the fullest, giving the audience a privileged view that even those in the best seats in the house didn’t have. It combined live action and animated effects, and expanded the language of the documentary. “U2 meant that there were no more excuses. You absolutely can do live action,” he adds.

The CG movie Toy Story is also embracing the stereoscopic trend, and soon Buzz, Woody and the gang will debut in stereo 3D.

For 3ality, the real frontier is broadcast. One day soon there will be a lot of 3D movie theaters out there, and right now, people can’t make 3D movies fast enough.

Ironically, there’s too much—and not enough. The theaters want more 3D movies, but sometimes they hit at the same time. For instance, Coraline was still doing well at the box office, but it was bumped off 3D screens to make room for the Jonas Brothers. This resulted in a heart-breaking situation for the makers of a movie that—because it was stop motion—took more than four years to produce, averaging just 90 seconds of footage a week.

Likewise, concert movies tend to have very long legs, with audiences coming back for more, but they don’t usually have big opening weekends. This isn’t a great situation when there aren’t enough 3D studios for the movies to find their audiences. And in the spaces between the 3D releases, the theaters aren’t getting the premium they could for a stereo 3D production. The studios and theaters are leaving money on the table. There’s an imbalance until there is enough content and enough screens to keep the pipeline full.

3D Everywhere

In the interim, there’s broadcast as a source of instant content. One look at a football game in 3D, and even the non-sports fan is hooked. There’s an undeniable sense of immediacy in 3D. 3ality CEO Sandy Climan says that hearing non-sports fans talk about how great a football game is in 3D tells him that stereoscopic success is here to stay this time. He is also excited about the possibility of offering theatrical and opera performances in 3D. Not everyone can afford to go to New York City to see a first-run play or to the Met to catch Aida. If 3D happens the way Climan hopes it will, movie theaters will be full again, and there will be all kinds of people watching all kinds of content—and the audiences will expect the same kind of immediacy at home watching TV.

Climan will tell you that there are no limits for 3D content in terms of genre or venue. His company has 3D trucks to enable capture on site at sports events or live performances.

On another front, 3ality collaborated on an episode of the TV series Chuck that aired the day after the Super Bowl, which had 3D advertisements, including promotions for Monsters vs. Aliens and PepsiCo’s SoBe, complete with dancing lizards (see “Super Sunday’s Best,” April 2009). Climan sees the 3D episode of Chuck, which used anaglyph technology, as another landmark. It’s live action and it’s a comedy (and, in another genre blender, it’s also faux-horror).

According to Climan, stereo 3D will take off when it does not cost much more than 2D. He maintains that 3ality can deliver 3D shots on a 2D schedule. “For Chuck, we were able to do 40 setups a day,” he says.

Unfortunately, 3D TV is a minefield with competing standards and no agreement on how broadcast 3D might work. SMPTE has volunteered to come up with standards for 3D TV, but in its first round of meetings, was defeated by the warring factions and wound up simply out­lining the various options and stating that SMPTE would define the 3D master, leaving broadcasters, TV manufacturers, and studios to sort out what happens next.

Stereo 3D movies are being offered to audiences now on disc in the red and green anaglyph format—much to the consternation of filmmakers, who want their movie to look as good at home as it did in the theater. So-called 3D-ready TVs are available to enthusiasts now—they’re TVs based on TI’s DLP chip, and new 120hz TVs with LED backlights are on the way to consumers for later this year or next year.

But, there has to be content. First, in 2010 or so, the Blu-ray guys will offer 3D Blu-ray players. Panasonic has already announced an initiative to provide end-to-end products supporting 3D, from cameras to TVs to Blu-ray players. Sony, Samsung, and the Blu-ray forum members will follow, and it’s hoped (but not really believed) they’ll manage to avoid yet another ridiculous standards war.

The PC already has 3D games, thanks to driver tweaks that create and enhance the 3D effect. Nvidia is offering a graphics board, Samsung monitor, and shutter-glasses system. Another company, iZ3D, is offering a monitor and passive glasses system. Nvidia and iZ3D are in relatively friendly competition since both want to see 3D take off on PCs. Furthermore, PC vendors think that the personal computer might function as the Trojan horse, which will smuggle 3D media, games, movies, and TV into the home.

Broadcast will eventually come in a few years’ time, and by then, it is expected that a de-facto winner will manifest itself. The trouble is, it might not be the same system chosen by early enthusiasts.  

So, here we are at a place in time where so much work has been done, money has been spent, and powerful people have shot off their mouths that it seems there’s no turning back. Furthermore, the Web and television technology has opened up new possibilities. And, best of all, people are showing a willingness to pay for new entertainment experiences. Wide vistas of opportunity are ahead—so long as nobody screws anything up.

Kathleen Maher is a contributing editor to Computer Graphics World, a senior analyst at Jon Peddie Research, a Tiburon, California-based consultancy specializing in graphics and multimedia, and editor in chief of JPR’s “TechWatch.” She can be reached at

Drive time with Lenny Lipton

By Kathleen Maher

I have talked with 3D guru Lenny Lipton on several occasions, and interviewed him years ago for an article about the use of stereo 3D in CAD. I happened to see him at a conference on stereoscopic movies some months back, and recently chatted with him about the medium on his long commute through the hills and valleys of his newly adopted hometown of Los Angeles, where he is truly at home.

Lenny Lipton wrote the book on stereo 3D—several times. He’s been through several “rebirths” of ­­3D movies, and he’s been working diligently in Hollywood to try and ensure that the moguls don’t screw it up this time.  

Lipton does not wait around to see what will happen next; he invents things and pushes people. His company, Stereo­Graphics, concentrated on visualization for scientific and engineering applications, and he holds significant patents for projection, displays, and glasses in stereographic 3D technology. RealD technology acquired StereoGraphics in 2005, and brought Lipton on as the company’s CTO. Now he has ventured out on his own as a consultant to help moviemakers create better 3D content.

As a consultant, Lipton has been working on many of the latest 3D movies, including Henry Selick’s Coraline. He also worked with Selick on The Nightmare Before Christmas, which probably gives him more “cred” with his kids than most dads can hope to get. “The big thing about stereographic 3D,” Lipton says, “is that it’s more fun than 2D.” Lipton had just met with John Leonetti, the cinematographer on the upcoming film Piranha 3D, and he’s excited about the 3D quality the technicians were developing for that film.

As far as Lipton is concerned, stereoscopic 3D entertainment content is following a predictable path that’s been enabled by technology. “There’s no going back. The dominos are falling,” he says in his rapid-fire delivery that betrays his Brooklyn origins.

Lipton observes that CG animated movies was the first genre to give itself up to 3D because there’s more control available to the artists. “You can’t make an animated movie now and not make it 3D. Horror is next, and the demographic is going up.” My Bloody Valentine, for instance, is the first R-rated movie in 3D. “It’s happening genre by genre.” He doesn’t see 3D being limited by any genre; rather, it’s more a function of time.

“3D is unburdening itself of the baggage of prior failures,” says Lipton.

A student of film history, Lipton points out that even widescreen had its rocky period, where it was deemed only suitable for big-screen adventures, and expensive failures slowed acceptance. Now, it is just 3D’s turn.

Lipton, who is not at all afraid of calling a spade a spade or an idiot an idiot, takes issue with people who talk about how 3D should look. Right now, almost everyone who talks about 3D cautions against having 3D jump out at the screen. He disagrees. Rather, he says with some impatience, “everything needs to be appropriate. It needs to be right. Everything can’t be loud. Everything can’t be red.” But, Lipton notes, when you have a movie like My Bloody Valentine (which he did not work on) having things leap out from the screen is pretty much required.

Live action is the next challenge. In an interview Lipton gave to fellow 3D pioneer and enthusiast 3D comic-book author Ray Zone, he jokes, “I used to think that projection was the death of the [stereoscopic 3D] medium. I can see now it was a combination of projection and cinematography.” But, as the uneven quality of the films being released today can attest, there’s still a lot of variables out there. “The system is not a product,” he says, and that’s a problem with which the industry has to come to grips.

Lipton is happy to work within the system. “Shakespeare was a commercial artist,” he points out. Lipton is just trying to make the system work better, and every now and then, art might happen. The industry needs more trained professionals and tools. There is now a trained corps of artists working in CG and live-action 3D, and Lipton names several, including Rob Engle, who was the cinematographer on Beowulf and The Jonas Brothers, and Eric Breviq who directed Journey to the Center of the Earth and, incidentally, also was the special effects supervisor on Disney’s Captain Eo theme-park attraction, which demonstrated remarkable 3D effects in 1986.

“There are people who are absolute experts at stereoscopic images. In CG, there’s an advantage because they can totally control the space. Their cameras can be set up anywhere they want, and their sequences and scenes are very balanced,” says Lipton. But in live action, there’s the added problem of getting balanced data from two cameras. Color timing becomes an additional variable.

In Coraline, Lipton suggested to cinematographer Pete Kozachik that they take advantage of the fact that the film is stop motion, and simply slide the camera to capture both the left and right eye. In that way, they’ll at least eliminate the variation between cameras. The production developed 3D sliders to move the cameras and capture left- and right-eye views. In a fascinating bit of, ahem, convergence, Lipton also suggested they consider the distance between the eyes of their puppets as they calculated the optimum interocular distance for 3D effects within Selick’s carefully constructed world. Delightedly, Lipton says, “it worked out at about the same ratio as humans.”  

Lipton also reassured Kozachik about the ability to rely on post for 3D adjustments. “The tools are getting there,” says Lipton, noting that the Avid Media Composer now has 3D editing features, as does Quantel’s Pablo, which accommodates 4k workflows. Next, he says, “the studio cameras need to be better. They need to look like cameras. Cinematographers don’t like the 3D cameras that are available now.”

Lipton, who has worked in stereoscopy for a long time, was a consultant on the stop-motion 3D stereo film Coraline.

Right now, maintains Lipton, 3D is being made by talented technologists, but in order to become mainstream, 3D tools have to be more accessible to all filmmakers.

Lipton brings up the example of director James Cameron, whom he sees as an exceptional filmmaker because of his understanding of the technology as well as the storytelling. Cameron’s film Avatar, long in production, carries the weight of the budding 3D industry with it: It is hoped that Avatar will be a huge hit, and, in fact, one of the reasons it’s being held up is to ensure that there are enough screens for it to break big when it is released at the end of the year.

So, as Avatar slouches toward the big screen, Lipton says Cameron is putting the time to good use. “He’s doing a remarkable job,” says Lipton. Cameron has developed his own pipeline and workflow. He’s using motion capture, which lets him keep working after the actors go home.” And, motion capture gives Cameron more control over the camera and lighting. “Avatar will close the gap,” says Lipton.

But, says Lipton: “Cameron is a genius. You can’t expect the director to be an expert. Now, we’ve got to bring it to the next level. The director shouldn’t have to be an expert in stereoscopic 3D capture any more than he has to know about camera emulsions. He needs to concentrate on telling a story.”  

The future looks bright for stereo 3D. “From this point on, cinematographers and directors will have experience,” Lipton points out. “They will be making stereoscopic movies, and they will learn. And the audiences will learn, and it will be a partnership with the audience. We’ll all contribute to creating the glamour of stereoscopic film.”