Harmonix re-creates the sights and sounds from the Beatles for its latest Rock Band title.
Their music was legendary. Their faces recognizable the world over. They sparked controversy. They inspired trends. They were The Beatles.
The English rock band formed in 1960, and soon after became one of the most critically acclaimed acts and commercial successes the world has ever known. Despite its meteor-like rise to fame, its years of successful record releases and sold-out concerts, the group disbanded after just a decade.
Today, some 50 years later, The Beatles’ music remains as popular as ever, as vinyl records and eight-track tapes of hits such as Yellow Submarine, All You Need Is Love, Back in the USSR, Can’t Buy Me Love, Hey Jude, and I Am the Walrus have given way to CDs and MP3 downloads of the songs. And now, a new generation of fans—in addition to those of old—can do what every Beatles lover has always dreamt of doing: They can be a Beatle and play alongside their idols in the music video game The Beatles: Rock Band.
Developed by Harmonix Music Systems, published by MTV Games, and distributed by Electronic Arts, the title is the third major console release in the Rock Band series. Like with the earlier titles, players use instrument-shaped controllers (including special ones modeled after those used by the actual band) to simulate their performance. What’s different about this release, though, is that rather than a collective of songs from various bands, this one contains all Beatles music and visuals. The game disc contains 45 Beatles songs, to which users can play along using the instrument peripherals. (Harmonix has made three additional albums available for download to play with the game: Abbey Road, Sgt. Pepper’s Lonely Hearts Club Band, and Rubber Soul.)
“The Beatles: Rock Band is a journey through The Beatles’ career, starting from their early days in The Cavern Club and going all the way through to their final public performance on the rooftop of the Apple offices,” says Josh Randall, project lead and creative director at Harmonix. “You get to see them in their authentically replicated outfits at a number of familiar venues.”
Prior to Rock Band, no song by The Beatles had been featured in any of the popular music video games, which began striking a chord with players a few years ago. So what prompted this virtual Beatlemania? According to Randall, it all began with a chance meeting between MTV president Van Toffler and Dhani Harrison, son of George Harrison, in December 2006, shortly after MTV’s acquisition of Harmonix. Toffler introduced the younger Harrison to Alex Rigopulos, CEO and co-founder of Harmonix, and in a casual meeting, the two discussed the idea of making a game based on The Beatles. Harrison then helped introduce the concept to Apple Corps, the music production company established by The Beatles.
A five-song demo finished in early 2008 helped seal the deal with Apple shareholders Paul McCartney, Ringo Starr, Yoko Ono (on behalf of John Lennon), and Olivia Harrison (on behalf of George Harrison), all of whom acted as creative partners in the venture.
Rock Icons
Work on the project did not follow a traditional schedule. Rather, five artists spent several months working on preproduction, creating concept art and test models, and brainstorming. “We worked closely with Apple and had to get a lot of the early art approved,” notes Randall. “They wanted to be comfortable with the way they were being represented.”
In addition to the character representations, the band members and heirs approved the songs and venues that would be used, as well. They also provided feedback on all the artwork and storyboards for the animation sequences. Members of the Harmonix team even made trips to the UK to meet with Olivia Harrison, while Yoko Ono met with the artists locally. “John [Lennon] had an enormous amount of charisma, and reproducing that in the game was really difficult,” says Ryan Lesser, Harmonix art director. “She came in and gave constructive criticism that helped us get the character to where it was in the final product.”
The art team also delved into mountains of research material. In addition to reviewing personal photos, books, and other materials, the Harmonix crew watched a number of documentaries, including The Beatles Anthology, giving them an accurate portrait of the looks and mannerisms of their larger-than-life subjects. The references additionally provided a time capsule for the artists in terms of the venues, outfits, and hairstyles for the various stages in the band’s career. Experts both inside and outside the studio provided answers that enabled the Harmonix team to meticulously replicate the outfits that the performers wore for each of their concerts, as well as the instruments they used for recordings and live performances.
“People here became these uber Beatles fans—more than we already were,” says Randall.
Despite the plethora of available source material, capturing the likeness of the four musicians was incredibly difficult, says Lesser. “We knew early on that was the most important thing, to get their likeness correct,” he adds. To this end, a small team of character artists worked on the models throughout the entire year and a half of production, tweaking them to fit perfectly within each scene.
“These are the four most recognizable people in the world, and everyone knows how they look, how they move, and what happens to their faces when they smile,” says Randall, “all those details we knew we had to get right. Even with all the reference, it was hard to pin down these guys’ looks. They appeared different picture to picture.”
Lesser agrees, adding, that at times the group believed it had the perfect model, and then another reference would make it look like “a different person.”
Creating Legends
Using Autodesk’s 3ds Max and Pixologic’s ZBrush, the artists crafted the character models, which are stylized as opposed to hyper-realistic, keeping in tune with the look of the other Rock Band music video games from Harmonix. “The hyper-realistic look doesn’t seem as fun or interesting to us,” says Lesser. “And the Uncanny Valley problem would have been more of a problem for us than you have in most games because we were not inventing new characters. These were people who folks could describe in detail with their eyes closed.”
According to Lesser, Harmonix had a solid technique in place from its other titles for crafting the heads and bodies of the characters. Nevertheless, the modelers started to develop a slightly new way of working on the heads, which included prototyping in ZBrush, doing quick sculpts, before handing the model over to artist Shawn Witt, who was proficient at paint-overs. He would then adjust the model and get it closer to the desired look. At that point, Jed Wahl and Brian Parnell transferred the model back into Max for further work.
The Harmonix artists replicated historical venues from The Beatles’ concerts, such as this one, from the band’s appearance on The Ed Sullivan Show.
|
“We have a style for the game that is related to the Rock Band series, which is that things are typically more smooth and planar, with hard angles and edges on the models,” explains Lesser. “With things like clothing and hair, we found we could use Max and the smoothing ability it offered to get the kind of end results we wanted much faster and with sharper detail than when we would go into ZBrush, which is where we typically do our flesh-based work.” Harmonix utilized 3ds Max’s poly editing tools, specifically chamfer and turbosmooth to attain the smooth, stylized surfaces of the clothing and hair. For this particular group of assets, the technique proved to be effective and fast compared to Harmonix’s typical pipeline through ZBrush.
Character lead Steve Kimura and art lead Dare Matheson directed the artists, trying to get them to find that sweet spot between the stylized and realistic representations. Once the group was satisfied with the model, it was sent off to Apple for approval. And as Lesser notes, “They [the shareholders] were very good at telling us what was missing. They were invested in this, so they were big participants in the process.”
Initially, the artists planned to use more than one model for each character spanning the seven or so years depicted in the game. But once the modelers changed the hairstyles, there was no need to re-sculpt the faces, explains Randall.
In terms of the clothing, the modelers had used ZBrush in the past, but came up with a technique that enables them to do that work in 3ds Max. “Retaining the model inside Max kept us moving and gave us the style we want,” Randall says. The dynamics for the cloth occur in real time, thanks to in-house coding scripted mainly by James Fleming. Driving the dynamics is Milo, Harmonix’s proprietary game engine.
You Move Me
Animation proved just as important to the game’s focus as the modeling. All the body animations were done with MotionBuilder and brought into the game engine. Depending on the needs of the face, clothing, hairstyle, mouth, hands, and dynamics, each character model had approximately 115 bones.
In addition, the crew used motion capture for both body and facial movement, particularly for the simulated musical performances. Mainly they worked with a group of tribute bands and performers, who played through the songs.
The mocap was done at Curious Pictures in New York City with a 23-camera Vicon system using two distinct camera setups for body and facial capture, and then processed within Autodesk’s MotionBuilder and delivered to Harmonix in that format. The animators worked in MotionBuilder, where they spliced the mocapped actions into various sequences.
“We paid a lot of attention to how The Beatles interacted with their instruments, microphones, and with each other,” says Lesser. “We do that in our other games, but in particular, the way The Beatles reacted to their stage surroundings and to one another was unique. And between the mocap, animation, and technology, we made sure their mannerisms came through in the performances depicted in the game.”
Moving into new territory, the animators also used facial capture for the first time. “We typically hand-animate our face movements, but for this game, we wanted a little less of an exaggerated, stylized facial animation style. We wanted more of a humanistic motion,” notes Lesser.
The facial capture also was performed at Curious with the Vicon system, albeit in a much smaller 18-camera capture volume to pick up more detail. Rather than import the data into 3ds Max, the animators decided to use Maya instead, marking the first project for which Harmonix used that software. Lesser states, “Curious has been a longtime partner of ours, and their facial-capture specialists were writing their tools in Maya. It made sense for us to delve into that software so that we could achieve the highest-quality results as soon as possible.”
The game artists used Autodesk’s 3ds Max and Pixologic’s ZBrush to re-create The Beatles characters. The game developer chose a stylized look, keeping in tune with the aesthetic used for Harmonix’s other music video game titles.
|
According to Randall, when each of the band members sang, their faces moved in unique ways—for instance, George tended to sing out the side of his mouth. While the facial capture gave the animators a good starting-off point, they still had to do extensive tweaking “to add that ‘Beatles thing’ to each of the characters.”
To accommodate the facial-capture data, the team had to ensure that the generic facial capture that they applied to their models actually fit the virtual Beatles’ faces. New animation rigs had to be set up since the group was using Maya for the first time.
According to Matheson, the biggest challenge the Harmonix artists faced was making sure the characters exuded the charm and personality of “four of the most beloved musical artists ever.” While this was an aesthetic challenge, it also became a technical challenge. “We went to great lengths to get body and face motion capture that ‘felt’ right for each of the guys. But even so, we spent a lot of time hand-adjusting the face positions and movements to get it just right,” he says.
The new techniques also gave the artists more control over the animation than they have ever had before. For instance, they were able to add more apparent joyfulness in the faces. They also had more control over the characters’ positions and postures on stage, and were able to use normal maps to create exaggerated expressions.
“This all added up to an incredible amount of control, which required a lot of additional time spent on each character and song by the artists,” explains Matheson. “The implementation of these additional controls were invaluable in achieving the level of success I believe we had in bringing these 3D avatars to life as animated Beatles that lived stylistically somewhere between the older cartoon depictions [in the earlier concepts] and [the final] depictions, which might comfortably lean against the protective railing overlooking the Uncanny Valley.”
According to Lesser, no one virtual Beatle received more attention than another. However, Ringo’s drum animations and body movements proved more difficult to pin down because the artists wanted to show his particular style while keeping the drummer technology intact.
“There is a complicated underlying system of controls that makes Ringo do his thing, so we were constantly fighting against the inherent roboticness of such a system and the needs of showing Ringo to be the relaxed, loose player he was,” says Matheson. John, Paul, and George, meanwhile, each had their own particular way of playing guitar or bass, as well as singing, which required careful tuning on the part of the Harmonix crew.
On Location
The Beatles: Rock Band takes players to many locales, from concert stages to fantasy dreamscapes, all modeled in 3ds Max and textured in Adobe’s Photoshop. Using the available reference material, the artists accurately re-created the historical venues to enable players to feel as if they are actually there. “We stylized them a bit to make them feel grandiose when shown on a small television screen,” admits Lesser.
The historical venues—like the clothing and the instruments—represented part of The Beatles history, and thus had to be meticulously re-created to give the game an air of authenticity. For example, the set of The Ed Sullivan Show was resurrected digitally from photos and videos. Other historical venues in the game include The Cavern Club, Shea Stadium, Budokan, Abbey Road Studios, and the rooftop at Apple headquarters.
Surprisingly, the contrived dreamscapes, on the other hand, proved more challenging, according to Matheson. Unlike the historical venues, which supported multiple songs and were fairly in line with venues in Harmonix’s past releases, these settings were not based on reality—in fact, far from it.
“We had to flex totally different muscles for those,” Matheson says.
These fantastic sets, filled with vivid colors, stylized lighting, and geometric forms, were used especially during the later years, when the band stopped touring and had been camped out in recording studios. “So we did this thing which is new for the Rock Band games, where we bring the band into these imaginary dreamscapes, which are fantasy locations based around the lyrical content or vibe of the song,” describes Randall. For instance, the sequence for Octopus’s Garden contains an underwater reef setting. The most complex, says Matheson, was Sgt. Pepper’s Lonely Hearts Club Band/With a Little Help from My Friends because the artists were working on two songs at once and had to seamlessly move the whole stage, including the characters, at the switchover.
“We were moving between different spaces and switching outfits. All this required a new authoring system that would allow artists to control what is seen at any point within a song,” says Matheson.
The imagery for these sequences was created in 3ds Max, though some of it has a 2.5D look for more of a kaleidoscope effect. The animation was done in Milo with new animation tools built in-engine. “We created a whole new generation of animation system within Milo that is very much like other modern animation systems, with complex curve editors and triggering systems—things that were new to our engine and modern in the regard to what you would see in Max or Maya or After Effects,” Lesser says.
Rock On
With The Beatles, Harmonix had to consider the rich and deep history of the people, places, and songs associated with the four rockers that had to be faithfully and respectfully brought into the digital world. In order to do that, the Harmonix team spent as much time making a game with four musicians as it did with many rock stars in the Rock Band series of games.
In addition to historical venues, the game contains fantasy backgrounds contrived by Harmonix to augment the lyrics or mood of a particular song. These backdrops were built in 3ds Max, while the animation was done within the Milo game engine using new tools created by Harmonix.
|
While the initial team was small, the numbers soon swelled as the work amped up, expanding to an impressive 200-person crew at the high point of the production.
In the end, the game was welcomed by fans and critics alike. Perhaps more impressive is the fact that Paul McCartney utilized game footage at various performances. (In fact, this is how the game first leaked to the world?…?in Sir Paul’s live concert.) Furthermore, the game’s release coincided with that of the new, remastered CD versions of The Beatles albums, showing that indeed the music and aura of this band are as relevant and exciting today as they were decades ago. Only today, fans can accompany the stars on a virtual stage and enjoy the music in a way that was unheard of when The Beatles first took the stage so many years ago.