Riot Games' League of Legends is one of the most popular and beloved games in the world, still drawing approximately eight million peak concurrent players daily now a decade after release.
League of Legends fans are just as passionate about the game's esports scene, which has regional leagues all around the world, and pulled in some 205 million peak viewers for the 2018 World Championship.
At that enormous South Korean event, Riot Games introduced the world to K/DA, a fictional K-pop group comprised of in-game champions featuring bold new character skins—and a song, "Pop/Stars," that you just can't get out of your head. In addition to a music video with stunning animation (which has 260+ million views and counting), Riot Games used augmented reality (AR) to depict the champions on the world stage performing the song.
Fans fell head-over-heels in love with K/DA, and so Riot Games decided to bring that AR experience to a whole new level. For China's League of Legends Pro League (LPL) finals in September, Riot enlisted the talents of Cubic Motion alongside The Future Group, Animatrik, and Stype to bring K/DA heroine Akali to life during the live broadcast, both for a dance number and real-time interview.
Thanks to Cubic Motion's intricate facial capture and animation technology, League of Legends fans saw Akali dance, talk, and react like never before.
Bringing Akali to Life
Riot Games approached Cubic Motion with a desire to do more with the K/DA augmented reality concept, including a real-time interview with a character. Cubic Motion's skilled team of computer vision scientists and facial animation experts set to work on creating a series of internal demos to demonstrate its capabilities to the various stakeholders, proving that real-time facial capture and performance could be done live. The two companies then took part in real-time, transatlantic video conferencing to work creatively together and pin down the overall look.
Cubic Motion took a big step forward with its rigging capabilities for this project, taking the existing Akali character model and rebuilding the facial rig from the inside out. Along with look development, rigging proved to be the most difficult challenge in realizing Akali as a believable character entering the real world, that could be driven live by a real performer backstage.
Akali was a true collaboration between the studios. The Future Group's Pixotope virtual production system was used, mocap specialists Animatrik handled body capture work, and Stype provided the live camera tracking hardware. Cubic Motion's facial animation expertise was its most significant contribution, but the team also worked on the dance number and did clean-up of the body work.
It was all done with an eye towards creating the most seamless, lifelike character they could, and to enhance the viewing experience of the die-hard League of Legends audience.
With help from Nvidia's RTX series graphics cards, Akali was realized with stunning real-time facial animation and real-time ray-tracing, breaking new barriers for augmented reality as the digital human took the stage at Shanghai's Mercedes-Benz Arena.
Even while dancing and twirling a glowing saber, Akali looked perfectly in place on the stadium floor amidst the real-life dancers—before disappearing into a puff of smoke at the end. Afterwards, her conversation with the interviewer was fluid and natural, as if the real champion was there in the center of the action greeting thousands of excited fans.
Real Characters, Anywhere
As with the original World Championship performance, the reaction was wildly positive as Riot Games introduced a new vision of what's possible with AR characters at its world-class events.
International fans from Europe and North America commented online that they hoped to see Akali and her K-pop comrades take the stage in their regions, as well—and the K/DA legend only grows with each new AR advancement.
For Cubic Motion, it was also a bold new advancement in its capabilities. The international scale of the Siren project and its demos helped the team navigate the complexities of coordinating a large-scale collaboration for this project. The League of Legends Pro League activation showcased the talents of Cubic Motion's growing onsite production team, which is taking the technology to new places all across the world while keeping the intricate facial quality that the studio is known for.
Thanks to the collaboration with Riot Games and other partners, Cubic Motion showed that it can train up a team of industry specialists to deploy this kind of technology anywhere and put on a world-renowned show. The technology is accessible and it is available for all kinds of real-time activations
Excitement around digital humans is growing exponentially as the results become all the more natural and believable, and the ability to pull a beloved character out of its video game universe and into the real world holds incredible potential ahead.