Framestore Employs Deep Fake Tech for Groundbreaking Campaign
May 18, 2021

Framestore Employs Deep Fake Tech for Groundbreaking Campaign

Kiyan Prince, a football prodigy, tragically died after being stabbed at the age of 15 in May 2006 while trying to prevent another child from being bullied. Now, 15 years later, using their expertise in cutting-edge art and AI, Framestore has collaborated with marketing agency Engine Creative to support a groundbreaking effort to bring Kiyan Prince back to life virtually, and to ultimately help raise awareness of knife crime.
The campaign, called “Long Live the Prince,” will see Kiyan Prince sign for his former football club, Queens Park Rangers, where he will wear the squad number 30 to reflect the age he would have been today. In a world first, Kiyan will also be introduced as a non-playable character in EA Sports FIFA21 – one of the world’s biggest video game franchises. Together with this, Kiyan will be a collectable footballer within the Match Attax card trading game, which is being released by Topps on May 18. Brands such as Adidas and JD Sports will be sponsoring Kiyan as a pro-footballer, and proceeds will raise money for the Kiyan Prince Foundation.

Working pro bono, Framestore’s co-founder and chief creative officer, Mike McGee; project lead, Karl Woolley; creative technologist, Johannes Saam; global head of creatures, Markus Schmidt; and senior designer, David Lochhead collaborated with Engine Creative and expert likeness artist Chris Scalf to lead the development of the assets needed for the campaign. To show the 30-year-old likeness of Kiyan, the creative team had access to a leading expert on human aging: professor Hassan Ugail of the University of Bradford, who used state-of-the art aging-projection software to create a scientifically accurate image of how Kiyan would look today. Once completed Framestore set about creating the suite of images to accompany the campaign.

Watch video.

“We've created many humans, digi-doubles and superheroes in the past, but Kiyan was on a different level. Creating Kiyan as the 30-year-old man he would have been today was a sensitive and delicate process, and we took our responsibility seriously,” says Woolley. “Making sure we did Kiyan and his family justice by realizing him as he would have been today was the biggest challenge of this project and, quite possibly, of all my projects to date.”

As the end sequence for the promo film wasn’t concepted until after the planning for Kiyan’s likeness had been completed, the creative team was left with no time to go down the traditional and more laborious digi-double route. Alongside this, there were no source images of Kiyan at 30 to train the deep fake on, either.

To pull off the promo’s end sequence, Framestore’s creative technologist, Johannes Saam, partnered with Sydney-based Matt Hermans, founder of the creative company Electric Lens Co., to work on the promo video assets and began the challenging task of bringing life and movement to 30-year-old Kiyan. Using a mixture of cutting-edge development and some secret sauce, the team was able to incorporate Scalf’s stills into the process to create synthetic data, which they were able to train the deep fake on.

“Deep Fakes are often seen as gimmicks, to service a trend, but when we heard about the project and the potential to use deep fake for good, we knew we had to find a solution, employing new neural rendering-based techniques,” says Saam. 

Based on a first pass of the geometry and painted reference frames, a virtual dataset was assembled with an automated pipeline rendering 6,000 pictures as a base layer, which was then iteratively refined. 

“The first pass added facial animations to the training material. After that, the pipeline was fed a detailed lookdev pass with facial hair and additional texture detail to keep the machine in line with our 2D likeness,” explains Saam. “Based on this dataset, we used a pre-trained neural network and the Kiyan-specific images to focus the deep fake network on translating the human facial performance into a digital representation of Kiyan.” 

This network replaced the entire head frame-by-frame and was then composited over the photography, as well as removal of sponsor logos from the shirt using Foundry’s Nuke’s brand-new CopyCat node.

All proceeds raised will go directly to Kiyan Prince Foundation, a charity run by Kiyan’s father, Dr. Mark Prince OBE, who’s been tirelessly campaigning to prevent knife crime since his son’s tragic death..