BAFTA-winning VFX house
Union has teamed up with
Clear Angle Studios to develop a new crowd pipeline for an upcoming feature film that could dramatically change the way digital crowds and digi-doubles are created.
Gaussian Splats
The technology utilises Gaussian Splats, a hot topic that has recently been making waves mainly in the capture of environments. Gaussian Splats recreate objects and people using millions of tiny volumetric points that blend together into incredibly photoreal assets instead of building traditional CG characters from scratch.
For digital characters Splats preserve an exact performance, capturing amazing detail including the nuanced motion of hair and cloth. This reduces the need for manual cleanup, texturing, lighting and animation. These ultra realistic captures are ideal for use in VFX, especially for crowd simulations.
Clear Angle’s new Volumetric Capture Rig (VCR) uses 40 cameras, running at up to 60 frames per second, with controllable LED lighting to capture performances within a 3m by 3m area.
The scanned data then runs through a custom training pipeline developed jointly by the two studios before being brought into Union’s VFX pipeline as fully usable USD CG assets.
The resulting splats can be viewed from any camera angle, dropped into any environment, and rendered through standard CG renderers.
True Relighting
Until now, true relighting has been a major challenge when attempting to use splats in this way:
David Schneider, VFX & Technical Supervisor at Union, said:
“There are experiments out there that ‘relight’ splats using relighting baked in at training time or render-time relighting, resulting in the introduction of artefacts.
“Our solution allows the splats to be lit with the same lighting setups used in other CG workflows without specialised render support. They behave like traditional CG assets, but are far more realistic and come complete with animation, hair and fabric movement. They can also be lit and rendered within CG environments and alongside traditional CG assets.
“This results in incredibly realistic splat crowds that accurately match your choice of lighting environments while keeping everything looking natural and consistent.
“Splats hold up way closer to camera than traditional CG crowd, allowing productions to create more convincing crowds with fewer unique elements.”
Variation
Union leverages image segmentation technology, which allows artists to isolate and adjust costume colours and materials, creating much more visual variety in crowds.
In practice, as the VCR can comfortably capture small groups of performers together, a small number of 20-second captures can now become a crowd of tens of thousands by applying time-offset animation, varied placement, and segmentation.
Endless Possibilities
This approach allows almost infinite possibilities:
- • The capture area can accommodate devices that enable movement such as a treadmill, bike rollers, a gimbal or wires; action captured can then be placed into a CG environment creating believable movement.
- • No camera move is impossible.
- • It can always be the golden hour — lighting is fully controllable in post.
- • As CG Assets, the splats can be used in FX Simulations and more.
“It’s really exciting technology,” said Tim Caplan, CEO and Founder, Union VFX
“As far as we know, we’re the only studio currently offering truly relightable Gaussian Splats in production. Being able to relight splats opens up huge creative possibilities for digi-doubles with realistic hair and cloth simulation, giving far greater realism and utility than building them traditionally.
The costume variation tools, alongside the human imperfection of the captured material, make our digital crowds feel far more natural and diverse than a traditional CG approach.
The earlier productions start thinking about how they can utilise this tech, the more time and money they can save across the production process.”
Dominic Ridley, CEO of Clear Angle Studios, adds:
“Our entire ethos is about being faithful to reality. The VCR delivers a dataset that fully preserves the talent's original, nuanced performance in all its glory. By providing VFX teams with robust, accurate data as a foundation, we make life easier downstream and ensure the final result is completely grounded in truth.”
Watch Union's explainer video here:
About Union VFX
Union VFX was founded by Tim Caplan and Adam Gascoyne in 2008, and remains an independently owned studio based in Soho, London & Montreal, Canada.
A BAFTA award winning, Emmy nominated, and Oscar shortlisted visual effects team who built its reputation working with some of the great British filmmakers, including Danny Boyle, Martin McDonaugh, Sam Mendes and many more.
Union’s unique approach puts creative and innovative storytelling at the heart of everything we do. The team thrives on creative problem-solving and technical collaboration. By welcoming a mixture of projects with a variety of challenges, Union continually maintains a creative and technical edge. More about Union VFX:
https://www.unionvfx.com