The Third Floor joined early during pre-production to serve as the visualization team, with Justin Summers in London heading up previs. Leading into and during the shoot, The Third Floor’s UK virtual production supervisor Kaya Jabar worked across departments to help plan and shoot live performers in sync with Dumbo’s digital flights. After principal photography, Summers and The Third Floor temped Dumbo and other CG into the live-action plates to produce postvis across the film.
“Beginning with previs, we would visualize the key scenes as written in the script, working with visual effects supervisor Richard Stammers with input from Tim or film editor Chris Lebenzon,” says Summers. “We created previs assets and animation directly from script pages and used a stylized black-and-white look to focus attention on staging, cameras and creature performances.”
Previs shots are usually composed and animated on a per-shot basis, with artists placing a single camera or a handful of cameras for the coverage of an action beat or two. On Dumbo, The Third Floor instead created expansive 3D “masterscenes” that visualized a broad range of script action and staging within a single 3D Maya file.
Summers says, “The idea was for us to allow exploration of the narrative of the scene. Artists would animate the staging according to the script and we would add cameras at different points throughout the masterscene, both for global coverage and to test out specific framings from different vantage points. It was a sort of lighter virtual production approach that gave Tim flexibility to see and place witness cameras versus looking at an edit of previs and providing notes.”
Having collaborated on Miss Peregrine’s Home for Peculiar Children and, earlier,
Alice in Wonderland, The Third Floor came in with some familiarity to the camera styles and moves favored by the director. This helped as the team visualized a canvas of action for key scenes like the First Show and the initial introduction of Dumbo.
With the titular character to be a CG creation, the previs provided a good way to explore Dumbo’s look and performance prior to final shot production. How would he operate within the tent when flying? How would he interact with other characters? How would he fit with circus props and the train? What could the performance look like when he first takes flight in front of the audience?
“For the pivotal flying moments, we’d begin with a look-dev of Dumbo in full flying mode in the confines of a restricted space — the circus tent,” says Summers. “For any scene we visualized, we tried get a good sense of all the aspects of his flying.”
Scenes of Dumbo with seated riders leveraged a virtual production shoot to capture live actors and stunt players on a practical elephant motion buck. Fresh from helping create believable dragon rides on HBO’s Game of Thrones, Jabar worked across departments to accurately bring MPC’s animation of the flying elephants to the on-set motion base, also solving camera moves for a Spydercam system to create the paths of Dumbo’s flight through the frame.
“When working with creature interaction through motion control, the first challenge is always defining the movement of the equipment,” describes Jabar. “We liaised closely with MPC to analyze their Dumbo flight cycles and the rhythm of the animation and produce feedback about what was possible and what would need to be created through intricate camera counter-movement. Once we had a good flight cycle, the stunt department would come in and analyze how their bodies reacted to the motions. The biggest challenge is always translating enough movement to the equipment so our eyes believe the actors are reacting to the creature in perfect sync.”
The team next created a CG simulation of the stage rig that would convert MPC shots into the final physical exports. The CG stage rig was designed to preserve and distribute motion between the six-axis gimbal, Dumbo’s mechanical head, a stabilized Libra head cradling the Alexa Mini and Spydercam’s winch system.
Once the Dumbo animatronic was put down to the measured specs, Jabar and the team incorporated a Lidar scan of the shooting stage to render accurate lens passes of each setup. “To ensure nothing was left to chance or had to be simulated ‘on the day,’” she says, “we also exported forward kinematic data for each axis of the equipment involved — one positional or rotation value per frame of animation for 14 axes, across three systems and three different file types! We additionally worked with the AD to make sure everyone knew the file they needed to load per shot, tracking the status of each file on a master whiteboard.”
Shots with dialogue action for Collette (Eva Green) while she sits on Dumbo presented a particular challenge for the motion-base shoot. Dumbo’s performance was closely tied into the actress’s rhythm of delivery, so the team worked with Green to time her lines and then adjust the solves, adding pauses and edits to the Dumbo performance where needed. The adjustments were then tracked back to the original MPC animation frame for frame. Frame in and out points were meticulously maintained and everything was tied to timecode, with renders of how the two correlated to the MPC files driving the shoot.
Synchronization with the equipment was done through Spydercam’s proprietary Moto system, which triggered everyone on set, from video assist to lights. This was especially important due to the speeds some of the camera moves had to achieve — some upwards of 26 feet per second, where a single frame delay would be the difference between having the subject’s body in frame on a fly-by and looking at the ceiling. “The Spydercam system is second to none for repeatability and fine control,” says Jabar. “It allowed us to turn the stage into a huge motion control volume and achieve all the stage flying elements on schedule by leveraging detailed pre-planning.”
Once principal filming wrapped, The Third Floor created postvis for shots that included Dumbo and other elephants and animals, replacing stand-in props that had been used on set or adding temp versions of the characters and performances where they were needed within the plates.
According to Summers, “While the action and performances for some scenes had been visualized pre-shoot in previs with Tim that then fed into postvis, there were also scenes where Dumbo’s performance needed to be post-produced to fit the performances of the actors and stand-ins that had been recorded using props and eyeline references, or we temped Dumbo’s animation in post over the reference from a suited actor. For some shots, there was not a live-action representation or a previs to work from as we post-vised the plates.”
The Third Floor’s postvis artists used 3D Equalizer to hand over fully tracked Maya scenes lined up against a 3D Lidar scan of the set. Postvis shot creators then animated Dumbo into the postvis using a projection of the plate in Maya for reference. Finally, postvis artists or dedicated Nuke compositors at The Third Floor created finished postvis, compositing the elements, cleaning up the plate and adding any bells and whistles needed for storytelling.
With animals throughout the movie being digital, The Third Floor first worked on adding elephants to the clean postvis plates, focusing mainly on Dumbo. As postvis progressed, other animals were added, mainly as background elements, but some characters — like the monkey Barrymore — featured in sequences of their own.
In the postvis blocking, artists had to consider timing for a CG Dumbo that sometimes needed to move at a quick pace against more quiet crowd action in the background plate. And in both postvis and previs, capturing a sense of the required emotion and mood was always front of mind.
“It was particularly fun working on performance-based sequences that showcased Dumbo’s developing playfulness,” Summers concludes. “And it was great helping realizing homage moments, like the opening scene where the circus train crosses the American landscape. I’d say the single biggest challenge across the board in visualization was making an elephant fly in a believable way. There is nothing cartoony about Tim’s take on Dumbo and he wanted him to look and feel physically real, especially when flying. We really had to suspend everything we knew about the physical rules of our world as we made ourselves believe elephants could fly.”