Smoke and Mirrors? Not by a long shot
Sheffield, UK, and Chatsworth, Calif. – Visual effects and post-production company Smoke & Mirrors goes live with a new organisation wide, Digital intermediate, Visual FX, and Petabyte media archive storage environment at its offices in New York and Soho, London.
The storage deployment, a world first for the media industry, enables upwards of six users - working on broadcast, film, and commercial productions - to simultaneously read and write to the same digital video sequences at resolutions of over 2K in real-time, which requires 300 megabytes per second for each stream. This real-time, shared file capability allows access to files instantaneously and eliminates data copy across the workflow, allowing editors to transform the original film to digital format and edit the content instantaneously.
Replacing a previously manual process, which involved scanning, couriering, and uploading digital format content to storage at the London and New York offices, the new storage system provides considerable competitive advantage to Smoke & Mirrors’ and enables the company to significantly increase staff productivity and deliver content to customers faster.
The storage system also enables Smoke & Mirrors to pool, new disparate storage hardware and make it available to all staff via personal 2D and 3D workstations. A more secure storage system, this replaces myriad local and network attached storage drives.
In addition, the storage system also enables automatic archiving and retrieval of old digital production data to tape. This saves Smoke & Mirrors, on average, three person days per month spent searching for old content to archive and also enables Smoke & Mirrors to retrieve old content for customers instantly – a task which previously took several days.
Built using IBM General Parallel File System (GPFS), IBM Tivoli Storage Manager, and DataDirect Networks (DDN) S2A6620 hardware, the pioneering storage system was designed, installed, and configured by UK High Performance Computing and storage system integrator OCF.
The Storage System
‘Dark fibre’ connects Telecine film scanners, in use to transform original film to digital data formats, from a nearby post production facility to the offices of Smoke & Mirrors in London and New York. Transformed digital data is transferred over 10GB Ethernet at a high-speed rate of 750MB per second, delivering 30+ frames of film per second to Smoke & Mirrors. The digital data is delivered directly to the new central storage pool. Two IBM GPFS System x3650 M2, with Nehalem Dual quad cores handle the file transfer over 20 port Qlogic fibre switches into two DDN S2A6620 storage subsystems providing 50 TB of storage capacity.
Mid tier - 3d and 2d Render Farm
The second element of the storage system consists of four IBM GPFS System x3650 M2, with Nehalem Dual quad cores, over DataDirect Network’s DDN6620 storage hardware and Qlogic switches. Collectively this provides 200 TB of storage capacity accessible locally by multiple 2D and 3D rendering workstations.
End tier - Tape Library
All digital data is backed up to a 900 slot Quantum Tape Library, providing 700 TB of storage capacity with IBM GPFS and IBM Tivoli Storage Manager (TSM) working side by side.
GPFS tells TSM what files need backing up and what files still need to be accessed (according to workflow rules).
Commenting on the new architecture, Barry Evans technical director OCF, says: “One of the greatest challenges for the media sector is its silo approach to unstructured data.”
“GPFS can potentially become a really big thing in the media space simply because of its standard functionality – it knows where data is held, can handle data lifecycle management effectively and offers tight integration with Tivoli Storage Manager.
Because this solution is all based on non proprietary technology, Smoke & Mirrors has the perfect architecture that can be expanded easily as the business grows.”