Storage Requirements for Digital Content
Driven by the demand for image quality in theaters and homes, feature-film resolutions are on an upward trend. In the high-end feature-film market, 2k resolution is common, and 4
k resolution is gaining ground. With increasing resolution and storage demands, new solutions will be needed to store and move those assets throughout the studio and into theaters and the home entertainment market.
Nearly all feature-film production is still captured on film and must be converted to a digital format with a film scanner before postproduction begins. Once the film is ready for distribution, the digital content must be copied back to film.
Almost all content creators now use nonlinear editing of digitized content, and most special effects today are done with digital techniques. This streamlines the editing process, resulting in faster editing at a lower cost.
Nonlinear editing is generally done with uncompressed or slightly compressed content, since heavy compression increases the overhead of editing and can cause timing problems. For a large facility with several editing chairs, shared network storage allows the local disk storage to be kept at about 30 minutes per station.
Storage networking has been decreasing in price due to the maturity of Fibre Channel SAN components and the growing use of iSCSI SANs and network-attached storage (NAS).
The high-end segment of the nonlinear editing market requires expensive components to support bandwidth and latency requirements for 2k and 4
k resolution. RAM is often used as a buffer in various parts of nonlinear editing systems to help reduce the impact of system latencies.
Compositing and special effects are increasingly a staple of films and other digital content. Demand for more sophisticated results will increase storage requirements (capacity and performance). Special effects are often done with clustered computers connected to storage networks. These are generally based on open computer architectures with proprietary data management software.
Preserving new digital content and converting historical analog content to digital form will be the single largest driver of digital storage capacity. Much of the storage for archiving will be on removable media such as tape and optical disks that can be put on a shelf or in a library until needed. Digital preservation allows content to be available for research and distribution.
Many major digital conversion and preservation efforts are under way worldwide. For example, there are very large libraries of material being converted to digital archives, such as the 100,000-hour CNN library. Other major networks such as CBS and Sony/Columbia are also digitally archiving materials; CBS, has more than 1,045,000 tapes and 150,000,000 feet of film, and Sony/Columbia’ has 600,000-plus reels and tapes to convert.
One of the biggest issues for archiving is the obsolescence of the storage media technology. Tapes or optical disks get outdated, and if the digital content that they contain is not transferred to new media, it will be difficult to preserve, cannot be easily read, and likely will be lost.
As the size of the digital archive increases, it will become more difficult to transfer digital content fast enough to preserve that content. A serious issue in the future will be having sufficient bandwidth available to convert from old media to new media. Archiving will not be a static or occasional process. Format conversion of large data stores may eventually require almost continuous transfer operations. When the archive load becomes too large, choices will have to be made about which content to transfer and preserve on the new format.
Between 2004 and 2010, analysts expect a 900-fold increase in the required digital storage capacity for the digital creation and distribution markets. With the growth in storage demand for high-resolution content and the ease with which digital footage can be acquired, digital storage demands for content acquisition should match those for archiving and preservation by 2010. Analysts also expect that extensive digital conversion projects will occur in the intervening period.
In 2004, analysts estimate that 60 percent of the total storage media shipped for digital entertainment content was on tape, with 40 percent on optical disks.
By 2010, the change in segment demand will also change the mix in digital storage media, and tape usage and optical disks should decrease to 40 percent and 55 percent, respectively, with hard disk drives comprising a 4 percent market share.
Thomas Coughlin is president of Coughlin Associates, a data storage consulting firm.