I can remember the day, just like it was yesterday, when I saw my first 1gb disk drive. It was the size of a small toaster (5¼-inch wide and two full-height spaces tall). You could have used it as a boat anchor. This was amazing technology. We hadn't seen anything that large. Most disk controllers on the market couldn't support it. They didn't understand the large block count geometry.
Of course, today, you can go buy 6,000gb drives as easily as you can buy a toaster.
On the day I saw that 1gb drive, I immediately referred back to the day I saw my first 20mb drive. It was in the used, broken PC I had received as a gift, and I was amazed at how much stuff this thing could hold! 20mb was a lot of space.
This story plays out over and over in the technology space. In fact, if you assume the late '80s as the beginning of the personal computer boom, we're approaching the 30-year mark for the standard "back when I was a kid, we had floppy disks and we liked it" comment. It needs to stop.
However, there are times when these technological shifts mean something. When a common technology simply going 10X faster or being 10X bigger (or 10X smaller) actually changes our lives and requires something more than just a whistle and some nostalgia about our first computer.
Some things that changed my life: The mouse, Ethernet, Windowing graphics with icons (Windows obviously, but HP Vue and Irix Indigo Magic, as well), the hard disk (versus floppies), always-on Internet, e-mail with attachments, YouTube, and a phone with enough memory to store my music library.
Teton Gravity, a small tree customer, illustrates the majesty of shooting in 4k.
For those in the production/postproduction industries, 4k digital video formats look to be one of these technological shifts.
When Small Tree started selling Ethernet equipment back in 2005, we didn't spend much time worrying about video formats. We sold 1gb Ethernet cards, and the most anyone could hope for over a link like that was 70mb/sec (under the best conditions). This either meant a crummy-quality codec or you weren't editing in real time. Then, Apple ProRes came along. We didn't know anything about ProRes. It was just another proprietary codec. Without even realizing it, our customer base expanded rapidly because customers could now play out, in real time, high-quality, HD video over our gigabit Ethernet ports. Storage didn't just become important; it became our main business.
Living with 4K
4k digital codecs hit with a similar pattern. I certainly knew there were 4k codecs out there. There are many cameras that support them. I even had conversations about running 4k video tests to see how many 350mb/sec streams we could handle. However, it never struck me as a serious contender for "common" workflows since very few customers would have 10gbE links, very few would have the capacity required to store video at 350mb/sec, and the common demands of today's collaborative workflows pretty much ruled out 4k codecs as a digital intermediate.
Going all the way back to my days at Silicon Graphics, I've had countless conversations with customers wanting uncompressed HD or 4k, but then balking at the price when we got into the specifics. 10gbE links to each workstation (perhaps two), eight spindles (disks) for two 350mb/sec streams, a 10gbE switch, perhaps as much as 192tb of storage just to handle six editors. It adds up. A budget system like this might run $90,000 today (and easily $200,000 just a couple of years ago).
Of course, ProRes came along and changed this. At this year's NAB, it was apparent that facilities are working in 4k, and they are doing it (sometimes reluctantly) with ProRes.
Why does this matter? The main reason is the bandwidth and recent advances in the 10gb Ethernet space. Depending on frame rate, a ProRes 4k stream can be 150mb/sec. This means that gigabit Ethernet is no longer sufficient. It means there are now real and compelling reasons to move to 10gbE for current systems and future deployments.
Last year, a workflow requiring 10gb Ethernet throughout a facility might have been a deal-breaker. 10gb cards were expensive, and switch performance was hit or miss. Many vendors had offerings, but not all of them were good ones.
Today, Tomorrow
Today, 10gBase-T Ethernet is fairly common and runs over Cat6 cabling. There are inexpensive switches from the likes of Netgear, and there are now Thunderbolt 10gbE devices that can be used on Mac and Windows clients alike. This fundamental shift in the cost and deployment of 10gb Ethernet has made the 4k shift viable and inevitable. There's no excuse not to begin planning for 4k workflows by deploying 10gb Ethernet.
To go one further, it also resurrects a long-standing plumbing problem. If you are feeding your clients with 1gb links, what kind of link do you run back to your server? Clearly, it can't be another 1gb link. You can't feed 10 editors expecting 1gbE performance with a single 1gbE port on the server. Servers need 10gbE to feed those editors' workstations. So with our new paradigm of 10gb Ethernet to the desktop, what will we storage manufacturers put in our servers?
Have no fear. The Ethernet standards people already have the answer: 40gb Ethernet. It exists, and we've been working on the drivers for Intel's 40gbE controllers. The media type won't be familiar to most people (it'll be QSFP copper or optical), but for running short distances from a server to a switch, it's perfect.
The inexorable advance of technology in speed and power isn't going to stop, and we all need to get used to it. It just keeps moving. We shouldn't be spending a lot of time marveling over 10tb disk drives. In fact, we should come to expect that next year, we'll have 20tb disk drives.
What's been missing for many of us is an imperative to move with these new technologies.
First and foremost, adopting new technology is risky. For most computer and technology companies, their main competition isn't the guy down the street, it's the status quo. Many users are happy to leave well enough alone, and without an imperative to change, they do just that.
I believe these new 4k compressed codecs are going to be the biggest technology revolution driver we've seen since ProRes was first released. I believe the combination of 4k ProRes and 10gb Ethernet over Thunderbolt is going to move us to a new plateau.
However, keep in mind that it never stops. Don't marvel at today's impressive storage numbers and speeds. Think about next year's. Put your seat belts on, and let's get ready for 100gb Ethernet!
Steve Modica is CTO of Small Tree Communications (www.small-tree.com). He brings over 26 years of experience in the field of high-performance operating systems to the company. Prior to founding Small Tree, he worked at Silicon Graphics, managing a group of device driver engineers and managing third-party vendor relationships.