One of the hats I wear as a technologist is that of adviser to a small independent Television production company. A discussion we had last week centred around the viability of equipping a video editing suite against the expense of hiring it. The economics of that discussion are not the subject of this article, but the conversation did touch upon the different technologies available and the possible futures.
Photographers and Filmmakers have similar challenges around workflow and since these two industries are increasingly overlapping with the video capabilities available in modern DSLR’s it makes sense to discuss this storage question here. I wrote about the RAID storage solution I use for my photography last week, but that was obtained several years ago and although ethernet is fast, moving video data off a RAID and into an editing tool such as Final Cut Pro is demanding in ways that still photography just isn’t. What is required is super fast connectivity between storage and workstation. Fibre fits the bill, but for the photographer it is unnecessary. (It is appropriate for a professional video editing house using racked storage arrayed across multiple hard drives). For Photographers and Video makers, eSATA and Thunderbolt both offer very fast connectivity between storage and workstation. The problem here is that various vendors have adopted different technologies. G-Technology, the makers of the G Drive I raved about in my previous post use eSATA, and Promise, the makers of the Promise Pegasus solution have adopted Apple’s Thunderbolt technology.
eSATA boasts data transfer speeds of up to 200MB/second which is plenty fast enough to edit video. Compare with Ethernet at 125MB/second, Firewire 800 at 98MB/second, Thunderbolt weighs in with a staggering 1.25GB/second. Now be aware that data will only move at the speed the slowest component dictates, so if you are using a 7200 rpm disk drive it will deliver approximately 50-60MB/second. SSD drives on the other hand offer 100-500 MB/second depending on type. These figures are approximate and in the case of the traditional spinning disk drives, factor in the delay in locating the data on the disk.
At one level the debate boils down to identifying the slowest component on the network. If you’re going to invest in high speed data transfer and then use wireless to connect your laptop, you’ll see a data transfer of only 75 MB/second, making your investment in fast storage completely redundant!
So which one is king? Thunderbolt is clearly the fastest by a long way. It is also the newest which means it has the least support in the market at the moment. It does offer one very interesting capability though, which Apple have been quick to exploit. You can daisy chain Thunderbolt devices together in a way which has been commonplace in storage for years, but Apple have extended the principle to include other hardware components, notably monitors and desktop computers. This means that you can plug a Thunderbolt enabled storage component into your Apple Thunderbolt enabled monitor into your AppleMac Mini Server. Hmmm…. attach that server to your Ethernet LAN and you have a solution only limited by the speed of Ethernet. Install FinalCut Pro on the AppleMac Mini and you have all the speed that Thunderbolt allows.
On the other hand, eSATA boasts wider support, is plenty fast enough for local editing and substantially cheaper. You’ll need to buy an eSATA card or adapter to realise the full speed, but the drives all support Firewire 800 as a matter of course. As a photographer, I’d go with eSATA as the data speed is sufficient to make photo editing seamless. As a videomaker I would seriously consider whether my requirements justify the additional expense of Thunderbolt. Having said that, I wouldn’t underestimate Apple’s ability to push Thunderbolt to the forefront. They succeeded with Firewire, but at the moment, and this is to my mind a big disadvantage, the Mac Pro cannot support Thunderbolt. That support will be forthcoming when the next overhaul of the Mac Pro architecture is completed.
In conclusion, Thunderbolt is exciting, no question, but for my purposes, unless I win a lottery sometime soon I’ll be sticking with my ReadyNAS Ethernet based RAID and my eSATA back ups.
An upfront statement – I have spent the last fifteen years in the IT industry and if there is one thing I’ve learned it’s that the importance of backups cannot be understated. In fact I’d go so far as to say that amateur or professional, if you don’t have your photographs backed up, you will lose them sooner or later!
So I wanted to write about workflow. This is the process that moves your images from camera/card to computer to print/client/website. Most of us constantly reuse cards and the wise ones reformat the card once the content is uploaded. But what if the computer fails – or what if one of those files is inadvertantly deleted or corrupted?
I use 8Gb or 16Gb SanDisk cards in the camera and like most people probably, upload the pictures directly into the laptop for sorting and processing. I do not delete the data from the card at upload time. I rotate the cards so that the oldest spare is always next in the camera – this means I have a window of opportunity to retrieve any corrupt data in the laptop, from the original card. A better way of doing this, and one which I use when I’m away from home for any length of time, is to upload the pictures from the card into a standalone drive. I have a Nexto Extreme, 250 Gb SATA disk which comes with firewire and card reading slots. As you can see, I can store a lot of 8 Gb cards on there, 30 in fact. I copy the files from the Nexto, onto the laptop for processing. Thus maintaining two copies.
Naturally I fill up the Nexto sooner rather than later, so I maintain two further separate collections on a ReadyNAS RAID device that will store up to 3 Terabytes of data. I have over a thousand music albums on there as well as about five years worth of photography. Its only 16% full. What are the two collections? One is the original RAW files, filed by date and event. The other is the exported, finished article. For those who are not familiar with RAID, it is a technology that maintains data across an array of several disks (I have four) in such a way that should a drive fail or even two, the data can be rebuilt. Once the original is uploaded to the ReadyNAS, I can relax about the card and the Nexto.
The other area where the data is vulnerable is on the hard drive of the computer. There are a number of reasons why you shouldn’t co-locate photographs with the operating system, the most obvious being the risk of the operating system crashing and wrecking data (Yes, Microsoft, I’m looking at you!). Apart from that, the fact is you will eventually fill up the drive and slow the computer to a crawl. To avoid co-location, you can partition the drive, install a second hard drive or as I did recently, invest in a G Drive 3Tb 3200 eSATA device or equivalent. This is a wonderful piece of kit. Over Firewire, it’s acceptably fast. Over eSATA it’s as fast as the internal drive. It’s quiet, unobtrusive and portable. The best bit is I can keep my entire Lightroom catalog on the disk and move between laptop and workstation as I please.
This is a setup that works for me. It might seem like overkill, but the bottom line is this:
1. Copy your original RAW files to a second disk
2. Don’t process Photographs on your OS disk
3. Keep two copies of your work at all times.
You will sleep much better at night, trust me!