Backup Strategies for Creatives
/
Why do we backup? So we can recover.
Why do we back up? For security? For confidence? For redundancy?
We back up because we are smart enough to understand the science of storage. It's not a question of WHETHER storage will fail, it's a question of WHEN. Manufacturers of high capacity storage like traditional hard drives provide MTBF numbers, if you can find them. MTBF stands for Mean Time Before Failure. That bafflegab means that half of the measured devices will have failed by the time specified. For a spinning hard drive that's typically about six years. SSDs are rated higher, and depending upon who you talk to, but current reasonable estimates are ten years.
Some manufacturers such as Seagate, eschew MTBF numbers entirely using their own measurements. Typical manufacturer numbers are 1.5M hours for a spinning drive and 2M hours for an SSD in constant use. Do the math and that's nowhere near the estimates that I provided in the first paragraph. In a study conducted by Carnegie Mellon University, manufacturer guidance is out on average by a factor of 15 times. That's a high amount of leeway and suggests that maybe the manufacturer numbers are likely inaccurate. I'm not saying that they are lying, but their testing conditions are optimized and since there have not been hard drives for 171 years, their estimates are extrapolations based on discovered failure rates.
This hard drive has died. Yours can too.
All disks have bad sectors. Higher end storage maps these out as part of the configuration but in our "right freaking now" society, this step is usually passed over. Drives are meant to be kept spinning, but our real world usage doesn't do this. We shut machines down, we encounter power faults, we move machines around while they are running. None of these activities are optimal for drive life. Certainly Solid State Drives (SSDs) are improving in quality and reliability, and definitely outperform spinning drives, but the capacity is not yet there in an affordable format.
So what to do? Assume that there will be data loss. It may not be the entire disk, but when you lose something, it will be something important rather than something trivial. The key to to have backups.
Information Technology professionals, and that's what I've done for most of my working life, know that backups are integral to long term data success. We tend to value a three tier model. I personally believe that if there are not three copies of any data item, there is NO backup.
For years, people were using Compact Discs, DVDs and even Blu-Ray discs as backup targets. I cannot recommend this to anyone. While the media is inexpensive, media drives are vanishing from computers as the entire need for this kind of media wanes rapidly. Consumer blank media is not the same quality as commercial grade write once media. We have seen surface rot, surfaces becoming unlaminated and other disasters that make these mechanisms unsuitable as backup targets. For those of you still thinking about tape, stop. It's too slow and is non-linear so restoration will be a major pain, presuming that your new computer can even support a tape subsystem.
Consider the following structure. You keep your photo and video originals on a hard drive. You may also keep interim edit steps on the same or a different hard drive. I don't care whether this is a spinning drive or an SSD. That's copy number one.
Next you attach an external drive or drive array. This secondary storage can be attached by USB, Thunderbolt or be on a local area network in the form of a SAN device (storage area network). If your computer supports USB3, the external should be USB3. USB2 is slow, unreliable and a poor data replication design. No professional who truly understands storage architecture is going to be excited about USB2. For many people, the storage attachment of choice was IEEE 1394 also known as FireWire. FireWire and USB2 were like Beta and VHS. One was way better and the other one proved to be the market success. FireWire support from manufacturers is now non-existent. If your secondary storage is FireWire, it's time to be planning a change. If you use Apple gear, Thunderbolt and Thunderbolt II are the best choices, although USB3 can also be fine. Thunderbolt support on Windows platforms is growing but is still not ubiquitous.
When looking at secondary storage, there are two options. Buy a fully enclosed device or put your own together. If you can manage a screwdriver, option two is better because you will get to choose the drive that goes in the case. Some vendors put a lot into making the case look pretty but have skimped for years on the drive inside. For the best performance in your backup system you want a fast spinning drive with a large cache. My recommendation at time of writing is the Western Digital Black series with 64mb of cache. Try to avoid consumer and "green" drives as they operate more slowly and will spin down and spin up a lot. While this consumes marginally less power, it also is harder on the drive than just spinning the same speed all the time. If you must buy a complete system, the higher end Western Digital and GTech kits are fine. It's your money, but I would be inclined to steer away from Seagate products as well as LaCIE due to a long series of negative experiences.
If you are looking at a Network Attached Storage system that can be shared by all the people in your home, I heartily recommend the line from Synology. I have no experience with QNAP but have read good reports. A very popular lineup is from Drobo Systems. While I own many Drobos, all but one has failed on me, and I have had to go both in and out of warranty for service. I love the concept, I just don't recommend the solution. Photographers who follow Scott Kelby may recall his rather vocal dumping of all things Drobo a couple of years back. NAS is a bit more work to set up, but delivers the potential for more functionality beyond just pure backup.
Ok, so now you have your data in two places. Excellent! Unfortunately that data is still in the same location and disaster recovery professionals, and I am one, will advise that your offsite storage should be at least 10km away on a different power grid and different external network subnet. I understand that sounds a lot like bafflegab and I am sorry, but it's a what's so. Fortunately, this is not hard at all.
Cloud storage doesn't look exactly like this in real life
For the last several years, I have paid a subscription to a company called Crashplan. I have "cloud" backup for all the machines in the house, with no limitations on space or how the storage connects to the computers. I load the agent and it just works. I get regular status updates on all the machines and can be very selective of what get's backed up to the cloud and what does not. The one drawback to cloud backup is that you need a decent Internet upload connection because a lot of data will take months to get up to your offsite storage. There are many cloud storage providers. Amazon is one of the biggest, but working with them is pricey and complex. I have heard of many different ones, but will not recommend any that I haven't used personally. Plans change, but in the past, many of the alternatives wanted more money if you had external drives or NAS devices that you were backing up. The message being sent is that they want your money, but not your files. Some of the alternates also placed data caps on storage AND throttled your uploads to keep load on their own downlinks low. No thank you.
Once you have an offsite setup running, you have three copies and you are now backed up. Good. Next try to restore something from each of your backup targets. Be random in your attempts and that will tell you how reliable your backup is. If you don't test restores, you have no backup.
Lastly, I am writing about backing up your creative work. This is separate and independent of a machine level backup. If you have a Mac and don't use Time Machine, give your head a shake and start now. It is by far the simplest backup method out there, just be sure to format the external drive as Mac OS X with a GUID filesystem if you want it to be bootable. For my Windows machines, I use Acronis True Image. I actually DO have Windows machines, and Acronis has been my go to answer for years. It just works, and I can make the backup a proper clone drive if I need to, and I've needed to.
In summary, backup for creatives is not an option and you want to have three levels of data integrity and to check that it is working. I want to thank my very good friend Valerie for the question that inspired this article.
