Here’s your New Year’s reminder: back your stuff up. Do it redundantly, with extreme paranoia, and do it now. If it helps, I’ll even say “Pretty please.”
You put so much time into your work, and harddrives fail all the time. So really, this is not paranoia. It’s wisdom. My preference is that every file, including my Lightroom catalog files, be on three harddrives, one of which is always off-site. So at home I work on an iMac, my files are all on a G-Speed Q RAID drive. Every week, or after any real work session, I run a Super Duper backup of that RAID to one of two G-Tech 4TB drives. I could automate this, I suppose, but I’m really organized, so I’ve never felt the need. The other drive is in a Pelican case at my manager’s office, off-site. He comes over, brings the drive and we swap the week-old one for a new one, always leap-frogging the current drive. This means I’ve always got my files on 3 drives, or at minimum 2, and one of those is always off-site.
There are other solutions out there for off-site back-up. I’ve not tried any of them, though all my personal and business administration files go to DropBox. I’ve heard good things about Backblaze, Amazon Glacier, and Crashplan, all of which are off-site and cloud-based.
The storm, Sandy, that the east-coast of America dealt with recently is a good reminder that the unexpected happens. That’s one of the reasons I keep a drive in a Pelican case. It’s why you should have your stuff off-site and redundant. I know, we’d all rather have a new lens than to have to shell out for drives, but it doesn’t take much to lose it all.
Want more? Check out this video from Chase Jarvis. It’s probably way more than you need, but you’re bound to learn something – if it’s good enough for Chase, it’s worth listening to.
Already backing up your stuff? Well done! Tell us about your strategy and then go calibrate your monitor. 🙂 Happy New Year.
Comments
Good article David. Just a month ago or so I had this sudden panic attack where I realized all my photos were only stored on machines right in front of me on my desk. The thought of losing all that work is just absurd. So now I store all my files on the harddrive of my Mac, on an external hard-disk that’s sitting right next to it, and through an online backup service in the cloud. This works really well for me, and it’s quite simple as well: you just install some software that checks the folders you designated each day at a fixed time and backs them up in the background. I don’t even really notice the software is working.
For documents etc I use the same strategy you do, Dropbox is just great. Additionally, the more devices you use Dropbox on, the more redundant backups you add, so it gets even safer the more you use it, just for that totally unlikely case where Dropbox loses all their files…
hi everybody, please don’t forget to read use-agreements for every on-line/cloud/whatsoever utilities (dropbox, google drive etc). There can be many “dangerous” reservations for your work/content (i.e. heard some bad things about Google Drive).
🙂 have a safe backups 🙂
Crashplan has an “initial feed” service. They will send you a 1T hard drive you do your initial backup to and you mail it back. When they get and process it, all you are going to do is incremental backups from this time on. You save money and bandwidth.
Crashplan does data-deduplication so if you have copies of images on your hard drives or if you move a folder of images, it will recognize the same files and will not send their content to the servers extra time.
Also, Crashplan will backup incremental changes to your open files, like LR catalog file you have open while you work. It will backup those changes instantly. Time Machine had issues with this use case.
As for sharing my backup, I have a mac mini server that has several attached external hard drives that Time Machine backs up all home computers to including the iMac that is used for working with photos. Also, Crashplan client is making a local backup of the iMac to another external hard drive attached to it. And yes, Crashplan is backing up all home computers to their cloud service.
This way, I have three copies of images and LR catalog and all of them have versioned backups so I can go back in time if needed. One of the backups is “in a remote location” at Crashplan data centers and is maintained automatically and instantly.
-Sergey
Brilliant. Thank you for taking the time to leave the comment, Sergey. Some great ideas here.
I too still need reminding! I’m currently investigating hard drive back-up vs Cloud. I’m guessing it’s going to take quite a few days to upload 250Gb of photos onto a cloud server??
My question may be silly and probably half-answered, but you back up everything more than once and only after you go through your images and delete the one you don’t like (out of focus or…).
Photographers prefer to back up everything, and buying more space than taking the time to cycle through the whole project deleting the bad images and only backing up the good ones?
My process, and from what I’ve read what many other photographers do, is to backup the “current state” of their library. As you cull photos, for bad focus or whatever, you’ll eventually purge those from your library. Most folks I know do this in Lightroom, where the file isn’t deleted until you specify to purge (otherwise it still exists but is hidden from view as “rejected”). Once those deletes propagate to the backup, they will be lost forever. I’m OK with this.
Some may not be, and that is where the method of backup can help. There are ways to preserve deleted files while doing backups. The problem becomes how long do you keep those deleted files. Don’t forget, most backup systems aren’t smart enough to know that a moved or renamed folder isn’t a delete. This can start piling up lots of files and lots of disk space which translates to time and/or money (pick your mix).
I think that the really careful folks just keep multiple copies of their “current state” backups. They might keep multiple copies of their backups over time. For example: last week, last month, 3 months ago, and 1 year ago (4 copies total). That way if they realized they lost a file they could go back in time and grab it again. They are probably also careful about how frequently they purge their rejects from Lightroom.
I suppose one could always select never to purge the rejected files in lightroom, but then the cost in time and money (disk space) can start to really add up.
The most important thing is to keep a redundant backup of your current work (two different disk drives, usually “mirrored”), and then off-site backups of that; however frequently (or however many copies) you need. This protects you against both disk failure and on-site disaster (theft, fire, or other).
If you have any other specific questions I can try to answer.
-Shawn
I can´t believe I still need to be reminded about sufficient backup….But I do!! Thanks!!
I have a similar multi-drive setup but I really neet to get a better routine in order. I also have a bootable Super Duper backup drive for the Mac, I keep all non system/image files in Dropbox, and allow Crashplan to do it’s thing.
Pingback: Things You’ll Find Interesting January 2, 2013 | Chuq Von Rospach, Photographer and Author
Put a fair amount of effort in to our backup/storage system this past year.
We use a Synology 412+ as our primary store which is configured as RAID-5. Daily this gets backed up to a Synology 212j which is configured as RAID-0. I have not added the archival mechanism in yet, that is, the offsite storage of the backup done on a regular basis.
On an hourly basis, all of our Macs are backed up using TimeMachine as well.
Presently I have 5.4T of RAID-5 (9T total) and 5.4T (6T total) of RAID-0 for storage. I highly recommend the Synology as a NAS unit, very fast and has worked flawlessly.
Hi David,
Thanks a lot for that post. I am with you on this one that the importance of backups can’t be repeated often enough.
I have an on-going project of doing a portrait of my kids every sunday. My daughter is now 3 years and my son is 1 year old. Lots of image data coming together in the end. I would never ever forgive myself in case I lose even one of those pictures. Never!
That’s why I have sat down and thought about my backup strategy including on-site and off-site backups. Have written about it in my blog-post “Backup Early and Often” (http://bit.ly/TmqrZI).
Am also using G-Tech products and am really happy with them. Lately, I have checked out CrashPlan for off-site backups, cool stuff.
Cheers,
Tobi
Hey David,
Great post. I’ve been using Crashplan for a few months now and it’s worked great (in addition to all my other backups.).
The point about US servers above is good, but for what it’s worth, Crashplan encrypts the data with 448bit encryption using a key that’s not publicly known so even if someone were to go snooping they shouldn’t be able to read it, at least not easily. Still worth thinking about if you have sensitive data.
-Andrew
What product can one get where you download from a SD or CF card straight onto a external hard drive without the use of a computer….?
Try this, Donna – http://www.hypershop.com/HyperDrive-s/119.htm
I have a mac mini server at home that hosts my media library and is the hub of my backup plan. The server has 2 internal drives. Drive A is the server HD and drive B is a time machine backup for my MBP and my wife’s Air. So when our laptops are on the wireless network, they back up automatically to Drive B.
Next, a 3TB (Drive C for this case) external backs up both the A and B drives from the server.
Lastly, a 750 gig G-Drive Mini gets a copy of drive A and B (server + our laptops) and lives in a pelican 1020 case, off site.
All vital docs are in dropbox. I also back up the LR catalog to dropbox.
It sounds complex, but it is quite seamless and automatic, with the exception of the g-drive mini, which I copy manually.
I’m always looking for ways to simplify the process and love learning about what works for other people!
You’ve inspired me to Backblaze my stuff. I have two harddrives at the moment, but have wanted to add a third option for a while now. I keep hearing good things about Backblaze so I think I’ll that dip later today.
Hey David,
Because you mentioned this stuff a few years ago I was reminded to create my own similar backup plan. I had to go the “cheap” route though, using much smaller Synology NAS, backed up nightly to an external hard drive, and swapped whenever I remember to do it to a secret offsite location (my office). Thanks for mentioning it again, everyone needs a backup plan these days, better sooner than later.
My backup plan never has and never will include “the cloud” though, for a couple of very valid reasons. I’ll mention them, but I don’t feel like debating them:
1) Bandwidth: Here in the USA service providers are few and far between, and they have what we all want: data. It’s a matter of time before they realize we’re all addicted to the Internet and streaming movies, music, and backing up lots and lots of data. I anticipate that they’ll soon realize demand is higher than supply and raise prices, possibly even capping monthly data volumes (currently at 300GB a month on my ISP, possible to get lower). I don’t want my strategy to suddenly get very expensive or need to drastically change.
2) Legal Action: Stories involving the service called “Megaupload” make me hesitant to think that my data would be retained if legal action (government or private) were to be taken against my backup service for any reason (ex. media piracy). I don’t think Megaupload was a backup service and may have promoted piracy, but that may not stop the RIAA or Department of Justice from taking some kind of legal action whilst trying to pursue pirates.
Call me paranoid, but I feel safer being able to physically handle my data backups.
-Shawn
P.S.
The “cheap” version of the Pellican is an old cardboard box with a piece of foam that once housed a video card or some other now-forgotten computer hardware.
‘Cloud” services are a privacy nightmare. You may want to reconsider what you are using DropBox for in the future. Personally, I wouldn’t put anything on those servers that meant anything to me.
For instance, you are Canadian and if you have your “personal and business administration files” stored on DropBox. Their servers are located in the US and are subject to the Patriot Act, basically, the US government can have a look at your files for whatever reason they see fit.
Don’t take what I have to say as legal advice as I am not a lawyer. I have however, educated myself with respect to privacy laws and again, these ‘cloud’ services are just not the way to go in my opinion. Look into it…
Thanks, this is something I hadn’t thought of in my own post regarding the cloud below.
Yes, I concur. The Patriot Act is a dangerous piece of legislation, essentially allowing US authorities to do just about anything they want with anything stored within their jurisdiction.
Its important to remember that the same applies to your personal laptop or storage devices when travelling in the US. They are subject to US privacy laws, not ours; so you are not protected.
So using Dropbox, Amazon S3 or any of the other services should be done with this in mind.
i agree this is something that needs to be very well understood by the users of any “cloud” file system.
I happen to own a web hosting company (incorporated in Canada, with servers in both US and Canada), and we’ve run into this issue numerous times over the years.
If your “cloud” storage location is in the USA, then the feds don’t need a warrant to look at your data. It’s on US soil, so they can look at it. It’s just that simple. Yes, “technically” they’re supposed to have some sort of reason, like they suspect you of being a terrorist or money-laundering or whatever… but the reality is that we’ve had to turn over our US-based servers to the feds with little more then a “we just want to see what xyz person is doing”. We have no choice in the matter.
Now, that does not mean I don’t think US-based services are evil. Not at all. It just makes me want to remind everyone that you should *encrypt* your data before sending it off into the cloud.
There are numerous free and for-pay utilities that will let you use very high levels of (for now) unbreakable encryption, so your data can be safely stored. Do not rely on the encryption provided by the service provider, since it’s very likely they will have your key stashed somewhere.
Yes, the feds can still grab a copy of your encrypted data and try to crack it. No, it’s not likely they will be able to crack it, as long as you’ve used a good encryption key, etc. (And it’s very unlikely they would allocate the money and resources necessary to even try, unless they really do think you’re a terrorist or laundering vast amounts of $$.)
TL;DR – Your data located in a US-based cloud service can be taken by the US government at any time for no reason. Use proper encryption before sending your data. Do not rely on the service provider’s encryption.
I use BackBlaze w/ my iMac and have had good success. (I can also recommend Carbonite.) Using TimeMachine to regularly make a local backup copy–buttressed by a remote online backup–has been a solid solution for me.
As a mild tangent, David, do you use an iMac for color-critical work (e.g., prepping files for print)? I’ve had difficulty getting good results, so I drive an external matte screen LCD monitor with the iMac and use it.
Finally, Happy New Year to everyone!
Chris – Yes, I use my iMac, calibrated with a Spider-Pro or Elite. I’ve had consistently excellent results but I’m also not known for being super picky about colour, so I guess we’d have to agree on a definition of “colour-critical” For me my calibrated iMac works beautifully.
Thanks, David! Interesting to hear, b/c your images always look marvelous–both on screen and in print. I’ve struggled not only with color but with the deceptive sharpness/contrastiness of the glossy screen. Perhaps I’m too busy seeing detail and missing the bigger picture (something I struggle with). Time to finish up Within the Frame….
Thanks for this important reminder. A backup (AND RESTORE) plan should be on everybody’s new year’s resolution list.
Nice feature for Mac users: Since OS X Mountain Lion (10.8), Apple’s time machine backup works on more than one drive. So you can have one at home and one at work and wherever you plug it in to your (portable) machine, it starts off from the last state one that specific drive.
For users of earlier versions of OS X, there’s Tedium (http://www.tediumapp.com/). It lets you switch between backup drives quite easily. Time Machine has supported that feature for some time, but it wasn’t visible in the user interface, hence the need for this extra app. I feel really safe, now that I have working TM backups at home and (off-site) at the office.
Btw. fun fact I read about once: there has been a study on the optimum distance of off-site backup media. It took into account all kinds of disasters, including a nuclear attack and weighed it against the cost of having the backup at a certain distance. The result was something around 10 kilometers.
Wise words. I currently need to get a third backup drive to leave offsite but I have two at home (one in fire proof safe).
I only off site once a month, as I only have one drive for it so prefer to keep it away if I can. I’m not super prolific so it works well enough, but seeing as we’ve just had a big event (Christmas) I’m doing one 2 weeks early tonight!
One question: Your off site drives are 4TB, how big is your RAID then? If more than 4TB how do you deal with this problem?
Sound advice.
I have a post suggestion. You mention working on the iMac at home. I know you use Lightroom 4. I’d be interested in hearing how you balance and sync mobile work and office work (laptop LR4 edits and desktop edits). Do you export mobile edits as a catalog when you return home and import them onto the iMac? Do you work off a catalog that’s stored externally on a drive you bring with you on travel?
I’m a new LR4 user and still trying to work out a good system for syncing my libraries across systems, so hearing other’s methods is useful for me.
Good idea, Matt. Might be a nice compliment to this one. No idea when it might happen now, though. About to get busy and begin travelling again. The short answer is, I travel with my 11″ MacBook Air. When I get home I export all files as DNG to a small drive, then on the iMac I simply import the DNGs. Only downside would be for those folks doing a ton of virtual copies, I’m not sure they’d get copied over.
That is correct: virtual copies are part of the catalog, so they will not come along unless you export as catalog.
Matt, it seems like you know how to do this, but for anyone who might not:
select the images (including VCs) that you want to move to your desktop machine and choose File > Export As Catalog…
Make sure to check “Export negative files” (as well as “Export selected photos only” if you’re doing just a subset).
Copy the folder that is created (it will contain your catalog file, the images, and the previews if you exported them). On your desktop machine, choose File > Import from Another Catalog… and select the .lrcat file.
Thanks for the reminder David!
Can you tell which Pelican you are using to keep your G-Drives in?
Pelican 1400. Perfect fit for the drives I use.
Thank you sir!!
I thought that the 1400 might be the one, just wanted to be sure.
Also I love my Saddleback Briefcase (thanks to you)!
Good advice, David, but please don’t confuse Dropbox with backup. If you accidentally delete a file that is in dropbox, it is gone. Backblaze (and probably Crashplan – I haven’t used it) will let you retrieve previously deleted files.
That’s true if yo u are their “Free’ customer. But if you are a paid customer with add on Packrat you can recover any deleted file.
https://www.dropbox.com/help/113/en
You do have 30 days to restore a deleted dropbox file but the same could be said for physical backups as well. If you accidentally delete a file and run (in this case) SuperDuper, that file will be removed from all backups too.