There is a tightness in my chest, and I am crying right now. I have just suffered a catastrophic data loss for the second time in my life. Fool me once, shame on, shame on, fool me can't get fooled again, or something like that. In college, a freak transformer explosion and subsequent power surge killed my hard drive. From that point on, I swore to always back up my data, and mostly I did just that. As of mid this summer, I had a ridiculous mirrored RAID drive setup with external SATA drives and all sorts of doohickies. I had about 1 terabyte of data backed up locally and had started to upload it offsite to a service called Mozy. But then I started selling off my desktop in preparation for my move from Boston to NYC. I purchased a LaCie 1TB Big Disk and put all my media files and documents from my "Atlas" drive on it. That drive literally held my world on its shoulders. I reasoned that after the move, I would re-establish my redundant data setup. I was not given the time. Two days after moving in, the drive started clicking. I knew that sound from my college crash, and raced to B&H Photo Video in Midtown. I purchased a Drobo storage device (a redundant storage array), hoping to save my Atlas drive. I was too late. I took the drive to Tekserve on 23rd St. It would cost $2,000, but can you put a price tag on your memories and thousands of hours of media production? They couldn't recover it. They sent it on to DriveSavers who said it may cost up to $6,000. I had recently closed out my Discover Card, but decided it was worth going back into nasty credit card debt. Then today, I got the phone call. "We have some bad news." They could recover nothing. They will just charge $400 for the attempt. It's funny, I struggled with the decision to send them the drive considering the cost but it is so clear now that I would rather have paid $10,000 to get my data back. On the technical side, here is what happened. That LaCie big disk is actually two 500GB drives "striped" together in an array. One of those drives failed and because the data is stretched across both, you can get nothing even from the good drive. Fortunately, I managed to get some of my data uploaded to Mozy as of late May 2007. So I've managed to recover all my digital photos as well as my "Documenz" folder which includes my books, jokes, financial filings, scripts and everything else a digital paper version of a file cabinet would have. Over the past year, I have been using Google Docs for most of my day to day creative documents with columns, joke ideas, etc, so that's all good. Unfortunately, I have lost much, much, much more, so much that I cannot even be sure how much.

• My iTunes music and video library. (~300GB) I estimate I had about $1500 worth of purchased music and videos in there plus hundreds of gigs of ripped CDs. The good news is I saved all the original CDs and can re-rip them. I had also "acquired" a massive music collection from a friend which ended up creating more problems than it solved. There was a lot of music I never really wanted to own permanently. I can repurchase the iTunes music at far less than the cost of the data recovery, though I'll see about begging Apple for a restoration. I've head that happens sometimes.
• My video projects (~500GB). This includes imported MiniDV footage and many edited and rendered Final Cut and iMovie projects made since January 2005. The good news is I have all the original MiniDVs and I can download the most valuable rendered projects back from YouTube (I hope) and blip.tv which hosts a bunch. The bad news is video is the most time intensive, high learning curve activity I have ever engaged in. Much of my knowledge in those project files has to be relearned.
• My audio projects (??GB). This includes raw audio for my podcast, including dozens of unedited, unreleased interviews. I've often felt bad that I never got to many of these. Now I have a pretty good excuse.
• My old computer files. About two months ago, I extracted data from my old college computer hard drives and put them on the Atlas drive. This had emails, papers, mp3s, etc. I was so excited to have found this time capsule, but now it's gone.
• My mother. At the end of it all, I am pained by the loss of the above items, but nothing can represent the sense of anguish I feel at having lost audio of my mother who passed away two years ago. We had taken a cross-country drive together, and I recorded hours of conversation. I only got to podcast a little bit of it (which can be redownloaded from my webhost) but the unedited stuff is beyond valuation. It's like losing her all over again.

I certainly blame Lacie for the drive that failed, but my data is my responsibility. I will mourn this loss forever, and I really will never let it happen again. I'm trying to be open minded about this. It's the most aggressive "spring cleaning" I've ever done. Even with my mother's memories, I have thousands of photos and a bit of video. Mostly I have her in my heart, and if I think about it, I just happen to live in an era where it's possible to capture image and sound in such high fidelity. Most of the people that ever lived had no such technology to remind them of their lost loved ones. The best memories are always going to be with me. Now, here's the plan

• I have the Drobo with 1.3 terabytes of capacity to be the home of New_Atlas. This drive will also be mirrored on a 1TB external Glyph and online via Mozy or a similar service. Any recommendations?
• My MacBook Pro internal drive will be mirrored on the Drobo/Glyph/Mozy setup as well
• I'll keep a smaller subset of high priority files for more frequent offsite backup

I urge everyone reading this to backup your most important files right now. Not tonight. Not tomorrow. But right now. Do a local backup. Upload files to a server. Email them to yourselves. Print things out and put them in a lockbox. If you're interested in the Drobo, I have a discount code you can use for $25 off. It's EVBARATUNDE, and yes I get some money out of it. Mostly, I want you to avoid what I'm going through. Update: September 10, 2007 @ 11:13am Wow, I never expected such a massive response to this, of all my posts. Most of yall found me through reddit, it seems. Thanks for dropping by and thanks for all the very useful suggestions. I wanted to provide a few more details of what went down and why I wrote this in the first place

• The more technical side of the failure is that the "master" drive is fine but the "slave" drive suffered a series of head crashes. Basically, a major mechanical failure happened, and the platters inside the drive collapsed. It does not appear to be due to physical impact but just a mechanical malfunction
• I appreciate just about all the comments except for those telling me I'm an idiot. I know that. This single-point-of-failure system was temporary during my move. The odds of the drive failing in such a short period of time are low, but it happened. Remind me to drop in on your house and mock you when you suffer your own tragedy
• I wrote this post to 1) provide an emotional outlet for me 2) see how others might be handling their own data backups in this era of digital memories but mostly 3) to scare people into backing up their stuff as soon as possible in one way or another. I really don't want this to happen to others. It costs too much in time and emotional energy.

If you're interested in what I do when I'm not lamenting the loss of my digital existence, here are a few posts to give you a flavor for what I'm about. If you like what you see/hear/read, subscribe to the feeds or join the email list (both at the top of the blog page in the left and right columns)

• Facebook Follies (or the Dangers of Investing in Someone Else’s Platform)
• gcc 004: I Hate the Smugness of Apple (VIDEO)
• Iraq is to Vietnam as Dubya is to WTF!?!? (Huffington Post)
• Jesse Jackson Really Needs to Read a Book… explaining what satire is
• Me. Public Radio. Monday Apr 24. Racism In America.
• and my standup set on YouTube

Update: September 11, 2007 @ 8:50am First up, welcome digg users who put me on the front page. I am so glad this story is making people back up their stuff. This is unbelievable. A few more updates

• Apple's iTunes folks restored almost all my purchased items, 83GB worth in over 900 tracks.
• A 1TB Glyph drive has arrived which I will use to back up the drobo in a "spanning" setup. It's two drives, but in this case the Glyph will fill up one drive then the other, sequentially. I'll store this in a fire-proof box in my home. I'm also gonna store this all on Mozy, so that's three places (two on-site and one remote) with ALL my data. I'll make smaller backup sets of really important stuff
• Today is my birthday, and getting Dugg is the best web gift ever... way better than a $1 Facebook "gift" :) And yes, it really is my birthday. Check the vid...

Update: September 11, 2007 @ 1:05pm  Another update. I've been reading the comments further and want to point people to a few more resources

• The Infrant ReadyNAS is a tool many have mentioned for hard drive backup. I studied it vs. the Drobo and chose Drobo, but that may not be right for everyone or even for me.
• An eBook recommended by Macworld called Take Control of Mac OS X Backups, 2nd Edition. Here is a free article in two parts which goes through a lot of the same material. There is some great content here even if you don't have a Mac

Technorati Tags: backup, data, hard drive failure