[geeks] Cheap/reliable backup?

[Jonathan Patschke]

On Mon, 2 Dec 2013, microcode at zoho.com wrote:

> Is there a cheap/reliable way to backup data from a disk?

I use both automated backups and "RAID" to do different things, with lots
of scripts and manual labor to make the solutions more reliable.

On my Mac, I use Time Machine, which is effectively rsnapshot with a
pretty[0] UI (and some truly horrific file system hacks to make it work
well with HFS+).

That's not entirely accurate.  I have a Mac Pro which has an unmirrored
boot disk, a mirrored "Vault" filesystem, and a Time Machine disk.  TM
backs up most of the boot disk (modulo huge VM disks, ~/tmp, and other
things that don't need backing-up).  Stuff only goes on the "Vault"
filesystem if it's something that leans more towards data loss through
hardware failure, rather than user error.  Occasionally, I take a copy of
"Vault" and put the copy in a fire-proof box.

Photos go onto the Mac, into the Time Machine-managed area while I'm
working on them, and into the Vault area once they're in a fixed state.
If I had time anymore to do music, I'd do the same thing with recordings;
sheet music lives in git.

My notebook uses Time Machine; data only lives on the notebook
temporarily, as it's either all code (git) or documents/photos (rsync to
big Mac).

On my big FreeBSD box, I run zfs, so I do hourly/daily/weekly snapshot
rotations.  Periodically[1], I'll take the most recent snapshot and
extract it to an external disk that I keep at the office.  Every few
weeks, I stop by the bank to pull the disk out of the safe deposit box,
bring it home, flush the most recent snapshot[2] to _that_ and then take
it back to the bank the next day.

I used to use rsnapshot, and it worked really well.  If you don't run ZFS,
I highly recommend rsnapshot.

My code and other texty things live in either svn (old) or git (new), and
the master repositories all live on the server and get archived in a
last-30-changed-days rotation (in case the repo metadata gets screwed
somehow).  That collection of compressed archives goes somewhere on the
big server so it can participate in the offsite-backup game.

Customer data gets rsynced onto mirrored disks, and occasionally I'll make
an additional mirror that goes to the bank vault.  My customers have no
SLA with me; this is me being nice, as they sometimes delete things, and I
like to be a hero.

For virtual BSD/Linux/Solaris machines, I rsync /etc, /usr/etc, /var, and
/home to subdirs on the big FreeBSD server.

On my Windows VMs, the only data I care about is source code, which is in
svn or git already.

I used to use tape, but quickly outgrew DDS2, then DLT-IV.  BluRay looked
like The Thing for a while, and it really could be, if there were software
that streamed FS backups to a BluRay library in a WORM-fashion with SSD as
an index.  Eventually, I got into the habit of buying slow high-capacity
disks when they go on deep discount at $hatefulElectronicsStore.  Combined
with a $40 eSATA "toaster" style dock, I have a backup device.

Disk is easy, disk is random access, disk is cheap.  Just don't ever trust
only one.

Off-site is key.  A flood that will swap your server will probably swamp
your file cabinet.  If you can't do off-site, get a fireproof box with
good gaskets.

Nothing I've mentioned above is particularly expensive relative to the
cost of the things that necessitate it[3].  They all require dedication
and habit, and for that, I really miss Tivoli Storage Manager, the IBM
Shark, and the cubicle-sized Ultrium library at $employer[-1].
Thankfully, I'm not responsible for backups at the day job.


[0] That is to say "pretty ugly and almost useless," IMO, but it does the
     job.
[1] When I remember...
[2] Not the latest snapshot of _everything_, the latest snapshot of stuff
     that would devastate me to lose: source code, home dirs, mail,
     databases.
[3] In the US, we have a culture of tipping service staff, and fitting
     into that culture requires factoring the tip into the price of a night
     out.  If you can't afford the tip, you can't afford the dinner or the
     valet or the whatever.  In much the same way, backups are the "tip" of
     data management.
-- 
Jonathan Patschke | "No matter how much the government controls...any
Elgin, TX         %  problem will be blamed on whatever small zone of
USA               |  freedom that remains."         --Sheldon Richman