Borg. I did it.
I'm not sure why, but I guess I had to try.
I had and still have a good data backup system that I created on my own. I have had to restore whole directories from it, and it works. I trust it, but of course it isn't perfect.
The system that I wrote is slow and also redundant. In a way, those characteristics are positive, slow less so, but I can live with slow if that's what it takes to do it right.
Redundancy means safety. Because with redundancy I know that I have multiple copies of irreplaceable data, saved daily. I have the space to store multiple copies of my data, so the only real issue with my in-house data backup system is that writing four to five GB of data to external drives takes a bit of time, and I have been doing it twice a day.
But then I read about other ways to do this sort of thing, and there are several well-known systems, Borg probably being the best-known, so I just got a bit itchy to try something, and started with Borg because it looked like the most complete system and the one with the longest and best track record. Maybe I was most interested because Borg is a "de-duplicating" backup method. In other words, Borg does not re-copy the same data on each backup. If Borg has safely backed up a directory or file, and that directory or file has not changed, then Borg leaves it alone.
Learning about Borg was confusing though, and despite me having it working now, and working well, there are still some things that I cannot figure out. They're minor but annoying in that I spent a lot of time on them without getting to solutions.
The main issue is that in my original backup process, I created a list of directories to back up, and put that list into a file. When the process runs, it reads in turn from that file the name of each directory to back up. If I ever decide to add or delete a directory from the process, then all I need to do is to edit that file and the backup process follows the new instructions on its own without needing to be edited.
I could not get Borg to work this way. The best I could do was to get Borg to read the contents of my backup list file and back that up as a single line of data, and then quit. This was after trying everything that I could think of.
Part of the issue may be with bash. Bash is at best obtuse, with a heavy overlay of flat-out weird, and I will never be an expert. And I don't want to be one. Given that though, I have written a bunch of bash scripts to do various things, and my original backup process is written as a set of bash scripts, and it works. Borg resists.
My original process goes through my list of directories to back up and uses tar to put each directory into a separate tar file. That's the backup script itself. I have a second script for the second half of the process.
The second script pushes the backed-up data to whatever external drive is plugged in. All I need to do is to tell this second script which backup this is. I have separate external directories for Sunday through Saturday, another one for mid-month, and another for month-end. Other than that single instruction, entered numerically from a menu as a number from 1 through 9, everything is automatic.
Borg is even simpler and unbelievably faster, but I had to embed the list of directories to back up as literals into the Borg script. So to add, rename, or delete a directory from the Borg version of the backup process, I need to edit the Borg script and not just an external, read-only file.
This is annoying. I'm sure that there is a way to get Borg to read from an external file, but not here, not now, not by me, and no, it's not in the documentation. Not clearly enough to be recognizable and understandable by me.
I did test Borg's integrity by restoring a backup and comparing the total size of the restored data, and checking some of the filenames and file sizes.
That all looked good, but I've decided to keep using my original system for a monthly backup, and for a couple of weekly backups, like Wednesdays and Saturdays. Makes me feel safer. It can't hurt to have a little redundant data lying around, at least until I completely trust Borg, and restoring data from a Borg backup requires Borg to be installed. Restoring from tar backups requires only generic tar software, which is almost universally installed.
But, even given Borg's confusing and incomplete documentation, I do like it a lot.
Actual data backups usually take somewhere in the low hundredths of a second, and writing new data into my repositories takes only a couple of seconds more. I'm almost always done with a backup to one external drive within 15 seconds at most, and that includes connecting the drive, running my Borg script, and unplugging the drive again, even my very slow thumb drive.
Can't beat that one.
So for now, unless I need other features, and happen to find something else that is significantly better, I'm relying on Borg for my routine backup needs. Nothing else does look any better, and most tools look worse.
I did check Restic, which got a slightly higher rating on one review site, but Restic looks far more confusing, much more oriented toward business and server system backups, has even more confusing documentation, and is far less mature. Methinks I don't want to mess with it.
True, Restic is written in Go, while Borg is written in Python, so Restic may be much faster, and may be certifiably much faster, but Borg does my small backups so fast that I would never notice a difference. I don't have terabytes of data to wrangle. With a de-duplicating process, it's more like a few hundred kilobytes per day, if even that much.
So then. I was happy to give up trying to understand Restic at all, and decided to just stick with Borg, which I already had working.
I use Borg daily, it works, and there is no fussing around. Good enough.
Done.
Refs:
BorgBackup – Deduplicating archiver with compression and authenticated encryption
Borg Documentation Borg Documentation — Borg - Deduplicating Archiver 1.2.6 documentation
BorgBase - Simple and Secure Offsite Backups
borgmatic
Have anything worth adding? Then try sosayseff@gmail.com
Me? Temporarily feeling good about myself.
Etc...
so says eff: sporadic spurts of grade eff distraction
definitions: outdoor terms
fiyh: dave's little guide to ultralight backpacking stoves
boyb: dave's little guide to backpacks
snorpy bits: nibbling away at your sanity
last seen receding: missives from a certain mobile homer
noseyjoe: purposefully poking my proboscis into technicals