Tree @master (Download .tar.gz)
Space Madness is a backup system. It is evolving. It is my attempt to describe the backup system that I try to use. This repository contains those notes, and some tools that may or may not help in impementing this backup system.
The main tool used here is
rsync. If it is not installed as a standard part
of your operating system, install it first.
The main unit of organizing files in Space Madness is the depository, which
is a directory tree. The subdirectories in a depository are called subjects.
Subjects are hierarchical (for example, you might have
A subject is expected to have the same content in all the different depositories in which it appears. (This is not 100% true, but it's useful to think of them this way.)
There are three kinds of depositories:
This contains the canonical copy of the files it contains. You might have more than one canonical depository, but their contents do not overlap; e.g. one canonical depository for text files, another canonical depository for movies.
Canonical depositories can be stored on removable media, e.g. a USB stick or external hard drive. This is convenient if the files are used on more than one computer, e.g. your laptop and your desktop.
This is a redundant copy of a canonical depository or part of a canonical depository. Each canonical depository should have at least one cache depository containing an entire copy (that's what makes this a backup system.) But it might also have cache depositories for convenience of access. For example, I might want to keep a copy of my software documentation on my netbook.
Files saved to a cache depository are in jeopardy; they are apt to be overwritten or deleted when the cache is updated. So don't do that.
This contains files which have not yet been put into a canonical depository (perhaps because the filesystem on which the canonical depository is stored is not available at the moment) but which are intended to be put there soon.
With these definitions, we can say that a canonical depository is backed up if and only if every subject in it exists in at least one cache depository which resides on a different physical medium than the canonical depository.
Note that two partitions on the same hard disk are not different physical media!
The goal of the backup system is that every interesting file is in a canonical depository, and that every canonical depository is backed up.
To accomplish this goal, there are two important actions on depositories.
Update a cache depository from a canonical depository
To update a cache depository from a canonical depository,
rsync --archive --verbose --delete $CANONICAL/subject/ $CACHE/subject/
To check if a cache depository is up-to-date with a canonical depository,
rsync --archive --verbose --delete --dry-run $CANONICAL/subject/ $CACHE/subject/
We might provide a lightweight wrapper for those.
Add the contents of an incoming depository to a canonical depository
To update a canonical depository from an incoming depository,
deposit $CANONICAL $INCOMING
In fact this is virtually the same as saying
rsync --archive --verbose $INCOMING/ $CANONICAL/
but incorporates some consistency checking, i.e. that you don't trample
something in the canonical depository with something in the incoming with the
same name. (Also,
rsync is pretty picky about directory names, and if
you forget the trailing slash, bad things can happen. So it helps there too.)
deposit tool takes an additional flag,
--clean, which deletes all the
files (but not the directories) in the incoming depository after they are
copied over. However, as a safety check, it will not function unless the files
have already been copied into the canonical depository, and are identical.
It is an excellent idea to update a cache depository from the canonical
depository after running
deposit (and before running
Advanced Topics in Backup Subjects
Because subjects are hierarchical, it is also possible to think of (and treat) a subject as a sort of sub-depository.
It is up to you to decide if you have any backup use cases that are complex enough to justify doing that.
It is also useful to keep certain backups in version-controlled repositories,
e.g. git repos. In this case, a repo directory can usually be treated as
a subject. Instead of depositing files with
deposit, one can push changes
to the repo-subject residing in the canonical depository.
Need to research this, but the basic idea would be to
chmod -R a
cache depository so that you don't accidentally change files in it
or add files to it. (But then changing the permissions back when you
need to update it from a canonical depository.)
It is unlikely that you will always use a backup system perfectly, and even less likely that you started backing up your files with a perfect backup system. It is in fact likely that at some point you just copied important files from one place to another to ensure that you had a backup copy.
Thus, Space Madness includes some tools to assist with cleaning up backups and making them meaningful.
We should eventually document them here.
git clone https://git.catseye.tc/Space-Madness/
- Add --dry-run option to `deposit` utility. Chris Pressey 6 years ago
- Define 'backed up' for depositories, and talk a bit about repos. Chris Pressey 6 years ago
- Add headings and more notes to README and reformat to 80 columns. Chris Pressey 6 years ago
- Explain things more fully in the README. Chris Pressey 6 years ago
- Initial import of tools, and rudimentary documentation in README. Chris Pressey 6 years ago
- Initial commit Chris Pressey 6 years ago