How I Keep my Life in Git

git secret_cabal greet

After watching My life in git, after subversion, after CVS. from DebConf, I’ve realized it’s been a while since I talked about the way I keep everything¹ I do in git, and I don’t think I’ve ever done it online, so it looked like a good time for a blog post.

Beyond git itself (of course), I use a few git-related programs:

• myrepos (also known as mr) to manage multiple git repositories with one command;
• vcsh to make it easy to keep dot-files under git;
• git annex to store media files (anything that is big and will not change);
• etckeeper to keep an history of the /etc directory;
• gitolite and cgit to host my git repositories;

and some programs that don’t use git directly, but easily interact with it:

• ansible to keep track of the system configuration of all machines;
• lesana as a project tracker and journal and to inventory the things made of atoms that are hard ² to store in git.

All of these programs are installed from Debian packages, on stable (plus rarely backports) or testing, depending on the machine.

I’m also grateful to the vcs-home people, who wrote most of the tools I use, and sometimes hang around their IRC channel.

And now, on to what I’m actually doing.

With the git repositories I’ve decided to err for too much granularity rather than too little³, so of course each project has its own repository, and so do different kinds of media files, dot-files that are related to different programs etc.

Most of the repositories are hosted on two gitolite servers: one runs on the home server, for stuff that should remain private, and the other one is on my VPS for things that are public (or may become public in the future), and also has a web interface with cgit. Of course things where I’m collaborating with other people are sometimes hosted elsewhere, mostly on salsa, sourcehut or on $DAYJOB related gitlab instances.

The .mr directory is where everything is managed: I don’t have a single .mrconfig file but a few different ones, that in turn load all files in a directory with the same name:

• collections.mr for the media file annexes and inventories (split into different files, so that computers with little disk space can only get the inventories);
• private.mr for stuff that should only go on my own personal machine, not on shared ones;
• projects.mr for the actual projects, with different files for the kinds of projects (software, docs, packaging, crafts, etc.);
• setup.mr with all of the vcsh repositories, including the one that tracks the mr files (I’ll talk about the circular dependency later);
• work.mr for repositories that are related to $DAYJOB.

Then there are the files in the .mr/machines directory, each one of which has the list of repositories that should be on every specific machine, including a generic workstation, but also specific machines such as e.g. the media center which has a custom set of repositories.

The dot files from my home directory are kept in vcsh, so that it’s easy to split them out into different repositories, and I’m mostly used the simplest configuration described in the 30 Second How-to in its homepage; vcsh gives some commands to work on all vcsh repositories at the same time, but most of the time I work on a single repository, and use mr to act on more than one repo.

The media collections are also pretty straightforward git-annex repositories, one for each kind of media (music, movies and other videos, e-books, pictures, etc.) and I don’t use any auto-syncing features but simply copy and move files around between clones with the git annex copy, git annex move and git annex get commands.

There isn’t much to say about the project repositories (plain git), and I think that the way I use my own program lesana for inventories and project tracking is worth an article of its own, here I’ll just say that the file format used has been designed (of course) to work nicely with git.

On every machine I install etckeeper so that there is a history of the changes in the /etc directory, but that’s only a local repository, not stored anywhere else, and is used mostly in case something breaks with an update or in similar situation. The authoritative source for the configuration of each machine is an ansible playbook (of course saved in git) which can be used to fully reconfigure the machine from a bare Debian installation.

When such a reconfiguration from scratch happens, it will be in two stages: first a run of ansible does the system-wide configuration (including installing packages, creating users etc.), and then I login on the machine and run mr to set up my own home. Of course there is a chicken-and-egg problem in that I need the mr configuration to know where to get the mr configuration, and that is solved by having setup two vcsh repositories from an old tarball export: the one with the ssh configuration to access the repositories and the one with the mr files.

So, after a machine has been configured with ansible what I’ll actually do is to login, use vcsh pull to update those two repositories and then run mr to checkout everything else.

And that’s it, if you have questions on something feel free to ask me on the fediverse or via email (contacts are in the about page)

Update (2023-09-12 17:00ish): The ~/.mr directory is not special for mr, it’s just what I use and then I always run mr -c ~/.mr/some/suitable/file.mr, with the actual file being different whether I’m registering a new repo or checking out / updating them. I could include some appropriate ~/.mr/machines/some_machine.mr in ~/.mrconfig, but I’ve never bothered to do so, since it wouldn’t cover all usecases anyway. Thanks to the person on #vcs-home@OFTC who asked me the question :)