Thicket data repository for the EEG
anil.recoil.org
1{
2 "id": "https://www.tunbury.org/2025/06/14/borg-backup",
3 "title": "Borg Backup",
4 "link": "https://www.tunbury.org/2025/06/14/borg-backup/",
5 "updated": "2025-06-14T00:00:00",
6 "published": "2025-06-14T00:00:00",
7 "summary": "Our PeerTube installation at watch.ocaml.org holds hundreds of videos we wouldn’t want to lose! It’s a VM hosted at Scaleway so the chances of a loss are pretty small, but having a second copy would give us extra reassurance. I’m going to use Borg Backup.",
8 "content": "<p>Our PeerTube installation at <a href=\"https://watch.ocaml.org/\">watch.ocaml.org</a> holds hundreds of videos we wouldn’t want to lose! It’s a VM hosted at Scaleway so the chances of a loss are pretty small, but having a second copy would give us extra reassurance. I’m going to use <a href=\"https://www.borgbackup.org\">Borg Backup</a>.</p>\n\n<p>Here’s the list of features (taken directly from their website):</p>\n\n<ul>\n <li>Space-efficient storage of backups.</li>\n <li>Secure, authenticated encryption.</li>\n <li>Compression: lz4, zstd, zlib, lzma or none.</li>\n <li>Mountable backups with FUSE.</li>\n <li>Easy installation on multiple platforms: Linux, macOS, BSD, …</li>\n <li>Free software (BSD license).</li>\n <li>Backed by a large and active open source community.</li>\n</ul>\n\n<p>We have several OBuilder workers with one or more unused hard disks, which would make ideal backup targets.</p>\n\n<p>In this case, I will format and mount <code>sdc</code> as <code>/home</code> on one of the workers.</p>\n\n<div><div><pre><code>parted /dev/sdc mklabel gpt\nparted /dev/sdc mkpart primary ext4 0% 100%\nmkfs.ext4 /dev/sdc1\n</code></pre></div></div>\n\n<p>Add this to /etc/fstab and run <code>mount -a</code>.</p>\n\n<div><div><pre><code>/dev/sdc1 /home ext4 defaults 0 2\n</code></pre></div></div>\n\n<p>Create a user <code>borg</code>.</p>\n\n<div><div><pre><code>adduser <span>--disabled-password</span> <span>--gecos</span> <span>'@borg'</span> <span>--home</span> /home/borg borg\n</code></pre></div></div>\n\n<p>On both machines, install the application <code>borg</code>.</p>\n\n<div><div><pre><code>apt <span>install </span>borgbackup\n</code></pre></div></div>\n\n<p>On the machine we want to backup, generate an SSH key and copy it to the <code>authorized_keys</code> file for user <code>borg</code> on the target server. Ensure that <code>chmod</code> and <code>chown</code> are correct.</p>\n\n<div><div><pre><code>ssh-keygen <span>-t</span> ed25519 <span>-f</span> ~/.ssh/borg_backup_key\n</code></pre></div></div>\n\n<p>Add lines to the <code>.ssh/config</code> for ease of connection. We can now <code>ssh backup-server</code> without any prompts.</p>\n\n<div><div><pre><code>Host backup-server\n HostName your.backup.server.com\n User borg\n IdentityFile ~/.ssh/borg_backup_key\n ServerAliveInterval 60\n ServerAliveCountMax 3\n</code></pre></div></div>\n\n<p>Borg supports encrypting the backup at rest on the target machine. The data is publicly available in this case, so encryption seems unnecessary.</p>\n\n<p>On the machine to be backed up, run.</p>\n\n<div><div><pre><code>borg init <span>--encryption</span><span>=</span>none backup-server:repo\n</code></pre></div></div>\n\n<p>We can now perform a backup or two and see how the deduplication works.</p>\n\n<div><div><pre><code><span># borg create backup-server:repo::test /var/lib/docker/volumes/postgres --compression lz4 --stats --progress</span>\n<span>------------------------------------------------------------------------------</span>\nRepository: ssh://backup-server/./repo\nArchive name: <span>test\n</span>Archive fingerprint: 627242cb5b65efa23672db317b4cdc8617a78de4d8e195cdd1e1358ed02dd937\nTime <span>(</span>start<span>)</span>: Sat, 2025-06-14 13:32:27\nTime <span>(</span>end<span>)</span>: Sat, 2025-06-14 13:32:38\nDuration: 11.03 seconds\nNumber of files: 3497\nUtilization of max. archive size: 0%\n<span>------------------------------------------------------------------------------</span>\n Original size Compressed size Deduplicated size\nThis archive: 334.14 MB 136.28 MB 132.79 MB\nAll archives: 334.14 MB 136.28 MB 132.92 MB\n\n Unique chunks Total chunks\nChunk index: 942 1568\n<span>------------------------------------------------------------------------------</span>\n<span># borg create backup-server:repo::test2 /var/lib/docker/volumes/postgres --compression lz4 --stats --progress</span>\n<span>------------------------------------------------------------------------------</span>\nRepository: ssh://backup-server/./repo\nArchive name: test2\nArchive fingerprint: 572bf2225b3ab19afd32d44f058a49dc2b02cb70c8833fa0b2a1fb5b95526bff\nTime <span>(</span>start<span>)</span>: Sat, 2025-06-14 13:33:05\nTime <span>(</span>end<span>)</span>: Sat, 2025-06-14 13:33:06\nDuration: 1.43 seconds\nNumber of files: 3497\nUtilization of max. archive size: 0%\n<span>------------------------------------------------------------------------------</span>\n Original size Compressed size Deduplicated size\nThis archive: 334.14 MB 136.28 MB 9.58 MB\nAll archives: 668.28 MB 272.55 MB 142.61 MB\n\n Unique chunks Total chunks\nChunk index: 971 3136\n<span>------------------------------------------------------------------------------</span>\n<span># borg list backup-server:repo</span>\n<span>test </span>Sat, 2025-06-14 13:32:27 <span>[</span>627242cb5b65efa23672db317b4cdc8617a78de4d8e195cdd1e1358ed02dd937]\ntest2 Sat, 2025-06-14 13:33:05 <span>[</span>572bf2225b3ab19afd32d44f058a49dc2b02cb70c8833fa0b2a1fb5b95526bff]\n</code></pre></div></div>\n\n<p>Let’s run this every day via by placing a script <code>borgbackup</code> in <code>/etc/cron.daily</code>. The paths given are just examples…</p>\n\n<div><div><pre><code><span>#!/bin/bash</span>\n\n<span># Configuration</span>\n<span>REPOSITORY</span><span>=</span><span>\"backup-server:repo\"</span>\n\n<span># What to backup</span>\n<span>BACKUP_PATHS</span><span>=</span><span>\"\n/home\n\"</span>\n\n<span># What to exclude</span>\n<span>EXCLUDE_ARGS</span><span>=</span><span>\"\n--exclude '*.tmp'\n--exclude '*.log'\n\"</span>\n\n<span># Logging function</span>\nlog<span>()</span> <span>{</span>\n logger <span>-t</span> <span>\"borg-backup\"</span> <span>\"</span><span>$1</span><span>\"</span>\n <span>echo</span> <span>\"</span><span>$(</span><span>date</span> <span>'+%Y-%m-%d %H:%M:%S'</span><span>)</span><span> - </span><span>$1</span><span>\"</span>\n<span>}</span>\n\nlog <span>\"========================================\"</span>\nlog <span>\"Starting Borg backup\"</span>\n\n<span># Check if borg is installed</span>\n<span>if</span> <span>!</span> <span>command</span> <span>-v</span> borg &> /dev/null<span>;</span> <span>then\n </span>log <span>\"ERROR: borg command not found\"</span>\n <span>exit </span>1\n<span>fi</span>\n\n<span># Test repository access</span>\n<span>if</span> <span>!</span> borg info <span>\"</span><span>$REPOSITORY</span><span>\"</span> &> /dev/null<span>;</span> <span>then\n </span>log <span>\"ERROR: Cannot access repository </span><span>$REPOSITORY</span><span>\"</span>\n log <span>\"Make sure repository exists and SSH key is set up\"</span>\n <span>exit </span>1\n<span>fi</span>\n\n<span># Create backup</span>\nlog <span>\"Creating backup archive...\"</span>\n<span>if </span>borg create <span>\\</span>\n <span>\"</span><span>$REPOSITORY</span><span>::backup-{now}\"</span> <span>\\</span>\n <span>$BACKUP_PATHS</span> <span>\\</span>\n <span>$EXCLUDE_ARGS</span> <span>\\</span>\n <span>--compression</span> lz4 <span>\\</span>\n <span>--stats</span> 2>&1 | logger <span>-t</span> <span>\"borg-backup\"</span><span>;</span> <span>then\n </span>log <span>\"Backup created successfully\"</span>\n<span>else\n </span>log <span>\"ERROR: Backup creation failed\"</span>\n <span>exit </span>1\n<span>fi</span>\n\n<span># Prune old backups</span>\nlog <span>\"Pruning old backups...\"</span>\n<span>if </span>borg prune <span>\"</span><span>$REPOSITORY</span><span>\"</span> <span>\\</span>\n <span>--keep-daily</span><span>=</span>7 <span>\\</span>\n <span>--keep-weekly</span><span>=</span>4 <span>\\</span>\n <span>--keep-monthly</span><span>=</span>6 <span>\\</span>\n <span>--stats</span> 2>&1 | logger <span>-t</span> <span>\"borg-backup\"</span><span>;</span> <span>then\n </span>log <span>\"Pruning completed successfully\"</span>\n<span>else\n </span>log <span>\"WARNING: Pruning failed, but backup was successful\"</span>\n<span>fi</span>\n\n<span># Monthly repository check (on the 1st of each month)</span>\n<span>if</span> <span>[</span> <span>\"</span><span>$(</span><span>date</span> +%d<span>)</span><span>\"</span> <span>=</span> <span>\"01\"</span> <span>]</span><span>;</span> <span>then\n </span>log <span>\"Running monthly repository check...\"</span>\n <span>if </span>borg check <span>\"</span><span>$REPOSITORY</span><span>\"</span> 2>&1 | logger <span>-t</span> <span>\"borg-backup\"</span><span>;</span> <span>then\n </span>log <span>\"Repository check passed\"</span>\n <span>else\n </span>log <span>\"WARNING: Repository check failed\"</span>\n <span>fi\nfi\n\n</span>log <span>\"Backup completed successfully\"</span>\nlog <span>\"========================================\"</span>\n</code></pre></div></div>\n\n<p>Check the logs…</p>\n\n<div><div><pre><code>journalctl <span>-t</span> borg-backup\n</code></pre></div></div>",
9 "content_type": "html",
10 "author": {
11 "name": "Mark Elvers",
12 "email": "mark.elvers@tunbury.org",
13 "uri": null
14 },
15 "categories": [
16 "borg",
17 "tunbury.org"
18 ],
19 "source": "https://www.tunbury.org/atom.xml"
20}