···
1
-
<chapter xmlns="http://docbook.org/ns/docbook"
2
-
xmlns:xlink="http://www.w3.org/1999/xlink"
3
-
xmlns:xi="http://www.w3.org/2001/XInclude"
5
-
xml:id="module-services-garage">
6
-
<title>Garage</title>
8
-
<link xlink:href="https://garagehq.deuxfleurs.fr/">Garage</link>
9
-
is an open-source, self-hostable S3 store, simpler than MinIO, for geodistributed stores.
10
-
The server setup can be automated using
11
-
<link linkend="opt-services.garage.enable">services.garage</link>. A
12
-
client configured to your local Garage instance is available in
13
-
the global environment as <literal>garage-manage</literal>.
16
-
The current default by NixOS is <literal>garage_0_8</literal> which is also the latest
17
-
major version available.
19
-
<section xml:id="module-services-garage-upgrade-scenarios">
20
-
<title>General considerations on upgrades</title>
1
+
<chapter xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xml:id="module-services-garage">
2
+
<title>Garage</title>
23
-
Garage provides a cookbook documentation on how to upgrade:
24
-
<link xlink:href="https://garagehq.deuxfleurs.fr/documentation/cookbook/upgrading/">https://garagehq.deuxfleurs.fr/documentation/cookbook/upgrading/</link>
4
+
<link xlink:href="https://garagehq.deuxfleurs.fr/">Garage</link> is
5
+
an open-source, self-hostable S3 store, simpler than MinIO, for
6
+
geodistributed stores. The server setup can be automated using
7
+
<link linkend="opt-services.garage.enable">services.garage</link>. A
8
+
client configured to your local Garage instance is available in the
9
+
global environment as <literal>garage-manage</literal>.
28
-
<para>Garage has two types of upgrades: patch-level upgrades and minor/major version upgrades.</para>
30
-
<para>In all cases, you should read the changelog and ideally test the upgrade on a staging cluster.</para>
32
-
<para>Checking the health of your cluster can be achieved using <literal>garage-manage repair</literal>.</para>
37
-
<para>Until 1.0 is released, patch-level upgrades are considered as minor version upgrades.
38
-
Minor version upgrades are considered as major version upgrades.
39
-
i.e. 0.6 to 0.7 is a major version upgrade.</para>
45
-
<title>Straightforward upgrades (patch-level upgrades)</title>
12
+
The current default by NixOS is <literal>garage_0_8</literal> which
13
+
is also the latest major version available.
15
+
<section xml:id="module-services-garage-upgrade-scenarios">
16
+
<title>General considerations on upgrades</title>
18
+
Garage provides a cookbook documentation on how to upgrade:
19
+
<link xlink:href="https://garagehq.deuxfleurs.fr/documentation/cookbook/upgrading/" role="uri">https://garagehq.deuxfleurs.fr/documentation/cookbook/upgrading/</link>
23
+
Garage has two types of upgrades: patch-level upgrades and
24
+
minor/major version upgrades.
27
+
In all cases, you should read the changelog and ideally test the
28
+
upgrade on a staging cluster.
31
+
Checking the health of your cluster can be achieved using
32
+
<literal>garage-manage repair</literal>.
37
+
Until 1.0 is released, patch-level upgrades are considered as
38
+
minor version upgrades. Minor version upgrades are considered as
39
+
major version upgrades. i.e. 0.6 to 0.7 is a major version
43
+
<itemizedlist spacing="compact">
46
+
<emphasis role="strong">Straightforward upgrades (patch-level
47
+
upgrades).</emphasis> Upgrades must be performed one by one,
48
+
i.e. for each node, stop it, upgrade it : change
49
+
<link linkend="opt-system.stateVersion">stateVersion</link> or
50
+
<link linkend="opt-services.garage.package">services.garage.package</link>,
51
+
restart it if it was not already by switching.
56
+
<emphasis role="strong">Multiple version upgrades.</emphasis>
57
+
Garage do not provide any guarantee on moving more than one
58
+
major-version forward. E.g., if you're on
59
+
<literal>0.7</literal>, you cannot upgrade to
60
+
<literal>0.9</literal>. You need to upgrade to
61
+
<literal>0.8</literal> first. As long as
62
+
<link linkend="opt-system.stateVersion">stateVersion</link> is
63
+
declared properly, this is enforced automatically. The module
64
+
will issue a warning to remind the user to upgrade to latest
65
+
Garage <emphasis>after</emphasis> that deploy.
70
+
<section xml:id="module-services-garage-advanced-upgrades">
71
+
<title>Advanced upgrades (minor/major version upgrades)</title>
73
+
Here are some baseline instructions to handle advanced upgrades in
74
+
Garage, when in doubt, please refer to upstream instructions.
76
+
<itemizedlist spacing="compact">
79
+
Disable API and web access to Garage.
85
+
<literal>garage-manage repair --all-nodes --yes tables</literal>
87
+
<literal>garage-manage repair --all-nodes --yes blocks</literal>.
92
+
Verify the resulting logs and check that data is synced
93
+
properly between all nodes. If you have time, do additional
94
+
checks (<literal>scrub</literal>,
95
+
<literal>block_refs</literal>, etc.).
100
+
Check if queues are empty by
101
+
<literal>garage-manage stats</literal> or through monitoring
107
+
Run <literal>systemctl stop garage</literal> to stop the
108
+
actual Garage version.
113
+
Backup the metadata folder of ALL your nodes, e.g. for a
114
+
metadata directory (the default one) in
115
+
<literal>/var/lib/garage/meta</literal>, you can run
116
+
<literal>pushd /var/lib/garage; tar -acf meta-v0.7.tar.zst meta/; popd</literal>.
121
+
Run the offline migration:
122
+
<literal>nix-shell -p garage_0_8 --run "garage offline-repair --yes"</literal>,
123
+
this can take some time depending on how many objects are
124
+
stored in your cluster.
129
+
Bump Garage version in your NixOS configuration, either by
131
+
<link linkend="opt-system.stateVersion">stateVersion</link> or
133
+
<link linkend="opt-services.garage.package">services.garage.package</link>,
134
+
this should restart Garage automatically.
140
+
<literal>garage-manage repair --all-nodes --yes tables</literal>
142
+
<literal>garage-manage repair --all-nodes --yes blocks</literal>.
147
+
Wait for a full table sync to run.
47
-
Upgrades must be performed one by one, i.e. for each node, stop it, upgrade it : change <link linkend="opt-system.stateVersion">stateVersion</link> or <link linkend="opt-services.garage.package">services.garage.package</link>, restart it if it was not already by switching.
152
+
Your upgraded cluster should be in a working state, re-enable API
54
-
<title>Multiple version upgrades</title>
156
+
<section xml:id="module-services-garage-maintainer-info">
157
+
<title>Maintainer information</title>
56
-
Garage do not provide any guarantee on moving more than one major-version forward.
57
-
E.g., if you're on <literal>0.7</literal>, you cannot upgrade to <literal>0.9</literal>.
58
-
You need to upgrade to <literal>0.8</literal> first.
60
-
As long as <link linkend="opt-system.stateVersion">stateVersion</link> is declared properly,
61
-
this is enforced automatically. The module will issue a warning to remind the user to upgrade to latest
62
-
Garage <emphasis>after</emphasis> that deploy.
69
-
<section xml:id="module-services-garage-advanced-upgrades">
70
-
<title>Advanced upgrades (minor/major version upgrades)</title>
71
-
<para>Here are some baseline instructions to handle advanced upgrades in Garage, when in doubt, please refer to upstream instructions.</para>
74
-
<listitem><para>Disable API and web access to Garage.</para></listitem>
75
-
<listitem><para>Perform <literal>garage-manage repair --all-nodes --yes tables</literal> and <literal>garage-manage repair --all-nodes --yes blocks</literal>.</para></listitem>
76
-
<listitem><para>Verify the resulting logs and check that data is synced properly between all nodes.
77
-
If you have time, do additional checks (<literal>scrub</literal>, <literal>block_refs</literal>, etc.).</para></listitem>
78
-
<listitem><para>Check if queues are empty by <literal>garage-manage stats</literal> or through monitoring tools.</para></listitem>
79
-
<listitem><para>Run <literal>systemctl stop garage</literal> to stop the actual Garage version.</para></listitem>
80
-
<listitem><para>Backup the metadata folder of ALL your nodes, e.g. for a metadata directory (the default one) in <literal>/var/lib/garage/meta</literal>,
81
-
you can run <literal>pushd /var/lib/garage; tar -acf meta-v0.7.tar.zst meta/; popd</literal>.</para></listitem>
82
-
<listitem><para>Run the offline migration: <literal>nix-shell -p garage_0_8 --run "garage offline-repair --yes"</literal>, this can take some time depending on how many objects are stored in your cluster.</para></listitem>
83
-
<listitem><para>Bump Garage version in your NixOS configuration, either by changing <link linkend="opt-system.stateVersion">stateVersion</link> or bumping <link linkend="opt-services.garage.package">services.garage.package</link>, this should restart Garage automatically.</para></listitem>
84
-
<listitem><para>Perform <literal>garage-manage repair --all-nodes --yes tables</literal> and <literal>garage-manage repair --all-nodes --yes blocks</literal>.</para></listitem>
85
-
<listitem><para>Wait for a full table sync to run.</para></listitem>
89
-
Your upgraded cluster should be in a working state, re-enable API and web access.
93
-
<section xml:id="module-services-garage-maintainer-info">
94
-
<title>Maintainer information</title>
97
-
As stated in the previous paragraph, we must provide a clean upgrade-path for Garage
98
-
since it cannot move more than one major version forward on a single upgrade. This chapter
99
-
adds some notes how Garage updates should be rolled out in the future.
101
-
This is inspired from how Nextcloud does it.
105
-
While patch-level updates are no problem and can be done directly in the
106
-
package-expression (and should be backported to supported stable branches after that),
107
-
major-releases should be added in a new attribute (e.g. Garage <literal>v0.8.0</literal>
108
-
should be available in <literal>nixpkgs</literal> as <literal>pkgs.garage_0_8_0</literal>).
109
-
To provide simple upgrade paths it's generally useful to backport those as well to stable
110
-
branches. As long as the package-default isn't altered, this won't break existing setups.
111
-
After that, the versioning-warning in the <literal>garage</literal>-module should be
112
-
updated to make sure that the
113
-
<link linkend="opt-services.garage.package">package</link>-option selects the latest version
118
-
If major-releases will be abandoned by upstream, we should check first if those are needed
119
-
in NixOS for a safe upgrade-path before removing those. In that case we shold keep those
120
-
packages, but mark them as insecure in an expression like this (in
121
-
<literal><nixpkgs/pkgs/tools/filesystem/garage/default.nix></literal>):
159
+
As stated in the previous paragraph, we must provide a clean
160
+
upgrade-path for Garage since it cannot move more than one major
161
+
version forward on a single upgrade. This chapter adds some notes
162
+
how Garage updates should be rolled out in the future. This is
163
+
inspired from how Nextcloud does it.
166
+
While patch-level updates are no problem and can be done directly
167
+
in the package-expression (and should be backported to supported
168
+
stable branches after that), major-releases should be added in a
169
+
new attribute (e.g. Garage <literal>v0.8.0</literal> should be
170
+
available in <literal>nixpkgs</literal> as
171
+
<literal>pkgs.garage_0_8_0</literal>). To provide simple upgrade
172
+
paths it's generally useful to backport those as well to stable
173
+
branches. As long as the package-default isn't altered, this won't
174
+
break existing setups. After that, the versioning-warning in the
175
+
<literal>garage</literal>-module should be updated to make sure
177
+
<link linkend="opt-services.garage.package">package</link>-option
178
+
selects the latest version on fresh setups.
181
+
If major-releases will be abandoned by upstream, we should check
182
+
first if those are needed in NixOS for a safe upgrade-path before
183
+
removing those. In that case we shold keep those packages, but
184
+
mark them as insecure in an expression like this (in
185
+
<literal><nixpkgs/pkgs/tools/filesystem/garage/default.nix></literal>):
127
-
sha256 = "0000000000000000000000000000000000000000000000000000";
191
+
version = "0.7.3";
192
+
sha256 = "0000000000000000000000000000000000000000000000000000";
135
-
Ideally we should make sure that it's possible to jump two NixOS versions forward:
136
-
i.e. the warnings and the logic in the module should guard a user to upgrade from a
137
-
Garage on e.g. 22.11 to a Garage on 23.11.
198
+
Ideally we should make sure that it's possible to jump two NixOS
199
+
versions forward: i.e. the warnings and the logic in the module
200
+
should guard a user to upgrade from a Garage on e.g. 22.11 to a