pyrox.dev
1# Writing Tests {#sec-writing-nixos-tests}
2
3A NixOS test is a Nix expression that has the following structure:
4
5```nix
6import ./make-test-python.nix {
7
8 # Either the configuration of a single machine:
9 machine =
10 { config, pkgs, ... }:
11 { configuration…
12 };
13
14 # Or a set of machines:
15 nodes =
16 { machine1 =
17 { config, pkgs, ... }: { … };
18 machine2 =
19 { config, pkgs, ... }: { … };
20 …
21 };
22
23 testScript =
24 ''
25 Python code…
26 '';
27}
28```
29
30The attribute `testScript` is a bit of Python code that executes the
31test (described below). During the test, it will start one or more
32virtual machines, the configuration of which is described by the
33attribute `machine` (if you need only one machine in your test) or by
34the attribute `nodes` (if you need multiple machines). For instance,
35[`login.nix`](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/login.nix)
36only needs a single machine to test whether users can log in
37on the virtual console, whether device ownership is correctly maintained
38when switching between consoles, and so on. On the other hand,
39[`nfs/simple.nix`](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/nfs/simple.nix),
40which tests NFS client and server functionality in the
41Linux kernel (including whether locks are maintained across server
42crashes), requires three machines: a server and two clients.
43
44There are a few special NixOS configuration options for test VMs:
45
46`virtualisation.memorySize`
47
48: The memory of the VM in megabytes.
49
50`virtualisation.vlans`
51
52: The virtual networks to which the VM is connected. See
53 [`nat.nix`](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/nat.nix)
54 for an example.
55
56`virtualisation.writableStore`
57
58: By default, the Nix store in the VM is not writable. If you enable
59 this option, a writable union file system is mounted on top of the
60 Nix store to make it appear writable. This is necessary for tests
61 that run Nix operations that modify the store.
62
63For more options, see the module
64[`qemu-vm.nix`](https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/virtualisation/qemu-vm.nix).
65
66The test script is a sequence of Python statements that perform various
67actions, such as starting VMs, executing commands in the VMs, and so on.
68Each virtual machine is represented as an object stored in the variable
69`name` if this is also the identifier of the machine in the declarative
70config. If you didn\'t specify multiple machines using the `nodes`
71attribute, it is just `machine`. The following example starts the
72machine, waits until it has finished booting, then executes a command
73and checks that the output is more-or-less correct:
74
75```py
76machine.start()
77machine.wait_for_unit("default.target")
78if not "Linux" in machine.succeed("uname"):
79 raise Exception("Wrong OS")
80```
81
82The first line is actually unnecessary; machines are implicitly started
83when you first execute an action on them (such as `wait_for_unit` or
84`succeed`). If you have multiple machines, you can speed up the test by
85starting them in parallel:
86
87```py
88start_all()
89```
90
91The following methods are available on machine objects:
92
93`start`
94
95: Start the virtual machine. This method is asynchronous --- it does
96 not wait for the machine to finish booting.
97
98`shutdown`
99
100: Shut down the machine, waiting for the VM to exit.
101
102`crash`
103
104: Simulate a sudden power failure, by telling the VM to exit
105 immediately.
106
107`block`
108
109: Simulate unplugging the Ethernet cable that connects the machine to
110 the other machines.
111
112`unblock`
113
114: Undo the effect of `block`.
115
116`screenshot`
117
118: Take a picture of the display of the virtual machine, in PNG format.
119 The screenshot is linked from the HTML log.
120
121`get_screen_text_variants`
122
123: Return a list of different interpretations of what is currently
124 visible on the machine\'s screen using optical character
125 recognition. The number and order of the interpretations is not
126 specified and is subject to change, but if no exception is raised at
127 least one will be returned.
128
129 ::: {.note}
130 This requires passing `enableOCR` to the test attribute set.
131 :::
132
133`get_screen_text`
134
135: Return a textual representation of what is currently visible on the
136 machine\'s screen using optical character recognition.
137
138 ::: {.note}
139 This requires passing `enableOCR` to the test attribute set.
140 :::
141
142`send_monitor_command`
143
144: Send a command to the QEMU monitor. This is rarely used, but allows
145 doing stuff such as attaching virtual USB disks to a running
146 machine.
147
148`send_key`
149
150: Simulate pressing keys on the virtual keyboard, e.g.,
151 `send_key("ctrl-alt-delete")`.
152
153`send_chars`
154
155: Simulate typing a sequence of characters on the virtual keyboard,
156 e.g., `send_chars("foobar\n")` will type the string `foobar`
157 followed by the Enter key.
158
159`execute`
160
161: Execute a shell command, returning a list `(status, stdout)`.
162 If the command detaches, it must close stdout, as `execute` will wait
163 for this to consume all output reliably. This can be achieved by
164 redirecting stdout to stderr `>&2`, to `/dev/console`, `/dev/null` or
165 a file. Examples of detaching commands are `sleep 365d &`, where the
166 shell forks a new process that can write to stdout and `xclip -i`, where
167 the `xclip` command itself forks without closing stdout.
168 Takes an optional parameter `check_return` that defaults to `True`.
169 Setting this parameter to `False` will not check for the return code
170 and return -1 instead. This can be used for commands that shut down
171 the VM and would therefore break the pipe that would be used for
172 retrieving the return code.
173
174`succeed`
175
176: Execute a shell command, raising an exception if the exit status is
177 not zero, otherwise returning the standard output. Commands are run
178 with `set -euo pipefail` set:
179
180 - If several commands are separated by `;` and one fails, the
181 command as a whole will fail.
182
183 - For pipelines, the last non-zero exit status will be returned
184 (if there is one, zero will be returned otherwise).
185
186 - Dereferencing unset variables fail the command.
187
188 - It will wait for stdout to be closed. See `execute` for the
189 implications.
190
191`fail`
192
193: Like `succeed`, but raising an exception if the command returns a zero
194 status.
195
196`wait_until_succeeds`
197
198: Repeat a shell command with 1-second intervals until it succeeds.
199
200`wait_until_fails`
201
202: Repeat a shell command with 1-second intervals until it fails.
203
204`wait_for_unit`
205
206: Wait until the specified systemd unit has reached the "active"
207 state.
208
209`wait_for_file`
210
211: Wait until the specified file exists.
212
213`wait_for_open_port`
214
215: Wait until a process is listening on the given TCP port (on
216 `localhost`, at least).
217
218`wait_for_closed_port`
219
220: Wait until nobody is listening on the given TCP port.
221
222`wait_for_x`
223
224: Wait until the X11 server is accepting connections.
225
226`wait_for_text`
227
228: Wait until the supplied regular expressions matches the textual
229 contents of the screen by using optical character recognition (see
230 `get_screen_text` and `get_screen_text_variants`).
231
232 ::: {.note}
233 This requires passing `enableOCR` to the test attribute set.
234 :::
235
236`wait_for_console_text`
237
238: Wait until the supplied regular expressions match a line of the
239 serial console output. This method is useful when OCR is not
240 possibile or accurate enough.
241
242`wait_for_window`
243
244: Wait until an X11 window has appeared whose name matches the given
245 regular expression, e.g., `wait_for_window("Terminal")`.
246
247`copy_from_host`
248
249: Copies a file from host to machine, e.g.,
250 `copy_from_host("myfile", "/etc/my/important/file")`.
251
252 The first argument is the file on the host. The file needs to be
253 accessible while building the nix derivation. The second argument is
254 the location of the file on the machine.
255
256`systemctl`
257
258: Runs `systemctl` commands with optional support for
259 `systemctl --user`
260
261 ```py
262 machine.systemctl("list-jobs --no-pager") # runs `systemctl list-jobs --no-pager`
263 machine.systemctl("list-jobs --no-pager", "any-user") # spawns a shell for `any-user` and runs `systemctl --user list-jobs --no-pager`
264 ```
265
266`shell_interact`
267
268: Allows you to directly interact with the guest shell. This should
269 only be used during test development, not in production tests.
270 Killing the interactive session with `Ctrl-d` or `Ctrl-c` also ends
271 the guest session.
272
273To test user units declared by `systemd.user.services` the optional
274`user` argument can be used:
275
276```py
277machine.start()
278machine.wait_for_x()
279machine.wait_for_unit("xautolock.service", "x-session-user")
280```
281
282This applies to `systemctl`, `get_unit_info`, `wait_for_unit`,
283`start_job` and `stop_job`.
284
285For faster dev cycles it\'s also possible to disable the code-linters
286(this shouldn\'t be commited though):
287
288```nix
289import ./make-test-python.nix {
290 skipLint = true;
291 machine =
292 { config, pkgs, ... }:
293 { configuration…
294 };
295
296 testScript =
297 ''
298 Python code…
299 '';
300}
301```
302
303This will produce a Nix warning at evaluation time. To fully disable the
304linter, wrap the test script in comment directives to disable the Black
305linter directly (again, don\'t commit this within the Nixpkgs
306repository):
307
308```nix
309 testScript =
310 ''
311 # fmt: off
312 Python code…
313 # fmt: on
314 '';
315```