pyrox.dev
1<chapter xmlns="http://docbook.org/ns/docbook"
2 xmlns:xlink="http://www.w3.org/1999/xlink"
3 xml:id="chap-functions">
4
5<title>Functions reference</title>
6
7<para>
8 The nixpkgs repository has several utility functions to manipulate Nix expressions.
9</para>
10
11<section xml:id="sec-pkgs-overridePackages">
12 <title>pkgs.overridePackages</title>
13
14 <para>
15 This function inside the nixpkgs expression (<varname>pkgs</varname>)
16 can be used to override the set of packages itself.
17 </para>
18 <para>
19 Warning: this function is expensive and must not be used from within
20 the nixpkgs repository.
21 </para>
22 <para>
23 Example usage:
24
25 <programlisting>let
26 pkgs = import <nixpkgs> {};
27 newpkgs = pkgs.overridePackages (self: super: {
28 foo = super.foo.override { ... };
29 };
30in ...</programlisting>
31 </para>
32
33 <para>
34 The resulting <varname>newpkgs</varname> will have the new <varname>foo</varname>
35 expression, and all other expressions depending on <varname>foo</varname> will also
36 use the new <varname>foo</varname> expression.
37 </para>
38
39 <para>
40 The behavior of this function is similar to <link
41 linkend="sec-modify-via-packageOverrides">config.packageOverrides</link>.
42 </para>
43
44 <para>
45 The <varname>self</varname> parameter refers to the final package set with the
46 applied overrides. Using this parameter may lead to infinite recursion if not
47 used consciously.
48 </para>
49
50 <para>
51 The <varname>super</varname> parameter refers to the old package set.
52 It's equivalent to <varname>pkgs</varname> in the above example.
53 </para>
54
55</section>
56
57<section xml:id="sec-pkg-override">
58 <title><pkg>.override</title>
59
60 <para>
61 The function <varname>override</varname> is usually available for all the
62 derivations in the nixpkgs expression (<varname>pkgs</varname>).
63 </para>
64 <para>
65 It is used to override the arguments passed to a function.
66 </para>
67 <para>
68 Example usages:
69
70 <programlisting>pkgs.foo.override { arg1 = val1; arg2 = val2; ... }</programlisting>
71 <programlisting>pkgs.overridePackages (self: super: {
72 foo = super.foo.override { barSupport = true ; };
73})</programlisting>
74 <programlisting>mypkg = pkgs.callPackage ./mypkg.nix {
75 mydep = pkgs.mydep.override { ... };
76})</programlisting>
77 </para>
78
79 <para>
80 In the first example, <varname>pkgs.foo</varname> is the result of a function call
81 with some default arguments, usually a derivation.
82 Using <varname>pkgs.foo.override</varname> will call the same function with
83 the given new arguments.
84 </para>
85
86</section>
87
88<section xml:id="sec-pkg-overrideDerivation">
89 <title><pkg>.overrideDerivation</title>
90
91 <warning>
92 <para>Do not use this function in Nixpkgs as it evaluates a Derivation
93 before modifying it, which breaks package abstraction and removes
94 error-checking of function arguments. In addition, this
95 evaluation-per-function application incurs a performance penalty,
96 which can become a problem if many overrides are used.
97 It is only intended for ad-hoc customisation, such as in
98 <filename>~/.nixpkgs/config.nix</filename>.
99 </para>
100 </warning>
101
102 <para>
103 The function <varname>overrideDerivation</varname> creates a new derivation
104 based on an existing one by overriding the original's attributes with
105 the attribute set produced by the specified function.
106 This function is available on all
107 derivations defined using the <varname>makeOverridable</varname> function.
108 Most standard derivation-producing functions, such as
109 <varname>stdenv.mkDerivation</varname>, are defined using this
110 function, which means most packages in the nixpkgs expression,
111 <varname>pkgs</varname>, have this function.
112 </para>
113
114 <para>
115 Example usage:
116
117 <programlisting>mySed = pkgs.gnused.overrideDerivation (oldAttrs: {
118 name = "sed-4.2.2-pre";
119 src = fetchurl {
120 url = ftp://alpha.gnu.org/gnu/sed/sed-4.2.2-pre.tar.bz2;
121 sha256 = "11nq06d131y4wmf3drm0yk502d2xc6n5qy82cg88rb9nqd2lj41k";
122 };
123 patches = [];
124});</programlisting>
125 </para>
126
127 <para>
128 In the above example, the <varname>name</varname>, <varname>src</varname>,
129 and <varname>patches</varname> of the derivation will be overridden, while
130 all other attributes will be retained from the original derivation.
131 </para>
132
133 <para>
134 The argument <varname>oldAttrs</varname> is used to refer to the attribute set of
135 the original derivation.
136 </para>
137
138 <note>
139 <para>
140 A package's attributes are evaluated *before* being modified by
141 the <varname>overrideDerivation</varname> function.
142 For example, the <varname>name</varname> attribute reference
143 in <varname>url = "mirror://gnu/hello/${name}.tar.gz";</varname>
144 is filled-in *before* the <varname>overrideDerivation</varname> function
145 modifies the attribute set. This means that overriding the
146 <varname>name</varname> attribute, in this example, *will not* change the
147 value of the <varname>url</varname> attribute. Instead, we need to override
148 both the <varname>name</varname> *and* <varname>url</varname> attributes.
149 </para>
150 </note>
151
152</section>
153
154<section xml:id="sec-lib-makeOverridable">
155 <title>lib.makeOverridable</title>
156
157 <para>
158 The function <varname>lib.makeOverridable</varname> is used to make the result
159 of a function easily customizable. This utility only makes sense for functions
160 that accept an argument set and return an attribute set.
161 </para>
162
163 <para>
164 Example usage:
165
166 <programlisting>f = { a, b }: { result = a+b; }
167c = lib.makeOverridable f { a = 1; b = 2; }</programlisting>
168
169 </para>
170
171 <para>
172 The variable <varname>c</varname> is the value of the <varname>f</varname> function
173 applied with some default arguments. Hence the value of <varname>c.result</varname>
174 is <literal>3</literal>, in this example.
175 </para>
176
177 <para>
178 The variable <varname>c</varname> however also has some additional functions, like
179 <link linkend="sec-pkg-override">c.override</link> which can be used to
180 override the default arguments. In this example the value of
181 <varname>(c.override { a = 4; }).result</varname> is 6.
182 </para>
183
184</section>
185
186
187<section xml:id="sec-fhs-environments">
188 <title>buildFHSUserEnv</title>
189
190 <para>
191 <function>buildFHSUserEnv</function> provides a way to build and run
192 FHS-compatible lightweight sandboxes. It creates an isolated root with
193 bound <filename>/nix/store</filename>, so its footprint in terms of disk
194 space needed is quite small. This allows one to run software which is hard or
195 unfeasible to patch for NixOS -- 3rd-party source trees with FHS assumptions,
196 games distributed as tarballs, software with integrity checking and/or external
197 self-updated binaries. It uses Linux namespaces feature to create
198 temporary lightweight environments which are destroyed after all child
199 processes exit, without root user rights requirement. Accepted arguments are:
200 </para>
201
202 <variablelist>
203 <varlistentry>
204 <term><literal>name</literal></term>
205
206 <listitem><para>Environment name.</para></listitem>
207 </varlistentry>
208
209 <varlistentry>
210 <term><literal>targetPkgs</literal></term>
211
212 <listitem><para>Packages to be installed for the main host's architecture
213 (i.e. x86_64 on x86_64 installations). Along with libraries binaries are also
214 installed.</para></listitem>
215 </varlistentry>
216
217 <varlistentry>
218 <term><literal>multiPkgs</literal></term>
219
220 <listitem><para>Packages to be installed for all architectures supported by
221 a host (i.e. i686 and x86_64 on x86_64 installations). Only libraries are
222 installed by default.</para></listitem>
223 </varlistentry>
224
225 <varlistentry>
226 <term><literal>extraBuildCommands</literal></term>
227
228 <listitem><para>Additional commands to be executed for finalizing the
229 directory structure.</para></listitem>
230 </varlistentry>
231
232 <varlistentry>
233 <term><literal>extraBuildCommandsMulti</literal></term>
234
235 <listitem><para>Like <literal>extraBuildCommands</literal>, but
236 executed only on multilib architectures.</para></listitem>
237 </varlistentry>
238
239 <varlistentry>
240 <term><literal>extraOutputsToInstall</literal></term>
241
242 <listitem><para>Additional derivation outputs to be linked for both
243 target and multi-architecture packages.</para></listitem>
244 </varlistentry>
245
246 <varlistentry>
247 <term><literal>extraInstallCommands</literal></term>
248
249 <listitem><para>Additional commands to be executed for finalizing the
250 derivation with runner script.</para></listitem>
251 </varlistentry>
252
253 <varlistentry>
254 <term><literal>runScript</literal></term>
255
256 <listitem><para>A command that would be executed inside the sandbox and
257 passed all the command line arguments. It defaults to
258 <literal>bash</literal>.</para></listitem>
259 </varlistentry>
260 </variablelist>
261
262 <para>
263 One can create a simple environment using a <literal>shell.nix</literal>
264 like that:
265 </para>
266
267<programlisting><![CDATA[
268{ pkgs ? import <nixpkgs> {} }:
269
270(pkgs.buildFHSUserEnv {
271 name = "simple-x11-env";
272 targetPkgs = pkgs: (with pkgs;
273 [ udev
274 alsaLib
275 ]) ++ (with pkgs.xorg;
276 [ libX11
277 libXcursor
278 libXrandr
279 ]);
280 multiPkgs = pkgs: (with pkgs;
281 [ udev
282 alsaLib
283 ]);
284 runScript = "bash";
285}).env
286]]></programlisting>
287
288 <para>
289 Running <literal>nix-shell</literal> would then drop you into a shell with
290 these libraries and binaries available. You can use this to run
291 closed-source applications which expect FHS structure without hassles:
292 simply change <literal>runScript</literal> to the application path,
293 e.g. <filename>./bin/start.sh</filename> -- relative paths are supported.
294 </para>
295</section>
296
297<section xml:id="sec-pkgs-dockerTools">
298 <title>pkgs.dockerTools</title>
299
300 <para>
301 <varname>pkgs.dockerTools</varname> is a set of functions for creating and
302 manipulating Docker images according to the
303 <link xlink:href="https://github.com/docker/docker/blob/master/image/spec/v1.md#docker-image-specification-v100">
304 Docker Image Specification v1.0.0
305 </link>. Docker itself is not used to perform any of the operations done by these
306 functions.
307 </para>
308
309 <warning>
310 <para>
311 The <varname>dockerTools</varname> API is unstable and may be subject to
312 backwards-incompatible changes in the future.
313 </para>
314 </warning>
315
316 <section xml:id="ssec-pkgs-dockerTools-buildImage">
317 <title>buildImage</title>
318
319 <para>
320 This function is analogous to the <command>docker build</command> command,
321 in that can used to build a Docker-compatible repository tarball containing
322 a single image with one or multiple layers. As such, the result
323 is suitable for being loaded in Docker with <command>docker load</command>.
324 </para>
325
326 <para>
327 The parameters of <varname>buildImage</varname> with relative example values are
328 described below:
329 </para>
330
331 <example xml:id='ex-dockerTools-buildImage'><title>Docker build</title>
332 <programlisting>
333 buildImage {
334 name = "redis"; <co xml:id='ex-dockerTools-buildImage-1' />
335 tag = "latest"; <co xml:id='ex-dockerTools-buildImage-2' />
336
337 fromImage = someBaseImage; <co xml:id='ex-dockerTools-buildImage-3' />
338 fromImageName = null; <co xml:id='ex-dockerTools-buildImage-4' />
339 fromImageTag = "latest"; <co xml:id='ex-dockerTools-buildImage-5' />
340
341 contents = pkgs.redis; <co xml:id='ex-dockerTools-buildImage-6' />
342 runAsRoot = '' <co xml:id='ex-dockerTools-buildImage-runAsRoot' />
343 #!${stdenv.shell}
344 mkdir -p /data
345 '';
346
347 config = { <co xml:id='ex-dockerTools-buildImage-8' />
348 Cmd = [ "/bin/redis-server" ];
349 WorkingDir = "/data";
350 Volumes = {
351 "/data" = {};
352 };
353 };
354 }
355 </programlisting>
356 </example>
357
358 <para>The above example will build a Docker image <literal>redis/latest</literal>
359 from the given base image. Loading and running this image in Docker results in
360 <literal>redis-server</literal> being started automatically.
361 </para>
362
363 <calloutlist>
364 <callout arearefs='ex-dockerTools-buildImage-1'>
365 <para>
366 <varname>name</varname> specifies the name of the resulting image.
367 This is the only required argument for <varname>buildImage</varname>.
368 </para>
369 </callout>
370
371 <callout arearefs='ex-dockerTools-buildImage-2'>
372 <para>
373 <varname>tag</varname> specifies the tag of the resulting image.
374 By default it's <literal>latest</literal>.
375 </para>
376 </callout>
377
378 <callout arearefs='ex-dockerTools-buildImage-3'>
379 <para>
380 <varname>fromImage</varname> is the repository tarball containing the base image.
381 It must be a valid Docker image, such as exported by <command>docker save</command>.
382 By default it's <literal>null</literal>, which can be seen as equivalent
383 to <literal>FROM scratch</literal> of a <filename>Dockerfile</filename>.
384 </para>
385 </callout>
386
387 <callout arearefs='ex-dockerTools-buildImage-4'>
388 <para>
389 <varname>fromImageName</varname> can be used to further specify
390 the base image within the repository, in case it contains multiple images.
391 By default it's <literal>null</literal>, in which case
392 <varname>buildImage</varname> will peek the first image available
393 in the repository.
394 </para>
395 </callout>
396
397 <callout arearefs='ex-dockerTools-buildImage-5'>
398 <para>
399 <varname>fromImageTag</varname> can be used to further specify the tag
400 of the base image within the repository, in case an image contains multiple tags.
401 By default it's <literal>null</literal>, in which case
402 <varname>buildImage</varname> will peek the first tag available for the base image.
403 </para>
404 </callout>
405
406 <callout arearefs='ex-dockerTools-buildImage-6'>
407 <para>
408 <varname>contents</varname> is a derivation that will be copied in the new
409 layer of the resulting image. This can be similarly seen as
410 <command>ADD contents/ /</command> in a <filename>Dockerfile</filename>.
411 By default it's <literal>null</literal>.
412 </para>
413 </callout>
414
415 <callout arearefs='ex-dockerTools-buildImage-runAsRoot'>
416 <para>
417 <varname>runAsRoot</varname> is a bash script that will run as root
418 in an environment that overlays the existing layers of the base image with
419 the new resulting layer, including the previously copied
420 <varname>contents</varname> derivation.
421 This can be similarly seen as
422 <command>RUN ...</command> in a <filename>Dockerfile</filename>.
423
424 <note>
425 <para>
426 Using this parameter requires the <literal>kvm</literal>
427 device to be available.
428 </para>
429 </note>
430 </para>
431 </callout>
432
433 <callout arearefs='ex-dockerTools-buildImage-8'>
434 <para>
435 <varname>config</varname> is used to specify the configuration of the
436 containers that will be started off the built image in Docker.
437 The available options are listed in the
438 <link xlink:href="https://github.com/docker/docker/blob/master/image/spec/v1.md#container-runconfig-field-descriptions">
439 Docker Image Specification v1.0.0
440 </link>.
441 </para>
442 </callout>
443
444 </calloutlist>
445
446 <para>
447 After the new layer has been created, its closure
448 (to which <varname>contents</varname>, <varname>config</varname> and
449 <varname>runAsRoot</varname> contribute) will be copied in the layer itself.
450 Only new dependencies that are not already in the existing layers will be copied.
451 </para>
452
453 <para>
454 At the end of the process, only one new single layer will be produced and
455 added to the resulting image.
456 </para>
457
458 <para>
459 The resulting repository will only list the single image
460 <varname>image/tag</varname>. In the case of <xref linkend='ex-dockerTools-buildImage'/>
461 it would be <varname>redis/latest</varname>.
462 </para>
463
464 <para>
465 It is possible to inspect the arguments with which an image was built
466 using its <varname>buildArgs</varname> attribute.
467 </para>
468
469 </section>
470
471 <section xml:id="ssec-pkgs-dockerTools-fetchFromRegistry">
472 <title>pullImage</title>
473
474 <para>
475 This function is analogous to the <command>docker pull</command> command,
476 in that can be used to fetch a Docker image from a Docker registry.
477 Currently only registry <literal>v1</literal> is supported.
478 By default <link xlink:href="https://hub.docker.com/">Docker Hub</link>
479 is used to pull images.
480 </para>
481
482 <para>
483 Its parameters are described in the example below:
484 </para>
485
486 <example xml:id='ex-dockerTools-pullImage'><title>Docker pull</title>
487 <programlisting>
488 pullImage {
489 imageName = "debian"; <co xml:id='ex-dockerTools-pullImage-1' />
490 imageTag = "jessie"; <co xml:id='ex-dockerTools-pullImage-2' />
491 imageId = null; <co xml:id='ex-dockerTools-pullImage-3' />
492 sha256 = "1bhw5hkz6chrnrih0ymjbmn69hyfriza2lr550xyvpdrnbzr4gk2"; <co xml:id='ex-dockerTools-pullImage-4' />
493
494 indexUrl = "https://index.docker.io"; <co xml:id='ex-dockerTools-pullImage-5' />
495 registryVersion = "v1";
496 }
497 </programlisting>
498 </example>
499
500 <calloutlist>
501 <callout arearefs='ex-dockerTools-pullImage-1'>
502 <para>
503 <varname>imageName</varname> specifies the name of the image to be downloaded,
504 which can also include the registry namespace (e.g. <literal>library/debian</literal>).
505 This argument is required.
506 </para>
507 </callout>
508
509 <callout arearefs='ex-dockerTools-pullImage-2'>
510 <para>
511 <varname>imageTag</varname> specifies the tag of the image to be downloaded.
512 By default it's <literal>latest</literal>.
513 </para>
514 </callout>
515
516 <callout arearefs='ex-dockerTools-pullImage-3'>
517 <para>
518 <varname>imageId</varname>, if specified this exact image will be fetched, instead
519 of <varname>imageName/imageTag</varname>. However, the resulting repository
520 will still be named <varname>imageName/imageTag</varname>.
521 By default it's <literal>null</literal>.
522 </para>
523 </callout>
524
525 <callout arearefs='ex-dockerTools-pullImage-4'>
526 <para>
527 <varname>sha256</varname> is the checksum of the whole fetched image.
528 This argument is required.
529 </para>
530
531 <note>
532 <para>The checksum is computed on the unpacked directory, not on the final tarball.</para>
533 </note>
534
535 </callout>
536
537 <callout arearefs='ex-dockerTools-pullImage-5'>
538 <para>
539 In the above example the default values are shown for the variables
540 <varname>indexUrl</varname> and <varname>registryVersion</varname>.
541 Hence by default the Docker.io registry is used to pull the images.
542 </para>
543 </callout>
544 </calloutlist>
545
546 </section>
547
548 <section xml:id="ssec-pkgs-dockerTools-exportImage">
549 <title>exportImage</title>
550
551 <para>
552 This function is analogous to the <command>docker export</command> command,
553 in that can used to flatten a Docker image that contains multiple layers.
554 It is in fact the result of the merge of all the layers of the image.
555 As such, the result is suitable for being imported in Docker
556 with <command>docker import</command>.
557 </para>
558
559 <note>
560 <para>
561 Using this function requires the <literal>kvm</literal>
562 device to be available.
563 </para>
564 </note>
565
566 <para>
567 The parameters of <varname>exportImage</varname> are the following:
568 </para>
569
570 <example xml:id='ex-dockerTools-exportImage'><title>Docker export</title>
571 <programlisting>
572 exportImage {
573 fromImage = someLayeredImage;
574 fromImageName = null;
575 fromImageTag = null;
576
577 name = someLayeredImage.name;
578 }
579 </programlisting>
580 </example>
581
582 <para>
583 The parameters relative to the base image have the same synopsis as
584 described in <xref linkend='ssec-pkgs-dockerTools-buildImage'/>, except that
585 <varname>fromImage</varname> is the only required argument in this case.
586 </para>
587
588 <para>
589 The <varname>name</varname> argument is the name of the derivation output,
590 which defaults to <varname>fromImage.name</varname>.
591 </para>
592 </section>
593
594 <section xml:id="ssec-pkgs-dockerTools-shadowSetup">
595 <title>shadowSetup</title>
596
597 <para>
598 This constant string is a helper for setting up the base files for managing
599 users and groups, only if such files don't exist already.
600 It is suitable for being used in a
601 <varname>runAsRoot</varname> <xref linkend='ex-dockerTools-buildImage-runAsRoot'/> script for cases like
602 in the example below:
603 </para>
604
605 <example xml:id='ex-dockerTools-shadowSetup'><title>Shadow base files</title>
606 <programlisting>
607 buildImage {
608 name = "shadow-basic";
609
610 runAsRoot = ''
611 #!${stdenv.shell}
612 ${shadowSetup}
613 groupadd -r redis
614 useradd -r -g redis redis
615 mkdir /data
616 chown redis:redis /data
617 '';
618 }
619 </programlisting>
620 </example>
621
622 <para>
623 Creating base files like <literal>/etc/passwd</literal> or
624 <literal>/etc/login.defs</literal> are necessary for shadow-utils to
625 manipulate users and groups.
626 </para>
627
628 </section>
629
630</section>
631
632</chapter>