doc/stdenv.xml at 18.03-beta · pyrox.dev/nixpkgs

pyrox.dev / nixpkgs
lol
nixpkgs / doc / stdenv.xml
at 18.03-beta 88 kB view raw
   1<chapter xmlns="http://docbook.org/ns/docbook"
   2         xmlns:xlink="http://www.w3.org/1999/xlink"
   3         xml:id="chap-stdenv">
   4
   5<title>The Standard Environment</title>
   6
   7
   8<para>The standard build environment in the Nix Packages collection
   9provides an environment for building Unix packages that does a lot of
  10common build tasks automatically.  In fact, for Unix packages that use
  11the standard <literal>./configure; make; make install</literal> build
  12interface, you don’t need to write a build script at all; the standard
  13environment does everything automatically.  If
  14<literal>stdenv</literal> doesn’t do what you need automatically, you
  15can easily customise or override the various build phases.</para>
  16
  17
  18<section xml:id="sec-using-stdenv"><title>Using
  19<literal>stdenv</literal></title>
  20
  21<para>To build a package with the standard environment, you use the
  22function <varname>stdenv.mkDerivation</varname>, instead of the
  23primitive built-in function <varname>derivation</varname>, e.g.
  24
  25<programlisting>
  26stdenv.mkDerivation {
  27  name = "libfoo-1.2.3";
  28  src = fetchurl {
  29    url = http://example.org/libfoo-1.2.3.tar.bz2;
  30    sha256 = "0x2g1jqygyr5wiwg4ma1nd7w4ydpy82z9gkcv8vh2v8dn3y58v5m";
  31  };
  32}</programlisting>
  33
  34(<varname>stdenv</varname> needs to be in scope, so if you write this
  35in a separate Nix expression from
  36<filename>pkgs/all-packages.nix</filename>, you need to pass it as a
  37function argument.)  Specifying a <varname>name</varname> and a
  38<varname>src</varname> is the absolute minimum you need to do.  Many
  39packages have dependencies that are not provided in the standard
  40environment.  It’s usually sufficient to specify those dependencies in
  41the <varname>buildInputs</varname> attribute:
  42
  43<programlisting>
  44stdenv.mkDerivation {
  45  name = "libfoo-1.2.3";
  46  ...
  47  buildInputs = [libbar perl ncurses];
  48}</programlisting>
  49
  50This attribute ensures that the <filename>bin</filename>
  51subdirectories of these packages appear in the <envar>PATH</envar>
  52environment variable during the build, that their
  53<filename>include</filename> subdirectories are searched by the C
  54compiler, and so on.  (See <xref linkend="ssec-setup-hooks"/> for
  55details.)</para>
  56
  57<para>Often it is necessary to override or modify some aspect of the
  58build.  To make this easier, the standard environment breaks the
  59package build into a number of <emphasis>phases</emphasis>, all of
  60which can be overridden or modified individually: unpacking the
  61sources, applying patches, configuring, building, and installing.
  62(There are some others; see <xref linkend="sec-stdenv-phases"/>.)
  63For instance, a package that doesn’t supply a makefile but instead has
  64to be compiled “manually” could be handled like this:
  65
  66<programlisting>
  67stdenv.mkDerivation {
  68  name = "fnord-4.5";
  69  ...
  70  buildPhase = ''
  71    gcc foo.c -o foo
  72  '';
  73  installPhase = ''
  74    mkdir -p $out/bin
  75    cp foo $out/bin
  76  '';
  77}</programlisting>
  78
  79(Note the use of <literal>''</literal>-style string literals, which
  80are very convenient for large multi-line script fragments because they
  81don’t need escaping of <literal>"</literal> and <literal>\</literal>,
  82and because indentation is intelligently removed.)</para>
  83
  84<para>There are many other attributes to customise the build.  These
  85are listed in <xref linkend="ssec-stdenv-attributes"/>.</para>
  86
  87<para>While the standard environment provides a generic builder, you
  88can still supply your own build script:
  89
  90<programlisting>
  91stdenv.mkDerivation {
  92  name = "libfoo-1.2.3";
  93  ...
  94  builder = ./builder.sh;
  95}</programlisting>
  96
  97where the builder can do anything it wants, but typically starts with
  98
  99<programlisting>
 100source $stdenv/setup
 101</programlisting>
 102
 103to let <literal>stdenv</literal> set up the environment (e.g., process
 104the <varname>buildInputs</varname>).  If you want, you can still use
 105<literal>stdenv</literal>’s generic builder:
 106
 107<programlisting>
 108source $stdenv/setup
 109
 110buildPhase() {
 111  echo "... this is my custom build phase ..."
 112  gcc foo.c -o foo
 113}
 114
 115installPhase() {
 116  mkdir -p $out/bin
 117  cp foo $out/bin
 118}
 119
 120genericBuild
 121</programlisting>
 122
 123</para>
 124
 125</section>
 126
 127
 128<section xml:id="sec-tools-of-stdenv"><title>Tools provided by
 129<literal>stdenv</literal></title>
 130
 131<para>The standard environment provides the following packages:
 132
 133<itemizedlist>
 134
 135  <listitem><para>The GNU C Compiler, configured with C and C++
 136  support.</para></listitem>
 137
 138  <listitem><para>GNU coreutils (contains a few dozen standard Unix
 139  commands).</para></listitem>
 140
 141  <listitem><para>GNU findutils (contains
 142  <command>find</command>).</para></listitem>
 143
 144  <listitem><para>GNU diffutils (contains <command>diff</command>,
 145  <command>cmp</command>).</para></listitem>
 146
 147  <listitem><para>GNU <command>sed</command>.</para></listitem>
 148
 149  <listitem><para>GNU <command>grep</command>.</para></listitem>
 150
 151  <listitem><para>GNU <command>awk</command>.</para></listitem>
 152
 153  <listitem><para>GNU <command>tar</command>.</para></listitem>
 154
 155  <listitem><para><command>gzip</command>, <command>bzip2</command>
 156  and <command>xz</command>.</para></listitem>
 157
 158  <listitem><para>GNU Make.  It has been patched to provide
 159  <quote>nested</quote> output that can be fed into the
 160  <command>nix-log2xml</command> command and
 161  <command>log2html</command> stylesheet to create a structured,
 162  readable output of the build steps performed by
 163  Make.</para></listitem>
 164
 165  <listitem><para>Bash.  This is the shell used for all builders in
 166  the Nix Packages collection.  Not using <command>/bin/sh</command>
 167  removes a large source of portability problems.</para></listitem>
 168
 169  <listitem><para>The <command>patch</command>
 170  command.</para></listitem>
 171
 172</itemizedlist>
 173
 174</para>
 175
 176<para>On Linux, <literal>stdenv</literal> also includes the
 177<command>patchelf</command> utility.</para>
 178
 179</section>
 180
 181
 182<section xml:id="ssec-stdenv-dependencies"><title>Specifying dependencies</title>
 183
 184<para>
 185  As described in the Nix manual, almost any <filename>*.drv</filename> store path in a derivation's attribute set will induce a dependency on that derivation.
 186  <varname>mkDerivation</varname>, however, takes a few attributes intended to, between them, include all the dependencies of a package.
 187  This is done both for structure and consistency, but also so that certain other setup can take place.
 188  For example, certain dependencies need their bin directories added to the <envar>PATH</envar>.
 189  That is built-in, but other setup is done via a pluggable mechanism that works in conjunction with these dependency attributes.
 190  See <xref linkend="ssec-setup-hooks"/> for details.
 191</para>
 192<para>
 193  Dependencies can be broken down along three axes: their host and target platforms relative to the new derivation's, and whether they are propagated.
 194  The platform distinctions are motivated by cross compilation; see <xref linkend="chap-cross"/> for exactly what each platform means.
 195  <footnote><para>
 196    The build platform is ignored because it is a mere implementation detail of the package satisfying the dependency:
 197    As a general programming principle, dependencies are always <emphasis>specified</emphasis> as interfaces, not concrete implementation.
 198  </para></footnote>
 199  But even if one is not cross compiling, the platforms imply whether or not the dependency is needed at run-time or build-time, a concept that makes perfect sense outside of cross compilation.
 200  For now, the run-time/build-time distinction is just a hint for mental clarity, but in the future it perhaps could be enforced.
 201</para>
 202<para>
 203  The extension of <envar>PATH</envar> with dependencies, alluded to above, proceeds according to the relative platforms alone.
 204  The process is carried out only for dependencies whose host platform matches the new derivation's build platform–i.e. which run on the platform where the new derivation will be built.
 205  <footnote><para>
 206    Currently, that means for native builds all dependencies are put on the <envar>PATH</envar>.
 207    But in the future that may not be the case for sake of matching cross:
 208    the platforms would be assumed to be unique for native and cross builds alike, so only the <varname>depsBuild*</varname> and <varname>nativeBuildDependencies</varname> dependencies would affect the <envar>PATH</envar>.
 209  </para></footnote>
 210  For each dependency <replaceable>dep</replaceable> of those dependencies, <filename><replaceable>dep</replaceable>/bin</filename>, if present, is added to the <envar>PATH</envar> environment variable.
 211</para>
 212<para>
 213  The dependency is propagated when it forces some of its other-transitive (non-immediate) downstream dependencies to also take it on as an immediate dependency.
 214  Nix itself already takes a package's transitive dependencies into account, but this propagation ensures nixpkgs-specific infrastructure like setup hooks (mentioned above) also are run as if the propagated dependency.
 215</para>
 216<para>
 217  It is important to note dependencies are not necessary propagated as the same sort of dependency that they were before, but rather as the corresponding sort so that the platform rules still line up.
 218  The exact rules for dependency propagation can be given by assigning each sort of dependency two integers based one how it's host and target platforms are offset from the depending derivation's platforms.
 219  Those offsets are given are given below in the descriptions of each dependency list attribute.
 220  Algorithmically, we traverse propagated inputs, accumulating every propagated dep's propagated deps and adjusting them to account for the "shift in perspective" described by the current dep's platform offsets.
 221  This results in sort a transitive closure of the dependency relation, with the offsets being approximately summed when two dependency links are combined.
 222  We also prune transitive deps whose combined offsets go out-of-bounds, which can be viewed as a filter over that transitive closure removing dependencies that are blatantly absurd.
 223</para>
 224<para>
 225  We can define the process precisely with <link xlink:href="https://en.wikipedia.org/wiki/Natural_deduction">Natural Deduction</link> using the inference rules.
 226  This probably seems a bit obtuse, but so is the bash code that actually implements it!
 227  <footnote><para>
 228    The <function>findInputs</function> function, currently residing in <filename>pkgs/stdenv/generic/setup.sh</filename>, implements the propagation logic.
 229  </para></footnote>
 230  They're confusing in very different ways so...hopefully if something doesn't make sense in one presentation, it does in the other!
 231  <programlisting>
 232let mapOffset(h, t, i) = i + (if i &lt;= 0 then h else t - 1)
 233
 234propagated-dep(h0, t0, A, B)
 235propagated-dep(h1, t1, B, C)
 236h0 + h1 in {-1, 0, 1}
 237h0 + t1 in {-1, 0, 1}
 238-------------------------------------- Transitive property
 239propagated-dep(mapOffset(h0, t0, h1),
 240               mapOffset(h0, t0, t1),
 241               A, C)</programlisting>
 242  <programlisting>
 243let mapOffset(h, t, i) = i + (if i &lt;= 0 then h else t - 1)
 244
 245dep(h0, _, A, B)
 246propagated-dep(h1, t1, B, C)
 247h0 + h1 in {-1, 0, 1}
 248h0 + t1 in {-1, 0, -1}
 249-------------------------------------- Take immediate deps' propagated deps
 250propagated-dep(mapOffset(h0, t0, h1),
 251               mapOffset(h0, t0, t1),
 252               A, C)</programlisting>
 253  <programlisting>
 254propagated-dep(h, t, A, B)
 255-------------------------------------- Propagated deps count as deps
 256dep(h, t, A, B)</programlisting>
 257  Some explanation of this monstrosity is in order.
 258  In the common case, the target offset of a dependency is the successor to the target offset: <literal>t = h + 1</literal>.
 259  That means that:
 260  <programlisting>
 261let f(h, t, i) = i + (if i &lt;= 0 then h else t - 1)
 262let f(h, h + 1, i) = i + (if i &lt;= 0 then h else (h + 1) - 1)
 263let f(h, h + 1, i) = i + (if i &lt;= 0 then h else h)
 264let f(h, h + 1, i) = i + h
 265  </programlisting>
 266  This is where the "sum-like" comes from above:
 267  We can just sum all the host offset to get the host offset of the transitive dependency.
 268  The target offset is the transitive dep is simply the host offset + 1, just as it was with the dependencies composed to make this transitive one;
 269  it can be ignored as it doesn't add any new information.
 270</para>
 271<para>
 272  Because of the bounds checks, the uncommon cases are <literal>h = t</literal> and <literal>h + 2 = t</literal>.
 273  In the former case, the motivation for <function>mapOffset</function> is that since its host and target platforms are the same, no transitive dep of it should be able to "discover" an offset greater than its reduced target offsets.
 274  <function>mapOffset</function> effectively "squashes" all its transitive dependencies' offsets so that none will ever be greater than the target offset of the original <literal>h = t</literal> package.
 275  In the other case, <literal>h + 1</literal> is skipped over between the host and target offsets.
 276  Instead of squashing the offsets, we need to "rip" them apart so no transitive dependencies' offset is that one.
 277</para>
 278<para>
 279Overall, the unifying theme here is that propagation shouldn't be introducing transitive dependencies involving platforms the needing package is unaware of.
 280The offset bounds checking and definition of <function>mapOffset</function> together ensure that this is the case.
 281Discovering a new offset is discovering a new platform, and since those platforms weren't in the derivation "spec" of the needing package, they cannot be relevant.
 282From a capability perspective, we can imagine that the host and target platforms of a package are the capabilities a package requires, and the depending package must provide the capability to the dependency.
 283</para>
 284
 285<variablelist>
 286  <title>Variables specifying dependencies</title>
 287
 288   <varlistentry>
 289     <term><varname>depsBuildBuild</varname></term>
 290     <listitem>
 291       <para>
 292         A list of dependencies whose host and target platforms are the new derivation's build platform.
 293         This means a <literal>-1</literal> host and <literal>-1</literal> target offset from the new derivation's platforms.
 294         They are programs/libraries used at build time that furthermore produce programs/libraries also used at build time.
 295         If the dependency doesn't care about the target platform (i.e. isn't a compiler or similar tool), put it in <varname>nativeBuildInputs</varname>instead.
 296         The most common use for this <literal>buildPackages.stdenv.cc</literal>, the default C compiler for this role.
 297         That example crops up more than one might think in old commonly used C libraries.
 298       </para>
 299       <para>
 300         Since these packages are able to be run at build time, that are always added to the <envar>PATH</envar>, as described above.
 301         But since these packages are only guaranteed to be able to run then, they shouldn't persist as run-time dependencies.
 302         This isn't currently enforced, but could be in the future.
 303       </para>
 304     </listitem>
 305  </varlistentry>
 306
 307  <varlistentry>
 308    <term><varname>nativeBuildInputs</varname></term>
 309    <listitem>
 310      <para>
 311        A list of dependencies whose host platform is the new derivation's build platform, and target platform is the new derivation's host platform.
 312        This means a <literal>-1</literal> host offset and <literal>0</literal> target offset from the new derivation's platforms.
 313        They are programs/libraries used at build time that, if they are a compiler or similar tool, produce code to run at run time—i.e. tools used to build the new derivation.
 314        If the dependency doesn't care about the target platform (i.e. isn't a compiler or similar tool), put it here, rather than in <varname>depsBuildBuild</varname> or <varname>depsBuildTarget</varname>.
 315        This would be called <varname>depsBuildHost</varname> but for historical continuity.
 316      </para>
 317      <para>
 318         Since these packages are able to be run at build time, that are added to the <envar>PATH</envar>, as described above.
 319         But since these packages only are guaranteed to be able to run then, they shouldn't persist as run-time dependencies.
 320         This isn't currently enforced, but could be in the future.
 321       </para>
 322    </listitem>
 323  </varlistentry>
 324
 325  <varlistentry>
 326    <term><varname>depsBuildTarget</varname></term>
 327    <listitem>
 328      <para>
 329        A list of dependencies whose host platform is the new derivation's build platform, and target platform is the new derivation's target platform.
 330        This means a <literal>-1</literal> host offset and <literal>1</literal> target offset from the new derivation's platforms.
 331        They are programs used at build time that produce code to run at run with code produced by the depending package.
 332        Most commonly, these would tools used to build the runtime or standard library the currently-being-built compiler will inject into any code it compiles.
 333        In many cases, the currently-being built compiler is itself employed for that task, but when that compiler won't run (i.e. its build and host platform differ) this is not possible.
 334        Other times, the compiler relies on some other tool, like binutils, that is always built separately so the dependency is unconditional.
 335      </para>
 336      <para>
 337        This is a somewhat confusing dependency to wrap ones head around, and for good reason.
 338        As the only one where the platform offsets are not adjacent integers, it requires thinking of a bootstrapping stage <emphasis>two</emphasis> away from the current one.
 339        It and it's use-case go hand in hand and are both considered poor form:
 340        try not to need this sort dependency, and try not avoid building standard libraries / runtimes in the same derivation as the compiler produces code using them.
 341        Instead strive to build those like a normal library, using the newly-built compiler just as a normal library would.
 342        In short, do not use this attribute unless you are packaging a compiler and are sure it is needed.
 343     </para>
 344     <para>
 345       Since these packages are able to be run at build time, that are added to the <envar>PATH</envar>, as described above.
 346       But since these packages only are guaranteed to be able to run then, they shouldn't persist as run-time dependencies.
 347       This isn't currently enforced, but could be in the future.
 348     </para>
 349    </listitem>
 350  </varlistentry>
 351
 352  <varlistentry>
 353    <term><varname>depsHostHost</varname></term>
 354    <listitem><para>
 355      A list of dependencies whose host and target platforms match the new derivation's host platform.
 356      This means a both <literal>0</literal> host offset and <literal>0</literal> target offset from the new derivation's host platform.
 357      These are packages used at run-time to generate code also used at run-time.
 358      In practice, that would usually be tools used by compilers for metaprogramming/macro systems, or libraries used by the macros/metaprogramming code itself.
 359      It's always preferable to use a <varname>depsBuildBuild</varname> dependency in the derivation being built than a <varname>depsHostHost</varname> on the tool doing the building for this purpose.
 360    </para></listitem>
 361  </varlistentry>
 362
 363  <varlistentry>
 364    <term><varname>buildInputs</varname></term>
 365    <listitem>
 366      <para>
 367        A list of dependencies whose host platform and target platform match the new derivation's.
 368        This means a <literal>0</literal> host offset and <literal>1</literal> target offset from the new derivation's host platform.
 369        This would be called <varname>depsHostTarget</varname> but for historical continuity.
 370        If the dependency doesn't care about the target platform (i.e. isn't a compiler or similar tool), put it here, rather than in <varname>depsBuildBuild</varname>.
 371      </para>
 372      <para>
 373        These often are programs/libraries used by the new derivation at <emphasis>run</emphasis>-time, but that isn't always the case.
 374        For example, the machine code in a statically linked library is only used at run time, but the derivation containing the library is only needed at build time.
 375        Even in the dynamic case, the library may also be needed at build time to appease the linker.
 376      </para>
 377    </listitem>
 378  </varlistentry>
 379
 380  <varlistentry>
 381    <term><varname>depsTargetTarget</varname></term>
 382    <listitem><para>
 383      A list of dependencies whose host platform matches the new derivation's target platform.
 384      This means a <literal>1</literal> offset from the new derivation's platforms.
 385      These are packages that run on the target platform, e.g. the standard library or run-time deps of standard library that a compiler insists on knowing about.
 386      It's poor form in almost all cases for a package to depend on another from a future stage [future stage corresponding to positive offset].
 387      Do not use this attribute unless you are packaging a compiler and are sure it is needed.
 388    </para></listitem>
 389  </varlistentry>
 390
 391  <varlistentry>
 392    <term><varname>depsBuildBuildPropagated</varname></term>
 393    <listitem><para>
 394      The propagated equivalent of <varname>depsBuildBuild</varname>.
 395      This perhaps never ought to be used, but it is included for consistency [see below for the others].
 396    </para></listitem>
 397  </varlistentry>
 398
 399  <varlistentry>
 400    <term><varname>propagatedNativeBuildInputs</varname></term>
 401    <listitem><para>
 402      The propagated equivalent of <varname>nativeBuildInputs</varname>.
 403      This would be called <varname>depsBuildHostPropagated</varname> but for historical continuity.
 404      For example, if package <varname>Y</varname> has <literal>propagatedNativeBuildInputs = [X]</literal>, and package <varname>Z</varname> has <literal>buildInputs = [Y]</literal>, then package <varname>Z</varname> will be built as if it included package <varname>X</varname> in its <varname>nativeBuildInputs</varname>.
 405      If instead, package <varname>Z</varname> has <literal>nativeBuildInputs = [Y]</literal>, then <varname>Z</varname> will be built as if it included <varname>X</varname> in the <varname>depsBuildBuild</varname> of package <varname>Z</varname>, because of the sum of the two <literal>-1</literal> host offsets.
 406    </para></listitem>
 407  </varlistentry>
 408
 409  <varlistentry>
 410    <term><varname>depsBuildTargetPropagated</varname></term>
 411    <listitem><para>
 412      The propagated equivalent of <varname>depsBuildTarget</varname>.
 413      This is prefixed for the same reason of alerting potential users.
 414    </para></listitem>
 415  </varlistentry>
 416
 417  <varlistentry>
 418    <term><varname>depsHostHostPropagated</varname></term>
 419    <listitem><para>
 420      The propagated equivalent of <varname>depsHostHost</varname>.
 421    </para></listitem>
 422  </varlistentry>
 423
 424  <varlistentry>
 425    <term><varname>propagatedBuildInputs</varname></term>
 426    <listitem><para>
 427      The propagated equivalent of <varname>buildInputs</varname>.
 428      This would be called <varname>depsHostTargetPropagated</varname> but for historical continuity.
 429    </para></listitem>
 430  </varlistentry>
 431
 432  <varlistentry>
 433    <term><varname>depsTargetTarget</varname></term>
 434    <listitem><para>
 435      The propagated equivalent of <varname>depsTargetTarget</varname>.
 436      This is prefixed for the same reason of alerting potential users.
 437    </para></listitem>
 438  </varlistentry>
 439
 440</variablelist>
 441
 442</section>
 443
 444
 445<section xml:id="ssec-stdenv-attributes"><title>Attributes</title>
 446
 447<variablelist>
 448  <title>Variables affecting <literal>stdenv</literal>
 449  initialisation</title>
 450
 451  <varlistentry>
 452    <term><varname>NIX_DEBUG</varname></term>
 453    <listitem><para>
 454      A natural number indicating how much information to log.
 455      If set to 1 or higher, <literal>stdenv</literal> will print moderate debug information during the build.
 456      In particular, the <command>gcc</command> and <command>ld</command> wrapper scripts will print out the complete command line passed to the wrapped tools.
 457      If set to 6 or higher, the <literal>stdenv</literal> setup script will be run with <literal>set -x</literal> tracing.
 458      If set to 7 or higher, the <command>gcc</command> and <command>ld</command> wrapper scripts will also be run with <literal>set -x</literal> tracing.
 459    </para></listitem>
 460  </varlistentry>
 461
 462</variablelist>
 463
 464<variablelist>
 465  <title>Variables affecting build properties</title>
 466
 467  <varlistentry>
 468    <term><varname>enableParallelBuilding</varname></term>
 469    <listitem>
 470      <para>If set to <literal>true</literal>, <literal>stdenv</literal> will
 471      pass specific flags to <literal>make</literal> and other build tools to
 472      enable parallel building with up to <literal>build-cores</literal>
 473      workers.</para>
 474
 475      <para>Unless set to <literal>false</literal>, some build systems with good
 476      support for parallel building including <literal>cmake</literal>,
 477      <literal>meson</literal>, and <literal>qmake</literal> will set it to
 478      <literal>true</literal>.</para>
 479    </listitem>
 480  </varlistentry>
 481
 482  <varlistentry>
 483    <term><varname>preferLocalBuild</varname></term>
 484    <listitem><para>If set, specifies that the package is so lightweight
 485    in terms of build operations (e.g. write a text file from a Nix string
 486    to the store) that there's no need to look for it in binary caches --
 487    it's faster to just build it locally. It also tells Hydra and other
 488    facilities that this package doesn't need to be exported in binary
 489    caches (noone would use it, after all).</para></listitem>
 490  </varlistentry>
 491
 492</variablelist>
 493
 494<variablelist>
 495  <title>Special variables</title>
 496
 497  <varlistentry>
 498    <term><varname>passthru</varname></term>
 499    <listitem><para>This is an attribute set which can be filled with arbitrary
 500    values. For example:
 501
 502<programlisting>
 503passthru = {
 504  foo = "bar";
 505  baz = {
 506    value1 = 4;
 507    value2 = 5;
 508  };
 509}
 510</programlisting>
 511
 512</para>
 513
 514    <para>Values inside it are not passed to the builder, so you can change
 515    them without triggering a rebuild. However, they can be accessed outside of a
 516    derivation directly, as if they were set inside a derivation itself, e.g.
 517    <literal>hello.baz.value1</literal>. We don't specify any usage or
 518    schema of <literal>passthru</literal> - it is meant for values that would be
 519    useful outside the derivation in other parts of a Nix expression (e.g. in other
 520    derivations). An example would be to convey some specific dependency of your
 521    derivation which contains a program with plugins support.  Later, others who
 522    make derivations with plugins can use passed-through dependency to ensure that
 523    their plugin would be binary-compatible with built program.</para></listitem>
 524  </varlistentry>
 525
 526</variablelist>
 527
 528</section>
 529
 530
 531<section xml:id="sec-stdenv-phases"><title>Phases</title>
 532
 533<para>The generic builder has a number of <emphasis>phases</emphasis>.
 534Package builds are split into phases to make it easier to override
 535specific parts of the build (e.g., unpacking the sources or installing
 536the binaries).  Furthermore, it allows a nicer presentation of build
 537logs in the Nix build farm.</para>
 538
 539<para>Each phase can be overridden in its entirety either by setting
 540the environment variable
 541<varname><replaceable>name</replaceable>Phase</varname> to a string
 542containing some shell commands to be executed, or by redefining the
 543shell function
 544<varname><replaceable>name</replaceable>Phase</varname>.  The former
 545is convenient to override a phase from the derivation, while the
 546latter is convenient from a build script.
 547
 548However, typically one only wants to <emphasis>add</emphasis> some
 549commands to a phase, e.g. by defining <literal>postInstall</literal>
 550or <literal>preFixup</literal>, as skipping some of the default actions
 551may have unexpected consequences.
 552</para>
 553
 554
 555<section xml:id="ssec-controlling-phases"><title>Controlling
 556phases</title>
 557
 558<para>There are a number of variables that control what phases are
 559executed and in what order:
 560
 561<variablelist>
 562  <title>Variables affecting phase control</title>
 563
 564  <varlistentry>
 565    <term><varname>phases</varname></term>
 566    <listitem>
 567      <para>Specifies the phases.  You can change the order in which
 568      phases are executed, or add new phases, by setting this
 569      variable.  If it’s not set, the default value is used, which is
 570      <literal>$prePhases unpackPhase patchPhase $preConfigurePhases
 571      configurePhase $preBuildPhases buildPhase checkPhase
 572      $preInstallPhases installPhase fixupPhase $preDistPhases
 573      distPhase $postPhases</literal>.
 574      </para>
 575
 576      <para>Usually, if you just want to add a few phases, it’s more
 577      convenient to set one of the variables below (such as
 578      <varname>preInstallPhases</varname>), as you then don’t specify
 579      all the normal phases.</para>
 580    </listitem>
 581  </varlistentry>
 582
 583  <varlistentry>
 584    <term><varname>prePhases</varname></term>
 585    <listitem>
 586      <para>Additional phases executed before any of the default phases.</para>
 587    </listitem>
 588  </varlistentry>
 589
 590  <varlistentry>
 591    <term><varname>preConfigurePhases</varname></term>
 592    <listitem>
 593      <para>Additional phases executed just before the configure phase.</para>
 594    </listitem>
 595  </varlistentry>
 596
 597  <varlistentry>
 598    <term><varname>preBuildPhases</varname></term>
 599    <listitem>
 600      <para>Additional phases executed just before the build phase.</para>
 601    </listitem>
 602  </varlistentry>
 603
 604  <varlistentry>
 605    <term><varname>preInstallPhases</varname></term>
 606    <listitem>
 607      <para>Additional phases executed just before the install phase.</para>
 608    </listitem>
 609  </varlistentry>
 610
 611  <varlistentry>
 612    <term><varname>preFixupPhases</varname></term>
 613    <listitem>
 614      <para>Additional phases executed just before the fixup phase.</para>
 615    </listitem>
 616  </varlistentry>
 617
 618  <varlistentry>
 619    <term><varname>preDistPhases</varname></term>
 620    <listitem>
 621      <para>Additional phases executed just before the distribution phase.</para>
 622    </listitem>
 623  </varlistentry>
 624
 625  <varlistentry>
 626    <term><varname>postPhases</varname></term>
 627    <listitem>
 628      <para>Additional phases executed after any of the default
 629      phases.</para>
 630    </listitem>
 631  </varlistentry>
 632
 633</variablelist>
 634
 635</para>
 636
 637</section>
 638
 639
 640<section xml:id="ssec-unpack-phase"><title>The unpack phase</title>
 641
 642<para>The unpack phase is responsible for unpacking the source code of
 643the package.  The default implementation of
 644<function>unpackPhase</function> unpacks the source files listed in
 645the <envar>src</envar> environment variable to the current directory.
 646It supports the following files by default:
 647
 648<variablelist>
 649
 650  <varlistentry>
 651    <term>Tar files</term>
 652    <listitem><para>These can optionally be compressed using
 653    <command>gzip</command> (<filename>.tar.gz</filename>,
 654    <filename>.tgz</filename> or <filename>.tar.Z</filename>),
 655    <command>bzip2</command> (<filename>.tar.bz2</filename> or
 656    <filename>.tbz2</filename>) or <command>xz</command>
 657    (<filename>.tar.xz</filename> or
 658    <filename>.tar.lzma</filename>).</para></listitem>
 659  </varlistentry>
 660
 661  <varlistentry>
 662    <term>Zip files</term>
 663    <listitem><para>Zip files are unpacked using
 664    <command>unzip</command>.  However, <command>unzip</command> is
 665    not in the standard environment, so you should add it to
 666    <varname>buildInputs</varname> yourself.</para></listitem>
 667  </varlistentry>
 668
 669  <varlistentry>
 670    <term>Directories in the Nix store</term>
 671    <listitem><para>These are simply copied to the current directory.
 672    The hash part of the file name is stripped,
 673    e.g. <filename>/nix/store/1wydxgby13cz...-my-sources</filename>
 674    would be copied to
 675    <filename>my-sources</filename>.</para></listitem>
 676  </varlistentry>
 677
 678</variablelist>
 679
 680Additional file types can be supported by setting the
 681<varname>unpackCmd</varname> variable (see below).</para>
 682
 683<para></para>
 684
 685<variablelist>
 686  <title>Variables controlling the unpack phase</title>
 687
 688  <varlistentry>
 689    <term><varname>srcs</varname> / <varname>src</varname></term>
 690    <listitem><para>The list of source files or directories to be
 691    unpacked or copied.  One of these must be set.</para></listitem>
 692  </varlistentry>
 693
 694  <varlistentry>
 695    <term><varname>sourceRoot</varname></term>
 696    <listitem><para>After running <function>unpackPhase</function>,
 697    the generic builder changes the current directory to the directory
 698    created by unpacking the sources.  If there are multiple source
 699    directories, you should set <varname>sourceRoot</varname> to the
 700    name of the intended directory.</para></listitem>
 701  </varlistentry>
 702
 703  <varlistentry>
 704    <term><varname>setSourceRoot</varname></term>
 705    <listitem><para>Alternatively to setting
 706    <varname>sourceRoot</varname>, you can set
 707    <varname>setSourceRoot</varname> to a shell command to be
 708    evaluated by the unpack phase after the sources have been
 709    unpacked.  This command must set
 710    <varname>sourceRoot</varname>.</para></listitem>
 711  </varlistentry>
 712
 713  <varlistentry>
 714    <term><varname>preUnpack</varname></term>
 715    <listitem><para>Hook executed at the start of the unpack
 716    phase.</para></listitem>
 717  </varlistentry>
 718
 719  <varlistentry>
 720    <term><varname>postUnpack</varname></term>
 721    <listitem><para>Hook executed at the end of the unpack
 722    phase.</para></listitem>
 723  </varlistentry>
 724
 725  <varlistentry>
 726    <term><varname>dontMakeSourcesWritable</varname></term>
 727    <listitem><para>If set to <literal>1</literal>, the unpacked
 728    sources are <emphasis>not</emphasis> made
 729    writable.  By default, they are made writable to prevent problems
 730    with read-only sources.  For example, copied store directories
 731    would be read-only without this.</para></listitem>
 732  </varlistentry>
 733
 734  <varlistentry>
 735    <term><varname>unpackCmd</varname></term>
 736    <listitem><para>The unpack phase evaluates the string
 737    <literal>$unpackCmd</literal> for any unrecognised file.  The path
 738    to the current source file is contained in the
 739    <varname>curSrc</varname> variable.</para></listitem>
 740  </varlistentry>
 741
 742</variablelist>
 743
 744</section>
 745
 746
 747<section xml:id="ssec-patch-phase"><title>The patch phase</title>
 748
 749<para>The patch phase applies the list of patches defined in the
 750<varname>patches</varname> variable.</para>
 751
 752<variablelist>
 753  <title>Variables controlling the patch phase</title>
 754
 755  <varlistentry>
 756    <term><varname>patches</varname></term>
 757    <listitem><para>The list of patches.  They must be in the format
 758    accepted by the <command>patch</command> command, and may
 759    optionally be compressed using <command>gzip</command>
 760    (<filename>.gz</filename>), <command>bzip2</command>
 761    (<filename>.bz2</filename>) or <command>xz</command>
 762    (<filename>.xz</filename>).</para></listitem>
 763  </varlistentry>
 764
 765  <varlistentry>
 766    <term><varname>patchFlags</varname></term>
 767    <listitem><para>Flags to be passed to <command>patch</command>.
 768    If not set, the argument <option>-p1</option> is used, which
 769    causes the leading directory component to be stripped from the
 770    file names in each patch.</para></listitem>
 771  </varlistentry>
 772
 773  <varlistentry>
 774    <term><varname>prePatch</varname></term>
 775    <listitem><para>Hook executed at the start of the patch
 776    phase.</para></listitem>
 777  </varlistentry>
 778
 779  <varlistentry>
 780    <term><varname>postPatch</varname></term>
 781    <listitem><para>Hook executed at the end of the patch
 782    phase.</para></listitem>
 783  </varlistentry>
 784
 785</variablelist>
 786
 787</section>
 788
 789
 790<section xml:id="ssec-configure-phase"><title>The configure phase</title>
 791
 792<para>The configure phase prepares the source tree for building.  The
 793default <function>configurePhase</function> runs
 794<filename>./configure</filename> (typically an Autoconf-generated
 795script) if it exists.</para>
 796
 797<variablelist>
 798  <title>Variables controlling the configure phase</title>
 799
 800  <varlistentry>
 801    <term><varname>configureScript</varname></term>
 802    <listitem><para>The name of the configure script.  It defaults to
 803    <filename>./configure</filename> if it exists; otherwise, the
 804    configure phase is skipped.  This can actually be a command (like
 805    <literal>perl ./Configure.pl</literal>).</para></listitem>
 806  </varlistentry>
 807
 808  <varlistentry>
 809    <term><varname>configureFlags</varname></term>
 810    <listitem><para>A list of strings passed as additional arguments to the
 811    configure script.</para></listitem>
 812  </varlistentry>
 813
 814  <varlistentry>
 815    <term><varname>configureFlagsArray</varname></term>
 816    <listitem><para>A shell array containing additional arguments
 817    passed to the configure script.  You must use this instead of
 818    <varname>configureFlags</varname> if the arguments contain
 819    spaces.</para></listitem>
 820  </varlistentry>
 821
 822  <varlistentry>
 823    <term><varname>dontAddPrefix</varname></term>
 824    <listitem><para>By default, the flag
 825    <literal>--prefix=$prefix</literal> is added to the configure
 826    flags.  If this is undesirable, set this variable to
 827    true.</para></listitem>
 828  </varlistentry>
 829
 830  <varlistentry>
 831    <term><varname>prefix</varname></term>
 832    <listitem><para>The prefix under which the package must be
 833    installed, passed via the <option>--prefix</option> option to the
 834    configure script.  It defaults to
 835    <option>$out</option>.</para></listitem>
 836  </varlistentry>
 837
 838  <varlistentry>
 839    <term><varname>dontAddDisableDepTrack</varname></term>
 840    <listitem><para>By default, the flag
 841    <literal>--disable-dependency-tracking</literal> is added to the
 842    configure flags to speed up Automake-based builds.  If this is
 843    undesirable, set this variable to true.</para></listitem>
 844  </varlistentry>
 845
 846  <varlistentry>
 847    <term><varname>dontFixLibtool</varname></term>
 848    <listitem><para>By default, the configure phase applies some
 849    special hackery to all files called <filename>ltmain.sh</filename>
 850    before running the configure script in order to improve the purity
 851    of Libtool-based packages<footnote><para>It clears the
 852    <varname>sys_lib_<replaceable>*</replaceable>search_path</varname>
 853    variables in the Libtool script to prevent Libtool from using
 854    libraries in <filename>/usr/lib</filename> and
 855    such.</para></footnote>.  If this is undesirable, set this
 856    variable to true.</para></listitem>
 857  </varlistentry>
 858
 859  <varlistentry>
 860    <term><varname>dontDisableStatic</varname></term>
 861    <listitem><para>By default, when the configure script has
 862    <option>--enable-static</option>, the option
 863    <option>--disable-static</option> is added to the configure flags.</para>
 864    <para>If this is undesirable, set this variable to
 865    true.</para></listitem>
 866  </varlistentry>
 867
 868  <varlistentry>
 869    <term><varname>configurePlatforms</varname></term>
 870    <listitem><para>
 871      By default, when cross compiling, the configure script has <option>--build=...</option> and <option>--host=...</option> passed.
 872      Packages can instead pass <literal>[ "build" "host" "target" ]</literal> or a subset to control exactly which platform flags are passed.
 873      Compilers and other tools should use this to also pass the target platform, for example.
 874      <footnote><para>Eventually these will be passed when in native builds too, to improve determinism: build-time guessing, as is done today, is a risk of impurity.</para></footnote>
 875    </para></listitem>
 876  </varlistentry>
 877
 878  <varlistentry>
 879    <term><varname>preConfigure</varname></term>
 880    <listitem><para>Hook executed at the start of the configure
 881    phase.</para></listitem>
 882  </varlistentry>
 883
 884  <varlistentry>
 885    <term><varname>postConfigure</varname></term>
 886    <listitem><para>Hook executed at the end of the configure
 887    phase.</para></listitem>
 888  </varlistentry>
 889
 890</variablelist>
 891
 892
 893</section>
 894
 895
 896<section xml:id="build-phase"><title>The build phase</title>
 897
 898<para>The build phase is responsible for actually building the package
 899(e.g. compiling it).  The default <function>buildPhase</function>
 900simply calls <command>make</command> if a file named
 901<filename>Makefile</filename>, <filename>makefile</filename> or
 902<filename>GNUmakefile</filename> exists in the current directory (or
 903the <varname>makefile</varname> is explicitly set); otherwise it does
 904nothing.</para>
 905
 906<variablelist>
 907  <title>Variables controlling the build phase</title>
 908
 909  <varlistentry>
 910    <term><varname>dontBuild</varname></term>
 911    <listitem><para>Set to true to skip the build phase.</para></listitem>
 912  </varlistentry>
 913
 914  <varlistentry>
 915    <term><varname>makefile</varname></term>
 916    <listitem><para>The file name of the Makefile.</para></listitem>
 917  </varlistentry>
 918
 919  <varlistentry>
 920    <term><varname>makeFlags</varname></term>
 921    <listitem><para>A list of strings passed as additional flags to
 922    <command>make</command>.  These flags are also used by the default
 923    install and check phase.  For setting make flags specific to the
 924    build phase, use <varname>buildFlags</varname> (see below).
 925
 926<programlisting>
 927makeFlags = [ "PREFIX=$(out)" ];
 928</programlisting>
 929
 930    <note><para>The flags are quoted in bash, but environment variables can
 931    be specified by using the make syntax.</para></note></para></listitem>
 932  </varlistentry>
 933
 934  <varlistentry>
 935    <term><varname>makeFlagsArray</varname></term>
 936    <listitem><para>A shell array containing additional arguments
 937    passed to <command>make</command>.  You must use this instead of
 938    <varname>makeFlags</varname> if the arguments contain
 939    spaces, e.g.
 940
 941<programlisting>
 942makeFlagsArray=(CFLAGS="-O0 -g" LDFLAGS="-lfoo -lbar")
 943</programlisting>
 944
 945    Note that shell arrays cannot be passed through environment
 946    variables, so you cannot set <varname>makeFlagsArray</varname> in
 947    a derivation attribute (because those are passed through
 948    environment variables): you have to define them in shell
 949    code.</para></listitem>
 950  </varlistentry>
 951
 952  <varlistentry>
 953    <term><varname>buildFlags</varname> / <varname>buildFlagsArray</varname></term>
 954    <listitem><para>A list of strings passed as additional flags to
 955    <command>make</command>.  Like <varname>makeFlags</varname> and
 956    <varname>makeFlagsArray</varname>, but only used by the build
 957    phase.</para></listitem>
 958  </varlistentry>
 959
 960  <varlistentry>
 961    <term><varname>preBuild</varname></term>
 962    <listitem><para>Hook executed at the start of the build
 963    phase.</para></listitem>
 964  </varlistentry>
 965
 966  <varlistentry>
 967    <term><varname>postBuild</varname></term>
 968    <listitem><para>Hook executed at the end of the build
 969    phase.</para></listitem>
 970  </varlistentry>
 971
 972</variablelist>
 973
 974
 975<para>
 976You can set flags for <command>make</command> through the
 977<varname>makeFlags</varname> variable.</para>
 978
 979<para>Before and after running <command>make</command>, the hooks
 980<varname>preBuild</varname> and <varname>postBuild</varname> are
 981called, respectively.</para>
 982
 983</section>
 984
 985
 986<section xml:id="ssec-check-phase"><title>The check phase</title>
 987
 988<para>The check phase checks whether the package was built correctly
 989by running its test suite.  The default
 990<function>checkPhase</function> calls <command>make check</command>,
 991but only if the <varname>doCheck</varname> variable is enabled.</para>
 992
 993<variablelist>
 994  <title>Variables controlling the check phase</title>
 995
 996  <varlistentry>
 997    <term><varname>doCheck</varname></term>
 998    <listitem><para>
 999      Controls whether the check phase is executed.
1000      By default it is skipped, but if <varname>doCheck</varname> is set to true, the check phase is usually executed.
1001      Thus you should set <programlisting>doCheck = true;</programlisting> in the derivation to enable checks.
1002      The exception is cross compilation.
1003      Cross compiled builds never run tests, no matter how <varname>doCheck</varname> is set,
1004      as the newly-built program won't run on the platform used to build it.
1005    </para></listitem>
1006  </varlistentry>
1007
1008  <varlistentry>
1009    <term><varname>makeFlags</varname> /
1010    <varname>makeFlagsArray</varname> /
1011    <varname>makefile</varname></term>
1012    <listitem><para>See the build phase for details.</para></listitem>
1013  </varlistentry>
1014
1015  <varlistentry>
1016    <term><varname>checkTarget</varname></term>
1017    <listitem><para>The make target that runs the tests.  Defaults to
1018    <literal>check</literal>.</para></listitem>
1019  </varlistentry>
1020
1021  <varlistentry>
1022    <term><varname>checkFlags</varname> / <varname>checkFlagsArray</varname></term>
1023    <listitem><para>A list of strings passed as additional flags to
1024    <command>make</command>.  Like <varname>makeFlags</varname> and
1025    <varname>makeFlagsArray</varname>, but only used by the check
1026    phase.</para></listitem>
1027  </varlistentry>
1028
1029  <varlistentry>
1030    <term><varname>preCheck</varname></term>
1031    <listitem><para>Hook executed at the start of the check
1032    phase.</para></listitem>
1033  </varlistentry>
1034
1035  <varlistentry>
1036    <term><varname>postCheck</varname></term>
1037    <listitem><para>Hook executed at the end of the check
1038    phase.</para></listitem>
1039  </varlistentry>
1040
1041</variablelist>
1042
1043
1044</section>
1045
1046
1047<section xml:id="ssec-install-phase"><title>The install phase</title>
1048
1049<para>The install phase is responsible for installing the package in
1050the Nix store under <envar>out</envar>.  The default
1051<function>installPhase</function> creates the directory
1052<literal>$out</literal> and calls <command>make
1053install</command>.</para>
1054
1055<variablelist>
1056  <title>Variables controlling the install phase</title>
1057
1058  <varlistentry>
1059    <term><varname>makeFlags</varname> /
1060    <varname>makeFlagsArray</varname> /
1061    <varname>makefile</varname></term>
1062    <listitem><para>See the build phase for details.</para></listitem>
1063  </varlistentry>
1064
1065  <varlistentry>
1066    <term><varname>installTargets</varname></term>
1067    <listitem><para>The make targets that perform the installation.
1068    Defaults to <literal>install</literal>.  Example:
1069
1070<programlisting>
1071installTargets = "install-bin install-doc";</programlisting>
1072
1073    </para></listitem>
1074  </varlistentry>
1075
1076  <varlistentry>
1077    <term><varname>installFlags</varname> / <varname>installFlagsArray</varname></term>
1078    <listitem><para>A list of strings passed as additional flags to
1079    <command>make</command>.  Like <varname>makeFlags</varname> and
1080    <varname>makeFlagsArray</varname>, but only used by the install
1081    phase.</para></listitem>
1082  </varlistentry>
1083
1084  <varlistentry>
1085    <term><varname>preInstall</varname></term>
1086    <listitem><para>Hook executed at the start of the install
1087    phase.</para></listitem>
1088  </varlistentry>
1089
1090  <varlistentry>
1091    <term><varname>postInstall</varname></term>
1092    <listitem><para>Hook executed at the end of the install
1093    phase.</para></listitem>
1094  </varlistentry>
1095
1096</variablelist>
1097
1098
1099</section>
1100
1101
1102<section xml:id="ssec-fixup-phase"><title>The fixup phase</title>
1103
1104<para>The fixup phase performs some (Nix-specific) post-processing
1105actions on the files installed under <filename>$out</filename> by the
1106install phase.  The default <function>fixupPhase</function> does the
1107following:
1108
1109<itemizedlist>
1110
1111  <listitem><para>It moves the <filename>man/</filename>,
1112  <filename>doc/</filename> and <filename>info/</filename>
1113  subdirectories of <envar>$out</envar> to
1114  <filename>share/</filename>.</para></listitem>
1115
1116  <listitem><para>It strips libraries and executables of debug
1117  information.</para></listitem>
1118
1119  <listitem><para>On Linux, it applies the <command>patchelf</command>
1120  command to ELF executables and libraries to remove unused
1121  directories from the <literal>RPATH</literal> in order to prevent
1122  unnecessary runtime dependencies.</para></listitem>
1123
1124  <listitem><para>It rewrites the interpreter paths of shell scripts
1125  to paths found in <envar>PATH</envar>.  E.g.,
1126  <filename>/usr/bin/perl</filename> will be rewritten to
1127  <filename>/nix/store/<replaceable>some-perl</replaceable>/bin/perl</filename>
1128  found in <envar>PATH</envar>.</para></listitem>
1129
1130</itemizedlist>
1131
1132</para>
1133
1134<variablelist>
1135  <title>Variables controlling the fixup phase</title>
1136
1137  <varlistentry>
1138    <term><varname>dontStrip</varname></term>
1139    <listitem><para>If set, libraries and executables are not
1140    stripped.  By default, they are.</para></listitem>
1141  </varlistentry>
1142  <varlistentry>
1143    <term><varname>dontStripHost</varname></term>
1144    <listitem><para>
1145      Like <varname>dontStripHost</varname>, but only affects the <command>strip</command> command targetting the package's host platform.
1146      Useful when supporting cross compilation, but otherwise feel free to ignore.
1147    </para></listitem>
1148  </varlistentry>
1149  <varlistentry>
1150    <term><varname>dontStripTarget</varname></term>
1151    <listitem><para>
1152      Like <varname>dontStripHost</varname>, but only affects the <command>strip</command> command targetting the packages' target platform.
1153      Useful when supporting cross compilation, but otherwise feel free to ignore.
1154    </para></listitem>
1155  </varlistentry>
1156
1157  <varlistentry>
1158    <term><varname>dontMoveSbin</varname></term>
1159    <listitem><para>If set, files in <filename>$out/sbin</filename> are not moved
1160    to <filename>$out/bin</filename>. By default, they are.</para></listitem>
1161  </varlistentry>
1162
1163  <varlistentry>
1164    <term><varname>stripAllList</varname></term>
1165    <listitem><para>List of directories to search for libraries and
1166    executables from which <emphasis>all</emphasis> symbols should be
1167    stripped.  By default, it’s empty.  Stripping all symbols is
1168    risky, since it may remove not just debug symbols but also ELF
1169    information necessary for normal execution.</para></listitem>
1170  </varlistentry>
1171
1172  <varlistentry>
1173    <term><varname>stripAllFlags</varname></term>
1174    <listitem><para>Flags passed to the <command>strip</command>
1175    command applied to the files in the directories listed in
1176    <varname>stripAllList</varname>.  Defaults to <option>-s</option>
1177    (i.e. <option>--strip-all</option>).</para></listitem>
1178  </varlistentry>
1179
1180  <varlistentry>
1181    <term><varname>stripDebugList</varname></term>
1182    <listitem><para>List of directories to search for libraries and
1183    executables from which only debugging-related symbols should be
1184    stripped.  It defaults to <literal>lib bin
1185    sbin</literal>.</para></listitem>
1186  </varlistentry>
1187
1188  <varlistentry>
1189    <term><varname>stripDebugFlags</varname></term>
1190    <listitem><para>Flags passed to the <command>strip</command>
1191    command applied to the files in the directories listed in
1192    <varname>stripDebugList</varname>.  Defaults to
1193    <option>-S</option>
1194    (i.e. <option>--strip-debug</option>).</para></listitem>
1195  </varlistentry>
1196
1197  <varlistentry>
1198    <term><varname>dontPatchELF</varname></term>
1199    <listitem><para>If set, the <command>patchelf</command> command is
1200    not used to remove unnecessary <literal>RPATH</literal> entries.
1201    Only applies to Linux.</para></listitem>
1202  </varlistentry>
1203
1204  <varlistentry>
1205    <term><varname>dontPatchShebangs</varname></term>
1206    <listitem><para>If set, scripts starting with
1207    <literal>#!</literal> do not have their interpreter paths
1208    rewritten to paths in the Nix store.</para></listitem>
1209  </varlistentry>
1210
1211  <varlistentry>
1212    <term><varname>forceShare</varname></term>
1213    <listitem><para>The list of directories that must be moved from
1214    <filename>$out</filename> to <filename>$out/share</filename>.
1215    Defaults to <literal>man doc info</literal>.</para></listitem>
1216  </varlistentry>
1217
1218  <varlistentry>
1219    <term><varname>setupHook</varname></term>
1220    <listitem><para>A package can export a <link
1221    linkend="ssec-setup-hooks">setup hook</link> by setting this
1222    variable.  The setup hook, if defined, is copied to
1223    <filename>$out/nix-support/setup-hook</filename>.  Environment
1224    variables are then substituted in it using <function
1225    linkend="fun-substituteAll">substituteAll</function>.</para></listitem>
1226  </varlistentry>
1227
1228  <varlistentry>
1229    <term><varname>preFixup</varname></term>
1230    <listitem><para>Hook executed at the start of the fixup
1231    phase.</para></listitem>
1232  </varlistentry>
1233
1234  <varlistentry>
1235    <term><varname>postFixup</varname></term>
1236    <listitem><para>Hook executed at the end of the fixup
1237    phase.</para></listitem>
1238  </varlistentry>
1239
1240  <varlistentry xml:id="stdenv-separateDebugInfo">
1241    <term><varname>separateDebugInfo</varname></term>
1242    <listitem><para>If set to <literal>true</literal>, the standard
1243    environment will enable debug information in C/C++ builds. After
1244    installation, the debug information will be separated from the
1245    executables and stored in the output named
1246    <literal>debug</literal>. (This output is enabled automatically;
1247    you don’t need to set the <varname>outputs</varname> attribute
1248    explicitly.) To be precise, the debug information is stored in
1249    <filename><replaceable>debug</replaceable>/lib/debug/.build-id/<replaceable>XX</replaceable>/<replaceable>YYYY…</replaceable></filename>,
1250    where <replaceable>XXYYYY…</replaceable> is the <replaceable>build
1251    ID</replaceable> of the binary — a SHA-1 hash of the contents of
1252    the binary. Debuggers like GDB use the build ID to look up the
1253    separated debug information.</para>
1254
1255    <para>For example, with GDB, you can add
1256
1257<programlisting>
1258set debug-file-directory ~/.nix-profile/lib/debug
1259</programlisting>
1260
1261    to <filename>~/.gdbinit</filename>. GDB will then be able to find
1262    debug information installed via <literal>nix-env
1263    -i</literal>.</para>
1264
1265    </listitem>
1266  </varlistentry>
1267
1268</variablelist>
1269
1270</section>
1271
1272<section xml:id="ssec-installCheck-phase"><title>The installCheck phase</title>
1273
1274<para>The installCheck phase checks whether the package was installed
1275correctly by running its test suite against the installed directories.
1276The default <function>installCheck</function> calls <command>make
1277installcheck</command>.</para>
1278
1279<variablelist>
1280  <title>Variables controlling the installCheck phase</title>
1281
1282  <varlistentry>
1283    <term><varname>doInstallCheck</varname></term>
1284    <listitem><para>
1285      Controls whether the installCheck phase is executed.
1286      By default it is skipped, but if <varname>doInstallCheck</varname> is set to true, the installCheck phase is usually executed.
1287      Thus you should set <programlisting>doInstallCheck = true;</programlisting> in the derivation to enable install checks.
1288      The exception is cross compilation.
1289      Cross compiled builds never run tests, no matter how <varname>doInstallCheck</varname> is set,
1290      as the newly-built program won't run on the platform used to build it.
1291    </para></listitem>
1292  </varlistentry>
1293
1294  <varlistentry>
1295    <term><varname>preInstallCheck</varname></term>
1296    <listitem><para>Hook executed at the start of the installCheck
1297    phase.</para></listitem>
1298  </varlistentry>
1299
1300  <varlistentry>
1301    <term><varname>postInstallCheck</varname></term>
1302    <listitem><para>Hook executed at the end of the installCheck
1303    phase.</para></listitem>
1304  </varlistentry>
1305
1306</variablelist>
1307
1308</section>
1309
1310<section xml:id="ssec-distribution-phase"><title>The distribution
1311phase</title>
1312
1313<para>The distribution phase is intended to produce a source
1314distribution of the package.  The default
1315<function>distPhase</function> first calls <command>make
1316dist</command>, then it copies the resulting source tarballs to
1317<filename>$out/tarballs/</filename>.  This phase is only executed if
1318the attribute <varname>doDist</varname> is set.</para>
1319
1320<variablelist>
1321  <title>Variables controlling the distribution phase</title>
1322
1323  <varlistentry>
1324    <term><varname>distTarget</varname></term>
1325    <listitem><para>The make target that produces the distribution.
1326    Defaults to <literal>dist</literal>.</para></listitem>
1327  </varlistentry>
1328
1329  <varlistentry>
1330    <term><varname>distFlags</varname> / <varname>distFlagsArray</varname></term>
1331    <listitem><para>Additional flags passed to
1332    <command>make</command>.</para></listitem>
1333  </varlistentry>
1334
1335  <varlistentry>
1336    <term><varname>tarballs</varname></term>
1337    <listitem><para>The names of the source distribution files to be
1338    copied to <filename>$out/tarballs/</filename>.  It can contain
1339    shell wildcards.  The default is
1340    <filename>*.tar.gz</filename>.</para></listitem>
1341  </varlistentry>
1342
1343  <varlistentry>
1344    <term><varname>dontCopyDist</varname></term>
1345    <listitem><para>If set, no files are copied to
1346    <filename>$out/tarballs/</filename>.</para></listitem>
1347  </varlistentry>
1348
1349  <varlistentry>
1350    <term><varname>preDist</varname></term>
1351    <listitem><para>Hook executed at the start of the distribution
1352    phase.</para></listitem>
1353  </varlistentry>
1354
1355  <varlistentry>
1356    <term><varname>postDist</varname></term>
1357    <listitem><para>Hook executed at the end of the distribution
1358    phase.</para></listitem>
1359  </varlistentry>
1360
1361</variablelist>
1362
1363
1364</section>
1365
1366
1367</section>
1368
1369
1370<section xml:id="ssec-stdenv-functions"><title>Shell functions</title>
1371
1372<para>The standard environment provides a number of useful
1373functions.</para>
1374
1375<variablelist>
1376
1377
1378  <varlistentry xml:id='fun-makeWrapper'>
1379    <term><function>makeWrapper</function>
1380    <replaceable>executable</replaceable>
1381    <replaceable>wrapperfile</replaceable>
1382    <replaceable>args</replaceable></term>
1383    <listitem><para>Constructs a wrapper for a program with various
1384    possible arguments. For example:
1385
1386<programlisting>
1387# adds `FOOBAR=baz` to `$out/bin/foo`’s environment
1388makeWrapper $out/bin/foo $wrapperfile --set FOOBAR baz
1389
1390# prefixes the binary paths of `hello` and `git`
1391# Be advised that paths often should be patched in directly
1392# (via string replacements or in `configurePhase`).
1393makeWrapper $out/bin/foo $wrapperfile --prefix PATH : ${lib.makeBinPath [ hello git ]}
1394</programlisting>
1395
1396    There’s many more kinds of arguments, they are documented in
1397    <literal>nixpkgs/pkgs/build-support/setup-hooks/make-wrapper.sh</literal>.</para>
1398
1399    <para><literal>wrapProgram</literal> is a convenience function you probably
1400    want to use most of the time.</para>
1401
1402    </listitem>
1403  </varlistentry>
1404
1405
1406  <varlistentry xml:id='fun-substitute'>
1407    <term><function>substitute</function>
1408    <replaceable>infile</replaceable>
1409    <replaceable>outfile</replaceable>
1410    <replaceable>subs</replaceable></term>
1411
1412    <listitem>
1413      <para>Performs string substitution on the contents of
1414      <replaceable>infile</replaceable>, writing the result to
1415      <replaceable>outfile</replaceable>.  The substitutions in
1416      <replaceable>subs</replaceable> are of the following form:
1417
1418        <variablelist>
1419          <varlistentry>
1420            <term><option>--replace</option>
1421            <replaceable>s1</replaceable>
1422            <replaceable>s2</replaceable></term>
1423            <listitem><para>Replace every occurrence of the string
1424            <replaceable>s1</replaceable> by
1425            <replaceable>s2</replaceable>.</para></listitem>
1426          </varlistentry>
1427
1428          <varlistentry>
1429            <term><option>--subst-var</option>
1430            <replaceable>varName</replaceable></term>
1431            <listitem><para>Replace every occurrence of
1432            <literal>@<replaceable>varName</replaceable>@</literal> by
1433            the contents of the environment variable
1434            <replaceable>varName</replaceable>.  This is useful for
1435            generating files from templates, using
1436            <literal>@<replaceable>...</replaceable>@</literal> in the
1437            template as placeholders.</para></listitem>
1438          </varlistentry>
1439
1440          <varlistentry>
1441            <term><option>--subst-var-by</option>
1442            <replaceable>varName</replaceable>
1443            <replaceable>s</replaceable></term>
1444            <listitem><para>Replace every occurrence of
1445            <literal>@<replaceable>varName</replaceable>@</literal> by
1446            the string <replaceable>s</replaceable>.</para></listitem>
1447          </varlistentry>
1448
1449        </variablelist>
1450
1451      </para>
1452
1453      <para>Example:
1454
1455<programlisting>
1456substitute ./foo.in ./foo.out \
1457    --replace /usr/bin/bar $bar/bin/bar \
1458    --replace "a string containing spaces" "some other text" \
1459    --subst-var someVar
1460</programlisting>
1461
1462      </para>
1463
1464      <para><function>substitute</function> is implemented using the
1465      <command
1466      xlink:href="http://replace.richardlloyd.org.uk/">replace</command>
1467      command.  Unlike with the <command>sed</command> command, you
1468      don’t have to worry about escaping special characters.  It
1469      supports performing substitutions on binary files (such as
1470      executables), though there you’ll probably want to make sure
1471      that the replacement string is as long as the replaced
1472      string.</para>
1473
1474    </listitem>
1475  </varlistentry>
1476
1477
1478  <varlistentry xml:id='fun-substituteInPlace'>
1479    <term><function>substituteInPlace</function>
1480    <replaceable>file</replaceable>
1481    <replaceable>subs</replaceable></term>
1482    <listitem><para>Like <function>substitute</function>, but performs
1483    the substitutions in place on the file
1484    <replaceable>file</replaceable>.</para></listitem>
1485  </varlistentry>
1486
1487
1488  <varlistentry xml:id='fun-substituteAll'>
1489    <term><function>substituteAll</function>
1490    <replaceable>infile</replaceable>
1491    <replaceable>outfile</replaceable></term>
1492    <listitem><para>Replaces every occurrence of
1493    <literal>@<replaceable>varName</replaceable>@</literal>, where
1494    <replaceable>varName</replaceable> is any environment variable, in
1495    <replaceable>infile</replaceable>, writing the result to
1496    <replaceable>outfile</replaceable>.  For instance, if
1497    <replaceable>infile</replaceable> has the contents
1498
1499<programlisting>
1500#! @bash@/bin/sh
1501PATH=@coreutils@/bin
1502echo @foo@
1503</programlisting>
1504
1505    and the environment contains
1506    <literal>bash=/nix/store/bmwp0q28cf21...-bash-3.2-p39</literal>
1507    and
1508    <literal>coreutils=/nix/store/68afga4khv0w...-coreutils-6.12</literal>,
1509    but does not contain the variable <varname>foo</varname>, then the
1510    output will be
1511
1512<programlisting>
1513#! /nix/store/bmwp0q28cf21...-bash-3.2-p39/bin/sh
1514PATH=/nix/store/68afga4khv0w...-coreutils-6.12/bin
1515echo @foo@
1516</programlisting>
1517
1518    That is, no substitution is performed for undefined variables.</para>
1519
1520    <para>Environment variables that start with an uppercase letter or an
1521    underscore are filtered out,
1522    to prevent global variables (like <literal>HOME</literal>) or private
1523    variables (like <literal>__ETC_PROFILE_DONE</literal>) from accidentally
1524    getting substituted.
1525    The variables also have to be valid bash “names”, as
1526    defined in the bash manpage (alphanumeric or <literal>_</literal>,
1527    must not start with a number).</para>
1528  </listitem>
1529  </varlistentry>
1530
1531
1532  <varlistentry xml:id='fun-substituteAllInPlace'>
1533    <term><function>substituteAllInPlace</function>
1534    <replaceable>file</replaceable></term>
1535    <listitem><para>Like <function>substituteAll</function>, but performs
1536    the substitutions in place on the file
1537    <replaceable>file</replaceable>.</para></listitem>
1538  </varlistentry>
1539
1540
1541  <varlistentry xml:id='fun-stripHash'>
1542    <term><function>stripHash</function>
1543    <replaceable>path</replaceable></term>
1544    <listitem><para>Strips the directory and hash part of a store
1545    path, outputting the name part to <literal>stdout</literal>.
1546    For example:
1547
1548<programlisting>
1549# prints coreutils-8.24
1550stripHash "/nix/store/9s9r019176g7cvn2nvcw41gsp862y6b4-coreutils-8.24"
1551</programlisting>
1552
1553    If you wish to store the result in another variable, then the
1554    following idiom may be useful:
1555
1556<programlisting>
1557name="/nix/store/9s9r019176g7cvn2nvcw41gsp862y6b4-coreutils-8.24"
1558someVar=$(stripHash $name)
1559</programlisting>
1560
1561    </para></listitem>
1562  </varlistentry>
1563
1564
1565  <varlistentry xml:id='fun-wrapProgram'>
1566    <term><function>wrapProgram</function>
1567    <replaceable>executable</replaceable>
1568    <replaceable>makeWrapperArgs</replaceable></term>
1569    <listitem><para>Convenience function for <literal>makeWrapper</literal>
1570    that automatically creates a sane wrapper file
1571
1572    It takes all the same arguments as <literal>makeWrapper</literal>,
1573    except for <literal>--argv0</literal>.</para>
1574
1575    <para>It cannot be applied multiple times, since it will overwrite the wrapper
1576    file.</para>
1577    </listitem>
1578  </varlistentry>
1579
1580
1581</variablelist>
1582
1583</section>
1584
1585
1586<section xml:id="ssec-setup-hooks"><title>Package setup hooks</title>
1587
1588<para>
1589  Nix itself considers a build-time dependency merely something that should previously be built and accessible at build time—packages themselves are on their own to perform any additional setup.
1590  In most cases, that is fine, and the downstream derivation can deal with it's own dependencies.
1591  But for a few common tasks, that would result in almost every package doing the same sort of setup work---depending not on the package itself, but entirely on which dependencies were used.
1592</para>
1593<para>
1594  In order to alleviate this burden, the <firstterm>setup hook></firstterm>mechanism was written, where any package can include a shell script that [by convention rather than enforcement by Nix], any downstream reverse-dependency will source as part of its build process.
1595  That allows the downstream dependency to merely specify its dependencies, and lets those dependencies effectively initialize themselves.
1596  No boilerplate mirroring the list of dependencies is needed.
1597</para>
1598<para>
1599  The Setup hook mechanism is a bit of a sledgehammer though: a powerful feature with a broad and indiscriminate area of effect.
1600  The combination of its power and implicit use may be expedient, but isn't without costs.
1601  Nix itself is unchanged, but the spirit of adding dependencies being effect-free is violated even if the letter isn't.
1602  For example, if a derivation path is mentioned more than once, Nix itself doesn't care and simply makes sure the dependency derivation is already built just the same—depending is just needing something to exist, and needing is idempotent.
1603  However, a dependency specified twice will have its setup hook run twice, and that could easily change the build environment (though a well-written setup hook will therefore strive to be idempotent so this is in fact not observable).
1604  More broadly, setup hooks are anti-modular in that multiple dependencies, whether the same or different, should not interfere and yet their setup hooks may well do so.
1605</para>
1606<para>
1607  The most typical use of the setup hook is actually to add other hooks which are then run (i.e. after all the setup hooks) on each dependency.
1608  For example, the C compiler wrapper's setup hook feeds itself flags for each dependency that contains relevant libaries and headers.
1609  This is done by defining a bash function, and appending its name to one of
1610  <envar>envBuildBuildHooks</envar>`,
1611  <envar>envBuildHostHooks</envar>`,
1612  <envar>envBuildTargetHooks</envar>`,
1613  <envar>envHostHostHooks</envar>`,
1614  <envar>envHostTargetHooks</envar>`, or
1615  <envar>envTargetTargetHooks</envar>`.
1616  These 6 bash variables correspond to the 6 sorts of dependencies by platform (there's 12 total but we ignore the propagated/non-propagated axis).
1617</para>
1618<para>
1619  Packages adding a hook should not hard code a specific hook, but rather choose a variable <emphasis>relative</emphasis> to how they are included.
1620  Returning to the C compiler wrapper example, if it itself is an <literal>n</literal> dependency, then it only wants to accumulate flags from <literal>n + 1</literal> dependencies, as only those ones match the compiler's target platform.
1621  The <envar>hostOffset</envar> variable is defined with the current dependency's host offset <envar>targetOffset</envar> with its target offset, before it's setup hook is sourced.
1622  Additionally, since most environment hooks don't care about the target platform,
1623  That means the setup hook can append to the right bash array by doing something like
1624  <programlisting language="bash">
1625addEnvHooks "$hostOffset" myBashFunction
1626  </programlisting>
1627</para>
1628<para>
1629  The <emphasis>existence</emphasis> of setups hooks has long been documented and packages inside Nixpkgs are free to use these mechanism.
1630  Other packages, however, should not rely on these mechanisms not changing between Nixpkgs versions.
1631  Because of the existing issues with this system, there's little benefit from mandating it be stable for any period of time.
1632</para>
1633<para>
1634  Here are some packages that provide a setup hook.
1635  Since the mechanism is modular, this probably isn't an exhaustive list.
1636  Then again, since the mechanism is only to be used as a last resort, it might be.
1637<variablelist>
1638
1639  <varlistentry>
1640    <term>Bintools Wrapper</term>
1641    <listitem>
1642      <para>
1643        Bintools Wrapper wraps the binary utilities for a bunch of miscellaneous purposes.
1644        These are GNU Binutils when targetting Linux, and a mix of cctools and GNU binutils for Darwin.
1645        [The "Bintools" name is supposed to be a compromise between "Binutils" and "cctools" not denoting any specific implementation.]
1646        Specifically, the underlying bintools package, and a C standard library (glibc or Darwin's libSystem, just for the dynamic loader) are all fed in, and dependency finding, hardening (see below), and purity checks for each are handled by Bintools Wrapper.
1647        Packages typically depend on CC Wrapper, which in turn (at run time) depends on Bintools Wrapper.
1648      </para>
1649      <para>
1650        Bintools Wrapper was only just recently split off from CC Wrapper, so the division of labor is still being worked out.
1651        For example, it shouldn't care about about the C standard library, but just take a derivation with the dynamic loader (which happens to be the glibc on linux).
1652        Dependency finding however is a task both wrappers will continue to need to share, and probably the most important to understand.
1653        It is currently accomplished by collecting directories of host-platform dependencies (i.e. <varname>buildInputs</varname> and <varname>nativeBuildInputs</varname>) in environment variables.
1654        Bintools Wrapper's setup hook causes any <filename>lib</filename> and <filename>lib64</filename> subdirectories to be added to <envar>NIX_LDFLAGS</envar>.
1655        Since CC Wrapper and Bintools Wrapper use the same strategy, most of the Bintools Wrapper code is sparsely commented and refers to CC Wrapper.
1656        But CC Wrapper's code, by contrast, has quite lengthy comments.
1657        Bintools Wrapper merely cites those, rather than repeating them, to avoid falling out of sync.
1658      </para>
1659      <para>
1660        A final task of the setup hook is defining a number of standard environment variables to tell build systems which executables full-fill which purpose.
1661        They are defined to just be the base name of the tools, under the assumption that Bintools Wrapper's binaries will be on the path.
1662        Firstly, this helps poorly-written packages, e.g. ones that look for just <command>gcc</command> when <envar>CC</envar> isn't defined yet <command>clang</command> is to be used.
1663        Secondly, this helps packages not get confused when cross-compiling, in which case multiple Bintools Wrappers may simultaneously be in use.
1664        <footnote><para>
1665          Each wrapper targets a single platform, so if binaries for multiple platforms are needed, the underlying binaries must be wrapped multiple times.
1666          As this is a property of the wrapper itself, the multiple wrappings are needed whether or not the same underlying binaries can target multiple platforms.
1667        </para></footnote>
1668        <envar>BUILD_</envar>- and <envar>TARGET_</envar>-prefixed versions of the normal environment variable are defined for the additional Bintools Wrappers, properly disambiguating them.
1669      </para>
1670      <para>
1671        A problem with this final task is that Bintools Wrapper is honest and defines <envar>LD</envar> as <command>ld</command>.
1672        Most packages, however, firstly use the C compiler for linking, secondly use <envar>LD</envar> anyways, defining it as the C compiler, and thirdly, only so define <envar>LD</envar> when it is undefined as a fallback.
1673        This triple-threat means Bintools Wrapper will break those packages, as LD is already defined as the actual linker which the package won't override yet doesn't want to use.
1674        The workaround is to define, just for the problematic package, <envar>LD</envar> as the C compiler.
1675        A good way to do this would be <command>preConfigure = "LD=$CC"</command>.
1676      </para>
1677    </listitem>
1678  </varlistentry>
1679
1680  <varlistentry>
1681    <term>CC Wrapper</term>
1682    <listitem>
1683      <para>
1684        CC Wrapper wraps a C toolchain for a bunch of miscellaneous purposes.
1685        Specifically, a C compiler (GCC or Clang), wrapped binary tools, and a C standard library (glibc or Darwin's libSystem, just for the dynamic loader) are all fed in, and dependency finding, hardening (see below), and purity checks for each are handled by CC Wrapper.
1686        Packages typically depend on CC Wrapper, which in turn (at run time) depends on Bintools Wrapper.
1687      </para>
1688      <para>
1689        Dependency finding is undoubtedly the main task of CC Wrapper.
1690        This works just like Bintools Wrapper, except that any <filename>include</filename> subdirectory of any relevant dependency is added to <envar>NIX_CFLAGS_COMPILE</envar>.
1691        The setup hook itself contains some lengthy comments describing the exact convoluted mechanism by which this is accomplished.
1692      </para>
1693      <para>
1694        CC Wrapper also like Bintools Wrapper defines standard environment variables with the names of the tools it wraps, for the same reasons described above.
1695        Importantly, while it includes a <command>cc</command> symlink to the c compiler for portability, the <envar>CC</envar> will be defined using the compiler's "real name" (i.e. <command>gcc</command> or <command>clang</command>).
1696        This helps lousy build systems that inspect on the name of the compiler rather than run it.
1697      </para>
1698    </listitem>
1699  </varlistentry>
1700
1701  <varlistentry>
1702    <term>Perl</term>
1703    <listitem>
1704      <para>
1705        Adds the <filename>lib/site_perl</filename> subdirectory of each build input to the <envar>PERL5LIB</envar> environment variable.
1706        For instance, if <varname>buildInputs</varname> contains Perl, then the <filename>lib/site_perl</filename> subdirectory of each input is added to the <envar>PERL5LIB</envar> environment variable.
1707      </para>
1708    </listitem>
1709  </varlistentry>
1710
1711  <varlistentry>
1712    <term>Python</term>
1713    <listitem><para>Adds the
1714    <filename>lib/${python.libPrefix}/site-packages</filename> subdirectory of
1715    each build input to the <envar>PYTHONPATH</envar> environment
1716    variable.</para></listitem>
1717  </varlistentry>
1718
1719  <varlistentry>
1720    <term>pkg-config</term>
1721    <listitem><para>Adds the <filename>lib/pkgconfig</filename> and
1722    <filename>share/pkgconfig</filename> subdirectories of each
1723    build input to the <envar>PKG_CONFIG_PATH</envar> environment
1724    variable.</para></listitem>
1725  </varlistentry>
1726
1727  <varlistentry>
1728    <term>Automake</term>
1729    <listitem><para>Adds the <filename>share/aclocal</filename>
1730    subdirectory of each build input to the <envar>ACLOCAL_PATH</envar>
1731    environment variable.</para></listitem>
1732  </varlistentry>
1733
1734  <varlistentry>
1735    <term>Autoconf</term>
1736    <listitem><para>The <varname>autoreconfHook</varname> derivation adds
1737    <varname>autoreconfPhase</varname>, which runs autoreconf, libtoolize and
1738    automake, essentially preparing the configure script in autotools-based
1739    builds.</para></listitem>
1740  </varlistentry>
1741
1742  <varlistentry>
1743    <term>libxml2</term>
1744    <listitem><para>Adds every file named
1745    <filename>catalog.xml</filename> found under the
1746    <filename>xml/dtd</filename> and <filename>xml/xsl</filename>
1747    subdirectories of each build input to the
1748    <envar>XML_CATALOG_FILES</envar> environment
1749    variable.</para></listitem>
1750  </varlistentry>
1751
1752  <varlistentry>
1753    <term>teTeX / TeX Live</term>
1754    <listitem><para>Adds the <filename>share/texmf-nix</filename>
1755    subdirectory of each build input to the <envar>TEXINPUTS</envar>
1756    environment variable.</para></listitem>
1757  </varlistentry>
1758
1759  <varlistentry>
1760    <term>Qt 4</term>
1761    <listitem><para>Sets the <envar>QTDIR</envar> environment variable
1762    to Qt’s path.</para></listitem>
1763  </varlistentry>
1764
1765  <varlistentry>
1766    <term>gdk-pixbuf</term>
1767    <listitem><para>Exports <envar>GDK_PIXBUF_MODULE_FILE</envar>
1768    environment variable the the builder. Add librsvg package
1769    to <varname>buildInputs</varname> to get svg support.</para></listitem>
1770  </varlistentry>
1771
1772  <varlistentry>
1773    <term>GHC</term>
1774    <listitem><para>Creates a temporary package database and registers
1775    every Haskell build input in it (TODO: how?).</para></listitem>
1776  </varlistentry>
1777
1778  <varlistentry>
1779    <term>GStreamer</term>
1780    <listitem><para>Adds the
1781    GStreamer plugins subdirectory of
1782    each build input to the <envar>GST_PLUGIN_SYSTEM_PATH_1_0</envar> or
1783    <envar>GST_PLUGIN_SYSTEM_PATH</envar> environment variable.</para></listitem>
1784  </varlistentry>
1785
1786  <varlistentry>
1787    <term>paxctl</term>
1788    <listitem><para>Defines the <varname>paxmark</varname> helper for
1789    setting per-executable PaX flags on Linux (where it is available by
1790    default; on all other platforms, <varname>paxmark</varname> is a no-op).
1791    For example, to disable secure memory protections on the executable
1792    <replaceable>foo</replaceable>:
1793    <programlisting>
1794      postFixup = ''
1795        paxmark m $out/bin/<replaceable>foo</replaceable>
1796      '';
1797    </programlisting>
1798    The <literal>m</literal> flag is the most common flag and is typically
1799    required for applications that employ JIT compilation or otherwise need to
1800    execute code generated at run-time.  Disabling PaX protections should be
1801    considered a last resort: if possible, problematic features should be
1802    disabled or patched to work with PaX.</para></listitem>
1803  </varlistentry>
1804
1805  <varlistentry>
1806    <term>autoPatchelfHook</term>
1807    <listitem><para>This is a special setup hook which helps in packaging
1808    proprietary software in that it automatically tries to find missing shared
1809    library dependencies of ELF files. All packages within the
1810    <envar>runtimeDependencies</envar> environment variable are unconditionally
1811    added to executables, which is useful for programs that use
1812    <citerefentry>
1813      <refentrytitle>dlopen</refentrytitle>
1814      <manvolnum>3</manvolnum>
1815    </citerefentry>
1816    to load libraries at runtime.</para></listitem>
1817  </varlistentry>
1818
1819</variablelist>
1820
1821</para>
1822
1823</section>
1824
1825
1826<section xml:id="sec-purity-in-nixpkgs"><title>Purity in Nixpkgs</title>
1827
1828<para>[measures taken to prevent dependencies on packages outside the
1829store, and what you can do to prevent them]</para>
1830
1831<para>GCC doesn't search in locations such as
1832<filename>/usr/include</filename>.  In fact, attempts to add such
1833directories through the <option>-I</option> flag are filtered out.
1834Likewise, the linker (from GNU binutils) doesn't search in standard
1835locations such as <filename>/usr/lib</filename>.  Programs built on
1836Linux are linked against a GNU C Library that likewise doesn't search
1837in the default system locations.</para>
1838
1839</section>
1840
1841<section xml:id="sec-hardening-in-nixpkgs"><title>Hardening in Nixpkgs</title>
1842
1843<para>There are flags available to harden packages at compile or link-time.
1844These can be toggled using the <varname>stdenv.mkDerivation</varname> parameters
1845<varname>hardeningDisable</varname> and <varname>hardeningEnable</varname>.
1846</para>
1847
1848<para>
1849Both parameters take a list of flags as strings. The special
1850<varname>"all"</varname> flag can be passed to <varname>hardeningDisable</varname>
1851to turn off all hardening. These flags can also be used as environment variables
1852for testing or development purposes.
1853</para>
1854
1855<para>The following flags are enabled by default and might require disabling with
1856<varname>hardeningDisable</varname> if the program to package is incompatible.
1857</para>
1858
1859<variablelist>
1860
1861  <varlistentry>
1862    <term><varname>format</varname></term>
1863    <listitem><para>Adds the <option>-Wformat -Wformat-security
1864    -Werror=format-security</option> compiler options. At present,
1865    this warns about calls to <varname>printf</varname> and
1866    <varname>scanf</varname> functions where the format string is
1867    not a string literal and there are no format arguments, as in
1868    <literal>printf(foo);</literal>. This may be a security hole
1869    if the format string came from untrusted input and contains
1870    <literal>%n</literal>.</para>
1871
1872    <para>This needs to be turned off or fixed for errors similar to:</para>
1873
1874    <programlisting>
1875/tmp/nix-build-zynaddsubfx-2.5.2.drv-0/zynaddsubfx-2.5.2/src/UI/guimain.cpp:571:28: error: format not a string literal and no format arguments [-Werror=format-security]
1876         printf(help_message);
1877                            ^
1878cc1plus: some warnings being treated as errors
1879    </programlisting></listitem>
1880  </varlistentry>
1881
1882  <varlistentry>
1883    <term><varname>stackprotector</varname></term>
1884    <listitem>
1885    <para>Adds the <option>-fstack-protector-strong
1886    --param ssp-buffer-size=4</option>
1887    compiler options. This adds safety checks against stack overwrites
1888    rendering many potential code injection attacks into aborting situations.
1889    In the best case this turns code injection vulnerabilities into denial
1890    of service or into non-issues (depending on the application).</para>
1891
1892    <para>This needs to be turned off or fixed for errors similar to:</para>
1893
1894    <programlisting>
1895bin/blib.a(bios_console.o): In function `bios_handle_cup':
1896/tmp/nix-build-ipxe-20141124-5cbdc41.drv-0/ipxe-5cbdc41/src/arch/i386/firmware/pcbios/bios_console.c:86: undefined reference to `__stack_chk_fail'
1897    </programlisting></listitem>
1898  </varlistentry>
1899
1900  <varlistentry>
1901    <term><varname>fortify</varname></term>
1902    <listitem>
1903    <para>Adds the <option>-O2 -D_FORTIFY_SOURCE=2</option> compiler
1904    options. During code generation the compiler knows a great deal of
1905    information about buffer sizes (where possible), and attempts to replace
1906    insecure unlimited length buffer function calls with length-limited ones.
1907    This is especially useful for old, crufty code. Additionally, format
1908    strings in writable memory that contain '%n' are blocked. If an application
1909    depends on such a format string, it will need to be worked around.
1910    </para>
1911
1912    <para>Additionally, some warnings are enabled which might trigger build
1913    failures if compiler warnings are treated as errors in the package build.
1914    In this case, set <option>NIX_CFLAGS_COMPILE</option> to
1915    <option>-Wno-error=warning-type</option>.</para>
1916
1917    <para>This needs to be turned off or fixed for errors similar to:</para>
1918
1919    <programlisting>
1920malloc.c:404:15: error: return type is an incomplete type
1921malloc.c:410:19: error: storage size of 'ms' isn't known
1922    </programlisting>
1923    <programlisting>
1924strdup.h:22:1: error: expected identifier or '(' before '__extension__'
1925    </programlisting>
1926    <programlisting>
1927strsep.c:65:23: error: register name not specified for 'delim'
1928    </programlisting>
1929    <programlisting>
1930installwatch.c:3751:5: error: conflicting types for '__open_2'
1931    </programlisting>
1932    <programlisting>
1933fcntl2.h:50:4: error: call to '__open_missing_mode' declared with attribute error: open with O_CREAT or O_TMPFILE in second argument needs 3 arguments
1934    </programlisting>
1935    </listitem>
1936  </varlistentry>
1937
1938  <varlistentry>
1939    <term><varname>pic</varname></term>
1940    <listitem>
1941    <para>Adds the <option>-fPIC</option> compiler options. This options adds
1942    support for position independent code in shared libraries and thus making
1943    ASLR possible.</para>
1944    <para>Most notably, the Linux kernel, kernel modules and other code
1945    not running in an operating system environment like boot loaders won't
1946    build with PIC enabled. The compiler will is most cases complain that
1947    PIC is not supported for a specific build.
1948    </para>
1949
1950    <para>This needs to be turned off or fixed for assembler errors similar to:</para>
1951
1952    <programlisting>
1953ccbLfRgg.s: Assembler messages:
1954ccbLfRgg.s:33: Error: missing or invalid displacement expression `private_key_len@GOTOFF'
1955    </programlisting>
1956    </listitem>
1957  </varlistentry>
1958
1959  <varlistentry>
1960    <term><varname>strictoverflow</varname></term>
1961    <listitem>
1962    <para>Signed integer overflow is undefined behaviour according to the C
1963    standard. If it happens, it is an error in the program as it should check
1964    for overflow before it can happen, not afterwards. GCC provides built-in
1965    functions to perform arithmetic with overflow checking, which are correct
1966    and faster than any custom implementation. As a workaround, the option
1967    <option>-fno-strict-overflow</option> makes gcc behave as if signed
1968    integer overflows were defined.
1969    </para>
1970
1971    <para>This flag should not trigger any build or runtime errors.</para>
1972    </listitem>
1973  </varlistentry>
1974
1975  <varlistentry>
1976    <term><varname>relro</varname></term>
1977    <listitem>
1978    <para>Adds the <option>-z relro</option> linker option. During program
1979    load, several ELF memory sections need to be written to by the linker,
1980    but can be turned read-only before turning over control to the program.
1981    This prevents some GOT (and .dtors) overwrite attacks, but at least the
1982    part of the GOT used by the dynamic linker (.got.plt) is still vulnerable.
1983    </para>
1984
1985    <para>This flag can break dynamic shared object loading. For instance, the
1986    module systems of Xorg and OpenCV are incompatible with this flag. In almost
1987    all cases the <varname>bindnow</varname> flag must also be disabled and
1988    incompatible programs typically fail with similar errors at runtime.</para>
1989    </listitem>
1990  </varlistentry>
1991
1992  <varlistentry>
1993    <term><varname>bindnow</varname></term>
1994    <listitem>
1995    <para>Adds the <option>-z bindnow</option> linker option. During program
1996    load, all dynamic symbols are resolved, allowing for the complete GOT to
1997    be marked read-only (due to <varname>relro</varname>). This prevents GOT
1998    overwrite attacks. For very large applications, this can incur some
1999    performance loss during initial load while symbols are resolved, but this
2000    shouldn't be an issue for daemons.
2001    </para>
2002
2003    <para>This flag can break dynamic shared object loading. For instance, the
2004    module systems of Xorg and PHP are incompatible with this flag. Programs
2005    incompatible with this flag often fail at runtime due to missing symbols,
2006    like:</para>
2007
2008    <programlisting>
2009intel_drv.so: undefined symbol: vgaHWFreeHWRec
2010    </programlisting>
2011    </listitem>
2012  </varlistentry>
2013
2014</variablelist>
2015
2016<para>The following flags are disabled by default and should be enabled
2017with <varname>hardeningEnable</varname> for packages that take untrusted
2018input like network services.
2019</para>
2020
2021<variablelist>
2022
2023  <varlistentry>
2024    <term><varname>pie</varname></term>
2025    <listitem>
2026    <para>Adds the <option>-fPIE</option> compiler and <option>-pie</option>
2027    linker options. Position Independent Executables are needed to take
2028    advantage of Address Space Layout Randomization, supported by modern
2029    kernel versions. While ASLR can already be enforced for data areas in
2030    the stack and heap (brk and mmap), the code areas must be compiled as
2031    position-independent. Shared libraries already do this with the
2032    <varname>pic</varname> flag, so they gain ASLR automatically, but binary
2033    .text regions need to be build with <varname>pie</varname> to gain ASLR.
2034    When this happens, ROP attacks are much harder since there are no static
2035    locations to bounce off of during a memory corruption attack.
2036    </para>
2037    </listitem>
2038  </varlistentry>
2039
2040</variablelist>
2041
2042<para>For more in-depth information on these hardening flags and hardening in
2043general, refer to the
2044<link xlink:href="https://wiki.debian.org/Hardening">Debian Wiki</link>,
2045<link xlink:href="https://wiki.ubuntu.com/Security/Features">Ubuntu Wiki</link>,
2046<link xlink:href="https://wiki.gentoo.org/wiki/Project:Hardened">Gentoo Wiki</link>,
2047and the <link xlink:href="https://wiki.archlinux.org/index.php/DeveloperWiki:Security">
2048Arch Wiki</link>.
2049</para>
2050
2051</section>
2052
2053</chapter>