pyrox.dev
1<chapter xmlns="http://docbook.org/ns/docbook"
2 xmlns:xlink="http://www.w3.org/1999/xlink"
3 xml:id="chap-cross">
4 <title>Cross-compilation</title>
5 <section xml:id="sec-cross-intro">
6 <title>Introduction</title>
7
8 <para>
9 "Cross-compilation" means compiling a program on one machine for another
10 type of machine. For example, a typical use of cross compilation is to
11 compile programs for embedded devices. These devices often don't have the
12 computing power and memory to compile their own programs. One might think
13 that cross-compilation is a fairly niche concern, but there are advantages
14 to being rigorous about distinguishing build-time vs run-time environments
15 even when one is developing and deploying on the same machine. Nixpkgs is
16 increasingly adopting the opinion that packages should be written with
17 cross-compilation in mind, and nixpkgs should evaluate in a similar way (by
18 minimizing cross-compilation-specific special cases) whether or not one is
19 cross-compiling.
20 </para>
21
22 <para>
23 This chapter will be organized in three parts. First, it will describe the
24 basics of how to package software in a way that supports cross-compilation.
25 Second, it will describe how to use Nixpkgs when cross-compiling. Third, it
26 will describe the internal infrastructure supporting cross-compilation.
27 </para>
28 </section>
29<!--============================================================-->
30 <section xml:id="sec-cross-packaging">
31 <title>Packaging in a cross-friendly manner</title>
32
33 <section xml:id="sec-cross-platform-parameters">
34 <title>Platform parameters</title>
35
36 <para>
37 Nixpkgs follows the
38 <link xlink:href="https://gcc.gnu.org/onlinedocs/gccint/Configure-Terms.html">common
39 historical convention of GNU autoconf</link> of distinguishing between 3
40 types of platform: <wordasword>build</wordasword>,
41 <wordasword>host</wordasword>, and <wordasword>target</wordasword>. In
42 summary, <wordasword>build</wordasword> is the platform on which a package
43 is being built, <wordasword>host</wordasword> is the platform on which it
44 is to run. The third attribute, <wordasword>target</wordasword>, is
45 relevant only for certain specific compilers and build tools.
46 </para>
47
48 <para>
49 In Nixpkgs, these three platforms are defined as attribute sets under the
50 names <literal>buildPlatform</literal>, <literal>hostPlatform</literal>,
51 and <literal>targetPlatform</literal>. All three are always defined as
52 attributes in the standard environment, and at the top level. That means
53 one can get at them just like a dependency in a function that is imported
54 with <literal>callPackage</literal>:
55<programlisting>{ stdenv, buildPlatform, hostPlatform, fooDep, barDep, .. }: ...buildPlatform...</programlisting>
56 , or just off <varname>stdenv</varname>:
57<programlisting>{ stdenv, fooDep, barDep, .. }: ...stdenv.buildPlatform...</programlisting>
58 .
59 </para>
60
61 <variablelist>
62 <varlistentry>
63 <term>
64 <varname>buildPlatform</varname>
65 </term>
66 <listitem>
67 <para>
68 The "build platform" is the platform on which a package is built. Once
69 someone has a built package, or pre-built binary package, the build
70 platform should not matter and be safe to ignore.
71 </para>
72 </listitem>
73 </varlistentry>
74 <varlistentry>
75 <term>
76 <varname>hostPlatform</varname>
77 </term>
78 <listitem>
79 <para>
80 The "host platform" is the platform on which a package will be run. This
81 is the simplest platform to understand, but also the one with the worst
82 name.
83 </para>
84 </listitem>
85 </varlistentry>
86 <varlistentry>
87 <term>
88 <varname>targetPlatform</varname>
89 </term>
90 <listitem>
91 <para>
92 The "target platform" attribute is, unlike the other two attributes, not
93 actually fundamental to the process of building software. Instead, it is
94 only relevant for compatibility with building certain specific compilers
95 and build tools. It can be safely ignored for all other packages.
96 </para>
97 <para>
98 The build process of certain compilers is written in such a way that the
99 compiler resulting from a single build can itself only produce binaries
100 for a single platform. The task specifying this single "target platform"
101 is thus pushed to build time of the compiler. The root cause of this
102 mistake is often that the compiler (which will be run on the host) and
103 the the standard library/runtime (which will be run on the target) are
104 built by a single build process.
105 </para>
106 <para>
107 There is no fundamental need to think about a single target ahead of
108 time like this. If the tool supports modular or pluggable backends, both
109 the need to specify the target at build time and the constraint of
110 having only a single target disappear. An example of such a tool is
111 LLVM.
112 </para>
113 <para>
114 Although the existence of a "target platfom" is arguably a historical
115 mistake, it is a common one: examples of tools that suffer from it are
116 GCC, Binutils, GHC and Autoconf. Nixpkgs tries to avoid sharing in the
117 mistake where possible. Still, because the concept of a target platform
118 is so ingrained, it is best to support it as is.
119 </para>
120 </listitem>
121 </varlistentry>
122 </variablelist>
123
124 <para>
125 The exact schema these fields follow is a bit ill-defined due to a long and
126 convoluted evolution, but this is slowly being cleaned up. You can see
127 examples of ones used in practice in
128 <literal>lib.systems.examples</literal>; note how they are not all very
129 consistent. For now, here are few fields can count on them containing:
130 </para>
131
132 <variablelist>
133 <varlistentry>
134 <term>
135 <varname>system</varname>
136 </term>
137 <listitem>
138 <para>
139 This is a two-component shorthand for the platform. Examples of this
140 would be "x86_64-darwin" and "i686-linux"; see
141 <literal>lib.systems.doubles</literal> for more. This format isn't very
142 standard, but has built-in support in Nix, such as the
143 <varname>builtins.currentSystem</varname> impure string.
144 </para>
145 </listitem>
146 </varlistentry>
147 <varlistentry>
148 <term>
149 <varname>config</varname>
150 </term>
151 <listitem>
152 <para>
153 This is a 3- or 4- component shorthand for the platform. Examples of
154 this would be "x86_64-unknown-linux-gnu" and "aarch64-apple-darwin14".
155 This is a standard format called the "LLVM target triple", as they are
156 pioneered by LLVM and traditionally just used for the
157 <varname>targetPlatform</varname>. This format is strictly more
158 informative than the "Nix host double", as the previous format could
159 analogously be termed. This needs a better name than
160 <varname>config</varname>!
161 </para>
162 </listitem>
163 </varlistentry>
164 <varlistentry>
165 <term>
166 <varname>parsed</varname>
167 </term>
168 <listitem>
169 <para>
170 This is a nix representation of a parsed LLVM target triple with
171 white-listed components. This can be specified directly, or actually
172 parsed from the <varname>config</varname>. [Technically, only one need
173 be specified and the others can be inferred, though the precision of
174 inference may not be very good.] See
175 <literal>lib.systems.parse</literal> for the exact representation.
176 </para>
177 </listitem>
178 </varlistentry>
179 <varlistentry>
180 <term>
181 <varname>libc</varname>
182 </term>
183 <listitem>
184 <para>
185 This is a string identifying the standard C library used. Valid
186 identifiers include "glibc" for GNU libc, "libSystem" for Darwin's
187 Libsystem, and "uclibc" for µClibc. It should probably be refactored to
188 use the module system, like <varname>parse</varname>.
189 </para>
190 </listitem>
191 </varlistentry>
192 <varlistentry>
193 <term>
194 <varname>is*</varname>
195 </term>
196 <listitem>
197 <para>
198 These predicates are defined in <literal>lib.systems.inspect</literal>,
199 and slapped on every platform. They are superior to the ones in
200 <varname>stdenv</varname> as they force the user to be explicit about
201 which platform they are inspecting. Please use these instead of those.
202 </para>
203 </listitem>
204 </varlistentry>
205 <varlistentry>
206 <term>
207 <varname>platform</varname>
208 </term>
209 <listitem>
210 <para>
211 This is, quite frankly, a dumping ground of ad-hoc settings (it's an
212 attribute set). See <literal>lib.systems.platforms</literal> for
213 examples—there's hopefully one in there that will work verbatim for
214 each platform that is working. Please help us triage these flags and
215 give them better homes!
216 </para>
217 </listitem>
218 </varlistentry>
219 </variablelist>
220 </section>
221
222 <section xml:id="sec-cross-specifying-dependencies">
223 <title>Specifying Dependencies</title>
224
225 <para>
226 In this section we explore the relationship between both runtime and
227 buildtime dependencies and the 3 Autoconf platforms.
228 </para>
229
230 <para>
231 A runtime dependency between 2 packages implies that between them both the
232 host and target platforms match. This is directly implied by the meaning of
233 "host platform" and "runtime dependency": The package dependency exists
234 while both packages are running on a single host platform.
235 </para>
236
237 <para>
238 A build time dependency, however, implies a shift in platforms between the
239 depending package and the depended-on package. The meaning of a build time
240 dependency is that to build the depending package we need to be able to run
241 the depended-on's package. The depending package's build platform is
242 therefore equal to the depended-on package's host platform. Analogously,
243 the depending package's host platform is equal to the depended-on package's
244 target platform.
245 </para>
246
247 <para>
248 In this manner, given the 3 platforms for one package, we can determine the
249 three platforms for all its transitive dependencies. This is the most
250 important guiding principle behind cross-compilation with Nixpkgs, and will
251 be called the <wordasword>sliding window principle</wordasword>.
252 </para>
253
254 <para>
255 Some examples will probably make this clearer. If a package is being built
256 with a <literal>(build, host, target)</literal> platform triple of
257 <literal>(foo, bar, bar)</literal>, then its build-time dependencies would
258 have a triple of <literal>(foo, foo, bar)</literal>, and <emphasis>those
259 packages'</emphasis> build-time dependencies would have triple of
260 <literal>(foo, foo, foo)</literal>. In other words, it should take two
261 "rounds" of following build-time dependency edges before one reaches a
262 fixed point where, by the sliding window principle, the platform triple no
263 longer changes. Indeed, this happens with cross compilation, where only
264 rounds of native dependencies starting with the second necessarily coincide
265 with native packages.
266 </para>
267
268 <note>
269 <para>
270 The depending package's target platform is unconstrained by the sliding
271 window principle, which makes sense in that one can in principle build
272 cross compilers targeting arbitrary platforms.
273 </para>
274 </note>
275
276 <para>
277 How does this work in practice? Nixpkgs is now structured so that
278 build-time dependencies are taken from <varname>buildPackages</varname>,
279 whereas run-time dependencies are taken from the top level attribute set.
280 For example, <varname>buildPackages.gcc</varname> should be used at build
281 time, while <varname>gcc</varname> should be used at run time. Now, for
282 most of Nixpkgs's history, there was no <varname>buildPackages</varname>,
283 and most packages have not been refactored to use it explicitly. Instead,
284 one can use the six (<emphasis>gasp</emphasis>) attributes used for
285 specifying dependencies as documented in
286 <xref linkend="ssec-stdenv-dependencies"/>. We "splice" together the
287 run-time and build-time package sets with <varname>callPackage</varname>,
288 and then <varname>mkDerivation</varname> for each of four attributes pulls
289 the right derivation out. This splicing can be skipped when not cross
290 compiling as the package sets are the same, but is a bit slow for cross
291 compiling. Because of this, a best-of-both-worlds solution is in the works
292 with no splicing or explicit access of <varname>buildPackages</varname>
293 needed. For now, feel free to use either method.
294 </para>
295
296 <note>
297 <para>
298 There is also a "backlink" <varname>targetPackages</varname>, yielding a
299 package set whose <varname>buildPackages</varname> is the current package
300 set. This is a hack, though, to accommodate compilers with lousy build
301 systems. Please do not use this unless you are absolutely sure you are
302 packaging such a compiler and there is no other way.
303 </para>
304 </note>
305 </section>
306
307 <section xml:id="sec-cross-cookbook">
308 <title>Cross packaging cookbook</title>
309
310 <para>
311 Some frequently problems when packaging for cross compilation are good to
312 just spell and answer. Ideally the information above is exhaustive, so this
313 section cannot provide any new information, but its ludicrous and cruel to
314 expect everyone to spend effort working through the interaction of many
315 features just to figure out the same answer to the same common problem.
316 Feel free to add to this list!
317 </para>
318
319 <qandaset>
320 <qandaentry xml:id="cross-qa-build-c-program-in-build-environment">
321 <question>
322 <para>
323 What if my package's build system needs to build a C program to be run
324 under the build environment?
325 </para>
326 </question>
327 <answer>
328 <para>
329<programlisting>depsBuildBuild = [ buildPackages.stdenv.cc ];</programlisting>
330 Add it to your <function>mkDerivation</function> invocation.
331 </para>
332 </answer>
333 </qandaentry>
334 <qandaentry xml:id="cross-qa-fails-to-find-ar">
335 <question>
336 <para>
337 My package fails to find <command>ar</command>.
338 </para>
339 </question>
340 <answer>
341 <para>
342 Many packages assume that an unprefixed <command>ar</command> is
343 available, but Nix doesn't provide one. It only provides a prefixed one,
344 just as it only does for all the other binutils programs. It may be
345 necessary to patch the package to fix the build system to use a prefixed
346 `ar`.
347 </para>
348 </answer>
349 </qandaentry>
350 <qandaentry xml:id="cross-testsuite-runs-host-code">
351 <question>
352 <para>
353 My package's testsuite needs to run host platform code.
354 </para>
355 </question>
356 <answer>
357 <para>
358<programlisting>doCheck = stdenv.hostPlatform != stdenv.buildPlatfrom;</programlisting>
359 Add it to your <function>mkDerivation</function> invocation.
360 </para>
361 </answer>
362 </qandaentry>
363 </qandaset>
364 </section>
365 </section>
366<!--============================================================-->
367 <section xml:id="sec-cross-usage">
368 <title>Cross-building packages</title>
369
370 <note>
371 <para>
372 More information needs to moved from the old wiki, especially
373 <link xlink:href="https://nixos.org/wiki/CrossCompiling" />, for this
374 section.
375 </para>
376 </note>
377
378 <para>
379 Nixpkgs can be instantiated with <varname>localSystem</varname> alone, in
380 which case there is no cross compiling and everything is built by and for
381 that system, or also with <varname>crossSystem</varname>, in which case
382 packages run on the latter, but all building happens on the former. Both
383 parameters take the same schema as the 3 (build, host, and target) platforms
384 defined in the previous section. As mentioned above,
385 <literal>lib.systems.examples</literal> has some platforms which are used as
386 arguments for these parameters in practice. You can use them
387 programmatically, or on the command line:
388<programlisting>
389nix-build <nixpkgs> --arg crossSystem '(import <nixpkgs/lib>).systems.examples.fooBarBaz' -A whatever</programlisting>
390 </para>
391
392 <note>
393 <para>
394 Eventually we would like to make these platform examples an unnecessary
395 convenience so that
396<programlisting>
397nix-build <nixpkgs> --arg crossSystem.config '<arch>-<os>-<vendor>-<abi>' -A whatever</programlisting>
398 works in the vast majority of cases. The problem today is dependencies on
399 other sorts of configuration which aren't given proper defaults. We rely on
400 the examples to crudely to set those configuration parameters in some
401 vaguely sane manner on the users behalf. Issue
402 <link xlink:href="https://github.com/NixOS/nixpkgs/issues/34274">#34274</link>
403 tracks this inconvenience along with its root cause in crufty configuration
404 options.
405 </para>
406 </note>
407
408 <para>
409 While one is free to pass both parameters in full, there's a lot of logic to
410 fill in missing fields. As discussed in the previous section, only one of
411 <varname>system</varname>, <varname>config</varname>, and
412 <varname>parsed</varname> is needed to infer the other two. Additionally,
413 <varname>libc</varname> will be inferred from <varname>parse</varname>.
414 Finally, <literal>localSystem.system</literal> is also
415 <emphasis>impurely</emphasis> inferred based on the platform evaluation
416 occurs. This means it is often not necessary to pass
417 <varname>localSystem</varname> at all, as in the command-line example in the
418 previous paragraph.
419 </para>
420
421 <note>
422 <para>
423 Many sources (manual, wiki, etc) probably mention passing
424 <varname>system</varname>, <varname>platform</varname>, along with the
425 optional <varname>crossSystem</varname> to nixpkgs: <literal>import
426 <nixpkgs> { system = ..; platform = ..; crossSystem = ..;
427 }</literal>. Passing those two instead of <varname>localSystem</varname> is
428 still supported for compatibility, but is discouraged. Indeed, much of the
429 inference we do for these parameters is motivated by compatibility as much
430 as convenience.
431 </para>
432 </note>
433
434 <para>
435 One would think that <varname>localSystem</varname> and
436 <varname>crossSystem</varname> overlap horribly with the three
437 <varname>*Platforms</varname> (<varname>buildPlatform</varname>,
438 <varname>hostPlatform,</varname> and <varname>targetPlatform</varname>; see
439 <varname>stage.nix</varname> or the manual). Actually, those identifiers are
440 purposefully not used here to draw a subtle but important distinction: While
441 the granularity of having 3 platforms is necessary to properly *build*
442 packages, it is overkill for specifying the user's *intent* when making a
443 build plan or package set. A simple "build vs deploy" dichotomy is adequate:
444 the sliding window principle described in the previous section shows how to
445 interpolate between the these two "end points" to get the 3 platform triple
446 for each bootstrapping stage. That means for any package a given package
447 set, even those not bound on the top level but only reachable via
448 dependencies or <varname>buildPackages</varname>, the three platforms will
449 be defined as one of <varname>localSystem</varname> or
450 <varname>crossSystem</varname>, with the former replacing the latter as one
451 traverses build-time dependencies. A last simple difference then is
452 <varname>crossSystem</varname> should be null when one doesn't want to
453 cross-compile, while the <varname>*Platform</varname>s are always non-null.
454 <varname>localSystem</varname> is always non-null.
455 </para>
456 </section>
457<!--============================================================-->
458 <section xml:id="sec-cross-infra">
459 <title>Cross-compilation infrastructure</title>
460
461 <para>
462 To be written.
463 </para>
464
465 <note>
466 <para>
467 If one explores nixpkgs, they will see derivations with names like
468 <literal>gccCross</literal>. Such <literal>*Cross</literal> derivations is
469 a holdover from before we properly distinguished between the host and
470 target platforms —the derivation with "Cross" in the name covered the
471 <literal>build = host != target</literal> case, while the other covered the
472 <literal>host = target</literal>, with build platform the same or not based
473 on whether one was using its <literal>.nativeDrv</literal> or
474 <literal>.crossDrv</literal>. This ugliness will disappear soon.
475 </para>
476 </note>
477 </section>
478</chapter>