pyrox.dev
1<chapter xmlns="http://docbook.org/ns/docbook"
2 xmlns:xlink="http://www.w3.org/1999/xlink"
3 xml:id="chap-cross">
4
5<title>Cross-compilation</title>
6
7<section xml:id="sec-cross-intro">
8 <title>Introduction</title>
9 <para>
10 "Cross-compilation" means compiling a program on one machine for another type of machine.
11 For example, a typical use of cross compilation is to compile programs for embedded devices.
12 These devices often don't have the computing power and memory to compile their own programs.
13 One might think that cross-compilation is a fairly niche concern, but there are advantages to being rigorous about distinguishing build-time vs run-time environments even when one is developing and deploying on the same machine.
14 Nixpkgs is increasingly adopting the opinion that packages should be written with cross-compilation in mind, and nixpkgs should evaluate in a similar way (by minimizing cross-compilation-specific special cases) whether or not one is cross-compiling.
15 </para>
16
17 <para>
18 This chapter will be organized in three parts.
19 First, it will describe the basics of how to package software in a way that supports cross-compilation.
20 Second, it will describe how to use Nixpkgs when cross-compiling.
21 Third, it will describe the internal infrastructure supporting cross-compilation.
22 </para>
23</section>
24
25<!--============================================================-->
26
27<section xml:id="sec-cross-packaging">
28 <title>Packaging in a cross-friendly manner</title>
29
30 <section>
31 <title>Platform parameters</title>
32 <para>
33 Nixpkgs follows the <link xlink:href="https://gcc.gnu.org/onlinedocs/gccint/Configure-Terms.html">common historical convention of GNU autoconf</link> of distinguishing between 3 types of platform: <wordasword>build</wordasword>, <wordasword>host</wordasword>, and <wordasword>target</wordasword>.
34
35 In summary, <wordasword>build</wordasword> is the platform on which a package is being built, <wordasword>host</wordasword> is the platform on which it is to run. The third attribute, <wordasword>target</wordasword>, is relevant only for certain specific compilers and build tools.
36 </para>
37
38 <para>
39 In Nixpkgs, these three platforms are defined as attribute sets under the names <literal>buildPlatform</literal>, <literal>hostPlatform</literal>, and <literal>targetPlatform</literal>.
40 All three are always defined as attributes in the standard environment, and at the top level. That means one can get at them just like a dependency in a function that is imported with <literal>callPackage</literal>:
41 <programlisting>{ stdenv, buildPlatform, hostPlatform, fooDep, barDep, .. }: ...buildPlatform...</programlisting>, or just off <varname>stdenv</varname>:
42 <programlisting>{ stdenv, fooDep, barDep, .. }: ...stdenv.buildPlatform...</programlisting>.
43 </para>
44 <variablelist>
45 <varlistentry>
46 <term><varname>buildPlatform</varname></term>
47 <listitem><para>
48 The "build platform" is the platform on which a package is built.
49 Once someone has a built package, or pre-built binary package, the build platform should not matter and be safe to ignore.
50 </para></listitem>
51 </varlistentry>
52 <varlistentry>
53 <term><varname>hostPlatform</varname></term>
54 <listitem><para>
55 The "host platform" is the platform on which a package will be run.
56 This is the simplest platform to understand, but also the one with the worst name.
57 </para></listitem>
58 </varlistentry>
59 <varlistentry>
60 <term><varname>targetPlatform</varname></term>
61 <listitem>
62 <para>
63 The "target platform" attribute is, unlike the other two attributes, not actually fundamental to the process of building software.
64 Instead, it is only relevant for compatibility with building certain specific compilers and build tools.
65 It can be safely ignored for all other packages.
66 </para>
67 <para>
68 The build process of certain compilers is written in such a way that the compiler resulting from a single build can itself only produce binaries for a single platform.
69 The task specifying this single "target platform" is thus pushed to build time of the compiler.
70 The root cause of this mistake is often that the compiler (which will be run on the host) and the the standard library/runtime (which will be run on the target) are built by a single build process.
71 </para>
72 <para>
73 There is no fundamental need to think about a single target ahead of time like this.
74 If the tool supports modular or pluggable backends, both the need to specify the target at build time and the constraint of having only a single target disappear.
75 An example of such a tool is LLVM.
76 </para>
77 <para>
78 Although the existance of a "target platfom" is arguably a historical mistake, it is a common one: examples of tools that suffer from it are GCC, Binutils, GHC and Autoconf.
79 Nixpkgs tries to avoid sharing in the mistake where possible.
80 Still, because the concept of a target platform is so ingrained, it is best to support it as is.
81 </para>
82 </listitem>
83 </varlistentry>
84 </variablelist>
85 <para>
86 The exact schema these fields follow is a bit ill-defined due to a long and convoluted evolution, but this is slowly being cleaned up.
87 You can see examples of ones used in practice in <literal>lib.systems.examples</literal>; note how they are not all very consistent.
88 For now, here are few fields can count on them containing:
89 </para>
90 <variablelist>
91 <varlistentry>
92 <term><varname>system</varname></term>
93 <listitem>
94 <para>
95 This is a two-component shorthand for the platform.
96 Examples of this would be "x86_64-darwin" and "i686-linux"; see <literal>lib.systems.doubles</literal> for more.
97 This format isn't very standard, but has built-in support in Nix, such as the <varname>builtins.currentSystem</varname> impure string.
98 </para>
99 </listitem>
100 </varlistentry>
101 <varlistentry>
102 <term><varname>config</varname></term>
103 <listitem>
104 <para>
105 This is a 3- or 4- component shorthand for the platform.
106 Examples of this would be "x86_64-unknown-linux-gnu" and "aarch64-apple-darwin14".
107 This is a standard format called the "LLVM target triple", as they are pioneered by LLVM and traditionally just used for the <varname>targetPlatform</varname>.
108 This format is strictly more informative than the "Nix host double", as the previous format could analogously be termed.
109 This needs a better name than <varname>config</varname>!
110 </para>
111 </listitem>
112 </varlistentry>
113 <varlistentry>
114 <term><varname>parsed</varname></term>
115 <listitem>
116 <para>
117 This is a nix representation of a parsed LLVM target triple with white-listed components.
118 This can be specified directly, or actually parsed from the <varname>config</varname>.
119 [Technically, only one need be specified and the others can be inferred, though the precision of inference may not be very good.]
120 See <literal>lib.systems.parse</literal> for the exact representation.
121 </para>
122 </listitem>
123 </varlistentry>
124 <varlistentry>
125 <term><varname>libc</varname></term>
126 <listitem>
127 <para>
128 This is a string identifying the standard C library used.
129 Valid identifiers include "glibc" for GNU libc, "libSystem" for Darwin's Libsystem, and "uclibc" for µClibc.
130 It should probably be refactored to use the module system, like <varname>parse</varname>.
131 </para>
132 </listitem>
133 </varlistentry>
134 <varlistentry>
135 <term><varname>is*</varname></term>
136 <listitem>
137 <para>
138 These predicates are defined in <literal>lib.systems.inspect</literal>, and slapped on every platform.
139 They are superior to the ones in <varname>stdenv</varname> as they force the user to be explicit about which platform they are inspecting.
140 Please use these instead of those.
141 </para>
142 </listitem>
143 </varlistentry>
144 <varlistentry>
145 <term><varname>platform</varname></term>
146 <listitem>
147 <para>
148 This is, quite frankly, a dumping ground of ad-hoc settings (it's an attribute set).
149 See <literal>lib.systems.platforms</literal> for examples—there's hopefully one in there that will work verbatim for each platform that is working.
150 Please help us triage these flags and give them better homes!
151 </para>
152 </listitem>
153 </varlistentry>
154 </variablelist>
155 </section>
156
157 <section>
158 <title>Specifying Dependencies</title>
159 <para>
160 In this section we explore the relationship between both runtime and buildtime dependencies and the 3 Autoconf platforms.
161 </para>
162 <para>
163 A runtime dependency between 2 packages implies that between them both the host and target platforms match.
164 This is directly implied by the meaning of "host platform" and "runtime dependency":
165 The package dependency exists while both packages are running on a single host platform.
166 </para>
167 <para>
168 A build time dependency, however, implies a shift in platforms between the depending package and the depended-on package.
169 The meaning of a build time dependency is that to build the depending package we need to be able to run the depended-on's package.
170 The depending package's build platform is therefore equal to the depended-on package's host platform.
171 Analogously, the depending package's host platform is equal to the depended-on package's target platform.
172 </para>
173 <para>
174 In this manner, given the 3 platforms for one package, we can determine the three platforms for all its transitive dependencies.
175 This is the most important guiding principle behind cross-compilation with Nixpkgs, and will be called the <wordasword>sliding window principle</wordasword>.
176 </para>
177 <para>
178 Some examples will probably make this clearer.
179 If a package is being built with a <literal>(build, host, target)</literal> platform triple of <literal>(foo, bar, bar)</literal>, then its build-time dependencies would have a triple of <literal>(foo, foo, bar)</literal>, and <emphasis>those packages'</emphasis> build-time dependencies would have triple of <literal>(foo, foo, foo)</literal>.
180 In other words, it should take two "rounds" of following build-time dependency edges before one reaches a fixed point where, by the sliding window principle, the platform triple no longer changes.
181 Indeed, this happens with cross compilation, where only rounds of native dependencies starting with the second necessarily coincide with native packages.
182 </para>
183 <note><para>
184 The depending package's target platform is unconstrained by the sliding window principle, which makes sense in that one can in principle build cross compilers targeting arbitrary platforms.
185 </para></note>
186 <para>
187 How does this work in practice? Nixpkgs is now structured so that build-time dependencies are taken from <varname>buildPackages</varname>, whereas run-time dependencies are taken from the top level attribute set.
188 For example, <varname>buildPackages.gcc</varname> should be used at build time, while <varname>gcc</varname> should be used at run time.
189 Now, for most of Nixpkgs's history, there was no <varname>buildPackages</varname>, and most packages have not been refactored to use it explicitly.
190 Instead, one can use the six (<emphasis>gasp</emphasis>) attributes used for specifying dependencies as documented in <xref linkend="ssec-stdenv-dependencies"/>.
191 We "splice" together the run-time and build-time package sets with <varname>callPackage</varname>, and then <varname>mkDerivation</varname> for each of four attributes pulls the right derivation out.
192 This splicing can be skipped when not cross compiling as the package sets are the same, but is a bit slow for cross compiling.
193 Because of this, a best-of-both-worlds solution is in the works with no splicing or explicit access of <varname>buildPackages</varname> needed.
194 For now, feel free to use either method.
195 </para>
196 <note><para>
197 There is also a "backlink" <varname>targetPackages</varname>, yielding a package set whose <varname>buildPackages</varname> is the current package set.
198 This is a hack, though, to accommodate compilers with lousy build systems.
199 Please do not use this unless you are absolutely sure you are packaging such a compiler and there is no other way.
200 </para></note>
201 </section>
202
203 <section>
204 <title>Cross packagaing cookbook</title>
205 <para>
206 Some frequently problems when packaging for cross compilation are good to just spell and answer.
207 Ideally the information above is exhaustive, so this section cannot provide any new information,
208 but its ludicrous and cruel to expect everyone to spend effort working through the interaction of many features just to figure out the same answer to the same common problem.
209 Feel free to add to this list!
210 </para>
211 <qandaset>
212 <qandaentry>
213 <question><para>
214 What if my package's build system needs to build a C program to be run under the build environment?
215 </para></question>
216 <answer><para>
217 <programlisting>depsBuildBuild = [ buildPackages.stdenv.cc ];</programlisting>
218 Add it to your <function>mkDerivation</function> invocation.
219 </para></answer>
220 </qandaentry>
221 <qandaentry>
222 <question><para>
223 My package fails to find <command>ar</command>.
224 </para></question>
225 <answer><para>
226 Many packages assume that an unprefixed <command>ar</command> is available, but Nix doesn't provide one.
227 It only provides a prefixed one, just as it only does for all the other binutils programs.
228 It may be necessary to patch the package to fix the build system to use a prefixed `ar`.
229 </para></answer>
230 </qandaentry>
231 <qandaentry>
232 <question><para>
233 My package's testsuite needs to run host platform code.
234 </para></question>
235 <answer><para>
236 <programlisting>doCheck = stdenv.hostPlatform != stdenv.buildPlatfrom;</programlisting>
237 Add it to your <function>mkDerivation</function> invocation.
238 </para></answer>
239 </qandaentry>
240 </qandaset>
241 </section>
242</section>
243
244<!--============================================================-->
245
246<section xml:id="sec-cross-usage">
247 <title>Cross-building packages</title>
248 <note><para>
249 More information needs to moved from the old wiki, especially <link xlink:href="https://nixos.org/wiki/CrossCompiling" />, for this section.
250 </para></note>
251 <para>
252 Nixpkgs can be instantiated with <varname>localSystem</varname> alone, in which case there is no cross compiling and everything is built by and for that system,
253 or also with <varname>crossSystem</varname>, in which case packages run on the latter, but all building happens on the former.
254 Both parameters take the same schema as the 3 (build, host, and target) platforms defined in the previous section.
255 As mentioned above, <literal>lib.systems.examples</literal> has some platforms which are used as arguments for these parameters in practice.
256 You can use them programmatically, or on the command line: <programlisting>
257nix-build <nixpkgs> --arg crossSystem '(import <nixpkgs/lib>).systems.examples.fooBarBaz' -A whatever</programlisting>
258 </para>
259 <note>
260 <para>
261 Eventually we would like to make these platform examples an unnecessary convenience so that <programlisting>
262nix-build <nixpkgs> --arg crossSystem.config '<arch>-<os>-<vendor>-<abi>' -A whatever</programlisting>
263 works in the vast majority of cases.
264 The problem today is dependencies on other sorts of configuration which aren't given proper defaults.
265 We rely on the examples to crudely to set those configuration parameters in some vaguely sane manner on the users behalf.
266 Issue <link xlink:href="https://github.com/NixOS/nixpkgs/issues/34274">#34274</link> tracks this inconvenience along with its root cause in crufty configuration options.
267 </para>
268 </note>
269 <para>
270 While one is free to pass both parameters in full, there's a lot of logic to fill in missing fields.
271 As discussed in the previous section, only one of <varname>system</varname>, <varname>config</varname>, and <varname>parsed</varname> is needed to infer the other two.
272 Additionally, <varname>libc</varname> will be inferred from <varname>parse</varname>.
273 Finally, <literal>localSystem.system</literal> is also <emphasis>impurely</emphasis> inferred based on the platform evaluation occurs.
274 This means it is often not necessary to pass <varname>localSystem</varname> at all, as in the command-line example in the previous paragraph.
275 </para>
276 <note>
277 <para>
278 Many sources (manual, wiki, etc) probably mention passing <varname>system</varname>, <varname>platform</varname>, along with the optional <varname>crossSystem</varname> to nixpkgs:
279 <literal>import <nixpkgs> { system = ..; platform = ..; crossSystem = ..; }</literal>.
280 Passing those two instead of <varname>localSystem</varname> is still supported for compatibility, but is discouraged.
281 Indeed, much of the inference we do for these parameters is motivated by compatibility as much as convenience.
282 </para>
283 </note>
284 <para>
285 One would think that <varname>localSystem</varname> and <varname>crossSystem</varname> overlap horribly with the three <varname>*Platforms</varname> (<varname>buildPlatform</varname>, <varname>hostPlatform,</varname> and <varname>targetPlatform</varname>; see <varname>stage.nix</varname> or the manual).
286 Actually, those identifiers are purposefully not used here to draw a subtle but important distinction:
287 While the granularity of having 3 platforms is necessary to properly *build* packages, it is overkill for specifying the user's *intent* when making a build plan or package set.
288 A simple "build vs deploy" dichotomy is adequate: the sliding window principle described in the previous section shows how to interpolate between the these two "end points" to get the 3 platform triple for each bootstrapping stage.
289 That means for any package a given package set, even those not bound on the top level but only reachable via dependencies or <varname>buildPackages</varname>, the three platforms will be defined as one of <varname>localSystem</varname> or <varname>crossSystem</varname>, with the former replacing the latter as one traverses build-time dependencies.
290 A last simple difference then is <varname>crossSystem</varname> should be null when one doesn't want to cross-compile, while the <varname>*Platform</varname>s are always non-null.
291 <varname>localSystem</varname> is always non-null.
292 </para>
293</section>
294
295<!--============================================================-->
296
297<section xml:id="sec-cross-infra">
298 <title>Cross-compilation infrastructure</title>
299 <para>To be written.</para>
300 <note><para>
301 If one explores nixpkgs, they will see derivations with names like <literal>gccCross</literal>.
302 Such <literal>*Cross</literal> derivations is a holdover from before we properly distinguished between the host and target platforms
303 —the derivation with "Cross" in the name covered the <literal>build = host != target</literal> case, while the other covered the <literal>host = target</literal>, with build platform the same or not based on whether one was using its <literal>.nativeDrv</literal> or <literal>.crossDrv</literal>.
304 This ugliness will disappear soon.
305 </para></note>
306</section>
307
308</chapter>