pyrox.dev
1# Dhall {#sec-language-dhall}
2
3The Nixpkgs support for Dhall assumes some familiarity with Dhall's language
4support for importing Dhall expressions, which is documented here:
5
6* [`dhall-lang.org` - Installing packages](https://docs.dhall-lang.org/tutorials/Language-Tour.html#installing-packages)
7
8## Remote imports {#ssec-dhall-remote-imports}
9
10Nixpkgs bypasses Dhall's support for remote imports using Dhall's
11semantic integrity checks. Specifically, any Dhall import can be protected by
12an integrity check like:
13
14```dhall
15https://prelude.dhall-lang.org/v20.1.0/package.dhall
16 sha256:26b0ef498663d269e4dc6a82b0ee289ec565d683ef4c00d0ebdd25333a5a3c98
17```
18
19… and if the import is cached then the interpreter will load the import from
20cache instead of fetching the URL.
21
22Nixpkgs uses this trick to add all of a Dhall expression's dependencies into the
23cache so that the Dhall interpreter never needs to resolve any remote URLs. In
24fact, Nixpkgs uses a Dhall interpreter with remote imports disabled when
25packaging Dhall expressions to enforce that the interpreter never resolves a
26remote import. This means that Nixpkgs only supports building Dhall expressions
27if all of their remote imports are protected by semantic integrity checks.
28
29Instead of remote imports, Nixpkgs uses Nix to fetch remote Dhall code. For
30example, the Prelude Dhall package uses `pkgs.fetchFromGitHub` to fetch the
31`dhall-lang` repository containing the Prelude. Relying exclusively on Nix
32to fetch Dhall code ensures that Dhall packages built using Nix remain pure and
33also behave well when built within a sandbox.
34
35## Packaging a Dhall expression from scratch {#ssec-dhall-packaging-expression}
36
37We can illustrate how Nixpkgs integrates Dhall by beginning from the following
38trivial Dhall expression with one dependency (the Prelude):
39
40```dhall
41-- ./true.dhall
42
43let Prelude = https://prelude.dhall-lang.org/v20.1.0/package.dhall
44
45in Prelude.Bool.not False
46```
47
48As written, this expression cannot be built using Nixpkgs because the
49expression does not protect the Prelude import with a semantic integrity
50check, so the first step is to freeze the expression using `dhall freeze`,
51like this:
52
53```ShellSession
54$ dhall freeze --inplace ./true.dhall
55```
56
57… which gives us:
58
59```dhall
60-- ./true.dhall
61
62let Prelude =
63 https://prelude.dhall-lang.org/v20.1.0/package.dhall
64 sha256:26b0ef498663d269e4dc6a82b0ee289ec565d683ef4c00d0ebdd25333a5a3c98
65
66in Prelude.Bool.not False
67```
68
69To package that expression, we create a `./true.nix` file containing the
70following specification for the Dhall package:
71
72```nix
73# ./true.nix
74
75{ buildDhallPackage, Prelude }:
76
77buildDhallPackage {
78 name = "true";
79 code = ./true.dhall;
80 dependencies = [ Prelude ];
81 source = true;
82}
83```
84
85… and we complete the build by incorporating that Dhall package into the
86`pkgs.dhallPackages` hierarchy using an overlay, like this:
87
88```nix
89# ./example.nix
90
91let
92 nixpkgs = builtins.fetchTarball {
93 url = "https://github.com/NixOS/nixpkgs/archive/94b2848559b12a8ed1fe433084686b2a81123c99.tar.gz";
94 hash = "sha256-B4Q3c6IvTLg3Q92qYa8y+i4uTaphtFdjp+Ir3QQjdN0=";
95 };
96
97 dhallOverlay = self: super: {
98 true = self.callPackage ./true.nix { };
99 };
100
101 overlay = self: super: {
102 dhallPackages = super.dhallPackages.override (old: {
103 overrides =
104 self.lib.composeExtensions (old.overrides or (_: _: {})) dhallOverlay;
105 });
106 };
107
108 pkgs = import nixpkgs { config = {}; overlays = [ overlay ]; };
109
110in
111 pkgs
112```
113
114… which we can then build using this command:
115
116```ShellSession
117$ nix build --file ./example.nix dhallPackages.true
118```
119
120## Contents of a Dhall package {#ssec-dhall-package-contents}
121
122The above package produces the following directory tree:
123
124```ShellSession
125$ tree -a ./result
126result
127├── .cache
128│ └── dhall
129│ └── 122027abdeddfe8503496adeb623466caa47da5f63abd2bc6fa19f6cfcb73ecfed70
130├── binary.dhall
131└── source.dhall
132```
133
134… where:
135
136* `source.dhall` contains the result of interpreting our Dhall package:
137
138 ```ShellSession
139 $ cat ./result/source.dhall
140 True
141 ```
142
143* The `.cache` subdirectory contains one binary cache product encoding the
144 same result as `source.dhall`:
145
146 ```ShellSession
147 $ dhall decode < ./result/.cache/dhall/122027abdeddfe8503496adeb623466caa47da5f63abd2bc6fa19f6cfcb73ecfed70
148 True
149 ```
150
151* `binary.dhall` contains a Dhall expression which handles fetching and decoding
152 the same cache product:
153
154 ```ShellSession
155 $ cat ./result/binary.dhall
156 missing sha256:27abdeddfe8503496adeb623466caa47da5f63abd2bc6fa19f6cfcb73ecfed70
157 $ cp -r ./result/.cache .cache
158
159 $ chmod -R u+w .cache
160
161 $ XDG_CACHE_HOME=.cache dhall --file ./result/binary.dhall
162 True
163 ```
164
165The `source.dhall` file is only present for packages that specify
166`source = true;`. By default, Dhall packages omit the `source.dhall` in order
167to conserve disk space when they are used exclusively as dependencies. For
168example, if we build the Prelude package it will only contain the binary
169encoding of the expression:
170
171```ShellSession
172$ nix build --file ./example.nix dhallPackages.Prelude
173
174$ tree -a result
175result
176├── .cache
177│ └── dhall
178│ └── 122026b0ef498663d269e4dc6a82b0ee289ec565d683ef4c00d0ebdd25333a5a3c98
179└── binary.dhall
180
1812 directories, 2 files
182```
183
184Typically, you only specify `source = true;` for the top-level Dhall expression
185of interest (such as our example `true.nix` Dhall package). However, if you
186wish to specify `source = true` for all Dhall packages, then you can amend the
187Dhall overlay like this:
188
189```nix
190 dhallOverrides = self: super: {
191 # Enable source for all Dhall packages
192 buildDhallPackage =
193 args: super.buildDhallPackage (args // { source = true; });
194
195 true = self.callPackage ./true.nix { };
196 };
197```
198
199… and now the Prelude will contain the fully decoded result of interpreting
200the Prelude:
201
202```ShellSession
203$ nix build --file ./example.nix dhallPackages.Prelude
204
205$ tree -a result
206result
207├── .cache
208│ └── dhall
209│ └── 122026b0ef498663d269e4dc6a82b0ee289ec565d683ef4c00d0ebdd25333a5a3c98
210├── binary.dhall
211└── source.dhall
212
213$ cat ./result/source.dhall
214{ Bool =
215 { and =
216 \(_ : List Bool) ->
217 List/fold Bool _ Bool (\(_ : Bool) -> \(_ : Bool) -> _@1 && _) True
218 , build = \(_ : Type -> _ -> _@1 -> _@2) -> _ Bool True False
219 , even =
220 \(_ : List Bool) ->
221 List/fold Bool _ Bool (\(_ : Bool) -> \(_ : Bool) -> _@1 == _) True
222 , fold =
223 \(_ : Bool) ->
224…
225```
226
227## Packaging functions {#ssec-dhall-packaging-functions}
228
229We already saw an example of using `buildDhallPackage` to create a Dhall
230package from a single file, but most Dhall packages consist of more than one
231file and there are two derived utilities that you may find more useful when
232packaging multiple files:
233
234* `buildDhallDirectoryPackage` - build a Dhall package from a local directory
235
236* `buildDhallGitHubPackage` - build a Dhall package from a GitHub repository
237
238The `buildDhallPackage` is the lowest-level function and accepts the following
239arguments:
240
241* `name`: The name of the derivation
242
243* `dependencies`: Dhall dependencies to build and cache ahead of time
244
245* `code`: The top-level expression to build for this package
246
247 Note that the `code` field accepts an arbitrary Dhall expression. You're
248 not limited to just a file.
249
250* `source`: Set to `true` to include the decoded result as `source.dhall` in the
251 build product, at the expense of requiring more disk space
252
253* `documentationRoot`: Set to the root directory of the package if you want
254 `dhall-docs` to generate documentation underneath the `docs` subdirectory of
255 the build product
256
257The `buildDhallDirectoryPackage` is a higher-level function implemented in terms
258of `buildDhallPackage` that accepts the following arguments:
259
260* `name`: Same as `buildDhallPackage`
261
262* `dependencies`: Same as `buildDhallPackage`
263
264* `source`: Same as `buildDhallPackage`
265
266* `src`: The directory containing Dhall code that you want to turn into a Dhall
267 package
268
269* `file`: The top-level file (`package.dhall` by default) that is the entrypoint
270 to the rest of the package
271
272* `document`: Set to `true` to generate documentation for the package
273
274The `buildDhallGitHubPackage` is another higher-level function implemented in
275terms of `buildDhallPackage` that accepts the following arguments:
276
277* `name`: Same as `buildDhallPackage`
278
279* `dependencies`: Same as `buildDhallPackage`
280
281* `source`: Same as `buildDhallPackage`
282
283* `owner`: The owner of the repository
284
285* `repo`: The repository name
286
287* `rev`: The desired revision (or branch, or tag)
288
289* `directory`: The subdirectory of the Git repository to package (if a
290 directory other than the root of the repository)
291
292* `file`: The top-level file (`${directory}/package.dhall` by default) that is
293 the entrypoint to the rest of the package
294
295* `document`: Set to `true` to generate documentation for the package
296
297Additionally, `buildDhallGitHubPackage` accepts the same arguments as
298`fetchFromGitHub`, such as `hash` or `fetchSubmodules`.
299
300## `dhall-to-nixpkgs` {#ssec-dhall-dhall-to-nixpkgs}
301
302You can use the `dhall-to-nixpkgs` command-line utility to automate
303packaging Dhall code. For example:
304
305```ShellSession
306$ nix-shell -p haskellPackages.dhall-nixpkgs nix-prefetch-git
307[nix-shell]$ dhall-to-nixpkgs github https://github.com/Gabriella439/dhall-semver.git
308{ buildDhallGitHubPackage, Prelude }:
309 buildDhallGitHubPackage {
310 name = "dhall-semver";
311 githubBase = "github.com";
312 owner = "Gabriella439";
313 repo = "dhall-semver";
314 rev = "2d44ae605302ce5dc6c657a1216887fbb96392a4";
315 fetchSubmodules = false;
316 hash = "sha256-n0nQtswVapWi/x7or0O3MEYmAkt/a1uvlOtnje6GGnk=";
317 directory = "";
318 file = "package.dhall";
319 source = false;
320 document = false;
321 dependencies = [ (Prelude.overridePackage { file = "package.dhall"; }) ];
322 }
323```
324
325:::{.note}
326`nix-prefetch-git` is added to the `nix-shell -p` invocation above, because it has to be in `$PATH` for `dhall-to-nixpkgs` to work.
327:::
328
329The utility takes care of automatically detecting remote imports and converting
330them to package dependencies. You can also use the utility on local
331Dhall directories, too:
332
333```ShellSession
334$ dhall-to-nixpkgs directory ~/proj/dhall-semver
335{ buildDhallDirectoryPackage, Prelude }:
336 buildDhallDirectoryPackage {
337 name = "proj";
338 src = ~/proj/dhall-semver;
339 file = "package.dhall";
340 source = false;
341 document = false;
342 dependencies = [ (Prelude.overridePackage { file = "package.dhall"; }) ];
343 }
344```
345
346### Remote imports as fixed-output derivations {#ssec-dhall-remote-imports-as-fod}
347
348`dhall-to-nixpkgs` has the ability to fetch and build remote imports as
349fixed-output derivations by using their Dhall integrity check. This is
350sometimes easier than manually packaging all remote imports.
351
352This can be used like the following:
353
354```ShellSession
355$ dhall-to-nixpkgs directory --fixed-output-derivations ~/proj/dhall-semver
356{ buildDhallDirectoryPackage, buildDhallUrl }:
357 buildDhallDirectoryPackage {
358 name = "proj";
359 src = ~/proj/dhall-semver;
360 file = "package.dhall";
361 source = false;
362 document = false;
363 dependencies = [
364 (buildDhallUrl {
365 url = "https://prelude.dhall-lang.org/v17.0.0/package.dhall";
366 hash = "sha256-ENs8kZwl6QRoM9+Jeo/+JwHcOQ+giT2VjDQwUkvlpD4=";
367 dhallHash = "sha256:10db3c919c25e9046833df897a8ffe2701dc390fa0893d958c3430524be5a43e";
368 })
369 ];
370 }
371```
372
373Here, `dhall-semver`'s `Prelude` dependency is fetched and built with the
374`buildDhallUrl` helper function, instead of being passed in as a function
375argument.
376
377## Overriding dependency versions {#ssec-dhall-overriding-dependency-versions}
378
379Suppose that we change our `true.dhall` example expression to depend on an older
380version of the Prelude (19.0.0):
381
382```dhall
383-- ./true.dhall
384
385let Prelude =
386 https://prelude.dhall-lang.org/v19.0.0/package.dhall
387 sha256:eb693342eb769f782174157eba9b5924cf8ac6793897fc36a31ccbd6f56dafe2
388
389in Prelude.Bool.not False
390```
391
392If we try to rebuild that expression the build will fail:
393
394```ShellSession
395$ nix build --file ./example.nix dhallPackages.true
396builder for '/nix/store/0f1hla7ff1wiaqyk1r2ky4wnhnw114fi-true.drv' failed with exit code 1; last 10 log lines:
397
398 Dhall was compiled without the 'with-http' flag.
399
400 The requested URL was: https://prelude.dhall-lang.org/v19.0.0/package.dhall
401
402
403 4│ https://prelude.dhall-lang.org/v19.0.0/package.dhall
404 5│ sha256:eb693342eb769f782174157eba9b5924cf8ac6793897fc36a31ccbd6f56dafe2
405
406 /nix/store/rsab4y99h14912h4zplqx2iizr5n4rc2-true.dhall:4:7
407[1 built (1 failed), 0.0 MiB DL]
408error: build of '/nix/store/0f1hla7ff1wiaqyk1r2ky4wnhnw114fi-true.drv' failed
409```
410
411… because the default Prelude selected by Nixpkgs revision
412`94b2848559b12a8ed1fe433084686b2a81123c99is` is version 20.1.0, which doesn't
413have the same integrity check as version 19.0.0. This means that version
41419.0.0 is not cached and the interpreter is not allowed to fall back to
415importing the URL.
416
417However, we can override the default Prelude version by using `dhall-to-nixpkgs`
418to create a Dhall package for our desired Prelude:
419
420```ShellSession
421$ dhall-to-nixpkgs github https://github.com/dhall-lang/dhall-lang.git \
422 --name Prelude \
423 --directory Prelude \
424 --rev v19.0.0 \
425 > Prelude.nix
426```
427
428… and then referencing that package in our Dhall overlay, by either overriding
429the Prelude globally for all packages, like this:
430
431```nix
432 dhallOverrides = self: super: {
433 true = self.callPackage ./true.nix { };
434
435 Prelude = self.callPackage ./Prelude.nix { };
436 };
437```
438
439… or selectively overriding the Prelude dependency for just the `true` package,
440like this:
441
442```nix
443 dhallOverrides = self: super: {
444 true = self.callPackage ./true.nix {
445 Prelude = self.callPackage ./Prelude.nix { };
446 };
447 };
448```
449
450## Overrides {#ssec-dhall-overrides}
451
452You can override any of the arguments to `buildDhallGitHubPackage` or
453`buildDhallDirectoryPackage` using the `overridePackage` attribute of a package.
454For example, suppose we wanted to selectively enable `source = true` just for the Prelude. We can do that like this:
455
456```nix
457 dhallOverrides = self: super: {
458 Prelude = super.Prelude.overridePackage { source = true; };
459
460 …
461 };
462```
463
464[semantic-integrity-checks]: https://docs.dhall-lang.org/tutorials/Language-Tour.html#installing-packages