pyrox.dev
1# Dhall {#sec-language-dhall}
2
3The Nixpkgs support for Dhall assumes some familiarity with Dhall's language
4support for importing Dhall expressions, which is documented here:
5
6* [`dhall-lang.org` - Installing packages](https://docs.dhall-lang.org/tutorials/Language-Tour.html#installing-packages)
7
8## Remote imports {#ssec-dhall-remote-imports}
9
10Nixpkgs bypasses Dhall's support for remote imports using Dhall's
11semantic integrity checks. Specifically, any Dhall import can be protected by
12an integrity check like:
13
14```dhall
15https://prelude.dhall-lang.org/v20.1.0/package.dhall
16 sha256:26b0ef498663d269e4dc6a82b0ee289ec565d683ef4c00d0ebdd25333a5a3c98
17```
18
19… and if the import is cached, then the interpreter will load the import from
20the cache instead of fetching the URL.
21
22Nixpkgs uses this trick to add all of a Dhall expression's dependencies into the
23cache so that the Dhall interpreter never needs to resolve any remote URLs. In
24fact, Nixpkgs uses a Dhall interpreter with remote imports disabled when
25packaging Dhall expressions to enforce that the interpreter never resolves a
26remote import. This means that Nixpkgs only supports building Dhall expressions
27if all of their remote imports are protected by semantic integrity checks.
28
29Instead of remote imports, Nixpkgs uses Nix to fetch remote Dhall code. For
30example, the Prelude Dhall package uses `pkgs.fetchFromGitHub` to fetch the
31`dhall-lang` repository containing the Prelude. Relying exclusively on Nix
32to fetch Dhall code ensures that Dhall packages built using Nix remain pure and
33also behave well when built within a sandbox.
34
35## Packaging a Dhall expression from scratch {#ssec-dhall-packaging-expression}
36
37We can illustrate how Nixpkgs integrates Dhall by beginning from the following
38trivial Dhall expression with one dependency (the Prelude):
39
40```dhall
41-- ./true.dhall
42
43let Prelude = https://prelude.dhall-lang.org/v20.1.0/package.dhall
44
45in Prelude.Bool.not False
46```
47
48As written, this expression cannot be built using Nixpkgs because the
49expression does not protect the Prelude import with a semantic integrity
50check, so the first step is to freeze the expression using `dhall freeze`,
51like this:
52
53```ShellSession
54$ dhall freeze --inplace ./true.dhall
55```
56
57… which gives us:
58
59```dhall
60-- ./true.dhall
61
62let Prelude =
63 https://prelude.dhall-lang.org/v20.1.0/package.dhall
64 sha256:26b0ef498663d269e4dc6a82b0ee289ec565d683ef4c00d0ebdd25333a5a3c98
65
66in Prelude.Bool.not False
67```
68
69To package that expression, we create a `./true.nix` file containing the
70following specification for the Dhall package:
71
72```nix
73# ./true.nix
74
75{ buildDhallPackage, Prelude }:
76
77buildDhallPackage {
78 name = "true";
79 code = ./true.dhall;
80 dependencies = [ Prelude ];
81 source = true;
82}
83```
84
85… and we complete the build by incorporating that Dhall package into the
86`pkgs.dhallPackages` hierarchy using an overlay, like this:
87
88```nix
89# ./example.nix
90
91let
92 nixpkgs = fetchTarball {
93 url = "https://github.com/NixOS/nixpkgs/archive/94b2848559b12a8ed1fe433084686b2a81123c99.tar.gz";
94 hash = "sha256-B4Q3c6IvTLg3Q92qYa8y+i4uTaphtFdjp+Ir3QQjdN0=";
95 };
96
97 dhallOverlay = self: super: { true = self.callPackage ./true.nix { }; };
98
99 overlay = self: super: {
100 dhallPackages = super.dhallPackages.override (old: {
101 overrides = self.lib.composeExtensions (old.overrides or (_: _: { })) dhallOverlay;
102 });
103 };
104
105 pkgs = import nixpkgs {
106 config = { };
107 overlays = [ overlay ];
108 };
109
110in
111pkgs
112```
113
114… which we can then build using this command:
115
116```ShellSession
117$ nix build --file ./example.nix dhallPackages.true
118```
119
120## Contents of a Dhall package {#ssec-dhall-package-contents}
121
122The above package produces the following directory tree:
123
124```ShellSession
125$ tree -a ./result
126result
127├── .cache
128│ └── dhall
129│ └── 122027abdeddfe8503496adeb623466caa47da5f63abd2bc6fa19f6cfcb73ecfed70
130├── binary.dhall
131└── source.dhall
132```
133
134… where:
135
136* `source.dhall` contains the result of interpreting our Dhall package:
137
138 ```ShellSession
139 $ cat ./result/source.dhall
140 True
141 ```
142
143* The `.cache` subdirectory contains one binary cache product encoding the
144 same result as `source.dhall`:
145
146 ```ShellSession
147 $ dhall decode < ./result/.cache/dhall/122027abdeddfe8503496adeb623466caa47da5f63abd2bc6fa19f6cfcb73ecfed70
148 True
149 ```
150
151* `binary.dhall` contains a Dhall expression which handles fetching and decoding
152 the same cache product:
153
154 ```ShellSession
155 $ cat ./result/binary.dhall
156 missing sha256:27abdeddfe8503496adeb623466caa47da5f63abd2bc6fa19f6cfcb73ecfed70
157 $ cp -r ./result/.cache .cache
158
159 $ chmod -R u+w .cache
160
161 $ XDG_CACHE_HOME=.cache dhall --file ./result/binary.dhall
162 True
163 ```
164
165The `source.dhall` file is only present for packages that specify
166`source = true;`. By default, Dhall packages omit the `source.dhall` in order
167to conserve disk space when they are used exclusively as dependencies. For
168example, if we build the Prelude package it will only contain the binary
169encoding of the expression:
170
171```ShellSession
172$ nix build --file ./example.nix dhallPackages.Prelude
173
174$ tree -a result
175result
176├── .cache
177│ └── dhall
178│ └── 122026b0ef498663d269e4dc6a82b0ee289ec565d683ef4c00d0ebdd25333a5a3c98
179└── binary.dhall
180
1812 directories, 2 files
182```
183
184Typically, you only specify `source = true;` for the top-level Dhall expression
185of interest (such as our example `true.nix` Dhall package). However, if you
186wish to specify `source = true` for all Dhall packages, then you can amend the
187Dhall overlay like this:
188
189```nix
190{
191 dhallOverrides = self: super: {
192 # Enable source for all Dhall packages
193 buildDhallPackage = args: super.buildDhallPackage (args // { source = true; });
194
195 true = self.callPackage ./true.nix { };
196 };
197}
198```
199
200… and now the Prelude will contain the fully decoded result of interpreting
201the Prelude:
202
203```ShellSession
204$ nix build --file ./example.nix dhallPackages.Prelude
205
206$ tree -a result
207result
208├── .cache
209│ └── dhall
210│ └── 122026b0ef498663d269e4dc6a82b0ee289ec565d683ef4c00d0ebdd25333a5a3c98
211├── binary.dhall
212└── source.dhall
213
214$ cat ./result/source.dhall
215{ Bool =
216 { and =
217 \(_ : List Bool) ->
218 List/fold Bool _ Bool (\(_ : Bool) -> \(_ : Bool) -> _@1 && _) True
219 , build = \(_ : Type -> _ -> _@1 -> _@2) -> _ Bool True False
220 , even =
221 \(_ : List Bool) ->
222 List/fold Bool _ Bool (\(_ : Bool) -> \(_ : Bool) -> _@1 == _) True
223 , fold =
224 \(_ : Bool) ->
225…
226```
227
228## Packaging functions {#ssec-dhall-packaging-functions}
229
230We already saw an example of using `buildDhallPackage` to create a Dhall
231package from a single file, but most Dhall packages consist of more than one
232file and there are two derived utilities that you may find more useful when
233packaging multiple files:
234
235* `buildDhallDirectoryPackage` - build a Dhall package from a local directory
236
237* `buildDhallGitHubPackage` - build a Dhall package from a GitHub repository
238
239The `buildDhallPackage` is the lowest-level function and accepts the following
240arguments:
241
242* `name`: The name of the derivation
243
244* `dependencies`: Dhall dependencies to build and cache ahead of time
245
246* `code`: The top-level expression to build for this package
247
248 Note that the `code` field accepts an arbitrary Dhall expression. You're
249 not limited to just a file.
250
251* `source`: Set to `true` to include the decoded result as `source.dhall` in the
252 build product, at the expense of requiring more disk space
253
254* `documentationRoot`: Set to the root directory of the package if you want
255 `dhall-docs` to generate documentation underneath the `docs` subdirectory of
256 the build product
257
258The `buildDhallDirectoryPackage` is a higher-level function implemented in terms
259of `buildDhallPackage` that accepts the following arguments:
260
261* `name`: Same as `buildDhallPackage`
262
263* `dependencies`: Same as `buildDhallPackage`
264
265* `source`: Same as `buildDhallPackage`
266
267* `src`: The directory containing Dhall code that you want to turn into a Dhall
268 package
269
270* `file`: The top-level file (`package.dhall` by default) that is the entry point
271 to the rest of the package
272
273* `document`: Set to `true` to generate documentation for the package
274
275The `buildDhallGitHubPackage` is another higher-level function implemented in
276terms of `buildDhallPackage` that accepts the following arguments:
277
278* `name`: Same as `buildDhallPackage`
279
280* `dependencies`: Same as `buildDhallPackage`
281
282* `source`: Same as `buildDhallPackage`
283
284* `owner`: The owner of the repository
285
286* `repo`: The repository name
287
288* `rev`: The desired revision (or branch, or tag)
289
290* `directory`: The subdirectory of the Git repository to package (if a
291 directory other than the root of the repository)
292
293* `file`: The top-level file (`${directory}/package.dhall` by default) that is
294 the entry point to the rest of the package
295
296* `document`: Set to `true` to generate documentation for the package
297
298Additionally, `buildDhallGitHubPackage` accepts the same arguments as
299`fetchFromGitHub`, such as `hash` or `fetchSubmodules`.
300
301## `dhall-to-nixpkgs` {#ssec-dhall-dhall-to-nixpkgs}
302
303You can use the `dhall-to-nixpkgs` command-line utility to automate
304packaging Dhall code. For example:
305
306```ShellSession
307$ nix-shell -p haskellPackages.dhall-nixpkgs nix-prefetch-git
308[nix-shell]$ dhall-to-nixpkgs github https://github.com/Gabriella439/dhall-semver.git
309{ buildDhallGitHubPackage, Prelude }:
310 buildDhallGitHubPackage {
311 name = "dhall-semver";
312 githubBase = "github.com";
313 owner = "Gabriella439";
314 repo = "dhall-semver";
315 rev = "2d44ae605302ce5dc6c657a1216887fbb96392a4";
316 fetchSubmodules = false;
317 hash = "sha256-n0nQtswVapWi/x7or0O3MEYmAkt/a1uvlOtnje6GGnk=";
318 directory = "";
319 file = "package.dhall";
320 source = false;
321 document = false;
322 dependencies = [ (Prelude.overridePackage { file = "package.dhall"; }) ];
323 }
324```
325
326:::{.note}
327`nix-prefetch-git` is added to the `nix-shell -p` invocation above, because it has to be in `$PATH` for `dhall-to-nixpkgs` to work.
328:::
329
330The utility takes care of automatically detecting remote imports and converting
331them to package dependencies. You can also use the utility on local
332Dhall directories, too:
333
334```ShellSession
335$ dhall-to-nixpkgs directory ~/proj/dhall-semver
336{ buildDhallDirectoryPackage, Prelude }:
337 buildDhallDirectoryPackage {
338 name = "proj";
339 src = ~/proj/dhall-semver;
340 file = "package.dhall";
341 source = false;
342 document = false;
343 dependencies = [ (Prelude.overridePackage { file = "package.dhall"; }) ];
344 }
345```
346
347### Remote imports as fixed-output derivations {#ssec-dhall-remote-imports-as-fod}
348
349`dhall-to-nixpkgs` has the ability to fetch and build remote imports as
350fixed-output derivations by using their Dhall integrity check. This is
351sometimes easier than manually packaging all remote imports.
352
353This can be used like the following:
354
355```ShellSession
356$ dhall-to-nixpkgs directory --fixed-output-derivations ~/proj/dhall-semver
357{ buildDhallDirectoryPackage, buildDhallUrl }:
358 buildDhallDirectoryPackage {
359 name = "proj";
360 src = ~/proj/dhall-semver;
361 file = "package.dhall";
362 source = false;
363 document = false;
364 dependencies = [
365 (buildDhallUrl {
366 url = "https://prelude.dhall-lang.org/v17.0.0/package.dhall";
367 hash = "sha256-ENs8kZwl6QRoM9+Jeo/+JwHcOQ+giT2VjDQwUkvlpD4=";
368 dhallHash = "sha256:10db3c919c25e9046833df897a8ffe2701dc390fa0893d958c3430524be5a43e";
369 })
370 ];
371 }
372```
373
374Here, `dhall-semver`'s `Prelude` dependency is fetched and built with the
375`buildDhallUrl` helper function, instead of being passed in as a function
376argument.
377
378## Overriding dependency versions {#ssec-dhall-overriding-dependency-versions}
379
380Suppose that we change our `true.dhall` example expression to depend on an older
381version of the Prelude (19.0.0):
382
383```dhall
384-- ./true.dhall
385
386let Prelude =
387 https://prelude.dhall-lang.org/v19.0.0/package.dhall
388 sha256:eb693342eb769f782174157eba9b5924cf8ac6793897fc36a31ccbd6f56dafe2
389
390in Prelude.Bool.not False
391```
392
393If we try to rebuild that expression the build will fail:
394
395```ShellSession
396$ nix build --file ./example.nix dhallPackages.true
397builder for '/nix/store/0f1hla7ff1wiaqyk1r2ky4wnhnw114fi-true.drv' failed with exit code 1; last 10 log lines:
398
399 Dhall was compiled without the 'with-http' flag.
400
401 The requested URL was: https://prelude.dhall-lang.org/v19.0.0/package.dhall
402
403
404 4│ https://prelude.dhall-lang.org/v19.0.0/package.dhall
405 5│ sha256:eb693342eb769f782174157eba9b5924cf8ac6793897fc36a31ccbd6f56dafe2
406
407 /nix/store/rsab4y99h14912h4zplqx2iizr5n4rc2-true.dhall:4:7
408[1 built (1 failed), 0.0 MiB DL]
409error: build of '/nix/store/0f1hla7ff1wiaqyk1r2ky4wnhnw114fi-true.drv' failed
410```
411
412… because the default Prelude selected by Nixpkgs revision
413`94b2848559b12a8ed1fe433084686b2a81123c99` is version 20.1.0, which doesn't
414have the same integrity check as version 19.0.0. This means that version
41519.0.0 is not cached, and the interpreter is not allowed to fall back to
416importing the URL.
417
418However, we can override the default Prelude version by using `dhall-to-nixpkgs`
419to create a Dhall package for our desired Prelude:
420
421```ShellSession
422$ dhall-to-nixpkgs github https://github.com/dhall-lang/dhall-lang.git \
423 --name Prelude \
424 --directory Prelude \
425 --rev v19.0.0 \
426 > Prelude.nix
427```
428
429… and then referencing that package in our Dhall overlay, by either overriding
430the Prelude globally for all packages, like this:
431
432```nix
433{
434 dhallOverrides = self: super: {
435 true = self.callPackage ./true.nix { };
436
437 Prelude = self.callPackage ./Prelude.nix { };
438 };
439}
440```
441
442… or selectively overriding the Prelude dependency for just the `true` package,
443like this:
444
445```nix
446{
447 dhallOverrides = self: super: {
448 true = self.callPackage ./true.nix {
449 Prelude = self.callPackage ./Prelude.nix { };
450 };
451 };
452}
453```
454
455## Overrides {#ssec-dhall-overrides}
456
457You can override any of the arguments to `buildDhallGitHubPackage` or
458`buildDhallDirectoryPackage` using the `overridePackage` attribute of a package.
459For example, suppose we wanted to selectively enable `source = true` just for the Prelude. We can do that like this:
460
461```nix
462{
463 dhallOverrides = self: super: {
464 Prelude = super.Prelude.overridePackage { source = true; };
465
466 # ...
467 };
468}
469```
470
471[semantic-integrity-checks]: https://docs.dhall-lang.org/tutorials/Language-Tour.html#installing-packages