···
This document describes the Nixpkgs infrastructure for building Common Lisp
4
-
libraries that use ASDF (Another System Definition Facility). It lives in
5
-
`pkgs/development/lisp-modules`.
4
+
systems that use [ASDF](https://asdf.common-lisp.dev/) (Another System
5
+
Definition Facility). It lives in `pkgs/development/lisp-modules`.
## Overview {#lisp-overview}
The main entry point of the API are the Common Lisp implementation packages
10
-
(e.g. `abcl`, `ccl`, `clasp-common-lisp`, `clisp` `ecl`, `sbcl`)
11
-
themselves. They have the `pkgs` and `withPackages` attributes, which can be
12
-
used to discover available packages and to build wrappers, respectively.
10
+
themselves (e.g. `abcl`, `ccl`, `clasp-common-lisp`, `clisp`, `ecl`,
11
+
`sbcl`). They have the `pkgs` and `withPackages` attributes, which can be used
12
+
to discover available packages and to build wrappers, respectively.
14
-
The `pkgs` attribute set contains packages that were automatically imported from
15
-
Quicklisp, and any other manually defined ones. Not every package works for all
16
-
the CL implementations (e.g. `nyxt` only makes sense for `sbcl`).
14
+
The `pkgs` attribute set contains packages that were automatically
15
+
[imported](#lisp-importing-packages-from-quicklisp) from Quicklisp, and any
16
+
other [manually defined](#lisp-defining-packages-inside) ones. Not every package
17
+
works for all the CL implementations (e.g. `nyxt` only makes sense for `sbcl`).
18
-
The `withPackages` function is of primary utility. It is used to build runnable
19
-
wrappers, with a pinned and pre-built ASDF FASL available in the `ASDF`
20
-
environment variable, and `CL_SOURCE_REGISTRY`/`ASDF_OUTPUT_TRANSLATIONS`
21
-
configured to find the desired systems on runtime.
23
-
With a few exceptions, the primary thing that the infrastructure does is to run
24
-
`asdf:load-system` for each system specified in the `systems` argument to
25
-
`build-asdf-system`, and save the FASLs to the Nix store. Then, it makes these
26
-
FASLs available to wrappers. Any other use-cases, such as producing SBCL
27
-
executables with `sb-ext:save-lisp-and-die`, are achieved via overriding the
19
+
The `withPackages` function is of primary utility. It is used to build
20
+
[runnable wrappers](#lisp-building-wrappers), with a pinned and pre-built
21
+
[ASDF FASL](#lisp-loading-asdf) available in the `ASDF` environment variable,
22
+
and `CL_SOURCE_REGISTRY`/`ASDF_OUTPUT_TRANSLATIONS` configured to
23
+
[find the desired systems on runtime](#lisp-loading-systems).
In addition, Lisps have the `withOverrides` function, which can be used to
31
-
substitute any package in the scope of their `pkgs`. This will be useful
32
-
together with `overrideLispAttrs` when dealing with slashy ASDF systems, because
33
-
they should stay in the main package and be build by specifying the `systems`
26
+
[substitute](#lisp-including-external-pkg-in-scope) any package in the scope of
27
+
their `pkgs`. This will also be useful together with `overrideLispAttrs` when
28
+
[dealing with slashy systems](#lisp-dealing-with-slashy-systems), because they
29
+
should stay in the main package and be built by specifying the `systems`
argument to `build-asdf-system`.
## The 90% use case example {#lisp-use-case-example}
···
* (load (sb-ext:posix-getenv "ASDF"))
* (asdf:load-system 'alexandria)
···
sbcl' = sbcl.withPackages (ps: [ ps.alexandria ]);
56
-
buildInputs = [ sbcl' ];
52
+
packages = [ sbcl' ];
···
## Importing packages from Quicklisp {#lisp-importing-packages-from-quicklisp}
70
-
The library is able to very quickly import all the packages distributed by
71
-
Quicklisp by parsing its `releases.txt` and `systems.txt` files. These files are
72
-
available from [http://beta.quicklisp.org/dist/quicklisp.txt].
66
+
To save some work of writing Nix expressions, there is a script that imports all
67
+
the packages distributed by Quicklisp into `imported.nix`. This works by parsing
68
+
its `releases.txt` and `systems.txt` files, which are published every couple of
69
+
months on [quicklisp.org](http://beta.quicklisp.org/dist/quicklisp.txt).
The import process is implemented in the `import` directory as Common Lisp
75
-
functions in the `org.lispbuilds.nix` ASDF system. To run the script, one can
72
+
code in the `org.lispbuilds.nix` ASDF system. To run the script, one can
execute `ql-import.lisp`:
76
+
cd pkgs/development/lisp-modules
nix-shell --run 'sbcl --script ql-import.lisp'
1. Download the latest Quicklisp `systems.txt` and `releases.txt` files
85
-
2. Generate an SQLite database of all QL systems in `packages.sqlite`
83
+
2. Generate a temporary SQLite database of all QL systems in `packages.sqlite`
3. Generate an `imported.nix` file from the database
88
-
The maintainer's job there is to:
86
+
(The `packages.sqlite` file can be deleted at will, because it is regenerated
87
+
each time the script runs.)
90
-
1. Re-run the `ql-import.lisp` script
91
-
2. Add missing native dependencies in `ql.nix`
92
-
3. For packages that still don't build, package them manually in `packages.nix`
89
+
The maintainer's job is to:
91
+
1. Re-run the `ql-import.lisp` script when there is a new Quicklisp release
92
+
2. [Add any missing native dependencies](#lisp-quicklisp-adding-native-dependencies) in `ql.nix`
93
+
3. For packages that still don't build, [package them manually](#lisp-defining-packages-inside) in `packages.nix`
Also, the `imported.nix` file **must not be edited manually**! It should only be
95
-
generated as described in this section.
96
+
generated as described in this section (by running `ql-import.lisp`).
### Adding native dependencies {#lisp-quicklisp-adding-native-dependencies}
···
The previous implementation of `lisp-modules` didn't fully trust the Quicklisp
data, because there were times where the dependencies specified were not
111
-
complete, and caused broken builds. It instead used a `nix-shell` environment to
112
+
complete and caused broken builds. It instead used a `nix-shell` environment to
discover real dependencies by using the ASDF APIs.
The current implementation has chosen to trust this data, because it's faster to
···
129
-
- `+` in names are converted to `_plus{_,}`: `cl+ssl`->`cl_plus_ssl`, `alexandria+`->`alexandria_plus`
130
-
- `.` to `_dot_`: `iolib.base`->`iolib_dot_base`
130
+
- `+` in names is converted to `_plus{_,}`: `cl+ssl`->`cl_plus_ssl`, `alexandria+`->`alexandria_plus`
131
+
- `.` in names is converted to `_dot_`: `iolib.base`->`iolib_dot_base`
- names starting with a number have a `_` prepended (`3d-vectors`->`_3d-vectors`)
- `_` in names is converted to `__` for reversibility
## Defining packages manually inside Nixpkgs {#lisp-defining-packages-inside}
137
-
New packages, that for some reason are not in Quicklisp, and so cannot be
138
-
auto-imported, can be written in the `packages.nix` file.
138
+
Packages that for some reason are not in Quicklisp, and so cannot be
139
+
auto-imported, or don't work straight from the import, are defined in the
140
+
`packages.nix` file.
In that file, use the `build-asdf-system` function, which is a wrapper around
`mkDerivation` for building ASDF systems. Various other hacks are present, such
as `build-with-compile-into-pwd` for systems which create files during
145
+
compilation (such as cl-unicode).
145
-
The `build-asdf-system` function is documented with comments in
146
-
`nix-cl.nix`. Also, `packages.nix` is full of examples of how to use it.
147
+
The `build-asdf-system` function is documented
148
+
[here](#lisp-defining-packages-outside). Also, `packages.nix` is full of
149
+
examples of how to use it.
## Defining packages manually outside Nixpkgs {#lisp-defining-packages-outside}
Lisp derivations (`abcl`, `sbcl` etc.) also export the `buildASDFSystem`
151
-
function, which is the same as `build-asdf-system`, except for the `lisp`
152
-
argument which is set to the given CL implementation.
154
+
function, which is similar to `build-asdf-system` from `packages.nix`, but is
155
+
part of the public API.
157
+
It takes the following arguments:
159
+
- `pname`: the package name
160
+
- `version`: the package version
161
+
- `src`: the package source
162
+
- `patches`: patches to apply to the source before build
163
+
- `nativeLibs`: native libraries used by CFFI and grovelling
164
+
- `javaLibs`: Java libraries for ABCL
165
+
- `lispLibs`: dependencies on other packages build with `buildASDFSystem`
166
+
- `systems`: list of systems to build
It can be used to define packages outside Nixpkgs, and, for example, add them
155
-
into the package scope with `withOverrides` which will be discussed later on.
169
+
into the package scope with `withOverrides`.
### Including an external package in scope {#lisp-including-external-pkg-in-scope}
···
201
-
## Overriding packages in scope {#lisp-overriding-packages-in-scope}
203
-
Packages can be woven into a new scope by using `withOverrides`:
207
-
sbcl' = sbcl.withOverrides (self: super: {
208
-
alexandria = super.alexandria.overrideLispAttrs (oldAttrs: rec {
209
-
pname = "alexandria";
211
-
src = fetchFromGitLab {
212
-
domain = "gitlab.common-lisp.net";
213
-
owner = "alexandria";
214
-
repo = "alexandria";
215
-
rev = "v${version}";
216
-
hash = "sha256-1Hzxt65dZvgOFIljjjlSGgKYkj+YBLwJCACi5DZsKmQ=";
220
-
in builtins.elemAt sbcl'.pkgs.bordeaux-threads.lispLibs 0
### Dealing with slashy systems {#lisp-dealing-with-slashy-systems}
Slashy (secondary) systems should not exist in their own packages! Instead, they
···
243
-
See the respective section on using `withOverrides` for how to weave it back
235
+
See the [respective section](#lisp-including-external-pkg-in-scope) on using
236
+
`withOverrides` for how to weave it back into `ecl.pkgs`.
Note that sometimes the slashy systems might not only have more dependencies
than the main one, but create a circular dependency between `.asd`
···
implementations (`abcl`, `ecl`, `sbcl` etc.):
256
-
sbcl.withPackages (ps: [ ps.alexandria ps.bordeaux-threads ])
248
+
nix-shell -p 'sbcl.withPackages (ps: [ ps.alexandria ps.bordeaux-threads ])'
259
-
Such a wrapper can then be executed like this:
251
+
Such a wrapper can then be used like this:
255
+
* (load (sb-ext:posix-getenv "ASDF"))
256
+
* (asdf:load-system 'alexandria)
257
+
* (asdf:load-system 'bordeaux-threads)
### Loading ASDF {#lisp-loading-asdf}
pyrox.dev