···

       
1116
       
 
       
### New packages

     

       
1117
       
 
       


     

       
1118
       
 
       
New packages are a common type of pull requests.

     

       
1119
       
-
       
These pull requests consists in adding a new nix-expression for a package.

     

       
1120
       
 
       


     

       
1121
       
 
       
Review process:

     

       
1122
       
 
       


     
···

       
1131
       
 
       
  - Maintainers must be set.

     

       
1132
       
 
       
    This can be the package submitter or a community member that accepts taking up maintainership of the package.

     

       
1133
       
 
       
  - The `meta.mainProgram` must be set if a main executable exists.

     

       
1134
       
-
       
- Ensure any special packaging choices and required context are documented in i.e. the name of a patch or in a comment.

     

       
1135
       
 
       
  - If a special version of a package is pinned, document why, so others know if/when it can be unpinned.

     

       
1136
       
 
       
  - If any (especially opinionated) patch or `substituteInPlace` is applied, document why.

     

       
1137
       
 
       
  - If any non-default build flags are set, document why.

     
···

       
1204
       
 
       


     

       
1205
       
 
       
[github.com/NixOS/nixpkgs/issues?q=is:issue+is:open+"Vulnerability+roundup"](https://github.com/NixOS/nixpkgs/issues?q=is%3Aissue+is%3Aopen+%22Vulnerability+roundup%22)

     

       
1206
       
 
       


     

       
1207
       
-
       
Each issue correspond to a vulnerable version of a package; As a consequence:

     

       
1208
       
 
       


     

       
1209
       
 
       
- One issue can contain several CVEs;

     

       
1210
       
 
       
- One CVE can be shared across several issues;

     

···

       
1116
       
 
       
### New packages

     

       
1117
       
 
       


     

       
1118
       
 
       
New packages are a common type of pull requests.

     

       
1119
       
+
       
These pull requests consist in adding a new nix-expression for a package.

     

       
1120
       
 
       


     

       
1121
       
 
       
Review process:

     

       
1122
       
 
       


     
···

       
1131
       
 
       
  - Maintainers must be set.

     

       
1132
       
 
       
    This can be the package submitter or a community member that accepts taking up maintainership of the package.

     

       
1133
       
 
       
  - The `meta.mainProgram` must be set if a main executable exists.

     

       
1134
       
+
       
- Ensure any special packaging choices and required context are documented in, i.e., the name of a patch or in a comment.

     

       
1135
       
 
       
  - If a special version of a package is pinned, document why, so others know if/when it can be unpinned.

     

       
1136
       
 
       
  - If any (especially opinionated) patch or `substituteInPlace` is applied, document why.

     

       
1137
       
 
       
  - If any non-default build flags are set, document why.

     
···

       
1204
       
 
       


     

       
1205
       
 
       
[github.com/NixOS/nixpkgs/issues?q=is:issue+is:open+"Vulnerability+roundup"](https://github.com/NixOS/nixpkgs/issues?q=is%3Aissue+is%3Aopen+%22Vulnerability+roundup%22)

     

       
1206
       
 
       


     

       
1207
       
+
       
Each issue corresponds to a vulnerable version of a package; as a consequence:

     

       
1208
       
 
       


     

       
1209
       
 
       
- One issue can contain several CVEs;

     

       
1210
       
 
       
- One CVE can be shared across several issues;

     

···

       
11
       
 
       
  - `ungoogled-chromium`: A patch set for Chromium, that has its own entry in Chromium's `upstream-info.nix`.

     

       
12
       
 
       
  - `chromedriver`: Updated via Chromium's `upstream-info.nix` and not built

     

       
13
       
 
       
    from source. Must match Chromium's major version.

     

       
14
       
-
       
  - `electron-source`: Various version of electron that are built from source using Chromium's

     

       
15
       
 
       
    `-unwrapped` derivation, due to electron being based on Chromium.

     

       
16
       
 
       


     

       
17
       
 
       
# Upstream links

     

···

       
11
       
 
       
  - `ungoogled-chromium`: A patch set for Chromium, that has its own entry in Chromium's `upstream-info.nix`.

     

       
12
       
 
       
  - `chromedriver`: Updated via Chromium's `upstream-info.nix` and not built

     

       
13
       
 
       
    from source. Must match Chromium's major version.

     

       
14
       
+
       
  - `electron-source`: Various versions of electron that are built from source using Chromium's

     

       
15
       
 
       
    `-unwrapped` derivation, due to electron being based on Chromium.

     

       
16
       
 
       


     

       
17
       
 
       
# Upstream links

     

···

       
1
       
 
       
# K3s

     

       
2
       
 
       


     

       
3
       
-
       
K3s is a simplified [Kubernetes](https://wiki.nixos.org/wiki/Kubernetes) version that bundles  Kubernetes cluster components into a few small binaries optimized for Edge and IoT devices.

     

       
4
       
 
       


     

       
5
       
 
       
## Usage

     

       
6
       
 
       


     

···

       
1
       
 
       
# K3s

     

       
2
       
 
       


     

       
3
       
+
       
K3s is a simplified [Kubernetes](https://wiki.nixos.org/wiki/Kubernetes) version that bundles Kubernetes cluster components into a few small binaries optimized for Edge and IoT devices.

     

       
4
       
 
       


     

       
5
       
 
       
## Usage

     

       
6
       
 
       


     

···

       
24
       
 
       
Nixpkgs follows active minor version release channels (typically 4 at a time) and sets aliases for

     

       
25
       
 
       
`rke2_stable` and `rke2_latest` accordingly.

     

       
26
       
 
       


     

       
27
       
-
       
Patch releases should be backported to to the latest stable release branch, however, new minor

     

       
28
       
 
       
versions are not backported.

     

       
29
       
 
       


     

       
30
       
 
       
For further information visit the

     

···

       
24
       
 
       
Nixpkgs follows active minor version release channels (typically 4 at a time) and sets aliases for

     

       
25
       
 
       
`rke2_stable` and `rke2_latest` accordingly.

     

       
26
       
 
       


     

       
27
       
+
       
Patch releases should be backported to the latest stable release branch; however, new minor

     

       
28
       
 
       
versions are not backported.

     

       
29
       
 
       


     

       
30
       
 
       
For further information visit the

     

···

       
39
       
 
       


     

       
40
       
 
       
    When an end-of-life toolchain is removed, builders that pin the EOL version (according to 3.) will automatically be bumped to the then oldest pinned builder (e.g. Go 1.22 is EOL, `buildGo122Module` is bumped to `buildGo123Module`).

     

       
41
       
 
       


     

       
42
       
-
       
    If the package won't build with anymore with that builder, the package is marked broken.

     

       
43
       
 
       
    It is the package maintainers responsibility to fix the package and get it working with a supported Go toolchain.

     

       
44
       
 
       


     

       
45
       
 
       
[1]: http://go.dev/doc/go1compat

     

···

       
39
       
 
       


     

       
40
       
 
       
    When an end-of-life toolchain is removed, builders that pin the EOL version (according to 3.) will automatically be bumped to the then oldest pinned builder (e.g. Go 1.22 is EOL, `buildGo122Module` is bumped to `buildGo123Module`).

     

       
41
       
 
       


     

       
42
       
+
       
    If the package won't build with that builder anymore, the package is marked broken.

     

       
43
       
 
       
    It is the package maintainers responsibility to fix the package and get it working with a supported Go toolchain.

     

       
44
       
 
       


     

       
45
       
 
       
[1]: http://go.dev/doc/go1compat

     

···

       
110
       
 
       


     

       
111
       
 
       
## Limitations

     

       
112
       
 
       


     

       
113
       
-
       
There's some limitations as to which packages can be defined using this structure:

     

       
114
       
 
       


     

       
115
       
 
       
- Only packages defined using `pkgs.callPackage`.

     

       
116
       
 
       
  This excludes packages defined using `pkgs.python3Packages.callPackage ...`.

     

···

       
110
       
 
       


     

       
111
       
 
       
## Limitations

     

       
112
       
 
       


     

       
113
       
+
       
There are some limitations as to which packages can be defined using this structure:

     

       
114
       
 
       


     

       
115
       
 
       
- Only packages defined using `pkgs.callPackage`.

     

       
116
       
 
       
  This excludes packages defined using `pkgs.python3Packages.callPackage ...`.

     

···

       
87
       
 
       


     

       
88
       
 
       
Check if the desired functionality was added.

     

       
89
       
 
       


     

       
90
       
-
       
You can check if the extensions was recognized by running:

     

       
91
       
 
       


     

       
92
       
 
       
```sh

     

       
93
       
 
       
./result/bin/az extension list

     

···

       
87
       
 
       


     

       
88
       
 
       
Check if the desired functionality was added.

     

       
89
       
 
       


     

       
90
       
+
       
You can check if the extensions were recognized by running:

     

       
91
       
 
       


     

       
92
       
 
       
```sh

     

       
93
       
 
       
./result/bin/az extension list

     

···

       
1
       
 
       
This package supports the following parameters:

     

       
2
       
 
       


     

       
3
       
 
       
- withWayland (default: true): when false, set QT_QPA_PLATFORM to xcb

     

       
4
       
-
       
- spaceNavSupport (enabled by default on linux): whether to enable

     

       
5
       
 
       
  [spacenavd support](https://spacenav.sourceforge.net/)

     

       
6
       
 
       
- ifcSupport (default: false): whether to enable ifc support through

     

       
7
       
 
       
  ifcopenshell

     

···

       
1
       
 
       
This package supports the following parameters:

     

       
2
       
 
       


     

       
3
       
 
       
- withWayland (default: true): when false, set QT_QPA_PLATFORM to xcb

     

       
4
       
+
       
- spaceNavSupport (enabled by default on Linux): whether to enable

     

       
5
       
 
       
  [spacenavd support](https://spacenav.sourceforge.net/)

     

       
6
       
 
       
- ifcSupport (default: false): whether to enable ifc support through

     

       
7
       
 
       
  ifcopenshell

     

···

       
34
       
 
       


     

       
35
       
 
       
## Remove release

     

       
36
       
 
       


     

       
37
       
-
       
Kanidm versions are supported for 30 days after the release of new versions. Following the example above, 1.5.x superseding 1.4.x in 30 days, do the following near the end of the 30 day window

     

       
38
       
 
       


     

       
39
       
 
       
1. Update `pkgs/by-name/ka/kanidm/1_4.nix` by adding `unsupported = true;`

     

       
40
       
 
       
1. Update `pkgs/top-level/release.nix` and add `kanidm_1_4-1.4.6` and `kanidmWithSecretProvisioning_1_4-1.4.6` to `permittedInsecurePackages`

     

···

       
34
       
 
       


     

       
35
       
 
       
## Remove release

     

       
36
       
 
       


     

       
37
       
+
       
Kanidm versions are supported for 30 days after the release of new versions. Following the example above, 1.5.x superseding 1.4.x in 30 days, do the following near the end of the 30-day window

     

       
38
       
 
       


     

       
39
       
 
       
1. Update `pkgs/by-name/ka/kanidm/1_4.nix` by adding `unsupported = true;`

     

       
40
       
 
       
1. Update `pkgs/top-level/release.nix` and add `kanidm_1_4-1.4.6` and `kanidmWithSecretProvisioning_1_4-1.4.6` to `permittedInsecurePackages`

     

···

       
7
       
 
       


     

       
8
       
 
       
The current state of `nixos-rebuild` is dire: it is one of the most critical

     

       
9
       
 
       
pieces of code we have in NixOS, but it has tons of issues:

     

       
10
       
-
       
- The code is written in Bash, and while this by itself is not necessary bad,

     

       
11
       
 
       
  it means that it is difficult to do refactorings due to the lack of tooling

     

       
12
       
 
       
  for the language

     

       
13
       
 
       
- The code itself is a hacky mess. Changing even one line of code can cause

     
···

       
18
       
 
       
  builds Flakes inside a temporary directory and reads the resulting symlink

     

       
19
       
 
       
  since the code seems to predate `--print-out-paths` flag

     

       
20
       
 
       


     

       
21
       
-
       
Given all of those above, improvements in the `nixos-rebuild` are difficult to

     

       
22
       
 
       
do. A full rewrite is probably the easier way to improve the situation since

     

       
23
       
 
       
this can be done in a separate package that will not break anyone. So this is

     

       
24
       
 
       
an attempt of the rewrite.

     
···

       
44
       
 
       
```

     

       
45
       
 
       


     

       
46
       
 
       
The command above will build, run the unit tests and linters, and also check if

     

       
47
       
-
       
the code is formatted. However, sometimes is more convenient to run just a few

     

       
48
       
 
       
tests to debug, in this case you can run:

     

       
49
       
 
       


     

       
50
       
 
       
```console

     
···

       
97
       
 
       
  please open an issue

     

       
98
       
 
       
- We do some additional validation of flags, like exiting with an error when

     

       
99
       
 
       
  `--build-host` or `--target-host` is used with `repl`, since the user could

     

       
100
       
-
       
  assume that the `repl` would be run remotely while it always run the local

     

       
101
       
 
       
  machine. `nixos-rebuild` silently ignored those flags, so this

     

       
102
       
 
       
  [may cause some issues](https://github.com/NixOS/nixpkgs/pull/363922) for

     

       
103
       
 
       
  wrappers

     
···

       
108
       
 
       
  may be difficult to fix, so right now I only recommend using

     

       
109
       
 
       
  `nixos-rebuild-ng` if you are testing in a VM or in a filesystem with

     

       
110
       
 
       
  snapshots like BTRFS or ZFS. Those bugs are unlikely to be unfixable but the

     

       
111
       
-
       
  errors can be difficult to understand. If you want to go anyway,

     

       
112
       
 
       
  `nix-collect-garbage -d` and `nix store repair` are your friends

     

       
113
       
 
       


     

       
114
       
 
       
## TODON'T

     

···

       
7
       
 
       


     

       
8
       
 
       
The current state of `nixos-rebuild` is dire: it is one of the most critical

     

       
9
       
 
       
pieces of code we have in NixOS, but it has tons of issues:

     

       
10
       
+
       
- The code is written in Bash, and while this by itself is not necessarily bad,

     

       
11
       
 
       
  it means that it is difficult to do refactorings due to the lack of tooling

     

       
12
       
 
       
  for the language

     

       
13
       
 
       
- The code itself is a hacky mess. Changing even one line of code can cause

     
···

       
18
       
 
       
  builds Flakes inside a temporary directory and reads the resulting symlink

     

       
19
       
 
       
  since the code seems to predate `--print-out-paths` flag

     

       
20
       
 
       


     

       
21
       
+
       
Given all of the above, improvements in the `nixos-rebuild` are difficult to

     

       
22
       
 
       
do. A full rewrite is probably the easier way to improve the situation since

     

       
23
       
 
       
this can be done in a separate package that will not break anyone. So this is

     

       
24
       
 
       
an attempt of the rewrite.

     
···

       
44
       
 
       
```

     

       
45
       
 
       


     

       
46
       
 
       
The command above will build, run the unit tests and linters, and also check if

     

       
47
       
+
       
the code is formatted. However, sometimes it's more convenient to run just a few

     

       
48
       
 
       
tests to debug, in this case you can run:

     

       
49
       
 
       


     

       
50
       
 
       
```console

     
···

       
97
       
 
       
  please open an issue

     

       
98
       
 
       
- We do some additional validation of flags, like exiting with an error when

     

       
99
       
 
       
  `--build-host` or `--target-host` is used with `repl`, since the user could

     

       
100
       
+
       
  assume that the `repl` would be run remotely while it always runs the local

     

       
101
       
 
       
  machine. `nixos-rebuild` silently ignored those flags, so this

     

       
102
       
 
       
  [may cause some issues](https://github.com/NixOS/nixpkgs/pull/363922) for

     

       
103
       
 
       
  wrappers

     
···

       
108
       
 
       
  may be difficult to fix, so right now I only recommend using

     

       
109
       
 
       
  `nixos-rebuild-ng` if you are testing in a VM or in a filesystem with

     

       
110
       
 
       
  snapshots like BTRFS or ZFS. Those bugs are unlikely to be unfixable but the

     

       
111
       
+
       
  errors can be difficult to understand. If you want to go on,

     

       
112
       
 
       
  `nix-collect-garbage -d` and `nix store repair` are your friends

     

       
113
       
 
       


     

       
114
       
 
       
## TODON'T

     

···

       
40
       
 
       
    - It would also require keeping an impure or otherwise continuously updated reference to those other revisions.

     

       
41
       
 
       
    - The static mapping acts like a semi-automatically updated cache that we drag along with version history.

     

       
42
       
 
       
    - Other setups, such as a dedicated service to cache a history of moved content, are more complicated and would still be impure.

     

       
43
       
-
       
- Checking in large amounts of data that is touched often, bears a risk of more merge conflicts or related build failures.

     

       
44
       
 
       


     

       
45
       
 
       
The solution picked here is to have a static mapping of the historical locations checked into the Git tree, such that it can be read during the build process.

     

       
46
       
 
       
This also ensures that an improper redirect mapping will cause `nixos-render-docs` to fail the build and thus enforce that redirects stay up-to-date with every commit.

     

···

       
40
       
 
       
    - It would also require keeping an impure or otherwise continuously updated reference to those other revisions.

     

       
41
       
 
       
    - The static mapping acts like a semi-automatically updated cache that we drag along with version history.

     

       
42
       
 
       
    - Other setups, such as a dedicated service to cache a history of moved content, are more complicated and would still be impure.

     

       
43
       
+
       
- Checking in large amounts of data that is touched often bears a risk of more merge conflicts or related build failures.

     

       
44
       
 
       


     

       
45
       
 
       
The solution picked here is to have a static mapping of the historical locations checked into the Git tree, such that it can be read during the build process.

     

       
46
       
 
       
This also ensures that an improper redirect mapping will cause `nixos-render-docs` to fail the build and thus enforce that redirects stay up-to-date with every commit.

     

···

       
1
       
 
       
# switch-to-configuration-ng

     

       
2
       
 
       


     

       
3
       
-
       
This program implements the switching/updating of NixOS systems. It starts with the exising running configuration at `/run/current-system` and handles the migration to a new configuration, built from a NixOS configuration's `config.system.build.toplevel` derivation.

     

       
4
       
 
       


     

       
5
       
 
       
For more information on what happens during a switch, see [what-happens-during-a-system-switch](../../../../nixos/doc/manual/development/what-happens-during-a-system-switch.chapter.md).

     

       
6
       
 
       


     

···

       
1
       
 
       
# switch-to-configuration-ng

     

       
2
       
 
       


     

       
3
       
+
       
This program implements the switching/updating of NixOS systems. It starts with the existing running configuration at `/run/current-system` and handles the migration to a new configuration, built from a NixOS configuration's `config.system.build.toplevel` derivation.

     

       
4
       
 
       


     

       
5
       
 
       
For more information on what happens during a switch, see [what-happens-during-a-system-switch](../../../../nixos/doc/manual/development/what-happens-during-a-system-switch.chapter.md).

     

       
6
       
 
       


     

···

       
12
       
 
       
(versions.dat).

     

       
13
       
 
       


     

       
14
       
 
       
The makeDotElm function lets us retrieve these dependencies in the

     

       
15
       
-
       
standard nix way. we have to copy them in (rather than symlink) and

     

       
16
       
 
       
make them writable because the elm compiler writes other .dat files

     

       
17
       
 
       
alongside the source code. versions.dat was produced during an

     

       
18
       
 
       
impure build of this same code; the build complains that it can't

     

···

       
12
       
 
       
(versions.dat).

     

       
13
       
 
       


     

       
14
       
 
       
The makeDotElm function lets us retrieve these dependencies in the

     

       
15
       
+
       
standard nix way. We have to copy them in (rather than symlink) and

     

       
16
       
 
       
make them writable because the elm compiler writes other .dat files

     

       
17
       
 
       
alongside the source code. versions.dat was produced during an

     

       
18
       
 
       
impure build of this same code; the build complains that it can't

     

···

       
9
       
 
       
[julia]: https://julialang.org

     

       
10
       
 
       


     

       
11
       
 
       
For Nixpkgs, the manual is as always your primary reference, and for the Julia

     

       
12
       
-
       
side of things you probably want to familiarise yourself with the [README

     

       
13
       
 
       
][readme], [build instructions][build], and [release process][release_process].

     

       
14
       
 
       
Remember that these can change between Julia releases, especially if the LTS and

     

       
15
       
 
       
release branches have deviated greatly. A lot of the build process is

     

···

       
9
       
 
       
[julia]: https://julialang.org

     

       
10
       
 
       


     

       
11
       
 
       
For Nixpkgs, the manual is as always your primary reference, and for the Julia

     

       
12
       
+
       
side of things, you probably want to familiarise yourself with the [README

     

       
13
       
 
       
][readme], [build instructions][build], and [release process][release_process].

     

       
14
       
 
       
Remember that these can change between Julia releases, especially if the LTS and

     

       
15
       
 
       
release branches have deviated greatly. A lot of the build process is

     

···

       
2
       
 
       


     

       
3
       
 
       
- Run `update-git.py`.

     

       
4
       
 
       
  This will set the github revision and sha256 for `llvmPackages_git.llvm` to whatever the latest chromium build is using.

     

       
5
       
-
       
  For a more recent, commit run `nix-prefetch-github` and change the rev and sha256 accordingly.

     

       
6
       
 
       


     

       
7
       
 
       
- That was the easy part.

     

       
8
       
 
       
  The hard part is updating the patch files.

     
···

       
50
       
 
       
    which can be an easily missed reason for failures.

     

       
51
       
 
       
    For cases where the hunk is no longer needed you can simply remove it from the patch.

     

       
52
       
 
       


     

       
53
       
-
       
 This is fine for small corrections, but when more serious changes are needed its better to use git.

     

       
54
       
 
       


     

       
55
       
 
       
 1. Clone the LLVM monorepo at https://github.com/llvm/llvm-project/

     

       
56
       
 
       


     
···

       
68
       
 
       


     

       
69
       
 
       
Use CMake's [`GNUInstallDirs`](https://cmake.org/cmake/help/latest/module/GNUInstallDirs.html) to support multiple outputs.

     

       
70
       
 
       


     

       
71
       
-
       
Previously, LLVM Just hard-coded `bin`, `include`, and `lib${LLVM_TARGET_PREFIX}`.

     

       
72
       
 
       
We are making it use these variables.

     

       
73
       
 
       


     

       
74
       
 
       
For the older LLVM versions, these patches live in https://github.com/Ericson2314/llvm-project branches `split-prefix`.

     

···

       
2
       
 
       


     

       
3
       
 
       
- Run `update-git.py`.

     

       
4
       
 
       
  This will set the github revision and sha256 for `llvmPackages_git.llvm` to whatever the latest chromium build is using.

     

       
5
       
+
       
  For a more recent commit, run `nix-prefetch-github` and change the rev and sha256 accordingly.

     

       
6
       
 
       


     

       
7
       
 
       
- That was the easy part.

     

       
8
       
 
       
  The hard part is updating the patch files.

     
···

       
50
       
 
       
    which can be an easily missed reason for failures.

     

       
51
       
 
       
    For cases where the hunk is no longer needed you can simply remove it from the patch.

     

       
52
       
 
       


     

       
53
       
+
       
 This is fine for small corrections, but when more serious changes are needed, it's better to use git.

     

       
54
       
 
       


     

       
55
       
 
       
 1. Clone the LLVM monorepo at https://github.com/llvm/llvm-project/

     

       
56
       
 
       


     
···

       
68
       
 
       


     

       
69
       
 
       
Use CMake's [`GNUInstallDirs`](https://cmake.org/cmake/help/latest/module/GNUInstallDirs.html) to support multiple outputs.

     

       
70
       
 
       


     

       
71
       
+
       
Previously, LLVM just hard-coded `bin`, `include`, and `lib${LLVM_TARGET_PREFIX}`.

     

       
72
       
 
       
We are making it use these variables.

     

       
73
       
 
       


     

       
74
       
 
       
For the older LLVM versions, these patches live in https://github.com/Ericson2314/llvm-project branches `split-prefix`.

     

···

       
1
       
-
       
# Cuda modules

     

       
2
       
 
       


     

       
3
       
 
       
> [!NOTE]

     

       
4
       
 
       
> This document is meant to help CUDA maintainers understand the structure of

     
···

       
43
       
 
       


     

       
44
       
 
       
## Distinguished packages

     

       
45
       
 
       


     

       
46
       
-
       
### Cuda compatibility

     

       
47
       
 
       


     

       
48
       
-
       
[Cuda Compatibility](https://docs.nvidia.com/deploy/cuda-compatibility/),

     

       
49
       
 
       
available as `cudaPackages.cuda_compat`, is a component which makes it possible

     

       
50
       
 
       
to run applications built against a newer CUDA toolkit (for example CUDA 12) on

     

       
51
       
 
       
a machine with an older CUDA driver (for example CUDA 11), which isn't possible

     

       
52
       
-
       
out of the box. At the time of writing, Cuda Compatibility is only available on

     

       
53
       
 
       
the Nvidia Jetson architecture, but Nvidia might release support for more

     

       
54
       
 
       
architectures in the future.

     

       
55
       
 
       


     

       
56
       
-
       
As Cuda Compatibility strictly increases the range of supported applications, we

     

       
57
       
 
       
try our best to enable it by default on supported platforms.

     

       
58
       
 
       


     

       
59
       
 
       
#### Functioning

     
···

       
64
       
 
       
`cuda_compat` isn't a complete drop-in replacement for the driver (and that's

     

       
65
       
 
       
the point, otherwise, it would just be a newer driver).

     

       
66
       
 
       


     

       
67
       
-
       
Nvidia's recommendation is to set `LD_LIBRARY_PATH` to points to `cuda_compat`'s

     

       
68
       
 
       
driver. This is fine for a manual, one-shot usage, but in general setting

     

       
69
       
 
       
`LD_LIBRARY_PATH` is a red flag. This is global state which short-circuits most

     

       
70
       
-
       
of other dynamic libraries resolution mechanisms and can break things in

     

       
71
       
 
       
non-obvious ways, especially with other Nix-built software.

     

       
72
       
 
       


     

       
73
       
-
       
#### Cuda compat with Nix

     

       
74
       
 
       


     

       
75
       
 
       
Since `cuda_compat` is a known derivation, the easy way to do this in Nix would

     

       
76
       
 
       
be to add `cuda_compat` as a dependency of CUDA libraries and applications and

     

       
77
       
-
       
let Nix does its magic by filling the `DT_RUNPATH` fields. However,

     

       
78
       
 
       
`cuda_compat` itself depends on `libnvrm_mem` and `libnvrm_gpu` which are loaded

     

       
79
       
 
       
dynamically at runtime from `/run/opengl-driver`. This doesn't please the Nix

     

       
80
       
 
       
sandbox when building, which can't find those (a second minor issue is that

     

···

       
1
       
+
       
# CUDA Modules

     

       
2
       
 
       


     

       
3
       
 
       
> [!NOTE]

     

       
4
       
 
       
> This document is meant to help CUDA maintainers understand the structure of

     
···

       
43
       
 
       


     

       
44
       
 
       
## Distinguished packages

     

       
45
       
 
       


     

       
46
       
+
       
### CUDA Compatibility

     

       
47
       
 
       


     

       
48
       
+
       
[CUDA Compatibility](https://docs.nvidia.com/deploy/cuda-compatibility/),

     

       
49
       
 
       
available as `cudaPackages.cuda_compat`, is a component which makes it possible

     

       
50
       
 
       
to run applications built against a newer CUDA toolkit (for example CUDA 12) on

     

       
51
       
 
       
a machine with an older CUDA driver (for example CUDA 11), which isn't possible

     

       
52
       
+
       
out of the box. At the time of writing, CUDA Compatibility is only available on

     

       
53
       
 
       
the Nvidia Jetson architecture, but Nvidia might release support for more

     

       
54
       
 
       
architectures in the future.

     

       
55
       
 
       


     

       
56
       
+
       
As CUDA Compatibility strictly increases the range of supported applications, we

     

       
57
       
 
       
try our best to enable it by default on supported platforms.

     

       
58
       
 
       


     

       
59
       
 
       
#### Functioning

     
···

       
64
       
 
       
`cuda_compat` isn't a complete drop-in replacement for the driver (and that's

     

       
65
       
 
       
the point, otherwise, it would just be a newer driver).

     

       
66
       
 
       


     

       
67
       
+
       
Nvidia's recommendation is to set `LD_LIBRARY_PATH` to point to `cuda_compat`'s

     

       
68
       
 
       
driver. This is fine for a manual, one-shot usage, but in general setting

     

       
69
       
 
       
`LD_LIBRARY_PATH` is a red flag. This is global state which short-circuits most

     

       
70
       
+
       
of other dynamic library resolution mechanisms and can break things in

     

       
71
       
 
       
non-obvious ways, especially with other Nix-built software.

     

       
72
       
 
       


     

       
73
       
+
       
#### CUDA Compat with Nix

     

       
74
       
 
       


     

       
75
       
 
       
Since `cuda_compat` is a known derivation, the easy way to do this in Nix would

     

       
76
       
 
       
be to add `cuda_compat` as a dependency of CUDA libraries and applications and

     

       
77
       
+
       
let Nix do its magic by filling the `DT_RUNPATH` fields. However,

     

       
78
       
 
       
`cuda_compat` itself depends on `libnvrm_mem` and `libnvrm_gpu` which are loaded

     

       
79
       
 
       
dynamically at runtime from `/run/opengl-driver`. This doesn't please the Nix

     

       
80
       
 
       
sandbox when building, which can't find those (a second minor issue is that

     

···

       
4
       
 
       


     

       
5
       
 
       
# How to run tests

     

       
6
       
 
       


     

       
7
       
-
       
You may need to make yourself familiar with [package tests](../../../README.md#package-tests), and [Writing larger package tests](../../../README.md#writing-larger-package-tests), then run tests locally with:

     

       
8
       
 
       


     

       
9
       
 
       
```shell

     

       
10
       
 
       
$ export NIXPKGS_ALLOW_UNFREE=1

     

···

       
4
       
 
       


     

       
5
       
 
       
# How to run tests

     

       
6
       
 
       


     

       
7
       
+
       
You may need to make yourself familiar with [package tests](../../../README.md#package-tests) and [Writing larger package tests](../../../README.md#writing-larger-package-tests), then run tests locally with:

     

       
8
       
 
       


     

       
9
       
 
       
```shell

     

       
10
       
 
       
$ export NIXPKGS_ALLOW_UNFREE=1

     

···

       
3
       
 
       
## Introduction

     

       
4
       
 
       


     

       
5
       
 
       
Gradle build scripts are written in a DSL, computing the list of Gradle

     

       
6
       
-
       
dependencies is a turing-complete task, not just in theory but in

     

       
7
       
 
       
practice. Fetching all of the dependencies often requires building some

     

       
8
       
 
       
native code, running some commands to check the host platform, or just

     

       
9
       
 
       
fetching some files using either JVM code or commands like `curl` or

     
···

       
22
       
 
       
Obviously, this is horrible for reproducibility. Additionally, Gradle

     

       
23
       
 
       
doesn't offer a way to export the list of dependency URLs and hashes (it

     

       
24
       
 
       
does in a way, but it's far from being complete, and as such is useless

     

       
25
       
-
       
for nixpkgs). Even if did, it would be annoying to use considering

     

       
26
       
 
       
fetching non-Gradle dependencies in Gradle scripts is commonplace.

     

       
27
       
 
       


     

       
28
       
 
       
That's why the setup hook uses mitm-cache, a program designed for

     
···

       
89
       
 
       
and finally download `b.jar.sha1`, locate it in its cache, and then

     

       
90
       
 
       
*not* download `b.jar`. This means `b.jar` won't be stored in the MITM

     

       
91
       
 
       
cache. Then, consider that on a later invocation, the fetching order

     

       
92
       
-
       
changed, whether it was because of a running on different system,

     

       
93
       
 
       
changed behavior after a Gradle update, or any other source of

     

       
94
       
 
       
nondeterminism - `b.jar` is fetched before `a.jar`. Gradle will first

     

       
95
       
 
       
fetch `b.jar.sha1`, not find it in its cache, attempt to fetch `b.jar`,

     
···

       
103
       
 
       
via CLI arguments.

     

       
104
       
 
       


     

       
105
       
 
       
**Caveat**: Gradle .module files also contain file hashes, in md5, sha1,

     

       
106
       
-
       
sha256, sha512 formats. It posed no problem as of yet, but it might in

     

       
107
       
 
       
the future. If it does pose problems, the deps derivation code can be

     

       
108
       
 
       
extended to find all checksums in .module files and copy existing files

     

       
109
       
 
       
there if their hash matches.

     
···

       
173
       
 
       
The mitm-cache lockfile format is described in the [mitm-cache

     

       
174
       
 
       
README](https://github.com/chayleaf/mitm-cache#readme).

     

       
175
       
 
       


     

       
176
       
-
       
The nixpkgs Gradle lockfile format is more complicated:

     

       
177
       
 
       


     

       
178
       
 
       
```json

     

       
179
       
 
       
{

     

       
180
       
-
       
  "!comment": "This is a nixpkgs Gradle dependency lockfile. For more details, refer to the Gradle section in the nixpkgs manual.",

     

       
181
       
 
       
  "!version": 1,

     

       
182
       
 
       
  "https://oss.sonatype.org/content/repositories/snapshots/com/badlogicgames/gdx-controllers": {

     

       
183
       
 
       
    "gdx-controllers#gdx-controllers-core/2.2.4-20231021.200112-6/SNAPSHOT": {

     
···

       
223
       
 
       
  discern where the repo base ends and the group ID begins).

     

       
224
       
 
       


     

       
225
       
 
       
`compress-deps-json.py` converts the JSON from mitm-cache format into

     

       
226
       
-
       
nixpkgs Gradle lockfile format. `fetch.nix` does the opposite.

     

       
227
       
 
       


     

       
228
       
 
       
## Security Considerations

     

       
229
       
 
       


     
···

       
242
       
 
       
  doesn't match)

     

       
243
       
 
       


     

       
244
       
 
       
Please be mindful of the above when working on Gradle support for

     

       
245
       
-
       
nixpkgs.

     

···

       
3
       
 
       
## Introduction

     

       
4
       
 
       


     

       
5
       
 
       
Gradle build scripts are written in a DSL, computing the list of Gradle

     

       
6
       
+
       
dependencies is a Turing-complete task, not just in theory but also in

     

       
7
       
 
       
practice. Fetching all of the dependencies often requires building some

     

       
8
       
 
       
native code, running some commands to check the host platform, or just

     

       
9
       
 
       
fetching some files using either JVM code or commands like `curl` or

     
···

       
22
       
 
       
Obviously, this is horrible for reproducibility. Additionally, Gradle

     

       
23
       
 
       
doesn't offer a way to export the list of dependency URLs and hashes (it

     

       
24
       
 
       
does in a way, but it's far from being complete, and as such is useless

     

       
25
       
+
       
for Nixpkgs). Even if it did, it would be annoying to use considering

     

       
26
       
 
       
fetching non-Gradle dependencies in Gradle scripts is commonplace.

     

       
27
       
 
       


     

       
28
       
 
       
That's why the setup hook uses mitm-cache, a program designed for

     
···

       
89
       
 
       
and finally download `b.jar.sha1`, locate it in its cache, and then

     

       
90
       
 
       
*not* download `b.jar`. This means `b.jar` won't be stored in the MITM

     

       
91
       
 
       
cache. Then, consider that on a later invocation, the fetching order

     

       
92
       
+
       
changed, whether it was because of running on a different system,

     

       
93
       
 
       
changed behavior after a Gradle update, or any other source of

     

       
94
       
 
       
nondeterminism - `b.jar` is fetched before `a.jar`. Gradle will first

     

       
95
       
 
       
fetch `b.jar.sha1`, not find it in its cache, attempt to fetch `b.jar`,

     
···

       
103
       
 
       
via CLI arguments.

     

       
104
       
 
       


     

       
105
       
 
       
**Caveat**: Gradle .module files also contain file hashes, in md5, sha1,

     

       
106
       
+
       
sha256, sha512 formats. It has posed no problem as of yet, but it might in

     

       
107
       
 
       
the future. If it does pose problems, the deps derivation code can be

     

       
108
       
 
       
extended to find all checksums in .module files and copy existing files

     

       
109
       
 
       
there if their hash matches.

     
···

       
173
       
 
       
The mitm-cache lockfile format is described in the [mitm-cache

     

       
174
       
 
       
README](https://github.com/chayleaf/mitm-cache#readme).

     

       
175
       
 
       


     

       
176
       
+
       
The Nixpkgs Gradle lockfile format is more complicated:

     

       
177
       
 
       


     

       
178
       
 
       
```json

     

       
179
       
 
       
{

     

       
180
       
+
       
  "!comment": "This is a Nixpkgs Gradle dependency lockfile. For more details, refer to the Gradle section in the Nixpkgs manual.",

     

       
181
       
 
       
  "!version": 1,

     

       
182
       
 
       
  "https://oss.sonatype.org/content/repositories/snapshots/com/badlogicgames/gdx-controllers": {

     

       
183
       
 
       
    "gdx-controllers#gdx-controllers-core/2.2.4-20231021.200112-6/SNAPSHOT": {

     
···

       
223
       
 
       
  discern where the repo base ends and the group ID begins).

     

       
224
       
 
       


     

       
225
       
 
       
`compress-deps-json.py` converts the JSON from mitm-cache format into

     

       
226
       
+
       
Nixpkgs Gradle lockfile format. `fetch.nix` does the opposite.

     

       
227
       
 
       


     

       
228
       
 
       
## Security Considerations

     

       
229
       
 
       


     
···

       
242
       
 
       
  doesn't match)

     

       
243
       
 
       


     

       
244
       
 
       
Please be mindful of the above when working on Gradle support for

     

       
245
       
+
       
Nixpkgs.

     

···

       
15
       
 
       
There are effectively two steps when updating the standard environment:

     

       
16
       
 
       


     

       
17
       
 
       
1. Update the definition of llvmPackages in `all-packages.nix` for Darwin to match the value of

     

       
18
       
-
       
   llvmPackages.latest in `all-packages.nix`. Timing-wise, this done currently using the spring

     

       
19
       
 
       
   release of LLVM and once llvmPackages.latest has been updated to match. If the LLVM project

     

       
20
       
 
       
   has announced a release schedule of patch updates, wait until those are in nixpkgs. Otherwise,

     

       
21
       
 
       
   the LLVM updates will have to go through staging instead of being merged into master; and

     

···

       
15
       
 
       
There are effectively two steps when updating the standard environment:

     

       
16
       
 
       


     

       
17
       
 
       
1. Update the definition of llvmPackages in `all-packages.nix` for Darwin to match the value of

     

       
18
       
+
       
   llvmPackages.latest in `all-packages.nix`. Timing-wise, this is done currently using the spring

     

       
19
       
 
       
   release of LLVM and once llvmPackages.latest has been updated to match. If the LLVM project

     

       
20
       
 
       
   has announced a release schedule of patch updates, wait until those are in nixpkgs. Otherwise,

     

       
21
       
 
       
   the LLVM updates will have to go through staging instead of being merged into master; and

     

···

       
32
       
 
       


     

       
33
       
 
       
If you're updating `nixVersions.stable`, follow all the steps mentioned above, but use the **staging** branch for your pull request (or **staging-next** after coordinating with the people in matrix `#staging:nixos.org`)

     

       
34
       
 
       
This is necessary because, at the end of the staging-next cycle, the NixOS tests are built through the [staging-next-small](https://hydra.nixos.org/jobset/nixos/staging-next-small) jobset.

     

       
35
       
-
       
Especially nixos installer test are important to look at here.

     

       
36
       
 
       


     

       
37
       
 
       
There is a script to update minor versions:

     

       
38
       
 
       


     

···

       
32
       
 
       


     

       
33
       
 
       
If you're updating `nixVersions.stable`, follow all the steps mentioned above, but use the **staging** branch for your pull request (or **staging-next** after coordinating with the people in matrix `#staging:nixos.org`)

     

       
34
       
 
       
This is necessary because, at the end of the staging-next cycle, the NixOS tests are built through the [staging-next-small](https://hydra.nixos.org/jobset/nixos/staging-next-small) jobset.

     

       
35
       
+
       
Especially NixOS installer tests are important to look at here.

     

       
36
       
 
       


     

       
37
       
 
       
There is a script to update minor versions:

     

       
38
       
 
       


     

···

       
13
       
 
       


     

       
14
       
 
       
### `workDir` attribute

     

       
15
       
 
       


     

       
16
       
-
       
The Nixpkgs for Nix does inherit the `workDir` attribute that determines the location of the subproject to build.

     

       
17
       
 
       
It is compared to this directory to produce the correct relative path, similar to upstream.

     

···

       
13
       
 
       


     

       
14
       
 
       
### `workDir` attribute

     

       
15
       
 
       


     

       
16
       
+
       
The Nixpkgs for Nix inherits the `workDir` attribute that determines the location of the subproject to build.

     

       
17
       
 
       
It is compared to this directory to produce the correct relative path, similar to upstream.