Merge pull request #123111 from hercules-ci/writeDirectReferencesToFile

writeDirectReferencesToFile and maintenance

Changed files
+155
doc
pkgs
build-support
test
+47
doc/builders/trivial-builders.chapter.md
···
## `symlinkJoin` {#trivial-builder-symlinkJoin}
This can be used to put many derivations into the same directory structure. It works by creating a new derivation and adding symlinks to each of the paths listed. It expects two arguments, `name`, and `paths`. `name` is the name used in the Nix store path for the created derivation. `paths` is a list of paths that will be symlinked. These paths can be to Nix store derivations or any other subdirectory contained within.
+
+
## `writeReferencesToFile` {#trivial-builder-writeReferencesToFile}
+
+
Writes the closure of transitive dependencies to a file.
+
+
This produces the equivalent of `nix-store -q --requisites`.
+
+
For example,
+
+
```nix
+
writeReferencesToFile (writeScriptBin "hi" ''${hello}/bin/hello'')
+
```
+
+
produces an output path `/nix/store/<hash>-runtime-deps` containing
+
+
```nix
+
/nix/store/<hash>-hello-2.10
+
/nix/store/<hash>-hi
+
/nix/store/<hash>-libidn2-2.3.0
+
/nix/store/<hash>-libunistring-0.9.10
+
/nix/store/<hash>-glibc-2.32-40
+
```
+
+
You can see that this includes `hi`, the original input path,
+
`hello`, which is a direct reference, but also
+
the other paths that are indirectly required to run `hello`.
+
+
## `writeDirectReferencesToFile` {#trivial-builder-writeDirectReferencesToFile}
+
+
Writes the set of references to the output file, that is, their immediate dependencies.
+
+
This produces the equivalent of `nix-store -q --references`.
+
+
For example,
+
+
```nix
+
writeDirectReferencesToFile (writeScriptBin "hi" ''${hello}/bin/hello'')
+
```
+
+
produces an output path `/nix/store/<hash>-runtime-references` containing
+
+
```nix
+
/nix/store/<hash>-hello-2.10
+
```
+
+
but none of `hello`'s dependencies, because those are not referenced directly
+
by `hi`'s output.
+29
pkgs/build-support/trivial-builders.nix
···
done < graph
'';
+
/*
+
Write the set of references to a file, that is, their immediate dependencies.
+
+
This produces the equivalent of `nix-store -q --references`.
+
*/
+
writeDirectReferencesToFile = path: runCommand "runtime-references"
+
{
+
exportReferencesGraph = ["graph" path];
+
inherit path;
+
}
+
''
+
touch ./references
+
while read p; do
+
read dummy
+
read nrRefs
+
if [[ $p == $path ]]; then
+
for ((i = 0; i < nrRefs; i++)); do
+
read ref;
+
echo $ref >>./references
+
done
+
else
+
for ((i = 0; i < nrRefs; i++)); do
+
read ref;
+
done
+
fi
+
done < graph
+
sort ./references >$out
+
'';
+
/* Print an error message if the file with the specified name and
* hash doesn't exist in the Nix store. This function should only
+20
pkgs/build-support/trivial-builders/test.nix
···
+
{ lib, nixosTest, path, writeText, hello, figlet, stdenvNoCC }:
+
+
nixosTest {
+
name = "nixpkgs-trivial-builders";
+
nodes.machine = { ... }: {
+
virtualisation.writableStore = true;
+
+
# Test runs without network, so we don't substitute and prepare our deps
+
nix.binaryCaches = lib.mkForce [];
+
environment.etc."pre-built-paths".source = writeText "pre-built-paths" (
+
builtins.toJSON [hello figlet stdenvNoCC]
+
);
+
};
+
testScript = ''
+
machine.succeed("""
+
cd ${lib.cleanSource path}
+
./pkgs/build-support/trivial-builders/test.sh 2>/dev/console
+
""")
+
'';
+
}
+57
pkgs/build-support/trivial-builders/test.sh
···
+
#!/usr/bin/env bash
+
+
# -------------------------------------------------------------------------- #
+
#
+
# trivial-builders test
+
#
+
# -------------------------------------------------------------------------- #
+
#
+
# This file can be run independently (quick):
+
#
+
# $ pkgs/build-support/trivial-builders/test.sh
+
#
+
# or in the build sandbox with a ~20s VM overhead
+
#
+
# $ nix-build -A tests.trivial-builders
+
#
+
# -------------------------------------------------------------------------- #
+
+
# strict bash
+
set -euo pipefail
+
+
# debug
+
# set -x
+
# PS4='+(${BASH_SOURCE}:${LINENO}): ${FUNCNAME[0]:+${FUNCNAME[0]}(): }'
+
+
cd "$(dirname ${BASH_SOURCE[0]})" # nixpkgs root
+
+
testDirectReferences() {
+
expr="$1"
+
diff -U3 \
+
<(sort <$(nix-build --no-out-link --expr "with import ../../.. {}; writeDirectReferencesToFile ($expr)")) \
+
<(nix-store -q --references $(nix-build --no-out-link --expr "with import ../../.. {}; ($expr)") | sort)
+
}
+
+
testDirectReferences 'hello'
+
testDirectReferences 'figlet'
+
testDirectReferences 'writeText "hi" "hello"'
+
testDirectReferences 'writeText "hi" "hello ${hello}"'
+
testDirectReferences 'writeText "hi" "hello ${hello} ${figlet}"'
+
+
+
+
testClosure() {
+
expr="$1"
+
diff -U3 \
+
<(sort <$(nix-build --no-out-link --expr "with import ../../.. {}; writeReferencesToFile ($expr)")) \
+
<(nix-store -q --requisites $(nix-build --no-out-link --expr "with import ../../.. {}; ($expr)") | sort)
+
}
+
+
testClosure 'hello'
+
testClosure 'figlet'
+
testClosure 'writeText "hi" "hello"'
+
testClosure 'writeText "hi" "hello ${hello}"'
+
testClosure 'writeText "hi" "hello ${hello} ${figlet}"'
+
+
+
echo 'OK!'
+2
pkgs/test/default.nix
···
cuda = callPackage ./cuda { };
+
trivial = callPackage ../build-support/trivial-builders/test.nix {};
+
writers = callPackage ../build-support/writers/test.nix {};
}