pyrox.dev
1# darwin.linux-builder {#sec-darwin-builder}
2
3`darwin.linux-builder` provides a way to bootstrap a Linux remote builder on a macOS machine.
4
5This requires macOS version 12.4 or later.
6
7The remote builder runs on host port 31022 by default.
8You can change it by overriding `virtualisation.darwin-builder.hostPort`.
9See the [example](#sec-darwin-builder-example-flake).
10
11You will also need to be a trusted user for your Nix installation. In other
12words, your `/etc/nix/nix.conf` should have something like:
13
14```
15extra-trusted-users = <your username goes here>
16```
17
18To launch the remote builder, run the following flake:
19
20```ShellSession
21$ nix run nixpkgs#darwin.linux-builder
22```
23
24That will prompt you to enter your `sudo` password:
25
26```
27+ sudo --reset-timestamp /nix/store/…-install-credentials.sh ./keys
28Password:
29```
30
31… so that it can install a private key used to `ssh` into the build server.
32After that the script will launch the virtual machine and automatically log you
33in as the `builder` user:
34
35```
36<<< Welcome to NixOS 22.11.20220901.1bd8d11 (aarch64) - ttyAMA0 >>>
37
38Run 'nixos-help' for the NixOS manual.
39
40nixos login: builder (automatic login)
41
42
43[builder@nixos:~]$
44```
45
46> Note: When you need to stop the VM, run `shutdown now` as the `builder` user.
47
48To delegate builds to the remote builder, add the following options to your
49`nix.conf` file:
50
51```
52# - Replace ${ARCH} with either aarch64 or x86_64 to match your host machine
53# - Replace ${MAX_JOBS} with the maximum number of builds (pick 4 if you're not sure)
54builders = ssh-ng://builder@linux-builder ${ARCH}-linux /etc/nix/builder_ed25519 ${MAX_JOBS} - - - c3NoLWVkMjU1MTkgQUFBQUMzTnphQzFsWkRJMU5URTVBQUFBSUpCV2N4Yi9CbGFxdDFhdU90RStGOFFVV3JVb3RpQzVxQkorVXVFV2RWQ2Igcm9vdEBuaXhvcwo=
55
56# Not strictly necessary, but this will reduce your disk utilization
57builders-use-substitutes = true
58```
59
60To allow Nix to connect to a remote builder not running on port 22, you will also need to create a new file at `/etc/ssh/ssh_config.d/100-linux-builder.conf`:
61
62```
63Host linux-builder
64 Hostname localhost
65 HostKeyAlias linux-builder
66 Port 31022
67```
68
69… and then restart your Nix daemon to apply the change:
70
71```ShellSession
72$ sudo launchctl kickstart -k system/org.nixos.nix-daemon
73```
74
75## Example flake usage {#sec-darwin-builder-example-flake}
76
77```
78{
79 inputs = {
80 nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-22.11-darwin";
81 darwin.url = "github:lnl7/nix-darwin/master";
82 darwin.inputs.nixpkgs.follows = "nixpkgs";
83 };
84
85 outputs = { self, darwin, nixpkgs, ... }@inputs:
86 let
87
88 inherit (darwin.lib) darwinSystem;
89 system = "aarch64-darwin";
90 pkgs = nixpkgs.legacyPackages."${system}";
91 linuxSystem = builtins.replaceStrings [ "darwin" ] [ "linux" ] system;
92
93 darwin-builder = nixpkgs.lib.nixosSystem {
94 system = linuxSystem;
95 modules = [
96 "${nixpkgs}/nixos/modules/profiles/macos-builder.nix"
97 { virtualisation.host.pkgs = pkgs; }
98 ];
99 };
100 in {
101
102 darwinConfigurations = {
103 machine1 = darwinSystem {
104 inherit system;
105 modules = [
106 {
107 nix.distributedBuilds = true;
108 nix.buildMachines = [{
109 hostName = "ssh://builder@localhost";
110 system = linuxSystem;
111 maxJobs = 4;
112 supportedFeatures = [ "kvm" "benchmark" "big-parallel" ];
113 }];
114
115 launchd.daemons.darwin-builder = {
116 command = "${darwin-builder.config.system.build.macos-builder-installer}/bin/create-builder";
117 serviceConfig = {
118 KeepAlive = true;
119 RunAtLoad = true;
120 StandardOutPath = "/var/log/darwin-builder.log";
121 StandardErrorPath = "/var/log/darwin-builder.log";
122 };
123 };
124 }
125 ];
126 };
127 };
128
129 };
130}
131```
132
133## Reconfiguring the remote builder {#sec-darwin-builder-reconfiguring}
134
135Initially you should not change the remote builder configuration else you will not be
136able to use the binary cache. However, after you have the remote builder running locally
137you may use it to build a modified remote builder with additional storage or memory.
138
139To do this, you just need to set the `virtualisation.darwin-builder.*` parameters as
140in the example below and rebuild.
141
142```
143 darwin-builder = nixpkgs.lib.nixosSystem {
144 system = linuxSystem;
145 modules = [
146 "${nixpkgs}/nixos/modules/profiles/macos-builder.nix"
147 {
148 virtualisation.host.pkgs = pkgs;
149 virtualisation.darwin-builder.diskSize = 5120;
150 virtualisation.darwin-builder.memorySize = 1024;
151 virtualisation.darwin-builder.hostPort = 33022;
152 virtualisation.darwin-builder.workingDirectory = "/var/lib/darwin-builder";
153 }
154 ];
155```
156
157You may make any other changes to your VM in this attribute set. For example,
158you could enable Docker or X11 forwarding to your Darwin host.
159
160## Troubleshooting the generated configuration {#sec-darwin-builder-troubleshoot}
161
162The `linux-builder` package exposes the attributes `nixosConfig` and `nixosOptions` that allow you to inspect the generated NixOS configuration in the `nix repl`. For example:
163
164```
165$ nix repl --file ~/src/nixpkgs --argstr system aarch64-darwin
166
167nix-repl> darwin.linux-builder.nixosConfig.nix.package
168«derivation /nix/store/...-nix-2.17.0.drv»
169
170nix-repl> :p darwin.linux-builder.nixosOptions.virtualisation.memorySize.definitionsWithLocations
171[ { file = "/home/user/src/nixpkgs/nixos/modules/profiles/macos-builder.nix"; value = 3072; } ]
172
173```