pyrox.dev
1# darwin.linux-builder {#sec-darwin-builder}
2
3:::{.warning}
4By default, `darwin.linux-builder` uses a publicly-known private SSH **host key** (this is different from the SSH key used by the user that connects to the builder).
5
6Given the intended use case for it (a Linux builder that runs **on the same machine**), this shouldn't be an issue.
7However, if you plan to deviate from this use case in any way (e.g. by exposing this builder to remote machines), you should understand the security implications of doing so and take any appropriate measures.
8:::
9
10`darwin.linux-builder` provides a way to bootstrap a Linux remote builder on a macOS machine.
11
12This requires macOS version 12.4 or later.
13
14The remote builder runs on host port 31022 by default.
15You can change it by overriding `virtualisation.darwin-builder.hostPort`.
16See the [example](#sec-darwin-builder-example-flake).
17
18You will also need to be a trusted user for your Nix installation. In other
19words, your `/etc/nix/nix.conf` should have something like:
20
21```
22extra-trusted-users = <your username goes here>
23```
24
25To launch the remote builder, run the following flake:
26
27```ShellSession
28$ nix run nixpkgs#darwin.linux-builder
29```
30
31That will prompt you to enter your `sudo` password:
32
33```
34+ sudo --reset-timestamp /nix/store/…-install-credentials.sh ./keys
35Password:
36```
37
38… so that it can install a private key used to `ssh` into the build server.
39After that the script will launch the virtual machine and automatically log you
40in as the `builder` user:
41
42```
43<<< Welcome to NixOS 22.11.20220901.1bd8d11 (aarch64) - ttyAMA0 >>>
44
45Run 'nixos-help' for the NixOS manual.
46
47nixos login: builder (automatic login)
48
49
50[builder@nixos:~]$
51```
52
53> Note: When you need to stop the VM, run `shutdown now` as the `builder` user.
54
55To delegate builds to the remote builder, add the following options to your
56`nix.conf` file:
57
58```
59# - Replace ${ARCH} with either aarch64 or x86_64 to match your host machine
60# - Replace ${MAX_JOBS} with the maximum number of builds (pick 4 if you're not sure)
61builders = ssh-ng://builder@linux-builder ${ARCH}-linux /etc/nix/builder_ed25519 ${MAX_JOBS} - - - c3NoLWVkMjU1MTkgQUFBQUMzTnphQzFsWkRJMU5URTVBQUFBSUpCV2N4Yi9CbGFxdDFhdU90RStGOFFVV3JVb3RpQzVxQkorVXVFV2RWQ2Igcm9vdEBuaXhvcwo=
62
63# Not strictly necessary, but this will reduce your disk utilization
64builders-use-substitutes = true
65```
66
67To allow Nix to connect to a remote builder not running on port 22, you will also need to create a new file at `/etc/ssh/ssh_config.d/100-linux-builder.conf`:
68
69```
70Host linux-builder
71 Hostname localhost
72 HostKeyAlias linux-builder
73 Port 31022
74```
75
76… and then restart your Nix daemon to apply the change:
77
78```ShellSession
79$ sudo launchctl kickstart -k system/org.nixos.nix-daemon
80```
81
82## Example flake usage {#sec-darwin-builder-example-flake}
83
84```nix
85{
86 inputs = {
87 nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-22.11-darwin";
88 darwin.url = "github:lnl7/nix-darwin/master";
89 darwin.inputs.nixpkgs.follows = "nixpkgs";
90 };
91
92 outputs =
93 {
94 self,
95 darwin,
96 nixpkgs,
97 ...
98 }@inputs:
99 let
100
101 inherit (darwin.lib) darwinSystem;
102 system = "aarch64-darwin";
103 pkgs = nixpkgs.legacyPackages."${system}";
104 linuxSystem = builtins.replaceStrings [ "darwin" ] [ "linux" ] system;
105
106 darwin-builder = nixpkgs.lib.nixosSystem {
107 system = linuxSystem;
108 modules = [
109 "${nixpkgs}/nixos/modules/profiles/nix-builder-vm.nix"
110 {
111 virtualisation = {
112 host.pkgs = pkgs;
113 darwin-builder.workingDirectory = "/var/lib/darwin-builder";
114 darwin-builder.hostPort = 22;
115 };
116 }
117 ];
118 };
119 in
120 {
121
122 darwinConfigurations = {
123 machine1 = darwinSystem {
124 inherit system;
125 modules = [
126 {
127 nix.distributedBuilds = true;
128 nix.buildMachines = [
129 {
130 hostName = "localhost";
131 sshUser = "builder";
132 sshKey = "/etc/nix/builder_ed25519";
133 system = linuxSystem;
134 maxJobs = 4;
135 supportedFeatures = [
136 "kvm"
137 "benchmark"
138 "big-parallel"
139 ];
140 }
141 ];
142
143 launchd.daemons.darwin-builder = {
144 command = "${darwin-builder.config.system.build.macos-builder-installer}/bin/create-builder";
145 serviceConfig = {
146 KeepAlive = true;
147 RunAtLoad = true;
148 StandardOutPath = "/var/log/darwin-builder.log";
149 StandardErrorPath = "/var/log/darwin-builder.log";
150 };
151 };
152 }
153 ];
154 };
155 };
156
157 };
158}
159```
160
161## Reconfiguring the remote builder {#sec-darwin-builder-reconfiguring}
162
163Initially you should not change the remote builder configuration else you will not be
164able to use the binary cache. However, after you have the remote builder running locally
165you may use it to build a modified remote builder with additional storage or memory.
166
167To do this, you just need to set the `virtualisation.darwin-builder.*` parameters as
168in the example below and rebuild.
169
170```nix
171{
172 darwin-builder = nixpkgs.lib.nixosSystem {
173 system = linuxSystem;
174 modules = [
175 "${nixpkgs}/nixos/modules/profiles/nix-builder-vm.nix"
176 {
177 virtualisation.host.pkgs = pkgs;
178 virtualisation.darwin-builder.diskSize = 5120;
179 virtualisation.darwin-builder.memorySize = 1024;
180 virtualisation.darwin-builder.hostPort = 33022;
181 virtualisation.darwin-builder.workingDirectory = "/var/lib/darwin-builder";
182 }
183 ];
184 };
185}
186```
187
188You may make any other changes to your VM in this attribute set. For example,
189you could enable Docker or X11 forwarding to your Darwin host.
190
191## Troubleshooting the generated configuration {#sec-darwin-builder-troubleshoot}
192
193The `linux-builder` package exposes the attributes `nixosConfig` and `nixosOptions` that allow you to inspect the generated NixOS configuration in the `nix repl`. For example:
194
195```
196$ nix repl --file ~/src/nixpkgs --argstr system aarch64-darwin
197
198nix-repl> darwin.linux-builder.nixosConfig.nix.package
199«derivation /nix/store/...-nix-2.17.0.drv»
200
201nix-repl> :p darwin.linux-builder.nixosOptions.virtualisation.memorySize.definitionsWithLocations
202[ { file = "/home/user/src/nixpkgs/nixos/modules/profiles/nix-builder-vm.nix"; value = 3072; } ]
203
204```