pyrox.dev
1# darwin.linux-builder {#sec-darwin-builder}
2
3:::{.warning}
4By default, `darwin.linux-builder` uses a publicly-known private SSH **host key** (this is different from the SSH key used by the user that connects to the builder).
5
6Given the intended use case for it (a Linux builder that runs **on the same machine**), this shouldn't be an issue.
7However, if you plan to deviate from this use case in any way (e.g. by exposing this builder to remote machines), you should understand the security implications of doing so and take any appropriate measures.
8:::
9
10`darwin.linux-builder` provides a way to bootstrap a Linux remote builder on a macOS machine.
11
12This requires macOS version 12.4 or later.
13
14The remote builder runs on host port 31022 by default.
15You can change it by overriding `virtualisation.darwin-builder.hostPort`.
16See the [example](#sec-darwin-builder-example-flake).
17
18You will also need to be a trusted user for your Nix installation. In other
19words, your `/etc/nix/nix.conf` should have something like:
20
21```
22extra-trusted-users = <your username goes here>
23```
24
25To launch the remote builder, run the following flake:
26
27```ShellSession
28$ nix run nixpkgs#darwin.linux-builder
29```
30
31That will prompt you to enter your `sudo` password:
32
33```
34+ sudo --reset-timestamp /nix/store/…-install-credentials.sh ./keys
35Password:
36```
37
38… so that it can install a private key used to `ssh` into the build server.
39After that the script will launch the virtual machine and automatically log you
40in as the `builder` user:
41
42```
43<<< Welcome to NixOS 22.11.20220901.1bd8d11 (aarch64) - ttyAMA0 >>>
44
45Run 'nixos-help' for the NixOS manual.
46
47nixos login: builder (automatic login)
48
49
50[builder@nixos:~]$
51```
52
53> Note: When you need to stop the VM, run `shutdown now` as the `builder` user.
54
55To delegate builds to the remote builder, add the following options to your
56`nix.conf` file:
57
58```
59# - Replace ${ARCH} with either aarch64 or x86_64 to match your host machine
60# - Replace ${MAX_JOBS} with the maximum number of builds (pick 4 if you're not sure)
61builders = ssh-ng://builder@linux-builder ${ARCH}-linux /etc/nix/builder_ed25519 ${MAX_JOBS} - - - c3NoLWVkMjU1MTkgQUFBQUMzTnphQzFsWkRJMU5URTVBQUFBSUpCV2N4Yi9CbGFxdDFhdU90RStGOFFVV3JVb3RpQzVxQkorVXVFV2RWQ2Igcm9vdEBuaXhvcwo=
62
63# Not strictly necessary, but this will reduce your disk utilization
64builders-use-substitutes = true
65```
66
67To allow Nix to connect to a remote builder not running on port 22, you will also need to create a new file at `/etc/ssh/ssh_config.d/100-linux-builder.conf`:
68
69```
70Host linux-builder
71 Hostname localhost
72 HostKeyAlias linux-builder
73 Port 31022
74```
75
76… and then restart your Nix daemon to apply the change:
77
78```ShellSession
79$ sudo launchctl kickstart -k system/org.nixos.nix-daemon
80```
81
82## Example flake usage {#sec-darwin-builder-example-flake}
83
84```nix
85{
86 inputs = {
87 nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-22.11-darwin";
88 darwin.url = "github:lnl7/nix-darwin/master";
89 darwin.inputs.nixpkgs.follows = "nixpkgs";
90 };
91
92 outputs = { self, darwin, nixpkgs, ... }@inputs:
93 let
94
95 inherit (darwin.lib) darwinSystem;
96 system = "aarch64-darwin";
97 pkgs = nixpkgs.legacyPackages."${system}";
98 linuxSystem = builtins.replaceStrings [ "darwin" ] [ "linux" ] system;
99
100 darwin-builder = nixpkgs.lib.nixosSystem {
101 system = linuxSystem;
102 modules = [
103 "${nixpkgs}/nixos/modules/profiles/macos-builder.nix"
104 { virtualisation = {
105 host.pkgs = pkgs;
106 darwin-builder.workingDirectory = "/var/lib/darwin-builder";
107 darwin-builder.hostPort = 22;
108 };
109 }
110 ];
111 };
112 in {
113
114 darwinConfigurations = {
115 machine1 = darwinSystem {
116 inherit system;
117 modules = [
118 {
119 nix.distributedBuilds = true;
120 nix.buildMachines = [{
121 hostName = "localhost";
122 sshUser = "builder";
123 sshKey = "/etc/nix/builder_ed25519";
124 system = linuxSystem;
125 maxJobs = 4;
126 supportedFeatures = [ "kvm" "benchmark" "big-parallel" ];
127 }];
128
129 launchd.daemons.darwin-builder = {
130 command = "${darwin-builder.config.system.build.macos-builder-installer}/bin/create-builder";
131 serviceConfig = {
132 KeepAlive = true;
133 RunAtLoad = true;
134 StandardOutPath = "/var/log/darwin-builder.log";
135 StandardErrorPath = "/var/log/darwin-builder.log";
136 };
137 };
138 }
139 ];
140 };
141 };
142
143 };
144}
145```
146
147## Reconfiguring the remote builder {#sec-darwin-builder-reconfiguring}
148
149Initially you should not change the remote builder configuration else you will not be
150able to use the binary cache. However, after you have the remote builder running locally
151you may use it to build a modified remote builder with additional storage or memory.
152
153To do this, you just need to set the `virtualisation.darwin-builder.*` parameters as
154in the example below and rebuild.
155
156```nix
157 {
158 darwin-builder = nixpkgs.lib.nixosSystem {
159 system = linuxSystem;
160 modules = [
161 "${nixpkgs}/nixos/modules/profiles/macos-builder.nix"
162 {
163 virtualisation.host.pkgs = pkgs;
164 virtualisation.darwin-builder.diskSize = 5120;
165 virtualisation.darwin-builder.memorySize = 1024;
166 virtualisation.darwin-builder.hostPort = 33022;
167 virtualisation.darwin-builder.workingDirectory = "/var/lib/darwin-builder";
168 }
169 ];
170 };
171 }
172```
173
174You may make any other changes to your VM in this attribute set. For example,
175you could enable Docker or X11 forwarding to your Darwin host.
176
177## Troubleshooting the generated configuration {#sec-darwin-builder-troubleshoot}
178
179The `linux-builder` package exposes the attributes `nixosConfig` and `nixosOptions` that allow you to inspect the generated NixOS configuration in the `nix repl`. For example:
180
181```
182$ nix repl --file ~/src/nixpkgs --argstr system aarch64-darwin
183
184nix-repl> darwin.linux-builder.nixosConfig.nix.package
185«derivation /nix/store/...-nix-2.17.0.drv»
186
187nix-repl> :p darwin.linux-builder.nixosOptions.virtualisation.memorySize.definitionsWithLocations
188[ { file = "/home/user/src/nixpkgs/nixos/modules/profiles/macos-builder.nix"; value = 3072; } ]
189
190```