pyrox.dev
1# BEAM Languages (Erlang, Elixir & LFE) {#sec-beam}
2
3## Introduction {#beam-introduction}
4
5In this document and related Nix expressions, we use the term, _BEAM_, to describe the environment. BEAM is the name of the Erlang Virtual Machine and, as far as we're concerned, from a packaging perspective, all languages that run on the BEAM are interchangeable. That which varies, like the build system, is transparent to users of any given BEAM package, so we make no distinction.
6
7## Structure {#beam-structure}
8
9All BEAM-related expressions are available via the top-level `beam` attribute, which includes:
10
11- `interpreters`: a set of compilers running on the BEAM, including multiple Erlang/OTP versions (`beam.interpreters.erlangR19`, etc), Elixir (`beam.interpreters.elixir`) and LFE (Lisp Flavoured Erlang) (`beam.interpreters.lfe`).
12
13- `packages`: a set of package builders (Mix and rebar3), each compiled with a specific Erlang/OTP version, e.g. `beam.packages.erlangR19`.
14
15The default Erlang compiler, defined by `beam.interpreters.erlang`, is aliased as `erlang`. The default BEAM package set is defined by `beam.packages.erlang` and aliased at the top level as `beamPackages`.
16
17To create a package builder built with a custom Erlang version, use the lambda, `beam.packagesWith`, which accepts an Erlang/OTP derivation and produces a package builder similar to `beam.packages.erlang`.
18
19Many Erlang/OTP distributions available in `beam.interpreters` have versions with ODBC and/or Java enabled or without wx (no observer support). For example, there's `beam.interpreters.erlangR22_odbc_javac`, which corresponds to `beam.interpreters.erlangR22` and `beam.interpreters.erlangR22_nox`, which corresponds to `beam.interpreters.erlangR22`.
20
21## Build Tools {#build-tools}
22
23### Rebar3 {#build-tools-rebar3}
24
25We provide a version of Rebar3, under `rebar3`. We also provide a helper to fetch Rebar3 dependencies from a lockfile under `fetchRebar3Deps`.
26
27We also provide a version on Rebar3 with plugins included, under `rebar3WithPlugins`. This package is a function which takes two arguments: `plugins`, a list of nix derivations to include as plugins (loaded only when specified in `rebar.config`), and `globalPlugins`, which should always be loaded by rebar3. Example: `rebar3WithPlugins { globalPlugins = [beamPackages.pc]; }`.
28
29When adding a new plugin it is important that the `packageName` attribute is the same as the atom used by rebar3 to refer to the plugin.
30
31### Mix & Erlang.mk {#build-tools-other}
32
33Erlang.mk works exactly as expected. There is a bootstrap process that needs to be run, which is supported by the `buildErlangMk` derivation.
34
35For Elixir applications use `mixRelease` to make a release. See examples for more details.
36
37There is also a `buildMix` helper, whose behavior is closer to that of `buildErlangMk` and `buildRebar3`. The primary difference is that mixRelease makes a release, while buildMix only builds the package, making it useful for libraries and other dependencies.
38
39## How to Install BEAM Packages {#how-to-install-beam-packages}
40
41BEAM builders are not registered at the top level, simply because they are not relevant to the vast majority of Nix users. To install any of those builders into your profile, refer to them by their attribute path `beamPackages.rebar3`:
42
43```ShellSession
44$ nix-env -f "<nixpkgs>" -iA beamPackages.rebar3
45```
46
47## Packaging BEAM Applications {#packaging-beam-applications}
48
49### Erlang Applications {#packaging-erlang-applications}
50
51#### Rebar3 Packages {#rebar3-packages}
52
53The Nix function, `buildRebar3`, defined in `beam.packages.erlang.buildRebar3` and aliased at the top level, can be used to build a derivation that understands how to build a Rebar3 project.
54
55If a package needs to compile native code via Rebar3's port compilation mechanism, add `compilePort = true;` to the derivation.
56
57#### Erlang.mk Packages {#erlang-mk-packages}
58
59Erlang.mk functions similarly to Rebar3, except we use `buildErlangMk` instead of `buildRebar3`.
60
61#### Mix Packages {#mix-packages}
62
63`mixRelease` is used to make a release in the mix sense. Dependencies will need to be fetched with `fetchMixDeps` and passed to it.
64
65#### mixRelease - Elixir Phoenix example
66
67Here is how your `default.nix` file would look.
68
69```nix
70with import <nixpkgs> { };
71
72let
73 packages = beam.packagesWith beam.interpreters.erlang;
74 src = builtins.fetchgit {
75 url = "ssh://git@github.com/your_id/your_repo";
76 rev = "replace_with_your_commit";
77 };
78
79 pname = "your_project";
80 version = "0.0.1";
81 mixEnv = "prod";
82
83 mixDeps = packages.fetchMixDeps {
84 pname = "mix-deps-${pname}";
85 inherit src mixEnv version;
86 # nix will complain and tell you the right value to replace this with
87 sha256 = lib.fakeSha256;
88 # if you have build time environment variables add them here
89 MY_ENV_VAR="my_value";
90 };
91
92 nodeDependencies = (pkgs.callPackage ./assets/default.nix { }).shell.nodeDependencies;
93
94 frontEndFiles = stdenvNoCC.mkDerivation {
95 pname = "frontend-${pname}";
96
97 nativeBuildInputs = [ nodejs ];
98
99 inherit version src;
100
101 buildPhase = ''
102 cp -r ./assets $TEMPDIR
103
104 mkdir -p $TEMPDIR/assets/node_modules/.cache
105 cp -r ${nodeDependencies}/lib/node_modules $TEMPDIR/assets
106 export PATH="${nodeDependencies}/bin:$PATH"
107
108 cd $TEMPDIR/assets
109 webpack --config ./webpack.config.js
110 cd ..
111 '';
112
113 installPhase = ''
114 cp -r ./priv/static $out/
115 '';
116
117 outputHashAlgo = "sha256";
118 outputHashMode = "recursive";
119 # nix will complain and tell you the right value to replace this with
120 outputHash = lib.fakeSha256;
121
122 impureEnvVars = lib.fetchers.proxyImpureEnvVars;
123 };
124
125
126in packages.mixRelease {
127 inherit src pname version mixEnv mixDeps;
128 # if you have build time environment variables add them here
129 MY_ENV_VAR="my_value";
130 preInstall = ''
131 mkdir -p ./priv/static
132 cp -r ${frontEndFiles} ./priv/static
133 '';
134}
135```
136
137Setup will require the following steps:
138
139- Move your secrets to runtime environment variables. For more information refer to the [runtime.exs docs](https://hexdocs.pm/mix/Mix.Tasks.Release.html#module-runtime-configuration). On a fresh Phoenix build that would mean that both `DATABASE_URL` and `SECRET_KEY` need to be moved to `runtime.exs`.
140- `cd assets` and `nix-shell -p node2nix --run node2nix --development` will generate a Nix expression containing your frontend dependencies
141- commit and push those changes
142- you can now `nix-build .`
143- To run the release, set the `RELEASE_TMP` environment variable to a directory that your program has write access to. It will be used to store the BEAM settings.
144
145#### Example of creating a service for an Elixir - Phoenix project
146
147In order to create a service with your release, you could add a `service.nix`
148in your project with the following
149
150```nix
151{config, pkgs, lib, ...}:
152
153let
154 release = pkgs.callPackage ./default.nix;
155 release_name = "app";
156 working_directory = "/home/app";
157in
158{
159 systemd.services.${release_name} = {
160 wantedBy = [ "multi-user.target" ];
161 after = [ "network.target" "postgresql.service" ];
162 requires = [ "network-online.target" "postgresql.service" ];
163 description = "my app";
164 environment = {
165 # RELEASE_TMP is used to write the state of the
166 # VM configuration when the system is running
167 # it needs to be a writable directory
168 RELEASE_TMP = working_directory;
169 # can be generated in an elixir console with
170 # Base.encode32(:crypto.strong_rand_bytes(32))
171 RELEASE_COOKIE = "my_cookie";
172 MY_VAR = "my_var";
173 };
174 serviceConfig = {
175 Type = "exec";
176 DynamicUser = true;
177 WorkingDirectory = working_directory;
178 # Implied by DynamicUser, but just to emphasize due to RELEASE_TMP
179 PrivateTmp = true;
180 ExecStart = ''
181 ${release}/bin/${release_name} start
182 '';
183 ExecStop = ''
184 ${release}/bin/${release_name} stop
185 '';
186 ExecReload = ''
187 ${release}/bin/${release_name} restart
188 '';
189 Restart = "on-failure";
190 RestartSec = 5;
191 StartLimitBurst = 3;
192 StartLimitInterval = 10;
193 };
194 # disksup requires bash
195 path = [ pkgs.bash ];
196 };
197
198 environment.systemPackages = [ release ];
199}
200```
201
202## How to Develop {#how-to-develop}
203
204### Creating a Shell {#creating-a-shell}
205
206Usually, we need to create a `shell.nix` file and do our development inside of the environment specified therein. Just install your version of Erlang and any other interpreters, and then use your normal build tools. As an example with Elixir:
207
208```nix
209{ pkgs ? import "<nixpkgs"> {} }:
210
211with pkgs;
212
213let
214
215 elixir = beam.packages.erlangR22.elixir_1_9;
216
217in
218mkShell {
219 buildInputs = [ elixir ];
220
221 ERL_INCLUDE_PATH="${erlang}/lib/erlang/usr/include";
222}
223```
224
225#### Elixir - Phoenix project
226
227Here is an example `shell.nix`.
228
229```nix
230with import <nixpkgs> { };
231
232let
233 # define packages to install
234 basePackages = [
235 git
236 # replace with beam.packages.erlang.elixir_1_11 if you need
237 beam.packages.erlang.elixir
238 nodejs-15_x
239 postgresql_13
240 # only used for frontend dependencies
241 # you are free to use yarn2nix as well
242 nodePackages.node2nix
243 # formatting js file
244 nodePackages.prettier
245 ];
246
247 inputs = basePackages ++ lib.optionals stdenv.isLinux [ inotify-tools ]
248 ++ lib.optionals stdenv.isDarwin
249 (with darwin.apple_sdk.frameworks; [ CoreFoundation CoreServices ]);
250
251 # define shell startup command
252 hooks = ''
253 # this allows mix to work on the local directory
254 mkdir -p .nix-mix .nix-hex
255 export MIX_HOME=$PWD/.nix-mix
256 export HEX_HOME=$PWD/.nix-mix
257 export PATH=$MIX_HOME/bin:$HEX_HOME/bin:$PATH
258 # TODO: not sure how to make hex available without installing it afterwards.
259 mix local.hex --if-missing
260 export LANG=en_US.UTF-8
261 export ERL_AFLAGS="-kernel shell_history enabled"
262
263 # postges related
264 # keep all your db data in a folder inside the project
265 export PGDATA="$PWD/db"
266
267 # phoenix related env vars
268 export POOL_SIZE=15
269 export DB_URL="postgresql://postgres:postgres@localhost:5432/db"
270 export PORT=4000
271 export MIX_ENV=dev
272 # add your project env vars here, word readable in the nix store.
273 export ENV_VAR="your_env_var"
274 '';
275
276in mkShell {
277 buildInputs = inputs;
278 shellHook = hooks;
279}
280```
281
282Initializing the project will require the following steps:
283
284- create the db directory `initdb ./db` (inside your mix project folder)
285- create the postgres user `createuser postgres -ds`
286- create the db `createdb db`
287- start the postgres instance `pg_ctl -l "$PGDATA/server.log" start`
288- add the `/db` folder to your `.gitignore`
289- you can start your phoenix server and get a shell with `iex -S mix phx.server`