pyrox.dev
1<section xmlns="http://docbook.org/ns/docbook"
2 xmlns:xlink="http://www.w3.org/1999/xlink"
3 xml:id="sec-beam">
4
5 <title>BEAM Languages (Erlang, Elixir & LFE)</title>
6 <section xml:id="beam-introduction">
7 <title>Introduction</title>
8 <para>
9 In this document and related Nix expressions, we use the term,
10 <emphasis>BEAM</emphasis>, to describe the environment. BEAM is the name
11 of the Erlang Virtual Machine and, as far as we're concerned, from a
12 packaging perspective, all languages that run on the BEAM are
13 interchangeable. That which varies, like the build system, is transparent
14 to users of any given BEAM package, so we make no distinction.
15 </para>
16 </section>
17 <section xml:id="beam-structure">
18 <title>Structure</title>
19 <para>
20 All BEAM-related expressions are available via the top-level
21 <literal>beam</literal> attribute, which includes:
22 </para>
23 <itemizedlist>
24 <listitem>
25 <para>
26 <literal>interpreters</literal>: a set of compilers running on the
27 BEAM, including multiple Erlang/OTP versions
28 (<literal>beam.interpreters.erlangR19</literal>, etc), Elixir
29 (<literal>beam.interpreters.elixir</literal>) and LFE
30 (<literal>beam.interpreters.lfe</literal>).
31 </para>
32 </listitem>
33 <listitem>
34 <para>
35 <literal>packages</literal>: a set of package sets, each compiled with
36 a specific Erlang/OTP version, e.g.
37 <literal>beam.packages.erlangR19</literal>.
38 </para>
39 </listitem>
40 </itemizedlist>
41 <para>
42 The default Erlang compiler, defined by
43 <literal>beam.interpreters.erlang</literal>, is aliased as
44 <literal>erlang</literal>. The default BEAM package set is defined by
45 <literal>beam.packages.erlang</literal> and aliased at the top level as
46 <literal>beamPackages</literal>.
47 </para>
48 <para>
49 To create a package set built with a custom Erlang version, use the
50 lambda, <literal>beam.packagesWith</literal>, which accepts an Erlang/OTP
51 derivation and produces a package set similar to
52 <literal>beam.packages.erlang</literal>.
53 </para>
54 <para>
55 Many Erlang/OTP distributions available in
56 <literal>beam.interpreters</literal> have versions with ODBC and/or Java
57 enabled. For example, there's
58 <literal>beam.interpreters.erlangR19_odbc_javac</literal>, which
59 corresponds to <literal>beam.interpreters.erlangR19</literal>.
60 </para>
61 <para xml:id="erlang-call-package">
62 We also provide the lambda,
63 <literal>beam.packages.erlang.callPackage</literal>, which simplifies
64 writing BEAM package definitions by injecting all packages from
65 <literal>beam.packages.erlang</literal> into the top-level context.
66 </para>
67 </section>
68 <section xml:id="build-tools">
69 <title>Build Tools</title>
70 <section xml:id="build-tools-rebar3">
71 <title>Rebar3</title>
72 <para>
73 By default, Rebar3 wants to manage its own dependencies. This is perfectly
74 acceptable in the normal, non-Nix setup, but in the Nix world, it is not.
75 To rectify this, we provide two versions of Rebar3:
76 <itemizedlist>
77 <listitem>
78 <para>
79 <literal>rebar3</literal>: patched to remove the ability to download
80 anything. When not running it via <literal>nix-shell</literal> or
81 <literal>nix-build</literal>, it's probably not going to work as
82 desired.
83 </para>
84 </listitem>
85 <listitem>
86 <para>
87 <literal>rebar3-open</literal>: the normal, unmodified Rebar3. It
88 should work exactly as would any other version of Rebar3. Any Erlang
89 package should rely on <literal>rebar3</literal> instead. See <xref
90 linkend="rebar3-packages"/>.
91 </para>
92 </listitem>
93 </itemizedlist>
94 </para>
95 </section>
96 <section xml:id="build-tools-other">
97 <title>Mix & Erlang.mk</title>
98 <para>
99 Both Mix and Erlang.mk work exactly as expected. There is a bootstrap
100 process that needs to be run for both, however, which is supported by the
101 <literal>buildMix</literal> and <literal>buildErlangMk</literal>
102 derivations, respectively.
103 </para>
104 </section>
105</section>
106
107<section xml:id="how-to-install-beam-packages">
108 <title>How to Install BEAM Packages</title>
109 <para>
110 BEAM packages are not registered at the top level, simply because they are
111 not relevant to the vast majority of Nix users. They are installable using
112 the <literal>beam.packages.erlang</literal> attribute set (aliased as
113 <literal>beamPackages</literal>), which points to packages built by the
114 default Erlang/OTP version in Nixpkgs, as defined by
115 <literal>beam.interpreters.erlang</literal>.
116
117 To list the available packages in
118 <literal>beamPackages</literal>, use the following command:
119 </para>
120
121 <programlisting>
122$ nix-env -f "<nixpkgs>" -qaP -A beamPackages
123beamPackages.esqlite esqlite-0.2.1
124beamPackages.goldrush goldrush-0.1.7
125beamPackages.ibrowse ibrowse-4.2.2
126beamPackages.jiffy jiffy-0.14.5
127beamPackages.lager lager-3.0.2
128beamPackages.meck meck-0.8.3
129beamPackages.rebar3-pc pc-1.1.0
130 </programlisting>
131 <para>
132 To install any of those packages into your profile, refer to them by their
133 attribute path (first column):
134 </para>
135 <programlisting>
136$ nix-env -f "<nixpkgs>" -iA beamPackages.ibrowse
137 </programlisting>
138 <para>
139 The attribute path of any BEAM package corresponds to the name of that
140 particular package in <link xlink:href="https://hex.pm">Hex</link> or its
141 OTP Application/Release name.
142 </para>
143</section>
144<section xml:id="packaging-beam-applications">
145 <title>Packaging BEAM Applications</title>
146 <section xml:id="packaging-erlang-applications">
147 <title>Erlang Applications</title>
148 <section xml:id="rebar3-packages">
149 <title>Rebar3 Packages</title>
150 <para>
151 The Nix function, <literal>buildRebar3</literal>, defined in
152 <literal>beam.packages.erlang.buildRebar3</literal> and aliased at the
153 top level, can be used to build a derivation that understands how to
154 build a Rebar3 project. For example, we can build <link
155 xlink:href="https://github.com/erlang-nix/hex2nix">hex2nix</link> as
156 follows:
157 </para>
158 <programlisting>
159 { stdenv, fetchFromGitHub, buildRebar3, ibrowse, jsx, erlware_commons }:
160
161 buildRebar3 rec {
162 name = "hex2nix";
163 version = "0.0.1";
164
165 src = fetchFromGitHub {
166 owner = "ericbmerritt";
167 repo = "hex2nix";
168 rev = "${version}";
169 sha256 = "1w7xjidz1l5yjmhlplfx7kphmnpvqm67w99hd2m7kdixwdxq0zqg";
170 };
171
172 beamDeps = [ ibrowse jsx erlware_commons ];
173 }
174 </programlisting>
175 <para>
176 Such derivations are callable with
177 <literal>beam.packages.erlang.callPackage</literal> (see <xref
178 linkend="erlang-call-package"/>). To call this package using the normal
179 <literal>callPackage</literal>, refer to dependency packages via
180 <literal>beamPackages</literal>, e.g.
181 <literal>beamPackages.ibrowse</literal>.
182 </para>
183 <para>
184 Notably, <literal>buildRebar3</literal> includes
185 <literal>beamDeps</literal>, while
186 <literal>stdenv.mkDerivation</literal> does not. BEAM dependencies added
187 there will be correctly handled by the system.
188 </para>
189 <para>
190 If a package needs to compile native code via Rebar3's port compilation
191 mechanism, add <literal>compilePort = true;</literal> to the derivation.
192 </para>
193 </section>
194 <section xml:id="erlang-mk-packages">
195 <title>Erlang.mk Packages</title>
196 <para>
197 Erlang.mk functions similarly to Rebar3, except we use
198 <literal>buildErlangMk</literal> instead of
199 <literal>buildRebar3</literal>.
200 </para>
201 <programlisting>
202 { buildErlangMk, fetchHex, cowlib, ranch }:
203
204 buildErlangMk {
205 name = "cowboy";
206 version = "1.0.4";
207
208 src = fetchHex {
209 pkg = "cowboy";
210 version = "1.0.4";
211 sha256 = "6a0edee96885fae3a8dd0ac1f333538a42e807db638a9453064ccfdaa6b9fdac";
212 };
213
214 beamDeps = [ cowlib ranch ];
215
216 meta = {
217 description = ''
218 Small, fast, modular HTTP server written in Erlang
219 '';
220 license = stdenv.lib.licenses.isc;
221 homepage = https://github.com/ninenines/cowboy;
222 };
223 }
224 </programlisting>
225 </section>
226 <section xml:id="mix-packages">
227 <title>Mix Packages</title>
228 <para>
229 Mix functions similarly to Rebar3, except we use
230 <literal>buildMix</literal> instead of <literal>buildRebar3</literal>.
231 </para>
232 <programlisting>
233 { buildMix, fetchHex, plug, absinthe }:
234
235 buildMix {
236 name = "absinthe_plug";
237 version = "1.0.0";
238
239 src = fetchHex {
240 pkg = "absinthe_plug";
241 version = "1.0.0";
242 sha256 = "08459823fe1fd4f0325a8bf0c937a4520583a5a26d73b193040ab30a1dfc0b33";
243 };
244
245 beamDeps = [ plug absinthe ];
246
247 meta = {
248 description = ''
249 A plug for Absinthe, an experimental GraphQL toolkit
250 '';
251 license = stdenv.lib.licenses.bsd3;
252 homepage = https://github.com/CargoSense/absinthe_plug;
253 };
254 }
255 </programlisting>
256 <para>
257 Alternatively, we can use <literal>buildHex</literal> as a shortcut:
258 </para>
259 <programlisting>
260 { buildHex, buildMix, plug, absinthe }:
261
262 buildHex {
263 name = "absinthe_plug";
264 version = "1.0.0";
265
266 sha256 = "08459823fe1fd4f0325a8bf0c937a4520583a5a26d73b193040ab30a1dfc0b33";
267
268 builder = buildMix;
269
270 beamDeps = [ plug absinthe ];
271
272 meta = {
273 description = ''
274 A plug for Absinthe, an experimental GraphQL toolkit
275 '';
276 license = stdenv.lib.licenses.bsd3;
277 homepage = https://github.com/CargoSense/absinthe_plug;
278 };
279 }
280 </programlisting>
281 </section>
282 </section>
283</section>
284<section xml:id="how-to-develop">
285 <title>How to Develop</title>
286 <section xml:id="accessing-an-environment">
287 <title>Accessing an Environment</title>
288 <para>
289 Often, we simply want to access a valid environment that contains a
290 specific package and its dependencies. We can accomplish that with the
291 <literal>env</literal> attribute of a derivation. For example, let's say
292 we want to access an Erlang REPL with <literal>ibrowse</literal> loaded
293 up. We could do the following:
294 </para>
295 <programlisting>
296 $ nix-shell -A beamPackages.ibrowse.env --run "erl"
297 Erlang/OTP 18 [erts-7.0] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]
298
299 Eshell V7.0 (abort with ^G)
300 1> m(ibrowse).
301 Module: ibrowse
302 MD5: 3b3e0137d0cbb28070146978a3392945
303 Compiled: January 10 2016, 23:34
304 Object file: /nix/store/g1rlf65rdgjs4abbyj4grp37ry7ywivj-ibrowse-4.2.2/lib/erlang/lib/ibrowse-4.2.2/ebin/ibrowse.beam
305 Compiler options: [{outdir,"/tmp/nix-build-ibrowse-4.2.2.drv-0/hex-source-ibrowse-4.2.2/_build/default/lib/ibrowse/ebin"},
306 debug_info,debug_info,nowarn_shadow_vars,
307 warn_unused_import,warn_unused_vars,warnings_as_errors,
308 {i,"/tmp/nix-build-ibrowse-4.2.2.drv-0/hex-source-ibrowse-4.2.2/_build/default/lib/ibrowse/include"}]
309 Exports:
310 add_config/1 send_req_direct/7
311 all_trace_off/0 set_dest/3
312 code_change/3 set_max_attempts/3
313 get_config_value/1 set_max_pipeline_size/3
314 get_config_value/2 set_max_sessions/3
315 get_metrics/0 show_dest_status/0
316 get_metrics/2 show_dest_status/1
317 handle_call/3 show_dest_status/2
318 handle_cast/2 spawn_link_worker_process/1
319 handle_info/2 spawn_link_worker_process/2
320 init/1 spawn_worker_process/1
321 module_info/0 spawn_worker_process/2
322 module_info/1 start/0
323 rescan_config/0 start_link/0
324 rescan_config/1 stop/0
325 send_req/3 stop_worker_process/1
326 send_req/4 stream_close/1
327 send_req/5 stream_next/1
328 send_req/6 terminate/2
329 send_req_direct/4 trace_off/0
330 send_req_direct/5 trace_off/2
331 send_req_direct/6 trace_on/0
332 trace_on/2
333 ok
334 2>
335 </programlisting>
336 <para>
337 Notice the <literal>-A beamPackages.ibrowse.env</literal>. That is the key
338 to this functionality.
339 </para>
340 </section>
341 <section xml:id="creating-a-shell">
342 <title>Creating a Shell</title>
343 <para>
344 Getting access to an environment often isn't enough to do real
345 development. Usually, we need to create a <literal>shell.nix</literal>
346 file and do our development inside of the environment specified therein.
347 This file looks a lot like the packaging described above, except that
348 <literal>src</literal> points to the project root and we call the package
349 directly.
350 </para>
351 <programlisting>
352{ pkgs ? import "<nixpkgs"> {} }:
353
354with pkgs;
355
356let
357
358 f = { buildRebar3, ibrowse, jsx, erlware_commons }:
359 buildRebar3 {
360 name = "hex2nix";
361 version = "0.1.0";
362 src = ./.;
363 beamDeps = [ ibrowse jsx erlware_commons ];
364 };
365 drv = beamPackages.callPackage f {};
366
367in
368
369 drv
370 </programlisting>
371 <section xml:id="building-in-a-shell">
372 <title>Building in a Shell (for Mix Projects)</title>
373 <para>
374 We can leverage the support of the derivation, irrespective of the build
375 derivation, by calling the commands themselves.
376 </para>
377 <programlisting>
378# =============================================================================
379# Variables
380# =============================================================================
381
382NIX_TEMPLATES := "$(CURDIR)/nix-templates"
383
384TARGET := "$(PREFIX)"
385
386PROJECT_NAME := thorndyke
387
388NIXPKGS=../nixpkgs
389NIX_PATH=nixpkgs=$(NIXPKGS)
390NIX_SHELL=nix-shell -I "$(NIX_PATH)" --pure
391# =============================================================================
392# Rules
393# =============================================================================
394.PHONY= all test clean repl shell build test analyze configure install \
395 test-nix-install publish plt analyze
396
397all: build
398
399guard-%:
400 @ if [ "${${*}}" == "" ]; then \
401 echo "Environment variable $* not set"; \
402 exit 1; \
403 fi
404
405clean:
406 rm -rf _build
407 rm -rf .cache
408
409repl:
410 $(NIX_SHELL) --run "iex -pa './_build/prod/lib/*/ebin'"
411
412shell:
413 $(NIX_SHELL)
414
415configure:
416 $(NIX_SHELL) --command 'eval "$$configurePhase"'
417
418build: configure
419 $(NIX_SHELL) --command 'eval "$$buildPhase"'
420
421install:
422 $(NIX_SHELL) --command 'eval "$$installPhase"'
423
424test:
425 $(NIX_SHELL) --command 'mix test --no-start --no-deps-check'
426
427plt:
428 $(NIX_SHELL) --run "mix dialyzer.plt --no-deps-check"
429
430analyze: build plt
431 $(NIX_SHELL) --run "mix dialyzer --no-compile"
432
433 </programlisting>
434 <para>
435 Using a <literal>shell.nix</literal> as described (see <xref
436 linkend="creating-a-shell"/>) should just work. Aside from
437 <literal>test</literal>, <literal>plt</literal>, and
438 <literal>analyze</literal>, the Make targets work just fine for all of the
439 build derivations.
440 </para>
441 </section>
442</section>
443</section>
444<section xml:id="generating-packages-from-hex-with-hex2nix">
445 <title>Generating Packages from Hex with <literal>hex2nix</literal></title>
446 <para>
447 Updating the <link xlink:href="https://hex.pm">Hex</link> package set
448 requires <link
449 xlink:href="https://github.com/erlang-nix/hex2nix">hex2nix</link>. Given the
450 path to the Erlang modules (usually
451 <literal>pkgs/development/erlang-modules</literal>), it will dump a file
452 called <literal>hex-packages.nix</literal>, containing all the packages that
453 use a recognized build system in <link
454 xlink:href="https://hex.pm">Hex</link>. It can't be determined, however,
455 whether every package is buildable.
456 </para>
457 <para>
458 To make life easier for our users, try to build every <link
459 xlink:href="https://hex.pm">Hex</link> package and remove those that fail.
460 To do that, simply run the following command in the root of your
461 <literal>nixpkgs</literal> repository:
462 </para>
463 <programlisting>
464$ nix-build -A beamPackages
465 </programlisting>
466 <para>
467 That will attempt to build every package in
468 <literal>beamPackages</literal>. Then manually remove those that fail.
469 Hopefully, someone will improve <link
470 xlink:href="https://github.com/erlang-nix/hex2nix">hex2nix</link> in the
471 future to automate the process.
472 </para>
473</section>
474</section>