pyrox.dev
1<section xmlns="http://docbook.org/ns/docbook"
2 xmlns:xlink="http://www.w3.org/1999/xlink"
3 xml:id="sec-beam">
4
5 <title>Beam Languages (Erlang & Elixir)</title>
6 <section xml:id="beam-introduction">
7 <title>Introduction</title>
8 <para>
9 In this document and related Nix expressions we use the term
10 <emphasis>Beam</emphasis> to describe the environment. Beam is
11 the name of the Erlang Virtial Machine and, as far as we know,
12 from a packaging perspective all languages that run on Beam are
13 interchangable. The things that do change, like the build
14 system, are transperant to the users of the package. So we make
15 no distinction.
16 </para>
17 </section>
18<section xml:id="build-tools">
19 <title>Build Tools</title>
20 <section xml:id="build-tools-rebar3">
21 <title>Rebar3</title>
22 <para>
23 By default Rebar3 wants to manage it's own dependencies. In the
24 normal non-Nix, this is perfectly acceptable. In the Nix world it
25 is not. To support this we have created two versions of rebar3,
26 <literal>rebar3</literal> and <literal>rebar3-open</literal>. The
27 <literal>rebar3</literal> version has been patched to remove the
28 ability to download anything from it. If you are not running it a
29 nix-shell or a nix-build then its probably not going to work for
30 you. <literal>rebar3-open</literal> is the normal, un-modified
31 rebar3. It should work exactly as would any other version of
32 rebar3. Any Erlang package should rely on
33 <literal>rebar3</literal> and thats really what you should be
34 using too.
35 </para>
36 </section>
37 <section xml:id="build-tools-other">
38 <title>Mix & Erlang.mk</title>
39 <para>
40 Both Mix and Erlang.mk work exactly as you would expect. There
41 is a bootstrap process that needs to be run for both of
42 them. However, that is supported by the
43 <literal>buildMix</literal> and <literal>buildErlangMk</literal> derivations.
44 </para>
45 </section>
46
47</section>
48
49<section xml:id="how-to-install-beam-packages">
50 <title>How to install Beam packages</title>
51 <para>
52 Beam packages are not registered in the top level simply because
53 they are not relevant to the vast majority of Nix users. They are
54 installable using the <literal>beamPackages</literal> attribute
55 set.
56
57 You can list the avialable packages in the
58 <literal>beamPackages</literal> with the following command:
59 </para>
60
61 <programlisting>
62$ nix-env -f "<nixpkgs>" -qaP -A beamPackages
63beamPackages.esqlite esqlite-0.2.1
64beamPackages.goldrush goldrush-0.1.7
65beamPackages.ibrowse ibrowse-4.2.2
66beamPackages.jiffy jiffy-0.14.5
67beamPackages.lager lager-3.0.2
68beamPackages.meck meck-0.8.3
69beamPackages.rebar3-pc pc-1.1.0
70 </programlisting>
71 <para>
72 To install any of those packages into your profile, refer to them by
73 their attribute path (first column):
74 </para>
75 <programlisting>
76$ nix-env -f "<nixpkgs>" -iA beamPackages.ibrowse
77 </programlisting>
78 <para>
79 The attribute path of any Beam packages corresponds to the name
80 of that particular package in Hex or its OTP Application/Release name.
81 </para>
82</section>
83<section xml:id="packaging-beam-applications">
84 <title>Packaging Beam Applications</title>
85 <section xml:id="packaging-erlang-applications">
86 <title>Erlang Applications</title>
87 <section xml:id="rebar3-packages">
88 <title>Rebar3 Packages</title>
89 <para>
90 There is a Nix functional called
91 <literal>buildRebar3</literal>. We use this function to make a
92 derivation that understands how to build the rebar3 project. For
93 example, the epression we use to build the <link
94 xlink:href="https://github.com/erlang-nix/hex2nix">hex2nix</link>
95 project follows.
96 </para>
97 <programlisting>
98 {stdenv, fetchFromGitHub, buildRebar3, ibrowse, jsx, erlware_commons }:
99
100 buildRebar3 rec {
101 name = "hex2nix";
102 version = "0.0.1";
103
104 src = fetchFromGitHub {
105 owner = "ericbmerritt";
106 repo = "hex2nix";
107 rev = "${version}";
108 sha256 = "1w7xjidz1l5yjmhlplfx7kphmnpvqm67w99hd2m7kdixwdxq0zqg";
109 };
110
111 beamDeps = [ ibrowse jsx erlware_commons ];
112 }
113 </programlisting>
114 <para>
115 The only visible difference between this derivation and
116 something like <literal>stdenv.mkDerivation</literal> is that we
117 have added <literal>erlangDeps</literal> to the derivation. If
118 you add your Beam dependencies here they will be correctly
119 handled by the system.
120 </para>
121 <para>
122 If your package needs to compile native code via Rebar's port
123 compilation mechenism. You should add <literal>compilePort =
124 true;</literal> to the derivation.
125 </para>
126 </section>
127 <section xml:id="erlang-mk-packages">
128 <title>Erlang.mk Packages</title>
129 <para>
130 Erlang.mk functions almost identically to Rebar. The only real
131 difference is that <literal>buildErlangMk</literal> is called
132 instead of <literal>buildRebar3</literal>
133 </para>
134 <programlisting>
135 { buildErlangMk, fetchHex, cowlib, ranch }:
136 buildErlangMk {
137 name = "cowboy";
138 version = "1.0.4";
139 src = fetchHex {
140 pkg = "cowboy";
141 version = "1.0.4";
142 sha256 =
143 "6a0edee96885fae3a8dd0ac1f333538a42e807db638a9453064ccfdaa6b9fdac";
144 };
145 beamDeps = [ cowlib ranch ];
146
147 meta = {
148 description = ''Small, fast, modular HTTP server written in
149 Erlang.'';
150 license = stdenv.lib.licenses.isc;
151 homepage = "https://github.com/ninenines/cowboy";
152 };
153 }
154 </programlisting>
155 </section>
156 <section xml:id="mix-packages">
157 <title>Mix Packages</title>
158 <para>
159 Mix functions almost identically to Rebar. The only real
160 difference is that <literal>buildMix</literal> is called
161 instead of <literal>buildRebar3</literal>
162 </para>
163 <programlisting>
164 { buildMix, fetchHex, plug, absinthe }:
165 buildMix {
166 name = "absinthe_plug";
167 version = "1.0.0";
168 src = fetchHex {
169 pkg = "absinthe_plug";
170 version = "1.0.0";
171 sha256 =
172 "08459823fe1fd4f0325a8bf0c937a4520583a5a26d73b193040ab30a1dfc0b33";
173 };
174 beamDeps = [ plug absinthe];
175
176 meta = {
177 description = ''A plug for Absinthe, an experimental GraphQL
178 toolkit'';
179 license = stdenv.lib.licenses.bsd3;
180 homepage = "https://github.com/CargoSense/absinthe_plug";
181 };
182 }
183 </programlisting>
184 </section>
185 </section>
186</section>
187<section xml:id="how-to-develop">
188 <title>How to develop</title>
189 <section xml:id="accessing-an-environment">
190 <title>Accessing an Environment</title>
191 <para>
192 Often, all you want to do is be able to access a valid
193 environment that contains a specific package and its
194 dependencies. we can do that with the <literal>env</literal>
195 part of a derivation. For example, lets say we want to access an
196 erlang repl with ibrowse loaded up. We could do the following.
197 </para>
198 <programlisting>
199 ~/w/nixpkgs ❯❯❯ nix-shell -A beamPackages.ibrowse.env --run "erl"
200 Erlang/OTP 18 [erts-7.0] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]
201
202 Eshell V7.0 (abort with ^G)
203 1> m(ibrowse).
204 Module: ibrowse
205 MD5: 3b3e0137d0cbb28070146978a3392945
206 Compiled: January 10 2016, 23:34
207 Object file: /nix/store/g1rlf65rdgjs4abbyj4grp37ry7ywivj-ibrowse-4.2.2/lib/erlang/lib/ibrowse-4.2.2/ebin/ibrowse.beam
208 Compiler options: [{outdir,"/tmp/nix-build-ibrowse-4.2.2.drv-0/hex-source-ibrowse-4.2.2/_build/default/lib/ibrowse/ebin"},
209 debug_info,debug_info,nowarn_shadow_vars,
210 warn_unused_import,warn_unused_vars,warnings_as_errors,
211 {i,"/tmp/nix-build-ibrowse-4.2.2.drv-0/hex-source-ibrowse-4.2.2/_build/default/lib/ibrowse/include"}]
212 Exports:
213 add_config/1 send_req_direct/7
214 all_trace_off/0 set_dest/3
215 code_change/3 set_max_attempts/3
216 get_config_value/1 set_max_pipeline_size/3
217 get_config_value/2 set_max_sessions/3
218 get_metrics/0 show_dest_status/0
219 get_metrics/2 show_dest_status/1
220 handle_call/3 show_dest_status/2
221 handle_cast/2 spawn_link_worker_process/1
222 handle_info/2 spawn_link_worker_process/2
223 init/1 spawn_worker_process/1
224 module_info/0 spawn_worker_process/2
225 module_info/1 start/0
226 rescan_config/0 start_link/0
227 rescan_config/1 stop/0
228 send_req/3 stop_worker_process/1
229 send_req/4 stream_close/1
230 send_req/5 stream_next/1
231 send_req/6 terminate/2
232 send_req_direct/4 trace_off/0
233 send_req_direct/5 trace_off/2
234 send_req_direct/6 trace_on/0
235 trace_on/2
236 ok
237 2>
238 </programlisting>
239 <para>
240 Notice the <literal>-A beamPackages.ibrowse.env</literal>.That
241 is the key to this functionality.
242 </para>
243 </section>
244 <section xml:id="creating-a-shell">
245 <title>Creating a Shell</title>
246 <para>
247 Getting access to an environment often isn't enough to do real
248 development. Many times we need to create a
249 <literal>shell.nix</literal> file and do our development inside
250 of the environment specified by that file. This file looks a lot
251 like the packageing described above. The main difference is that
252 <literal>src</literal> points to project root and we call the
253 package directly.
254 </para>
255 <programlisting>
256{ pkgs ? import "<nixpkgs"> {} }:
257
258with pkgs;
259
260let
261
262 f = { buildRebar3, ibrowse, jsx, erlware_commons }:
263 buildRebar3 {
264 name = "hex2nix";
265 version = "0.1.0";
266 src = ./.;
267 erlangDeps = [ ibrowse jsx erlware_commons ];
268 };
269 drv = beamPackages.callPackage f {};
270
271in
272 drv
273 </programlisting>
274 <section xml:id="building-in-a-shell">
275 <title>Building in a shell</title>
276 <para>
277 We can leveral the support of the Derivation, regardless of
278 which build Derivation is called by calling the commands themselv.s
279 </para>
280 <programlisting>
281# =============================================================================
282# Variables
283# =============================================================================
284
285NIX_TEMPLATES := "$(CURDIR)/nix-templates"
286
287TARGET := "$(PREFIX)"
288
289PROJECT_NAME := thorndyke
290
291NIXPKGS=../nixpkgs
292NIX_PATH=nixpkgs=$(NIXPKGS)
293NIX_SHELL=nix-shell -I "$(NIX_PATH)" --pure
294# =============================================================================
295# Rules
296# =============================================================================
297.PHONY= all test clean repl shell build test analyze configure install \
298 test-nix-install publish plt analyze
299
300all: build
301
302guard-%:
303 @ if [ "${${*}}" == "" ]; then \
304 echo "Environment variable $* not set"; \
305 exit 1; \
306 fi
307
308clean:
309 rm -rf _build
310 rm -rf .cache
311
312repl:
313 $(NIX_SHELL) --run "iex -pa './_build/prod/lib/*/ebin'"
314
315shell:
316 $(NIX_SHELL)
317
318configure:
319 $(NIX_SHELL) --command 'eval "$$configurePhase"'
320
321build: configure
322 $(NIX_SHELL) --command 'eval "$$buildPhase"'
323
324install:
325 $(NIX_SHELL) --command 'eval "$$installPhase"'
326
327test:
328 $(NIX_SHELL) --command 'mix test --no-start --no-deps-check'
329
330plt:
331 $(NIX_SHELL) --run "mix dialyzer.plt --no-deps-check"
332
333analyze: build plt
334 $(NIX_SHELL) --run "mix dialyzer --no-compile"
335
336 </programlisting>
337 <para>
338 If you add the <literal>shell.nix</literal> as described and
339 user rebar as follows things should simply work. Aside from the
340 <literal>test</literal>, <literal>plt</literal>, and
341 <literal>analyze</literal> the talks work just fine for all of
342 the build Derivations.
343 </para>
344 </section>
345</section>
346</section>
347<section xml:id="generating-packages-from-hex-with-hex2nix">
348 <title>Generating Packages from Hex with Hex2Nix</title>
349 <para>
350 Updating the Hex packages requires the use of the
351 <literal>hex2nix</literal> tool. Given the path to the Erlang
352 modules (usually
353 <literal>pkgs/development/erlang-modules</literal>). It will
354 happily dump a file called
355 <literal>hex-packages.nix</literal>. That file will contain all
356 the packages that use a recognized build system in Hex. However,
357 it can't know whether or not all those packages are buildable.
358 </para>
359 <para>
360 To make life easier for our users, it makes good sense to go
361 ahead and attempt to build all those packages and remove the
362 ones that don't build. To do that, simply run the command (in
363 the root of your <literal>nixpkgs</literal> repository). that follows.
364 </para>
365 <programlisting>
366$ nix-build -A beamPackages
367 </programlisting>
368 <para>
369 That will build every package in
370 <literal>beamPackages</literal>. Then you can go through and
371 manually remove the ones that fail. Hopefully, someone will
372 improve <literal>hex2nix</literal> in the future to automate
373 that.
374 </para>
375</section>
376</section>