Thicket data repository for the EEG
anil.recoil.org
1{
2 "id": "https://ryan.freumh.org/hillingar.html",
3 "title": "Hillingar",
4 "link": "https://ryan.freumh.org/hillingar.html",
5 "updated": "2025-02-15T00:00:00",
6 "published": "2022-12-14T00:00:00",
7 "summary": "<div>\n \n <span>Published 14 Dec 2022.</span>\n \n \n <span>Last update 15 Feb 2025.</span>\n \n </div>\n \n <div> Tags: <a href=\"/projects.html\" title=\"All pages tagged 'projects'.\">projects</a>. </div>\n \n \n\n \n\n<blockquote>\n<p><span><a href=\"https://github.com/RyanGibb/hillingar\">Hillingar</a>, an <a href=\"https://en.wikipedia.org/wiki/Hillingar_effect\">arctic mirage</a>\n<span><a href=\"#ref-lehnNovayaZemlyaEffect1979\">[1]</a></span></span></p>\n</blockquote>\n<h2>Introduction</h2>\n<p><span>The Domain Name System (DNS) is a\ncritical component of the modern Internet, allowing domain names to be\nmapped to IP addresses, mailservers, and more<a href=\"#fn1\">1</a>.\nThis allows users to access services independent of their location in\nthe Internet using human-readable names. We can host a DNS server\nourselves to have authoritative control over our domain, protect the\nprivacy of those using our server, increase reliability by not relying\non a third party DNS provider, and allow greater customization of the\nrecords served (or the behaviour of the server itself). However, it can\nbe quite challenging to deploy one’s own server reliably and\nreproducibly, as I discovered during my master’s thesis <span><a href=\"#ref-gibbSpatialNameSystem2022\">[2]</a></span>. The Nix deployment system aims to\naddress this. With a NixOS machine, deploying a DNS server is as simple\nas:</span></p>\n<div><pre><code><span><a href=\"#cb1-1\"></a><span>{</span></span>\n<span><a href=\"#cb1-2\"></a> <span>services</span>.<span>bind</span> <span>=</span> <span>{</span></span>\n<span><a href=\"#cb1-3\"></a> <span>enable</span> <span>=</span> <span>true</span><span>;</span></span>\n<span><a href=\"#cb1-4\"></a> <span>zones</span>.<span>"freumh.org"</span> <span>=</span> <span>{</span></span>\n<span><a href=\"#cb1-5\"></a> <span>master</span> <span>=</span> <span>true</span><span>;</span></span>\n<span><a href=\"#cb1-6\"></a> <span>file</span> <span>=</span> <span>"freumh.org.zone"</span><span>;</span></span>\n<span><a href=\"#cb1-7\"></a> <span>};</span></span>\n<span><a href=\"#cb1-8\"></a> <span>};</span></span>\n<span><a href=\"#cb1-9\"></a><span>}</span></span></code></pre></div>\n<p><span>Which we can then query\nwith</span></p>\n<div><pre><code><span><a href=\"#cb2-1\"></a><span>$</span> dig ryan.freumh.org @ns1.ryan.freumh.org +short</span>\n<span><a href=\"#cb2-2\"></a><span>135.181.100.27</span></span></code></pre></div>\n<p><span>To enable the user to query our domain\nwithout specifying the nameserver, we have to create a glue record with\nour registrar pointing <code>ns1.freumh.org</code> to\nthe IP address of our DNS-hosting machine.</span></p>\n<p><span>You might notice this configuration is\nrunning the venerable bind<a href=\"#fn2\">2</a>, which is written in C.\nAs an alternative, using functional, high-level, type-safe programming\nlanguages to create network applications can greatly benefit safety and\nusability whilst maintaining performant execution <span><a href=\"#ref-madhavapeddyMelangeCreatingFunctional2007\">[3]</a></span>. One such language is\nOCaml.</span></p>\n<p><span>MirageOS<a href=\"#fn3\">3</a> is\na deployment method for these OCaml programs <span><a href=\"#ref-madhavapeddyUnikernelsLibraryOperating2013\">[4]</a></span>. Instead of running them as a\ntraditional Unix process, we instead create a specialised ‘unikernel’\noperating system to run the application, which allows dead code\nelimination improving security with smaller attack surfaces and improved\nefficiency.</span></p>\n<p><span>However, to deploy a Mirage unikernel\nwith NixOS, one must use the imperative deployment methodologies native\nto the OCaml ecosystem, eliminating the benefit of reproducible systems\nthat Nix offers. This blog post will explore how we enabled reproducible\ndeployments of Mirage unikernels by building them with Nix.</span></p>\n<p><span>At this point, the curious reader\nmight be wondering, what is ‘Nix’? Please see the separate webpage on <a href=\"nix.html\">Nix</a> for more.</span></p>\n<h2>MirageOS</h2>\n\n\n<img src=\"./images/mirage-logo.svg\">\n\n<a href=\"#fn4\">4</a>\n\n<p><span>MirageOS is a library operating system\nthat allows users to create unikernels, which are specialized operating\nsystems that include both low-level operating system code and high-level\napplication code in a single kernel and a single address space.<span><a href=\"#ref-madhavapeddyUnikernelsLibraryOperating2013\">[4]</a></span>.</span></p>\n<p><span>It was the first such ‘unikernel creation\nframework’, but comes from a long lineage of OS research, such as the\nexokernel library OS architecture <span><a href=\"#ref-englerExokernelOperatingSystem1995\">[5]</a></span>. Embedding application code in the\nkernel allows for dead-code elimination, removing OS interfaces that are\nunused, which reduces the unikernel’s attack surface and offers improved\nefficiency.</span></p>\n\n\n<img src=\"./images/mirage-diagram.svg\">\n\nContrasting software layers in existing VM appliances\nvs. unikernel’s standalone kernel compilation approach <span><a href=\"#ref-madhavapeddyUnikernelsLibraryOperating2013\">[4]</a></span>\n\n<p><span>Mirage unikernels are written OCaml<a href=\"#fn5\">5</a>. OCaml is more practical for systems\nprogramming than other functional programming languages, such as\nHaskell. It supports falling back on impure imperative code or mutable\nvariables when warranted.</span></p>\n<h2>Deploying Unikernels</h2>\n<p><span>Now that we understand what\nNix and Mirage are, and we’ve motivated the desire to deploy Mirage\nunikernels on a NixOS machine, what’s stopping us from doing just that?\nWell, to support deploying a Mirage unikernel, like for a DNS server, we\nwould need to write a NixOS module for it.</span></p>\n<p><span>A paired-down<a href=\"#fn6\">6</a>\nversion of the bind NixOS module, the module used in our Nix expression\nfor deploying a DNS server on NixOS (<a href=\"#cb1\">§</a>),\nis:</span></p>\n<div><pre><code><span><a href=\"#cb3-1\"></a><span>{</span> <span>config</span><span>,</span> <span>lib</span><span>,</span> <span>pkgs</span><span>,</span> <span>...</span> <span>}</span>:</span>\n<span><a href=\"#cb3-2\"></a></span>\n<span><a href=\"#cb3-3\"></a><span>with</span> lib<span>;</span></span>\n<span><a href=\"#cb3-4\"></a></span>\n<span><a href=\"#cb3-5\"></a><span>{</span></span>\n<span><a href=\"#cb3-6\"></a> <span>options</span> <span>=</span> <span>{</span></span>\n<span><a href=\"#cb3-7\"></a> <span>services</span>.<span>bind</span> <span>=</span> <span>{</span></span>\n<span><a href=\"#cb3-8\"></a> <span>enable</span> <span>=</span> mkEnableOption <span>"BIND domain name server"</span><span>;</span></span>\n<span><a href=\"#cb3-9\"></a></span>\n<span><a href=\"#cb3-10\"></a> <span>zones</span> <span>=</span> mkOption <span>{</span></span>\n<span><a href=\"#cb3-11\"></a> <span>...</span></span>\n<span><a href=\"#cb3-12\"></a> <span>};</span></span>\n<span><a href=\"#cb3-13\"></a> <span>};</span></span>\n<span><a href=\"#cb3-14\"></a> <span>};</span></span>\n<span><a href=\"#cb3-15\"></a></span>\n<span><a href=\"#cb3-16\"></a> <span>config</span> <span>=</span> mkIf cfg.enable <span>{</span></span>\n<span><a href=\"#cb3-17\"></a> <span>systemd</span>.<span>services</span>.<span>bind</span> <span>=</span> <span>{</span></span>\n<span><a href=\"#cb3-18\"></a> <span>description</span> <span>=</span> <span>"BIND Domain Name Server"</span><span>;</span></span>\n<span><a href=\"#cb3-19\"></a> <span>after</span> <span>=</span> <span>[</span> <span>"network.target"</span> <span>];</span></span>\n<span><a href=\"#cb3-20\"></a> <span>wantedBy</span> <span>=</span> <span>[</span> <span>"multi-user.target"</span> <span>];</span></span>\n<span><a href=\"#cb3-21\"></a></span>\n<span><a href=\"#cb3-22\"></a> <span>serviceConfig</span> <span>=</span> <span>{</span></span>\n<span><a href=\"#cb3-23\"></a> <span>ExecStart</span> <span>=</span> <span>"</span><span>${</span>pkgs.bind.out<span>}</span><span>/sbin/named"</span><span>;</span></span>\n<span><a href=\"#cb3-24\"></a> <span>};</span></span>\n<span><a href=\"#cb3-25\"></a> <span>};</span></span>\n<span><a href=\"#cb3-26\"></a> <span>};</span></span>\n<span><a href=\"#cb3-27\"></a><span>}</span></span></code></pre></div>\n<p><span>Notice the reference to <code>pkgs.bind</code>. This is the Nixpkgs repository Nix\nderivation for the <code>bind</code> package. Recall\nthat every input to a Nix derivation is itself a Nix derivation (<a href=\"#nixpkgs\">§</a>); in order to use a package in a Nix expression –\ni.e., a NixOS module – we need to build said package with Nix. Once we\nbuild a Mirage unikernel with Nix, we can write a NixOS module to deploy\nit.</span></p>\n<h2>Building Unikernels</h2>\n<p><span>Mirage uses the package manager\nfor OCaml called opam<a href=\"#fn7\">7</a>. Dependencies in opam, as is common\nin programming language package managers, have a file which – among\nother metadata, build/install scripts – specifies dependencies and their\nversion constraints. For example<a href=\"#fn8\">8</a></span></p>\n<pre><code>...\ndepends: [\n "arp" { ?monorepo & >= "3.0.0" & < "4.0.0" }\n "ethernet" { ?monorepo & >= "3.0.0" & < "4.0.0" }\n "lwt" { ?monorepo }\n "mirage" { build & >= "4.2.0" & < "4.3.0" }\n "mirage-bootvar-solo5" { ?monorepo & >= "0.6.0" & < "0.7.0" }\n "mirage-clock-solo5" { ?monorepo & >= "4.2.0" & < "5.0.0" }\n "mirage-crypto-rng-mirage" { ?monorepo & >= "0.8.0" & < "0.11.0" }\n "mirage-logs" { ?monorepo & >= "1.2.0" & < "2.0.0" }\n "mirage-net-solo5" { ?monorepo & >= "0.8.0" & < "0.9.0" }\n "mirage-random" { ?monorepo & >= "3.0.0" & < "4.0.0" }\n "mirage-runtime" { ?monorepo & >= "4.2.0" & < "4.3.0" }\n "mirage-solo5" { ?monorepo & >= "0.9.0" & < "0.10.0" }\n "mirage-time" { ?monorepo }\n "mirageio" { ?monorepo }\n "ocaml" { build & >= "4.08.0" }\n "ocaml-solo5" { build & >= "0.8.1" & < "0.9.0" }\n "opam-monorepo" { build & >= "0.3.2" }\n "tcpip" { ?monorepo & >= "7.0.0" & < "8.0.0" }\n "yaml" { ?monorepo & build }\n]\n...\n</code></pre>\n<p><span>Each of these dependencies will\nhave its own dependencies with their own version constraints. As we can\nonly link one dependency into the resulting program, we need to solve a\nset of dependency versions that satisfies these constraints. This is not\nan easy problem. In fact, it’s NP-complete <span><a href=\"#ref-coxVersionSAT2016\">[6]</a></span>. Opam uses the Zero Install<a href=\"#fn9\">9</a> SAT solver for dependency\nresolution.</span></p>\n<p><span>Nixpkgs has many OCaml\npackages<a href=\"#fn10\">10</a> which we could provide as build\ninputs to a Nix derivation<a href=\"#fn11\">11</a>. However, Nixpkgs has\none global coherent set of package versions<a href=\"#fn12\">12</a><span>, <a href=\"#fn13\">13</a></span>. The support for installing\nmultiple versions of a package concurrently comes from the fact that\nthey are stored at a unique path and can be referenced separately, or\nsymlinked, where required. So different projects or users that use a\ndifferent version of Nixpkgs won’t conflict, but Nix does not do any\ndependency version resolution – everything is pinned<a href=\"#fn14\">14</a>.\nThis is a problem for opam projects with version constraints that can’t\nbe satisfied with a static instance of Nixpkgs.</span></p>\n<p><span>Luckily, a project from Tweag\nalready exists (<code>opam-nix</code>) to deal with\nthis<a href=\"#fn15\">15</a><span>, <a href=\"#fn16\">16</a></span>. This project uses the opam\ndependency versions solver inside a Nix derivation, and then creates\nderivations from the resulting dependency versions<a href=\"#fn17\">17</a>.</span></p>\n<p><span>This still doesn’t support\nbuilding our Mirage unikernels, though. Unikernels quite often need to\nbe cross-compiled: compiled to run on a platform other than the one\nthey’re being built on. A common target, Solo5<a href=\"#fn18\">18</a>,\nis a sandboxed execution environment for unikernels. It acts as a\nminimal shim layer to interface between unikernels and different\nhypervisor backends. Solo5 uses a different <code>glibc</code> which requires cross-compilation. Mirage\n4<a href=\"#fn19\">19</a> supports cross compilation with\ntoolchains in the Dune build system<a href=\"#fn20\">20</a>. This uses a host\ncompiler installed in an opam switch (a virtual environment) as normal,\nas well as a target compiler<a href=\"#fn21\">21</a>. But the\ncross-compilation context of packages is only known at build time, as\nsome metaprogramming modules may require preprocessing with the host\ncompiler. To ensure that the right compilation context is used, we have\nto provide Dune with all our sources’ dependencies. A tool called <code>opam-monorepo</code> was date to do just that<a href=\"#fn22\">22</a>.</span></p>\n<p><span>We extended the <code>opam-nix</code> project to support the <code>opam-monorepo</code> workflow with this pull request:\n<a href=\"https://github.com/tweag/opam-nix/pull/18\">github.com/tweag/opam-nix/pull/18</a>.</span></p>\n<p><span>This is very low-level support\nfor building Mirage unikernels with Nix, however. In order to provide a\nbetter user experience, we also date the Hillingar Nix flake: <a href=\"https://github.com/RyanGibb/hillingar\">github.com/RyanGibb/hillingar</a>.\nThis wraps the Mirage tooling and <code>opam-nix</code>\nfunction calls so that a simple high-level flake can be dropped into a\nMirage project to support building it with Nix. To add Nix build support\nto a unikernel, simply:</span></p>\n<div><pre><code><span><a href=\"#cb5-1\"></a><span># create a flake from hillingar's default template</span></span>\n<span><a href=\"#cb5-2\"></a><span>$</span> nix flake new . <span>-t</span> github:/RyanGibb/hillingar</span>\n<span><a href=\"#cb5-3\"></a><span># substitute the name of the unikernel you're building</span></span>\n<span><a href=\"#cb5-4\"></a><span>$</span> sed <span>-i</span> <span>'s/throw "Put the unikernel name here"/"<unikernel-name>"/g'</span> flake.nix</span>\n<span><a href=\"#cb5-5\"></a><span># build the unikernel with Nix for a particular target</span></span>\n<span><a href=\"#cb5-6\"></a><span>$</span> nix build .#<span><</span>target<span>></span></span></code></pre></div>\n<p><span>For example, see the flake for\nbuilding the Mirage website as a unikernel with Nix: <a href=\"https://github.com/RyanGibb/mirage-www/blob/master/flake.nix\">github.com/RyanGibb/mirage-www/blob/master/flake.nix</a>.</span></p>\n<h2>Dependency Management</h2>\n<p><span>To step back for a moment and\nlook at the big picture, we can consider a number of different types of\ndependencies at play here:</span></p>\n<ol>\n<li>System dependencies: Are dependencies installed through the system\npackage manager – <code>depexts</code> in opam\nparlance. This is Nix for Hillingar, but another platform’s package\nmanagers include <code>apt</code>, <code>pacman</code>, and <code>brew</code>.\nFor unikernels, these are often C libraries like <code>gmp</code>.</li>\n<li>Library dependencies: Are installed through the programming language\npackage manager. For example <code>opam</code>, <code>pip</code>, and <code>npm</code>.\nThese are the dependencies that often have version constraints and\nrequire resolution possibly using a SAT solver.</li>\n<li>File dependencies: Are dependencies at the file system level of\ngranularity. For example, C files, Java (non-inner) classes, or OCaml\nmodules. Most likely this will be for a single project, but in a\nmonorepo, these could span many projects which all interoperate (e.g.,\nNixpkgs). This is the level of granularity that builds systems often\ndeal with, like Make, Dune, and Bazel.</li>\n<li>Function dependencies: Are dependencies between functions or another\nunit of code native to a language. For example, if function <code>a</code> calls function <code>b</code>, then <code>a</code>\n‘depends’ on <code>b</code>. This is the level of\ngranularity that compilers and interpreters are normally concerned with.\nIn the realms of higher-order functions this dependance may not be known\nin advance, but this is essentially the same problem that build systems\nface with dynamic dependencies <span><a href=\"#ref-mokhovBuildSystemsCarte2018\">[7]</a></span>.</li>\n</ol>\n<p><span>Nix deals well with system\ndependencies, but it doesn’t have a native way of resolving library\ndependency versions. Opam deals well with library dependencies, but it\ndoesn’t have a consistent way of installing system packages in a\nreproducible way. And Dune deals with file dependencies, but not the\nothers. The OCaml compiler keeps track of function dependencies when\ncompiling and linking a program.</span></p>\n<h3>Cross-Compilation</h3>\n<p><span>Dune is used to support\ncross-compilation for Mirage unikernels (<a href=\"#building-unikernels\">§</a>). We encode the cross-compilation\ncontext in Dune using the <code>preprocess</code>\nstanza from Dune’s DSL, for example from <a href=\"https://github.com/mirage/mirage-tcpip/blob/3ab30ab7b43dede75abf7b37838e051e0ddbb23a/src/tcp/dune#L9-L10\"><code>mirage-tcpip</code></a>:</span></p>\n<pre><code>(library\n (name tcp)\n (public_name tcpip.tcp)\n (instrumentation\n (backend bisect_ppx))\n (libraries logs ipaddr cstruct lwt-dllist mirage-profile tcpip.checksum\n tcpip duration randomconv fmt mirage-time mirage-clock mirage-random\n mirage-flow metrics)\n (preprocess\n (pps ppx_cstruct)))\n</code></pre>\n<p><span>Which tells Dune to preprocess\nthe opam package <code>ppx_cstruct</code> with the host\ncompiler. As this information is only available from the build manager,\nthis requires fetching all dependency sources to support\ncross-compilation with the <code>opam-monorepo</code>\ntool:</span></p>\n<blockquote>\n<p><span>Cross-compilation - the details\nof how to build some native code can come late in the pipeline, which\nisn’t a problem if the sources are available<a href=\"#fn23\">23</a>.</span></p>\n</blockquote>\n<p><span>This means we’re essentially\nencoding the compilation context in the build system rules. To remove\nthe requirement to clone dependency sources locally with <code>opam-monorepo</code> we could try and encode the\ncompilation context in the package manager. However, preprocessing can\nbe at the OCaml module level of granularity. Dune deals with this level\nof granularity with file dependencies, but opam doesn’t. Tighter\nintegration between the build and package manager could improve this\nsituation, like Rust’s Cargo. There are some plans towards modularising\nopam and creating tighter integration with Dune.</span></p>\n<p><span>There is also the possibility of\nusing Nix to avoid cross-compilation. Nixpkg’s cross compilation<a href=\"#fn24\">24</a> will not innately help us here, as\nit simply specifies how to package software in a cross-compilation\nfriendly way. However, Nix remote builders would enable reproducible\nbuilds on a remote machine<a href=\"#fn25\">25</a> with Nix installed\nthat may sidestep the need for cross-compilation in certain\ncontexts.</span></p>\n<h3>Version Resolution</h3>\n<p><span>Hillingar uses the Zero Install\nSAT solver for version resolution through opam. While this works, it\nisn’t the most principled approach for getting Nix to work with library\ndependencies. Some package managers are just using Nix for system\ndependencies and using the existing tooling as normal for library\ndependencies<a href=\"#fn26\">26</a>. But generally, <code>X2nix</code> projects are numerous and created in an\n<em>ad hoc</em> way. Part of this is dealing with every language’s\necosystems package repository system, and there are existing\napproaches<a href=\"#fn27\">27</a><span>, <a href=\"#fn28\">28</a></span> aimed at reducing code\nduplication, but there is still the fundamental problem of version\nresolution. Nix uses pointers (paths) to refer to different versions of\na dependency, which works well when solving the diamond dependency\nproblem for system dependencies, but we don’t have this luxury when\nlinking a binary with library dependencies.</span></p>\n\n\n<img src=\"images/version-sat.svg\">\n\nThe diamond dependency problem <span><a href=\"#ref-coxVersionSAT2016\">[6]</a></span>.\n\n<p><span>This is exactly why opam uses a\nconstraint solver to find a coherent package set. But what if we could\nsplit version-solving functionality into something that can tie into any\nlanguage ecosystem? This could be a more principled, elegant, approach\nto the current fragmented state of library dependencies (program\nlanguage package managers). This would require some ecosystem-specific\nlogic to obtain, for example, the version constraints and to create\nderivations for the resulting sources, but the core functionality could\nbe ecosystem agnostic. As with <code>opam-nix</code>,\nmaterialization<a href=\"#fn29\">29</a> could be used to commit a lock file\nand avoid IFD. Although perhaps this is too lofty a goal to be\npractical, and perhaps the real issues are organisational rather than\ntechnical.</span></p>\n<p><span>Nix allows multiple versions of\na package to be installed simultaneously by having different derivations\nrefer to different paths in the Nix store concurrently. What if we could\nuse a similar approach for linking binaries to sidestep the version\nconstraint solving altogether at the cost of larger binaries? Nix makes\na similar tradeoff makes with disk space. A very simple approach might\nbe to programmatically prepend/append functions in <code>D</code> with the dependency version name <code>vers1</code> and <code>vers2</code>\nfor calls in the packages <code>B</code> and <code>C</code> respectively in the diagram above.</span></p>\n<blockquote>\n<p><span>Another way to avoid\nNP-completeness is to attack assumption 4: what if two different\nversions of a package could be installed simultaneously? Then almost any\nsearch algorithm will find a combination of packages to build the\nprogram; it just might not be the smallest possible combination (that’s\nstill NP-complete). If <code>B</code> needs <code>D</code> 1.5 and <code>C</code> needs\nD 2.2, the build can include both packages in the final binary, treating\nthem as distinct packages. I mentioned above that there can’t be two\ndefinitions of <code>printf</code> built into a C\nprogram, but languages with explicit module systems should have no\nproblem including separate copies of <code>D</code>\n(under different fully-qualified names) into a program. <span><a href=\"#ref-coxVersionSAT2016\">[6]</a></span></span></p>\n</blockquote>\n<p><span>Another wackier idea is, instead\nof having programmers manually specific constraints with version\nnumbers, to resolve dependencies purely based on typing<a href=\"#fn30\">30</a>.\nThe issue here is that solving dependencies would now involve type\nchecking, which could prove computationally expensive.</span></p>\n<h3>Build Systems</h3>\n<p><span>The build script in a Nix derivation\n(if it doesn’t invoke a compiler directly) often invokes a build system\nlike Make, or in this case Dune. But Nix can also be considered a build\nsystem with a suspending scheduler and deep constructive trace\nrebuilding <span><a href=\"#ref-mokhovBuildSystemsCarte2018\">[7]</a></span>. But Nix is at a coarse-grained\npackage level, invoking these finer-grained build systems to deal with\nfile dependencies.</span></p>\n<p><span>In Chapter 10 of the original Nix\nthesis <span><a href=\"#ref-dolstraPurelyFunctionalSoftware2006\">[8]</a></span>, low-level build management using\nNix is discussed, proposing extending Nix to support file dependencies.\nFor example, to build the ATerm library:</span></p>\n<div><pre><code><span><a href=\"#cb7-1\"></a><span>{</span><span>sharedLib</span> <span>?</span> <span>true</span><span>}</span>:</span>\n<span><a href=\"#cb7-2\"></a></span>\n<span><a href=\"#cb7-3\"></a><span>with</span> <span>(</span><span>import</span> <span>../../../lib</span><span>);</span></span>\n<span><a href=\"#cb7-4\"></a></span>\n<span><a href=\"#cb7-5\"></a><span>rec</span> <span>{</span></span>\n<span><a href=\"#cb7-6\"></a> <span>sources</span> <span>=</span> <span>[</span></span>\n<span><a href=\"#cb7-7\"></a> <span>./afun.c</span> <span>./aterm.c</span> <span>./bafio.c</span> <span>./byteio.c</span> <span>./gc.c</span> <span>./hash.c</span></span>\n<span><a href=\"#cb7-8\"></a> <span>./list.c</span> <span>./make.c</span> <span>./md5c.c</span> <span>./memory.c</span> <span>./tafio.c</span> <span>./version.c</span></span>\n<span><a href=\"#cb7-9\"></a> <span>];</span></span>\n<span><a href=\"#cb7-10\"></a></span>\n<span><a href=\"#cb7-11\"></a> <span>compile</span> <span>=</span> <span>main</span><span>:</span> compileC <span>{</span><span>inherit</span> main sharedLib<span>;};</span></span>\n<span><a href=\"#cb7-12\"></a></span>\n<span><a href=\"#cb7-13\"></a> <span>libATerm</span> <span>=</span> makeLibrary <span>{</span></span>\n<span><a href=\"#cb7-14\"></a> <span>libraryName</span> <span>=</span> <span>"ATerm"</span><span>;</span></span>\n<span><a href=\"#cb7-15\"></a> <span>objects</span> <span>=</span> <span>map</span> compile sources<span>;</span></span>\n<span><a href=\"#cb7-16\"></a> <span>inherit</span> sharedLib<span>;</span></span>\n<span><a href=\"#cb7-17\"></a> <span>};</span></span>\n<span><a href=\"#cb7-18\"></a><span>}</span></span></code></pre></div>\n<p><span>This has the advantage over\ntraditional build systems like Make that if a dependency isn’t\nspecified, the build will fail. And if the build succeeds, the build\nwill succeed. So it’s not possible to make incomplete dependency\nspecifications, which could lead to inconsistent builds.</span></p>\n<p><span>A downside, however, is that Nix\ndoesn’t support dynamic dependencies. We need to know the derivation\ninputs in advance of invoking the build script. This is why in Hillingar\nwe need to use IFD to import from a derivation invoking opam to solve\ndependency versions.</span></p>\n<p><span>There is prior art that aims to\nsupport building Dune projects with Nix in the low-level manner\ndescribed called <a href=\"https://gitlab.com/balsoft/tumbleweed\">tumbleweed</a>. While this\nproject is now abandoned, it shows the difficulties of trying to work\nwith existing ecosystems. The Dune build system files need to be parsed\nand interpreted in Nix, which either requires convoluted and error-prone\nNix code or painfully slow IFD. The former approach is taken with\ntumbleweed which means it could potentially benefit from improving the\nNix language. But fundamentally this still requires the complex task of\nreimplementing part of Dune in another language.</span></p>\n<p><span>I would be very interested if anyone\nreading this knows if this idea went anywhere! A potential issue I see\nwith this is the computational and storage overhead associated with\nstoring derivations in the Nix store that are manageable for\ncoarse-grained dependencies might prove too costly for fine-grained file\ndependencies.</span></p>\n<p><span>While on the topic of build systems,\nto enable more minimal builds tighter integration with the compiler\nwould enable analysing function dependencies<a href=\"#fn31\">31</a>.\nFor example, Dune could recompile only certain functions that have\nchanged since the last invocation. Taking granularity to such a fine\ndegree will cause a great increase in the size of the build graph,\nhowever. Recomputing this graph for every invocation may prove more\ncostly than doing the actual rebuilding after a certain point. Perhaps\npersisting the build graph and calculating differentials of it could\nmitigate this. A meta-build-graph, if you will.</span></p>\n<h2>Evaulation</h2>\n<p><span>Hillingar’s primary limitations are (1)\ncomplex integration is required with the OCaml ecosystem to solve\ndependency version constraints using <code>opam-nix</code>, and (2) that cross-compilation\nrequires cloning all sources locally with <code>opam-monorepo</code> (<a href=\"#dependency-management\">§</a>). Another issue that proved an\nannoyance during this project is the Nix DSL’s dynamic typing. When\nwriting simple derivations this often isn’t a problem, but when writing\ncomplicated logic, it quickly gets in the way of productivity. The\nruntime errors produced can be very hard to parse. Thankfully there is\nwork towards creating a typed language for the Nix deployment system,\nsuch as Nickel<a href=\"#fn32\">32</a>. However, gradual typing is hard,\nand Nickel still isn’t ready for real-world use despite being\nopen-sourced (in a week as of writing this) for two years.</span></p>\n<p><span>A glaring omission is that despite it\nbeing the primary motivation, we haven’t actually written a NixOS module\nfor deploying a DNS server as a unikernel. There are still questions\nabout how to provide zonefile data declaratively to the unikernel, and\nmanage the runtime of deployed unikernels. One option to do the latter\nis Albatross<a href=\"#fn33\">33</a>, which has recently had support for\nbuilding with nix added<a href=\"#fn34\">34</a>. Albatross aims to provision\nresources for unikernels such as network access, share resources for\nunikernels between users, and monitor unikernels with a Unix daemon.\nUsing Albatross to manage some of the inherent imperative processes\nbehind unikernels, as well as share access to resources for unikernels\nfor other users on a NixOS system, could simplify the creation and\nimprove the functionality of a NixOS module for a unikernel.</span></p>\n<p><span>There also exists related work in the\nreproducible building of Mirage unikernels. Specifically, improving the\nreproducibility of opam packages (as Mirage unikernels are opam packages\nthemselves)<a href=\"#fn35\">35</a>. Hillingar differs in that it only\nuses opam for version resolution, instead using Nix to provide\ndependencies, which provides reproducibility with pinned Nix derivation\ninputs and builds in isolation by default.</span></p>\n<h2>Conclusion</h2>\n<p><span>To summarise, this project was motivated\n(<a href=\"#introduction\">§</a>) by deploying unikernels on NixOS (<a href=\"#deploying-unikernels\">§</a>). Towards this end, we added support\nfor building MirageOS unikernels with Nix; we extended <code>opam-nix</code> to support the <code>opam-monorepo</code> workflow and created the Hillingar\nproject to provide a usable Nix interface (<a href=\"#building-unikernels\">§</a>). This required scrutinising the OCaml\nand Nix ecosystems along the way in order to marry them; some thoughts\non dependency management were developed in this context (<a href=\"#dependency-management\">§</a>). Many strange issues and edge cases\nwere uncovered during this project but now that we’ve encoded them in\nNix, hopefully, others won’t have to repeat the experience!</span></p>\n<p><span>While only the first was the primary\nmotivation, the benefits of building unikernels with Nix are:</span></p>\n<ul>\n<li>Reproducible and low-config unikernel deployment using NixOS modules\nis enabled.</li>\n<li>Nix allows reproducible builds pinning system dependencies and\ncomposing multiple language environments. For example, the OCaml package\n<code>conf-gmp</code> is a ‘virtual package’ that\nrelies on a system installation of the C/Assembly library <code>gmp</code> (The GNU Multiple Precision Arithmetic\nLibrary). Nix easily allows us to depend on this package in a\nreproducible way.</li>\n<li>We can use Nix to support building on different systems (<a href=\"#cross-compilation\">§</a>).</li>\n</ul>\n<p><span>While NixOS and MirageOS take\nfundamentally very different approaches, they’re both trying to bring\nsome kind of functional programming paradigm to operating systems. NixOS\ndoes this in a top-down manner, trying to tame Unix with functional\nprinciples like laziness and immutability<a href=\"#fn36\">36</a>;\nwhereas, MirageOS does this by throwing Unix out the window and\nrebuilding the world from scratch in a very much bottom-up approach.\nDespite these two projects having different motivations and goals,\nHillingar aims to get the best from both worlds by marrying the\ntwo.</span></p>\n\n\n<p><span>I want to thank some people for their\nhelp with this project:</span></p>\n<ul>\n<li>Lucas Pluvinage for invaluable help with the OCaml ecosystem.</li>\n<li>Alexander Bantyev for getting me up to speed with the <code>opam-nix</code> project and working with me on the\n<code>opam-monorepo</code> workflow integration.</li>\n<li>David Allsopp for his opam expertise.</li>\n<li>Jules Aguillon and Olivier Nicole for their fellow\nNix-enthusiasm.</li>\n<li>Sonja Heinze for her PPX insights.</li>\n<li>Anil Madhavapeddy for having a discussion that led to the idea for\nthis project.</li>\n<li>Björg Bjarnadóttir for her Icelandic language consultation.</li>\n<li>And finally, everyone at Tarides for being so welcoming and\nhelpful!</li>\n</ul>\n<p><span>This work was completed with the support\nof <a href=\"https://tarides.com/\">Tarides</a>, and a version of this\nblog post can be found <a href=\"https://tarides.com/blog/2022-12-14-hillingar-mirageos-unikernels-on-nixos\">on\nthe Tarides website</a>.</span></p>\n<p><span>If you have any questions or comments on\nthis feel free to <a href=\"about.html#contact\">get in\ntouch</a>.</span></p>\n<p><span>If you have a unikernel, consider trying\nto build it with Hillingar, and please report any problems at <a href=\"https://github.com/RyanGibb/hillingar/issues\">github.com/RyanGibb/hillingar/issues</a>!</span></p>\n\n\n<h2>References</h2>\n<p><span><span></span></span></p>\n<div>\n<div>\n<span><div>[1] </div><div>W. H. Lehn, <span>“The <span>Novaya\nZemlya</span> effect: <span>An</span> arctic mirage,”</span> <em>J. Opt.\nSoc. Am., JOSA</em>, vol. 69, no. 5, pp. 776–781, May 1979, doi: <a href=\"https://doi.org/10.1364/JOSA.69.000776\">10.1364/JOSA.69.000776</a>.\n[Online]. Available: <a href=\"https://opg.optica.org/josa/abstract.cfm?uri=josa-69-5-776\">https://opg.optica.org/josa/abstract.cfm?uri=josa-69-5-776</a>.\n[Accessed: Oct. 05, 2022]</div></span>\n</div>\n<div>\n<span><div>[2] </div><div>R. T. Gibb, <span>“Spatial <span>Name\nSystem</span>,”</span> Nov. 30, 2022. [Online]. Available: <a href=\"http://arxiv.org/abs/2210.05036\">http://arxiv.org/abs/2210.05036</a>.\n[Accessed: Jun. 30, 2023]</div></span>\n</div>\n<div>\n<span><div>[3] </div><div>A. Madhavapeddy, A. Ho, T. Deegan, D. Scott,\nand R. Sohan, <span>“Melange: Creating a \"functional\" internet,”</span>\n<em>SIGOPS Oper. Syst. Rev.</em>, vol. 41, no. 3, pp. 101–114, Mar.\n2007, doi: <a href=\"https://doi.org/10.1145/1272998.1273009\">10.1145/1272998.1273009</a>.\n[Online]. Available: <a href=\"https://doi.org/10.1145/1272998.1273009\">https://doi.org/10.1145/1272998.1273009</a>.\n[Accessed: Feb. 10, 2022]</div></span>\n</div>\n<div>\n<span><div>[4] </div><div>A. Madhavapeddy <em>et al.</em>,\n<span>“Unikernels: Library operating systems for the cloud,”</span>\n<em>SIGARCH Comput. Archit. News</em>, vol. 41, no. 1, pp. 461–472, Mar.\n2013, doi: <a href=\"https://doi.org/10.1145/2490301.2451167\">10.1145/2490301.2451167</a>.\n[Online]. Available: <a href=\"https://doi.org/10.1145/2490301.2451167\">https://doi.org/10.1145/2490301.2451167</a>.\n[Accessed: Jan. 25, 2022]</div></span>\n</div>\n<div>\n<span><div>[5] </div><div>D. R. Engler, M. F. Kaashoek, and J. O’Toole,\n<span>“Exokernel: An operating system architecture for application-level\nresource management,”</span> <em>SIGOPS Oper. Syst. Rev.</em>, vol. 29,\nno. 5, pp. 251–266, Dec. 1995, doi: <a href=\"https://doi.org/10.1145/224057.224076\">10.1145/224057.224076</a>.\n[Online]. Available: <a href=\"https://doi.org/10.1145/224057.224076\">https://doi.org/10.1145/224057.224076</a>.\n[Accessed: Jan. 25, 2022]</div></span>\n</div>\n<div>\n<span><div>[6] </div><div>R. Cox, <span>“Version\n<span>SAT</span>,”</span> Dec. 13, 2016. [Online]. Available: <a href=\"https://research.swtch.com/version-sat\">https://research.swtch.com/version-sat</a>.\n[Accessed: Oct. 16, 2022]</div></span>\n</div>\n<div>\n<span><div>[7] </div><div>A. Mokhov, N. Mitchell, and S. Peyton Jones,\n<span>“Build systems à la carte,”</span> <em>Proc. ACM Program.\nLang.</em>, vol. 2, pp. 1–29, Jul. 2018, doi: <a href=\"https://doi.org/10.1145/3236774\">10.1145/3236774</a>. [Online].\nAvailable: <a href=\"https://dl.acm.org/doi/10.1145/3236774\">https://dl.acm.org/doi/10.1145/3236774</a>.\n[Accessed: Oct. 11, 2022]</div></span>\n</div>\n<div>\n<span><div>[8] </div><div>E. Dolstra, <span>“The purely functional\nsoftware deployment model,”</span> [s.n.], S.l., 2006 [Online].\nAvailable: <a href=\"https://edolstra.github.io/pubs/phd-thesis.pdf\">https://edolstra.github.io/pubs/phd-thesis.pdf</a></div></span>\n</div>\n</div>\n\n\n\n\n<ol>\n<li><p><span><a href=\"./dns-loc-rr.html\">DNS LOC</a></span><a href=\"#fnref1\">↩︎</a></p></li>\n<li><p><span><a href=\"https://www.isc.org/bind/\">ISC bind</a> has many <a href=\"https://www.cvedetails.com/product/144/ISC-Bind.html?vendor_id=64\">CVE’s</a></span><a href=\"#fnref2\">↩︎</a></p></li>\n<li><p><span><a href=\"https://mirage.io\">mirage.io</a></span><a href=\"#fnref3\">↩︎</a></p></li>\n<li><p><span>Credits to Takayuki\nImada</span><a href=\"#fnref4\">↩︎</a></p></li>\n<li><p><span>Barring the use of <a href=\"https://mirage.io/blog/modular-foreign-function-bindings\">foreign\nfunction interfaces</a> (FFIs).</span><a href=\"#fnref5\">↩︎</a></p></li>\n<li><p><span>The full module\ncan be found <a href=\"https://github.com/NixOS/nixpkgs/blob/fe76645aaf2fac3baaa2813fd0089930689c53b5/nixos/modules/services/networking/bind.nix\">here</a></span><a href=\"#fnref6\">↩︎</a></p></li>\n<li><p><span><a href=\"https://opam.ocaml.org/\">opam.ocaml.org</a></span><a href=\"#fnref7\">↩︎</a></p></li>\n<li><p><span>For <a href=\"https://github.com/mirage/mirage-www\">mirage-www</a> targetting\n<code>hvt</code>.</span><a href=\"#fnref8\">↩︎</a></p></li>\n<li><p><span><a href=\"https://0install.net\">0install.net</a></span><a href=\"#fnref9\">↩︎</a></p></li>\n<li><p><span><a href=\"https://github.com/NixOS/nixpkgs/blob/9234f5a17e1a7820b5e91ecd4ff0de449e293383/pkgs/development/ocaml-modules/\">github.com/NixOS/nixpkgs\npkgs/development/ocaml-modules</a></span><a href=\"#fnref10\">↩︎</a></p></li>\n<li><p><span>NB they are not\nas complete nor up-to-date as those in <code>opam-repository</code> <a href=\"https://github.com/ocaml/opam-repository\">github.com/ocaml/opam-repository</a>.</span><a href=\"#fnref11\">↩︎</a></p></li>\n<li><p><span>Bar some\nexceptional packages that have multiple major versions packaged, like\nPostgres.</span><a href=\"#fnref12\">↩︎</a></p></li>\n<li><p><span>In fact Arch has\nthe same approach, which is why it <a href=\"nix.html#nixos\">doesn’t\nsupport partial upgrades</a>.</span><a href=\"#fnref13\">↩︎</a></p></li>\n<li><p><span>This has led to\nmuch confusion with how to install a specific version of a package <a href=\"https://github.com/NixOS/nixpkgs/issues/9682\">github.com/NixOS/nixpkgs/issues/9682</a>.</span><a href=\"#fnref14\">↩︎</a></p></li>\n<li><p><span><a href=\"https://github.com/tweag/opam-nix\">github.com/tweag/opam-nix</a></span><a href=\"#fnref15\">↩︎</a></p></li>\n<li><p><span>Another project,\n<a href=\"https://github.com/timbertson/opam2nix\">timbertson/opam2nix</a>,\nalso exists but depends on a binary of itself at build time as it’s\nwritten in OCaml as opposed to Nix, is not as minimal (higher LOC\ncount), and it isn’t under active development (with development focused\non <a href=\"https://github.com/timbertson/fetlock\">github.com/timbertson/fetlock</a>)</span><a href=\"#fnref16\">↩︎</a></p></li>\n<li><p><span>Using something\ncalled <a href=\"https://nixos.wiki/wiki/Import_From_Derivation\">Import\nFrom Derivation (IFD)</a>. Materialisation can be used to create a kind\nof lock file for this resolution, which can be committed to the project\nto avoid having to do IFD on every new build. An alternative may be to\nuse opam’s built-in version pinning[fn:47].</span><a href=\"#fnref17\">↩︎</a></p></li>\n<li><p><span><a href=\"https://github.com/Solo5/solo5\">github.com/Solo5/solo5</a></span><a href=\"#fnref18\">↩︎</a></p></li>\n<li><p><span><a href=\"https://mirage.io/blog/announcing-mirage-40\">mirage.io/blog/announcing-mirage-40</a></span><a href=\"#fnref19\">↩︎</a></p></li>\n<li><p><span><a href=\"https://dune.build\">dune.build</a></span><a href=\"#fnref20\">↩︎</a></p></li>\n<li><p><span><a href=\"https://github.com/mirage/ocaml-solo5\">github.com/mirage/ocaml-solo5</a></span><a href=\"#fnref21\">↩︎</a></p></li>\n<li><p><span><a href=\"https://github.com/tarides/opam-monorepo\">github.com/tarides/opam-monorepo</a></span><a href=\"#fnref22\">↩︎</a></p></li>\n<li><p><span><a href=\"https://github.com/tarides/opam-monorepo/blob/feeb325c9c8d560c6b92cbde62b6a9c5f20ed032/doc/faq.mld#L42\">github.com/tarides/opam-monorepo</a></span><a href=\"#fnref23\">↩︎</a></p></li>\n<li><p><span><a href=\"https://nixos.org/manual/nixpkgs/stable/#chap-cross\">nixos.org/manual/nixpkgs/stable/#chap-cross</a></span><a href=\"#fnref24\">↩︎</a></p></li>\n<li><p><span><a href=\"https://nixos.org/manual/nix/stable/advanced-topics/distributed-builds.html\">nixos.org/manual/nix/stable/advanced-topics/distributed-builds.html</a></span><a href=\"#fnref25\">↩︎</a></p></li>\n<li><p><span><a href=\"https://docs.haskellstack.org/en/stable/nix_integration/\">docs.haskellstack.org/en/stable/nix_integration</a></span><a href=\"#fnref26\">↩︎</a></p></li>\n<li><p><span><a href=\"https://github.com/nix-community/dream2nix\">github.com/nix-community/dream2nix</a></span><a href=\"#fnref27\">↩︎</a></p></li>\n<li><p><span><a href=\"https://github.com/timbertson/fetlock\">github.com/timbertson/fetlock</a></span><a href=\"#fnref28\">↩︎</a></p></li>\n<li><p><span><a href=\"https://github.com/tweag/opam-nix/blob/4e602e02a82a720c2f1d7324ea29dc9c7916a9c2/README.md#materialization\"><span>https://github.com/tweag/opam-nix#materialization</span></a></span><a href=\"#fnref29\">↩︎</a></p></li>\n<li><p><span><a href=\"https://twitter.com/TheLortex/status/1571884882363830273\">twitter.com/TheLortex/status/1571884882363830273</a></span><a href=\"#fnref30\">↩︎</a></p></li>\n<li><p><span><a href=\"https://signalsandthreads.com/build-systems/#4305\">signalsandthreads.com/build-systems/#4305</a></span><a href=\"#fnref31\">↩︎</a></p></li>\n<li><p><span><a href=\"https://www.tweag.io/blog/2020-10-22-nickel-open-sourcing/\">www.tweag.io/blog/2020-10-22-nickel-open-sourcing</a></span><a href=\"#fnref32\">↩︎</a></p></li>\n<li><p><span><a href=\"https://hannes.robur.coop/Posts/VMM\">hannes.robur.coop/Posts/VMM</a></span><a href=\"#fnref33\">↩︎</a></p></li>\n<li><p><span><a href=\"https://github.com/roburio/albatross/pull/120\">https://github.com/roburio/albatross/pull/120</a></span><a href=\"#fnref34\">↩︎</a></p></li>\n<li><p><span><a href=\"https://hannes.nqsb.io/Posts/ReproducibleOPAM\">hannes.nqsb.io/Posts/ReproducibleOPAM</a></span><a href=\"#fnref35\">↩︎</a></p></li>\n<li><p><span><a href=\"https://www.tweag.io/blog/2022-07-14-taming-unix-with-nix/\">tweag.io/blog/2022-07-14-taming-unix-with-nix</a></span><a href=\"#fnref36\">↩︎</a></p></li>\n</ol>",
8 "content": "<div>\n \n <span>Published 14 Dec 2022.</span>\n \n \n <span>Last update 15 Feb 2025.</span>\n \n </div>\n \n <div> Tags: <a href=\"/projects.html\" title=\"All pages tagged 'projects'.\">projects</a>. </div>\n \n \n\n \n\n<blockquote>\n<p><span><a href=\"https://github.com/RyanGibb/hillingar\">Hillingar</a>, an <a href=\"https://en.wikipedia.org/wiki/Hillingar_effect\">arctic mirage</a>\n<span><a href=\"#ref-lehnNovayaZemlyaEffect1979\">[1]</a></span></span></p>\n</blockquote>\n<h2>Introduction</h2>\n<p><span>The Domain Name System (DNS) is a\ncritical component of the modern Internet, allowing domain names to be\nmapped to IP addresses, mailservers, and more<a href=\"#fn1\">1</a>.\nThis allows users to access services independent of their location in\nthe Internet using human-readable names. We can host a DNS server\nourselves to have authoritative control over our domain, protect the\nprivacy of those using our server, increase reliability by not relying\non a third party DNS provider, and allow greater customization of the\nrecords served (or the behaviour of the server itself). However, it can\nbe quite challenging to deploy one’s own server reliably and\nreproducibly, as I discovered during my master’s thesis <span><a href=\"#ref-gibbSpatialNameSystem2022\">[2]</a></span>. The Nix deployment system aims to\naddress this. With a NixOS machine, deploying a DNS server is as simple\nas:</span></p>\n<div><pre><code><span><a href=\"#cb1-1\"></a><span>{</span></span>\n<span><a href=\"#cb1-2\"></a> <span>services</span>.<span>bind</span> <span>=</span> <span>{</span></span>\n<span><a href=\"#cb1-3\"></a> <span>enable</span> <span>=</span> <span>true</span><span>;</span></span>\n<span><a href=\"#cb1-4\"></a> <span>zones</span>.<span>"freumh.org"</span> <span>=</span> <span>{</span></span>\n<span><a href=\"#cb1-5\"></a> <span>master</span> <span>=</span> <span>true</span><span>;</span></span>\n<span><a href=\"#cb1-6\"></a> <span>file</span> <span>=</span> <span>"freumh.org.zone"</span><span>;</span></span>\n<span><a href=\"#cb1-7\"></a> <span>};</span></span>\n<span><a href=\"#cb1-8\"></a> <span>};</span></span>\n<span><a href=\"#cb1-9\"></a><span>}</span></span></code></pre></div>\n<p><span>Which we can then query\nwith</span></p>\n<div><pre><code><span><a href=\"#cb2-1\"></a><span>$</span> dig ryan.freumh.org @ns1.ryan.freumh.org +short</span>\n<span><a href=\"#cb2-2\"></a><span>135.181.100.27</span></span></code></pre></div>\n<p><span>To enable the user to query our domain\nwithout specifying the nameserver, we have to create a glue record with\nour registrar pointing <code>ns1.freumh.org</code> to\nthe IP address of our DNS-hosting machine.</span></p>\n<p><span>You might notice this configuration is\nrunning the venerable bind<a href=\"#fn2\">2</a>, which is written in C.\nAs an alternative, using functional, high-level, type-safe programming\nlanguages to create network applications can greatly benefit safety and\nusability whilst maintaining performant execution <span><a href=\"#ref-madhavapeddyMelangeCreatingFunctional2007\">[3]</a></span>. One such language is\nOCaml.</span></p>\n<p><span>MirageOS<a href=\"#fn3\">3</a> is\na deployment method for these OCaml programs <span><a href=\"#ref-madhavapeddyUnikernelsLibraryOperating2013\">[4]</a></span>. Instead of running them as a\ntraditional Unix process, we instead create a specialised ‘unikernel’\noperating system to run the application, which allows dead code\nelimination improving security with smaller attack surfaces and improved\nefficiency.</span></p>\n<p><span>However, to deploy a Mirage unikernel\nwith NixOS, one must use the imperative deployment methodologies native\nto the OCaml ecosystem, eliminating the benefit of reproducible systems\nthat Nix offers. This blog post will explore how we enabled reproducible\ndeployments of Mirage unikernels by building them with Nix.</span></p>\n<p><span>At this point, the curious reader\nmight be wondering, what is ‘Nix’? Please see the separate webpage on <a href=\"nix.html\">Nix</a> for more.</span></p>\n<h2>MirageOS</h2>\n\n\n<img src=\"./images/mirage-logo.svg\">\n\n<a href=\"#fn4\">4</a>\n\n<p><span>MirageOS is a library operating system\nthat allows users to create unikernels, which are specialized operating\nsystems that include both low-level operating system code and high-level\napplication code in a single kernel and a single address space.<span><a href=\"#ref-madhavapeddyUnikernelsLibraryOperating2013\">[4]</a></span>.</span></p>\n<p><span>It was the first such ‘unikernel creation\nframework’, but comes from a long lineage of OS research, such as the\nexokernel library OS architecture <span><a href=\"#ref-englerExokernelOperatingSystem1995\">[5]</a></span>. Embedding application code in the\nkernel allows for dead-code elimination, removing OS interfaces that are\nunused, which reduces the unikernel’s attack surface and offers improved\nefficiency.</span></p>\n\n\n<img src=\"./images/mirage-diagram.svg\">\n\nContrasting software layers in existing VM appliances\nvs. unikernel’s standalone kernel compilation approach <span><a href=\"#ref-madhavapeddyUnikernelsLibraryOperating2013\">[4]</a></span>\n\n<p><span>Mirage unikernels are written OCaml<a href=\"#fn5\">5</a>. OCaml is more practical for systems\nprogramming than other functional programming languages, such as\nHaskell. It supports falling back on impure imperative code or mutable\nvariables when warranted.</span></p>\n<h2>Deploying Unikernels</h2>\n<p><span>Now that we understand what\nNix and Mirage are, and we’ve motivated the desire to deploy Mirage\nunikernels on a NixOS machine, what’s stopping us from doing just that?\nWell, to support deploying a Mirage unikernel, like for a DNS server, we\nwould need to write a NixOS module for it.</span></p>\n<p><span>A paired-down<a href=\"#fn6\">6</a>\nversion of the bind NixOS module, the module used in our Nix expression\nfor deploying a DNS server on NixOS (<a href=\"#cb1\">§</a>),\nis:</span></p>\n<div><pre><code><span><a href=\"#cb3-1\"></a><span>{</span> <span>config</span><span>,</span> <span>lib</span><span>,</span> <span>pkgs</span><span>,</span> <span>...</span> <span>}</span>:</span>\n<span><a href=\"#cb3-2\"></a></span>\n<span><a href=\"#cb3-3\"></a><span>with</span> lib<span>;</span></span>\n<span><a href=\"#cb3-4\"></a></span>\n<span><a href=\"#cb3-5\"></a><span>{</span></span>\n<span><a href=\"#cb3-6\"></a> <span>options</span> <span>=</span> <span>{</span></span>\n<span><a href=\"#cb3-7\"></a> <span>services</span>.<span>bind</span> <span>=</span> <span>{</span></span>\n<span><a href=\"#cb3-8\"></a> <span>enable</span> <span>=</span> mkEnableOption <span>"BIND domain name server"</span><span>;</span></span>\n<span><a href=\"#cb3-9\"></a></span>\n<span><a href=\"#cb3-10\"></a> <span>zones</span> <span>=</span> mkOption <span>{</span></span>\n<span><a href=\"#cb3-11\"></a> <span>...</span></span>\n<span><a href=\"#cb3-12\"></a> <span>};</span></span>\n<span><a href=\"#cb3-13\"></a> <span>};</span></span>\n<span><a href=\"#cb3-14\"></a> <span>};</span></span>\n<span><a href=\"#cb3-15\"></a></span>\n<span><a href=\"#cb3-16\"></a> <span>config</span> <span>=</span> mkIf cfg.enable <span>{</span></span>\n<span><a href=\"#cb3-17\"></a> <span>systemd</span>.<span>services</span>.<span>bind</span> <span>=</span> <span>{</span></span>\n<span><a href=\"#cb3-18\"></a> <span>description</span> <span>=</span> <span>"BIND Domain Name Server"</span><span>;</span></span>\n<span><a href=\"#cb3-19\"></a> <span>after</span> <span>=</span> <span>[</span> <span>"network.target"</span> <span>];</span></span>\n<span><a href=\"#cb3-20\"></a> <span>wantedBy</span> <span>=</span> <span>[</span> <span>"multi-user.target"</span> <span>];</span></span>\n<span><a href=\"#cb3-21\"></a></span>\n<span><a href=\"#cb3-22\"></a> <span>serviceConfig</span> <span>=</span> <span>{</span></span>\n<span><a href=\"#cb3-23\"></a> <span>ExecStart</span> <span>=</span> <span>"</span><span>${</span>pkgs.bind.out<span>}</span><span>/sbin/named"</span><span>;</span></span>\n<span><a href=\"#cb3-24\"></a> <span>};</span></span>\n<span><a href=\"#cb3-25\"></a> <span>};</span></span>\n<span><a href=\"#cb3-26\"></a> <span>};</span></span>\n<span><a href=\"#cb3-27\"></a><span>}</span></span></code></pre></div>\n<p><span>Notice the reference to <code>pkgs.bind</code>. This is the Nixpkgs repository Nix\nderivation for the <code>bind</code> package. Recall\nthat every input to a Nix derivation is itself a Nix derivation (<a href=\"#nixpkgs\">§</a>); in order to use a package in a Nix expression –\ni.e., a NixOS module – we need to build said package with Nix. Once we\nbuild a Mirage unikernel with Nix, we can write a NixOS module to deploy\nit.</span></p>\n<h2>Building Unikernels</h2>\n<p><span>Mirage uses the package manager\nfor OCaml called opam<a href=\"#fn7\">7</a>. Dependencies in opam, as is common\nin programming language package managers, have a file which – among\nother metadata, build/install scripts – specifies dependencies and their\nversion constraints. For example<a href=\"#fn8\">8</a></span></p>\n<pre><code>...\ndepends: [\n "arp" { ?monorepo & >= "3.0.0" & < "4.0.0" }\n "ethernet" { ?monorepo & >= "3.0.0" & < "4.0.0" }\n "lwt" { ?monorepo }\n "mirage" { build & >= "4.2.0" & < "4.3.0" }\n "mirage-bootvar-solo5" { ?monorepo & >= "0.6.0" & < "0.7.0" }\n "mirage-clock-solo5" { ?monorepo & >= "4.2.0" & < "5.0.0" }\n "mirage-crypto-rng-mirage" { ?monorepo & >= "0.8.0" & < "0.11.0" }\n "mirage-logs" { ?monorepo & >= "1.2.0" & < "2.0.0" }\n "mirage-net-solo5" { ?monorepo & >= "0.8.0" & < "0.9.0" }\n "mirage-random" { ?monorepo & >= "3.0.0" & < "4.0.0" }\n "mirage-runtime" { ?monorepo & >= "4.2.0" & < "4.3.0" }\n "mirage-solo5" { ?monorepo & >= "0.9.0" & < "0.10.0" }\n "mirage-time" { ?monorepo }\n "mirageio" { ?monorepo }\n "ocaml" { build & >= "4.08.0" }\n "ocaml-solo5" { build & >= "0.8.1" & < "0.9.0" }\n "opam-monorepo" { build & >= "0.3.2" }\n "tcpip" { ?monorepo & >= "7.0.0" & < "8.0.0" }\n "yaml" { ?monorepo & build }\n]\n...\n</code></pre>\n<p><span>Each of these dependencies will\nhave its own dependencies with their own version constraints. As we can\nonly link one dependency into the resulting program, we need to solve a\nset of dependency versions that satisfies these constraints. This is not\nan easy problem. In fact, it’s NP-complete <span><a href=\"#ref-coxVersionSAT2016\">[6]</a></span>. Opam uses the Zero Install<a href=\"#fn9\">9</a> SAT solver for dependency\nresolution.</span></p>\n<p><span>Nixpkgs has many OCaml\npackages<a href=\"#fn10\">10</a> which we could provide as build\ninputs to a Nix derivation<a href=\"#fn11\">11</a>. However, Nixpkgs has\none global coherent set of package versions<a href=\"#fn12\">12</a><span>, <a href=\"#fn13\">13</a></span>. The support for installing\nmultiple versions of a package concurrently comes from the fact that\nthey are stored at a unique path and can be referenced separately, or\nsymlinked, where required. So different projects or users that use a\ndifferent version of Nixpkgs won’t conflict, but Nix does not do any\ndependency version resolution – everything is pinned<a href=\"#fn14\">14</a>.\nThis is a problem for opam projects with version constraints that can’t\nbe satisfied with a static instance of Nixpkgs.</span></p>\n<p><span>Luckily, a project from Tweag\nalready exists (<code>opam-nix</code>) to deal with\nthis<a href=\"#fn15\">15</a><span>, <a href=\"#fn16\">16</a></span>. This project uses the opam\ndependency versions solver inside a Nix derivation, and then creates\nderivations from the resulting dependency versions<a href=\"#fn17\">17</a>.</span></p>\n<p><span>This still doesn’t support\nbuilding our Mirage unikernels, though. Unikernels quite often need to\nbe cross-compiled: compiled to run on a platform other than the one\nthey’re being built on. A common target, Solo5<a href=\"#fn18\">18</a>,\nis a sandboxed execution environment for unikernels. It acts as a\nminimal shim layer to interface between unikernels and different\nhypervisor backends. Solo5 uses a different <code>glibc</code> which requires cross-compilation. Mirage\n4<a href=\"#fn19\">19</a> supports cross compilation with\ntoolchains in the Dune build system<a href=\"#fn20\">20</a>. This uses a host\ncompiler installed in an opam switch (a virtual environment) as normal,\nas well as a target compiler<a href=\"#fn21\">21</a>. But the\ncross-compilation context of packages is only known at build time, as\nsome metaprogramming modules may require preprocessing with the host\ncompiler. To ensure that the right compilation context is used, we have\nto provide Dune with all our sources’ dependencies. A tool called <code>opam-monorepo</code> was date to do just that<a href=\"#fn22\">22</a>.</span></p>\n<p><span>We extended the <code>opam-nix</code> project to support the <code>opam-monorepo</code> workflow with this pull request:\n<a href=\"https://github.com/tweag/opam-nix/pull/18\">github.com/tweag/opam-nix/pull/18</a>.</span></p>\n<p><span>This is very low-level support\nfor building Mirage unikernels with Nix, however. In order to provide a\nbetter user experience, we also date the Hillingar Nix flake: <a href=\"https://github.com/RyanGibb/hillingar\">github.com/RyanGibb/hillingar</a>.\nThis wraps the Mirage tooling and <code>opam-nix</code>\nfunction calls so that a simple high-level flake can be dropped into a\nMirage project to support building it with Nix. To add Nix build support\nto a unikernel, simply:</span></p>\n<div><pre><code><span><a href=\"#cb5-1\"></a><span># create a flake from hillingar's default template</span></span>\n<span><a href=\"#cb5-2\"></a><span>$</span> nix flake new . <span>-t</span> github:/RyanGibb/hillingar</span>\n<span><a href=\"#cb5-3\"></a><span># substitute the name of the unikernel you're building</span></span>\n<span><a href=\"#cb5-4\"></a><span>$</span> sed <span>-i</span> <span>'s/throw "Put the unikernel name here"/"<unikernel-name>"/g'</span> flake.nix</span>\n<span><a href=\"#cb5-5\"></a><span># build the unikernel with Nix for a particular target</span></span>\n<span><a href=\"#cb5-6\"></a><span>$</span> nix build .#<span><</span>target<span>></span></span></code></pre></div>\n<p><span>For example, see the flake for\nbuilding the Mirage website as a unikernel with Nix: <a href=\"https://github.com/RyanGibb/mirage-www/blob/master/flake.nix\">github.com/RyanGibb/mirage-www/blob/master/flake.nix</a>.</span></p>\n<h2>Dependency Management</h2>\n<p><span>To step back for a moment and\nlook at the big picture, we can consider a number of different types of\ndependencies at play here:</span></p>\n<ol>\n<li>System dependencies: Are dependencies installed through the system\npackage manager – <code>depexts</code> in opam\nparlance. This is Nix for Hillingar, but another platform’s package\nmanagers include <code>apt</code>, <code>pacman</code>, and <code>brew</code>.\nFor unikernels, these are often C libraries like <code>gmp</code>.</li>\n<li>Library dependencies: Are installed through the programming language\npackage manager. For example <code>opam</code>, <code>pip</code>, and <code>npm</code>.\nThese are the dependencies that often have version constraints and\nrequire resolution possibly using a SAT solver.</li>\n<li>File dependencies: Are dependencies at the file system level of\ngranularity. For example, C files, Java (non-inner) classes, or OCaml\nmodules. Most likely this will be for a single project, but in a\nmonorepo, these could span many projects which all interoperate (e.g.,\nNixpkgs). This is the level of granularity that builds systems often\ndeal with, like Make, Dune, and Bazel.</li>\n<li>Function dependencies: Are dependencies between functions or another\nunit of code native to a language. For example, if function <code>a</code> calls function <code>b</code>, then <code>a</code>\n‘depends’ on <code>b</code>. This is the level of\ngranularity that compilers and interpreters are normally concerned with.\nIn the realms of higher-order functions this dependance may not be known\nin advance, but this is essentially the same problem that build systems\nface with dynamic dependencies <span><a href=\"#ref-mokhovBuildSystemsCarte2018\">[7]</a></span>.</li>\n</ol>\n<p><span>Nix deals well with system\ndependencies, but it doesn’t have a native way of resolving library\ndependency versions. Opam deals well with library dependencies, but it\ndoesn’t have a consistent way of installing system packages in a\nreproducible way. And Dune deals with file dependencies, but not the\nothers. The OCaml compiler keeps track of function dependencies when\ncompiling and linking a program.</span></p>\n<h3>Cross-Compilation</h3>\n<p><span>Dune is used to support\ncross-compilation for Mirage unikernels (<a href=\"#building-unikernels\">§</a>). We encode the cross-compilation\ncontext in Dune using the <code>preprocess</code>\nstanza from Dune’s DSL, for example from <a href=\"https://github.com/mirage/mirage-tcpip/blob/3ab30ab7b43dede75abf7b37838e051e0ddbb23a/src/tcp/dune#L9-L10\"><code>mirage-tcpip</code></a>:</span></p>\n<pre><code>(library\n (name tcp)\n (public_name tcpip.tcp)\n (instrumentation\n (backend bisect_ppx))\n (libraries logs ipaddr cstruct lwt-dllist mirage-profile tcpip.checksum\n tcpip duration randomconv fmt mirage-time mirage-clock mirage-random\n mirage-flow metrics)\n (preprocess\n (pps ppx_cstruct)))\n</code></pre>\n<p><span>Which tells Dune to preprocess\nthe opam package <code>ppx_cstruct</code> with the host\ncompiler. As this information is only available from the build manager,\nthis requires fetching all dependency sources to support\ncross-compilation with the <code>opam-monorepo</code>\ntool:</span></p>\n<blockquote>\n<p><span>Cross-compilation - the details\nof how to build some native code can come late in the pipeline, which\nisn’t a problem if the sources are available<a href=\"#fn23\">23</a>.</span></p>\n</blockquote>\n<p><span>This means we’re essentially\nencoding the compilation context in the build system rules. To remove\nthe requirement to clone dependency sources locally with <code>opam-monorepo</code> we could try and encode the\ncompilation context in the package manager. However, preprocessing can\nbe at the OCaml module level of granularity. Dune deals with this level\nof granularity with file dependencies, but opam doesn’t. Tighter\nintegration between the build and package manager could improve this\nsituation, like Rust’s Cargo. There are some plans towards modularising\nopam and creating tighter integration with Dune.</span></p>\n<p><span>There is also the possibility of\nusing Nix to avoid cross-compilation. Nixpkg’s cross compilation<a href=\"#fn24\">24</a> will not innately help us here, as\nit simply specifies how to package software in a cross-compilation\nfriendly way. However, Nix remote builders would enable reproducible\nbuilds on a remote machine<a href=\"#fn25\">25</a> with Nix installed\nthat may sidestep the need for cross-compilation in certain\ncontexts.</span></p>\n<h3>Version Resolution</h3>\n<p><span>Hillingar uses the Zero Install\nSAT solver for version resolution through opam. While this works, it\nisn’t the most principled approach for getting Nix to work with library\ndependencies. Some package managers are just using Nix for system\ndependencies and using the existing tooling as normal for library\ndependencies<a href=\"#fn26\">26</a>. But generally, <code>X2nix</code> projects are numerous and created in an\n<em>ad hoc</em> way. Part of this is dealing with every language’s\necosystems package repository system, and there are existing\napproaches<a href=\"#fn27\">27</a><span>, <a href=\"#fn28\">28</a></span> aimed at reducing code\nduplication, but there is still the fundamental problem of version\nresolution. Nix uses pointers (paths) to refer to different versions of\na dependency, which works well when solving the diamond dependency\nproblem for system dependencies, but we don’t have this luxury when\nlinking a binary with library dependencies.</span></p>\n\n\n<img src=\"images/version-sat.svg\">\n\nThe diamond dependency problem <span><a href=\"#ref-coxVersionSAT2016\">[6]</a></span>.\n\n<p><span>This is exactly why opam uses a\nconstraint solver to find a coherent package set. But what if we could\nsplit version-solving functionality into something that can tie into any\nlanguage ecosystem? This could be a more principled, elegant, approach\nto the current fragmented state of library dependencies (program\nlanguage package managers). This would require some ecosystem-specific\nlogic to obtain, for example, the version constraints and to create\nderivations for the resulting sources, but the core functionality could\nbe ecosystem agnostic. As with <code>opam-nix</code>,\nmaterialization<a href=\"#fn29\">29</a> could be used to commit a lock file\nand avoid IFD. Although perhaps this is too lofty a goal to be\npractical, and perhaps the real issues are organisational rather than\ntechnical.</span></p>\n<p><span>Nix allows multiple versions of\na package to be installed simultaneously by having different derivations\nrefer to different paths in the Nix store concurrently. What if we could\nuse a similar approach for linking binaries to sidestep the version\nconstraint solving altogether at the cost of larger binaries? Nix makes\na similar tradeoff makes with disk space. A very simple approach might\nbe to programmatically prepend/append functions in <code>D</code> with the dependency version name <code>vers1</code> and <code>vers2</code>\nfor calls in the packages <code>B</code> and <code>C</code> respectively in the diagram above.</span></p>\n<blockquote>\n<p><span>Another way to avoid\nNP-completeness is to attack assumption 4: what if two different\nversions of a package could be installed simultaneously? Then almost any\nsearch algorithm will find a combination of packages to build the\nprogram; it just might not be the smallest possible combination (that’s\nstill NP-complete). If <code>B</code> needs <code>D</code> 1.5 and <code>C</code> needs\nD 2.2, the build can include both packages in the final binary, treating\nthem as distinct packages. I mentioned above that there can’t be two\ndefinitions of <code>printf</code> built into a C\nprogram, but languages with explicit module systems should have no\nproblem including separate copies of <code>D</code>\n(under different fully-qualified names) into a program. <span><a href=\"#ref-coxVersionSAT2016\">[6]</a></span></span></p>\n</blockquote>\n<p><span>Another wackier idea is, instead\nof having programmers manually specific constraints with version\nnumbers, to resolve dependencies purely based on typing<a href=\"#fn30\">30</a>.\nThe issue here is that solving dependencies would now involve type\nchecking, which could prove computationally expensive.</span></p>\n<h3>Build Systems</h3>\n<p><span>The build script in a Nix derivation\n(if it doesn’t invoke a compiler directly) often invokes a build system\nlike Make, or in this case Dune. But Nix can also be considered a build\nsystem with a suspending scheduler and deep constructive trace\nrebuilding <span><a href=\"#ref-mokhovBuildSystemsCarte2018\">[7]</a></span>. But Nix is at a coarse-grained\npackage level, invoking these finer-grained build systems to deal with\nfile dependencies.</span></p>\n<p><span>In Chapter 10 of the original Nix\nthesis <span><a href=\"#ref-dolstraPurelyFunctionalSoftware2006\">[8]</a></span>, low-level build management using\nNix is discussed, proposing extending Nix to support file dependencies.\nFor example, to build the ATerm library:</span></p>\n<div><pre><code><span><a href=\"#cb7-1\"></a><span>{</span><span>sharedLib</span> <span>?</span> <span>true</span><span>}</span>:</span>\n<span><a href=\"#cb7-2\"></a></span>\n<span><a href=\"#cb7-3\"></a><span>with</span> <span>(</span><span>import</span> <span>../../../lib</span><span>);</span></span>\n<span><a href=\"#cb7-4\"></a></span>\n<span><a href=\"#cb7-5\"></a><span>rec</span> <span>{</span></span>\n<span><a href=\"#cb7-6\"></a> <span>sources</span> <span>=</span> <span>[</span></span>\n<span><a href=\"#cb7-7\"></a> <span>./afun.c</span> <span>./aterm.c</span> <span>./bafio.c</span> <span>./byteio.c</span> <span>./gc.c</span> <span>./hash.c</span></span>\n<span><a href=\"#cb7-8\"></a> <span>./list.c</span> <span>./make.c</span> <span>./md5c.c</span> <span>./memory.c</span> <span>./tafio.c</span> <span>./version.c</span></span>\n<span><a href=\"#cb7-9\"></a> <span>];</span></span>\n<span><a href=\"#cb7-10\"></a></span>\n<span><a href=\"#cb7-11\"></a> <span>compile</span> <span>=</span> <span>main</span><span>:</span> compileC <span>{</span><span>inherit</span> main sharedLib<span>;};</span></span>\n<span><a href=\"#cb7-12\"></a></span>\n<span><a href=\"#cb7-13\"></a> <span>libATerm</span> <span>=</span> makeLibrary <span>{</span></span>\n<span><a href=\"#cb7-14\"></a> <span>libraryName</span> <span>=</span> <span>"ATerm"</span><span>;</span></span>\n<span><a href=\"#cb7-15\"></a> <span>objects</span> <span>=</span> <span>map</span> compile sources<span>;</span></span>\n<span><a href=\"#cb7-16\"></a> <span>inherit</span> sharedLib<span>;</span></span>\n<span><a href=\"#cb7-17\"></a> <span>};</span></span>\n<span><a href=\"#cb7-18\"></a><span>}</span></span></code></pre></div>\n<p><span>This has the advantage over\ntraditional build systems like Make that if a dependency isn’t\nspecified, the build will fail. And if the build succeeds, the build\nwill succeed. So it’s not possible to make incomplete dependency\nspecifications, which could lead to inconsistent builds.</span></p>\n<p><span>A downside, however, is that Nix\ndoesn’t support dynamic dependencies. We need to know the derivation\ninputs in advance of invoking the build script. This is why in Hillingar\nwe need to use IFD to import from a derivation invoking opam to solve\ndependency versions.</span></p>\n<p><span>There is prior art that aims to\nsupport building Dune projects with Nix in the low-level manner\ndescribed called <a href=\"https://gitlab.com/balsoft/tumbleweed\">tumbleweed</a>. While this\nproject is now abandoned, it shows the difficulties of trying to work\nwith existing ecosystems. The Dune build system files need to be parsed\nand interpreted in Nix, which either requires convoluted and error-prone\nNix code or painfully slow IFD. The former approach is taken with\ntumbleweed which means it could potentially benefit from improving the\nNix language. But fundamentally this still requires the complex task of\nreimplementing part of Dune in another language.</span></p>\n<p><span>I would be very interested if anyone\nreading this knows if this idea went anywhere! A potential issue I see\nwith this is the computational and storage overhead associated with\nstoring derivations in the Nix store that are manageable for\ncoarse-grained dependencies might prove too costly for fine-grained file\ndependencies.</span></p>\n<p><span>While on the topic of build systems,\nto enable more minimal builds tighter integration with the compiler\nwould enable analysing function dependencies<a href=\"#fn31\">31</a>.\nFor example, Dune could recompile only certain functions that have\nchanged since the last invocation. Taking granularity to such a fine\ndegree will cause a great increase in the size of the build graph,\nhowever. Recomputing this graph for every invocation may prove more\ncostly than doing the actual rebuilding after a certain point. Perhaps\npersisting the build graph and calculating differentials of it could\nmitigate this. A meta-build-graph, if you will.</span></p>\n<h2>Evaulation</h2>\n<p><span>Hillingar’s primary limitations are (1)\ncomplex integration is required with the OCaml ecosystem to solve\ndependency version constraints using <code>opam-nix</code>, and (2) that cross-compilation\nrequires cloning all sources locally with <code>opam-monorepo</code> (<a href=\"#dependency-management\">§</a>). Another issue that proved an\nannoyance during this project is the Nix DSL’s dynamic typing. When\nwriting simple derivations this often isn’t a problem, but when writing\ncomplicated logic, it quickly gets in the way of productivity. The\nruntime errors produced can be very hard to parse. Thankfully there is\nwork towards creating a typed language for the Nix deployment system,\nsuch as Nickel<a href=\"#fn32\">32</a>. However, gradual typing is hard,\nand Nickel still isn’t ready for real-world use despite being\nopen-sourced (in a week as of writing this) for two years.</span></p>\n<p><span>A glaring omission is that despite it\nbeing the primary motivation, we haven’t actually written a NixOS module\nfor deploying a DNS server as a unikernel. There are still questions\nabout how to provide zonefile data declaratively to the unikernel, and\nmanage the runtime of deployed unikernels. One option to do the latter\nis Albatross<a href=\"#fn33\">33</a>, which has recently had support for\nbuilding with nix added<a href=\"#fn34\">34</a>. Albatross aims to provision\nresources for unikernels such as network access, share resources for\nunikernels between users, and monitor unikernels with a Unix daemon.\nUsing Albatross to manage some of the inherent imperative processes\nbehind unikernels, as well as share access to resources for unikernels\nfor other users on a NixOS system, could simplify the creation and\nimprove the functionality of a NixOS module for a unikernel.</span></p>\n<p><span>There also exists related work in the\nreproducible building of Mirage unikernels. Specifically, improving the\nreproducibility of opam packages (as Mirage unikernels are opam packages\nthemselves)<a href=\"#fn35\">35</a>. Hillingar differs in that it only\nuses opam for version resolution, instead using Nix to provide\ndependencies, which provides reproducibility with pinned Nix derivation\ninputs and builds in isolation by default.</span></p>\n<h2>Conclusion</h2>\n<p><span>To summarise, this project was motivated\n(<a href=\"#introduction\">§</a>) by deploying unikernels on NixOS (<a href=\"#deploying-unikernels\">§</a>). Towards this end, we added support\nfor building MirageOS unikernels with Nix; we extended <code>opam-nix</code> to support the <code>opam-monorepo</code> workflow and created the Hillingar\nproject to provide a usable Nix interface (<a href=\"#building-unikernels\">§</a>). This required scrutinising the OCaml\nand Nix ecosystems along the way in order to marry them; some thoughts\non dependency management were developed in this context (<a href=\"#dependency-management\">§</a>). Many strange issues and edge cases\nwere uncovered during this project but now that we’ve encoded them in\nNix, hopefully, others won’t have to repeat the experience!</span></p>\n<p><span>While only the first was the primary\nmotivation, the benefits of building unikernels with Nix are:</span></p>\n<ul>\n<li>Reproducible and low-config unikernel deployment using NixOS modules\nis enabled.</li>\n<li>Nix allows reproducible builds pinning system dependencies and\ncomposing multiple language environments. For example, the OCaml package\n<code>conf-gmp</code> is a ‘virtual package’ that\nrelies on a system installation of the C/Assembly library <code>gmp</code> (The GNU Multiple Precision Arithmetic\nLibrary). Nix easily allows us to depend on this package in a\nreproducible way.</li>\n<li>We can use Nix to support building on different systems (<a href=\"#cross-compilation\">§</a>).</li>\n</ul>\n<p><span>While NixOS and MirageOS take\nfundamentally very different approaches, they’re both trying to bring\nsome kind of functional programming paradigm to operating systems. NixOS\ndoes this in a top-down manner, trying to tame Unix with functional\nprinciples like laziness and immutability<a href=\"#fn36\">36</a>;\nwhereas, MirageOS does this by throwing Unix out the window and\nrebuilding the world from scratch in a very much bottom-up approach.\nDespite these two projects having different motivations and goals,\nHillingar aims to get the best from both worlds by marrying the\ntwo.</span></p>\n\n\n<p><span>I want to thank some people for their\nhelp with this project:</span></p>\n<ul>\n<li>Lucas Pluvinage for invaluable help with the OCaml ecosystem.</li>\n<li>Alexander Bantyev for getting me up to speed with the <code>opam-nix</code> project and working with me on the\n<code>opam-monorepo</code> workflow integration.</li>\n<li>David Allsopp for his opam expertise.</li>\n<li>Jules Aguillon and Olivier Nicole for their fellow\nNix-enthusiasm.</li>\n<li>Sonja Heinze for her PPX insights.</li>\n<li>Anil Madhavapeddy for having a discussion that led to the idea for\nthis project.</li>\n<li>Björg Bjarnadóttir for her Icelandic language consultation.</li>\n<li>And finally, everyone at Tarides for being so welcoming and\nhelpful!</li>\n</ul>\n<p><span>This work was completed with the support\nof <a href=\"https://tarides.com/\">Tarides</a>, and a version of this\nblog post can be found <a href=\"https://tarides.com/blog/2022-12-14-hillingar-mirageos-unikernels-on-nixos\">on\nthe Tarides website</a>.</span></p>\n<p><span>If you have any questions or comments on\nthis feel free to <a href=\"about.html#contact\">get in\ntouch</a>.</span></p>\n<p><span>If you have a unikernel, consider trying\nto build it with Hillingar, and please report any problems at <a href=\"https://github.com/RyanGibb/hillingar/issues\">github.com/RyanGibb/hillingar/issues</a>!</span></p>\n\n\n<h2>References</h2>\n<p><span><span></span></span></p>\n<div>\n<div>\n<span><div>[1] </div><div>W. H. Lehn, <span>“The <span>Novaya\nZemlya</span> effect: <span>An</span> arctic mirage,”</span> <em>J. Opt.\nSoc. Am., JOSA</em>, vol. 69, no. 5, pp. 776–781, May 1979, doi: <a href=\"https://doi.org/10.1364/JOSA.69.000776\">10.1364/JOSA.69.000776</a>.\n[Online]. Available: <a href=\"https://opg.optica.org/josa/abstract.cfm?uri=josa-69-5-776\">https://opg.optica.org/josa/abstract.cfm?uri=josa-69-5-776</a>.\n[Accessed: Oct. 05, 2022]</div></span>\n</div>\n<div>\n<span><div>[2] </div><div>R. T. Gibb, <span>“Spatial <span>Name\nSystem</span>,”</span> Nov. 30, 2022. [Online]. Available: <a href=\"http://arxiv.org/abs/2210.05036\">http://arxiv.org/abs/2210.05036</a>.\n[Accessed: Jun. 30, 2023]</div></span>\n</div>\n<div>\n<span><div>[3] </div><div>A. Madhavapeddy, A. Ho, T. Deegan, D. Scott,\nand R. Sohan, <span>“Melange: Creating a \"functional\" internet,”</span>\n<em>SIGOPS Oper. Syst. Rev.</em>, vol. 41, no. 3, pp. 101–114, Mar.\n2007, doi: <a href=\"https://doi.org/10.1145/1272998.1273009\">10.1145/1272998.1273009</a>.\n[Online]. Available: <a href=\"https://doi.org/10.1145/1272998.1273009\">https://doi.org/10.1145/1272998.1273009</a>.\n[Accessed: Feb. 10, 2022]</div></span>\n</div>\n<div>\n<span><div>[4] </div><div>A. Madhavapeddy <em>et al.</em>,\n<span>“Unikernels: Library operating systems for the cloud,”</span>\n<em>SIGARCH Comput. Archit. News</em>, vol. 41, no. 1, pp. 461–472, Mar.\n2013, doi: <a href=\"https://doi.org/10.1145/2490301.2451167\">10.1145/2490301.2451167</a>.\n[Online]. Available: <a href=\"https://doi.org/10.1145/2490301.2451167\">https://doi.org/10.1145/2490301.2451167</a>.\n[Accessed: Jan. 25, 2022]</div></span>\n</div>\n<div>\n<span><div>[5] </div><div>D. R. Engler, M. F. Kaashoek, and J. O’Toole,\n<span>“Exokernel: An operating system architecture for application-level\nresource management,”</span> <em>SIGOPS Oper. Syst. Rev.</em>, vol. 29,\nno. 5, pp. 251–266, Dec. 1995, doi: <a href=\"https://doi.org/10.1145/224057.224076\">10.1145/224057.224076</a>.\n[Online]. Available: <a href=\"https://doi.org/10.1145/224057.224076\">https://doi.org/10.1145/224057.224076</a>.\n[Accessed: Jan. 25, 2022]</div></span>\n</div>\n<div>\n<span><div>[6] </div><div>R. Cox, <span>“Version\n<span>SAT</span>,”</span> Dec. 13, 2016. [Online]. Available: <a href=\"https://research.swtch.com/version-sat\">https://research.swtch.com/version-sat</a>.\n[Accessed: Oct. 16, 2022]</div></span>\n</div>\n<div>\n<span><div>[7] </div><div>A. Mokhov, N. Mitchell, and S. Peyton Jones,\n<span>“Build systems à la carte,”</span> <em>Proc. ACM Program.\nLang.</em>, vol. 2, pp. 1–29, Jul. 2018, doi: <a href=\"https://doi.org/10.1145/3236774\">10.1145/3236774</a>. [Online].\nAvailable: <a href=\"https://dl.acm.org/doi/10.1145/3236774\">https://dl.acm.org/doi/10.1145/3236774</a>.\n[Accessed: Oct. 11, 2022]</div></span>\n</div>\n<div>\n<span><div>[8] </div><div>E. Dolstra, <span>“The purely functional\nsoftware deployment model,”</span> [s.n.], S.l., 2006 [Online].\nAvailable: <a href=\"https://edolstra.github.io/pubs/phd-thesis.pdf\">https://edolstra.github.io/pubs/phd-thesis.pdf</a></div></span>\n</div>\n</div>\n\n\n\n\n<ol>\n<li><p><span><a href=\"./dns-loc-rr.html\">DNS LOC</a></span><a href=\"#fnref1\">↩︎</a></p></li>\n<li><p><span><a href=\"https://www.isc.org/bind/\">ISC bind</a> has many <a href=\"https://www.cvedetails.com/product/144/ISC-Bind.html?vendor_id=64\">CVE’s</a></span><a href=\"#fnref2\">↩︎</a></p></li>\n<li><p><span><a href=\"https://mirage.io\">mirage.io</a></span><a href=\"#fnref3\">↩︎</a></p></li>\n<li><p><span>Credits to Takayuki\nImada</span><a href=\"#fnref4\">↩︎</a></p></li>\n<li><p><span>Barring the use of <a href=\"https://mirage.io/blog/modular-foreign-function-bindings\">foreign\nfunction interfaces</a> (FFIs).</span><a href=\"#fnref5\">↩︎</a></p></li>\n<li><p><span>The full module\ncan be found <a href=\"https://github.com/NixOS/nixpkgs/blob/fe76645aaf2fac3baaa2813fd0089930689c53b5/nixos/modules/services/networking/bind.nix\">here</a></span><a href=\"#fnref6\">↩︎</a></p></li>\n<li><p><span><a href=\"https://opam.ocaml.org/\">opam.ocaml.org</a></span><a href=\"#fnref7\">↩︎</a></p></li>\n<li><p><span>For <a href=\"https://github.com/mirage/mirage-www\">mirage-www</a> targetting\n<code>hvt</code>.</span><a href=\"#fnref8\">↩︎</a></p></li>\n<li><p><span><a href=\"https://0install.net\">0install.net</a></span><a href=\"#fnref9\">↩︎</a></p></li>\n<li><p><span><a href=\"https://github.com/NixOS/nixpkgs/blob/9234f5a17e1a7820b5e91ecd4ff0de449e293383/pkgs/development/ocaml-modules/\">github.com/NixOS/nixpkgs\npkgs/development/ocaml-modules</a></span><a href=\"#fnref10\">↩︎</a></p></li>\n<li><p><span>NB they are not\nas complete nor up-to-date as those in <code>opam-repository</code> <a href=\"https://github.com/ocaml/opam-repository\">github.com/ocaml/opam-repository</a>.</span><a href=\"#fnref11\">↩︎</a></p></li>\n<li><p><span>Bar some\nexceptional packages that have multiple major versions packaged, like\nPostgres.</span><a href=\"#fnref12\">↩︎</a></p></li>\n<li><p><span>In fact Arch has\nthe same approach, which is why it <a href=\"nix.html#nixos\">doesn’t\nsupport partial upgrades</a>.</span><a href=\"#fnref13\">↩︎</a></p></li>\n<li><p><span>This has led to\nmuch confusion with how to install a specific version of a package <a href=\"https://github.com/NixOS/nixpkgs/issues/9682\">github.com/NixOS/nixpkgs/issues/9682</a>.</span><a href=\"#fnref14\">↩︎</a></p></li>\n<li><p><span><a href=\"https://github.com/tweag/opam-nix\">github.com/tweag/opam-nix</a></span><a href=\"#fnref15\">↩︎</a></p></li>\n<li><p><span>Another project,\n<a href=\"https://github.com/timbertson/opam2nix\">timbertson/opam2nix</a>,\nalso exists but depends on a binary of itself at build time as it’s\nwritten in OCaml as opposed to Nix, is not as minimal (higher LOC\ncount), and it isn’t under active development (with development focused\non <a href=\"https://github.com/timbertson/fetlock\">github.com/timbertson/fetlock</a>)</span><a href=\"#fnref16\">↩︎</a></p></li>\n<li><p><span>Using something\ncalled <a href=\"https://nixos.wiki/wiki/Import_From_Derivation\">Import\nFrom Derivation (IFD)</a>. Materialisation can be used to create a kind\nof lock file for this resolution, which can be committed to the project\nto avoid having to do IFD on every new build. An alternative may be to\nuse opam’s built-in version pinning[fn:47].</span><a href=\"#fnref17\">↩︎</a></p></li>\n<li><p><span><a href=\"https://github.com/Solo5/solo5\">github.com/Solo5/solo5</a></span><a href=\"#fnref18\">↩︎</a></p></li>\n<li><p><span><a href=\"https://mirage.io/blog/announcing-mirage-40\">mirage.io/blog/announcing-mirage-40</a></span><a href=\"#fnref19\">↩︎</a></p></li>\n<li><p><span><a href=\"https://dune.build\">dune.build</a></span><a href=\"#fnref20\">↩︎</a></p></li>\n<li><p><span><a href=\"https://github.com/mirage/ocaml-solo5\">github.com/mirage/ocaml-solo5</a></span><a href=\"#fnref21\">↩︎</a></p></li>\n<li><p><span><a href=\"https://github.com/tarides/opam-monorepo\">github.com/tarides/opam-monorepo</a></span><a href=\"#fnref22\">↩︎</a></p></li>\n<li><p><span><a href=\"https://github.com/tarides/opam-monorepo/blob/feeb325c9c8d560c6b92cbde62b6a9c5f20ed032/doc/faq.mld#L42\">github.com/tarides/opam-monorepo</a></span><a href=\"#fnref23\">↩︎</a></p></li>\n<li><p><span><a href=\"https://nixos.org/manual/nixpkgs/stable/#chap-cross\">nixos.org/manual/nixpkgs/stable/#chap-cross</a></span><a href=\"#fnref24\">↩︎</a></p></li>\n<li><p><span><a href=\"https://nixos.org/manual/nix/stable/advanced-topics/distributed-builds.html\">nixos.org/manual/nix/stable/advanced-topics/distributed-builds.html</a></span><a href=\"#fnref25\">↩︎</a></p></li>\n<li><p><span><a href=\"https://docs.haskellstack.org/en/stable/nix_integration/\">docs.haskellstack.org/en/stable/nix_integration</a></span><a href=\"#fnref26\">↩︎</a></p></li>\n<li><p><span><a href=\"https://github.com/nix-community/dream2nix\">github.com/nix-community/dream2nix</a></span><a href=\"#fnref27\">↩︎</a></p></li>\n<li><p><span><a href=\"https://github.com/timbertson/fetlock\">github.com/timbertson/fetlock</a></span><a href=\"#fnref28\">↩︎</a></p></li>\n<li><p><span><a href=\"https://github.com/tweag/opam-nix/blob/4e602e02a82a720c2f1d7324ea29dc9c7916a9c2/README.md#materialization\"><span>https://github.com/tweag/opam-nix#materialization</span></a></span><a href=\"#fnref29\">↩︎</a></p></li>\n<li><p><span><a href=\"https://twitter.com/TheLortex/status/1571884882363830273\">twitter.com/TheLortex/status/1571884882363830273</a></span><a href=\"#fnref30\">↩︎</a></p></li>\n<li><p><span><a href=\"https://signalsandthreads.com/build-systems/#4305\">signalsandthreads.com/build-systems/#4305</a></span><a href=\"#fnref31\">↩︎</a></p></li>\n<li><p><span><a href=\"https://www.tweag.io/blog/2020-10-22-nickel-open-sourcing/\">www.tweag.io/blog/2020-10-22-nickel-open-sourcing</a></span><a href=\"#fnref32\">↩︎</a></p></li>\n<li><p><span><a href=\"https://hannes.robur.coop/Posts/VMM\">hannes.robur.coop/Posts/VMM</a></span><a href=\"#fnref33\">↩︎</a></p></li>\n<li><p><span><a href=\"https://github.com/roburio/albatross/pull/120\">https://github.com/roburio/albatross/pull/120</a></span><a href=\"#fnref34\">↩︎</a></p></li>\n<li><p><span><a href=\"https://hannes.nqsb.io/Posts/ReproducibleOPAM\">hannes.nqsb.io/Posts/ReproducibleOPAM</a></span><a href=\"#fnref35\">↩︎</a></p></li>\n<li><p><span><a href=\"https://www.tweag.io/blog/2022-07-14-taming-unix-with-nix/\">tweag.io/blog/2022-07-14-taming-unix-with-nix</a></span><a href=\"#fnref36\">↩︎</a></p></li>\n</ol>",
9 "content_type": "html",
10 "categories": [],
11 "source": "https://ryan.freumh.org/atom.xml"
12}