Self-host your own digital island

add docs

+9 -32
README.md
···
-
Warning: work in progress.
-
# Eilean
+
# Eilean 🏝️
-
Create your own digital island powered by Nix.
+
Eilean enables you to host your own digital 'island', where you control you're own online infrastructure.
+
The use of federated protocols allows you to 'bridge' your island to others.
-
Self hosting is hard.
-
There's a lot of complexity to manage.
+
Eilean uses [NixOS](https://nixos.org/) to enable reproducible deployments of services such as webservers, mailservers, federated communication servers, Virtual Private Network servers, and more.
+
However, they still require a lot of manual configuration for domain names, DNS records, user accounts, databases, HTTP proxies, TLS certificates, and more.
-
While NixOS enables reproducible deployments of many services, they still require much manual configuration.
-
Adding domain names, DNS records, user accounts, databases, HTTP proxies, SMTP servers all add additional complexity.
-
-
Eilean aims to be a personal or community 'island in a box' that requires minimal configuration.
-
It can achieve this by sharing configration between many NixOS modules (each for an individuial service).
+
Eilean aims to be a optioned framework to allow the simple deployment of these services on a single machine, and a library of documentation for common issues in managing runtime state like secrets, databases, and upgrades.
-
### Usage
+
By using Nix Eilean modules are extensive to other configurations outside this deployment scenario, such as offloading a particularly resource heavy service to a dedicated machine.
-
Some familiarity with Nix is required.
+
Contributions for additional services welcome.
-
TODO...
-
-
##### Networking
-
-
Requires a public IPv4 and IPv6 address.
-
-
##### DNS
-
-
Hosting DNS allows necessary records to be specified decleratively and enabled when the module that required then is.
-
You will need to point your domain's NS record to your IP address with your registar using a glue record.
-
-
need to update SOA serial No
-
-
##### Email
-
-
Hosting email allows for an easy, and cheap, SMTP server for services that require it.
-
Recieving EMail shouldn't pose an issues.
-
Sending email to users on your own domain shouldn't pose any issues, if for example users are signing up to services like Mastodon using an EMail account on the same Eilean.
-
Sending mail will require TCP port 25 to be unblocked by your network provider, and your IP address to not be blacklisted (e.g. check [here](https://mxtoolbox.com/blacklists.aspx)).
-
+
For instructions on getting started see [docs/getting_started.md](./docs/getting_started.md).
+9
docs/adding_eilean.md
···
+
+
# Adding Eilean to an Existing NixOS System
+
+
If you already have a NixOS system and want to use Eilean you can add to your configuration.
+
Note this requires a flake-enabled system.
+
+
Add `github:RyanGibb/eilean-nix` as an input to your flake, and import `eilean.nixosModules.default`.
+
You should then be able to use the configuration options in `config.eilean`.
+
See [../template/flake.nix](../template/flake.nix) for an example.
+199
docs/getting_started.md
···
+
+
# Setup
+
+
This guide walks a user through the first time setup of a server running Eilean.
+
If you already have a NixOS system, please feel look at [adding eilean](adding_eilean.md) to an existing system.
+
Some familiarity with networking, operating systems, and Linux is necessary.
+
Some familiarity with Nix is beneficial.
+
+
## Step 1: Find a server
+
+
- Option 1 (recommended): a Virtual Private Server (VPS) with a cloud provider such as Hetzner, Vultr, or Digital Ocean. Get an IPv4 address^[1].
+
You can use this referral link to get started on Hetzner: https://hetzner.cloud/?ref=XydbkWdf49TY.
+
+
- Option 2: your own hardware, such as an old PC or laptop, Raspberry Pi, or a custom-build server.
+
Note you'll need a static IPv4^[1] address for reliable hosting^[2]. If you're behind Network Address Translation (NAT) you'll need to set up port forwarding for every service you want to run.
+
+
The resource requirements depend on the number of services you want to run and
+
resource requirements
+
under 10G disk space with no services...
+
with all enabled...
+
+
+
[1]: You could just use an IPv6 address, but much of the Internet is still [IPv4-only](https://stats.labs.apnic.net/ipv6).
+
+
[2]: If you don't have a static address, Dynamic DNS is possible but takes some time to propagate. Email reputation is tied to your IP address; using a residential address assigned by your ISP may get your mail blocked.
+
+
Resource requirements depends how many service you want to run and how much load they'll be under, but 2 GB RAM and 20 GB disk should be a good starting point.
+
+
## Step 2: Install NixOS with Eilean
+
+
Most service providers don't offer a NixOS image, so we'll install it manually.
+
+
- Create the server with a generic linux distribution such as Debian.
+
- Mount the NixOS ISO, either from your provider directly, or by uploading it yourself. For Herzner:
+
- Create a new instance and power it off
+
- Switch to the ISO-Images tab and mount the NixOS minimal ISO
+
- Open the remote console (>_ button) and power the machine on
+
- Follow the usual installation guide
+
- Unmount the ISO and reboot
+
- Install NixOS.
+
+
The [official manual](https://nixos.org/manual/nixos/stable/index.html#sec-installation-manual) contains detailed instructions, but the minimum to get your disk partioned is:
+
```
+
DISK=/dev/sda
+
+
parted $DISK -- mklabel gpt
+
parted $DISK -- mkpart ESP fat32 1MB 512MB
+
parted $DISK -- mkpart primary 512MiB 100%
+
parted $DISK -- set 1 esp on
+
mkfs.fat -F 32 -n boot ${DISK}1
+
mkfs.ext4 -L nixos ${DISK}2
+
```
+
+
We can then mount the primary and boot partions and generate a configuration for you (possible virtualised) hardware:
+
```
+
mount /dev/disk/by-label/nixos /mnt
+
mkdir /mnt/boot
+
mount /dev/disk/by-label/boot /mnt/boot
+
nixos-generate-config --root /mnt
+
```
+
+
It's possible to install Nix with the default configuration at this point.
+
However, Eilean comes with a simple template that's recommended to get you started:
+
```
+
cd /mnt/etc/nixos
+
rm configuration.nix
+
nix flake init -t github:RyanGibb/eilean-nix
+
```
+
+
Note that we are using the `hardware-configuration.nix` generated by `nixos-generate-config`.
+
+
Eilean uses [flakes](https://www.tweag.io/blog/2020-05-25-flakes/).
+
Without going into too much depth, they enable hermetic evaluation of Nix expressions and provide a standard way to compose Nix projects^[3].
+
+
[3]: [tweag.io/blog/2020-05-25-flakes](https://www.tweag.io/blog/2020-05-25-flakes/).
+
+
You can edit the resulting `configuration.nix`.
+
Check out the `TODO`'s for a place to start.
+
The website [search.nixos.org](https://search.nixos.org/) is a great place to find information on configuration options and packages available, and with `man 5 configuration.nix` you can see the configuration options locally.
+
One thing you should do at this point is generate a password hash with `mkpasswd` and add it to `root.initialHashedPassword`.
+
+
Now, we can install NixOS and reboot:
+
```
+
nixos-install --root /mnt/ --flake /mnt/etc/nixos#eilean --no-root-passwd
+
reboot
+
```
+
+
Upon boot you should be able to login as root.
+
You may need to run `passwd <username>` (where `<username>` is `eilean` by default) to be able to log in as `<username>`^[4].
+
You should be able to edit `/etc/nixos/configuration.nix` and rebuild you system with `sudo nixos-rebuild switch`.
+
+
[4]: [github.com/NixOS/nixpkgs/issues/55424](https://github.com/NixOS/nixpkgs/issues/55424)
+
+
By default DHCP will be enabled so your machine will discovery it's IP address, however some providers don't enable DHCPv6 or SLAAC so you need to manually configure the IP address.
+
For example a Hetzner VPS IPv6 address can be found in the networking tab and enabled with:
+
```
+
networking = {
+
interfaces."enp1s0" = {
+
ipv6.addresses = [{
+
address = "<address>";
+
prefixLength = 64;
+
}];
+
};
+
defaultGateway6 = {
+
address = "fe80::1";
+
interface = "enp1s0";
+
};
+
};
+
```
+
+
Replace `enp1s0` with your network interface (try `ip addr`).
+
+
It's recommended to track `/etc/nixos` in a git repository.
+
Note that untracked files aren't seen by Nix flakes, so `git add` any new files you create to use them in Nix.
+
It may be useful to change the user and group from root so your user account can edit these files:
+
```
+
sudo chgrp -R eilean /etc/nixos
+
sudo chgrp -R users /etc/nixos
+
```
+
+
Eilean creates a set of NixOS modules under `eilean`.
+
Check out the default configuration, and files in [modules](../modules/), to see what options there are.
+
(Documentation and man pages are on the way).
+
+
## Step 3: Get a Domain
+
+
From your favourite registrar, e.g. [gandi.net](https://www.gandi.net/), purchase a domain.
+
Eilean automates Domain Name System (DNS) record creation and maintence by hosting DNS on the server and managing records decleratively.
+
+
Create a Glue record with your registrar with `ns1.<domain>` pointing to your public IP address from step 1.
+
Next, add this as an external nameserver.
+
+
If your domain TLD requires two nameservers, you can create a duplicate `ns2.<domain>`.
+
If you use a domain name other than this pattern, be sure to update `eilean.dns.nameservers`.
+
+
You may need to wait up to 24 hours for DNS records to propagate at this point.
+
+
You can check if this is working with:
+
```
+
dig <domain>
+
```
+
+
## Step 4: Configure Eilean
+
+
Once your domain is setup, replace these default values of Eilean with your IPv4 and IPv6 network addresses, and your public network interface:
+
```
+
eilean.serverIpv4 = "203.0.113.0";
+
eilean.serverIpv6 = "2001:DB8:0:0:0:0:0:0";
+
eilean.publicInterface = "enp1s0";
+
```
+
+
You should be able to get these from `ip addr`.
+
+
Set `eilean.username` to what you want your username to be on email, matrix, and any other services.
+
A first name might be a good choice.
+
+
Now, enable services at will!
+
It's a good idea to enable one service at a time initially or else if you run into issues, e.g. DNS record propitiation, then you may get rate limited by Let's Encrypt for TLS certificate provisioning.
+
+
## Further Information
+
+
### Website
+
+
TODO
+
+
### Secrets
+
+
TODO
+
+
### Backups
+
+
TODO
+
+
##### Email
+
+
Hosting email allows for an easy, and cheap, SMTP server for services that require it.
+
Receiving EMail shouldn't pose an issues.
+
Sending email to users on your own domain shouldn't pose any issues, if for example users are signing up to services like Mastodon using an EMail account on the same Eilean.
+
Sending mail will require TCP port 25 to be unblocked by your network provider, and your IP address to not be blacklisted (e.g. check [here](https://mxtoolbox.com/blacklists.aspx)).
+
+
not managed:
+
- multiple domains
+
- multiple servers /load balance
+
+
### Matrix
+
+
TODO
+
+
### Mastodon
+
+
TODO
+
+
### Gitea
+
+
TODO
+
+
### Wireguard/Headscale
+
+
TODO
+83
docs/nix.md
···
+
+
# Nix
+
+
Nix is a software deployment system that uses cryptographic hashes to compute unique paths for components (i.e., packages) that are stored in a read-only directory: the Nix store, at `/nix/store/<hash>-<name>`.
+
This provides several benefits, including concurrent installation of multiple versions of a package, atomic upgrades, and multiple user environments.
+
+
Nix uses a declarative domain-specific language (DSL), also called Nix, to build and configure software.
+
The Nix DSL is a functional language with a syntax Turing complete but lacks a type system.
+
We use the DSL to write derivations for software, which describe how to build said software with input components and a build script.
+
This Nix expression is then 'instantiated' to create 'store derivations' (`.drv` files), which is the low-level representation of how to build a single component.
+
This store derivation is 'realised' into a built artefact, hereafter referred to as 'building'.
+
+
Possibly the simplest Nix derivation uses `bash` to create a single file containing `Hello, World!`:
+
```nix
+
{ pkgs ? import <nixpkgs> { } }:
+
+
builtins.derivation {
+
name = "hello";
+
system = builtins.currentSystem;
+
builder = "${nixpkgs.bash}/bin/bash";
+
args = [ "-c" ''echo "Hello, World!" > $out'' ];
+
}
+
```
+
Note that `derivation` is a function that we're calling with one argument, which is a set of attributes.
+
+
We can instantiate this Nix derivation to create a store derivation:
+
```
+
$ nix-instantiate default.nix
+
/nix/store/5d4il3h1q4cw08l6fnk4j04a19dsv71k-hello.drv
+
$ nix show-derivation /nix/store/5d4il3h1q4cw08l6fnk4j04a19dsv71k-hello.drv
+
{
+
"/nix/store/5d4il3h1q4cw08l6fnk4j04a19dsv71k-hello.drv": {
+
"outputs": {
+
"out": {
+
"path": "/nix/store/4v1dx6qaamakjy5jzii6lcmfiks57mhl-hello"
+
}
+
},
+
"inputSrcs": [],
+
"inputDrvs": {
+
"/nix/store/mnyhjzyk43raa3f44pn77aif738prd2m-bash-5.1-p16.drv": [
+
"out"
+
]
+
},
+
"system": "x86_64-linux",
+
"builder": "/nix/store/2r9n7fz1rxq088j6mi5s7izxdria6d5f-bash-5.1-p16/bin/bash",
+
"args": [ "-c", "echo \"Hello, World!\" > $out" ],
+
"env": {
+
"builder": "/nix/store/2r9n7fz1rxq088j6mi5s7izxdria6d5f-bash-5.1-p16/bin/bash",
+
"name": "hello",
+
"out": "/nix/store/4v1dx6qaamakjy5jzii6lcmfiks57mhl-hello",
+
"system": "x86_64-linux"
+
}
+
}
+
}
+
```
+
+
And build the store derivation:
+
```sh
+
$ nix-store --realise /nix/store/5d4il3h1q4cw08l6fnk4j04a19dsv71k-hello.drv
+
/nix/store/4v1dx6qaamakjy5jzii6lcmfiks57mhl-hello
+
$ cat /nix/store/4v1dx6qaamakjy5jzii6lcmfiks57mhl-hello
+
Hello, World!
+
```
+
+
Most Nix tooling does these two steps together:
+
```
+
nix-build default.nix
+
this derivation will be built:
+
/nix/store/q5hg3vqby8a9c8pchhjal3la9n7g1m0z-hello.drv
+
building '/nix/store/q5hg3vqby8a9c8pchhjal3la9n7g1m0z-hello.drv'...
+
/nix/store/zyrki2hd49am36jwcyjh3xvxvn5j5wml-hello
+
```
+
+
Nix realisations (hereafter referred to as 'builds') are done in isolation to ensure reproducibility.
+
Projects often rely on interacting with package managers to make sure all dependencies are available and may implicitly rely on system configuration at build time.
+
To prevent this, every Nix derivation is built in isolation (without network access or access to the global file system) with only other Nix derivations as inputs.
+
+
> The name Nix is derived from the Dutch word *niks*, meaning nothing; build actions do not see anything that has not been explicitly declared as an input.
+
+
For more information, see:
+
- The Nix paper: https://edolstra.github.io/pubs/nspfssd-lisa2004-final.pdf
+
- The Nix PhD thesis: https://edolstra.github.io/pubs/phd-thesis.pdf
+
+19
docs/nixos.md
···
+
+
# NixOS
+
+
[NixOS](nixos.org) is a Linux distribution built with [Nix](./nix.md) from a modular, purely functional specification.
+
It has no traditional filesystem hierarchy (FSH), like `/bin`, `/lib`, `/usr`, but instead stores all components in `/nix/store`.
+
The system configuration is managed by Nix and configured with Nix expressions.
+
[NixOS modules](https://nixos.org/manual/nixos/stable/index.html#sec-writing-modules) are Nix files containing chunks of system configuration that can be composed to build a full NixOS system.
+
While many NixOS modules are provided in the [Nixpkgs](./nixpkgs.md) repository, they can also be written by an individual user.
+
For example, the expression used to deploy a DNS server is a NixOS module.
+
Together these modules form the configuration which builds the Linux system as a Nix derivation.
+
+
NixOS minimises global mutable state that -- without knowing it -- you might rely on being set up in a certain way.
+
For example, you might follow instructions to run a series of shell commands and edit some files to get a piece of software working.
+
You may subsequently be unable to reproduce the result because you've forgotten some intricacy or are now using a different version of the software.
+
Nix forces you to encode this in a reproducible way, which is extremely useful for replicating software configurations and deployments, aiming to solve the 'It works on my machine' problem.
+
Docker is often used to fix this configuration problem, but Nix aims to be more reproducible.
+
+
Nix provides safe and reliable atomic upgrades and rollbacks.
+
And every new system configuration build creates a GRUB entry, so you can boot previous systems even from your UEFI/BIOS.
+19
docs/nixpkgs.md
···
+
+
### Nixpkgs
+
+
Nixpkgs^[ [github.com/nixos/nixpkgs](https://github.com/nixos/nixpkgs) ] is a large repository of software packaged in [Nix](./nix.md), where every package is a Nix derivation.
+
It also stores all the default [./NixOS](./nixos.md) modules.
+
+
There is also a command line package manager that installs packages from Nixpkgs, which is why people sometimes refer to Nix as a package manager.
+
+
While Nix, and therefore Nix package management, is primarily source-based (since derivations describe how to build software from source), binary deployment is an optimisation of this.
+
Since packages are built in isolation and entirely determined by their inputs, binaries can be transparently deployed by downloading them from a remote server instead of building the derivation locally.
+
+
Nix supports atomic upgrades and rollbacks.
+
The pointers to the new packages are only updated when the install succeeds.
+
+
Due to every
+
+
+
While Nixpkgs also has one global coherent package set, one can use multiple instances of Nixpkgs (i.e., channels) at once to support partial upgrades, as the Nix store allows multiple versions of a dependency to be stored.
+
This also supports atomic upgrades, as all the software's old versions can be kept until garbage collection.
+9 -7
template/configuration.nix
···
eilean = {
# TODO replace these values
-
username = "user";
-
secretsDir = "/secrets";
-
serverIpv4 = "203.0.113.0";
-
serverIpv6 = "2001:DB8:0:0:0:0:0:0";
-
publicInterface = "enp1s0";
+
# serverIpv4 = "203.0.113.0";
+
# serverIpv6 = "2001:DB8:0:0:0:0:0:0";
+
# publicInterface = "enp1s0";
+
+
# TODO replace with your desired username
+
# username = "user";
# TODO enable desired services
-
dns.enable = true;
# mailserver.enable = true;
# matrix.enable = true;
# mastodon.enable = true;
# gitea.enable = true;
-
# dns.enable = true;
+
# headscale.enable = true;
+
+
# secretsDir = "/secrets";
};
# This value determines the NixOS release from which the default