Thicket data repository for the EEG
anil.recoil.org
1{
2 "id": "https://www.tunbury.org/2025/04/16/ubuntu-cloud-init",
3 "title": "Ubuntu cloud-init",
4 "link": "https://www.tunbury.org/2025/04/16/ubuntu-cloud-init/",
5 "updated": "2025-04-16T00:00:00",
6 "published": "2025-04-16T00:00:00",
7 "summary": "Testing cloud-init is painful on real (server) hardware, as the faster the server, the longer it seems to take to complete POST. Therefore, I highly recommend testing with a virtual machine before moving to real hardware.",
8 "content": "<p>Testing cloud-init is painful on real (server) hardware, as the faster the server, the longer it seems to take to complete POST. Therefore, I highly recommend testing with a virtual machine before moving to real hardware.</p>\n\n<p>I have set up a QEMU machine to simulate the Dell R640 machines with 10 x 8T disks. I’ll need to set up and tear this machine down several times for testing, so I have wrapped the setup commands into a <code>Makefile</code>. QCOW2 is a thin format, so you don’t actually need 80T of disk space to do this!</p>\n\n<p>The Dell machines use EFI, so I have used EFI on the QEMU machine. Note the <code>OVMF</code> lines in the configuration. Ensure that you emulate a hard disk controller, which is supported by the EFI BIOS. For example, <code>-device megasas,id=scsi0</code> won’t boot as the EFI BIOS can’t see the drives. I have enabled VNC access, but I primarily used the serial console to interact with the machine.</p>\n\n<div><div><pre><code>machine: disk0.qcow2 disk1.qcow2 disk2.qcow2 disk3.qcow2 disk4.qcow2 disk5.qcow2 disk6.qcow2 disk7.qcow2 disk8.qcow2 disk9.qcow2 OVMF_VARS.fd\n\tqemu-system-x86_64 -m 8G -smp 4 -machine accel=kvm,type=pc -cpu host -display none -vnc :0 \\\n\t\t-drive if=pflash,format=raw,readonly=on,file=/usr/share/OVMF/OVMF_CODE.fd \\\n\t\t-drive if=pflash,format=raw,file=OVMF_VARS.fd \\\n\t\t-serial stdio \\\n\t\t-device virtio-scsi-pci,id=scsi0 \\\n\t\t-device scsi-hd,drive=drive0,bus=scsi0.0,channel=0,scsi-id=0,lun=0 \\\n\t\t-drive file=disk0.qcow2,if=none,id=drive0 \\\n\t\t-device scsi-hd,drive=drive1,bus=scsi0.0,channel=0,scsi-id=1,lun=0 \\\n\t\t-drive file=disk1.qcow2,if=none,id=drive1 \\\n\t\t-device scsi-hd,drive=drive2,bus=scsi0.0,channel=0,scsi-id=2,lun=0 \\\n\t\t-drive file=disk2.qcow2,if=none,id=drive2 \\\n\t\t-device scsi-hd,drive=drive3,bus=scsi0.0,channel=0,scsi-id=3,lun=0 \\\n\t\t-drive file=disk3.qcow2,if=none,id=drive3 \\\n\t\t-device scsi-hd,drive=drive4,bus=scsi0.0,channel=0,scsi-id=4,lun=0 \\\n\t\t-drive file=disk4.qcow2,if=none,id=drive4 \\\n\t\t-device scsi-hd,drive=drive5,bus=scsi0.0,channel=0,scsi-id=5,lun=0 \\\n\t\t-drive file=disk5.qcow2,if=none,id=drive5 \\\n\t\t-device scsi-hd,drive=drive6,bus=scsi0.0,channel=0,scsi-id=6,lun=0 \\\n\t\t-drive file=disk6.qcow2,if=none,id=drive6 \\\n\t\t-device scsi-hd,drive=drive7,bus=scsi0.0,channel=0,scsi-id=7,lun=0 \\\n\t\t-drive file=disk7.qcow2,if=none,id=drive7 \\\n\t\t-device scsi-hd,drive=drive8,bus=scsi0.0,channel=0,scsi-id=8,lun=0 \\\n\t\t-drive file=disk8.qcow2,if=none,id=drive8 \\\n\t\t-device scsi-hd,drive=drive9,bus=scsi0.0,channel=0,scsi-id=9,lun=0 \\\n\t\t-drive file=disk9.qcow2,if=none,id=drive9 \\\n\t\t-net nic,model=virtio-net-pci,macaddr=02:00:00:00:00:01 \\\n\t\t-net bridge,br=br0\n\ndisk%.qcow2:\n\tqemu-img create -f qcow2 $@ 8T\n\nOVMF_VARS.fd:\n\tcp /usr/share/OVMF/OVMF_VARS.fd OVMF_VARS.fd\n\nclean:\n\trm *.qcow2 OVMF_VARS.fd\n</code></pre></div></div>\n\n<p>We are using <a href=\"https://netboot.xyz\">netboot.xyz</a> to network boot the machine via PXE. The easiest way to use netboot.xyz is to use it within the prebuilt Docker container. This can be set up using a <code>docker-compose.yml</code> file. Start the container with <code>docker compose up -d</code>.</p>\n\n<div><div><pre><code>version: \"2.1\"\nservices:\n netbootxyz:\n image: ghcr.io/netbootxyz/netbootxyz\n container_name: netbootxyz\n environment:\n - NGINX_PORT=80 # optional\n - WEB_APP_PORT=3000 # optional\n volumes:\n - /netbootxyz/config:/config # optional\n - /netbootxyz/assets:/assets # optional\n ports:\n - 3000:3000 # optional, destination should match ${WEB_APP_PORT} variable above.\n - 69:69/udp\n - 8080:80 # optional, destination should match ${NGINX_PORT} variable above.\n restart: unless-stopped\n</code></pre></div></div>\n\n<p>We have a Ubiquiti EdgeMax providing DHCP services. The DHCP options should point new clients to the Docker container.</p>\n\n<div><div><pre><code>set service dhcp-serverbootfile-server doc.caelum.ci.dev\nset service dhcp-server global-parameters \"class &quot;BIOS-x86&quot; { match if option arch = 00:00; filename &quot;netboot.xyz.kpxe&quot;; }\"\nset service dhcp-server global-parameters \"class &quot;UEFI-x64&quot; { match if option arch = 00:09; filename &quot;netboot.xyz.efi&quot;; }\"\nset service dhcp-server global-parameters \"class &quot;UEFI-bytecode&quot; { match if option arch = 00:07; filename &quot;netboot.xyz.efi&quot;; }\"\n</code></pre></div></div>\n\n<p>I also recommend staging the Ubuntu installation ISO, <code>vmlinuz</code>, and <code>initrd</code> locally, as this will speed up the machine’s boot time. The files needed are:</p>\n\n<ul>\n <li>https://releases.ubuntu.com/24.04.2/ubuntu-24.04.2-live-server-amd64.iso</li>\n <li>https://github.com/netbootxyz/ubuntu-squash/releases/download/24.04.2-dac09526/vmlinuz</li>\n <li>https://github.com/netbootxyz/ubuntu-squash/releases/download/24.04.2-dac09526/initrd</li>\n</ul>\n\n<p>Create a <code>user-data</code> file containing the following cloud-init configuration. In this case, it primarily includes the storage configuration. The goal here is to configure each disk identically, with a tiny EFI partition, an MD RAID partition and a rest given over to the ZFS datastore. Additionally, create empty files <code>meta-data</code> and <code>vendor-data</code>. None of the files have an extension. The encrypted password is <code>ubuntu</code>.</p>\n\n<div><div><pre><code>#cloud-config\nautoinstall:\n version: 1\n storage:\n config:\n - { ptable: gpt, path: /dev/sda, preserve: false, name: '', grub_device: false, id: disk-sda, type: disk }\n - { ptable: gpt, path: /dev/sdb, wipe: superblock-recursive, preserve: false, name: '', grub_device: false, id: disk-sdb, type: disk }\n - { ptable: gpt, path: /dev/sdc, wipe: superblock-recursive, preserve: false, name: '', grub_device: false, id: disk-sdc, type: disk }\n - { ptable: gpt, path: /dev/sdd, wipe: superblock-recursive, preserve: false, name: '', grub_device: false, id: disk-sdd, type: disk }\n - { ptable: gpt, path: /dev/sde, wipe: superblock-recursive, preserve: false, name: '', grub_device: false, id: disk-sde, type: disk }\n - { ptable: gpt, path: /dev/sdf, wipe: superblock-recursive, preserve: false, name: '', grub_device: false, id: disk-sdf, type: disk }\n - { ptable: gpt, path: /dev/sdg, wipe: superblock-recursive, preserve: false, name: '', grub_device: false, id: disk-sdg, type: disk }\n - { ptable: gpt, path: /dev/sdh, wipe: superblock-recursive, preserve: false, name: '', grub_device: false, id: disk-sdh, type: disk }\n - { ptable: gpt, path: /dev/sdi, wipe: superblock-recursive, preserve: false, name: '', grub_device: false, id: disk-sdi, type: disk }\n - { ptable: gpt, path: /dev/sdj, wipe: superblock-recursive, preserve: false, name: '', grub_device: false, id: disk-sdj, type: disk }\n - { device: disk-sda, size: 512M, wipe: superblock, flag: boot, number: 1, preserve: false, grub_device: true, offset: 1048576, id: efi-0, type: partition }\n - { device: disk-sdb, size: 512M, wipe: superblock, flag: boot, number: 1, preserve: false, grub_device: true, offset: 1048576, id: efi-1, type: partition }\n - { device: disk-sdc, size: 512M, wipe: superblock, flag: boot, number: 1, preserve: false, grub_device: false, offset: 1048576, id: efi-2, type: partition }\n - { device: disk-sdd, size: 512M, wipe: superblock, flag: boot, number: 1, preserve: false, grub_device: false, offset: 1048576, id: efi-3, type: partition }\n - { device: disk-sde, size: 512M, wipe: superblock, flag: boot, number: 1, preserve: false, grub_device: false, offset: 1048576, id: efi-4, type: partition }\n - { device: disk-sdf, size: 512M, wipe: superblock, flag: boot, number: 1, preserve: false, grub_device: false, offset: 1048576, id: efi-5, type: partition }\n - { device: disk-sdg, size: 512M, wipe: superblock, flag: boot, number: 1, preserve: false, grub_device: false, offset: 1048576, id: efi-6, type: partition }\n - { device: disk-sdh, size: 512M, wipe: superblock, flag: boot, number: 1, preserve: false, grub_device: false, offset: 1048576, id: efi-7, type: partition }\n - { device: disk-sdi, size: 512M, wipe: superblock, flag: boot, number: 1, preserve: false, grub_device: false, offset: 1048576, id: efi-8, type: partition }\n - { device: disk-sdj, size: 512M, wipe: superblock, flag: boot, number: 1, preserve: false, grub_device: false, offset: 1048576, id: efi-9, type: partition }\n - { device: disk-sda, size: 16G, wipe: superblock, number: 2, preserve: false, grub_device: false, id: md-0, type: partition }\n - { device: disk-sdb, size: 16G, wipe: superblock, number: 2, preserve: false, grub_device: false, id: md-1, type: partition }\n - { device: disk-sdc, size: 16G, wipe: superblock, number: 2, preserve: false, grub_device: false, id: md-2, type: partition }\n - { device: disk-sdd, size: 16G, wipe: superblock, number: 2, preserve: false, grub_device: false, id: md-3, type: partition }\n - { device: disk-sde, size: 16G, wipe: superblock, number: 2, preserve: false, grub_device: false, id: md-4, type: partition }\n - { device: disk-sdf, size: 16G, wipe: superblock, number: 2, preserve: false, grub_device: false, id: md-5, type: partition }\n - { device: disk-sdg, size: 16G, wipe: superblock, number: 2, preserve: false, grub_device: false, id: md-6, type: partition }\n - { device: disk-sdh, size: 16G, wipe: superblock, number: 2, preserve: false, grub_device: false, id: md-7, type: partition }\n - { device: disk-sdi, size: 16G, wipe: superblock, number: 2, preserve: false, grub_device: false, id: md-8, type: partition }\n - { device: disk-sdj, size: 16G, wipe: superblock, number: 2, preserve: false, grub_device: false, id: md-9, type: partition }\n - { device: disk-sda, size: -1, wipe: superblock, number: 3, preserve: false, grub_device: false, id: zfs-0, type: partition }\n - { device: disk-sdb, size: -1, wipe: superblock, number: 3, preserve: false, grub_device: false, id: zfs-1, type: partition }\n - { device: disk-sdc, size: -1, wipe: superblock, number: 3, preserve: false, grub_device: false, id: zfs-2, type: partition }\n - { device: disk-sdd, size: -1, wipe: superblock, number: 3, preserve: false, grub_device: false, id: zfs-3, type: partition }\n - { device: disk-sde, size: -1, wipe: superblock, number: 3, preserve: false, grub_device: false, id: zfs-4, type: partition }\n - { device: disk-sdf, size: -1, wipe: superblock, number: 3, preserve: false, grub_device: false, id: zfs-5, type: partition }\n - { device: disk-sdg, size: -1, wipe: superblock, number: 3, preserve: false, grub_device: false, id: zfs-6, type: partition }\n - { device: disk-sdh, size: -1, wipe: superblock, number: 3, preserve: false, grub_device: false, id: zfs-7, type: partition }\n - { device: disk-sdi, size: -1, wipe: superblock, number: 3, preserve: false, grub_device: false, id: zfs-8, type: partition }\n - { device: disk-sdj, size: -1, wipe: superblock, number: 3, preserve: false, grub_device: false, id: zfs-9, type: partition }\n - { name: md0, raidlevel: raid5, devices: [ md-0, md-1, md-2, md-3, md-4, md-5, md-6, md-7, md-8, md-9 ], spare_devices: [], preserve: false, wipe: superblock, id: raid-0, type: raid }\n - { fstype: fat32, volume: efi-0, preserve: false, id: efi-dos-0, type: format }\n - { fstype: fat32, volume: efi-1, preserve: false, id: efi-dos-1, type: format }\n - { fstype: ext4, volume: raid-0, preserve: false, id: root-ext4, type: format }\n - { path: /, device: root-ext4, id: mount-2, type: mount }\n - { path: /boot/efi, device: efi-dos-0, id: mount-0, type: mount }\n - { path: /boot/efi-alt, device: efi-dos-1, id: mount-1, type: mount }\n identity:\n hostname: ubuntu-server\n password: \"$6$exDY1mhS4KUYCE/2$zmn9ToZwTKLhCw.b4/b.ZRTIZM30JZ4QrOQ2aOXJ8yk96xpcCof0kxKwuX1kqLG/ygbJ1f8wxED22bTL4F46P0\"\n username: ubuntu\n ssh:\n install-server: yes\n authorized-keys:\n - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIA7UrJmBFWR3c7jVzpoyg4dJjON9c7t9bT9acfrj6G7i\n allow-pw: no\n packages:\n - zfsutils-linux\n user-data:\n disable_root: false\n</code></pre></div></div>\n\n<p>The binaries and configuration files should be stored in the assets folder used by netbootxyz.</p>\n\n<div><div><pre><code>/netbootxyz/assets/r640/initrd\n/netbootxyz/assets/r640/meta-data\n/netbootxyz/assets/r640/ubuntu-24.04.2-live-server-amd64.iso\n/netbootxyz/assets/r640/user-data\n/netbootxyz/assets/r640/vendor-data\n/netbootxyz/assets/r640/vmlinuz\n</code></pre></div></div>\n\n<p>The kernel command line used for iPXE needs to include <code>autoinstall</code> and <code>ds=nocloud;s=http://your_server</code>. We could modify one of the existing <code>ipxe</code> scripts to do this, but it is more flexible to create <code>/netbootxyz/config/menus/MAC-020000000001.ipxe</code> where <code>020000000001</code> represents the MAC address <code>02:00:00:00:00:01</code> and should be updated to reflect the actual server’s MAC address.</p>\n\n<div><div><pre><code>#!ipxe\n\n# Set a timeout (in milliseconds) for automatic selection\nset timeout 30000\n\n# Define a title for the menu\n:start\nmenu Boot Menu\nitem --key 1 local Boot from local hdd\nitem --key 2 ubuntu Autoinstall Ubuntu Noble\nitem --key r reboot Reboot system\nitem --key x exit Exit to iPXE shell\nchoose --timeout ${timeout} --default local option && goto ${option}\n\n# boot local system\n:local\necho Booting from local disks ...\nexit 1\n\n# Ubuntu boot configuration\n:ubuntu\nimgfree\necho Autoinstall Ubuntu Noble...\nset base-url http://doc.caelum.ci.dev:8080/r640\nkernel ${base-url}/vmlinuz\ninitrd ${base-url}/initrd\nimgargs vmlinuz root=/dev/ram0 ramdisk_size=3500000 cloud-config-url=/dev/null ip=dhcp url=${base-url}/ubuntu-24.04.2-live-server-amd64.iso initrd=initrd.magic console=ttyS0,115200n8 autoinstall ds=nocloud;s=${base-url}\nboot || goto failed\n\n# Error handling\n:failed\necho Boot failed, waiting 5 seconds...\nsleep 5\ngoto start\n\n# Reboot option\n:reboot\nreboot\n\n# Exit to shell\n:exit\necho Exiting to iPXE shell...\nexit\n</code></pre></div></div>\n\n<p>With this setup, we can now boot a machine from the network and automatically install Ubuntu with our chosen disk configuration.</p>",
9 "content_type": "html",
10 "author": {
11 "name": "Mark Elvers",
12 "email": "mark.elvers@tunbury.org",
13 "uri": null
14 },
15 "categories": [
16 "Netboot.xyz,Ubuntu",
17 "tunbury.org"
18 ],
19 "source": "https://www.tunbury.org/atom.xml"
20}