Tutorial: minimum viable Mirage unikernel web server on GCE
If tens of millions of lines of C code in your application stack keeps you from sleeping easily at night, as it does me, you may be interested in changing your web services to all-OCaml based Mirage unikernels. The complete argument for considering this can be made elsewhere.
What I'd like to do here is provide a minimum example that runs on a commonly available cloud provider, Google Compute Engine (GCE) on the most affordable GCE instance type, f1-micro. This is assuming you already have an OCaml opam environment setup. This also assumes you're on Ubuntu on x86-64.
Firstly, you'll need something called Solo5, for gluing unikernels into virtualization platforms:
cd ~/src
git clone https://github.com/Solo5/solo5
cd solo5
./configure.sh
make
# no install needed, we'll reference the tools here soon
Now get the Mirage tutorial code then build and package the unikernel.
The example we're using here is a very basic, static content, HTTPS-enabled web site:
opam install mirage --yes
cd ~/src/
git clone https://github.com/mirage/mirage-skeleton
cd mirage-skeleton/applications/static_website_tls
mirage configure -t virtio --dhcp true
make depends
make
# yields a payload called https.virtio
virtio is the virtualization platform you want Mirage to target. It's an extension to Linux KVM that allows for efficient sharing of I/O devices with the hypervisor. Then, we use Solo5 to package the unikernel:
~/src/solo5/scripts/virtio-mkimage/solo5-virtio-mkimage.sh -f tar -- \
image.tar.gz https.virtio --ipv4-only=true
image.tar.gz is the kernel that our GCE instance will boot into.
The rest of these instructions are how to set up the GCE instance to boot this image.
Go to https://cloud.google.com/ and create an account if you don't have one. Then set up the Google Cloud SDK by following the instructions here.
gcloud init
export MY_PROJECT=mirage-test-12345
# This name above needs to be unique across all gcloud users in the world.
export MY_REGION=us-west1
gcloud projects create $MY_PROJECT
You must enable billing for this project to proceed. Visit https://cloud.google.com/ , log into the Console, select $MY_PROJECT from the dropdown in the top left, and then click Billing. Connect it to a credit card you have on file.
gcloud config set project $MY_PROJECT
gcloud compute addresses create my-address1 --region $MY_REGION
export MY_IP_ADDRESS=..EXTERNAL_IP from output of previous step..
If you have a DNS domain somewhere, you might want to assign a friendly name to the $MY_IP_ADDRESS above.
Now, upload image.tar.gz to a GCE storage "bucket", then turn it into a VM image.
gsutil mb gs://$MY_PROJECT
gsutil cp image.tar.gz gs://$MY_PROJECT
gcloud compute images create mirage-test --source-uri gs://$MY_PROJECT/image.tar.gz
Your GCE instance is locked down to the world by default. Create firewall rules to allow access to the pre-configured static_website_tls ports 8080 and 4433.
gcloud compute firewall-rules create http --allow tcp:8080
gcloud compute firewall-rules create https --allow tcp:4443
Now boot the image!
gcloud compute instances create mirage-test --image mirage-test \
--address $MY_IP_ADDRESS --zone $REGION-a --machine-type f1-micro
If your request can't be satisfied in the current GCE region try changing the -a to a -b, or -b to -c, or so on, until you find an availability zone with enough dimension. You should see output like this on success:
NAME ZONE MACHINE_TYPE PREEMPTIBLE INTERNAL_IP EXTERNAL_IP STATUS
mirage-test us-west1-c f1-micro 10.168.0.2 $MY_IP_ADDRESS RUNNING
Visit https://$MY_IP_ADDRESS:4433/ and accept the untrusted certificate. You should see the Mirage Hello World.
If you have any trouble, visit the GCE console, click on your mirage-test instance, and view the serial console. Successful output looks like this:
Serial port 1 (console) output for mirage-test
SeaBIOS (version 1.8.2-google)
Total RAM Size = 0x0000000026600000 = 614 MiB
CPUs found: 1 Max CPUs supported: 1
found virtio-scsi at 0:3
virtio-scsi vendor='Google' product='PersistentDisk' rev='1' type=0 removable=0
virtio-scsi blksize=512 sectors=2097152 = 1024 MiB
drive 0x000f2510: PCHS=0/0/0 translation=lba LCHS=1024/32/63 s=2097152
Sending Seabios boot VM event.
Booting from Hard Disk 0...
SYSLINUX 6.04 20200816 Copyright (C) 1994-2015 H. Peter Anvin et al
Loading unikernel.bin... ok
| ___|
__| _ \ | _ \ __ \
\__ \ ( | | ( | ) |
____/\___/ _|\___/____/
Solo5: Bindings version v0.6.8
Solo5: Memory map: 613 MB addressable:
Solo5: reserved @ (0x0 - 0xfffff)
Solo5: text @ (0x100000 - 0x488fff)
Solo5: rodata @ (0x489000 - 0x52dfff)
Solo5: data @ (0x52e000 - 0x82ffff)
Solo5: heap >= 0x830000 < stack < 0x265fd000
Solo5: Clock source: KVM paravirtualized clock
Solo5: PCI:00:03: unknown virtio device (0x8)
Solo5: PCI:00:04: virtio-net device, base=0xc040, irq=11
Solo5: PCI:00:04: configured, mac=42:01:0a:a8:00:05, features=0x204399a7
Solo5: PCI:00:05: unknown virtio device (0x4)
Solo5: Application acquired 'service' as network device
2021-11-09 23:31:32 -00:00: INF [netif] Plugging into service with mac 42:01:0a:a8:00:05 mtu 1500
2021-11-09 23:31:32 -00:00: INF [ethernet] Connected Ethernet interface 42:01:0a:a8:00:05
2021-11-09 23:31:32 -00:00: INF [ipv6] IP6: Starting
2021-11-09 23:31:32 -00:00: INF [dhcp_client_lwt] Lease obtained! IP: 10.168.0.5, routers: 10.168.0.1
2021-11-09 23:31:32 -00:00: INF [ARP] Sending gratuitous ARP for 10.168.0.5 (42:01:0a:a8:00:05)
2021-11-09 23:31:32 -00:00: INF [udp] UDP interface connected on 10.168.0.5
2021-11-09 23:31:32 -00:00: INF [tcpip-stack-direct] stack assembled: mac=42:01:0a:a8:00:05,ip=10.168.0.5
2021-11-09 23:31:32 -00:00: INF [https] listening on 4433/TCP
2021-11-09 23:31:32 -00:00: INF [http] listening on 8080/TCP
2021-11-09 23:37:58 -00:00: INF [https] [7] serving //34.102.123.54:4433/.
2021-11-09 23:37:58 -00:00: INF [https] [7] serving //34.102.123.54:4433/favicon.ico.
2021-11-09 23:38:45 -00:00: INF [https] [8] closing
Costs
As of this writing, in the Google Cloud us-west1 region, an f1-micro instance costs about $3.88 per month and the final deployed image consumes 4.4MB (that’s megabytes with an M) of storage, though GCE rounds this up to a whole gigabyte, which costs $0.02 per month.
The future
~~In a follow-up post I'll describe the workflow for updating the image and also for integrating Let's Encrypt support.~~
To see how to update your running image, and replace the static TLS web server with an untrusted certificate to one that fetches a certificate from Let's Encrypt, see this follow-up post
Thank yous!
The information here was adapted from instructions in the DreamOS project provided by the illustrious @dinosaure and with some hand-holding from the OCaml Labs Slack #mirage community.