Demystifying Open-Source Orchestration of Unikernels With Unik

Abstract

As the cloud-native ecosystem continues to evolve, many alternative solutions are popping-up, that challenges the status quo of application deployment methodologies. One of these solutions that is quickly gaining traction is Unikernels, which are executable images that can run natively on a hypervisor without the need for a separate operating system.

For cloud-native platforms to integrate unikernels into their ecosystem, they are required to provide unikernels with the same services they provide for containers. In this post, we are going to introduce UniK, an open-source orchestration system for unikernels.

UniK (pronounced you-neek) is a tool for simplifying compilation and orchestration of unikernels. Similar to the way Docker builds and orchestrates containers, UniK automates compilation of popular languages (C/C++, Golang, Java, Node.js. Python) into unikernels. UniK deploys unikernels as virtual machines on various cloud platforms.

UniK Design

The UniK Daemon consists of 3 major components.

• The API server
• Compilers
• Providers

The API Server handles requests from the CLI or any HTTP Client, then determines which is the appropriate provider and/or compiler to service the request.

When the API Server receives a build request (POST /images/:image_name/create), it calls the specified compiler to build the raw image, and then passes the raw image to the specified provider, which processes the raw image with the Stage() method, turning it into an infrastructure-specific bootable image (e.g. an Amazon AMI on AWS)

Let’s go ahead and try spinning up a unikernel ourselves.

Deploying a GO HTTP daemon with UniK

In this tutorial, we are going to be:

1. Installing UniK
2. Writing a simple HTTP Daemon in Go
3. Compiling to a unikernel and launching an instance on Virtualbox

Install, configure, and launch UniK

Prerequisites

Ensure that each of the following are installed

• Docker installed and running with at least 6GB available space for building images
• jq
• make
• Virtualbox

Install UniK

$ git clone https://github.com/solo-io/unik.git
$ cd unik
$ make binary

Note: make will take quite a few minutes the first time it runs. The UniK Makefile is pulling all of the Docker images that bundle UniK’s dependencies.

Then, place the unik executable in your $PATH to make running UniK commands easier:

$ mv _build/unik /usr/local/bin/

Configure a Host-only network on VirtualBox

1. Open Virtualbox.
2. Open Preferences > Network > Host-only Networks.
3. Click the green Add button on the right side of the UI.
4. Record the name of the new Host-only adapter (e.g. “vboxnet0”). You will need this in your UniK configuration.
5. Ensure that the Virtualbox DHCP Server is Enabled for this Host-only Network
6. With the Host-only Network selected, Click the edit button (screwdriver image)
7. In the Adapter tab, note the IPv4 address and netmask of the adapter.
8. In the DHCP Server tab, check the Enable Server box.
9. Set Server Address an IP on the same subnet as the Adapter IP. For example, if the adapter IP is 192.168.100.1, make set the DHCP server IP as 192.168.100.X, where X is a number between 2-254.
10. Set Server Mask to the netmask you just noted.
11. Set Upper/Lower Address Bound to a range of IPs on the same subnet. We recommend using the range X-254 where X is one higher than the IP you used for the DHCP server itself. E.g., if your DHCP server is 192.168.100.2, you can set the lower and upper bounds to 192.168.100.3 and 192.168.100.254, respectively.

Configure UniK daemon

UniK configuration files are stored in $HOME/.unik. Create this directory, if it is not present:

$mkdir $HOME/.unik

Using a text editor, create and save the following to $HOME/.unik/daemon-config.yaml:

providers:
  virtualbox:
    - name: my-vbox
      adapter_type: host_only
      adapter_name: NEW_HOST_ONLY_ADAPTER

Replacing NEW_HOST_ONLY_ADAPTER with the name of the network adapter you created.

Launch UniK and automatically deploy the Virtualbox Instance Listener

• Open a new terminal window/tab. This terminal will be where we leave the UniK daemon running.
• cd to the _build directory created by make
• run unik daemon --debug (the --debug flag is optional, if you want to see more verbose output)
• UniK will compile and deploy its own 30 MB unikernel. This unikernel is the Unik Instance Listener. The Instance Listener uses udp broadcast to detect (the IP address) and bootstrap instances running on Virtualbox.
• After this is finished, UniK is running and ready to accept commands.
• Open a new terminal window and type unik target --host localhost to set the CLI target to the your local machine.

Write a Go HTTP server

Open a new terminal window, but leave the window with the daemon running. This window will be used for running UniK CLI commands.

1. Create httpd.go file

Create a file httpd.go using a text editor. Copy and paste the following code in that file:

package main

import (
    "fmt"
    "net/http"
)

func main() {
    http.HandleFunc("/", handler)
    http.ListenAndServe(":8080", nil)
}

func handler(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "my first unikernel!")
}

2. Run the code

Try running this code with go run httpd.go. Visit http://localhost:8080/ to see that the server is running.

3. Create a dummy Godeps file

We need to create a dummy Godeps file. This is necessary to tell the Go compiler how Go projects and their dependencies are structured. Fortunately, with this example, our project has no dependencies, and we can just fill out a simple Godeps file without installing godep.

Note: For Go projects with imported dependencies, and nested packages, you will need to install Godeps and run GO15VENDOREXPERIMENT=1 godep save ./... in your project. See Compiling Go Apps with UniK for more information.

To create the dummy Godeps file, create a folder named Godeps in the same directory as httpd.go. Inside, create a file named Godeps.json and paste the following inside:

{
	"ImportPath": "my_httpd",
	"GoVersion": "go1.6",
	"GodepVersion": "v63",
	"Packages": [
		"./.."
	],
	"Deps": [
		{
			"ImportPath": "github.com/solo-io/unik/docs/examples",
			"Rev": "f8cc0dd435de36377eac060c93481cc9f3ae9688"
		}
	]
}

For the purposes of this example, what matters here is my_httpd. It instructs the Go compiler that the project should be installed from $GOPATH/src/my_httpd.

4. Great! Now we’re ready to compile this code to a unikernel.

Compile an image to run on VirtualBox

1. Compile sources

Run the following command from the directory where your httpd.go is located:

unik build --name myImage --path ./ --base rump --language go --provider virtualbox

This command will instruct UniK to compile the sources found in the working directory (./) using the rump-go-virtualbox compiler.

2. Watch output

You can watch the output of the build command in the terminal window running the daemon.

3. Locate disk image

When build finishes, the resulting disk image will reside at $HOME/.unik/virtualbox/images/myImage/boot.vmdk

4. Run instance of disk image

Run an instance of this image with:

unik run --instanceName myInstance --imageName myImage

5. Check IP

When the instance finishes launching, let’s check its IP and see if it is running our application. Run unik instances. The instance IP Address should be listed.

6. View the instance!

Direct your browser to http://instance-ip:8080 and see that your instance is running!

Finishing up

To clean up your image and the instance you created

unik rmi --force --image myImage

And we’re done. We hope you will benefit from this post and tutorial and stay tuned for future posts!