My VM is Lighter (and Safer) than yours

What kind of paper is this?

• It's a "I want the best of both worlds" paper.
• I want the isolation of VMs, but the performance of containers.
• Perhaps that makes it a kind of synthesis paper (bringing together two ideas?)

The Fairy Tale: Once upon a time, people began deploying services in virtual machines, for ease of deployment and isolation. Then people developed containers to improve the overhead of these deployments. However, containers do not provide the same isolation guarantees as VMs (and some people really want such guarantees), so folks were left running containers in VMs (which kind of seems to defeat the purpose). LightVM is an attempt to get the lightweight cost of containers in a VM framework. If such a thing were possible, everyone would live happily ever after.

Requirements

• Fast VM creation (100s of ms)
• Thousands of VMs per host (Xen's original goal was 100)
• Quick pause/restart
• What are the obstacles? Big footprint
• Obvious approach: reduce both image size and VM footprint

Approach

• Leverage the fact that most containers run only a single
• Unikernel: application is linked directly with the (small) OS
• Tinyx: Tool to build tiny Linux installations application.
  • Takes an application and a target hypervisor
  • Uses objdump and the package manager to figure out dependencies
  • Install everything on a Debian overlap file system
  • Remove all the cruft: cache files, install files, etc
  • Then overlay the OverlayFS on a BusyBox image
  • Incrementally figure out which kernel build options are necessary.

Identify Xen Overheads

• Nice methodology: time 1000 create/boot VMs of 1) Debian, 2) Tinyx, 3) MiniOS, 4) Docker, 5) Process
• Then instrument Xen and figure out where the time is going
• This shows the Xenstore and devices are the culprit
• Device cost is constant
• Xenstore increases as the number of instances increases
• Some of the Xenstore overhead is just dumb (comparing against existing names -- um, how about an index?).

LightVM

• Redesign of Xen control plane
• Instead of a XenStore server; just communicate shared information via shared memory.
• How much of this re-engineering just helps with create/startup? That is, does it actually make the running VMs also lighter?
• Key idea in noxs: Hypervisor already has a lot of this information, so let it manage it instead of a server in Dom0.
• User a page shared between Guest and Xen instead

Eval

• What is the "fair" way to measure create time after the decoupling of shell creation and VM creation?
• Ah, the different combinations provide nice insight!
• Microbenchmark (i.e., create) is pretty compelling.
• Next they look at save/restore/migrate
• Then they look at Memory Footprint
• Finally CPU Usage
• Yay! Then they do use cases (maybe some real workloads???)
  1. Personal Firewall: Scenario is on a mobile gateway; each user gets a lightVM to implement its firewall; the VM can migrate when the user moves. Use ClickOS. (They show that they can run this workload, but they don't A) compare to containers or B) show where regular VMs would break.)
  2. JIT Service Instantiation: Let a phone have its own VM in a nearby cloud for processing ofload
  3. TLS Termination: Nice experiment, because it exposes a weakness in the unikernel implementation.
  4. Lightweight Compute Service: (Think lambda.)