Arrakis: The Operating System is the Control Plane

What kind of paper is this?

• Design and Implementation
• Different strategy than a lot: let's take a particular environment (network servers) and design an OS specifically for them.

The Story

• Once upon a time people built general purpose operating systems. However, as these systems grew to embody more and more functionality, the raw performance for IO intensive applications got worse. At one point, it was proposed that we give applications direct access to the hardware for such data movement, but this didn't catch on. Now, however, with even faster networks and persistent stores, the authors revisted this idea. They designed a system that provides such direct access, producing significant performance improvements for this important class of servers. And now everyone who uses such services lives happily ever after.

Today's stacks are deeply layered and introduce overhead

• Networking
  • SW demultiplexing
  • Security checks
  • Scheduling
  • Context switching
  • Cache and lock contension
  • Queue management
• Storage (done with fsync) (Used to not be such a big deal when disks were the bottleneck; with faster persistence, CPU time is a greater concern.)
  • Data copies
  • Parameter checking
  • Allocation
  • VFS
  • Snapshots
  • Metadata updates
• Arakis solution: use SR-IOV to remove kernel mediation from the data plane; eliminates some overhead entirely and reduces other overheads. Eschewing POSIX produces true zero-copy.

Virtual IO

• SR-IOV is designed to support multiple VMs sharing an IO device
• Single device (physical function) can be dynamically turned into multiple PCI devices (virtual functions).
• Hypervisor creates the virtual functions and install filters to demultiplex operations to the right virtual function (guest OS).
• Arrakis uses SR-IOV and IOMMUs to give applications (instead of guest VMs) direct access to IO devices.
• Note: This is not so much a new idea as the modern embodiment of an idea that's been around for quite some time.

Goals

• Remove kernel from data plane operations
• Require no changes for application programming (but allow changes to produce even better performance).
• Provide abstractions useful across many devices.
• Enabling technology: While SR-IOV is designed for VMs, in Arrakis, each application can have its own networking stack and virtual device.

Evaluation

• Read-heavy memcached
• Write-heavy Redis
• Direct HTTP requests
• HTTP requests via a load balancer
• Nice focused questions for the eval:
  1. What are performance overheads and how to they compare to Linux?
  2. Is Arrakis latency/throughput better?
  3. Can Arrakis provide high performance directly to the application?
  4. What is the advantage of departing from POSIX?

Results

• Is 128 entries in memcached realistic? If not, is it likely to affect performance? Is the equal load assumption valid?
• Why do we only see throughput numbers?

Feature Analysis: What did I like?

1. Getting the OS out of the way for data operations has repeatedly been shown to be a win.
2. Designing for idealized HW also makes the OS way easier to deal with.