Memory Resource Management in VMware ESX Server
Waldspurger, 2002
Memory Resource Management in VMware ESX Server
What kind of paper is this?
- Tackles a very specific problem.
- Memory usage in a VMM based machine
- Introduces a number of techniques to address that particular problem
Difference between VMware Workstation and VMware ESX Server
- Workstation: Uses host OS to virtualize
- ESX: Provides hypervisor to provide virtualization
- Both cases: Pure virtualization (not paravirtualization)
Big Picture
- Target per-VM memory allocation
- Allocation a function of load and parameterization
- Try to reclaim memory from VMs
- In background, try to share pages between VMs
Terminology
- machine address: traditional hardware address
- physical address: a VMs (false) belief of a hardware address;
abstraction provided to the VM, so it thinks it has traditional machine
addresses
Hardware/Software Interface
- VM thinks in terms of its page tables that map virtual addresses to
physical addresses.
- Shadow page table maps virtual addresses to machine addresses, so that
memory translation works while guest applications run.
Ballooning
- ESX server installs a "device driver" in the Guest.
- To take memory away from the guest, "inflate" the balloon, thus
putting memory pressure on the guest.
- The guest will allocate the balloon a new physical page (either by
paging or from a free list).
- The balloon tells the server what page got allocated
- The server can then use the page allocated to the balloon
- When you deflate the balloon, the pages go back to the guest OS.
- Performance shows that ballooning tracks non-ballooned performance
When Ballooning Doesn't Work
- Resort to swapping
- Parts of guest memory are simply swapped out and reused
- Use random page replacement so as not to interfere with native
paging algorithms
Sharing
- Goal: share pages without OS modification (as in Disco)
- Compute hash on each page
- Check for matching hash; if found, compare whole page
- Mark shared pages COW
- Find pages to share via
- Zero page is widely shared
Share-based Memory Allocation
- Desire proportional share allocation, but...
- Want to value active pages more than idle ones
- Tax idle pages so they appear to be more costly
- Reclaim idle pages
- Use statistical sampling (page invalidation) to determine what fraction
of a VMs pages are active/inactive.
Allocation Policies
- Admission Control
- If there isn't enough memory available, don't start up a VM.
- Similarly, reserve some disk space for swap
- Dynamic Reallocation (moving memory around).
- Four threshholds that determine reallocation
- High: Don't do anything
- Soft: Use ballooning
- Hard: Start paging
- Low: Page and block all piggy VMs