HBench-OS Operating System Benchmarks

Download | Documentation | Publications | Contact the authors

HBench-OS is a suite of portable benchmarks designed to measure the performance of primitive functionality provided by an operating system/hardware platform. HBench-OS was designed primarily as a research tool for the systems research and OS development communities, and thus its major design goal is to provide a flexible system for reproducible, accurate, and statistically-sound performance analysis and evaluation. These characteristics make it ideal for both performance research as well as more traditional consumer-oriented performance measurement and analysis.

HBench-OS's major features include:

a broad range of benchmarks, covering hardware capabilities, low-level OS abstractions, and high-level OS primitives
an accurate timing system that automatically measures and removes its own overhead
most benchmarks dynamically determine how long to run to overcome low timing resolution
all benchmark data is preserved for analysis
support for user-defined statistical analysis policies, including viewing raw data distributions
basic automatic analysis of benchmark data
support for obtaining cold-cache results (no loops) when timing resolution is high enough
a flexible control system that allows you to configure exactly which benchmarks are executed (and their parameters) in a given run
built-in support for multiple benchmarking runs, including statistical processing across multiple runs
a machine-independent framework that supports high-resolution timing via hardware cycle counters (currently supports Intel Pentium and later chips)
a machine-independent framework to support hardware event counters for detailed profiling of the benchmark tests (currently supports Intel Pentium Pro under modified NetBSD/i386 only)

Read the HBench-OS README file.

Visit the page of collected HBench-OS results. If you would like to share your results, please mail them to hbench-results@eecs.harvard.edu.

Downloading HBench-OS

HBench-OS is available either via http or ftp by clicking on the appropriate link below. It is distributed as a .tar.gz file; after extracting the file using gunzip and tar, please read the included README file for information on configuring and using the benchmarks.

Download now:

Version 1.0:

http://www.eecs.harvard.edu/vino/perf/hbench/hbench-OS-1.0.tar.gz

ftp://ftp.eecs.harvard.edu/pub/hbench-OS/hbench-OS-1.0.tar.gz

Also available:

NetBSD/i386 drivers for Pentium and Pentium Pro hardware performance monitoring counters

More Information on System Performance Measurement Using HBench-OS

The following papers and slides give more technical details on HBench-OS, and describe analysis methodologies that use HBench-OS as the foundation for a complete model of computer system performance:

Brown, A., and M. Seltzer. "Operating System Benchmarking in the Wake of Lmbench: A Case Study of the Performance of NetBSD on the Intel x86 Architecture." Proceedings of the 1997 ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems, Seattle, AW, June 1997, 214-224. [slides]
Brown, A., "A Decompositional Approach to Computer System Performance Evaluation." Harvard Senior Thesis, 7 April 1997. [pdf] [.ps.gz]

Author Contact Information

HBench-OS was written by Aaron B. Brown, with the guidance and assistance of Margo Seltzer, margo@eecs.harvard.edu.

Any questions/comments/bug reports/patches that you have should be sent to the author at margo@eecs.harvard.edu.

Last modified: 17 April 2003.