Reimplementing the Cedar File System Using Logging and Group Commit

What kind of paper is this?

• Design
• Analysis
• Analytic Modeling

Overall Design

• Log-based recovery for meta-data (a la JFS, Veritas).
• Group Commit to reduce number of writes.
• Base design on performance model.
• Increased robustness.

Terminology

• Header sectors = inodes
• Data sectors = file blocks
• Bit vector = bitmaps
• Scavenge = fsck

Requirements

• Robust against HW failure
  • Lose one or two sectors and just lose data.
  • Never lose meta-data due to single failure.
• High performance (near hardware speed).
• Robust against SW failures.
  • Memory smashes.
  • Internal errors.
• Fast recovery (no fsck).
• Off-the-shelf components.

Design

• CFS
  • Three main components: file name table, file headers, disk labels.
  • File Name Table
    1. File name Table maps name to unique id.
    2. Name table contains address of header block (i.e. inode address).
  • File Headers
    1. Run Table: addresses of disk blocks (d_ib and d_db in inode).
    2. Length of file.
    3. Time stamps.
    4. Name of file (redundant with above).
  • Labels
    1. Sector type (header, free, data).
    2. Page number
    3. UID
• FSD
  • Merge File Name Table and File Headers
  • Replicate this merged structured (called File Name Table).
  • Leaders are a new data structure that checks for integrity of FNT.
    1. Used only for SW checking.
    2. Contains: UID, preamble of run table, checksum of run table.
  • Log all changes to either FNT or Leaders.
  • Replicate log.
  • Group commit bundles log writes.
  • Write log at least once per half-second.
  • If sectors fill before half-second, write log.
• Comparison of file creation under CFS and FSD

  CFS File Create	FSD File Create
  Read VAM (bitmap)	Read VAM (bitmap)
  Find 3 blocks file header (2 blocks) new file data	Find 2 blocks new file data Leader page
  Read labels to ensure that they are free (2)	Log write
  Write the labels to claim them for the file
  Write the header
  Add entry to the file name table	Update File Name Table
  Write the data	Writer the leader
  Rewrite the header	Write the data
  	Asynchronous log write

Detail Design Discussion

• Locality: put information logged and written close together.
  • The header and FNT are now one object; single read; makes ls fast.
  • Does not facilitate UNIX hard links (each file has one and only 1 name).
• Robustness
  • Leader pages are not necessary for normal operation.
  • Provide sanity checks on recovery.
• Logging
  • Avoid multiple synchronous writes.
  • Log only meta-data writes (FNT and Leaders (asynchronously)).
  • Log writes are localized.
  • Log writes allow other data writes to be grouped (i.e. directory updates).
  • Cedar uses redo-only log (i.e. never undo operations, just redo them).
  • Did not log bitmaps (could have with pretty minimal modification).
  • Implement log as circular buffer.
  • Recover log space in thirds (i.e. no periodic checkpoints, just periodically flush pages to disk that haven’t been written in 2/3 of a log period).
  • Log robustness through duplicate writes to non-adjacent blocks.
  • Physical logging.
• Free bit maps
  • Since all ops are async, need to delay free of blocks.
  • Keep a shadow of blocks that will be freed upon next write.
  • Reuse blocks only after log records are safely on disk.
• Allocation
  • Partition disk into big-file areas and small-file areas.
  • Difficult to allocate files correctly.
  • Since they are caching server files, simplifies this chore.
  • They encountered same potential fragmentation problems as FFS.
• Protection against errors
  • Logging makes multi-page B-tree operations atomic.
  • Logging protects against partial writes of FNT.
  • Replication of FNT protects against bad pages in it.
  • Recovery takes care of bad blocks in VAM (reconstructed).
  • Boot pages are replicated.

Performance

• Model performance to make design decisions.
• Calculate average time for each operation based on:
  • the number of short seeks,
  • the number of long seeks,
  • the number of half revolutions,
  • the number of lost revoluations, and
  • transfer time.
• Estimate caching behavior with probabilistic model.
• Calibrate model on CFS and 4.2BSD FFS.
• Predictions were with in 5% (this is very good).
• Model ignores CPU. Is this OK?
• Performance measurements
  • how many runs?
  • standard deviations?
  • table 5 compares FFS and FSD in terms of CPU and disk utilization