Case study 01 / Software Engineering
A database, from storage to consensus.
A Go key-value database that connects an embedded LSM storage engine to a Raft-replicated cluster.
The problem
What does a database need to keep a write correct when processes crash, leaders change, or a follower falls behind?
What I built
Built an embedded LSM engine with a write-ahead log, immutable SSTables, Bloom filters, range scans, and compaction. A separate replica mode uses that engine as the state machine for a three-node Raft cluster.
How it works
The replicated path persists and orders commands through Raft before applying them to the LSM engine. Snapshots bound the retained log and help lagging followers recover. The embedded mode retains its own WAL-based recovery path.
- gRPC client
- Raft quorum
- Committed log
- LSM state
Engineering decisions
One ordering authority
Replica mode uses the Raft log index as the LSM sequence number. This keeps replicated ordering and recovery under one authority.
Correctness through failure
The test suite exercises majority commit, conflicting logs, stale leaders, quorum loss, and snapshot recovery. Reads confirm a current-term quorum before serving linearizable results.
Expose the bottleneck
The documented local benchmark loses throughput with four clients. Per-update durable syncs and a single Raft event loop expose a clear opportunity for group commit.
Results & evidence
Single-client median
89.6 ops/s
Five local runs; 1,000 writes of 128 bytes per run on Apple M4, 16 GiB RAM, loopback gRPC. Reported in the project README.
Leader failover median
296 ms
Same single-client profile; time until a write succeeds after leader termination. This is a local development measurement.
Results are documented in the linked project artifacts. They have not been independently reproduced for this portfolio.
Limits & lessons
The local results are not production capacity. Four-client throughput fell to 28.6 ops/s in the reported runs. Embedded and replica durability guarantees differ; the project documentation describes both.
A useful database benchmark explains its durability settings and failure behavior alongside throughput.