WAL, Group Commit, and the Cost of fsync TODO
Concept
A write-ahead log (WAL) records changes sequentially to a log file before modifying the actual data pages, guaranteeing durability and crash recovery — and as a side effect, it turns random writes into sequential ones. For a commit to be truly durable, its log record has to actually reach storage, which requires an fsync (or fdatasync) call to force the OS page cache and device cache to flush — and that call is the dominant cost in commit latency. Group commit batches the log records of several transactions that arrive within a short time window and durably writes them with a single fsync, so the number of fsyncs scales with time rather than with transaction count. The result: a slight increase in any single transaction's latency, but a large gain in overall throughput — a classic latency-for-throughput batching tradeoff. Conversely, an asynchronous-commit setting skips waiting on fsync entirely for speed, at the cost of accepting the loss of some recent commits on crash.
When a database suddenly slows down, the culprit is often fsync-per-commit rather than the query plan — and conversely, a setting that looks fast might actually have quietly traded away durability.
Code & Formula
# WAL·그룹 커밋·fsync 비용 — 변경을 로그에 먼저 순차 기록하고, 여러 트랜잭션을 모아 fsync 한 번으로 묶어 내구화한다.
# 트랜잭션마다 fsync하는 방식과, 짧은 창 안의 여러 트랜잭션을 묶어 fsync 한 번으로 처리하는 그룹 커밋을 비교한다.
class WAL:
def __init__(self):
self.log = []
self.fsync_calls = 0
def append(self, record):
self.log.append(record) # 순차 기록 (아직 장치까지 내구화되지는 않음)
def flush(self):
self.fsync_calls += 1 # fsync: 실제 장치까지 강제로 내려보내는, 비용이 큰 호출
txns = [f"txn-{i}: UPDATE balance SET ..." for i in range(12)]
# 방식 1: 트랜잭션마다 즉시 fsync
wal1 = WAL()
for t in txns:
wal1.append(t)
wal1.flush()
print(f"개별 커밋: fsync 호출 {wal1.fsync_calls}회 (트랜잭션 수만큼)")
# 방식 2: 그룹 커밋 — 4개씩 모아 fsync 한 번
wal2 = WAL()
GROUP_SIZE = 4
for i in range(0, len(txns), GROUP_SIZE):
for t in txns[i:i + GROUP_SIZE]:
wal2.append(t)
wal2.flush() # 그룹 전체를 한 번의 fsync로 내구화
print(f"그룹 커밋(그룹 크기 {GROUP_SIZE}): fsync 호출 {wal2.fsync_calls}회")
def replay(wal):
return list(wal.log) # 크래시 복구: 로그를 처음부터 재생해 마지막 커밋 상태를 되살린다
print(f"\n복구 재생 결과 (마지막 3건): {replay(wal2)[-3:]}")
docs/code/algorithms/algorithms-70.py
Exercise
Run the same insert workload against local Postgres with synchronous_commit on and off, compare TPS and p99 latency, and also measure the numbers when transactions are batched together.
Practical Connection
When an indexer writes thousands of events per block, batching commits at the block level instead of committing per event is exactly the same throughput trick, for exactly the same reason.
If you study this on a given day, add a note link and a ✅ to this line in the source curriculum (docs/knowledge/dev-100-curriculum.md) and this spot will lead straight to the note body. You can also write directly on this page — but regenerating overwrites it, so it's safer to keep anything you want to save as markdown under docs/algorithms/.