Capacity Planning, SLOs, and Error Budgets TODO
Concept
An SLO is a target set on a service-level indicator (SLI) measured from the user's perspective, and the error budget is the total amount of failure that target allows. If the availability target is set at 99.9%, the remaining 0.1% is the budget available for that period, and it becomes an explicit trade-off mechanism between shipping speed and stability. When budget remains you can deploy more aggressively; once it's exhausted, the policy is to halt feature releases and put the effort into reliability work instead. Capacity planning combines this with load forecasting to determine the maximum load the system can absorb while holding the target latency, and how much headroom is needed. From a queuing-theory perspective, wait time diverges sharply as utilization approaches 1, so headroom should be sized against peak and tail load, not average utilization.
Without agreeing on "how stable does this need to be" as a number, every priority fight between incident response and feature work turns into an emotional argument.
Code & Formula
# 용량 계획·SLO와 에러 예산 — 가용성 목표에서 허용 실패량(에러 예산)을 계산하고,
# 실측 실패율로 예산 소진율을 구해 배포를 계속할지 판단한다.
def error_budget_minutes(slo_percent, period_days=30):
"""기간 동안 허용되는 다운타임(분)"""
period_minutes = period_days * 24 * 60
allowed_failure_ratio = 1 - slo_percent / 100
return period_minutes * allowed_failure_ratio
def budget_status(slo_percent, downtime_minutes_so_far, days_elapsed, period_days=30):
total_budget = error_budget_minutes(slo_percent, period_days)
consumed_ratio = downtime_minutes_so_far / total_budget
# 지금까지 경과한 기간 대비 정상 소진 속도(1.0이면 딱 예산대로 소진 중)
expected_ratio_by_now = days_elapsed / period_days
burn_rate = consumed_ratio / expected_ratio_by_now if expected_ratio_by_now else 0
return total_budget, consumed_ratio, burn_rate
slo = 99.9 # 월 가용성 목표
total_budget = error_budget_minutes(slo)
print(f"SLO {slo}% -> 30일 에러 예산: {total_budget:.1f}분")
scenarios = [
("정상 운영", 10.0, 15), # 15일 경과, 다운타임 10분
("장애 다발", 35.0, 15), # 같은 15일에 다운타임 35분
]
for name, downtime, days in scenarios:
budget, consumed, burn = budget_status(slo, downtime, days)
action = "배포 계속 (여유 있음)" if burn < 1.0 else "기능 출시 중단, 신뢰성 작업 우선"
print(f"[{name}] {days}일 경과, 다운타임 {downtime}분 -> "
f"소진율 {consumed*100:.1f}%, burn rate {burn:.2f}x -> {action}")
# 용량 계획: 이용률이 1에 가까워질수록 대기시간이 급격히 발산 (M/M/1 근사)
def expected_wait_factor(utilization):
"""대기행렬 이론: 대기시간은 rho / (1 - rho) 에 비례해 발산"""
if utilization >= 1:
return float("inf")
return utilization / (1 - utilization)
print("\n이용률별 대기시간 배율 (M/M/1 근사):")
for rho in [0.5, 0.7, 0.9, 0.95, 0.99]:
print(f" rho={rho:.2f} -> wait factor={expected_wait_factor(rho):.2f}")
docs/code/algorithms/algorithms-49.py
Exercise
For a service you run, define two SLIs (availability and p99 latency), compute a 30-day SLO and error budget, then plug in the last 30 days of real measurements to work out the budget burn rate.
Practical Connection
For paths mixed with external factors — RPC node dependencies, oracle response latency, settlement transaction confirmation time — pre-defining targets and budgets is what lets you judge where "normal" ends.
If you study this on a given day, add a note link and a ✅ to this line in the source curriculum (docs/knowledge/dev-100-curriculum.md) and this spot will lead straight to the note body. You can also write directly on this page — but regenerating overwrites it, so it's safer to keep anything you want to save as markdown under docs/algorithms/.