Rate limiting strategies: token bucket vs. sliding window
January 28, 2025 · sydep team · 2 min read
Every public API needs rate limiting, but "add a rate limiter" hides a real design decision: which algorithm, and where does it live?
Fixed window: simple, but bursty at the edges
A fixed window counter (e.g. "100 requests per minute, reset on the minute") is trivial to implement and cheap to check, but it allows up to 2x the intended rate at window boundaries — a client can send 100 requests in the last second of one window and another 100 in the first second of the next.
Sliding window: smooths the boundary problem
A sliding window log or sliding window counter tracks requests over a rolling interval rather than a fixed clock-aligned one, which eliminates the boundary-burst problem. The trade-off is more state to track per client (a log of timestamps, or a weighted average between two fixed windows).
Token bucket: the most common production choice
A token bucket adds tokens to a per-client bucket at a fixed rate, up to a cap, and each request consumes a token. This naturally allows short bursts (up to the bucket size) while enforcing a long-run average rate — which usually matches how real clients actually behave (bursty, not perfectly uniform). It's also cheap: just a counter and a last-refill timestamp per client, refilled lazily on each request rather than on a timer.
See the full component breakdown in the rate limiting pattern reference.
Where to enforce it
- At the edge (API gateway / CDN) — cheapest to scale, protects everything behind it, but usually only supports coarse-grained rules (per IP, per API key).
- In a shared service (e.g. Redis-backed counters) — needed once you have multiple application instances that all need to agree on the same client's usage in real time.
- In-process, per-instance — simplest to build, but only enforces a limit per instance, which means the effective limit scales with your fleet size unless you also cap it there deliberately.
Most production systems layer two of these: a coarse edge limit for abuse protection, and a precise shared-state limiter for billing-relevant quotas.
Try wiring a rate limiter in front of an API in the canvas to see how it composes with the rest of a request path.