sydepsystem design atlas

Distributed Rate Limiter

Count and throttle requests globally across many gateway instances.

IntermediatePlatform
ClientsClientGateway InstancesNetworkingDistributed Counter StoreStorage

Each gateway instance increments a shared rate-limit counter in a distributed store; the store exposes atomic increments and expiry semantics so all instances enforce the same quota without each one needing a local-only counter.

When to use it

  • Traffic is spread across many instances and the limit must be enforced globally, not per-node
  • Burstiness and cross-instance fairness matter for the API contract

Trade-offs

  • Shared limit state introduces a coordination dependency and potential hot-key bottlenecks
  • Expressing the right sliding-window semantics can be harder than a simple fixed window

Components used

Third-party ClientAPI GatewayCache

How it works

  • Counters live in a shared fast store so every gateway instance sees the same tally, rather than each enforcing its own local limit.
  • Increment-and-check runs as one atomic operation — typically a script executed inside the store — to avoid read-modify-write races.
  • Instances may cache a small local allowance to reduce round trips, trading a little precision for a lot of latency.

Used in the wild

  • API quotas enforced consistently across a horizontally scaled gateway fleet.
  • Login attempt limiting that cannot be bypassed by hitting a different server.
  • Fair-use enforcement on expensive shared resources.

Good to know

  • Purely local limits multiply by instance count: ten instances each allowing 100 requests per second permit 1000, and the effective limit changes every time you autoscale.
  • The counter store becomes a hard dependency on every request. Decide in advance whether it failing means fail-open (let traffic through) or fail-closed (reject everything) — the wrong choice turns a cache blip into a full outage.