Distributed Rate Limiter
Count and throttle requests globally across many gateway instances.
IntermediatePlatform
Each gateway instance increments a shared rate-limit counter in a distributed store; the store exposes atomic increments and expiry semantics so all instances enforce the same quota without each one needing a local-only counter.
When to use it
- Traffic is spread across many instances and the limit must be enforced globally, not per-node
- Burstiness and cross-instance fairness matter for the API contract
Trade-offs
- Shared limit state introduces a coordination dependency and potential hot-key bottlenecks
- Expressing the right sliding-window semantics can be harder than a simple fixed window
Components used
Third-party ClientAPI GatewayCache
How it works
- Counters live in a shared fast store so every gateway instance sees the same tally, rather than each enforcing its own local limit.
- Increment-and-check runs as one atomic operation — typically a script executed inside the store — to avoid read-modify-write races.
- Instances may cache a small local allowance to reduce round trips, trading a little precision for a lot of latency.
Used in the wild
- API quotas enforced consistently across a horizontally scaled gateway fleet.
- Login attempt limiting that cannot be bypassed by hitting a different server.
- Fair-use enforcement on expensive shared resources.
Good to know
- Purely local limits multiply by instance count: ten instances each allowing 100 requests per second permit 1000, and the effective limit changes every time you autoscale.
- The counter store becomes a hard dependency on every request. Decide in advance whether it failing means fail-open (let traffic through) or fail-closed (reject everything) — the wrong choice turns a cache blip into a full outage.
Related system design
URL Shortener
Create short aliases, then redirect quickly from a cache-backed read path.
Pastebin / Snippet Hosting
Store snippets as text or markdown, retrieve them by id, and optionally render preview.
Web Crawler / Search Indexer
Discover pages, crawl them, extract content, and index for search.
Distributed Job Scheduler
Queue jobs, dispatch them across workers, and persist execution state.