sydepsystem design atlas

Circuit Breaker

Stop calling a failing dependency instead of piling up timeouts.

StarterPlatform
CallerComputeCircuit BreakerNetworkingDownstream ServiceCompute

A breaker in front of a downstream call trips open after repeated failures, failing fast and letting the dependency recover instead of every caller queueing up on a timeout, then half-opens to test recovery.

When to use it

  • A downstream dependency occasionally becomes slow or unavailable
  • Cascading timeouts from that dependency risk exhausting the caller's own threads/connections

Trade-offs

  • Requests fail fast (with a fallback or error) while the breaker is open, even if the call might have succeeded
  • Thresholds need tuning per dependency or the breaker trips too eagerly or too late

Components used

Managed App ServiceService Mesh

How it works

  • A proxy tracks failures to a dependency. Under a threshold it stays closed and calls pass through normally.
  • Once failures exceed the threshold the breaker opens and calls fail immediately without attempting the network, freeing threads and connections.
  • After a cooldown it moves to half-open and permits a trial request. Success closes it; failure reopens it.

Used in the wild

  • Calls to third-party APIs that may degrade without warning.
  • Preventing cascading failure when one slow service exhausts callers' thread pools.
  • Giving a struggling downstream service room to recover instead of hammering it.

Good to know

  • Michael Nygard popularised it in Release It! (2007). The insight is that failing fast is kinder to both caller and callee than waiting on a timeout.
  • Timeouts alone are insufficient: a 30-second timeout under load still ties up every worker thread for 30 seconds. The breaker exists to stop the call being made at all.