sydepsystem design atlas

Canary Release

Send a small slice of traffic to the new version before a full rollout.

IntermediatePlatform
Traffic SplitterNetworkingStable (95%)ComputeCanary (5%)ComputeMetricsObservabilityAuto-rollback TriggerObservability

The router sends a small percentage of live traffic to the canary version while watching error rates and latency; the rollout ramps up automatically or is rolled back the moment metrics regress.

When to use it

  • You want real production signal on a new version before it affects everyone
  • Automated metrics (error rate, latency) can gate the rollout

Trade-offs

  • Needs solid per-version observability, or a bad canary goes unnoticed
  • Slower full rollout than an all-at-once deploy

Components used

Load BalancerContainer ServiceMetrics StoreAlerting & On-call

How it works

  • The new version is deployed alongside the old one and receives a small traffic share — often 1% to 5%.
  • Error rates, latency and business metrics are compared between cohorts. If the canary looks healthy, its share is increased in stages.
  • Any regression triggers an automatic rollback, and only the small exposed cohort was ever affected.

Used in the wild

  • High-traffic services where a bad release would otherwise affect everyone at once.
  • Changes whose risk is hard to assess in staging because production traffic is genuinely different.
  • Progressive delivery pipelines that promote automatically based on metrics rather than human approval.

Good to know

  • Named for the canaries carried into coal mines to detect carbon monoxide — the bird succumbed first, warning the miners.
  • At 1% traffic you need meaningful volume before any statistic is significant. On low-traffic services, a canary can look perfectly healthy simply because nothing exercised the broken path.