sydepsystem design atlas

Scatter-Gather

Broadcast a request to several services, then merge their replies.

IntermediateMessagingPlatform
CoordinatorNetworkingProvider AComputeProvider BComputeProvider CComputeResponse AggregatorCompute

A coordinator sends the same request to multiple services in parallel, waits for their responses, and combines them into a single result -- the shape behind price comparison, multi-provider search and quorum reads.

When to use it

  • A single answer requires combining results from independent services
  • Latency should track the slowest required reply, not the sum of all of them

Trade-offs

  • Overall latency is bounded by the slowest responder unless you time out partial results
  • Partial failures need an explicit policy: wait for all, quorum, or best-effort

Components used

API GatewayServerless FunctionWorkflow Orchestrator

How it works

  • A coordinator broadcasts one request to several services in parallel and collects their responses.
  • Results are merged into a single reply once all have answered or a timeout fires.
  • Total latency is that of the slowest responder, not the sum — which is the reason to do it at all.

Used in the wild

  • Travel or price comparison querying many suppliers at once.
  • Federated search across several independent indexes.
  • Dashboard endpoints assembling data from multiple owning services.

Good to know

  • Partial failure is the design question, not an edge case. Decide up front whether a missing responder means a degraded result or a failed request.
  • The tail latency problem is brutal: with ten parallel calls each having a 1% chance of being slow, roughly one in ten requests hits at least one straggler.