sydepsystem design atlas

Ad Click Aggregation & Analytics

Ingest high-volume click events and aggregate them for billing and reporting.

IntermediateCommerceData
Ad ServerEdge & IoTClick StreamMessagingDedup & AggregateAnalytics & DataBilling CountersStorageAnalytics WarehouseAnalytics & DataReportingAnalytics & Data

Clicks are ingested at high volume through a stream, deduplicated and aggregated into rollups for billing (advertisers pay per click) and separately loaded into a warehouse for slower, exploratory analytics.

When to use it

  • Click volume is high enough that per-click synchronous processing would not scale
  • Aggregated counts need to be both fast (for billing) and durable/queryable (for reporting)

Trade-offs

  • Deduplicating clicks (bots, double-fires) is inherently imperfect and needs ongoing tuning
  • Billing aggregates and warehouse totals can diverge slightly depending on when each was computed

Components used

Edge FunctionEvent StreamStream ProcessingKey-Value StoreData WarehouseBI & Visualisation

How it works

  • Click events arrive at enormous volume and are written straight to a durable log rather than a database, decoupling ingestion from processing.
  • A stream processor aggregates counts into time windows, while raw events are also archived for reprocessing and fraud investigation.
  • Serving reads precomputed rollups, because scanning raw events per query would be impossibly expensive.

Used in the wild

  • Advertising platforms billing per click or impression.
  • Product analytics and funnel measurement.
  • Any high-cardinality event counting where approximate live numbers and exact billing numbers are both required.

Good to know

  • Because clicks turn into invoices, at-least-once delivery is not acceptable on its own — duplicate counting means overbilling. Deduplication on an event id within a window is mandatory.
  • Click fraud means a meaningful share of traffic is not human, so the pipeline needs filtering before aggregation rather than after.