Medallion Lakehouse (Bronze/Silver/Gold)
Progressively refine raw data through bronze, silver and gold layers.
AdvancedData
Raw data lands untouched in a bronze layer, gets cleaned and conformed into silver, then aggregated into business-ready gold tables -- each layer independently reprocessable from the one before it.
When to use it
- Multiple teams need both raw and curated views of the same source data
- You want to reprocess history when transformation logic changes, without re-ingesting
Trade-offs
- Three storage layers and orchestration between them instead of one pipeline
- Gold-layer consumers are only as fresh as the slowest upstream layer
Components used
Event StreamData LakeIngestion / ETL PipelineStream ProcessingData WarehouseBI & Visualisation
How it works
- Bronze holds raw ingested data exactly as it arrived, append-only and never edited, so any downstream mistake can be reprocessed from source.
- Silver applies cleaning, deduplication, type enforcement and joins, producing a validated and queryable representation.
- Gold contains business-level aggregates shaped for specific consumers — dashboards, ML features, reports.
Used in the wild
- Analytics platforms consolidating many source systems with inconsistent quality.
- Regulated environments needing provable lineage from a report back to raw source data.
- Teams wanting warehouse semantics over cheap object storage rather than a proprietary warehouse.
Good to know
- The naming is Databricks marketing that stuck; the underlying staged-refinement idea long predates it and appears elsewhere as raw/refined/curated or staging/core/mart.
- Keeping bronze immutable is what makes the architecture forgiving. Teams that 'clean up' bronze to save storage discover they can no longer reprocess when a silver bug is found months later.
Related patterns
Retrieval-Augmented Generation (RAG)
Ground an LLM's answers in retrieved, up-to-date, private documents.
Vector Search + Rerank
Cheaply retrieve a broad candidate set, then precisely re-rank the top results.
Feature Store
Compute features once, serve them consistently to training and inference.
CQRS
Separate models and stores for writes and reads.