Retrieval-Augmented Generation (RAG)
Ground an LLM's answers in retrieved, up-to-date, private documents.
IntermediateAI / MLData
Documents are chunked and embedded into a vector database ahead of time. At query time, the user's question is embedded, the most relevant chunks are retrieved and injected into the prompt, and the model answers grounded in that context instead of parametric memory alone.
When to use it
- Answers must be grounded in private, proprietary or frequently changing documents
- You need citations/traceability back to source content, not just a generated answer
Trade-offs
- Answer quality depends heavily on chunking and retrieval quality, not just the model
- Adds an ingestion pipeline and a vector database to keep in sync with source content
Components used
Object StorageIngestion / ETL PipelineVector DatabaseSingle-Page AppAI GatewayLLM / Model Endpoint
How it works
- Documents are chunked, embedded into vectors and stored in a vector index ahead of time. This ingestion path runs on its own schedule, independent of query traffic.
- At query time the question is embedded with the same model, the nearest chunks are fetched, and those chunks are pasted into the prompt as context before the LLM answers.
- The model is never fine-tuned on your data. It only ever sees the retrieved snippets, which is why answer quality is bounded by retrieval quality rather than model size.
Used in the wild
- Support assistants that must answer from current product documentation rather than year-old training data.
- Internal search over wikis, tickets and design docs where users want an answer, not ten blue links.
- Any regulated domain where every claim must be traceable to a source document you can cite.
Good to know
- The term comes from a 2020 Facebook AI paper by Lewis et al. The original design jointly trained the retriever and generator; almost nobody does that now — modern RAG bolts a frozen LLM onto an off-the-shelf vector store.
- The single most common failure is not the model but the chunker. Splitting on a fixed token count cuts tables and code blocks in half, and no amount of prompt engineering recovers the lost context.
Related patterns
Agentic Tool-Use Loop
An agent plans, calls tools, observes results, and iterates to a goal.
Vector Search + Rerank
Cheaply retrieve a broad candidate set, then precisely re-rank the top results.
Model Serving with A/B Testing
Route inference traffic across model versions to compare live performance.
Fine-Tuning Pipeline
Curate a training set, fine-tune a base model, and register the result.