sydepsystem design atlas

Fine-Tuning Pipeline

Curate a training set, fine-tune a base model, and register the result.

AdvancedAI / ML
Curated DatasetStorageFine-tuning JobAI / MLEvaluationComputeModel RegistryAI / ML

Labelled examples are curated and validated, used to fine-tune a base model, evaluated against a held-out set, and -- if it clears the bar -- registered as a new versioned model ready for serving.

When to use it

  • A base model's generic behaviour needs specialising to a narrow domain or house style
  • You have (or can generate) enough quality-labelled examples to fine-tune meaningfully

Trade-offs

  • Requires an ongoing data curation and evaluation discipline, not a one-off run
  • A fine-tuned model needs re-training as the base model or requirements evolve

Components used

Object StorageModel TrainingBatch / Scheduled JobModel Registry

How it works

  • Raw examples are curated, filtered and split into train and eval sets. This curation step dominates final quality far more than any hyperparameter.
  • A training job fine-tunes a base checkpoint, then an evaluation gate compares the result against the incumbent on a held-out set.
  • Passing models are written to a registry with their dataset version and metrics, so any deployed model can be traced back to exactly what it learned from.

Used in the wild

  • Teaching a model a house style, tone or strict output format that prompting keeps failing to enforce.
  • Compressing a large expensive model into a small fast one for a narrow task.
  • Domain adaptation for specialist vocabulary — legal, clinical or industrial jargon.

Good to know

  • Fine-tuning teaches form far more reliably than it teaches facts. If the goal is up-to-date knowledge, RAG is usually the correct tool and fine-tuning is an expensive detour.
  • LoRA and similar adapter methods train a small number of extra weights instead of the whole network, which is why fine-tuning stopped requiring a cluster around 2021.