Customisation: fine-tuning and when not to
The escalation ladder’s top rung — expensive, powerful, and reached far less often than assumed.
Fine-tuning has gravity: it sounds like the serious option, so teams reach for it first and discover expensively that it was the third-best tool for their problem. This chapter is the escalation ladder with prices attached — and the mechanics, for the genuine top-rung cases.
The ladder, with an honest toll booth
Prompting changes in minutes and costs nothing but tokens. Retrieval changes what the model reads — hours to improve, and the only correct answer when the problem is knowledge (facts, freshness, your private data). Fine-tuning changes what the model is — days, datasets, regression risk, and a model you now operate. The toll booth question, asked with evals in hand: is the model failing on knowledge, or on behaviour? Missing facts → retrieval, full stop (fine-tuning is a terrible database: lossy to write, impossible to update, prone to confident blur). Wrong form — format drift despite good prompts, a voice that will not hold, a narrow task where a small model must match a big one, prompts so long that baking the behaviour in pays for itself in tokens — that is the fine-tuning shape.
What tuning actually is
Supervised fine-tuning continues training on your examples — prompt/ideal-response pairs — nudging weights toward your distribution. Everything the deep learning chapter said applies, sharpened: quality dominates quantity (500 excellent, consistent examples beat 50,000 scraped ones — the model learns your annotators’ disagreements as faithfully as their intentions), and overfitting now has a new face called catastrophic forgetting: tune hard on your narrow task and general capability quietly erodes — which is why the eval suite must cover what you did not train, not just what you did. LoRA made all this affordable: freeze the base model, train small low-rank adapter matrices alongside — a fraction of the memory, adapters you can swap per task over one shared base. It is the default; full fine-tuning is the exception that needs a reason.
The alignment layer, briefly
Base models complete text; assistants behave. The gap is closed by preference tuning — RLHF and its simpler successor DPO: train on human choices between candidate outputs, shaping tone, helpfulness and refusal. Karpathy’s deep-dive in the reading walks the full pipeline. You will mostly consume this layer rather than build it, but knowing it exists explains model behaviour — including why a model sometimes refuses oddly or hedges — better than any prompt archaeology. Distillation rounds out the kit: generate outputs from a large model on your task, fine-tune a small one on them — how narrow tasks get big quality at small-model prices.
Where it bites
- Fine-tuning to teach facts — six weeks to build a worse, staler retrieval system.
- No held-out evals before tuning — you cannot measure what the tune bought or broke.
- Inconsistent training data — the model becomes fluently inconsistent at scale.
- Forgetting the operational tail: every tuned model is a deployment you now version, eval, serve and retire.