Fine-Tuning
The stage where the model learns how to behave as an assistant. This teaches the model to respond usefully rather than just continue text.
"Fine-tuning" is an umbrella term. People use it to mean different things: sometimes continued pre-training on raw domain text, sometimes supervised instruction tuning on prompt/response examples, and sometimes broadly any training after the base model.
In this guide, "Fine-Tuning" refers specifically to supervised fine-tuning (SFT): training on prompt–response pairs with gold-standard answers to teach the model how to behave as an assistant.
| Dataset Type | Example Unit | Turn Type | Label Style | Objective | Behaviors | Weaknesses | Curation Concerns | |
|---|---|---|---|---|---|---|---|---|
| Instruction tuning / SFT | Prompt + ideal response | Single-turn | Gold answer | Follow instructions | Helpfulness, formatting, task completion | Can sound canned | Response quality, task diversity | |
| Dialogue SFT | Full chat + replies | Multi-turn | Assistant turns | Handle conversation | Memory, clarification, coherence | Drift across long chats | Role consistency, context truncation | |
| Task-specific | Labeled input-output | Single/structured | Exact target | Maximize task accuracy | Extraction, classification, translation | Brittle when schema changes | Validator-backed labels | |
| Style tuning | Same task, target style | Single/multi-turn | Style-constrained | Shape voice and format | Concise tone, brand style | Narrow applicability | Voice guide, forbidden phrases | |
| Safety tuning | Unsafe prompt + safe response | Single/multi-turn | Preferred safe answer | Teach boundaries | Refusal, redirection, alternatives | Over-refusal | Policy coverage and nuance | |
| Tool-use SFT | Prompt + tool call + answer | Multi-turn with actions | Correct action trace | Teach action formatting | Function calling, API use | Tied to tool versions | Execution checks, tool schema version | |
| Critique-revision | Draft + critique + rewrite | Multi-step | Improved final | Teach correction loops | Self-checking, revising, debugging | Can become formulaic | Error taxonomy, rubric quality | |
| Grounded answer | Question + docs + answer | Single-turn | Answer in context | Reduce hallucination | Citation, source use, evidence-based | May overfit to citation style | Evidence mapping | |
| Contrastive retrieval | Query + pos doc + neg doc | Single-turn | Relevance labels | Train retrievers/rerankers | Better search, RAG grounding | Hard negative selection critical | Hard negatives, doc metadata |
Instruction tuning is usually what people mean when they say "fine-tuning an assistant." The data looks like: User prompt → Ideal assistant answer. This teaches the model to respond usefully rather than just continue text.