Curating Training
Data for LLMs
How to build training datasets for each stage of LLM development: pre-training corpora, fine-tuning pairs, preference rankings, RL trajectories, and safety data. What to collect, how to structure it, and what to watch out for.
Generally, there are three categories of training data:
What phase of training is the data for? Pre-training, fine-tuning, preference learning, RL, or safety.
What shape is each example? Document chunks, prompt-response pairs, ranked answers, or action trajectories.
What should the model learn? Knowledge, helpfulness, safety, tool use, planning, or self-correction.
5 questions to figure out what kind of data you need and where to start.
Jump directly to pre-training, fine-tuning, preference, RL, or safety data.
Map a target behavior (safety, tool use, reasoning) to the dataset types that train it.
Every dataset type, format, signal, and behavior in one dense reference table.