InstructGPT and Instruction Following

By Satwik · February 22, 2026

InstructGPT was OpenAI's demonstration that a smaller, aligned model could be preferred by users over a much larger base model. Starting from GPT-3, they applied supervised fine-tuning on human demonstrations followed by reinforcement learning from human feedback to make the model follow instructions and be more helpful and truthful. Human raters preferred outputs from a 1.3B InstructGPT model over the 175B GPT-3 base model in many comparisons.

The result mattered because it decoupled usefulness from raw scale. Alignment tuning, not just more parameters, drove the perceived quality gap. This made instruction tuning the standard recipe and set the template that ChatGPT would later ride to mass adoption.

For safety work, InstructGPT is a double-edged artifact. On one hand it reduced some toxic and untruthful outputs relative to the base model and showed that human feedback could steer behavior at low cost. On the other hand it introduced the alignment tax and reward-hacking concerns, and it demonstrated that a thin tuning layer sits between a helpful assistant and a raw completion engine. That layer is exactly what jailbreaks target.

InstructGPT also formalized an operational pattern that has security consequences: a pipeline of human labelers producing preference data. Label quality, labeler incentives, and the instruction set itself become part of the trust base. If you can influence the feedback data, you can influence model behavior. The paper is short but it is one of the most consequential of the year for how deployed assistants are built.