OpenAI Codex

By Satwik · February 6, 2026

Codex (Chen et al., 2021) was a GPT model fine-tuned on a large corpus of public code, primarily Python, and released with the HumanEval benchmark for measuring functional correctness. Rather than scoring generated code by text similarity, HumanEval runs it against unit tests and asks whether it actually passes, which set a more honest standard for code models. Codex could turn docstrings into working functions and became the engine behind developer tooling.

The paper is unusually candid about limits. Accuracy dropped on longer, multi-step specifications, and the model could produce subtly wrong code that looked right. Sampling many candidates and filtering raised the effective success rate but is not free.

Reading angle

Codex is a landmark for anyone reading with a security lens. The authors themselves discuss misalignment, the model sometimes writes bad code when its context contains bugs, essentially imitating the flawed style it is prompted with, and they flag the risk of generating insecure patterns. Because the training data is public code, including code with vulnerabilities, the model can reproduce those vulnerabilities. There are also concerns about license and provenance of suggested code. The broader point is that a code assistant sits inside the trust boundary of software development: its outputs get compiled, run, and shipped. Treat Codex-style suggestions as untrusted input requiring review, not as authoritative answers. The HumanEval methodology, execute and test rather than eyeball, is itself a good habit to carry into evaluating any code-producing system.