Research Notes

T5 and Text-to-Text

By Satwik · January 22, 2026

T5 (Raffel et al., 2019) unified NLP under one deceptively simple frame: every task is text-to-text. Translation, classification, summarization, question answering, even regression are all cast as feeding the model a text string and training it to produce a text string. One model, one loss, one decoding procedure, for everything.

Why it mattered

The reframing removed the zoo of task-specific heads and output formats. A sentiment task becomes "produce the word positive or negative"; a similarity score becomes text digits. This uniformity let the authors run a large, systematic study, the real substance of the paper, comparing pretraining objectives, architectures, datasets, and scales on equal footing. That empirical rigor was as valuable as the framing itself.

T5 also introduced the C4 corpus, a cleaned crawl of the web, and used a span-corruption pretraining objective. Scaled up, it reached state-of-the-art across many benchmarks, but the enduring contribution is methodological: a common interface makes fair comparison possible.

Reading it now, the text-to-text frame is the ancestor of the instruction-following interface that dominates today, where you simply describe what you want in text. That is powerful and, from a security standpoint, precarious. When the interface for every task is free-form text, the boundary between data and instruction dissolves entirely, which is the root condition for prompt injection. T5 did not create that risk, but its elegant unification made the porous data-instruction boundary the standard way we interact with models.