Research Notes

DeepSeek V2 and V3 and Efficient Scaling

By Satwik · May 10, 2026

DeepSeek released V2 in mid-2024 and V3 in December 2024, and together they reset expectations about how cheaply a frontier-class model could be trained. Both are large mixture-of-experts models with only a small fraction of parameters active per token. V2 introduced Multi-head Latent Attention, which compresses the key-value cache dramatically and cuts inference memory. V3, at 671 billion total parameters with roughly 37 billion active, matched leading models on many benchmarks while reportedly training for a small fraction of the compute budgets rumored for Western frontier runs.

Why it mattered: the efficiency story challenged the assumption that frontier capability required enormous, exclusive compute. Through architectural work, MoE, latent attention, and an auxiliary-loss-free load-balancing scheme, plus low-precision training, DeepSeek showed strong results on a leaner budget, and released the weights openly. That combination pressured both the economics and the geopolitics of the field.

The security and governance angle is layered. Open weights from any lab carry the recall-impossible, guardrails-removable properties we always flag. There is an added supply-chain and provenance dimension: organizations must reason about where a model was trained, what data and controls stood behind it, and what that means for sensitive deployments. Efficiency also lowers barriers broadly, which cuts both ways, democratizing capability for defenders and adversaries alike.

For our purposes, DeepSeek V2 and V3 are important less for any single benchmark and more for what they signaled: that algorithmic efficiency can substitute for raw compute faster than expected, compressing timelines and diffusing frontier capability. Any forecast that assumed compute would remain a durable moat had to be revised.