Malicious Model Uploads on Model Hubs
By Satwik ยท May 26, 2026
Beyond behaviorally poisoned weights, model hubs face a more direct threat: models whose files execute code when loaded. Several security teams, including JFrog and Protect AI, reported through 2024 that public repositories on Hugging Face contained models carrying malicious payloads. The most common vector abused Python's pickle serialization format, which many model files use. Deserializing a pickle can run arbitrary code by design, so a crafted model file can open a reverse shell or run commands the moment a victim loads it with a standard call.
Researchers found real uploads attempting exactly this, some establishing outbound connections to attacker-controlled addresses on load. Because loading a model feels routine and trusted, developers often do it without the caution they would apply to running an unknown executable, which is effectively what an unsafe pickle is.
Why it matters: the risk is not hypothetical model misbehavior but immediate code execution on the machine that loads the model, often a developer workstation or a training server with broad access. The AI supply chain thus inherits classic deserialization vulnerabilities at scale.
The defensive lesson is to treat model files as untrusted code. Prefer safe serialization formats such as safetensors that carry only tensor data and cannot execute code. Scan model artifacts with tooling that inspects for unsafe pickle opcodes, load unknown models in sandboxed or isolated environments, and pin to verified publishers. Hubs have added automated scanning and safetensors adoption, but the burden of not executing an unknown binary ultimately rests with whoever loads it.