The AI Supply Chain: Model Weights, Malicious Uploads, and Slopsquatting
By Satwik ยท July 1, 2026
The components an AI system depends on, model weights, hubs, datasets, and generated package names, form a supply chain with weak provenance and novel poisoning paths. Package hallucination and slopsquatting show how the model itself becomes a vector for installing attacker-controlled code.
The supply chain got a new layer
Traditional software supply-chain security worries about dependencies: the libraries, containers, and build tools you pull in and implicitly trust. AI systems inherit all of that and add new links: pretrained model weights, model hubs, training and fine-tuning datasets, embedding stores, and, most novel, artifacts that the model itself generates and a developer then trusts. Each link has weaker provenance than the code you write, and several have almost none.
The unifying weakness is trust by default. Teams download a multi-gigabyte weights file from a hub and load it, download a dataset and train on it, or paste model-suggested code and run it, all with far less scrutiny than they would apply to a pull request. Attackers have noticed.
Malicious model uploads and weight-format risk
Model hubs let anyone publish weights, and consumers pick models by name, popularity, and task fit, rarely by verified provenance. This enables two problems.
First, the serialization format can carry code. The Python `pickle` format, historically common for model checkpoints, executes arbitrary code on deserialization. A weights file is not inert data; loading a malicious pickle can run an attacker's payload on your machine, exactly the classic deserialization vulnerability, now wearing a `.bin` or `.pt` extension. Safer formats such as safetensors exist precisely to make weights a pure data blob with no code execution path, and using them is a baseline control. But many pipelines still load legacy formats without sandboxing.
Second, the weights themselves can be malicious even when the file format is safe. A published model can be fine-tuned to contain a backdoor: it behaves normally on ordinary inputs but produces attacker-chosen output when it sees a trigger phrase, or it carries a subtle bias, an inserted vulnerability generator, or a jailbroken disposition. You cannot easily audit a billion-parameter tensor for hidden behavior. Typosquatted model names ("meta-llama" versus a look-alike org) and re-uploaded "optimized" copies of popular models are effective distribution tricks. Downstream, a fine-tune of a poisoned base inherits the backdoor.
Dataset and pipeline poisoning
Upstream of weights sits data. Models trained or fine-tuned on web-scraped or community-contributed corpora can be poisoned by planting content designed to install a behavior or a backdoor. Because training data is vast and unaudited, a small volume of carefully crafted poison can survive into the model. RAG pipelines face the same risk at retrieval time: a poisoned document in the knowledge base is a supply-chain compromise of the context, not just a runtime injection. Embedding models and tokenizers pulled from hubs are dependencies too.
Package hallucination and slopsquatting
The most distinctly new link is the model as a code source. LLM coding assistants routinely suggest imports and install commands, and they hallucinate: they confidently recommend packages that do not exist. Studies of code-generating models have found a substantial fraction of suggested dependencies are non-existent, and, critically, that the same fabricated names recur across many generations. Hallucinations are not uniformly random; models gravitate to plausible, repeatable names.
That repeatability is the exploit. An attacker observes which non-existent package names a popular model tends to hallucinate, then registers those names on the public registry (PyPI, npm) with malicious code inside. This is slopsquatting: squatting on the "slop" that models reliably invent, as opposed to typosquatting real names. A developer asks an assistant for code, the assistant says `pip install <hallucinated-name>`, the developer runs it, and now attacker code executes in their environment and, via CI, potentially in production. The model has been turned into a delivery mechanism for a dependency the attacker planted in advance. Because the hallucination is reproducible, the attacker does not need to compromise anything; they just wait where the model points.
Defenses: provenance, pinning, and verification
The supply chain is secured the way all supply chains are, by replacing default trust with verified provenance and least privilege, adapted to the new artifacts.
For weights: prefer non-executable formats (safetensors) and refuse to load pickle from untrusted sources; if you must, deserialize in a sandbox. Verify integrity with hashes and, where available, signatures and attestations. Pin to specific, vetted model versions from known publishers rather than pulling "latest" from an arbitrary uploader. Scan model files with tooling that detects embedded code and known-malicious patterns, and evaluate models behaviorally for backdoors, including trigger and anomaly testing, before trusting them in sensitive roles.
For data: treat training and retrieval corpora as security-relevant, control who can contribute, track provenance of documents, and monitor for poisoning.
For package hallucination: never run model-suggested install commands blind. Verify that every suggested package exists, is the intended maintainer's, has a plausible history, and is pinned by version and hash in a lockfile. Maintain an internal allowlist or private mirror so developers install only vetted dependencies, and gate new dependencies through review. Configure assistants and CI to flag or block unrecognized packages.
Across all links, apply least privilege: the process that loads weights, runs generated code, or installs packages should have minimal access, so a compromised artifact cannot reach secrets or production. The AI supply chain is longer and murkier than the classic one, and the only durable answer is to stop trusting components you have not verified.