Security teams have spent a decade learning that a package install runs code. The same lesson applies to model weights, and it is less widely absorbed. A data scientist who would never pipe a script from the internet into a shell will download a checkpoint from an unfamiliar account and call torch.load on it, because it feels like opening a file of numbers. For pickle-based formats, it is closer to running a program.
How a pickle turns into code execution
Python's documentation opens its pickle page with a warning that the module is not secure, that you should only unpickle data you trust, and that malicious pickle data can execute arbitrary code during unpickling [1]. The reason is the object-reconstruction protocol. A class can define __reduce__, which returns a callable and a tuple of arguments; when the pickle is loaded, the unpickler calls that callable with those arguments to rebuild the object [1].
An attacker does not need a real class. They write a pickle whose reduce step names a function such as os.system or builtins.exec, with a shell command or Python source as the argument. Hugging Face's explanation of the format shows the mechanics at the opcode level: GLOBAL and STACK_GLOBAL tell the unpickler to import something, and REDUCE tells it to call that thing with the arguments on the stack [2]. By default the unpickler will import any class or function it finds referenced in the data [1].
A .bin or .pt file from a model hub goes to torch.load or pickle.load, the __reduce__ step calls an attacker-chosen function, and code runs on your host as a shell or a beacon. Safetensors stores tensors only, so it has no step that can run code.
Timing is what makes this dangerous. The payload runs inside the load call, before your code inspects the model, before evaluation, before anything you would think of as using it. Wrapping the load in a try block does not help, because the code has already run by the time an exception could be raised.
What has been found on public hubs
In February 2024, JFrog reported finding around 100 malicious models on Hugging Face after excluding false positives, with PyTorch models the most common and TensorFlow Keras models close behind [3]. Their worked sample, a PyTorch model in a repository called baller423/goober2, used __reduce__ to open a reverse shell when loaded with torch.load(), launching /bin/sh or powershell.exe depending on the platform. The destination address belonged to a South Korean research network, which JFrog treated as possible evidence of researcher activity rather than proof of a criminal campaign [3].
JFrog also noted the limit of the hub's own control: Hugging Face's pickle scanning marks unsafe models but does not block them, so users can still download and run them [3]. Hugging Face says as much in its documentation, describing its pickle import scan as not 100% foolproof and making safety the user's responsibility [2]. A warning label is useful to a careful person. It does nothing for an automated job that downloads by repository name.
In February 2025, ReversingLabs described a technique it named nullifAI in two models on Hugging Face, glockr1/ballr7 and a second repository with a long string of zeros as its name [4]. Both were PyTorch files compressed with 7z instead of PyTorch's usual ZIP, so torch.load could not open them as-is. More importantly, both were deliberately broken pickles: the malicious code sat at the start of the stream, and because deserialization executes opcodes as it reads them, the payload ran before the parser hit the error [4]. ReversingLabs judged them more likely a proof of concept for testing the method than an active attack. Hugging Face removed them within 24 hours of the January 20, 2025 report and updated Picklescan to handle broken pickles [4].
Why scanners miss things
Model scanners face the same structural problem as antivirus: they look for known-bad patterns in a format designed to run code. ReversingLabs pointed to three specific weaknesses. Picklescan relied on a blocklist of dangerous functions, which they called not scalable or adaptable; it validated the file first and scanned second, so a broken file produced an error instead of a finding; and an unusual container format kept the scanner from seeing the pickle at all [4].
- Blocklists lag. Any importable function that can run a command or evaluate a string is a candidate, and Python's own documentation warns that even a restricted unpickler needs care because functions like builtins.eval can be abused [1].
- Parsers and loaders disagree. If a scanner refuses a malformed file but the loader executes it up to the point of failure, the attacker lives in the gap [4].
- Containers hide contents. A PyTorch checkpoint is an archive around a pickle; a scanner that does not unpack the archive the way the loader does sees nothing [4].
- Pickle is not the only carrier. JFrog's malicious set included Keras models as well as PyTorch ones, so a pickle-only scanner covers only part of the problem [3].
Scanning is still worth doing, because it catches the commodity samples and gives you a record. The mistake is to treat a clean scan as permission to load an untrusted pickle in a privileged environment. The control that does not depend on the scanner being right is to stop executing the format.
What PyTorch 2.6 changed, and what it did not
PyTorch 2.6.0, released in January 2025, flipped torch.load to use weights_only=True by default, a change listed as backwards incompatible in the release notes [5]. With that setting the unpickler is restricted to tensors, primitive types, dictionaries and any types explicitly allowed through torch.serialization.add_safe_globals() [6]. That removes the most direct path, where a checkpoint names os.system and the loader calls it.
It is a meaningful improvement with clear limits. Code pinned to PyTorch versions before 2.6 keeps the old default. Checkpoints that serialize a whole nn.Module instead of a state dictionary fail under the new default, and the usual workaround is to pass weights_only=False, which restores full pickle behavior for that call. The same documentation still says, in bold, never to load data from an untrusted source [6]. And joblib, widely used for scikit-learn models, is pickle-based and has no equivalent restriction.
| Loading path | Can loading run arbitrary code? | What to do |
|---|---|---|
| pickle.load on a .pkl file | Yes; the unpickler imports and calls what the file names [1] | Never on untrusted files; convert in an isolated job or reject |
| torch.load with weights_only=False | Yes; full pickle behavior [6] | Treat as code execution; allow only for files you produced |
| torch.load with weights_only=True (default from 2.6) | Restricted to tensors, primitives, dicts and allowlisted types [5][6] | Use it, keep add_safe_globals lists short, and still isolate untrusted files |
| joblib.load on a scikit-learn model | Yes; the format is pickle-based | Same handling as raw pickle |
| 7z-compressed or deliberately broken PyTorch file | The payload can run before the loader fails, and scanners may miss it [4] | Reject archives that do not match the expected container |
| safetensors load | No code step; the format stores tensors [7] | Preferred format for anything entering production |
A worked example: the fine-tune from a new account
Consider an illustrative health-tech company, Ostrander Health, whose ML team needs a named-entity model for clinical notes. An engineer finds a fine-tuned checkpoint on a public hub from an account created last month, with a convincing model card. The evaluation script runs on a shared GPU host that has a cloud role with read access to a training-data bucket and outbound internet for pulling dependencies.
The checkpoint is a full serialized module, so torch.load fails under the default and the engineer adds weights_only=False, the usual workaround. The load call executes a reduce step that starts a small background process. The process requests the host's cloud credentials from the instance metadata endpoint and posts them to a remote server, then exits. The model loads, evaluates reasonably, and the engineer moves on.
The blast radius is everything that GPU host's role can reach, which here includes training data that may contain patient records. Nothing about the model's accuracy would reveal the compromise. The signals were all upstream: an unapproved source, a pickle format, a disabled safe loader, and a host allowed to talk to the internet.
An intake pipeline that does not trust the file
The fix is to give model files the same path to production as third-party code: a single intake route, run by a service with no access to anything valuable, that produces a converted and pinned artifact that is the only thing other systems may load.
Models enter only from approved sources, are scanned for pickle opcodes, converted to safetensors, pinned and signed by hash, and loaded in isolation with no egress. Each stage removes a way the file could run code in a place that matters.
- Approved source: allow downloads only from named organizations or accounts you have reviewed, through a proxy or mirror, and block direct hub access from training and serving hosts. Hugging Face recommends loading from users and organizations you trust and relying on signed commits, which establish origin but not safety [2].
- Scan: disassemble pickle content without executing it, as Hugging Face's own scanner does with pickletools, and reject any import outside a short allowlist of tensor-rebuilding functions rather than looking for known-bad ones [2]. Reject containers that do not match the expected format.
- Convert: rebuild the weights as safetensors inside a disposable sandbox with no network and no credentials, using weights_only=True; conversion has to read the original file, so this is the one place an untrusted pickle is opened, and it must be a place where code execution gains nothing [6][7].
- Pin and sign: record the SHA-256 of the converted file, sign it in your artifact store, and make every consumer verify the hash before loading. A model reference in code should be a digest, not a repository name and a branch.
- Isolated load: serve and evaluate models on hosts with outbound network denied by default, no instance-metadata access unless required, and the narrowest cloud role that still works. If a payload ever slips through, it has nowhere to send what it finds.
The egress rule carries more weight than it looks. Both published payloads above were reverse shells to hard-coded addresses [3][4]. A model-serving host that can reach only its own artifact store and its callers turns a successful payload into a failed connection and an alert, which is a much better day than a quiet credential theft.
Checks to run this quarter
- Search your code for torch.load calls that pass weights_only=False, pickle.load and joblib.load on downloaded files, and pinned PyTorch versions below 2.6; each one is a code-execution call site to justify or remove.
- Inventory where models come from: list every hub organization, bucket and URL your pipelines pull from, and compare it with the approved list.
- Build a harmless test pickle whose reduce step writes a marker file, and confirm your scanner rejects it, your conversion sandbox contains it, and your serving hosts would never load it.
- Repeat the test with the payload placed before a deliberately corrupted opcode, the nullifAI pattern, and confirm the scanner reports a rejection rather than an error you ignore [4].
- From a model-serving host, try to reach an arbitrary internet address and the instance metadata endpoint. Both should fail unless you have a written reason.
- Confirm every production model reference is a hash and that a changed file with the same name fails verification.
The pipeline is verified when the test pickle never runs anywhere except the sandbox, and when a model swapped behind the same repository name is refused. Repeat the scanner tests whenever you upgrade the scanner or PyTorch, since both change what they accept.
Where a runtime gateway helps
Intertrace does not scan model files, and the intake pipeline above is yours to run. What the gateway adds is on the other side of the model: inference and service calls through it carry an itr_rt_* runtime key, so tenancy comes from the key, every decision is a structured event you can export to your SIEM, and the enforcer fails closed when it cannot decide. Discovery's read-only scans of cloud accounts and GitHub repositories surface AI services and AI usage in code and CI, which helps you find the hosts and pipelines that load models before you decide which of them need the egress rules above.
