Your AI's fake package is real now

// the names repeat

A 2024 study (arXiv 2406.10279, to appear at USENIX Security 2025) generated 576,000 code samples with 16 popular code models, in Python and JavaScript, and checked every package the code needed against PyPI and npm. Then it asked a sharper question: if a prompt produced an invented name once, does it produce it again?

"We randomly sampled 500 prompts that generated package hallucinations during our initial testing and then repeated those queries 10 times per prompt." … "43% of hallucinated packages were repeated in all 10 queries, while 39% did not repeat at all across the 10 queries."

That repeat test was Python only, on four models (GPT-4 Turbo, GPT-3.5, CodeLlama 7B and DeepSeek 6.7B). The split matters both ways. Many names repeat, which is what makes them findable. Many do not, and the paper adds that "81% of hallucinated packages are generated by only one model", so these are not one universal list. They are names someone can collect by asking a given model the same kind of question.

One more number for scale, and its caveat: across all 16 models, "440,445 (19.7%)" of 2.23 million suggested packages did not exist, "including 205,474 unique non-existent packages". That one-in-five is pooled and weighted toward open models. The GPT models tested came in at 5.2% (about one in twenty), and GPT-4 Turbo was lowest at 3.59%. The models were the leaders "as of January 20, 2024"; Claude was not tested, and today's assistants should not be assumed to invent names at these rates.

Why "it exists" proves nothing

The paper spells out the attack, and the qualifier is the point:

"An adversary can exploit package hallucinations, especially if they are repeated, by publishing a package to an open-source repository with the same name as the hallucinated or fictitious package and containing some malicious code/functionality."

So the check most people reach for, "does this package exist?", is the one the attacker made true in advance. The authors say so directly: cross-checking a suggested name against a list of known packages is "ineffective … as an adversary may already have published the hallucinated package with malicious code", and "the mere presence of a package in an open-source repository does not confirm its credibility."

These are mostly not typos of real packages, either. Only "13.4% (10,263 of 76,489)" sat one or two characters away from a real name. Someone looking for them does not guess misspellings; they ask the model.

The decoy: an empty box with downloads

In March 2024, Bar Lanyado at Lasso Security described a test he ran on one invented name, huggingface-cli:

"I have decided to upload an empty package by the same name and see what happens." … "In three months the fake and empty package got more than 30k authentic downloads!"

He uploaded a dummy control package as well, to estimate how many downloads were just scanners. He also found the name in the wild: "instructions for installing this package can be found in the README of a repository dedicated to research conducted by Alibaba." Nothing here says that company was compromised; the package was empty. What it shows is that a name with no maintainer behind it can collect real installs and turn up in someone's install steps.

Two honest caveats. The downloads are his count, not shown to be AI-driven: in 2024 Hugging Face's own docs installed huggingface_hub[cli] and the command it gave you was huggingface-cli, so a human could easily type the wrong thing too (today the CLI is hf, shipped with the core huggingface_hub package). And the decoy is gone: the PyPI entry for huggingface-cli returns 404 as of 2026-10-01. Do not install it, or any name you saw only in a chat.

The fix: the tool's own official docs

The only check that counts is where the name came from. Take the install line from the tool's own official docs, the maintainer's install page, never from the chat and never from a third-party README (a third-party README is exactly where the fake name above turned up).

# not this: the name came from the chat $ pip install quickcsv-tools # illustrative name, not a real package # this: the line on the maintainer's install page $ pip install -U "huggingface_hub" # Hugging Face CLI guide, checked 2026-10-01

This matters more now than in 2024 for one reason: coding agents run installs for you. If an agent installs straight from its own suggestion, the check above never happens. The habit is the same either way: the name comes from the official docs.

Where this breaks

Old models. The study's models are from early 2024 and Claude was not tested. The repeat rate is a property of those models on those prompts.
Not every name repeats. 39% never came back in ten tries, and Lasso's own repeat rates were lower (19.6% for GPT-4 and 13.6% for GPT-3.5, on 20 questions asked 100 times each).
Two registries. The paper covers PyPI and npm. Go modules and .NET work differently; say "registries like npm and PyPI".
No documented compromise. No real attack through an invented name is documented in these sources, and the Lasso package was empty. "The code is theirs" is the risk, not a reported incident.

Related: why models invent things in the first place; prompt injection, the other way a stranger's text reaches your agent; hiding your .env from Claude Code.

Sources: Spracklen et al., "We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs", arXiv 2406.10279v3 (USENIX Security 2025) · Bar Lanyado, "Diving Deeper into AI Package Hallucinations", Lasso Security, 2024-03-28 · Hugging Face huggingface_hub CLI guide, v0.22.2 and current · PyPI JSON API for huggingface-cli (404). All verified 2026-10-01.

One concept a week. Free.

The deeper, copy-paste version of each ToolCall short — in your inbox.

// total: 0.00 · spam: void · unsubscribe: one click