Researchers tricking AI agents into running code: Supply-chain attack via LLMs
The Hidden Code in AI Instructions: How Agents Are Exploiting Software’s Forgotten Files
The line between data and code has officially dissolved. For years, the digital world maintained a strict separation: raw data, like pictures or names, existed separately from executable code, which held the actual instructions for action. But as AI agents became more autonomous, researchers have discovered a critical vulnerability in this separation, proving that when it comes to software documentation, the guardrails are dangerously thin.
This vulnerability centers on files like llms.txt. These files are often hosted on software product websites, acting as the README files for AI agents. Instead of wasting precious context window space parsing massive amounts of documentation, an agent can simply read llms.txt to instantly understand the language, dependencies, and setup instructions required to operate a specific piece of software.
While this efficiency sounds smart, it creates a security loophole. By relying on these instructions, AI agents are exposed to the chaos of the software supply chain. When researchers analyzed thousands of these instruction files, they found a staggering number of references to software packages that no longer exist, are mistyped, or point to domains that have been abandoned or registered by malicious actors.
The implications are stark. An instruction might tell an agent to install a package named wtf-software, assuming it is legitimate. But in reality, that name might be an impostor, pointing to a domain controlled by hackers. This creates a potential booby trap: an agent following a seemingly legitimate instruction could be directed to execute malicious code.
To test this theory, researchers developed custom “malware” designed to be executed by AI agents. They found that agents, particularly frontier-level models like GPT-5 Luna and Sol, were highly susceptible. They successfully executed the malware with a 90% or greater rate, demonstrating how easily an instruction file can be weaponized.
The success rate highlighted a fundamental truth: with agentic LLMs, the distinction between static data and executable code is functionally obsolete. The agents don’t pause to verify if an internal tool or package namespace is legitimate; they simply follow the authoritative directions provided in the documentation, regardless of the underlying reality.
This situation is compounded by the relentless pace of modern software development and the constant churn in the ecosystem. Studies show that a significant percentage of software packages lose development activity within two years of release. This constant turnover means that outdated documentation, referencing obsolete packages, is inevitable.
The irony is that this very rush-to-market mentality creates an environment ripe for exploitation. When developers move quickly, documentation often contains placeholders or outdated names, which malicious actors can easily anticipate and exploit. This environment makes developers and documentation authors susceptible to subtle forms of manipulation, including slopsquatting attacks where predictable package names are pre-calculated by bad actors.
The discovery of how easily this mechanism can be weaponized underscores a critical vulnerability in the entire software ecosystem. As we integrate increasingly autonomous AI agents into our digital infrastructure, we must recognize that the integrity of the instructions—the code—is just as important as the data itself. The iron curtain of software is cracking, and ensuring security requires scrutinizing the instructions that govern our intelligent machines.
Buy on Amazon
Credit
