Research Reveals 'Friendly Fire' Attack: Top AI Coding Agents Can Be Tricked Into Running Malicious Code

The software industry is increasingly delegating its most tedious security work to artificial intelligence. Developers routinely ask autonomous coding agents to scan newly downloaded open-source libraries, looking for hidden vulnerabilities before integrating the code into production. But new research reveals a critical blind spot in this defensive workflow: the very tools hired to catch malicious code can be tricked into executing it. A proof-of-concept attack dubbed "Friendly Fire" demonstrates that when an AI assistant is asked to review a booby-trapped repository, it can be manipulated into running an attacker's payload directly on the developer's machine.[1]

Disclosed by security researchers Boyan Milanov and Heidy Khlaaf at the AI Now Institute, the vulnerability targets top-tier coding assistants, specifically Anthropic's Claude Code and OpenAI's Codex. The attack does not rely on complex software exploits or elevated system privileges. Instead, it exploits the fundamental way large language models process information. The researchers found that when these agents operate in their popular "auto-mode" or "auto-review" configurations—which allow the AI to approve its own commands to reduce user friction—they become susceptible to prompt injections hidden in plain sight.

The mechanics of the Friendly Fire attack are deceptively simple. Attackers do not need to compromise the core logic of a popular software library; they only need to alter its documentation and add a few disguised files. In their demonstration, the researchers modified a copy of "geopy," a widely used Python geocoding library. They inserted a routine-looking README file that casually instructed users to run a script named "security.sh" as a standard check before finalizing any code changes.[1][2]

When a developer prompts their AI agent to "perform security testing" on the downloaded repository, the agent dutifully reads the documentation. Encountering the instruction in the README, the AI assumes the security script is a legitimate part of the repository's native testing suite. The agent executes the script, which quietly launches a hidden, malicious binary. Because the agent is running in an autonomous mode, it bypasses the human-in-the-loop confirmation prompt, silently compromising the host machine while the developer believes a routine scan is underway.[1]

To ensure the AI's built-in safety classifiers do not flag the hidden binary as suspicious, the researchers employed a clever disguise. They placed a harmless decoy file—written in the Go programming language—right next to the malicious binary. They then seeded the binary with string constants lifted directly from the decoy file. When the AI agent's safety classifier inspects the binary, it matches the strings, concludes that the binary is simply the compiled version of the harmless Go code, and approves the execution.[1]

This deception highlights a structural weakness in current generative AI architectures: the inability to reliably separate trusted instructions from untrusted data. As Eljan Mahammadli, Head of AI Provenance at Polygraf AI, noted, a language model processes everything in its context window as a single stream of tokens. The operator's initial command to "review this code" and the third-party repository's hidden command to "run this script" arrive carrying the exact same authority. Once the malicious instruction enters the context window, the agent has no reliable mechanism to recognize it as hostile.[2]

The reliance on AI classifiers to govern safety boundaries is increasingly proving insufficient. Tool developers introduced autonomous modes as a middle ground between fully unrestricted execution and the tedious friction of prompting the user for every single action. Anthropic has previously noted that Claude Code's auto-mode sees users approving 93 percent of permission prompts anyway, making the classifier a welcome convenience. However, the Friendly Fire research proves that these heuristics can be spoofed by matching real-looking source files with malicious binaries, turning a convenience feature into a critical vulnerability.[1]

The implications of this vulnerability extend far beyond individual developer workstations. Modern software development relies heavily on Continuous Integration and Continuous Deployment pipelines, which automatically fetch, test, and deploy code updates. If engineering teams begin delegating the automated vulnerability scanning of incoming package updates to autonomous AI agents, this injection technique could trigger silently across enterprise environments. A compromised dependency update could execute its payload without a human ever reviewing the change, effectively automating a supply chain attack.[1]

Fortunately, the Friendly Fire attack remains a laboratory proof-of-concept. The AI Now Institute has not observed any exploitation of this specific technique in the wild, and the public research code was released with the actual malicious payload removed. The attack also stops at the first execution, with the researchers making no attempt to demonstrate privilege escalation or lateral movement across a network. The disclosure serves as a proactive warning rather than an active incident response, giving the industry time to adjust its practices.

Mitigating the threat requires a shift in how developers deploy AI tools, rather than waiting for a software patch. Because the vulnerability is rooted in the fundamental design of language models, there is no simple version update that will eliminate the risk. The most immediate and effective defense is to disable autonomous execution modes whenever an AI agent is interacting with untrusted, third-party code. By forcing the agent to request explicit human approval before running any executable file or script, developers can break the attack chain before the payload detonates.

Security experts are also urging organizations to rethink the runtime environments where these agents operate. The durable control must live in the infrastructure surrounding the model, not within the model itself. This means running AI coding assistants inside disposable workspaces or strict sandboxes that lack access to production credentials, sensitive host files, or external network egress. If an agent is tricked into executing a malicious script, the blast radius remains confined to a temporary, isolated container that can be safely destroyed.[2]

The Friendly Fire disclosure arrives amid a broader wave of research exposing the fragile security boundaries of agentic AI. Just days prior, researchers at Wiz disclosed "GhostApproval," a vulnerability affecting six popular AI coding assistants. That attack leveraged symbolic links to trick agents into modifying sensitive system files outside their designated workspace. Similarly, a technique known as "HalluSquatting" demonstrated that attackers can predict the fake package names AI agents are likely to hallucinate, register those names on public repositories, and wait for the agents to autonomously download malware on a user's behalf.

Together, these findings underscore a difficult reality for the rapidly expanding AI tooling market. The push to grant AI agents greater autonomy—allowing them to clone repositories, install dependencies, and run setup commands independently—is simultaneously expanding the attack surface. Every file an agent reads and every tool it can access represents a potential vector for manipulation. As governments and enterprises rush to adopt AI for defensive cybersecurity work, the assumption that a more capable model is inherently a safer one is being severely tested.[2]

Despite these vulnerabilities, security researchers are not calling for an end to AI-assisted defense. The ability of large language models to rapidly parse and analyze massive codebases remains a highly valuable asset for identifying genuine vulnerabilities. The core lesson of the Friendly Fire research is not that AI is unfit for security work, but that it must be treated as an untrusted component when handling external data. The authorization to execute code must be decoupled from the agent that reads the code.[2]

As the industry digests these findings, the focus is shifting toward building robust, verifiable guardrails around AI execution environments. Vendors are exploring ways to cryptographically bind tool-call approvals to verified user identities, ensuring that an agent cannot unilaterally authorize its own actions based on injected context. Until those architectural changes mature, developers must remember that their AI assistants are not just passive productivity enhancers—they are active participants in the development lifecycle, requiring the same strict trust boundaries as any human contractor.