How 'Model Distillation' Became the AI Industry's Biggest Security Gray Area

The White House has leveled a severe accusation against one of China's leading artificial intelligence startups, alleging that Beijing-based Moonshot AI illicitly copied the capabilities of Anthropic's most advanced AI system. According to Michael Kratsios, Director of the White House Office of Science and Technology Policy, Moonshot utilized a technique known as "covert industrial distillation" to train its newly released Kimi K3 model on Anthropic's proprietary Fable model. The accusation highlights a growing crisis in the AI industry: the realization that a company's most valuable intellectual property can be systematically extracted without anyone ever breaching a server or stealing a line of code.[1][6]

The controversy centers on Moonshot AI's Kimi K3, a massive 2.8-trillion-parameter open-source model released in mid-July 2026. Upon its debut, Kimi K3 shocked the industry by benchmarking neck-and-neck with the most powerful proprietary systems from US developers, including Anthropic and OpenAI. For a three-year-old startup to match the capabilities of labs backed by billions in Big Tech funding raised immediate questions about how the model was trained. The US government now claims the answer lies in model distillation, a process where a newer AI learns by copying the inner workings and reasoning patterns of a more mature system.[1][2][6]

In traditional machine learning, model distillation is a standard and legitimate practice. It involves taking a massive, highly capable "teacher" model and using it to train a smaller, more efficient "student" model. The teacher model generates millions of high-quality answers, reasoning traces, and coding solutions, which are then fed into the student model as training data. When a company does this internally with its own models, it is considered one of the smartest ways to ship a fast, cost-effective AI product.[3][4]

However, the technique becomes highly controversial—and potentially malicious—when applied to a competitor's proprietary system. In a "black-box distillation attack," the attacker does not need access to the target model's underlying weights, source code, or internal architecture. Instead, they simply open an account and query the target model's public Application Programming Interface (API) at a massive scale. By feeding the target model millions of carefully crafted prompts and recording the responses, the attacker can essentially outsource their data labeling and capability development to their rival.[3][4][6]

The economics of this strategy are overwhelmingly favorable to the attacker. Training a frontier AI model from scratch requires hundreds of millions of dollars in specialized compute hardware and painstaking human data curation. In contrast, distilling a competitor's model via API queries might cost only a few hundred thousand dollars in usage fees. To the victim company, the attack often looks indistinguishable from a highly active, paying enterprise customer utilizing the service exactly as designed.[3][4][6]

White House officials were quick to draw a line between standard industry practices and the actions allegedly taken by Moonshot AI. Kratsios noted that while legitimate AI distillation plays a vital role in creating efficient models, "large-scale, covert industrial distillation aimed at stealing proprietary U.S. technology and undermining American research is unacceptable." Under Secretary of State Jacob Helberg echoed the sentiment, describing the incident as an assault on economies that rely on fair and honest competition.[1]

Despite the strong rhetoric from Washington, the legal reality of model distillation is incredibly murky. Legal scholars point out that distillation sits in an uncomfortable space between imitation and outright appropriation. Because the attacker never copies files, extracts weights, or duplicates source code, traditional intellectual property laws struggle to classify the behavior as theft. Copyright law fundamentally protects the expression of ideas, not the ideas themselves or functional methods of operation.[5][6]

Experts often compare model distillation to reverse engineering, or to a student attending a brilliant professor's public lectures. If the student takes meticulous notes and later builds a condensed course based on what they learned, they haven't stolen the professor's brain—they have simply learned from observing their behavior. In the context of AI, the student model is statistically inferring patterns by treating the teacher model as a black box, a process that many argue does not meet the legal threshold for copyright infringement.[5]

With copyright law offering little protection, the real battleground has shifted to contract law. Most frontier AI developers, including Anthropic and OpenAI, have updated their Terms of Service to explicitly prohibit users from using their model's outputs to train competing artificial intelligence systems. While this gives companies grounds to terminate accounts and potentially sue for breach of contract, enforcing these terms globally—especially against state-backed or international competitors—is notoriously difficult.[4][5][6]

The Moonshot AI incident is not the first time US labs have faced this threat. In recent years, Anthropic, Google, and OpenAI have all reported variations of distillation attacks originating from competing labs. In one notable case, Anthropic alleged that attackers used approximately 24,000 fraudulent accounts to conduct 16 million API exchanges in an attempt to extract reasoning traces from its Claude models. As these attacks become more sophisticated, attackers are developing internal platforms that quickly switch between multiple access methods to evade detection.[1][3][6]

To protect their billion-dollar investments, AI companies are racing to develop technical countermeasures. Security teams are implementing behavior-aware rate limiting to identify accounts that query the API in patterns indicative of automated dataset generation. Developers are also experimenting with subtle cryptographic watermarks embedded in the text outputs, which can later be detected if a competitor's model begins reproducing those specific linguistic quirks. However, determined attackers continually find ways to strip these watermarks or obfuscate their queries.[4][6]

The geopolitical stakes of the Moonshot AI dispute are particularly high, coming just weeks after the US Commerce Department imposed unprecedented export controls on Anthropic's Fable 5 and Mythos 5 models. In June 2026, the US government forced Anthropic to suspend global access to its top models for all foreign nationals, citing severe national security concerns. The revelation that a Chinese lab may have bypassed these restrictions by distilling the model's capabilities through API access highlights the immense difficulty of containing digital intelligence.[1][6]

As the AI industry moves forward, the line between learning and stealing will remain one of its most contentious debates. For policymakers, the challenge is to strengthen guardrails against industrial-scale model theft without outlawing the legitimate distillation techniques that allow open-source developers to build smaller, more accessible AI tools. Until a clear legal framework emerges, frontier AI labs will have to accept that their most dangerous competitors might also be their highest-paying API customers.[1][4][6]