In the realm of technology, the concept of AI distillation has transitioned from a simple research concept to a burgeoning shadow economy that puts at risk the foundational business models associated with trillions in AI investments.
This technique operates by training one AI model based on the outputs generated by another. The boundaries of what's permissible in this practice are often blurred and heavily debated, as various AI firms incorporate model outputs from their competitors in different phases of their development.
Major US AI companies invest heavily in data, talent, and computing resources, aiming to create cutting-edge models that command high prices. If these models can be duplicated swiftly and inexpensively through distillation, the financial returns on such investments might dwindle significantly.
"Typically, AI companies distill other AI companies," remarked Elon Musk during a legal dispute with OpenAI earlier this year.
The implications of this technique could undermine the industry's profitability, allowing competitors to swiftly develop models that rival the performance of advanced systems produced by firms like Anthropic, OpenAI, and Google.
As Xiaoyin Qu, a former senior product manager at Meta, expressed on X, this situation can be frustrating for those who have invested time and resources into creating their models. He stated, "They hired the best talents, burned billions and built the newest model, only to have Chinese free models wiping out all your margins."
Distillation poses significant challenges to the economic landscape of AI. Anthropic recently highlighted these concerns, accusing Alibaba of employing unethical distillation practices by creating numerous fake accounts to extract its responses in large volumes.
"This distorts the economic rationale that supports American AI leadership, effectively transforming billions in research and development, computational resources, and other US investments into a subsidy for our overseas competitors," wrote Sarah Heck, Anthropic's head of policy, in a letter directed at leading US legislators.
OpenAI has also raised alarms, suggesting that distillation could eventually yield models that surpass current leading systems by merging outputs from various US models, allowing adversaries to reproduce and even enhance capabilities beyond any single original model.
Investor sentiment has turned cautious, as recent weeks saw a dip in AI stocks following the unveiling of new models from Chinese firms, including GLM-5.2 from Z.ai, a company previously known as Zhipu AI. Some AI researchers suspect that this model was enhanced through distillation of knowledge derived from American systems.
"Yes, they distilled Claude and GPT 5.5," commented Patrick Toulme, an AI chip software engineer at Google, referencing models from Anthropic and OpenAI.
Initially, distillation was a non-controversial concept. Back in 2015, prominent AI researchers at Google detailed a technique in which a group trained a smaller model using data from its own larger model.
However, with the advent of ChatGPT in 2022, the landscape shifted drastically, leading to a global race in AI development. Distillation has since evolved, moving beyond a company’s internal models to become a method for accelerating progress by leveraging the outputs of rival technologies.
Zhang Chi, an AI researcher who recently collaborated on large language models at ByteDance, noted in a podcast discussion that many Chinese AI firms depend significantly on distillation instead of developing their own high-quality training data. Instead of employing specialists to create detailed content, these companies utilize models like ChatGPT, Claude, or Gemini to generate responses that serve as training material.
On another podcast, Yao Shunyu from Google DeepMind mentioned that due to limited access to advanced AI chips, Chinese developers have turned distillation into a competitive edge. He categorized distillation into two streams—"dumb distillation," which merely replicates another model's outputs, and "smart distillation," where multiple AI systems generate, assess, and refine each other's responses to generate superior training data.
As translated from the podcast, Yao indicated that Chinese laboratories might take the lead in developing more sophisticated distillation techniques.
Earlier this year, during a legal inquiry, an attorney for OpenAI posed a question to Elon Musk about whether xAI had engaged in distilling OpenAI’s models. Musk acknowledged that this practice is common across the sector, responding "Partly" when pressed if xAI had participated in such actions.
Anthropic, OpenAI, and Google have repeatedly raised concerns regarding what they refer to as unethical distillation, accusing various Chinese firms—such as DeepSeek, Moonshot, MiniMax, and Alibaba—of misusing the technique and pushing for regulatory intervention.
Despite this outcry, American companies have also leveraged each other's advancements in improving their own models. For instance, Google has engaged Scale AI gig workers to refine ChatGPT answers as part of its effort to catch up with OpenAI.
Many scholars do not classify these practices as distillation. However, terms of service from AI labs prohibit using their platforms to develop competing models in any capacity.
The debate around "distillation panic" reveals differing opinions on where authentic research ends and unethical practices begin. AI researcher Nathan Lambert has expressed concerns that confusing these two realms may lead to overly broad restrictions that could hinder smaller AI startups and academic researchers reliant on distillation for building and analyzing AI models without excessive costs.
Preventing distillation may prove challenging.
"It's always a sort of cat-and-mouse scenario," Zilan Qian from the Oxford China Policy Lab explained to Business Insider. As long as AI model outputs are publicly accessible, "people will likely find a way to obtain them."
In response to regulatory measures, Anthropic has tightened restrictions on its models by blocking users in China, mandating overseas verifications, and requiring identity confirmations through government-issued documentation along with live selfies.
The consequence of these limitations, according to Qian, has been an emergence of "transfer stations," networks of intermediary services that allow Chinese developers to circumvent these barriers while offering their services at significantly reduced costs.
Some of these intermediaries operate with a plethora of fake or recycled accounts, or employ individuals from economically disadvantaged regions to complete required identity checks. By funneling all requests through these networks, operators can accumulate users' prompts and AI-generated responses, compiling valuable datasets for subsequent training or resale.
Qian argues that imposing stricter restrictions may inadvertently enhance the profitability of these alternative channels, affirming, "History shows that blocking access rarely deters determined users. It raises the cost of acquiring access, inadvertently creating lucrative opportunities for those skilled in bypassing such barriers."
Recent actions by Anthropic seem to have produced counterproductive results. The company discreetly reduced the quality of its model responses concerning AI development before partially reversing that policy after facing pushback from developers.
Reports from The Information further indicated that Anthropic had abandoned spyware intended to monitor Chinese users.
Many researchers argue that the increased enforcement has unintentionally nudged developers toward cheaper distilled open-source models, a trend that the leading firms are keen to prevent.
