OpenAI reported that it successfully identified and disrupted a coordinated effort aimed at extracting protected reasoning from its AI models. This activity has been traced back to a key group connected with the Chinese startup Moonshot AI, the creators of the Kimi platform.
The campaign commenced in early July and escalated rapidly, resulting in an alarming 16,000 requests from over 4,000 users within just two days, according to OpenAI. After further investigation, the company traced the activity to a broader network of more than 15,000 users and confirmed that it had completely neutralized the operation by July 28.
These revelations emerged shortly after OpenAI's competitor, Anthropic, made allegations against several Chinese AI firms, including Moonshot AI and Alibaba. Anthropic claimed these companies were covertly leveraging its Claude model to enhance their own AI systems, highlighting escalating worries among U.S. AI firms about the potential for rivals to exploit their models, thereby accelerating their development of competing technologies at lower costs.
OpenAI characterized the identified efforts as "adversarial distillation,” a process where the outputs or reasoning from one AI model are utilized to improve another. The extraction of this kind of reasoning presents risks, as it could allow competitors to replicate cutting-edge capabilities without investing the necessary resources into creating and securing advanced models, raising both safety and national security concerns.
Importantly, OpenAI clarified that its encryption, databases, and stored user conversations were not compromised during this incident. The operators instead altered their interactions with the AI models in an attempt to reveal concealed reasoning in a way that was accessible to them.
While OpenAI could not determine whether all the individuals involved were part of a single organization, it did connect a significant portion of the activity to those affiliated with Moonshot AI. The company has conveyed its findings to other AI developers through the Frontier Model Forum and has engaged with government information-sharing initiatives.
As of now, Moonshot AI has not provided a response to requests for comments from CNBC.
