OpenAI disrupts model distillation campaign linked to Moonshot AI

FoxTPNL / Wikimedia Commons (CC BY 4.0)

OpenAI disrupts model distillation campaign linked to Moonshot AI

The company said operators used thousands of accounts to extract protected reasoning from its models in an effort consistent with training or improving rival systems.

OpenAI said it disrupted a coordinated model-distillation campaign that attempted to extract protected reasoning from its systems at scale, attributing a core cluster of the activity to individuals associated with Moonshot AI, the developer of Kimi.

The activity began in early July and initially remained limited before surging on July 24 and 25, when OpenAI observed roughly 16,000 requests using a relevant extraction pattern across more than 4,000 users.

A broader investigation identified related prompt activity across a cluster of more than 15,000 users. OpenAI said it had fully disrupted the operation by July 28.

The company described the activity as consistent with adversarial distillation, in which outputs or internal reasoning from one AI model are systematically used without authorization to help reproduce or improve another model.

Advertisement

OpenAI stressed that the operators did not break its encryption, compromise a database or directly access stored user conversations.

Instead, they manipulated model interactions to make protected reasoning visible. One technique involved copying encrypted reasoning from one conversation and asking a model in another conversation to decrypt and transcribe it.

Independent security researchers separately disclosed related cross-model and conversation-compaction vulnerabilities to OpenAI. The company said it confirmed those attack paths were real and used the findings to accelerate its mitigations.

OpenAI said it remains unclear whether all of the operators involved during the period belonged to a single actor. However, it attributed a core group of the activity to individuals associated with Moonshot AI.

The company warned that adversarial distillation could allow other developers to transfer advanced model capabilities without making equivalent investments in training or safety, particularly as frontier models become more capable in cybersecurity and other dual-use areas.

OpenAI responded by banning or restricting accounts, strengthening signup and infrastructure controls and expanding monitoring for related networks.

It also closed a pathway that allowed someone already in possession of another user’s encrypted reasoning to replay it and recover the underlying content. Additional safeguards were added to detect streamed outputs that might expose hidden reasoning.

OpenAI said it shared details of the campaign with other AI developers through the Frontier Model Forum and with government information-sharing channels.

The company expects distillation attempts to become more sophisticated and said it is continuing to expand protections across first-party products, cloud partners and tool-enabled systems.

Disclosure: This article was edited by Diego Almada Lopez. For more information on how we create and review content, see our Editorial Policy.
OpenAI disrupts model distillation campaign linked to Moonshot AI
OpenAI disrupts model distillation campaign linked to Moonshot AI

The company said operators used thousands of accounts to extract protected reasoning from its models in an effort consistent with training or improving rival systems.

FoxTPNL / Wikimedia Commons (CC BY 4.0)

OpenAI said it disrupted a coordinated model-distillation campaign that attempted to extract protected reasoning from its systems at scale, attributing a core cluster of the activity to individuals associated with Moonshot AI, the developer of Kimi.

The activity began in early July and initially remained limited before surging on July 24 and 25, when OpenAI observed roughly 16,000 requests using a relevant extraction pattern across more than 4,000 users.

A broader investigation identified related prompt activity across a cluster of more than 15,000 users. OpenAI said it had fully disrupted the operation by July 28.

The company described the activity as consistent with adversarial distillation, in which outputs or internal reasoning from one AI model are systematically used without authorization to help reproduce or improve another model.

Advertisement

OpenAI stressed that the operators did not break its encryption, compromise a database or directly access stored user conversations.

Instead, they manipulated model interactions to make protected reasoning visible. One technique involved copying encrypted reasoning from one conversation and asking a model in another conversation to decrypt and transcribe it.

Independent security researchers separately disclosed related cross-model and conversation-compaction vulnerabilities to OpenAI. The company said it confirmed those attack paths were real and used the findings to accelerate its mitigations.

OpenAI said it remains unclear whether all of the operators involved during the period belonged to a single actor. However, it attributed a core group of the activity to individuals associated with Moonshot AI.

The company warned that adversarial distillation could allow other developers to transfer advanced model capabilities without making equivalent investments in training or safety, particularly as frontier models become more capable in cybersecurity and other dual-use areas.

OpenAI responded by banning or restricting accounts, strengthening signup and infrastructure controls and expanding monitoring for related networks.

It also closed a pathway that allowed someone already in possession of another user’s encrypted reasoning to replay it and recover the underlying content. Additional safeguards were added to detect streamed outputs that might expose hidden reasoning.

OpenAI said it shared details of the campaign with other AI developers through the Frontier Model Forum and with government information-sharing channels.

The company expects distillation attempts to become more sophisticated and said it is continuing to expand protections across first-party products, cloud partners and tool-enabled systems.

Disclosure: This article was edited by Diego Almada Lopez. For more information on how we create and review content, see our Editorial Policy.