Kimi K3's Success: Not an Anthropic Fable Exploit, Experts Confirm

Michael Kratsios, the White House science advisor, has alleged that Moonshot, the Chinese firm responsible for the Kimi K3—currently the largest open-weight large language model (LLM) available—developed its model by illicitly replicating Anthropic’s Fable LLM and utilizing semiconductor chips prohibited for export to China.

Kratsios stated, “Large-scale, covert industrial distillation aimed at stealing proprietary U.S. technology and undermining American research is unacceptable.” This statement comes amidst reports of potential bans on Chinese open-weight AI models, which have caused significant disruption within the AI industry. Moonshot declined to comment on its training methodologies, and Kratsios did not elaborate on the evidence supporting his claims.

These remarks by Kratsios mirrored earlier comments from Treasury Secretary Scott Bessent, who noted, “we are finding watermarks of our U.S. large language models on many of the Chinese models, and that that’s unacceptable.” The precise nature of these "watermarks" remains undefined, and the Treasury Department has not provided a response to inquiries on the matter.

Nevertheless, experts express skepticism regarding whether distillation—defined as the process of querying an LLM to discern its internal mechanisms and replicate its functionalities—is solely responsible for the advanced capabilities demonstrated by Kimi K3.

Braden Hancock, a researcher at the Laude Institute and co-founder of Snorkel AI, conveyed to TechCrunch his doubt: “I don’t think you get a model this strong and this quickly on the heels of Fable doing strictly distillation.” He further emphasized the time constraint, stating, “There’s just not even frankly time, right? Fable’s only been publicly available since July 1st. You can’t distill that much data, train a model, and release it in two weeks.”

Nathan Lambert, an AI researcher at the Allen Institute for AI, remarked in a recent podcast that “I’ve been of the opinion that distillation has becoming less and less impactful over time as the Chinese models get closer to the frontier and the training regime shifts to [reinforcement learning].” He added, “[I]f it were the case, everyone would be easily able to catch up to a GLM or to a K3 by using its data for distillation. But we have not, or we won’t see this, from supervised fine-tuning alone.”

Effective distillation necessitates a laboratory systematically querying a target model to generate data suitable for post-training. This can involve explicitly prompting the model to explain its chain-of-thought to comprehend its problem-solving approach. Alternatively, the model's prompts and responses are employed to train a new model through a process known as supervised fine-tuning (SFT).

This fine-tuning process is precisely what can lead to a model, ostensibly developed by a third party, presenting itself as another, such as Claude. According to Lambert, fine-tuning is where the “model picks up its manners.”

However, Lambert contends that the advantages of SFT are diminishing as models grow in complexity. To replicate Fable-like capabilities through distillation would likely demand reinforcement learning (RL) techniques. Often, this entails an agent of the larger model evaluating the smaller model’s responses and making adjustments based on that assessment.

Such advanced techniques also necessitate substantial infrastructure. Extensive reinforcement learning operations can demand tens of millions of agents. Utilizing a leading lab’s API for this purpose “would be insanely expensive and potentially it would probably be a time bottleneck because these models are pretty slow and to be frank might not even give you a performance uplift.”

It appears plausible that earlier frontier models contributed to Kimi’s development; Anthropic had previously publicly accused Moonshot, DeepSeek, and MiniMax earlier this year of systematically distilling its models. Anthropic reported discovering millions of interactions between its models and users identified at these companies via IP addresses and other metadata. These queries were deemed “distinct from normal usage patterns, reflecting deliberate capability extraction rather than legitimate use.” Anthropic did not respond to TechCrunch’s inquiries specifically regarding Fable distillation.

Nevertheless, distillation is widely recognized as a prevalent practice among AI companies globally, extending beyond China. Elon Musk testified earlier this year that his company, SpaceX AI, distilled OpenAI models to develop Grok, asserting that this practice is common within the industry. The distinction between distillation and, for instance, developing synthetic data sets, can often be quite ambiguous.

Hancock commented, “[I]n general, Americans are understating the technical expertise of these Chinese teams.” He highlighted that “One of the founders of Moonshot was a CMU PhD student. These are legitimate researchers and engineers doing solid work.” He concluded, “…if American models ground to a halt, I think China’s progress would slow, but would still continue. They’re not just riding coattails here.”

Furthermore, it is challenging to separate the claims of distillation from the second component of Kratsios’ allegations—that Moonshot acquired advanced Nvidia Grace Blackwell 300 chips and accessed GB300-equipped servers in Thailand. These chips are explicitly banned for export to China, though a black market for them reportedly exists, as noted by Sam Bresnick, a research fellow at Georgetown’s Center for Security and Emerging Technology. Notably, in May, the founder of Supermicro, a U.S. server manufacturer, faced indictment for allegedly smuggling advanced chips into China.

Bresnick advocated, “I am a proponent of know your customer laws for data centers across the world.” He asserted, “If you are letting a company conduct huge training runs on your state-of-the-art hardware, there needs to be a reporting mechanism for who that company is and what they’re doing.”

In 2024, President Joe Biden’s Department of Commerce proposed federal know-your-customer regulations for data centers; however, no subsequent progress seems to have occurred under Donald Trump's administration. Nevertheless, exporters shipping advanced semiconductor chips internationally are expected to verify that these components are exclusively utilized for approved purposes.