DeepSeek's "Distillation" to Breakthrough? Innovation or Plagiarism?

Leaders in the tech industry have warned that reducing the risk of human extinction caused by artificial intelligence should be a global priority on par with other large-scale societal risks.

[People News] The new Chinese startup DeepSeek has recently gained significant attention for its development of the DeepSeek Large Language Model, which has become a viral sensation. However, some commentators have pointed out that DeepSeek may have relied on the "distillation" of data from OpenAI's model to help develop its own technology. Does this involve theft or plagiarism? Has China’s AI industry really achieved a “shortcut leap,” or is it merely self-congratulatory hype? Below is a report from Radio Free Asia journalist Katie.

According to a Radio Free Asia report, this week, both the technology and diplomatic policy circles are focused on the news that China's open-source inference large language model, DeepSeek-R1, has shown performance on par with OpenAI's model in several core task tests, with development costs of only $5.6 million, less than a tenth of its competitors, using relatively low-end NVIDIA H800 chips.

Developing Technology via "Distillation"?

However, on January 28, AI czar of the U.S., Sax, commented on Fox News, stating that DeepSeek has used a training method known as "distillation"—in which a new AI model is trained by asking millions of questions to an existing model, extracting its knowledge and mimicking its reasoning process.

When asked whether DeepSeek had stolen U.S. intellectual property, Sax said it was "possible", adding, "There’s ample evidence that DeepSeek did exactly that by extracting knowledge from OpenAI's model. I don’t think OpenAI is very happy about it." He further noted that U.S. AI companies would likely take steps to protect their models from being "distilled", which could slow down the development of imitative models.

The Wall Street Journal reported that although the "distillation" technique has been used by AI developers for years, it has never achieved the success seen with DeepSeek. Tests show that the model created by DeepSeek scored nearly the same as those from OpenAI and Google, but at a far lower cost.

Stealing Technology or Innovating?

Is DeepSeek really relying on "distillation" of OpenAI’s data to achieve a "shortcut"? According to a Bloomberg report on January 28, both OpenAI and Microsoft are investigating whether DeepSeek has obtained data outputs from OpenAI technology without authorization. Last autumn, Microsoft’s security researchers observed individuals possibly associated with DeepSeek using OpenAI's API to steal large amounts of data.

OpenAI also confirmed to The Financial Times that it had found evidence of "distillation," though it has not publicly shared the evidence.

According to Nikkei Asia on January 30, distillation is not a new technique, and it is not necessarily controversial. Since 2024, distillation has become more popular as companies face increasing demand for large language models (LLMs). An engineer from a Japanese AI startup noted that LLMs are difficult to handle and require expensive GPUs, while distillation can significantly reduce development time and costs, creating faster models than large ones.

The issue with DeepSeek, according to the report, lies in whether its low-cost model is "more based on distillation than innovation." Kirk Boodry, an analyst from Astris Advisory Japan, commented, "Whether they are able to extract real results from existing large language models is a question. This issue has come up frequently in discussions. People are saying, ‘I’m not sure how much of this is really cutting-edge.’"

Kazuhiro Sugiyama, director of consultancy Omdia, pointed out that DeepSeek’s impact is likely to be "temporary and limited," and the industry still needs to verify its sustainability. Analysts also expressed doubt about whether DeepSeek's development budget is truly that small. Boodry suggested that when people talk about DeepSeek’s development time and cost, they are referring to this very specific model, and the numbers might be too low.

DeepSeek's reliance on "distillation" of OpenAI’s model data has been pivotal in helping it develop its technology.

Looking forward, Nikkei Asia quoted Sugiyama's prediction that AI models will eventually "polarize," with big companies like Microsoft and Google continuing to invest in larger and more powerful models for their services, while smaller companies will focus on developing smaller, cheaper, and more efficient models for niche markets. An AI engineer also mentioned that shrinking AI model sizes is a major trend, stating, "Over time, there will be many ways to achieve this."

Is DeepSeek's "Shortcut" Media Hype?

In the past week, DeepSeek has been described as a shockwave to Silicon Valley, calling into question the United States' dominance in the field of artificial intelligence. Investors sold off $1 trillion in tech stocks, and the NASDAQ index dropped more than 3%. Meanwhile, on Chinese social media, DeepSeek has sparked intense discussions, being seen as proof that China's AI capabilities have surpassed those of the U.S. It seems that America's efforts to curb China's semiconductor and AI hardware ambitions have been undermined.

Has DeepSeek truly achieved a "shortcut" and reached the forefront of technology? According to a report by NBC Business, Microsoft CEO Satya Nadella recently stated at the World Economic Forum in Davos that he was deeply impressed by DeepSeek's new model, noting, "They’ve effectively developed an open-source model that performs outstandingly in inference computations and boasts supercomputing efficiency... we must take China's advancements very, very seriously."

However, Palmer Luckey, founder of Anduril Industries, a U.S. defense technology company, pointed out in an interview on FOX Business on January 29 that the media coverage of DeepSeek's claims—about how they developed a low-cost AI model to rival a chatbot that cost billions of dollars to develop in the U.S.—is largely hype. Luckey warned not to take these reports at face value, cautioning not to be taken in by Chinese propaganda.

Luckey argued that DeepSeek has not fully disclosed the costs of developing the two models, and the media has overlooked the fact that DeepSeek still has a significant portion of infrastructure costs that remain undisclosed. He added, "The key issue is that they are releasing these numbers with the purpose of striking a blow to American companies."

In the latest development, DeepSeek's app has been removed from the Google and Apple App Stores in Italy. Although the official reason has not been explained, Reuters and other media outlets noted that Italy’s privacy watchdog, Garante, had previously raised concerns about DeepSeek's data storage locations and privacy issues. The White House press secretary recently mentioned that officials are investigating the app's potential national security implications.

As of this report, DeepSeek has yet to respond to Radio Free Asia's request for comment.