DeepSeek’s popularity fuels concerns over misinformation

Since DeepSeek-R1 entered public view, its generated content has frequently trended on Chinese social media. Topics such as “#DeepSeek Comments on Jobs AI Cannot Replace” and “#DeepSeek Recommends China’s Most Livable Cities” have sparked widespread discussion. Meanwhile, organizations throughout Chinese society have rushed to embrace the new technologies that DeepSeek has helped spotlight. Shenzhen’s Futian District recently introduced 70 “AI digital employees” developed using DeepSeek, demonstrating AI’s increasing implementation and broad application.

Yet as society embraces this new wave of innovation, a troubling pattern is emerging: AI-generated misinformation is flooding public networks. One viral case involved a Weibo user who discovered that Tiger Brokers, a Beijing-based fintech firm, had integrated DeepSeek for financial analysis. Out of curiosity, the user tested it on Alibaba, prompting the AI to analyze how its valuation logic shifted from e-commerce to a tech company. One of the AI’s reasoning points was that Alibaba’s domestic and international e-commerce businesses contributed 55% of its revenue, peaking at 80%, while its cloud intelligence group’s revenue share exceeded 20%. Surprised by these figures, the user cross-checked them against Alibaba’s financial reports, only to find that the AI had fabricated the data.

While DeepSeek-R1, a reasoning-focused model, performs similarly to conventional models on basic tasks, its approach actually differs significantly. Standard models rely on pattern matching for quick translations or summaries. Reasoning models, however, activate multi-step logic chains even for simple queries — a process that enhances explainability but risks “overthinking.”

Testing shows that these extended reasoning chains increase the risks of hallucination. The Vectara HHEM benchmark reveals DeepSeek-R1’s hallucination rate is 14.3%, nearly four times higher than DeepSeek-V3’s 3.9%. This disparity likely stems from R1’s training framework, which prioritizes user-pleasing outputs through reward-punishment mechanisms, sometimes fabricating content to confirm user biases.

AI systems don’t store facts — they predict plausible text sequences. Their core function isn’t verifying the truth but generating statistically likely continuations. In creative contexts, this means freely blending historical records with fabricated narratives to maintain story coherence. Such mechanisms inherently risk factual distortion. As AI-generated content floods online spaces, a dangerous feedback loop emerges: synthetic outputs are increasingly scraped back into training datasets. This erodes the boundary between authentic and artificial information, challenging public discernment. High-engagement domains – politics, history, culture, entertainment – face particular contamination risks.

Addressing this crisis demands accountability. AI developers must implement safeguards such as digital watermarks, while content creators should clearly label unverified AI outputs. Otherwise, the proliferation of synthetic misinformation, amplified by AI’s industrial-scale efficiency, will persistently test society’s ability to separate fact from algorithmic fiction.