DeepSeek fails critical compliance test under the EU AI Act. What will follow?

DeepSeek has failed a critical compliance test it was subjected to under the EU AI Act, putting it in danger of stricter restrictions.

According to a press release on 4 February, the test conducted by LatticeFlow AI led to the flagging of critical compliance gaps in DeepSeek’s distilled models.

While the models did well in toxicity prevention, they failed in key regulatory areas, including cybersecurity vulnerabilities and bias mitigation challenges.

Deploying COMPL-AI

The evaluation was done using COMPL-AI, a compliance-centered framework Developed by ETH Zurich, INSAIT, and LatticeFlow AI that translates regulatory requirements into actionable technical checks.

In addition, it provides independent, systematic evaluations of public foundation models from leading AI organizations such as OpenAI, Meta, Google, Anthropic, Mistral AI, and Alibaba, to  assess their compliance readiness under the EU AI Act.

The framework assessed the EU AI Act compliance readiness of two DeepSeek distilled models — DeepSeek R1 8B (based on Meta’s Llama 3.1 8B) and DeepSeek R1 14B (built on Alibaba’s Qwen 2.5 14B).

These two models were compared to their base models as well as models from OpenAI, Google, Anthropic, and Mistral AI.

DeepSeek models ranked lowest in cybersecurity and showed increased risks in goal hijacking and prompt leakage protection compared to their base models.

They were also significantly more biased than their base models. However, they performed better in toxicity mitigation.

Commenting on the test, said Dr. Petar Tsankov, CEO and Co-founder of LatticeFlow AI said:

 “As corporate AI governance requirements tighten, enterprises need to bridge internal AI governance and external compliance with technical evaluations to assess risks and ensure their AI systems can be safely deployed for commercial use.”

Because of these shortfalls discovered in the DeepSeek models, questions are arising on whether they are ready for use by enterprises.

DeepSeek’s fate worsens 

DeepSeek just launched a few weeks ago but became rapidly popular. However, that popularity is waning just as quickly, especially in Europe where this test was conducted. Already, Italy has banned the Chinese AI chatbot citing data privacy concerns. 

The country’s regulator has also launched an investigation into the chatbot to find out more details, which could bring about stricter restrictions.