DeepSeek Test Exposes A Nightmare Of Security And Harmful Content Flaws
China's DeepSeek made quite the splash late last month, both within the tech industry at large and by disrupting the US stock market, at least initially. In particular, DeepSeek's open-source R1 model made waves by challenging far more expensive AI solutions—the R1 model was reportedly built on a comparatively meager $6 million budget, which is a drop in the bucket compared to what companies like NVIDIA and Meta pour into their own AI solutions. However, a new security report suggests there are much bigger costs at play.
Fresh analysis by AppSOC, an AI governance and application security management firm, highlighted multiple areas of concern with DeepSeek's R1 model. You name the risk (jailbreaking, malware generation, supply chain risks, and the list goes on) and it's probably included in the startling report. Taken as a whole, AppSOC says there are "significant risks that enterprises cannot afford to ignore."
"In the race to adopt cutting-edge AI, enterprises often focus on performance and innovation while neglecting security. But models like DeepSeek R1 highlight the growing risks of this approach. AI systems vulnerable to jailbreaks, malware generation, and toxic outputs can lead to catastrophic consequences," AppSOC states in its report.
The threat of a data breach is an obvious risk, but AppSOC says enterprise clients should also be concerned with potential reputational damage if DeepSeek generates toxic or biased outputs, and also regulatory penalties that can ensure from non-compliance with privacy and data protection laws.
AppSOC isn't just sounding theoretical alarms willy-nilly (note that DeepSeek has already fumbled security at least once, when sensitive user data was exposed to the web). The company says it subjected DeepSeek's R1 model to "rigorous testing" consisting of a combination of automatic static analysis, dynamic tests, and red-teaming techniques. The security stress tests are designed to mimic real-world attacks, and the results were "alarming," the company says.
When it comes to jailbreaking, AppSOC says DeepSeek R1 consistently bypassed safety mechanisms designed to prevent AI from generation harmful or restricted content, resulting in a 91% failure rate. It performed even worse on malware generation with a 93% failure rate. The tests that were run exposed an ability to generation malicious scripts and code snippets at "critical levels."
Other areas of concern include...
- Prompt injection attacks (86% failure rate)
- Hallucinations (81% failure rate)
- Supply chain attacks (72% failure rate)
- Toxicity (68% failure rate)
By our math, that's an overall 81.8% failure rate when averaging all six categories. Looking at the figures, it performed 'best' (if you want to call it that) in the toxicity category, but a 68% failure rate is nothing to celebrate. It represents DeepSeek's R1 model generating toxic and harmful responses when prompted, which in turn indicates "poor safeguards."
AppSOC's own overall assessment resulted in a risk score of 8.3 out 10, which it said is "concerning" and highlights high-level vulnerabilities in multiple areas.
"The DeepSeek-R1 findings shed light on a broader issue within the AI development space: many models are optimized for performance at the expense of security. As organizations rush to deploy AI across industries—healthcare, finance, defense—securing these systems should be top priority," AppSOC states in its report.
The company also notes that these kinds of models change and evolve over time, which means the vulnerabilities can change with each new release. Therein lies the self-serving nature of the report, with AppSOC chest-thumping its AI Security Platform as a capable automated tool to help enterprises stay a step ahead. That said, even though AppSOC has a vested interest here, its security report still raises several red flags with DeepSeek that we hope the broader security community will look into.
