[Dev]olution

Open Weight AI Isn't the Risk You Think It Is


Listen Later

Everyone's asking the wrong question about open AI models and has turned the debate into a geography problem. Is it Chinese? Is it American? Can we trust it?


But the biggest AI security incident in recent memory had nothing to do with where a model came from.


In this minisode, Nicky Pike breaks down the three layers of risk hidden behind open-weight AI models and why most security conversations focus on the easiest layer to defend. From poisoned training data and sleeper-agent research to malicious Hugging Face repositories that fooled hundreds of thousands of developers, this episode separates real threats from political headlines.

If you're evaluating DeepSeek, Qwen, Llama, or any open model, this episode will change how you think about trust, ownership, and security.


In this episode, you’ll learn:

  1. Why self-hosting only protects you if you control outbound access
  2. How malicious repositories create bigger risks than model origins
  3. What every engineering leader should inspect before deploying open models

Episode highlights:
(00:00) The repo that fooled 244,000 developers

(00:35) Why everyone wants open models

(01:20) The wrong question CIOs keep asking

(02:10) The three layers of AI risk

(04:00) Anthropic's sleeper agent experiment

(06:25) How 250 documents can poison a model

(08:00) The Hugging Face malware problem

(09:20) Why banning models misses the real threat

(10:15) Self-hosting isn't a force field

(11:20) What ownership actually gives you

(13:20) 4 practical defenses that work

(16:20) Why open models are getting too good to ignore

Resources:

  • Sleeper Agents: Training Deceptive LLMs That Persist Through Safety Training
  • A Small Number of Samples Can Poison LLMs of Any Size
  • Examining Backdoor Data Poisoning at Scale
  • JFrog Security Research on Malicious Models in Hugging Face

...more
View all episodesView all episodes
Download on the App Store

[Dev]olutionBy Coder