Sign up to save your podcasts
Or
Share AI Is a Weapon You Might Be Pointing at Yourself | OWASP Top 10 LLMs
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
AI Is a Weapon You Might Be Pointing at Yourself | OWASP Top 10 LLMs
A lawyer submitted six court cases to a federal judge in New York.
ChatGPT wrote every single one of them. None of them existed.
When opposing counsel said they couldn't find the cases, the lawyer went back to ChatGPT to verify whether the cases were real. ChatGPT said yes. Absolutely. You can find them on Westlaw and LexisNexis.
He submitted them anyway — under oath.
That's hallucination. That's number nine on the OWASP Top 10 for LLM Applications. And it cost him $5,000, a formal apology to every federal judge whose name appeared in the fake rulings, and probably a lot more in embarrassment.
This week on Security Bros, Rocky and John Giglio go deep on the OWASP Top 10 for LLM Applications — the 2025 edition, built by 600+ researchers across 18 countries. If you're building with AI, deploying AI, or just using it every day at work, this list is the closest thing the security world has to a peer-reviewed warning label.
They break down all 10 vulnerabilities in plain English, connect each one to real stories, and don't sugarcoat any of it:
- A world-famous white hat hacker who jailbreaks ChatGPT to write his own attack tools
- Samsung engineers who handed proprietary source code to ChatGPT — and how long it took after the ban was lifted for it to happen again (spoiler: 20 days, three incidents)
- Air Canada's chatbot that gave a grieving customer wrong information about bereavement fares — and the company's legal defense that the chatbot was "a separate legal entity"
- How DeepSeek may have reverse-engineered Claude's reasoning by querying it at scale — and what Anthropic is doing about it
- The invisible text on a webpage that hijacks your AI agent without you ever knowing
The lesson running through all of it: your security policy will never beat convenience without technical controls. You have to make the secure path the easy path.
Subscribe so you don't miss the follow-up deep dives on Claude Cowork security, AI-ready DLP, and the excessive agency problem that's about to blow up as agentic AI goes mainstream.
Resources mentioned:
- OWASP Top 10 for LLM Applications: https://owasp.org/www-project-top-10-for-large-language-model-applications/
- Previous episode: OWASP Top 10 for Web Applications
https://youtu.be/oCuYgphY6iY
00:00 The Lawyer Who Asked AI If AI Was Lying to Him
00:33 Meet the Security Bros + What We're Covering Today
01:41 What Is OWASP? (600 Researchers, 18 Countries, One List)
04:38 #1 Prompt Injection — The Attack That's Everywhere Right Now
07:28 #2 Sensitive Information Disclosure — You Think It's Private. It's Not.
08:29 #3 Supply Chain Risk — What's Really Inside That Open Source Model?
10:27 #4 Data & Model Poisoning — The Sleeper Agent Attack
13:31 #5 Output Handling — Nobody Reviews AI Code. Nobody.
14:05 #6 Excessive Agency — When Your AI Has Too Much Power
18:12 #7 System Prompt Leakage — Stop Putting Secrets in the Instructions
20:37 #8 Vector & Embedding Weaknesses — How RAG Gets Poisoned
23:30 #9 Hallucination — AI Makes Things Up. Confidently.
25:58 #10 Unbounded Consumption — How DeepSeek May Have Stolen Claude's Brain
29:59 Real Story: Samsung's 3 Data Leaks in 20 Days
36:03 Real Story: Air Canada's "Separate Legal Entity" Defense
40:30 Real Story: The $5K Fine & Apology Letters to Federal Judges
45:09 Key Takeaways — Make the Secure Path the Easy Path
View all episodes
By Security BrosA lawyer submitted six court cases to a federal judge in New York.
ChatGPT wrote every single one of them. None of them existed.
When opposing counsel said they couldn't find the cases, the lawyer went back to ChatGPT to verify whether the cases were real. ChatGPT said yes. Absolutely. You can find them on Westlaw and LexisNexis.
He submitted them anyway — under oath.
That's hallucination. That's number nine on the OWASP Top 10 for LLM Applications. And it cost him $5,000, a formal apology to every federal judge whose name appeared in the fake rulings, and probably a lot more in embarrassment.
This week on Security Bros, Rocky and John Giglio go deep on the OWASP Top 10 for LLM Applications — the 2025 edition, built by 600+ researchers across 18 countries. If you're building with AI, deploying AI, or just using it every day at work, this list is the closest thing the security world has to a peer-reviewed warning label.
They break down all 10 vulnerabilities in plain English, connect each one to real stories, and don't sugarcoat any of it:
- A world-famous white hat hacker who jailbreaks ChatGPT to write his own attack tools
- Samsung engineers who handed proprietary source code to ChatGPT — and how long it took after the ban was lifted for it to happen again (spoiler: 20 days, three incidents)
- Air Canada's chatbot that gave a grieving customer wrong information about bereavement fares — and the company's legal defense that the chatbot was "a separate legal entity"
- How DeepSeek may have reverse-engineered Claude's reasoning by querying it at scale — and what Anthropic is doing about it
- The invisible text on a webpage that hijacks your AI agent without you ever knowing
The lesson running through all of it: your security policy will never beat convenience without technical controls. You have to make the secure path the easy path.
Subscribe so you don't miss the follow-up deep dives on Claude Cowork security, AI-ready DLP, and the excessive agency problem that's about to blow up as agentic AI goes mainstream.
Resources mentioned:
- OWASP Top 10 for LLM Applications: https://owasp.org/www-project-top-10-for-large-language-model-applications/
- Previous episode: OWASP Top 10 for Web Applications
https://youtu.be/oCuYgphY6iY
00:00 The Lawyer Who Asked AI If AI Was Lying to Him
00:33 Meet the Security Bros + What We're Covering Today
01:41 What Is OWASP? (600 Researchers, 18 Countries, One List)
04:38 #1 Prompt Injection — The Attack That's Everywhere Right Now
07:28 #2 Sensitive Information Disclosure — You Think It's Private. It's Not.
08:29 #3 Supply Chain Risk — What's Really Inside That Open Source Model?
10:27 #4 Data & Model Poisoning — The Sleeper Agent Attack
13:31 #5 Output Handling — Nobody Reviews AI Code. Nobody.
14:05 #6 Excessive Agency — When Your AI Has Too Much Power
18:12 #7 System Prompt Leakage — Stop Putting Secrets in the Instructions
20:37 #8 Vector & Embedding Weaknesses — How RAG Gets Poisoned
23:30 #9 Hallucination — AI Makes Things Up. Confidently.
25:58 #10 Unbounded Consumption — How DeepSeek May Have Stolen Claude's Brain
29:59 Real Story: Samsung's 3 Data Leaks in 20 Days
36:03 Real Story: Air Canada's "Separate Legal Entity" Defense
40:30 Real Story: The $5K Fine & Apology Letters to Federal Judges
45:09 Key Takeaways — Make the Secure Path the Easy Path