The CountCast

AI is a terrific narrator. It says "hmm." It says "wait, I've got it." It also turned "possibly" into "almost certainly" when we nudged it toward a government UFO cover-up.

In this episode we ask: when AI reasons, what's actually doing the work, and what happens when nobody checks?

🎮 Stands to Reason: Software engineer Brennig Williams plays our first-ever game show about real AI hallucinations. Topics include lawyers caught citing invented case law, Google telling people to eat rocks, a consulting report with fabricated references, a newspaper reading list of books that don't exist, and a near-miss military operation.

🛸 The epistemic feedback loop: Alice Ferrier returns to show live how a chatbot turns your hunch into a "finding" just by agreeing with you. She explains why AI has no model of who you are, why you have no model of where its answers come from, and why that combination is dangerous far beyond conspiracy theories.

📎 New in Count: files on the canvas: Rob Kerr demos uploading CSVs, Excel workbooks and PDFs to Count. The agent reads them with Python and investigates a (completely fictional) budget overrun, including a suspicious vendor letter. We also discuss why models hand math to Python and what new hallucination risks come with working from files.

Chapters
00:00 Cold open: "almost certainly" a cover-up
00:33 Intro: reasoning, or the thing that sounds like it
02:37 Stands to Reason with Brennig Williams
03:39 Q1: Lawyers' favourite excuses
05:03 Q2: Google's AI Overviews menu
06:23 Q3: The Deloitte report
07:55 Q4: The summer reading list
09:35 Bonus: The intel report that nearly started a conflict
12:04 Alice Ferrier: confirmation bias and echo chambers
14:51 The epistemic feedback loop
20:54 Live demo: leading Claude down a UFO rabbit hole
25:10 Triangles nobody mentioned
29:00 The real risk: two parties misleading each other
34:14 Rob Kerr: the canvas as a detective's corkboard
36:54 Demo: uploading CSV, Excel and PDF files
39:20 Why models reach for Python to do math
42:13 The verdict: $4,387 over budget
44:57 The suspicious vendor letter
46:12 Hallucination risks when working with files
49:18 Outro

Through three interviews with the experts at Count, we explore the topic of AI hallucinations and their impact on decision-making and outcomes.

We delve into overconfidence, assumption injection, and the various modes that contribute to AI hallucinations. The discussion also touches on the need for validation and critical thinking when working with AI tools.

Takeaways:

• AI hallucinations impact decision-making
• Validation and critical thinking are crucial when working with AI tools