Chaos Agents

Chaos Agents

Download on the App Store

All Chaos Agents episodes

  • AI Targeting, Why Benchmarks Lie, and Bringing Back AIM

    Welcome to Season 2 of Chaos Agents.

    We’re changing things up this season. AI is moving too fast for us to wait weeks between conversations, so we’re following it in real time—getting together a couple times a week to dig into the stories, research, experiments, and weird things happening in AI that we can’t stop thinking about.

    For our first episode back: who checks the AI?

    We dig into a devastating story about an AI-assisted military targeting system and what can happen when automation moves faster than the human guardrails around it.

    Then Becca goes deep on AI benchmarks: when a local model scores 0% on tool use, the model itself might not actually be the problem. The serving layer, the harness, formatting, and a missing standard can all turn a successful tool call into an apparent failure.

    We also get into a surprisingly powerful way of working with agents: give them a goal, let them iterate, and see how far they can get.

    And for One Cool Thing: someone is rebuilding Visual FoxPro for the modern era, which sends us down a nostalgia rabbit hole about the old internet, AIM away messages, and the software we’d bring back if we could.

    Plus: Becca and Sara reveal their deeply important AOL screen names.

    The links for today's stories are in today's newsletter except for:

    Circe : Sara's AI Harness Project

    22 min
  • Superpowers, Subagents, and Why Coding Isn’t the Job Anymore - With Jesse Vincent

    In the last episode of Season 1, Sara and Becca talk with Jesse Vincent about the shift from writing code to managing AI agents.

    They break down how tools like Superpowers turn development into a system of planning, testing, and review, where subagents implement work and other agents validate it. The conversation explores what happens when coding becomes less about syntax and more about taste, judgment, and outcomes.

    They also get into the reality of open source in the AI era, from “slop PRs” to agents impersonating maintainers, and why specs and tests may now matter more than the code itself. Along the way, they unpack how AI behaves like a team of eager but chaotic interns, and what it means to manage them effectively.

    If you’ve ever felt like coding is changing faster than you can keep up, this episode explains what comes next.

    Links mentioned:

    ⚡ Superpowers (Jesse’s tool) https://github.com/obra/superpowers 🛠️

    🧠 Claude Code (agentic dev tooling) https://docs.anthropic.com/en/docs/claude-code 🤖

    📊 Influence (book on persuasion principles) https://www.influenceatwork.com/book/ 📘

    42 min
  • Learn Anything, Build Anything, and the AI Barrier That Isn’t Real - With Navarrow Wright

    Most people are using AI, but very few actually understand how to use it well.

    In this episode, Sara and Becca talk with Navarrow Wright about what is really happening with AI adoption and why the biggest barrier is mindset, not technology. They break down why prompting matters, why AI is not one-size-fits-all, and how access to these tools is more open than any previous tech shift.

    They also explore the real opportunity: using AI as a thought partner. From tools like NotebookLM to building without traditional technical skills, this episode shows what becomes possible when people actually understand how to use these systems.

    If you feel like everyone else “gets AI” and you do not, this is a great place to start.

    Links mentioned:

    • 📓 NotebookLM
    • 📚 Gemini AI course
    • 🧠 Prompting guide (OpenAI)
    • 💻 Navarrow: IG, YT

    56 min
  • Algorithmic Transference, Generative UI, and Why the Text Box Won - With Louise Macfadyen

    We’re joined by Louise Macfadyen, author of Designing AI Interfaces, to talk about how we actually design for AI, and why most companies are getting it wrong.

    We unpack “algorithmic transference” (why one bad AI experience makes you distrust all of them), the limits of the chat interface, and whether the text box is here to stay. Louise walks us through the history from command line → Google search → LLMs, and why designing for ambiguity is still one of the hardest problems in product.

    We also get into generative UI (and why it might be a trap), how large companies should actually integrate AI without breaking user trust, and why understanding user intent matters more than ever. Plus: vibe coding, personal AI “jigs,” and the tension between personalization and privacy.

    If you build products, or just use AI, this will change how you think about interfaces.

    📘 Designing AI Interfaces · 🧠 Algorithmic Transference · 🛠️ Claude Code

    50 min
  • Personal Agents, Opinion Geometry, and AI That Learns Too Much - With Hayden Helm

    This week, Becca turns her birthday into a Twister masterclass by watching the movie with an AI “director’s commentary” running in real time (including some behind-the-scenes facts that sound… borderline illegal). Then Sara drops a mildly unhinged hot take: we might have already hit AGI—and we’ll never know it, because we don’t even agree on what “counts.”

    From there, we’re joined by Hayden Helm, CEO and founder of Helivan, to talk about the next wave of agentic AI: personal bots that don’t just answer questions, but act—with tools, permissions, and the ability to change over time.

    We get into:

    1. Why agent behavior drift is the real risk (even when the bot still sounds “nice”)
    2. The rising need for agent observability: detecting change, quantifying it, and rolling back when things go sideways
    3. What happens when agents learn from the environment (and other agents), not just from you
    4. Hayden’s work on “likeness” and opinion geometry—making messy human-ish behavior measurable
    5. Why manual inspection won’t scale in a world of millions of autonomous interactions

    It’s a conversation about trust, safety, and the new security surface area we’re creating—one helpful assistant at a time.

    🛠 Agent Infrastructure

    1. OpenClaw
    2. Helivan

    🔐 Agent Payments

    1. Coinbase x402 Protocol / Agentic Wallets

    🧨 Agent Behavior Incident

    1. “An AI Agent Published a Hit Piece on Me”

    58 min
  • Agentic Romance Scams, Open Source Safety, and Anything Bad on the Internet - With Juliet Shen

    Juliet Shen (Roost, formerly Snap, Grindr, Google) joins us to break down trust & safety — aka “anything bad that happens on the internet” — and why AI is changing the game for both attackers and defenders.

    We talk safety-by-design (how to bake guardrails into product), the real human cost of content moderation, and where AI can actually help without pretending it solves everything. Juliet explains how agentic AI scales old harms like romance scams from a handful of conversations to thousands at once — and why open-source safety infrastructure matters when bad actors share tactics faster than platforms do.

    We also dig into what Roost is building: Osprey (an investigation + rules engine) and Coop (a flexible review tool), plus the building blocks smaller teams need to ship products responsibly before the “weird edge cases” arrive.

    If you build products, this one’s a must.

    🛡️ Roost (GitHub + tools directory)

    👥 Trust & Safety Professional Association (TSPA)

    🧩 Prosocial Design Network

    📚 Behind the Screen (Sarah T. Roberts — content moderation + human cost)

    47 min
  • Agents in Crypto, Agents at Work, and the Weird New Middle Management - With Meghan Heintz

    We’re joined by Meghan Heintz, founding engineer at Herd Labs, to break down where crypto and AI agents actually work. We talk prediction markets, smart contracts, wallets, rug pulls, and how AI can finally explain what happened on-chain.

    Then we zoom out to the human side: benchmarking and evals for agents, misinformation, the Mom Test, and what it feels like to manage coding agents instead of junior engineers.

    Links mentioned: Polymarket · Dune Analytics · Herd Labs · The Mom Test

    51 min
  • Can You Build Anything in a Week? GPUs, Code Gen, and the End of Engineers - With Harper Reed

    Becca just got back from NeurIPS, the academic AI conference that feels like an adult science fair. We dig into research on training large AI models across cheap GPUs and slow internet connections—and why that could dramatically lower the barrier to building AI.

    Then we’re joined by Harper Reed, CEO of 2389, for a wide-ranging conversation about code generation, coaching-based engineering teams, and why “production code” might have always been a myth. We talk vibe coding (begrudgingly), the shifting role of software engineers, taste vs. technical skill, and what happens when you can build almost anything in a week.

    Smart, funny, and a little unsettling—Chaos Agents at full volume.

    🎓 Academic AI & research culture
    1. NeurIPS (Conference on Neural Information Processing Systems)
    2. NeurIPS 2024 Accepted Papers

    🧠 Distributed training, GPUs & efficiency
    1. NVIDIA H100 Tensor Core GPU (referenced GPU class)
    2. Pluralis Research (distributed training across low-bandwidth networks)

    ⚙️ Core AI concepts mentioned
    1. GPU vs CPU explained (parallel vs sequential compute)
    2. Data Parallelism vs Model Parallelism (training overview)

    🧑‍💻 Code generation & developer tools
    1. Claude Code (Anthropic code-gen tooling)
    2. Cursor (AI-first code editor, discussed implicitly)

    🛠️ Agent workflows & infrastructure
    1. Matrix (open-source, decentralized chat protocol)
    2. Model Context Protocol (MCP) overview

    🧩 Utilities & recommendations
    1. Jesse Vincent’s Superpowers (Claude workflow enhancer)
    2. Fly.io (deployment platform referenced)
    3. Netlify (deployment & hosting)

    🧪 Related Chaos Agents context
    1. Perceptrons & early neural networks (referenced conceptually)

    59 min
  • The Magic Cycle, AI Detectors, and the End of Writing as Proof - With Clay Shirky

    Sara’s back from visiting her New Jersey Christian high school—where she gets hit with a genuinely spicy question: How do you reconcile AGI with faith? From there, we go straight into the bigger theme of the episode: education is getting stress-tested by AI in real time.

    Becca breaks down Google’s “magic cycle” — the uncomfortable lesson of inventing transformative research (Transformers, BERT) and then watching someone else ship it to the world. Sara shares what she’s learning about research workflows moving beyond “just chat,” including multi-agent setups for planning, searching, reading, and synthesis.

    Then we’re joined by Clay Shirky, Vice Provost for AI & Technology in Education at NYU, to talk about what’s actually happening on campuses: why students integrated AI “sideways” before institutions could respond, why AI detectors are a trap (and who they harm most), and why the real shift isn’t assignments — it’s assessment.

    We dig into what comes next: oral exams, in-class scaffolding, and designing learning around productive struggle—not just output. And we end in a place that’s both funny and unsettling: the rise of AI “personalities,” RLHF as “reinforcement learning for human flattery,” and what it means when a machine is always on your side.

    Because whether we like it or not: a well-written paragraph is no longer proof of human thought.

    🧠 Foundational AI papers & breakthroughs
    1. Attention Is All You Need (Transformers, 2017)
    2. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

    🧪 Google’s “Magic Cycle” framing
    1. Accelerating the magic cycle of research breakthroughs and real-world applications Google Research
    2. How AI Drives Scientific Research with Real-World Benefit (Google Blog)

    🚨 Shipping pressure: Bard + “code red” era
    1. Reuters: Alphabet shares dive after Bard flubs info, ~$100B market cap hit (https://www.reuters.com/technology/google-ai-chatbot-bard-offers-inaccurate-information-company-ad-2023-02-08/) Reuters
    2. Google Blog: Bard updates from Google I/O 2023 (https://blog.google/technology/ai/google-bard-updates-io-2023/) blog.google

    🎓 AI in higher ed: assessment, detectors, bias
    1. Clay Shirky’s NYT essay
    2. Stanford HAI: AI detectors biased against non-native English writers
    3. Peer-reviewed paper (PMC): GPT detectors misclassify non-native English writing

    🪞 RLHF, sycophancy, and “the tool likes you too much”
    1. OpenAI: Aligning language models to follow instructions (RLHF explainer)
    2. OpenAI: Sycophancy in GPT-4o—what happened & what we’re doing


    54 min
  • Speed vs Quality, Hallucinations, and the AI Learning Rabbit Hole - With Nir Zicherman

    Sara breaks down perceptrons (1957!) as the tiny “matrix of lights” idea that eventually became neural networks—then we jump straight into modern AI chaos.

    Oboe’s Nir Zuckerman walks us through the messy reality of building consumer-grade AI for education: every feature is a tradeoff between loading fast and being good, and “just use a better model” doesn’t magically solve it. We talk guardrails, web search, multi-model pipelines, and why learning tools should feel lightweight—more like curiosity than homework. Also: Becca’s “how does a computer work?” obsession and a book recommendation that might change your life.

    🧠 AI Concepts & Foundations
    1. Perceptron (Wikipedia)
    2. Neural Networks Explained
    3. Scaling Laws for Neural Language Models
    4. FLOPS (Floating Point Operations Per Second)

    🎓 Learning, Education & AI
    1. Oboe
    2. AI as a Personal Tutor (Overview)
    3. Why Tutors Are So Effective

    🏗️ Building AI Products
    1. Speed vs Quality Tradeoffs in LLM Apps
    2. LLM Orchestration Patterns
    3. Retrieval-Augmented Generation (RAG)
    4. LLM Hallucinations: Causes & Mitigation

    📚 Books Mentioned
    1. Code: The Hidden Language of Computer Hardware and Software
    2. Perceptrons

    🧪 History of AI
    1. Frank Rosenblatt and the Perceptron
    2. The AI Winter Explained
    3. Early Neural Network Research (1950s–1980s)


    49 min