The BugBash Podcast

You hit 100% test coverage, deploy to production, and the system still falls over. An old story, maybe aside from the 100% coverage percent part.

In this episode, David sits down with Marco Primi and Sean Allen from the Antithesis Red Team to talk about a likely missing piece in your testing strategy: workloads.

They break down exactly what workloads are, why they are so helpful for testing distributed systems, and the biggest pitfalls engineers hit when trying them out. They talk about why you have to let go of deterministic, "happy path" testing, and how even a workload as "dumb" as a basic `while true` loop can expose silent data loss that your unit tests would never catch.

Why do LLMs struggle to build complex architecture? 

According to Unmesh Joshi, Distinguished Engineer at ThoughtWorks, it often comes down to a lack of shared abstractions, of a shared vocabulary.

Today he joins Will Wilson and David Wynn to unpack how we learn about, build, and test distributed systems. We cover the value of building miniature versions of systems like Kafka to grasp the basics, why Deterministic Simulation Testing (DST) is a highly effective tool for teaching as well as finding bugs , and why Guy Steele's 1998 keynote holds the secret to successfully collaborating with modern AI tools.

Can you stress-test a React UI completely hands-free? In this episode, Oskar Wickström and I are joined by Jonathan Chan to discuss the origins of Fuzzmap.

From a janitor’s closet in Figma—where end-to-end tests once lived on a "spicy pillow" MacBook—Jonathan shares how he applied coverage-guided fuzzing (inspired by AFL) to the world of web front-ends. We dive into the gnarly hacks required to instrument React, the challenges of visualizing complex state machines, and the future of "full-stack" fuzzing in the age of LLMs and team agents.

With the "Agentic Era" coming online, teams are onboarding new AI "minions" into our codebases every thirty minutes. So a critical question emerges: Are we actually becoming more productive, or just more busy?Jessica Kerr (Jessitron) joins me to explore symmathesis—the theory of learning systems made of learning parts. We dive into viewing software as a teammate, how the definition of "legacy code" is shifting to anything an AI can't understand, and why our most important human contribution is moving from writing lines of code to orchestrating the flows of learning across the entire system.We dive into the shifting boundaries between the social and technical worlds, and why the "productivity unlock" of AI might actually make software engineering a more human endeavor than ever before.

Today we’re talking with Rain from Oxide Computing, tracing their journey from the massive, data-driven scale of Meta to the high-stakes, air-gapped world of shipping a 'cloud in a box'. We talk about moving from an environment of over 10,000 engineers—where a 10% tooling improvement is worth a thousand people—to a culture where you’re shipping hardware that simply cannot be patched once it leaves the building. Rain shares how they realized that technical writing is a senior engineer’s ultimate force multiplier for justifying complex designs. We also dive deep into the 'invisible superpower' of property-based testing and using Oracles to kill bugs in the developer’s inner loop long before they ever have a chance to reach production. Mentioned: https://rfd.shared.oxide.computer/rfd/0576 Further reading: https://oxide-and-friends.transistor.fm/episodes/engineering-rigor-in-the-llm-age

Everyone wants reliable software. But nobody wants to test messy legacy code. Today, Lewis Campbell from Outdata joins the show to share a practical approach for bringing deterministic simulation testing to existing systems. 

We get into why React components are terrible for business logic, and why front end static typing stops non-determinism at the door. We also take on non-tech aspects, like the politics of updating old code bases, the danger of hiding data conflicts, and how to ease your team into property-based testing. There’s even an Internet-hungry sheep named Angus.

Building the rr replay debugger wasn't about academic purity. It was about survival.

Today, creator Rob O'Callahan reveals the "necessary” complexity that makes time-travel debugging possible on Firefox. We dig into the engineering nightmares behind the tool: from patching 2-byte system calls to the constant fear of silent hardware failures.

Rob explains how rr became a "protected species" inside the Linux kernel—and why messy, manual hacks often beat six months of theoretical work.

It’s been nine years since Designing Data-Intensive Applications became the standard text for distributed systems. Today, Martin Kleppmann and Chris Riccomini join the show to pull back the curtain on the upcoming Second Edition. After all, the era of local disks are giving way to cloud-native object storage. So we discuss why modern databases are being rebuilt entirely on top of S3.

From there, we revisit the CAP theorem—and why it might be time to retire it in favor of "offline availability." We also get into a surprisingly practical debate on AI, exploring how LLMs might be terrible at creative design but perfect as "test oracles" for verifying complex system migrations.

You’ll want to stick around for this one.

Large Language Models can generate code in a flash, but that code is notoriously unreliable. Traditional unit tests often can’t put enough guardrails in place to ensure correctness… even if they’re written by the LLM itself.

This is where property-based testing (PBT) becomes essential.

Today, we're joined by David R. MacIver, creator of the PBT library Hypothesis, and now an Antithesis employee! We discuss how to build robust feedback loops that are needed to make AI-generated code trustworthy.

We'll cover why standard AI coding benchmarks are flawed, how Hypothesis makes PBT approachable, and the challenge of getting developers to think in "invariants." David also shares his perspective on the future of AI in software engineering.

If you want to build a reliability backstop for your code, vibed or otherwise, stick around.

Software testing research is exploding, but in practice, most companies' testing approaches seem stuck in the past. Where does that gap come from?

It often boils down to the distance between academic promises and the practical needs of developers who need usable tools and fast results.

In this episode, David talks with Rohan Padhye, head of the PASTA research group at Carnegie Mellon University, who has lived on both sides of that divide. They explore how fuzz testing crossed that chasm—from industry curiosity to academic focus and back again—and what it will take for other techniques to do the same.

Rohan shares insights on designing testable software, building a robust testing culture, and what truly makes a "good" property for finding bugs.