JOSH: It's Wednesday, August 5. This is Build or Be Replaced — powered by ScanBrief.dev. I'm Josh, here with Erik Anderson.
ERIK: Today's theme is simple. Smaller models, stricter guardrails, and fewer excuses.
JOSH: Stick around — Erik's got an AI pro tip at the end about making agents prove their work before they touch your repo.
[pause]
JOSH: First headline. Mistral has Shieldstral, a 3B open-weights model for multimodal moderation. Why does that matter?
ERIK: Moderation is turning into infrastructure. A 3B model that can read plain-language policy and classify text or images means teams can stop hard-coding every weird rule into brittle filters.
JOSH: Second. Stateless MCP is getting attention again.
ERIK: Good. Persistent agent state is where a lot of weird bugs and security problems hide. Stateless flows are boring in the best way. Input, tool call, output, audit trail. Done.
JOSH: Third. Uber has ADR trending on GitHub. AI agent observability, threat detection, security benchmarking. Is that enterprise theater or real?
ERIK: Real, if it actually sits in the path. Agents are now production actors. If you don't observe them like services, you're basically letting interns run kubectl with vibes.
[pause]
JOSH: ScanBrief had a lot today. What made Shieldstral stand out?
ERIK: The size. 3B parameters is the headline for me. Not because tiny is always better, but because safety classifiers should be cheap enough to run everywhere. You don't want every moderation check hitting a giant hosted model that costs real money and adds latency.
JOSH: So this isn't about replacing the main model?
ERIK: No. It's about putting a bouncer at the door. Your main model can be Claude, GPT, Gemini, qwen3-coder, whatever. But before you send user input into the expensive brain, or before you let the agent take action, a small classifier can ask, does this violate my policy.
JOSH: And the policy is written in plain language?
ERIK: That's the interesting part. Traditional moderation is usually a pile of categories. Hate, violence, self-harm, sexual content, malware. Fine. Useful. But real businesses don't think like that. They think, don't generate regulated advice. Don't discuss customer secrets. Don't produce exploit steps. Don't make claims our legal team hates.
JOSH: That's much closer to how people actually work.
ERIK: Exactly. If I can say, "this agent may summarize a firewall incident but may not reveal customer IP space," that's useful. If I can say, "this workflow can inspect Terraform but cannot propose IAM wildcard permissions," now we're talking about real guardrails.
JOSH: How would you use something like that in your own setup?
ERIK: PrimeBus already treats events as the source of truth. An error hits the bus, an agent picks it up, it proposes a fix, tests run, Gandalf reviews it, then the auto-merger decides whether it's safe. The current numbers are 2295 attempts since 2026-06-05, 1507 merged, 788 blocked by Gandalf, 0 escalated to me.
JOSH: Zero escalated to you is the crazy part.
ERIK: That's the point. The blocked count is the story. Those aren't failures. That's the guardrail doing its job. A small policy model fits right there. Before an agent opens a PR, before it changes a config, before it comments on an incident, ask a cheap model if the action matches policy.
JOSH: What does that mean for teams that don't have your PrimeBus setup?
ERIK: Start with one path. Pick the scariest agent action. Maybe it can write to GitHub. Maybe it can restart a Kubernetes deployment. Maybe it can update a ServiceNow ticket. Put a policy check in front of that action. Don't start with the whole company. Start with the one button that makes you nervous.
JOSH: That feels practical.
ERIK: It is. Everyone wants a magical AI governance platform. Duuude, put a small model before the dangerous tool call. Log the input, the policy, the decision, and the action. That's already better than half the agent demos I see.
[beat]
JOSH: Is open weights important here?
ERIK: Very. Safety is one of those places where companies need inspection. If the model is making allow or deny decisions, I want to know what I'm running, where it's running, and what leaves my network. Especially for healthcare, finance, government, anything with customer data.
JOSH: So local moderation becomes part of the stack.
ERIK: Yep. Like auth. Like logging. Like rate limiting. Not sexy. Very necessary. In my Echo and Neo lab, I've got 160 services running on the production server right now. You don't keep that sane with hope. You keep it sane with events, policy, and boring checks that run every time.
[pause]
JOSH: The second story is stateless MCP. This one sounds nerdy even by our standards.
ERIK: It is nerdy. That's why I like it.
JOSH: Give me the human version.
ERIK: MCP lets models talk to tools. File systems, databases, browsers, ticket systems, internal APIs. Stateful MCP means the server remembers context between calls. Stateless MCP means every request carries what it needs, and the server doesn't keep a hidden little memory pile.
JOSH: Why is hidden memory bad?
ERIK: Hidden memory is where surprise comes from. Maybe the server remembers a prior user. Maybe it keeps stale credentials. Maybe the model thinks it's operating on one project and the server has state from another. Now your agent makes a confident mess.
JOSH: Wait, really?
ERIK: Absolutely. State is useful, but it has to be deliberate. Network engineers already know this. If a device has config drift, stale sessions, or some half-applied change, you're going to have a bad day. Agent tools are the same thing with nicer marketing.
JOSH: What's the benefit of stateless then?
ERIK: Replayability. Auditability. Security. If every request says, "here's the repo, here's the task, here's the allowed tools, here's the policy, here's the auth scope," then you can replay that later and understand what happened. That's gold for incident review.
JOSH: How does that compare to how you run PrimeBus?
ERIK: PrimeBus is event-driven, but the events are explicit. A service emits telemetry. Another service subscribes. The payload says what happened. Right now 140 distinct projects have emitted telemetry to PrimeBus. That works because the event carries context. I don't want agents guessing based on some foggy memory.
JOSH: So stateless doesn't mean no context.
ERIK: Correct. It means no invisible context. Big difference. You can pass a full task packet. You can include prior logs, test output, a diff, a ticket, whatever. But it should be in the request where you can inspect it.
JOSH: That sounds like more tokens.
ERIK: It can be. But tokens are cheaper than mystery. Also, you don't need to pass the whole universe. Pass the right slice. ScanBrief scored 103 items across 56 sources today. It doesn't send me the entire internet. It scores, filters, ranks, and gives me the signal.
[beat]
JOSH: Where do teams get this wrong?
ERIK: They treat agent memory like a product feature before they have basic controls. "Our agent remembers your preferences." Great. Does it remember secrets? Does it remember the wrong tenant? Can I delete it? Can I prove what it used to make a decision?
JOSH: That's the uncomfortable version.
ERIK: That's the real version. Agent systems need clean boundaries. Tool scope. Fresh auth. Short-lived context. Logs that humans can read. And for anything important, a second check before execution.
JOSH: Is this where Uber's ADR fits in?
ERIK: Yeah, it lines up. ADR is about observability, benchmarking, and threat detection for AI agents. That's the enterprise version of what I'm saying. Don't only ask whether the agent produced a nice answer. Ask what tools it touched, what data it saw, what threat pattern it matched, and whether the action should have happened.
JOSH: Agents as production services.
ERIK: Exactly. If a normal service starts making weird outbound calls, you want alerts. If an AI agent starts pulling sensitive files it doesn't need, same deal. The agent doesn't get a special pass because it speaks politely.
[pause]
JOSH: Let's stay on ADR for a minute. What does agent observability actually need?
ERIK: Three layers. First, tool calls. What did it call, with what arguments, and what came back. Second, intent. What was the agent trying to do. Third, outcome. Did it change code, send email, create a ticket, deploy something, or just draft text.
JOSH: Most logs don't capture intent.
ERIK: Right. Traditional logs say request received, response sent, 200 OK. That's not enough for agents. If Claude edits a Terraform file, I want the diff, the reason, the test result, and the policy verdict. If Selenium logs into a site and clicks around, I want screenshots or checkpoints. If NSO config gets touched, I want a dry run and rollback plan.
JOSH: That sounds heavy.
ERIK: It's heavy if you bolt it on after the agent is already doing real work. It's manageable if you design for it from day one. My 12 agents in the Bobaverse fleet run through named paths. Neo, Homer, Bill, Echo, Gandalf. Claude and GPT both have jobs. When something acts, it emits telemetry. That's how you keep the system from becoming a haunted spreadsheet.
JOSH: There it is.
ERIK: I mean it. People are building agent systems that are basically cron jobs with a chat box. No trace. No owner. No kill switch. Then they act shocked when nobody wants it near production.
JOSH: What should a normal engineering team do this week?
ERIK: Put IDs on everything. Agent ID. User ID. Tool ID. Task ID. Request ID. Commit SHA if code changed. Ticket ID if it touched a workflow. Then log the decision chain in plain text. Not a novel. Just enough for another engineer to reconstruct what happened.
JOSH: And security benchmarking?
ERIK: Test your agent like you test an API. Prompt injection. Bad tool args. Secret access. Cross-tenant data. Weird file paths. Malicious markdown. Fake instructions inside documents. If your agent reads web pages, assume the page is trying to manipulate it.
JOSH: That's bleak.
ERIK: That's Tuesday in security. The model is not the perimeter. The tool boundary is the perimeter. If the agent can call a tool, that tool needs checks like a real API endpoint.
[beat]
JOSH: This connects to the Interpol headline too. AI fueling more cybercrime in Africa as scams surge.
ERIK: Yep. Scammers don't need frontier models. They need cheap generation, translation, personalization, and volume. That's why small models matter on both sides. Builders use them for defense. Attackers use them for noise.
JOSH: So more AI means more filtering.
ERIK: More filtering, more provenance, more verification. Email-reactor style workflows are a good example. Don't ask, "is this email scary." Ask, who sent it, does the domain match, is the link new, is the payment language weird, did this person ever request this before. Then let the model reason over facts instead of vibes.
JOSH: That sounds like automation before AI.
ERIK: Exactly. AI is better when automation does the boring evidence gathering. Give the model structured facts. Then ask for a decision. That's how you get useful output instead of a confident paragraph.
[pause]
JOSH: Third deep topic. Pi's minimalism. Four tools, tiny prompts, faster coding assistant. Why are people excited about less?
ERIK: Because big agent stacks got ridiculous. Every demo added memory, browser, planner, critic, worker, supervisor, database, vector store, seven prompts, and a mascot. Then it takes 90 seconds to rename a variable.
JOSH: That's not ideal.
ERIK: No. A coding agent needs fewer tools than people think. Read files. Search. Edit. Run tests. Maybe inspect git. That's enough for a lot of work. The magic is not the tool count. It's whether the agent can make a tight loop.
JOSH: Read, change, test.
ERIK: Yep. Fast loop beats fancy architecture. Hermes on my Mac M3 runs qwen3-coder at 70 tok/sec. That matters because latency changes behavior. If the model is fast, you ask it smaller questions, check more often, and throw away bad paths sooner.
JOSH: So local models are not just a privacy story.
ERIK: They're a workflow story. If an agent can run locally, cheap, and fast, you can put it in places where a big hosted model would be annoying. Pre-commit review. Log triage. Config linting. Tiny refactors. First-pass test fixes.
JOSH: But doesn't a smaller agent make more mistakes?
ERIK: Sometimes. That's why you don't give it the nuclear codes. You give it narrow tasks and measurable checks. In HumanDesignApp, the iMessage state machine is a perfect example. The agent doesn't need to understand the meaning of life. It needs to preserve state transitions, message templates, and test behavior. Smaller scope. Better results.
JOSH: That's a good line. Smaller scope, better results.
ERIK: That's the whole thing. People keep asking for smarter models when they need tighter jobs. A small model with a clean task, good files, and a test command can beat a giant model wandering through the repo like it's touring a museum.
[beat]
JOSH: How do you decide when to use Claude versus a local model?
ERIK: Claude gets the messy reasoning. Architecture, weird bugs, tradeoffs, code review, anything where I want deep context. Local models get repetitive work. Summaries, first-pass patches, boilerplate, small inspections, things I can verify with tests.
JOSH: Model selection by risk.
ERIK: Risk and cost. If the action is reversible and tests are clear, use the cheap fast thing. If the action is high-impact or ambiguous, use the better model and add review. This is not emotional. It's routing.
JOSH: That's where most people overthink it.
ERIK: They do. They make it philosophical. I don't care if the model feels smart. I care if the diff is correct, the test passes, and the guardrail catches nonsense. That's engineering.
JOSH: What should builders copy from Pi's approach?
ERIK: Cut the prompt down. Seriously. If your system prompt is a constitution, your agent is probably fighting itself. Give it the role, the boundary, the tools, and the done condition. Then let the repo and the tests carry the truth.
JOSH: Done condition is underrated.
ERIK: Huge. "Fix the bug" is weak. "Make this failing test pass without changing public behavior, then show the diff and test output" is usable. That's the difference between a helper and a liability.
[pause]
ERIK: This episode is sponsored by Prime Automation Solutions. If you're still doing it manually, we automate it. Also, special on a website — $250. primeautomationsolutions.com
[pause]
JOSH: Alright, what's the AI pro tip today?
ERIK: Make your agent write a receipt before it changes anything. One file. Plain text or JSON. Task, files it plans to touch, tool calls it expects to make, tests it will run, and what "done" means. Then after the work, make it compare the receipt to reality.
JOSH: So it's a pre-flight and post-flight check.
ERIK: Exactly. If it said it would only touch one parser and it changed six UI files, stop. If it said it would run tests and didn't, stop. If it can't explain why the diff matches the task, stop. This works with Claude, Codex, local qwen3-coder, whatever. Agents need receipts. That's your tip. Use it.
[pause]
ERIK: If you're building toward financial independence through automation, my first book walks through the whole path. Free chapter at erikandersonbook.com.
[pause]
JOSH: One more thing — we started a Discord for builders. If you're shipping AI, automation, or anything that makes a human obsolete — come hang out. Link at buildorbereplaced.dev.
ERIK: Post what you built. We'll post what we're building. Real wins, real builds, no fluff.
[pause]
ERIK: Build or be replaced.
JOSH: If you want these signals in your inbox every morning, scanbrief.dev. See you tomorrow.