JOSH: It's Friday, July 10. This is Build or Be Replaced — powered by ScanBrief.dev. I'm Josh, here with Erik Anderson.
ERIK: Today's theme is trust. Not the soft kind. The kind you enforce with roles, logs, tests, and hard stops.
JOSH: Stick around — Erik's got an AI pro tip at the end about making agents prove their work before they touch anything important.
JOSH: [pause]
JOSH: First headline. OpenAI released GPT-5.6 and a new ChatGPT Work product. Bigger model, more workplace agent features. What's your read?
ERIK: Better models are useful. But a smarter model inside a sloppy workflow is still sloppy. The win is the whole loop: task, tool access, diff, tests, review, approval.
JOSH: Second headline. European lawmakers moved forward with private-message scanning rules critics call Chat Control. That sounds less like software and more like policy with a database attached.
ERIK: Exactly. The intent and the architecture are different things. Once you build the power to inspect private communication, the next fight is who gets to use it.
JOSH: Third headline. Anthropic rolled out a Claude reflection feature, basically a usage dashboard for how people use the assistant. Helpful or weird?
ERIK: Both. Seeing your AI usage is good. But if a system can summarize your habits, it can reveal more than people think. Metadata is not harmless just because it looks tidy.
JOSH: [pause]
JOSH: Start with GPT-5.6. Everyone wants to know if this is the next big jump.
JOSH: [beat]
ERIK: The jump I care about isn't a leaderboard number.
ERIK: It's whether the model can work inside a real system without acting like it owns the place.
ERIK: That's where builders need to be ruthless. GPT-5.6 may be stronger. ChatGPT Work may be useful. Fine. I want to know what it can read, what it can write, what tools it can call, what gets logged, and where the stop sign lives.
JOSH: So you're not buying the model-release hype by itself.
ERIK: No. Model releases matter, but workflows eat model releases for breakfast.
ERIK: A model can be brilliant in a chat window and still be dangerous in production. Those are different games.
ERIK: In a chat window, the cost of a bad answer is usually annoyance. In an automation pipeline, the cost of a bad action can be a broken deploy, a bad firewall rule, a deleted record, or some 2:00 AM nonsense you now own.
JOSH: That's the part people skip. They see the demo, not the blast radius.
ERIK: Yep. The demo is always clean. The real world has stale docs, flaky tests, weird permissions, half-migrated services, and one repo that was last touched by a person who left three jobs ago.
ERIK: That's why I treat the model like a worker, not a wizard.
ERIK: Give it a job. Give it the narrowest access that job needs. Make it write down what it did. Then make something else judge the result.
JOSH: Something else meaning another model?
ERIK: Could be. Could be Claude reviewing GPT. Could be GPT reviewing Claude. Could be a deterministic script. Could be a human gate. The important part is separation.
ERIK: The actor doesn't approve itself.
ERIK: That's not an AI principle. That's basic operations. Same reason you don't let the person who wrote the change be the only reviewer on a risky production diff.
JOSH: How does that look in your systems?
ERIK: PrimeBus carries the event. The agent picks up the task. It tries the fix. The reviewer checks the diff, test output, and repo rules. If the change doesn't clear the bar, it gets blocked.
ERIK: No speech about confidence. No "trust me bro." Pass the checks or sit down.
JOSH: That is very Gandalf.
ERIK: That's why the gate is named Gandalf. The whole job is to say no.
JOSH: What should a normal team copy from that without building your whole lab?
ERIK: Pick one narrow workflow.
ERIK: Not "let AI run engineering." That's how people create a very expensive problem with nice markdown.
ERIK: Pick one repeatable failure. A broken test from a dependency bump. A stale Terraform provider. A lint issue. A missing import. Something boring and common.
ERIK: Have the agent propose the fix. Make it open a pull request. Require three things in the PR body: what failed, what changed, and how it was verified.
ERIK: Then add a second check. If you don't have another model in the loop, use a script. If you don't have the script yet, use a human checklist. The structure matters more than the fancy part.
JOSH: Where does ChatGPT Work fit into that?
ERIK: Workplace agents are going to push this problem into every department.
ERIK: Engineering has been arguing about code agents. But now the same pattern hits Gmail, Slack, Drive, CRM records, spreadsheets, support queues.
ERIK: If an agent can draft a document, fine. If it can send that document to a customer, different story. If it can update a CRM field that changes commission or support priority, now we're in controls territory.
ERIK: The mistake is treating "work assistant" as one permission. It isn't. Reading a doc, editing a doc, sending a doc, and triggering a downstream workflow are four different powers.
JOSH: That's the line. Tool access is not one switch.
ERIK: Exactly. Give me scoped tools, scoped data, scoped actions, audit logs, and revocation.
ERIK: Also, don't give the agent your whole company drive because it made a nice spreadsheet once. That's not trust. That's surrender with a better UI.
JOSH: [pause]
JOSH: That connects hard to the European message-scanning story. Different topic, same trust boundary.
ERIK: Same muscle.
ERIK: People hear privacy and think it's a values debate. It is, but it's also engineering. What can the system see? Who can request access? What gets stored? Who audits it? What happens when it's wrong?
ERIK: Those are architecture questions.
JOSH: The pitch is child safety. Nobody wants to be on the wrong side of that.
ERIK: The goal matters. But good intent does not make a surveillance mechanism harmless.
ERIK: You have to separate the stated reason from the capability being created.
ERIK: If you build infrastructure that can scan private messages, you've created a tool. After that, every future government, vendor, agency, lawsuit, panic, and political fight can ask to expand the tool.
JOSH: And the expansion is always framed as reasonable.
ERIK: Always. "Just this one category." "Just this one threat." "Just this one emergency." Then the exception becomes the baseline.
ERIK: Engineers should understand this better than anyone. Temporary scripts become permanent services. Debug flags become product features. One-off access becomes an integration.
ERIK: Systems have gravity.
JOSH: That's bleak, but accurate.
ERIK: It's not even bleak. It's normal. That's why you design boundaries early.
ERIK: In network automation, I don't give a script broad device access because I'm feeling optimistic. I define the role. I define the commands. I log the change. I keep rollback close.
ERIK: Cisco NSO is powerful because it models intent and state. But if you point bad intent at a good tool, congratulations, you automated the bad thing.
JOSH: So the builder lesson is don't create invisible inspection.
ERIK: Yes.
ERIK: If your product reads user data, say what it reads. If an agent acts for a user, log the action. If humans can review private material, make that explicit and auditable.
ERIK: Hidden access rots systems. It also rots trust.
JOSH: How does that show up in smaller teams?
ERIK: Internal tools are the sneaky place.
ERIK: Somebody builds an admin panel. Then it can view customer records. Then it can impersonate users. Then logs are spotty because "it's just internal." That's how bad patterns start.
ERIK: Internal does not mean harmless.
ERIK: PrimeDash is boring on purpose. It shows me service health and status. It doesn't need to reach into every database and read everything. The dashboard should answer the operational question, not become a universal skeleton key.
JOSH: Wait, really? You don't want the all-powerful admin console?
ERIK: Absolutely not.
ERIK: The all-powerful admin console is how teams turn one compromised account into a company event.
ERIK: Give tools the power they need. No more. And when you add more, write down why.
JOSH: There's the AI connection again.
ERIK: Yep. Same thing with agents.
ERIK: People are wiring agents into email, calendars, repos, ticket queues, payment tools, and production APIs. That's going to create huge wins and some extremely dumb outages.
ERIK: The winners won't be the teams with the fanciest prompts. It'll be the teams with clean permissions, review paths, and logs that tell the truth.
JOSH: [pause]
JOSH: Third story. Claude reflection. It shows people how they've been using Claude over time. Part productivity mirror, part "your chatbot knows your habits."
JOSH: [beat]
ERIK: I like the idea, with caution.
ERIK: Builders need feedback loops. If you're spending half your Claude time rewriting the same prompt, you should know that. If you're using it for shallow work and avoiding the hard thinking, you should know that too.
ERIK: A usage mirror can be useful.
JOSH: What's the caution?
ERIK: Summaries are data.
ERIK: People act like a summary is safer than the raw log. Sometimes it is. Sometimes it compresses the sensitive part into one clean sentence.
ERIK: If Claude can tell you, "you mostly ask about legal disputes, job changes, medical symptoms, and financial stress," that's not nothing. That's a profile.
JOSH: That's wild. The dashboard may be more revealing than the chats.
ERIK: It can be.
ERIK: Metadata is usually where the story lives. Who you talk to, when you work, what topics repeat, what tools you connect, what you avoid. You don't need every message to understand a person.
JOSH: Should people turn it off?
ERIK: Depends on the work.
ERIK: For normal personal usage, I like reflection if the controls are clear and the user understands what's included. For sensitive work, legal work, medical work, customer data, or anything under contract, be careful.
ERIK: The same advice applies to team AI logs. Don't store everything forever because storage is cheap. Cheap storage has very expensive consequences.
JOSH: How do you handle that in your own workflow?
ERIK: I separate work by sensitivity.
ERIK: Some tasks can go through general AI tools. Some stay in local systems. Some get redacted before the model sees them. Some don't go to a model at all.
ERIK: That sounds slower, but it's not. Once you build the routing, it becomes normal.
ERIK: ScanBrief can score public news. That's fine. PAS website-pipeline can generate site copy for a public business. Fine. But credentials, private customer context, and production secrets don't get tossed into a prompt because I'm in a hurry.
JOSH: That's the boring rule that saves you later.
ERIK: Boring rules pay rent.
ERIK: Also, reflection is a good reminder that AI usage itself needs review. Not just output quality. Usage.
ERIK: Ask what the model is doing in your process. Is it drafting? Reviewing? Deciding? Acting? Those are different roles.
ERIK: If it drafts, you review. If it reviews, something else decides. If it decides, you need an appeal path. If it acts, you need logs and rollback.
JOSH: That sounds like your AI pro tip trying to sneak in early.
ERIK: It is. I have no shame.
JOSH: Before we get there, tie the three stories together for me.
ERIK: GPT-5.6 says agents are getting more capable. Chat Control says inspection powers are politically tempting. Claude reflection says even usage patterns become sensitive data.
ERIK: Same lesson across all three: define the boundary before the system grows around you.
ERIK: The future is not "use AI" or "don't use AI." That's too simple.
ERIK: The future is whether you can build systems where AI has a job, a lane, a log, and a boss.
JOSH: [pause]
ERIK: This episode is sponsored by Prime Automation Solutions. If you're still doing it manually, we automate it. Also, special on a website — $250. primeautomationsolutions.com
JOSH: [pause]
JOSH: Alright, what's the AI pro tip today?
ERIK: Make your agent produce an evidence packet before it does anything irreversible.
ERIK: Not a paragraph. A packet.
ERIK: Three fields minimum: intended action, source evidence, rollback plan.
ERIK: If it's editing code, the source evidence is the failing test, the file touched, and the test it ran after. If it's sending an email, the source evidence is the user request, the recipient, and the exact draft. If it's changing infrastructure, the source evidence is the ticket, the target, the diff, and the rollback command.
ERIK: Then add one rule: no evidence packet, no action.
JOSH: That's simple enough to use today.
ERIK: Exactly. You can add it to a prompt. You can add it to a GitHub Action. You can add it to a Slack approval workflow.
ERIK: The trick is making the agent prove its path before it gets power.
ERIK: Don't ask, "are you sure?" Models are always sure when the sentence sounds nice.
ERIK: Ask, "show me the evidence, show me the action, show me how we undo it."
ERIK: That's your tip. Use it.
JOSH: [pause]
JOSH: Binge all five episodes this weekend plus our YouTube shorts — links at buildorbereplaced.dev.
JOSH: [pause]
JOSH: One more thing — we started a Discord for builders. If you're shipping AI, automation, or anything that makes a human obsolete — come hang out. Link at buildorbereplaced.dev.
ERIK: Post what you built. We'll post what we're building. Real wins, real builds, no fluff.
JOSH: [pause]
ERIK: Build or be replaced.
JOSH: If you want these signals in your inbox every morning, scanbrief.dev. See you tomorrow.