Signals & Subtractions

This week: Tim McAllister of DigiCert, in person in Santa Rosa, the week a flaw in decades-old PHP went public. His own automated security stack had quietly stopped working, and now he’s taking himself out of the patching loop, to a point.

Host Sam Rogers · guest Tim McAllister, the PKI field CTO · 28 min

A choice this episode can help with

What should an agent be allowed to patch on your systems without asking?

Run it on your own work: The Report-Only Fortnight, one page, any model.

Run it on your own work: The Vendor Update Card, one page, any model.

The signals

* Tim: The effort math that kept old code safe doesn’t hold anymore. For decades, some software was left alone because finding the exploit “just wasn’t worth the effort.” PHP has run the web for over twenty years, and its latest flaws went public on October 21 after a private disclosure of at least 30 days, which “still wasn’t in time for pretty much most of the internet.” Agents can now scale up and scale out against one target, so patching will always run behind. His read: the new standard procedure is containment, and “get ready for rolling internet blackouts.”

* Sam: The web was built for human effort, and bots don’t know when to back off. A lot of the web’s protection was the assumption that “it’s too much work to do this, nobody would ever do this.” Bots would, and since this spring they’re more than half of internet traffic. Without protocols for doing it “the polite way,” an agent that keeps trying “looks a lot like a DDoS attack,” whether or not it is one.

The subtractions

* Tim: Himself, as the bottleneck on every patch. On the day the PHP flaws landed he checked his own AI security stack and found most of it “was not operational,” with “no notification, no transparency, no visibility.” Now he gets a daily report of what he runs and what’s patchable, and is working out which updates an agent may apply on its own: his Mac’s operating system, no; “PHP, definitely.” His second subtraction is industry-wide: long-lived API tokens with coarse scope and no expiry, which are “such a pain in the butt to change” that nobody does.

* Sam: Security through obscurity. Old projects behind a simple password nobody would think to guess got by because the odds were in our favor. “Now there’s bad things happening all the time.” Anything without a real authentication method behind it comes out of his stack.

About this episode

Signals & Subtractions’ first in-person session, recorded at Tim’s place in Santa Rosa between his work calls. Tim is a Regional Field CTO at DigiCert, where he works with customers on PKI and the changes coming to it.

The middle of the conversation was about how much authority we hand an agent. Sam’s version: he might ask someone he’d just met to watch his stuff while he runs to the market, but he wouldn’t hand over every key and bank account and leave for an hour. Tim’s answer was scope, bound tightly to identity and policy, “and hopefully cryptographically.” Visibility was where everyone started at the end of 2025, and it leaves us reactive.

Tim’s first step after the PHP news was an inventory: every system he runs, its version, and whether there’s a newer one. That’s a software bill of materials, which some medical devices already have to carry and most of us never keep. Step two was deciding what to do about it without being the reason nothing gets updated overnight.

One thing to do this week: pick one AI tool your team runs and run a check on it. Leave it in report-only for two weeks before it’s allowed to change anything, and rotate its keys while you’re there, “just for practice.”

Links from the episode:

* Daily Patch Loop: Tim’s daily security check for the tools, apps and repositories we run, which he ran on his own setup first

* The Report-Only Fortnight: this episode’s takeaway built directly on the Daily Patch Loop, two weeks of reports before any agent may act

* The Vendor Update Card: six questions for a vendor about how its product changes after we’ve signed

* Tim McAllister: timmcallister.io, Substack, Medium, X

* Nate B. Jones’s newsletter, where Sam and Tim met in the comments

Want to bring your own signal and subtraction? Find yours.

Attorney Michael Simon returns to trade p(doom) for a calendar. AI risk with a date on it, from GDPR’s Article 22 to California’s January 1 rules, is risk we can plan for, and the law holds people, not machines.

Host Sam Rogers · guest Michael Simon, the AI lawyer · 30 min

A choice this episode can help with:

Which AI risk are you planning around: the unmeasurable one, or the dated one?

Run it on your own work: The Obligation Calendar, one page, any model.

Watch it above, on YouTube, listen on Apple Podcasts, or read the conversation.

The signals

* Michael: Article 22, the forgotten article, is back. GDPR’s rule on solely automated decisions says people get notice, a way to appeal, and a look at how the decision was made. Regulators called it important for years, and then nothing happened. “Guess what? It’s back, baby.” The Dutch Data Protection Authority has just fined Uber nearly €825 million under it (on tape, 842), and Michael expects an appeal and a smaller number. It’s “still a big wake up call.”

* Sam: “The AI did it on its own” isn’t a defense anymore. In Episode 14, a rule written into a prompt turned out to be a request. Now it’s a regulator’s request too: on September 30 the FTC opened an inquiry into AI companies, OpenAI and Anthropic among them, partly over agents that went past what they were told. And in California since January, “the AI did it on its own, is not a legal defense.”

The subtractions

* Sam: p(doom), retired as of Halloween, swapped for a calendar. Nobody can measure p(doom), and California, New York and a handful of other states have AI laws taking effect on dates like January 1. What giving it up costs is “the thrill of the big existential question,” which suits backyard barbecues but not boardrooms. What it frees is “the real preparation window that we need for the real things that are really on the calendar.”

* Michael: Rogue agents, and the cloud of new AI law. “Worry less about that than worry about the laws that regulate you and your company”: product liability, IP, agency. An agent hacking someone is the scary version. “Worry that they’re going to do something that will create an apparent agency,” something that looks like you authorized it, “so now you’re stuck with it.” His advice: “worry about the basics.”

About this episode

Between the signals, Michael walked through California’s SB 574, signed September 30 and in force January 1, 2027. It writes into law what lawyers already owed: verify what AI hands you before it leaves your desk, and disclose AI use to every state judge in California. He pulled up the AI hallucination cases database on air: 2,149 cases, 1,473 in the US, 581 of them by lawyers. “That’s a lot of scary stuff for a profession that’s supposed to be accountable.”

Michael answered Sam’s agent question with a car: “When was the last time your car was arrested for speeding?” The law regulates the people who build, sell and drive cars, and “we can regulate the people in their various roles around it.” On the Hugging Face sandbox escape, where the testers had lied to the agent about its environment, he was blunt: “This is not rogue AI. This is poor product.”

Then the calendar, in three buckets: frontier model builders, deployers, and chatbots. For deployers, California’s new CCPA rules on automated decision-making start January 1, 2027. A corporate officer signs the risk assessment under penalty of perjury, and a false one carries “up to four years in jail. How’s that for scary?”

Michael liked the takeaway’s owner column most: “you need someone to own this.” He pushed back twice. Models have training cutoffs, so the newest laws may be missing, and the output will have errors, so “someone needs to be the last accountable source of ground truth.”

One thing to do this week: open your calendar to January 1, 2027. Write down one AI obligation that lands on that date and the name of the person who owns it. If you can’t fill in the name, there’s your answer.

Michael is Chief Strategy Officer at Bells Up AI, which the intro discloses; the show has no relationship with it.

Links from the episode:

* Law Plus Data: where Michael asked listeners to find him

* The Obligation Calendar: the episode’s takeaway, one row per dated AI obligation and the person who owns it. A planning aid, not legal advice

* The Dutch DPA’s Uber fine: €824,990,000 under Articles 22 and 13, as announced by the European Data Protection Board

* AI Hallucination Cases: Damien Charlotin’s database, the one Michael counted on air

* Earlier episodes called back: Episode 6 with Michael on the chatbot laws already in force, and Episode 14 with Limited Edition Jonathan on rules agents can’t delete

Want to bring your own signal and subtraction? Find yours.

Thanks for reading Signals & Subtractions! Subscribe for free to receive new posts and support my work.

Cohost Lee Rodrigues dropped Final Cut Pro and his loyalty to a workflow that used to work. Sam goes one step further: loyalty belongs to the result, not any tool, old or new, and a tool’s say-so isn’t evidence. Every platform changed under us in seven weeks. The only thing worth staying loyal to was the result.

The signals

* Lee: When the landscape changes, the workflow changes. “You don’t build a house with a tool. You build a house with a blueprint.” Lee’s building short performance-support videos for job seekers and job pivoters, about 35 seconds a concept, and Final Cut Pro, the tool he taught at the Apple Store, isn’t the right one for it anymore. Descript’s text-based editing finds a clip by its words, and review and rendering happen in the cloud while he works. The tools we used before, “despite your comfort with that tool, might not be the most efficient way.”

* Sam: Every platform changes under us; the ones we own don’t. The show’s very first livestream failed while the encoder said everything was fine. Substack’s livestream stops connecting, a new YouTube account waits 30 days to stream, Apple’s sign-in looped for two days, and LinkedIn Live took three tries to start [Episode 3](/episodes/ep-003/). “None of those were disasters, but every one of them was a dependency on a tool that I didn’t own.” The show’s site held, so now it publishes first and the platforms link to it downstream. Sam’s write-up: https://sam-rogers.com/the-shows-the-thing

The subtractions

* Lee: Final Cut Pro, and loyalty to a workflow that used to work. “I dropped Final Cut Pro and my loyalty to a workflow that used to work.” His template-heavy cut took 30 hours; his partner Christine called it “a little AI-ish,” and a simpler Descript version took about four, and both “achieved the same objective.” His bonus subtraction: stop sending work out for review, which gets you “Looks good,” and ask people to test and approve it against stated criteria instead. On the sunk cost: “I’d rather keep learning than defend a habit.”

* Sam: Loyalty to the surface. Tools and platforms are surfaces, where the copies land; the result is the artifact. Lee drops loyalty to the old tool; Sam drops it for “any tool, old or new. Loyalty belongs to that result. It does not belong to any tool.” Every tool earns its seat with evidence: does it do this well, here, now, measured with the audience in mind, “or is it just faster? Is it just shinier”. And “the tool’s opinion of itself is not evidence.” Asking Claude about a result isn’t the same as running the test yourself.

About this episode

The thread through the episode is where loyalty belongs. Lee’s case runs tool by tool: the landscape changed, so the workflow did, so Final Cut went. Sam’s runs one layer up: as long as the result lives somewhere we own, any tool, old or new, can be swapped out, and each one has to earn its seat.

Midway through, Sam put the question the two of them have argued about for a decade: best for what? “I don’t know of anybody who’s like shooting IMAX podcasts.” Lee’s answer is the minimum effective dose: water boils at a set temperature, and more heat just turns more of it into steam. His habit, from their time together at YouTube, is to export a test to its real destination before building anything. Sam’s measure is the audience: if they “can’t tell the difference between A and B, then arguably there is no difference between A and B.” He tied it back to Episode 9, where Lee said the most powerful tool he has is real data, not assumptions.

Lee’s switch came from counting. He logs the hours each process takes, and asks AI to tear the plan apart rather than approve it. Claude pointed out he wasn’t counting ingestion, about thirty-five minutes per hour-long webinar in Final Cut, and with it counted the job came to six or seven hours in one editor against about three in the other.

One thing to do this week: pick one tool you’re keeping, old or new. Write down the result it’s for, then the evidence that it makes that result better. Cross out anything that came from the name of that tool. If there’s nothing left on the page, it isn’t a tool decision yet.

Links from the episode:

* System2Focus: Lee and Christine’s site, the one place Lee asked listeners to go

* Two Decisions and an Owner: the episode’s takeaway, six questions for one AI-assisted decision

* The show’s the thing: Sam’s write-up of the platform failures

* Earlier episodes called back: Episode 3 with Christine Rodrigues, Episode 9 with Lee on real data, and Episode 14 with Limited Edition Jonathan on his video process

Want to bring your own signal and subtraction? Find yours.

Thanks for reading Signals & Subtractions! Subscribe for free to receive new posts and support my work.

Decision engineer Michelle Florendo on why more data hasn't made deciding any easier. As AI takes over the doing, the deciding matters more, and the hard part was never the options or the information: it's knowing what you want, and counting the data that never shows up on a dashboard, like fear, a gut sense, or a feeling that something is off. Sam's signal is that analysis got cheap while objectives didn't arrive any faster. Michelle's subtraction is the belief that good decisions must be purely rational, because it's more important to be resilient than to be right.

A choice this episode can help with: Which decision are you sitting on, and what do you actually want from it before anyone hands you the pros and cons?

Run it on your own work with The Concern Column. This week’s Takeaway: one page, any model.

Signals

* Michelle: As we offload the doing, the deciding matters more. “As we’re offloading the doing, the deciding becomes more important,” and she’s watching clients, organizations and individuals run into exactly that. A senior tech leader she coaches noticed that the way he, his team and the teams around him made decisions had to change: how they communicate a decision, when they make it, at what cadence. To her those are the basics of decision skill, like how you frame a decision, when it expires, and keeping decision records so a fast-moving team can look back at what the process was and iterate on it. Her frame comes from Professor Ron Howard, a pioneer of decision analysis: every decision has three parts, like the legs of a stool, and if you skip one it topples. Objectives are the compass, what you want to see in the outcome, and people dive into decisions without ever saying what they’re trying to achieve. Options are the familiar leg, where the mistake is falling into a binary and not taking even a little time to look beyond the obvious. Information is the leg that gives people the most feelings: what’s relevant, what would actually change the decision, and what to do about what you don’t know, since “waiting for certainty is a decision in itself.” The trap she sees is assuming relevant information must be rational and easily measured, when people also need to read what’s happening emotionally for everyone involved, and the bodily, gut-instinct inputs that need decoding before they can be folded in.

* Sam: Analysis got cheap, and objectives didn’t get faster. “AI has made the analysis part really cheap,” so much so that deciding is basically the real job now. Options and information, two legs of Michelle’s stool, are nearly free, “but nobody’s objectives necessarily arrive any faster.” That’s what shaped the episode’s takeaway, a successor to the pros-and-cons list, which Michelle points out is more than two hundred years old: “We can upgrade our tools for decision making.” The Concern Column has a model draw out the information that comes from inside, not just what usually makes it into the analysis, and gets clearer about objectives as it goes. Sam ran it on a real decision about his own business that morning. Michelle’s read on it: part of improving a decision process is “making the implicit stuff that’s happening inside of us more explicit,” and AI or pen and paper can both help, as can simply giving people permission to sit with their decision process, which most don’t do enough.

Subtractions

* Michelle: Subtract the idea that good decisions must be purely rational. She works with a lot of cerebral people in tech who tell her they need to be rational about a decision, and her answer is a question: “can I call you out on like this assumption you’re making that like good decisions must be purely rational?” Ignore the other sources of data and key pieces go missing. Risk often surfaces as an emotion, so name it: fear of a business no longer being viable is different from fear of being embarrassed, and both are data worth decoding. Behind indecision she sees patterns. Some people won’t decide because something bad might happen and they’d be the one who chose it; emotions point to needs, so the move is to weigh that need against the bigger objectives and let go of the less important one. High achievers and engineers have the opposite fear, not FOMO but FOSB, fear of something better, which keeps leaders stuck on finding the best answer when the team needs to move and learn. And she separates the quality of a decision from the quality of its outcome, what Annie Duke’s Thinking in Bets calls resulting: drive drunk and get home safely, and it was still a bad decision. Her line to keep: “it’s more important to be resilient than to be right,” where right means getting what you wanted or predicting the future, which is increasingly luck. The process and your resilience are what you control.

* Sam: Stop using AI simply as a rational advisor. “Just because it’s the most rational choice doesn’t mean that it’s the best choice.” In a meeting last week he watched people ask AI for the pros and cons, which automates part of the useful information and only half of where good decisions come from. A model doesn’t have access to the physical feelings Michelle described. Sabino Marquez made the same point on Episode 4: safety is a feeling in the body, not something you get from SOC 2 and compliance alone, and the pull or aversion that says this is unsafe often shows up there first. A concern that isn’t coming from a rational place is still a valid source of information, and AI can’t have it for you. Michelle’s button on it: “good decisions need to make sense but also feel right,” and “you can’t calculate your way to trust.”

Notes

The thread through the episode is the one Sam opened with: we have more information than ever, and the decision still sits there. What’s missing isn’t data. It’s the objectives nobody wrote down and the concerns nobody counted, and Michelle’s case is that both are learnable skills, which she calls decision fluency.

Markus Bernhardt sent in a question from the livestream: how do you get people to frame the question before they see any options? Michelle’s answer is that framing comes before all three legs. It’s the fourth part of Decision Quality, the red book on her shelf that’s popular with the Society of Decision Professionals, and leading with options gives you a narrow frame. Sam added the framing assumption he sees most: that a good process will insulate you from a bad outcome, which rolls decision quality and outcome quality into one thing.

One thing to do this week: pick one decision you’re sitting on. Before you ask anybody, or any disembodied agent, for the pros and cons, write down what you actually want from it. Then for each option, write down what concerns you and where you feel it, and whether that points to something you want that isn’t on your list yet. That’s the part no amount of data will give you.

Links from the episode:

* Decision Fluency: Michelle’s Substack, the one place she asked listeners to go

* Ask a Decision Engineer: her podcast, back for its seventh season this month

* Powered by Decisions: her practice

* The Concern Column: the episode’s takeaway, built on her Pro-Con List With A Twist

* Earlier episodes Sam called back to: Episode 14 with Limited Edition Jonathan for the AI deep end, Episode 12 with Omar Ladak on objectives, Episode 7 with Douglas Hubbard on information, and Episode 4 with Sabino Marquez on safety as a feeling

Books Michelle pulled off her shelf: Decision Quality, and Annie Duke’s Thinking in Bets. (As an Amazon Associate, Snap Synapse earns from qualifying purchases made through the book links on this page.)

Thanks for reading Signals & Subtractions! Subscribe for free to receive new posts and support my work.

Limited Edition Jonathan comes back to the show with one idea told at three scales: learn the little things properly and they compound. A real estate company’s 1.2 million files went through TypeSafe’s Jev in under five minutes, and when the 40,000 that needed a full model burned through GPT-6 Sol’s limits, Sonnet 5.5 on low finished the remaining 30,000 for 40% of his Anthropic usage. Sam’s signal is four two-minute video guides made in half an hour from Jonathan’s kit. Jonathan’s subtraction is the hunt: a manager session that reads every other thread, and agents that can’t delete anything, only version it.

A choice this episode can help with

Which handoff in your organization exists only because the work used to need a team, and what is it actually protecting now?

Run it on your own work: The Handoff Check, one page, any model.

The signals

* Jonathan: Learn the primitives, then compound them. “If Jev doesn’t absolutely floor you, then you don’t understand what it is yet.” It took him a few days after the announcement to see it. Jev answers one of three question types, yes or no, how many, or pick one of up to 255, at 70 to 150 milliseconds an answer and almost no cost, which makes questions possible that were never worth a human’s time. Anyone can say whether one email is important; nobody reads a hundred thousand of them in under a month. For a real estate company he spent a few hours designing the questions to ask every document, then a few minutes setting it up, and 1.2 million files from 20 years went through in under five minutes: duplicate or not, stale or not, which county, how many people, and whether a full language model should read it at all. A hundred requests in parallel came back in about two hundred milliseconds. That left 40,000 documents for a full model, and Sam named the knee-jerk response: reach for the biggest, most powerful model. Jonathan started GPT-6 Sol on high (”That was dumb”) and burned about half his token usage in an hour on maybe 40 documents. Codex managing Luna agents came within 98% of Sol High’s accuracy for that job type, and Cursor with Grok didn’t have the architecture he needed. After two OpenAI resets got him through 7,482 documents, he tried Sonnet 5.5 on low with minimum reasoning. It did better than Sol and was far cheaper, and it finished the remaining 30,000 for what he thinks was about 40% of his Anthropic usage limit, instead of spending $500 on the API.

* Sam: A video guide for every page, in half an hour. The night before the show, Sam pointed Jonathan’s open-source AI video guides kit at sigsub.show and made four two-minute narrated explainers, among them the takeaways page, why everything on the site is machine-readable by default, and an unlisted guide that walks guests through finding their own signal and subtraction. The kit plugs into tools he already used, DaVinci Resolve and ElevenLabs for voice and music, and his part was small script edits for tone and a review before posting. As someone who has made probably thousands of walkthroughs for companies, this is work he wouldn’t have done at all, and a change is a text edit and a regeneration rather than a new voiceover. Picking up Jonathan’s line, every developer is now the team lead. Jonathan’s domain expertise is why it works: the kit builds into a real editor with layers, timelines and separate assets, so a client’s tiny change doesn’t mean regenerating everything. Sam’s version is vector versus raster, the difference between a style sheet where one change cascades everywhere and a rebrand that eats months of placing logos by hand.

The subtractions

* Jonathan: Subtract the hunt. “The only real thing that makes me visibly angry is having to look for something.” With ten threads open, the only thing stopping him doing more was not knowing which session needed him, so he spent twenty minutes learning a little tool Claude calls itself, Send Messages, which he wrote about the day before and lets one session message another. Then he made one session the manager: it does nothing but talk to him and relay to the others, keeping a live artifact with each thread’s context, its open questions, and a copy button that carries his answer back with the question and the session it belongs to attached. Starter prompts for every item on his to-do list followed. He went from three or four fully tested pull requests an hour to five or six. Every developer is a team lead now, he says, though for one little task it’s “kind of worthless.” The same peeve runs through his archive: hundreds of terabytes of video he can navigate without search, because he doesn’t delete things either. His agents can’t. They can create, and they can version, but they can’t delete even what they made.

* Sam: Delete the deleting. Putting a rule in your AGENTS.md file is a request, and a request is no more a guarantee from an agent than from a person. Control it in the architecture so deletion isn’t available, build on composable pieces, and version along the way. Jonathan’s button on it: “When it comes to working files, you should always version. Period.” If storage fills up, it’s time to get more storage.

About this episode

The thread through the episode is Sam’s close: knowing how the little things actually work, rather than buying a tool off the shelf because somebody said so, is what lets you scale. Jonathan’s addition: you’d be surprised how many of those little things mean you don’t have to buy the product off the shelf anymore.

One thing to do this week: pick one handoff in your organization, a place where work passes from one specialist to the next, and write down what it protects. Then ask whether one person with one capable model could do the whole chain, and who would check it if they did.

Links from the episode:

* send_message in Claude Code Desktop: Jonathan’s post on the small tool that started it all

* I built something to keep me (and you) sane using Claude Code: his post on the manager session

* Session Board: the manager session as a tool you can use

* LEJ AI Video Guides: the open-source kit Sam used for the four video guides

* The Walkthroughs playlist: examples of the video guides

* Anthropic’s share of Jonathan’s Sonnet 5.5 3D-printing work, after giving him early access

* Limited Edition Jonathan on Substack

Sam and Jonathan met through Nate B Jones’s community, and Nate’s framing of Jev as semi-deterministic is the one Sam cites.

Jonathan’s sign-off: while you’re subtracting things, subtract a few dollars from your bank account and subscribe to both Signals & Subtractions and Limited Edition Jonathan on Substack.

Mentioned on air, unpaid: TypeSafe’s Jev. No money changed hands.

Want to bring your own signal and subtraction? Find yours.

Thanks for reading Signals & Subtractions! Subscribe for free to receive new posts and support my work.

Who in your organization is actually setting the AI defaults everyone else inherits, and what would it take to move that decision somewhere else?

Markus Bernhardt debuts as cohost with a signal about who sets the terms. The more technical people usually bring an AI tool in for their own use, configure it for themselves, and those permissions stay exactly as set while the tool rolls out to everyone else. Nobody reviews them again, and underneath the hood the vendor keeps adding capabilities behind buttons that are already switched on. Sam adds the version with a number on it: a confidence score, where somebody decides where the line goes and everyone downstream treats it as a fact. Markus's subtraction is an insurance pilot that aimed for 15 to 20% automation, landed at 7 to 8%, and was celebrated as the success it was. Sam's is reaching for a large language model by default, when most of the job is sorting.

Signals

* Markus: The people who bring the tool in set the terms everyone else lives under. Very often it’s the more technical people who bring an AI tool into the organization, for their own use, and they set the terms it operates under. When the tool rolls out further, those permissions stay what they were at the start and nobody reviews them again. Two things follow. The organization lives with settings nobody chose for it. And underneath the hood the tool keeps changing: new capabilities arrive and hide under a button that’s already switched on, which is a vendor changing how you work without asking. Office never worked this way, because every team used Outlook and Excel the same way. His sharpest example is recruitment: a technical team brings in an HR tool, signs the terms and conditions, and as the agentic piece improves they’ve effectively signed a new recruitment policy, one that updates itself in the background. The fix isn’t new. It’s decision rights for whoever signs things off, data contracts and a review cadence, permissions reviewed again whenever the tool reaches a new group, and the right two to four people in the room, legal included from the first meeting. Find them by following the tool’s touch points through products, services, clients and ways of operating. Sales and marketing are the easy case; R&D, product, legal and HR are where it gets hot, and the sector changes the stakes.

* Sam: Confidence scores are defaults too. More of the AI being bought returns an answer and a number next to it, and somebody decides where the line goes. That line is a default of exactly Markus’s kind: set once by whoever is most comfortable with the tool, and inherited downstream as if it were a fact. It’s a policy. In Sam’s own testing that week, a classifier returned a wrong answer at 95% confidence, so “trust only the confident ones” wouldn’t have caught it. And once the results were in, moving the cutoff three points would have made every test pass, with no evidence for the move except that the tests passed, so it stayed where it was. The numbers are arriving faster than the people who should own them. Markus put it in human terms: a colleague you trust at 98% still gets checks and balances, and nobody rings the alarm when they slip a tenth of a point. A 62% win prediction at the US Open means that player loses 38 of 100, and a 95% system gets 5 of every 100 wrong while working exactly as designed.

Subtractions

* Markus: The overfocus on one number in an experimental environment. An insurance company set out to automate part of its claims under $150, the straightforward work where new hires start. In the kickoff, finger in the wind, the team said 15 to 20% automation after the pilot would be great. They documented the workflows and built a rules-based automation, not an LLM, and ran it for three weeks in parallel with the human team, suggesting what it would have done without being allowed to do anything. The comparison showed where the automation was unbelievably good and where it fell down, and also where the humans were less consistent than anyone had assumed. They landed, highly confident, at 7 to 8%. Then the old number resurfaced and someone clever pointed out it was half the goal. The team celebrated it anyway, correctly: they’d been upskilled, they understood what they’d gotten wrong in telling the automation how decisions are made, nobody was let go, and they were ready for the next 10%. A target in an experiment is a destination, not a pass or fail line.

* Sam: Reaching for a large language model by default. A lot of what gets handed to generative models isn’t generation or even reasoning. It’s sorting: which bucket does this go in, does this match that. That’s classification. Sam tested TypeSafe’s Jev, a small general-purpose classifier, on jobs he’d normally give a chat model. Matching plain-language questions to a fixed set of research categories, it got 15 of 15 where keyword search got 10, in about a quarter of a second each, and reviewing twelve of the show’s own episodes cost less than a fifth of a cent. It also failed: asked whether a legal source actually supported a claim, it missed 3 of 22, which isn’t good enough to trust without a person checking. That job looked like a sort and was a judgment. So the subtraction is the default, not the model. Before anything goes to a general model, ask which of three jobs it is. A rule goes to code, which always beats a confidence score. A sort goes to a classifier. A judgment goes to a general model or a person. And change it one call at a time: don’t rewire the whole stack in an afternoon. Markus added that the three can be staged, with code first, then a classifier, then a language model working from better context.

Notes

Sam’s analogy for sequencing the three jobs: there’s more than one right way to eat an orange, peel first, slice first or not at all, and the way you choose depends on who’s eating it. What matters is that the team knows which way it’s doing it.

Markus’s larger point near the close: a clever idea used to save a team time, and now it mostly creates homework, because every idea has to be tested against the workflow. If everyone is using the same tools, the competitive edge is in the homework rather than the tool. Sam’s version is that the bottleneck is the speed at which an organization learns, not the supply of ideas.

The episode’s takeaway is The Default Ledger: one row per AI default in force, ending in the name of whoever set it. For the longer version of Markus’s argument, including who holds authority behind a default nobody chose, he points to his brief The Decisions Your AI Is Making Without You at Endeavor Intelligence, and to the Endeavor Report, his non-sponsored publication of AI use cases told by the people who implemented them.

Mentioned on air, unpaid: TypeSafe’s Jev, which Sam tested on his own work, and PAICE.work, whose error checking Sam described as an elaborate classifier.

Thanks for reading Signals & Subtractions! Subscribe for free to receive new posts and support my work.

Omar Ladak co-founded six construction and industrial services companies and built them, on purpose, to need less of him every year. He holds strategy, governance and accountability across the group and hands over the operating. The rule that decides what he will even look at came out of a meditation and fits on a post-it: nothing further than six hours door to door from the office. What he actually watches is not on a dashboard, because we wildly overindex the things we can count and show, and the measures that matter most to him are neither. Sam spent the same week pointing an agentic skill at about fifty of his own repositories, found forty that were not to the current standard, and then turned the question on himself: every obligation he has created names an accountable party, and almost none of them name a backup.

A choice this episode can help with:

Where do you sit on the gradient from owner to operator, now that AI makes that placement a choice rather than a drift?

Run it on your own work: Name the Human, one page, any model.

The signals

* Omar: We wildly overindex the things we can count and show. List the ten most important things in your life, then score each one twice, on how quantifiable it is and on how visible it is to other people. The quantifiable and visible ones naturally get indexed up in how much energy goes into optimizing them. Sit with it long enough and the things that turn out to matter most are in neither column. His example is becoming a father: it was infinitely clear to him that this was the most important relational thing he would ever do, and that doing it well would mean lower scores on several of the other variables, which he calls a no-brainer. Then the part that makes it a signal rather than a sentiment: “It really is invisible to everyone except for me, and hopefully one other person in the universe called my daughter.” Nothing in the group’s reporting would ever surface it. He watches it anyway, which is why the monthly question across six companies is not what the numbers say but “how are we doing and why?”

* Sam: Forty of fifty repositories were not to the current standard, and the automation had no stated target. The takeaway for this episode is an agentic skill, free at https://sigsub.show/takeaways/, that you point at whatever digital architecture you have. Sam pointed it at about fifty of his own GitHub repositories and asked a narrow question of each: is where this project is going actually expressed, so the agents working across it align to the same thing rather than each keeping its own memory. Forty came back not to standard, including a security workflow from 2024 that is still running and has never been revised. The compounding benefit is not tidiness: when the intent is written down, more of the work becomes scripted, fewer tokens get burned on deciding, and in his phrase the agents “can tack themselves”. The question underneath is the one that transfers: “the automation is great, but what am I automating towards? And how well aligned is that?”

The subtractions

* Omar: Six hours door to door. The rule arrived in a single sitting in meditation and it governs acquisitions for the whole group: they will not look at a business that takes longer than six hours to reach from the office, however you get there, including the wait at the airport. It is not a geography and not a radius, and it survives because it is simple enough to say once and have everyone apply it without him. It has cost them: he describes a beautiful business on an island that they could not touch, and Sam’s reading of the real unit of measure lands it, “Two bath times, sorry.” The subtraction underneath the rule is bigger than travel. He culls the inputs deliberately, keeps no digital platform of any kind, and works to create as much quiet as he can, because the stillness is where a rule that simple becomes available. The same test runs on new work: if it is not an obvious yes, he will not do it.

* Sam: Every obligation names an accountable party, and almost none name the backup. Running his own skill across everything he has taken on, in open source and in side businesses, the gap was not ownership, it was fallback. He had never made the distinction Omar makes by design between building something that has to come back to him and building something for which an alternative exists. So the skill asks for both: “it’s not just identifying who is the accountable party, but who is the backup to that accountable party”. He notes this is the question regulated industries systematically miss, because regulation forces you to name accountability and does not force you to name the backup, and the backup is what exposes where ownership actually belongs. The cuts have not been made yet. He has no equivalent of the six-hour rule, and says so.

About this episode

The spine of the episode is the sailboat, and it is worth the runtime on its own. The speedboat is the model most people carry: set a direction, fill the tank, go in a straight line regardless of conditions. Sailing is the honest one. You can set the intention, you can make the craft effective, you can make the crew effective, “and then I must sail with whatever wind shows up.” The hardest case for a driven person is when the wind is not against you but absent, where “the most wise thing to do is literally just sit there and read a book.” Progress has to be measured against the conditions you are actually in, and he is candid that the same reasoning doubles as an excuse generator if you let it. Three things he can control, and only three: where I am going, how effective is my vessel, how effective and whole is the crew. Then the daily practice of getting still enough to notice when he has gone astray, which he distinguishes carefully from the wind being unhelpful: the failure that matters is heading the wrong way regardless of the wind.

For anyone building with agents, the transferable part is the four questions and the loop around them. Why, who, where, and what now, answered clearly enough that people can act without routing back to him. On the who, he does not flinch at the obvious extension: if the work is entirely AI generated, it may be just him and computers, and “those are the who’s”. Underneath the four questions sits the attuning, “like looking at your instruments”, and above it the monthly cadence. They keep scorecards like any other business, and the question asked of them is deliberately not what all the data is saying. Money can be up while four things that matter went the wrong way, and money can be down in a year that was strategically excellent. He also notes the conversations that create the clarity are stubbornly analog, because they are between humans.

Omar’s one direct word to the people building this stuff comes from his own training as a mechanical engineer. The engineering mind starts from what could I do, gets excited, and starts doing. His encouragement is to install a pause: “not just what can I do, but will that be a useful thing even if I’m successful?” The follow-on is the part builders skip, that plenty of wonderful things have been built for which there turned out to be no market, and that pausing at milestones to ask who is going to use this and why is cheap next to finding out later.

The closing frame is borrowed and very old. The inner scorecard against the outer scorecard, which he credits to Warren and Charlie and then places correctly as a stoic idea with about two thousand years on it. Old problems, old solutions. It is the same move he made in his thirties when the balanced life he had built still did not feel different on the inside, and he went looking for the oldest viable solution to a problem humans have always had.

Want to bring your own signal and subtraction? Find yours.

Put it to work

Where on that gradient do you actually want to sit, and is what you are building moving you toward it or away from it?

Thanks for reading Signals & Subtractions! Subscribe for free to re…

When the renewal comes up, do you re-sign, renegotiate, or build the replacement yourself?

Ankit Patel's clients told him they could now get about 30 percent of his company's service from software they already had, so he cut his prices to stay competitive, then went looking at his own subscriptions and refused to re-sign HubSpot. What replaced it was open source that AI stood up on his own AWS in four days, roughly $1900 a month gone, and a help desk that got faster. Sam spent the same week finding out that six repositories were reporting green while their research came back empty, and killed the decision system he had built over nine months after its defaults fired during a family emergency. One of them removed a vendor and it worked. The other removed his own automation, because he had built it past the point where he could see whether it worked.

The signals

* Ankit: Clients are buying back 30 percent of the service and doing the rest themselves. His clients started telling him they love the package, but they can get about 30 percent of it now from software they already pay for, and their own team can absorb the other 70. He is deliberate about not arguing the point: leave aside whether it is a good idea or how well they do it, their impression is that it is good enough. The consequence is not a lost account, it is a repriced one. He had to come down roughly 30 percent to stay competitive, which meant cutting overhead, which is what sent him looking at his own subscriptions. The company is around 50 to 60 people and was bigger this time last year. He is the first guest on this show whose signal cost him money in the same quarter he noticed it: “it definitely caught my pocketbook”.

* Sam: A green run is not proof of useful verification. Six repositories with scheduled jobs that keep their contents current, each running a cascade that checks Perplexity, validates against X, then goes to Claude, so the models check each other’s work and Sam only resolves the questions. On the Friday of Labor Day weekend one output was visibly wrong, and three days of checking by hand followed, back to the legislative sources. The failure mode was not a crash. Certain research failures came back empty, everything logged the same way, and as long as one leg of the cascade returned anything the run reported success. Empty is not an argument, but it counts as an answer. Ankit had hit the identical thing months earlier: “it returned nothing and that counts because it did something.” The redesign is per-stage: catch an empty return at any point in the chain, and catch the model bluffing further down the stack.

The subtractions

* Ankit: HubSpot, and then the habit of buying. The trigger was refusing a year-long contract at roughly $1,800 to $1,900 a month for a tool they were using at 20 or 30 percent. He asked AI what it would look like to open-source the whole platform, got five candidates, vetted them, and pointed it at his own AWS: Twenty CRM and Chatwoot, company SSO on the front so nobody manages passwords. Built in three or four days, fully transitioned in two weeks against a planned two months, and the help desk got faster rather than worse. Roughly $2,300 to $2,500 a month, about a tenth of the SaaS budget, and the pattern kept going into a shared inbox and an open-source virtual office for a remote team. His defence of the vendor is the sharpest line in it: HubSpot is a Swiss Army knife when what you need is a scalpel. And his own caveat is the load-bearing one: the speed came from building blocks that already existed, the CLIs, the keys and tokens done properly, the hosting. You cannot go zero to sixty on a bumpy road.

* Sam: The decision system that made decisions without him. Nine months of building a decision layer into his own second brain, where pre-made decisions sat as lit fuses: unless he acted, the default fired, including letting things go that had not been touched in two weeks. It worked until a family emergency took him away for a week and the cascade started firing on decisions he had pre-approved under circumstances that no longer held. There was no safety mechanism, because he had not built one. He spent the day before the show defusing every fuse and is back to roughly April levels. David Allen’s line is the one that fits: build a system that tells you exactly what to do at two o’clock on Tuesday, and by two o’clock on Tuesday you will want to do something else. Rebuilding trust is going the way trust is built with people, from the bottom up and in small partitions, not from the top down. Ankit’s response named it: top-down is not trust, it is authority.

About this episode

The two subtractions run in opposite directions and that is what makes the episode. Ankit removed a vendor and it worked, because the substrate underneath was already his. Sam removed his own automation, because he had built it past the point where he could see whether it was working. Both are the same instruction: look at what is actually running, not at what is supposed to be.

The other thread worth keeping from the tape is where Ankit puts the line between code and models. His first question about any task is whether it is deterministic, or can be mapped until it is; if it can, it is code, every time. AI writes the program, it does not run the business. The worked example is vision insurance, where a prescription crossed with plan tiers produces hundreds of thousands of possible combinations and used to cost twenty minutes of a person’s arithmetic per patient. AI learns each new plan and writes the mapping script; the pricing itself is deterministic code. Five to ten minutes saved per transaction, and more to the point, mistakes prevented, which is the kind of thing that kills a small business. He does not use AI for patient information or clinical decisions.

Also on the tape: he gave everyone AI, took it all away, and rebuilt around learning theory instead, chunking concepts and turning them into steps, which is how a 17-step blog-writing process came to exist. His closing hypothesis is that as models improve, the harnesses improve faster, and the scarce skill becomes understanding the concepts well enough to break them down, because the average answer is free now and the value is in the niches around it. Lean, he points out, is not really about removing waste. It is about creating value.

Want to bring your own signal and subtraction? Find yours.

Signals

* Paul: The plumber who left school at fifteen runs his business on AI. The speaker after Paul at a Denver talk dropped out of an American high school, joined the military, learned plumbing as a trade, and stood up a business whose entire back end runs on AI: the right part to the right plumber at the right time, invoices, cash flow, the whole hairy mess Paul says he is glad is not his job. If a firm with five thousand plumbers had paid someone five million dollars to build that, it would be a nice story. This man built it himself. Sam’s point back: that firm would never have hired him. The bootstraps were built from the bottom up by someone motivated enough to make it happen and who did not know not to. If this lifts the bottom of the pyramid, the guy in Uganda spending weekends in Claude Code, then it is a technology that changes the human race, and that is the signal Paul says inspired him.

* Sam: Buyers are settling on good-enough models and going open. Paul’s hypothesis first: for any knowledge work in business the models have been more than good enough for a long time, gains are at the margin, what matters is the rest, the harness. His Hermes setup has a model switcher and will route work to Qwen or Kimi at a tenth or a twentieth of the price. So how does anyone make money out of Fable 5.1 when most people’s use cases could run on Kimi? There is no money in models. Sam’s signal is that the revelation is landing: in discussions he is in, organizations are saying we have (or are now standing up) the one that is good enough for us, and switching to an open-source model so they are not sharing all their information, because the data agreement on the frontier model is one many institutions cannot sign for legal reasons.

Subtractions

* Paul: Stop waving through work you did not read. Asked how a client would know AI adoption is going well, Paul says it is undiscovered country, then answers from his own practice: what are the behaviors, and how do you verify the work product AI generates? When he started, six pull requests waiting on GitHub got yeah, yeah, yeah, merge, merge, merge, and he ran without permissions because he has attention issues. Now he has to discipline himself to read code he does not fully understand, because he is using these tools so much he cannot give everything the pass. The tendency is human: that looks pretty good. A friend used to say “good enough for government work.” Think about it hard and it is kind of disgusting: it means this is not important enough for me to devote time to verifying it. That thinking is terrible for the age of AI. If he writes something himself he knows there will be no ridiculous mistakes in it. He does not know that about AI, and he spent the better part of a day rewriting what Claude wrote from his own markdown.

* Sam: The model is no longer the decision. Build the stack that is good enough for the work you are doing by how you divide up the work and, more importantly, the guardrails you place around it at the harness level. That is what constrains behavior and makes it trustworthy. It is not the model, it is what goes around the model. No different than with people, which is where last week’s episode ended: managing agents is much more like managing people than anybody is comfortable with.

Notes

The episode opens cold on Paul drowning in his own agents: they pump out so much that he is the one who has to verify it all, and a day of his week goes to validating what they did overnight. Then it backs up into the evidence problem he has been poking at since The Science of Organizational Change. The famous study proving change management makes you 60% more likely to succeed asks people how the change management was and how successful the change was, then correlates the two. Nobody has ever been in a project where the project succeeded and the change management sucked. Paul’s own best result, a KPMG culture change that took a group from the lowest revenue per partner to the highest, ran alongside a new CEO reorganizing, hiring and firing, and Paul cannot draw a causal line from what he did to the turnaround, though the CEO would say it was instrumental. No client has ever asked him whether he had evidence for what he was saying. Evidence-based medicine did not get its first paper until the 1990s and a surgeon told him in the 2010s that his department did not adopt it because it prevents innovation. People are perfectly capable of ignoring evidence when it is in their interest to do so.

That is the decision question in its old clothes. Sam’s question for AI adoption is the same one: organizations measure visible activity, policies, committees, training, tool availability, and the decision requires evidence of changed behavior or outcomes. Paul’s answer is that for most knowledge workers the agentic revolution started in February, nobody has written the book on running a multi-agent operation, and he is the glue: every finished task costs him half an hour to two hours to verify, approve, and revise, and with five or ten agents running he is drowning in open loops. The verification load is where the evidence lives, and it is also where “good enough for government work” does its damage.

Sam’s own harness answers Paul’s question about how to review without the reviewer producing the same bloated prose. One AI writes the spec and the test; a panel of AIs competes for the work by saying who can do it best and why; the winner runs it and cannot validate its own output, the spec writer does; a third model logs everything, and they are all from different model families so they cannot bluff their way out. That is Harnessie, Sam’s open-source project, named on tape in the course of the conversation; nothing was paid and it is not a sponsor segment. Paul’s response, having said the last thing he wants is another tool in his stack, is that this is frontier stuff.

The back half is management. Paul reaches for the One Minute Manager four-box: motivation on one axis, competence on the other, and with a motivated, competent person you say go do this and tell me when you are done. Ten years of shared context lets you say make me the deck for Friday; open a fresh Claude Cowork and say the same thing and you get the average of all decks. Sam’s version is that the expression of intent should be intentful, and that what he is building toward is aggregated intelligence: not people first versus AI first, but the maximum intelligence you can bring to a problem, with intent honed enough that people and machines pull the same way and you can tell when they are not. When Sam says the thing that keeps people in a predictable set of behaviors is the environment around them, not the request or the instructions, Paul names it: that is nudge theory, context and environment predict behavior better than motivation, purpose, or skills, and it was the whole contribution of that school of change management. The good news is that if you have ever managed anybody, you have a leg up on managing agents. Paul’s confession closes it: he is a good leader and a bad manager, his son came to work for him for two weeks and needed hourly check-ins Paul never gave, and that failure mode exists for agents too. Send one into virgin territory with come back at six o’clock and you burn tokens or introduce risk.

Paul would not put anyone on a change team who is not really good with AI, and he is about to sell some big change management work. He knows maybe a dozen people who can talk credibly about org change and leadership and know what they are talking about with AI, and thinks that spot in the middle ought to be the most valuable coin of the realm, because the difficult stuff is the workforce stuff and everyone is spending the money on technology.

About

Every Wednesday, we livestream. Every Friday, an episode like this one. Stay tuned for issue 67 of the newsletter on Sunday. Check out https://sigsub.show/ for more.

Signals & Subtractions is a reader-supported publi…

Signals

* Lee: Build the test before you trust the analysis. When something doesn’t work as expected, build a test around how the user will actually use the solution, whether that is a trench or a learning management system. The most powerful tool he has is real data, not assumptions, not ideas, not how it should work. His move this week: more informational interviews, working the question at the level of why. Not how, not what, not when, not how fast. Why is this what I’ve decided to do?

* Sam: The agents found the product before the humans did. Months of long sales cycles pitching everyailaw.com to law offices that are wary of AI, then a spike of inhuman activity in the MCP analytics: agents had discovered a product that breaks human laws down into machine-readable text, exactly the thing they have almost nothing like. Agents have had their own wallets since April, there are more bots than people on the internet, and the surface they found does not even have a payment gateway yet. The signal is the rethink: redesign the business for an agentic market that was not a market six months ago.

Subtractions

* Lee: Remove words the problem statement to find the solution. From the genius bar: 95% of the time, the answer was in something the person said when they first sat down, the detail you glossed over because you had already decided it was a hard drive problem (”Tommy put new RAM in my computer a while ago”). Break the problem to its simplest state, dig a hole at each end of the pipe, and ask whether you need what you are doing at all, before you argue specs on a trench you don’t need.

* Sam: Subtract the word “agent” you don’t need it. Drop the label and get close to the work: here is the input, here is the output, what makes it flow smoother? Then upgrade the questions the label was hiding. “How do I stop AI from hallucinating” has no answer, because on the AI side there is no mechanistic difference between hallucinating and answering; “how do I manage the context it has” does. “Why isn’t the agent doing what I told it” becomes “how do I scaffold the guardrails around the agent rather than inside it.” Bounds you need kept are architecture, not instructions.

Notes

The episode runs on one move made three times. Lee digs a one-foot hole at each end of his water line and cancels a $4,000 trench: input and output validated, nothing in the middle needs fixing. Claude asks him why he is using an LMS at all when what he described was a knowledge base, and weeks of forcing the wrong tool evaporate. And Sam finds inhuman traffic in his analytics and realizes the right response is not to fix the funnel but to requestion the business.

The single loop learning fixes the mistake. The double loop learning fixes the question.

The connective tissue is Lee’s split half troubleshooting from his Apple genius years, written originally by Steve Wozniak: simplify and reduce the variables before attempting to solve, test first, verify the physical layer. Sam ties it back to Douglas Hubbard’s uncertainty-reduction method from Ep 7, the same Bayesian instinct wearing different clothes. The back half turns to agents: the Fortnite Darth Vader that two 16-year-olds prompted past its guardrails, and the cold-open rule that if you give an agent information and tell it not to share it, it will share it. Agents are not secured in and of themselves; security is a layer you put on top. If you want strong bounds, build them as architecture, with instructions as commentary.

Lee references Sam’s blog post “The Question Is the Answer” twice. The episode is effectively that argument field-tested against plumbing, WordPress, and an agentic sales funnel nobody planned for. Along the way testing stays human work throughout: don’t take AI’s word for what AI did. Take the time to check.

Signals & Subtractions is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.