Token Intelligence

AI is dangerous because it agrees with you


Listen Later

AI's sycophantic design means bad thinking upstream produces confident, polished output downstream. The problem isn't just the design of the tool; it's the thinking you bring to it.

Summary

Eric opens with a quote from historian David McCullough: "Writing is just a great deal of hard thinking." The insight applies well beyond writing. Whether you're drafting a strategy, debugging code, or analyzing data, the output of AI is only as good as the thinking that precedes it. When the upstream thinking is flawed, everything downstream inherits the flaw.

John illustrates this with the story of Amazon's competitors and the assumption that undid them: people need to see, touch, or try a product before buying it. Amazon didn't dispute that instinct; they found a proxy for it through honest reviews and generous return policies. Their competitors never questioned the assumption, looked to each other for validation, and paid for it for a decade.

From there, Eric and John surface a subtler problem: AI makes this worse. Because AI is sycophantic by design, it tends to validate whatever framing you bring to it, fill in ambiguous gaps with its best guess, and carry that direction forward with confidence. John shares two real examples from his own team that week, including one where a bug report that arrived with a proposed solution led everyone, human and AI both, down the wrong path for hours. The fix was simple: go back to the problem before the proposed solution, and if you're correcting the AI too often, that's a sign to start the conversation over.

Key takeaways

Bad upstream thinking scales with AI: AI doesn't correct your assumptions; it amplifies them. The sycophantic nature of the tools means flawed framing gets carried forward with speed and confidence.

Separate the problem from the proposed solution: When a bug report or task arrives with a built-in answer, the first step is to go back to the raw problem and reproduce it, not to chase the suggested fix.

Good thinking feels like a lot of hard work before you pick up the tool: The people who get the best results from AI come with a well-formed thesis before opening a chat window, not after.

Disagreeing with AI more than agreeing is a healthy sign: The best AI users are the ones telling the tool it's wrong more often than nodding along. That friction is a feature, not a bug.

Ambiguity in a plan is filled in by AI's best guess: When a complex system has gaps, AI will resolve them toward what it believes your intent to be, which may not be right. Sharper goals produce better results.

If you're constantly correcting, start over: Steering an AI conversation that started from the wrong premise is harder than reframing and starting fresh. The instinct to push through is often the wrong call.

Amazon’s competitors failed by not questioning assumptions: The retail incumbents looked to each other for validation instead of questioning the underlying belief. AI will do the same; it mirrors the assumptions in the room.

Notable mentions and links

David McCullough's observation that "writing is just a great deal of hard thinking" anchors the episode's core argument, with Eric citing it as one of his favorite quotes about the craft; McCullough was a two-time Pulitzer Prize-winning historian known for books on John Adams, Harry Truman, the Panama Canal, and the Wright Brothers.

... (Read more at the episode page)

...more
View all episodesView all episodes
Download on the App Store

Token IntelligenceBy Eric Dodds & John Wessel