
Sign up to save your podcasts
Or


Show Notes
Everyone Is Quoting the Same Number and Getting Different Answers
References from this episode:
Cloudflare, Crawl-to-Refer Ratios on Cloudflare Radar - the primary source for everything in this episode. The full formula, the June 19 to 26 2025 sample week, the user-agent aggregation, the Google ASN exclusion, and the native-app Referer caveat that says the calculations may overstate the ratios by an amount that is unclear.
Cloudflare, Control Content Use for AI Training - the separate June 2025 figures I mentioned, where Anthropic sits at 73,000 to one against the 70,900 in the post above. Same publisher, same month, different window.
Cloudflare, From Googlebot to GPTBot: Who's Crawling and Who's Clicking - background on the crawl-to-click gap and how the ratios have moved since launch.
Cloudflare Radar AI Insights - the live dashboard. Worth pulling your own figure here rather than quoting somebody else's, and worth noting which window you selected when you do.
Google, Search updates from I/O 2026 - the source for the AI Mode scale claims, including the growth rate published without a base.
The Machine Layer - my book, if you want to go further into how these systems get measured and mismeasured. Available here.
The written version is on the Substack. If you have hit a metric in this space that fell apart once you asked what was in the denominator, leave a comment there or reach out directly. Specific examples are more useful than the general argument.
When AI Has Nothing On Your Company, It Describes Someone Else
References from this episode:
Mallen et al., ACL 2023, When Not to Trust Language Models - the finding I mentioned about models struggling with less popular factual knowledge, and about scale improving recall of popular facts while doing very little for the tail of the distribution.
Longpre et al., EMNLP 2021, Entity-Based Knowledge Conflicts in Question Answering - the work on how heavily models lean on memorized information rather than reading what is in front of them, and where the hallucinate-rather-than-read framing comes from.
Sciavolino et al., EMNLP 2021, Simple Entity-Centric Questions Challenge Dense Retrievers - the retrieval half of the argument. Dense retrievers underperform sparse methods on entity-rich questions and generalize reliably only to common entities.
Stanford HAI AI Index - the report the vendor article credited a hallucination rate range to, linked at its front page rather than at any page inside it. Free to download, which is part of why the paywall defense does not hold for this one.
ACL Anthology - free, open, and fully indexed. Worth a look if you want to check the taxonomy paper I could not find, or just to see what a real citation trail looks like.
The Machine Layer - my book, if you want to go further into how these systems build and hold their picture of a company. Available here.
The written version of this episode is on the Substack, and if you have caught a model describing your company from evidence you could not trace, leave a comment there or reach out directly. I am collecting examples.
Episode description
AI answer systems do not simply make search easier. They relocate the work. Traditional search exposed candidate sources before the consumer assembled an answer; generative search increasingly performs retrieval, comparison, and synthesis first, leaving the consumer to judge a finished response afterward. This episode looks at that shift through Jakob Nielsen's cognitive-load framework, why verification becomes more important when synthesis happens out of view, and what SEOs should care about when their information may be extracted, compressed, and separated from the page that originally gave it meaning. The goal is not to make content 'easy for AI.' It is to make meaning harder to lose between retrieval and belief.
What gets covered
· Why cognitive load is better treated as a limited budget than as something that should always be minimized
· How traditional search made the consumer do much of the query formulation, source selection, comparison, and synthesis work
· Why generative search changes the sequence by putting synthesis before much of the consumer's verification
· Research showing that citations can increase trust in AI search results even when the references are wrong or hallucinated
· Why this is a human cognitive-load problem, not an argument that LLMs experience human psychology
· What can happen when retrieval separates a claim from the qualifications, units, chronology, evidence, or entity relationships that make it accurate
· Nielsen's idea of load laundering, and the SEO parallel: removing words is not the same thing as removing informational dependency
· A practical content question for the AI-answer era: if the rest of the page disappeared, would this passage still mean what you intended?
· Why retrieval is no longer the end of the SEO journey when information can be extracted, recombined with other sources, and presented inside an answer the consumer must decide whether to trust
· Why compression that preserves the relationships required for accuracy is a more useful standard than maximum simplicity or simply writing less
Research referenced
Jakob Nielsen, Cognitive Load Is a Budget, Not an Enemy: Design for the Brain's 4 Chunks: uxtigers.com/post/cognitive-load
Research on the distribution of cognitive load during web search: arxiv.org/abs/1005.1340
ACL 2026 study comparing traditional and generative web search: aclanthology.org/2026.findings-acl.526
Microsoft Research analysis of 200,000 anonymized Bing Copilot conversations: arxiv.org/abs/2507.07935
Haiwen Li and Sinan Aral, on trust, citations, and hallucinated references in AI search: arxiv.org/abs/2504.06435
Research on context fragmentation in long-document RAG: aclanthology.org/2025.emnlp-main.63
About
Duane Forrester Decodes is published weekly at duaneforresterdecodes.substack.com.
His current book, The Machine Layer, is available on Amazon.
The Keyword Universe Was Always Smaller Than We Thought
AI answers are settling phrases, not ranking them. The space of real opportunities was never as large as the space of phrasings.
Everything referenced in this episode:
The GEO Tooling and Data Market Has a Trust Problem. The practitioner survey this episode responds to.
Cross-model agreement audit, June 2026. 3,750 responses across three models and 250 category queries. Source of the 41.6 percent, 91.6 percent, 8 percent and 20 percent figures.
Persona conditioning audit. 2,000 runs across ten buyer personas. Source of the 80 percent and 75 percent figures.
Brand recognition experiment, Trine University and Texas A&M. One real brand against nine validated fictional ones with identical specs, across 670 trials.
Google Search Central, AI features documentation. Google on query fan-out in AI Overviews and AI Mode.
Kandpal et al., ICML 2023. Recall tracks how many relevant documents a model saw during pretraining.
Mallen et al., ACL 2023. Scaling improves recall at the popular end and leaves the sparse end roughly where it started.
Ahrefs featured snippets study. Click-through impact of snippets at the time.
The Machine Layer. More on how these systems build and hold their picture of a brand.
I run CitationIQ, an AI optimization data platform, so I have a commercial interest in the answers to things like this. Nearly all published measurement in this space comes from companies selling measurement, mine included. Hold every number loosely.
If you are testing this against your own data and getting a different answer, I want to hear about it. Leave a comment or reach out directly.
Show Notes: The GEO Tooling and Data Market Has a Trust Problem
I surveyed 163 people who work on AI search visibility about the platforms built to measure it. They rate the underlying data at 4.20 out of 5. They rate paying for a platform at 3.19.
When I asked what's actually wrong, price came fifth. Trust and ROI came first and second: 57% raised one of those two, against 7% who raised cost.
This episode covers what they said, what their answers reveal when cross-referenced against each other, and one argument of mine about why the most common objection may have no answer at all.
Method
Fielded 14 July to 3 August 2026. N=163. Self-selected sample recruited through my own network, two newsletter sends, two industry newsletters, and paid promotion on LinkedIn and X. Not a probability sample, so it describes engaged practitioners in one corner of the industry rather than the industry. Roughly ±7 points at this size.
123 respondents (75%) wrote open text. Themes were coded by hand, and many people raised more than one issue, so shares add to more than 100%.
Full numbers and methodology are in the written version.
Disclosure
I build in this category. That's a conflict, and it's most relevant to the section on what vendors can and can't publish about their own methodology. Everything presented as a survey finding comes from the responses. Where I'm making an argument of my own, I say so.
Source
IBISWorld, SEO & Internet Marketing Consultants in the US (NAICS OD4523, August 2025): 714,838 people employed across 362,753 businesses. That category bundles social and display in with SEO, so I use it as a ceiling on who could have answered, not as a count of GEO practitioners.
Everything else comes from the survey responses.
Coming next
Two arguments I cut from this piece because they need their own space: what it means when every AI answer converges on the same thing, and why "snake oil" is three separate accusations wearing one coat.
One question
If you can't have a vendor's methodology, and you can't, what would make you believe a number? Nobody in 123 responses proposed an answer. I'd like to hear yours.
Episode description
"Influence the parametric side" sounds like something you assign to someone with a deadline. It isn't. This episode prices what actually sits behind that phrase: years of separate parties describing a company in their own words, most of it produced before language models were a consideration, by functions that report to marketing but never owned the output. Research from ICML, ACL, TACL, and Johns Hopkins on how training corpora become model knowledge, why volume of self-published content doesn't substitute for variety of independent description, and why the same property that makes parametric standing impossible to author is what makes it hard to lose.
What gets covered
● Why the cost of parametric standing isn't measured in years, and what it's actually measured in
● The difference between training data and parametric standing, and why conflating them causes most of the confusion in this conversation
● Kandpal and colleagues on document count and accuracy, including why a bigger model won't fix a thin footprint
● Allen-Zhu and Li on why repetition from one source does not produce what variety from many sources produces
● The corpus diffusion problem: the most common domain in C4 accounts for less than five hundredths of one percent of documents
● Why polite crawling and robots.txt mean the busiest sites on the internet are underrepresented in training corpora
● Effective cutoffs versus published cutoffs, and why a model's picture of a company is older than its stated date
● The functions that built parametric standing, from PR and analyst relations to crisis communications and review operations
● Why a company controls the review response but never the review
● Knowledge editing research showing that even researchers with direct parameter access can't cleanly change a single fact
● Why the same property that makes this impossible to author is what makes it durable
Research referenced
Kandpal and colleagues, on long-tail knowledge and pretraining document counts, ICML: arxiv.org/abs/2211.08411
Mallen and colleagues, on entity popularity and parametric reliability, ACL: arxiv.org/abs/2212.10511
Allen-Zhu and Li, on knowledge extractability and phrasing variation, ICML: arxiv.org/abs/2309.14316
Elazar and colleagues, What's In My Big Data, ICLR: arxiv.org/abs/2310.20707
Dodge and colleagues, documenting the Colossal Clean Crawled Corpus, EMNLP: aclanthology.org/2021.emnlp-main.98
Cheng and colleagues, on effective knowledge cutoffs, Johns Hopkins: arxiv.org/abs/2403.12958
Cohen and colleagues, on ripple effects in knowledge editing, TACL: aclanthology.org/2024.tacl-1.16
Common Crawl published crawl statistics: commoncrawl.github.io/cc-crawl-statistics/plots/domains
Related
The companion article to this episode, which drew the line between retrieval and parametric memory: Entity Mapping Works on Google. Does Any of It Reach ChatGPT?
About
Duane Forrester Decodes is published weekly at duaneforresterdecodes.substack.com.
The Machine Layer is available on Amazon.
Show notes below, ordered as the links appear in the piece, with the anchor described so listeners know what each one is when they can't see the article's hyperlinked text. The two arXiv papers are the load-bearing sources, so I gave them the fuller descriptions.
Entity Mapping Works on Google. Does Any of It Reach ChatGPT?
Entity mapping treats Google and ChatGPT as one system. They are not, and this episode is about where the line falls and what your entity work is actually worth on each side.
Read the full article: [link to the Substack post]
Referenced in this episode
Google's 2012 "things not strings" announcement, the launch of the Knowledge Graphhttps://blog.google/products/search/introducing-knowledge-graph-things-not/
Schema.org, the structured data vocabularyhttps://schema.org
Wikidata, the collaborative entity databasehttps://www.wikidata.org
Research on how factual knowledge is stored in language models, distributed across parameters rather than held in any single onehttps://arxiv.org/abs/2602.22787
Research finding that entity knowledge and relational knowledge sit in different parts of a model and do not map onto each otherhttps://arxiv.org/abs/2409.00617
Connect
Have a comment or a situation you want to compare notes on? Leave a comment on the post, or reach out directly. I also have an opening in my consulting roster starting mid-August.
Google Went "Not Provided" in 2011 and Blinded Us. ChatGPT Just Shipped Its Version.
The organic attribution you keep asking for structurally isn't coming, and OpenAI's own documentation shows why. Here's what to measure instead.
This week I'm digging into the question that keeps surfacing, unprompted, in my reader survey: attribution. The organic, item-level attribution we were trained to expect isn't coming from ChatGPT or the other answer engines, and OpenAI's own documentation shows why. They've built the measurement for advertisers and handed organic operators a robots.txt file. I walk through the three different things people actually mean when they say attribution, the first-party measurement you can run today without anyone's permission, a one-question test for any vendor in the space, and why the deeper problem underneath all of it is a trust and literacy gap that belongs to us, not to a platform.
Referenced in this episode:
The split I kept pointing at between the visibility switches and the measurement stack lives right in OpenAI's crawler documentation, where you'll find the ads bot sitting alongside the search and training bots.
https://developers.openai.com/api/docs/bots
And here's the paid half I described, OpenAI's own server-to-server conversions pipeline, the pixel and the events API tied to an ads manager account. That's closed-loop attribution that already exists, just behind an ad account.
https://developers.openai.com/ads/conversions-api
The optimistic end of that wildly uneven conversion range comes from Microsoft Clarity's study of more than 1,200 publisher sites, where AI-referred visitors converted to sign-ups at roughly eleven times the organic-search rate.
https://clarity.microsoft.com/blog/ai-traffic-converts-at-3x-the-rate-of-other-channels-study/
And the sober counterweight, the peer-reviewed study in Marketing Science across 973 sites and twenty billion dollars in revenue, found organic LLM traffic converting below every traditional channel except paid social. Both are true, which is the point.
https://pubsonline.informs.org/doi/10.1287/mksc.2025.0489
Finally, if you want the longer argument for why the underlying systems, not the dashboards, are where this literacy has to live, that's the whole spine of my book, The Machine Layer.
https://www.amazon.com/Machine-Layer-Visible-Trusted-Search/dp/B0G2WZKM59/ref=sr_1_1
This week I'm asking whether the fingerprint your SEO work has been pressing into search for years actually carries into the answer engines, or whether they start fresh every time. We walk through where the record provably persists, which is Google's stack and Bing's, where it enters the pipeline and then goes dark, such as in the Bing-to-ChatGPT path, and the layer nobody outside the lab can prove yet, whether the models keep a native, per-domain record of their own the same way traditional search does. Then a practical sorting of your existing work into what pays forward, what does double duty, and what's genuinely new, plus why recovery isn't one thing but a curve.
Links and references
When I said Google's AI features are rooted in its core ranking systems, that's straight from Google's own blog post on AI Mode:
https://blog.google/products/search/ai-mode-search/
On Google measuring sections of a site independently, here's the site reputation abuse policy clarification I referenced:
https://developers.google.com/search/blog/2024/11/site-reputation-abuse
When I mentioned IndexNow as the pipe that pushes freshness signals to the index, here's the protocol itself:
https://www.indexnow.org/
Web IQ came out at Build in 2026, and here’s the link to their own post explaining what it is and what it does: https://www.microsoft.com/en-us/webiq
And if you want the longer argument, my book, The Machine Layer:
https://www.amazon.com/Machine-Layer-Visible-Trusted-Search/dp/B0G2WZKM59/ref=sr_1_1
The written version of this episode is available as the full article on this same Substack.
Show Notes
This week: the web is filling up with machine-written content just as machine readers are about to become its dominant audience, and the two trends feed each other. Retrieval systems already lean toward AI-written text, which quietly narrows the pool of sources behind AI answers while accuracy still looks fine, a state researchers call deceptively healthy. We walk through the mechanism in plain language, why it can't hold, and four things you can do to be the human, evidence-bearing source that survives the shakeout.
Links and references
On how much of the web is already machine-written, the Graphite analysis as reported by Axios:
https://www.axios.com/2025/10/14/ai-generated-writing-humans
On the coming explosion in machine-driven queries, Jordi Ribas of Microsoft, on X:
https://x.com/JordiRib1/status/2061866606670581871
On retrieval systems preferring AI-written text, the SIGIR study that named the effect, invisible relevance bias:
https://arxiv.org/abs/2311.14084
On what happens as synthetic content accumulates in the pool, the 2026 Web Conference paper on retrieval collapse:
https://arxiv.org/abs/2602.16136
On why a system that feeds on its own output degrades over time, the Nature research on model collapse:
https://www.nature.com/articles/s41586-024-07566-y
On the platforms' stated neutrality about how content is made, Google's own guidance on its AI features:
https://developers.google.com/search/docs/fundamentals/ai-optimization-guide
And if you want the longer argument, my book, The Machine Layer:
https://www.amazon.com/Machine-Layer-Visible-Trusted-Search/dp/B0G2WZKM59/ref=sr_1_1
From the publisher's feed