What does it really mean to run SEO as an experiment instead of a checklist?
In this episode, we’re joined by Will Critchlow and Mat Reiss to explore the evolution of SEO and AEO/GEO A/B testing — from enterprise-level experimentation frameworks to a super simple way for any team to get started: title tags.
Will breaks down the fundamentals of SEO and AEO/GEO testing and how it differs from traditional user-based A/B testing. Because Google effectively acts as the “audience,” experiments must split pages rather than users and be implemented server-side so crawlers can see the changes. We discuss why organic traffic is a stronger measure of impact than rankings alone, how counterfactual forecasting helps isolate true lift, and why modern SEO increasingly feels like experimenting against a black-box, AI-driven system.
Then I called up Mat to help us get real practical, really quickly — with “title tags”, what he calls the “last mile” of search performance. He explains why small copy changes can meaningfully increase click-through rate without improving rankings, shares a case study that drove a 30% traffic lift from a simple tweak, and outlines how teams of different sizes can begin testing title tags without heavy engineering resources.
In this episode:
How SEO and AEO/GEO A/B testing differs from traditional experimentation
Why organic traffic is often a more reliable KPI than rankings
The statistical and structural realities of page-based testing
Why title tags are an underused but high-impact growth lever
A real-world example of a 30% traffic lift from a simple title change
When SEO testing becomes viable for small and mid-sized e-commerce teams