Today I unpack the intense industry reaction to a piece of Albertsons Media Collective, Ovative Group, and Kellogg School of Management research that challenged how we measure iROAS in retail media. When a single campaign’s return can swing wildly — up to 6.5x — based purely on methodology, it raises some uncomfortable but necessary questions. I walk through the biggest critiques that surfaced from practitioners and academics, and why this debate matters more than ever for brands trying to make sense of their performance data.

I also share how the authors of the study responded to the pushback, and what they didn’t address. More importantly, we get into what this all means for real-world advertisers, especially mid-market brands navigating opaque reporting standards. If you’ve ever questioned whether your campaign results are telling the full story, this episode will give you a fresh lens to evaluate retail media measurement.

This episode is sponsored by Mirakl Ads

Timeline

[00:00] – Why controversial research is better than silence, and what sparked this debate
[00:31] – The headline finding: iROAS can vary dramatically depending on measurement methodology
[01:24] – Key critique: Are these methodologies even comparable in the first place?
[02:07] – The BSTS debate: causal method or misapplied forecasting tool?
[03:08] – A practical question: What are brands actually using iROAS for?
[04:30] – The authors respond: what the study was, and wasn’t, designed to prove
[05:39] – What mid-market brands should do next: pushing for transparency and better disclosure

Links & Resources

• The white paper in question, titled iROAS Demystified, is a collaboration between Albertsons Media Collective, Ovative Group, and professors from Northwestern University's Kellogg School of Management. 
• My original article on the white paper: iROAS results can be easily gamed. But we shouldn't just throw it away
• The LinkedIn comment thread in which the research was fiercely debated
• Check out Prof. Koen Pauwels' Substack Amazon Days
• Read my related articles:
  • Why ROAS Refuses To Die
  • Retail Media's Measurement Problem: It's Not Just the Retailers
  • 'Not 100% altruistic': ad measurement veteran on what retail media is really optimizing for
• On Thursday May 7 I am hosting a LinkedIn Live event: How Kroger Precision Marketing Is Plugging Into National Brand Budgets ! Join me, Christine Foster (head of Kroger Precision Marketing) and Jon Flugstad (Metarouter) to hash out the details of Kroger's partnership with DV360 and how this development is a sign of maturity in the retail media ecosystem. Bring your questions, we'll be doing this LIVE.
• I'm keynoting at Xnurta's Signal to Scale summit on May 19 in Chicago. This is the room to be in if you work in retail media, care about what AI actually changes, and want to leave with something you can put to work. A full day focused on agentic AI, modern measurement, and the strategies the best teams are already building toward. Reserve your spot here.
• I'm sharing comedy skits on Instagram about retail media! Follow along: @retailmediabreakfastclub
• Subscribe to Retail Media Breakfast Club's daily newsletter
• Follow Kiri Masters on LinkedIn