Experiment Nation: The Podcast

S5E4 - Ethical Testing & Design in Experimentation: Leaks vs. Puddles Framework (w/ Namata Agarwal)


Listen Later

In this episode of Experiment Nation, host Jason sits down with Namata Agarwal, a seasoned product designer with over 17 years of experience building digital products, websites, and apps.Together, they explore:- Ethical testing: how to balance user needs with business goals.- Leaks vs. puddles framework: a practical way to identify urgent vs. non-urgent product problems.- The role of design in CRO and experimentation.- Frameworks, guardrails, and collaboration strategies for building an experimentation culture.Whether you’re a CRO specialist, product manager, or experimentation nerd, this episode is packed with actionable insights you can bring to your own team.πŸ‘‰ Don’t forget to like, comment, and subscribe to stay updated on the latest in experimentation and growth!πŸ“– Check out Prove It or Lose It: The (Mostly) No-Nonsense Guide to Surviving Experimentation Program Drama .⏱️ Chapters with Timestamps0:00 – Why design must be involved early0:22 – Welcome & intro to Jason0:43 – Guest intro: Namata Agarwal0:55 – Topics: Ethical testing & Leaks vs. Puddles1:18 – What is ethical testing?2:22 – Balancing business needs vs. user needs2:54 – Handling stakeholder pushback4:01 – Misaligned metrics and real solutions5:17 – The Leaks vs. Puddles analogy explained6:32 – Why better hypothesis building matters7:48 – Using Airtable & frameworks for test prioritization8:40 – Why experimentation must be collaborative9:24 – Reframing experiments around the user journey10:00 – Unpredictable user journeys & design challenges11:12 – Involving guest collaborators in experimentation11:50 – Mapping user journeys with Miro & qualitative research12:50 – Jobs To Be Done & other qualitative frameworks14:44 – Turning qualitative insights into hypotheses15:32 – How product design elevates CRO strategies16:11 – The importance of forms in experimentation17:36 – Why design should be included from day one18:20 – Design is not just visualsβ€”it’s about impact19:11 – Governance, responsibility, and inclusivity in testing19:46 – Leak vs. Puddle framework deep dive22:58 – Running collaborative workshops25:45 – The β€œStinky Fish” workshop format27:21 – Micro vs. macro frameworks for problem-solving28:05 – Balancing bold ideas vs. safe bets29:19 – Planning tests like an investment portfolio30:13 – Testing across 13 sites at scale30:26 – What Namata would A/B test in daily life31:57 – Art, painting, and creative balance32:42 – Martial arts & keeping calm32:56 – Wrap up & final thoughts33:14 – How to connect with Namata



This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit experimentnation.substack.com
...more
View all episodesView all episodes
Download on the App Store

Experiment Nation: The PodcastBy Rommil from Experiment Nation

  • 5
  • 5
  • 5
  • 5
  • 5

5

2 ratings