Wout Laban is the senior manager of customer success for Leanplum. Leanplum helps mobile teams orchestrate multi-channel campaigns, from messaging to the in-app experience, all from a single mobile marketing platform.

Questions Wout Laban Answered in this Episode:

• What are some of the most fundamental things a mobile marketer should know about A/B testing, and how do I apply that?
• What is the difference between optimization and experimentation? 
• What changes in optimization are worthwhile?
• How does an increase in advertising spend or world events affect A/B testing? 
• What does perfect A/B testing look like for a brand?
• What’s a typical challenge you see brands face when starting out with A/B testing?

Timestamp:

• 8:22 The fundamentals of A/B testing
• 9:08 Experimentation vs optimization 
• 10:00 Optimization that works across industries
• 12:20 Defining success metric criteria for testing
• 16:17 Setting up to excel at A/B testing
• 18:37 Onboarding and conversion 

Quotes:

(8:22-8:42) “It starts and ends with having a very structured strategy or process towards testing. There are so many things that can influence a test that you really want to make sure that you’re setting up a controlled environment that allows you to look at the data correctly and make decisions based on that. Because that’s essentially what you’re doing, right? You’re running an ‘A’ versus a ‘B’ and at the end, you want to make a decision.”

(9:13-9:44) “So experimentation or an experiment is really something that’s totally new out-of-the-box that’s like, ‘Hey, what if we did this in a totally different way?’ And that can be part of the user journey and how you engage with the user. It can be in the user acquisition and how do we reel users in. Optimization is really where we bring out the small, little tweaks, like flipping out a word or flipping out a small part of the image or making the gradient tone a little bit stronger. Is this going to help us further optimize a strategy that’s already working? So, experimentation is finding a strategy, optimization is optimizing the strategy.”

Mentioned in this Episode:

• Wout Laban - LinkedIn
• Leanplum