From A to B

Are synthetic users actually valuable? Or are they just another part of the AI hype cycle?

Nils 'Dolla Billz' Stotz joined this ep to flesh out his research into synthetic users, and I gave him significant pushback. We ended up agreeing on more than I was anticipating tbh, but he is clearly doing the research to see how synthetic users are valuable now! And he shared what he's currently doing today.

We got into:

- Exactly where synthetic users excel and are valuable, but where they hold significant limitations as well

- Holding synthetic users to the standards of humans (and where they are actually creating double standards)

- The issues with 'predicting' in CRO (and why are we too focused on it)

This is the post I referenced in the episode for the homie Craig Sullivan (who also deserves a follow): https://tinyurl.com/FromAtoBCraigSullivan

Go follow Nils on LinkedIn, his content is great and he posts basically every day.

https://www.linkedin.com/in/nilsstotz/

Also go follow Shiva Manjunath on LinkedIn:

⁠⁠⁠⁠⁠https://www.linkedin.com/in/shiva-manjunath/⁠⁠⁠⁠⁠

Subscribe to our newsletter for more memes, clips, and awesome content!

⁠⁠⁠https://fromatob.beehiiv.com/

What do you .... actually do in your first 30 days as a CRO? Besides like health insurance and get a laptop, and fight with IT to get access to all the tools?

Gertrud “Valhalla” Anirudhan joined this ep to share her wisdom, AND journey, into this, as she recently joined a company and actually went through those 30 days. She's ex-Speero, so you know we talked mad shit about Ben Labay ;)

We got into:

- What you should aim to accomplish in 30 days, tactically (not just 'build a culture of experimentation)

- We debated a bit of 'tactics' vs. 'politics' you should balance in 30 days (I won obviously, but she made a strong case)

- She shared really cool stories of her experiences, one of which is ... making a case for copying competitors? (I totally agreed with her, which for y'all who know, I hate copying competitors.

Banger episode, even cooler human being. Go follow Gertrud on LinkedIn, she's amazing.

https://www.linkedin.com/in/gertrudvahtra/

Also go follow Shiva Manjunath on LinkedIn:

⁠⁠⁠⁠https://www.linkedin.com/in/shiva-manjunath/⁠⁠⁠⁠

Subscribe to our newsletter for more memes, clips, and awesome content!

⁠⁠https://fromatob.beehiiv.com/

This is a bit of a unique episode - not your usual "how to build a culture of experimentation" type episode. And you shouldn't skip this episode because 'it's not experimentation so i don't care' - this episode is SUPER important for anyone in an individual contributor role to hear her wisdom.

Mallory "Take the Mantle" Randall is a seasoned veteran who's gone through all of the different stages in her career. And she's learned super valuable career growth things, moving from individual contributor, to managing a small team, to managing MANY teams as VP.

So many CROs and experimentation people fixate too much on their program, they don't realize there are people outside of the organization with different goals, trying to accomplish different things. This is why her POV is so important.

We got into:

- When you start your job, understanding your peers is just as important (if not more important) than actually getting your program off the ground

- Peers need you to push back - understanding HOW to push back, thoughtfully (and respectfully) is key. And picking and choosing your battles will help you grow in your career and get more things done

- There are limitations to 'being right' - just because you're right, doesn't mean people will listen to you.

Go follow Mallory on LinkedIn (we told her she's going to post more though leadership now, so stay tuned for amazing content coming from her):⁠

https://www.linkedin.com/in/mallorysiler/

Also go follow Shiva Manjunath on LinkedIn:

⁠⁠⁠https://www.linkedin.com/in/shiva-manjunath/⁠⁠⁠

Subscribe to our newsletter for more memes, clips, and awesome content!

⁠⁠https://fromatob.beehiiv.com/

We are temporarily renaming this podcast From D to A - From Data to Action. Because data is almost useless unless you do something meaningful with it.

Josh "One of a Kind" Silverbauer hopped on the pod to dig deeper into the 'analytics' and 'data' part of product building. Identifying better ways to tell the 'data' story, and better ways to communicate that data so you actually can get shit done.

We got into:

- Josh's Pro Tip for consultants and agencies (it blew my mind, I'm serious)

- How 'data' isn't enough, and how presenting trust and authority is critical to how you communicate with data

- What data engineers critically need to do when communicating data

- His honest views on dashboards (is it beyond theater...?)

Go follow Josh on LinkedIn:⁠

https://www.linkedin.com/in/jsilverbauer/

AND HIS BAND....?!

https://open.spotify.com/artist/7ek3eQ8HOLZgtE8FgAy0G2

Also go follow Shiva Manjunath on LinkedIn:

⁠⁠⁠https://www.linkedin.com/in/shiva-manjunath/⁠⁠⁠

Subscribe to our newsletter for more memes, clips, and awesome content!

⁠⁠https://fromatob.beehiiv.com/

In this episode, Shiva Manjunath and Slobodan Manić go over the important lessons the Knicks (winning in 5) taught us about B2B.

It's a super serious episode and we go in depth with a LOT of important things you need to learn, and know, about B2B.

Are fake door tests ethical? We are super purposefully kinda misleading people to see if something is worth building.... (cue Wile E. Coyote reference).

Jon "Metal as Fuck" Crowder had a SUPER interesting post about the ethics of fake door testing, which I had to have him on the pod to discuss this topic. But we also got into a more broader conversation around ethics of A/B testing too!

We got into:

- Are fake door tests truly ethical? (Or legal...?)

- Should there be a 'board of ethics' for A/B testing? (yes. kinda)

- Jon's philosophy around CRO (and how it's influenced by his ethics)

Go follow Jon on LinkedIn:⁠

https://www.linkedin.com/in/jc-awip/

And check out his website: https://www.anotherwebispossible.co.uk/

Also go follow Shiva Manjunath on LinkedIn:

⁠⁠⁠https://www.linkedin.com/in/shiva-manjunath/⁠⁠⁠

Subscribe to our newsletter for more memes, clips, and awesome content!

⁠⁠https://fromatob.beehiiv.com/

Juliana "Run the Jewels" Jackson is someone who I consider a thought leader in the AI/digital marketing space, and I needed her POV on how to understand the impact of LLMs to user behavior.

So I had her on the pod. And she shared some insightful information into how LLMs actually work, how it's meaningfully (or maybe not so meaningfully) changing behavior, and what you should be doing about it.

(PS vote for me for a thought leadership award for Kameleoon https://tinyurl.com/voteforshiva)

We got into:

- How LLMs are evolving user behavior on your site, and what you really should be focusing on (spoiler: it's not what you think...)

- Are we putting too much confidence in AI? (and how do we right-size it)

- How LLMs are free research for you (and how to utilize it better)

Go follow Juliana on LinkedIn:⁠

https://www.linkedin.com/in/juliana-jackson/

And subscribe to her newsletter (it's amazing): https://julianajackson.substack.com/

Also go follow Shiva Manjunath on LinkedIn:

⁠⁠⁠https://www.linkedin.com/in/shiva-manjunath/⁠⁠⁠

Subscribe to our newsletter for more memes, clips, and awesome content!

⁠⁠https://fromatob.beehiiv.com/

Are manipulation and bribery the keys to building a culture of experimentation?

No.

(Well, maybe)

Jenni "Broadway" Bruckman (Director of Digital Product at YETI) joined this week to help broad the topic of building "cultures" which foster data-driven thinking and experimentation. It's different than focusing on explicitly building a culture of experimentation, in that your goal is to basically build a culture of fun, transparency, and engagement. The testing and data comes naturally from there!

We got into:

- Tips on navigating stakeholders (especially in a remote-first workplace)

- Better ways to run meetings and tell stories with data

-The value of your personal brand in scaling testing programs

Go follow Jenni on LinkedIn:⁠

https://www.linkedin.com/in/jennibruckman/

Also go follow Shiva Manjunath on LinkedIn:

⁠⁠⁠https://www.linkedin.com/in/shiva-manjunath/⁠⁠⁠

Subscribe to our newsletter for more memes, clips, and awesome content!

⁠⁠https://fromatob.beehiiv.com/

A/B testing tools - all they do is randomize traffic. So why are we spending thousands of dollars for it?

Jonas "Elvis" Alves (of AB Smartly) joined the pod to discuss some of the major challenges in building A/B testing tools. It kinda started in a way I had no idea it would - that you're not buying a testing tool, but buying a... trust platform?

Jonas throws some shade at other testing tools, but I bleep it out, don't worry ;) . (And thankfully, he doesn't sales pitch at all. )

We got into:

- Jonas makes the case for BOTH building in-house and buying an external tool

- How to earn back trust once it's broken, through data or experimentation

- Booking.com ran A/B tests on A/B testers (it's super meta and super cool)

Go follow Jonas on LinkedIn:⁠

https://www.linkedin.com/in/jonasalves/

Also go follow Shiva Manjunath on LinkedIn:

⁠⁠⁠https://www.linkedin.com/in/shiva-manjunath/⁠⁠⁠

Subscribe to our newsletter for more memes, clips, and awesome content!

⁠⁠https://fromatob.beehiiv.com/

Is 'ethics' something we've undervaluing when it comes to experimentation? Should we actually care about this nerd stuff of 'ethics'?!

Well, yes. Obviously. AJ "Does No Wrong" Long joined Shiva to chat about this specifically, as someone who works in the fantasy sports realm of things, ethics can be a contentious subject.

We got into:

- Why "users first" is really the only way to foolproof ethnical experimentation (and low-key, the only great way to run a business long term)

- How PrizePicks fosters 'responsibility' in fantasy sports (really cool things they're doing there)

- How to set (and enforce) guardrails in programs. Which is stupidly important in experimentation, when people want to break it down and hack it as much as possible

Go follow AJ on LinkedIn:⁠

https://www.linkedin.com/in/angusjlong/

Also go follow Shiva Manjunath on LinkedIn:

⁠⁠⁠https://www.linkedin.com/in/shiva-manjunath/⁠⁠⁠

Subscribe to our newsletter for more memes, clips, and awesome content!

⁠⁠https://fromatob.beehiiv.com/