Today we check up with Frances to see if she's recruited her power user for our live test and discuss the importance of user testing and how to recruit users.
—
Transcript
Pablo: Welcome to Prototype Thinking in Action. I'm Pablo, your host, and this is the podcast where we talk to founders, dive into one big challenge that they are facing, and help them solve it by using the prototype thinking method. Let's get started.
Pablo: Frances, welcome back, it's lovely to have you again with us. I know that J gave
you homework, so to speak, which is always the challenging bit of the process, specifically because this homework was about talking to your power users, your early adopters, those users that we can get really valuable feedback from. So can you tell us how that went?
Frances: So entrepreneurs aren't good at homework, as we all know. I did not recruit power users because I feel weird about it, like I don't I don't know them. I can easily look up the stats on my email list on who has like the highest opens and click through rates, but I don't know these folks, they're parents obviously, they're really busy, and I feel like I can't ask them to do something for me without giving them something. I've been hesitating.
J: I noticed that because we emailed you a few times being like, hey Frances, do you
have your user? I noticed that when I send you emails about other things you respond to them instantly, but you ghosted five messages from our team about user recruitment, so it sounds like there was something here that was tough for you.
Frances: Yeah, I apologize for being a fuckboy on a dating app.
J: No no no no, I mean that, the data, right, like that is the most interesting thing. When things don't go as expected, like that is the juiciest part.
Frances: You know, yeah, like I have no problem talking to people in real life, but this feels like a heavy ask for someone to like do something for me and they don't know me despite the fact that I do believe they've been getting value out of the newsletter. I have this fear that I'm going to look at the data and the people with the highest clicks and opens, the most engaged users
that we are looking for are people that I've had no interaction with whatsoever. I just don't know them. They're complete strangers, and I don't have anything to offer them, to give them back
in return for their time.
Pablo: This is fantastic, Frances, because you're not alone in this. J knows that I've been teaching entrepreneurship and innovation for many many years at business schools and teaching corporations how to do these kind of customer discovery processes and
everyone kind of like gets maybe the logic behind it and the value in talking to customers but then when when they have to take that that leap, that plunge, they are like, Damn, this is not as easy as it seemed in the workshop or in the explanation.
J: Let's dive into this. First of all, I'm curious. Like right now you said that you were nervous about looking at the data and what if they're strangers. Can you go look at the data right now?
Frances: Okay, I'll look at the data.
J: Okay, I actually think it's better if they're strangers. I'm hoping they're strangers, but we'll see.
Frances: Finding the data that we're looking for is actually not as easy as I thought it was going to be.
J: Okay, why don't I give you the homework of going and digging through your email platform and seeing if you can pull a few power users from it and if the data is available and if not, you're just going to have to add a recruitment message at the bottom of a newsletter.
Pablo: I was precisely going to suggest the same thing as J. People will self-select in a way. This is the good thing about having this sort of ask, Frances, if people don't want to talk to you, you're not forcing them to talk to you.
J: Yeah, power users are the easiest users to recruit.
Pablo: Correct. There's a definition that I teach my students, the definition of early adopter and the short version is that an early adopter is the person who has the problem that you're solving, they're aware that they have the problem that you want to solve, and they're looking for a solution, and when you're building a product or trying to design a solution around that, they're eager to talk to you because you are actually going to be doing ad hoc personalized customized consulting for them.
Frances: Yeah, I mean, I've been on the other end of this and have totally been that person, but I've also been a founder, so I'm like oh of course I want to offer feedback, but wouldn't that person be like biased?
J: I think there's a few threads here, right? One is, why do we want to be talking to power users right now, and the answer is, when we want to know what's successful about the past usage of our product, we talk to the people who are already experiencing success. When we want to
know about what's failing, we talk to the people who are experiencing failure. So for example if you came to me and said we have a retention problem, people come, they look at one email and then they never click again, which is not the case with you, then I would tell you, you want
to talk to your reluctant adopter because you want to talk to the people you are failing to retain to figure out why they are dropping. But in this case, what we're trying to do is refine your value proposition for the purposes of a tighter marketing strategy, so for that reason, you want to talk to the people that you are already succeeding with so that you can replicate your successes.
Frances: Say someone responds to that request to speak to users and they actually have like relatively low open rates or clickthrough rates. Or would that just not happen?
J: No, I mean, they certainly could, right? Like they could respond because they didn't like it and they have a beef to talk to you about. So let's talk about how to think about the activity of user recruitment. To begin with, we have a principle that we call user intimacy, and it's essentially like, in early stage the relationship that you want to be having with your users is very close one-on-one with a lot of high bandwidth social and emotional and personal back and forth where you're being authentic about what you're building you're being authentic about what the vulnerabilities are and you're very very socially present for them and you're holding the space for them to show up and be authentically themselves as well in how they're responding to you. So in some sense you're not asking them to come and user test, you're asking them for the favor of coming and having a serious one-on-one conversation with you about something that matters to you and perhaps it also matters a bit to them. So for example, Frances, you're on your prototype thinking journey, and you have been following us. We met you because you
followed me on LinkedIn and you've been following us on substack so imagine that after following us for a bit I reached out to you and I said, hey, Frances, I've noticed that you've been, you know, following me a lot and commenting a lot on my posts. I need to ask you for a big
favor. I am building out a new self-serve prototype thinking process for founders and I'm trying to figure out the best way to design it and which parts of it work for people and I would love to just ask you for your help. Do you think you have 30 minutes to do a one-on-one video call with me so I can learn more about what you need and what you're thinking and so we can kind of
iterate on a few options together and do some co-design and I can build a solution that better works for people like you? And I'm only talking to five or six people at this early stage and I looked at all of the people who have been corresponding with our company and you seem like a perfect fit, so it would really help me out a lot to have your opinion.
Frances: That I would totally feel more comfortable messaging with versus a blanket like is anyone interested in like a 30 minute blah blah blah.
J: Right? Like how does that make you feel when I ask you that?
Frances: Like flattered.
J: Flattered, yes!
Frances: Yeah, what you were saying about that intimacy with users is really interesting because that's kind of what I try to manufacture.
J: Yeah, so even in the worst case, if you're not able to pull individual user information, imagine that the Shitty Mom Guide subscribers got a separate email that said, hey quick request, Shitty Mom Guide needs your help. We’ve been growing and we want to figure out how to refine and improve the parts of Shitty Mom Guide that are more valuable and we just need to talk to a very small number of people at this point. Are you somebody who's been reading Shitty Mom Guide every week? If so, it would help us a lot if you'd be willing to talk to us and we'd love to invite you to be a part of the design of the future of our newsletter in the next year.
Frances: I love that email. Do I have to offer them anything?
J: No, the only type of users that you do truly need to cash incentive are people who are sufficiently low income that they cannot afford to spend time to talk to you unless there's a cash reward.
Pablo: Jay, do you think it's counterproductive to offer them anything? Is it better actually not to? What do you think?
J: So this is how I think about recruitment from a user point of view. People get inundated with user testing requests, right, or user survey requests, and you generally hate them. Why do you hate them? It's because they treat you like a number. I get this lovely email from some platform I'm subscribed to and they're like hey we would love your feedback. I'm like I have so much feedback for you, I would love to give you my feedback and I click through and it's like please answer these three survey questions about things that are not things I care about. Or even like I get into like a one-on-one user test they want me to click on a bunch of stuff and tell them what I'm doing next and I'm like this is not the conversation I want to be having. And the fact is that people actually are dying to have conversations with businesses that are out there to help them and everyone has an opinion and everyone just wants to feel valued but most of how people get taught to do user testing today actively tries to treat users like a number because they're trying to be scientific. But we're not trying to be scientific, we're trying to be intimate here, so it actively makes you feel worse than average to go to a typical user test and get shoehorned through their anonymous process but you're offering something much, much cooler than that. You're like this is half an hour to an hour, I want to give you my undivided attention, no one but you and I want to listen and hear about how you think about things and just talk about your point of view and toss some ideas back and forth and work with you. And most people love that.
Pablo: This is awesome. That's great.
J: People are incentivized to user test for three reasons. The first is people love helping people. I always ask people to open a user recruitment request with just saying, can I ask you for a favor? The best way to build social capital for your company is to ask them to help you because they keep feeling good after they help you. Always say I just honestly need help from you, like I can't do this without you, I need your help. The second is being valued by you, demonstrating in your recruitment language that you're there to listen to them and really like receive their advice. People love giving advice and how often do you get to give advice to somebody who actually is going to listen to you? And then the last piece is exclusivity, the sense that there's something special about you and we're honored to be talking to you. It's not like we're trying to survey 2,000 people and you're going to be a dot on my graph, it's like I'm only talking to six people right now you're going to have your true territory in my brain.
Pablo: Connecting this with what you just said, J, one of the dangers some of my teams found out or one of the dangers that they would run into when they were running customer discovery interviews was that essentially they would take the opinion from randos that were not relevant, that really are not interested in the problem or the solution.
J: It’s okay to fire a user, right? Like when you're testing with consumers, about one in four consumers that you recruit to talk to is a dud. Like they turn out to just not be
able to be authentic or they give you endless advice about something you don't care about or they turn out not to be a qualified user. That's why you just pad a few extra users on your schedule. If you are having a user conversation and you're finding that it's not valuable, you don't have to keep having it. You just have 5 to 10 minutes of the conversation and go, hey, you know, that was super valuable, thank you so much for your time, I got what I needed, I really
appreciate it, bye. Right? And just don't spend the rest of your user test time on them.
Frances: I'll either put a blurb in the next newsletter or maybe just send out a plain text email.
J: I actually think the ladder would be more helpful because then it gets visibility on its own and you can do that social capital building in a way that, by the end of a newsletter, nobody's
ready to receive social capital.
Frances: Yeah, okay, this is scary. Like I used to send out plain text emails all the time at Awkward Essentials and they were the most, people loved them, it was super interesting, but I've done it and I did it with like 200k people, but I don't know, it's like, it's different, it's scary.
J: Pablo, you had said something earlier. I don't remember the exact wording of it but it's essentially like, what if I'm talking to users and they're not qualified, right? And the answer is like you just ask them. Right? And it's like, what if I'm imposing on them? Well, users are capable of consenting to having this conversation with you.
Pablo: Yeah, here here, certainly.
J: And like that's the thing about looking at it from an intimacy perspective, right? Is that I've
noticed there's this thing that happens with founders and team leads of all types no matter how social and gregarious they are naturally, is that when it comes to user testing they just kind of freeze up and they feel like they can't ask about whatever they want and I think it's because there's this like scientificization of user testing that has been drilled into us, right? And we're
Like, what do we do, I'm talking to this person, they're not giving me the answers I want, I don't even know if they’re the right person, like I don't know if they open my newsletter. I'm like, just ask them if they open your newsletter, you know. Like at the early stage and especially with what we know are Frances’s strengths at communication and back and forth and relationship and intimacy building, just play your strengths. You're trying to be these people's friends, right? Just talk to them like they're your friends, trust that they're able to consent to the relationship, ask them what you really want to know. And this goes for the user test too. I see people freezing up in user tests, they're like, I don't really know why this person isn't responding to anything that we're putting in front of them, and I'll just talk to the user and be like, hey I've noticed the last few things we put in front of you, you've been kind of like “blah.” Tell me what's going on with that.
Pablo: I think that's fantastic advice, J. Like don't tell them anything you wouldn't be telling your friend. You don't have to pretend that you're something else.
Frances: Just be normal.
Pablo: Yes, just be normal.
J: Just be normal, yeah.
Pablo: It's great.
J: Yeah.
Frances: Okay, cool.
J: How's that feeling for you?
Frances: No, this is so interesting cuz I think you're right. When people think about user testing, they think about everyone in a room and there's this…
J: Guy in a white coat and a clipboard.
Frances: Like, yeah!
J: I was in one of those, actually.
Frances: I think you assume that if you take that friendly approach, it's biased. Guy in the white coat in a room, that's the unbiased like systemized way to get the right answers, and so this is the opposite. It's really interesting.
J: Yeah, it is, right? And I get that question a lot. Well, won't this bias the user? And I'm like, well they're all different, we're only talking to six of them, bias is all over the place in early stage, there's no such thing as no bias because what we're trying to do is build qualitative deep understanding. Essentially what we're trying to do is find the right biases, the right heuristics to be applying. When you're running high volume tests and you know that the framework that you're testing is valid and you're trying to get mathematical data, then bias matters.
Pablo: So Frances, now that we've been hammering you with this, you can be honest with us. Tell us, how are you feeling about this? Are you feeling more confident?
Frances: It still makes me nervous, but I think that's just like any new thing that you launch, right? Like it's something you care about. I have my homework. I think it was just like, what's the best way to like do it.
J: Definitely. And you're just going to play to your strengths.
Frances: Yeah, the context around like it being a more like intimate ask versus like, “hi users, I would like data” is helpful.
Pablo: Thank you for listening to Prototype Thinking in Action. Please don't forget to Like and Subscribe wherever you get your podcasts, join our email list, and follow us for exclusive access to tools, case studies, and live workshops. All links can be found in the description below. See you next time.
Get full access to Prototype Thinking Labs at prototypethinking.substack.com/subscribe
