Awkward Silences

We've been more and more curious about how user research can be used by more than just researchers. This week, Erin and JH chatted with Jon MacDonald, founder of The Good, about how marketers can use research to improve conversion rates.

He talked about what conversion rate optimization really is, how to get stakeholders to see the value of research work, and why the key to conversion success is really just giving users what they came to your site for in the first place.

Highlights

• [1:43] Everyone expects Jon to have great stories about one quick thing that 20x'ed conversion rates, but unfortunately the truth is that his work is a lot of research and testing to make meaningful changes. 
• [7:05] The trick to better conversion rates is giving the customers what they came to your site for in the first place.
• [9:33] Why onsite surveys may be hurting your customer experience, instead of getting you valuable qualitative data.
• [12:21] How Jon changed a real stakeholder's mind with feedback and business impact.
• [13:51] Jon suggests starting with quantitative data to identify the right areas to test, then following up with qualitative to learn what's wrong.
• [15:37] How Jon worked with baseball bat maker Easton to improve their buying process and doubled their conversion rates.
• [21:14] Why recruiting is one of the most important factors of research. 
• [27:18] The gang talks about A/B testing, including the tools they use.
• [35:11] How Jon cuts through the noise created by lots of different data sources.
• [38:58] Following up on your work at a regular cadence is a huge part of success.

When Andrew joined Adobe back in November, he faced an interesting design challenge. How do you manage a vast library of help content, spanning tons of different products in different industries, so that users can easily find what they need to know to fix their problems? Andrew has learned a lot about leveraging the power of community to problem solve, experimenting with different formats to make technical explanations more accessible, and proving the value of great help content. 

Erin and JH chatted with him about how he positions help content to stakeholders, tackles creating content for products that have evolved from box software to the cloud, and uses research to focus on the right things at the right time. 

Highlights

• [1:43] Great product doesn't need as much help content, but people will always need help, and the more powerful your product is, the more help they will need.
• [8:53] By connecting customers with the right information through communities, they're 3x less likely to reach out to support for help.
• [10:59] Early engagement = better retention, so Adobe segments out its customers to focus on what gets them started.
• [13:35] Adobe has segments and chapters to keep everyone on the same page and communicating well.
• [25:20] People who get value from help content actually end up having a higher lifetime value. It's all about trust and mutual respect.
• [27:41] How Adobe creates help content for different languages and cultures.
• [32:09] How Andrew is thinking about simplifying solutions, and making help content work smarter, not harder
• [37:58] Help content and product can work together to create even better solutions for users.
• [39:32] Quantitative information can tell you what some of the issues are, while qualitative can help your team dig deeper into why they're there
• [43:03] Building out recommendations for other things you may be looking for helps your help content build a story for the user. 

Jonathon Hensley spends a lot of time creating strategies for businesses. So Erin and JH chatted with him about what's changed since COVID-19 and why alignment matters more than ever right now.  

The takeaway? Everyone needs to be aligned around a common goal. They also need to understand how every project supports that larger goal. In a world that's moving faster than ever, user research is a pivotal part of making sure each effort is serving that goal well.

Highlights

• [7:06] The key to individual alignment is clarity on your goals and purpose, and providing people with the ability to fulfill those things.
• [10:46] Coming into research with bias or trying to validate an idea you already have can be a sign of misalignment
• [16:46] Right now, people want something concrete. While making a plan can feel good now, having a strategy and goals you're clearly aligned around is the key to long term, agile success.
• [25:13] Strategy is equating things to value and sharing values across your organization
• [30:47] The strategy, not the plan, will keep everyone working towards the same goals
• [37:52] If you can't afford to fail, research is one of the things that will ensure you don't.

There's a lot of data out there. Keeping track of Google Analytics, NPS scores, site metrics, usability test results, industry data, and everything else can be downright overwhelming. Which is why Hannah Shamji, Head of Research at Copyhackers, likes to say she's doing data-informed work, not data-driven work.

For Hannah, her team, and her clients, working with tons of data can be overwhelming. Since you can usually find at least one graph to support a research point, it's important to put data in context. Hannah outlined how she gets in the zone with large amounts of data, puts things in context while doing her best to stay unbiased, and frames data around her research questions.

Highlights

[2:12] The difference between being data-informed and data-driven.

[6:21] Why it's important to put data in context and pull from many different sources.

[9:25] How Hannah approaches data through the lens of her research question.

[16:40] How Hannah tries to build data narratives that tell both sides of the story.

[23:21] Digging deep into data is a little bit like meditating.

[27:07] Hannah, Erin, and JH chat about data and COVID-19. (This episode was recorded on April 24, 2020.)

About our Guest

Hannah Shamji is a Consumer Psychologist, formerly the Head of Research & Insights at Copyhackers. There, she helped clients create great, data-informed, copy and marketing strategies. She blends qualitative and quantitative research to tell client stories.

UX benchmarking may seem like a lot of work, but Kate Moran is here to show you how to do it effectively. She's VP of Research & Content at Neilsen Norman Group and leads UX teams to better benchmarking, teaches newbies how to get started, and explains this complicated subject with clarity. She joined Erin and JH on our very first live episode to explain how UX benchmarking can help teams show the ROI of their work. 

She walked through how benchmarking can help get stakeholders on board, how to choose the right metrics early on, and most importantly, how to translate that to real ROI. 

Our very first live podcast was a great learning experience and a ton of fun! We really enjoyed the interactive aspect, and our audience asked a lot of thoughtful questions. 

Highlights

• [2:01] Kate explains what UX benchmarking is
• [3:37] How to choose benchmarking metrics
• [12:01] The difference between summative and formative studies, and why you need to distinguish between them.
• [17:21] Why context matters when evaluating benchmarking metrics
• [21:28] How to translate benchmarking results to ROI
• [29:11] Kate talks about case studies from NNg's ROI for Usability report
• [35:34] Q&A - How do you limit bias in unmoderated studies with non-users and users?
• [38:16] Q&A - How do you measure time spent on a task? Stopwatches aren't great.
• [39:59] Q&A - How do session replay tools fit into this?
• [41:10] Q&A - What happens when your stakeholders have different metrics for success?
• [44:57] Q&A - If a participant thinks they completed a task successfully, is that a success?
• [46:42] Q&A - How do you benchmark for emotional aspects, like how fun a product is?
• [49:07] Parting words of wisdom

Kate's recommended resources 

• NN/g full-day seminar: Measuring UX & ROI
• NN/g articles about benchmarking, ROI, and quantitative research
• Jeff Sauro’s articles about quantitative UX research (he also has very useful confidence interval and statistical significance calculators): measuringu.com 
• Jeff Sauro’s book about benchmarking: Benchmarking the User Experience
• Two great books for getting started with statistics: Straightforward Statistics and The Cartoon Guide to Statistics

Sales demos are a great opportunity to get to know your customers. The person on the other end is interested in your product, looking for a solution to a problem, and likely have some pain points with their current solutions.

That's why Jane Portman, co-founder of Userlist, uses demos as an opportunity to connect with potential customers, keep pain points top of mind, and learn how to make her product even better.  She chatted with Erin and JH about why she's doing customer research and sales demos at the same time, how constantly talking to customers helps her develop a better product, and how she came up with the podcast name UI Breakfast. 

Highlights

• [2:16] During the MVP phase, all new customers had to go through sales demos to start using Userlist. 
• [4:24] Making early customers go through demos ensured that Userlist's customers were all well informed about the capabilities and what to expect from the product.
• [5:49] How do you combine meaningful research with sales demos?
• [8:35] Because Jane and her team are talking to people all the time, they're learning as things change.
• [11:57] The specific questions Jane asks in her demo calls.
• [14:40] If something is coming up in calls all the time, you can't forget about it. Since Jane and her co-founder are always hearing about pain points, they can focus on building solutions before logging insights.
• [20:43] Asking your most active customers for feedback as you go is helpful for product teams who like to stay in touch with customers. 
• [24:03] How do you stay objective when doing research in a demo?

About our Guest

Jane Portman is the CEO & co-founder of Userlist and the host of the UI Breakfast podcast. She's passionate about helping founders connect with their customers and learning more about their stories. 

Recommended Reading

• User Onboarding: The Ultimate Guide for SaaS Founders
• How to Design and Conduct a Customer Interview

We’ve heard from a lot of designers and user researchers on the show, but we’re always looking for fresh perspectives on how research can help your business. So this week, Erin and JH chatted with Aazar Shad, Head of Growth at Userpilot, about how research methods are essential to his growth strategy.

Aazar started using research methods to find our who his users were, but continued using them to grow Userpilot’s business. He talked about how secondary research helped him find the best ways to connect with his target audience, continuous interviews help him identify where to go next, and how he honed his research skills over time. 

Highlights

• [3:44] User research is essential to acquisition 
• [5:13] Aazar found that his usual toolkit wouldn’t sway the product managers he was targeting, so he’d need to meet them where they were: Google Search
• [13:03] Asking users how they would describe Userpilot to other people helped to better understand what they might search.
• [13:38] Setting notifications for keywords to conversations he may want to jump in on in Slack helped Aazar be in the right place at the right time.
• [22:39] Aazar found that asking less questions, but digging deeper during his user interviews helped him learn more about his users.
• [29:56] Asking other people to review the feedback you get from research helps to identify trends with less bias.

Related links 

• How to use Slack for social media marketing & user acquisition
• SEO strategies for leads
• Conferences as a growth channel

About our guest

Aazar Shad is a growth marketing leader and founder of The Performers, a paid social mastermind group (at the time of our interview, he was Head of Growth at Userpilot). He is the host of the podcast Growth Marketing Stories

With so many research tools on the market, it can be hard to nail down exactly which ones are right for your team. This week on the pod, Erin and JH chatted with Daniel Loewus-Deitch and Leo Smith, who are the Directors of User Experience and Research, respectively, at a large insurance company. They wanted to learn more about how Daniel and Leo choose the tools with the best ROI for their team.

Daniel and Leo have spent a lot of time building out their tool stack. Since they have a lot of experience working for large organizations with many people conducting research and even more consuming it, it was important to them to get it right. In this episode, they talked about how they evaluate the ROI of tools, the summit they assembled to identify the tools their team could and would use, and how important it is to leave your assumptions at the door when tool-hunting.

Highlights

• [10:15] Leo used to spend 20-30 hours just on recruiting.
• [12:53] Sometimes the simplest tool is the most effective.
• [14:23] It's important to consider how accessible the tools you're using are to everyone on your team. Even if you choose the best tools, your team won't use them if it's not easy to do so.
• [16:23] How the team defined their user-centered design process, mapped tools to the right parts of it, and moved forward from there.
• [20:43] Why Leo and Daniel's team prefer a customized toolset over an all-in-one solution.
• [24:07] Applying the thinking behind design systems to a larger ecosystem is helping the team build a better toolkit.
• [33:56] The shiny new tools everyone is eager to try.
• [38:58] Usability test the tools your team will be using.
• [42:09] Going for an all in one tool is like going to the Cheesecake Factory, lots of choices, all pretty mediocre. Choosing a few specialized tools is like going to a farmer's market, less choice from each vendor, but better results.

Tools mentioned in the episode

• Loom is great for recording your screen and sharing it with your team. It can also be used for usability testing. 
• Dovetail is a research repository tool that makes it easy to organize and analyze your insights.  
• Lookback is a great tool for conducting usability tests and taking notes live. 
• Descript is a video, audio, and text editor. We’ve recently switched to it for editing the podcast, and I am absolutely in love. It does really good automatic transcription and makes pulling clips incredibly easy. 
• Rev is a transcription tool that provides really accurate, done-by-a-human transcription. 
• Zapier is a workflow automation tool that helps us make connections between apps that don’t naturally talk to each other. 
• Userbit is a research repository tool that makes it easy to code your analysis into deliverables. ‍
• Dedoose is a research repository tool that makes the academic process of analyzing research more collaborative. 

About our guests

Daniel Loewus-Deitch is the Assistant VP, User Experience at Unum. He has over 20 years of experience in UX, and has worked at companies like IBM and Microsoft. Daniel is interested in holistic wellness and technological harmony. 💻🎵.

Leo Smith is the Director of UX Research at SS&C Technologies. He has over 20 years of experience in UXR, and has worked for companies of all shapes and sizes in roles ranging from research to design. Leo is also a certified Hatha Yoga Instructor 🧘.

After three failed MVPs, Jonathan Anderson and the team at Candu realized they needed a better strategy for understanding how users interact with their product. So they started doing some user interviews. And they kept doing them. Every day for a year before launching their product. Jonathan chatted with Erin and JH about what he learned from those interviews, how it changed the direction of his company, and how he went from a total newbie to a research pro.

Highlights

• [5:36] Doing one interview a day every day keeps the Candu team curious about what the users have to say, rather than hearing the same things all in one day.
• [8:29] Jonathan and his team always ask users what they would expect the prototype to do.
• [11:34] How do you know when you've done enough interviews?
• [13:31] Creating low-fidelity designs to use, even if it's just drawing within Zoom, is incredibly helpful to Candu's design team.
• [15:51] After their third failed MVP, Jonathan and his team decided they need to make research a priority to build something truly great.
• [16:41] Candu built out a panel of trusted partners who gave great feedback and wanted to be a part of building something new. They supplemented this with new people to get great perspectives regularly.
• [21:55] When Jonathan started, he really didn't know how to do a user interview. Learning to step back from his excitement and be objective was important in evaluating feedback.
• [23:53] Jonathan shares his secret to identifying good research participants
• [26:12] Asking people about their process and how they currently solve thier problem can be illuminating, both for your process and finding the right people to interview.
• [29:02] Research shifted Candu's entire outlook as a company

This week on the pod, Erin and JH talk to Cat Noone, CEO of Stark, a suite of tools designed to help teams ship accessible work. They chatted about how accessibility is constantly evolving, what teams can do to get started, and inclusive design.

Highlights

• [1:50] Accessibility is continually changing and evolving, so it's important to think of it that way.
• [3:01] Accessibility is a side effect of inclusive design.
• [12:59] Identify other people in your organization that may be able to work with you on accessibility and create a bridge between teams.
• [15:08] Accessibility helps everyone, and framing it that way can help teams to understand its importance.
• [23:09] Ethics change team culture, exposure changes executive's minds, profit and customer loss changes action. 
• [31:11] If you can, speak up about having the tools to do your job well.