In this episode, Carol and Erin dive into all things research sample sizes with Lauren Stern. No matter your experience with user research, you'll need to recruit folks, making this an evergreen topic. Lauren has coached both new-to-research and junior UXRs on this critical topic and she shares some of her best advice.

Lauren shares importance considerations and nuances around different types of studies and even analysis approaches. She also unpacks her approach for international samples, the impact of drop-off rates, and participant compensation strategies.

Going beyond sample sizes, we conclude with a discussion of how to better engage with stakeholders when advocating sample sizes, making these conversations about "how many to recruit" more informed. She also shares resources to use when making the case to stakeholders.

Episode Highlights

• 05:41 - Crafting research goals and parameters: a collaborative journey
• 13:48 - Flexible research design: navigating sample sizes and methodologies
• 19:57 - Tailoring sample sizes to research objectives: finding the right fit 
• 26:30 - Qualitative confidence and stakeholder expectations
• 34:11 - Diverse methodologies in quantitative research: beyond surveys
• 43:05 - Departing research wisdom on sample sizes generally

About Our Guest
Lauren Stern is a mixed-methods research leader focused on creating the most human-centered technology possible. Over the last ten years her work has explored how perception and social cognition shape our experiences with automated systems from military zones to living rooms. Whether exploring individual experiences in the field or looking at large-scale data collections, she loves the puzzle of study design and coaching new researchers through the process.

Resources on Sample Sizes

• This free calculator gives sample ranges based on your study needs.
• Need some research-backed sample size help? Bookmark this today.
• You must pay participants. This calculator helps ensure an fair amount.

Carol and Erin welcome George Whitfield, an expert in applying AI to the analysis of qualitative data. George discusses the intricate challenges of leveraging language models to interpret expansive open-ended data (like interview transcripts), emphasizing the importance of context and not just keyword or topic identification.

They'll dig into the crucial role of human oversight in AI, what preliminary analysis might look like using AI, how to check and refine the work of an AI assistant without derailing your project delivery date, and recommendations for etiquette regarding the reporting of AI-informed results.

The episode closes with an exploration of the limits of AI and where user experience researchers can play a larger role in its development. George believes AI can (and should) inspire new directions of research, but not dictate them.

Episode Highlights

• 03:48 - Innovating consumer insights using AI
• 12:21 - Importance of human involvement in AI tools
• 20:04 - Enhance discussion sections with  AI tools
• 26:50 - AI-inspired insights provide inspiration, not guidance
• 34:12 - Interpretation beyond analyzing transcripts
• 36:46 - Applying engineering rigor to the process of building a business

About Our Guest
George Whitfield is an Entrepreneur-in-Residence at the Martin Trust Center for MIT Entrepreneurship, Lecturer at the MIT Sloan School of Management, and CEO at FindOurView. As CEO of his most recent company FindOurView, he launched a Gen AI product to help user researchers synthesize insights faster from high volumes of customer interviews. George holds 4 patents and has 3 degrees from MIT including a Bachelors in Electrical Engineering and Computer Science and a Masters and Ph.D. in Materials Science and Engineering.

Resources on Qual Data and AI

• Our AI in UX Report shares findings from a survey of over 1,000 researchers
• What does it mean to "code" qualitative data? This breakdown explains it all.
• Interested in trying an AI tool for your analysis? Here are 20 worth considering.

In this episode Carol and Erin are joined by Ruby Pryor, founder of Rex, a consulting firm specializing in UX research and strategic design.

They explore the world of measuring the impact of UX research. with Pryor introducing a four-level impact assessment framework. Ruby shares a four-level assessment framework and the conversation moves to prioritization, making "strategic" decisions, and increasing your research influence.

This episode also gets into what it means to demonstrate the "business value" of one's work, specifically how researchers can and should quantify their impact in terms that are tangible to the company. Ruby will share ways to score early wins, build momentum, and overcome communication frictions to find shared value.

Episode Highlights

• 03:14 - UX researcher impact: insights, optimization, prioritization, strategy
• 09:14 - Understanding organizational structures and strategy development collaboration
• 20:22 - Challenge of quantifying impact and strategic level
• 21:33 - Measuring strategy impact: challenges and indicators
• 30:04 - Prioritizing investments based on strong market indicators
• 38:51 - Stakeholders prioritize growth, revenue, and cost reduction

About Our Guest
Ruby Pryor is the founder of Rex, a service design and UX research consulting firm. Her previous roles include UX research at Grab, strategic design at Boston Consulting Group and management consulting at Nous Group. She has taught courses on increasing the impact of UX to learners from 5 continents and has spoken about design and UX at conferences in Asia and Europe.

Resources on UX Research Impact

• How to Track the Impact of Your UX Research
• Leading an Impactful User Research Team
• Use Stakeholder Interviews to Drive Impact

In this episode of Awkward Silences, Carol and Erin dive into the world of conference networking and planning with Bryan Dosono, a staff UX research lead at eBay known for his extensive experience curating and organizing conference programs.

Bryan share insights on how to prepare for conferences such as using apps to schedule meetings and emphasizing networking over solely focusing on content. He also discusses strategic scheduling to align conference participation with career goals and offers advice on making spontaneous, serendipitous connections.

The conversation also covers practical tips for adding value during small talk, how to approach and connect with other researchers, and the importance of being open, vulnerable, and willing to face rejection in the pursuit of meaningful interactions. Bryan provides guidance for both newcomers to the UXR field and seasoned professionals: overcoming imposter syndrome and leveraging transferable skills from related fields.

Episode Highlights

• 03:58 - Navigating the Unpredictable: Strategies for Introverts at Networking Events.
• 08:24 - Strategic Networking: Maximizing Opportunities Before and During Conferences.
• 12:33 - Networking strategies for academic and professional growth.
• 22:36 - Balancing Attendance: Prioritizing Conferences with Active Roles.
• 28:03 - Maximizing Remote Networking: Strategies for Engagement in Virtual Conferences.
• 33:07 - Conference Insights: Making the Most of Your Experience through Pre-Planning.

About Our Guest
Bryan Dosono, PhD, is a user experience research leader in the consumer technology space. He applies human-computer interaction research methods with visual storytelling to modernize the design of global marketplaces and online communities. He currently volunteers as a Conference Chair at UXPA International and serves on the User Interviews Research Council.

Looking for a UX community? Here are 16 to join for networking and development.

The decision to leave academia can be difficult. Often, it involves giving up a stable career path and a lifetime commitment to a particular field of study. 

But as Joe Stubenrauch explains, the move also offers researchers the opportunity to pursue new career paths, improve work-life balance and geographic flexibility, and reinvent their lives.

And he should know: Formerly a professor of history, he walked away from tenure to join a big tech company as a UX Researcher at AWS. In this episode, Joe discusses his decision to walk transition to the private sector and shares thoughtful advice for others considering a similar move.

Highlights from the episode

• [04:06] Joe reflects on feeling burnt out, bored, and doubtful in academia 
• [11:23] What makes you happy? Joe weighs the pros and cons of things like mental health, geographic mobility, and proximity to family and friends.
• [19:41] Breaking into a new field, experimenting, and talking to people. 
• [34:47] The importance of having a portfolio that includes compelling stories 
• [41:35] Tactical advice for job seekers (resumes, LinkedIn profiles, job interview strategies, etc)
• [44:23] Working in a team environment as a former academic

About our guest

Joe Stubenrauch is a former professor of British history at Baylor University, and the author of a prize-winning book published by Oxford University Press. During the height of the pandemic, he walked away from tenure and joined a big tech company in order to redesign his life. Now as a UX Researcher at AWS, Joe has found unexpected similarities between his work as Victorianist and his work in the cloud. He also writes regularly about the transition from academia to industry and is obsessed with how people can reinvent their lives and careers.

You can follow Joe on LinkedIn.

How often do you think about content design? The answer, most likely, is: “not enough”.

The work of a Content Designer involves fitting the right words in the right places, understanding the nuances of things log in vs. sign in, and knowing the right words to use to engage customers. Content Design is an important part of the user experience—and the ROI is high. Like, “millions of $s saved through content-testing” high.

Erica Jorgensen is a Staff Content Designer at Chewy.com and the author of Strategic Content Design: Tools and Research Techniques for Better UX. She joined Erin and Carol on the podcast to discuss the ins and outs of content design, the importance of clarity for effective communication, and how to involve your audience in content design.

Highlights from the episode:

• 00:02:03: What is content design in the context of UX research?
• 00:06:48: Gaining clarity by thinking about the words you use frequently 
• 00:14:00: How Erica used simplified language to get customers to buy more insurance plans
• 00:26:42: Using cloze testing to ensure general content clarity
• 00:29:26: The importance of syllables and length 
• 00:39:46: Resources and guides for better content design
• 00:42:56: How Microsoft saved $2 million through content testing

  Sources and people mentioned

• BrainTraffic
• Chewy.com
• Cloze testing — “Cloze Test for Reading Comprehension,” NN/g.
• Curtis Kopf, Chief Experience Officer at REI
• Dylan Romero, Affiliate Instructor at University of Washington, UX at Microsoft
• Flesch–Kincaid – Wikipedia 
• Hemingway Editor
• Jorgensen, Erica. Strategic Content Design: Tools and Research Techniques for Better UX. (2023) — GET 15% OFF AT ROSENFELD MEDIA WITH CODE AS15
• MeasuringU (Jeff Sauro)
• Merriam-Webster Dictionary
• Metts, Michael and Andy Wefle. Writing Is Designing: Words and the User Experience. (2020)
• Scott Kubie
• System Usability Scale
• Tracy Vandygam, Senior Content Designer at Mozilla
• Visual Thesaurus
• Winters, Sarah. Content Design. (2017)

About our guest

Erica Jorgensen is a staff content designer at Chewy.com and the author of Strategic Content Design: Tools and Research Techniques for Better UX, published in April 2023 by Rosenfeld Media. 

She's a content designer, content strategist, and team leader determined to bring greater respect to the content field. To that end, Erica speaks frequently at conferences including UXDX USA, UX Lisbon, Microsoft Design Week, the Web Directions Summit, and Button: The Content Design Conference, and on podcasts like The Content Strategy Podcast with Kristina Halvorson and Content Insights podcast with Larry Swanson. In addition to working in content roles for companies of all sizes, she has taught at the University of Washington and Seattle’s School of Visual Concepts. 

Erica earned her B.A. from the University of Connecticut and M.A. from the University of Missouri’s School of Journalism. In her free time, you can find her exploring Washington State’s wineries or hiking with her husband and rescue dog, Rufus.

Maria Guidice, author of Changemakers: How Leaders Can Design Change in an Insanely Complex World, joins Erin May and Carol Guest in this episode of Awkward Silences. Together, they examine the qualities of effective change agents and discuss how designers, researchers, and passionate folks can hone and apply these qualities to drive change within their organization. 

Throughout the episode, Maria draws on her experience leading global teams at Facebook and Autodesk and a decade of research into the interconnectedness of leadership, design, and change to offer practical advice on how to approach change through a designer mindset.

In this episode, Maria, Erin and Carol cover:

• What is a “changemaker”?
• The qualities of effective leaders and change agents.
• How designers can employ their skills to drive change at scale.
• Treating design as a mindset.
• Building support and working with people who share a creative and open mindset.
• Why we need to believe in progress and continuous improvement.

Highlights

00:04:23 – Design is about helping people make sense of the world

00:09:34 – Maria’s research into the connection between leadership, design, and change

00:13:50 – The importance of understanding people’s resistance to change

00:16:28 – What is a changemaker?

00:18:04 – Design as a noun, a verb, a mindset that can lead change at scale 

00:24:30 – Qualities of effective change agents

00:33:19 – Why Maria hates the term “change management”

00:36:12 – Embracing failure as a learning opportunity

00:39:46 – The importance of passion, purpose, and a belief in continuous progress

About our guest

For three decades, creative teams and business leaders have sought the provocative vision and mentorship of Maria Giudice, GEW DEECE. After founding the pioneering experience design firm Hot Studio and leading global teams at Facebook and Autodesk, Maria’s mission today is to build the next generation of creative leaders. 

Through one-on-one coaching, group coaching, and team-building workshops, Maria unlocks the potential hidden in executives and the people they lead. A popular speaker at design and business conferences, Maria is also the author of four design books, including Rise of the DEO: Leadership by Design, and most recently Changemakers: How Leaders Can Design Change in an Insanely Complex World.

What company isn’t “customer obsessed”? Well, a lot, as it turns out. 

That’s not for lack of caring—but building a customer-centric culture is easier said than done. And how you do this depends on whether you’re starting from square one or trying to change a company’s culture from the inside.

According to Prayag Narula, CEO and Co-Founder of Marvin, changing a culture towards customer centricity requires starting from the bottom-up. Building something new? Leaders should be obsessing over customer feedback, actively placing research and customer insights front and center from the very beginning. 

In this episode of Awkward Silences, Prayag sits down with Erin and Carol to discuss his strategies for building a customer-centric product culture at Marvin. Tune in to learn more about what it means to obsess over customers, doing research versus using research, how customer centricity is put into practice at Marvin, and more. 👇

Highlights: 

[00:00:31] The origins of Marvin

[00:04:40] Building a culture around customer centricity

[00:08:35] Why leaders should obsess over a customer centric culture

[00:13:59] How the rest of the team can keep customers front and center

[00:18:04] Balancing doing research and using research

[00:21:46] “Capital R” versus “lower case r” researchers 

[00:28:21] Getting researchers more involved in strategic conversations

[00:30:35] Practicing customer centricity at Marvin

[00:39:36] The Marvin repository

[00:46:53] Prayag’s parting words of wisdom: talk to your market

Sources mentioned in the episode:

• userinterviews.com/awkward
• heymarvin.com

About Our Guest

Prayag Narula is an entrepreneur and trained researcher with a passion for building technology he always wished for when conducting user research. He is currently the Co-Founder and CEO of Marvin, a qualitative data analysis platform and research repository for user-centric teams across the globe. He is further the Co-Founder and a Board Member for LeadGenius, a demand generation automation company automating and accelerating outbound sales and marketing for mid-marketing and enterprise companies across the world. 

AI is a big part of Notion's strategy for the future of knowledge work, and they're exploring ways to use AI to make people's lives easier.

Linus Lee, a Research Engineer at Notion, spends his time researching and learning about AI—how users interact with it, the challenges of building trustworthy AI systems, and its - potential to help people work more effectively.

In this episode of Awkward Silences, he sits down with Erin and Carol to discuss Notion’s approach to designing interfaces for Generative AI tools. 

Tune into this episode to learn more about the challenges of instructing AI, the importance of user feedback to the AI design process, security considerations, use cases, and more. 

Highlights: 

[00:05:03] Tools Linus is excited to see come to market

[00:06:50] Generative AI and its function at Notion

[00:10:22] Recognizing AI is useful versus finding the use cases for it

[00:13:58] Lessons from the Notion design process

[00:22:09] Breadth versus depth in workflows

[00:26:10] Learning about models in the wild

[00:28:52] Security considerations with AI products

[00:32:13] Understanding the difference between AI generated and human written text

[00:36:34] Should we be scared of AI?

About Our Guest

Linus Lee is a Research Engineer at Notion, prototyping new software interfaces for augmenting our collaborative work and creativity with AI. He has spent the last few years experimenting with AI-augmented tools for thinking, like a canvas for exploring the latent space of neural networks and writing tools where ideas connect themselves. Before Notion, Linus spent a year as an independent researcher in New York City. 

According to Andrea Amorós, Associate Principal Research at ADP, giving research a seat at the big table is all about building meaningful connections with the people you’re presenting to.

Andrea joins the Awkward Silences podcast to provide insight into how she learned to make research meaningful for stakeholders at higher levels of the organization. Having been hired to conduct discovery but with no plan set up for her, she took the initiative to interview the leaders within the company to understand their needs and identify opportunities for strategic research. 

Tune into this episode to hear Andrea’s advice on building relationships with stakeholders, approaching push-back, and creating meaningful change in the current research landscape.

Highlights: 

[00:00:50] Andrea’s motivation to give research a seat at the big table

[00:07:46] What bringing research to the big table really means

[00:11:06] Presenting research to bigger audiences versus smaller audiences

[00:14:08] Making presentations meaningful to the audience

[00:17:47] How Andrea made it to the big table

[00:22:26] How product-specific research can be escalated to the big table

[00:24:53] The marginalization of research today

[00:30:27] How changes in research apply to other fields

[00:32:23] Andrea’s advice for building courage

[00:36:17] The possibility of getting to the big table without presentations

[00:38:46] Navigating push-back

Sources mentioned in the episode:

• 📘 Awkward Silences blog
• 🔗 Andrea's LinkedIn
• 📘 Andrea's article: How to Present Research Insights to Executives with Ease
• 🎁 Get 3 free participants for your next research project with User Interviews
• ✍️ Take our survey to let us know what you want to see next

About Our Guest

Andrea M. Amorós is the Associate Principal Researcher at ADP, a payroll solutions provider leader globally. She has always been a really deep thinker, curious about understanding human behavior and the unconscious mind. Her expertise lies in conducting strategic research and discovering patterns at scale to improve people's lives. She has a background in Design Engineering and has worked in various startups, consultancies and large-scale corporations.