Insights Unlocked

Why AI in user research isn’t replacing real people (yet) with Mario Callegaro


Listen Later

Episode web page: https://bit.ly/3KUDMzV


Episode description:

In this episode of Insights Unlocked, host Amrit Bhachu sits down with Mario Callegaro, founder of Callegaro Research and a leading expert in survey methodology and AI-augmented research. Mario brings a wealth of experience from his 15 years at Google and shares practical, thoughtful guidance on how to responsibly integrate AI tools into UX, market, and survey research workflows.

From the evolving role of the researcher to the risks and potential of synthetic users, this episode unpacks what’s changing, what to be cautious of, and what skills are now essential in the age of AI.

What you’ll learn

  • How AI supports every phase of the research workflow: planning, execution, and activation

  • The role of prompt engineering in getting better insights from AI tools

  • Why synthetic users and synthetic data can't yet replace real human participants

  • What researchers are gaining—and potentially losing—by relying on AI

  • The current limitations of large language models in capturing nuance and variability

  • Practical advice on testing and validating AI tools with known datasets

  • Why transparency, iteration, and responsible experimentation are key

    Resources & links

    • Mario on LinkedIn (https://www.linkedin.com/in/mcallegaro/)

    • Amrit on LinkedIn (https://www.linkedin.com/in/amritsbhachu/)

    • Callegaro Research (https://callegaroresearch.com/)

    • Quantitative User Experience Association (https://www.quantuxcon.org/)

    • There’s an AI for that (https://theresanaiforthat.com/)

    • Mike on LinkedIn (https://www.linkedin.com/in/mikemace/)

    • Nathan Isaacs on LinkedIn (https://www.linkedin.com/in/nathanisaacs/)

    • Learn more about Insights Unlocked: https://www.usertesting.com/podcast

      ...more
      View all episodesView all episodes
      Download on the App Store

      Insights UnlockedBy UserTesting

      • 4.8
      • 4.8
      • 4.8
      • 4.8
      • 4.8

      4.8

      35 ratings


      More shows like Insights Unlocked

      View all
      Planet Money by NPR

      Planet Money

      30,727 Listeners

      Hidden Brain by Hidden Brain, Shankar Vedantam

      Hidden Brain

      43,615 Listeners

      Marketplace by Marketplace

      Marketplace

      8,764 Listeners

      Up First from NPR by NPR

      Up First from NPR

      56,489 Listeners

      Masters of Scale by WaitWhat

      Masters of Scale

      3,991 Listeners

      Creative Confidence Podcast by IDEO U

      Creative Confidence Podcast

      127 Listeners

      The Indicator from Planet Money by NPR

      The Indicator from Planet Money

      9,535 Listeners

      Crime Junkie by Audiochuck

      Crime Junkie

      368,887 Listeners

      The Journal. by The Wall Street Journal & Spotify Studios

      The Journal.

      6,071 Listeners

      Consider This from NPR by NPR

      Consider This from NPR

      6,395 Listeners

      NN/G UX Podcast by Nielsen Norman Group

      NN/G UX Podcast

      107 Listeners

      IMO with Michelle Obama and Craig Robinson by Higher Ground

      IMO with Michelle Obama and Craig Robinson

      13,061 Listeners

      The Artificial Intelligence Show by Paul Roetzer and Mike Kaput

      The Artificial Intelligence Show

      197 Listeners

      New Heights with Jason & Travis Kelce by Wondery

      New Heights with Jason & Travis Kelce

      18,144 Listeners

      The AI Daily Brief: Artificial Intelligence News and Analysis by Nathaniel Whittemore

      The AI Daily Brief: Artificial Intelligence News and Analysis

      633 Listeners