Two Psychologists Four Beers

Episode 99: Is MTurk Too Good To Be True?


Listen Later

In a recent article, psychologists Webb and Tangney document their experience collecting psychology data online using Amazon's crowdsourcing platform MTurk. Alarmingly, the authors conclude that ultimately only 2.6% of their sample was valid data from human beings. Yoel and Alexa weigh in on these findings, discussing what researchers can reasonably expect from online studies and platforms, and how their personal experiences have informed their own practices. They also consider a response written by Cuskley and Sulik, who argue that researchers, not recruitment platforms, are responsible for ensuring the quality of data collected online. Questions that arise include: What studies do people want to do? Does anyone read the fine print? And what are the ethics of mouse-hunting?

Links:

  • Two Psychologists Four Beers on Untappd
  • Too Good to Be True: Bots and Bad Data From Mechanical Turk - Margaret A. Webb, June P. Tangney, 2022
  • PsyArXiv Preprints | The burden for high-quality online data collection lies with researchers, not recruitment platforms
  • Living in Harmony with House Mice and Rats | PETA
  • ...more
    View all episodesView all episodes
    Download on the App Store

    Two Psychologists Four BeersBy Yoel Inbar, Michael Inzlicht, and Alexa Tullett

    • 4.5
    • 4.5
    • 4.5
    • 4.5
    • 4.5

    4.5

    155 ratings


    More shows like Two Psychologists Four Beers

    View all
    Fresh Air by NPR

    Fresh Air

    38,053 Listeners

    The New Yorker Radio Hour by WNYC Studios and The New Yorker

    The New Yorker Radio Hour

    6,795 Listeners

    99% Invisible by Roman Mars

    99% Invisible

    26,206 Listeners

    On the Media by WNYC Studios

    On the Media

    9,231 Listeners

    Political Gabfest by Slate Podcasts

    Political Gabfest

    8,475 Listeners

    Very Bad Wizards by Tamler Sommers & David Pizarro

    Very Bad Wizards

    2,672 Listeners

    EconTalk by Russ Roberts

    EconTalk

    4,270 Listeners

    Uncomfortable Conversations with Josh Szeps by Josh Szeps

    Uncomfortable Conversations with Josh Szeps

    834 Listeners

    Weird Studies by SpectreVision Radio

    Weird Studies

    594 Listeners

    Sean Carroll's Mindscape: Science, Society, Philosophy, Culture, Arts, and Ideas by Sean Carroll

    Sean Carroll's Mindscape: Science, Society, Philosophy, Culture, Arts, and Ideas

    4,168 Listeners

    Blocked and Reported by Katie Herzog and Jesse Singal

    Blocked and Reported

    3,819 Listeners

    Hard Fork by The New York Times

    Hard Fork

    5,533 Listeners

    The Ezra Klein Show by New York Times Opinion

    The Ezra Klein Show

    16,096 Listeners

    On with Kara Swisher by Vox Media

    On with Kara Swisher

    3,505 Listeners

    Nullius in Verba by Smriti Mehta and Daniël Lakens

    Nullius in Verba

    12 Listeners