Emily and Alex are joined this week by Dr. Jeremy G. Kahn to discuss evaluation metrics and some new AI hell.

Jeremy G. Kahn has a PhD in computational linguistics, with a focus on information-theoretic and empirical engineering approaches to dealing with natural language (in text and speech). He’s gregarious, polyglot, a semi-auto-didact, and occasionally prolix. He also likes comic books, coffee, progressive politics, information theory, lateral thinking, science fiction, science fact, linear thinking, bicycles, beer, meditation, love, play, and inquiry. He lives in Seattle with his wife Dorothy and son Elliott.

References:

WikidataCon

Gish Gallop:

The Bender Rule:

DJ Khaled - You Played Yourself:

Jeff Kao's interrogation of public comment periods:

Emily's blog post response to NYT piece:

Follow Jeremy:

You can check out future streams at https://twitch.tv/KateSilver538.

Follow Emily at https://twitter.com/EmilyMBender and https://dair-community.social/@EmilyMBender

Research Fellow Adrienne Williams discusses massive amounts of wage theft committed by Amazon against its delivery drivers. She demonstrates how extra standup meetings, time taken for remedial actions by Amazon’s “Mentor” application, and AI-powered cameras in trucks penalized drivers, and quantifies the scale of wage theft, which is at the levels of upwards of thousands of dollars in stolen wages per year, per worker.

Learn more about DAIR at https://dair-institute.org and follow us on social media:

Research Fellow Dr. Asmelash Teka discusses his work with Lesan.ai, a startup focused on providing a broad range of technology for languages used in Ethiopia and Eritrea. He presents here machine translation and automated speech recognition for Ge’ez based languages, and the need to keep data fuelling these technologies in the hands of people who speak these languages rather than a few multinational corporations.

Learn more about DAIR at https://dair-institute.org and follow us on social media:

Research Fellow Raesetje Sefala on her work on using computer vision to detect the persistence of spatial apartheid in South Africa. Although Apartheid legally ended in 1991, the difference between townships, where non-white people were forced to live in, and suburbs is still clear. Those who grow up in townships, like Raesetje, know that the services they receive are much worse than those in the suburbs. Raesetje here shows her dataset, in which she and others raised in townships annotated geo-located satellite images with their neighborhood types, including townships and suburbs. She also demonstrats her computer vision tool, which labels different neighborhood types. These tools will allow those who live in townships to advocate for better resources, including better government service allocation.

Learn more about DAIR at https://dair-institute.org and follow us on social media:

DAIR Executive Director Timnit Gebru holds a conversation with Research Fellow Meron Estefanos, in which they talked about Meron’s activism as a refugee advocate who has rescued more than 16,000 victims of trafficking. Meron discusses the extent to which she has faced transnational repression in the course of her work, and how calls to attack her on social media result in violence: for example, after calls to injure her circulated on Facebook and Twitter, she was attacked eight times in a single evening. Timnit concludes with a discussion of how refugees are the first to experience the harms of experimental AI technology on the ground. You can learn more about Meron’s work in the 2013 documentary Sound of Torture and this episode of the podcast This American Life.

Learn more about DAIR at https://dair-institute.org and follow us on social media:

Research engineer Dylan Baker talks about how billionaires have taken over the imaginary of technological futures, and how we need to take back that imaginary. Instead of continually playing on the back foot, what would it mean to build something new? He concludes by introducing our forthcoming Possible Futures series where we will further explore this question.

Learn more about DAIR at https://dair-institute.org and follow us on social media:

Emily and Alex discuss MetaAI's bullshit science paper generator, Galactica, along with its defenders. Also, more fresh AI hell.

Imre Lakatos on research programs

Shah, Chirag and Emily M. Bender. 2022. Situating Search. Proceedings of the 2022 ACM SIGIR Conference on Human Information Interaction and Retrieval (CHIIR ’22). https://dl.acm.org/doi/10.1145/3498366.3505816

UW RAISE: https://www.raise.uw.edu/

Stochastic Parrots:

The Octopus Paper:

Palestinian man arrested because of bad MT:

Katherine McKittrick - Dear Science and Other Stories

Sokal Hoax

Safiya Noble - Algorithms of Oppression

Latanya Sweeney "Discrimination in Online Ad Delivery"

Michelle Dion

The Subjects and Stages of AI Dataset Development: A Framework for Dataset Accountability

Sealioning:

Grover:

Ben Dickson's coverage of Grover:

Sociology of science work by Roberta Sinatra

You can check out future streams at https://twitch.tv/KateSilver538.

Follow Emily at https://twitter.com/EmilyMBender and https://dair-community.social/@EmilyMBender

Emily and Alex discuss Sam Bankman-Fried's Future Fund and their essay contest. Also, we introduce our "What in the Fresh AI Hell?" segment!

Citations:

Super Forecasting

Bill Howe - Applied AI in High-Expertise Settings or Curation as Programming

Samir Passi and Solon Barocos - "Problem Formulation in Fairness"

Vinod Prabharakan, William Isaac, Donald Martin Jr

David Ribes, Andrew S Hoffman, Steven C Slota and Geoffrey C Bowker

Shoshana Zuboff, The Age of Surveillance Capitalism

Lee Vinsel on Criti-hype:

You can check out future streams at https://twitch.tv/KateSilver538.

Follow Emily at https://twitter.com/EmilyMBender and https://dair-community.social/@EmilyMBender

Emily and Alex are joined this week by Drs. Johnathan Flowers, Negar Rostamzedah, and Jennifer Lena to discuss the hype around AI Art.

Dr. Johnathan Flowers is an assistant professor in the department of philosophy at California State University, Northridge. His research interest is at the intersection of American Pragmatism, Philosophy of Disability, and Philosophy of Race, Gender and Sexuality as they apply to socio-technical systems. Flowers also explores the impacts of cultural narratives on the perception and development of sociotechnical systems.

Dr. Jennifer Lena is a professor at Teachers College, Columbia University, where she runs the Arts Administration program. She’s published books on music genres, the legitimation of art, and the measurement of culture.

Dr. Negar Rostamzadeh is a Senior Research Scientist at Google Responsible AI team. Her recent research is at the intersection of computer vision and sociotechnical research. She studies creative computer vision technologies and the broader social impact of them. She is also substantially engaged with the creative ML community and organized a series of workshops with artists on creative computer vision technologies and their broader impact at computer vision venues.

References:

Kevin Roose, "An A.I.-Generated Picture Won an Art Prize. Artists Aren’t Happy."

Jo Lawson-Tancred, "Robot Artist Ai-Da Just Addressed U.K. Parliament About the Future of A.I. and ‘Terrified’ the House of Lords"

Marco Donnarumma, "AI Art Is Soft Propaganda for the Global North"

Jane Recker, "U.S. Copyright Office Rules A.I. Art Can’t Be Copyrighted"

Richard Whiddington, "Shutterstock Inks Deal With DALL-E Creator to Offer A.I.-Generated Stock Images. Not All Artists Are Rejoicing."

Stephen Cave and Kanta Dihal, "The Whiteness of AI"

Follow our guests:

You can check out future streams at https://twitch.tv/KateSilver538.

Follow Emily at https://twitter.com/EmilyMBender and https://dair-community.social/@EmilyMBender

One of DAIR's pillars is to mitigate harms stemming from the development of artificial intelligence and technology. In this video, our researchers discuss our approaches for assessing and calling out the harms of these technologies.

Learn more about DAIR at https://dair-institute.org and follow us on social media: