NLP Highlights

103 - Processing Language in Social Media, with Brendan O'Connor

01.27.2020 - By Allen Institute for Artificial IntelligencePlay

Download our free app to listen on your phone

Download on the App StoreGet it on Google Play

We talked to Brendan O’Connor for this episode about processing language in social media. Brendan started off by telling us about his projects that studied the linguistic and geographical patterns of African American English (AAE), and how obtaining data from Twitter made these projects possible. We then talked about how many tools built for standard English perform very poorly on AAE, and why collecting dialect-specific data is important. For the rest of the conversation, we discussed the issues involved in scraping data from social media, including ethical considerations and the biases that the data comes with.

Brendan O’Connor is an Assistant Professor at the University of Massachusetts, Amherst.

Warning: This episode contains explicit language (one swear word).

More episodes from NLP Highlights