NLP Highlights

By Allen Institute for Artificial Intelligence

What's NLP Highlights about?

Welcome to the NLP highlights podcast, where we invite researchers to talk about their work in various areas in natural language processing. All views expressed belong to the hosts/guests, and do not represent their employers.

Download our free app to listen on your phone

NLP Highlights episodes:

02.29.2024
Are LLMs safe?
Curious about the safety of LLMs?
01.08.2024
"Imaginative AI" with Mohamed Elhoseiny
This podcast episode features Dr. Mohamed Elhoseiny, a true luminary in the realm of computer vision with over a decade of groundbreaking research. As an Assistant Professor at KAUST, Dr. Elhoseiny's work delves into the intersections of Computer Vision, Language...
12.28.2023
142 - Science Of Science, with Kyle Lo
Our first guest with this new format is Kyle Lo, the most senior lead scientist in the Semantic Scholar team at Allen Institute for AI (AI2), who kindly agreed to share his perspective on #Science of #Science (#scisci) on our...
06.29.2023
141 - Building an open source LM, with Iz Beltagy and Dirk Groeneveld
In this special episode of NLP Highlights, we discussed building and open sourcing language models. What is the usual recipe for building large language models? What does it mean to open source them? What new research questions can we answer...
06.06.2023
140 - Generative AI and Copyright, with Chris Callison-Burch
In this special episode, we chatted with Chris Callison-Burch about his testimony in the recent U.S. Congress Hearing on the Interoperability of AI and Copyright Law. We started by asking Chris about the purpose and the structure of this hearing....
03.24.2023
139 - Coherent Long Story Generation, with Kevin Yang
How can we generate coherent long stories from language models? Ensuring that the generated story has long range consistency and that it conforms to a high level plan is typically challenging. In this episode, Kevin Yang describes their system that...
01.20.2023
138 - Compositional Generalization in Neural Networks, with Najoung Kim
Compositional generalization refers to the capability of models to generalize to out-of-distribution instances by composing information obtained from the training data. In this episode we chatted with Najoung Kim, on how to explicitly evaluate specific kinds of compositional generalization in...
01.13.2023
137 - Nearest Neighbor Language Modeling and Machine Translation, with Urvashi Khandelwal
We invited Urvashi Khandelwal, a research scientist at Google Brain to talk about nearest neighbor language and machine translation models. These models interpolate parametric (conditional) language models with non-parametric distributions over the closest values in some data stores built from...
05.19.2022
136 - Including Signed Languages in NLP, with Kayo Yin and Malihe Alikhani
In this episode, we talk with Kayo Yin, an incoming PhD at Berkeley, and Malihe Alikhani, an assistant professor at the University of Pittsburgh, about opportunities for the NLP community to contribute to Sign Language Processing (SLP). We talked about...
03.02.2022
135 - PhD Application Series: After Submitting Applications
This episode is the third in our current series on PhD applications. We talk about what the PhD application process looks like after applications are submitted. We start with a general overview of the timeline, then talk about how to approach...
10.19.2021
134 - PhD Application Series: PhDs in Europe versus the US, with Barbara Plank and Gonçalo Correia
This episode is the second in our current series on PhD applications. How do PhD programs in Europe differ from PhD programs in the US, and how should people decide between them? In this episode, we invite Barbara Plank (Professor at...
10.06.2021
133 - PhD Application Series: Preparing Application Materials, with Nathan Schneider and Roma Patel
This episode is the first in our current series on PhD applications. How should people prepare their applications to PhD programs in NLP? In this episode, we invite Nathan Schneider (Professor of Linguistics and Computer Science at Georgetown University) and Roma...
09.27.2021
132 - Alexa Prize Socialbot Grand Challenge and Alquist 4.0, with Petr Marek
In this episode, we discussed the Alexa Prize Socialbot Grand Challenge and this year's winning submission, Alquist 4.0, with Petr Marek, a member of the winning team. Petr gave us an overview of their submission, the design choices that led...
08.20.2021
131 - Opportunities and Barriers between HCI and NLP, with Nanna Inie and Leon Derczynski
What can NLP researchers learn from Human Computer Interaction (HCI) research? We chatted with Nanna Inie and Leon Derczynski to find out. We discussed HCI's research processes including methods of inquiry, the data annotation processes used in HCI, and how...
08.09.2021
130 - Linking human cognitive patterns to NLP Models, with Lisa Beinborn
In this episode, we talk with Lisa Beinborn, an assistant professor at Vrije Universiteit Amsterdam, about how to use human cognitive signals to improve and analyze NLP models. We start by discussing different kinds of cognitive signals—eye-tracking, EEG, MEG, and...
07.02.2021
129 - Transformers and Hierarchical Structure, with Shunyu Yao
In this episode, we talk to Shunyu Yao about recent insights into how transformers can represent hierarchical structure in language. Bounded-depth hierarchical structure is thought to be a key feature of natural languages, motivating Shunyu and his coauthors to show...
06.19.2021
128 - Dynamic Benchmarking, with Douwe Kiela
We discussed adversarial dataset construction and dynamic benchmarking in this episode with Douwe Kiela, a research scientist at Facebook AI Research who has been working on a dynamic benchmarking platform called Dynabench. Dynamic benchmarking tries to address the issue of...
06.08.2021
127 - Masakhane and Participatory Research for African Languages, with Tosin Adewumi and Perez Ogayo
We invited members of Masakhane, Tosin Adewumi and Perez Ogayo, to talk about their EMNLP Findings paper that discusses why typical research is limited for low-resourced NLP and how participatory research can help. As a result of participatory research, Masakhane...
05.24.2021
126 - Optimizing Continuous Prompts for Generation, with Lisa Li
We invited Lisa Li to talk about her recent work, Prefix-Tuning: Optimizing Continuous Prompts for Generation. Prefix tuning is a lightweight alternative to finetuning, and the idea is to tune only a fixed-length task-specific continuous vector, and to keep the...
05.04.2021
125 - VQA for Real Users, with Danna Gurari
How can we build Visual Question Answering systems for real users? For this episode, we chatted with Danna Gurari, about her work in building datasets and models towards VQA for people who are blind. We talked about the differences between...
04.14.2021
124 - Semantic Machines and Task-Oriented Dialog, with Jayant Krishnamurthy and Hao Fang
We invited Jayant Krishnamurthy and Hao Fang, researchers at Microsoft Semantic Machines to discuss their platform for building task-oriented dialog systems, and their recent TACL paper on the topic. The paper introduces a new formalism for task-oriented dialog to effectively...
04.05.2021
123 - Robust NLP, with Robin Jia
In this episode, Robin Jia talks about how to build robust NLP systems. We discuss the different senses in which a system can be robust, reasons to care about system robustness, and the challenges involved in evaluating robustness of NLP...
11.12.2020
122 - Statutory Reasoning in Tax Law, with Nils Holzenberger
We invited Nils Holzenberger, a PhD student at JHU to talk about a dataset involving statutory reasoning in tax law Holzenberger et al. released recently. This dataset includes difficult textual entailment and question answering problems that involve reasoning about how...
10.30.2020
121 - Language and the Brain, with Alona Fyshe
We invited Alona Fyshe to talk about the link between NLP and the human brain. We began by talking about what we currently know about the connection between representations used in NLP and representations recorded in the brain. We also...
10.03.2020
120 - Evaluation of Text Generation, with Asli Celikyilmaz
We invited Asli Celikyilmaz for this episode to talk about evaluation of text generation systems. We discussed the challenges in evaluating generated text, and covered human and automated metrics, with a discussion of recent developments in learning metrics. We also...
09.03.2020
119 - Social NLP, with Diyi Yang
In this episode, Diyi Yang gives us an overview of using NLP models for social applications, including understanding social relationships, processes, roles, and power. As NLP systems are getting used more and more in the real world, they additionally...
08.26.2020
118 - Coreference Resolution, with Marta Recasens
In this episode, we talked about Coreference Resolution with Marta Recasens, a Research Scientist at Google. We discussed the complexity involved in resolving references in language, the simplification of the problem that the NLP community has focused on by talking...
08.13.2020
117 - Interpreting NLP Model Predictions, with Sameer Singh
We interviewed Sameer Singh for this episode, and discussed an overview of recent work in interpreting NLP model predictions, particularly instance-level interpretations. We started out by talking about why it is important to interpret model outputs and why it is...
07.03.2020
116 - Grounded Language Understanding, with Yonatan Bisk
We invited Yonatan Bisk to talk about grounded language understanding. We started off by discussing an overview of the topic, its research goals, and the the challenges involved. In the latter half of the conversation, we talked about ALFRED (Shridhar...
06.17.2020
115 - AllenNLP, interviewing Matt Gardner
In this special episode, Carissa Schoenick, a program manager and communications director at AI2 interviewed Matt Gardner about AllenNLP. We chatted about the origins of AllenNLP, the early challenges in building it, and the design decisions behind the library. Given...
05.26.2020
114 - Behavioral Testing of NLP Models, with Marco Tulio Ribeiro
We invited Marco Tulio Ribeiro, a Senior Researcher at Microsoft, to talk about evaluating NLP models using behavioral testing, a framework borrowed from Software Engineering. Marco describes three kinds of black-box tests the check whether NLP models satisfy certain necessary...
05.22.2020
113 - Managing Industry Research Teams, with Fernando Pereira
We invited Fernando Pereira, a VP and Distinguished Engineer at Google, where he leads NLU and ML research, to talk about managing NLP research teams in industry. Topics we discussed include prioritizing research against product development and effective collaboration with...
05.13.2020
112 - Alignment of Multilingual Contextual Representations, with Steven Cao
We invited Steven Cao to talk about his paper on multilingual alignment of contextual word embeddings. We started by discussing how multilingual transformers work in general, and then focus on Steven’s work on aligning word representations. The core idea is...
04.27.2020
111 - Typologically diverse, multi-lingual, information-seeking questions, with Jon Clark
We invited Jon Clark from Google to talk about TyDi QA, a new question answering dataset, for this episode. The dataset contains information seeking questions in 11 languages that are typologically diverse, i.e., they differ from each other in terms...
04.06.2020
110 - Natural Questions, with Tom Kwiatkowski and Michael Collins
In this episode, Tom Kwiatkowski and Michael Collins talk about Natural Questions, a benchmark for question answering research. We discuss how the dataset was collected to reflect naturally-occurring questions, the criteria used for identifying short and long answers, how this...
03.30.2020
109 - What Does Your Model Know About Language, with Ellie Pavlick
How do we know, in a concrete quantitative sense, what a deep learning model knows about language? In this episode, Ellie Pavlick talks about two broad directions to address this question: structural and behavioral analysis of models. In structural analysis,...
03.23.2020
108 - Data-To-Text Generation, with Verena Rieser and Ondřej Dušek
In this episode we invite Verena Rieser and Ondřej Dušek on to talk to us about the complexities of generating natural language when you have some kind of structured meaning representation as input. We talk about when you might...
02.24.2020
107 - Multi-Modal Transformers, with Hao Tan and Mohit Bansal
In this episode, we invite Hao Tan and Mohit Bansal to talk about multi-modal training of transformers, focusing in particular on their EMNLP 2019 paper that introduced LXMERT, a vision+language transformer. We spend the first third of the episode...
02.17.2020
106 - Ethical Considerations In NLP Research, with Emily Bender
In this episode, we talked to Emily Bender about the ethical considerations in developing NLP models and putting them in production. Emily cited specific examples of ethical issues, and talked about the kinds of potential concerns to keep in mind,...
02.10.2020
105 - Question Generation, with Sudha Rao
In this episode we invite Sudha Rao to talk about question generation. We talk about different settings where you might want to generate questions: for human testing scenarios (rare), for data augmentation (has been done a bunch for SQuAD-like...
02.03.2020
104 - Model Distillation, with Victor Sanh and Thomas Wolf
In this episode we talked with Victor Sanh and Thomas Wolf from HuggingFace about model distillation, and DistilBERT as one example of distillation. The idea behind model distillation is compressing a large model by building a smaller model, with much...
01.27.2020
103 - Processing Language in Social Media, with Brendan O'Connor
We talked to Brendan O’Connor for this episode about processing language in social media. Brendan started off by telling us about his projects that studied the linguistic and geographical patterns of African American English (AAE), and how obtaining data from...
01.20.2020
102 - Biomedical NLP research at the National Institute of Health with Dina Demner-Fushman
What exciting NLP research problems are involved in processing biomedical and clinical data? In this episode, we spoke with Dina Demner-Fushman, who leads NLP and IR research at the Lister Hill National Center for Biomedical Communications, part of the National...
01.14.2020
101 - The lottery ticket hypothesis, with Jonathan Frankle
In this episode, Jonathan Frankle describes the lottery ticket hypothesis, a popular explanation of how over-parameterization helps in training neural networks. We discuss pruning methods used to uncover subnetworks (winning tickets) which were initialized in a particularly effective way. We...
01.08.2020
100 - NLP Startups, with Oren Etzioni
For our 100th episode, we invite AI2 CEO Oren Etzioni to talk to us about NLP startups. Oren has founded several successful startups, is himself an investor in startups, and helps with AI2's startup incubator. Some of our discussion topics...
12.16.2019
99 - Evaluating Protein Transfer Learning, With Roshan Rao And Neil Thomas
For this episode, we chatted with Neil Thomas and Roshan Rao about modeling protein sequences and evaluating transfer learning methods for a set of five protein modeling tasks. Learning representations using self-supervised pretaining objectives has shown promising results in transferring...
12.09.2019
98 - Analyzing Information Flow In Transformers, With Elena Voita
What function do the different attention heads serve in multi-headed attention models? In this episode, Lena describes how to use attribution methods to assess the importance and contribution of different heads in several tasks, and describes a gating mechanism...
11.27.2019
97 - Automated Analysis Of Historical Printed Documents, With Taylor Berg-Kirkpatrick
In this episode, we talk to Taylor Berg-Kirkpatrick about optical character recognition (OCR) on historical documents. Taylor starts off by describing some practical issues related to old scanning processes of documents that make performing OCR on them a difficult problem....
11.12.2019
96 - Question Answering as an Annotation Format, with Luke Zettlemoyer
In this episode, we chat with Luke Zettlemoyer about Question Answering as a format for crowdsourcing annotations of various semantic phenomena in text. We start by talking about QA-SRL and QAMR, two datasets that use QA pairs to annotate predicate-argument...
10.07.2019
95 - Common sense reasoning, with Yejin Choi
In this episode, we invite Yejin Choi to talk about common sense knowledge and reasoning, a growing area in NLP. We start by discussing a working definition of “common sense” and the practical utility of studying it. We then...

What's NLP Highlights about?

Download our free app to listen on your phone

NLP Highlights episodes:

Are LLMs safe?

"Imaginative AI" with Mohamed Elhoseiny

142 - Science Of Science, with Kyle Lo

141 - Building an open source LM, with Iz Beltagy and Dirk Groeneveld

140 - Generative AI and Copyright, with Chris Callison-Burch

139 - Coherent Long Story Generation, with Kevin Yang

138 - Compositional Generalization in Neural Networks, with Najoung Kim

137 - Nearest Neighbor Language Modeling and Machine Translation, with Urvashi Khandelwal

136 - Including Signed Languages in NLP, with Kayo Yin and Malihe Alikhani

135 - PhD Application Series: After Submitting Applications

134 - PhD Application Series: PhDs in Europe versus the US, with Barbara Plank and Gonçalo Correia

133 - PhD Application Series: Preparing Application Materials, with Nathan Schneider and Roma Patel

132 - Alexa Prize Socialbot Grand Challenge and Alquist 4.0, with Petr Marek

131 - Opportunities and Barriers between HCI and NLP, with Nanna Inie and Leon Derczynski

130 - Linking human cognitive patterns to NLP Models, with Lisa Beinborn

129 - Transformers and Hierarchical Structure, with Shunyu Yao

128 - Dynamic Benchmarking, with Douwe Kiela

127 - Masakhane and Participatory Research for African Languages, with Tosin Adewumi and Perez Ogayo

126 - Optimizing Continuous Prompts for Generation, with Lisa Li

125 - VQA for Real Users, with Danna Gurari

124 - Semantic Machines and Task-Oriented Dialog, with Jayant Krishnamurthy and Hao Fang

123 - Robust NLP, with Robin Jia

122 - Statutory Reasoning in Tax Law, with Nils Holzenberger

121 - Language and the Brain, with Alona Fyshe

120 - Evaluation of Text Generation, with Asli Celikyilmaz

119 - Social NLP, with Diyi Yang

118 - Coreference Resolution, with Marta Recasens

117 - Interpreting NLP Model Predictions, with Sameer Singh

116 - Grounded Language Understanding, with Yonatan Bisk

115 - AllenNLP, interviewing Matt Gardner

114 - Behavioral Testing of NLP Models, with Marco Tulio Ribeiro

113 - Managing Industry Research Teams, with Fernando Pereira

112 - Alignment of Multilingual Contextual Representations, with Steven Cao

111 - Typologically diverse, multi-lingual, information-seeking questions, with Jon Clark

110 - Natural Questions, with Tom Kwiatkowski and Michael Collins

109 - What Does Your Model Know About Language, with Ellie Pavlick

108 - Data-To-Text Generation, with Verena Rieser and Ondřej Dušek

107 - Multi-Modal Transformers, with Hao Tan and Mohit Bansal

106 - Ethical Considerations In NLP Research, with Emily Bender

105 - Question Generation, with Sudha Rao

104 - Model Distillation, with Victor Sanh and Thomas Wolf

103 - Processing Language in Social Media, with Brendan O'Connor

102 - Biomedical NLP research at the National Institute of Health with Dina Demner-Fushman

101 - The lottery ticket hypothesis, with Jonathan Frankle

100 - NLP Startups, with Oren Etzioni

99 - Evaluating Protein Transfer Learning, With Roshan Rao And Neil Thomas

98 - Analyzing Information Flow In Transformers, With Elena Voita

97 - Automated Analysis Of Historical Printed Documents, With Taylor Berg-Kirkpatrick

96 - Question Answering as an Annotation Format, with Luke Zettlemoyer

95 - Common sense reasoning, with Yejin Choi