Data Skeptic

By Kyle Polich

Technology

What's Data Skeptic about?

The Data Skeptic Podcast features interviews and discussion of topics related to data science, statistics, machine learning, artificial intelligence and the like, all from the perspective of applying critical thinking and the scientific method to evaluate the veracity of claims and efficacy of approaches.

4.4

439 ratings

Download our free app to listen on your phone

LAST EPISODE

AI Platforms

12.04.2023

Our guest today is Eric Boyd, the Corporate Vice President of AI at Microsoft. Eric joins us to share how organizations can leverage AI for faster development. Eric shared the benefits of using natural language to build products. He discussed ...

Data Skeptic episodes:

12.04.2023 AI Platforms Our guest today is Eric Boyd, the Corporate Vice President of AI at Microsoft. Eric joins us to share how organizations can leverage AI for faster development. Eric shared the benefits of using natural language to build products. He discussed ...
11.27.2023 Deploying LLMs We are excited to be joined by Aaron Reich and Priyanka Shah. Aaron is the CTO at Avanade, while Priyanka leads their AI/IoT offering for the SEA Region. Priyanka is also the MVP for Microsoft AI. They join us to ...
11.20.2023 A Survey Assessing Github Copilot In this episode, we are joined by Jenny Liang, a PhD student at Carnegie Mellon University, where she studies the usability of code generation tools. She discusses her recent survey on the usability of AI programming assistants. Jenny discussed the ...
11.13.2023 Program Aided Language Models We are joined by Aman Madaan and Shuyan Zhou. They are both PhD students at the Language Technology Institute at Carnegie Mellon University. They join us to discuss their latest published paper, PAL: Program-aided Language Models. Aman and Shuyan started ...
11.06.2023 Which Programming Language is ChatGPT Best At In this episode, we have Alessio Buscemi, a software engineer at Lifeware SA. Alessio was a post-doctoral researcher at the University of Luxembourg. He joins us to discuss his paper, A Comparative Study of Code Generation using ChatGPT 3.5 across ...
10.31.2023 GraphText On the show today, we are joined by Jianan Zhao, a Computer Science student at Mila and the University of Montreal. His research focus is on graph databases and natural language processing. He joins us to discuss how to use ...
10.23.2023 arXiv Publication Patterns Today, we are joined by Rajiv Movva, a PhD student in Computer Science at Cornell Tech University. His research interest lies in the intersection of responsible AI and computational social science. He joins to discuss the findings of this work ...
10.16.2023 Do LLMs Make Ethical Choices We are excited to be joined by Josh Albrecht, the CTO of Imbue. Imbue is a research company whose mission is to create AI agents that are more robust, safer, and easier to use. He joins us to share findings ...
10.09.2023 Emergent Deception in LLMs On today’s show, we are joined by Thilo Hagendorff, a Research Group Leader of Ethics of Generative AI at the University of Stuttgart. He joins us to discuss his research, Deception Abilities Emerged in Large Language Models. Thilo discussed how ...
10.02.2023 Agents with Theory of Mind Play Hanabi Nieves Montes, a Ph.D. student at the Artificial Intelligence Research Institute in Barcelona, Spain, joins us. Her PhD research revolves around value-based reasoning in relation to norms. She shares her latest study, Combining theory of mind and abductive reasoning in ...
09.25.2023 LLMs for Evil We are joined by Maximilian Mozes, a PhD student at the University College, London. His PhD research focuses on Natural Language Processing (NLP), particularly the intersection of adversarial machine learning and NLP. He joins us to discuss his latest research, ...
09.11.2023 The Defeat of the Winograd Schema Challenge Our guest today is Vid Kocijan, a Machine Learning Engineer at Kumo AI. Vid has a Ph.D. in Computer Science at the University of Oxford. His research focused on common sense reasoning, pre-training in LLMs, pretraining in knowledge-based completion, and ...
09.04.2023 LLMs in Social Science Today, We are joined by Petter Törnberg, an Assistant Professor in Computational Social Science at the University of Amsterdam and a Senior Researcher at the University of Neuchatel. His research is centered on the intersection of computational methods and their ...
08.28.2023 LLMs in Music Composition In this episode, we are joined by Carlos Hernández Oliván, a Ph.D. student at the University of Zaragoza. Carlos’s interest focuses on building new models for symbolic music generation. Carlos shared his thoughts on whether these models are genuinely creative. ...
08.21.2023 Cuttlefish Model Tuning Hongyi Wang, a Senior Researcher at the Machine Learning Department at Carnegie Mellon University, joins us. His research is in the intersection of systems and machine learning. He discussed his research paper, Cuttlefish: Low-Rank Model Training without All the Tuning, ...
08.15.2023 Which Professions Are Threatened by LLMs On today’s episode, we have Daniel Rock, an Assistant Professor of Operations Information and Decisions at the Wharton School of the University of Pennsylvania. Daniel’s research focuses on the economics of AI and ML, specifically how digital technologies are changing ...
08.08.2023 Why Prompting is Hard We are excited to be joined by J.D. Zamfirescu-Pereira, a Ph.D. student at UC Berkeley. He focuses on the intersection of human-computer interaction (HCI) and artificial intelligence (AI). He joins us to share his work in his paper, Why Johnny ...
07.31.2023 Automated Peer Review In this episode, we are joined by Ryan Liu, a Computer Science graduate of Carnegie Mellon University. Ryan will begin his Ph.D. program at Princeton University this fall. His Ph.D. will focus on the intersection of large language models and ...
07.24.2023 Prompt Refusal The creators of large language models impose restrictions on some of the types of requests one might make of them. LLMs commonly refuse to give advice on committing crimes, producting adult content, or respond with any details about a variety ...
07.18.2023 A Long Way Till AGI Our guest today is Maciej Świechowski. Maciej is affiliated with QED Software and QED Games. He has a Ph.D. in Systems Research from the Polish Academy of Sciences. Maciej joins us to discuss findings from his study, Deep Learning and ...
07.11.2023 Brain Inspired AI Today on the show, we are joined by Lin Zhao and Lu Zhang. Lin is a Senior Research Scientist at United Imaging Intelligence, while Lu is a Ph.D. candidate at the Department of Computer Science and Engineering at the University ...
07.03.2023 Computable AGI On today’s show, we are joined by Michael Timothy Bennett, a Ph.D. student at the Australian National University. Michael’s research is centered around Artificial General Intelligence (AGI), specifically the mathematical formalism of AGIs. He joins us to discuss findings from ...
06.26.2023 AGI Can Be Safe We are joined by Koen Holtman, an independent AI researcher focusing on AI safety. Koen is the Founder of Holtman Systems Research, a research company based in the Netherlands. Koen started the conversation with his take on an AI apocalypse ...
06.19.2023 AI Fails on Theory of Mind Tasks An assistant professor of Psychology at Harvard University, Tomer Ullman, joins us. Tomer discussed the theory of mind and whether machines can indeed pass it. Using variations of the Sally-Anne test and the Smarties tube test, he explained how LLMs ...
06.12.2023 AI for Mathematics Education The application of LLMs cuts across various industries. Today, we are joined by Steven Van Vaerenbergh, who discussed the application of AI in mathematics education. He discussed how AI tools have changed the landscape of solving mathematical problems. He also ...
06.06.2023 Evaluating Jokes with LLMs Fabricio Goes, a Lecturer in Creative Computing at the University of Leicester, joins us today. Fabricio discussed what creativity entails and how to evaluate jokes with LLMs. He specifically shared the process of evaluating jokes with GPT-3 and GPT-4. He ...
05.29.2023 Why Machines Will Never Rule the World Barry Smith and Jobst Landgrebe, authors of the book “Why Machines will never Rule the World,” join us today. They discussed the limitations of AI systems in today’s world. They also shared elaborate reasons AI will struggle to attain the ...
05.23.2023 A Psychopathological Approach to Safety in AGI While the possibilities with AGI emergence seem great, it also calls for safety concerns. On the show, Vahid Behzadan, an Assistant Professor of Computer Science and Data Science, joins us to discuss the complexities of modeling AGIs to accurately achieve ...
05.15.2023 The NLP Community Metasurvey Julian Michael, a postdoc at the Center for Data Science, New York University, joins us today. Julian’s conversation with Kyle was centered on the NLP community metasurvey: a survey aimed at understanding expert opinions on controversial NLP issues. He shared ...
05.10.2023 Skeptical Survey Interpretation Kyle shares his own perspectives on challenges getting insight from surveys. The discussion ranges from commentary on the market research industry to specific advice for detecting disingenuous or fraudulent responses and filtering them from your analysis. Finally, he shares some ...
05.01.2023 The Gallup Poll Jeff Jones, a Senior Editor at Gallup, joins us today. His conversation with Kyle spanned a range of topics on Gallup’s poll creation process. He discussed how Gallup generates unbiased questionnaires, gets respondents, analyzes results, and everything in between.
04.25.2023 Inclusive Study Group Formation at Scale Gireeja Ranade, a University of California at Berkeley professor, speaks with us today. She presented her study on implementing inclusive study groups at scale and shared the observed student performance improvements after the intervention.
04.21.2023 The PhilPapers Survey Today, we are joined by David Bourget. David is an Associate Professor in Philosophy at Western University in London, Ontario. David is also the co-director of the PhilPapers Foundation and Director of the Center for Digital Philosophy. He joins us ...
04.10.2023 Non-Response Bias Today’s show focused on an essential part of surveys — missing values. This is typically caused by a low response rate or non-response from respondents. Yajuan Si is a Research Associate Professor at the Survey Research Center at the University ...
04.03.2023 Measuring Trust in Robots with Likert Scales We are joined by two guests today, Mariah, a Ph.D. student in the CORE Robotics Lab at Georgia Tech, and Matthew Gombolay, the Director of the CORE Robotics Lab. They both discuss practices for measuring a respondent’s perception in a ...
03.27.2023 CAREER Prediction Ever wondered what your next career would be? Today, Keyon Vafa, a computer science Ph.D. student at Columbia University, joins us to discuss his latest research on developing a machine-learning model for career prediction. Keyon extensively spoke about how the ...
03.21.2023 The Panel Study of Income Dynamics Noura Insolera, a Research Investigator with the Panel Study of Income Dynamics (PSID), joins us to share how PSID conducts longitudinal household surveys. She also shared some interesting findings from their data exploration, particularly on the observation and trends in ...
03.14.2023 Survey Design Working Session Susan Gerbic joins Kyle to review some of the surveys Data Skeptic has launch, draft a new survey about podcast listening habits, and then review the results of that survey. You can see those results at the link below. https://survey.dataskeptic.com/survey/result/1675102237053 ...
03.06.2023 Bot Detection and Dyadic Surveys The use of social bots to fill out online surveys is becoming prevalent. Today, we speak with Sara Bybee, a postdoctoral research scholar at the University of Utah. Sara shares from her research, how she detected social bots, the strategies ...
02.20.2023 Reproducible ESP Testing Our guest today is Zoltán Kekecs, a Ph.D. holder in Behavioural Science. Zoltán highlights the problem of low replicability in journal papers and illustrates how researchers can better ensure complete replication of their research and findings. He used Bem’s experiment ...
02.13.2023 A Survey of Data Science Methodologies On the show, Iñigo Martinez, a Ph.D. student at the University of Navarra shares his survey results which investigated how data practitioners perform data science projects. He revealed the methodologies typically used by data practitioners and the success factors in ...
02.06.2023 Opinion Dynamics Models On the show today, Dino Carpentras, a post-doctoral researcher at the Computational Social Science group at ETH Zürich joins us to discuss how opinion dynamics models are built and validated. He explained how quantifying opinions is complex, and strategies to ...
01.30.2023 Casual Affective Triggers Crafting survey questions is one thing but getting your audience to fill it is yet another. On the show today, we speak with Alexander Nolte, an Associate Professor at the University of Tartu. Alexander discussed the use of Casual Affective ...
01.23.2023 Conversational Surveys Traditional surveys have straight-jacket questions to be answered, thus restricting the information that can be gotten. Today, Ziang Xiao, a Postdoc Researcher in the FATE group at Microsoft Research Montréal, talks about conversational surveys, a type of survey that asks ...
01.17.2023 Do Results Generalize for Privacy and Security Surveys Today, Jenny Tang, a Ph.D. student of societal computing at Carnegie Mellon University discusses her work on the generalization of privacy and security surveys on platforms such as Amazon MTurk and Prolific. Jenny shared the drawbacks of using such online ...
01.10.2023 4 out of 5 Data Scientists Agree This episode kicks off the new season of the show, Data Skeptic: Surveys. Linhda rejoins the show for a conversation with Kyle about her experience taking surveys and what questions she has for the season. Lastly, Kyle announces the launch ...
12.26.2022 Crowdfunded Board Games It may be intuitive to think crowdfunding a project drives its innovation and novelty, but there are no empirical studies that prove this. On the show, Johannes Wachs shares his research that sought to determine whether crowdfunding truly drives innovation. ...
12.19.2022 Russian Election Interference Effectiveness There were reports of Russia’s interference in the 2016 US elections. In today’s episode, Koustuv Saha, a researcher at Microsoft Research walks us through the effect of targeted ads for political campaigns. Using practical examples, he discusses how targeted ads ...
12.15.2022 Placement Laundering Fraud There is an unsung kind of ad fraud brewing in the ad tech space — placement laundering fraud. On the show, Jeff Kline discusses what placement laundering fraud is, how it can be identified, and possible solutions to it. Listen ...
12.12.2022 Data Clean Rooms Bosko Milekic, the Co-founder of Optable, a data collaboration platform for the media and advertising industry, joins us today. Bosko talked about the clean rooms, the technology driving data privacy during collaboration. He discussed why clean rooms are gaining widespread ...

Older episodes

Data Skeptic

What's Data Skeptic about?

Download our free app to listen on your phone

Data Skeptic episodes:

AI Platforms

Deploying LLMs

A Survey Assessing Github Copilot

Program Aided Language Models

Which Programming Language is ChatGPT Best At

GraphText

arXiv Publication Patterns

Do LLMs Make Ethical Choices

Emergent Deception in LLMs

Agents with Theory of Mind Play Hanabi

LLMs for Evil

The Defeat of the Winograd Schema Challenge

LLMs in Social Science

LLMs in Music Composition

Cuttlefish Model Tuning

Which Professions Are Threatened by LLMs

Why Prompting is Hard

Automated Peer Review

Prompt Refusal

A Long Way Till AGI

Brain Inspired AI

Computable AGI

AGI Can Be Safe

AI Fails on Theory of Mind Tasks

AI for Mathematics Education

Evaluating Jokes with LLMs

Why Machines Will Never Rule the World

A Psychopathological Approach to Safety in AGI

The NLP Community Metasurvey

Skeptical Survey Interpretation

The Gallup Poll

Inclusive Study Group Formation at Scale

The PhilPapers Survey

Non-Response Bias

Measuring Trust in Robots with Likert Scales

CAREER Prediction

The Panel Study of Income Dynamics

Survey Design Working Session

Bot Detection and Dyadic Surveys

Reproducible ESP Testing

A Survey of Data Science Methodologies

Opinion Dynamics Models

Casual Affective Triggers

Conversational Surveys

Do Results Generalize for Privacy and Security Surveys

4 out of 5 Data Scientists Agree

Crowdfunded Board Games

Russian Election Interference Effectiveness

Placement Laundering Fraud

Data Clean Rooms