The Test Set by Posit

Marco Gorelli: Narwhals, ecosystem glue, and the value of boring work


Listen Later

You’ve probably used Narwhals without realizing it. It’s the compatibility layer helping apps and libraries like Plotly play nice with Pandas, Polars, Arrow, and more — while keeping computation native instead of converting everything to Pandas. In this episode, Marco Gorelli explains how his weekend experiment turned into essential ecosystem infrastructure and why data types, not APIs, are where interoperability gets tricky. Plus what it takes to build trust and community around an open-source project.


Episode Notes
Marco shares the Narwhals origin story (including the meme-powered name), the hard edge cases that live in data types and null semantics, and why he’s cautious about using AI for code generation when correctness hinges on tiny details. We also jam on proactive “GitHub surfing,” conference talks as trust-building exercises, celebrating contributors, and how early commit messages capture the genuine excitement of building something new.


What’s Inside

  • Narwhals 101: You’ve probably used it (even if you didn’t know it)
  • The real interoperability traps: data types, null semantics, and “looks-the-same” operations
  • Why expression systems won, and how they shaped Marco’s approach — with nods to Ibis, Polars, and Pandas
  • Open source as social work: proactive outreach, trust-building, and a Discord-powered community
  • Extending Narwhals to new engines, starting with the Daft plugin
...more
View all episodesView all episodes
Download on the App Store

The Test Set by PositBy Posit, PBC

  • 5
  • 5
  • 5
  • 5
  • 5

5

24 ratings


More shows like The Test Set by Posit

View all
Freakonomics Radio by Freakonomics Radio + Stitcher

Freakonomics Radio

32,011 Listeners

Planet Money by NPR

Planet Money

30,680 Listeners

Odd Lots by Bloomberg

Odd Lots

1,929 Listeners

Bloomberg Law by Bloomberg

Bloomberg Law

376 Listeners

Data Skeptic by Kyle Polich

Data Skeptic

480 Listeners

Talk Python To Me by Michael Kennedy

Talk Python To Me

585 Listeners

Super Data Science: ML & AI Podcast with Jon Krohn by Jon Krohn

Super Data Science: ML & AI Podcast with Jon Krohn

303 Listeners

CoRecursive: Coding Stories by Adam Gordon Bell - Software Developer

CoRecursive: Coding Stories

190 Listeners

DataFramed by DataCamp

DataFramed

269 Listeners

Practical AI by Practical AI LLC

Practical AI

207 Listeners

The Pie: An Economics Podcast by Becker Friedman Institute at UChicago

The Pie: An Economics Podcast

174 Listeners

Dwarkesh Podcast by Dwarkesh Patel

Dwarkesh Podcast

517 Listeners

R Weekly Highlights by Eric Nantz

R Weekly Highlights

6 Listeners

Hard Fork by The New York Times

Hard Fork

5,509 Listeners

The Pragmatic Engineer by Gergely Orosz

The Pragmatic Engineer

64 Listeners