Talk Python To Me

#540: Modern Python monorepo with uv and prek


Listen Later

Monorepos -- you've heard the talks, you've read the blog posts, maybe you've seen a few tantalizing glimpses into how Google or Meta organize their massive codebases. But it's often in the abstract and behind closed doors. What if you could crack open a real, production monorepo, one with over a million lines of Python and over 100 of sub-packages, and actually see how it's built, step by step, using modern tools and standards? That's exactly what Apache Airflow gives us.


On this episode, I sit down with Jarek Potiuk and Amogh Desai, two of Airflow's top contributors, to go inside one of the largest open-source Python monorepos in the world and learn how they manage it with uv, pyproject.toml, and the latest packaging standards, so you can apply those same patterns to your own projects.

Episode sponsors

Agentic AI Course
Python in Production
Talk Python Courses

Links from the show
Guests
Amogh Desai: github.com
Jarek's GitHub: github.com

definition of a monorepo: monorepo.tools
airflow: airflow.apache.org
Activity: github.com
OpenAI: airflowsummit.org
Part 1. Pains of big modular Python projects: medium.com
Part 2. Modern Python packaging standards and tools for monorepos: medium.com
Part 3. Monorepo on steroids - modular prek hooks: medium.com
Part 4. Shared “static” libraries in Airflow monorepo: medium.com
PEP-440: peps.python.org
PEP-517: peps.python.org
PEP-518: peps.python.org
PEP-566: peps.python.org
PEP-561: peps.python.org
PEP-660: peps.python.org
PEP-621: peps.python.org
PEP-685: peps.python.org
PEP-723: peps.python.org
PEP-735: peps.python.org
uv: docs.astral.sh
uv workspaces: blobs.talkpython.fm
prek.j178.dev: prek.j178.dev
your presentation at FOSDEM26: fosdem.org
Tallyman: github.com

Watch this episode on YouTube: youtube.com
Episode #540 deep-dive: talkpython.fm/540
Episode transcripts: talkpython.fm

Theme Song: Developer Rap
🥁 Served in a Flask 🎸: talkpython.fm/flasksong

---== Don't be a stranger ==---
YouTube: youtube.com/@talkpython

Bluesky: @talkpython.fm
X.com: @talkpython

Michael on Bluesky: @mkennedy.codes
Michael on Mastodon: @[email protected]
Michael on X.com: @mkennedy
...more
View all episodesView all episodes
Download on the App Store

Talk Python To MeBy Michael Kennedy

  • 4.8
  • 4.8
  • 4.8
  • 4.8
  • 4.8

4.8

579 ratings


More shows like Talk Python To Me

View all
The Changelog: Software Development, Open Source by Changelog Media

The Changelog: Software Development, Open Source

288 Listeners

Software Engineering Daily by Software Engineering Daily

Software Engineering Daily

629 Listeners

Super Data Science: ML & AI Podcast with Jon Krohn by Jon Krohn

Super Data Science: ML & AI Podcast with Jon Krohn

305 Listeners

Python Bytes by Michael Kennedy and Brian Okken

Python Bytes

214 Listeners

NVIDIA AI Podcast by NVIDIA

NVIDIA AI Podcast

343 Listeners

Syntax - Tasty Web Development Treats by Wes Bos & Scott Tolinski - Full Stack JavaScript Web Developers

Syntax - Tasty Web Development Treats

986 Listeners

Practical AI by Practical AI LLC

Practical AI

209 Listeners

Django Chat by William Vincent and Carlton Gibson

Django Chat

75 Listeners

The Real Python Podcast by Real Python

The Real Python Podcast

140 Listeners

Last Week in AI by Skynet Today

Last Week in AI

314 Listeners

Dwarkesh Podcast by Dwarkesh Patel

Dwarkesh Podcast

551 Listeners

Big Technology Podcast by Alex Kantrowitz

Big Technology Podcast

513 Listeners

Latent Space: The AI Engineer Podcast by Latent.Space

Latent Space: The AI Engineer Podcast

102 Listeners

This Day in AI Podcast by Michael Sharkey, Chris Sharkey

This Day in AI Podcast

228 Listeners

The AI Daily Brief: Artificial Intelligence News and Analysis by Nathaniel Whittemore

The AI Daily Brief: Artificial Intelligence News and Analysis

685 Listeners