core.py

Who says we can't do short episodes? Well, it was a challenge! But with the help of some gentle conference schedule pressure, here's our first sub-hour episode. We're discussing the language summit, answering audience questions, and behaving like serious professionals, as usual.

## Outline

(00:00:00) INTRO

(00:02:01) PEP 602 tweaks: 2 years of bug fix releases

(00:02:45) CalVer for Python versions?

(00:04:51) In Python there's this thing called the C API

(00:08:51) More about PyREPL

(00:12:17) Evolving pdb

(00:14:49) Memhive

(00:16:18) The boring 3.13

(00:18:16) Q&A

(00:18:43) Fashion + favorite bug

(00:22:12) What happens in 2100 with CalVer?

(00:23:02) Subinterpreters or free-threading?

(00:23:41) Where else to find information on current development?

(00:25:40) Will distros provide free-threading builds?

(00:26:23) Will the podcast run out of content?

(00:28:37) OUTRO

Oof, no episode in April, huh? Yeah, we're getting close to Python 3.13 beta 1. PyCon US is also coming up real soon. Let's use this opportunity then to talk about a feature we're teaming up on: a better interactive interpreter!

## Outline

(00:00:00)  INTRO

(00:01:53)  PART 1: History of Terminals

(00:03:20)  /dev/tty

(00:04:51)  The first cool word

(00:05:45)  Chrząszcz

(00:06:20)  Control code characters in ASCII

(00:11:54)  PART 2: Python REPL Today

(00:12:34)  There is no REPL

(00:15:28)  So what is there instead?

(00:19:13)  readline

(00:25:38)  Source in the REPL

(00:31:13)  Implementing a REPL from scratch? Prepare to support arg: 5

(00:36:09)  PART 3: PR OF THE WEEK

(00:37:09)  Introducing: Complaining Pablo

(00:38:23)  Tests are always green if you skip them

(00:39:57)  Getting dirty with escape sequences

(00:41:28)  Typing finds bugs

(00:42:29)  Shiny new features of the new REPL

(00:45:55)  Contributing back to PyPy

(00:48:10)  We still have two weeks, right?

(00:49:59)  Is Python synthwave enough?

(00:51:57)  Do we have a bug?

(00:55:31)  What's lurking in pydoc?

(00:59:38)  PART 4: WHAT'S HAPPENING IN CPYTHON?

(01:02:39)  PEP 744: The JIT

(01:06:05)  Incremental GC is now actually in

(01:08:21)  Tier 2 interpreter updates

(01:10:29)  Python supported on iOS with PEP 730

(01:13:11)  Better error messages for name shadowing

(01:15:17)  Queue.shutdown()

(01:17:14)  ctypes adopts heap types

(01:18:26)  Free-threading updates

(01:20:14)  Dataclass creation is faster

(01:20:44)  OUTRO

Let's talk about the Steering Council, running a small consultancy business, the Walrus, and pet peeves with our special guest today!

## Outline

(00:00:00)  INTRO

(00:00:56)  PART 1: Emily Morehouse

(00:02:15)  Running a small consultancy business

(00:04:39)  What features of JS do you miss in Python?

(00:05:50)  Łukasz outnumbered in a world of Steering Council members

(00:06:12)  Upgrading to new Python versions

(00:07:00)  It depends on who deployed the project

(00:09:44)  Second term as a Steering Council member

(00:11:33)  Barry, play some bass for us

(00:13:04)  Let's hear a recent war story

(00:15:17)  Is this progress bar even working?

(00:17:40)  The Villain Origin Story

(00:21:37)  Emily, The Bringer of Doom

(00:22:37)  Consensus within the Steering Council

(00:25:52)  Syntax changes in Python are rare, right? Right?

(00:28:22)  On implementing PEP 572

(00:32:52)  How would PyCon 2020 in Pittsburgh feel?

(00:34:18)  How can you be mad about the Walrus?

(00:36:10)  Favorite parts of the standard library

(00:38:10)  Is hacking on Python a good experience to newcomers?

(00:40:26)  Emily's pet peeve about Python, take 1

(00:42:17)  Emily's favorite change in Python in recent years

(00:44:34)  Emily's pet peeve about Python, take 2

(00:46:34)  Łukasz's pet peeve

(00:48:25)  Surprise extra question

(00:49:42)  At core.py we are professionals

(00:51:00)  PART 2: PR of the Week

(00:54:00)  CALL TO ACTION: Upgrade Python.org to Django 4!

(00:56:22)  PART 3: What's Going On in CPython?

(00:56:38)  Faster Python updates

(01:00:10)  Free threading: GIL can be disabled but we're not done yet!

(01:04:17)  New defaults for SSL context flags

(01:05:39)  python -m asyncio and sys.__interactivehook__

(01:06:24)  Surprise question: what is sys.__interactivehook__ even doing?

(01:08:11)  OUTRO

The suspense was killing us! OK, the old parser was then... but what about NOW? We're finally answering this question... in more detail than you dared to ask for. PEG, memoization, funky secrets, and how a certain auto-formatter self-inflicted an existential crisis on itself. It's all there, told in barely 100 minutes! Can you believe it?

# Timestamps

(00:00:00)  INTRO

(00:00:54)  PART 1: What even is PEG?

(00:04:02)  You can't prove anything!

(00:05:03)  What's a "parsing expression"?

(00:08:23)  Our old LL1 parser wasn't doing its job

(00:09:37)  "Soft keywords" in LL1: A Horror Story

(00:13:16)  PART 2: How PEG was adopted by Python

(00:17:10)  Why not LALR?

(00:22:11)  The PEG paper wasn't enough either, if we're honest

(00:26:26)  Less obvious advantages of the new parser

(00:31:28)  Black is stuck with LL1, can it cope?

(00:36:24)  Hedging against Łukasz, the bringer of doom

(00:41:14)  PART 3: How does the PEG parser of CPython work?

(00:44:30)  Pedantic Pablo on "exponential"

(00:45:14)  Fresh news from literally yesterday last week

(00:46:39)  Pedantic Pablo on "infinite"

(00:47:32)  Memoization in the PEG parser

(00:50:41)  Parse once, and if it fails, try again!

(00:52:14)  How to model a grammar of programming mistakes?

(00:56:36)  Why is there C code in my grammar file?

(00:59:57)  Bro, do you even lift?

(01:01:45)  How soft keywords work today: it's not free lunch

(01:04:29)  Funky grammar secrets

(01:09:07)  PART 4: PR OF THE WEEK

(01:09:15)  audioop.c license shenanigans

(01:14:56)  The secret profiler inside CPython (tests)

(01:22:45)  PART 5: WHAT'S GOING ON IN CPYTHON?

(01:23:30)  Free-threading changes

(01:28:15)  Faster Python changes

(01:35:39)  End of an era: docs get rid of Python 2 migration info

(01:36:45)  Python --help output is now nicer

(01:38:43) SQLite as a dbm backend

(01:41:08)  OUTRO

Context-free grammars, non-deterministic finite automatons, left-to-right leftmost derivations... what even is all that?! Today we're talking about how Python parses your source code. We start gently with how this worked in the past. Come listen to Łukasz's high-level explanations and Pedantic Pablo's "well actuallys".

# Timestamps

(00:00:00) INTRO

(00:01:35) You can still download Python 1.0!

(00:02:19) The original tokenizer

(00:03:10) What even is a tokenizer?

(00:04:08) FUN FACTS ABOUT THE TOKENIZER

(00:04:34) Circumflex

(00:05:16) Python's invisible braces

(00:08:29) Backticks in the syntax

(00:11:00) Where are the comments stored?

(00:12:27) GRAMMAR

(00:13:37) What is a grammar?

(00:16:25) The long-forgotten 'access' keyword

(00:20:25) Making LL1 do things it wasn't meant to do

(00:23:24) SURPRISE QUESTION 1: soft keywords

(00:24:46) What's a context-free grammar?

(00:26:51) A note about backslashes

(00:29:33) The Dragon Book(s)

(00:31:27) PARSING: What is it?

(00:35:23) How to generate a parser?

(00:39:00) LL Cool Parser

(00:41:15) What if we used LR?

(00:44:01) Let's have three tokenizers!

(00:47:50) 2to3 and its legacy

(00:52:38) Black and its blib2to3

(00:54:04) The pesky 'with' statement and the death of LL1

(01:00:05) PR OF THE WEEK: GH-113745

(01:05:41) SURPRISE QUESTION 2: Subclasses of SyntaxError

(01:07:02) WHAT'S GOING ON IN CPYTHON?

(01:09:16) Sam Gross nominated as a core dev

(01:10:13) Free-threading progress

(01:13:11) Faster CPython changes

(01:17:29) ntpath.isreserved()

(01:20:11) Pablo and the DWARF

(01:22:02) OUTRO

Happy New Year! In this episode we're talking about exceptions, how they work, and how they evolved. Expect the unexpected.

# Timestamps

(00:00:00)  INTRO

(00:01:43)  How does a 'try' block work?

(00:04:00)  How many 'try' blocks can you fit on a bus?

(00:05:56)  How does Python store the current exception?

(00:09:30)  Pre-history: exceptions as strings

(00:12:59)  Try out string exceptions with CentOS 5

(00:14:28)  PEP 341: Combining 'finally' and 'except' in one 'try' block

(00:16:15)  Core Hacker in Residence

(00:16:51)  PEP 3109: Raising exceptions in Python 3K

(00:19:17)  Automatic tuple unpacking for raised exceptions?!

(00:21:55)  PEP 3110: Catching exceptions in Python 3K

(00:26:00)  Foreshadowing: exception groups

(00:27:10)  PEP 3134: Exception chaining

(00:29:12)  __cause__, __context__, __traceback__

(00:31:50)  Back in the day we had to walk uphill both ways

(00:32:56)  PEP 409: Suppressing exception chaining

(00:34:44)  Raise from None? Or raise from Ellipsis?

(00:37:11)  __supress_context__

(00:38:13)  Semantic difference between 'pass', 'None', and '....'

(00:41:02)  NotImplemented vs NotImplementedError

(00:43:02)  Zero-cost exceptions in Python 3.11

(00:51:12)  Reconstructing exception table entries dynamically

(00:52:51)  Objects/exception_handling_notes.txt

(00:54:19)  PEP 654: Historical context

(00:56:58)  PEP 654: BaseExceptionGroup and ExceptionGroup

(00:58:29)  PEP 654: except*

(01:03:23)  PEP 678: Exception notes

(01:06:09)  PEP 657: Cooler Errors

(01:08:36)  A message to language implementers

(01:09:15)  Fine-grained error locations in tracebacks

(01:12:05)  This is useful for tracing coverage and Specializator

(01:13:50)  Hacker in Residence fired?

(01:16:25)  WHAT'S GOING ON IN CPYTHON

(01:16:58)  Copy&patch JIT PR open

(01:21:30)  Free-threading progress: GC split, stop-the-world

(01:24:22)  The buildbots are red

(01:25:27)  Faster CPython changes: interpreter code generator refactors

(01:26:27)  Eval Game crash fix

(01:28:41)  Three developers in residence!

(01:29:54)  OUTRO

# Links

https://compilercrim.es/rust-np/

This time we're hosting a special guest: Carl Meyer from Meta. What is Cinder, how does it work, and how does it intersect with the future of Python 3? Find out in today's episode. 100% serious stuff!

# Timestamps

(00:00:00)  INTRO

(00:00:53)  Carl Meyer's war story

(00:02:27)  CINDER

(00:03:22)  Static Python makes things significantly faster

(00:08:15)  Cinder JIT and how it's tuned for Instagram

(00:11:57)  Strict Python and the joy of import side effects

(00:16:35)  The static typing controversy

(00:18:52)  Upstreaming changes from Cinder?

(00:22:53)  PEP 709: Comprehension inlining

(00:28:35)  pip install CinderX

(00:31:19)  Immortal instances

(00:35:15)  asyncio.eager_task_factory()

(00:39:39)  Carl's pet peeve with Python

(00:44:49)  PR OF THE WEEK: PyPy's REPL in CPython

(00:52:07)  WHAT'S GOING ON IN CPYTHON

(00:52:22)  Python 3.12.1

(00:53:17)  Python 3.11.7

(00:54:45)  multiprocessing.SharedMemory track

(00:56:49)  Fine-grained error locations for multi-line expressions

(01:00:03)  libedit tab completion is fixed

(01:02:14)  Colored exception tracebacks

(01:05:11)  Removing testing modules from sys.modules, correctly

(01:06:47)  SBOMs are a very serious matter

(01:09:08)  Arrays by value on ARM

(01:12:24)  Remove development environments and CAPS LOCK

(01:15:30)  Interpreter cases generator refactored

(01:16:17)  Free-threading news

(01:20:01)  OUTRO

What makes Python an interpreter? Today we're talking about ceval.c, the wonders of frame evaluation, and how it changed over the years.

# Timestamps

(00:00:00)  INTRO

(00:00:59)  BACK TO PYTHON 2.6

(00:02:53)  Stack virtual machine

(00:04:41)  First encounter with opcodes

(00:08:06)  What even is frame evaluation?

(00:12:51)  Stack! Which stack?

(00:15:46)  PRESENT DAY

(00:16:41)  Computed gotos

(00:21:22)  PEP 523: JIT me, maybe

(00:26:53)  Let's generate the interpreter

(00:29:08)  The JIT is coming

(00:33:13)  Python function call inlining

(00:37:23)  Instrumentation: DTrace, PEP 669

(00:41:50)  lltrace and pystats

(00:44:02)  Eval breaker

(00:47:54)  Signal handling

(00:50:47)  Recursion limits

(00:54:27)  String concatenation special case

(00:58:24)  WHAT'S GOING ON IN CPYTHON?

(00:58:42)  3.12.0a2

(00:59:12)  Critical section API adoption

(00:59:34)  PyOnceFlag

(01:00:28)  PyDict_GetItemRef()

(01:03:36)  PyList_Extend() and PyDict_Pop()

(01:04:18)  Parser: better error messages for non-matching elif/else

(01:05:39)  glob.translate()

(01:07:22)  TLS-PSK in the ssl module

(01:08:35)  IDLE debugger improvements

(01:10:50)  First micro-op in the Tier 2 interpreter

(01:11:18)  OUTRO

What happens when you type “import abc”? Why does it say the module is frozen? What significant changes landed in Python in the past two weeks? And why does the “PR of the Week” jingle go so hard? Find out in this week’s episode!

# Timestamps

(00:00:00)  INTRO

(00:01:12)  IMPORTS

(00:02:21)  Here be dragons

(00:02:42)  High level summary

(00:05:12)  Finders and Loaders

(00:06:29)  Loading encrypted modules

(00:08:13)  Debugging by altering someone else's code

(00:10:08)  Support for site-packages

(00:14:22)  Nefarious uses of imports

(00:17:28)  Tagged strings?

(00:21:00)  FROZEN MODULES

(00:23:38)  Deepfreeze origins: importlib

(00:26:01)  Deepfreeze evolution

(00:31:32)  First world problems with build speed

(00:33:05)  WHAT'S GOING ON IN CPYTHON

(00:34:16)  Biased reference counts landed, buildbots

(00:35:51)  There is no nogil, there is only free-threaded

(00:36:54)  mimalloc landed

(00:42:29)  Critical sections landed

(00:45:04)  Tuples can now be shared across subinterpreters

(00:48:31)  JIT beginnings: Tier 2 interpreter merged with Tier 1

(00:50:54)  perf support with DWARF: look ma, no frame pointers

(00:55:05)  Yo DAWG, I herd you liked codepoints

(00:58:46)  Thread.join() waits for the OS thread now

(01:01:04)  PR OF THE WEEK

(01:02:06)  Pablo: gh-109181 -- traceback object creation is now lazy = much faster

(01:05:25)  Łukasz: gh-111710 -- stdlib coverage now gathered with sys.monitoring

(01:10:20)  WRAP UP

We've read the PEP on making the Global Interpreter Lock optional so you don't have to.

Timestamps

(00:00:00) Intro

(00:00:50) CURRENT STATE OF THINGS

(00:00:58) Reference counting

(00:01:35) Garbage collection

(00:02:33) What is the Global Interpreter Lock?

(00:03:57) The GIL and threading

(00:07:24) Current ways around the GIL

(00:09:26) HISTORICAL ATTEMPTS TO REMOVE THE GIL

(00:09:46) 1999: Greg Stein's attempt at Python 1.6

(00:10:46) Jython doesn't have the GIL

(00:11:19) 2015: Larry Hastings' Gilectomy at Python 3.5

(00:12:34) Pablo says removing the GIL is actually very easy

(00:13:24) Łukasz is skeptical

(00:14:20) 2021: Sam Gross' nogil at Python 3.9

(00:18:22) 2023: PEP 703 for Python 3.13

(00:20:05) PEP 703 IN DETAIL

(00:20:20) Biased Reference Counting

(00:25:19) Other needed speedups: deferred refcounts, immortalization, GC

(00:28:41) mimalloc

(00:34:46) More GC changes

(00:38:20) eval breaker

(00:42:08) Thread-safe standard collections

(00:45:26) Fast paths vs. slow paths

(00:49:47) Reading freed memory with mimalloc is kinda okay?

(00:58:06) Specializations become harder to implement without the GIL

(01:01:27) PEP 703 terms of acceptance

(01:04:08) No free lunch

(01:09:25) It's now or never

(01:11:59) Outro