The Python Podcast.__init__

The Journey To Replace Python's Parser And What It Means For The Future


Listen Later

Summary

The release of Python 3.9 introduced a new parser that paves the way for brand new features. Every programming language has its own specific syntax for representing the logic that you are trying to express. The way that the rules of the language are defined and validated is with a grammar definition, which in turn is processed by a parser. The parser that the Python language has relied on for the past 25 years has begun to show its age through mounting technical debt and a lack of flexibility in defining new syntax. In this episode Pablo Galindo and Lysandros Nikolaou explain how, together with Python’s creator Guido van Rossum, they replaced the original parser implementation with one that is more flexible and maintainable, why now was the time to make the change, and how it will influence the future evolution of the language.

Announcements
  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app or want to try a project you hear about on the show, you’ll need somewhere to deploy it, so take a look at our friends over at Linode. With the launch of their managed Kubernetes platform it’s easy to get started with the next generation of deployment and scaling, powered by the battle tested Linode platform, including simple pricing, node balancers, 40Gbit networking, dedicated CPU and GPU instances, and worldwide data centers. Go to pythonpodcast.com/linode and get a $60 credit to try out a Kubernetes cluster of your own. And don’t forget to thank them for their continued support of this show!
  • You listen to this show to learn and stay up to date with the ways that Python is being used, including the latest in machine learning and data analysis. For more opportunities to stay up to date, gain new skills, and learn from your peers there are a growing number of virtual events that you can attend from the comfort and safety of your home. Go to pythonpodcast.com/conferences to check out the upcoming events being offered by our partners and get registered today!
  • Your host as usual is Tobias Macey and today I’m interviewing Pablo Galindo and Lysandros Nikolaou about their work on replacing the parser in CPython and what that means for the language
  • Interview
    • Introductions
    • How did you get introduced to Python?
    • Can you start by discussing the role of the parser in the lifecycle of a Python program?
    • What were the limitations of the previous parser, and how did that contribute to complexity and technical debt in the CPython runtime?
    • What are the options for styles of parsers, and what are the benefits of using a PEG style grammar?
    • How does the new parser impact the approachability of the CPython code for new contributors?
    • What was the process for reimplementing the parser and guarding against regressions in the syntax?
    • As developers switch to the 3.9 release, what potential edge cases/bugs might they see from introducing the new parser?
    • What new syntax options does this parser provide for the Python language?
      • Are there any specific features that are planned for implementation in the 3.10 release that are enabled by the new parser grammar?
      • As the language evolves due to new capabilities offered by the updated parser, how will that impact other implementations such as PyPy?
      • What were the most interesting, unexpected, or challenging aspects of this project?
      • What other aspects of the CPython code do you think should be reconsidered or reimplemented in light of the changes in computing and the usage of the language?
      • Keep In Touch
        • Pablo
          • pablogsal on GitHub
          • @pyblogsal on Twitter
          • LinkedIn
          • Lysandros
            • LinkedIn
            • lysnikolaou on GitHub
            • @lysnikolaou on Twitter
            • Picks
              • Tobias
                • Annual Python Developer Survey
                • Jessica Jones TV show
                • Pablo
                  • Raised By Wolves TV Series
                  • Lysandros
                    • Afterlife TV show
                    • Links
                      • PEP 617 – New PEG Parser for CPython
                      • Podcast Episode About Parsers
                      • CPython
                      • Bloomberg
                      • PEG Parsers
                      • Seafair
                      • LL(1) Parsers
                      • Łukasz Langa
                      • Parser Generator
                      • Concrete Syntax Tree
                      • Abstract Syntax Tree
                      • PyPy
                      • RustPython
                        • Podcast Episode
                        • IronPython
                        • Structural Pattern Matching – PEP 622
                        • Pylint
                        • ASTroid
                          • Podcast Episode
                          • Hy
                            • Podcast Episode
                            • Walrus Operator/Assignment Expressions
                            • C99
                            • Reference Counting
                            • Cycle Hunting/Generational Garbage Collection
                            • The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

                              ...more
                              View all episodesView all episodes
                              Download on the App Store

                              The Python Podcast.__init__By Tobias Macey

                              • 4.4
                              • 4.4
                              • 4.4
                              • 4.4
                              • 4.4

                              4.4

                              100 ratings


                              More shows like The Python Podcast.__init__

                              View all
                              TED Talks Daily by TED

                              TED Talks Daily

                              11,267 Listeners

                              6 Minute English by BBC Radio

                              6 Minute English

                              1,777 Listeners

                              The Changelog: Software Development, Open Source by Changelog Media

                              The Changelog: Software Development, Open Source

                              285 Listeners

                              Data Skeptic by Kyle Polich

                              Data Skeptic

                              474 Listeners

                              Talk Python To Me by Michael Kennedy

                              Talk Python To Me

                              584 Listeners

                              Software Engineering Daily by Software Engineering Daily

                              Software Engineering Daily

                              630 Listeners

                              The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) by Sam Charrington

                              The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)

                              429 Listeners

                              Super Data Science: ML & AI Podcast with Jon Krohn by Jon Krohn

                              Super Data Science: ML & AI Podcast with Jon Krohn

                              295 Listeners

                              Python Bytes by Michael Kennedy and Brian Okken

                              Python Bytes

                              212 Listeners

                              Syntax - Tasty Web Development Treats by Wes Bos & Scott Tolinski - Full Stack JavaScript Web Developers

                              Syntax - Tasty Web Development Treats

                              985 Listeners

                              DataFramed by DataCamp

                              DataFramed

                              267 Listeners

                              Practical AI by Practical AI LLC

                              Practical AI

                              196 Listeners

                              The Real Python Podcast by Real Python

                              The Real Python Podcast

                              137 Listeners

                              Last Week in AI by Skynet Today

                              Last Week in AI

                              275 Listeners

                              Latent Space: The AI Engineer Podcast by swyx + Alessio

                              Latent Space: The AI Engineer Podcast

                              64 Listeners