The Real Python Podcast

The Real Python Podcast

By Real PythonTechnology
Download on the App Store

The Real Python Podcast episodes

  • Leveraging Documents and Data to Create a Custom LLM Chatbot

    How do you customize a LLM chatbot to address a collection of documents and data? What tools and techniques can you use to build embeddings into a vector database? This week on the show, Calvin Hendryx-Parker is back to discuss developing an AI-powered, Large Language Model-driven chat interface.

    Calvin is the co-founder and CTO of Six Feet Up, a Python and AI consultancy. He shares a recent project for a family-owned seed company that wanted to build a tool for customers to access years of farm research. These documents were stored as brochure-style PDFs and spanned 50 years.

    We discuss several of the tools used to augment a LLM. Calvin covers working with LangChain and vectorizing data with ChromaDB. We talk about the obstacles and limitations of capturing documentation.

    Calvin also shares a smaller project that you can try out yourself. It takes the information from a conference website and creates a chatbot using Django and Python prompt-toolkit.

    This episode is sponsored by Mailtrap.

    Course Spotlight: Command Line Interfaces in Python

    Command line arguments are the key to converting your programs into useful and enticing tools that are ready to be used in the terminal of your operating system. In this course, you’ll learn their origins, standards, and basics, and how to implement them in your program.

    Topics:

    • 00:00:00 – Introduction
    • 00:02:21 – Background on the project
    • 00:03:51 – Complexity of adding documents
    • 00:09:01 – Retrieval-augmented generation and providing links
    • 00:13:46 – Updating information and larger conversation context
    • 00:18:08 – Sponsor: Mailtrap
    • 00:18:43 – Working with context
    • 00:21:02 – Temperature adjustment
    • 00:22:07 – Rally Conference Chatbot Project
    • 00:26:20 – Vectorization using ChromaDB
    • 00:32:49 – Employing Python prompt-toolkit
    • 00:35:07 – Learning libraries on the fly
    • 00:37:38 – Video Course Spotlight
    • 00:39:00 – Problems with tables in documents
    • 00:42:30 – Everything looks like a chat box
    • 00:44:26 – Finding the right fit for a client and customer
    • 00:49:05 – What are questions you ask a new client now?
    • 00:51:54 – Canada Air anecdote
    • 00:56:20 – How do you stay up to date on these topics?
    • 01:01:03 – What are you excited about in the world of Python?
    • 01:03:22 – What do you want to learn next?
    • 01:04:58 – How can people follow your work online?
    • 01:05:31 – IndyPy
    • 01:07:13 – Thanks and goodbye
    • Show Links:

      • Transforming Agricultural Data with AI — Six Feet Up
      • Build ChatGPT-like Apps with AI — Six Feet Up
      • Innovate with AI: Build ChatGPT-like Apps - YouTube
      • What is retrieval-augmented generation? - IBM Research Blog
      • rally-llm-presentation - sixfeetup - GitHub
      • Python Prompt Toolkit 3.0 — Documentation
      • Chroma - the AI-native open-source embedding database
      • Embeddings and Vector Databases With ChromaDB – Real Python
      • LangChain
      • Build an LLM RAG Chatbot With LangChain – Real Python
      • Air Canada must pay after chatbot lies to grieving passenger - The Register
      • I’d Buy That for a Dollar: Chevy Dealership’s AI Chatbot Goes Rogue
      • Omnivore
      • TLDR AI - Get smarter about AI in 5 minutes
      • Tech Brew
      • Simon Willison’s Weblog
      • llm: Access large language models from the command-line - simonw - GitHub
      • PyCon US 2024
      • Syntorial: The Ultimate Synthesizer Tutorial
      • Blog — Six Feet Up
      • Calvin Hendryx-Parker - LinkedIn
      • Eclipse Insights: How AI is Transforming Solar Astronomy - YouTube
      • Level up your Python skills with our expert-led courses:

        • How to Work With a PDF in Python
        • Command Line Interfaces in Python
        • Sneaky REST APIs With Django Ninja
        • Support the podcast & join our community of Pythonistas

          1 hr 9 min
        • Build a Video Game With Python Turtle & Visualize Data in Seaborn

          Can you build a Space Invaders clone using Python’s built-in turtle module? What advantages does the Seaborn data visualization library provide compared to Matplotlib? Christopher Trudeau is back on the show this week, along with special guest Real Python core team member Bartosz Zaczyński. We’re sharing another batch of PyCoder’s Weekly articles and projects.

          Bartosz shares a Real Python step-by-step project for building a video game using the Python turtle module. The turtle module provides an interactive environment that lets users create pictures and shapes on a virtual canvas. The project takes you through game design concepts such as animating sprites, detecting a collision, and building a game loop.

          We discuss another Real Python resource, “Visualizing Data in Python With Seaborn.” Seaborn is a significant next step if you’ve already been working with Matplotlib. It produces impressive visualizations and offers a functional or object-based approach to designing plots.

          We also share several other articles and projects from the Python community, including a news roundup, finding Python Easter eggs, exploring whether Python has pointers, styling Excel cells with OpenPyXL, using weird tests to capture tacit knowledge, inspecting and running Django commands in a TUI, building reactive web UIs in Python, and a project for predictable Python datetimes.

          This week’s episode is brought to you by Posit.

          Course Spotlight: Python Turtle for Beginners

          In this step-by-step course, you’ll learn the basics of Python programming with the help of a simple and interactive Python library called turtle. If you’re a beginner to Python, then this tutorial will definitely help you on your journey as you take your first steps into the world of programming.

          Topics:

          • 00:00:00 – Introduction
          • 00:03:06 – Python 3.13.0 alpha 5 is now available
          • 00:03:38 – Allow disabling the GIL with flags
          • 00:04:17 – Django security releases issued: 5.0.3, 4.2.11, and 3.2.25
          • 00:04:32 – The Python Coding Book
          • 00:05:03 – Finding Python Easter Eggs – Code Conversation
          • 00:12:48 – Sponsor: Posit
          • 00:13:34 – Visualizing Data in Python With Seaborn
          • 00:18:18 – Does Python have pointers?
          • 00:21:43 – Build a Python Turtle Game: Space Invaders Clone
          • 00:30:42 – Video Course Spotlight
          • 00:32:04 – Styling Excel Cells with OpenPyXL and Python
          • 00:35:22 – Use weird tests to capture tacit knowledge
          • 00:37:09 – whenever: Strict, predictable, and typed datetimes
          • 00:42:25 – hyperdiv: Build Reactive Web UIs in Python
          • 00:46:19 – django-tui: Inspect and run Django Commands in a TUI
          • 00:48:40 – Thanks and goodbye
          • News:

            • Python Insider: Python 3.13.0 alpha 5 is now available
            • Allow disabling the GIL with Flags - cpython - GitHub
            • Django security releases issued: 5.0.3, 4.2.11, and 3.2.25 - Weblog - Django
            • The Python Coding Book – The Python Coding Place
            • Show Links:

              • Finding Python Easter Eggs – Code Conversation - Video Course – Python has its fair share of hidden surprises, commonly known as Easter eggs. From clever jokes to secret messages, these little mysteries are often meant to be discovered by curious developers like you!
              • Visualizing Data in Python With Seaborn – In this tutorial, you’ll learn how to use the Python seaborn library to produce statistical data analysis plots to allow you to better visualize your data. You’ll learn how to use both its traditional classic interface and more modern objects interface.
              • Does Python have pointers? - Ned Batchelder – Depending on how you’re using the term “pointer” changes the answer to the question. Read on to better understand the programming terminology and whether Python has pointers.
              • Build a Python Turtle Game: Space Invaders Clone – In this step-by-step tutorial, you’ll use Python’s turtle module to write a Space Invaders clone. You’ll learn about techniques used in animations and games, and consolidate your knowledge of key Python topics.
              • Styling Excel Cells with OpenPyXL and Python - Many Python libraries that deal with Excel only handle data, but OpenPyXL gives you the ability to style your cells in many different ways. Learn how to give your spreadsheets pizazz!
              • Use weird tests to capture tacit knowledge - Applied Cartography – Sometimes adding code in one place means configuration elsewhere also needs to be updated. One way of ensuring this is happening properly in a large project is to use unit tests. This post covers a few examples, complete with pytest code.
              • Projects:

                • whenever: ⏰ Strict, predictable, and typed datetimes
                • hyperdiv: Build Reactive Web UIs in Python
                • django-tui: Inspect and run Django Commands in a text-based user interface (TUI)
                • Additional Links:

                  • What’s the Zen of Python? – Real Python
                  • Using Python for Data Analysis – Real Python
                  • The Beginner’s Guide to Python Turtle – Real Python
                  • Roamer - The History of Turtle Robots
                  • Tutorial — openpyxl 3.1.2 documentation
                  • Ten Python datetime pitfalls, and what libraries are (not) doing about it - Arie Bovenberg
                  • The science behind why people hate Daylight Saving Time so much - Ars Technica
                  • Textual
                  • Level up your Python skills with our expert-led courses:

                    • Histogram Plotting in Python: NumPy, Matplotlib, Pandas & Seaborn
                    • Pointers and Objects in Python
                    • Python Turtle for Beginners
                    • Support the podcast & join our community of Pythonistas

                      50 min
                    • Using Python in Bioinformatics and the Laboratory

                      How is Python being used to automate processes in the laboratory? How can it speed up scientific work with DNA sequencing? This week on the show, Chemical Engineering PhD Student Parsa Ghadermazi is here to discuss Python in bioinformatics.

                      Parsa provides background on his research and the bioinformatic techniques used to discover gut microbes’ role in human health and diseases. We talk about automating lab experiments with liquid handling robots and Python.

                      We dig into libraries to shatter and reassemble DNA sequences. Parsa also shares current projects from the Chan Lab at Colorado State University and his GitHub repository.

                      Course Spotlight: Building Python Project Documentation With MkDocs

                      In this video course, you’ll learn how to build professional documentation for a Python package using MkDocs and mkdocstrings. These tools allow you to generate nice-looking and modern documentation from Markdown files and, more importantly, from your code’s docstrings.

                      Topics:

                      • 00:00:00 – Introduction
                      • 00:01:51 – Engineering Background and Current PhD Program
                      • 00:05:52 – What is Bioinformatics?
                      • 00:08:11 – Where do you use Python in the lab?
                      • 00:10:35 – Using lab robotics
                      • 00:15:22 – Python development environment
                      • 00:16:33 – Lab robotics allow for precision
                      • 00:19:03 – How are using these tools for research?
                      • 00:22:14 – What are the techniques for measurements?
                      • 00:26:17 – Video Course Spotlight
                      • 00:27:33 – How is the data output from the machine?
                      • 00:29:20 – Moving into DNA sequencing and extraction
                      • 00:32:08 – Shattering to work with smaller DNA sequences
                      • 00:34:34 – Python libraries for DNA re-assembly
                      • 00:36:28 – Building ADToolbox
                      • 00:40:24 – How do you store the data?
                      • 00:41:32 – Inferring microbial interactions
                      • 00:44:02 – Types of hardware used for these projects
                      • 00:47:07 – What are you excited about in the world of Python?
                      • 00:48:09 – What do you want to learn next?
                      • 00:49:16 – How can people follow your work online?
                      • 00:50:00 – Thanks and goodbye
                      • Show Links:

                        • ParsaGhadermazi - GitHub
                        • Bioinformatics - Wikipedia
                        • Opentrons - Lab Automation - Lab Robots for Life Scientists
                        • Tutorial — Opentrons Python API V2 Documentation
                        • Serial Dilutions and Plating: Microbial Enumeration - Microbiology - JoVE
                        • What is Chromatogram & How to Read a Chromatogram?
                        • Episode #186: Exploring Python in Excel
                        • megahit: Ultra-fast and memory-efficient (meta-)genome assembler
                        • DRAM: Distilled and Refined Annotation of Metabolism - GitHub
                        • humann: HUMAnN 3.0 - HMP Unified Metabolic Analysis Network
                        • MetaPhlAn4 – The Huttenhower Lab
                        • ADToolbox - Tools for modeling and optimizing the anaerobic digestion process
                        • SPAM-DFBA - Algoritm for inferring microbial interactions
                        • Microbial interactions from a new perspective - Bioinformatics - Oxford Academic
                        • Alpine — Research Computing University of Colorado Boulder documentation
                        • MkDocs
                        • Build Your Python Project Documentation With MkDocs
                        • scikit-bio
                        • Episode #190: Great Starting Points for Contributing to Open Source
                        • Chan Lab at Colorado State University - GitHub
                        • ParsaGhadermazi - GitHub
                        • Parsa Ghadermazi - LinkedIn
                        • Level up your Python skills with our expert-led courses:

                          • Reading and Writing CSV Files
                          • Data Cleaning With pandas and NumPy
                          • Building Python Project Documentation With MkDocs
                          • Support the podcast & join our community of Pythonistas

                            51 min
                          • Exploring Duck Typing in Python & Dynamics of Monkey Patching

                            What are the advantages of determining the type of an object by how it behaves? What coding circumstances are not a good fit for duck typing? Christopher Trudeau is back on the show this week, bringing another batch of PyCoder’s Weekly articles and projects.

                            Christopher covers a recent Real Python tutorial by Leodanis Pozo Ramos titled Duck Typing in Python: Writing Flexible and Decoupled Code. The tutorial explains the concepts of duck typing within object-oriented programming and its use within Python’s built-in tools.

                            We discuss a recent article on monkey patching in Python. This practice of dynamically modifying a class or module’s behavior at runtime allows for testing, debugging, and experimentation.

                            We also share several other articles and projects from the Python community, including a news roundup, why names are not the same as objects in Python, using IPython Jupyter magic commands, a discussion about becoming a senior developer, a data exploration challenge, a Python evaluation game, and a terminal UI for regex testing.

                            This week’s episode is brought to you by Sentry.

                            Course Spotlight: Pointers and Objects in Python

                            In this video course, you’ll learn about Python’s object model and see why pointers don’t really exist in Python. You’ll also cover ways to simulate pointers in Python without managing memory.

                            Topics:

                            • 00:00:00 – Introduction
                            • 00:02:38 – Listener feedback
                            • 00:04:02 – DjangoCon US Call for Proposals
                            • 00:04:38 – White House Recommends Use of Python
                            • 00:05:41 – JupyterLab 4.1 and Notebook 7.1 Released
                            • 00:06:05 – What’s in a Name?
                            • 00:11:52 – Duck Typing in Python: Writing Flexible and Decoupled Code
                            • 00:15:07 – Sponsor: Sentry
                            • 00:16:11 – Using IPython Jupyter Magic Commands
                            • 00:22:31 – A Guide to Monkey Patching
                            • 00:25:27 – Falsehoods Junior Developers Believe About Becoming Senior
                            • 00:33:01 – Video Course Spotlight
                            • 00:34:11 – Falsehoods continued
                            • 00:43:43 – Where in the data?
                            • 00:46:55 – the eval game
                            • 00:48:03 – rexi: Terminal UI for Regex Testing
                            • 00:49:43 – Thanks and goodbye
                            • News:

                              • DjangoCon US Call for Proposals
                              • White House Recommends Use of Python
                              • JupyterLab 4.1 and Notebook 7.1 Released
                              • Show Links:

                                • What’s in a Name? – An article about names in Python, and why they’re not the same as objects. The article discusses reference counts and namespaces.
                                • Duck Typing in Python: Writing Flexible and Decoupled Code – In this tutorial, you’ll learn about duck typing in Python. It’s a typing system based on objects’ behaviors rather than on inheritance. By taking advantage of duck typing, you can create flexible and decoupled sets of Python classes that you can use together or individually.
                                • Using IPython Jupyter Magic Commands – “IPython Jupyter Magic commands (e.g. lines in notebook cells starting with % or %%) can decorate a notebook cell, or line, to modify its behavior.” This article shows you how to define them and where they can be useful.
                                • Monkeying Around With Python: A Guide to Monkey Patching – Monkey patching is the practice of modifying live code. This article shows you how it’s done and why and when to use the practice.
                                • Discussion:

                                  • Falsehoods Junior Developers Believe About Becoming Senior – This opinion piece by Vadim discusses how newer developers perceive what it means to be a senior developer, and how they’re often wrong.
                                  • Projects:

                                    • Where in the data?
                                    • the eval game
                                    • rexi: Terminal UI for Regex Testing
                                    • Additional Links:

                                      • Pointers in Python: What’s the Point? – Real Python
                                      • Unlock IPython’s Magical Toolbox for Your Coding Journey – Real Python
                                      • Episode #88: Discussing Type Hints, Protocols, and Ducks in Python – The Real Python Podcast
                                      • saul.pw
                                      • BlueBird Shell
                                      • Level up your Python skills with our expert-led courses:

                                        • Python Type Checking
                                        • Pointers and Objects in Python
                                        • Testing Your Code With pytest
                                        • Support the podcast & join our community of Pythonistas

                                          51 min
                                        • Building a Healthy Developer Mindset While Learning Python

                                          How do you get yourself unstuck when facing a programming problem? How do you develop a positive developer mindset while learning Python? This week on the show, Bob Belderbos from Pybites is here to talk about learning Python and building healthy developer habits.

                                          Bob created the Pybites learning platform with his friend Julian Sequeira. They initially focused on exercises and coding challenges to motivate new Python students. As they grew their community, they created a podcast and moved into coaching.

                                          They noticed that most new developers share common struggles of tutorial paralysis, imposter syndrome, and motivation. Bob discusses techniques for developing a positive mindset, overcoming coding blocks, and delivering projects.

                                          Course Spotlight: Grow Your Python Portfolio With 13 Intermediate Project Ideas

                                          Get started on 13 Python project ideas that are just right for intermediate Python developers. They’ll challenge you enough to help you become a better Pythonista.

                                          Topics:

                                          • 0:00:00 – Introduction
                                          • 0:02:06 – How did you start Pybites?
                                          • 0:04:32 – Building a community through challenges
                                          • 0:06:17 – When did you start your podcast?
                                          • 0:08:22 – Defining developer mindset
                                          • 0:11:31 – Learning Python outside of a classroom
                                          • 0:16:15 – Podcast is a good place to discuss mindset
                                          • 0:19:37 – Video Course Spotlight
                                          • 0:20:56 – Sharing Python tips
                                          • 0:30:14 – Sharing content as a creator
                                          • 0:35:03 – Writing cleaner code
                                          • 0:40:20 – Moving from challenges to projects
                                          • 0:47:08 – Helping yourself when you’re stuck
                                          • 0:51:31 – Dealing with imposter syndrome
                                          • 0:55:50 – What are you excited about in the world of Python?
                                          • 0:57:43 – What do you want to learn next?
                                          • 0:59:15 – How can people follow your work online?
                                          • 0:59:51 – Thanks and goodbye
                                          • Show Links:

                                            • Pybites - We Create Python Developers
                                            • Pybites Podcast
                                            • PyBites Platform - Real World Python Exercises
                                            • Pybites Community
                                            • Pybites Python Tips Book - Pybites
                                            • itertools — Functions creating iterators for efficient looping - Python documentation
                                            • string — Common string operations - Python documentation
                                            • Python enumerate(): Simplify Loops That Need Counters – Real Python
                                            • ast — Abstract Syntax Trees - Python documentation
                                            • Building Maintainable Software, Java Edition
                                            • Refactoring - Improving the Design of Existing Code by Martin Fowler
                                            • A Mind For Numbers - Barbara Oakley
                                            • Real Imposters Don’t Experience Imposter Syndrome
                                            • Rust Programming Language
                                            • Bob Belderbos (@bbelderbos) - X
                                            • Bob Belderbos - LinkedIn
                                            • Level up your Python skills with our expert-led courses:

                                              • Grow Your Python Portfolio With 13 Intermediate Project Ideas
                                              • Python Basics: Code Your First Python Program
                                              • Building Python Project Documentation With MkDocs
                                              • Support the podcast & join our community of Pythonistas

                                                1 hr 1 min
                                              • Automate Tasks With Python & Building a Small Search Engine

                                                What are the typical computer tasks you do manually every week? Could you automate those tasks with a Python script? Christopher Trudeau is back on the show this week, bringing another batch of PyCoder’s Weekly articles and projects.

                                                We discuss a recent Hacker News thread about frequently used automation scripts. We share the kinds of tasks we’ve automated with Python in our work and personal lives.

                                                Christopher shares a tutorial about building a micro-search engine from scratch using Python. The post takes you through coding the components of a crawler, index, and ranker. The finished engine is designed to search the posts of the blogs you follow.

                                                We also share several other articles and projects from the Python community, including a news roundup, how a Polars query works under the hood, using Python for data analysis, understanding open-source licensing, summarizing the significant changes between Python versions, a robust TUI hex editor, and a lightweight dataframe library with a universal interface for data wrangling.

                                                This week’s episode is brought to you by Intel.

                                                Course Spotlight: Building Command Line Interfaces With argparse

                                                In this step-by-step Python video course, you’ll learn how to take your command line Python scripts to the next level by adding a convenient command line interface that you can write with argparse.

                                                Topics:

                                                • 00:00:00 – Introduction
                                                • 00:02:23 – uv: Python Packaging in Rust
                                                • 00:02:43 – Rye Grows With uv
                                                • 00:03:20 – Python 3.13.0 Alpha 4 Is Now Available
                                                • 00:03:45 – A Bird’s Eye View of Polars
                                                • 00:07:28 – Polars: Why We Have Rewritten the String Data Type
                                                • 00:09:33 – A Search Engine in 80 Lines of Python
                                                • 00:13:14 – Using Python for Data Analysis
                                                • 00:18:22 – Sponsor: Intel
                                                • 00:18:53 – Understanding Open Source Licensing
                                                • 00:21:54 – Summary of Major Changes Between Python Versions
                                                • 00:23:19 – What Python automation scripts do you reuse frequently at work?
                                                • 00:34:21 – Video Course Spotlight
                                                • 00:35:52 – hexabyte: A modern, modular, and robust TUI hex editor
                                                • 00:39:56 – ibis: The Flexibility of Python With the Scale of Modern SQL
                                                • 00:43:31 – Thanks and goodbye
                                                • News:

                                                  • uv: Python Packaging in Rust – uv is an extremely fast Python package installer and resolver, designed as a drop-in alternative to pip and pip-tools. This post introduces you to uv and shows some of its performance numbers.
                                                  • Rye Grows With uv - Armin Ronacher’s Thoughts and Writings
                                                  • Python 3.13.0 Alpha 4 Is Now Available
                                                  • Show Links:

                                                    • A Bird’s Eye View of Polars – This post on the Polars blog introduces you to how Polars works, showing the steps from queries, plans, optimizations, and then the final execution.
                                                    • Polars: Why We Have Rewritten the String Data Type – A large refactor on the string data type is underway in Polars. This deep dive explains why and what is changing.
                                                    • A Search Engine in 80 Lines of Python – In this post Alex explains how he built a micro-search engine from scratch using Python. The resulting search engine is used to search in the posts of the blogs he follows.
                                                    • Using Python for Data Analysis – In this tutorial, you’ll learn the importance of having a structured data analysis workflow, and you’ll get the opportunity to practice using Python for data analysis while following a common workflow process.
                                                    • Understanding Open Source Licensing – This article discusses the importance of open-source licensing in software development and its implications for stakeholders.
                                                    • Summary of Major Changes Between Python Versions – This article is a quick reference covering the major changes introduced with each new version of Python. Can’t remember when the walrus operator was introduced? This is the place to look that up.
                                                    • Discussion:

                                                      • What Python automation scripts do you reuse frequently at work? - Hacker News
                                                      • Projects

                                                        • hexabyte: A modern, modular, and robust TUI hex editor
                                                        • ibis: The Flexibility of Python With the Scale of Modern SQL
                                                        • Additional Links:

                                                          • Episode #193: Wes McKinney on Improving the Data Stack & Composable Systems – The Real Python Podcast
                                                          • Open Source Licensing: Software Freedom and Intellectual Property Law
                                                          • What’s in which Python - Ned Batchelder
                                                          • Automate the Boring Stuff with Python - Al Sweigart
                                                          • Working With Files in Python – Real Python
                                                          • Build Command-Line Interfaces With Python’s argparse – Real Python
                                                          • Level up your Python skills with our expert-led courses:

                                                            • Practical Recipes for Working With Files in Python
                                                            • Defining Python Functions With Optional Arguments
                                                            • Building Command Line Interfaces With argparse
                                                            • Support the podcast & join our community of Pythonistas

                                                              45 min
                                                            • Wes McKinney on Improving the Data Stack & Composable Systems

                                                              How do you avoid the bottlenecks of data processing systems? Is it possible to build tools that decouple storage and computation? This week on the show, creator of the pandas library Wes McKinney is here to discuss Apache Arrow, composable data systems, and community collaboration.

                                                              Wes briefly describes the humble beginnings of the pandas project in 2008 and moving the project to open source in 2011. Since then, he’s been thinking about improvements across the data processing ecosystem.

                                                              Wes collaborated with members of the broader data science community to build the in-memory analytics infrastructure of Apache Arrow. Arrow avoids the bottlenecks of repeated data serialization and format conversion. He shares examples of Arrow’s use across the spectrum in tools like Polars and DuckDB.

                                                              Wes advocates moving from vertically integrated tools toward composable data systems. We discuss his work on Ibis, a portable dataframe API for data manipulation and exploration in Python. Ibis supports multiple backends by decoupling the API from the execution engine.

                                                              This week’s episode is brought to you by Posit Connect.

                                                              Course Spotlight: Unleashing the Power of the Console With Rich

                                                              Rich is a powerful library for creating text-based user interfaces (TUIs) in Python. It enhances code readability by pretty-printing complex data structures and adds visual appeal with colored text, tables, animations, and more.

                                                              Topics:

                                                              • 00:00:00 – Introduction
                                                              • 00:02:26 – Dealing with limitations in early data science
                                                              • 00:04:53 – Making pandas open source
                                                              • 00:07:10 – Making changes to an existing platform
                                                              • 00:12:34 – Decoupling storage and computation
                                                              • 00:23:04 – Sponsor: Posit Connect
                                                              • 00:23:54 – Apache Arrow solving multiple issues
                                                              • 00:27:40 – DuckDB efficient analytic SQL database
                                                              • 00:30:24 – Polars dataframe library
                                                              • 00:31:04 – pandas 2.0 adding Arrow
                                                              • 00:35:56 – Video Course Spotlight
                                                              • 00:37:20 – Apache Software Foundation background
                                                              • 00:41:29 – Shifting from developer to organizer and collaborator
                                                              • 00:45:56 – Creating a portable query layer with Ibis
                                                              • 00:55:34 – Casualties of the language wars
                                                              • 00:57:57 – What’s your role at Posit?
                                                              • 01:01:23 – What are you excited about in the world of Python?
                                                              • 01:04:52 – What do you want to learn next?
                                                              • 01:06:21 – How can people follow your work online?
                                                              • 01:08:20 – Thanks and goodbye
                                                              • Show Links:

                                                                • Wes McKinney - Personal Website
                                                                • Wes McKinney - The Road to Composable Data Systems: Thoughts on the Last 15 Years and the Future
                                                                • Wes McKinney - Leveling Up the Data Stack: Thoughts on the Last 15 Years - YouTube
                                                                • Apache Hadoop
                                                                • Cloudera - The hybrid data company
                                                                • Wes McKinney - Apache Arrow and the “10 Things I Hate About pandas”
                                                                • Voltron Data - The Leading Designer and Builder of Enterprise Data Systems
                                                                • Apache Arrow
                                                                • DuckDB - An in-process SQL OLAP database management system
                                                                • DuckDB-Wasm - Efficient Analytical SQL in the Browser
                                                                • Polars - Dataframes for the new era
                                                                • pandas 2.2.0 documentation
                                                                • Episode #167: Exploring pandas 2.0 & Targets for Apache Arrow – The Real Python Podcast
                                                                • ASF - Welcome to The Apache Software Foundation!
                                                                • Ursa Labs Blog
                                                                • Ibis - The Portable Python dataframe Library
                                                                • Python dataframe interchange protocol
                                                                • Hadley Wickham
                                                                • Rust Programming Language
                                                                • italki - Best language learning app with certificated tutors
                                                                • Wes McKinney - LinkedIn
                                                                • Wes McKinney (@wesmckinn) - X
                                                                • Posit - The Open-Source Data Science Company
                                                                • Level up your Python skills with our expert-led courses:

                                                                  • The pandas DataFrame: Working With Data Efficiently
                                                                  • Data Cleaning With pandas and NumPy
                                                                  • Unleashing the Power of the Console With Rich
                                                                  • Support the podcast & join our community of Pythonistas

                                                                    1 hr 10 min
                                                                  • Practical Python Decorator Uses & Avoiding datetime Pitfalls

                                                                    What are real-life examples of using Python decorators? How can you harness their power in your code? Christopher Trudeau is back on the show this week, bringing another batch of PyCoder’s Weekly articles and projects.

                                                                    We discuss a recent article series that digs into Python decorators. The first two articles discuss the basics of constructing decorators. The third part describes how popular Python libraries use decorators with call interception, function registration, and enriching the behavior of a function.

                                                                    Christopher shares a piece about the common pitfalls of working with the Pythondatetime library. The article considers how current third-party libraries don’t address most of these quirks and offers a potential solution with a new library.

                                                                    We also share several other articles and projects from the Python community, including a couple of news items, a discussion about the popularity of the Rust language, handling unset values in FastAPI with Pydantic, working with Python’s mini-language for formatting strings, mocking Django queryset functions, and a modern replacement for the Requests library.

                                                                    This week’s episode is brought to you by Sentry.

                                                                    Course Spotlight: Python Decorators 101

                                                                    In this course on Python decorators, you’ll learn what they are and how to create and use them. Decorators provide a simple syntax for calling higher-order functions in Python. By definition, a decorator is a function that takes another function and extends the behavior of the latter function without explicitly modifying it.

                                                                    Topics:

                                                                    • 00:00:00 – Introduction
                                                                    • 00:02:53 – Django security releases issued: 5.0.2, 4.2.10, and 3.2.24
                                                                    • 00:03:10 – Python 3.12.2 and 3.11.8 are now available
                                                                    • 00:03:21 – Introducing PSF Grants Program Office Hours
                                                                    • 00:04:19 – Python’s Format Mini-Language for Tidy Strings
                                                                    • 00:12:22 – Ten Python datetime Pitfalls
                                                                    • 00:18:34 – Sponsor: Sentry
                                                                    • 00:19:37 – Real Life Use of Decorators
                                                                    • 00:29:18 – Handling Unset Values in FastAPI With Pydantic
                                                                    • 00:35:43 – Video Course Spotlight
                                                                    • 00:37:06 – The Python Rust-Aissance
                                                                    • 00:50:19 – django-mock-queries: Mock Django Queryset Functions
                                                                    • 00:53:09 – niquests: Requests but Multiplexed
                                                                    • 00:55:55 – Thanks and goodbye
                                                                    • News:

                                                                      • Django security releases issued: 5.0.2, 4.2.10, and 3.2.24
                                                                      • Python 3.12.2 and 3.11.8 are now available
                                                                      • Introducing PSF Grants Program Office Hours
                                                                      • Show Links:

                                                                        • Python’s Format Mini-Language for Tidy Strings – In this tutorial, you’ll learn about Python’s format mini-language. See how to use it for creating working format specifiers and build nicely formatted strings and messages in your code.
                                                                        • Ten Python datetime Pitfalls – It’s no secret that the Python datetime library has its quirks. Not only are there probably more than you think, but third-party libraries don’t address most of them! Arie created a new library to explore what a better datetime library could look like.
                                                                        • Real Life Use of Decorators – Part 3 in a series on how Python decorators are used. This part covers real-life use cases including call interception, function registration, and behavioral enrichment.
                                                                        • Handling Unset Values in FastAPI With Pydantic – When using the HTTP PATCH method only those fields that got changed are updated. Pydantic sets fields not given as arguments as None so there is no way to distinguish between an explicit None value and an unset field. This post explains how you process this scenario.
                                                                        • Discussion:

                                                                          • The Python Rust-Aissance – Companies like Polars are showing how with Rust, Python developers now have a better, smoother path towards building high-performance libraries.
                                                                          • Rye: A Python Developer Experience Vision Continued
                                                                          • PyO3: Rust bindings for the Python interpreter
                                                                          • Rust in Linux: Where we are and where we’re going next
                                                                          • RustPython: OSS CPython Written in Rust
                                                                          • granian: Rust HTTP Server for Python Applications
                                                                          • Projects:

                                                                            • django-mock-queries: Mock Django Queryset Functions for Testing
                                                                            • niquests: Requests but Multiplexed
                                                                            • Additional Links:

                                                                              • Backus–Naur form - Wikipedia
                                                                              • ANTLR - (ANother Tool for Language Recognition) A Powerful Parser Generator
                                                                              • Xmas Decoration, Part 1 - Bite code!
                                                                              • Xmas Decoration, Part 2 - Bite code!
                                                                              • Falsehoods programmers believe about time - @noahsussman - Infinite Undo
                                                                              • 10 Reasons You Should Quit Your HTTP Client - Ahmed TAHRI - Dev Genius
                                                                              • HTTPX - Fully Featured HTTP Client for Python 3
                                                                              • Level up your Python skills with our expert-led courses:

                                                                                • Python 3's F-Strings: An Improved String Formatting Syntax
                                                                                • Python Decorators 101
                                                                                • Using Python's datetime Module
                                                                                • Support the podcast & join our community of Pythonistas

                                                                                  57 min
                                                                                • Focusing on Data Science & Less on Engineering and Dependencies

                                                                                  How do you manage the dependencies of a large-scale data science project? How do you migrate that project from a laptop to cloud infrastructure or utilize GPUs and multiple instances in parallel? This week on the show, Savin Goyal returns to discuss the updates to the open-source framework Metaflow.

                                                                                  Savin briefly describes the Metaflow platform and the goal of simplifying engineering overhead for data scientists and programmers. We discuss how the platform captures snapshots of a project as you work, allowing you to go back in time or share the state of your project with another team member.

                                                                                  We dig into the complicated process of managing dependencies for machine learning and data science projects. Savin describes how the required external libraries can be specified within a flow with the new @pypi or @conda decorators. This allows a project to scale from a local machine to the cloud or multiple instances with all dependencies included.

                                                                                  He talks about starting a new company, Outerbounds, with fellow co-workers from Netflix. Their vision is to continue to build the Metaflow open-source platform and offer customers scalable enterprise-grade infrastructure.

                                                                                  This week’s episode is brought to you by Intel.

                                                                                  Course Spotlight: Everyday Project Packaging With pyproject.toml

                                                                                  In this Code Conversation video course, you’ll learn how to package your everyday projects with pyproject.toml. Playing on the same team as the import system means you can call your project from anywhere, ensure consistent imports, and have one file that’ll work for many build systems.

                                                                                  Topics:

                                                                                  • 00:00:00 – Introduction
                                                                                  • 00:02:25 – Update on Metaflow
                                                                                  • 00:04:13 – What is Outerbounds?
                                                                                  • 00:07:26 – An ML platform to serve data scientists needs
                                                                                  • 00:13:02 – Dependency reproducibility via @conda and @pypi decorators
                                                                                  • 00:26:18 – Sponsor: Intel
                                                                                  • 00:27:10 – Storing lock files along with snapshots
                                                                                  • 00:29:17 – Working alongside code and dependency management systems
                                                                                  • 00:34:03 – Scaling a project from laptop to the cloud
                                                                                  • 00:40:13 – Video Course Spotlight
                                                                                  • 00:41:41 – Getting visibility on processes
                                                                                  • 00:47:23 – Adjusting your project due to GPU availability
                                                                                  • 00:52:27 – Example of jumping back into a project one year later
                                                                                  • 00:55:54 – What are you excited about in the world of Python?
                                                                                  • 00:57:39 – What do you want to learn next?
                                                                                  • 00:59:35 – How can people follow your work online?
                                                                                  • 01:00:19 – Thanks and goodbye
                                                                                  • Show Links:

                                                                                    • Metaflow - a framework for real-life ML, AI, and data science
                                                                                    • Infrastructure for ML, AI, and Data Science - Outerbounds
                                                                                    • Human-Friendly, Production-Ready Data Science with Metaflow- Savin Goyal | SciPy 2022 - YouTube
                                                                                    • Episode #61: Scaling Data Science and Machine Learning Infrastructure Like Netflix – The Real Python Podcast
                                                                                    • New in Metaflow: The Long-Awaited @pypi Decorator - Outerbounds
                                                                                    • Managing Dependencies - Metaflow Docs
                                                                                    • Secure ML with Secure Software Dependencies - Outerbounds
                                                                                    • Directed acyclic graph (DAG) - Wikipedia article
                                                                                    • Visualizing Results - Metaflow Docs
                                                                                    • Seamless Data and ML Pipelines with Airflow and Metaflow - Outerbounds
                                                                                    • Episode #142: Orchestrating Large and Small Projects With Apache Airflow – The Real Python Podcast
                                                                                    • Savin (@SavinGoyal) - X
                                                                                    • Savin Goyal - LinkedIn
                                                                                    • Building the ML-driven future - Outerbounds Blog
                                                                                    • Level up your Python skills with our expert-led courses:

                                                                                      • Histogram Plotting in Python: NumPy, Matplotlib, Pandas & Seaborn
                                                                                      • Combining Data in pandas With concat() and merge()
                                                                                      • Everyday Project Packaging With pyproject.toml
                                                                                      • Support the podcast & join our community of Pythonistas

                                                                                        1 hr 2 min
                                                                                      • Great Starting Points for Contributing to Open Source

                                                                                        What’s it like to sit down for your first developer sprint at a conference? How do you find an appropriate issue to work on as a new open-source contributor? This week on the show, author and software engineer Stefanie Molin is here to discuss starting to contribute to open-source projects.

                                                                                        Stefanie is a data scientist and software engineer on Bloomberg’s Security Data Science team. She recently wrote an article titled “5 Ways to Get Started in Open Source.” We discuss finding ways to contribute that fit your interests and developer skills. We dig into the experience of participating in community sprints at a conference.

                                                                                        Stefanie is the author of Hands-On Data Analysis with Pandas. We also discuss the different processes between writing technical articles and authoring a book.

                                                                                        This week’s episode is brought to you by Intel.

                                                                                        Course Spotlight: Documenting Python Projects With Sphinx and Read the Docs

                                                                                        In this video series, you’ll create project documentation from scratch using Sphinx, the de facto standard for Python. You’ll also hook your code repository up to Read The Docs to automatically build and publish your code documentation.

                                                                                        Topics:

                                                                                        • 00:00:00 – Introduction
                                                                                        • 00:02:55 – Being asked about how to get started
                                                                                        • 00:07:13 – Differences in short vs long form writing
                                                                                        • 00:09:03 – What was your introduction to contributing?
                                                                                        • 00:17:52 – What are additional benefits of contributing?
                                                                                        • 00:22:32 – Sponsor: Intel
                                                                                        • 00:23:22 – Sprints as an entry point
                                                                                        • 00:34:36 – Other requirements of a sprint
                                                                                        • 00:36:05 – Differences in conferences
                                                                                        • 00:41:52 – Other sprint experiences
                                                                                        • 00:42:50 – Contributing examples to documentation
                                                                                        • 00:45:59 – Video Course Spotlight
                                                                                        • 00:47:11 – Looking for good first issues
                                                                                        • 00:52:04 – Is this a bug?
                                                                                        • 00:54:10 – Proposing a new feature
                                                                                        • 00:56:36 – Data Morph and working on personal projects
                                                                                        • 01:07:29 – Showing up in the Python community
                                                                                        • 01:12:14 – What are you excited about in the world of Python?
                                                                                        • 01:14:59 – How can people follow the work you do online?
                                                                                        • 01:15:28 – What do you want to learn next?
                                                                                        • 01:18:23 – Thanks and goodbye
                                                                                        • Show Links:

                                                                                          • 5 Ways to Get Started in Open Source - by Stefanie Molin - Level Up Coding
                                                                                          • seaborn: Statistical Data Visualization - Documentation
                                                                                          • Docstring Validation using Pre-Commit Hook - numpydoc
                                                                                          • Hands-On Data Analysis with Pandas: A Python data science handbook for data collection, wrangling, analysis, and visualization, 2nd Edition - Amazon.com
                                                                                          • Episode #173: Getting Involved in Open Source & Generating QR Codes With Python – The Real Python Podcast
                                                                                          • Episode #8: Docker + Python for Data Science and Machine Learning With Tania Allard – The Real Python Podcast
                                                                                          • Mentored Sprints for Diverse Beginners - PyCon US 2023
                                                                                          • Episode #177: Welcoming PyPI’s Safety & Security Engineer Mike Fiedler – The Real Python Podcast
                                                                                          • Data Morph: Moving Beyond the Datasaurus Dozen - Level Up Coding
                                                                                          • Data Morph: A Cautionary Tale of Summary Statistics – Slides
                                                                                          • Anscombe’s quartet - Wikipedia
                                                                                          • How to Set Up Pre-Commit Hooks - Stefanie Molin
                                                                                          • Dog Speaks In Italian Accent To Sound Like His Owner - Bored Panda
                                                                                          • Stefanie Molin (@StefanieMolin) / X
                                                                                          • Stefanie Molin - Personal Website
                                                                                          • Level up your Python skills with our expert-led courses:

                                                                                            • Data Cleaning With pandas and NumPy
                                                                                            • Building Python Project Documentation With MkDocs
                                                                                            • Documenting Python Projects With Sphinx and Read the Docs
                                                                                            • Support the podcast & join our community of Pythonistas

                                                                                              1 hr 20 min

                                                                                            About The Real Python Podcast

                                                                                            From the publisher's feed

                                                                                            A weekly Python podcast hosted by Christopher Bailey with interviews, coding tips, and conversation with guests from the Python community.

                                                                                            More shows like The Real Python Podcast

                                                                                            The Changelog: Software Development, Open Source by Changelog Media

                                                                                            The Changelog: Software Development, Open Source

                                                                                            290 Listeners

                                                                                            Software Engineering Daily by Software Engineering Daily

                                                                                            Software Engineering Daily

                                                                                            624 Listeners

                                                                                            Talk Python To Me by Michael Kennedy

                                                                                            Talk Python To Me

                                                                                            582 Listeners

                                                                                            Soft Skills Engineering by Jamison Dance and Dave Smith

                                                                                            Soft Skills Engineering

                                                                                            296 Listeners

                                                                                            Super Data Science: ML & AI Podcast with Jon Krohn by Jon Krohn

                                                                                            Super Data Science: ML & AI Podcast with Jon Krohn

                                                                                            301 Listeners

                                                                                            Python Bytes by Michael Kennedy and Calvin Hendryx-Parker

                                                                                            Python Bytes

                                                                                            212 Listeners

                                                                                            Syntax - Tasty Web Development Treats by Wes Bos & Scott Tolinski - Full Stack JavaScript Web Developers

                                                                                            Syntax - Tasty Web Development Treats

                                                                                            983 Listeners

                                                                                            Darknet Diaries by Jack Rhysider

                                                                                            Darknet Diaries

                                                                                            8,075 Listeners

                                                                                            Tech Brew Ride Home by Morning Brew

                                                                                            Tech Brew Ride Home

                                                                                            961 Listeners

                                                                                            Practical AI by Daniel Whitenack and Chris Benson

                                                                                            Practical AI

                                                                                            203 Listeners

                                                                                            AWS Podcast by Amazon Web Services

                                                                                            AWS Podcast

                                                                                            202 Listeners

                                                                                            Django Chat by William Vincent and Carlton Gibson

                                                                                            Django Chat

                                                                                            75 Listeners

                                                                                            Last Week in AI by Skynet Today

                                                                                            Last Week in AI

                                                                                            311 Listeners

                                                                                            Machine Learning Street Talk (MLST) by Machine Learning Street Talk (MLST)

                                                                                            Machine Learning Street Talk (MLST)

                                                                                            98 Listeners

                                                                                            The Pragmatic Engineer by Gergely Orosz

                                                                                            The Pragmatic Engineer

                                                                                            74 Listeners