What are the typical computer tasks you do manually every week? Could you automate those tasks with a Python script? Christopher Trudeau is back on the show this week, bringing another batch of PyCoder’s Weekly articles and projects.

We discuss a recent Hacker News thread about frequently used automation scripts. We share the kinds of tasks we’ve automated with Python in our work and personal lives.

Christopher shares a tutorial about building a micro-search engine from scratch using Python. The post takes you through coding the components of a crawler, index, and ranker. The finished engine is designed to search the posts of the blogs you follow.

We also share several other articles and projects from the Python community, including a news roundup, how a Polars query works under the hood, using Python for data analysis, understanding open-source licensing, summarizing the significant changes between Python versions, a robust TUI hex editor, and a lightweight dataframe library with a universal interface for data wrangling.

This week’s episode is brought to you by Intel.

Course Spotlight: Building Command Line Interfaces With argparse

In this step-by-step Python video course, you’ll learn how to take your command line Python scripts to the next level by adding a convenient command line interface that you can write with argparse.

Topics:

News:

Show Links:

Discussion:

Projects

Additional Links:

Level up your Python skills with our expert-led courses:

Support the podcast & join our community of Pythonistas

How do you avoid the bottlenecks of data processing systems? Is it possible to build tools that decouple storage and computation? This week on the show, creator of the pandas library Wes McKinney is here to discuss Apache Arrow, composable data systems, and community collaboration.

Wes briefly describes the humble beginnings of the pandas project in 2008 and moving the project to open source in 2011. Since then, he’s been thinking about improvements across the data processing ecosystem.

Wes collaborated with members of the broader data science community to build the in-memory analytics infrastructure of Apache Arrow. Arrow avoids the bottlenecks of repeated data serialization and format conversion. He shares examples of Arrow’s use across the spectrum in tools like Polars and DuckDB.

Wes advocates moving from vertically integrated tools toward composable data systems. We discuss his work on Ibis, a portable dataframe API for data manipulation and exploration in Python. Ibis supports multiple backends by decoupling the API from the execution engine.

This week’s episode is brought to you by Posit Connect.

Course Spotlight: Unleashing the Power of the Console With Rich

Rich is a powerful library for creating text-based user interfaces (TUIs) in Python. It enhances code readability by pretty-printing complex data structures and adds visual appeal with colored text, tables, animations, and more.

Topics:

Show Links:

Level up your Python skills with our expert-led courses:

Support the podcast & join our community of Pythonistas

What are real-life examples of using Python decorators? How can you harness their power in your code? Christopher Trudeau is back on the show this week, bringing another batch of PyCoder’s Weekly articles and projects.

We discuss a recent article series that digs into Python decorators. The first two articles discuss the basics of constructing decorators. The third part describes how popular Python libraries use decorators with call interception, function registration, and enriching the behavior of a function.

Christopher shares a piece about the common pitfalls of working with the Pythondatetime library. The article considers how current third-party libraries don’t address most of these quirks and offers a potential solution with a new library.

We also share several other articles and projects from the Python community, including a couple of news items, a discussion about the popularity of the Rust language, handling unset values in FastAPI with Pydantic, working with Python’s mini-language for formatting strings, mocking Django queryset functions, and a modern replacement for the Requests library.

This week’s episode is brought to you by Sentry.

Course Spotlight: Python Decorators 101

In this course on Python decorators, you’ll learn what they are and how to create and use them. Decorators provide a simple syntax for calling higher-order functions in Python. By definition, a decorator is a function that takes another function and extends the behavior of the latter function without explicitly modifying it.

Topics:

News:

Show Links:

Discussion:

Projects:

Additional Links:

Level up your Python skills with our expert-led courses:

Support the podcast & join our community of Pythonistas

How do you manage the dependencies of a large-scale data science project? How do you migrate that project from a laptop to cloud infrastructure or utilize GPUs and multiple instances in parallel? This week on the show, Savin Goyal returns to discuss the updates to the open-source framework Metaflow.

Savin briefly describes the Metaflow platform and the goal of simplifying engineering overhead for data scientists and programmers. We discuss how the platform captures snapshots of a project as you work, allowing you to go back in time or share the state of your project with another team member.

We dig into the complicated process of managing dependencies for machine learning and data science projects. Savin describes how the required external libraries can be specified within a flow with the new @pypi or @conda decorators. This allows a project to scale from a local machine to the cloud or multiple instances with all dependencies included.

He talks about starting a new company, Outerbounds, with fellow co-workers from Netflix. Their vision is to continue to build the Metaflow open-source platform and offer customers scalable enterprise-grade infrastructure.

This week’s episode is brought to you by Intel.

Course Spotlight: Everyday Project Packaging With pyproject.toml

In this Code Conversation video course, you’ll learn how to package your everyday projects with pyproject.toml. Playing on the same team as the import system means you can call your project from anywhere, ensure consistent imports, and have one file that’ll work for many build systems.

Topics:

Show Links:

Level up your Python skills with our expert-led courses:

Support the podcast & join our community of Pythonistas

What’s it like to sit down for your first developer sprint at a conference? How do you find an appropriate issue to work on as a new open-source contributor? This week on the show, author and software engineer Stefanie Molin is here to discuss starting to contribute to open-source projects.

Stefanie is a data scientist and software engineer on Bloomberg’s Security Data Science team. She recently wrote an article titled “5 Ways to Get Started in Open Source.” We discuss finding ways to contribute that fit your interests and developer skills. We dig into the experience of participating in community sprints at a conference.

Stefanie is the author of Hands-On Data Analysis with Pandas. We also discuss the different processes between writing technical articles and authoring a book.

This week’s episode is brought to you by Intel.

Course Spotlight: Documenting Python Projects With Sphinx and Read the Docs

In this video series, you’ll create project documentation from scratch using Sphinx, the de facto standard for Python. You’ll also hook your code repository up to Read The Docs to automatically build and publish your code documentation.

Topics:

Show Links:

Level up your Python skills with our expert-led courses:

Support the podcast & join our community of Pythonistas

How does a debugger work? What can you learn about Python by building one from scratch? Christopher Trudeau is back on the show this week, bringing another batch of PyCoder’s Weekly articles and projects.

Christopher shares a two-part tutorial on building a debugger in Python? Creating a simple one requires less code than you might think.

We also talk about an article from Itamar Turner-Trauring about how to prepare for the upcoming changes to NumPy. The new version is not backward compatible and will require some inspection of your project dependencies. Itamar includes advice, techniques, and tools for updating your code.

We also share several other articles and projects from the Python community, including a couple of news items, a discussion about managing advice as a new developer, moving to Python as a former R developer, building a Markov chain to generate readable nonsense, optimizing fonts to individual glyphs on your website, and a project for working with units of measurement in Python.

This week’s episode is brought to you by Posit Connect.

Course Spotlight: Create a Tic-Tac-Toe Python Game Engine With an AI Player

In this video course, you’ll create a universal game engine in Python for tic-tac-toe with two computer players, one of which will be an AI player using the powerful minimax algorithm. You’ll give your game library a text-based graphical interface and explore two front ends.

Topics:

News:

Show Links:

Discussion:

Projects:

Additional Links:

Level up your Python skills with our expert-led courses:

Support the podcast & join our community of Pythonistas

How can you measure the quality of a large language model? What tools can measure bias, toxicity, and truthfulness levels in a model using Python? This week on the show, Jodie Burchell, developer advocate for data science at JetBrains, returns to discuss techniques and tools for evaluating LLMs With Python.

Jodie provides some background on large language models and how they can absorb vast amounts of information about the relationship between words using a type of neural network called a transformer. We discuss training datasets and the potential quality issues with crawling uncurated sources.

We dig into ways to measure levels of bias, toxicity, and hallucinations using Python. Jodie shares three benchmarking datasets and links to resources to get you started. We also discuss ways to augment models using agents or plugins, which can access search engine results or other authoritative sources.

This week’s episode is brought to you by Intel.

Course Spotlight: Learn Text Classification With Python and Keras

In this course, you’ll learn about Python text classification with Keras, working your way from a bag-of-words model with logistic regression to more advanced methods, such as convolutional neural networks. You’ll see how you can use pretrained word embeddings, and you’ll squeeze more performance out of your model through hyperparameter optimization.

Topics:

Background Links:

Dataset Links:

Tutorials and Documentation for Python Packages:

Measurement Links:

Training Data for LLMs:

Agents and Plugin Links:

Additional Links:

Level up your Python skills with our expert-led courses:

Support the podcast & join our community of Pythonistas

Do you need to transfer an extensive data collection for a science project? What’s the best way to send executable code over the wire for distributed processing? What are the different ways to serialize data in Python? Christopher Trudeau is back on the show this week, bringing another batch of PyCoder’s Weekly articles and projects.

Christopher shares a tutorial by Real Python author Bartosz Zaczyński called “Serialize Your Data With Python.” This comprehensive guide moves beyond XML and JSON to explore multiple data formats and their potential use cases. It’s a deep dive into the topic and provides a thorough resource for future reference.

We also discuss a Real Python tutorial about naming conventions in Python that use single and double underscores. The piece covers differentiating between public and non-public names in APIs, writing safe classes for subclassing purposes, and avoiding name clashes with keywords.

We also share several other articles and projects from the Python community, including a couple of release announcements and news items, a discussion about never being taught how to construct quality software, building a small REPL in Python, using the key parameter in Python functions and methods, a framework for RESTful APIs using Flask and SQLAlchemy, and a Rust-based HTML sanitizer for your Python projects.

Course Spotlight: Writing Beautiful Pythonic Code With PEP 8

Learn how to write high-quality, readable code by using the Python style guidelines laid out in PEP 8. Following these guidelines helps you make a great impression when sharing your work with potential employers and collaborators. This course outlines the key guidelines laid out in PEP 8. It’s aimed at beginner to intermediate programmers.

Topics:

News:

Show Links:

Discussion:

Projects:

Additional Links:

Level up your Python skills with our expert-led courses:

Support the podcast & join our community of Pythonistas

Are you interested in using your Python skills within Excel? Would you like to share a data science project or visualization as a single Office file? This week on the show, we speak with Principal Architect John Lam and Sr. Cloud Developer Advocate Sarah Kaiser from Microsoft about Python in Excel.

John shares the multi-year journey of adding Python to Excel. He describes how the project moved beyond writing user functions in Python to something much more elaborate. He details assembling a team with diverse skills in interface design, languages, and security.

Sarah discusses the instant convenience of having familiar Python and pandas techniques at your fingertips inside Excel. We cover typical data science workflows and the potential of interactive visualizations within a spreadsheet. We also share multiple resources for you to learn more.

Note: Python in Excel is currently a preview accessible by joining the Microsoft 365 Insider Program and selecting the Beta Channel.

Course Spotlight: Data Cleaning With pandas and NumPy

In this video course, you’ll learn how to clean up messy data using pandas and NumPy. You’ll become equipped to deal with a range of problems, such as missing values, inconsistent formatting, malformed records, and nonsensical outliers.

Topics:

Show Links:

Level up your Python skills with our expert-led courses:

Support the podcast & join our community of Pythonistas

Three members of the Real Python team are joining us this week: Kate Finegan, Tappan Moore, and Philipp Acsany. We wanted to share a year-end wrap-up with tutorials, step-by-step projects, code conversations, and video courses that showcase what our team created this year.

Kate helps to shepherd articles through the multi-stage editing process. She and the rest of the team ensure these resources impart crucial Python knowledge and provide a thorough didactic experience. Kate was also instrumental in helping introduce a new group of tutorial authors to the Real Python editorial process and house style.

Philipp returns to the podcast after our conversation earlier this year, and it was great to talk to him about onboarding new video instructors. Tappan edits all our video courses and ensures that the sound, picture, and animations are just right. He also helped provide feedback to the new instructors on our video creation process.

We hope you enjoy this review! We look forward to bringing you another year full of great guests, articles, and topics.

Course Spotlight: Recursion in Python

A recursive function is one that calls itself. In this video course, you’ll see what recursion is, how it works in Python, and under what circumstances you should use it.

Topics:

Show Links:

Additional Links:

Level up your Python skills with our expert-led courses:

Support the podcast & join our community of Pythonistas