March 01, 2024

Automate Tasks With Python & Building a Small Search Engine

44 minutes

What are the typical computer tasks you do manually every week? Could you automate those tasks with a Python script? Christopher Trudeau is back on the show this week, bringing another batch of PyCoder’s Weekly articles and projects.

We discuss a recent Hacker News thread about frequently used automation scripts. We share the kinds of tasks we’ve automated with Python in our work and personal lives.

Christopher shares a tutorial about building a micro-search engine from scratch using Python. The post takes you through coding the components of a crawler, index, and ranker. The finished engine is designed to search the posts of the blogs you follow.

We also share several other articles and projects from the Python community, including a news roundup, how a Polars query works under the hood, using Python for data analysis, understanding open-source licensing, summarizing the significant changes between Python versions, a robust TUI hex editor, and a lightweight dataframe library with a universal interface for data wrangling.

This week’s episode is brought to you by Intel.

Course Spotlight: Building Command Line Interfaces With argparse

In this step-by-step Python video course, you’ll learn how to take your command line Python scripts to the next level by adding a convenient command line interface that you can write with argparse.

Topics:

00:00:00 – Introduction

00:02:23 – uv: Python Packaging in Rust

00:02:43 – Rye Grows With uv

00:03:20 – Python 3.13.0 Alpha 4 Is Now Available

00:03:45 – A Bird’s Eye View of Polars

00:07:28 – Polars: Why We Have Rewritten the String Data Type

00:09:33 – A Search Engine in 80 Lines of Python

00:13:14 – Using Python for Data Analysis

00:18:22 – Sponsor: Intel

00:18:53 – Understanding Open Source Licensing

00:21:54 – Summary of Major Changes Between Python Versions

00:23:19 – What Python automation scripts do you reuse frequently at work?

00:34:21 – Video Course Spotlight

00:35:52 – hexabyte: A modern, modular, and robust TUI hex editor

00:39:56 – ibis: The Flexibility of Python With the Scale of Modern SQL

00:43:31 – Thanks and goodbye

News:

uv: Python Packaging in Rust – uv is an extremely fast Python package installer and resolver, designed as a drop-in alternative to pip and pip-tools. This post introduces you to uv and shows some of its performance numbers.

Rye Grows With uv - Armin Ronacher’s Thoughts and Writings

Python 3.13.0 Alpha 4 Is Now Available

Show Links:

A Bird’s Eye View of Polars – This post on the Polars blog introduces you to how Polars works, showing the steps from queries, plans, optimizations, and then the final execution.

Polars: Why We Have Rewritten the String Data Type – A large refactor on the string data type is underway in Polars. This deep dive explains why and what is changing.

A Search Engine in 80 Lines of Python – In this post Alex explains how he built a micro-search engine from scratch using Python. The resulting search engine is used to search in the posts of the blogs he follows.

Using Python for Data Analysis – In this tutorial, you’ll learn the importance of having a structured data analysis workflow, and you’ll get the opportunity to practice using Python for data analysis while following a common workflow process.

Understanding Open Source Licensing – This article discusses the importance of open-source licensing in software development and its implications for stakeholders.

Summary of Major Changes Between Python Versions – This article is a quick reference covering the major changes introduced with each new version of Python. Can’t remember when the walrus operator was introduced? This is the place to look that up.

Discussion:

What Python automation scripts do you reuse frequently at work? - Hacker News

Projects

hexabyte: A modern, modular, and robust TUI hex editor

ibis: The Flexibility of Python With the Scale of Modern SQL

Additional Links:

Episode #193: Wes McKinney on Improving the Data Stack & Composable Systems – The Real Python Podcast

Open Source Licensing: Software Freedom and Intellectual Property Law

What’s in which Python - Ned Batchelder

Automate the Boring Stuff with Python - Al Sweigart

Working With Files in Python – Real Python

Build Command-Line Interfaces With Python’s argparse – Real Python

Level up your Python skills with our expert-led courses:

Defining Python Functions With Optional Arguments

Practical Recipes for Working With Files in Python

Building Command Line Interfaces With argparse

Support the podcast & join our community of Pythonistas

...more

View all episodes

By Real Python

4.7

139139 ratings