Have you worked on a project that needed an orchestration tool? How do you define the workflow of an entire data pipeline or a messaging system with Python? This week on the show, Calvin Hendryx-Parker is back to talk about using Apache Airflow and orchestrating Python projects.

Calvin is the co-founder and CTO of Six Feet Up and a Python Web Conference co-organizer. He’s recently been working on a massive project that requires thousands of jobs involving transferring and transforming data. Through his research into orchestration systems, he found Apache Airflow.

Airflow is an open-source tool to define, schedule, and monitor workflows. The platform is pure Python and integrates with a wide variety of services. We discuss how workflows are defined by creating directed acyclic graphs (DAG).

Calvin talks about how a recent project outgrew the system and how his team built a clever solution using Python. We also discuss the upcoming Python Web Conference and what virtual attendees can expect.

Course Spotlight: Python Basics: Object-Oriented Programming

In this video course, you’ll get to know OOP, or object-oriented programming. You’ll learn how to create a class, use classes to create new objects, and instantiate classes with attributes.

Topics:

Show Links:

Level up your Python skills with our expert-led courses:

Support the podcast & join our community of Pythonistas