Buğra Öztürk, Senior Data Engineer at Mollie and Committer and PMC member on the Apache Airflow project, joins us to walk through Airflow CTL — what it is, how it differs from the existing Airflow CLI and where it is headed under AIP-94.

Key Takeaways:

00:00 Introduction.

03:10 Buğra has contributed to Airflow since 2022, from docs changes up to Committer and PMC member — a path he hopes inspires others to start small and contribute. 

04:05 Airflow CTL solves secure user interaction by abstracting database credentials behind the public core API. 

05:13 Airflow CLI and Airflow CTL are complementary — CLI handles administration and database management while CTL handles secure user interactions via the API. 

07:08 Airflow CTL authenticates via the API, acquires a JWT token and stores it securely in the OS keyring — running on the user's machine and never requiring direct database access.

08:21 Concrete use cases include local DAG development without the UI and CI/CD automation using headless mode with short-lived JWT tokens.

10:08 AIP-94 describes the long-term vision — decoupling all remote commands from the Airflow CLI and routing them through Airflow CTL. 

13:12 Airflow CTL is currently at 0.X and already being used in CI and deployment automations. The move to 1.0 with full CLI parity is the next milestone under AIP-94.  

16:09 Multi-team deployment becoming generally available in a future Airflow release is Buğra's most-anticipated upcoming feature beyond Airflow CTL.

Resources Mentioned:

Buğra Öztürk

https://www.linkedin.com/in/bugraozturk93/

Mollie

https://www.linkedin.com/company/mollie/

Mollie | Website

https://www.mollie.com/

Apache Airflow CTL 

https://airflow.apache.org/

AIP-94 on Airflow Confluence

https://lists.apache.org/thread/d2o1pr78wxdp1wozq519stp0pkcv6k6c

Apache Airflow GitHub

https://www.github.com/apache/airflow

Thanks for listening to “The Data Flowcast: Mastering Apache Airflow® for Data Engineering and AI.” If you enjoyed this episode, please leave a 5-star review to help get the word out about the show. And be sure to subscribe so you never miss any of the insightful conversations.

#AI #Automation #Airflow #MachineLearning